Best Practices: Tank Monitor Import Frequency

Where to locate the setting/General notes

• The setting can be found in Cargas by navigating to: Admin -> Setup -> Tank Monitors, then scrolling down to your Tank Monitor settings
• NOTE: This setting only applies to FTP-based monitor integrations. It does not apply to API-based integrations (e.g., Otodata).
• For FTP integrations, Cargas imports a bulk data file from the monitor vendor’s FTP server. The frequency you set in Cargas controls how often we reach out to the FTP server to retrieve available data.
• The frequency set in Cargas does not control how often the vendor updates the file on their FTP server.

Wesroc

• Wesroc monitors may report several times per day. However, they only update the FTP server once per day. As a result, increasing the frequency in Cargas will only cause the same data to be re-imported multiple times until the next FTP update.
• Example:
  • Wesroc updates the FTP server at 6:00 AM daily.
  • If Cargas is set to import every 15 minutes, you will import the same data 95 times between 6:15 AM and the next day's 6:00 AM update.
  • New data is only available once per day, no matter how often we poll the FTP server.
• You can reach out to your contact at Wesroc to adjust when the update their FTP server. 

Other FTP Integrations

• The other FTP Integrations include (TankSure, Centeron, Enertrac, Silicon Controls, Ecogreen, Skybitz)
• These will work similar to Wesroc as listed above with minor differences with how often they update their FTP server that we access. 

Why the Frequency matters and the effects it can have

While frequent polling might seem beneficial, it has significant downsides:

• System Impact of High Frequency

  • Repeatedly importing the same unchanged data bloats the TankMonitorImportHistory table.
  • Cargas retains the last 3 months of import history.
  • A high import frequency (e.g., every 15 minutes) results in significantly more rows than a lower frequency (e.g., hourly).

• Performance Degradation

  • The ArchiveTankMonitorImport process regularly clears out data older than 3 months.
  • As the Import History table grows, this archival process:
    • Takes longer to scan and delete outdated rows.
    • Is more prone to deadlocks, which can:
      • Slow or block other processes.
      • Prevent new monitor imports from being processed.

• Summary

  • A higher import frequency does NOT mean fresher data if the FTP server is not updated more frequently.
  • High frequency causes redundant imports, bloats system tables, slows archive processes, and risks deadlocks.
  • Recommended Frequency is 60 minutes