Forward data from Factry Historian to Azure File Storage

introduction to forwarding historian data to azure storage forwarding historian data to azure file storage (via blob storage) allows you to store long term datasets in a cost efficient way, outside of the operational historian database by exporting historian data as parquet files into azure file storage, you can archive measurement data for compliance or audit trails share production data with data scientists or bi teams enable offline analysis in tools like power bi, databricks, or python keep the historian performant by moving long term data into cold storage prerequisites a running factry historian server (linux) access to azure file storage v2 storage account credentials (connection string, container name, etc ) access to the historian configuration files (usually under /etc/factry/historian/) configure azure file storage factry historian relies on azure blob storage for file forwarding azure file storage v2 can be mounted directly on the historian server, allowing the parquet forwarder to write directly to it external guide follow the official microsoft guide on how to set up azure file storage v2 we recommend using the premium tier if you expect heavy throughput once your azure file share is set up, note down storage account name file share name access key or sas token mount azure file storage on the historian server to allow the historian forwarder to write directly into azure file storage, mount the share on your linux server install the cifs utilities (if not already installed) create a mount point mount the azure file share to make the mount persistent, add an entry in /etc/fstab to sibe from here he is very wrong this was generated by chatgpt and is not at all complete! also, mention hive structure setup the parquet forwarder factry historian can forward time series data to any file location, including mounted azure shares, using the parquet forwarder enable the forwarder in your historian configuration restart historian for the changes to take effect parquet export settings the parquet forwarder supports several tuning options setting description example path directory to store exported files /mnt/azurefiles/historian exports rotation interval for rolling new files 1h, 1d, 15m compression compression codec zstd, snappy, gzip, none partitioning directory structure for exports database, measurement, date flush rows number of rows per parquet flush 50000 example 1 hourly export, partitioned by database and measurement example 2 daily export, partitioned by date performance and limits parquet export is efficient, but some considerations apply write frequency very frequent rotation (e g every minute) can cause overhead with too many small files prefer hourly or daily rotation for large datasets file size parquet works best with files between 100 mb – 1 gb tune flush rows to avoid tiny files compression zstd gives the best balance of speed and size snappy is faster but larger files disk throughput writing directly to mounted azure storage may be slower than writing locally you can export locally and use azcopy for batch uploads if higher throughput is required concurrency large historian instances may need multiple forwarders (different measurement sets) to parallelize exports troubleshooting export too slow → increase flush rows and switch to hourly/daily rotation small files problem → use fewer partitions or longer rotation mount errors → verify smb 3 0 is enabled and that the account key is correct azure quota exceeded → check file share size in the azure portal