Parquet Export

apache parquet is an open source, column oriented data file format designed for efficient data storage and retrieval it provides efficient data compression and encoding schemes with enhanced performance to handle complex data in bulk creating parquet exports is currently an experimental feature, which means the feature is still subject to change to create a parquet export, make sure ‘experimental features’ are enabled, then navigate to configuration > parquet export the screen shows an overview of the created export schedules, and allows for the creation of a one of export or the creation of an export schedule currently the parquet export function only exports data that has a good quality status one of export to create a one of export task that only runs once, click the parquet export button in the upper right corner a modal will open where the export task can be configured select a time range for which to export data choose a directory where the exported parquet file(s) will be saved choose an interval to split up the range into multiple files select the label(s) of the measurement(s) that need to be exported export on a schedule to create a recurring export, press the create task scheduler button in the top right corner fill in the task details choose a name and description for the scheduler write an {{rrule}} to define when and how often the task should be scheduled fill in the export details fill in the start offset the start offset determines the start of the interval for the exported data, relative to the rrule’s trigger event fill in the period for the data to export or fill in the stop offset choose an interval to split up the range into multiple files specify a directory where the parquet files should be saved select the labels of the measurements that should be exported click save an rrule is way to define a recurrence set the rule defines a pattern to generate a series of timestamps for events their syntax is defined by section 3 8 5 3 of rfc 5545 use the rrule tool for help in creating rrules example the following is an example configuration of an exporter the used rrule is rrule\ freq=hourly;interval=1;byminute=0;bysecond=0 , which triggers every hour, on the hour by setting the start offset to 1h10m and the period to 1h , the scheduler will export an hour’s worth of data, starting from 10 minutes before the previous hour until 10 minutes before the current hour setting the stop offset to 10m would be equivalent to setting the period to 1h in this example trigger at export start time export stop time 12 00 10 50 11 50 13 00 11 50 12 50 14 00 12 50 13 50 export progress creating a parquet export can take some time, depending on the amount of data that needs to be exported task progress can be viewed in the tasks docid\ ouywpxc3neevmwrszfhle screen