REST API collector
The REST API collector can read data points from a REST API endpoint. The collector supports reading data points in JSON and CSV format.
Basic settings
TimestampLayout
Description: The timestamp layout is used to correctly interpret the timestamp(s) of the data points read for a measurement. A list of available timestamp formats, along with their string representations, can be found here. Required: No If Empty: The default RFC3339 timestamplayout e.g. 2006-01-02T15:04:05Z07:00 is used.
For timestamps in UNIX format, one of the following values can be used: UNIX (time in s), UNIXMILLIS (time in ms), UNIXMICROS (time in μs), or UNIXNANOS (time in ns).
For other timestamp formats, see the Read data from JSONRead data from JSON examples.
Warning: The collector TimestampLayout setting is the default TimestampLayout setting for all measurements on the collector. It can be overwritten per measurement by setting the TimestampLayout setting on the measurement.
URL
Description: The URL to the REST API endpoint. Required: Yes.
The collector URL setting is the default URL setting for all measurements on the collector. It can be overwritten per measurement by setting the URL setting on the measurement, but it must still be set on the collector itself: an empty collector URL fails validation at startup with invalid config: URL is missing and the collector never polls.
QueryParameters
Description: A JSON object to configure default query parameters for all requests. This is only used when the Method is set to GET. The query parameters must contain <startTime> and <stopTime> as values of a JSON key. If these are not present, the collector will go into an error status. Required: Yes, when the Method is GET. Default: {} Example:
{
"id": "temperature",
"start": "<startTime>",
"stop": "<stopTime>"
}The keywords <startTime> and <stopTime> are replaced by the collector by the start time and stop time when sending a request. For each request, the start time and stop time are derived from the Interval setting.
Warning: The collector QueryParameters setting is the default QueryParameters setting for all measurements on the collector. It can be overwritten per measurement by setting the QueryParameters setting on the measurement. The collector's own request is validated on its own, before any measurement setting is merged in, so the collector QueryParameters must contain both keywords even when every measurement overrides them. Leaving it at \{} with Method GET fails at startup with <startTime> is missing from QueryParameters.
Body
Description: A JSON object to configure the default Body for all requests. Only in use when the Method is set to POST. The Body must contain <startTime> and <stopTime> as values of a JSON key. If these are not present, the collector will go into an error status. Required: Yes, when the Method is POST. Example:
{
"id": "temperature",
"start": "<startTime>",
"stop": "<stopTime>"
}Warning: The collector Body setting is the default Body setting for all measurements on the collector. It can be overwritten per measurement by setting the Body setting on the measurement. The collector's own request is validated on its own, before any measurement setting is merged in, so the collector Body must contain both keywords even when every measurement overrides them. Leaving it empty with Method POST fails at startup with <startTime> is missing from Body.
Method
Description: The default HTTP(S) request method to be used. Required: No. Options: GET | POST Default: the HTTP(S) method GET is used.
The collector Method setting is the default Method setting for all measurements on the collector. It can be overwritten per measurement by setting the Method setting on the measurement.
Interval
Description: The interval (in seconds) at which API requests should be sent. Use -1 to leave empty. Required: No. If Empty: The Interval from the measurement settings is used. Example: 86400 This sends one request each day.
The polling schedule uses the interval rounded up to a value that divides a week evenly, so an interval that does not is polled slightly less often than configured. The queried time period is still computed from the value you set, which leaves a gap between consecutive requests. Prefer an interval that divides a week, such as 60, 900, 3600 or 86400.
The collector Interval setting is the default Interval setting for all measurements on the collector. It can be overwritten per measurement by setting the Interval setting on the measurement.
IntervalOffset
Description: The default offset in seconds at which requests are sent, relative to the start of each Interval window. The windows are aligned on the local week, which starts Sunday at 00:00:00, and an offset larger than the interval is taken modulo the interval. Use -1 to leave empty. Required: No. If Empty: The IntervalOffset from the measurement settings is used. Example: 1 With one request each day, this sends the request the 1st second after midnight, instead of at midnight itself.
The collector IntervalOffset setting is the default IntervalOffset setting for all measurements on the collector. It can be overwritten per measurement by setting the IntervalOffset setting on the measurement.
Headers
Description: A JSON object to configure default headers for all requests. Use {} to leave empty. Required: No, except with Method POST, which needs a Content-Type header.
With Method POST the body is marshalled according to the Content-Type header. Without it every request fails with no Content-Type header specified in the client. Only application/json, application/xml and application/x-www-form-urlencoded are implemented.
Example:
{
"Authorization": "Bearer xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"referer": "API-collector"
}The collector Headers setting is the default Headers setting for all measurements on the collector. It can be overwritten per measurement by setting the Headers setting on the measurement.
Historian re-run settings
StartTimeHistoric
To start a historic re-run of the data points for all measurements on the collector (in parallel with the realtime run), set the StartTimeHistoric setting.
Description: An optional historic start time. The timestamp Layout must match the TimestampLayout setting. Required: No. If Empty: No historic re-run of the data is started and only live data will be processed.
Empty the StartTimeHistoric when the historic data reading is finished (see the collector logs), to prevent it from being repeated on resuming the collector after a restart or a pause.
StopTimeHistoric
The StopTimeHistoric is only used if the StartTimeHistoric is not empty
Description: An optional historic stop time. The timestamp Layout must match the TimestampLayout setting. Required: No. If Empty: The run for processing historic data will go up to now.
IntervalHistoric
The IntervalHistoric is only used if the StartTimeHistoric is not empty.
Description: An optional interval in seconds that indicates the time difference between the start time and the stop time for each request in a historic re-run (requests are executed each second). Required: No. If Empty: The interval 900 is set (15 minutes). Example: 86400 This sends a request for a time period of a day (time between start time and stop time of each request in the historic re-run) at each second (fixed for historic re-run).
Advanced settings
Only collector-specific advanced settings are listed below.
ReadTimeout
Description: The HTTP(S) request timeout in seconds. If the collector does not receive a response in this timeframe after the request, the request will be disregarded and an error status is written on the measurements in that request. Required: No. Default: 5 seconds. Set to 0 for no timeout.
IdleTimeout
Description: Maximum time (in seconds) an idle keep-alive HTTP connection can stay open without any data being sent or received. Once this time passes, the connection will automatically close. Required: No. Default: 0 seconds (means no limit)
LookBackIntervals
Description: The amount of intervals (from the Interval setting) to additionally look back before the start time of each live request. It is a count of intervals, not a number of seconds, and it does not apply to historic re-runs, which carry an explicit start and stop time. Required: No. Default: 0 (means only the current interval will be used as time period for each request) Example: If the Interval is set to 900 (15 minutes), and the LookBackIntervals is set to 3. Then for each request, the start time will be set 45 minutes before the start time determined by Interval, while the stop time of the request is kept. This results in each request setting a time period of 1 hour.
The LookBackIntervals is typically used when the REST API server does not hold the data points in realtime, but only holds data points that from some time ago e.g. it does only hold the data points 30 minutes after the fact.
StopTimeShift
Description: An optional shift in seconds, subtracted from the stop time of each live query. Useful when the source REST API needs a small delay before its latest data becomes available. Can be negative to shift the stop time into the future. Does not apply to historic runs. Required: No. Default: 0 seconds (means no shift is applied to the stop time) Example: If the Interval is set to 900 (15 minutes), and the StopTimeShift is set to 1. Then for each request, the stop time is set 1 second earlier than the stop time determined by Interval, while the start time is kept. This avoids querying up to now, which some APIs reject.
SkipVerifySSLCertificate
Description: When enabled, the certificate chain and host name of the server are not verified, for both data and authentication requests. Only use this against a server with a self-signed certificate on a trusted network. Required: No. Default: false
QueryTimezone
Description: Specifies the time zone used to interpret query timestamps (for example, the start and stop time parameters in a request) before formatting them according to the TimestampLayout setting.
The time zone must be specified using a valid name from the IANA Time Zone database, which you can consult on wikipedia. Required: No. Default: UTC
AuthType
Description: Specifies the type of authentication used before making API request. None (the default) means no authentication request is made and data requests proceed directly. Required: No. Default: None Options: None | oAuth2
AuthURL
Description: The endpoint (URL) used to send authentication requests. Only used if AuthType is not None. Required: No.
AuthInterval
Description: How often (in seconds) the system should re-authenticate. Only used if AuthType is not None. Required: No. Default: 3600 (1 hour)
AuthID
Description: The unique identifier (such as a client ID or username) used in authentication requests. Only used if AuthType is not None. Required: No.
AuthSecret
Description: The secret key or password used for authentication requests. Only used if AuthType is not None. Required: No.
AuthUsername
Description: The username used for authentication requests. Only used if AuthType is not None and AuthGrantType is password. Required: No.
AuthPassword
Description: The password used for authentication requests. Only used if AuthType is not None and AuthGrantType is password. Required: No.
AuthGrantType
Description: Specifies the OAuth grant type for authentication requests. Only used if AuthType is not None. Options: password | client_credentials Required: No. Default: password
AuthScope
Description: A comma-separated list of authorization scopes requested during authentication. Only used if AuthType is not None. Required: No.
Measurement settings
Every collector setting marked as overridable above (TimestampLayout, URL, QueryParameters, Body, Method, Interval, IntervalOffset and Headers) can also be set on an individual measurement, where it overrides the collector default.
CollectionType
Description: How the points of this measurement are stored. With polled every point is stored. With monitored a point is dropped when its value, status and attributes are identical to the previous point. Required: No. Default: polled Options: polled | monitored
The settings that describe how to read values out of the response are documented per data format, see REST-API response format below.
REST-API response format
For the REST-API collector to read data points from:
- a JSON response, see JSON Settings.
- a CSV response, see Read data from CSVRead data from CSV.
Next steps
To learn how to configure a REST-API collector: see Configuring a REST API collectorConfiguring a REST API collector.