As software engineers and manufacturing industry professionals, it is important to understand the use of timestamps. This article explains the basics of what timestamps are, the features it boasts, and a possible solution to any associated headaches.
What is a Timestamp?
A timestamp is a way of recording the date and time of each data point or event in a data acquisition system. Having correct timestamps in process control data acquisition is essential for ensuring the validity, reliability, and usability of the data. It also helps to enable effective process monitoring, control, and optimization.
Precise timestamps ensure accurate analysis and interpretation of the data. This is especially crucial because it aids in the identification of trends, patterns, anomalies, and correlations. Timestamps also facilitate synchronization and integration of data from multiple sources. This includes different sensors, instruments, devices, or systems.
Types of Timestamps:
According to the OPC UA Standard, each value of an OPC UA variable is associated with 2 timestamps: Source and Server timestamps.
- The Source Timestamp is used to reflect the timestamp that was applied to a variable value by the data source. This indicates the time at which data values were measured at the lowest-level data source. It’s important to consider that this data source can be located in the same machine that runs the OPC UA Server, or it can be in a different device with its own system clock.
- The Server Timestamp is used to reflect the time that the Server received a variable value or deemed it as accurate.
If the server reads data values itself, then the source and server timestamps will usually be the same. If an OPC UA Server gets data from another device that supports timestamps, then the source timestamp can be significantly different than the server timestamp.
Time Zones: A Mystery
No matter the situation, system clocks in devices that are a part of the data acquisition path should be in sync. Ideally, system clocks should be synchronized with time servers, such as an NTP protocol.
To avoid confusion and errors during conversions, all timestamps in OPC UA are UTC timestamps. Interestingly, this means that there are no time zones or daylight savings. To offset this, timestamp values are usually converted to the user’s local time by the application displaying them.
When an OPC UA client creates a subscription with monitored items, the device can define which timestamps it needs to receive. This can be a source or server timestamp, and sometimes even both.
The Solution:
Our bestselling product Idako: Industrial Data Collector, creates subscriptions and monitored items that request both timestamps. This allows you to have the choice of freedom between different timestamp formats. In addition, Idako is scalable, energy efficient, and robust when handling an unlimited number of tags. When data values are forwarded to the destination time-series database or MQTT broker, the format of timestamps depends on the destination database type.
Different Scenarios, Depending on the Destination Database:
- When the destination is an SQL database or Snowflake, the source timestamp is written in a “time” column of the values table. If the source timestamp is not defined, then a server timestamp is used. It is also possible to write a client timestamp in the column “client_time”. The “client_time” is the time when data values were received by the Idako from the OPC UA Server. This timestamp is especially useful when the server or source timestamps are unreliable and not accurate enough.
- When the destination is InfluxDB, Confluent, or Apache Kafka, then the source timestamp is used as a record timestamp. If the payload is composed using a template, then all three timestamps can be included in the payload using placeholders like “[SourceTimestamp]”, “[ServerTimestamp]”, and/ or “[ClientTimestamp]”.
- When the destination is an MQTT broker, timestamps cannot be a part of the published messages. This is because the MQTT protocol does not exactly specify how timestamps should be transported from the publisher to the broker. So, they can be only included in the payload. In this case, the payload should be defined using a template, with placeholders for timestamps like: “[SourceTimestamp]”, “[ServerTimestamp]”, and/ or “[ClientTimestamp]“.
Duplicate Records and High-Resolution:
In some cases, duplicate records (with the same value and timestamp for the same variable) can be written on the database. This can occur when Idako disconnects from the server and reconnects, or when it restarts. In these cases, a variable data value can be the same, and the source timestamp can also be the same as before it was reconnected. This might cause duplicate record errors in SQL databases if the values table is configured to have a unique index by “source_id” and time column values. To resolve this issue, our product Idako has configuration settings that allow duplicate records (refer to our User Manual for details).
Furthermore, OPC UA allows the definition of timestamps with high resolution: down to a 10 picosecond precision. Other target databases usually do not support such high resolutions, so this is a very impressive feature. The Idako configures the precision with which timestamps are written. This can be seconds, milliseconds, or microseconds. Different storage destinations have different ways to represent timestamps.
Our product Idako is a master in fine-tuning the timestamp format. This can occur in the following ways:
- An integer number representing a Unix epoch time (depending on the number of seconds, milliseconds, or microseconds precision since Jan. 1st, 1970).
- A string value formatted with the ISO-8601 standard
- An OPC UA DateTime value (which is an integer number of 100 nanosecond intervals passed since Jan. 1st, 1601).
If you enjoyed learning more about timestamps and its uses, feel free to comment and ask questions. If you’re interested in finding out more about the Idako, email us at sales@onewayautomation.com to begin the purchase process.