Industrial Data Collector User Manual

Integration tool to collect, store, visualize and analyze OPC UA data.

Contents:

• Introduction
• Licensing
  • Licence for Industrial Data Collector.
  • Annual Maintenance and Upgrades License.
  • Licenses of open source dependencies.
• Features
  • Deploy
  • Activate
  • Configure
  • Collect
  • Store and Forward
  • Visualize
  • Analyze
  • High Availability
  • Troubleshoot
  • Target Platforms
  • More features are coming
• Deploy
  • Setup using Docker
    • Pulling docker image of Industrial Data Collector and running it.
    • Using docker-compose.
    • Running all supported databases and demo OPC UA Server in Docker
    • Breaking change in Docker container version 4.2.0
  • Setup in Windows using the installer
  • Portable Setup in Windows
    • 1. Download portable distribution package
    • 2. Install required prerequisites
      • a). Install Visual C++ 2019 redistributables (64 bit version)
      • b). Install Microsoft ODBC Driver 17 for SQL Server.
      • c). Install / Configure database to store time-series data.
    • 3. Start Industrial Data Collector.
    • 4. Running Industrial Data Collector as a Windows service.
    • 5. Install Grafana
  • Setup in Ubuntu 2022.04
  • Setup in RedHat Enterprise Linux
  • Setup in 64-bit Raspberry Pi.
  • Setup in Raspberry 32-bit Pi.
    • Prepare Micro SD card with 32 bit debian-bullseye OS.
    • Install Docker and Docker Compose
    • Get docker compose files for Industrial Data Collector and start it:
  • Install as Azure IoT Edge module.
  • Installing as Kafka Source Connector on Confluent and Redpanda platforms.
    • Installing as Confluent / Kafka Source Connector
    • Installing as Redpanda / Kafka Source Connector
  • Installing older versions.
  • Upgrading from older versions.
  • Upgrading older Windows installations to version 4.2.0 or later.
    • Steps before upgrade.
    • Upgrading Docker container.
    • Upgrading Windows and Ubuntu installations.
    • Steps after upgrade.
    • Versions the upgrade is supported from.
• How to deploy Industrial Data Collector as Azure IoT Edge Module
  • Install Azure IoT Edge runtime in the device.
    • Install prerequisites.
    • Install Azure IoT Edge runtime
    • Provision IoT Edge device with its cloud identity (using symmetric key).
    • Restart IoT Edge runtime.
  • Deploy Industrial Data Collector
• Activate
• Upload Annual Maintenance and Upgrades License.
• Reactivate.
• Upgrade product edition with a new key.
• How to setup configuration GUI endpoint to use https.
  • How to generate and install SSL certificate issued by Let’s Encrypt.
  • Requirements for SSL certificate files generated by third-party tools.
  • Configuring of the instance to use https.
• Configure
  • Application Instances.
  • Configuration files.
  • Configuration steps.
    • 1. Connecting to the Configuration database.
      • 1.1 Connecting to PostgreSQL database in secured mode
    • 2. User authentication and authorization.
      • 2.1. Versions before 4.0.0
      • 2.2. Versions 4.0.0 or newer.
    • 3. Collector Configurations.
    • 4. Time-Series database connection settings.
    • 5. Configuration of application instance.
    • 6. Configuration of the instances as High Availability Cluster Nodes.
• Collect
  • Connections to OPC UA Servers.
  • Advanced connection settings.
    • Periodical reading and logging of variable values.
    • Enabling support for complex type values.
    • Browsing OPC UA Server nodes with large number of child nodes.
  • Connecting in secured mode and certificates.
    • Configure OPC UA servers to trust Industrial Data Collector instance certificate.
    • Configuring idako: Industrial Data Collector to trust to server instance certificates.
  • Connections from Docker container.
  • Connecting to OPC UA Servers running in Docker container, accessed over VPN or NAT or using port forwarding.
  • Connecting to classic OPC DA Servers.
  • Browsing OPC UA Server address space and selecting variables to log.
    • Browsing and selecting of variables manually using web GUI.
    • Auto-discovery and selection of variables
    • Debugging Python scripts
  • Logged Variables table.
    • Columns.
    • Grouping of logged variables
    • Filtering by OPC UA Server.
    • How to set default values for logging options.
    • Display real time data.
    • Other options.
    • Client-side deadbands.
    • Logging values of complex data type.
• Write values to OPC UA Variables
• Time-Series Databases
  • Common settings
    • Store and Forward Settings and statistics.
    • Settings for OPC UA to TSDB mapping.
    • Handling of invalid UTF-8 strings.
  • SQLite
  • Snowflake
  • TimescaleDB/PostgreSQL.
    • Installation.
    • Configuration.
    • Mapping from OPC UA to SQL database, table and columns.
      • Schema of the values table.
      • Accessing Display Name or OPC UA Node Id of variables from time-series database of SQL type.
      • Writing time-series data to the table with name different than default name values.
    • Connection in secured mode
    • Using PostgreSQL instance hosted at Microsoft Azure
    • Using instance of TimescaleDB hosted by Timescale
    • Configuring retention policy in TimescaleDB.
  • InfluxDB version 1.7.
    • Installation.
    • Configuration.
      • Example file for automatic deployments.
      • Configuration using web GUI.
    • Mapping OPC UA to InfluxDb.
      • The measurement part.
      • The tags part
      • Placeholders
      • Spaces in measurement and tag keys.
      • The InfluxDB fields part.
      • Timestamp precision.
    • Logging application status
  • InfluxDB version 2.0.
    • Installation.
    • Configuration.
      • Example configuration file used in case of automatic deployment.
      • Steps to configure InfluxDb2 running in Docker container.
      • Grafana data source plugin for InfluxDB 2.0
    • Mapping OPC UA to InfluxDb.
  • Confluent Platform: Cloud and Enterprise
    • Configuration.
      • Configuration using web GUI.
      • Example file for automatic deployments.
    • Mapping from OPC UA to Confluent Topic, Key names and Partitions. Payload format.
      • Payload
  • Apache Kafka
    • Configuration.
      • Configuration using web GUI.
      • Example file for automatic deployments.
    • Mapping from OPC UA to Kafka Topic, Key names and Partitions. Payload format.
  • Microsoft SQL
  • MySQL
  • MemSQL
  • MQTT
    • Publish Topic Name composing
    • MQTT payload
    • Eclipse Mosquitto
    • Microsoft Azure IoT Hub
    • AWS IoT Broker
    • Google Cloud IoT Core MQTT Bridge.
    • IBM Watson IoT Platform
  • TSDB database access optimization.
• Store
• Visualize
  • InfluxDB 2.0 built-in visualization tools.
  • Visualization with Grafana.
  • Install Grafana
  • Setup Grafana data source plugins to get data via Industrial Data Collector REST API endpoint
    • Installing of the Infinity plugin.
    • Installing of the SimpleJson data source plugin.
  • Using Grafana data sources to read data from databases directly.
  • Setup Grafana dashboards.
    • Create new dashboard.
    • Reading values from Infinity data source plugin.
    • Reading values from SimpleJson data source plugin.
    • Reading values stored in PostgreSQL database.
• Analyze
  • Test use case
  • Step 1: Add server node into Address Space.
  • Step 2. Start logging of values for OPC UA variable node.
  • Step 3. Configure Grafana panel to view data values
  • Step 4. Analyze data visually.
  • Step 5. Open PgAdmin page and connect to the TSDB database.
  • Step 6. Run SQL queries.
  • Step 7. Look at log files for additional information.
  • Step 8. Conclusion.
• Roadmap
• Support
• How to:
  • Modify minimal refresh rate for Grafana
  • How to enable SQL Server Authentication and TCP in Microsoft SQL Server Express
    • Enabling SQL authentication Mode.
    • Enabling TCP connections.
  • Enable horizontal scrolling in the Logged Variables table.
  • Upgrade time-series database from InfluxDB 2.0 Beta version to InfluxDB 2.0.3 released version.
  • How to solve issue with duplicate records in SQLite or PostgreSQL/TimescaleDB.
  • How to install and run InfluxDB.
    • Windows 10
    • Linux (Ubuntu 18.04)
    • Installing SSL certificates from letsencrypt for InfluxDB (Ubuntu)
    • Running InfluxDB as Azure IoT Edge Module
  • How to quickly setup test configuration with 100K variables.
  • How to modify configuration options for large number of variables.
  • Import variables from CSV file
    • Import into SQLite configuration database.
    • Import into PostgreSQL configuration database.
  • How to create and use self-signed OPC UA application instance certificate.
  • How to monitor state of connection with OPC UA Servers
  • How to configure Industrial Data Collector and monitor its state using REST API.
  • How to add OPC UA Server connection settings and logged variables using Python scripts
  • How to connect to Ignition OPC UA Server in secured mode.
  • Connecting when Industrial Data Collector and OPC UA Server run in different network environments.
  • Network issues on Siemens Simatic IoT2050