Solving Archiving Challenges with PI UFL - Ensuring Complete Data Capture

Troubleshooting PI UFL: Understanding and Managing Data Archiving Challenges

In the world of industrial data management, ensuring that all relevant data is accurately captured and archived is crucial for long-term analysis and operational efficiency. One particular tool often used for this task is the PI Universal File and Stream Loading (UFL) interface, part of the OSIsoft PI System lineup. Despite its powerful capabilities, users might run into specific challenges with data archiving, especially when working with inputs like CSV files. Here, we discuss a common issue and share solutions that can help in effectively utilizing the PI UFL interface.

Common Archiving Issue

A frequently encountered problem arises when importing data from CSV files — only a subset of the data is being archived. This issue typically manifests itself when the archive mode is incorrectly configured, leading to only some of the values being stored, while others seem to disappear.

Understanding Archive Modes

The PI System's archive mode determines how data points are stored when multiple values have identical timestamps. By default, settings like ARCREPLACE might cause only the latest value for each timestamp to be retained, replacing earlier values. This might initially seem like a loss of data, when in fact, it's a typical configuration oversight.

Solution: Using ARCAPPENDX

Switching the archive mode to ARCAPPENDX can resolve this issue. This configuration ensures that events are added to the archive irrespective of whether an event with an identical timestamp already exists — devoid of any compression. By changing to ARCAPPENDX, users can see their entire dataset represented accurately in the PI System, capturing complete historical data.

Troubleshooting the 1000 Value Limit

Having resolved the timestamp issue, users might encounter new complications, such as what happens when dealing with larger datasets. A particular case is an inability to archive more than 1000 values per tag.

Possible Investigations:

1. Error Log Examination: Checking the UFL error logs is a crucial step in understanding why the limit is being enforced. These logs might provide specific messages or codes indicating why values beyond the 1000 marker are not processed.
2. Configuration Limits: While not currently known as a factor, older versions of PI UFL had a maximum file size limit that could potentially impact large datasets. Ensuring you are using an updated version and reviewing any configuration files for such limitations should be part of the troubleshooting process.
3. Data Validation: Make sure there is no issue in the CSV formatting or data cleanliness, which might impact how UFL processes the file.

Conclusion

By understanding your system’s archive mode settings and investigating possible limitations, you can overcome common data loss issues in PI UFL. For those facing difficulties, exploring both configurations and error logs will help pinpoint underlying problems, ensuring data integrity and completeness in your PI System.

Deploying these solutions can ensure the reliability and comprehensiveness of industrial data analytics, driving better operational insights and decisions.