Managing time-series data with data historians and advanced analytics software

What Is a Data Historian and How Is It Used?

Historians continue to get attention from vendors and customers new and old. For recent interest, there is the focus on time series data storage from cloud vendors like Microsoft and Azure, plus open source offerings funded by Silicon Valley venture funds. For the established market, there are articles on the importance of insights from historians, for example, Control Magazine’s article last spring. Since all of our business at Seeq involves time-series data stored in a historian – on-premise or in the cloud – we get a lot of questions about historians and wanted to pull a summary together for reference.

Historians by any name––data historians, process historians, operational historians or enterprise historians– are at least simple in what they do. Historians are database applications for storing time-series data, i.e., storing measurement samples over a period of time. In the early days – the 1980s into the 2000s – you could build your historian, typically on a SQL database, or buy one from any number of vendors. This buy vs. build discussion was a key competitive point for many historian vendors. You can still find white papers on the web from data historian vendors about why a dedicated historian product is a better option than building your own for example, “10 Advantages of a Process Historian vs. SQL Server”.

The first historian was Oil System Incorporated’s plant information system ─ later renamed OSIsoft PI System ─ which ran on DEC VAX/VMS minicomputers ─ the manufacturing system of choice in the 1980s. Over time every vendor of a DCS, HMI, or SCADA system either licensed the OSIsoft PI System (like Rockwell FactoryTalk, an OSIsoft licensee) or introduced their own, usually after an acquisition. GE Fanuc acquired Mountain Systems in 2003 (now GE Proficy), the early days of WonderWare in 1987 (now owned by AVEVA), Honeywell’s 2007 acquisition of InterPlant (now Professional Historian Database, or PHD), AspenTech’s InfoPlus 21, and too many others to count.

One of the challenges of the current historian market is trying to size it. Data historian vendors like Siemens, ABB, and AVEVA have multiple offerings from a history of acquisitions. AVEVA alone has WonderWare, eDNA, and Citec historians, for example. Other vendors include a historian as an optional product feature, such as SCADA vendor Inductive Automation’s Ignition system.

So, how many historian products are out there today? Only counting commercial offerings, a conservative estimate is at least in the dozens, certainly over 30. These include products you might not know unless you’ve been to Norway (erPrediktor), Australia (Onyx), or Ireland (AutomSoft), or if you’re a student of SCADA systems like Cygnet, Ellipse, and CopaDATA.

But why have a data historian, what's the benefit?

The answer is historians enable access to all the sensor data in your plant, which is invaluable for any number of analytics, investigation, and reporting requirements. Typical use cases include:

• Visibility to any asset or sensor in the plant
• Diagnostics or troubleshooting (what went bump in the night)
• Reporting to meet regulatory requirements
• Investigation of opportunities to improve outcomes (quality, yield, etc.)
• Monitoring or alerts based on set values
• Tracking downtime of assets or lines
• Production accounting
• Statistics on sensor values
• Optimization of resources – energy, water, etc

Over time, historians went from a build vs. buy decision to a “buying with” and “buying after” decision. Buying with means the inclusion of the historian with the DCS, HMI or SCADA system purchase, and buying after implies the acquisition of a historian separate from the control system, typically for better enterprise-class capabilities (scalability, reliability, etc.) and greater platform functionality. The leader in the “buy separate” scenario is OSIsoft which offers not just a historian, but a full suite of platform features for notifications, asset analytics, visualization and the broadest range of connectors to different PLC and control systems.

So that is where we are today with historians, but the world is moving ahead. Customers want more insights, faster from their data. Instead of just diagnostic (historical) and descriptive (reporting) analytics, they want predictive analytics and more value more quickly. They want to run their company on analytics, and they are under pressure to find valuable insights, for example, McKinsey’s article on the buried treasure in process data. Furthermore, the increase in data volumes driven by more sensors, wireless technologies, and IIoT, has moved data storage and analytics from important to imperative for many executives.

Stop copying and pasting data - start gaining data insights

As a result, there is a sense that historians are due for a refresh to enable a new class of analytics and insights. The issue, however, isn’t the historian itself, it’s the lack of innovation in the analytics applications that work with historian data. Note that every historian – EVERY historian – comes with at least two applications: a trending tool for visualization, often called “Insight,” and an Excel connector for analytics. Historians and spreadsheets have had a similar lifetime arc over the last 30+ years from invention to adoption to widespread use. As a result, spreadsheets are the overwhelming tool of choice for process engineers working with time-series data. Slow, underpowered, and lacking innovation? Yes. But at the same time, flexible, accessible, and known. But it’s 2019 and copying data into Excel for cleansing, calculating, contextualizing, and finding insights. There must be a better way as I have written previously.

How does advanced analytics software work with data historians?

Seeq is an advanced analytics application which enables self-service insights for process engineers. Seeq’s application includes publishing, knowledge capture, collaboration, and many other features and works with all leading historians. Whether you have 17 PI servers or 5 different historians from a history of acquisitions Seeq works with them all, so your past investments are made better with Seeq. Seeq is available as a SaaS application and can work with data in the cloud, so it supports your roadmap for data if it includes data aggregation or analytics beyond your plant.

So that is an introduction into historians. There is much more of course in the details of historian performance – compression algorithms, security, enterprise vs. plant historians, read and write performance – but the summary is they are the past, present and future starting point for all plant-based analytics. And spreadsheets, not the historians, are what’s blocking faster and better insights for production outcomes.