How ERP systems generate so much data

Those of us who “live and breathe” ERP systems are flooded with information about orders and transactions. There are transactions for shipments, receipts, labor, subcontracting, production orders, kanbans, purchase orders, sales orders, transfers, picking, packing and moving. And we have all kinds of codes and parameters to further define those orders and transactions, such as defect codes, error codes, good quantities, bad quantities and hold codes. Every single order and transaction is stamped with a date and time and who generated it. It’s no surprise then that the data stored in ERP databases is overwhelmingly filled with transaction history, not master data like items, customers or suppliers.

IBM has estimated that of the 2.5 quintillion bytes of data in the world today (that’s 18 zeroes folks), 90% was created in the last 2 years.

With the digital revolution, ERP systems will incorporate even MORE data!

The digital revolution is well underway with smart products, smart processes, smart machines and even smart inventory. Each point can capture every change of status and send it to massive data stores in the cloud. The connected world is upon us. It’s not just hype surrounding the “Internet of Things.”  IBM has estimated that of the 2.5 quintillion bytes of data in the world today (that’s 18 zeroes folks), 90% was created in the last 2 years.

We are awash in data from our formal ERP orders and transactions and from products and production equipment on the shop floor and now even connected to the internet and big data sources.

But…where is the insight?

A popular study conducted by Sidney Yoshida describes the “iceberg of ignorance” that inevitably occurs in an organization. That study found that executives were aware of only 4% of the problems known in the organization. Only 9% are known to managers and 74% are known to team leaders, but 100% of the problems are known to the workers in each process. And most of the time, the workers know WHY the problem exists and often times may have solutions.

But workers are paid to produce and they are singularly focused on production especially when every movement is time-studied and their actual work is constantly being compared to standards or to others on the team. Suggestion systems are great but for real insight we need to be much more focused.

Combining quantitative and qualitative data

What if we could combine the quantitative data from our ERP systems and IoT with qualitative feedback from operators on what the problems were and what solutions might be tried?   Maybe we would actually be able to learn from our data-rich environments.

Here’s what we can do now quantitatively:

  • Sift through massive amounts of “data”
  • Organize that data into what seem to be “facts”
  • Apply “models” to fit the facts
  • Project “trends” using the models
  • Isolate the “exceptions” to the trend

These data points can help us to optimize “what is. ” But we need people’s creativity and problem solving to break through to “what could be. ”

The Voice of the Operator

The leading edge of where we are going is in an idea we call “The Voice of the Operator. ” The goal is to enable continuous improvement by merging quantitative results with qualitative feedback and suggestions from the people who actually work in the process. It’s got the technology bits of cool stuff we don’t really need to know about (Data Lakes, Predictive Analytics, Natural Language Query) but in the end, we want to be able to ask questions of the data including operator feedback and then launch kaizen events for improvement.

Where’s the real problem? How do we fix it?

Intriguing data relationships don’t necessarily mean we understand the root cause. But they do help us focus on a particular problem and then conduct tests that will tell us whether we have the problem corrected or not. Root causes require insight and sometimes that insight can only come from people.

One of my favorite stories took place in a clothing manufacturing plant. The data clearly showed that the second shift operations had a much higher defect rate than the first shift. Everyone’s immediate response is that there was a training problem or poor management. In reality, the root cause was that the first shift guys were cherry-picking the easy jobs and leave the hard stuff for the second shift.

That’s the kind of insight that only comes from people.

Written By:  Phil Coy



Adam Jenkins

Author Adam Jenkins

More posts by Adam Jenkins