The Need for Metrics Standardization
by Walter Nijsen, CMRP
Understanding how our plants perform and how well we perform in relation to others often reveals opportunities for improvement, at least in principle. The key question first raised is often, “Are we comparing apples with apples?” If not (as in many cases), the whole exercise of comparison, and to some extent, measurement, becomes somewhat (or completely) meaningless.
On top of that, a question that really should be answered first is, “WHY should we measure?”, along with, “WHAT should be measured and HOW?”
The measures we believe are truly important are often referred to as Key Performance Indicators (KPI’s), since, apparently, as the wording implies, those contain key information on performance. But does it, and if so, what precisely is it indicating?
When measuring true performance, a number of questions and preliminary steps need to be taken first:
• Which KPI’s are useful at what stage?
• Is this a leading or a lagging indicator?
• What is the correct definition?
• How will we interpret the results?
• How will you benchmark KPI’s?
Why Should We Measure?
Joseph Juran famously said, “If you don’t measure it, you can’t manage it.”
Ron Moore said, “Your measurements should expose your weaknesses – those are your improvement opportunities.”
When asking this question to several persons in an organization, you will typically get different answers.
An operational leader or business leader could answer: “to measure our profit and losses, or to understand if we are achieving our goals”. A reliability improvement leader could answer: “to identify opportunities for improvement, or to measure the improvement progress”.
Both answers are correct and make sense, depending on your role and interests, because you want to measure and trend the results or the improvements at your facility.
“To compare and benchmark between industries or within the company” is also an expected answer.
On top of this, there is another important, and mostly forgotten, or at least not identified, reason why we should measure: “To share success, which encourages changes and improvements”.
To achieve reliability excellence many changes and improvements need to be made. Some are easy, and some are more difficult, but sharing success will help drive forward these changes. Benchmarking at a facility level, company level or industry level is a part of sharing those successes.
What Should We Measure?
If the “Why should we measure question?” is clear and understood, the answer to “What should we measure?” is simple. Let’s focus only on the maintenance and reliability process. At the end of the day, the financial results, product quality and availability will determine your profit and losses and your business growth. So KPI’s such as, OEE, maintenance cost as a percent of replacement asset value, quality index, on time delivery, production cost per unit produced need to be in place.
However, these indicators are lagging indicators, or results indicators, which give a snapshot or update for the moment, but will not tell you what the future results will be, nor if these results are sustainable.
Further, many persons or processes can influence these KPI’s. For example, maintenance cost is influenced by many things, e.g., amount of unplanned breakdowns, amount of pro-active work executed, quality of the executed work, efficiency of the executed work, etc. Therefore, it is important to also implement KPI’s, which tell you something about your potential performance in the future, or so-called leading or process indicators. These indicators are typically used to measure the process improvements that bring us to our new goals.
The leading indicators should show us the direction of future results, or in other words, the leading indicators will tell us if the lagging indicators will get better or worse.
There are three sets of measurable components that make up the maintenance and reliability process at Cargill (See Figure 1: Components of the Maintenance and Reliability Process).
• Behaviors and management processes (people skills, mission and vision)
• Operational execution (operations, design and maintenance)
• Manufacturing performance (availability, quality, cost and benefits)
Each component is a process on its own, which can be measured using both leading and lagging indicators. To determine the quality of each process, the results of each process need to be measured using lagging indicators. To assure good results, we must have good leading indicators – if you do the right things, the right things will happen for the business.
The components of the maintenance and reliability process can also be explained as:
approach, deployment and results.
Manufacturing performance is a (end) result of the (correct) deployment of operational execution. Operational execution is, in part, the deployment of maintenance planning and scheduling, defect elimination, predictive and preventive measuring and follow up.
To understand if these manufacturing performances (results) are sustainable, it is important not only to measure the deployment (operational execution) but also the approach (behaviors and management process).
Without having a clearly defined approach, the result can be, and often times is, based on individuals deploying to their best effort, but without any vision and strategy for the future.
In this context, and as a supply chain, the components of the maintenance and reliability process are both leading and lagging indicators depending on where in the process the indicators are being used.
This simplified view of leading and lagging measures betrays the full value the definition can have. There is a cause and effect relationship between leading and lagging; the action being measured will cause a resulting action or effect, which is also being measured.
This means that a given measure could be both a lagging measure for a previous cause in the chain, and a leading measure for a following effect. There are a series of causes and effects in the chain until the final lagging measures are reached.
The Leading and Lagging Indicator Mapping in Figure 2 shows the concept of an indicator being both leading and lagging.
Preventive Maintenance (PM) Compliance is a lagging indicator, or a result of how much PM work is completed when viewed in the context of work execution.
However, when viewed as an indicator of equipment reliability, PM compliance is a leading indicator of the reliability process. The better or higher an organization’s PM compliance, the more likely this will lead to or predict improved equipment reliability. Similarly, improved equipment reliability will lead to reduced Maintenance Costs, which is a lagging indicator of the maintenance process.
Before applying and implementing leading and lagging indictors, the maturity of the facility or company needs to be understood.
For example, if you are in a transition stage from reactive to pro-active, then KPI’s like training compliance and percent pro-active work completed are more applicable to use than inventory turns increasing, or maintenance rework reducing, which are typically results of a more mature reliability process.
To implement which KPI at which particular moment is unique for each business or location.
Some ground rules need to be considered:
- Focus on leading and lagging indicators for each reliability process component, see Figure 1
- Provide clear definitions and examples
- Assess the operational readiness to implement the KPI.
How Should We Measure?
When implementing key performance indicators and setting goals, it is a natural human behavior to produce the results you are aiming for. This induces a natural bias to get the results you are targeting. For example, at our company we are measuring percent pro-active work completed, and we are aiming to achieve 80% pro-active work.
Clear definitions with examples are set and the unit of measure is hours. When starting the measurement, plants were only asked to report the percent pro-active work done, and within several months, almost all plants were achieving this number, even though we knew it could not be possible considering the maturity stage of some plants. A thorough review showed that not all plants used “work hours complete” as a unit of measure; that is, some used number of work orders complete, some used actual cost, some included contractors, and some not.
So, we made a change in the reporting. Instead of asking plants to report the percent pro-active work completed, we asked them to report actual hours spent on pro-active work and the total actual hours worked.
This changed the results completely; some plants captured only 60% of the total hours, which were typically the hours spent on proactive work. The hours on reactive work were not included in the total hours show, and thus a much higher percentage of proactive work was reported done than actual.
Plants not capturing hours at all actually failed to report anything.
Lessons learned on how we should measure are:
- Provide clear definitions and examples, in multiple languages if applicable
- Understand the unit of measure and report the raw data, not the end results
- Use uniform reporting systems
Benchmarking and Standardization
Cargill is a leading company in the food industry with over 1500 locations in more than 80 countries. Comparing and benchmarking within the company and with other industries is a challenge.
During the last 10 years, a major change has been made within the company, from a focus on traditional lagging indicators to more leading indicators.
Cargill has learned a great deal in the last 10 years, and many others can learn and benefit from both these lessons learned and from Cargill’s experience. In fact, these lessons learned drive the process of standardization of key performance indicators.
The Society for Maintenance and Reliability Professionals (www.smrp.org) is a group “by practitioners for practitioners”, who in the last year, has developed standardized Maintenance and Reliability Key performance indicators. Each of these indicators have a clear definition, objective, formula, component definition, qualification and sample calculation developed by experts from several industries world wide, validated and evaluated by practitioners, and are ready for use.
A total of 77 key performance indicators are identified and under development, 21 KPI’s are finished and will be published soon by SMRP. Figure 3 provides several examples of the key performances indicators from the body of knowledge (BoK), including the reliability process component upon which they can be used.
Worldwide adoption of these metrics will benefit Cargill, but also all other industries. It will also create transparency and unique benchmarking opportunities within the maintenance and reliability industry.
Walter Nijsen, CMRP, holds the position of Asst M&R Leader for Cargill grain and oilseeds in Europe. Based in the Netherlands, he is responsible for developing and implementing the maintenance and reliability strategy for about 40 locations across West and East Europe
. During the last 5 years Walter has implemented maintenance best practices, and has been instrumental in building the overall reliability culture and vision for Cargill worldwide by actively participating in maintenance steering committees, conferences, facilitating trainings, developing systems and procedures. He holds a degree in Chemical Engineering and joined Cargill in 1995. He is certified in several predictive technologies, is a certified maintenance and reliability professional since 2003. He is an active member of the SMRP Best Practice Committee.
Comments