We Improve What We Measure
WRONG!
I was accosted yesterday by a well-meaning manager who admonished me to “make sure the metrics are the right ones!” He went on to remind me that “we improve what we measure.” I tried (and failed) to explain to him that the adage is wrong…close, but wrong. We actually improve the measures we measure. In other words, because we focus on the “score,” “rating,” or “grade” we obtain in the metric – and not on the underlying effort, process, or activity we’re trying to improve - we are only assured of improving that score. I call this “chasing data.” Rather than improving the underlying driver for the creation of the metric - we end up trying to improve the numbers. A simpler way to look at metrics are that they are NOT used to improve anything. They’re indicators. They provide insight and visibility which in turn can be used to improve things.
A friend told me a story about an innocent example. The call center was trying to improve the way they did business. They wanted to enhance the customer experience. So among other things, they began measuring the length of help calls - with the idea that the shorter the call, the more calls the analyst could answer, and the more customers he could assist. And they only focused on those that repeatedly had “longer than average” calls. My friend was the biggest culprit. His calls lasted five times longer than the rest of his coworkers! The boss called him on it. She wanted his numbers to improve!
What she forgot was why she was collecting the data. She also forgot that the measures only provided insight, not answers. She could have asked why he was so much slower. If she investigated (instead of reacting - or “managing using the data”) she would have found that because he was the most knowledgeable and experienced, all of the difficult calls were being forwarded to him. His co-workers were passing all of the most difficult, and thereby the most time consuming, cases to him.
The measure was fine. The use of it to “manage” was not.
In this case, the measure could be used, as an aggregate - not on an individual level. The average time to resolve trouble calls could be a very good indicator for the call center as a whole. The mistake was in believing that the measure told a full story and worse - that it was an answer.
Managers manage people - managers don’t manage metrics. If we could manage by metrics, we’d not need people with people skills as managers - we could use computers.