While I am waiting here in Istanbul for my delayed flight back home to Zurich, I have time to write a much belated blog post.
Last week, I attended, as a speaker, IBM’s EMEA Academic Days in Frankfurt. The main topic of the event was “Big Data Analytics“. I was particularly intrigued by the presentation of Professor Andy Neely from Cambridge who spoke about “Big Data and Analytics: Changing the Face of Business Performance Measurement“.
His main message was that (big) data and analytics should be used for creating a learning organization rather than controlling the organization. He gave some very intuitive examples from his research where he explained that one of the pitfalls of having a lot of data available and being able to analyze it was that it would be used to construct key performance indicators on how an organization performed and what needed to be managed to meet performance targets.
First, he explained that too many performance metrics actually confused people because it was no longer clear what was really important. Then secondly, that when performance metrics are introduced they may change peoples behaviors in sometimes counter productive ways to satisfy the metrics.
One such example is to measure in call centers the time it takes to resolve a client’s issue. Andy explained, that if the target is 2 minutes many agents will find a reason to hang up or terminate the call in some other way when 1:45 minutes is reached, just to meet the target. Whether the issue has actually been resolved or not becomes immaterial to the agent.
We all know that what one should really care about is doing the right things rather than doing things right.
He gave another example from the airline industry – how fitting since I am waiting here for my flight – on what drives client satisfaction with the airline. The assumption being that clients will fly more often with an airline where they have fond memories of the experience.
So the drivers for client satisfaction are: friendliness of the staff; check-in time; on-time departure; and quality of the food. Yet when he analyzed data collected by British Airways it turned out that on-time departure actually had a negative correlation to passenger satisfaction. The explanation he gave was that when the plane was late, the staff put in a lot extra effort to calm down the passengers and really treat them well, so they would not be unruly during the flight. When the departure was on time, the passengers felt the staff was less friendly. Apparently people remember the staff’s behavior a lot longer than the fact that their plane was late.
So the lesson was – one should use the various sources of data rather to create models, discuss the models and refine them, learn from the data and not use it exclusively to control people’s behavior.
I agree with Andy here. I wonder what you think and what your experiences are with performance metrics.