I’ll save you from the suspense: the “It” you are probably doing wrong is metrics. Specifically, performance metrics, by which I mean the measurements you put in place to guide or change people’s behaviour or decision making. A decade ago I gave a presentation at an international conference titled “What Gets Measured Gets Done”, and I have written about metrics before in this space, so you may have realised that I fancy myself a bit of an expert. Don’t believe me? Google “Lewis’s First Law of Metrics”. In case you don’t go to the effort, it states You get what you measure – that’s the risk you take. I should probably say that phrasing isn’t mine, it belongs to Bob Lewis (no relation) but is remarkably similar to ‘what gets measured gets done’. Bob also has a corollary – If you mismeasure, you mismanage – which essentially says that doing it wrong will cost you.

How do you know if you are doing it wrong? Ah, there’s the crux of the matter. Let’s look at some examples and see if we find some patterns.

Cost

I once was a Director of Lean for a division of a Fortune 500 company. Before I started the head of purchasing had negotiated price rebates from our yarn suppliers based on quarterly usage. The more we bought, the less it cost. If we took those yarn spools and aggregated them into what are called warp beams (a giant spool with a thousand threads wound around it), the accounting software changed the value of that yarn upward by 10% from the original negotiated price, 20% if we include the rebate. It was like printing our own money!

I know what you are thinking - Yeah, yeah, inventory is bad, blah, blah, blah. Not so fast. Did you think that holding costs and extra inventory was the only consequence? I haven’t even started yet. As Director of Lean I initiated some operating changes. Turns out we had eight months of inventory for a product that was going to end in five. Oops, write off $250,000. One of my kaizen teams worked out that we actually only need 3 days of completed warp beams (not eight months), so we began to empty the stock and stopped ordering yarn. Oh no, our rebates stopped coming from the yarn supplier! Without them, our finances look funny and we won’t have consecutive quarters of profit! Oh no, that means the CEO will miss out on his bonus! We better “pull ahead” next quarter’s “rebates” from our yarn supplier. In case you were wondering, “pulling ahead” “rebates” is not only impossible from the point of view of the meaning of those two phrases, but is illegal as well. The results? I was soon fired for pointing out the problem with pulling ahead rebates. Seven months later the US Securities and Exchange Commission found out about the “pulled ahead rebates”. Nine months later the Fortune 500 company was bankrupt, and the CEO was headed to prison.

The ultimate problem here is that the purchasing manager was measured by quarterly cost savings, the director of manufacturing and the CEO were both measured by quarterly profits. No thought at all was given to the long-term health of the business!

Quality

I worked at an automaker just after getting my master’s degree. Despite the fact that the plant I worked in was under threat of closure, we were determined to get our car to win the “JD Power Best In Class In Initial Quality” award, an award no vehicle from our company had ever won.  This award seemed to be owned by Toyota based on the number of years they had won it, and the only reason our car was in the top ten was that there were only ten competitor vehicles! My team was tasked with eliminating defects that had been reported in the JD Power surveys, and one particular problem that stood out was water leaks into the boot. One only had to go out in the shipping yard after a hard rain to find the evidence – leaks so bad sometimes there was standing water where the spare tire was.

The root cause was twofold. First, two sheet metal parts that supported the trunk seal in an area called the trough overlapped in the wrong way, channelling water between them rather than cascading it over. The second problem was that the design had called for this gap to be filled with a brushed-on paintable rubber, but the marketing folks said that the application looked amateurish and diminished the perceived quality. This is a part that you can only see with the deck-lid open, and it is an area that is going to collect dirt on most cars anyway, but engineering ordered the brushed on sealant replaced with an expanding foam welded between the parts. This foam failed almost 7 % of the time, and cost about ten times as much.

The problem here was that a subjective quality measure – does it look good to marketing guys – was used to make decisions that directly impacted objective quality measures – does the car leak? No customer had ever complained about the sealant or how it looked before (nor did they the entire time I was there), and in fact it was obvious and prevalent in the Toyota car on and near the same place. Did anyone track the number of water leaks after the initial change was made? No, 20 cars were tested and they were fine, so no further effort was put forward.

We went back to the sealant without consulting marketing, increased the number of cars going through the water test by 50%, made a heap of other improvements, and ended the year by winning the JD Power award, the first non-Japanese winner for more than a decade.

Delivery

I worked for a company that had an unbelievable track record for on-time delivery. 100% on time for  years. We made about 100,000 units a year, and we had this stat on our catalogue (yes, that long ago!) “More than a million on time shipments!” in a little bubble meant to inspire confidence.  The truth was we had an eight week lead time; we survived by inventory. For “stock” items, we’d ship it in three days. For special orders we told them it would take eight weeks, and then we acted like heroes when we shipped it in four. We were great at telling the customers when we would ship and then beating that time, but what did the customer’s want?

First I must say that shipping on time is a QUALITY metric, not a delivery metric. Second, shipping when we can is not the same as shipping when the customer wants it. The company’s new owners had a different idea, and a different measure. They wanted us to have zero inventory in finished goods each day. That meant we had to build to order. That meant that our lead time from order to shipping had to be less than three days! Lead time became our delivery metric. Two years later, you could order anything ‘standard’ before 1:00 pm and we would build it and ship to you the same day, zero finished goods inventory. Custom orders had a target of three days, mainly because it sometimes took a day to get special materials delivered. Our sales increased by 10%.

That’s not the end, though. A large industrial supply catalogue competed with us on the most common models, and they made their product in Mexico and shipped from several warehouses in the US. Every five years they put production of those models out to tender, and every time we would bid and lose. This time we bid, but also said “we’ll even pack them in your cardboard boxes and ship them from our production facility so you don’t need any inventory”. We still cost more than the competition, but we won the contract, which amounted to a 25% increase in sales each month. The 10% increase in our sales and the 25% increase above that from the new contract was the ‘cost’ of measuring the wrong thing in the wrong way.

What does it all mean?

These cases are all connected, and I would go on until you could see the pattern, but who has time to read pages and pages of this? In each case, a target and a measure was put in place with disregard to unintended results and the ultimate goal. Don’t talk to me about ‘balanced scorecards’ being the solution – every example above had balanced scorecard systems in place. Adding more metrics is not the answer – we all intuitively know that if you measure people on too many things they pay attention to none of them.

Time spent developing the RIGHT metric, even if it is complicated to gather data for or to calculate, will prevent problems like these from coming back to bite you. Focusing on profit (actually ‘paper profit’ through clever financial shenanigans) sent the first company to bankruptcy; a similar focus on financial Throughput (Sales less Material costs and Outside Services) would have kept the company in business, and most likely in profit for each quarter. Focusing on the customer’s objective quality experiences with the car led us to win a prestigious industry award, and turned a plant around from near closure (they are still open today). Focusing on lead time instead of being on-time transformed the last business into a super-competitive, customer centric company with incredibly high margins (35%).

Every company I visit seems to be doing some type of metric wrong; it is possible yours is not, but in my opinion that is unlikely. You have to identify the goal and all unintended consequences for each metric; being aware that they exist is the first step toward eliminating them.

What has been your experience with people or companies picking the wrong metric? Post yours in a comment so we can all learn from your experience!