Don’t Let Metrics Undermine Your Business
Idea in Brief
The Problem
Companies that work hard on their strategies and carefully monitor their progress often run into spectacular trouble.
Why It Happens
People have a behavioral tendency—known as surrogation—to confuse what’s being measured with the metric being used.
How to Fix It
To reduce the risk of surrogation, make sure that the people executing your strategy had a role in formulating it, don’t link incentives too tightly to strategy metrics, and use multiple metrics to assess performance.
Tying performance metrics to strategy has become an accepted best practice over the past few decades. Strategy is abstract by definition, but metrics give strategy form, allowing our minds to grasp it more readily. With metrics, Ford Motor Company’s onetime strategy “Quality is job one” could be translated into Six Sigma performance standards. Apple’s “Think different” and Samsung’s “Create the future” could be linked to the amount of sales from new products. If strategy is the blueprint for building an organization, metrics are the concrete, wood, drywall, and bricks.
But there’s a hidden trap in this organizational architecture: A company can easily lose sight of its strategy and instead focus strictly on the metrics that are meant to represent it. For an extreme example of this problem, look to Wells Fargo, where employees opened 3.5 million deposit and credit card accounts without customers’ consent in an effort to implement its now-infamous “cross-selling” strategy.
The costs from that debacle were enormous, and the bank has yet to see the end of the financial carnage. In addition to paying initial fines ($185 million), reimbursing customers for fees ($6.1 million), and eventually settling a class-action lawsuit to cover damages as far back as 2002 ($142 million), Wells Fargo has faced strong headwinds in attracting new retail customers. In April 2017, it reported that first-quarter credit card applications were down 42% year over year, with new checking-account openings down 35%. Meanwhile, more revelations about unauthorized mortgage modifications and fees, improper auto loan practices, and other missteps surfaced throughout 2017. In the fourth quarter the bank had to set aside a $3.25 billion accrual for future litigation expenses. In February 2018 the Federal Reserve prohibited Wells Fargo from growing its assets any further until it strengthened its governance and risk management. This was followed in April by a joint $1 billion fine from the Consumer Financial Protection Bureau (CFPB) and the Office of the Comptroller of the Currency (OCC), which led Wells Fargo to increase its litigation accrual by $800 million. While press releases from the CFPB and the OCC tie the agencies’ action only to mortgage fees and auto loan problems, the political context suggests that the penalty’s severity stems in part from public outrage over the original fake-accounts scandal. In the face of the bank’s prolonged difficulties, the CEO who’d taken the helm after the scandal, Timothy Sloan, resigned in March 2019.
Were these devastating outcomes simply the natural consequences of having a bad strategy? Closer examination suggests that Wells Fargo never actually had a cross-selling strategy. It had a cross-selling metric. In its third quarter 2016 earnings report, the bank mentions an effort to “best align our cross-sell metric with our strategic focus of long-term retail banking relationships” [emphasis added]. In other words, Wells Fargo had—and still has—a strategy of building long-term customer relationships, and management intended to track the degree to which it was accomplishing that goal by measuring cross-selling. With brutal irony, a focus on the metric unraveled many of the bank’s valuable long-term relationships.
Every day, across almost every organization, strategy is being hijacked by numbers, just as it was at Wells Fargo. It turns out that the tendency to mentally replace strategy with metrics—called surrogation—is quite pervasive. And it can destroy company value.
The Surrogation Snare
Of course, we all know that metrics are inherently imperfect at some level. In business the intent behind metrics is usually to capture some underlying intangible goal—and they almost always fail to do this as well as we would like. Your performance management system is full of metrics that are flawed proxies for what you care about.
Here’s a common scenario: A company selects “delighting the customer” as a strategic objective and decides to track progress on it using customer survey scores. The surveys do tell managers something about how well the firm is pleasing customers, but somehow employees start thinking the strategy is to maximize survey scores, rather than to deliver a great customer experience.
It’s easy to see how this could quickly become a problem, because there are plenty of ways to boost scores while actually displeasing customers. For example, what happened the last time you were urged to rate your experience a 10 on a satisfaction survey “because anything but a 10 is considered a failure”? That request may have turned negative feedback into a nonresponse or an artificially high score, and the pressure was probably off-putting. And think about all the pop-up windows, follow-up emails, and robocalls that pester you with surveys you would rather ignore. Such tactics tend to lower a customer’s satisfaction with a company, but surrogation can lead those charged with delighting the customer to use them despite the strategy.
Mike McQuadeSurrogation is especially harmful when the metric and the strategy are poorly aligned. The greater the mismatch, the larger the potential damage. When a production manager’s success at achieving the strategic objective “make high-quality products” is measured by using very precise quality standards (such as “ball bearings must be 10 millimeters in diameter, plus or minus 0.0001 millimeters”), surrogation might not be a problem. However, if success at the objective is measured by the number of customer returns, the production manager might find creative ways to avoid returns. For example, he or she might connect directly with the purchasing departments of clientele, offering to personally handle any product concerns so that returns are registered as rework rather than returns. Or the manager might be willing to gamble a bit, pushing beyond acceptable (or even safe) quality standards, knowing that while the lower quality will increase the likelihood of a return, it may not actually trigger one. Furthermore, when a single metric is used more widely—for example, to gauge the performance of multiple managers overseeing various components of a complex product—surrogation can have a far bigger impact and do much greater harm.
What Happened at Wells Fargo
Several explanations have been provided for how things went awry at Wells Fargo. The most widely accepted theory lays the blame on the company’s incentive system. In the words of Richard Cordray, the former CFPB director involved in imposing an early fine on the bank: “What happened here…is that Wells Fargo built an incentive-compensation program that made it possible for its employees to pursue underhanded sales practices.”
But was the compensation approach actually the root of Wells Fargo’s problems—or was it simply a symptom of a more insidious ailment? Another culprit might have been the combination of challenging sales quotas and relentless pressure to meet them. Indeed, employees under investigation cited pressure more often than incentives as a cause for misconduct. Another possible cause was a permissive sales culture. A key finding of an internal investigation was that management espoused the philosophy that “it was acceptable to sell 10 low-quality accounts to realize one good one.” The investigation found that managers referred to products that the customer did not need (or want) as “slippage” and that a certain amount of slippage was deemed “the cost of doing business in any retail environment.” But again, sales pressure and questionable culture could merely have been symptoms of a more pervasive and pernicious problem.
Incentives, pressure to meet quotas, and sales culture were all tied to a system employed throughout Wells Fargo at the time. In fact, it’s one found at almost every company. It’s the performance measurement system, used to monitor everyday business activities, from the organizational level on down to the individual-employee level. There could be no sales incentives at Wells Fargo without rigorous tracking of sales numbers. There would have been no accounts-per-household goals, pressure to meet them, or culture surrounding them if customers’ accounts were never counted. Ex-CEO John Stumpf’s now-infamous mantra, “Eight is great” (the goal was to have eight Wells Fargo products per customer), was based on this common denominator.
The mental tendency to replace strategy with metrics can destroy company value.
The real source of Wells Fargo’s problems was measurement. When the bank decided to actively track daily cross-sales numbers, employees rationally responded by working to maximize them. Throw in financial incentives, a permissive culture, and intense demands for performance, and they might even illegally open some unauthorized accounts, all in the name of advancing the “strategy” of cross-selling.
Don’t get us wrong. We’re not suggesting that measurement is a bad thing. It’s not, and there’s a reason it’s ubiquitous in business: It’s the only way we can make sense of our environment, our results, and our strategic objectives, which we must do if we are to succeed. Metrics provide clearly defined direction where strategy may otherwise seem too amorphous to have an impact. Because they can coordinate behaviors and actions, metrics are crucial. But as the Wells Fargo case shows, unless the inherent distortions of metrics are understood, they can be dangerous—and the distortions can be amplified precisely because the flawed metrics coordinate behaviors.
Guarding Against Surrogation
To prevent surrogation, we must first understand how it happens. Two recent studies on surrogation—one using fMRI machines to measure blood flow in the brain to better understand how people make decisions, and the other using video games to examine surrogation in a nonbusiness setting—suggest that surrogation is a common subconscious bias: Whenever metrics are present, people tend to surrogate. Nobel prize winner Daniel Kahneman and Yale professor Shane Frederick postulate that three conditions are necessary to produce the type of substitution we see with surrogation:
- The objective or strategy is fairly abstract.
- The metric of the strategy is concrete and conspicuous.
- The employee accepts, at least subconsciously, the substitution of the metric for the strategy.
Multiple research studies have helped demonstrate how these conditions combine to produce surrogation. Knowledge of them supplies us with the means to combat the problem. Just as fire is stifled when the heat, fuel, or oxygen necessary for combustion is removed, surrogation can be suppressed by cutting off one or more of its key ingredients. Here’s how to do that:
Get the people responsible for implementing strategy to help formulate it.
This helps reduce surrogation because those involved in executing the strategy will then be better able to grasp it, despite its abstract nature—and to avoid replacing it with metrics. It’s particularly crucial to bring the executives and senior managers who are charged with communicating strategy into this process. Research that one of us, Bill, did with Willie Choi of the University of Wisconsin and Gary Hecht of the University of Illinois, Urbana-Champaign, suggests that simply talking about strategy with people is not sufficient. In other words you can’t just invite them to boardroom briefings and hang signs around the building promoting the strategy—you need to involve people in its development.
Consider the experiences of one organization Bill advised, Intermountain Healthcare. Its goal is to provide high-quality, low-cost care. One of the battlegrounds for this type of “value-based care” is the treatment of lower back pain. It turns out that most lower back pain goes away on its own in a few weeks. Medication and surgery can help, but they can also hurt—and they can be very costly. The data suggests that once a patient presents with lower back pain, the ideal response is to wait. So, with the involvement and advice of practicing physicians, Intermountain recently formulated a strategy aimed at reducing unnecessary interventions. To measure performance on the strategy, Intermountain began tracking whether doctors waited at least four weeks after meeting with a patient with lower back pain to recommend an X-ray, MRI, or another, more invasive diagnosis or treatment method.
If you stop to think about it, the surrogation trap is everywhere. Even the most common performance metric—earnings—was conceived as a proxy for something bigger and more abstract: Hicksian income.
Don’t remember learning about Hicksian income in your accounting class? That’s probably because you didn’t take accounting in the mid-1900s, when our ideas about measuring value began to congeal. John Hicks described this fairly abstract metric as the amount of money that can be distributed to shareholders while still leaving the company’s value unchanged. In contemporary terms, it’s the value added by a company’s operations. Earnings were a clearly defined proxy for that value and were intended to make an abstract concept more concrete. But generating accounting earnings isn’t necessarily the same thing as creating value. After all, financial statements don’t present a complete picture of what’s happening in companies, especially if the numbers in them are manipulated. And investors who surrogate may support financial decisions that do not create value, such as cost cuts that undermine customer service and long-term financial performance.
While there is no formal evidence that investors surrogate, the outsize market reactions we see when a company misses an earnings target by a penny suggest that many are falling into the trap.
The danger with this metric, of course, is that doctors could begin to see “make patients wait” as the objective rather than providing high-quality care at low cost. But because Intermountain doctors helped develop the strategy, this type of surrogation was far less likely to happen. And because the physicians were also heavily involved in the rollout and training for the strategy and its metrics, they could help others avoid surrogation as well. Indeed, Nick Bassett, executive director of population health at Intermountain, says that “without question, when physicians are involved in designing objectives, they better understand those objectives, and when they understand the objectives, they have proven time and time again their ability to determine the right course of action, often in spite of a particular metric.”
Brett Muse, a doctor at Intermountain who played a large part in the strategy’s development and rollout, agrees. “When I get in front of physicians and throw data at them, they get glassy-eyed,” he says. Instead, he gets in front of the group and says, “‘Here’s a problem involving quality of care. Let’s try to solve this problem—and by the way, here’s some data we can look at to see how we’re doing.’”
Loosen the link between metrics and incentives.
Tying compensation to a metric-based target tends to increase surrogation—an unfortunate side effect of pay for performance. Besides tapping into any monetary motivations people might have, this approach makes the metric much more visible, which means employees are more likely to focus on it at the expense of the strategy.
To think about how to get around this problem, let’s look again at Intermountain’s lower-back-pain metric. If management had done the obvious and just informed physicians that they would be paid a small bonus each time they required a patient to wait four weeks before receiving any costly tests or treatments, it probably would have driven even the most well-meaning doctors away from the true strategy of reducing unnecessary interventions and toward maximization of the metric. But the people overseeing the program didn’t tie compensation to the metric, because they recognized that most doctors are already intrinsically motivated to provide high-value care. In addition, they set the target for the percentage of patients who waited four weeks before medical intervention at 80%. This served as a reminder to doctors that high-quality, low-cost care for most patients meant waiting for lower back pain to resolve itself, but for some patients—for example, those who waited a month before seeing the doctor in the first place—immediate treatment was warranted. The target reflected the imperfect nature of the metric and drew physicians’ attention back to the underlying strategy.
Use multiple metrics.
Another study Bill did with Choi and Hecht shows that people surrogate less when they’re compensated for meeting targets on multiple metrics of a strategy rather than just one. This approach highlights the fact that no single metric completely captures the strategy, which makes people more likely to consciously reject substituting it for the strategy. At Intermountain overall physician performance is assessed with a myriad of metrics, including patient satisfaction, condition-specific quality metrics (such as average A1C levels of diabetes patients), health outcomes (such as hospital readmittance), preventive efforts (such as appropriately timed mammograms), and total cost of care. No lone metric is used to quantify the competence or contribution of the medical staff. Multiple yardsticks do add complexity to the task of performance evaluation, but they’re essential to keeping people focused on the true strategy and avoiding surrogation.
Wells Fargo Revisited
To see if Wells Fargo remains vulnerable to surrogation, let’s look at the actions it has taken in the wake of its crisis. As far as we can tell, the bank is heading in the right direction with its damage-control efforts.
First, the new management’s emphasis on rebuilding trust with customers after the scandal has made the long-term relationship strategy much more clear and prominent. Second, the bank has stopped paying employees to cross-sell and has eliminated all sales goals. That may sound extreme, but it was appropriate for Wells Fargo because an obsession with sales quotas had become so entrenched at the bank. To address that issue, the cross-selling metric and everything related to it needed to go. Finally, Wells Fargo now gauges strategic success using at least a dozen metrics related to its customer focus, emphasizing that no single number tells the whole story and encouraging employees to consciously reject surrogation.
That progress notwithstanding, this episode in Wells Fargo’s history was devastating in terms of both quantifiable out-of-pocket costs and less measurable (but truly colossal) reputational costs, and there’s no indication yet that the bank is close to full recovery. However, at the very least, the new steps Wells Fargo has taken seem likely to remind tomorrow’s managers and employees that performance metrics are mere representations of strategy, not the strategy itself.
Conclusion
Many managers learn the hard way that surrogation can spoil strategy, and if you don’t take action to protect against it, it’s very likely that sooner or later personal experience will lead you to the same realization. If you’re using performance metrics, surrogation is probably already happening—the mere presence of a metric, even absent any compensation, is enough to induce some level of the behavior. So it’s time to take a hard look internally to see which metrics might be most prone to surrogation and consider where it might cause the most damage. As the Wells Fargo case illustrates, preventing the disease is far preferable to treating its symptoms.
from Hacker News https://ift.tt/2Z9zNXq