Fooled by Experience

Idea in Brief The Status Quo Experience: We think of it as our guide, a reliable source of insight and the foundation of our expertise. As we make decisions, we rely on our experience and on what advisers and confidants tell us about theirs. The Problem We view our experience through multiple filters that distort reality, limiting our ability to figure out what’s actually going on around us. As a consequence, our experiences fool us instead of making us wiser. The Solution We can base our decisions on a clearer view of the world if we focus not just on outcomes but on the processes that lead to them; learn from near misses; encourage disagreement and the search for disconfirming evidence; and broaden our perspective. May We rely on the weight of experience to make judgments and decisions. We interpret the past—what we’ve seen and what we’ve been told—to chart a course for the future, secure in the wisdom of our insights. After all, didn’t our ability to make sense of what we’ve been through get us where we are now? It’s reasonable that we go back to the same well to make new decisions. It could also be a mistake. Experience seems like a reliable guide, yet sometimes it fools us instead of making us wiser. The problem is that we view the past through numerous filters that distort our perceptions. As a result, our interpretations of experience are biased, and the judgments and decisions we base on those interpretations can be misguided. Even so, we persist in believing that we have gleaned the correct insights from our own experience and from the accounts of other people. If our goal is to improve decision making, we can use our knowledge of those filters to understand just what our experience has to teach us. Distilling a wide range of research on the subject, we focus in this article on the biases that result from three types of filters: the business environment, which favors the observation of outcomes (especially successes) over the processes that lead to them; our circle of advisers, who may be censoring the information they share with us; and our own limited reasoning abilities. We also consider techniques for overcoming those biases. We Focus on What We Can See In the business environment, the outcomes of decisions are highly visible, readily available for us to observe and judge. But the details of the decision process, which we can control far more than the result, typically don’t catch our attention. If the aim is to learn from experience—mistakes as well as successes—acknowledging that process is crucial. We celebrate successes and condemn failures—a response that disregards the underlying causes. Imagine that two firms use the same risky strategy, but one gets lucky and prospers while the other doesn’t. We celebrate the first one and condemn the second—a response that disregards the underlying causes. The tendency to overreward the results of a decision and underreward its quality is known as the outcome bias. This bias can influence our actions in subtle ways. A good outcome can lead us to stick with a questionable strategy, and a bad outcome can cause us to change or discard a strategy that may still be worthwhile. For example, in the NBA, coaches “are more likely to revise their strategy after a loss than a win—even for narrow losses, which are uninformative about team effectiveness,” a recent Management Science article shows. Further Reading The Five Traps of High-Stakes Decision Making Decision Making Magazine Article Michael C. Mankins Bet on process rather than luck or inspiration. Save Share A focus on outcomes can also influence our sense of ethics. A Harvard Business School working paper finds that “the same behaviors produce more ethical condemnation when they happen to produce bad rather than good outcomes, even if the outcomes are determined by chance.” In other words, if everything turns out OK, we’re more likely to think that the decision was not just effective but also morally sound. Our attention to outcomes—and disregard of the processes that create them—makes solutions seem more valuable than preventive actions. A decision maker who solves a burning problem can be identified and rewarded, while one who takes action to avoid the same problem is far harder to spot. Among outcomes, successes are more visible than failures. The business world is awash with success stories: The latest best-sellers, the biggest start-up, and winning corporate strategies are widely trumpeted, while failures quietly disappear. Relying on stories of successes and on analyses of what those successes have in common is as unreliable a practice as it is popular. In an article in Organization Science, Jerker Denrell points out that observing the common managerial practices of existing organizations can mislead us in a couple of ways. First, failures can share some of the same traits as successes. Second, if certain factors always lead to failure, we won’t be able to discover them by studying only successes. Approach with caution any list that purports to reveal, for instance, “10 common traits of successful companies”—whether it is punctuated with an exclamation point or comes with the seriousness of a legitimate study. Ignoring failures has another effect. In Fooled by Randomness, Nassim Nicholas Taleb argues that doing so masks the failure rate. If the rate is high, chances are there is no magic formula for success. And if there’s no magic formula, then a manager can’t be faulted for missing it. By concealing the prevalence of failures, the environment makes it more difficult for us to learn from them. Instead, we are fooled into thinking that we have more control over success than we actually do. We Trust Our Circle of Advisers Honest feedback—an unbiased, undistorted assessment of one’s experience—is essential for improving decisions. Yet decision makers are often surrounded by individuals who have incentives to feed them censored and self-serving information—and these people are not necessarily a crowd of yes-men. Censorship is a powerful tool for influencing opinion. Restricting the information that reaches decision makers installs a strong bias in their perceptions. Even if we are aware of the existence of censors, it can be difficult to think beyond the immediately available information. Our intuitions are often shaped by the evidence we recall, no matter its relevance—a tendency cognitive scientists call the availability bias. Individuals who are hoping for a raise, a promotion, or some other benefit may well choose to deliver partial and insincere feedback, omitting anything negative about a decision maker’s performance. As the organizational psychologist Lynn Offermann argues in “When Followers Become Toxic” (HBR, January 2004), leaders run the danger of being “surrounded by followers who fool them with flattery and isolate them from uncomfortable realities.” In this way, flattery can be an especially powerful filter. But your trusted advisers aren’t necessarily aiming to ingratiate themselves with you; they may just be trying to conform. One powerful way to do that is to agree with you. It is much easier to conform to others’ opinions than to voice objections. If all your advisers follow that approach, you won’t have any dissenters. Your demeanor can make matters worse. Shooting the messenger doesn’t facilitate healthy communication. Indeed, welcoming criticism is hard, especially for people with high status. Censorship and a desire for conformity give decision makers a distorted view of their strategic competence, a bias that can result in their downfall. A recent article in Administrative Science Quarterly abstract suggests that such misperceptions may reduce “the likelihood that CEOs will initiate needed strategic change in response to poor firm performance,” which, of course, can lead to their dismissal. Executives who are surrounded by people who agree with them may also experience decreased creativity and problem-solving abilities. Conceiving an idea, a strategy, or a methodology is rarely a solo act. A successful creative endeavor involves input from a diverse set of people. If everyone is simply trying to conform, the group cannot benefit from people’s backgrounds, perspectives, and experiences. We Overvalue Our Own Experience We can’t place all the blame for our distorted view of the world on the environment and our inner circle. Some of the blame lies with us. Our own reasoning abilities can sabotage how we collect information and evaluate evidence. We end up learning the wrong lessons from our experience—even when it’s possible to learn the right ones. One issue is that we tend to search for and use evidence that confirms our beliefs and hypotheses, and we gloss over or ignore information that contradicts them—an exercise of selectively building and interpreting experience known as the confirmation bias. We can easily support our beloved superstitions, spurious correlations, and bogus connections. This natural tendency of the human mind hinders competent decision making. Some see external, information-rich big data as a possible remedy, but data is subject to the same kinds of bias. If analysts cherry-pick information to suit managers’ expectations, managers will be reassured about their decisions and see no need to improve them. And once misleading insights are data-approved, they are even harder to challenge. Another issue is our memory. The philosopher John Stuart Mill wrote in On Liberty, “There are many truths of which the full meaning cannot be realized, until personal experience has brought it home.” Mill’s sentiment assumes that we record and remember events accurately. We don’t. Further Reading Predictable Surprises Organizational Culture Magazine Article Michael Watkins and Max H. Bazerman The signs of an impending crisis often lie all around us, yet we still don’t see them. Fortunately, there are ways to spot danger before it’s too late. Save Share In addition to the poor quality and reliability of our memory of experience, there is the crucial problem of quantity. The issue is sampling variability: A manager’s personal experience is inevitably based on small samples of incidents that are most likely unrepresentative of the whole context; there is no way that any one manager can experience the entire range of possibilities. Differences among incidents may be due to unknown factors or randomness. If managers read too much into those differences, they may be fooled into seeing patterns that do not actually exist and illusory relationships between unrelated variables. Our belief in the relationship between the past and the future also can interfere with our view of the world. Predictions based on experience make the crucial assumption that the future will resemble the past. One of us, Robin Hogarth, has done extensive research on how human intuition fares in prediction tasks. The findings suggest that not even experts with a tremendous amount of experience are proficient in foreseeing economic, social, and technological developments. Managers should be aware that just because something seems obvious after the fact does not mean that it could have been predicted. Decision makers often fall into this hindsight bias, which can lead to overconfidence and illusions of control. When it comes to effective decision making, not knowing that you cannot predict is a bigger sin than not being able to predict. How Not to Be Fooled The following techniques can uncover the real lessons experience offers. None are easy, but making the effort to adopt them can help you base decisions on a clearer view of the world. Sample failure. Failures and the processes that lead to them are doomed to stay in the dark unless special occasions are created to bring them to light. It is not easy for managers to share their experiences of defeat. One exception is Paul Biggar, a founder of Newstilt, who posted a detailed account of the fall of the news website, which stayed open for just two months in 2010. To give more people the opportunity to share their stories of failure, a group of entrepreneurs has been organizing FailCon, a conference dedicated to giving visibility to experiences that would otherwise remain hidden. To identify what could be done better in the future, companies can also conduct decision postmortems to analyze underlying processes. Of course, the goal of learning must dominate the natural tendency to assign blame. Don’t miss near misses. Another oft-ignored event is the near miss—a failure that’s disguised as a success, but only because there are generally no dire consequences. An executive at a chemical company told us of a near miss when a machine malfunctioned at a plant, causing a sudden burst of extremely hot gas. Luckily, no workers were nearby, but a quick inquiry revealed that some of the workers in the plant hadn’t been wearing protective gear at the time of the incident, even though they are required to put it on the minute they step onto the premises. Exposure to the gas without the safety gear would have resulted in serious physical injury. It might be easy to dismiss this episode as unimportant, since no one was hurt. But doing so would deprive the company of an opportunity to learn a valuable lesson without having to suffer dreadful consequences. Ironically, ignoring this near disaster—as so often happens—would lead workers to perceive it as inconsequential and thus would help perpetuate the same dangerous behavior. As Catherine Tinsley, Robin Dillon, and Peter Madsen have shown in “How to Avoid Catastrophe” (HBR, April 2011), risk-free, anonymous reporting channels can reduce close calls and disastrous mistakes in many sectors. Pursue prevention. Recognizing a potential problem requires a different approach than solving an actual problem. One strategy is to harness employees’ collective talents by allowing people to raise concerns about the firm’s operations. Many companies’ incentive mechanisms work exactly to the contrary, and employees often hesitate to speak up for fear of reprisal or being labeled a nuisance. But the signs of a blunder can be picked up more easily by lower-level managers and personnel who deal with daily operations than by their senior colleagues. Employees should be made to feel comfortable reporting issues to the very top—even obliged to do so. Disagree. As Peter Drucker wrote, “The first rule in decision making is that one does not make a decision unless there is disagreement.” To devise healthy strategies, executives need to hear many perspectives, including feedback that is critical of their own actions. Executives should surround themselves with people from diverse backgrounds and promote independent thinking in their team. Many executives task certain coworkers, friends, or family members with speaking frankly on important matters. Ed Catmull, the president of Pixar and Walt Disney Animation Studios, stresses the importance of building a brain trust, a group of advisers who will deflate egos and voice unpopular opinions. He argues in his September 2008 HBR article that disagreements in meetings end up benefiting everyone in the long run, because “it’s far better to learn about problems from colleagues when there’s still time to fix them than from the audience after it’s too late.” Disconfirm. Rather than finding clues that corroborate your hunch—all too easy in an information-rich world—start by asking yourself how you could know you were, in fact, wrong. What evidence would contradict your belief and how likely is it that you would see it? One technique is to use this thought experiment: Imagine that you are already in the future and things have not turned out as you had hoped. Now use your new hindsight to ask how this might have happened. If you do go the route of using big data, refrain from revealing your hopes and dreams to the data scientists you hire to collect and mine information. Ask questions in a way that prompts them to look for caveats that would endanger your mission but that doesn’t reveal what you actually hope they’ll find. Lose focus. It may seem that to mine our experience for valuable lessons, we must focus on the experiences we think really matter. In fact, a narrow perspective can be a serious impediment. In The Luck Factor, the psychologist Richard Wiseman shows that when people focus too much on an issue or a task, they inevitably miss out on unexpected opportunities. For a firm, spotting those opportunities is vital. A company that directs its R&D efforts on a single domain, a start-up that uses only a few channels of communication, or a manager who employs only people from a certain background will not be able to cope well with the complexity of today’s business environment. Being open to the unexpected is also crucial for individuals. A wide perspective can help, giving new meaning to our varied experiences and allowing us to learn from them and draw on them in surprising ways. The result is often serendipitous discovery and innovation. Curiosity prompted Alexander Fleming to inspect a moldy petri dish before cleaning it, a move that resulted in the discovery of penicillin. Similarly, a passion for hiking and hunting led George de Mestral to invent Velcro. Seeds that got stuck in his dog’s fur gave him the idea for the product. Managers who acknowledge the role of serendipity and luck have an advantage over those who have illusions of control and are overconfident about the accuracy of their judgments. Change is both inevitable and unpredictable. As Spyros Makridakis, Robin Hogarth, and Anil Gaba argue in Dance with Chance, managers who accept that can calibrate their intuitions accordingly and learn to see change as an opportunity rather than a shock. To do so, they must broaden their perspective. Welcoming diverse experiences will help decision makers manage the unknowns ahead and greatly increase the odds of being in the right place at the right time. The lessons experience seems to be teaching us, accepted uncritically, should almost never guide our actions. What we learn from experience is typically filtered: by the business environment, by the people around us, and by ourselves. If we keep the filters and their antidotes clearly in mind, we can discover what experience actually has to teach us. As the late Hillel Einhorn, one of the fathers of behavioral decision theory, asked, “If we believe we can learn from experience, can we also learn that we can’t?”

Fooled by Experience

Resumen.

Partner Center