Good Data Won’t Guarantee Good Decisions

Global businesses have entered a new era of decision making. The ability to gather, store, access, and analyze data has grown exponentially over the past decade, and companies now spend tens of millions of dollars to manage the information streaming in from suppliers and customers. For all the breathless promises about the return on investment in Big Data, however, companies face a challenge. Investments in analytics can be useless, even harmful, unless employees can incorporate that data into complex decision making. Our research offers a succinct warning to managers. At this very moment, there’s an odds-on chance that someone in your organization is making a poor decision on the basis of information that was enormously expensive to collect. To help organizations measure and improve employees’ facility with data-driven decision making, Corporate Executive Board created the Insight IQ, which assesses the ability to find and analyze relevant information. We evaluated 5,000 employees at 22 global companies and sorted them into three groups. “Unquestioning empiricists” trust analysis over judgment, and “visceral decision makers” go exclusively with their gut. “Informed skeptics”—the employees best equipped to make good decisions—effectively balance judgment and analysis, possess strong analytic skills, and listen to others’ opinions but are willing to dissent. They’re the kind of data-savvy workers every company should try to cultivate. However, we found that only 38% of employees, and 50% of senior managers, fall into this group. Our analysis also showed that functions whose employees had the highest average scores performed about 24% better than other functions across a wide range of metrics, including effectiveness, productivity, employee engagement, and market-share growth. In addition to quantifying this overall skill deficit, we identified four problems that prevent organizations from realizing better returns on their investment in Big Data: Analytic skills are concentrated in too few employees. When a new form of analytics enters the workplace, companies typically start by hiring experts versed in using it, reasoning that the skills will trickle down to all. But too many companies are stuck in the “expert” phase. They have a handful of highly skilled analytics professionals but have not begun to train everyone else to make use of their analytics methodology. IT needs to spend more time on the “I” and less on the “T.” Most IT functions “grew up” working with finance, supply chain, and HR, where business needs are clearly defined, stable, and relatively consistent over a wide group of users. Other departments may have diverse data demands or may need to use data in ways they can’t clearly articulate. Meeting these challenges requires anthropological skills and behavioral understanding—traits that are often in short supply in IT departments. Reliable information exists, but it’s hard to locate. Many organizations lack a coherent, accessible structure for the data they’ve collected. They’re like libraries with no card catalog and no covers on their books. The rise of social media, new selling channels, and devices such as tablets and smartphones has made it even harder to manage analytic content. Fewer than 44% of employees say they know where to find the information they need for their day-to-day work. Business executives don’t manage information as well as they manage talent, capital, and brand. Too many executives treat data as something for the IT department to handle or consider themselves too inexpert to get deeply involved in how data is shared across the organization. Managers need to wake up to the fact that their data investments are providing limited returns because their organization is underinvested in understanding the information. Developing More Informed Skeptics Companies that want to make better use of the data they gather should focus on two things: training workers to increase their data literacy and more efficiently incorporate information into decision making, and giving those workers the right tools. Trust the Data—and Your Gut Corporate Executive Board assessed 5,000 workers according to their ability to balance judgment and analysis. Companies need more “informed skeptics,” who can find the middle ground. Some of the training can take place in workshops. Employees need to recognize that not all numbers are created equal—some are more reliable than others. They must understand the factors and calculations behind the numbers and learn to think critically about the accuracy, sample sizes, biases, and quality of their data. Even people who took statistics in college could probably use a refresher to help them apply what they learned then to their current jobs. But workshops aren’t the only—or necessarily the best—way to teach this information. Ongoing coaching is often more effective. To create an environment in which employees get the help they need, companies must rethink the kinds of people they bring in as experts. Although hiring managers typically put a premium on analysts’ quantitative skills, outstanding coaching skills are more valuable. Instead of simply answering questions as they arise, people-oriented data experts can provide informal, ongoing training to employees in departments outside their own, increasing the organization’s overall Insight IQ. Many of the best data-driven cultures have formalized the decision-making process, setting up standard procedures so that employees can obtain and correctly use the most appropriate data. Companies should make performance metrics transparent and embed them in job objectives. They should also ensure that compensation systems reward dialogue and dissent. Great decisions often require diverse contributions, challenges, and second-guessing. Having the right tools to create and interpret data displays is also vital. Half of all employees find that information from corporate sources is in an unusable format. The best companies avoid this problem by deploying improved information filtering and better visualization—for example, they might provide charts instead of raw data. If given the option of good-enough data now or perfect data later, most executives choose the former, confident that they can apply judgment to bridge the gaps. They rarely drill down into information—but they want to know that it’s possible to do so. Let’s look at two companies that have shown an growing awareness of the payoffs from data literacy: Tiffany and Blue Cross and Blue Shield of North Carolina (BCBSNC). Companies introducing a new data-analysis tool often conduct one-off workshops that are overly focused on the tool itself, instead of on how managers can use it to improve their judgment—and because the training isn’t repeated, it’s apt to be quickly forgotten. Tiffany holds year-round workshops that teach employees to use broad categories of information (such as sales, merchandising, and financial data) and instruct them in creating useful queries and employing analytical techniques. Surveys indicate that only 25% of all knowledge workers receive effective training in information analysis and use. At Tiffany, nearly all knowledge workers participate in ongoing data education. As a result, they are better equipped to exploit information, and the IT team spends more time helping them derive value from the company’s data and less time answering simple data-support questions. To understand how many “business intelligence” tools BCBSNC required, the IT team identified 10 skills that knowledge workers need in order to collect, analyze, and display information for decision making. It routinely surveys the workforce to assess those skills and creates training and new tools, such as dashboards and scorecards, to close the gaps it finds. The company recognizes the trade-off between having tools that optimize individual worker efficiency and bearing the expense of creating and managing too many customized tools. To support a broad cross-section of workers at a reasonable cost, it maintains three to five “enterprise tools,” used by the majority of employees. It also supports a number of “specialty tools” for specific teams or functions. Thus it’s eliminating dozens of unnecessary tools while ensuring that knowledge workers have the ones they need, both now and as business, analysis approaches, and tools change. What’s Your Decision-Making Style? Here’s a self-diagnostic developed by Corporate Executive Board to help employees assess their analytic skills. 1) Your boss asks you to develop a proposal to launch a new product. You: a. Dig up data to generate some initial ideas, talk with your colleagues, and then write the proposal. b. Burn the midnight oil drafting the proposal, add some supporting charts, and get it to the boss as soon as possible. c. Find your group’s last product launch proposal, take a look at recent data, and model the new proposal on the old one. 2) Reviewing recent sales figures, you notice a spike in a division that’s been struggling. You: a. Look up some data, run some numbers, and make a couple of calls to figure out why sales are up. b. Are suspicious about the increase. c. Congratulate the division manager for turning things around. 3) You’re leading the search for a new team member and must develop guidelines for evaluating candidates. You: a. Pull the résumés of some past top performers to help you define an ideal candidate profile. b. Talk to several people you think might be interested in the job and try to understand what their profiles would look like. c. Check the criteria previously used to fill similar positions. 4) You’re evaluating options for a product redesign, and your market research is inconclusive. You: a. Choose the option you think your management team is most likely to make work. b. Rely on your best sense of what your customers will like. c. Commission more market research before making a decision. 5) Your boss asks you to prepare the department budget for the coming year. You: a. Review recent department budget trends and meet with team leaders to learn whether forecasts need to be adjusted for changing conditions. b. Ask your team leaders to provide their budget expectations, and aggregate the results. c. Project the budget on the basis of an extensive analysis of historical trends. If most of your answers are: As, you’re an informed skeptic Bs, you’re a visceral decision maker Cs, you’re an unquestioning empiricist In the case of a tie, go by your answer to question 3. Recent financial and business events show all too plainly what can happen when rich data and analytics collide with gaps in knowledge or lapses in judgment. Leaders need to ensure that their processes and human capabilities keep pace with the computing firepower and information they import. To overcome the insight deficit, Big Data—no matter how comprehensive or well analyzed—needs to be complemented by Big Judgment.

Good Data Won’t Guarantee Good Decisions

Partner Center