Just-in-Time Delivery Comes to Knowledge Management

Dr. Bob Goldszer is the associate chief medical officer and head of the Special Services Department at Brigham and Women’s in Boston, one of the nation’s leading hospitals. A professor at the Harvard Medical School, Goldszer has both an MD and an MBA. He’s a high-end knowledge worker at the top of the medical profession. Yet Dr. Goldszer has a big problem—one common to all physicians. There is so much knowledge available about his work that he cannot possibly absorb it all. He needs to know something about almost 10,000 different diseases and syndromes, 3,000 medications, 1,100 laboratory tests, and many of the 400,000 articles added each year to the biomedical literature. Even if he were to consult only those articles written by his colleagues at Partners HealthCare (the Boston-based umbrella organization that includes Brigham and Women’s, Massachusetts General, and several other hospitals and physicians’ groups), he would need to choose among 202 on hypertension, 139 on asthma, and 313 on diabetes. As a primary care physician, he must know something like a million facts, and those facts are constantly changing. Clearly, it is difficult for Goldszer to stay on top of even a fraction of all the new knowledge being generated in his field and still do his job. This is not a trivial problem. It is, quite literally, a matter of life and death. Over the past decade, researchers have done a series of studies on medical errors. The results are sobering. The Institute of Medicine’s 1999 report To Err Is Human suggests that more than a million injuries and as many as 98,000 deaths each year are attributable to medical errors. Partners’ own research in 1995 suggested that more than 5 % of patients had adverse reactions to drugs while under medical care; 43 % of those inpatient reactions were serious, life threatening, or fatal. Of the reactions that were preventable, more than half were caused by inappropriate drug prescriptions. About a third of the marginally abnormal pap smears and mammograms received no documented follow-up. A study of the six most common laboratory tests ordered by physicians in Brigham and Women’s surgical intensive care unit found that almost half of the tests ordered were clinically unnecessary. Another study at the Brigham found that more than half of the prescriptions for a particular heart medicine were inappropriate. Some of these mistakes result from carelessness, but far more of them, we believe, occur because the clinicians must track such massive amounts of complex information. The problem of staying on top of all the knowledge available in a given profession is not restricted to medicine, of course. Knowledge workers in many other fields have problems similar to Dr. Goldszer’s, though generally theirs are less life threatening. No matter what the industry, knowledge workers often can’t keep up with the knowledge being generated. And although failure to keep up with current information may not result in deaths, it can lead to less successful projects and products, wasted resources, and broken businesses. Knowledge management, which was all the rage in the mid- to late 1990s, is still a good idea, but it needs a new approach. In the early years of knowledge management, companies established employee networks and communities of practice, built knowledge repositories, and tried to encourage information sharing. Knowledge workers were expected to participate in these activities in addition to doing their regular jobs. That meant staying a little later each night to share what they’d learned in the course of doing their jobs and coming in a little earlier each morning to learn from others. As a result, the programs, many of which continue today, have been only marginally successful. Even the successful ones require motivational schemes and some arm-twisting from senior executives. But there is a better approach to information sharing and retrieval. The key to success, we’ve found, is to bake specialized knowledge into the jobs of highly skilled workers—to make the knowledge so readily accessible that it can’t be avoided. This is the main approach Partners HealthCare has taken to address Dr. Goldszer’s problem. Partners has made his job easier by helping him avoid mistakes, learn from other employees’ experiences, and access important information when he needs to make decisions. While there are several ways to bake knowledge into knowledge work, the most promising approach is to embed it into the technology that knowledge workers use to do their jobs. That approach ensures that knowledge management is no longer a separate activity requiring additional time and motivation. The key to success is to bake specialized knowledge into the jobs of highly skilled workers—to make the knowledge so readily accessible that it can’t be avoided. We believe that this method could revolutionize knowledge management in the same way that just-in-time systems revolutionized inventory management—and by following much the same philosophy. In this article, we’ll discuss how just-in-time knowledge has been embedded into Dr. Goldszer’s work and other physicians’ work at a few Partners hospitals. We’ll also consider the circumstances that make it possible—or impossible—to bake knowledge into the work processes of other high-end professionals. Partners’ Ambitious Project Embedding knowledge into everyday work processes is time-consuming and expensive. It’s not an undertaking that anyone in his right mind would tackle without a very good reason. A decade ago, Partners had that reason: Researchers at the Harvard School of Public Health and Harvard Medical School found that there were surprisingly high numbers of medical errors and adverse drug reactions at Partners hospitals. That these institutions could be unconsciously acting in direct opposition to their healing mission was deeply troubling. Under the direction of H. Richard Nesson, CEO of Brigham and Women’s at the time, Partners undertook an ambitious and risky project to link massive amounts of constantly updated clinical knowledge to the IT systems that supported doctors’ work processes. The project was ambitious because it had the potential to substantially improve the quality of physicians’ decision making—and hence improve the quality of patient care. But it was also risky because knowledge-based systems had a very spotty record of success in their first incarnation two decades ago and because Partners didn’t really know if it would be able to codify the millions of facts and data points that doctors use to make complex decisions about treatment. So the project was defined relatively narrowly at first. Partners professionals targeted an essential work process—physician order entry—and a problem that was well documented—errors in drug prescriptions and lab-test ordering. Drug interactions are relatively straightforward and easy to program; this fact, too, improved the project’s chances for success. The decision to focus on the order-entry system was important because the system is central to physicians delivering good medical care. When doctors order tests, medications, or other forms of treatment, they’re translating their judgments into actions. This is the moment when outside knowledge is most valuable. Without the system, doctors would have no easy way to access others’ knowledge in real time. Automated order entry addresses this need in several ways: It increases efficiency and safeguards against errors due to poorly written orders. Even more important, it allows physicians easy access to massive amounts of up-to-date medical knowledge while they go about their daily work. Indeed, the order-entry system forces physicians to engage with queries or recommendations (although, as we shall see, they can always override the system’s recommendations). Order entry is a key work process in this system, but it’s not the only one. Partners’ approach is built on a set of integrated information systems—including on-line referral and medical-records systems—that physicians can use to manage patient care. These all draw from a single database of clinical information and use a common logic engine that runs physicians’ orders through a series of checks and decision rules. Here’s how it works. Let’s say Dr. Goldszer has a patient, Mrs. Johnson, and she has a serious infection. He decides to treat the infection with ampicillin. As he logs on to the computer to order the drug, the system automatically checks her medical records for allergic reactions to any medications. She’s never taken that particular medication, but she once had an allergic reaction to penicillin, a drug chemically similar to ampicillin. The computer brings that reaction to Goldszer’s attention and asks if he wants to continue with the order. He asks the system what the allergic reaction was. It could have been something relatively minor, like a rash, or major, like going into shock. Mrs. Johnson’s reaction was a rash. Goldszer decides to override the computer’s recommendation and prescribe the original medication, judging that the positive benefit from the prescription outweighs the negative effects of a relatively minor and treatable rash. The system lets him do that, but it requires him to give a reason for overriding its recommendation. The fact that the order-entry system is linked not just with the clinical database but also with the patient’s records increases its usefulness by an order of magnitude. The system may inform Goldszer that a drug being prescribed is not economical or effective, but it can also tell him that the patient is taking another drug that interacts badly with the new medication or one that might exacerbate a condition other than the one being treated. When it comes to ordering tests for a patient, the system may note that a particular test is generally not useful in addressing the symptoms identified or that it has been performed on the patient enough times that a retest would not be useful. That’s a relatively simple explanation of what the integrated system does, but, in fact, the logic engine and the knowledge base can serve as very sophisticated screens for the physicians’ decisions. For instance, imagine that a patient with a history of sleep apnea is prescribed a narcotic to mitigate pain after surgery. Narcotics can cause people with sleep apnea to go into respiratory arrest, but, as long as the history of sleep apnea is noted in the patient’s medical records, the system will alert the physician to that potential problem. It also takes into account the patient’s age, likely metabolism, probability of renal failure, maximum allowable lifetime amounts of a chemotherapy agent, and hundreds of other factors. The logic engine and knowledge base at Partners are used more during order entry than at any other time. But they are used increasingly during normal review of patient medical records as well. For example, the system alerts the physician, as he or she reviews Mrs. Smith’s record, to follow up on her marginally abnormal mammogram or to recheck her cholesterol levels. In addition, it may remind a physician that a particular patient should receive a call or schedule a follow-up appointment. There are, of course, times when a physician isn’t treating a patient directly yet still needs to know that something has happened. For these times, Partners developed an event-detection system that alerts a physician when a hospitalized patient’s monitored health indicators depart significantly from what is expected. The physician is notified through a pager and can then visit the patient directly or call in a new treatment. Minor variations are routed to the nurses’ station, and the nurse can decide whether to call in the physician. The power of knowledge-based order-entry, referral, computerized medical-record, and event-detection systems is that they operate in real time. Knowledge is brought to bear immediately without the physician having to seek it out. In some situations, physicians can consult with other experts in real time, via teleconferencing and other technologies. Such practices are still in their early stages, but they show great promise. For example, if a patient on Nantucket island experiences what his doctor suspects is a stroke, he needs to be diagnosed and treated within an hour or his chances for full recovery drop precipitously. By the time he is flown to Cape Cod Hospital, it might be too late. If a specialist in Boston, or for that matter in Tel Aviv, can interview the patient over a videoconference screen, observe how he speaks and moves, and review scan results, the likelihood of effective treatment will go way up. Partners has also assembled many other knowledge resources that are not accessible in real time but are valuable nonetheless. These sources are more extensive than what’s in the clinical-information database. However, they’re like traditional knowledge-management systems in that users need to seek them out. The organization’s on-line sources (collectively called The Handbook) include on-line journals and databases, care protocols or guidelines for particular diseases, interpretive digests prepared by Partners physicians, formularies of approved drugs and details on their use, and even on-line textbooks. All of these resources are accessible through an integrated intranet portal. It’s an unusually good set of resources, but they’re not different in kind from those that practitioners at other hospitals can consult. The Handbook is accessed, across all Partners institutions, about 3,000 times a day. Contrast this with the 13,000 orders submitted a day at Brigham and Women’s alone; even though it’s invisible to the clinicians, the information embedded in the order-entry system is used far more intensively than The Handbook is. While Partners’ embedded-knowledge program has been under development for more than a decade, it’s still not complete. The on-line order-entry system and related knowledge are only accessible within the organization’s two flag-ship hospitals, Mass General and Brigham and Women’s. Medical knowledge has not yet been codified for all the diseases that Partners physicians treat. But the approach is clearly beneficial. A controlled study of the system’s impact on medication errors found that serious errors were reduced by 55 % . When Partners experts established that a new drug was particularly beneficial for heart problems, orders for that drug increased from 12 % to 81 % . When the system began recommending that a cancer drug be given fewer times per day, the percent of orders entered for the lower frequency changed from 6 % to 75 % . When the system began to remind physicians that patients requiring bed rest also needed the blood thinner heparin, the frequency of prescriptions for that drug increased from 24 % to 54 % . These improvements not only save lives, they also save money. For starters, the system now recommends cheaper as well as more effective drugs. Even more important, it helps prevent longer hospital stays and repeat tests that result from adverse drug events (ADE). That can save a facility large sums of money, since a 700-bed hospital will normally incur about $ 1 million per year in preventable ADE costs. Order entry with embedded knowledge is still rare enough that U.S. insurers have not yet seen their costs go down, nor have national malpractice figures changed. However, Partners, which insures itself for malpractice, has some early data suggesting that malpractice reserves can be smaller because of fewer drug-related claims. Keys to Success Developing a system like Partners’ isn’t easy—from either a technical or a managerial standpoint. Few off-the-shelf software packages used for knowledge-intensive business processes allow individuals and organizations to embed their own knowledge into systems. Partners had to develop most of its systems from scratch, creating a complex information and technology infrastructure that pulled together the knowledge base and logic modules with an integrated patient-record system, a clinical-decision support system, an event-management system, an intranet portal, and several other system capabilities. Other hospitals have some or all of these capabilities, but Partners’ real-time knowledge approaches are undoubtedly at the cutting edge. The technical underpinnings of an embedded-knowledge system are key, but just as important are the nontechnical, managerial aspects required to keep the system running smoothly. Several of these aspects—each of which would be relevant to any organization seeking to bake knowledge into its work—are described below. Support from the Best and Brightest. Building a system like Partners’ is a challenging IT project, to be sure. But then comes an even harder task: Convincing knowledge workers, no matter what environment or field they’re in, to support the system and the new way of working. The growing concern over medical errors provided that motivation at Partners; absent a similar sense of pressing need, it probably wouldn’t have gotten off the ground. An Expert and Up-to-Date Knowledge Base. If Partners’ clinical database included idiosyncratic, untested, or obsolete knowledge, it would put patients—and Partners itself—at high risk. Thus, only clinicians at the top of their game can create and maintain the knowledge repository. Partners has addressed this issue by forming several committees, and empowering existing ones, to identify, refine, and update the knowledge used in each domain. For instance, the medication recommendations in the system come from drug therapy committees. Teams of specialists design care protocols for particular diseases. And radiology utilization committees have developed logic to guide radiology test ordering. Participation in these groups is viewed as a prestigious activity, so busy physicians are willing to devote extra time to codifying the knowledge within their fields. Prioritized Processes and Knowledge Domains. Since these initiatives are difficult and expensive, they should only be undertaken for truly critical knowledge work processes. At Partners, it was relatively easy to identify which medical care processes were the most crucial, but important decisions still needed to be made about which disease domains and medical subprocesses to address—for example, ordering medications versus referring a patient to a specialist—and in what order. Fields with many disease variations and multiple treatment protocols, such as oncology, are more difficult to include in the knowledge systems. In general, it’s preferable to develop systems in fields with low levels of ambiguity, a well-established external knowledge base, and a relatively low number of possible choices facing the decision makers. Embedded-knowledge initiatives should only be undertaken for truly critical knowledge work processes. Final Decisions by the Experts. With high-end knowledge workers like physicians, it would be a mistake to remove them from the decision-making process; they might end up resenting or rejecting the system if it challenged their role—and with good reason. Because over-reliance on computerized knowledge can easily lead to mistakes, Partners’ system presents physicians with recommendations, not commands. The hope is that the physicians will combine their own knowledge with the system’s. Out of the 13,000 orders entered on an average day by physicians at Brigham and Women’s, 386 are changed as a result of a computer suggestion. When medication allergies or conflict warnings are generated, a third to a half of the orders are canceled. The hospital’s event-detection system generates more than 3,000 alerts per year; as a result of these alerts, treatments are changed 72 % of the time—a sign that the hybrid human-computer knowledge system at Partners is working as it should. A Culture of Measurement. In order to justify the time and money spent on an embedded-knowledge system, and to assess how well it’s working, an organization needs to have a measurement-oriented culture. Partners has always had a strong measurement culture because it is an academic medical center and because most of its senior clinicians are also researchers. Its knowledge management approach has only furthered the emphasis on measurement. The tracking mechanisms within the order-entry system can detect whether the physicians use the knowledge and change their treatment decisions, which is the only way to know that the system is working. The measures are used as justifications and progress reporting tools for efforts to reengineer and continuously improve care processes. The Right Information and IT People. Whenever knowledge technologies are applied to business problems, it’s tempting to attribute any success to the technology. But in the case of Partners’ system, and in many others we’ve seen, success is based mostly on the people behind the technology. An IT organization that knows the business and can work closely with key executives and knowledge-rich professionals is important. A “back room” IT group could never successfully build a system of this type. Also important is a staff that is skilled in information management. In health care, this discipline is called medical informatics, and Partners has recruited leaders in this field. It has several medical informatics departments, including Clinical and Quality Analysis, Medical Imaging, Telemedicine, and Clinical Information Systems R&D. The leaders of each of these departments are doctors, but they also have advanced degrees in fields such as computer science, statistics, and medical informatics. • • • In general, it’s easier to embed knowledge into the work of less-skilled workers; the higher you go, the harder it gets. But organizations are gradually learning how to make the concept work at all levels. Customer service representatives without a great deal of technical skill now have highly scripted jobs. Many highly skilled reps at high-tech firms like Hewlett-Packard, Dell, and Xerox work with computer systems that rapidly supply knowledge to help them resolve customers’ problems. Midlevel knowledge workers—programmers, engineers, designers—depend increasingly on knowledge repositories built into the technology they use to do their jobs. GM’s Vehicle Engineering Centers, for example, program information about the desirable dimensions of new vehicles and the parameters of existing components into the company’s computer-aided design systems so that car and truck designers can’t help but employ the knowledge. Baking knowledge into the work processes of high-end professionals like physicians is relatively new. Such professionals are different from other knowledge workers: They’re generally paid more and receive more intensive training; they make decisions based largely on intuition and years of experience; they’ve historically enjoyed high levels of autonomy; they’re sufficiently powerful that the organizations they work for are reluctant to tinker with their work processes; and, perhaps most important, they do most of their work away from a computer screen. All those factors make it harder to embed knowledge into their work processes. But the Partners example illustrates that it is indeed possible to inject knowledge directly and effectively into the work these professionals do, dramatically improving their performance. And for people like Dr. Goldszer and his patients, such improvements can make all the difference.

Just-in-Time Delivery Comes to Knowledge Management

Partner Center