search text   Advanced Search

This Article Extract Full Text (PDF) P3Rs: Submit a response Alert me when this article is cited Alert me when P3Rs are posted Alert me if a correction is posted Citation Map Services E-mail this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My File Cabinet Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via CrossRef Citing Articles via ISI Web of Science (4) Citing Articles via Google Scholar Google Scholar Articles by Sittig, D. F. Articles by Shabot, M. M. Search for Related Content PubMed PubMed Citation Articles by Sittig, D. F. Articles by Shabot, M. M. Related Collections Office Practice

Lessons From "Unexpected Increased Mortality After Implementation of a Commercially Sold Computerized Physician Order Entry System"

^a Department of Medical Informatics, Northwest Permanente PC, Portland, Oregon
^b Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, Oregon
^c School of Health Information Sciences, University of Texas, Houston, Texas
^d Thomson Micromedex, Denver, Colorado
^e Department of Medicine, University of Pennsylvania, School of Medicine, Philadelphia, Pennsylvania
^f Departments of Surgery and Enterprise Information Services, Cedars-Sinai Medical Center, Los Angeles, California

Abbreviations: CPOE, computerized physician order entry • CIS, clinical information system • ADE, adverse drug event

We are writing in response to the article "Unexpected Increased Mortality After Implementation of a Commercially Sold Computerized Physician Order Entry System" by Han et al.¹ The authors are to be congratulated for their courage in bringing their compelling account of computerized physician order entry (CPOE) implementation problems to the medical literature as they tried to interpret their results concerning mortality. Their article is as much a search for answers as it is a recitation of the shortfalls in their implementation process and computer systems. It is critically important to understand that the types of problems described by Han et al are not limited to their institution. In fact, setbacks and failures in the implementation of clinical information systems (CISs) and CPOE systems are all too common (eg, see refs ^2–4). Although it is tempting to focus solely on the role of new technology in the problems highlighted by this example, there are also important lessons to be learned about related organizational and workflow factors that affect the potential for danger associated with CPOE implementation.

There are many previous publications about troubled or failed implementations. The account by Han et al is unique in that an adverse change in mortality rate was associated in time with CIS and CPOE implementation. We may question the study's methodology and conclude that causality was not proven, yet the assignment of CPOE to a severity-adjusted odds ratio of 3.71 for patient death simply cannot be ignored. Regardless of what was or was not proven, if only one unnecessary death were caused by the implementation process or CIS and CPOE modules, that is one too many.

The question that must be asked is how can intelligent and well-intentioned leaders at all levels of an institution make the kind of implementation decisions that ultimately place excellent patient care in jeopardy? Clearly, that was not their intent, so how could it happen? What is it about CIS and CPOE that makes implementation so risky? Why are these implementations prone to causing emotional distress,^{5 rework,6 delay,7} user protest,⁷ temporary system withdrawal, and later repeat implementation,⁸ often at a cost of millions of dollars to the hospital or health system involved? How can institutions avoid these risks and additional costs? These are the questions that demand answers.

We posit that the primary reason CISs and CPOE are prone to failure is that they have the ability to profoundly alter patient care workflow processes. Although the intent of computerization is to improve patient care by making it safer and more efficient, the adverse effects and unintended consequences of workflow disruption may make the situation far worse.⁹ It is important to remember that the manual processes of patient care and documentation in place within an institution have been finely tuned over long periods of time, usually years to decades. Although paper charting forms, medication ordering, delivery and administration, and processes for patient admission and transfer are appropriate subjects for computerization, the transition from manual to computerized methods is notoriously complex. This is a severely underappreciated fact of CIS and CPOE implementation. In an ordinary business, employees, clients, revenue, and profits may be adversely affected by computerization. In a hospital, patients and caregivers are at risk. In other words, the stakes are much higher. Santayana once wrote, "Those who cannot remember the past are condemned to repeat it."¹⁰ So perhaps what we should do in retrospect is to learn from mistakes that occurred in this implementation and others to help ensure that organizations implementing CIS and CPOE in the future do not fall prey to Santayana's admonition.

We can learn many things from the Han et al study. Each of the following lessons begins with a direct quote from the article by Han et al.

1. "Hospital-wide implementation of CHP's [Children's Hospital of Pittsburgh's] CPOE system (along with its clinical applications platform) occurred over a 6-day period."—Although few organizations have the luxury of pioneering institutions that spent 10 years or more rolling out a CIS and CPOE, attempting such a project in a few days goes beyond challenging and borders on the temerarious.¹¹ Previous studies have shown that the workflow of clinicians changes significantly after implementation of these types of systems.¹² Given such a huge change, clinicians must be given time to adapt to their new routines and responsibilities in a setting that is carefully managed to ensure that patient care is not harmed in any way. Contrary to isolated claims of success by several Health Information Technology vendors, rapid implementation of any CIS, let alone one that changes the way that orders are written and conducted, should not be attempted unless planning has been thorough and resources are abundant.^13,14 Furthermore, time is needed during the implementation process to evaluate whether the changes in workflow are positive or negative, safe or unsafe, and more or less efficient. This cannot be done in a few days, even on a single ward. Experience with countless previous CIS and CPOE implementations has shown that not all changes in workflow represent improvements (eg, see refs ^{15 and 16)}.
   
2. "After CPOE implementation, order entry was not allowed until after the patient had physically arrived to the hospital and been fully registered into the system."—Although accurate patient registration is clearly important to patient safety, the care and treatment of a severely ill patient should never be made to wait for a computer system. Analysis by multidisciplinary teams regarding the workflows that were successful before system implementation should have led clinicians and system administrators to develop a means to allow clinicians to continue to treat patients in the best way possible. This might require using old-fashioned paper orders in emergency situations with subsequent entry into the CPOE system after the patient is stable. Under no circumstances can the care of a patient be subordinated to the idiosyncrasies of a computer system.
   
3. "As part of CPOE implementation, all medications, including vasoactive agents and antibiotics, became centrally located within the pharmacy department."—It is important to recognize that the relocation of all medications including ICU vasoactive drugs to a central pharmacy, even if this were done for administrative reasons without implementation of a CIS and CPOE, could account for many of the adverse effects noted in this study. Considering that the hospital was already undertaking a huge disruptive organizational change affecting every caregiver in the institution, it was unfortunate that they would also try to institute a significant policy change regarding pharmacy workflow to accommodate CPOE more effectively. The additive effects of the CIS implementation, CPOE implementation, and pharmacy centralization could have been predicted to dramatically slow the delivery of drugs to all patients. Han et al's selection of interfacility transport patients as the patient population probably magnified the ill effects, because these patients can be predicted to be more severely ill at admission than other patients. Piggybacking organizational changes with significant potential for adverse workflow effects onto a CIS/CPOE implementation should be avoided if at all possible. CIS and CPOE are disruptive in and of themselves; interrelated workflows should be enhanced before implementation, when possible, or at least remain stable through the implementation period to minimize this disruption. Many hospitals use small "tests of change" on a single hospital unit to evaluate a new care process for both efficacy and potential adverse effects or unintended consequences. If pharmacy centralization had been evaluated in a single ICU in advance of CIS and CPOE implementation, it is likely that the operational problems described by Han et al would have been appreciated so that appropriate solutions could have been put into place.
   
4. "Because pharmacy could not process medication orders until they had been activated, ICU nurses also spent significant amounts of time at a separate computer terminal and away from the bedside."—This was clearly an unintended consequence of computerization. Careful sociotechnical analysis is required before clinical systems are implemented to ensure that caregivers can do their basic job at least as well and as safely as they could before computerization. Whenever an organization commits to moving from a well-honed manual care delivery system to a new computer-based model, the organization should carefully review and modify all applicable practices, procedures, policies, and bylaws. Mock use, full dress drills, and trial use on individual patients and wards should precede wider implementation. The role of the computer in health care is not to ensure that rules and regulations that had never been completely followed are now rigorously followed. Rather, computerization highlights the need for review, careful consideration of purpose, and clear definition of intended policies. Allowing a computer to enforce rules and regulations without first working through all implications and potential unintended consequences for patient care is a prescription for disaster.
   
5. "After CPOE implementation, because order entry and activation occurred through a computer interface, often separated by several bed spaces or separate ICU pods, the opportunities for such face-to-face physician-nurse communication were diminished."—Clear, 2-way, face-to-face communication is the hallmark of high-quality, collaborative patient care. Assuming that ambiguities in the treatment process are removed because all the orders are now legible and available in a central database is inappropriate and potentially dangerous. As the importance and complexity of the information to be communicated increases, the necessity of face-to-face communication increases dramatically. Careful sociotechnical evaluation, or even trial use on a single ward, probably would have brought to light the need for better system design. In addition, recent advances in the capability and utility of mobile terminals, tablet computers, and other devices such as hands-free, wireless communication systems¹⁷ may allow caregivers to remain in personal contact while doing their computer work. In fact, careful attention to these details has been shown to bring care teams together and make them more, not less, effective.^18,19 Computer systems need to be designed and implemented in such a way as to foster appropriate levels of communication, not hinder it.
   
6. "This initial time burden seemed to change the organization of bedside care. Before CPOE implementation, physicians and nurses converged at the patient's bedside to stabilize the patient. After CPOE implementation, while 1 physician continued to direct medical management, a second physician was often needed solely to enter orders into the computer during the first 15 minutes to 1 hour if a patient arrived in extremis."—Again, it is apparent that the consequences of CPOE were not appreciated until after implementation. Doubling physician workload, while slowing the delivery of life-saving medications, treatments, and diagnostic studies could not have been the original intent. Careful pilot studies could have revealed these issues so that solutions could have been devised before hospital-wide implementation. Although several studies have shown a small, but significant, increase in the time required on the part of clinicians to enter orders using a computer system, no one has ever documented a "doubling" of physician workload (see ref ¹² for a review of several studies).
   
7. "The physical process of entering stabilization orders often required an average of ten "clicks" on the computer mouse per order, which translated to 1 to 2 minutes per single order as compared with a few seconds previously needed to place the same order by written form. However, no ICU-specific order sets had been programmed at the time of CPOE implementation."—Methods of entering frequently occurring orders should be as easy and fast as on paper, especially for sets of orders, with the added benefits of 100% legibility, instantaneous transmission to the ancillary department, dose-range checking, and potential drug, laboratory, and condition interaction checking.²⁰ Organizations must take the time to implement validated standard order sets for routinely occurring critical conditions to speed the ordering and care process.²¹ Simply training users to overcome a steep learning curve and time-consuming process of entering many individual orders in the midst of critical patient care is not an optimal, or even effective, solution. Therefore, organizations must work to ensure that the clinical content (eg, order sets), default settings, and anticipated screen flows are designed, implemented, and tested to optimize speed, usability, and patient safety. Again, this may require trial use of the system on one or a few well-defined and well-staffed wards by a variety of users and over a prolonged period of time. This cannot be done in a few days. A more reasonable estimate of the amount of time to fully develop and vet clinical policies and order sets and to configure, test, and implement CPOE systems is 1 to 3 years.⁶
   
8. "Because the vast majority of computer terminals were linked to the hospital computer system via wireless signal, communication bandwidth was often exceeded during peak operational periods, which created additional delays between each click on the computer mouse."—Technical issues such as this can also be anticipated and tested in advance. On top of all the other process and workflow changes involved in CPOE implementation, inadequate or unreliable computing capacity can be particularly frustrating to clinicians and other end-users. Testing a new CIS under peak load conditions is an important task that can not be overlooked.
   

Although it is not clear whether the increase in mortality rate was a direct result of the CPOE implementation or other concomitant organizational and system changes, the CPOE implementation may well have been responsible, and we applaud the authors for reporting their findings and their problems with implementation. Although it is easy to criticize organizations for reporting implementation decisions that in retrospect seem flawed, we must respect, appreciate, and encourage other institutions to share their experiences so that everyone can learn from them. Likewise, regardless of whether the technology has a direct role in adverse effects from a specific deployment, we can take the opportunity from case studies such as this one to learn how to develop better systems.

The complexity of the decisions that must be made by CIS implementation and management teams demands iterative ongoing dialogue and feedback over time. There is no substitute for careful workflow and sociotechnical analysis, and beyond those there is no substitute for trial or pilot use of a system to uncover hidden flaws, unintended consequences, and adverse effects.

One must avoid the inclination or temptation to blame the adverse effects noted in this article solely on the particular CIS or CPOE system used. This would be the equivalent to stating that a particular brand of tool from a hardware store was unsafe because an injury occurred while someone was misusing it.

To return to the central question, how can well-intentioned organizations avoid these problems? Beyond the solutions noted above, several collective publications on CIS and CPOE implementation are available to guide institutions in designing a safe and effective process.^22–25 The advice in these guides, along with a careful evaluation of caregiver workflows and trial implementation in limited hospital areas, should allow safe implementation for everyone involved, especially patients.

A very important lesson in the Han et al article is the need to measure overall hospital mortality and adverse-event rates when implementing major new systems. Indeed, these mortality rates could go up even when rates of adverse drug events (ADEs) go down, as concomitantly reported from Han et al's institution.²⁶ However, these particular findings need to be interpreted cautiously, because they relied on "self-reported" ADEs, which may have little to do with the true underlying rate.²⁷ Traditionally, the efficacy of CPOE systems has been measured in terms of ADE rates; Han et al have reminded us that we must consider the larger scope of patient outcomes to accurately evaluate safety and efficacy.

We believe that the problems observed in Han et al's and others' CPOE deployments can be overcome by systematically developing and applying human-centered design, implementation, and evaluation methods adapted to point-of-care CISs. Such a systematic approach, as advocated by experienced practitioners in the field of medical informatics, has been achieved in aviation, the military, nuclear power, and the consumer software industry; it can, and must, be achieved in health care as well.

	FOOTNOTES

 
Accepted Mar 2, 2006.

Address correspondence to Dean F. Sittig, PhD, Kaiser Permanente Center for Health Research, 3800 N Interstate Ave, Portland, OR 97227. E-mail: dean.f.sittig{at}kp.org

Financial Disclosure: Dr Shabot is a consultant for Philips Medical Systems and the Cerner Corp and has performed research funded by Eli Lilly & Company. The other authors have indicated they have no financial relationships relevant to this article to disclose.

Opinions expressed in these commentaries are those of the authors and not necessarily those of the American Academy of Pediatrics or its Committees

	REFERENCES

TOP REFERENCES

 

1. Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system [published correction appears in Pediatrics. 2006;117:594]. Pediatrics. 2005;116 :1506 –1512[Abstract/Free Full Text]
2. Southon G, Sauer C, Dampney K. Lessons from a failed information systems initiative: issues for complex organisations. Int J Med Inform. 1999;55 :33 –46[CrossRef][ISI][Medline]
3. Goddard BL. Termination of a contract to implement an enterprise electronic medical record system. J Am Med Inform Assoc. 2000;7 :564 –568[Abstract/Free Full Text]
4. Wager KA, Lee FW, White AW. Life After a Disastrous Electronic Medical Record Implementation: One Clinic's Experience. Charleston, SC: Idea Group Publishing; 2002
5. Sittig DF, Krall M, Kaalaas-Sittig J, Ash JS. Emotional aspects of computer-based provider order entry: a qualitative study. J Am Med Inform Assoc. 200;12:561–567
6. Payne TH, Hoey PJ, Nichol P, Lovis C. Preparation and use of preconstructed orders, order sets, and order menus in a computerized provider order entry system. J Am Med Inform Assoc. 2003;10 :322 –329[Abstract/Free Full Text]
7. Massaro TA. Introducing physician order entry at a major academic medical center: I. Impact on organizational culture and behavior. Acad Med. 1993;68 :20 –25[ISI][Medline]
8. Scott JT, Rundall TG, Vogt TM, Hsu J. Kaiser Permanente's experience of implementing an electronic medical record: a qualitative study. BMJ. 2005;331 :1313 –1316[Abstract/Free Full Text]
9. Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc. 2004;11 :104 –112[Medline]
10. Santayana G. Life of Reason: Reason in Common Sense. New York, NY: Scribner's; 1905:284
11. McDonald CJ, Overhage JM, Tierney WM, et al. The Regenstrief medical record system: a quarter century experience. Int J Med Inform. 1999;54 :225 –253[CrossRef][ISI][Medline]
12. Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc. 2005;12 :505 –516[Abstract/Free Full Text]
13. CPOE improves patient safety at top-ranked pediatric hospital [press release]. December 2, 2005. Available at: www.cerner.com/public/NewsReleases_1a.asp?id=257&cid=4668. Accessed June 14, 2006
14. Baker ML. Management plays key role in success of electronic patient record system Ziff Davis Internet. May 26, 2004. Available at: www.eweek.com/article2/0,1895,1600999,00.asp. Accessed June 14, 2006
15. Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS. Physicians' decisions to override computerized drug alerts in primary care. Arch Intern Med. 2003;163 :2625 –2631[Abstract/Free Full Text]
16. Nebeker JR, Hoffman JM, Weir CR, Bennett CL, Hurdle JF. High rates of adverse drug events in a highly computerized hospital. Arch Intern Med. 2005;165 :1111 –1116[Abstract/Free Full Text]
17. Breslin S, Greskovich W, Turisco F. Wireless technology improves nursing workflow and communications. Comput Inform Nurs. 2004;22 :275 –281[Medline]
18. Ash JS, Stavri PZ, Dykstra R, Fournier L. Implementing computerized physician order entry: the importance of special people. Int J Med Inform. 2003;69 :235 –250[CrossRef][ISI][Medline]
19. Reddy M, Pratt W, Dourish P, Shabot MM. Sociotechnical requirements analysis for clinical systems. Methods Inf Med. 2003;42 :437 –444[ISI][Medline]
20. Bates DW, Boyle DL, Teich JM. Impact of computerized physician order entry on physician time. Proc Annu Symp Comput Appl Med Care. 1994:996
21. Ali NA, Mekhjian HS, Kuehn PL, et al. Specificity of computerized physician order entry has a significant effect on the efficiency of workflow for critically ill patients. Crit Care Med. 2005;33 :110 –114[CrossRef][ISI][Medline]
22. Lee F, Teich JM, Spurr CD, Bates DW. Implementation of physician order entry: user satisfaction and self-reported usage patterns. J Am Med Inform Assoc. 1996;3 :42 –55[Medline]
23. Osheroff JA, Pifer EA, Teich JM, Sittig DF, Jenders RA. Improving Outcomes With Clinical Decision Support: An Implementer's Guide. Chicago, IL: Healthcare Information and Management Systems Society; 2005
24. Metzger J, Fortin J. Computerized Physician Order Entry in Community Hospitals: Lessons From the Field. Oakland, CA: California Healthcare Foundation; 2003
25. Drazen E, Kilbridge P, Turisco F. A Primer on Physician Order Entry. Oakland, CA: California Healthcare Foundation; 2000
26. Upperman JS, Staley P, Friend K, Neches W, Kazimer D, Benes J, Wiener ES. The impact of hospitalwide computerized physician order entry on medical errors in a pediatric hospital. J Pediatr Surg. 2005;40 :57 –59[CrossRef][ISI][Medline]
27. Classen DC, Pestotnik SL, Evans RS, Burke JP. Computerized surveillance of adverse drug events in hospital patients [published correction appears in JAMA. 1992;267:1922]. JAMA. 1991;266 :2847 –2851[Abstract]

This article has been cited by other articles:

				J. M. Walker, P. Carayon, N. Leveson, R. A. Paulus, J. Tooker, H. Chin, A. Bothe Jr., and W. F. Stewart EHR Safety: The Way Forward to Safe and Effective Systems J. Am. Med. Inform. Assoc., May 1, 2008; 15(3): 272 - 277. [Abstract] [Full Text] [PDF]

This Article Extract Full Text (PDF) P3Rs: Submit a response Alert me when this article is cited Alert me when P3Rs are posted Alert me if a correction is posted Citation Map Services E-mail this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My File Cabinet Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via CrossRef Citing Articles via ISI Web of Science (4) Citing Articles via Google Scholar Google Scholar Articles by Sittig, D. F. Articles by Shabot, M. M. Search for Related Content PubMed PubMed Citation Articles by Sittig, D. F. Articles by Shabot, M. M. Related Collections Office Practice

	Home | My Pediatrics | Journal Information | Current Issue | Past Issues | Subscriptions & Services | Contact Us Click here for faster international access © 2006 American Academy of Pediatrics. All rights reserved.