PEDIATRICS Vol. 117 No. 4 April 2006, pp. 1452-1455 (doi:10.1542/peds.2005-3163)
Perceived Increase in Mortality After Process and Policy Changes Implemented With Computerized Physician Order Entry
S. Trent Rosenbloom, MD, MPHDepartments of Biomedical Informatics and Pediatrics
Vanderbilt University School of Medicine
Nashville, TN 37232
School of Nursing
Vanderbilt University
Nashville, TN 37235
Frank E. Harrell, Jr, PhD
Department of Biostatistics
Vanderbilt University School of Medicine
Nashville, TN 37232
Christoph U. Lehmann, MD
Department of Pediatrics and Division of Health Information Sciences
Johns Hopkins University
Baltimore, MD 21287
Joseph H. Schneider, MD, MBA
Department of Pediatrics
University of Texas Southwestern Medical Center
Dallas, TX 75390
Children's Medical Center
Dallas, TX 75235
S. Andrew Spooner, MD, MS
Department of Pediatrics
University of Tennessee College of Medicine,
Memphis, TN 38163
Kevin B. Johnson, MD, MS
Departments of Biomedical Informatics and Pediatrics
Vanderbilt University School of Medicine
Nashville, TN 37232
To the Editor.—
Han et al1 describe a retrospective study in which the mortality rate for interfacility transfers into an ICU was compared before and after implementation of a computerized provider order entry (CPOE) system. The authors found that the mortality rate increased from 2.8% (30 deaths of 1394 patient transfers during 13 months) before CPOE implementation to 6.6% (36 deaths of 548 transfers during 5 months) after. The authors conclude that the increased mortality was associated directly with modifications in standard clinical processes, including the following changes: (1) not allowing order communication until the patient was physically present and registered in the admitting system; (2) relocating medication dispensing to a central (rather than a satellite) pharmacy; (3) increasing the physical separation of nursing and physician staff during the time that orders were generated; (4) implementing computerized order entry; and (5) system-wide provider role changes to support the CPOE system.
Perhaps the most important lesson from this study is that there exists an intimate association between care-delivery processes and health information technology. Any shift in the methods used to manage patient care (such as implementing and using a CPOE system) is associated with significant changes in clinical workflows, communication among providers, and distribution of responsibilities.2–4 Decades of research in medical informatics have underscored the importance of this observation, a message that was not lost on the authors. In this study, they note that the increased unadjusted mortality may reflect problems with the process of change, including the extremely rapid implementation plan adopted by their organization. The authors describe other major changes in workflow and patient care processes that occurred coincident with the CPOE system implementation. For example, their institution changed its policies to prohibit providers from entering patient orders before the patient had physically arrived and had been registered. This change was not a function of the CPOE system but rather of how the institution chose to implement it. The authors list other similar workflow changes that occurred coincident with the CPOE system implementation, such as the elimination of bedside stocks of critical drugs, resulting in the need to request these drugs when needed from a remote pharmacy.
The authors' findings merit additional scrutiny because, as they noted, there are numerous limitations to the study's design. The primary study outcome, mortality rates, naturally change over time with trends that are sensitive to many factors including policy changes at the hospital, staffing ratios, and seasonal variations in disease. In a pre-post single-crossover design, as this study used, there is no way to disentangle trends associated with time from the effects of the CPOE system-implementation process. In cases such as this, investigators must aggressively adjust for all possible measured confounders, taking pains to identify and include all that might reflect changes occurring over calendar time, including seasonal effects, primary diagnoses, a large number of comorbidities,5 staffing levels, and any changes in care-delivery and hospital-workflow processes. This limitation is amplified by the concern that many of the statistical methods used in this study have known problems that can invalidate apparently significant statistical associations.6–8 For observational studies such as this one, in which the number of deaths is limited, the preferred method for adjusting for confounding is propensity score analysis.9 The propensity model should include all measured confounders regardless of whether they are statistically significant. A more rigorous approach that pays attention to identifying and including all known confounders in an appropriate statistical analysis might influence the outcomes observed in this study. It is also important to realize the very unique population of critically ill children in this study. Interfacilty transfers occur under a variety of conditions. Mortality in this group may be a result of factors including suboptimal diagnosis and management at the transferring site, delays in transfer, problems during transit, and other factors remote from the study site.
Implementing health information technology such as CPOE and electronic health record systems is a complex process.10 In the commentary that accompanied the article by Han et al, Gesteland et al emphasized this lesson: "Deploying a sophisticated clinical-applications platform including CPOE in 6 days is an audacious task and leaves little margin for error in adapting highly evolved work processes to the new environment."11 There exist well-established approaches to enabling high-risk organizational change.12 Untoward effects have been well described when the process is not performed appropriately.13–15 Additional studies need to be designed and conducted to understand whether there are unique implications of process change in ICU and other acute care environments and to determine how pediatric health care affects these implications.16,17 However, as noted recently by the Institute of Medicine,18,19 the preponderance of evidence to date strongly suggests that CPOE systems can reduce medication-related errors of commission and may be useful for improving compliance with errors of omission.20,21 Readers of the article by Han et al should not abort plans to implement CPOE systems under the belief that CPOE itself can increase harm; rather, they should proceed with implementations cautiously, applying the lessons about systems implementation learned and published by others. In this regard, a more appropriate title for the article might have emphasized that any increase in observed mortality at the authors' institution was associated with the process of implementing change rather than with the CPOE system itself.
All systems designed to deliver health care, computerized or not, will fail at certain points. The process for implementing new systems should be designed to identify potential failures and to assure that they do not result in sentinel or catastrophic events. Key steps in the implementation of CPOE systems, for example, should include detailed flowcharted analyses of current and proposed workflow processes, failure analyses conducted throughout the design and implementation process, usability testing in a controlled environment, and a stepwise rollout in which each subsequent institutional unit learns from challenges experienced by the previous unit. Pertinent system evaluations should supplement classic technical software and hardware analyses with human-interface and human-interaction testing. There also should be extensive end-user testing in realistic situations that takes place with a "frozen" software design. Finally, there must be well-designed "break-the-glass" functionality that allows users to do what they think is best for the patient even when the computer system does not allow it (eg, writing orders before a critically ill patient is registered in the hospital).22
The CPOE system-implementation process described by Han et al did not incorporate steps or elements known to ensure system dependability and usability. For example, CPOE systems commonly include tools designed to improve safety and time efficiency, such as "order sets."23 Order sets allow providers to select and order multiple related items such as medications and diagnostics tests with only a few mouse clicks or keystrokes. According to the authors, their institution chose an implementation approach that did not include order sets, although the commercially available system they implemented offered the ability to construct them. The authors also describe unforeseen technical problems that occurred during the rapid implementation process (eg, overloaded wireless networks that slowed down CPOE systems). Technical problems such as these should have been evident in testing that typically takes place before system implementation.
Over the past 5 years, numerous investigators have outlined case studies documenting successful and failed CPOE implementations.14,24 Many successful implementations have received the Nicholas E. Davies award (these case studies are available at www.himss.org/ASP/daviesAward.asp). Fortunately, there now exists enough shared and published experience with CPOE implementation that each institution should not need to rediscover problems that others have found. However, with the increasing demand for technology infrastructure in medicine, there continues to be an insufficient number of individuals trained in biomedical informatics to provide consultation. Efforts such as the American Medical Informatics Association's 10 x 10 program and the National Library of Medicine's short course in medical informatics, fellowship training programs, and career development grants all promise to improve the penetration of informatics knowledge in the pediatric workforce. Awards such as the Davies award recognize best practices for CPOE and electronic health record system implementation. Ongoing efforts by the Certification Commission for Health Information Technology (CCHIT) are designed to inform purchasers about the quality of commercial systems. These nationally visible products and programs add to the knowledge that can be applied to CPOE and electronic health record system implementation.
ACKNOWLEDGMENTS
The project was supported by US National Library of Medicine grant 1K22 LM008576-01 (to Dr Rosenbloom).
REFERENCES
1. Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2005;116
:1506
–1512
2. Braude RM. People and organizational issues in health informatics. J Am Med Inform Assoc. 1997;4
:150
–151
3. Lorenzi NM, Riley RT. Knowledge and change in health care organizations. Stud Health Technol Inform. 2000;76 :63 –69[Medline]
4. Lorenzi NM, Riley RT. Organizational issues = change. Int J Med Inform. 2003;69 :197 –203[CrossRef][Medline]
5. Stukenborg GJ, Wagner DP, Harrell FE Jr, et al. Hospital discharge abstract data on comorbidity improved the prediction of death among patients hospitalized with aspiration pneumonia. J Clin Epidemiol. 2004;57 :522 –532[CrossRef][Medline]
6. Sun GW, Shook TL, Kay GL. Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis. J Clin Epidemiol. 1996;49 :907 –916[CrossRef][Web of Science][Medline]
7. Harrell FE. Regression Modeling Strategies, With Applications to Linear Models, Survival Analysis and Logistic Regression. New York, NY: Springer; 2001
8. Greenland S. When should epidemiologic regressions use random coefficients? Biometrics. 2000;56 :915 –921[CrossRef][Web of Science][Medline]
9. D'Agostino RB Jr. Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Stat Med. 1998;17 :2265 –2281[CrossRef][Web of Science][Medline]
10. Kuperman GJ, Gibson RF. Computer physician order entry: benefits, costs, and issues. Ann Intern Med. 2003;139
:31
–39
11. Gesteland PH, Nebeker JR, Gardner RM. These are the technologies that try men's souls: common-sense health information technology. Pediatrics. 2006;117
:216
–217
12. Stablein D, Welebob E, Johnson E, Metzger J, Burgess R, Classen DC. Understanding hospital readiness for computerized physician order entry. Jt Comm J Qual Saf. 2003;29 :336 –344[Medline]
13. N.V.: Cedars slips on CPOE; The editors and reporters of Modern Physician chose the following as the 12 most important stories of the past year. Modern Physician. 2003;7 :19
14. Shane R. Computerized physician order entry: challenges and opportunities. Am J Health Syst Pharm. 2002;59
:286
–288
15. Sittig DF, Stead WW. Computer-based physician order entry: the state of the art. J Am Med Inform Assoc. 1994;1
:108
–123
16. Johnson KB. Barriers that impede the adoption of pediatric information technology. Arch Pediatr Adolesc Med. 2001;155
:1374
–1379
17. Johnson KB, Davison CL. Information technology: its importance to child safety. Ambul Pediatr. 2004;4 :64 –72[CrossRef][Web of Science][Medline]
18. Institute of Medicine, Committee on Quality in Healthcare in America. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 1999
19. Aspden P, Corrigan JM, Wolcott J, Erickson SM, eds. Patient Safety: Achieving a New Standard for Care. 1st ed. Washington, DC: National Academies Press; 2004
20. Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc. 1999;6
:313
–321
21. Potts AL, Barr FE, Gregory DF, Wright L, Patel NR. Computerized physician order entry and medication errors in a pediatric critical care unit. Pediatrics. 2004;113
:59
–63
22. Bates DW, Kuperman GJ, Wang S, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003;10 :523 –530[CrossRef][Web of Science][Medline]
23. McAlearney AS, Chisolm D, Veneris S, Rich D, Kelleher K. Utilization of evidence-based computerized order sets in pediatrics. Int J Med Inform. 2005; In press
24. Aarts J, Berg M. A tale of two hospitals: a sociotechnical appraisal of the introduction of computerized physician order entry in two Dutch hospitals. Medinfo. 2004;11 :999 –1002[Medline]
PEDIATRICS (ISSN 1098-4275). ©2006 by the American Academy of Pediatrics
CiteULike 
Connotea 
Del.icio.us 
Digg 
Facebook 
Reddit 
Technorati 
Twitter What's this?
This article has been cited by other articles:
|
|
 |
|
  D. Ludwick, D. Manca, and J. Doucette Primary care physicians' experiences with electronic medical records: Implementation experience in community, urban, hospital, and academic family medicine Can Fam Physician, January 1, 2010; 56(1): 40 - 47. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. M Walker, P. Carayon, N. Leveson, R. A Paulus, J. Tooker, H. Chin, A. Bothe Jr, and W. F Stewart EHR Safety: The Way Forward to Safe and Effective Systems JAMIA, May 1, 2008; 15(3): 272 - 277. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. I Harrison, R. Koppel, and S. Bar-Lev Unintended Consequences of Information Technologies in Health Care--An Interactive Sociotechnical Analysis JAMIA, September 1, 2007; 14(5): 542 - 549. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. Basaviah and S. Frost Update in hospital medicine. Ann Intern Med, November 7, 2006; 145(9): 685 - 691. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
