 |
|
 |
|
 |
 |
|
 |
 |
 |
 |
 |
 |
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Frank E. Harrell, Jr, PhD
Department of Biostatistics
Vanderbilt University School of Medicine
Nashville, TN 37232
Christoph U. Lehmann, MD
Department of Pediatrics and Division of Health Information Sciences
Johns Hopkins University
Baltimore, MD 21287
Joseph H. Schneider, MD, MBA
Department of Pediatrics
University of Texas Southwestern Medical Center
Dallas, TX 75390
Children's Medical Center
Dallas, TX 75235
S. Andrew Spooner, MD, MS
Department of Pediatrics
University of Tennessee College of Medicine,
Memphis, TN 38163
Kevin B. Johnson, MD, MS
Departments of Biomedical Informatics and Pediatrics
Vanderbilt University School of Medicine
Nashville, TN 37232
To the Editor.—
Han et al1 describe a retrospective study in which the mortality rate for interfacility transfers into an ICU was compared before and after implementation of a computerized provider order entry (CPOE) system. The authors found that the mortality rate increased from 2.8% (30 deaths of 1394 patient transfers during 13 months) before CPOE implementation to 6.6% (36 deaths of 548 transfers during 5 months) after. The authors conclude that the increased mortality was associated directly with modifications in standard clinical processes, including the following changes: (1) not allowing order communication until the patient was physically present and registered in the admitting system; (2) relocating medication dispensing to a central (rather than a satellite) pharmacy; (3) increasing the physical separation of nursing and physician staff during the time that orders were generated; (4) implementing computerized order entry; and (5) system-wide provider role changes to support the CPOE system.
Perhaps the most important lesson from this study is that there exists an intimate association between care-delivery processes and health information technology. Any shift in the methods used to manage patient care (such as implementing and using a CPOE system) is associated with significant changes in clinical workflows, communication among providers, and distribution of responsibilities.2–4 Decades of research in medical informatics have underscored the importance of this observation, a message that was not lost on the authors. In this study, they note that the increased unadjusted mortality may reflect problems with the process of change, including the extremely rapid implementation plan adopted by their organization. The authors describe other major changes in workflow and patient care processes that occurred coincident with the CPOE system implementation. For example, their institution changed its policies to prohibit providers from entering patient orders before the patient had physically arrived and had been registered. This change was not a function of the CPOE system but rather of how the institution chose to implement it. The authors list other similar workflow changes that occurred coincident with the CPOE system implementation, such as the elimination of bedside stocks of critical drugs, resulting in the need to request these drugs when needed from a remote pharmacy.
The authors' findings merit additional scrutiny because, as they noted, there are numerous limitations to the study's design. The primary study outcome, mortality rates, naturally change over time with trends that are sensitive to many factors including policy changes at the hospital, staffing ratios, and seasonal variations in disease. In a pre-post single-crossover design, as this study used, there is no way to disentangle trends associated with time from the effects of the CPOE system-implementation process. In cases such as this, investigators must aggressively adjust for all possible measured confounders, taking pains to identify and include all that might reflect changes occurring over calendar time, including seasonal effects, primary diagnoses, a large number of comorbidities,5 staffing levels, and any changes in care-delivery and hospital-workflow processes. This limitation is amplified by the concern that many of the statistical methods used in this study have known problems that can invalidate apparently significant statistical associations.6–8 For observational studies such as this one, in which the number of deaths is limited, the preferred method for adjusting for confounding is propensity score analysis.9 The propensity model should include all measured confounders regardless of whether they are statistically significant. A more rigorous approach that pays attention to identifying and including all known confounders in an appropriate statistical analysis might influence the outcomes observed in this study. It is also important to realize the very unique population of critically ill children in this study. Interfacilty transfers occur under a variety of conditions. Mortality in this group may be a result of factors including suboptimal diagnosis and management at the transferring site, delays in transfer, problems during transit, and other factors remote from the study site.
Implementing health information technology such as CPOE and electronic health record systems is a complex process.10 In the commentary that accompanied the article by Han et al, Gesteland et al emphasized this lesson: "Deploying a sophisticated clinical-applications platform including CPOE in 6 days is an audacious task and leaves little margin for error in adapting highly evolved work processes to the new environment."11 There exist well-established approaches to enabling high-risk organizational change.12 Untoward effects have been well described when the process is not performed appropriately.13–15 Additional studies need to be designed and conducted to understand whether there are unique implications of process change in ICU and other acute care environments and to determine how pediatric health care affects these implications.16,17 However, as noted recently by the Institute of Medicine,18,19 the preponderance of evidence to date strongly suggests that CPOE systems can reduce medication-related errors of commission and may be useful for improving compliance with errors of omission.20,21 Readers of the article by Han et al should not abort plans to implement CPOE systems under the belief that CPOE itself can increase harm; rather, they should proceed with implementations cautiously, applying the lessons about systems implementation learned and published by others. In this regard, a more appropriate title for the article might have emphasized that any increase in observed mortality at the authors' institution was associated with the process of implementing change rather than with the CPOE system itself.
All systems designed to deliver health care, computerized or not, will fail at certain points. The process for implementing new systems should be designed to identify potential failures and to assure that they do not result in sentinel or catastrophic events. Key steps in the implementation of CPOE systems, for example, should include detailed flowcharted analyses of current and proposed workflow processes, failure analyses conducted throughout the design and implementation process, usability testing in a controlled environment, and a stepwise rollout in which each subsequent institutional unit learns from challenges experienced by the previous unit. Pertinent system evaluations should supplement classic technical software and hardware analyses with human-interface and human-interaction testing. There also should be extensive end-user testing in realistic situations that takes place with a "frozen" software design. Finally, there must be well-designed "break-the-glass" functionality that allows users to do what they think is best for the patient even when the computer system does not allow it (eg, writing orders before a critically ill patient is registered in the hospital).22
The CPOE system-implementation process described by Han et al did not incorporate steps or elements known to ensure system dependability and usability. For example, CPOE systems commonly include tools designed to improve safety and time efficiency, such as "order sets."23 Order sets allow providers to select and order multiple related items such as medications and diagnostics tests with only a few mouse clicks or keystrokes. According to the authors, their institution chose an implementation approach that did not include order sets, although the commercially available system they implemented offered the ability to construct them. The authors also describe unforeseen technical problems that occurred during the rapid implementation process (eg, overloaded wireless networks that slowed down CPOE systems). Technical problems such as these should have been evident in testing that typically takes place before system implementation.
Over the past 5 years, numerous investigators have outlined case studies documenting successful and failed CPOE implementations.14,24 Many successful implementations have received the Nicholas E. Davies award (these case studies are available at www.himss.org/ASP/daviesAward.asp). Fortunately, there now exists enough shared and published experience with CPOE implementation that each institution should not need to rediscover problems that others have found. However, with the increasing demand for technology infrastructure in medicine, there continues to be an insufficient number of individuals trained in biomedical informatics to provide consultation. Efforts such as the American Medical Informatics Association's 10 x 10 program and the National Library of Medicine's short course in medical informatics, fellowship training programs, and career development grants all promise to improve the penetration of informatics knowledge in the pediatric workforce. Awards such as the Davies award recognize best practices for CPOE and electronic health record system implementation. Ongoing efforts by the Certification Commission for Health Information Technology (CCHIT) are designed to inform purchasers about the quality of commercial systems. These nationally visible products and programs add to the knowledge that can be applied to CPOE and electronic health record system implementation.
ACKNOWLEDGMENTS
The project was supported by US National Library of Medicine grant 1K22 LM008576-01 (to Dr Rosenbloom).
REFERENCES
This article has been cited by other articles:
 |
|
 |
|
  J. M. Walker, P. Carayon, N. Leveson, R. A. Paulus, J. Tooker, H. Chin, A. Bothe Jr., and W. F. Stewart EHR Safety: The Way Forward to Safe and Effective Systems J. Am. Med. Inform. Assoc., May 1, 2008; 15(3): 272 - 277. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. I. Harrison, R. Koppel, and S. Bar-Lev Unintended Consequences of Information Technologies in Health Care An Interactive Sociotechnical Analysis J. Am. Med. Inform. Assoc., September 1, 2007; 14(5): 542 - 549. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. Basaviah and S. Frost Update in hospital medicine. Ann Intern Med, November 7, 2006; 145(9): 685 - 691. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
||
|
||
Home | My Pediatrics | Journal Information | Current Issue | Past Issues | Subscriptions & Services | Contact Us Click here for faster international access © 2006 American Academy of Pediatrics. All rights reserved. |
||
|
||
