Objective. Computerized physician order entry (CPOE) has the potential to reduce patient injury resulting from medication errors. We assessed the impact of a CPOE system on medication errors and adverse drug events (ADEs) in pediatric inpatients.
Design. A retrospective cohort study.
Setting. Tertiary care pediatric hospital.
Participants. Pediatric inpatients on 3 medical and 2 surgical wards.
Intervention. CPOE system implemented on 2 medical wards and compared with 1 medical and 2 surgical wards that continued to use hand written orders.
Outcome Measures. Rate of medication error and ADEs before and after CPOE implementation.
Results. In 6 years, a total of 804 medication errors were identified with 18 ADEs, resulting in patient injury among 36 103 discharges and 179 183 patient days. The overall medication error rate (MER) was 4.49 per 1000 patient days. Before the introduction of CPOE, the MERs of the intervention versus control wards were indistinguishable (ratio = 0.93; 95% confidence interval [CI] = 0.76, 1.13). After the introduction of CPOE, the MER was 40% lower on the intervention than on the control wards (ratio = 0.60; 95% CI = 0.48, 0.74). On average, 490 patient days are required to see the benefit of one less medication error using CPOE. We did not demonstrate a similar effect of CPOE for ADEs (ratio of rate ratios = 1.30; 95% CI 0.47, 3.52).
Conclusions. The introduction of a commercially available physician computer order entry system was associated with a significant decrease in the rate of medication errors but not ADEs in an inpatient pediatric population.
Medication errors are an internationally recognized source of significant morbidity and mortality in hospitalized patients. In the United States alone, the Institute of Medicine report To Err Is Human: Building a Safer Health System estimated that between 44 000 and 98 000 people die each year from medical errors, whereas in the United Kingdom and Australia adverse events causing patient harm occurred in 10% to 16% of admissions.1–3 Medication use accounts for at least 20% of adverse events in hospitalized patients.4 Although the majority of medication errors do not result in patient harm, errors that result in a serious adverse drug event (ADE) are associated with increased length of hospital stay, extra costs, and mortality.5
In adults, ADEs occur at a rate of 5 per 100 medication orders.6 A similar error rate has been observed in children who are at risk for unique medication errors such as large, 10-fold errors in dosing.7,8 Children are at risk for medication errors attributable to weight-based dosing, off-label drug usage and preparation, limited reserve to withstand a dosing error, and limited ability to communicate with health care personnel to prevent an error or signal that an error has occurred.9,10 A review of prospective studies of ADEs in hospitalized children noted an overall incidence of 9.5%, with severe reactions accounting for 12% of the total.11 In 2 academic pediatric hospitals, the medication error rate (MER) was 6 per 100 medication orders, the majority of which occurred during physician medication ordering.7 Many ADEs are preventable (51% to 93%), as medication errors often occur during drug ordering and may be corrected, especially if the error is detected early in the order process.5,6,11,12
The significant MER has focused attention on understanding the causes of ADEs and developing methods to prevent them. Advances in information technology, such as computerized physician order entry (CPOE), hold promise of reducing medication-related errors. CPOE has been given a “medium strength of evidence” rating by the Agency for Healthcare Research and Quality as a safety practice to implement, and the Leapfrog Group is promoting CPOE in hospitals as 1 of 3 changes to improve patient safety in America.13,14 Despite these recommendations, there has been limited evaluation and publication of the impact of CPOE on ADEs in part because of the methodologic difficulties in trial design.15 Studies demonstrating CPOE effectiveness have shown a decrease in the rate of serious medication errors but no effect on ADEs.16–19
A reduction in the rate of serious medication errors conferred by CPOE would clearly have significant implications for hospitalized children. We assessed the impact of a commercially available CPOE system on medication errors and ADEs in pediatric inpatients at a tertiary care pediatric hospital.
Setting and Study Design
This retrospective cohort study was conducted at the Children’s Hospital of Eastern Ontario (CHEO), a tertiary care pediatric teaching hospital affiliated with the University of Ottawa, Ottawa, Canada. The inpatient units at CHEO average 6000 admissions and 38 000 inpatient days per year.
In March 1996, the CHEO implemented, on 2 medical inpatient units, a commercially available CPOE system developed by Eclipsys. The CPOE system was originally introduced as Carevision and underwent periodic product upgrade and is now commercially available as Sunrise Clinical Manager. Clinical decision support was not provided and the CPOE system interfaced with the laboratory system but not with the pharmacy computer. The intervention group consisted of the 2 medical wards on which CPOE was implemented. The control group consisted of 1 medical and 2 surgical wards that continued to use handwritten orders. Each ward unit is clinically independent with respect to nursing staff. Medical patients on the intervention wards are cared for by a team approach consisting of pediatric generalists and subspecialists with assistance from pediatric residents, interns, and medical students. Approximately 30% of the patients cared for by the medical team are admitted off-service to the nonintervention medical and surgical wards.
Medication Error Definition and Surveillance
A medication error was defined as any event involving medication prescription, dispensing, administration, or monitoring of medications irrespective of outcome. Medication errors at CHEO are reported in a standardized fashion by the adverse event reporting system on all inpatient floors. This is a passive reporting system that was constant throughout the duration of the study. After a medication error is detected, an incident report is completed by the nurse and physician involved. The events of the medication error are documented and the severity of patient harm is rated as none, mild, moderate, or severe. The incident report is sent to the Pharmacy Department. Each report is reviewed by the institution Quality Improvement Committee. The medication errors are then entered into a spreadsheet database (Microsoft Excel 97; Microsoft, Redmond, WA). MERs in both the intervention and control groups were analyzed for the 3 years before (April 1, 1993 to March 31, 1996) and 3 years following (January 1, 1997 to December 31, 1999) the implementation of CPOE (Fig 1). From April 1, 1996 to December 31, 1996, following the introduction of CPOE, medication errors were not assessed to allow for the piloting and training of CPOE and a major software upgrade in November 1996.
Two physicians (N.P., W.J.K.) accessed the medication error database and reviewed all original incident reports. Severity was reclassified based on patient impact as an ADE, potential ADE, or other. An ADE was defined as a medication error resulting in an injury to the patient (example: a 10-fold insulin overdose resulting in hypoglycemia). A potential ADE was defined as a medication error with the potential for patient injury where no actual harm occurred. Twenty random incident reports were independently rated by each of the 2 reviewers with good agreement (κ = 0.64; 95% confidence interval [CI]: 0.45, 0.82). All disagreements were resolved through discussion and consensus.
The total number of patient days and discharges during the study period was obtained from the CHEO discharge database. The CHEO uses Med2020 WinRecs abstracting software to collect inpatient information on discharge from the hospital. This database has a minimum dataset with definitions for each data element to ensure data quality and consistency.
We computed the ratios of rates before and after CPOE implementation, per 1000 patient days for medication error, potential ADE, and ADE. CIs for these rate ratios were computed under the assumption that the number of events per 1000 patient days followed a Poisson distribution.20 To evaluate the impact of introducing CPOE, we computed the ratio of the rate ratio for the intervention wards and the rate ratio for the control inpatient wards. To provide a clinical interpretation of the effect of CPOE on medication errors, we calculated an analog of the number needed to treat (NNT), representing the expected number of patient days to prevent 1 additional medication error using CPOE.21 The calculation is as follows: NNT = 1/(B × (1 − R)) where R is the ratio of rates of medication errors in the intervention versus the control group, and B is a baseline rate, which we took to be the rate of medication errors in the control group following the introduction of CPOE.22
During the 6-year study period, there were 36 103 discharges and 179 183 patient days. A total of 804 medication errors were identified with 18 adverse events resulting in patient harm, yielding an overall MER of 4.49/1000 patient days (Table 1).
The MERs before the introduction of CPOE for the intervention and control wards were indistinguishable (P value = .50; ratio = 0.93; 95% CI = 0.76–1.13; Table 2). The wards that implemented CPOE experienced a 40% reduction in medication errors when compared with the control wards (P value < .001; ratio = 0.60; 95% CI = 0.48–0.74). This change in rate ratios following the introduction of CPOE is statistically significant (P < .01; ratio of rate ratios = 1.54; 95% CI = 1.27–1.88). Our NNT analog revealed that CPOE would prevent 1 medication error every 490 (95% CI = 382–781) patient days.
Table 2 presents the rate ratios for ADEs and potential ADEs. We did not demonstrate an effect of CPOE implementation on ADE (P value = .6; ratio of rate ratios = 1.30; 95% CI: 0.47–3.52), although the power to detect such an effect was low because of the small number of ADEs (18) reported. We did note a larger decrease in potential ADEs on the control as compared with the intervention wards, which was statistically significant (P value < .001; ratio of rate ratios = 0.24; 95% CI: 0.09–0.68).
To our knowledge, this study is the first evaluation of CPOE on medication errors and ADEs in a pediatric inpatient population. We observed a 40% decrease in the MER on the wards that implemented CPOE; however, we were unable to demonstrate a similar effect on actual or potential patient injury. The lack of effect on patient harm may be explained by insufficient power to demonstrate an effect due to the small number of ADEs observed. Another explanation is that our institution saw a decline in potential ADEs and ADEs in both the intervention (from 11–7) and control wards (from 19–5), perhaps the result of other system changes to reduce medical error, such as ward-based pharmacists.
Despite recommendations to incorporate CPOE into everyday practice and the potential benefit of CPOE in pediatrics, there has been limited evaluation of its effectiveness. Existing studies in adult populations have been performed with “home-grown” systems and have shown a similar beneficial effect on medication errors without a demonstrable decrease in actual patient harm.16–19 Children are a vulnerable population at risk for medication error from both a physiologic and developmental viewpoint.23 The rate of medication errors and ADEs for pediatric inpatients are comparable to rates in adults with potential ADEs 3 times more common in children.7 Thus, as a group, children are at risk for ADE with a need to reduce this risk at least as great as in adults. To place in context the observed decrease in medication errors, at our pediatric tertiary care hospital with 40 000 patient days per year, we saw 1 less medication error every 490 patient days (95% CI: 382–781) and would therefore prevent 82 (95% CI: 51–105) medication errors per year. Although we did not demonstrate an actual decrease in patient harm, all medication errors have the potential to harm a patient. Even at a rate of 1 ADE for every 100 medication errors, with longer exposure to CPOE, we would expect to see a reduction in actual patient injury.24
The finding of this study must be interpreted in light of its limitations. We used retrospective cohort study design because a clinical trial in which patients are randomized to receive CPOE would be costly and methodologically difficult to implement.15 Medication error data were collected prospectively and we included reporting of medication errors before CPOE implementation. The similarity of MERs and ADEs before CPOE implementation (MERs per 1000-patient days, 4.48 vs 4.80) are helpful to assure that the populations were similar with respect to medication error. Another limitation is the passive reporting system for medication error at our institution; however, the inpatient wards were unaware of the study and the policy and procedure for medication error reporting were unchanged and occurred in a standardized fashion throughout the study. Finally, although our MER and number of ADEs are low, they are similar to the rate of medication errors reported in the pediatric literature (0.51–17.7 per 1000 patient days), especially when we did not include medication errors occurring in the pharmacy, emergency department, or intensive care units.25–27
The introduction of CPOE is associated with a decrease in the rate of medication error. A similar effect on the more serious but rarer ADE was not discovered. CPOE holds promise as an intervention that may improve patient safety but requires further evaluation of the benefit and costs before widespread adoption.
We thank Nicholas J. Barrowman, PhD, for his statistical assistance and Alex E. MacKenzie, MD, PhD, for his review of the manuscript.
- Received December 10, 2002.
- Accepted February 21, 2003.
- Reprint requests to (W.J.K.) Division of Pediatric Medicine, Children’s Hospital of Eastern Ontario, 401 Smyth Rd, Ottawa, Ontario, Canada K1H 8L1. E-mail:
This work was presented in part at a meeting of the Pediatric Academic Society; May 2002; Baltimore, MD.
- ↵To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 1999
- An organisation with a memory. Report of an expert group on learning from adverse events in the NHS Department of Health. 2000. Available at: http://www.doh.gov.uk/cmo/orgmem.pdf. Accessed September 26, 2002
- ↵Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients: results from the Harvard Medical Practice Study II. N Engl J Med.1991;342 :377– 384
- ↵Conroy S, Choonara I, Impicciatore P, et al. Unlicensed and off-label drug use in pediatric wards in different European countries. BMJ.2000;320 :79– 82
- ↵Levine SR, Cohen MR, Blanchard NR, et al. Guidelines for preventing medication errors in pediatrics. J Pediatr Pharmacol Ther.2001;6 :426– 442
- ↵Shojania K, Duncan B, MacDonald K, Wachter RM, eds. Making Health Care Safer: A Critical Analysis of Patient Safety Practices. Rockville, MD: Agency for Healthcare Research and Quality; 2001. Evidence Report/Technology Assessment No. 43; AHRQ Publ. No. 01-E058
- ↵Deeks JJ, Altman DG, Bradburn MJ. Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. In: Egger M, Davey Smith G, Altman DG, eds. Systematic Reviews in Health Care. Meta-analysis in Context. 2nd ed. London, United Kingdom: BMJ Books; 2001:285–312
- ↵Folli HL, Poole RL, Benitz WE, Russo JC. Medication error prevention by clinical pharmacists in two children’s hospitals. Pediatrics.1987;79 :718– 722
- Ross LM, Wallace J, Paton JY. Medication errors in a pediatric teaching hospital in the UK: five years operational experience. Arch Dis Child.2000;83 :492– 497
- Copyright © 2003 by the American Academy of Pediatrics