Abstract
CONTEXT. Pediatric and intensive care patients are particularly at risk for medication errors. Computerized physician order entry systems could be effective in reducing medication errors and improving outcome. Effectiveness of computerized physician order entry systems has been shown in adult medical care. However, in critically ill patients and/or children, medication prescribing is a more complex process, and usefulness of computerized physician order entry systems has yet to be established.
OBJECTIVE. To evaluate the effects of computerized physician order entry systems on medication prescription errors, adverse drug events, and mortality in inpatient pediatric care and neonatal, pediatric or adult intensive care settings.
METHODS. PubMed, the Cochrane library, and Embase up to November 2007 were used as our data sources. Inclusion criteria were studies of (1) children 0 to 18 years old and/or ICU patients (including adults), (2) computerized physician order entry versus no computerized physician order entry as intervention, and (3) randomized trial or observational study design. All studies were validated, and data were analyzed.
RESULTS. Twelve studies, all observational, met our inclusion criteria. Eight studies took place at an ICU: 4 were adult ICUs, and 4 were PICUs and/or NICUs. Four studies were pediatric inpatient studies. Meta-analysis showed a significant decreased risk of medication prescription errors with use of computerized physician order entry. However, there was no significant reduction in adverse drug events or mortality rates. A qualitative assessment of studies revealed the implementation process of computerized physician order entry software as a critical factor for outcome.
CONCLUSIONS. Introduction of computerized physician order entry systems clearly reduces medication prescription errors; however, clinical benefit of computerized physician order entry systems in pediatric or ICU settings has not yet been demonstrated. The quality of the implementation process could be a decisive factor determining overall success or failure.
- CPOE
- hospital information systems
- medical record systems
- ICU
- children
- patient safety
- medication errors
- mortality
According to the Institute of Medicine, medical errors lead to 44.000 to 98.000 deaths in the United States annually.1 Currently, prevention of medical errors receives a large amount of attention and presents a major challenge to health care. In particular, critically ill patients are vulnerable and at risk for medication prescription errors (MPEs). Within this population, neonatal and pediatric patients present an even more vulnerable group. A study by Kaushal et al2,3 underlined this by showing that potentially harmful errors occurred 3 times more frequently in pediatric than in adult patients. Moreover, an increasing number of drugs, regimen complexity, and the continuously growing knowledge base of drug indications and adverse effects create the need for automated systems to deliver clinical support.4 Use of computerized physician order entry (CPOE) systems could possibly address these problems. For example, it has been shown that computer support in drug dosing has resulted in more patients with drug concentrations in the therapeutic range, reduced time to achieve therapeutic benefits, and resulted in fewer adverse effects of treatment in adults.5 Computer systems, therefore, may support doctors in tailoring drug doses more closely to the needs of individual patients.
CPOE can also improve patient safety in several ways. First, CPOEs are obviously more legible than handwritten ones. Furthermore, CPOE can force physicians to include dose, route of administration, and frequency in the order before authorizing the prescription, thus resulting in better structured and more complete medication prescriptions. CPOE systems can be linked to databases with background information and deliver decision support by warning for drug-dosage errors, interactions, or contraindications.6 However, although it is generally assumed that CPOE systems decrease medication error rates and improve clinical outcome, unfavorable findings associated with CPOE have been reported as well.7 In a study by Han et al,8 the mortality rate in a pediatric population increased after CPOE implementation. Therefore, specific settings such as pediatric or neonatal care or complex environments such as ICUs could determine the eventual clinical effect of CPOE systems.
We performed a systematic review of the use of CPOE systems in the most demanding and complex situations, that is, adult ICUs, PICUs, and NICUs, and in general pediatric and neonatal care. Meta-analysis was performed to estimate effects on MPEs, adverse drug events (ADEs), and mortality rate. Factors associated with success or failure of CPOE systems were identified.
METHODS
This systematic review was conducted according to the criteria as defined in the Quality of Reporting of Meta-analyses (QUORUM) and MOOSE (Meta-analysis of Observational Studies in Epidemiology) statements.9,10
Literature Search
Studies were identified by searching PubMed, the Cochrane library, and Embase up to November 2007. The literature search strategy was performed by using the following search terms: (child*[tiab] or paediatr*[tiab] or pediatr*[tiab] or infant*[tiab] or toddler*[tiab] or “pre school”[tiab] or preschool[tiab] or adolescent*[tiab] or pediatrics[Medical Subject Headings (MeSH)] or child[MeSH] or infant[MeSH] or adolescent[MeSH] or intensive care units[MeSH] or intensive care units, neonatal[MeSH] or intensive care, neonatal[MeSH] or intensive care[tiab]) and (CPOE[tiab] or “computerized physician order entry”[tiab] or “computerized provider order entry”[tiab] or “computerized prescribing”[tiab] or “electronic prescribing systems”[tiab] or “computerized order entry”[tiab] or “computer order entry”[tiab] or “medical order entry systems”[MeSH]).
Study Selection
After title screening, we examined abstracts and selected articles that met all of the following inclusion criteria: (1) hospitalized children 0 to 18 years old and/or ICU patients (including adults); (2) intervention CPOE compared with no CPOE; and (3) randomized trial or observational cohort study design. Exclusion criteria were descriptive studies (ie, case reports, narrative reviews, comments, etc) and CPOE research in populations targeted at specific diseases. Literature lists of included articles were searched for possible additional studies.
Definitions
A CPOE system was defined as a computer-based system that automates the medication-ordering process to ensure standardized, legible, and complete orders. A clinical decision-support system consists of at least basic dosing guidance for medication, formulary decision support, and drug allergy, duplicate therapy, and drug-drug interaction checking.11 Clinical decision-support systems are built into most CPOE systems.3 An MPE was defined as any error in prescription of medication irrespective of outcome. Potential ADEs were defined as medication errors with significant potential to harm a patient without reaching a patient, and ADEs were defined as actual harm that resulted from a medication error.2
Data Extraction
The following data were extracted: year of study, study design, study period, whether the study was performed in an academic hospital, patient population (adult ICU, PICU, NICU, or pediatric ward), software manufacturer, presence of decision regarding support. With respect to the implementation process, use of classroom training and individual training and on-site support present after CPOE implementation was assessed.
Validity Assessment
Observational studies were evaluated by applying criteria from the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) statement.12 We determined validity by assessing whether control and intervention groups were defined, whether possible sources of confounding, selection bias, or misclassification were identified and/or adjusted for, whether outcome measures were clearly defined, whether the exact study period was mentioned, whether the implementation process was described, and whether original outcome data were available in the publication. Validity of randomized trials was assessed by using the criteria published by Jadad et al.13
Data Analysis
All data were analyzed on an intention-to-treat basis. Risk rates for MPEs were calculated by dividing the number of errors by the total number of prescriptions in the intervention and control groups, respectively. Risk rates for ADEs and mortality were calculated by the number of incidents divided by the population at risk in the 2 groups: CPOE and no CPOE. Using the risk rates in both groups, relative risk (RR) estimates were calculated along with 95% confidence intervals (CI). Pooled RR estimates were calculated by using a random-effects model. Heterogeneity was assessed by the I[r]2 statistic.14I2 describes the percentage of total variation across studies resulting from heterogeneity rather than chance. I2 ranges from 0% to 100%; a value of 0% indicates no heterogeneity, and larger values indicate increasing heterogeneity. All analyses were conducted by using Excel 2007 (Microsoft, Redmond, WA).
RESULTS
Search
Our literature search yielded 122 citations that were screened for relevance, which left 12 articles that were included in the systematic review (Fig 1). We also cross-referenced the results of our literature search with lists of studies published in another systematic review.15 This did not yield any additional studies that were not already found in our search. Although the studies of Han et al8 and Upperman et al16 took place in the same hospital, the outcomes were different and both, therefore, were included.
Study selection.
Included Studies
Among the 12 included studies, which are summarized in Table 1, there were no randomized trials. There was 1 controlled cross-sectional trial.17 Eight studies were retrospective,8,16,18–23 and 3 studies were prospective cohort studies.7,24,25 Of the included studies, 4 were performed with adult ICU patients,7,17,22,23 and 8 were performed with pediatric patients.8,16,18–21,24,25 Of those 8 pediatric studies, 4 were performed on a PICU and/or NICU,18–20,25 1 on a ward with a PICU,24 and 3 on a pediatric ward.8,16,21
Included Studies
Three of the 12 studies reported mortality as outcome,8,19,20 1 analyzed workflow,22 7 of them studied MPEs and/or ADEs,7,16,17,21,23–25 and 1 study18 reported 3 outcomes: medication turnaround times, radiology procedure completion time, and MPEs (only gentamicin dosages). The definitions of MPEs and ADEs varied considerably among studies (see Table 3, which is published as supporting information at www.pediatrics.org/content/full/123/4/1184).
Different kinds of CPOE software systems were used: Siemens (Munich, Germany), Eclipsys (Atlanta, GA), Cerner (Kansas City, KS), PHAMIS (Seattle, WA), WizOrder (Nashville, TN), and homegrown systems. Because of a lack of consistency among studies, quantitative data analysis across vendors was not possible.
In 7 studies, implementation of decision support was explicitly mentioned,8,16–19,24,25 in 3 studies there was no decision support,7,21,22 and 2 studies did not describe whether decision support was available.20,23 A quantitative data analysis on decision support also was not possible, either because the studies poorly described the decision-support systems or because of the different levels of decision support among studies.
There was considerable variation in timing and length of the periods in which outcome was measured without or before CPOE and with CPOE among studies (Fig 2). Five of the studies started their intervention period right after CPOE implementation.8,18–20,23 Therefore, a so-called learning-curve in these studies was included in the measurements. The other 7 studies did not include the period right after CPOE implementation in the measurement period.
Distribution of the study periods.
Adult ICU Studies
All 4 adult ICU studies described an intervention and a control group, assessed potential confounding, and mentioned quantitative outcome data on number of MPEs, ADEs, and/or mortalities. Study periods varied among the ICU studies (Fig 2). For 2 of the studies, the implementation process was not described,7,17 for 1 study it was mentioned only briefly,23 and for only 1 study was it described extensively.22 An increase in MPEs was observed by Weant et al23 during the initial period after CPOE implementation. Three studies showed a clinical beneficial effect.7,17,22 In the study by Colpaert et al,17 CPOE only had a beneficial effect when potential ADEs were taken into account.
Pediatric, PICU, and NICU Studies
In all 8 studies the intervention and/or control group were clearly defined. All studies reported patient and clinical characteristics that implied comparability between the intervention and control groups. The original outcome data could be extracted from all studies except that of Upperman et al.16 In this study, only aggregate outcome estimates were reported. Again, study periods varied considerably (Fig 2). King et al21 did not describe their implementation process, Potts et al25 and Holdsworth et al24 mentioned it briefly, and the other 5 authors8,16,18–20 described their implementation process more extensively.
Of 5 studies with MPEs and/or ADEs as outcome measures, CPOE conferred a significant beneficial effect in 3 studies,18,24,25 and in 1 study a nonsignificant beneficial effect was reported.16 In the study by King et al,21 the overall result was beneficial: MPEs decreased, as did ADEs, but potential ADEs increased. In the 3 studies with mortality rate as main outcome,8,19,20 results varied; in the study by Han et al8 the mortality rate increased, whereas Del Beccaro et al19 reported a nonsignificant decrease in mortality rate, and Keene et al20 reported a significant decrease in mortality rate.
Implementation Process
Four studies described classroom training before implementation, extensive individualized instruction, and on-site support during and after CPOE implementation.18–20,22 Two of those studies showed a significant beneficial effect of CPOE.18,22 In the other 2 studies, mortality rates did not increase after CPOE implementation.19,20 Han et al8 and Upperman et al reported 3 hours of classroom computer practice 3 months before CPOE implementation. In the Upperman et al16 study, CPOE had a positive effect on ADEs, but in the Han et al8 study, introduction of a CPOE system increased mortality rates.
Meta-analysis
A meta-analysis was conducted to pool the outcome measures: MPEs, ADEs (potential and actual ADEs taken together), and mortality rate (Table 2). MPEs were pooled, taking all studies together. ADEs and mortality rates were pooled for pediatric and neonatal studies only. There was a significant reduction in MPEs (RR: 0.08 [95% CI: 0.01–0.77]), uniformly observed in all studies. The number of potential and actual ADEs showed a nonsignificant decrease with the use of CPOE (RR: 0.65 [95% CI 0.40–1.08]). However, there was significant heterogeneity (I2 = 65%) among the studies. Quantitative analysis to explore the causes for this heterogeneity was not possible because of the limited number of studies available. Mortality rates were not significantly influenced by CPOE (RR: 1.02 [95% CI: 0.52–1.94]). This was observed in all studies except for the study by Han et al.8 In that study, an RR of 2.35 (95% CI: 1.51–3.65) was observed. Even after adjustment for possible confounders, the mortality risk associated with CPOE remained elevated (odds ratio: 3.28 [95% CI: 1.94–5.55]).
Meta-analyses
DISCUSSION
In this systematic review we affirmed the important potential of CPOE systems to reduce MPEs. However, to what extent the application of CPOE systems actually results in clinical benefit remains to be established. Our meta-analysis showed a nonsignificant and heterogeneously distributed reduction in ADEs. Overall, mortality did not seem to be affected by the use of CPOE. The implementation process without individual practice and in-house support after CPOE-implementation could be related with unfavorable clinical outcome.
This is the first systematic review that concentrates on the effects of CPOE on pediatric care and critical care in general. It is necessary to specifically focus on these groups because of their high vulnerability and the complexity of their treatments. We pooled results on MPEs, taking pediatric non-ICU, PICU, NICU, and adult ICU studies together, because of the involved complexity of the prescription process mentioned above. We assumed that the effect of CPOE systems on MPEs would be mainly influenced by the level of demand posed by the setting in which the CPOE system was used and the complexity of the patients. These patients probably demand a nonordinary CPOE system to improve MPEs and patient outcome, including ADEs. Obviously though, pediatric non-ICU, PICU, NICU, and adult ICU patients are quite different, and it would be interesting to distinguish between these groups and study them in more detail with regard to clinical outcome. Unfortunately, only a limited set of clinical outcome data restricted to pediatric and neonatal patients was available.
It is evident that CPOE gives rise to better structured and more clearly legible prescriptions. The dramatic decrease in MPEs experienced after CPOE implementation in different studies clearly illustrates this aspect. Moreover, improvement in communication between physicians, nurses, and pharmacists has been shown as well.22 Ordering and prescribing by CPOE have been found to be more efficient than handwritten prescribing. Although it might be expected that CPOE systems can introduce new errors, in the present study this was not demonstrated. However, reductions in MPEs did not directly result in reduction in clinically relevant ADEs or improvement of clinical outcome.
The increase in mortality rates associated with the introduction of a CPOE system as reported by Han et al,8 has been discussed extensively in the literature.19,20,26 Del Beccaro et al19 studied the exact same CPOE system as Han et al but did not find a significant change in mortality rates. Ammenwerth et al26 compared these 2 studies and stated that there were important differences in design and implementation of these studies. Han et al studied CPOE use in a more critically ill and much younger patient population compared with Del Beccaro et al. Furthermore, Han et al only studied 5 months after CPOE implementation, whereas Del Beccaro et al extended their postimplementation study period to 13 months. The longer study period of Del Beccaro et al may have averaged out a potentially higher error rate in the first few months after CPOE implementation (learning curve). Besides Del Beccaro et al and Han et al, Keene et al20 also studied the effect of CPOE introduction in a critically ill pediatric population with comparable results to those of Del Beccaro et al. Potential causes of the increase in mortality rate in the study by Han et al have been hypothesized as slowing down of adequate patient treatment resulting from (1) the inability to register patients during transport to the hospital (medication could only be ordered when the patient had arrived in the hospital), (2) an increase in time needed to enter orders, (3) a reduction in verbal communication, (4) drug relocation from ward to central pharmacy, and (5) technical problems with network connections.8,19,20,26 Most of these causes cannot be attributed to the CPOE system itself but resulted from the implementation process.
As can be concluded from the previous paragraph, the implementation of a CPOE system could be critical. We argue that 3 hours of training 3 months before the implementation day (Del Beccaro et al19 and Han et al8) is far from sufficient. House staff cannot learn enough in just 3 hours, and 3 months later they probably will have forgotten most of what they did learn.
Seven systematic reviews about CPOE have been published as yet,3,11,15,27–30 but none of them concentrated on CPOE in a pediatric and ICU population, which represent the most demanding and complex situations. For 1 study the effect of CPOE on medication safety in general was described,3 for 1 clinical decision support and clinicians’ behavior were described,29 for 1 the effect on time records in clinical staff was studied,30 1 focused on the effect on pathology services,28 1 studied costs, adherence, and safety in a noncritical adult population,27 and 1 focused on decision support and examined cost-effectiveness.11 Only 1 of these 7 reviews examined the use of CPOE in a pediatric and/or critically ill population.15 However, this review did not assess the exclusive effects of CPOE systems on enhancing medication safety but, rather, investigated other interventions as well. In addition, this review applied other inclusion criteria and so included studies that differed from ours.
Ideally, a large randomized trial would provide valid evidence for the effect of CPOE systems on patient safety and clinical outcome. However, because of the nature of the intervention, a randomized trial would be practically nearly impossible to conduct. Therefore, most studies were based on a before/after design; however, this design permits limited conclusions about the causative nature of observed associations between CPOE introduction and change in outcome. More valid effect estimates could be obtained by using a “controlled before/after” design in a multicenter setting. An intervention setting with and a control setting without the intervention are both followed in time. Observed differences before and after the intervention, thus, can be adjusted for general changes in time in the control setting. Furthermore, in future studies, strict criteria should be used to define MPEs and ADEs, and methods of detecting and evaluating should be clearly described. We found definitions of detection and evaluation of MPEs and ADEs to vary widely among studies, which possibly led to variable results and making comparison between studies difficult (Table 3). Finally, intervention data should preferably be collected directly after CPOE implementation to make assessment of a potential learning curve possible.
CONCLUSIONS
CPOE systems indisputably reduce MPEs effectively. However, as to what extent this results in improved patient safety and better clinical outcome remains to be established. The implementation process of CPOE systems requires specific attention, because this may be associated with adverse outcome. Multicenter studies, preferably designed as controlled before/after studies, are needed to ascertain the role and requirements of CPOE systems in improving hospital care for pediatric and critically ill patients.
Footnotes
- Accepted August 6, 2008.
- Address correspondence to Casper W. Bollen, MD, PhD, Wilhelmina Children's Hospital, University Medical Center Utrecht, Pediatric Intensive Care Unit, Room KG 1.319.0, PO Box 85090, 3508 AB Utrecht, Netherlands. E-mail: c.w.bollen{at}umcutrecht.nl
The authors have indicated they have no financial relationships relevant to this article to disclose.
REFERENCES
- Copyright © 2009 by the American Academy of Pediatrics
In this issue
Jump to section
Related Articles
Cited By...
- Systematic review of computerized prescriber order entry and clinical decision support
- Preventable Adverse Drug Events Among Inpatients: A Systematic Review
- Computerised medical record systems that guide and protect - reflections on the Bawa-Garba case
- Exploring the roots of unintended safety threats associated with the introduction of hospital ePrescribing systems and candidate avoidance and/or mitigation strategies: a qualitative study
- Improving the Efficiency of Care for Pediatric Patients Hospitalized With Asthma
- Pediatric Medication Safety in Adult Community Hospital Settings: A Glimpse Into Nationwide Practice
- Stepped-wedge cluster randomised controlled trial to assess the effectiveness of an electronic medication management system to reduce medication errors, adverse drug events and average length of stay at two paediatric hospitals: a study protocol
- Perceptions and experiences of the implementation, management, use and optimisation of electronic prescribing systems in hospital settings: protocol for a systematic review of qualitative studies
- Drug-Drug Interactions Among Hospitalized Children Receiving Chronic Antiepileptic Drug Therapy
- Understanding and evaluating the effects of implementing an electronic paediatric prescribing system on care provision and hospital work in paediatric hospital ward settings: a qualitatively driven mixed-method study protocol
- Healthcare professionals' perceptions of the facilitators and barriers to implementing electronic systems for the prescribing, dispensing and administration of medicines in hospitals: a systematic review
- Patient Safety and Quality Improvement: Terminology
- Effect of computerized prescriber order entry on pharmacy: Experience of one health system
- Computerised provider order entry combined with clinical decision support systems to improve medication safety: a narrative review
- Interventions to Reduce Pediatric Medication Errors: A Systematic Review
- Clinical Decision Support Alert Appropriateness: A Review and Proposal for Improvement
- Facilitators and barriers to quality of care in maternal, newborn and child health: a global situational analysis through metareview
- Support tools for paediatric inpatient prescribers: a review
- An introduction to quality improvement in paediatrics and child health
- Clinical pharmacy interventions in paediatric electronic prescriptions
- Use of FMEA analysis to reduce risk of errors in prescribing and administering drugs in paediatric wards: a quality improvement report
- The effect of a multifaceted educational intervention on medication preparation and administration errors in neonatal intensive care
- Long-term Reduction in Adverse Drug Events: An Evidence-Based Improvement Model
- Preventability of adverse drug events involving multiple drugs using publicly available clinical decision support tools
- The pros and cons of electronic prescribing for children
- Topics in Neonatal Informatics: Computerized Physician Order Entry
- Decrease in Hospital-wide Mortality Rate After Implementation of a Commercially Sold Computerized Physician Order Entry System