Impact of Computerized Prescriber Order Entry on the Incidence of Adverse Drug Events in Pediatric Inpatients

METHODS

This study used the same prospective methodology that was used in a previous ADE study conducted on the same patient care units during 2000–2001.⁶ An ADE was defined as an injury from a medicine or lack of an intended medicine (eg, omission of an indicated medication). A potential ADE was defined as an error that had the potential to result in at least a significant injury.¹⁴ Potential ADEs included errors detected before drug administration, as well as errors that were administered without causing significant adverse consequences. A preventable ADE was defined as all of the ADEs that were associated with a medication error. These ADEs and potential ADEs may have been preventable by CPOE or any other medication management system or strategy. Errors that were corrected before the medication being entered onto the medication administration record, ostensibly prevented by pharmacists, physicians, and nurses, were excluded during both the pre-CPOE and post-CPOE data-collection periods. Although errors that were the result of omissions were included, errors that were the result of delays were not specifically collected

All of the pediatric patients admitted to either the PICU or the general pediatric unit at the New York Presbyterian Hospital, Weill Cornell Medical Center, Komansky Center for Children's Health consecutively between April 1, 2004, and October 5, 2004, were included in this study. For comparison purposes, data from a previous published study on these same units during the pre-CPOE period from September 2000 to May 2001 were also included.⁶ The PICU and the general pediatric unit are located within a large metropolitan tertiary care center and are composed of 20 and 30 beds, respectively. The PICU is a multidisciplinary unit that functions as a major referral center for a large health care network and a major cancer center and provides medical and surgical intensive care to all of the pediatric patients, including infants.

A single clinical pharmacist, serving as the primary reviewer during each study period, prospectively identified events and potential events. The primary reviewer identified recordable events on a daily basis in a prospective manner via review of physician and nursing notes, pharmacy records, medication administration records, and laboratory data. In addition, nursing, medical, and pharmacy personnel were interviewed to resolve questions raised during medical chart review. Nursing and pharmacy supervisory personnel were also interviewed weekly to obtain any reports of additional adverse events. This was the sole job function of the reviewer during the time of data collection. Demographic and other case-specific data, including case-mix index (CMI; an index used to assess case severity), were collected for all of the admitted patients. These data served as the denominator for all of the further analysis.

The proximal cause and systems failure were assigned by the primary reviewer using previously published definitions.¹⁵ Examples of proximal causes include lack of drug knowledge, lack of information about the patient, and drug stocking. Examples of systems failures include drug-knowledge dissemination, dose and frequency standardization, and medication order tracking. A rating of event severity was assigned by the primary reviewer according to a previously published scale as significant, serious, or life threatening. This severity rating was primarily determined by the magnitude of dosing error, therapeutic index of the drug, and route of administration.^16,17 Determination as to whether an ADE was the result of a medication error, categorization of error types, and the validation process for events were identical to a process used previously.⁶ This process involved the independent assessment by 2 secondary reviewers and consensus among the primary and secondary reviewers regarding all of the events. Events that were not the result of a medication error (adverse drug reactions) were also included for comparison purposes.

Data from the previous ADE study conducted during 2000–2001 (pre-CPOE) were used as the baseline data for comparison purposes.⁶ The only substantive global change on these 2 patient care units between the time of completion of the previous study and initiation of the current investigation was the introduction of a CPOE system with decision support. There were limited enhancements of the unit dose drug distribution system (eg, enhanced unit dose preparation for some oral liquids) and other minor improvements to the medication management system (eg, override use enhancement, addressing look-alike sound-alike issues, and ongoing formulary revision). These enhancements were consistent with evolutionary changes generated by any organization actively focusing on ongoing quality improvement. However, these changes were minor compared with the revolutionary change experienced with the implementation of the CPOE system.

The CPOE module for medication order entry within the electronic medical chart (Eclipsys System 2000, SCC 1.3 M01; Eclipsys, Boca Raton, FL) was modified from the commercial product. A list of ∼200 medications commonly used in pediatric patients was selected for incorporation into the system. A collaborative effort among pharmacists, physicians, informatics specialists, nurses, and performance improvement specialists was undertaken to create a pediatric dosing table (PDT) that was used as a source database for the dosing recommendations triggered by the system after drug selection. Data reflected in the PDT were adapted from standard pediatric and neonatal references and expert opinion.^18,19 Each drug in the PDT had ≥1 line to stratify recommendations based on combinations of parameters, such as dosage form, gestational age, postnatal age, and weight and/or body surface area. The PDT was not stratified for physiologic function or disease state.

Default doses were developed to address the most common indication for any particular drug. For drugs with multiple indications and/or variable dosing recommendations, the system provided users with alternative dosing information in a text box incorporated into the order default screen. In addition, the system populated all of the medication order fields in the order default screen with the recommended dosing parameters outlined in the PDT. Doses that exceeded a preestablished maximum triggered an alert to the prescriber but did not alter the order or prevent the order from being executed. A dose-rounding function served to round doses to a degree appropriate for pediatric and neonatal patients. An alert was triggered notifying the prescriber when the rounded dose varied >5% from the originally calculated dose. The dosing weight for each patient was a required field within the system, and the system did not permit medication order entry without this required information. The system also used standardized concentrations for medicated intravenous infusions. Pediatric generalists, specialists, and intensivists reviewed and approved the content of the PDT, and the process and content were approved by the institution's formulary and therapeutics committee and medical board. The CPOE system was developed, approved, extensively tested, and implemented over a period of 20 months after the preliminary (pre-CPOE) data-collection period. The majority of this time was spent on gaining consensus related to appropriate defaults within the PDT. System refinements and user acclimation occurred for an additional 15 months before the follow-up (post-CPOE) data-collection period. This development process led to the post-CPOE study period commencing ∼36 months after the pre-CPOE period.

Orders from the CPOE module were electronically interfaced to the pharmacy system (Cerner Pharmnet; Cerner Corporation, Kansas City, MO) where the order was reviewed by a pharmacist and transmitted to the medication profile in the automated dispensing machines (Pyxis Medication Station 2000; Cardinal Health, Dublin, OH) located on patient care areas. In addition, active orders were directly transmitted to the medication administration record within the electronic medical chart. The study was approved by the organization's institutional review board.

RESULTS

Data for the post-CPOE period were collected from 1210 patient admissions between April and October 2004. Data for the pre-CPOE period were previously published and were collected from 1197 consecutive patient admissions between September 2000 and May 2001.⁶ Demographic data for these 2 populations are provided in Table 1. Small differences were noted in the age, number of medications, CMI, and inpatient distribution between the patient care units for the no-ADE population. During the post-CPOE period, there was an increased proportion of preventable ADEs and a decreased proportion of potential ADEs in the ICU. The length of stay (LOS), CMI, and distribution of patients within each nursing unit were also different between the 2 time periods for the potential ADE group.

Preventable ADE and potential ADE incidence rates are also provided in Table 1. The RRs of an adverse event are further depicted in Fig 1, by both event type and severity. There was a significant reduction in total ADEs, preventable ADEs, and potential ADEs after the implementation of the CPOE system. The RR of a preventable ADE was 0.56 (95% CI: 0.34–0.91), and that of a potential ADE was 0.37 (95% CI: 0.25–0.55) after the implementation of CPOE. Furthermore, there were reductions in all of the significant events (combination of preventable ADEs and potential ADEs) between the pre-CPOE (n = 127) and post-CPOE time periods (n = 44; RR: 0.34; 95% CI: 0.24–0.49). Similar reductions were found for all of those events rated as serious or life-threatening between the 2 time periods (pre-CPOE [n = 13] and post-CPOE [n = 3]; RR: 0.23; 95% CI: 0.07–0.80).

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 1

RR (95% CI) of adverse events in comparison with baseline (pre-CPOE) data.

Using a number needed to treat analysis, it was determined that CPOE was associated with the avoidance of 1 potential ADE every 20.2 admissions (95% CI: 13.9–30.0 admissions) and 1 preventable ADE every 59.0 admissions (95% CI: 36.2–96.1 admissions). Similarly, a potential ADE was avoided every 212 patient-days (95% CI: 144–313 patient-days), and a preventable ADE was avoided every 865 patient-days (95% CI: 534–1400 patient-days). Using the average daily census observed during the post-CPOE study period (41.5 patients), CPOE was associated with the avoidance of a potential ADE every 5 days and a preventable ADE every 20 days.

Logistic regression analyses were used to determine the impact of CPOE on rates of potential and preventable ADEs, after controlling for LOS, number of medications, CMI, and patient care unit (Table 2). To perform the analysis, the number of medications and CMI were converted into categorical variables by dividing them into quartiles relevant to each period (cutoff points for quartiles shown in Table 2). CPOE remained a significant independent predictor of decreased occurrence of preventable (odds ratio [OR]: 0.76; 95% CI: 0.59–0.97) and potential ADEs (OR: 0.56; 95% CI: 0.46–0.70). The results also showed that, in addition to CPOE, exposure to the top quartiles of number of medications (pre-CPOE: >8 medications; post-CPOE: >10 medications) and CMI (pre-CPOE: >2.85; post-CPOE: >2.43) were significant predictors of potential and preventable adverse events.

The most common error types are presented in Table 3. The character of the errors changed after the implementation of the CPOE system, with a notable reduction in overall errors, dispensing errors, and drug-choice errors. However, some types of dosing errors, particularly underdoses, continued to represent a substantial proportion of the total errors. The continued prevalence of underdose errors was particularly evident in the ADE group. The most common type of preventable event in both time periods was inadequate analgesia (pre-CPOE period: n = 28; post-CPOE: n = 29). All of the other types of preventable events were a minority (ie, ≤5 events in both data sets). After analgesics, the drug class most associated with preventable events was antibiotics. After CPOE implementation, there were reductions in adverse events associated with certain antibiotic drug classes, including aminoglycosides (12 vs 0) and cephalosporins (14 vs 2). The most common types of adverse events that were decreased in relation to aminoglycosides were potential ADEs (n = 11) related to medication order tracking (n = 6) and dose and frequency standardization (n = 5). For cephalosporins, the reductions in adverse events again involved potential ADEs (n = 13) most commonly related to medication order tracking (n = 8). Problems with medication order tracking before CPOE were because of the pharmacy not receiving discontinuation orders and subsequently continuing to dispense discontinued medication.

There were some differences in the proximal cause and systems failures between the 2 time periods. Events resulting from lack of drug knowledge were less prevalent in the post-CPOE group (n = 34) compared with the pre-CPOE group (n = 54; RR: 0.62; 95% CI: 0.40–0.96). Drug stocking as a cause of these events was eliminated after CPOE implementation (35 vs 0; P < .001 [χ²]), because of automation of discontinuation orders that prevented the dispensing of discontinued medication. The most frequent systems failures attributed to these events pre-CPOE were dose and frequency standardization (25.4%), medication order tracking (24.6%), and dose and identity checking (9.7%). After CPOE implementation, errors associated with dose and scheduling were reduced, and there was a shift in systems failure attribution for these events to drug-knowledge dissemination (60%) and medication order tracking (10%).

There were no differences in patient disposition among patients with preventable ADEs or potential ADEs between pre-CPOE and post-CPOE time periods. Patients with preventable ADEs or potential ADEs were less likely to be routinely discharged and more likely to be discharged to either another institution or to care under a home health care program in the pre-CPOE time period. There was also a difference in patient disposition between the no-ADE groups in the 2 time periods, with the post-CPOE group having a greater chance for nonroutine discharge. However, further analysis of the no-ADE group between the 2 time periods did not reveal significant differences for individual types of discharge disposition (eg, discharge to another institution, discharge to home health care, or deceased).

DISCUSSION

The results of this study demonstrate that implementation of a CPOE system with substantive decision support on inpatient pediatric patient units was associated with a reduction in both preventable ADEs and potential ADEs. The reduction in these adverse events was most likely related to the implementation of the CPOE system within the institution, because this was the only major process modification between the 2 time periods. Although there were some minor differences between the 2 data sets for some of the study variables (eg, CMI, LOS, and patient care unit distribution), logistic regression findings demonstrate that, after controlling for these variables, CPOE remained a significant predictor of lower adverse event rates.

The LOS remained nearly identical between the ADE and potential ADE groups within each time period, but the ∼7-day reduction in the post-CPOE period does suggest that patients with adverse events or potential adverse events in the latter study group may have been different. A previous study at this institution in this same population determined that exposure to >8 medications was a strong predictor of these events.⁶ Consequently, this variable was compared in the no-ADE groups between the 2 time periods to explain the difference in LOS. It was shown that the percentage of patients receiving >8 medications increased from 16.1% to 31.9% (P < .001) in the no-ADE group after CPOE implementation. This indicated that the change in adverse event incidence between the 2 time periods was associated with a shift of patients at higher risk for an event to the no-ADE group rather than an actual difference in disease severity between the pre-CPOE and post-CPOE populations. In addition, LOS in the overall patient cohort was not different. One possible explanation for this finding is that those patients with more complicated illnesses were less likely to experience an adverse event after CPOE implementation and that such patients were being discharged more promptly.

In addition to an overall decrease in adverse events, specific types of errors decreased after implementation of the CPOE system, including drug-choice errors and dispensing errors. These changes are likely a direct result of the CPOE system. For instance, the majority of dispensing errors within the potential ADE category were because of a lack of diligence in communicating discontinuation orders, particularly related to discontinued antibiotics, in the manual system used previously.⁶ With the advent of the CPOE system, the discontinuation order became automated, and this potential system failure was addressed.

Underdosing of analgesic medications continued to represent a large number of the total dosing errors after the implementation of CPOE. This dosing error was also noted in the pre-CPOE data set with approximately the same incidence. The opiate analgesic doses were coded in the PDT using the lower end of the standard dose range from pediatric references with the intention for users to titrate doses based on pain scale assessments.^18,19 The system was not designed to improve the manual system and to use rules recommending dose escalations for single or sequential suboptimal pain scores. This may have contributed to the continued observation of inadequate analgesia in the post-CPOE population. The development of continuous pain score assessment and dosing escalation recommendations through alerting may be a worthwhile endeavor for implementation and future study. Underdosing of systemic analgesics is a well-documented problem in the pediatric arena and will likely require decision support and cultural change to produce a significant improvement in analgesic dosing.^20,21

This study also confirms findings from a previous study in pediatric inpatients.⁶ Preventable ADEs did not necessarily predict excess resource use. When compared with patients with potential ADEs, it was demonstrated that the mean LOS was not significantly greater for the preventable ADE group during either time period. In addition, LOS was substantially greater for both of these groups compared with the no-ADE group. Because patients with potential ADEs have not actually experienced an injury from a drug, these data suggest that excess resource use, as reflected by LOS in this group, likely reflects a more complicated patient. This is also supported by greater CMI values and medication exposures in the ADE and potential ADE populations. It is apparent that both of these groups are more complicated than most pediatric inpatients and that these differences do not seem to be because of ADEs themselves. ADEs may be a result of complicated illness rather than a causal factor in the pediatric inpatient population.

Several studies of CPOE have been performed in recent years in both the adult and pediatric populations.^{8,9,12,13,22–25} Most CPOE studies have noted a reduction in either medication errors and/or potential ADEs, but none have been reported to reduce the overall rate of ADEs or preventable ADEs.^{8,9,11–13,22–25} However, 1 previous study has demonstrated that a computer-assisted antibiotic management program led to reductions in errors and adverse events.²⁶

The majority of these studies have not included clinical outcomes such as ADEs as measured variables but rather surrogate outcomes, such as errors, laboratory test use, and compliance.¹¹ The lack of a documented impact on ADEs in the majority of these studies is thought to be because of the fact that these are rare events and represent a distinct minority (∼1%) of all medication errors. For instance, a previous time series analysis of the impact of CPOE on medication errors and events found only 5 ADEs during the baseline period and a total of 19 such events during 3 different follow-up periods that were each 7 to 10 weeks in length.⁹ Furthermore, many studies may not have been adequately powered to detect a difference in the rates of these relatively rare events. The fact that this study has demonstrated a difference in preventable ADEs and potential ADEs with CPOE likely reflects the number of events captured through rigorous prospective identification rather than relying on the low yield from a voluntary reporting system. Negative results of the impact of CPOE on ADEs have been reported when ADE incidence is based on voluntary reporting, likely because of the limited number of events identified with a voluntary reporting system.¹²

Implementation of CPOE has also not routinely demonstrated success in improving patient outcomes. In particular, a few recent studies have actually demonstrated a continued high incidence of ADEs, increased mortality, or facilitation of medication errors after implementation of CPOE.^22,23,25 The reasons for CPOE failure were postulated to be multifactorial, although common themes within these studies were hastened implementation timelines, lack of clinical decision support, and lack of adequate process design. Assessing outcomes after a prolonged system refinement and acclimation phase in addition to the development of extensive decision support were likely key factors that contributed to the findings of this study. The most important decision support modifications were likely the provision of pertinent patient demographic information during the ordering process, drug knowledge in order-detail screens, dosing recommendations, and notification to the pharmacy of discontinuation orders. Similar findings may not be reproducible with commercially available CPOE systems with nominal decision support. It would seem logical that the less comprehensive the dosing decision support the less likely a CPOE system will have the ability to reduce preventable ADEs. The risk of failing to customize existing systems to assist with prescribing for pediatric patients is likely substantial. The complexity of decision support development, however, may preclude its development in institutions where substantive resources are not available to dedicate to these functions. Highly advanced systems will require substantial resource commitment on an ongoing basis from a wide variety of providers (eg, pharmacists, physicians, information technology personnel, and nurses) focusing on system monitoring, assessment, expansion, and improvement on an ongoing basis.

Using new technologic advances to reduce the potential occurrence of ADEs is perhaps more important in the pediatric population than for any other patient group. Children are particularly vulnerable to errors within the health care system.^5,12 Although consequences of this vulnerability are not well studied, emerging evidence indicates that the most severe outcomes may be amplified in the pediatric population. An analysis of mortalities associated with medication errors demonstrated that children aged 0 to 9 years represented the second greatest number and percentage of deaths (after patients aged 70–79 years) and the largest number of life-years lost when distributed by 10-year age cohorts.^27,28 Thus, minimizing errors in the pediatric population may be of heightened importance, because the pediatric population's vulnerability to dosing errors is significant and the downstream benefit in terms of life-years gained from successful error prevention is greater than for the adult population. Despite these critical benefits from ADE prevention in children, it is unfortunate that all of the commercially available CPOE systems require significant modification to effectively serve pediatric populations.

This research did have some limitations. Because we used an observational pre-post comparison, differences in outcomes between the 2 study periods could represent a difference in patient populations or other factors, including potential bias in the post-CPOE data-collection period, that were not accounted for in the logistic regression. Possible confounding variables could have been the minor medication management enhancements discussed previously. Another limitation of this study was the potential for interreviewer variability between the 2 study periods, because the sole primary reviewers were different individuals in the 2 study periods. This potential variability was mitigated by having the reviewer for the first study period train and review cases with the primary reviewer for the second study period. In addition, the data-collection periods were not seasonally matched, because the pre-CPOE data collection was performed from September through May. However, based on the current ADE literature, there is no evidence that adverse event incidence is subject to seasonal variation. Because the study did not collect data on errors that were corrected before medication being entered into the medication administration record, it is unknown whether certain types of these errors may have also been impacted by the CPOE system. These errors were excluded because the purpose of the study was to examine the incidence of ADEs and potential ADEs that were not being prevented by the existing patient care system during both data-collection periods. In addition, errors that were the result of delays were not a specific focus of this research and would require additional inquiry to determine the impact of CPOE on delays in therapy.

CONCLUSIONS

This study demonstrated that a CPOE system with substantive decision support was associated with a reduction in both ADEs and potential ADEs among pediatric inpatients. Some ADEs continued to occur in this patient population, particularly underdosing of analgesic medications, suggesting that additional system modifications will be necessary to affect the remainder of these events. Preventable events did not predict excess resource use and may have instead represented a sign, rather than a cause, of more complicated illness.