Skip to main content

Advertising Disclaimer »

Main menu

  • Journals
    • Pediatrics
    • Hospital Pediatrics
    • Pediatrics in Review
    • NeoReviews
    • AAP Grand Rounds
    • AAP News
  • Authors/Reviewers
    • Submit Manuscript
    • Author Guidelines
    • Reviewer Guidelines
    • Open Access
    • Editorial Policies
  • Content
    • Current Issue
    • Online First
    • Archive
    • Blogs
    • Topic/Program Collections
    • AAP Meeting Abstracts
  • Pediatric Collections
    • COVID-19
    • Racism and Its Effects on Pediatric Health
    • More Collections...
  • AAP Policy
  • Supplements
    • Supplements
    • Publish Supplement
  • Multimedia
    • Video Abstracts
    • Pediatrics On Call Podcast
  • Subscribe
  • Alerts
  • Careers
  • Other Publications
    • American Academy of Pediatrics

User menu

  • Log in
  • Log out
  • My Cart

Search

  • Advanced search
American Academy of Pediatrics

AAP Gateway

Advanced Search

AAP Logo

  • Log in
  • Log out
  • My Cart
  • Journals
    • Pediatrics
    • Hospital Pediatrics
    • Pediatrics in Review
    • NeoReviews
    • AAP Grand Rounds
    • AAP News
  • Authors/Reviewers
    • Submit Manuscript
    • Author Guidelines
    • Reviewer Guidelines
    • Open Access
    • Editorial Policies
  • Content
    • Current Issue
    • Online First
    • Archive
    • Blogs
    • Topic/Program Collections
    • AAP Meeting Abstracts
  • Pediatric Collections
    • COVID-19
    • Racism and Its Effects on Pediatric Health
    • More Collections...
  • AAP Policy
  • Supplements
    • Supplements
    • Publish Supplement
  • Multimedia
    • Video Abstracts
    • Pediatrics On Call Podcast
  • Subscribe
  • Alerts
  • Careers

Discover Pediatric Collections on COVID-19 and Racism and Its Effects on Pediatric Health

American Academy of Pediatrics
Article

Adverse Events in the Neonatal Intensive Care Unit: Development, Testing, and Findings of an NICU-Focused Trigger Tool to Identify Harm in North American NICUs

Paul J. Sharek, Jeffrey D. Horbar, Wilbert Mason, Hema Bisarya, Cary W. Thurm, Gautham Suresh, James E. Gray, William H. Edwards, Donald Goldmann and David Classen
Pediatrics October 2006, 118 (4) 1332-1340; DOI: https://doi.org/10.1542/peds.2006-0565
Paul J. Sharek
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jeffrey D. Horbar
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Wilbert Mason
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Hema Bisarya
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Cary W. Thurm
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Gautham Suresh
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
James E. Gray
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
William H. Edwards
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Donald Goldmann
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
David Classen
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Info & Metrics
  • Comments
Loading
Download PDF

Abstract

OBJECTIVES. Currently there are few practical methods to identify and measure harm to hospitalized children. Patients in NICUs are at high risk and warrant a detailed assessment of harm to guide patient safety efforts. The purpose of this work was to develop a NICU-focused tool for adverse event detection and to describe the incidence of adverse events in NICUs identified by this tool.

METHODS. A NICU-focused trigger tool for adverse event detection was developed and tested. Fifty patients from each site with a minimum 2-day NICU stay were randomly selected. All adverse events identified using the trigger tool were evaluated for severity, preventability, ability to mitigate, ability to identify the event earlier, and presence of associated occurrence report. Each trigger, and the entire tool, was evaluated for positive predictive value. Study chart reviewers, in aggregate, identified 88.0% of all potential triggers and 92.4% of all potential adverse events.

RESULTS. Review of 749 randomly selected charts from 15 NICUs revealed 2218 triggers or 2.96 per patient, and 554 unique adverse events or 0.74 per patient. The positive predictive value of the trigger tool was 0.38. Adverse event rates were higher for patients <28 weeks' gestation and <1500 g birth weight. Fifty-six percent of all adverse events were deemed preventable; 16% could have been identified earlier, and 6% could have been mitigated more effectively. Only 8% of adverse events were identified in existing hospital-based occurrence reports. The most common adverse events identified were nosocomial infections, catheter infiltrates, and abnormal cranial imaging.

CONCLUSIONS. Adverse event rates in the NICU setting are substantially higher than previously described. Many adverse events resulted in permanent harm and the majority were classified as preventable. Only 8% were identified using traditional voluntary reporting methods. Our NICU-focused trigger tool appears efficient and effective at identifying adverse events.

  • trigger
  • adverse event
  • harm
  • NICU
  • patient safety

In the report To Err is Human,1 the Institute of Medicine concluded that between 44000 and 98000 lives are lost per year in US hospitals as a result of error. This estimate was developed in part from the seminal work of the Harvard Medical Practice Study, which estimated that 3.7% of all hospitalized patients in a New York state cohort experienced an adverse event (AE) related to medical therapy.2 Similar results were found in a cohort of Colorado and Utah patients.3 More recent data from the Harvard group using more sophisticated detection methods revealed a 6.5% rate of adverse drug events (ADEs) alone in the adult inpatient setting, with 33% of these events described as preventable.4 Application of these methods to a pediatric population revealed a slightly lower 2.3% ADE rate, with 19% described as preventable.5 Each of these more recent studies4,5 relied on voluntary and verbally solicited reports from hospital staff, medication administration records, and nonfocused retrospective chart review to identify medication errors and ADEs. A variety of methods have been used to identify medical errors and AEs, including chart review, voluntary reporting by health care providers, direct observation, and review of medical malpractice claims.

Recently, a different strategy, known as the trigger methodology, was shown to be superior to voluntary occurrence reports and conventional unfocused chart review in the identification of AEs in hospitalized adult patients.6,7 A trigger is defined as an “occurrence, prompt, or flag found on review of the medical record that ‘triggers’ further investigation to determine the presence or absence of an adverse event.”6,7 An example of a trigger would be the administration of naloxone to a patient, which would prompt a focused chart review for evidence of opioid-induced respiratory depression. The use of triggers, therefore, theoretically promotes a more focused and efficient chart review than an unfocused chart review and, thus, may identify more AEs. Studies using the trigger methodology have identified AE rates as much as 50 times higher than hospital-based occurrence reporting strategies8 and have identified AE rates in high-risk populations as high as 112 per 100 patients with ADE rates of 20 per 100 patients.9 The only study to date using a trigger tool in the pediatric population identified an ADE rate of 11.1 per 100 pediatric patients when applied to 960 charts from 12 children's hospitals in North America.10 Because both the Institute for Healthcare Improvement and the Institute of Medicine are recommending a transition from measuring error to that of measuring harm,6–8,11 the trigger methodology is one promising and practical approach for effectively measuring harm associated with hospital-based health care.6–8,11

Despite the ability of the trigger method to effectively identify harm, it is clear that one specific tool does not work for every environment. For example, direct application of the adult-focused ADE trigger tool to the pediatric population identified an ADE rate of 9.3 per 100 patients, whereas a version adapted for pediatric populations identified 11.1 ADEs per 100 pediatric patients.10 In addition, in the pediatric ADE study, the rate of ADEs in NICU patients was only 12 per 100 patients,10 a rate only slightly higher than the mean rate for pediatric patients in all areas of the hospital. Recognizing that NICU patients are at high risk for AEs and that the pediatric ADE trigger tool may not reliably identify NICU-specific ADEs, we embarked on a study to develop and test a NICU-specific trigger tool to identify both ADEs, as well as AEs. The aims of this study were to (1) develop and test a NICU population-focused trigger tool for AE detection, (2) determine rates of AEs in NICUs in North America, and (3) identify the most frequent AEs in NICUs to provide the basis for developing a cohesive strategy to proactively prevent harm in NICU populations.

METHODS

Design/Setting/Patients

This study was a cross-sectional study, using retrospective chart review, in 15 NICUs (14 in United States, 1 in Canada; see “Acknowledgments” for participating sites). Subjects were selected randomly from a master list of eligible patients generated at each site. Patients were eligible for the study if they were in the NICU for a minimum of 2 days and discharged, transferred out, or died in the NICU between November 1, 2004, and January 31, 2005. Patients were excluded if they were in the NICU for <2 days or were discharged, transferred out, or died before November 1, 2004, or after January 31, 2005.

Intervention

Phase I: Initial NICU Trigger Tool Development

A panel of 6 practicing neonatologists, with expertise in patient safety (J.D.H., G.S., J.E.G., W.H.E., Bonnie Taylor, MD, and Robert Ursprung, MD) from the Vermont Oxford Network, the Agency for Healthcare Research and Quality-funded Center for Patient Safety in Neonatal Intensive Care, and the Child Health Corporation of America (CHCA) were convened to develop a list of AEs, including ADEs, relevant to NICUs. An initial list of 64 potential AEs was developed, which was narrowed down to 24 high-risk or high-volume AEs by using a modified Delphi method.12 From this list, a draft trigger tool consisting of 38 triggers thought likely to identify these 24 AEs was created.

Phase II: Pilot Testing of the Draft NICU Trigger Tool

The draft NICU trigger tool was piloted in 4 sites, using 42 charts, to further refine the tool and establish the final tool for full testing. Triggers with a low positive predictive value (PPV), ambiguity, or resulting in extreme inefficiencies during chart review were removed based on the results of this pilot test. Twenty-one triggers were removed, forming the final trigger tool consisting of 17 triggers, for full testing in phase III (Table 1).

View this table:
  • View inline
  • View popup
TABLE 1

Final Trigger List Used in NICU Trigger Tool Study

Phase III: Application of Final NICU Trigger Tool

NICUs were recruited from both the Vermont Oxford Network and CHCA to participate in the full NICU trigger tool trial. A data collection tool and instruction manual with standard processes and detailed definitions for each trigger (eg, hyperglycemia was defined as >200 mg/dL; electrolyte abnormalities were defined as serum sodium >150 mEq/L or <120 mEq/L, serum calcium <7 mg/dL, and serum potassium >8 mEq/L, etc) and each associated AE were then developed. To enhance accuracy of AE identification, each site was encouraged to participate in a training exercise (May 2005) consisting of chart review and data collection from 4 standardized practice patient charts. The final NICU trigger tool and a detailed standardized instruction manual were used for guidance. Three conference calls (May 2005) with participating sites allowed the principal investigator to facilitate a discussion of results of the exercise, clarify definitions, refine the instruction manual, and discuss strategies to make the chart review more efficient for the full trial. Ten of the 15 sites participated in the training chart exercise, and all 15 sites participated in ≥1 of the 3 training Webcasts, which reviewed and discussed the data and lessons learned from this exercise.

The full NICU trigger tool trial was launched with a Webcast in June 2005. The final trigger tool, an instruction manual with definitions, a chart audit tool, and instructions for using a standard random selection strategy to identify subjects, were sent to each site. Specifically, all of the eligible subjects at each site were included in a master list from which 50 subjects were randomly selected using the randomization function in Excel (Microsoft, Redmond, WA). We defined an AE (drug and nondrug related) as “an injury, large or small, caused by the use (including nonuse) of a drug, test, or medical treatment.”7,8 Similarly, an ADE was defined as “an injury, large or small, caused by the use (including nonuse) of a drug.”7,8 Within the context of this definition, a nosocomial infection, for example, would be classified as an AE, because evidence exists that most infections can be prevented in the hospital setting when appropriate prevention techniques are used. Preventability was defined based on the reviewer's interpretation and the neonatologists confirmation of whether the AE could have been prevented.

A neonatal nurse or neonatologist trained in chart review methods was designated locally to be the chart reviewer for the site. We encouraged the use of senior neonatologists or nurses experienced in chart review for local chart review, although no formal criteria were mandated. We offered several conference calls to both nurses and neonatologists to promote a more clear understanding of the definitions and methods. Selected charts were reviewed locally for the presence or absence of each of the 17 triggers. Each trigger identified prompted additional in-depth review for the presence of associated AEs. In addition, the chart reviewer was instructed to identify any AE that they discovered that was not associated with a trigger. Each identified trigger and any AE (whether associated with a trigger or not) was recorded on a standard data collection sheet. Sections of the charts were reviewed in the following order: discharge summary; International Classification of Diseases, Ninth Revision codes (if available in the chart); laboratory reports; physician orders and medication administration records; nursing flow sheets; and nursing/multidisciplinary progress notes. All of the AEs were evaluated for severity using categories based on the system used for classifying medication errors by the National Coordinating Council for Medication Error Reporting and Prevention (Table 2). 13 All of the AEs were evaluated by the local chart reviewer and neonatologist using local definitions to determine whether these AEs could have been prevented, identified earlier, or mitigated more effectively and whether an associated occurrence report was filed. Each AE and associated characteristics identified by the primary chart reviewer were then reviewed for accuracy by a local neonatologist, who was provided a summary of the event and any relevant clinical information that would allow him/her to confirm (or reject) that an AE occurred. In case of discrepancy, the neonatologists' interpretations of the preventability, earlier identification, and more effective mitigation were considered final.

View this table:
  • View inline
  • View popup
TABLE 2

Severity Categories of AEs Based on the System Used for Classifying Medication Errors by the National Coordinating Council for Medication Error Reporting and Prevention13

Once completed, the data collection sheet was sent, without patient identifiers, to CHCA for database entry (August 2005). Data were reviewed for completeness, and all of the discrepancies or questions were referred back to the chart reviewer at the appropriate site for resolution. Accuracy of data entry into the database was reviewed for 100% of data collection sheets received by the study coordinator by cross-checking all of the data sheets with the data entered.

Institutional review board (IRB) approval was obtained at each participating site, and IRB approval for the entire collaborative was obtained from the Western Institutional Review Board (Olympia, WA), an IRB that specializes in collaborative project IRB oversight.

Outcomes

Analysis of outcomes included (1) triggers per patient (combined hospital rates, crude individual hospital rates, gestational age adjusted individual hospital rates, and birth weight adjusted individual hospital rates), (2) trigger-PPVs (defined as the number of times a specific trigger independently identified an AE divided by the number of times a trigger was identified) individually and for the trigger tool in total, (3) AE rates per patient (combined hospital rates, crude individual hospital rates, gestational age adjusted individual hospital rates, and birth weight adjusted individual hospital rates), (4) severity of AEs (defined as the highest level of harm applicable using the National Coordinating Council for Medication Error Reporting and Prevention severity scale),11 (5) mean and median time for chart review, and (6) percentage of AEs that were preventable, could have been identified earlier, could have been mitigated more effectively, and were associated with a hospital occurrence report.

Chart review accuracy was assessed and defined as the ratio of the number of correct rating responses provided by each reviewer over the total number of correct responses for a total of 4 standardized charts. Correct responses were based on a standard defined by the scoring of the 4 standardized charts created and reviewed by the principal investigator (P.J.S.) before the completion of the training exercise.

Accuracy of Chart Review

One chart reviewer from 9 of the 14 non–principal investigator sites reviewed and scored 4 standardized practice charts. In aggregate, these reviewers accurately identified 88.8% ± 2.5% of all potential triggers and 92.4% ± 1.2% of all potential AEs.

Statistical Analysis

Wilcoxon rank-sum and Kruskal-Wallis tests were used when comparing AE rates between 2 or >2 groups, respectively, of gestational ages or birth weights. Functional association of gestational age/weight variables with the number of unique AEs was summarized with R2 from least-squares fit regression. Comparison of preventability by AE severity rating was determined by χ2 analysis. AE rates by hospital were risk adjusted using standard regression methods. Gestational age and birth weight were independently considered as risk adjustment variables; birth weight was determined to provide a better risk adjustment based on the fit of the model to the population examined. Statistical significance was defined as P < .05, assuming that adjustment for type I error is not required. Statistical analysis was performed with JMP (SAS Institute, Inc, Cary, NC).

RESULTS

Aggregate Data

A total of 749 randomly selected charts from 15 NICUs reflecting a total of 17106 NICU days were evaluated. Demographic characteristics for the study population and characteristics of the 15 participating sites are listed in Table 3. A total of 2218 triggers were detected, resulting in a mean rate of 2.96 triggers per patient (range: 0–26). A total of 554 unique AEs were identified (287 of these unique AEs were identified with >1 trigger, resulting in a total of 841 “nonunique” AEs; this number is necessary to calculate PPVs), resulting in a mean rate of 0.74 AEs per patient (range: 0–11 AEs per patient). The mean PPV of the trigger tool overall was 0.38 (range of PPVs for each individual independent trigger: 0.08–1.0; Table 4). The mean time for chart review was 20.5 minutes (range: 3–150 minutes), with a median time of 15 minutes, and the mean length of stay was 22.8 days (range: 2 to 250 days) with a median length of stay of 12 days. The mean AE rate per 1000 patient days was 32.4.

View this table:
  • View inline
  • View popup
TABLE 3

Study Population Demographics (n = 749) and Hospital Characteristics (n = 15 sites)

View this table:
  • View inline
  • View popup
TABLE 4

Positive Predictive Values by Trigger

Detailed Data

AE rates were stratified by birth weight, gestational age, and gender. AEs were significantly different (P < .0001) across gestational age groups (0–24 weeks as group 1, 25–28 weeks as group 2, and >28 weeks as group 3; P < .0001 for all pairwise comparisons). The regression model evaluating AE rates by gestational age revealed an R2 of 0.173. Similarly, AEs were significantly different (P < .0001) across birth weight groups ≤500 g (group 1), 501 to 1000 g (group 2), 1001 to 1500 g (group 3), and >1500 g (group 4; group 1 versus group 4, P < .0001; group 2 versus group 4, P < .0001; group 3 versus group 4, P < .001). The regression model evaluating AE rates by birth weight revealed an R2 of 0.135. AE rates stratified by gender showed no differences between males and females (0.75 vs 0.73 AE per patient; P = .514). Birth weight-adjusted hospital-specific AE rates varied from 0.18 AE per patient to 1.28 AE per patient (Fig 1). Similar outcomes are reflected in gestational age-adjusted AE rates (Fig 2). Unique AE rates did not differ based on freestanding versus not freestanding children's hospital (0.79 vs 0.73; P = .20), availability of general surgery versus no general surgery (0.75 vs 0.75; P = .92), availability of cardiothoracic surgery versus no cardiothoracic surgery (0.77 vs 0.73; P = .54), or availability of extracorporeal membrane oxygenation (ECMO) versus no ECMO (0.77 vs 0.73; P = .54).

FIGURE 1
  • Download figure
  • Open in new tab
  • Download powerpoint
FIGURE 1

Absolute and birth weight–adjusted AE rates per hospital.

FIGURE 2
  • Download figure
  • Open in new tab
  • Download powerpoint
FIGURE 2

Absolute and gestational age–adjusted AE rates per hospital.

AE Characteristics

The frequencies of all of the unique AEs identified are listed in Table 5. The most common AEs were nosocomial infections (n = 154; 27.8%), catheter infiltrates (n = 86; 15.5%) abnormal cranial imaging (n = 58; 10.5%), and unplanned extubations requiring reintubation (n = 46; 8.3%). Of the 554 unique AEs identified in this study, 40% were determined to be worse than a category E (category E is defined as “contributed to or resulted in temporary harm to the patient and required intervention”; Table 2), with 22.7% of all of the AEs resulting in permanent harm (6.5%; n = 36), requiring intervention to save the patient's life (6.5%; n = 36) or contributing to or resulting in the patient's death (9.7%; n = 54; categories G, H, and I).13 Fifty-six percent were deemed preventable, 16% could have been identified earlier, 6% could have been mitigated more effectively, and only 8% had a voluntary hospital occurrence report associated with the event. Only 6.1% (n = 34) of the 554 unique AEs identified did not have a trigger associated with them, and these AEs were diffuse in nature, making expansion of the trigger tool to capture these AEs not feasible (Table 5). Preventability of AEs was less likely when permanent harm (categories G and I: 31% and 15%, respectively) occurred as compared with when temporary harm (categories E, F and H: 62%, 71%, 56%, respectively) occurred (P < .0001).

View this table:
  • View inline
  • View popup
TABLE 5

Frequency of Unique AEs

DISCUSSION

This study, reviewing 749 charts representing a total of 17106 patient days from 15 NICUs, is by far the largest detailed review of NICU-associated AEs yet published. These data form the basis for a better understanding of the frequency of AEs, the most common types of AEs, and the subset of patients at most risk for AEs within the NICU setting. This information should help identify strategies to reduce harm to NICU patients.

Multiple previous studies have identified AEs in pediatric inpatients5,14–17 including at least 2 that focused on patients <2 years of age.17,18 However, none of these previous studies evaluated AEs in NICU patient specifically. For example, in a recent article, Woods et al17 used nontrigger methods to identify AEs in children. They identified an AE rate of 0.63 AEs per 100 patients (0.0063 per patient) in children <1 year of age. In contrast, our data using the trigger tool reveal an AE rate of 74 events per 100 patients in the NICU. Although the population studied by Woods et al17 was clearly a lower-risk population, the large discrepancy in AE rates between the study by Woods et al17 and our study suggests that our methodology is likely more effective at identifying AEs. Kaushal et al,5 in a study of ADEs (a subset of AEs) in pediatric patients, described potential ADE rates of 46 per 100 NICU patients (n = 54 patients) but did not identify the actual ADE rates for the NICU population. These 2 studies used the methodology developed by Brennan et al,2 which identified AEs and ADEs using a combination of “voluntary and verbally solicited reports from house officers, nurses, and pharmacists and by medication order sheet, medication administration records, and chart review of all hospitalized patients.”5 The critical difference in this methodology compared with the trigger methodology is that the chart review component is done in an nontrigger-based (unfocused) fashion. Unfocused chart review can result in inefficient, inaccurate, and highly variable results.7 Finally, in a recent study,19 1230 voluntarily submitted occurrence reports within the NICU setting were reviewed. Participating centers were encouraged to submit near misses, as well as AEs, and outcome was assessed in 673 occurrences. Category E events were identified in 167 (25%) of the reported occurrences and category F through I events in only 14 (1.9%). This predominance of low severity AEs is not surprising in that hospital-based occurrence reports have been shown to reflect error much more frequently than harm.8 The focus of that study was to develop an effective voluntary anonymous, Internet-based occurrence database for neonates and to evaluate its feasibility rather than to study rates of AEs per se.

The NICU trigger tool, and trigger tools in general, seem to be effective, efficient, and more robust than the more traditional methods of occurrence reports,8,19,20 nontriggered chart review,2,3,17 and administrative data analysis.14,15 Effectiveness is reflected in the relatively high PPV of the tool in total (0.38) and the low number of AEs identified (6.1%) that were not associated with the tool (although, admittedly, we cannot rule out the possibility that certain AEs, particularly those not associated with the triggers tested, were not identified during the study). The efficiency of the NICU trigger tool is reflected in the mean chart review time of 20.5 minutes (median: 15 minutes). We attribute the effectiveness and efficiency of the NICU trigger tool to the methodology used to develop the tool, in particular, the pilot testing, standardized practice chart review, the standardized definitions, and the standardized approaches to chart evaluation required by the instruction manual.

The findings from this study support the notion that the NICU houses an extremely high-risk population. Although these data do not reveal AE rates as high as those identified in the adult ICU setting,9 they are substantially higher than AE rates identified in any previous study of children.2,10,14–19 Further evidence of the high risk nature of this population is that 40% of AEs fall into the severity categories of F through I, which reflects substantial and often permanent harm (Fig 3). These severity data are also substantial higher than any previous study of the pediatric population (eg, in the Child Health Accountability Initiative pediatric ADE trigger study, only 2.8% of all ADEs were of categories F through I)10,14–19 and are of a similar magnitude to that of adults in the ICU setting.9 The unique nature of the NICU population is further highlighted by the rapid increase in risk based on lower birth weights and gestational ages and the unusual finding that preventability of any given event is inversely proportional to its severity. This has not been the case in previous studies that evaluated the relationship between severity and preventability of AEs.2,9,10

FIGURE 3
  • Download figure
  • Open in new tab
  • Download powerpoint
FIGURE 3

Level of AE severity.13

There are several limitations of this study. The most important limitation is the lack of a gold standard for AE detection with which we can compare our results.2,3 We, therefore, made the assumption that the AEs identified in this study were the sum total of all of the AEs that occurred in these patients. This is the only way that we could calculate a PPV for each of the triggers and the tool itself. In addition, this assumption was required to calculate the accuracy rates of chart reviewers for the 4 standard practice charts. Because the standardized practice chart exercise was designed to be a teaching exercise rather than to formally assess interrater reliability, we were limited to assessing the accuracy of trigger selection and AE association. Second, the use of the trigger tool, and in particular the determination of an event as being an AE, is inherently subjective and susceptible to certain biases that could affect the outcomes in uncertain ways.21 We attempted to standardize the use and interpretation of the findings by requiring review of 4 standard practice charts; discussion of the findings via conference call; clear definitions for triggers, AEs, and severity ratings; a detailed instruction manual with specific instructions, processes, and definitions; open and frequently used access to the primary investigator for questions; and by requiring a local neonatologist (otherwise not involved in the trial) to confirm all of the AEs. Despite these safeguards, there remains some subjectivity in the identification and interpretation of these triggers and events.21 This subjectivity is likely an important component of the variation in adjusted AE rates (range: 0.18–1.28 per patient). Additional studies in the interrater reliability of this and other trigger tools is warranted, including interrater reliability testing between nurse chart reviewers and neonatologists. Finally, the classification of preventability (as well as the ability to identify sooner and the ability to mitigate more effectively) of an event is subjective21 and was left to local sites to determine. Although we suspect substantial variability in a chart reviewer's interpretation of preventability, the assignment of preventability in aggregate remains critical to producing buy-in to use of the tool and, ultimately, toward redesigning systems to minimize the risk of AEs in the future.

Despite these weaknesses, trigger tools represent the most robust methodology developed to date to assess AEs. First, their ability to identify AEs, as opposed to error, is substantially greater than previously developed strategies, such as administrative database analysis, hospital-based occurrence reports (eg, in this study, occurrence reports only identified 8% of all of the AEs identified by the trigger tool), and nontriggered chart review. Second, trigger tools provide a consistent methodology that allows routine determination of AE rates over time. The ability to track AE rates over time in a methodologically consistent way is critical to quality improvement efforts at a local level. Furthermore, using the Institute for Healthcare Improvement recommendations for surveillance methods (using a random selection of 20 appropriate charts per month), the time investment to track NICU-associated AE rates would amount to only 400 minutes per month.22 Third, trigger tools are associated with a standardized instruction manual with standard definitions and, therefore, can be used to identify best practice sites for benchmarking purposes. Finally, trigger tools can potentially be automated, which could allow AE identification in real time. The relatively high PPV of the NICU trigger tool (0.38) suggests that it could form the basis of an efficient strategy for tracking AE rates, as well as identifying AEs in real time. Ideally this real-time identification could be used to mitigate AEs before they fully evolve.19 An example of this might be identification of a rise in serum creatinine (possibly because of a medication error) that is still in the normal creatinine range for age but is flagged automatically by a computer-based preprogrammed “trigger” before extensive renal damage is done. Several studies have successfully used an automated identification system with a trigger tool in the adult setting.6,23,24 The national movement toward electronic medical charts should enhance this effort.

CONCLUSIONS

This study is the first attempt to develop and evaluate a trigger tool to enhance the detection of AEs in the high-risk NICU setting. Results of this study identify low birth-weight, early gestational age infants as most susceptible to AEs, with the most common events being nosocomial infection, catheter infiltrates, abnormal cranial imaging, and accidental extubations requiring reintubation. More than one half of all identified AEs were classified as preventable, with 40% of events falling into the more severe harm categories of F through I. One or more AEs contributed to 27 of the 30 patient deaths in this study. Compared with other identification strategies, such as administrative databases, hospital-based occurrence reports, and nontriggered chart review, the trigger methodology seems superior in identifying AEs in multiple settings. Our data support this claim, comparing the NICU trigger tool to occurrence reporting within the NICU setting. These data should provide the groundwork for aggressive, evidence-based prevention strategies to decrease the substantial risk to one of our most vulnerable pediatric patient populations.

Acknowledgments

This work was sponsored by grants from the Agency for Healthcare Research and Quality (5 U18 HS013698-04 to Dr Sharek, Principal Investigator, and P20 HS11583 to Dr Horbar, Principal Investigator).

Participants and associated hospitals included the following: Akron Children's Hospital: Judith A. Ohlinger, RNC, MSN, and Amy Knupp, RN, MSN, CNS; Arkansas Children's Hospital: Robert E. Lyle, MD, and Marla Harrison, MD; Baylor University Medical Center: Pam S. McKinley, RN, BSN, and Craig T. Shoemaker, MD; Columbus Children's Hospital: Tamara Wallace, RNC, MS, CNNP, and Richard McClead, MD; Children's Hospital of Los Angeles: Philippe S. Friedlich, MD, MS, and Cyndi Atkinson, RNC; Cincinnati Children's Hospital Medical Center: Jon Fridriksson, MD, and Debbie Hershberger, RN, MSN; Jackson Madison County General Hospital: Donna-Jean Walker, MD, and Betty Beverly-Brown, RN, MSHA; Lucile Packard Children's Hospital at Stanford: Karen Kruse, RN, and William Rhine, MD; Sunnybrook and Women's Health Sciences Centre: Maureen Reilly, RRT, and Tina Romani, RN; Children's Hospital at Providence: Jeannie Bieganski, RN, and Karen Thompson, RPh; University of Minnesota Children's Hospital: Margaret Harder, BSN, MA, and Marla Mills, RN, CNP; Vermont Children's Hospital at Fletcher Allen Health Care: Candice Bullock, RNC, BA, and Ken Schroeter, DO; Wesley Medical Center: Sue Laudert, MD; Woman's Hospital of Baton Rouge: Dallas Estey, MSN, APRN/NNP, and Cynthia A. Voelker, MD; and Yakima Valley Memorial Hospital: Elizabeth L. Engelhardt, MD.

Footnotes

    • Accepted June 1, 2006.
  • Address correspondence to Paul J. Sharek, MD, MPH, Division of General Pediatrics, Department of Pediatrics, Stanford University School of Medicine, 700 Welch Rd, Suite 227, Palo Alto, CA 94304. E-mail: psharek{at}lpch.org
  • The authors have indicated they have no financial relationships relevant to this article to disclose.

AE—adverse event • ADE—adverse drug event • CHCA—Child Health Corporation of America • IRB—institutional review board • PPV—positive predictive value • ECMO—extracorporeal membrane oxygenation

REFERENCES

  1. ↵
    Kohn LT, Corrigan JM, Donaldson MS, eds. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 1999
  2. ↵
    Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients: Results of the Harvard Medical Practice Study I. N Engl J Med.1991;324 :370– 376
    OpenUrlCrossRefPubMed
  3. ↵
    Thomas EJ, Studdert DM, Burstin HR, et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care.2000;38 :261– 271
    OpenUrlCrossRefPubMed
  4. ↵
    Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse drug events and potential adverse drug events. JAMA.1995;274 :29– 34
    OpenUrlCrossRefPubMed
  5. ↵
    Kaushal R, Bates DW, Landrigan C, et al. Medication errors and adverse drug events in pediatric inpatients. JAMA.2001;285 :2114– 2120
    OpenUrlCrossRefPubMed
  6. ↵
    Rozich JD, Haraden CR, Resar RK. Adverse drug event trigger tool: a practical methodology for measuring medication related harm, Qual Saf Health Care.2003;12 :194– 200
    OpenUrlAbstract/FREE Full Text
  7. ↵
    Resar RK, Rozich JD, Classen DC. Methodology and rationale for the measurement of harm with trigger tools. Qual Saf Health Care.2003;12 :39– 45
  8. ↵
    Rozich JD, Haraden CR, Smithson K, Simmonds T, Resar RK. The occurrence of adverse events in the ICU measured by a trigger tool methodology. Jt Comm J Qual Saf. 2006; In press
  9. ↵
    Classen DC, Pestotnik SL, Evans RS, Burke JP. Computerized surveillance of adverse drug events in hospital patients. JAMA.1991;266 :2847– 2851
    OpenUrlCrossRefPubMed
  10. ↵
    Takata G, Currier K. Enhancing patient safety through improved detection of adverse drug events. Presented at: 13th Annual Forum on Quality Improvement in Health Care (Institute for Healthcare Improvement); December 9–12, 2001; Orlando, FL
  11. ↵
    Institute of Medicine Report. Patient Safety: Achieving a New Standard of Care. Washington, DC: National Academy Press; 2004. Available at www.iom.edu/CMS/3809/4629/16663.aspx. Accessed May 10, 2006
  12. ↵
    Williams PL. Webb C. The Delphi technique: a methodologic discussion. J Adv Nur.1994;19 :180– 186
  13. ↵
    National Coordinating Council for Medication Error Reporting and Prevention. Taxonomy of medication errors. Available at www.nccmerp.org. Accessed May 10, 2006
  14. ↵
    Miller MR, Elixhauser A, Zhan C. Patient safety events during pediatric hospitalizations. Pediatrics.2003;111 :1358– 1366
    OpenUrlAbstract/FREE Full Text
  15. ↵
    Slonim AD, LaFleur BJ, Ahmed W, Joseph JG. Hospital-reported medical errors in children. Pediatrics.2003;111 :617– 621
    OpenUrlAbstract/FREE Full Text
  16. Miller MR, Zhan C. Pediatric patient safety in hospitals: a national picture in 2000. Pediatrics.2004;113 :1741– 1746
    OpenUrlAbstract/FREE Full Text
  17. ↵
    Woods D, Thomas E, Holl J, Altman S, Brennan T. Adverse events and preventable adverse events in children. Pediatrics.2005;115 :155– 160
    OpenUrlAbstract/FREE Full Text
  18. ↵
    Moore TJ, Weiss SR, Kaplan S, Blaisdell CJ. Reported adverse drug events in infants and children under 2 years of age. Pediatrics.2002;110 (5). Available at: www.pediatrics.org/cgi/content/full/110/5/e53
  19. ↵
    Suresh G, Horbar JD, Plsek P, et al. Voluntary anonymous reporting of medical errors for neonatal intensive care. Pediatrics.2004;113 :1609– 1618
    OpenUrlAbstract/FREE Full Text
  20. ↵
    Cullen DJ, Bates DW, Small SD, Cooper JB, Nemeskal AR, Leape LL. The incident reporting system does not detect adverse drug events: a problem for quality improvement. Jt Comm J Qual Improv.1995;21 :541– 548
    OpenUrlPubMed
  21. ↵
    Thomas EJ, Lipsitz SR, Studdert DM, Brennan TA. The reliability of medical record review for estimating adverse event rates. Ann Intern Med.2002;136 :812– 816
    OpenUrlCrossRefPubMed
  22. ↵
    Institute for Healthcare Improvement. Trigger tool for measuring adverse drug events. Available at www.ihi.org. Accessed May 10, 2006
  23. ↵
    Raschke RA, Gollihare B, Wunderlich TA, et al. A computer alert system to prevent injury from adverse drug events. JAMA.1998;280 :1317– 1320
    OpenUrlCrossRefPubMed
  24. ↵
    Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc.1998;5 :305– 314
    OpenUrlAbstract/FREE Full Text
  • Copyright © 2006 by the American Academy of Pediatrics
PreviousNext
Back to top

Advertising Disclaimer »

In this issue

Pediatrics
Vol. 118, Issue 4
October 2006
  • Table of Contents
  • Index by author
View this article with LENS
PreviousNext
Email Article

Thank you for your interest in spreading the word on American Academy of Pediatrics.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Adverse Events in the Neonatal Intensive Care Unit: Development, Testing, and Findings of an NICU-Focused Trigger Tool to Identify Harm in North American NICUs
(Your Name) has sent you a message from American Academy of Pediatrics
(Your Name) thought you would like to see the American Academy of Pediatrics web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Request Permissions
Article Alerts
Log in
You will be redirected to aap.org to login or to create your account.
Or Sign In to Email Alerts with your Email Address
Citation Tools
Adverse Events in the Neonatal Intensive Care Unit: Development, Testing, and Findings of an NICU-Focused Trigger Tool to Identify Harm in North American NICUs
Paul J. Sharek, Jeffrey D. Horbar, Wilbert Mason, Hema Bisarya, Cary W. Thurm, Gautham Suresh, James E. Gray, William H. Edwards, Donald Goldmann, David Classen
Pediatrics Oct 2006, 118 (4) 1332-1340; DOI: 10.1542/peds.2006-0565

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
Adverse Events in the Neonatal Intensive Care Unit: Development, Testing, and Findings of an NICU-Focused Trigger Tool to Identify Harm in North American NICUs
Paul J. Sharek, Jeffrey D. Horbar, Wilbert Mason, Hema Bisarya, Cary W. Thurm, Gautham Suresh, James E. Gray, William H. Edwards, Donald Goldmann, David Classen
Pediatrics Oct 2006, 118 (4) 1332-1340; DOI: 10.1542/peds.2006-0565
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
Print
Download PDF
Insight Alerts
  • Table of Contents

Jump to section

  • Article
    • Abstract
    • METHODS
    • RESULTS
    • DISCUSSION
    • CONCLUSIONS
    • Acknowledgments
    • Footnotes
    • REFERENCES
  • Figures & Data
  • Info & Metrics
  • Comments

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Cited By...

  • Decreasing Unplanned Extubation in the Neonatal ICU With a Focus on Endotracheal Tube Tip Position
  • Safety of Prescribing Off-Label Drugs for Noncritical Ill Children: A Cross-Sectional Study
  • Phased Quality Improvement Interventions in Reducing Unplanned Extubation in the Neonatal ICU
  • Outcomes, Resource Use, and Financial Costs of Unplanned Extubations in Preterm Infants
  • The Adverse Impact of Unplanned Extubation in a Cohort of Critically Ill Neonates
  • Adverse Events in Hospitalized Pediatric Patients
  • Development of a trigger tool to identify adverse events and no-harm incidents that affect patients admitted to home healthcare
  • Reducing Unplanned Extubations in the NICU Following Implementation of a Standardized Approach
  • Quality Improvement and Safety in the Neonatal Intensive Care Unit
  • Interventions to Improve Patient Safety During Intubation in the Neonatal Intensive Care Unit
  • Performance of the Global Assessment of Pediatric Patient Safety (GAPPS) Tool
  • Fractures Among Inpatients in a Pediatric Hospital
  • ReCAP: Detection of Potentially Avoidable Harm in Oncology From Patient Medical Records
  • Electronic health record-based triggers to detect adverse events after outpatient orthopaedic surgery
  • The Impact of Rudeness on Medical Team Performance: A Randomized Trial
  • A Trigger Tool to Detect Harm in Pediatric Inpatient Settings
  • CareTrack Kids--part 3. Adverse events in children's healthcare in Australia: study protocol for a retrospective medical record review
  • Application of a trigger tool in near real time to inform quality improvement activities: a prospective study in a general medicine ward
  • Parents' perspectives on safety in neonatal intensive care: a mixed-methods study
  • Exposure to Leadership WalkRounds in neonatal intensive care units is associated with a better patient safety culture and less caregiver burnout
  • Prevalence and severity of patient harm in a sample of UK-hospitalised children detected by the Paediatric Trigger Tool
  • Reducing Unplanned Extubations in the NICU
  • Strengths and weaknesses of working with the Global Trigger Tool method for retrospective record review: focus group interviews with team members
  • Unplanned Extubation in the Neonatal ICU: A Systematic Review, Critical Appraisal, and Evidence-Based Recommendations
  • Assessment of the global trigger tool to measure, monitor and evaluate patient safety in cancer patients: reliability concerns are raised
  • Experiences with global trigger tool reviews in five Danish hospitals: an implementation study
  • Adverse events among children in Canadian hospitals: the Canadian Paediatric Adverse Events Study
  • Long-term Reduction in Adverse Drug Events: An Evidence-Based Improvement Model
  • Assessment of adverse events in medical care: lack of consistency between experienced teams using the global trigger tool
  • Quality Improvement for Neonatal Nurses, Part I: A Framework for Advancing the Quality and Safety of Care
  • Patient safety factors in children dying in a paediatric intensive care unit (PICU): a case notes review study
  • Topics in Neonatal Informatics: Computerized Physician Order Entry
  • Principles of Pediatric Patient Safety: Reducing Harm Due to Medical Care
  • Description of the development and validation of the Canadian Paediatric Trigger Tool
  • 'Global Trigger Tool' Shows That Adverse Events In Hospitals May Be Ten Times Greater Than Previously Measured
  • Recognizing Hypoglycemia in Children Through Automated Adverse-Event Detection
  • Systematic review of medication safety assessment methods
  • Large-scale deployment of the Global Trigger Tool across a large hospital system: refinements for the characterisation of adverse events to support patient safety learning opportunities
  • Adverse events and comparison of systematic and voluntary reporting from a paediatric intensive care unit
  • Iatrogenic Events in Neonates: Beneficial Effects of Prevention Strategies and Continuous Monitoring
  • Identifying causes of adverse events detected by an automated trigger tool through in-depth analysis
  • Tailoring adverse drug event surveillance to the paediatric inpatient
  • Compatibility of drug infusions in the NICU
  • Critical incident reporting in neonatal practice
  • Network Analysis of Team Structure in the Neonatal Intensive Care Unit
  • Specialty-based, voluntary incident reporting in neonatal intensive care: description of 4846 incident reports
  • The preliminary development and testing of a global trigger tool to detect error and patient harm in primary-care records
  • An Intervention to Decrease Narcotic-Related Adverse Drug Events in Children's Hospitals
  • Iatrogenesis in Neonatal Intensive Care Units: Observational and Interventional, Prospective, Multicenter Study
  • Detection of adverse events in surgical patients using the Trigger Tool approach
  • An epistemology of patient safety research: a framework for study design and interpretation. Part 3. End points and measurement
  • Reevaluating the Safety Profile of Pediatrics: A Comparison of Computerized Adverse Drug Event Surveillance and Voluntary Reporting in the Pediatric Environment
  • Development, Testing, and Findings of a Pediatric-Focused Trigger Tool to Identify Medication-Related Harm in US Children's Hospitals
  • Adverse Medical Events in the NICU: Epidemiology and Prevention
  • Google Scholar

More in this TOC Section

  • Clinical Impact of a Diagnostic Gastrointestinal Panel in Children
  • Intrapartum Group B Streptococcal Prophylaxis and Childhood Allergic Disorders
  • Changes in Neurodevelopmental Outcomes From Age 2 to 10 Years for Children Born Extremely Preterm
Show more Articles

Similar Articles

Subjects

  • Fetus/Newborn Infant
    • Fetus/Newborn Infant
    • Neonatology
  • Journal Info
  • Editorial Board
  • Editorial Policies
  • Overview
  • Licensing Information
  • Authors/Reviewers
  • Author Guidelines
  • Submit My Manuscript
  • Open Access
  • Reviewer Guidelines
  • Librarians
  • Institutional Subscriptions
  • Usage Stats
  • Support
  • Contact Us
  • Subscribe
  • Resources
  • Media Kit
  • About
  • International Access
  • Terms of Use
  • Privacy Statement
  • FAQ
  • AAP.org
  • shopAAP
  • Follow American Academy of Pediatrics on Instagram
  • Visit American Academy of Pediatrics on Facebook
  • Follow American Academy of Pediatrics on Twitter
  • Follow American Academy of Pediatrics on Youtube
  • RSS
American Academy of Pediatrics

© 2021 American Academy of Pediatrics