A Trigger Tool to Detect Harm in Pediatric Inpatient Settings
OBJECTIVES: An efficient and reliable process for measuring harm due to medical care is needed to advance pediatric patient safety. Several pediatric studies have assessed the use of trigger tools in varying inpatient environments. Using the Institute for Healthcare Improvement’s adult-focused Global Trigger Tool as a model, we developed and pilot tested a trigger tool that would identify the most common causes of harm in pediatric inpatient environments.
METHODS: After formal training, 6 academic children’s hospitals used this novel pediatric trigger tool to review 100 randomly selected inpatient records per site from patients discharged during the month of February 2012.
RESULTS: From the 600 patient charts evaluated, 240 harmful events (“harms”) were identified, resulting in a rate of 40 harms per 100 patients admitted and 54.9 harms per 1000 patient days across the 6 hospitals. At least 1 harm was identified in 146 patients (24.3% of patients). Of the 240 total events, 108 (45.0%) were assessed to have been potentially or definitely preventable. The most common patient harms were intravenous catheter infiltrations/burns, respiratory distress, constipation, pain, and surgical complications.
CONCLUSIONS: Consistent with earlier rates of all-cause harm in adult hospitals, harm occurs at high rates in hospitalized children. Availability and use of an all-cause harm identification tool will establish the epidemiology of harm and will provide a consistent approach to assessing the effect of interventions on harms in hospitalized children.
What’s Known on This Subject:
Harm occurs at a high rate in adult inpatient populations. One single-center study, applying an adult-based surveillance tool, suggests that a pediatric inpatient population also has a high rate of harm.
What This Study Adds:
Harm occurred frequently in 6 freestanding children’s hospitals. Identification and understanding of the harm is the first step to making necessary improvements and to preventing future harm.
Despite the increased attention paid to patient safety over the past 15 years since the publication of “To Err Is Human,” harm rates for hospitalized patients remain high.1,2 Advanced harm detection techniques suggest that national rates of harm resulting in death among hospitalized patients are >4 times higher than originally reported. These rates suggest that almost half a million people die in the United States per year as a result of hospital-associated harm.3 This estimate would place patient harm as the third leading overall cause of death behind heart disease and cancer.4
Consistent and accurate detection of patient harm remains a challenge for most organizations. Although voluntary reports have been shown to capture only 2% to 8% of all harms, they remain the mainstay of harm detection in most hospitals.5 The Institute of Healthcare Improvement (IHI) noted recently that “public health researchers have established that only 10 to 20 percent of errors are ever reported and, of those, 90 to 95 percent cause no harm to patients.”6 Noting this gap, efforts to improve harm detection have accelerated, with the most visible perhaps being the development, testing, and dissemination of the adult-focused IHI Global Trigger Tool (GTT). A “trigger” is a medical record-based “hint” (such as the use of the antidote naloxone) that “triggers” the search of the medical record to determine whether an adverse event (such as a clinical overdose of an opiate, as opposed to a therapeutic use in response to a nonprescribed opiate use) might have occurred. The GTT, which includes 55 such triggers, has been used in such settings as the North Carolina Patient Safety Study, the Office of the Inspector General of the Centers for Medicare and Medicaid Services report on inpatient harm, and the Office of the Inspector General work in skilled nursing facilities.2,6–11 These efforts have reinforced the perspective that the trigger tool approach is at present the most reliable and consistent harm detection method.3,7,10
Although the IHI GTT was designed for the adult population, 1 children’s hospital tested it on its pediatric inpatient records and identified patient harm in more than one-third of admissions.12 In addition, there have been ongoing efforts to develop trigger tools tailored for pediatric settings. These have included trigger tools for discrete settings (eg, the NICU, PICU, and general pediatric non–intensive care inpatient environment) and specific types of harms (eg, adverse drug events). Such applications of focused trigger tools have identified dramatically higher rates of harm compared with voluntary reports of safety events.13–17 However, a single, pediatric-specific detection tool that can be applied across all settings, analogous to the IHI’s GTT, has not been available. Until a comprehensive standard harm measure is established in pediatrics, improvement efforts will continue to focus on a fraction of the harm that occurs to patients, leaving countless harms within the system unrecognized and unaddressed.18
Recently, a comprehensive pediatric trigger list for the inpatient environment using a modified Delphi technique was developed. The tool was modeled after the IHI GTT and named the Pediatric All-Cause Harm Measurement Tool (PACHMT).19 Of note, the PACHMT was developed anticipating future automation to allow integration into electronic health records. Using the PACHMT, we sought to pilot test the tool to estimate harm rates across 6 children’s hospitals and to ensure the feasibility and appropriateness of the tool as a way to estimate the rates of all-cause harm in hospitalized children. Although no tool can consistently identify all causes of harm, the PACHMT was designed to identify many of the most common causes of pediatric harm able to be recognized through trigger methodology.
Design, Setting, and Patients
As has been done in previous trigger tool studies, we used a cross-sectional study design, using retrospective chart review in 6 academic freestanding children’s hospitals with previous trigger tool experience from different regions of the country. Patient records were eligible for inpatients who were <22 years of age, had a length of stay between 24 hours and 6 months, and were discharged in February 2012.20 Any patients who were admitted for rehabilitation, to the normal newborn nursery, to day treatment areas, or with a primary discharge diagnosis related to psychiatric or obstetric care were excluded as is consistent with earlier study designs.12–17 A random sample of 100 patients who met the inclusion criteria was selected from each site for review. Institutional review board waiver or approval for this study was granted at each site.
The list of pediatric-specific triggers from the PACHMT was created via a modified Delphi process described previously.19 The PACHMT trigger list (Table 1), an instruction manual containing detailed definitions of triggers, a list of likely associated predefined harm events and case examples, and data collection forms were distributed to each hospital. Each site identified at least 1 nonphysician clinical reviewer, either a nurse or pharmacist, as well as 1 physician reviewer. All clinical reviewers and physician reviewers from each site completed standard training on trigger tool chart review methods incorporating the PACHMT and based on previous work by the Children’s Hospital Association and IHI.6,13,14,17 Training was taught by IHI GTT expert educators and consisted of 3 interactive educational webinars. During these sessions, these experts described trigger detection, introduced the PACHMT trigger list, presented the process for the standard application of the tool, reviewed examples, and facilitated active dialogue. The site reviewers independently completed standard case studies for detecting triggers, harm identification, and harm classification that were reviewed during training sessions. Calibration exercises, such as extensive case reviews and group review of event and harm classifications, during training assisted in promoting consistency in the use of the tool and study definitions.
The reviewer-physician team at each site then applied the standard trigger tool review method to review their 100 charts. There were no limits placed on the review time per chart. Once a trigger was identified, the reviewer would determine if there was harm. Harm was defined as an “unintended physical injury (resulting from or contributed to) by medical care that required additional monitoring, treatment, or hospitalization, or that resulted in death.”7 This definition is consistent with the Centers for Medicare and Medicaid Services, Office of the Inspector General, and the Agency for Healthcare Research and Quality’s definition of harm.7,11,21 Once harm was identified, its level of severity was assigned on the basis of the National Coordination Council for Medication Error Reporting and Prevention classification approach (Table 2).22
After harm identification and classification, reviewers were asked to assign a level of preventability or nonpreventability. Each harm was assigned as “definitely preventable,” “probably preventable,” “probably not preventable,” or “definitely not preventable.” Preventable events were defined as “events where definite breach of standard professional behavior or technique was identified; necessary precautions were not taken; event was preventable by modification of behavior, technique or care.”2 All other harms were considered not preventable.
After the reviewer’s assessment of a chart and PACHMT trigger application, physician reviewers reviewed a summary of the harm and either confirmed or modified the primary reviewer’s assessment. The physician reviewer’s assessment and scoring of the case was considered final.
The primary outcomes of this study were harms per patient and harms per 1000 patient days. Secondary outcomes of interest were (1) triggers per patient,(2) trigger-positive predictive values (defined as the number of times a specific trigger independently identified harm divided by the number of times a trigger was identified) individually and for PACHMT in total,(3) harm severity,(4) percentage of harms that were preventable, percentage of harms that were also identified in local occurrence reports, and median time for chart review.
Patient characteristics were described with means and 95% confidence intervals (CIs), as well as medians with interquartile ranges (IQRs). We summarized harms with frequencies and percentages, and computed rates with 95% Poisson CIs. Finally, for each trigger, we report positive predictive values with exact binomial 95% confidence intervals (CI). All analyses were performed with SAS v.9.3 (SAS, Cary, NC), and P < .05 were considered statistically significant.
The median age of patients whose chart was in the study was 4 years (IQR: 0.5–12.0), and 287 (47.8%) were female. The median length of stay was 4 days (IQR: 3–7), and 2.7% of patients had a length of stay >30 days (Table 3).
There were 1093 triggers detected resulting in identification of 204 (85.0%) of the total harms identified. The remaining 36 harms were identified during the chart review process but were not linked to any specific trigger. The positive predictive value of the aggregate PACHMT trigger list was 22.0% (95% CI: 19.0–25.1).
A total of 240 harms were identified from the 600 total patient records reviewed resulting in a rate of 40 harms per 100 patients admitted (95% CI: 35.2–45.4) and 54.9 harms per 1000 patient days (95% CI: 48.3–62.3). At least 1 harm was identified in 146 patients; thus, 24.3% of all patients experienced ≥1 harms. Fifty-one patients (8.5% of the total) had multiple harms. One hundred and eight harms (45.0%) were documented as either probably preventable or definitely preventable.
The mean time for chart review was 42 minutes with a median of 30 minutes (IQR: 15.5–60.0). Of the 240 harms identified, 22 (9.2%) of these were also identified within the hospital’s voluntary reporting system.
The most common harms were intravenous catheter infiltrations/burns, respiratory distress, constipation, pain, surgical complications and skin rash, bruising, or burn. A complete listing of the harms experienced is included in Table 4. Sixty-eight percent of the harms documented were rated as level E on the National Coordination Council for Medication Error Reporting and Prevention harm scale. Table 5 categorizes the identified harms by severity level and preventability as determined by the reviewers.
Thirty-six (70.6%) of the PACHMT triggers were identified at least once during the chart review. Table 6 lists the triggers in order of frequency and lists each trigger’s positive predictive value.
Using a novel pediatric-specific list of triggers, we found 40 harms per 100 admissions among children hospitalized at 6 large children’s hospitals. Consistent with earlier trigger-based harm detection studies, nearly one half of the harm was deemed preventable.2,13,14 One of every 4 pediatric admissions in our study had at least 1 identified harm. Our detected harm rate is similar to a recent single-site study, which detected a pediatric harm rate of 37 harm events for every 100 admissions using the adult-focused IHI GTT.12 This is the first multisite study describing all-cause inpatient pediatric harm using a novel pediatric-specific global trigger tool.
Consistent with previous trigger tool efforts, harms were identified substantially more frequently with the PACHMT than with voluntary reporting. Nevertheless, we consider these 2 approaches to patient safety measurement to be complementary. Although a trigger tool’s systematic measurement captures substantially more harm events, voluntary reporting often provides information related to near miss events that are not detected by the trigger tool methodology. We believe a combination of both approaches results in an enhanced understanding of a system’s potential fallibilities.23
There are several limitations to this study. First, as a pilot, our study had a relatively small sample size and only 1 physician reviewer (rather than the more typical design of 2 physician reviewers) to assess and rate the harms. Second, like adult trigger tool studies,6,19 our study lacked definitive evidence for which triggers were the best to include, and it does not capture all harms. In an effort to construct a parsimonious list of highly predictive triggers, several potential triggers such as hypocalcemia or hypokalemia were not included in the final trigger tool. Third, we did not undertake inter- or intrarater reliability testing, so we do not yet know the reliability and consistency of this tool between alternative users. Finally, although this was a relatively large, multicenter pilot study, larger studies are needed to better identify rare but clinically important harms, to define the operating characteristics of less common triggers, and to examine how trigger tools work in a more diverse set of hospitals. The federal Pediatric Quality Measures Program is developing such a measure at present, drawing on the work of PACHMT and other trigger tools.
The application of a novel pediatric global trigger tool identified 40 harms per 100 admissions in 6 freestanding children’s hospitals. Despite more than a decade of intense focus, harm continues to occur in large numbers in hospitalized children. The use of the PACHMT trigger tool will provide the foundation to capture harms in a rigorous and systematic way. Use of such trigger tools will lead to a better understanding of the epidemiology of harm in hospitalized children as well as allow tracking of change with patient-safety-focused interventions. Because this work was only intended to be a pilot study, future research should focus on editing the PACHMT to establish the next generation pediatric global trigger tool, conduct a formal study to establish harm rates and epidemiology, and determine the operating characteristics of this tool, with an eye toward integration into the electronic medical record and eventual automation.
The following participants and associated hospitals performed chart review and data collection, and we greatly appreciate their efforts: Boston Children’s Hospital: Annemarie Fayemi, RN, MS, Shannon Cottreau, RN, BSN, CPN, Christopher Landrigan, MD, MPH; Children’s Hospital Central California: Dana Sankrit, BS, RN, Richard Sakai, PharmD, Kathleen Murphy, MD; Children’s Hospital Colorado: Megan Dix, RN, Viktor Lasmarias, RN, Eric Tham, MD; Children’s National Medical Center: Valere Lemon MBA, BSN, RN, David C. Stockwell, MD, MBA; Cincinnati Children’s Hospital Medical Center: Carol Frese, BSN, RN, Deborah Hacker, ASN, RN, Eric Kirkendall, MD, MBI; Lucile Packard Children’s Hospital: Krisa Hoyle Elgin, MPH, FNP-BC, Paul Sharek, MD, MPH. We also thank John Mallet, Child Health Corporation of America, for data collection form and database design and for database query and management.
- Accepted February 27, 2015.
- Address correspondence to David C. Stockwell, MD, MBA, Children’s National Medical Center, 111 Michigan Ave, NW, Suite M-4800, Washington, DC 20010. E-mail:
Dr Stockwell co-conceptualized and designed the study and drafted the initial manuscript; Ms Bisarya designed the data collection instruments, coordinated and supervised data collection at all sites, and critically reviewed the manuscript; Dr Classen contributed to the original study design, conducted training for all sites assisted with the initial analysis, and reviewed and revised the manuscript; Drs Kirkendall and Landrigan, Ms Lemon, Drs Tham, Hyman, and Lehman, Ms Searles, and Dr Muething contributed to the initial study design, carried out the local data collection, assisted with the initial analysis, and reviewed and revised the manuscript; Dr Hall contributed to the study design, performed the data analysis, and critically reviewed the manuscript; Dr Schuster carried out local data collection and provided resources, assisted with interpretation of data, and reviewed and revised the manuscript; Dr Sharek co-conceptualized and designed the study and drafted and edited the initial manuscript; and all authors approved the final manuscript as submitted.
FINANCIAL DISCLOSURE: Dr Stockwell reports partial employment by Pascal Metrics, a federally certified Patient Safety Organization. Dr Classen reports employment by Pascal Metrics, a federally certified Patient Safety Organization. Dr Landrigan reports having served as a paid consultant to Virgin Pulse, to help develop a Sleep and Health Program. He is supported in part by the Children’s Hospital Association, for his work as an Executive Council member of the Pediatric Research in Inpatient Settings (PRIS) network. In addition, Dr Landrigan has consulted and received monetary awards, honoraria, and travel reimbursement from multiple academic and professional organizations for delivering lectures on sleep deprivation, physician performance, handoffs, and safety, and has served as a paid expert witness in cases regarding patient safety and sleep deprivation. Ms Lemon reports accepting a position at Pascal Metrics as client services manager after the study was completed. The other authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This work was partially supported by the Children’s Hospital Association. Drs Landrigan and Schuster and data collection at Boston Children’s Hospital were supported by grant U18 HS020513. The views expressed in this article are those of the authors and do not necessarily represent the views of the US Department of Health and Human Services or the Agency for Healthcare Research and Quality.
POTENTIAL CONFLICT OF INTERESTS: Dr Stockwell reports partial employment by Pascal Metrics, a federally certified Patient Safety Organization. Dr Classen reports employment by Pascal Metrics, a federally certified Patient Safety Organization. Dr Landrigan reports having served as a paid expert witness in cases regarding patient safety and sleep deprivation. The other authors have indicated they have no potential conflicts of interest relevant to this article to disclose.
- Kohn LT,
- Corrigan JM,
- Donaldson MS
- ↵Center for Disease Control and Prevention. FASTSTATs—Leading Causes of Death. Available at: http://www.cdc.gov/nchs/fastats/leading-causes-of-death.htm. Accessed January 5, 2015
- Stockwell DC,
- Slonim AD
- ↵Griffin FA, Resar RK. IHI Global Trigger Tool for Measuring Harms (IHI Innovation Series white paper). 2nd ed. Cambridge, MA: Institute for Healthcare Improvement; 2009. Available at http://www.IHI.org
- ↵Office of the Inspector General. Adverse Events in Hospitals: Methods for Identifying Events (OEI-06-08-00221). Washington, DC: Department of Health and Human Services; 2010. Available at: http://www.oig.hhs.gov/oei/reports/oei-06-08-00221.pdf. Accessed January 5, 2015
- Rozich JD,
- Haraden CR,
- Resar RK
- Resar RK,
- Rozich JD,
- Classen D
- Classen DC,
- Resar R,
- Griffin F,
- et al
- ↵Office of the Inspector General. Adverse Events in Skilled Nursing Facilities: National Incidence Among Medicare Beneficiaries (OEI-06-11-00370). Washington, DC: Department of Health and Human Services; 2014. Available at: http://oig.hhs.gov/oei/reports/oei-06-11-00370.pdf. Accessed January 5, 2015
- Kirkendall ES,
- Kloppenborg E,
- Papp J,
- et al
- Sharek PJ,
- Horbar JD,
- Mason W,
- et al
- ↵Stockwell DC, Bisarya H, Classen DC, et al. Development of an electronic pediatric all-cause harm measurement tool using a modified Delphi method [published online ahead of print August 26, 2014]. J Patient Safety. Available at: http://journals.lww.com/journalpatientsafety/Abstract/publishahead/Development_of_an_Electronic_Pediatric_All_Cause.99719.aspx. Accessed January 5, 2015
- American Academy of Pediatrics, Council on Child Health
- ↵Hunt DR, Verzier N, Abend SL, et al. Fundamentals of medicare patient safety surveillance: intent, relevance, and transparency. In: Henriksen K, Battles JB, Marks ES, et al., eds. Advances in Patient Safety: From Research to Implementation. Volume 2: Concepts and Methodology. Rockville, MD: Agency for Healthcare Research and Quality; 2005. Available at: http://www.ncbi.nlm.nih.gov/books/NBK20489. Accessed March 10, 2015
- ↵National Coordinating Council for Medication Error Reporting and Prevention. Types of Medication Errors. Available at: harm.nccmerp.org/medErrorCatIndex.html Accessed January 5, 2015
- Copyright © 2015 by the American Academy of Pediatrics