BACKGROUND AND OBJECTIVES: Awareness of the impact of preventable harm on patients and families has resulted in extensive efforts to make our health care systems safer. We determined that, in our hospital, patients experienced 1 of 9 types of preventable harm approximately every other day. In an effort to expedite early identification of patients at risk and provide timely intervention, we used the electronic health record’s (EHR) documentation to enable decision support, data capture, and auditing and implemented reporting tools to reduce rates of harm.
METHODS: Harm reduction strategies included aggregating data to generate a risk profile for hospital-acquired conditions (HACs) for all inpatients. The profile includes links to prevention bundles and available care guidelines. Additionally, lists of patients at risk for HACs autopopulate electronic audit tools contained within Research Electronic Data Capture, and data from observational audits and EHR documentation populate real-time dashboards of bundle compliance. Patient population summary reports promote the discussion of relevant HAC prevention measures during patient care and unit leadership rounds.
RESULTS: The hospital has sustained a >30% reduction in harm for 9 types of HAC since 2012. In 2014, the number of HACs with >80% bundle adherence doubled coincident with the progressive rollout of these EHR-based interventions.
CONCLUSIONS: Existing EHR documentation and reporting tools may be effective adjuncts to harm reduction initiatives. Additional study should include an evaluation of scalability across organizations, ongoing bundle adherence, and individual tests of change to isolate interventions with the highest impact on our results.
- API —
- application programming interface
- CAUTI —
- catheter-associated urinary tract infection
- CHCO —
- Children's Hospital Colorado
- CHSPS —
- Children’s Hospitals’ Solutions for Patient Safety
- CLABSI —
- central line–associated blood stream infection
- EDW —
- enterprise data warehouse
- EHR —
- electronic health record
- HAC —
- hospital-acquired condition
- REDCap —
- Research Electronic Data Capture
- SPC —
- statistical process control
Awareness of the impact of preventable harm on patients and families has resulted in extensive efforts to make our health care systems safer. Although several follow-up studies1–7 to the Institute of Medicine’s 1999 report, To Err Is Human,8 have not revealed dramatic reductions nationally in preventable morbidity or mortality, many reports have demonstrated the positive impacts local improvement initiatives can have on organizational patient safety outcomes.9–13 In 2012, Children's Hospital Colorado (CHCO) recognized that a patient was experiencing 1 of 9 types of often preventable harm approximately every other day. CHCO joined Children’s Hospitals’ Solutions for Patient Safety (CHSPS), a national network of children’s hospitals initially funded through the Center for Medicare and Medicaid Innovation14 to work collaboratively on reducing the incidence of hospital-acquired conditions (HACs) and readmissions.15
CHCO’s approach to harm reduction included improving adherence to evidence-based prevention measures (bundles), conducting organizational training in safety practices for staff and leaders, and strengthening our cause analysis program. Our aim was to reduce pressure injuries, falls, 3 types of hospital-acquired infections, venous thromboembolism, adverse drug events, codes outside the ICU, and patient identification care errors. Our multidisciplinary improvement teams included physicians, nurses, process improvement specialists, pharmacists, respiratory therapists, and parent partners. At CHCO, one of the unique aspects of our program was the innovative use of electronic health record (EHR) documentation for the purposes of risk identification, auditing tool development, clinical decision support, and dynamic reporting efforts. These EHR-based approaches were components of an effort to promote adherence to prevention bundles for patients at risk for specific HACs and were implemented along with other harm reduction strategies as elements of a comprehensive, organization-wide program starting in late 2012, with ongoing improvements over time that continue to this day.
CHCO is a 475+ bed freestanding, tertiary care teaching hospital with 16 satellite locations throughout Colorado and with a regional presence in surrounding states. CHCO has ∼700 000 outpatient visits, including 150 000 emergency department/urgent care visits annually, and admits 18 500 patients, resulting in over 110 000 inpatient days in our facilities. Our EHR (Epic, Epic Systems, Madison, WI) has been used for both clinical documentation and provider order entry since 2007 and is deployed throughout our network.
This improvement effort was implemented as part of the hospital’s participation in CHSPS. Systematic evaluation was solely for the purposes of the project’s quality improvement aims. The project was reviewed and approved as a quality improvement initiative by our Organizational Research Risk and Quality Improvement Review Panel. Data were managed in accordance with Health Insurance Portability and Accountability Act requirements.
Our team’s specific aim was to reduce 9 types of hospital-acquired harm for all inpatients and outpatients cared for at all CHCO locations by 40% over a 2-year period, consistent with the overall goals of the CHSPS network. A comprehensive program (called “Target Zero” at CHCO) that was implemented in early 2013 included training for all providers and staff in error prevention practices, leadership methods training for people in supervisory roles, and an enhanced cause analysis program.15 Multidisciplinary improvement teams of nurses, physicians, process improvement specialists, and other clinical support staff worked with clinicians in inpatient and outpatient settings to effect change and pursuit of the goal. Our goal was to reduce the incidence of 7 HACs (aligned with the CHSPS network’s goals) and 2 additional types of harm at CHCO as a part of our board of directors-approved quality and patient safety plan. These additional harm types included providing care to an unintended patient and preventing codes outside the ICU. For simplicity and readability, we will refer to this set of 9 preventable harm events as HACs, although patient identification errors and codes outside the ICU are not technically considered HACs. The EHR-based strategies we adopted and describe in this article were elements of this overall patient safety program in our hospital.
Numerous articles have demonstrated the linkage between the consistent implementation of evidence-based prevention (process) measures and a reduction in the incidence of HACs.16–19 In an effort to promote reliable bundle adherence at CHCO, we tested several innovative strategies by using EHR documentation.
First, we developed a patient-specific “Target Zero risk profile” (Fig 1) that identifies HACs for which each patient is at increased risk. The improvement team hypothesized that, based on practical analysis of clinical workflows, patient care teams would benefit from an automated tool within the EHR that identifies each patient’s HAC risks. A defined set of “triggers” placing patients at risk for developing specific HACs are routinely documented in structured fields in each patient’s medical record. An EHR-based display was built to show a real-time patient profile summarizing each patient’s risk when any of these triggers were documented in their record. For example, a patient documented as having an indwelling urinary catheter is automatically at risk for a catheter-associated urinary tract infection (CAUTI). Other examples include pressure ulcers (based on documented Braden Q score), falls (falls risk assessment), central line–associated blood stream infections (CLABSI, central line in place), and venous thromboembolism (internally developed risk assessment). Links to prevention bundles and available care guidelines are provided for relevant HACs. The auto-populated Target Zero risk profile can be used to improve each individual patient’s care by highlighting HAC prevention requirements, and to enhance the improvement teams’ observation and reporting efforts.
The second intervention, a comprehensive, easily filtered report (Fig 2) was developed to summarize current Target Zero risk profiles for all inpatients for use by clinicians, unit leaders, and improvement teams. The Target Zero risk profile report can sort groups of patients and their identified risks by various subgroupings, for example, by unit, care team, attending physician, or HAC risk. This summary list of patients with their various HAC risks can be used during medical team or unit based leadership rounds to enable discussions directed at ensuring that appropriate prevention measures are in place. Although not systematically measured, HAC improvement teams anecdotally reported their observations of how clinical teams were using these tools to promote bundle adherence for their at-risk patients.
The third intervention was spurred by the implementation and adoption of the previous 2. Pediatric residents and various care teams (eg, ICU) instituted a “pause to care” moment after rounding on each patient in which they confirmed that priority orders had been placed for things like influenza vaccine, and that HACs on the Target Zero risk profile were being addressed. This aligns with the organization’s broader use of a safety practice called “pause to care” that highlights the importance of stopping to check or review one’s own work for accuracy, especially when rushed or distracted. Figure 3 depicts the screen view in Epic to support this practice. Summary reports are also used by improvement team members to identify patients who are eligible for process measure auditing for particular HACs, described in greater detail below.
Finally, in collaboration with the CHSPS Network, CHCO adopted and implemented “bundles” of evidence-enabled preventive care guidelines for the set of defined HACs. HAC work groups were formed, each led by a clinical subject matter expert in partnership with a quality and process improvement specialist, front line staff, faculty, and parent partners. Each HAC team identified and approved the appropriate bundle elements and organized them into a standard bundle format (Fig 4) as a way for staff to leverage a consistent and reliable tool for evidence-based practice. Our organization adapted these bundle elements as practice recommendations and aligned organizational policies and procedures to them.
To ensure consistency in our process measurement and to permit valid conclusions about the impact of the bundle elements (process measures) on the HAC events (outcomes), strict data collection guidelines and processes were established. Initially, 3 basic types of data were identified as available through the bundle element auditing process: observation, documentation, and self-report. Observation data were the gold standard and deemed the most reliable. Documentation (EHR) data were used when observation data were not available, or to supplement directly observed practice; self-report data were used only in the absence of more reliable information.
In addition to the EHR-based efforts to improve process compliance, other interventions focused on team communication strategies, personal error prevention practices, patient safety culture, and leadership engagement in safety improvement efforts, all of which were strongly supported by the hospital’s senior management team and board of directors. Although we were unable to stratify the differential impact of these various interventions on the changes in the rates of adverse outcomes, we believe these electronic tools support the bundle compliance that may result in decreased patient harm rates. These tools were incorporated into individual patient care planning (risk profiles, pause to care tool), improvement team meetings and project planning sessions, unit leadership rounds, and governance committee discussions.
Compliance with process measures was determined by capturing adherence to evidence-based bundles for each identified HAC. Process measures are a coefficient of compliance expressed as a percentage, where the numerator is the number of compliant elements of the bundle and the denominator is the total opportunities for compliance. These improvement bundles are an “all or none” measure in which all intervention elements must be performed to be considered in compliance. We aimed to significantly improve bundle adherence for all HACs with the expectation of then seeing an associated reduction in our overall rates of harm. Outcome events are measured via standard processes either dictated by national guidelines (CAUTI, CLABSI)20 or standard definitions based on evidence or subject matter experts' recommendations. Outcome measures included both the raw number and rates of occurrence for the set of HACs described above. The identification of these occurrences was based on a combination of surveillance (eg, cultures, event tracking for codes), incident reporting, chart documentation, and some electronic triggers (eg, adverse drug events).
Data Collection, Reporting and Interpretation
Initially, auditors collected data via paper audit tools and manually entered the protected health information aggregate results into Excel for analysis and manual reporting. The analyzed audit data and resulting compliance measures were then provided to the organization in static Excel graphs via links on the hospital intranet. This process was time consuming and provided limited insight for unit leaders into their area’s specific performance. Our improvement teams partnered with analysts trained in both Epic-based (Clarity, Workbench) and other reporting tools (Tableau, Tableau Software, Seattle, WA) as well as with enterprise data warehouse (EDW) staff to improve and automate the flow of HAC-related data to create real time, “dynamic” dashboards displaying current data for both outcome and process measures for the defined set of HACs. An electronic audit form was designed by using Research Electronic Data Capture (REDCap).21–23 REDCap is an electronic data repository with versatile functionality that allows the user to build audit tools and safely store and deidentify protected health information. REDCap audit tools are efficiently completed on handheld devices, laptops, or bedside computers and automatically feed data into the EDW to populate the dashboard reports described above. The audit tools include branching logic that vary the questions to be asked (and teaching points to be made) based on the information being input by the auditor in real time.
To minimize manual data entry by the auditors, a real-time, patient-level report was developed within the EHR. HAC-specific reports identified patients at risk for a particular HAC and provided important patient identification information to guide the flow of auditing. Additionally, the reports contained HAC-related clinical documentation that aided in determining compliance with bundle elements. These functionalities dramatically reduced how much data auditors had to enter into the REDCap audit tool. An additional benefit was improved accuracy of the information collected, reducing the chance of data entry errors. These electronic innovations enabled an expansion of auditing via direct observation as opposed to using less accurate documentation reviews.
An application programming interface (API) was built to push relevant patient information from the EHR into REDCap on an hourly basis so it could be readily available for staff anytime they were performing HAC audits. The audit form was prepopulated with relevant patient information, and the additional observation or self-report audit data were entered to complete the audit. Audit data were exported to the EDW every hour by a second API. The patient identifiers included in each HAC audit tool facilitated linkages to other clinical and demographic data already housed in the EDW.
In cases where observation was unnecessary for audit completion (ie, when the completion of the process measure was in chart documentation), EHR reports were built and run automatically on a set schedule. APIs pulled these data into tables within the EDW, eliminating the intermediary step of using REDCap.
Data Visualization in Real-Time, Dashboard Format
The final step in this process was to build dynamic dashboards that summarized the observational audit and/or documentation data stored in the EDW. These HAC-specific dynamic dashboards were available from the hospital intranet, displaying adherence to each bundle element as well as overall (“all or none”) bundle compliance (Fig 5). The dashboard controls allowed users to easily and independently filter the data by clinical unit, date range, or other HAC-specific content to drill into the data of interest. The dashboards refreshed every 30 minutes, enabling timely interventions in response to then-current performance by leaders of clinical units and/or improvement teams. The dynamic dashboards were built by using Tableau version 8.1. Standard Statistical Process Control (SPC) charting rules for determining special cause were used as evidence of improvement when analyzing bundle compliance and HAC rates over time.24 Data review was incorporated into leader rounding and team huddles, improvement team meetings at both individual HAC team and overall project steering team levels, and governance committee reviews. Although we did not objectively measure the extent or ways in which these dashboards were used and therefore cannot prove a relationship between their use and the improvements we have seen, we believe they have been contributory and are pursuing opportunities to use these types of tools to even greater impact as we continue our improvement efforts.
HAC outcome data were stored and reported by using the same technology as bundle adherence (process data). Once the agreed-on definition of a HAC was determined to have been met in any particular case, the event was identified and logged into REDCap. The resulting list of HAC occurrences was pulled into the outcome dashboard (populated by the EDW) with the same filtering abilities as the process dynamic dashboard. For example, the pressure ulcer outcome dashboard was created to permit sorting by unit of occurrence, stage of pressure ulcer, likely contributing cause(s) (eg, device), and body location(s). Users are able to specifically filter and focus on any element or combination of elements of interest in the dashboard by simply clicking within the user interface. HAC outcome rates were all reported as SPC u-charts, created by using QI Charts version 2.0.22 (Scoville Associates, Raleigh-Durham, NC).
Over the past 2 years, and particularly between quarter 4, 2013 and quarter 4, 2014, compliance with process measures dramatically increased, as demonstrated in Fig 6. On average, we saw an 18% improvement in bundle adherence during this time frame and achieved >80% process adherence in 5 of the HACs as compared with 1 in the previous year. In association with these improvements, we saw a 30% reduction in our overall HAC rates that has been sustained over the past 2 years, as demonstrated in the statistical process control chart (Fig 7). Consistent with rules for special cause variation as defined above, we experienced a run of greater than 8 consecutive points below the mean of the initial period ending in December 2012. Updated mean and control limits were plotted for the period starting in January 2013 in the SPC chart after special cause was detected. We experienced a 21% reduction in the raw number of these HACs in 2013 and an additional 11% reduction in total occurrences (after volume adjustment) in 2014.
The described EHR-based interventions were among a number of changes implemented at CHCO over the past several years designed to reduce the rate of preventable harm experienced by our patients. The implementation of our risk profile presented a snapshot of our patients’ care requirements, enabling proactive care planning to improve HAC prevention practices. Care bundle adherence improved by 18% and CHCO achieved a >80% bundle adherence rate for 5 HACs during the measurement period. In addition, there was a 30% reduction in HAC rates overall. Publications by Leventhal, Thriemer, and Pageler25–27 all support this approach of using electronic tools, checklists, and real-time data to enhance improvement initiatives. Multiple technical and human interventions contributed to the successful reduction of HACs at CHCO. Data that demonstrated a correlation of explicit dashboard or documentation usage to the improvements seen would have strengthened the temporal relationship between their implementation and the results, although their relative contribution compared with other safety program components would have remained uncertain. We encourage others who might implement similar interventions to plan for and directly measure tool use, especially if implemented as an isolated system change.
Several limitations should be noted. First, this was conducted as a quality improvement project, not clinical research. As such, many cultural, training, and technological interventions were implemented in parallel, making it impossible to determine their differential impacts on success. Technological interventions, including documentation changes for risk identification, auditing tool development, and clinical decision support (in the form of linked care bundles and clinical care guidelines) likely enhanced other efforts, including improvements in team communication, use of personal error prevention practices, and promotion of a culture of patient safety.
Second, these EHR-based changes were implemented within a single organization. These changes would need to be tested in other organizations to additionally evaluate their effectiveness and applicability.
Third, the strength of the relationship between the specific elements in each adopted bundle and their related outcome is variable across the set of HACs. Evidence may be stronger for CLABSI prevention28 than for the other HACs, but in coming years, CHSPS will likely be able to use data from >110 hospitals to more strongly link bundle adherence with specifically defined elements to reduced rates of harm. Harm reduction results also do not always parallel improvements in process reliability due to the fact that not all complications are 100% preventable, and given <100% reliability, not all patients fully benefit from these risk reduction strategies.
The first 3 years of implementation of the Target Zero program have resulted in significant reductions in preventable harm at CHCO. We believe the combination of process reliability improvements, enabled in part by tools described in this paper, and enhanced patient safety culture were essential elements to achieving these improvements in outcomes. Our next steps are to work on improving and then sustaining reliable bundle adherence, additionally integrating the risk profile into individual patient care planning, and strengthening the role of patients and families in their own contributions to harm reduction. We seek to improve the specificity of clinical alerts and unit-level views of the dashboards to improve provider satisfaction with the user interface, and then anticipate sharing these assessment tools and clinical decision-support functionalities with other organizations. The ability to share EHR based tools, as this organization has done previously,29 will reduce the effort of participating organizations who wish to replicate this work, and will promote consistency in implementing interventions across sites, thereby enabling shared learning. Finally, we aim to use EHR documentation and risk profile information to simplify the completion of tasks by front line staff to reduce the complexity of their work and the documentation of completing bundle elements. We view all of these as being essential elements in our work to reach our goal of zero preventable harm.
We thank our colleagues in the Quality and Patient Safety and Information Technology Departments at CHCO for their contributions to this work, as well as the many physicians, nurses, respiratory therapists, and other staff who participated in this work. We also thank Marie St. Pierre for her assistance with references and Dr Joseph Albietz for his assistance with Epic figures.
- Accepted December 9, 2016.
- Address correspondence to Daniel Hyman MD, MMM, Children’s Hospital Colorado, 13123 E 16th Ave, B400, Aurora, CO 80045. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose
- Vincent C,
- Aylin P,
- Franklin BD, et al
- Mitchell I,
- Schuster A,
- Smith K, et al.
- Kohn LT,
- Corrigan JM,
- Donaldson MS; Institute of Medicine
- Quon BS,
- Goss CH
- Campbell SM,
- Braspenning J,
- Hutchinson A,
- Marshall MN
- Berwick DM,
- Godfrey AB,
- Roessner J
- Newman RE,
- Hedican EB,
- Herigon JC et al.
- Centers for Medicare and Medicaid Services
- Children’s Hospital Solutions for Patient Safety
- ↵SQUIRE. Revised standards for quality improvement reporting excellence 2.0. Available at: http://squire.citysoft.org/index.cfm?fuseaction=page.viewPage&pageID=471&nodeID=1#other. Accessed March 20, 2017
- Miller MR,
- Griswold M,
- Harris JM II, et al
- Miller MR,
- Niedner MF,
- Huskins WC, et al; National Association of Children’s Hospitals and Related Institutions Pediatric Intensive Care Unit Central Line-Associated Bloodstream Infection Quality Transformation Teams
- ↵REDCap [computer program]. Project-REDCap. Nashville, TN: Vanderbilt University; 2014. Available at: http://project-redcap.org/. Accessed September 11, 2015
- Provost L,
- Murray S
- Copyright © 2017 by the American Academy of Pediatrics