Abstract
BACKGROUND AND OBJECTIVE: Failure to recognize and treat clinical deterioration remains a source of serious preventable harm for hospitalized patients. We designed a system to identify, mitigate, and escalate patient risk by using principles of high-reliability organizations. We hypothesized that our novel care system would decrease transfers determined to be unrecognized situation awareness failures events (UNSAFE). These were defined as any transfer from an acute care floor to an ICU where the patient received intubation, inotropes, or ≥3 fluid boluses in first hour after arrival or before transfer.
METHODS: The setting for our observational time series study was a quaternary care children’s hospital. Before initiating tests of change, 2 investigators reviewed recent serious safety events (SSEs) and floor-to-ICU transfers. Collectively, 5 risk factors were associated with each event: family concerns, high-risk therapies, presence of an elevated early warning score, watcher/clinician gut feeling, and communication concerns. Using the model for improvement, an intervention was developed and tested to reliably and proactively identify patient risk and mitigate that risk through unit-based huddles. A 3-times daily inpatient huddle was added to ensure risks were escalated and addressed. Later, a “robust” and explicit plan for at-risk patients was developed and spread.
RESULTS: The rate of UNSAFE transfers per 10 000 non-ICU inpatient days was significantly reduced from 4.4 to 2.4 over the study period. The days between inpatient SSEs also increased significantly.
CONCLUSIONS: A reliable system to identify, mitigate, and escalate risk was associated with a near 50% reduction in UNSAFE transfers and SSEs.
- patient safety
- situation awareness
- rapid response systems
- clinical deterioration
- quality improvement
- high-reliability organizations
- hospital medicine
- ACA —
- apparent cause analysis
- EHR —
- electronic health record
- HRO —
- high-reliability organization
- MPS —
- manager of patient services
- MRT —
- medical response team
- PEWS —
- pediatric early warning score
- RRT —
- rapid response team
- SA —
- situation awareness
- SOD —
- safety officer of the day
- SSE —
- serious safety event
- UNSAFE —
- unrecognized situation awareness failure event
Rapid response teams (RRTs) are designed to identify and respond to events in the hours prearrest.1–13 Although interventions and contexts have varied substantially, these teams have demonstrated decreased codes outside the ICU and hospital-wide mortality in several studies.1–4,7,9,10,12,13 Variation in the effectiveness of RRTs may be due to insufficient processes around monitoring and risk identification.14 Potentially preventable morbidity and mortality from unrecognized deterioration remain, often due to ineffective clinical monitoring that we believe represents poor situation awareness (SA).14 SA (ie, “knowing what is going on”) exists at 3 levels and is defined as “the perception of elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future.”15–17 We believe improved SA drives better recognition of early deterioration and is essential in efforts to reduce “failure to rescue” from codes outside of the ICU, an event associated with a 50% to 67% mortality.18,19
High-reliability organizations (HROs) (eg, commercial aviation, nuclear power, and wilderness firefighting) deal with constant and catastrophic risk yet maintain exemplary safety records.20 Our institution began a journey to become an HRO with the Agency for Healthcare Research and Quality HRO Learning Network in September 2005.21 Learnings from this network fueled improvement work and introduced the concept of SA. Our organization has targeted serious safety event (SSE) reduction as a strategic improvement goal since 2006. Our efforts to reduce SSEs, defined as severe harm or death after variation from expected practice, have resulted in significant and sustained reduction.22 Before SA work, SSEs among inpatients had not decreased. Poor SA was a common etiology. To achieve our aim and facilitate rapid learning, our project team defined a precursor outcome measure that we believed would capture events that both represented SA failures and occurred more commonly than inpatient SSEs in our center.23 We prospectively defined unrecognized situation awareness failures events (UNSAFE) as the transfer of patients from the acute care floor to the ICU where the patient received tracheal intubation, initiation of vasoactive medications for hemodynamic support, or ≥3 fluid boluses in the first 60 minutes of ICU care or before arrival in the ICU. We believed these events represented potentially delayed transfers that are precursors to codes outside the ICU or SSEs. With focused improvement work beginning in November 2009, we aimed to decrease UNSAFE transfers by 50% by June 30, 2011. We hypothesized that a system of care that proactively identified, mitigated, and escalated risk would improve SA and decrease UNSAFE transfers and SSEs.
Methods
Setting
Cincinnati Children’s Hospital Medical Center is a 523-bed academic, quaternary-care, free-standing children’s hospital. Resident teaching teams care for the majority of hospitalized patients. Less commonly, direct hospitalist and nurse-practitioner models are used. Our RRT (called a medical response team [MRT]) has been in place since 2006 with defined activation criteria.2 A modified version of the Monaghan pediatric early warning score (PEWS) was tested and spread across the hospital in 2007.24
Human Subjects Protection
Our study was reviewed by the institutional review board and deemed exempt systems improvement. Individual patient care was discussed among clinicians in the course of identification and mitigation of risk. Medical record review was performed by the lead investigator by using a secure password-protected internal database and our hospital’s electronic health record (EHR).
Event Review
Two investigators (Dr Brady and Ms Goodfriend) reviewed 20 consecutive SSEs and 80 consecutive ICU transfers to identify potential predictors of deterioration. The presence of at least 1 of the following 5 risk factors was found in each case: (1) family concern about patient safety, (2) high-risk therapies including unfamiliar therapies on the unit (eg, insulin use outside of the diabetes unit), (3) elevated PEWS of ≥5, (4) watcher or a patient where a clinician had a “gut feeling” that the patient was at risk for deterioration or “close to the edge,” and (5) communication concern that may impact patient safety.
Intervention
The SA intervention included the following: (1) a formalized process where bedside nurses proactively identified these 5 factors, (2) unit-based huddles where charge nurses and physicians discussed identified factors and developed mitigation plans, (3) initiation of 3-times daily inpatient huddle where individual patient risk was discussed and specific predictions made, (4) development of a continuous learning system to evaluate SA and UNSAFE transfers, and 1 year later (5) development of a “robust” and explicit plan for patients identified as having 1 of the risk factors. Figure 1 provides a model of communication and action pathway for identification of patient risk. Figure 2 is the key driver diagram that illustrated the study team’s belief in hypothesized drivers needed to achieve the aim. Table 1 provides details on individual interventions.
Identify, mitigate, and escalate model illustrates which risk factors were systematically identified and how standardized communication about risk occurred throughout the center.
Key driver diagram illustrates the drivers (at right) that would lead to aim through improved situation awareness and no unrecognized clinical deterioration.
Specific Interventions, Settings, and Timing for Each Intervention
Proactive Identification of Risk
In our SA model (Fig 1), the fundamental job of the bedside nurses and interns was to identify any of the 5 risk factors. These clinicians have the most touch time with the patients but are generally the least experienced on the team. Tools were developed to support bedside nurses in identifying risk factors during their routine assessments. Structured yes/no questions (Fig 3) were prototyped, tested, and later spread.
Situation awareness algorithm illustrates the tool used during education and early phases and the specific questions and communication pathways.
Unit-Based Huddles
Unit-based huddles between the charge nurse and bedside nurse, regardless of identified risk, were scheduled every 4 hours. Huddles also were promoted whenever new risk factors were identified. The aim of the huddle was to trigger a bedside evaluation by experienced nurses and physicians of any patient with the identified risk. Huddles were led by a watchstander charge nurse and senior resident when risk was identified. The term “watchstander” was borrowed from the military to highlight that the primary job of the charge nurse and senior resident was to know which patients were at high risk for deterioration. When concerns were identified, the clinical care team discussed the risk at the patient’s bedside and developed a plan to mitigate that risk. This provided a standardized opportunity for more experienced clinicians to coach those less experienced in both patient management and communication/escalation techniques.
Three-Times Daily Inpatient Huddle
Our design later leveraged existing structures to proactively escalate identified and unresolved safety risks. Historically, our organization conducted an 8 am huddle that brought together a charge nurse from each inpatient unit and the manager of patient services (MPS) who oversaw the flow and staffing of inpatients. Before our work, there was no standardized discussion of patient safety. We piloted with 4 units and then spread to all noncritical care units a structured process where the charge nurse from each unit (1) reported on any risk factors present that were not fully addressed and (2) predicted any MRT activations. This process continued to be facilitated by the MPS, and a safety officer of the day (SOD) attended each inpatient huddle. The SOD was an experienced pediatrician who was given the authority by our hospital leadership to assist with mitigating any safety and/or communication concerns on the inpatient units. The MPS and SOD provided coaching on how to address concerns raised by the primary clinical team, including positive reinforcement of key behaviors and role modeling. The details of testing similar huddles at 4:30 pm and midnight are displayed in Table 1.
Continuous Learning System to Evaluate SA
The final component of our planned intervention was to develop a data system to rapidly identify process and outcome failures and direct this information to project leaders and leaders on individual units. To achieve this aim, (1) apparent cause analysis (ACA) forms were completed within 1 hour of each floor to ICU transfer to identify potential UNSAFE transfers and associated process failures, (2) a password-protected database was constructed to integrate information from these forms and the EHR, (3) process and outcome data were distributed each week to unit level clinical and medical directors with a story of patient-level SA, and (4) a control plan was designed with inpatient leaders to identify special cause on tracked process and outcome measures and target further interventions.
Robust Plan
One year after SA work began, the improvement team worked with 1 inpatient unit to develop and test a checklist tool to improve the mitigation/escalation process for patients with identified risk (Table 1). During multidisciplinary discussions, we proposed a “robust plan” bundle which included the following: (1) plan with proposed treatment change, (2) explicit communication with care team, (3) prediction of expected outcome, (4) outcome deadline, and (5) escalation plan (usually the MRT or discussion with SOD/MPS) if outcome was not achieved by a predefined deadline. This tool was tested, adapted, and spread throughout the remaining inpatient units (Fig 4). Subsequent testing integrated risk identification within the EHR and added focused discussion of a robust plan during safety rounds.
Situation awareness robust planning tool.
Study of the Intervention
In our observational time series study, data were collected on process measures of systematic identification, mitigation, and escalation of risk that we believed would improve SA and decrease UNSAFE transfers and SSEs. To evaluate the consistency of huddles and how well the identify, mitigate, and escalate intervention was implemented, data initially were collected from each unit on each nursing shift to measure the reliability that each shift identified all patients at risk and mitigated or escalated that risk. This was captured through a checklist-based form that followed the flow of algorithm of Fig 3 and was completed by each charge nurse. The tool was tested and evaluated with charge nurses from several units during early phases. Before spread throughout the hospital, 116 charge nurses received training on the process and tool through a 1.5-hour learning session. Validity of process data were evaluated through discussions during inpatient huddles by investigators, SOD, and MPS. UNSAFE transfers were identified from the ACA process and validated against review of the EHR for each ICU transfer. SSEs were captured through a safety reporting process as previously described.22
Analysis
Primary analysis of both process and outcome measures was performed by using statistical process control charts. For the primary outcome of UNSAFE transfers, results were tracked by using both a days-between t-chart and rate chart. Established rules for identifying special cause were employed.25–27
Results
After testing on 4 inpatient units from January 1, 2010, to March 21, 2010, the unit-level huddles and proactive inpatient huddles began on each of the 14 noncritical care inpatient units on March 22, 2010. The process measure evaluated the consistency of huddles and specifically how frequently patient risk was identified and mitigated or escalated each nursing shift on each unit. The number of units by week where ≥90% of weekly nursing shifts fully identified and mitigated or escalated patient risk were tracked on run charts and revealed both improved and sustained performance for 11 months of tracking (Fig 5). On each participating unit, 90% to 95% of identified risk was mitigated by the primary team with no escalation needed. Each inpatient huddle took less than 30 minutes. Although initially there was substantial variation in the number of patient risks that were escalated, a median of 2 risks for each huddle were escalated the first year. This increased over the study period with a median of 7.5 concerns escalated in May 2012.
Process measure run chart illustrating the number of units by week where ≥90% of weekly nursing shifts fully identified patients at risk (solid line/diamond) and where ≥90% of weekly shifts fully mitigated or escalated that risk (dotted line/circle).
The rate of UNSAFE transfers per 10 000 non-ICU inpatient days is displayed in Fig 6. An initial decrease in UNSAFE transfers occurred, though it did not meet rules for special cause and was not sustained. Analysis of UNSAFE transfers through an ongoing ACA process revealed that in the vast majority of UNSAFE transfers, patient risk had been identified but not fully mitigated on unit or escalated to the MRT or safety team. This led to focused improvement work on development of a robust plan as detailed above. After spread, the rate of UNSAFE transfers improved from a baseline of 4.4 to 2.4 transfers per 10 000 non-ICU inpatient days, meeting criteria for special cause variation with 8 points below the median line (Fig 6). Additionally, a significant change in the days-between inpatient SSEs from 100 days to >400 twice was observed in association with the intervention. Shortly before this work began, the number of MRT activations and PICU transfers per month increased significantly in association with a campaign to eliminate informal PICU consults (Fig 7). MRT-preventable codes outside the ICU were rare before and during the study period (Fig 8).
UNSAFE transfer rate chart. Rate of UNSAFE transfers per 10,000 non-ICU patient days at base location by month (n = non-ICU inpatient days by month).
Rate of MRT activations and transfers to ICU by month.
Rate of MRT preventable codes outside of the ICU by month.
Discussion
Our system to reliably identify, mitigate, and escalate patient risk was associated with a significant decrease in UNSAFE transfer and SSEs among inpatients. By providing a proactive and reliable model to identify and rapidly intervene on newly detected risk and early patient deterioration, we improved SA across our inpatient system. SA is achieved by scanning for important information, recognizing patterns and trends, and making short-term predictions.15–17 We believe that although SA is still uncommonly discussed in the medical literature, interventions such as rapid response systems and early warning scores (both in broad use) employ SA to identify deteriorating patients.28 We sought to build upon this work by addressing limitations in many rapid response systems’ inherently reactive (versus proactive) nature.14
We developed the concept of UNSAFE transfers to be used as the primary outcome measure for improvement efforts with the belief that these events represented potential precursor events to serious harm such as codes and because these events occurred sufficiently often in our center to enable rapid testing, learning, and adapting. Our time series design allowed us to learn sequentially, and we observed an initial decrease in the rate of UNSAFE transfers followed by a return to our previous baseline levels. With process measures in place that did not reveal a decrease in reliability of our intervention, we further studied where the process failures occurred regarding UNSAFE transfers. With this data, we learned that although 1 year into the intervention we had a system that identified risk, we did not systematically address this risk. We found that even on our most high risk patients, the language of plans included terms such as “continue to observe.” Without explicit and time-bound plans, clinicians were observing patients until they received aggressive resuscitation that met criteria for UNSAFE transfer.
Our second stage of interventions testing a robust plan and prediction was designed by a multidisciplinary team of leaders and front-line physicians, nurses, and respiratory therapists. The improvement team believed that a shared mental model or team SA would not be achieved without explicit prediction and contingency planning. We believed that this was because level 3 SA (the projection of current event status in the near future) was still often not achieved. This may have been due to the limits of individual clinicians in making near-term projections (eg, this tachycardic patient will be in shock within 4 hours if we do not aggressively hydrate) but was believed to more commonly result from doctors, nurses, and other members of the care team not explicitly sharing their mental model. Our basic theory was borrowed from hypothesis testing in the scientific method and explicit predictions in plan-do-study-act cycles. The goal was to increase accountability and to make disconcerting data (eg, patient did not improve as predicted) clear to each member of the team. The spread of this intervention and its integration into proactive inpatient huddle was associated with a sustained decrease in UNSAFE transfers.
One strength of our work is that we created a system of care built on reliable processes, not individual clinicians. We were able to build these processes into the workflows of busy clinicians and provide via the inpatient huddle a valued activity for charge nurses as they gained insight and assistance with their sickest patients. Our sustained reduction in UNSAFE transfers over the last 12 months is further evidence of the success of building interventions into work flow. Importantly, this intervention did not add additional clinicians to our system of care but instead clarified roles and processes for charge nurses, MPS, residents, and attending physicians. The additional responsibilities of SODs are on the order of 1.5 hours per day. We believe our work builds upon previous interventions to address patient deterioration such as rapid response systems and PEWS. The proactive and standardized nature of our intervention offers an important answer to afferent limb failures of the MRTs.14 Our intervention supplements the early warning score with other risk domains, most powerfully for us was that of the “watcher” or patient that a clinician has “a gut feeling is close to the edge.” We believe this employs the tacit knowledge of experienced clinicians and hence likely will achieve greater sensitivity than any numerical scoring tool, especially since we combined this concept with objective data.29,30 We also believe that assessment of risk as relayed by family and as emerges from communication problems has substantial face validity in identifying and predicting deterioration. Our final risk category was that of high risk therapy and borrowed from HRO thinking on the need for special oversight and procedures with new and unfamiliar therapies; for example, we believe the administration of insulin on short-stay surgery unit has a fundamentally different risk profile than doing so on diabetes unit. We therefore target these high risk therapies and address any knowledge gaps in close to real-time. Our intervention is perhaps most similar to that of the Rover team as described by Hueckel et al.31 Although both are proactive in assessment of risk, meaningful differences include our intervention’s staffing model and broader scanning for risk.
Because our goal was rapid improvement of a single site, we chose a time series design, which did not allow us to address secular trends or establish causality. We believe this design was appropriate for our innovative intervention that evolved and improved through iterative quality improvement methods. This design exposed the study to potential unmeasured confounding from safety work. We do not feel this was a particularly large risk because time series data reveal the rate of inpatient SSEs had not improved in previous years’ safety work and because there were no other large interventions targeted at this population. Additionally, it is uncertain how our results would generalize to medical centers with different patient populations, staffing models, quality improvement capabilities, and safety cultures. A final limitation is that we did not have a measure of SA to reveal that this improved as an effect of our intervention. Available measures of SA involve simulated events and typically require “pausing” the event to perform assessment.32 Clearly this was not possible or ethical in course of clinical care. A recent proposed measure of SA relied on accuracy of prediction that was fundamental to our work.33 We did use SA for much of our conceptual model, but we cannot say definitely if improved SA was the mediator between the identify, mitigate, and escalate intervention and our decreased rate of UNSAFE transfers.
Our institution additionally has applied and tested models to improve SA and decrease adverse events in diverse clinical settings including the operating room and resident psychiatry. We also believe improved SA and systems that proactively identify, mitigate, escalate, and predict risk have applicability to other untoward events such as flow and patient/family experience failures. Next steps include tailoring and testing interventions in these settings and in other hospital systems that have different patient populations and systems of care. Although we have begun to incorporate risk factors into the EHR, we believe simulation and human factors methods are needed to evaluate the role of technology and automation in improving SA and facilitating clinician workflows.
Conclusions
A reliable system to identify, mitigate, and escalate risk can be implemented in a children’s hospital and is associated with a reduction in safety events in a context where these events were already uncommon. We believe HRO thinking and SA are potentially transformative concepts for health care systems. Models to identify risk early and reliably intervene are likely generalizable both to different clinical systems and to modify different outcomes such as the patient/family experience and patient flow.
Acknowledgments
The authors thank Ralph Brueggemann, Ron Lu, and Ripal Soni for their expertise and work in database development; Kathy Dressman for her leadership with the robust plan project; Dick Gardner, Amy Bunger, and Nancy Burse for their work in developing and facilitating education; and Pam Kiessling, Mary Jo Giaccone, Ken Tegtmeyer, and Jan Jacob for their leadership and thinking on this work. Dr Brady also thanks Tom DeWitt for his exceptional mentorship.
Footnotes
- Accepted July 26, 2012.
- Address correspondence to Patrick Brady, MD, MSc, Cincinnati Children’s Hospital Medical Center, ML 9016, 3333 Burnet Ave, Cincinnati, OH 45229. E-mail: patrick.brady{at}cchmc.org
Dr Brady had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis; Drs Brady, Muething, and Kotagal, Mr Ashby, and Dr Wheeler contributed to the study concept and design; Drs Brady, Muething, and Kotagal, Mr Ashby, Ms Gallagher, Ms Hall, Ms Goodfriend, Dr White, Ms Bracke, Ms DeCastro, Ms Geiser, Ms Simon, Ms Tucker, Mr Olivea, and Dr Wheeler contributed to improvement project leadership and execution; Drs Brady and Ashby, Ms Gallagher, Ms Hall, Dr White, and Ms Bracke contributed to the acquisition of data; Dr Brady and Ms Bracke contributed to the statistical analysis; Drs Brady, Muething, and Kotagal, Mr Ashby, Ms Gallagher, Ms Hall, and Drs Conway and Wheeler contributed to the interpretation of data. Drs Brady and Wheeler drafted the article; and all authors contributed to critical revision of the article for important intellectual content.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Dr Brady was supported by funds from the Bureau of Health Professions, Health Resources and Services Administration, Department of Health and Human Services, under grant T32 HP10027. Portions of this project were supported by cooperative agreement number U19 HS021114 from the Agency for Healthcare Research and Quality.
References
- Copyright © 2013 by the American Academy of Pediatrics
In this issue
Jump to section
Related Articles
Cited By...
- Time series evaluation of improvement interventions to reduce alarm notifications in a paediatric hospital
- Reclaiming the systems approach to paediatric safety
- Optimising paediatric afferent component early warning systems: a hermeneutic systematic literature review and model development
- Reliability of instruments that measure situation awareness, team performance and task performance in a simulation setting with medical students
- Improving patient safety through an emergency call safety huddle
- Assessing risks to paediatric patients: conversation analysis of situation awareness in huddle meetings in England
- Development of the Huddle Observation Tool for structured case management discussions to improve situation awareness on inpatient clinical wards
- Paediatric early warning systems for detecting and responding to clinical deterioration in children: a systematic review
- Realistic evaluation of Situation Awareness for Everyone (SAFE) on paediatric wards: study protocol
- Implementation of the Safety Huddle
- Core Principles of Quality Improvement and Patient Safety
- Patient Safety and Quality Improvement: Reducing Risk of Harm
- A stepped wedge, cluster controlled trial of an intervention to improve safety and quality on medical wards: the HEADS-UP study protocol
- Cost-Benefit Analysis of a Medical Emergency Team in a Children's Hospital
- A qualitative study examining the influences on situation awareness and the identification, mitigation and escalation of recognised patient risk
- Huddling for high reliability and situation awareness