Quality Improvement Initiative to Reduce Serious Safety Events and Improve Patient Safety Culture
BACKGROUND AND OBJECTIVE: Many thousands of patients die every year in the United States as a result of serious and largely preventable safety events or medical errors. Safety events are common in hospitalized children. We conducted a quality improvement initiative to implement cultural and system changes with the goal of reducing serious safety events (SSEs) by 80% within 4 years at our large, urban pediatric hospital.
METHODS: A multidisciplinary SSE reduction team reviewed the safety literature, examined recent SSEs, interviewed internal leaders, and visited other leading organizations. Senior hospital leaders provided oversight, monitored progress, and helped to overcome barriers. Interventions focused on: (1) error prevention; (2) restructuring patient safety governance; (3) a new root cause analysis process and a common cause database; (4) a highly visible lessons learned program; and (5) specific tactical interventions for high-risk areas. Our outcome measures were the rate of SSEs and the change in patient safety culture.
RESULTS: SSEs per 10 000 adjusted patient-days decreased from a mean of 0.9 at baseline to 0.3 (P < .0001). The days between SSEs increased from a mean of 19.4 at baseline to 55.2 (P < .0001). After a worsening of patient safety culture outcomes in the first year of intervention, significant improvements were observed between 2007 and 2009.
CONCLUSIONS: Our multifaceted approach was associated with a significant and sustained reduction of SSEs and improvements in patient safety culture. Multisite studies are needed to better understand contextual factors and the significance of specific interventions.
- AHRQ —
- Agency for Healthcare Research and Quality
- CCHMC —
- Cincinnati Children’s Hospital Medical Center
- HSPSC —
- Hospital Survey on Patient Safety Culture
- SSE —
- serious safety event
A decade ago, 2 groundbreaking reports from the Institute of Medicine revealed that defective systems, rather than calloused or careless providers, were the cause of patient safety errors.1,2 Since then, many hospitals have implemented or revised systems to improve patient safety.3–5 In addition, many specialty societies and nongovernmental organizations have focused on improving patient safety.3,6–12 Although there have been some impressive improvements,4 patients continue to suffer from serious and preventable safety events.3,13 Recent multicenter data illustrate that patient harm remains common, and little, if at all, changed over a period from 2002 to 2007.14 Safety events in hospitalized children are common, and they are associated with increased length of stay, in-hospital mortality, and total charges.15–19 Up to 60% of such events may be preventable.17
Cincinnati Children’s Hospital Medical Center (CCHMC) has maintained a significant focus on improving patient safety for many years.20 Three strategic plans since 2001 confirmed quality as our core business strategy and committed us to transforming health care delivery and established organizational readiness via leadership focus, transparency, data infrastructure to support improvement, and continuously increasing improvement capability. Specific improvement initiatives led to measurable reductions in adverse events.22–23 However, during this same time, the number of serious safety events (SSEs) at CCHMC, defined as deviation from generally accepted performance standards resulting in severe or permanent harm,24 remained stable. Therefore, in 2006, leadership committed to an improvement effort focused on cultural and system changes, with the aim of reducing SSEs to 0.2 per 10 000 adjusted patient-days by June 30, 2010. We report here on the details and results of that initiative.
CCHMC is a large, urban pediatric academic medical center. In fiscal year 2010, CCHMC had >32 000 inpatient admissions and 125 000 emergency department (ED) visits and performed 31 000 surgical procedures. The hospital serves a diverse local, national, and international population; ∼43% of patients have Medicaid insurance.
In 2006, a SSE reduction team was formed, consisting of quality and safety leaders, risk management representatives, project managers, a senior decision analyst, and a consultant human factors expert. The faculty leader devoted 50% of his time. The other team members amounted to 4.75 full-time–equivalent employees. Senior hospital leaders provided oversight, monitored progress toward goals, and helped to overcome barriers.
The team reviewed the safety literature (with particular attention focused on reliability in health care),4,25,26 examined the 35 most-recent SSEs at CCHMC (applying methods from Healthcare Performance Improvement24 to analyze causes of the events), and created a common cause database. The team then reviewed CCHMC results of the Hospital Survey on Patient Safety Culture (HSPSC) developed by the Agency for Healthcare Research and Quality (AHRQ).27,28 Finally, the team hired a consultant from Healthcare Performance Improvement who conducted interviews with >100 CCHMC leaders, physicians, and clinical staff. Team members also participated with other health care organizations in AHRQ’s High Reliability Organization Learning Network,7 which included visits to organizations focused on reducing SSEs.29,30 Evidence from the available literature and expert opinion were then used to develop a key driver diagram (Fig 1).
A set of expected safety behaviors for all clinical employees were selected by a multidisciplinary panel of local clinicians on the basis of evidence obtained from a literature review and consultant expertise (Table 1).
An error prevention training program was designed for all direct patient caregivers, staff regularly assigned to a microsystem31 or clinical unit, and leaders with the opportunity to affect patient safety. Training objectives included recognizing the significance of safety and factors contributing to harm for patients and families, reviewing CCHMC patient safety history, identifying safety behaviors expected of all providers, practicing error prevention techniques, and discussing CCHMC accountability plans. To encourage teamwork, clinical microsystems trained together in interdisciplinary groups that included physicians. Didactic training involved dynamic lectures, videos, and interactive small-group discussions. Classes were taught by trained staff members. Continuing education credits were available to attendees. To date, >8100 employees have received training.
Volunteer safety coaches reinforced the reliable use4 of expected safety behaviors with their colleagues, encouraged open communication about safety, identified and shared safety improvement opportunities, and built accountability throughout the organization. More than 300 employees, from many disciplines and every inpatient unit, volunteered to be safety coaches. Safety coach groups were organized according to microsystem or department. They conducted behavioral observations and coached staff on performance of safety behaviors. Each safety coach completed at least 4 observations per month that included specific and immediate positive and negative feedback. Observation details were submitted online, creating a database used to plan additional interventions. Safety coaches on some units regularly led rounds to encourage open discussion of patient safety successes and failures. Safety coach leaders from each microsystem met monthly to share findings, provide support, and honor a coach for outstanding contributions.
Simulation training was used to improve communication and allow teams to practice expected safety behaviors.32–34 During simulations, clinicians learned to find and eliminate latent safety threats, develop communication and teamwork skills, make adjustments to training and practice, and respond to emergencies. Significant capital investment was dedicated to upgrade the simulation center. The initial focus was on high-risk areas and procedures, such as perioperative practices, extracorporeal membrane oxygenation, bone marrow transplantation, and the ED. Additional in situ or on-site simulations occurred regularly across many microsystems.
Safety training sessions were conducted for >600 leaders. They were held accountable for patient safety through rewards and recognition programs, yearly reviews, and performance-based privileging. For example, a portion of compensation for senior executives at the director level and above was tied to goal achievement.
Processes were developed across ICUs to solicit safety concerns from family members throughout the day and to “stop the line”35 until concerns were resolved. When a concern was identified, 2 staff members met with the family and developed a plan to address the concern before continuing care.
One unit implemented, and later spread, a successful pediatric early warning score to detect clinical deterioration and prevent cardiopulmonary arrests.36 Another team developed and tested processes across the entire delivery system for real-time situation awareness.7 The implementation of a medical emergency response team was associated with a reduction in the risk of respiratory and cardiopulmonary arrest outside of critical care areas.37
Restructuring Patient Safety Governance
A patient safety oversight group, consisting of 5 members of the senior leadership team, the patient safety officer (a physician), and the director of patient safety, focused on accountability, balancing quick fixes and long-term solutions in response to safety events, allocating organizational resources and quality improvement infrastructure to strategic priorities,20 and transparency to the organization and the public. They worked to remove barriers and maintain the emphasis on patient safety as the primary concern of the organization. They strategically used meetings with institutional leaders to keep patient safety on the agenda and as a means to drive accountability and execution on safety action plans developed from the cause analysis process (see following text). The patient safety officer called the chief executive officer and the board chairman if an SSE was identified. SSE details and progress on patient safety goals were the first agenda items at monthly meetings of senior leaders and board of trustee meetings.
Cause Analysis Program
A new root cause analysis process for SSEs was implemented based on a 3-meeting model developed by Healthcare Performance Improvement.24 After an extensive investigation, meeting 1 was devoted to establishing facts about the event; meeting 2 was used to determine root causes; and meeting 3 was used to make recommendations for changes and develop action plans. The focus was on root cause(s), systems thinking, effective investigation techniques, credible findings, and actionable improvement plans.
A common cause database was developed to support analysis of inappropriate actions, system and individual failure modes, and error types and to examine trends and inform improvement opportunities.
For each identified SSE, a senior executive was immediately assigned to oversee an analysis and completion of action plans to prevent reoccurrence of similar events.
Lessons Learned Program
Staff were given access to information, creating a highly visible, transparent feedback mechanism, emphasizing key concepts to promote and advance a culture of safety. The CCHMC intranet site, available to all employees, provided monthly updates on all patient safety goals, a patient safety tracker that displayed the number of days since the last SSE, safety stories, and a weekly report from the safety officer designed to reinforce expected safety behaviors, increase staff mindfulness and awareness, celebrate successful interventions, and share details of failures.
Tactical Interventions for High-Risk Areas
Interventions to reduce perioperative SSEs included a list of expected safety behaviors specific to operating room safety developed by surgeons, embedding “time outs” and “debriefs” into standard surgical practice, providing visual cues in the operating room, standardized checklists, smoothing patient flow into the ICU,38 parent review, real-time feedback, educational videos, and executive rounds.
The primary outcome measure was the rate of SSEs per 10 000 adjusted patient-days from January 2003 through June 2011. The secondary outcome was patient safety culture over 5 years (2005–2009).
Serious or permanent harm was defined by using a framework developed by Healthcare Performance Improvement24 (Table 2). Harm could result from errors of commission or omission. A review panel consisting of the chief of staff, patient safety officer, and 2 attorneys from risk management reviewed all potential SSEs identified by an anonymous, web-based patient safety reporting system, supplemented by a trigger tool system,19 telephone call alerts, and direct communications to the safety officer. The panel met biweekly and used a standardized evaluation to determine performance deviation and assessment of harm. Emphasis was on consistency and precedent in determination of whether an SSE had occurred. Clinicians with subspecialty knowledge expertise were consulted in most cases.
Each year since 2005, all patient care providers at CCHMC have been asked to anonymously complete the web-based AHRQ HSPSC.27,28 The survey asks respondents to share their experiences and perceptions over the previous 12 months regarding 7 unit-level and 3 hospital-level dimensions of safety culture and 4 outcome variables. Demographic data were not collected to protect respondent anonymity.
Human Subject Protection
The institutional review board determined that this was a local systems improvement project and did not require their review and approval.
Statistical process control charts were used to examine the impact of interventions on the number of SSEs.39,40 The number of events per 10 000 adjusted patient-days (including inpatient admissions, ED visits, and short stays) was tracked on a U (unit) chart based on count data with a Poisson distribution. The days between SSEs were tracked on a T (time between) chart, used to track rare events and with the expectation that the number of days between events would grow after intervention.41 The upper and lower control limits on the charts establish the margins within which the measurement will be found ∼99% of the time. Wilcoxon rank-sum test analyses of the change in the rate of events were also conducted.
For survey data, as recommended by AHRQ,28 the percentage of positive responses was calculated (strongly agree/agree or always/most of the time) across items in each HSPSC dimension. Responses to negatively worded questions were reversed before calculating the percentage of positive responses. For patient safety grade, a positive response was denoted by a response of excellent or very good. For the number of events reported, a positive response was assigned if a respondent submitted >1 report in the previous year. Finally, the mean percentages for each dimension and overall respondents were calculated. Scores for each dimension could range from 0% to 100%, with higher scores indicating a more positive response. The mean percentage over each year of the survey was adjusted to account for individual differences in how long the respondent worked at CCHMC, in the current hospital area or unit, and in the current specialty or profession; the number of weeks per year usually worked; and whether the respondent had direct or indirect patient contact.
All data analyses were performed by using SAS version 9.2 (SAS Institute, Inc, Cary, NC). Adjusted mean percentages and 95% confidence intervals were calculated by using the LSMEANS option in PROC GLM. Statistical differences across years were obtained by using pair-wise multiple comparison tests with Bonferroni corrections.
After intervention implementation, the number of SSEs per 10 000 adjusted patient-days (Fig 2) significantly decreased from a mean of 0.9 to 0.3 (P < .0001). Days between SSEs (Fig 3) increased from a mean of 19.4 to 55.2 (P < .0001). During this same time, patient volume increased; monthly average adjusted patient-days were 13 686 during the baseline period and 17 521 during the study period. The reduction in SSEs occurred gradually but then seemed to stabilize at 0.3 SSE per 10 000 adjusted patient-days. We believe this reflects a combined effect from system improvements and cultural change.
This combination of interventions, by nature, required time and relentless clarity of vision by the organization, with dedicated resources and focus on key clinical areas. As more employees were trained, we discussed methods to observe and measure performance of expected safety behaviors. We decided to focus, instead, on using the perceptions and qualitative feedback of safety coaches to reinforce behaviors and estimate the penetration of behaviors at the microsystem level. Senior leaders made transparent sharing of information regarding SSEs the expected norm, modeled the new culture through their actions, and directed attention to key clinical areas and processes by continuously focusing on results. The overall measure reflects performance across the organization, but the interventions required changes in dozens of clinical microsystems. Individual clinical areas adopted changes at varying paces, so persistence over time was vital to achieving the overall organizational goal. Specific tactical interventions were developed and implemented where they were needed, and results were monitored continuously at the microsystem level.31 Together, these interventions helped to focus leaders, physicians, and staff attention on patient safety and potential risks each day.
Patient Safety Culture
Response rates generally increased over time, especially for nurses, who received encouragement and incentives (eg, pizza lunches) to complete the survey. Between 2005 and 2007, there was a general decreasing trend in the mean percentage of respondents giving a positive response across several patient safety culture outcomes and dimensions (Table 3). However, significant improvements in positive responses in several outcomes and dimensions were seen between 2005 and 2009 and between 2007 and 2009.
After implementation of cultural and system changes to improve patient safety, we observed significant reductions in the number of SSEs and increases in the days between SSEs. Although we did not reach our goal of 0.2 events, both statistical process control and Wilcoxon rank-sum analyses confirm these improvements to be statistically significant. More importantly, they are unquestionably clinically significant, equating to 62 fewer SSEs over the last 5 years. During the initial phase of the interventions, results from the safety culture survey worsened. However, as the initiative progressed, there was improvement.
Our experience of significant reduction in SSEs is the first that we are aware of in the published literature. Although we do not have data on which drivers or interventions most affected this change, our bundle of an error prevention system, restructured safety governance, a cause analysis structure, lessons learned program, and specific tactical interventions was strongly associated with fewer SSEs. We believe this set of interventions, designed to reach every employee, is likely to achieve similar patient safety advances in other health systems if adjusted to the local context.
A more positive safety culture has been associated with fewer adverse events in hospitals.42 Many hospitals have measured and reported their patient safety culture; however, there are few reports of multiple surveys to determine the change in patient safety culture over time. Our leadership was acutely aware of the initial decrease in some patient safety culture outcomes in the second and third surveys. Such results have been reported previously and may be due to the increased focus on patient safety and error prevention and the perception that change is not happening fast enough.43–46 In addition, change takes time. Our leadership was confident that our interventions would eventually result in an improved safety culture.
Our overall response rates were calculated by using total employees as the denominator, rather than number of clinicians. Nonetheless, the improvement in the overall patient safety grade from 2007 (1 year after the interventions began) to 2009 was statistically significant. Although our study design did not allow us to attribute causation, our experience with this and other quality improvement initiatives leads us to believe that our interventions drove the culture change that was a prerequisite to safety outcomes improvement. In addition, our experience seems to affirm that process improvement initiatives alone are not sufficient to drive necessary culture change.
The major theoretical model for our approach was based on 5 key concepts guiding high reliability organizations: sensitivity to operations, reluctance to simplify, preoccupation with failure, deference to expertise, and resilience.4,5 These organizations strive to create a culture and processes that drastically reduce system failures and effectively respond when failures do occur.7 Our systematic, whole-system approach to reducing SSEs included a transparent, consistent method for identifying safety events; a standardized taxonomy for classifying failures; and a strict method to determine root causes, drive development of specific interventions, and focus organizational attention.
Some limitations of our study are that the interventions occurred at a single site, we do not have data from a control hospital, and multiple interventions were made simultaneously, resulting in what, in describing the etiology of disease, has been called a web of causation.31,47 We believe the simultaneous, multifaceted approach was crucial in changing behavior and culture and that no single change was responsible for the results obtained. In addition, this work was conducted over several years at a large pediatric medical center with a long history of improving care20 and a robust infrastructure for tracking outcomes and harm. However, we believe these methods can be generalized to other health care organizations if adapted to address contextual factors.48,49 Multisite studies and a detailed analysis of the percent penetration of the interventions will allow further improvement. For example, we are currently in a collaborative with all 8 children’s hospitals in Ohio to build improvement capability, reduce SSEs, and improve patient safety culture.
Improving patient safety is an ongoing challenge. We continue to focus on changing behavior and understand that this will require years of persistence. We believe additional factors and experience from outside health care will be important to further improvement. We have begun a new, organization-wide focus on situation awareness (ie, knowing what is going on around you and what is likely to happen next) to eliminate unrecognized clinical deterioration and delayed diagnosis as a cause of serious harm.7 We are also increasing emphasis on human factors engineering; that is, using knowledge about human behavior in system design and redesign.7,50
Our multifaceted approach, focusing on culture and system changes, was associated with a significant and sustained reduction of SSEs. This approach may be applicable to other sites. Multisite studies are warranted to better understand contextual factors and the significance of individual interventions.
- Accepted March 14, 2012.
- Address correspondence to Stephen E. Muething, MD, James M. Anderson Center for Health Systems Excellence, Cincinnati Children’s Hospital Medical Center, 3333 Burnet Ave, Cincinnati, OH 45229. E-mail:
The results of this initiative were presented at the 15th International Scientific Symposium on Improving the Quality and Value of Health Care; December 7, 2009; Orlando, FL.
Dr Goudie’s current affiliation is Center for Applied Research and Evaluation, Department of Pediatrics, University of Arkansas for Medical Sciences and Arkansas Children’s Hospital, Little Rock, AK. Dr Donnelly’s current affiliation is Nemours Children’s Hospital, Orlando, FL.
All of the listed authors are responsible for the reported research. Dr Muething, Ms Goodfriend, Mr Anderson, and Dr Kotagal contributed to the conception and design of the project; Dr Muething, Dr Goudie, Ms Goodfriend, and Ms Bracke contributed to the acquisition of the data; Dr Muething, Dr Goudie, Ms Schoettker, and Ms Bracke contributed to the analysis and interpretation of the data; Dr Muething, Dr Goudie, and Ms Schoettker drafted the manuscript; and Dr Muething, Dr Goudie, Ms Schoettker, Dr Donnelly, Ms Goodfriend, Ms Bracke, Dr Brady, Dr Wheeler, Mr Anderson, and Dr Kotagal contributed to the critical revision of the manuscript for important intellectual content. All authors approved the manuscript as submitted.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Dr Goudie received funding through a grant from the Agency for Healthcare Research and Quality (grant U18HS016957) for his work on this project. The funding source had no involvement in the study design; collection, analysis, or interpretation of the data; writing of the report; or decision to submit this article for publication. Funded by the National Institutes of Health (NIH).
- Institute of Medicine, Committee on Quality Health Care in America
- Institute of Medicine, Committee on Quality of Health Care in America
- Weick KE,
- Sutcliffe KM,
- Obstfeld D
- ↵American Board of Medical Specialties. ABMS performance improvement products. ABMS patient safety courses. Available at: www.abms.org/Products_and_Publications/Performance_Improvement/PSIP.aspx. Accessed June 22, 2010
- ↵Hines S, Luna K, Lofthus J, Marquardt M, Stelmokas D. Becoming a High Reliability Organization: Operational Advice for Hospital Leaders. Prepared by the Lewin Group under Contract No. 290-04-0011. Rockville, MD: Agency for Healthcare Research and Quality; 2008
- Institute for Healthcare Improvement. Closing the quality gap. Available at: www.ihi.org/NR/rdonlyres/9FEE1B6D-9B26-481D-B29C-A5D99E6CCFF8/0/IntrotoIHIFinal.pdf. Accessed June 22, 2010
- National Quality Forum. Serious reportable events in healthcare - 2011 update: a consensus report. Washington, DC; 2011. Available at: www.qualityforum.org/projects/hacs_and_sres.aspx. Accessed April 24, 2012
- National Quality Forum. Safety practices for better health care. Washington, DC; 2003. Report number NQFCE-05-03. Available at: www.ahrq.gov/qual/nqfpract.pdf. Accessed April 24, 2012
- American Hospital Association, Health Research and Educational Trust, Institute for Safe Medication Practices. Pathways for medication safety. Developing tools for medication safety. Chicago, IL: Health Research and Educational Trust; 2002. Available at: www.medpathways.info/medpathways/tools/content/phasetwo.pdf. Accessed November 30, 2011
- ↵The Joint Commission. 2011 National patient safety goals now available. Effective July 1, 2011. Available at: www.jointcommission.org/standards_information/npsgs.aspx. Accessed December 9, 2011
- Wachter RM
- Miller MR,
- Elixhauser A,
- Zhan C
- Slonim AD,
- LaFleur BJ,
- Ahmed W,
- Joseph JG
- Woods D,
- Thomas EJ,
- Holl J,
- Altman S,
- Brennan T
- Matlow AG,
- Cronin CM,
- Flintoft V,
- et al
- Muething SE,
- Conway PH,
- Kloppenborg E,
- et al
- Jacobs B
- Miller MR,
- Griswold M,
- Harris JM II,
- et al
- ↵Healthcare Performance Improvement, LLC. HPI: innovative solutions in healthcare performance. Available at: http://hpiresults.com/. Accessed November 30, 2011
- ↵Berwick D, Nolan T. High reliability health care presentation. Available at: www.ihi.org/knowledge/Pages/Presentations/HighReliabilityHealthCarePresentation.aspx. Accessed August 22, 2011
- Nolan T,
- Resar R,
- Haraden C,
- Griffins FA
- ↵Agency for Healthcare Research and Quality. Hospital survey and patient safety culture. Available at: www.ahrq.gov/qual/patientsafetyculture/hospsurvindex.htm. Accessed February 16, 2012
- ↵Sorra JS, Nieva VF. Hospital Survey on Patient Safety Culture. Rockville, MD: Agency for Healthcare Research and Quality; 2004. AHRQ Publication No. 04-0041
- Patterson MD, Blike GT, Nadkarni VM. In situ simulation: challenges and results. In Henriksen K, Battles JB, Keyes MA, Grady ML, eds. Advances in Patient Safety: New Directions and Alternative Approaches, Volume 3: Performance and Tools. Rockville, MD: Agency for Healthcare Research and Quality; 2008. Available at: www.ahrq.gov/downloads/pub/advances2/vol3/Advances-Patterson_48.pdf. Accessed August 22, 2011
- Ohno T
- Brilli RJ,
- Gibson R,
- Luria JW,
- et al
- Benneyan JC,
- Lloyd RC,
- Plsek PE
- ↵Provost L, Murray S. The Data Guide. Learning From Data to Improve Health Care. Austin, TX: Associates in Process Improvement, Corporate Transformation Concepts; 2007
- Adams-Pizarro I, Walker ZA, Robinson J, Kelly S, Toth M. Using the AHRQ hospital survey on patient safety culture as an intervention tool for regional clinical improvement collaboratives. In: Henriksen K, Battles JB, Keyes MA, Grady ML, eds. Advances in Patient Safety: New Directions and Alternative Approaches, Volume 2: Culture and Redesign. Rockville, MD: Agency for Healthcare Research and Quality; 2008
- ↵Jones KJ, Skinner A, Xu L, Sun J, Mueller K. The AHRQ hospital survey on patient safety culture: a tool to plan and evaluate patient safety programs. In: Henriksen K, Battles JB, Keyes MA, Grady ML, eds. Advances in Patient Safety: New Directions and Alternative Approaches, Volume 2: Culture and Redesign. Rockville, MD: Agency for Healthcare Research and Quality; 2008
- MacMahon B,
- Pugh TF
- Pawson R,
- Tilley N
- Copyright © 2012 by the American Academy of Pediatrics