BACKGROUND AND OBJECTIVE: Patient safety events are underreported by physicians. Baseline data demonstrated that physicians submitted 3% of event reports at Our Lady of the Lake Children’s Hospital. Our aim was to increase the proportion of safety reports filed by residents and faculty to 6% of all reports within a 9-month period.
METHODS: We used the Model for Improvement and serial Plan, Do, Study, Act cycles to test interventions we hypothesized would improve physician recognition and reporting of patient safety events. We tracked the percentage of Our Lady of the Lake Children’s Hospital event reports entered by residents or faculty over time as the primary outcome measure. Changes to teaching team processes included “patient safety rounds” prompted by text messages, an inpatient “superintendent” rotation with core patient safety responsibilities, and a “just-in-time” faculty development program called “QI on the Fly.”
RESULTS: Physician-reported events increased to a monthly average of 24% of all events reported, an improvement that has been sustained over 17 months. Resident reporting accounted for most of the increase in physician reports. Increased physician reporting was temporally associated with implementation of the “superintendent” rotation. The total number of events reported increased as a result of increased physician reporting.
CONCLUSIONS: Incorporating patient safety responsibilities into a teaching team’s workflow can increase physician safety event reporting. We plan additional Plan, Do, Study, Act cycles to spread this approach to other clinical settings and investigate the impact increased reporting might have on patient care.
- IHI —
- Institute for Healthcare Improvement
- OLOL —
- Our Lady of the Lake
- OLOLCH —
- Our Lady of the Lake Children’s Hospital
- PDSA —
- Plan, Do, Study, Act
- PGY —
- postgraduate year
- QI —
- quality improvement
- RCA —
- root cause analysis
- RCI —
- rapid cycle improvement
- super —
Preventable harm in health care is a major public health problem.1–4 Safety event reporting serves to identify and mitigate hazards to prevent harm.5 Event reporting systems are nearly universal in US hospitals because of their potential to improve care.6 Despite the availability of reporting systems and potential benefits attributed to event reporting, safety events remain underreported, particularly by physicians.7–10 Recently, the Accreditation Council for Graduate Medical Education has stressed the importance of patient safety education and physician reporting of safety events in improving clinical learning environments.11,12 A team at Our Lady of the Lake (OLOL) including representatives from the pediatric residency program and the OLOL Children’s Hospital (OLOLCH) patient safety team set out to analyze and improve local event reporting by physicians. Baseline data within the OLOLCH demonstrated that physicians, encompassing both residents and faculty, entered 3% of pediatric safety events in the hospital’s reporting system. Our aim was to increase the proportion of safety reports filed by residents and faculty to 6% of all reports within a 9-month period. In this report, we describe our workflow changes, additional resident education and faculty development efforts designed to meet this improvement aim, and discuss implications for fostering a culture of physician reporting within an embedded children’s hospital.
The OLOL pediatric residency is composed of 33 residents as well as a core and subspecialty/community faculty of >90 physicians. The program has primary inpatient responsibilities at OLOLCH, a 93-bed hospital within a regional medical center. The pediatric hospitalist service, consisting of 5 hospitalists and 2 physician assistants, oversees care of 90% of non-ICU admissions and delivers the inpatient resident training experience using a teaching-team model (2 teams, each composed of 1 faculty member, 1 postgraduate year (PGY) 2–3 resident, 1 PGY-1 resident, and up to 2 medical or midlevel students). The standard patient safety and quality improvement curriculum for residents is composed of 6 hour-long didactic sessions each year that cover a range of safety and quality topics. The curriculum includes instructions for safety-event reporting and intermittent feedback on institutional responses to select events reported by residents. This curriculum supplements cursory education on event reporting mechanisms that is provided annually to all employees. During the training program, residents also complete: (1) 12 Institute for Healthcare Improvement (IHI) Open School online modules covering patient safety, quality improvement, and family-centered care topics, and (2) 12 health care matrix exercises.13,14 The health care matrix is a tabular tool that helps residents dissect suboptimal care within the context of Accreditation Council for Graduate Medical Education core competencies and the Institute of Medicine’s dimensions of quality.14 The tool provides a foundation for reflection on care and planning improvement strategies. Baseline patient safety training for faculty has been less extensive. However, only a portion of faculty received additional structured training (consisting of voluntary completion of IHI modules, health care matrix exercise instructions, and sporadic participation in hospital-sponsored Lean Six Sigma activities).
The hospital uses the Quantros Safety and Risk Management web-based event reporting system. Reporting is voluntary, anonymity can be preserved, and any hospital team member can enter events. Starting in November 2013, physicians submitting Quantros reports were encouraged to indicate “resident,” “PGY-(level),” or “faculty” when completing event reports to ensure more accurate tracking of physician-reporting activity; although always an option, identifying one's role when reporting was not common prior to this study. Quantros reports are evaluated formally by the OLOLCH safety team. The Safety Team is a multidisciplinary group that meets weekly (safety team rounds) to investigate safety events, implement necessary process changes, and follow-up on report-based interventions. At baseline, safety team rounds were attended by the chief medical officer, medical director of patient safety, director of nursing, and nurse managers, whereas resident attendance was on a case-by-case basis only.
We used the Model for Improvement and serial Plan, Do, Study, Act (PDSA) cycles to conduct a time series study designed to test changes we predicted would improve physician recognition and reporting of adverse events and near misses.15–17 Qualitative feedback and process measures were tracked to ensure effectiveness of individual PDSA cycles while incorporating lessons learned during each cycle into subsequent interventions. Our institutional review board classified this project as quality improvement.
A summary of system changes is outlined in Table 1. Our initial intervention tested the integration of “patient safety rounds” on the teaching teams. Faculty and chief residents created a faculty development video that demonstrated ways to incorporate safety discussions on teaching rounds. This video, critical for faculty with limited previous exposure to patient safety conversations during team rounds, preceded 8 weeks of text messages sent to attending physicians weekly to prompt teaching team “patient safety rounds.” These text messages reminded faculty to translate adverse events and near misses identified during inpatient rounds into learning opportunities. We predicted that routine “patient safety rounds” would improve recognition of events by enhancing communication and education around patient safety.
The next series of cycles centered on the creation of an inpatient superintendent (“super”) rotation for PGY-3 residents that involved core patient safety responsibilities. In addition to providing clinical and educational support on the inpatient teaching teams, the super was tasked with reporting safety events identified during rounds and promoting team discussion of safety events. More specifically, the super resident was responsible for entering all of the safety events uncovered during each team’s family-centered rounds into Quantros, either during rounds on a mobile workstation or after bedside rounds. If the super did not have direct association with the event, the involved team member (student, resident, or attending physician) would detail the event for the super to ensure sufficient information was submitted (for review by the OLOLCH safety team). The super was designed to float between both inpatient teams during the rotation. Before the super position, this resident would otherwise be on an elective rotation. We predicted that the super rotation would increase reported events by integrating safety event reporting within the resident workflow while promoting safety discussions within the clinical context.
Initial PDSA cycles informed subsequent interventions by demonstrating the importance of: (1) providing continuous feedback to teaching teams on process changes that resulted from reported events; and (2) faculty development around patient safety topics. These observations prompted refinement of the super rotation. The super resident was added as a standing member to the OLOLCH safety team, which rounds weekly on all Quantros events. The super resident’s role is to provide front-line experiential input regarding safety events and relay feedback to the teaching team about process changes resulting from analysis of error reports. To augment event reporting feedback, data regarding the volume and nature of safety events reported were displayed in the resident workroom.
Our last series of cycles revolved around a new faculty development program that we called “quality improvement (QI) on the fly.” We focused on faculty development at this stage of the project in response to the decreased number of physician-entered events that coincided with decreased super coverage. The QI on the fly program provides faculty with education and teaching tips on patient safety topics associated with feedback on recently reported events. The e-mail–based curriculum incorporates insights from IHI, the Agency for Healthcare Research and Quality, and the broader patient safety literature. E-mails are sent to faculty at the beginning of each week and are followed by an end-of-week text message reminder to incorporate patient safety topics during clinical teaching. The goal of this program is to enhance patient safety knowledge of the faculty and residents while providing real-time feedback on event evaluations. We predicted that QI on the fly would improve faculty knowledge and behavior around reporting and, in turn, minimize dependence on residents to maintain physician reporting.
To select process measures and inform our change strategy, we constructed a driver diagram using key drivers of physician reporting (Fig 1).18–24 We then delineated operational definitions and data collection strategies (following the Standards for Quality Improvement Reporting Excellence, or SQUIRE, guidelines).25 The project outcome measure was defined as the percentage of physician (resident and faculty)-entered Quantros events at OLOLCH. The total number of safety events reported by all (physician and nonphysician) OLOLCH staff was tracked to calculate this percentage. As secondary outcome measures, we examined the frequency of the OLOLCH’s root cause analyses (RCAs) and rapid cycle improvements (RCIs) as well as the participation of residents in both.
Annotated run and control charts were used to demonstrate data longitudinally and to illustrate the timing of interventions (QI Macros version 2013.05, KnowWare, Inc, Denver, CO). The mean (center line) and upper and lower control limits were calculated and displayed as ±3 SD of the mean. Chart analysis was performed using standard rules to detect special cause variation.26 Additional descriptive statistics and Student’s t tests were performed using P ≤ 0.05 to determine significance (GraphPad Prism version 6.0, GraphPad Software, Inc, La Jolla, CA).
Physician-entered event reports increased from a baseline monthly mean of 3% to a sustained monthly mean of 24% (Fig 2). Control chart center lines and control limits were adjusted for special cause variation after observation of 8 successive points above the center line (June 2014 through January 2015).26 Although multiple interventions were made in rapid sequence, a substantial and persistent increase in reporting was temporally associated with implementation of the super rotation. Our initial success prompted an upward revision of our reporting goal from 6% to 15% 9 months into the project (Fig 2). Single points of special cause variation were noticed in December 2013 (within the baseline data) and January 2015 (within the postimplementation data). These outliers can potentially be explained by the timing of annual institutional training that includes instruction on mechanisms for safety reporting. A decrease in resident-reported events was observed in December 2014 when the inpatient service was not staffed by a super for over half of the month. The total number of OLOLCH Quantros events did not significantly increase after our interventions were initiated in April 2014 with safety rounds (P = .4). Resident-reported events accounted for the majority of physician reporting in all months except for June 2015, when all of the safety events reported by physicians were entered by faculty (Fig 2A). This aberrancy coincided with the absence of a super resident that month. Additionally, OLOLCH has performed 4 times as many RCAs and RCIs subsequent to the initiation of this program (2 in 2013 vs 9 in 2014 to 2015). Residents have participated in 78% of these events, whereas no resident involvement occurred before our interventions.
There is ample literature describing barriers to attending physician reporting of safety events (eg, fear of liability, peer disapproval and lack of perceived benefit).27 However, resident perceptions of event reporting indicate that lack of knowledge about what or how to report and not having the time to report are leading barriers to safety event reporting for physicians in training.19,28 Despite knowledge of faculty and resident barriers to safety event reporting, information is limited on successful strategies to improve physician reporting. Previous studies describe the use of educational strategies or incentives to improve reporting.27–31 We used a process improvement approach to increase physician reporting of safety events. Our change strategy was built on the premise that focusing on key reporting drivers (education, workflow, and culture) at the point of clinical teaching would most effectively change physician behavior around error identification and safety event reporting. The resulting increase in physician reporting has been dramatic and sustained, even leading to a revised goal based on our initial success.
Implementation of the super rotation appears to have been associated with the most substantial increase in physician-entered events. The goals and objectives for the super rotation were explicitly designed to integrate patient safety responsibilities into the teaching team workflow, relay real-time process improvement feedback to fellow residents, and initiate brief, germane discussions within the inpatient team. The value of the super role was underscored in December 2014 when the inpatient service was not staffed by a super for over half of the month: only 8% of all safety events were reported by physicians. Although our hospital was at capacity for most of the winter, physician reporting during months supported by a full-time super remained above average (November 2014: 25%, January 2015: 24%). This difference demonstrated the importance of the super on the inpatient teams, but also revealed a lack of safety culture penetration with regard to nonsuper residents and faculty. With additional exposure to the OLOLCH safety team, involvement in RCIs and RCAs, and their growing command of patient safety epistemology, residents appeared to integrate event reporting as standard clinical practice.
On the heels of increased and sustained resident engagement of safety event reporting, we uncovered preliminary evidence of greater faculty buy-in during June of 2015, a month without a scheduled super. Although the overall number of safety events reported by physicians diminished, all of the physician-reported events were placed by attending physicians. We found this encouraging and believe it suggests an increased sense of accountability in recognizing and reporting safety events on the part of our faculty. The residents are given more routine patient safety and quality improvement education than are our faculty. As such, residents often demonstrate for faculty the importance and logistics of reporting safety events. Such bidirectional teaching and learning between residents and faculty is critical to enhance a safety culture and we believe our faculty development model (QI on the fly) may have contributed to this aim.
Our group’s ultimate goal is to not only optimize the clinical learning environment for residents and faculty, but to also demonstrably improve patient care. Previous studies have shown that physicians are typically responsible for 1% to 2% of reports entered into safety reporting systems.7–9 It is evident that this reporting frequency is too low, but it is less clear what the physician-reporting benchmark should be. Ideally, all near-miss and safety events should be reported so that factors underlying these events can be identified and processes improved to reduce the chances of future events, thereby improving patient safety. We revised our initial aim statement 9 months into the project due to the remarkable increase in physician reporting. Although we have not identified a direct link to measurable improvement in patient outcomes during our project, the number of RCIs and RCAs performed in the OLOLCH has increased in association with our work. It is tempting to speculate that the increase in reporting associated with our project resulted in the increase in formal event response; however, we cannot demonstrate causality.
There are limitations to this work. First, this study was focused on a single residency program in a children’s hospital located within a regional medical center. As such, these results and the use of the super resident may not be easily generalized to other institutions or programs. However, we believe that our overall premise can be adapted broadly: placing an emphasis on patient safety and quality improvement at the context of clinical teaching and care. Secondly, our observed reporting increases may not be wholly attributable to our PDSA cycles, because other initiatives, such as interdisciplinary safety team rounds, an inaugural QI/research day, and resident lectures on patient safety, became more accessible within the hospital during this time. Our data suggest these factors did not increase reporting because they were largely present before our PDSA changes and the overall safety reporting did not appreciably change. Our baseline physician contribution to event reporting was 3%, already above the typical reported 1% to 2% average. This may offer evidence of the previous influence of existing initiatives on our physicians’ reporting behavior. Duplicative submissions detailing the same event accounted for <1 additional report per month and thus were not considered a significant reason for the overall increase in reporting. Although the total number of reported events did not overwhelm the safety team, the time to event resolution was not tracked, serving as an additional limitation of our report. Finally, although we attributed the most significant increase in reporting to the resident super rotation based on temporal trends, we made several RCIs. It is possible that the combination of interventions we introduced had a cumulative effect on physician reporting, possibly through supporting a culture that fosters reporting.
Incorporating patient safety responsibilities into a teaching team’s workflow can lead to a marked increase in physician event reporting. The combination of providing faculty with sustained resources while prompting safety and quality integration into their clinical teaching (QI on the fly) and the explicit accountability of senior residents (super residents) to assimilate reporting into workflow has generated consistent event reporting improvement. Going forward, we plan additional improvement cycles to: (1) spread the super model and QI on the fly program to other clinical settings in our subspecialty and primary care faculty; (2) improve the process for resident involvement in response to event reports, such as RCAs; and (3) assess potential measures of verifying reductions in patient harm as a result of increased physician reporting, strengthening the inferential leap from reporting behavior to patient outcomes. We predict that event reporting improvements that explicitly incorporate workflow redesign will be more sustainable than educational interventions or external incentives alone.
- Accepted July 27, 2016.
- Address correspondence to Alston Dunbar, MD, MBA, Our Lady of the Lake Children’s Hospital, 5000 Hennessy Blvd, Suite 6003, Baton Rouge, LA 70808. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relative to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Kohn LT
- Woods D,
- Thomas E,
- Holl J,
- Altman S,
- Brennan T
- Mitchell I,
- Schuster A,
- Smith K, et al
- Tuttle D,
- Holloway R,
- Baird T,
- Sheehan B,
- Skelton WK
- Institute for Healthcare Improvement
- Langley GJ,
- Moen R,
- Nolan KM, et al
- Provost LP
- Eccles M,
- Grimshaw J,
- Campbell M,
- Ramsay C
- Hatoun J,
- Suen W,
- Liu C, et al
- Taylor JA,
- Brownstein D,
- Christakis DA, et al
- Langley GJ,
- Moen R,
- Nolan KM, et al
- Siriwardena AN,
- Gillam S
- Ogrinc G,
- Davies L,
- Goodman D,
- Bataldan P,
- Provost LP,
- Murray SK
- Copyright © 2017 by the American Academy of Pediatrics