A Quality Improvement Collaborative to Improve the Discharge Process for Hospitalized Children
OBJECTIVE: To assess the impact of a quality improvement collaborative on quality and efficiency of pediatric discharges.
METHODS: This was a multicenter quality improvement collaborative including 11 tertiary-care freestanding children’s hospitals in the United States, conducted between November 1, 2011 and October 31, 2012. Sites selected interventions from a change package developed by an expert panel. Multiple plan–do–study–act cycles were conducted on patient populations selected by each site. Data on discharge-related care failures, family readiness for discharge, and 72-hour and 30-day readmissions were reported monthly by each site. Surveys of each site were also conducted to evaluate the use of various change strategies.
RESULTS: Most sites addressed discharge planning, quality of discharge instructions, and providing postdischarge support by phone. There was a significant decrease in discharge-related care failures, from 34% in the first project quarter to 21% at the end of the collaborative (P < .05). There was also a significant improvement in family perception of readiness for discharge, from 85% of families reporting the highest rating to 91% (P < .05). There was no improvement in unplanned 72-hour (0.7% vs 1.1%, P = .29) and slight worsening of the 30-day readmission rate (4.5% vs 6.3%, P = .05).
CONCLUSIONS: Institutions that participated in the collaborative had lower rates of discharge-related care failures and improved family readiness for discharge. There was no significant improvement in unplanned readmissions. More studies are needed to evaluate which interventions are most effective and to assess feasibility in non–children’s hospital settings.
- CHA —
- Children’s Hospital Association
- EMR —
- electronic medical record
- IHI —
- Institute for Healthcare Improvement
Although discharge from the hospital for many pediatric patients means the child is clinically improving, it also creates potential risk because of the transition of care.1 At a minimum this care may include medications and follow-up appointments, but it may also include home care, wound care, or therapy. The discharge process has historically been fragmented and variable, leading to errors.2–4 In 1 adult study, as many as 49% of patients had ≥1 medication error at discharge, which could increase their likelihood for readmission.5 In other studies, 10% to 20% of patients had an adverse event after discharge, with about half of these events deemed to be preventable.6,7
Most of the work on improving discharge processes to date has focused on the adult population. Examples of these projects include the Better Outcomes for Older Adults Through Safe Transitions Project, sponsored by the Society for Hospital Medicine; Project Re-Engineered Discharge, sponsored by the Agency for Healthcare Research and Quality, National Heart, Lung and Blood Institute, Blue Cross Blue Shield Foundation, and the Patient-Centered Outcomes Research Institute; and the State Action on Avoidable Rehospitalizations initiative of the Commonwealth Fund and the Institute for Healthcare Improvement (IHI). All these projects recommend strategies to improve the discharge process, including scheduling follow-up appointments before discharge, medication plans, written patient discharge instructions, patient education about diagnosis and medications, follow-up telephone calls to the patient, communication to the outpatient primary provider at discharge, and others.8–11 Recently White et al12 improved discharge efficiency in a children’s hospital by creating a common set of discharge goals for 11 different pediatric diseases. Although this intervention did decrease the length of stay, the readmission rate was not changed. To date, the only published pediatric discharge improvement collaborative focused on improving communication to primary care providers after hospital discharge.13
About 20% of older Medicare patients who are hospitalized are readmitted to the hospital within 30 days after discharge.14 Because of the high cost of readmissions, adult hospitals with high readmission rates receive reduced Medicare payments under the Affordable Care Act.15 Reimbursement rate penalties for Medicaid patients, including children, are already being implemented in some states. In an analysis of >550 000 pediatric admissions in 72 hospitals, Berry et al16 found that the 30-day unplanned readmission rate in pediatric patients was 6.5%, which is much lower than in adults. Recent publications have reported that most children who were readmitted had an underlying chronic disease, and only a small percentage of readmissions were found to be preventable.17,18 Interestingly, 1 study suggested that children who had a documented follow-up scheduled with their primary care provider were more likely to be readmitted to the hospital than those who did not.19
Because of the potential for errors and variability in the discharge process, Children’s Hospital Association (CHA) formed the first pediatric improvement collaborative to examine whether shared improvement strategies would affect discharge-related care failures, parent-reported readiness for discharge, and readmission.
The CHA invited its members to participate in a multicenter collaborative project addressing the discharge process for pediatric inpatients. Eleven hospitals participated in the collaborative. One hospital did not submit data on interventions and therefore was excluded from analysis. All hospitals were tertiary-care freestanding children’s hospitals in the United States that were members of the CHA. A specified target population was selected at the discretion of the participating site (Table 1). The participants selected populations by specific disease processes, level of clinical complexity, or specific units in the hospital.
The study was patterned after the standard methods used by the CHA in many of its other collaborative projects.20–24 The model for this improvement process was based on previous work developed by the IHI and has been used successfully in pediatric settings.25–29 A multidisciplinary advisory panel of experts with previous experience in discharge processes was recruited from across the CHA. The panel evaluated the existing literature and adopted tools and change concepts from previous discharge programs.2,3,8–11,30 They also incorporated lessons learned from previous CHA collaboratives, including the “Improving Inpatient Throughput” and “Improving Patient Handoffs” programs.20 This panel developed a change package covering 6 broad areas, which included the following strategies:
Proactive discharge planning throughout the hospitalization.
Arrange postdischarge treatment.
Communicate postdischarge plan to providers.
Communicate postdischarge plan to patients and families.
Sites formed multidisciplinary teams and were required to have an executive-level sponsor. The collaborative held 4 virtual learning sessions and monthly Web conferences. In between the learning sessions were 3 action periods, during which each site performed small tests of change using the plan–do–study–act method. During the learning sessions, training on quality improvement methods was provided by national experts. High performers also shared their successes, and participants were given opportunities to ask questions. Sites also presented their progress and challenges during monthly Web conferences. Teams could communicate with each other and share tools and resources via an electronic mailing list and a shared Web site. Teams were guided through improvement efforts by an experienced improvement coach.
Measures and Data Collection
The primary aim of the study was to reduce discharge-related care failures by 50% in 12 months. Discharge-related care failures were measured by using phone calls to families 2 to 7 days after discharge. Failure was a composite all-or-none measure; if any problem related to discharge occurred, the discharge was counted as a failure. Required components of the measure included understanding of diagnosis, receiving discharge instructions, receiving discharge education, compliance with instructions, receiving necessary equipment, having a plan to follow up pending tests, receiving help with appointments, and not needing a related unplanned visit. A discharge phone call script adapted by the expert panel from Project Re-Engineered Discharge was provided, and each site was permitted to modify the script to meet their local needs and capacity.10
All other measures were optional and selected by the individual sites depending on the change strategies targeted (see Supplemental Information and Supplemental Tables 1–6 for definition of measures). Readiness for discharge and readmission rates were priority measures and were highly recommended although not required. Patient and family readiness for discharge was defined as the percentage of families rating the highest category on the hospital’s standard patient satisfaction survey. Readmission at 72 hours and 30 days was defined as unplanned rehospitalization for the same diagnosis. Baseline data were collected from August through October 2011 if available. If baseline data were not available (eg, outreach calls), the first 3 months of project data were used as baseline. From November 2011 to October 2012, the hospitals participated by using Deming’s plan–do–study–act cycles to perform tests of change, implement improvements, and sustain results. Each site selected changes based on local capabilities and priorities. Standardized reporting of data occurred on a monthly basis via an electronic data repository managed by The CHA and did not include any patient identifiers. Monthly reports also included a narrative section that included information on successes, challenges, and next steps. In addition to collecting project measures, CHA staff scored each site based on improvement activity and performance by using the IHI Assessment Scale for Collaboratives. The scale rates teams between 0.5 and 5.0, with 0.5 defined as being signed up to participate and 5.0 demonstrating major change in all areas, outcome measures at national benchmark levels, and spread under way. (See Supplemental Table 7 for rating scale.)31
Measures were plotted on run charts (Minitab version 17.1, State College, PA), with the first 3 months of data reported used as baseline. Only months where ≥3 sites reported data were included. Run charts were interpreted according to standard probability-based rules for α level P < .05.32,33 Data for both individual hospital and overall hospital were also aggregated to the quarterly level for analysis in SAS version 9.3 (SAS Institute, Inc, Cary, NC). Comparisons between the entire baseline period and postbaseline values for the aggregated hospital data were made with χ2 tests. Within each quarter, first observation carried forward or last observation carried back imputation was conducted for missing data in SAS.
This study was determined to be exempt by the Children’s Hospital Los Angeles Institutional Review Board (CHLA-14-00111).
Elements of the collaborative change package were adopted by each institution at varying levels (Table 2). All sites chose to work on educating families on diagnosis and plans for discharge. Several sites also used discharge checklists, with discharge milestones and barriers. Eight out of 10 sites improved the written discharge instructions given to families. Some of these improvements included designing standardized discharge instructions for certain diagnoses, making instructions more user-friendly, and creating new discharge instruction forms in the electronic medical record. Almost all sites (9 of 10) used postdischarge follow-up phone calls to reinforce discharge instructions. Most sites also reported working on identifying and obtaining discharge medications. Few sites addressed communication with primary care providers.
Aggregate data for all hospitals combined are depicted in monthly run charts. Run charts with individual hospital trends are available online (Supplemental Figures 4–8). Eight hospitals reported rates of discharge-related care failures . Because precollaborative data were not available at most sites, the first quarter of the project was used as baseline data. The run chart demonstrated a shift, with 10 consecutive points below the baseline median line (Supplemental Figure 4). The statistical process control chart (Fig 1B) also confirms this finding, with 9 postintervention points below the baseline mean and the final postintervention point below the lower control limit. The aggregate rate of care failures was overall 34% in the first project quarter; the rate at the end of the collaborative was 21%, or a reduction of 40% (P < .05). Top-performing hospitals were able to achieve even lower care failure rates with the use of varying interventions (Fig 1B).
Only 4 hospitals reported data on family feeling ready for discharge (Fig 2). For these hospitals, there was a statistically significant increase in the percentage of patients who rated the readiness for discharge in the highest category. The precollaborative baseline was 85% of patients giving the highest rating; during the last quarter of the collaborative it was 91% (P < .05). The run chart showed a shift of 6 points above the median line in the last 2 quarters.
Five hospitals reported unplanned readmission rates for the same diagnosis, at 72 hours (Fig 3A) and at 30 days (Fig 3B). Four hospitals reported both rates. There was no improvement in unplanned 72-hour (0.7% vs 1.1%, P = .29) and slight worsening of the 30-day readmission rate (4.5% vs 6.3%, P = .05).
Of the 11 participating sites, 4 achieved an IHI Assessment Scale for Collaboratives score of 5.0 at the end of the collaborative (Hospitals A, B, C, D), indicating outstanding improvement. One site obtained a score of 4.5 (sustainable improvement, Hospital E), and 4 sites achieved a 4.0 (significant improvement, Hospitals F, G, H, I). Two sites were able to test changes but did not demonstrate measurable improvement. Common characteristics of the sites that achieved a score of 5.0 included strong multidisciplinary involvement; close collaboration with electronic medical record (EMR) teams; dedicated staff time for discharge phone calls, discharge education, and discharge rounding; and use of discharge checklists.
Adverse events related to poor hospital discharge planning are well described,34 and to our knowledge this is the first multicenter collaborative to target the hospital discharge process for pediatric inpatients. Because the discharge process is complex, involving multiple clinical microsystems, achieving large-scale change is particularly challenging. Although the collaborative did not meet its target of 50% reduction in care failures, significant progress was made. We found a decrease in discharge care failures and improvement in patient readiness for discharge. However, there was no impact on 72-hour unplanned readmissions and even a slight increase in the 30-day readmission rate.
A wide variety of change strategies were adopted by the participating sites to achieve results. One of the most commonly adopted strategies was proactive discharge planning throughout the hospitalization. Several change ideas were used to accomplish this planning, such as educating the patient and family about diagnosis and plans for discharge, including discharge planning in rounds, establishing and continuously updating anticipated discharge time, and ensuring that financial problems did not impede discharge. Other key change areas were improving communication of postdischarge plans to families and providing postdischarge support via outreach phone calls. Previous studies have shown that postdischarge contacts via home visits or follow-up phone calls were effective in decreasing health care utilization and improving satisfaction with care.35–38 Although most sites made postdischarge phone calls during the collaborative period, not all were able to continue doing so. The standardized phone call script used during the collaborative could take <5 minutes to 20 minutes, depending on the patient. If interpretation was needed, the call could take even longer. Some sites found this script unworkable and shortened it significantly. Follow-up studies must be done to evaluate the cost and benefit of phone calls to support their sustainability. Few sites chose to implement interventions related to communication and coordination with outpatient primary care physicians. Future efforts focused on this strategy may demonstrate more improvements in discharge-related outcomes.
Despite improvements in discharge-related outcome measures, there was no improvement in readmission rates during the collaborative. In fact, we saw a slight increase in 30-day unplanned readmissions. This could result from seasonal variability in readmissions. Also, readmission rates vary by diagnosis, leading to high variability in this measure. For example, 1 site focused on management of patients with sickle cell disease, who have 30-day readmission rates between 10% and 20%, and another site focused on patients with asthma, with much lower readmission rates of <2%.39–41 Also, our method was able to assess only revisits to the same facility.42 Another possibility is that improving throughput and discharge timeliness led to earlier discharge, with the unintended consequence of increased readmission; however, we did not collect data on length of stay. There is also significant variability in the definition of readmissions. We defined readmissions as unplanned readmissions for the same condition; however, even within these parameters, each site used different methods to collect the data. Even unplanned readmission may be unavoidable and therefore an insensitive measure for discharge quality. The 3M Potentially Preventable Readmissions algorithm is a promising tool that can be used in future improvement efforts, but it has not yet been prospectively evaluated and may still overestimate preventability.43 Average unplanned readmission rates were very low in the population studied: <1% for 3 days and 5% for 30 days. This finding adds to recent evidence that readmissions may not be a good indicator of hospital quality in the pediatric setting.44 Readmission rates are not solely an indicator of discharge quality; they are a measure of the entire health system, as well as socioeconomic factors and patient disease.38,45,46 There is also no consensus on the optimal readmission interval. The Centers for Medicare and Medicaid Services uses 30 days for adult readmissions measures; however, some studies have used 7, 14, or 15 days. Future studies should establish standardized frameworks and measures for evaluating discharge care quality.47,48
The limitations of this collaborative are consistent with other initiatives to improve care across multiple sites.49,50 First, the participating sites were all tertiary-care freestanding children’s hospitals, so the results may not be generalizable to community hospitals or pediatric care provided in general hospitals. Second, we were not able to measure the impact of specific change strategies, because each site chose different targets and implemented a bundle of several strategies simultaneously. Randomization of the interventions across sites would have increased our ability to draw conclusions about the effectiveness of individual interventions but would not have allowed sites to choose the strategies most relevant to their populations and feasible in their local environments. Third, for most measures sites did not have baseline data before implementing changes. In addition, charts had only 11 to 15 data points, with the first 3 points serving as baseline, leaving only 8 to 12 postintervention points. Therefore, we had insufficient points to accurately calculate control limits. Also, because prestudy baseline data were not available for most measures, it is possible that the teams may have made early improvements that were not reflected in the data. This discrepancy is likely to underestimate the true effect of the project. Nearly every site had difficulty obtaining data, and some sites were ultimately not able to submit data on some of the measures. Hospitals need better data systems and analytic resources to more effectively plan and monitor progress of quality improvement work. Finally, each site used different patient populations and different tools to collect data, making the data heterogeneous and difficult to compare.
Participating sites reported several benefits of the collaborative model that were consistent with previous studies.51,52 Teams enjoyed the opportunity to learn from national experts, share challenges and successes, learn and adapt from different settings and patient populations, and share tools such as checklists and call scripts. The collaborative approach also helped sites develop urgency for change at the institutional level and fostered friendly competition and accountability. Teams were also able to leverage collaborative participation to secure financial resources and staff time. Several innovations were also developed and tested during the collaborative period and made available to others. Some examples include sickle cell action plans, seizure actions plans, a “discharge lounge,” whiteboards in patient rooms with home schedules, and peer mentoring programs. Although teams cited difficulties in making timely modifications in the EMR, many sites shared the same EMR platform and were able to exchange technical assistance and screen shots of changes made such as automated discharge readiness reports, conditional discharge order sets, and standardized discharge instructions.
This study shows the potential benefit of the collaborative approach to improve quality of inpatient discharges by using an intervention bundle implemented in pediatric hospital settings. The spread of such interventions has the potential to improve care transition outcomes for all hospitalized children.
Expert panel members: Lori Armstrong, MSN, RN, NEA-BC; Mary Daymont, RN, MSN, CCM, CPUR; Pamela Kiessling, RN, MSN; Cheryl Missildine, RN, MSN, NEA-BC; Karen Tucker, MSN, RN. Data analysis: Cary Thurm, PhD, Children’s Hospital Association.
- Accepted March 14, 2016.
- Address correspondence to Susan Wu, MD, Division of Hospital Medicine, Department of Pediatrics, Children’s Hospital Los Angeles, 4650 Sunset Blvd, Los Angeles, CA 90027. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Funding for the collaborative was provided by the Children’s Hospital Association and participating member hospitals.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Vira T,
- Colquhoun M,
- Etchells E
- Forster AJ,
- Clark HD,
- Menard A, et al
- Williams MV,
- Budnitz T,
- Coleman EA, et al
- ↵Re-Engineered Discharge (RED) Toolkit. 2014. Agency for Healthcare Research & Quality (AHRQ). Available at: www.ahrq.gov/professionals/systems/hospital/red/toolkit/index.html. Accessed September 10, 2014
- Boutwell A,
- Jencks S,
- Nielsen GA,
- Rutherford P
- White CM,
- Statile AM,
- White DL, et al
- Shen MW,
- Hershey D,
- Bergert L,
- Mallory L,
- Fisher ES,
- Cooperberg D
- Centers for Medicare & Medicaid Services
- Bundy DG,
- Gaur AH,
- Billett AL,
- He B,
- Colantuoni EA,
- Miller MR; Children’s Hospital Association Hematology/Oncology CLABSI Collaborative
- Institute for Healthcare Improvement
- Kilo CM
- Billett AL,
- Colletti RB,
- Mandel KE, et al
- Boutwell A,
- Hwu S
- Institute for Healthcare Improvement
- Perla RJ,
- Provost LP,
- Murray SK
- Provost LP,
- Murray SK
- Bardach NS,
- Vittinghoff E,
- Asteria-Peñaloza R, et al
- Press MJ,
- Scanlon DP,
- Ryan AM, et al
- Auger KA,
- Simon TD,
- Cooperberg D, et al
- Øvretveit J,
- Bate P,
- Cleary P, et al
- Leape LL,
- Rogers G,
- Hanna D, et al
- Copyright © 2016 by the American Academy of Pediatrics