Improved Outcomes in a Quality Improvement Collaborative for Pediatric Inflammatory Bowel Disease
OBJECTIVES: Unintended variation in the care of patients with Crohn disease (CD) and ulcerative colitis (UC) may prevent achievement of optimal outcomes. We sought to improve chronic care delivery and outcomes for children with inflammatory bowel disease by using network-based quality improvement methods.
METHODS: By using a modified Breakthrough Series collaborative structure, 6 ImproveCareNow Network care centers tested changes in chronic illness care and collected data monthly. We used an interrupted time series design to evaluate the impact of these changes.
RESULTS: Data were available for 843 children with CD and 345 with UC. Changes in care delivery were associated with an increase in the proportion of visits with complete disease classification, measurement of thiopurine methyltransferase (TPMT) before initiation of thiopurines, and patients receiving an initial thiopurine dose appropriate to their TPMT status. These were significant in both populations for all process variables (P < .01) except for measurement of TPMT in CD patients (P = .12). There were significant increases in the proportion of CD (55%–68%) and UC (61%–72%) patients with inactive disease. There was also a significant increase in the proportion of CD patients not taking prednisone (86%–90%). Participating centers varied in the success of achieving these changes.
CONCLUSIONS: Improvements in the outcomes of patients with CD and UC were associated with improvements in the process of chronic illness care. Variation in the success of implementing changes suggests the importance of overcoming organizational factors related to quality improvement success.
- CD —
- Crohn disease
- IBD —
- inflammatory bowel disease
- PCDAI —
- Pediatric Crohn’s Disease Activity Index
- PGA —
- physician global assessment
- QI —
- quality improvement
- SPC —
- statistical process control
- sPCDAI —
- Short Pediatric Crohn’s Disease Activity Index
- TPMT —
- thiopurine methyltransferase
- UC —
- ulcerative colitis
As many as 1.4 million Americans suffer from inflammatory bowel disease (IBD), Crohn disease (CD), and ulcerative colitis (UC).1,2 Childhood IBD is particularly aggressive, and children experience significant psychosocial impact.3 Surgical intervention is frequently required.4 Despite therapeutic advances in the treatment of pediatric IBD, including widespread use of immunomodulators5–7 and anti–tumor necrosis factor α agents,8 there has been limited improvement in outcomes over the last several decades.9
There is widespread variation in the management of IBD due to a lack of consensus on best management practices and inadequate care delivery systems.10 This variation in delivery of care includes diagnostic and nutritional interventions, suboptimal dosing of medications, prolonged use of corticosteroids, failure to use steroid-sparing agents, inadequate attention to metabolic bone disease, and inadequate screening for colorectal cancer.11,12
The creation of networks of care centers has been an important avenue to accelerate research in pediatrics.13,14 Few centers have enough patients to determine if changes in care delivery are making a difference.15 Networks are increasingly being used as a means to enable multiple clinical centers to work together to apply quality improvement (QI) strategies to improve care and outcomes.14,16–20
Reducing unwanted variation and improving care is difficult without a system for creating new approaches to care, testing them, and translating them into the actual care of patients. Here, we report the effects of QI support by the ImproveCareNow Network (https://improvecarenow.org/) on the process of care delivery and the outcomes of IBD care.
We used an interrupted time series design, in which measurements are repeated over multiple time points, to assess the impact of changes aimed at improving the system of chronic illness care by pediatric gastroenterology centers participating in the ImproveCareNow Network. All participating centers and the data-coordinating center either received Institutional Review Board approval or were classified as exempt from review.
A detailed description and history of the ImproveCareNow Network has been previously published.20 In brief, participating centers received training and coaching in the Model for Improvement,21 changes consistent with the Chronic Illness Care Model,22–26 high reliability principles,27 and team building and were encouraged to audit and self-report on performance.
Participating Practice Centers and Patients
This report is based on an analysis of centers that joined the network in 2007. Participating centers were asked to contribute to the costs of creating the technical infrastructure (QI and data sharing) for the network. We included data from practice centers meeting the following criteria: (1) no extensive QI experience before formation of the network and (2) enrollment of at least 75% of their IBD patients. We included data from patients if they had at least 2 recorded visits, 1 of which was at least 90 days after diagnosis. Data from patients were included until a patient was no longer followed by the practice, withdrew consent, and, in the case of UC, underwent colectomy.
Interventions: QI and Chronic Care
As described previously,20 the network was designed between 2004 and 2006, measures and targets identified,28 and evidence-based changes in care delivery selected. We used a modified Breakthrough Series collaborative operational structure.29,30 Centers received monthly reports summarizing their performance and that of the entire network.
We used the Chronic Illness Care Model22–26 as the conceptual framework to develop changes in care delivery. Integrating evidence and consensus opinion, the network initially developed a set of recommendations to standardize diagnosis, classify disease severity, and evaluate nutritional and growth status. Teams developed and shared tools and information about how to simplify the process of implementing these changes and documenting their performance. As these care processes improved, the network developed standardized recommendations for initiating thiopurine treatment and managing nutrition and growth. Performance goals were expanded to include disease remission, use of corticosteroids, and nutritional and growth status. Additional interventions effective in improving chronic care22,23,31 were selected. Centers developed additional tools to support them, including: a population tracking and management tool; a Model IBD Care Guideline32 emphasizing reduced use of prednisone and improved use of immunomodulators and biologic agents; previsit planning templates to ensure appropriate medication dosing, nutrition and growth classification, and laboratory monitoring; and flow diagrams to illustrate the use of protocols and auditing. To promote development of high reliability processes, data elements for disease classification were reported as an all-or-nothing bundle.
Data Collection and Measures
Centers sought to enroll all of their patients with IBD and to collect data from all visits. Data on patient characteristics, disease status, and care provided were collected during each encounter by using structured clinical encounter forms. Data were entered into an Internet-based database hosted by Clinipace Worldwide (Chapel Hill, NC).
Consistent with the definition of a system,33 process measures of several dimensions of the chronic care model were selected to reflect the reliability of important interrelated care processes that, taken together, could be associated with improved outcomes, including (1) completion of a standardized assessment bundle, defined as the assessment and documentation at each visit of all of the following: disease severity, disease phenotype, extent of disease, plotting height, weight, and BMI on a growth chart, and assessment of nutritional and growth status; (2) measurement of thiopurine methyltransferase (TPMT) before treatment with a thiopurine; and (3) dose of thiopurine prescribed (for patients with an intermediate TPMT activity, azathioprine dose between 1.0 and 1.5 mg/kg per day or 6-mercaptopurine dose between 0.5 and 0.75 mg/kg per day; for patients with a normal to high TPMT activity, azathioprine dose between 2.0 and 3.0 mg/kg per day or 6-mercaptopurine dose between 1.0 and 1.5 mg/kg per day). Process measures were summarized as the proportion of visits each month in which the process was completed.
The primary patient outcomes were: (1) remission, measured as the proportion of patients whose disease was classified as quiescent by physician global assessment (PGA)20; (2) nutritional status, measured as BMI z-score; (3) growth, defined as the proportion of patients with a height velocity z-score ≥ −1; and (4) steroid-free treatment, measured as the proportion of patients not taking prednisone after 112 days (16 weeks) past diagnosis. Height velocity z-scores were calculated only for boys ≤17 years and girls ≤14 years to exclude patients who no longer had significant growth potential. Calculation of height velocity was also limited to patients who had been diagnosed for at least 112 days because any intervention would not be expected to affect height velocity for several months. In addition, patients were not included in this measurement until they had at least 2 postenrollment height measurements. Prednisone usage was only measured in patients who had been diagnosed for at least 112 days because the use of prednisone may be appropriate early in therapy, and the emphasis was on detecting prolonged or repeated courses of corticosteroids. The Short Pediatric Crohn’s Disease Activity Index (sPCDAI),34 for patients with CD, and the Pediatric Ulcerative Colitis Activity Index,35 for patients with UC, were measured as secondary outcome measures for the subset of patients with sufficient data to generate a score. For each outcome measure, the response for an individual patient at a visit was carried forward for each subsequent month until a new visit with a response occurred. Therefore, summary measures at each month reflected patient status as of the last visit.
To provide more reliable and stable baseline estimates of the outcome measures, data from a center were not included in analyses until the center had been in the network for at least 6 months and had enrolled at least 10 patients. For process measures, centers were included once they had been participating for 3 months. Patients who were no longer receiving care at the center were inactivated, and no visit-based data from the patient were used past the date of inactivation. For outcome measures, patient data were carried forward for 90 days past the date of inactivation. Data for UC patients who had a colectomy were not carried forward, nor were PCDAI scores34 for CD patients who underwent an ostomy. The PGA measure20 for CD patients with an ostomy was carried forward.
Statistical process control (SPC) methods were used to determine if there were changes in process and outcome measures across the participating centers.36 For analyses of remission rates and the percent of patients not taking prednisone, centerline (mean) and control limits (±3 SD) were calculated and displayed for the period from July 2007 through March 2010. The upper and lower control limits reflected the inherent variation in the data. Data values were added monthly, and the centerline and control limits were updated with each new monthly data point until 12 months had been plotted.37 The centerline was then held constant and not updated using new monthly data points. Control charts were then monitored for evidence of significant change by using standard SPC rules, including the presence of 1 point outside the upper or lower control limits. If it was predicted that a sustainable change had taken place, a new centerline was estimated, starting with the data point that was outside the previous limits.
To evaluate whether there was a significant time trend in the process and outcome measures, generalized linear mixed-effect models were used to account for clustering of data by center, with link functions dependent on variable type (eg, logit link for binary data). Models were run with terms for center, time, and center-by-time interaction. If the interaction term was significant, indicating that the change over time was different across centers, the time trend was estimated within each center and tested for statistical significance. Similar analyses were conducted to evaluate variation in patient characteristics at enrollment over time. All statistical analyses were performed by using SAS version 9.2 (SAS Institute, Cary, NC).
Eight practice centers enrolled in January 2007, and 2 additional centers joined in July 2007. One center was excluded because it had extensive experience with QI and previous improvements in measures of care delivery. Three additional centers were excluded; 1 dropped out of the network after 1 year, and 2 were unable to enroll at least 75% of their patients with IBD because they lacked the resources to participate fully, resulting in the inclusion of 6 centers.
The demographic and disease characteristics at enrollment for the 1188 children (843 with CD and 345 with UC) who composed the study population are summarized in Table 1. One hundred and twenty children with CD and 74 children with UC had been excluded because they did not meet inclusion criteria (>1 visit recorded and at least 1 visit at least 90 days after diagnosis).
Improvements in Processes of Care
As shown in Fig 1, by March 2010, changes in care delivery were associated with an increase in the proportion of visits with complete disease classification (CD: 55%–93%; UC: 62%–89%), measurement of TPMT before initiation of thiopurine (CD: 60%–80%; UC: 50%–73%), and patients receiving a starting dose of thiopurine appropriate to their TPMT status (CD: 48%–56%; UC: 23%–64%). For each of the above variables, there was a significant positive trend seen in both populations (P < .01), with the exception of measurement of TPMT in CD patients (P = .12). Although the rate of improvement for complete disease classification varied by center, all centers demonstrated significant improvement for CD patients. For UC patients, performance significantly increased at 3 centers and remained stable for 3 centers.
The proportion of CD patients with inactive disease increased over time. By October 2008, there was evidence of a significant improvement based on SPC criteria of 1 point outside control limits. On the basis of this finding and the concurrent improvements in the measures of the process of care, the centerline, with the associated control limits, was adjusted from 55% for the period July 2007 to September 2008 to 68% for the period October 2008 to March 2010. Figure 2A is a control chart annotated to show the association between changes undertaken in the network and changes in the proportion of CD patients with inactive disease.
The proportion of UC patients with inactive disease also increased (Fig 2B). By June 2009, there was evidence of a significant improvement based on SPC criteria of 1 point outside control limits. The centerline, with the associated control limits, was adjusted from 61% for the period July 2007 to May 2009 to 72% for the period June 2009 to March 2010. Figure 2 also shows the variation in improvement across centers. Centers with the lowest baseline rates showed larger rates of improvement. As shown in Fig 3, the changes observed in CD and UC disease activity were primarily associated with a decrease in the percentage of patients with mild disease. The proportion of CD patients not taking prednisone also showed evidence of significant improvement (Fig 4). The centerline was adjusted from 86% for the period July 2007 to October 2009 to 90% for the period November 2009 to March 2010. The proportion of UC patients not taking prednisone remained stable over time at 85%.
sPCDAI scores were analyzed as a secondary outcome. The proportion of patients in remission at the time of their last visit increased, as measured by sPCDAI, and demonstrated a significant positive trend over time increasing from 65% to 69% (P < .0001). Pediatric Ulcerative Colitis Activity Index scores also demonstrated a significant positive trend over time from 57% to 62% (P = .04).
BMI z-scores did not change and remained near the average value for a patient’s gender and age for CD patients (P = .51). For UC patients, BMI z-scores decreased significantly over time (0.40–0.29, P = .01) but remained above 0. The proportion of patients with normal height velocity showed no change over time for either disease group.
We examined whether the changes we observed could be attributed to other causes. To assess the impact of incomplete data in the registry, we selected 20 patients at random from each center. Overall, 87% of visits for these patients were entered into the registry. To determine if patients with active disease were more likely to be enrolled during the early part of the collaborative, thereby underestimating the proportion of patients in remission in centers’ populations, we calculated the remission rate of patients at enrollment by 3-month periods and found it to be stable over time (Fig 5). Similarly, we also examined other patient characteristics by 3-month intervals to determine if the population being enrolled was stable. There was no change in gender proportions or BMI at the time of enrollment (P > .10), except for a minor increase in BMI among UC patients (P = .03). As anticipated, the time from diagnosis to enrollment decreased over the course of the collaborative (P < .0001 for both CD and UC patients) because centers initially had predominantly previously diagnosed patients available for enrollment and later had predominantly newly diagnosed patients available for enrollment. Finally, to determine if disease activity improved as a function of time from diagnosis, we plotted mean duration of enrollment as a function of time in the collaborative. We found a very small association, indicating that time from diagnosis is unlikely to be a significant confounder.
After the creation of a collaborative improvement network, standardization of care, and the application of evidenced-based changes to improve chronic illness care, we observed improvements in specific care processes and an increase in the proportion of CD and UC patients in remission, as well as an increase in the percentage of CD patients not taking corticosteroids.
This project extends the findings of other investigators that redesigning specific elements of chronic care delivery leads to improvements in the quality and outcomes of care. For example, in 32 of 39 studies in a systematic review of diabetes care programs, interventions based on components of the Chronic Care Model improved at least 1 process or outcome measure for diabetic patients,23 although there was often a delay in seeing improvements in clinical outcomes.38 Other studies and reviews suggest that implementing more changes results in more improvement.39–42 Most of this work has taken place in adult primary care practices. Here, we demonstrated its relevance to pediatric subspecialty care.
There is mounting evidence that the use of collaborative improvement networks can improve patients’ outcomes. To date, much of the evidence comes from hospital-based networks.16–19 Primary care networks have demonstrated modest improvements in outcomes for patients with chronic illness.43 Our results provide an estimate of the magnitude of improvements in outcomes that may take place when there is particular emphasis on more consistent and reliable application of existing therapies.18,19,44
Our study has several potential limitations. First, the PGA of disease activity is a relatively subjective measure, which could result in misclassification error. It is unlikely that physicians systematically underestimated illness severity because accurate disease assessment was essential for efficient population management processes. Thus, any misclassification was likely stable over time. Second, we cannot determine if changes in some process measures simply reflected improved documentation. However, accurate documentation is essential to improve the chronic illness care processes. For example, without accurate information about drug doses, previsit planning and population management are difficult to accomplish. Third, improvements in outcome occurring over time could have taken place independent of changes in care delivery as part of the network. No external comparator group was available to help with this determination. However, not all centers showed improvement, and the improvement we observed took place over a relatively short period of time during which no new therapies were introduced into routine clinical practice. Finally, the processes we measured may not be directly responsible for the observed improvement in remission. Rather, as hypothesized by the Chronic Illness Care Model,22–26 these measures are tracers that indicate improvements in the overall systems of care delivery.
As anticipated, improvements did not occur equally across measures or across centers. Three centers were unable to participate fully in the network, demonstrating that a significant investment of time and resources is required simply to participate. Of those centers that were included, some began with high performance on specific measures and maintained that level of care, while others started at lower levels of process reliability and either improved or remained relatively stable. Such variation in the success of the center-based QI efforts likely reflects differences in the degree of implementation of the interventions or the impact of other contextual factors, such as focused leadership and availability of resources to support QI (eg, training, staff devoted to the effort, and allocated physician time).33,45–59 As the network has matured, we have increased the availability of focused coaching to centers which are not demonstrating improved performance to address issues of training, implementation, and leadership. A better understanding of these factors will allow more effective application of QI methods and, thereby, potentially even greater improvements in outcomes.
Similarly, improvements did not occur equally across measures. While improvements were noted in many process and outcome measures, there was not consistent improvement in anthropometrics. This lack of improvement could be attributed to several potential explanations, including (1) a lag period between improvement in remission rates and subsequently growth and nutrition parameters, (2) the fact that improvement occurred most prominently in those with mild disease and, therefore, with fewer growth issues at baseline, and (3) baseline z-scores were near or above 0, leaving little room for improvement.
These results suggest that collaborative QI methods focused on improving chronic illness care can lead to improved process and outcome measures in children with IBD, and likely in other chronic diseases as well. These improvements were likely the result of changes in the care delivery systems rather than a single specific intervention. We believe that improvement will be more consistent across centers as the package of care delivery system changes is more reliably and comprehensively implemented. Further study is needed to determine which combination of interventions is most important to improve the outcomes of these patients. Our data suggest that there are opportunities to substantially improve outcomes in pediatric IBD by using therapeutic interventions that are already available.
Currently, participating ImproveCareNow centers and their physician leaders are: Advocate Lutheran General Children’s Hospital, James Berman; Arkansas Children’s Hospital, George Fuchs; Arnold Palmer Children’s Hospital, Devendra Mehta/Jeffrey Bornstein; Children’s Healthcare of Atlanta/Emory Children’s Center, Bess T. Schoen; Children’s Hospital Boston, Leslie Higuchi; Children’s Hospital of the King’s Daughters, Marc Tsou; Children’s Hospital of Philadelphia, Andrew Grossman/Robert Baldassano; Children’s Mercy Hospital, William San Pablo; Cincinnati Children’s Hospital Medical Center, Charles Samson/Ted Denson; Great Ormond Street Hospital, Mamoun Elawad; Inova Pediatric Digestive Disease Center, Ian Leibowitz/Lynn Duffy; Levine Children’s Hospital, Victor Piñeiro; Maine Medical Center, Mark Integlia; Massachusetts General Children’s Hospital, George Russell/Esther Israel; Mayo Clinic, Jeanne Tung; Nationwide Children’s Hospital, Wallace Crandall/Brendan Boyle; Nemours Children’s Clinic, Fernando del Rosario/David Milov; Northwest Pediatric Gastroenterology, LLC, Jacqueline Fridge; Oakland Children’s Hospital, Sabina Ali; Oklahoma University Medical Center, John Grunow; Pediatric Gastroenterology and Nutrition Associates, Howard Baron; Texas Children’s Hospital, Seema Mehta/George Ferry; The Children’s Hospital, Denver, Deborah Neigut/Edward Hoffenberg; University of Michigan–Ann Arbor, Jeremy Adler; University of Minnesota, Boris Sudel; University of North Carolina at Chapel Hill, Michael Kappelman/Sandra Kim; UT Southwestern Medical Center, Ashish Patel; and Vermont Children’s Hospital, Richard Colletti.
We thank the staff of ImproveCareNow (Melissa Disorda) and the James M. Anderson Center for Health Systems Excellence (Sarah Myers, Debbie Ray, Divvie Powel, Harry Atherton, Diana McClendon, Julia Williams, Cole Jackson, and Pamela J. Schoettker), Lloyd Provost (Associates in Process Improvement), and Paul Miles (American Board of Pediatrics).
- Accepted October 31, 2011.
- Address correspondence to Wallace V. Crandall, MD, Division of Pediatric Gastroenterology, Hepatology and Nutrition, Nationwide Children’s Hospital, 700 Children’s Dr, Columbus, OH 43205. E-mail:
Drs Crandall, Colletti, Kappelman, Margolis, and King contributed to the conception and design of the manuscript; Drs Crandall, Colletti, Kappelman, Duffy, Grunow, Kim, Leibowitz, and Schoen contributed to the acquisition of the data; Drs Crandall, Colletti, Kappelman, Margolis, King, and Mr Pratt contributed to the analysis and interpretation of the data and drafted the manuscript; Drs Crandall, Colletti, Kappelman, Margolis, King, Duffy, Grunow, Kim, Leibowitz, Schoen and Mr Pratt contributed to the critical revision of the manuscript for important intellectual content; and Drs Margolis, King and Mr Pratt contributed statistical expertise.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Supported by funds from the participating centers of ImproveCareNow; a grant from the American Board of Pediatrics Foundation; a grant from the Agency for Healthcare Research and Quality (U18HS016957); and a Transformative RO1 (R01DK085719) from the National Institutes of Health. This research was also supported, in part, by grant KL2 RR025746 from the National Center for Research Resources. Funded by the National Institutes of Health (NIH).
- ↵Centers for Disease Control and Prevention. Chronic disease prevention and health promotion. Available at: www.cdc.gov/chronicdisease/overview/index.htm. Accessed January 19, 2010
- ↵Centers for Disease Control and Prevention. Inflammatory bowel disease (IBD). Available at: www.cdc.gov/nccdphp/dach/ibd.htm. Accessed March 23, 2010
- Mackner LM,
- Sisson DP,
- Crandall WV
- Markowitz J
- Horbar JD,
- Rogowski J,
- Plsek PE,
- et al.,
- NIC/Q Project Investigators of the Vermont Oxford Network
- ↵Simone JV, Lyons J. Superior cancer survival in children compared to adults: a superior system of cancer care? Available at: www.simoneconsulting.com/PDF/children_survival.pdf. Accessed January 18, 2010
- ↵NIH Office of Rare Diseases. Rare diseases and related terms. Available at: http://rarediseases.info.nih.gov/RareDiseaseList.aspx. Accessed March 27, 2010
- ↵Donovan EF, Lannon C, Bailit J, et al. A statewide initiative to reduce inappropriate scheduled births at 36(0/7)-38(6/7) weeks' gestation. Am J Obstet Gynecol. 2010;202(3):243.e1–e8
- Kaplan HC,
- Lannon CM,
- Walsh MC,
- Donovan EF,
- Ohio Perinatal Quality Collaborative
- Miller MR,
- Griswold M,
- Harris JM II,
- et al
- Langley GJ,
- Nolan KM,
- Nolan TW,
- Norman CL,
- Provost LP
- Wagner EH,
- Austin BT,
- Davis C,
- Hindmarsh M,
- Schaefer J,
- Bonomi A
- Crandall WV,
- Boyle BM,
- Colletti RB,
- Margolis PA,
- Kappelman MD
- ↵Institute for Healthcare Improvement. The Breakthrough Series: IHI's collaborative model for achieving breakthrough improvement. Available at: www.ihi.org/NR/rdonlyres/BCA88D8F-35EE-4251-BB93-E2252619A06D/0/BreakthroughSeriesWhitePaper2003.pdf. Accessed April 14, 2011
- ↵ImproveCareNow. About: helping children with Crohn's and colitis feel better now. Available at: http://improvecarenow.org/about. Accessed October 27, 2010
- Deming WE
- Benneyan JC,
- Lloyd RC,
- Plsek PE
- Shadish WR,
- Cook TD,
- Campbell DT
- Coleman K,
- Austin BT,
- Brach C,
- Wagner EH
- Shojania KG, Ranji SR, Shaw LK, et al. Closing the quality gap: a critical analysis of quality improvement strategies. Available at: www.ncbi.nlm.nih.gov/books/NBK43938/pdf/TOC.pdf. Accessed April 14, 2011
- Bradley EH,
- Curry LA,
- Webster TR,
- et al
- Bradley EH,
- Holmboe ES,
- Mattera JA,
- Roumanis SA,
- Radford MJ,
- Krumholz HM
- Juran JM,
- Gryna FM
- Øvretveit J,
- Bate P,
- Cleary P,
- et al
- Parker VA,
- Wubbenhorst WH,
- Young GJ,
- Desai KR,
- Charns MP
- Weiner BJ,
- Alexander JA,
- Shortell SM
- Copyright © 2012 by the American Academy of Pediatrics