BACKGROUND AND OBJECTIVES: Pediatric medical professionals have an increasing desire for quality improvement (QI) methods that produce sustainable changes in health care delivery. Previous reports have described QI in single settings or single coordinating entities that work with multiple sites. The objectives of this project are (1) to improve care for children with asthma across multiple practice settings and (2) to develop state-level expertise to support QI projects across entities in multiple states.
METHODS: Using a multiwave approach, the Chapter Quality Network of the American Academy of Pediatrics implemented statewide learning collaboratives in several states. For each cycle, a national leadership team coached multiple American Academy of Pediatrics chapter leadership teams, which, in turn, coached individual pediatric practices through 2 nested learning collaboratives. State chapters received data and reporting tools and a curriculum fostering QI learning and support change at the practice level. Practices implemented an asthma assessment tool and registry, analyzed work flows, and implemented self-management tools in plan-do-study-act cycles. Sixteen process and outcome measures, including optimal asthma care, were collected and analyzed by using run charts on a monthly dashboard. Chapter leaders provided feedback on sustainable QI change through surveys and interviews.
RESULTS: Optimal asthma care improved from 42% to 81% across the 4 waves. The percentage of patients rated by physicians as well controlled rose from 59% to 74%.
CONCLUSIONS: Asthma care can be improved by supporting practice change through statewide QI learning collaboratives.
- AAP —
- American Academy of Pediatrics
- BTS —
- Breakthrough Series Learning Collaborative
- CQN —
- Chapter Quality Network
- EMR —
- electronic medical record
- MOC —
- maintenance of certification
- NHLBI —
- National Heart, Lung, and Blood Institute
- QI —
- quality improvement
Primary care physicians in the United States are standing at the crossroads. They face an evolving practice landscape that ties payment and board certification to health care quality and delivery.1,2 Structural challenges in practices include the development of population registries, incorporation of quality improvement (QI) methods, and implementation of electronic medical records (EMRs).3 Given these changes, considerable support will likely be essential in bolstering primary care practices through redesigns that align care delivery with quality metric-driven payment systems.
QI methods are increasingly being used to transform care that improves patient safety, reduces unwarranted variations in care and outcomes, and produces sustainable changes in the health care delivery system.4–8 Previous reports, particularly those concerning pediatric primary care, have been limited to single settings or single coordinating entities working with multiple sites with centralized QI operations, thus limiting the potential for sustainability and diffusion.9–11 The facilitation of statewide QI efforts would ideally result in improved care and outcomes at the population level and foster sustainable QI programs at the state level.12,13
In this article, we describe the experience of a multiwave Chapter Quality Network (CQN) project on improving asthma care across multiple practices and states. The project focused on practice-level implementation of the National Heart, Lung, and Blood Institute’s (NHLBI) 2007 asthma guidelines.14 Although asthma is the most common serious chronic disorder of childhood, pediatric asthma care rendered by primary care physicians may be inadequate.15 We worked with American Academy of Pediatrics (AAP) state chapters to increase their capacity to support multiple pediatric practices in adopting practice-level changes that improved asthma care delivery. The question we sought to answer was the following: can AAP chapters, with support from the national organization, spread practice changes in asthma care across multiple practices through statewide QI learning collaboratives?
This project used a wave sequence approach to implement statewide learning collaboratives in 4 waves.16 In this section, we describe the CQN structure, practice-level interventions, implementation methods, and plan for evaluation across waves. Approval was obtained from the Institutional Review Board of the AAP.
The CQN structure rests on 3 linked frameworks: one providing a methodology for spreading practice changes, another outlining the key elements of chronic care, and a third providing a framework within which practices would test, implement, and adapt changes.
Institute for Healthcare Improvement Breakthrough Series Learning Collaborative Model
In the Breakthrough Series Learning Collaborative (BTS) model, a learning collaborative convenes multidisciplinary teams to improve care for a specific health condition.17 Through multiple in-person learning sessions and defined action periods, best practices and tests of change are spread and incorporated. The CQN featured a modified collaborative model with 4 learning sessions (Table 1).
Chronic Care Model
This framework describes key interventions to improve clinical outcomes of a chronic condition.18,19 Key components include self-management support, delivery system design, clinical decision support, and clinical information systems.
Institute for Healthcare Improvement Model for Improvement
The Institute for Healthcare Improvement Model for Improvement20 addresses 3 questions: (1) What are we trying to accomplish? (2) How will we know that a change is an improvement? (3) What change can we make that will result in an improvement?
The AAP formed a national leadership team consisting of experts in asthma care, QI, and primary care practice systems. The team developed a set of key drivers and associated interventions (Supplemental Information), as well as an implementation guide comprising resources, tools and methods, and a set of measures, with adaptations based on lessons learned during each wave. Data and reporting tools and a curriculum fostering QI learning by chapter leaders were also designed by the national team. These experts coached state chapter leaders throughout the recruitment and implementation phases, and they helped practices assess their performance and develop reliable care processes.21 Two nested BTS collaborative models ran simultaneously, which were the state chapter leadership learning network and a practice collaborative in each state.
For waves 1, 3, and 4, 4 AAP chapters were selected through a competitive application process. The number of chapters selected to participate was based on the capacity of the national QI coaches and resources. Six selection criteria had to be met: (1) identified leadership; (2) existing QI infrastructure; (3) experience conducting QI at the practice level; (4) evidence of partnership with state and private payers; (5) ability to recruit practices; and (6) demonstrated chapter-level communication strategies. Chapters participating in wave 1 were asked to join wave 2, in which the inclusion of a pediatric affordable care organization was tested. All chapters engaged a pediatrician leader, asthma expert, and project manager. The chapters were asked to recruit a minimum of 10 practices from various geographic locations in academic and nonacademic settings.
The chapter leadership learning collaborative was initiated at a face-to-face training session. State chapter leadership teams learned QI methodologies and leadership competencies, and they reviewed the project framework, interventions, and responsibilities for national and state leaders. State chapters organized monthly webinars and 2 in-person learning sessions for participating practices, reviewed state- and practice-level data reports, and provided practice-level coaching. Attendance for learning sessions is listed in Table 2.
Waves 1 to 3 ran for 12 months; wave 4 ran for 9 months because of the reduced number of maintenance of certification (MOC) points awarded by the American Board of Pediatrics for a single project. Practices were asked to incorporate optimal NHLBI asthma care practices, including assessment of control, a stepwise approach to treatment, appropriate use of controller medication, and an updated asthma action plan for self-management.14 The following specific interventions were included:
Asthma encounter form: A standardized asthma encounter form (Supplemental Information) that incorporated the NHBLI guidelines into a decision support tool.
Population registries: Children with asthma were identified with a billing query for asthma codes. In waves 1 and 2, practices created individual registries using Excel or other data management tools. A national asthma registry was developed for waves 3 and 4.
Workflow assessment: Practices were coached to conduct workflow assessments and tests of change to incorporate the encounter form into daily workflow.
Self-management skills: National and state leadership teams developed a robust repository of patient education materials in multiple languages using literacy guidelines. Practices were also trained on the fundamentals of motivational interviewing to help patients set goals.
Working as a QI practice team: Practices were coached on team roles to conduct plan-do-study-act cycles and review data.
All physician participants were eligible to earn MOC part 4 credit and continuing medical education credit. Eighty percent of participating physicians received MOC credit across the 4 waves (Table 2).
Sixteen measures, including outcome and process measures, were routinely collected by using the structured asthma encounter form (Supplemental Information). The measure of focus was optimal asthma care. This measure combined 4 asthma care practices suggested by NHLBI guidelines into a bundled measure. Optimal asthma care required 4 tasks at a visit: (1) physician assessment of asthma control using a standardized instrument; (2) updated asthma action plan; (3) use of the stepwise approach to initiate or adjust therapy; and (4) prescription of a controller medication for patients with persistent asthma. Data were collected at the point of care by using the asthma encounter form and entered by practices into a national database at least monthly. Each participating physician was asked to submit at least 5 encounters per month to ensure reliable implementation of project interventions. To encourage a reliable system of care, participating practices were advised to use the encounter form with each asthma patient at least once a year and more often for complex or severe asthma patients.
Patient-level data for QI reporting were aggregated monthly at the physician and practice levels. Practice-level data were compiled and presented to each practice and state leadership team in a dashboard format. This dashboard was a composite of run charts for each of the 16 measures (Supplemental Information). Measures were expressed as the percentage of asthma visits each month for which the guideline objectives were met. Practices received and reviewed their own performance data; state chapter leaders reviewed practice data individually and in the aggregate to inform ongoing plan-do-study-act cycles.
All members of chapter teams were provided surveys at the completion of each CQN wave to provide feedback on the project. The feedback was used to make adjustments in overall project operations as well as to help each chapter plan future QI initiatives.
Seven hundred and forty-nine pediatricians collected data during 45 431 patient encounters in 180 practices across 9 states during the 4 waves. The participating pediatricians represented ∼7.2% of pediatricians in those 9 states on the basis of the 2016 AAP Member Directory and the assumption that 60% of pediatricians are AAP members. Table 2 outlines the numbers of chapters, practices, physicians, and encounters for each wave; MOC credit awarded; and attendance levels for the in-person learning sessions (2 per wave) and webinars (2 per wave). Table 3 summarizes the practice demographics.
Optimal Asthma Care Bundle Measure
At the beginning of each wave, <50% of practices reported optimal asthma care at each encounter. In waves 1, 2, and 3, this rate improved to >80%. In wave 4, in which data were collected for only 9 months, the rate improved to 72% (Fig 1). The proportion of patients rated by physicians as well controlled rose from 59% to 74% across 4 waves (Fig 2).
A 1-way repeated measures analysis of variance was conducted to determine if there were statistically significant differences in optimal asthma care over the course of the project. The original data set was normally distributed for month 1 across the waves, but not for the final month. To normalize the data, an Arcsine transformation was performed. Subsequently, there were no outliers and the data were normally distributed (Shapiro-Wilk test, P > .05). The project elicited statistically significant changes in optimal asthma care (F1,123 = 63.281, P < .05). The normalized rate improved from 0.54 ± 0.045 in month 1 to 1.002 ± 0.40 in the final month of the project. Post hoc analysis revealed that chapter (F1,9 = 4.965, P < .05) had a significant impact on optimal asthma care improvements, whereas practice size (P > .05) and wave of participation (P > .05) did not.
Additional measures tracked during the project are included in Supplemental Information. Practices across all waves reported relatively high baseline use of the stepwise approach (81%–89%) and a controller medication for patients with persistent asthma (74%–88%). The proportion of patients with a current written asthma action plan rose from the range of 49% to 57% to the range of 75% to 91%. Physicians’ and parents’ ratings of asthma in children as well controlled increased modestly from the range of 58% to 63% to the range of 71% to 74% and from the range of 67% to 73% to the range of 78% to 87%, respectively. Patient receipt of self-management materials rose from the range of 57% to 62% to the range of 62% to 88%, with little change in wave 2. Missed school days, missed parent workdays, and emergency department visits remained relatively consistent through each wave.
Through planned care, QI activities, and learning collaboratives, project participants demonstrated continual improvement in reported optimal asthma care over a 9- to 12-month period. The improvement in care reported by practices was relatively consistent among practice types and sizes across multiple states. Improvement in asthma care using continuous QI and a learning collaborative framework has previously been reported for local and regional entities.9,10 Our findings are notable for achieving improvement in asthma care through training multiple state coordinating entities instead of directly leading the learning collaborative itself. Few other projects have achieved this level of results in pediatric practice improvement in asthma care on such a widespread scale across multiple states.21,22 This project may serve as a model for other statewide and national organizations attempting to achieve improvements in population health through support and training for statewide organizations that, in turn, support individual health care providers and practices. Although none of the waves achieved the preset goal of 90%, all showed substantial improvement and remarkable consistency in the rate of improvement; we therefore regarded the project as an overall success.
The key to improvement on the practice level appears to be system-level improvements. Other improvement activities included practice-based teams, redesign of patient flow, and plan- do-study-act cycles in implementation of consistent use of care protocols. The mechanism underlying the improvement in ratings cannot be clearly elucidated from the data, but existing measures provide some suggestions. We found that practices were already reporting high levels of use of the stepwise NHLBI approach. As the data illustrated, the challenges for practices centered on the reliability of providing asthma action plans and self-management materials. Overall improvement was reported for both measures by the end of each wave, with the sole exception of provision of self-management materials during wave 2. In addition, increasing numbers of physicians and parents rated asthma as well controlled, suggesting relatively consistent improvements in perceived management of the asthma. Further studies may examine more objective measures of control, including spirometry, which, according to participants, changed little during the waves.
A unique aspect of our work was that rather than directly work with individual practices, national leadership developed the frameworks and methodologies and then implemented a QI infrastructure on the state level that worked with individual practices. This approach reached multiple practice settings and children. Practices were able to share experiences and materials and compare results through the learning collaboratives. AAP chapters developed a variety of approaches combining teleconferences, webinars, and on-site visits to facilitate cross-practice learning.
The core national leadership team and project design were critical to the success of the overall project. The national team maintained data and reporting tools for individual practices and state chapters. However, the knowledge and experience gained by individual chapter leaders appeared to lead to subsequent QI projects, suggesting that the time and training invested in participating in the CQN could be sustained at the state level. The relatively consistent performance results over 4 waves suggest that an approach comprising a core national leadership team, invested state leaders, a core intervention package, a dedicated QI coach, a program implementation guide, data support, and 2 nested BTS collaborative models is a system that can be replicated for widespread practice improvement. In addition, all but 1 participating chapter engaged in discussions with state payers and other partners on continuing QI work and payment reform.
The overall success of the CQN asthma project is timely to health care delivery transformation.1 CQN uses concepts from innovative models prompted by the Affordable Care Act to improve asthma care. The project benefits from the use of patient registries, EMRs, and payment reform that enables population management of children with a chronic health condition. Further steps for improving asthma care include continued dissemination of the protocol among other providers and practices. Widespread use of EMRs and patient registries, assignment of dedicated staff, and payment incentives would enable practices to continue spreading the care protocol among additional providers and patients. Long-term population studies would enable the discovery of financial benefits through reduction of potentially preventable health encounters.
This project had multiple limitations. Overall success was limited to participating states and practices that may have been highly motivated. All data were self-reported by practices without external validation or consistent use of a patient registry that could automatically link to EMRs. The measures represent asthma visits, not single patients, and for any given month contain a mixture of patients being treated according to the intervention for the first time and patients who had made several visits since the start of the CQN. The 12-month window for data collection places limitations on any outcome data because for many patients the multiple data points necessary to observe changes in asthma control as a result of CQN interventions may not be available. Practices reported on patients who individual providers selected, and it is possible that other patients did not receive optimal asthma care, inflating the reported results. Our project was unable to link its findings to administratively determined health care outcomes of interest such as the number and cost of emergency department and inpatient visits that are potentially preventable. Finally, the project findings are specific to asthma, which has evidence- and consensus-based guidelines originating at the federal level. The project implementation strategies may not apply to other health care conditions.
Several challenges in this project were addressed in ways that may or may not be replicable. Physicians were self-selected, and patients were selected by participating physicians, which could lead to selection bias at the physician and patient levels. A randomized trial comparing physicians who participated in the QI initiative versus those who did not participate would have minimized the bias potential. Ongoing and future studies should examine results from a population level (for example, all children with asthma or a random selection of children with asthma). Population-level data on the sustainability of the intervention would also enable minimization of the Hawthorne, or observer, effect on the results. Data were collected manually at the practice level, and data analysis and reports were provided externally. Although it made data reporting and review easy for practice and chapter leaders, this strategy may not be available in all settings, and manual data entry poses a challenge to sustainability. Future improvements in EMR and registry capabilities may address this issue. Practices were asked to implement these strategies within their own setting. As a result, practice buy-in was variable and depended heavily on leadership interest and involvement. Chapter leaders reported that phone calls and in-person visits were helpful when providing support to practices, particularly for practices that were not consistently engaged in the monthly reporting or monthly practice webinars.
The experience of the CQN model to date may be considered and adapted by other large-scale health care delivery initiatives that use QI methods to improve health care delivery on a broad, possibly statewide, scale. The existing measure sets, implementation guides, data collection systems, and curriculum may be easily used by chapters depending on current infrastructure and training needs. Future studies may examine short- and long-term outcomes on a population level and test the CQN model in other high-priority medical conditions such as attention-deficit/hyperactivity disorder.
We thank the following individuals: Dr Keith Mandel, MD, CQN Asthma Quality Improvement Advisor; Dr Sandra Miller, MD, Wave 2 Oregon Chapter Leader; Dr Amy Belisle, MD, Wave 2 Maine Chapter Leader; Dr William Long, MD, Wave 3 Ohio Chapter Leader; Dr Christopher Landon, MD, Wave 4 California Chapter 2 Leader; Dr Gilbert Liu, MD, Wave 4 Kentucky Chapter Leader; Dr Dixie Griffin, MD, Wave 4 Georgia Chapter Leader; Dr Christina Schwindt, MD, Wave 4 California Chapter 4 Leader; and Dr Brad Weselmen, MD, Wave 4 Georgia Chapter Leader. We also thank Ms Suzanne Emmer and Ms Allie Stevens, MPH, MS, for editing.
- Accepted April 6, 2017.
- Address correspondence to Judith C. Dolins, MPH, The American Academy of Pediatrics, Community and Chapter Affairs and Quality Improvement, 141 Northwest Point Blvd, Elk Grove Village, IL 60007-1098. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: The Chapter Quality Network Asthma Project was funded by the Merck Childhood Asthma Network, American Board of Pediatrics Foundation, American Academy of Pediatrics Friends of Children Fund, The JPB Foundation, and GlaxoSmithKline. Ms Dolins and Mr Wise received salary support. Ms Powell and Dr Stemmler received consulting fees.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Homer CJ,
- Patel KK
- Powell J,
- DeWalt D,
- Norcross A,
- Parchman M.
- White CM,
- Schoettker PJ,
- Conway PH, et al
- Mangione-Smith R,
- Schonlau M,
- Chan KS, et al
- Vernacchio L,
- Frances ME,
- Epstein DM, et al
- Butts-Dion S,
- Crowe V,
- Birken S,
- Dolins J,
- Lannon C.
- National Asthma Education and Prevention Program, Third Expert Panel on the Diagnosis and Management of Asthma
- Massoud MR,
- Donohue KL,
- McCannon CJ
- Wagner EH,
- Austin BT,
- Davis C,
- Hindmarsh M,
- Schaefer J,
- Bonomi A
- The Improving Chronic Illness Care Program
- Langley G,
- Moen R,
- Nolan K,
- Nolan T,
- Norman C,
- Provost L
- Robert Wood Johnson Foundation
- Copyright © 2017 by the American Academy of Pediatrics