 |
|
 |
|
 |
 |
|
 |
 |
 |
 |
 |
 |
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ARTICLE |
Department of Pediatrics, University of Vermont College of Medicine, Vermont Department of Health, Burlington, Vermont
 |
ABSTRACT |
|---|
 TOP ABSTRACT METHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
METHODS. All pediatric practices in Vermont (n = 35) were invited to participate in a preventive services quality improvement initiative. Ninety-one percent agreed. Participating practices serve >80% of all Vermont children who are younger than 5 years. The main outcome measured was change in 9 preventive services areas: (1) immunizations up to date; (2) anemia screening; (3) tuberculosis risk assessment and indicated screening; (4) lead screening; (5) infant sleep position counseling; (6) environmental tobacco smoke–exposure risk assessment; (7) blood pressure screening; (8) vision screening; and (9) dental risk assessment.
RESULTS. All practices demonstrated improvement in 1 or more preventive services areas. The mean number of areas chosen was 5 (range: 1–9). Practices that selected a specific preventive service area as a quality improvement goal were more likely to demonstrate improvement in that area than practices that did not choose to focus on that preventive services area.
CONCLUSIONS. The work in this project has provided the evidence for an effective statewide pediatric quality improvement outreach program to improve preventive services for children who are younger than 5 years. Practices' decision to focus on a specific preventive service area as a quality improvement goal seems necessary for improvement in that area. This approach may be effective in other states or regions.
Key Words: preventive services • well-child care • quality improvement • immunizations • screening • maternal and child health
Abbreviations: QI—quality improvement • VPSI—Vermont Preventive Services Initiative • VCHIP—Vermont Child Health Improvement Program • VDH—Vermont Department of Health • AAP-VT—American Academy of Pediatrics, Vermont chapter • NICHQ—National Initiative for Children's Healthcare Quality • LS—learning session • PDSA—plan-do-study-act
Preventive services that are delivered in primary care health supervision visits are critical to protecting the health of children who are younger than 5 years. The majority of immunizations, the single most effective public health measure after the provision of clean water,1 now are delivered in primary care settings.2,3 The efficacy/importance of other preventive services, including risk assessment and screening for preventable conditions such as iron deficiency, lead toxicity, tuberculosis, amblyopia, and patient education and counseling regarding infant sleep position and injury prevention, are supported by many levels of evidence.4,5
Despite the effectiveness of preventive care, there is considerable room for improvement in the primary care delivery of these services. Variability and deficiencies in the provision of pediatric preventive care are well documented, both for individual elements of care (eg, screening, immunizations)6,7 and for the full spectrum of care.8,9
In the past 15 years, the challenge of changing provider behavior to improve health services has been met with quality improvement (QI) initiatives. Studies on QI initiatives that are designed to change provider behavior in varying settings have had somewhat mixed results.10 In the 1990s, investigators who were based at the University of North Carolina developed and implemented a preventive services QI strategy that was designed specifically for use in pediatric primary care. A pre–post evaluation of this approach in a small number of North Carolina pediatric practices showed clear improvements in the provision of preventive services for children.11 Subsequently, a large, randomized, controlled trial of this strategy has confirmed the efficacy of this approach.12
The efficacy of QI to improve pediatric preventive services, therefore, has been demonstrated in the intensively resourced environment of a randomized trial. However, evidence for large-scale effectiveness of the QI approach to improving pediatric preventive services on a population basis is lacking. Vermont provides an excellent setting to test the effectiveness of a broad QI intervention because the high insurance coverage of children who are on Medicaid (children covered up to 300% of poverty) has already ensured access to care, and the cohesive pediatric provider community has a long history of collaborations with the public health sector.
To address this issue, we tested the effectiveness of a statewide pediatric QI outreach program in improving preventive services for children who are younger than 5 years by tailoring the approach used in the North Carolina studies to an entire state. We hypothesized that a statewide QI outreach program would achieve measurable improvements in the provision of a broad range of preventive services for children who are younger than 5 years.
|
METHODS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
The VPSI represents a statewide public–private improvement partnership, with 2 state institutions (University of Vermont, Department of Pediatrics, and the VDH) partnering both with each other and with the AAP-VT to work on improving child health services. Of note, these organizations had collaborated in the early 1990s to agree on the menu of preventive services (periodicity schedule) that are offered to children under Early and Periodic Screening, Diagnostic and Treatment services and Medicaid managed care and then had decided to extend this schedule to all Vermont children. All partners, therefore, had a stake in the success of this work.
To lay the groundwork for the program, the plans for VPSI were presented at 2 AAP-VT meetings in the year before recruitment. Practices were recruited into the project through a letter that was signed by the AAP-VT chapter president and the executive director of VCHIP, followed by fax and telephone contacts. Institutional Review Board approval from the University of Vermont was received for this study. Informed consent was obtained from all primary care providers at each practice. Data that were collected through chart audits were aggregated and analyzed by practice. All data were collected before the implementation of Health Insurance Portability and Accountability Act.
Preventive Services Areas
The study team, in collaboration with representatives of AAP-VT and the VDH, selected 9 preventive services areas in 2 age groups as the focus of the project. Preventive services for 2-year-olds included (1) up to date on immunizations, (2) anemia screening, (3) tuberculosis risk assessment and indicated screening, (4) lead screening, (5) infant sleep position counseling, and (6) environmental tobacco smoke–exposure risk assessment. Preventive services for 4-year-olds included (7) blood pressure screening and (8) vision screening. Dental preventive services included (9) risk assessment for 2-year-olds and documented referral to or care by a dentist for 4-year-olds. Preventive services had to be completed within specific age ranges to qualify as being performed. The ranges of months used were as follows: anemia risk assessment between 5.5 and 24 months or hematocrit/hemoglobin level obtained between 6 and 24 months; tuberculosis risk assessment or screening (purified protein derivative standard [PPD]) between 0 and 24 months; lead risk assessment between 5.5 and 24 months or lead level obtained between 6 and 24 months; infant sleep position counseling between prenatal visit and 3 months; environmental tobacco smoke risk assessment between prenatal visit and 24 months; blood pressure obtained between 35 and 57 months; and vision screening between 35 and 57 months. These 9 preventive services areas are recommended by the US Preventive Services Task Force and the AAP as part of the provision of well-child care delivery and are mandated by Early and Periodic Screening, Diagnostic and Treatment services for all children who are insured by Medicaid.5,17 To demonstrate more than just improved documentation on the part of the health care provider, we chose a balanced set of measures to include documentation (eg, back-to-sleep counseling), process (eg, blood pressure and vision screening values documented), and outcome (eg, immunizations) measures.
Practice Participation and Data Collection
Of the 35 Vermont pediatric practices that were in operation from 2000 to 2003, 32 (91%) agreed to participate. One practice dropped out after the baseline chart abstraction; data from this practice were not analyzed, leaving 31 practices in the final analysis. On the basis of 2000 US census criteria, 73% of practices were rural. The majority of practices were physician owned and were group practices (Table 1). Insurance status was obtained in 92% of the medical charts reviewed, with 59% of the children privately insured, 41% covered by Medicaid, and <1% uninsured.
|
Project Intervention
The intervention was adapted from the Breakthrough Series Collaborative method,13 with the VCHIP project team initially receiving guidance and support from QI experts at the NICHQ. After baseline chart abstractions of preventive services rates at the participating practices, each practice received an in-person feedback session at the practice by a VCHIP pediatrician and nurse. The feedback included (1) the practice's baseline preventive services rates as compared with the aggregate data for all practices (an example of a feedback graph is shown in Fig 1), (2) an introduction to QI and the Breakthrough Series Collaborative method, and (3) a description of the benefits and obligations of project participation. Physicians, nurses, and administrative staff at the practice attended the feedback session. To participate in the intervention, practices were asked to form a QI team, identify specific preventive services QI goals, attend 1-day statewide trainings (learning sessions [LSs]), participate in monthly collaborative telephone calls, and report monthly on their improvement progress.
|
Statewide Trainings
Depending on when they entered the project, QI teams attended 3 or 4 face-to-face, 1-day LSs that were spaced 6 months apart. Trainings included (1) didactic sessions on evidence-based best practices in preventive services, (2) training in QI methods, (3) breakout sessions in which QI teams could apply the knowledge while assisted by VCHIP staff, and (4) opportunities for collaboration among the practice teams. Attendance ranged from 77 to 150 participants per training and, in addition to practice staff, included representatives from Medicaid, private insurers, public health, and community agencies. Three of the 4 LSs were held in conjunction with the AAP-VT meetings.
Table 2 summarizes the content covered at the 4 LSs. Across all 4 LSs, the VCHIP team emphasized the importance of using data to guide improvement activities and collaboration among the practices. Practices that enrolled after the first training attended a special training the day before LS2 to review the content from LS1 and to prepare for the next day's activities. Twenty-one practices attended LS1, an additional 4 practices joined at LS2, and 1 more practice joined for LS3. In total, 84% of pediatric practices attended at least 1 LS, with 35% having attended all 4 LSs; 58% attended at least 3, and 77% attended at least 2.
|
Monthly Reports
Practice teams submitted monthly reports via mail or fax, detailing chart abstraction results on a subset of 5 to 10 charts that corresponded to the changes that they had implemented in the previous month. VCHIP staff supported measurement activities by assisting teams in interpreting results and coaching teams in their ongoing PDSA cycles.
Practice Example
The following illustrates the manner in which 1 practice initiated the changes described above. Despite widespread recommendations that newborns should be placed on their back to sleep, only 8% of practices were documenting having counseled parents about this important topic. Providers stated that they often counseled but did not document it; they agreed that they were not consistent in counseling and documentation and that there was room for improvement. This example group practice, with 6 physician providers, 14 nurses, and 10 receptionists/clerical support staff, agreed on an aim statement "to provide comprehensive well-child care to newborns with a particular focus on the areas of newborn sleep position ("back to sleep"), newborn family history, and patient risk assessment." They specified the goal that 100% of their newborn families would be educated in the proper sleep position for infants as recommended by the AAP and documented that 100% of their newborn families and new patient transfers would complete a family/patient history information sheet and patient risk assessment. To accomplish these goals, they conducted a series of PDSA cycles, including obtaining provider agreement on use of a standard patient/family history/risk assessment form, incorporating the form into the workflow within the office, and educating staff on the use of the form. Steps involved 4 revisions to the form, a staff meeting to review use of the form, and an initial 2-week trial implementation. The practice held staff meetings to review proper infant sleep position, revised their well-child flow sheet, updated their patient handouts, and created prompts for discussion of sleep position with parents at 3 distinct visits: the prenatal visit, the first follow-up visit, and again at the 4-month visit. This team reported that one of their biggest challenges was disseminating information to all staff, including those who work part time; however, they also found that collecting data, both formal and informal, about their changes and sharing it across the practice was a key element to their success. This practice reported a baseline rate of <10% on documentation of sleep position counseling, and by the second training session, the team was reporting 100% compliance in their monthly measure. Staff reported enthusiasm, shared input in the process and the ability to transfer this improvement process to other areas in the practice.
Statistical Analyses
Overall mean change from baseline to follow-up on each of the preventive service variables was compared using the aggregated practice-level data. The initial statistical analyses used single-sample 2-tailed t tests on the difference score (change) for each preventive services variable. To address the question of whether focusing on a particular goal made a difference in the likelihood of improving, we conducted a second analysis using single-sample t tests of the change scores in preventive services areas, according to whether the practice had selected a specific preventive service area as a QI goal. These second analyses examined the change for each preventive service, looking at the group of practices that had identified the preventive service area as a goal and the group that had not.
Because clustering of patients' characteristics within practices represents a potential threat to the validity of these findings, both series of pre–post analyses were repeated using logistic-regression models that controlled for potentially high levels of intrapractice correlations (SAS Genmod).18 The logistic regressions used the method of generalized estimating equations and modeled before versus after change for each of the 9 preventive service variables while accounting for practice-level variability associated with 3 dichotomously coded practice characteristics: rural or suburban, physician owned or non–physician owned, and group or single practice. For the logistic-regression models that addressed the effects of practices' identified goals, the equations included a term for the interaction of change in preventive services and identified goal (coded dichotomously). For all models, statistical significance for change in preventive services was evaluated using 
2. To maintain statistical test validity for the analyses that focused on goals, we examined only preventive services that 8 or more practices selected as an improvement goal. The statistical significance level for all analyses was P < .05.
|
RESULTS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
|
|
|
|
|
DISCUSSION |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
The proliferation of preventive services guidelines for children challenges practices to provide recommended care.19,20 Despite that recommendations and guidelines for preventive care have been disseminated widely,5 we found tremendous variability within and among the practices at baseline. The intervention, although focused on 9 preventive services areas, allowed practices to choose the area(s) for improvement and to design their own approach. This improvement strategy values the practice as the expert; uses simple, incremental changes; and engages a multidisciplinary group in the practice. Most practices started with 1 area for improvement but very quickly took on new areas, often simultaneously and led by different team or staff members. VCHIP staff provided support that was tailored to the individual needs of the practices. This included providing practical tools (eg, designing or revising a form), effective strategies (eg, suggesting new systems for care delivery, how to prioritize and implement change, how to do a root-cause analysis to determine the source of the problem), measurement support (eg, providing simple measures to assess system changes, reviewing monthly data), and sharing learning (eg, connecting practices with common interests). In a state as rural as Vermont, this project brought together geographically scattered practices through either face-to-face training sessions or monthly conference calls and provided a forum for sharing and learning that otherwise would not have been possible.
A finding that is worthy of comment is that a practice's decision to focus on specific preventive services as their QI goals seemed necessary to their improving preventive services. We conclude this because although improvements were made when practices focused on specific preventive services, we found no evidence that focus on 1 group of services resulted in a decrease in rates of other preventive services. The implications of this finding are unclear. The most likely explanation is that this was a case of competing demands (ie, that focusing energies on certain topics precluded working on other topics). Practice systems for the delivery of preventive services areas may differ sufficiently from one another (eg, immunizations versus back-to-sleep counseling versus lead screening) to prevent translation of change strategies from 1 area to the next.
Overall, these results suggest that QI outreach was effective in this state. A comprehensive analysis of the effectiveness must take cost into account. The true total cost is challenging to define. We estimate that the cost of conducting the project, including the evaluation, was $350000, or approximately $11000 per practice. An additional unmeasured cost is that of the staff time devoted by each practice site in carrying out the QI work. With 27000 children in the state younger than the age of 5 years and the estimate that this project had an impact on 80% of that population, the relative cost per child of this endeavor is approximately $16. These costs included, however, not only the work of the project but also the start-up cost of establishing the VCHIP program. Subsequent QI efforts in other topic areas have cost less, because the VCHIP program is well established and the practices now are experienced in QI. Therefore, there are additional benefits, both to the practice and to the state, that have accrued with those initial investments. With the possible implementation of pay for performance, practices may recoup much of their in-kind investment in this work.
In addition to improvements in preventive services, practices benefited in other ways from their participation in the project. Practices learned basic QI skills that are transferable to other clinical areas. Although we did not specifically measure it, practices reported applying this method to other areas in the practice for improvement (eg, improving adolescent hepatitis B vaccination rates).
This study has some clear limitations. Because this intervention was implemented in an entire state, there was no comparison group. We cannot exclude, therefore, the possibility that a secular trend in improved preventive services accounted for the observed changes. However, we believe that the likelihood of such a trend's accounting for the observed change is remote. Although there has been some national trend in the improvement of varicella immunization rates, current evidence suggests that levels of lead screening, vision screening, and other preventive services have been stable. In fact, the immunization rates in Vermont for nonvaricella vaccines are already so high that a ceiling effect may have prevented any demonstration of a stronger impact on immunization rates.
Second, some of the observed improvement may reflect better documentation of activities that the practices were already conducting. For preventive services such as back-to-sleep counseling, this may be true. However, for varicella immunization and for tests such as lead and vision screening, for which a procedure is conducted with clear requirements for documentation of results (eg, immunization date, lead or vision value), we believe that improved documentation is unlikely to account for our findings. In fact, some of our study results may underestimate the true improvement that has taken place. Our postintervention audit of the charts of children who were 2 years of age occurred 
18 months after an initial audit and an even shorter time after the beginning of our QI process. Because the postintervention chart audit was completed 18 months after the first chart audits on charts of 2-year-olds, changes to counseling that was performed in the first year of life, especially the newborn period such as back-to-sleep counseling, likely would not be captured completely 18 months later.
To what extent is an improvement partnership program such as VCHIP necessary to making statewide change? Simpson21 recommended the development and support of knowledge brokers as an important step in the effort to accelerate improvement in children's health care. VCHIP serves such a role in Vermont. Although organizations such as health departments and AAP chapters have clear incentives to improve care, they typically lack sufficient resources to do so. An improvement partnership such as VCHIP can provide those resources. Specifically, VCHIP provided staff with expertise in QI to coach and support the practices, provided grant-writing skills to help secure funding, and, most important, brought the latest QI knowledge from national organizations such as the NICHQ to the local level for adoption by practices in the state.
The involvement of AAP-VT chapter leadership was critical, facilitating recruitment of 91% of the state's practices, which serve >80% of Vermont children who are younger than 5 years. Other regional collaborations with pediatric practices around QI in larger states, which have demonstrated similar outcomes, also have involved leadership from local professional organizations.11,12 We suspect that it is the involvement of provider leadership, not the size of the region, that is critical to the success of such efforts.
Although this project was deemed highly successful by the practices and organizations involved, it raises questions that should be examined. We do not know to what extent the changes implemented will be sustained by the practices. Stange et al22 found sustained changes 24 months later in preventive services among family practices after a practice-based intervention to improve preventive services. Every attempt was made to identify changes to the system to create sustainable, rather than transient, change. These improvements led to increases in the rates of preventive services, but we did not measure the impact of these changes on the cost of care. It is possible that improved compliance with the recommended services might increase health care costs, but we are unable to estimate the cost savings accrued in preventing serious problems in the future (eg, sudden infant death syndrome, lead toxicity, amblyopia). Practices shared with us how they were able to apply this approach to improvement to other areas in the practices. It is unclear what specific support and materials are needed to assist practices with improvement in other areas, such as asthma, attention-deficit/hyperactivity disorder, or developmental screening, once practices learn the model for improvement. The answers to these questions should be determined in future studies.
|
CONCLUSIONS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
|
ACKNOWLEDGMENTS |
|---|
We acknowledge the sponsors of the project and the contributions of our colleagues at our collaborating organizations, Charles Homer, Jayne Stuart, Peter Margolis, and Carole Lannon. Special thanks are extended to Jan Carney, former Vermont Commissioner of Health, for support and guidance during this project and Jeffrey Horbar and Lewis First, who provided expert advice. Most important, we recognize the dedication of the clinicians and staff of the pediatric practices whose participation made this work possible.
|
FOOTNOTES |
|---|
Address correspondence to Judith S. Shaw, RN, MPH, Vermont Child Health Improvement Program, Arnold 5459, One S Prospect St, Burlington, VT 05401. E-mail: judith.shaw{at}uvm.edu
The authors have indicated they have no financial relationships relevant to this article to disclose.
Preliminary findings from this study were presented at the Ambulatory Pediatric Association Presidential Plenary, meeting of the Pediatric Academic Societies; May 5, 2003; Seattle, WA.
|
REFERENCES |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
This article has been cited by other articles:
 |
|
 |
|
  E. J. Slora, J. M. Steffes, D. Harris, H. W. Clegg, D. Norton, P. M. Darden, S. A. Sullivan, and R. C. Wasserman Improving Pediatric Practice Immunization Rates Through Distance-Based Quality Improvement: A Feasibility Trial from PROS Clinical Pediatrics, February 1, 2008; 47(1): 25 - 36. [Abstract] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
||
|
||
Home | My Pediatrics | Journal Information | Current Issue | Past Issues | Subscriptions & Services | Contact Us Click here for faster international access © 2006 American Academy of Pediatrics. All rights reserved. |
||
|
||
