BACKGROUND AND OBJECTIVE: Many patients with Medicaid do not receive timely, comprehensive well-child care through the Early and Periodic Screening, Diagnosis, and Treatment (EPSDT) program. Missed opportunities for EPSDT well-child check-ups (WCCs) at acute visits contribute to this problem. The authors sought to reduce missed opportunities for WCCs at acute visits for patients overdue for those services.
METHODS: A quality improvement team developed key drivers and used a people-process-technology framework to devise 3 interventions: (1) an electronic indicator based on novel definitions of EPSDT status (up-to-date, due, overdue, no EPSDT), (2) a standardized scheduling process for acute visits based on EPSDT status, and (3) a dedicated nurse practitioner to provide WCCs at acute visits. Data were collected for 1 year after full implementation.
RESULTS: At baseline, 10.3 acute visits per month were converted to WCCs. After intervention, 86.7 acute visits per month were converted. Of 13 801 acute visits during the project, 31.2% were not up-to-date. Of those overdue for WCCs, 51.4% (n = 552) were converted to a WCC in addition to the acute visit. Including all patients who were not up-to-date, a total of 1047 acute visits (7.6% of all acute visits) were converted to comprehensive WCCs. Deferring needed WCCs at acute visits resulted in few patients who scheduled or completed future WCC visits.
CONCLUSIONS: Implementation of interventions focused on people-process-technology significantly increased WCCs at acute visits within a feasible and practical model that may be replicated at other academic general pediatrics practices.
- missed opportunities
- well-child care
- quality improvement
- outpatient clinics
- nurse practitioner
- AAP —
- American Academy of Pediatrics
- CHPCC —
- Children’s Hospital Primary Care Clinic
- EMR —
- electronic medical record
- EPSDT —
- Early and Periodic Screening, Diagnosis, and Treatment
- FTE —
- full-time equivalent
- NP —
- nurse practitioner
- REDCap —
- Research Electronic Data Capture
- QI —
- quality improvement
- WCC —
- well-child check-up
Early and Periodic Screening, Diagnosis, and Treatment (EPSDT) is Medicaid’s comprehensive and preventive child health benefit for children younger than 21 years.1 EPSDT screening exams are essentially synonymous with well-child check-up (WCC) visits and include a comprehensive health and development history, an unclothed physical exam, immunizations, laboratory tests, anticipatory guidance, and hearing, vision, and dental services.2 These EPSDT screening exams are the hallmark of the EPSDT program and serve as the foundation of well-child care for children with Medicaid.3 In addition to receiving immunizations and developmental screenings, children who have regular EPSDT care are less likely to have unexpected hospitalizations or emergency department visits.4–6 While there have been steady increases in EPSDT screening rates nationally, large gaps in coverage still remain particularly among older children.7,8 These aggregate screening rates do not address rates at the practice or provider level, nor do they give any sense of the number of missed opportunities for these services.
A missed opportunity is defined as any patient encounter where an intervention is recommended yet not performed. Many studies have examined missed opportunities for preventive services, primarily related to immunizations.9–15 However, immunization status alone is a poor indicator for other aspects of comprehensive well-child care provided through EPSDT.16 While families are willing to receive immunizations at acute visits,17 immunizing at acute visits has been shown to reduce the likelihood of patients returning for WCCs.18 Outside the work focused on immunizations and other piecemeal components of well-child care, little has been done looking at missed opportunities for complete well-child care at acute visits.
In 2007, Children’s Hospital Primary Care Clinic at Vanderbilt (CHPCC) embraced medical home principles as defined by the American Academy of Pediatrics (AAP),19 and this forced us to assess our ability to provide comprehensive and accessible care. Recognizing the contextual limitations of a resident continuity clinic as a medical home, this project was one of several implemented by clinic staff to deliver care more consistent with medical home principles. Initial data showed that 51% of patients in our practice were not up-to-date for WCCs and that many children had multiple missed opportunities for these services at acute visits. Our aim was to use quality improvement (QI) methods to develop a clinic system to improve WCC delivery at acute visits by reducing missed opportunities for children who were overdue for WCCs at acute visits by 50% within 1 year. We adopted a simple people-process-technology framework frequently used in informatics as the basis for our interventions and determined key drivers for the project (Fig 1).20
CHPCC serves as a continuity clinic for 72 pediatric residents and provides comprehensive primary care for 15 000 patients. We have 33 000 annual visits, roughly divided equally between well-child and acute care. The clinic has 22 exam rooms and is staffed by 4 supervisory attending physicians, 4 to 8 residents in continuity clinic, and 4 to 8 residents and 1 to 2 medical students in acute care. Nurse practitioners (NPs) provide some direct care for both well-child and acute care (3.25 full-time equivalents [FTEs]). Almost 80% of patients have TennCare, Tennessee’s form of Medicaid, and more than 30% of visits are with families who do not speak English as a primary language. We have a proprietary electronic medical record (EMR), StarPanel, which is fully supported at Vanderbilt.
Developing Interventions: People-Process-Technology
Before this project, clinic staff and providers would only occasionally review the EMR to determine the date of the last WCC during an acute visit. If a patient were noted to need a WCC, families were encouraged to schedule a future appointment. Only infrequently would comprehensive WCC services be provided at an acute visit. To improve this process, a QI team was created that included the clinic’s medical director, other clinic physicians, clinic manager, charge nurses, telephone triage nurses, front desk staff, and a physician member from the Department of Informatics. We developed key drivers and knew we needed a quick way to determine a child’s current EPSDT status. (We refer to EPSDT status instead of WCC status since our electronic indicator was labeled “EPSDT,” but the terms are considered equivalent in this project.) Our internal medicine colleagues at Vanderbilt had already developed patient-level electronic dashboards within the EMR for adult patients with diabetes and hypertension, so we turned to the principal developer of those tools (Dr Gregg) to create a pediatric preventive dashboard that would include EPSDT status as a visual, color-coded indicator to be shown each time a patient’s chart was opened.
We first had to define standard categories for EPSDT status. By using the Bright Futures/AAP Recommendations for Preventive Pediatric Health Care,21 we developed definitions for 4 categories of EPSDT status: (1) up-to-date (last WCC within recommended AAP interval for age, flagged green); (2) due (last WCC within 1× and 1.5× AAP interval for age, flagged yellow); (3) overdue (last WCC >1.5× AAP interval for age, flagged red); and (4) and no WCC (never had a documented WCC in our clinic, flagged white). As the recommended intervals between visits change as children grow, we developed age-specific cut-offs based on the best clinical judgment of the team as shown in Table 1.
The pediatrics preventive dashboard calculated current EPSDT status based on clinical documentation in the EMR. It also included indicators for blood pressure percentiles, BMI percentiles, height, weight, and influenza vaccine status (Fig 2). For the purposes of this QI project, we focused on reducing missed opportunities related only to EPSDT status.
The team then developed a standardized process for clinic staff to follow once an overdue patient called or presented for care. Since 80% of acute visits were scheduled through telephone triage nurses, we developed a protocol for triage nurses to schedule patients differently according to EPSDT status (Fig 3). To capture the 20% of patients who presented as walk-ins (including many patients who do not speak English), front desk staff were given a similar protocol. To encourage continuous care within the resident continuity clinic and medical home, our interventions were only aimed at patients who did not already have appointments scheduled with their primary care provider.
The QI team thought that interventions focused on technology and processes alone would likely not be sufficient to lead to provider behavior change required to capture missed opportunities. Increasing visit numbers coupled with sometimes unpredictable resident clinic staffing for acute visits led the team to determine that an additional provider would be necessary. Based on preliminary data showing that 8 to 15 acute patients per day would need WCCs and having used NPs within the practice for more than a decade, we developed a pro forma for a new NP position (0.75 FTE) that would spend roughly 50% of her/his time (0.375 FTE) doing WCCs at acute visits. We created a visit template that would allow for acute visits to be converted to an acute plus WCC visit (a “conversion visit”). When the NP was not in clinic or was fully scheduled, conversion visits were to be performed by resident physicians staffing acute visits.
This project evaluated the cumulative effect of our 3 main interventions: (1) electronic EPSDT status indicator; (2) standardized clinic processes; and (3) dedicated additional provider staffing. Interventions were introduced in a staggered fashion over several months. The NP was hired in January 2010, the electronic EPSDT status indicator began on March 10, 2010, and the full clinic triaging and scheduling process began on June 1, 2010. All clinical staff and providers were educated about the process through a didactic session in May 2010. The institutional review board reviewed the protocol and determined that, as QI work, it did not qualify as human research.
Measurement and Analysis
Baseline (all of 2009) and early intervention (January through May 2010) WCC conversion rates were determined through chart review of each acute visit billed as a WCC. For the full intervention period (June 2010 through May 2011), data were retrospectively collected by chart review from each weekday acute visit for patients older than 2 months. We excluded Saturday visits since patients from other practices are seen during that clinic. For each encounter during the full intervention period, we collected current EPSDT status, whether the acute visit was converted to a WCC, future appointment status, and whether a future appointment was completed. A conversion visit was defined as an acute appointment that was subsequently documented as a WCC within the EMR. Billing data were also used to confirm that a WCC was performed. A research assistant collected, stored, and managed data by using REDCap (Research Electronic Data Capture) electronic data capture tools hosted at Vanderbilt University. REDCap is a secure, Web-based application designed to support data capture for research studies.22
Each month, the number of conversion visits was plotted on a run chart. We used probability-based rules to detect nonrandom signals of change within the data including tests for shift (≥6 consecutive points above or below the median), trend (≥5 consecutive points all going up or all going down), runs (too many or too few crossings of the median line), or an astronomical value.23 A new median line was redrawn after a clear nonrandom shift and astronomical value were detected in June 2010.
Review of billing data for all acute visits from 2009 showed that we averaged 10.3 conversion visits per month before any of our interventions (median, 9.5). Monthly acute visits during that time ranged from 913 (June 2009) to 1651 (September 2009) (Fig 4).
During the phase from January 2010 through May 2010, there were 5472 weekday acute visits for patients 2 months and older, and the average number of conversion visits was 27.6 per month. After full implementation including the triaging and scheduling process, the average number of conversions per month rose to 86.7 with a median of 90.5. For the 12 months after full implementation in June 2010, there were 13 801 weekday acute visits for patients 2 months and older (Fig 5). Of those visits, 9629 (69.8%) were with patients who were up-to-date for WCCs at the time of the visit. The remaining 4172 visits were divided among encounters where patients were due for WCC (1910, 13.8%) or overdue for WCC (1490, 10.8%) or who had never had a WCC in clinic (772, 5.6%).
A total of 1047 conversion visits were completed representing 7.6% of all acute visits. For eligible patients overdue for WCCs at acute visits, 51.4% (552) were successfully converted, which slightly surpassed our aim of 50%. While our interventions were primarily intended for patients who were overdue for WCCs, there was spillover benefit for patients with no previous WCCs and also for those who were due for services. For eligible patients who were due, 29.1% (341) were converted, and for eligible patients with no previous history of any WCC in clinic, 28.6% (154) were converted to WCCs.
The study also demonstrated the effect of deferring WCCs for patients who were not up-to-date. If a patient already had a scheduled WCC appointment, the appointments were kept 65% to 77% of the time. However, if a WCC were deferred and scheduled in the future, only 53% to 58% of those appointments were completed. For patients who were not up-to-date and who did not already have an appointment scheduled, 28% to 46% of them did not schedule a future appointment.
Overall, resident physicians were less likely to convert acute visits to WCCs than was the NP (28.8% of eligible overdue opportunities for residents were converted vs 78.8% for the NP). Given the far greater volume of acute visits at which patients were seen by residents, however, almost half of all conversion visits for patients who were not up-to-date were completed by resident physicians (508 by residents, 539 by NP).
A complete financial analysis of the project is beyond the scope of this report (available on request), but this effort produced $45 000 in additional revenue, more than enough to offset the NP’s 0.375 FTE dedicated to conversion visits. Perfect provider billing that captured both well and acute services would have produced an additional $26 000 in revenue.
This project demonstrated that WCCs can be incorporated into acute visits by using QI techniques focused on key drivers related to people-processes-technology. This comprehensive approach to well-child care at acute visits eliminates concerns about families not returning for WCCs if only immunizations are given at acute visits and clearly builds on the data that families are willing to receive needed well-child care at acute visits.17 We found that deferring needed WCCs at acute visits meant that for many children, future appointments would not be scheduled or kept, underscoring the need to perform WCCs at acute visits when feasible.
One of the key innovations of this project was the ability to define EPSDT status in a way that was practical, easily derived, and usable at the patient and practice levels. The electronic EPSDT indicator allowed technology to drive the people and process changes. While electronic reminders alone have had mixed success driving focused preventive interventions in other settings,9,10,24–26 the integration into a broader process worked well here. We believe this was successful because the indicator was easily accessible by all clinic staff, was color-coded, and was clearly linked with well-defined processes for each staff member. This standardized definition of EPSDT status could readily be adopted by other general pediatrics practices and could also be used for appointment outreach and recall.
The project additionally demonstrated the success of integrating NPs into academic pediatrics practices, and it also shows some willingness of resident providers to provide well-child care at acute visits. Earlier work by Hull et al described the feasibility of using registered nurse–based protocols to incorporate EPSDT services.27 Their study was based on a much smaller convenience sample while ours showed success with large-scale implementation integrating both resident providers and NPs. By billing for both WCCs and acute visits at conversion visits (both allowed and encouraged by TennCare), we were able to provide all needed services within a model that may be financially sustainable.
There were limitations with this project. First, we based our interventions on EPSDT indicators that were only visible within our institution’s medical record: we could not determine if a child had received well-child care outside our system of care. This would potentially overestimate the number of children who needed WCCs. Second, we do not have retrospective data about EPSDT status for children seen at acute visits before the intervention; however, we do not believe there were significant access, demographic, or other changes in the period between baseline and intervention. Third, we did not study balancing or mitigating measures such as changes in cycle times, nursing workload, or patient and provider satisfaction with this project.
Additionally, while these visits were staffed, reviewed, and attested by attending pediatricians through the clinic’s usual review process, we did not perform an analysis to determine if there were differences between the quality of care delivered at typical WCCs compared with conversion visits. Future analysis should determine if there were differences in the level of well child care provided at conversion visits compared with typical check-ups. Furthermore, while we were able to determine that our system of care was potentially sustainable within the scope of our clinic, we were not able to perform a broader financial analysis of the larger societal or health system implications of providing well-child care at acute visits.
Despite these limitations, we believe this approach to recognizing and providing WCCs at acute visits could be replicated at other academic practices. Even for clinics without the ability to hire new practitioners, the processes for recognizing and treating patients who need well-child care at acute visits could lead to increased delivery of these services.
By determining key drivers focused on people-processes-technology, we developed a feasible and practical system of care that allowed for improved delivery of needed WCCs at acute visits. We demonstrated that families were willing to receive comprehensive well-child care at acute visits and that deferring needed check-ups meant that many patients would not return for care. Through our interventions, both NPs and resident physicians worked to make well-child care more accessible for our patients, and we were able to provide these services in a potentially financially sustainable way. We believe this approach could change how well-child care is delivered at academic practices and could be replicated at other academic practices that care for underserved children.
This project could not have been completed without Sara McFadden, our research assistant. The authors would like to thank Dr Ted Speroff for his assistance with data analysis, Carol Moore, FNP, for her dedication to clinic patients and well-child care, and the entire team at CHPCC.
- Accepted April 24, 2012.
- Address correspondence to Barron Lee Patterson, MD, FAAP, Department of Pediatrics, Vanderbilt University School of Medicine, 8236 Doctors’ Office Tower, 2200 Children’s Way, Nashville, TN 37232. E-mail:
Dr Patterson initially conceived of the project, served as a leader of the QI team, led project implementation, helped interpret the statistical analysis, and drafted and revised the manuscript. Dr Gregg was a member of the planning team, developed the software and concept behind the patient-level dashboard, and revised the manuscript. Ms Biggers was a leader of the QI team, developed the staffing and financial model for the project, gathered data, and helped revise the manuscript. Dr Barkin helped conceptualize the project, supervised and gave direction during the course of the entire project, and provided review and critique of each version of the manuscript. Each author accepts full responsibility for the final submitted manuscript.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose. Drs Patterson and Barkin and Ms Biggers have no conflicts of interest to disclose. Dr Gregg reports a conflict of interest outside of the submitted work. The software used in this project, StarPanel, which is the electronic medical record for Vanderbilt University Medical Center has been licensed to Informatics Corporation of America (ICA), which markets a version of the software. Dr Gregg is listed as one of the inventors under the licensing agreement and does occasional consulting work for ICA.
FUNDING: Dr Patterson received protected time and salary support for this project through the Katherine Dodd Faculty Scholar Program from the Vanderbilt University Department of Pediatrics. The REDCap database was supported by a Vanderbilt Institute for Clinical and Translational Research grant (1 UL1 RR024975 from NCRR/National Institutes of Health).
- ↵Centers for Medicare and Medicaid Services. Early and periodic screening diagnosis & treatment. Available at: www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Benefits/Early-Periodic-Screening-Diagnosis-and-Treatment.html. Accessed January 17, 2012
- ↵Maternal and Child Health Bureau. EPSDT & Title V collaboration to improve child health. Available at: mchb.hrsa.gov/epsdt/index.html. Accessed January 17, 2012
- Hakim RB,
- Bye BV
- Office of the Inspector General
- Centers for Medicare and Medicaid Services
- Fiks AG,
- Grundmeier RW,
- Biggs LM,
- Localio AR,
- Alessandrini EA
- Shaw JS,
- Samuels RC,
- Larusso EM,
- Bernstein HH
- McConnochie KM,
- Roghmann KJ
- Sabnis SS,
- Pomeranz AJ,
- Lye PS,
- Amateau MM
- Lieu TA,
- Black SB,
- Sorel ME,
- Ray P,
- Shinefield HR
- Szilagyi PG,
- Rodewald LE,
- Humiston SG,
- et al
- Udovic SL,
- Lieu TA,
- Black SB,
- Ray PM,
- Ray GT,
- Shinefield HR
- Fiks AG,
- Hunter KF,
- Localio AR,
- Grundmeier RW,
- Alessandrini EA
- Medical Home Initiatives for Children With Special Needs Project Advisory Committee, American Academy of Pediatrics
- ↵Alter S. The Work System Method: Connecting People, Processes and IT for Business Results. Larkspur, CA: Work System Press; 2006
- Hagan JF,
- Shaw JS,
- Duncan PM
- Provost LP,
- Murray SK
- Hull PC,
- Husaini BA,
- Tropez-Sims S,
- Reece M,
- Emerson J,
- Levine R
- Copyright © 2012 by the American Academy of Pediatrics