Abstract
BACKGROUND AND OBJECTIVES: It is unclear which specific well-child visits (WCVs) are most frequently missed and whether age-specific patterns of attendance differ by race or insurance type.
METHODS: We conducted a retrospective cohort study of children 0 to 6 years old between 2011 and 2016 within 2 health networks spanning 20 states. WCVs were identified by using International Classification of Diseases, Ninth and 10th Revisions and Current Procedural Terminology codes. We calculated adherence to the 13 American Academy of Pediatrics–recommended WCVs from birth to age 6 years. To address data completeness, we made 2 adherence calculations after a child’s last recorded WCV: 1 in which we assumed all subsequent WCVs were attended outside the network and 1 in which we assumed none were.
RESULTS: We included 152 418 children in our analysis. Most children were either publicly insured (77%) or uninsured (14%). The 2-, 4-, and 6-month visits were the most frequently attended (63% [assuming no outside care after the last recorded WCV] to 90% [assuming outside care]), whereas the 15- and 18-months visits (41%–75%) and 4-year visit (19%–49%) were the least frequently attended. Patients who were publicly insured and uninsured (versus privately insured) had higher odds of missing WCVs. Hispanic and Asian American (versus non-Hispanic white) patients had higher odds of attending WCVs.
DISCUSSION The 15- and 18-month WCVs as well as the 4-year WCV are the least frequently attended WCVs. The former represent opportunities to identify developmental delays, and the latter represents an opportunity to assess school readiness.
- AAP —
- American Academy of Pediatrics
- CI —
- confidence interval
- EHR —
- electronic health record
- FQHC —
- Federally Qualified Health Center
- MEPS —
- Medical Expenditure Panel Survey
- OR —
- odds ratio
- VCUHS —
- Virginia Commonwealth University Health System
- WCV —
- well-child visit
What’s Known on This Subject:
In young children, adherence to well-child visits (WCVs) has been reported to be lowest between 1 and 2 years of age and between 3 and 5 years of age. Missed WCVs have been associated with increased emergency department use and hospitalizations.
What This Study Adds:
Across patients of different races and with different insurance types, WCVs at 15 months, 18 months, and at 4 years were the most commonly missed. Children who missed these visits may lack developmental screenings and other preventive services typically performed at these ages.
The American Academy of Pediatrics (AAP) recommends at least 13 well-child visits (WCVs) between birth and 6 years of life.1 There are several components of WCVs that may contribute to improved health outcomes, including (1) timely receipt of age-appropriate vaccinations, (2) identification and management of acute and chronic illnesses, (3) education for parents about what to do for children who are otherwise healthy during such illnesses, and (4) screening for and management of developmental delays.2,3 It is estimated that children miss approximately one-third of WCVs,4 with African American children,4–6 children who are uninsured4 or publicly insured,6–8 and children from low-income families4,6 missing even greater proportions of WCVs. Missed WCVs have been associated with increased emergency department use9 and hospitalizations,2,10 associations that are especially pronounced among children from low-income families.10
Previous studies on adherence to the WCV schedule are limited by geographic representation,7 age of data,4,5,7,8,11 methodology, and/or quality of data.4 In the study that revealed the most age-specific detail in WCV attendance, researchers used data from the Medical Expenditure Panel Survey (MEPS) >15 years ago.4 This MEPS study revealed that between 2000 and 2002, adherence to the recommended number of WCVs was lowest among those aged between 13 and 18 years, followed by between 6 and 12 years.4 Among young children <6 years of age, adherence was lowest between 1 and 2 years of age and between 3 and 5 years of age.4 The MEPS data were generated by caregiver report; in the survey, caregivers were asked to identify which of their children’s visits to a medical provider were WCVs. Because the MEPS does not have longitudinal data on individual patients, researchers use an overlapping panel design to identify age-based trends. Because of the particular methodology used for the statistical analysis, MEPS reports compliance ratios over 1 to 6 years. To our knowledge, there are no longitudinal studies of clinic visits that examine the exact ages at which WCVs are most likely to be missed and how these differ by various demographic factors.
Our aim in this study was to determine if more recent electronic health record (EHR)–verified data could be used to determine the specific WCVs that are most frequently missed. On the basis of evidence that children of low-income homes are at higher risk for missed WCVs,4,6 we partnered with health care organizations with substantial populations of pediatric patients from low-income homes. Using this large preexisting research consortium comprised of children who are publicly insured, privately insured, and uninsured, we conducted a retrospective cohort study to estimate attendance for AAP-recommended WCVs between birth and age 6 years.
Methods
Setting
This study took place within a preexisting research consortium12 composed of (1) the OCHIN network (formerly known as the Oregon Community Health Information Network) and (2) the Virginia Commonwealth University Health System (VCUHS). Founded in 2001, OCHIN is a nonprofit, community-based, health information technology collaborative that serves 96 health systems (Federally Qualified Health Centers [FQHCs], community health centers, Critical Access Hospitals, and rural hospitals) in 19 states across the nation and links data from 581 clinics with 5027 primary care providers.12–16 VCUHS is an academic health system serving central and northeastern Virginia, which includes a large proportion of publicly insured children.
Patients and Participants
We included children aged 0 to 6 years with ≥1 WCV between January 1, 2011, and January 1, 2016. WCVs were identified by International Classification of Diseases, Ninth Revision, Clinical Modification or International Classification of Diseases, 10th Revision, Clinical Modification WCV diagnosis codes (eg, V20.2 and Z00.129) and Current Procedural Terminology claims codes (eg, 99381). We excluded patients without any WCVs because these patients may have used a different health system as their primary care medical home. Demographic variables, including age, sex, race, ethnicity, insurance type at last attended WCV within the study period, and residential zip code at last attended WCV within the study period, were queried for from the EHR. Zip codes were geocoded to estimate the distance between patients’ residences and the clinics where they attended their most recent WCVs.
Data Analysis
To capture the specific ages at which WCVs were missed, we used unique age ranges to define eligibility for each WCV (Supplemental Table 4). The duration of the age ranges increased with increasing age. For example, a WCV that was expected to occur at “2 months” could have taken place anywhere between 6 and 14 weeks of age, whereas the “3-year” WCV could have occurred between 34 and 46 months of age. We considered a patient’s entry time into the study to be his or her first WCV; recommended WCVs that would have occurred before a patient entered the study were not counted toward that patient’s total WCV count. Within the EHR, there was no way to distinguish between a missed WCV and a WCV that occurred outside the health care system. Because we wanted to study the phenomenon of missed WCVs, we calculated adherence to the AAP-recommended WCV schedule in 2 different ways: 1 in which all WCVs after the last recorded WCV were assumed to be attended outside the network and 1 in which none of the WCVs after the last recorded WCV were assumed to be attended outside the network. When assuming no subsequent outside care (ie, the lower limit of this range), we assumed that any missed WCVs from study entry to the end of the study period were not fulfilled outside the health network (ie, patients only received care as documented in the EHR). When assuming that subsequent outside care was received (ie, the upper limit of this range), we assumed that the exit point was the last recorded WCV within the study period, and any missed WCVs after the last recorded WCV were fulfilled outside the health network (eg, as might happen if a patient moved). This strategy allowed us to generate a reasonable range within which the true measure of adherence likely lies.
The overall proportion of total WCVs attended was estimated as the total number of WCVs attended by all patients divided by the total number of recommended WCVs for all patients adjusted for the study entry point. The proportion of WCVs attended within each age range was estimated as the total number of WCVs attended by all patients in that age range divided by the total number of recommended WCVs for all eligible patients in that age range. A multivariable analysis was used to (1) adjust the overall and age-specific proportions of WCVs attended for patient characteristics selected a priori (sex, race and/or ethnicity [as a single variable], insurance type at last WCV, and distance to clinic) and interpractice variability and (2) to examine the relative influence of patient characteristics on WCV attendance. When looking at adherence after the newborn period, we additionally adjusted the analyses for the age in weeks at which a child was first seen within the health network. For both the overall and age-specific analyses, a generalized linear mixed model was used to model the proportion of WCVs attended with fixed effects for patient sex, race and/or ethnicity, insurance type, distance from clinic, and (when appropriate) age in weeks at the first visit. Only 1 race and/or ethnicity was available per patient. The practice and health system network were included as random effects to account for any inherent attendance variability between practices and between health system networks. Resulting odds ratios (ORs) and their 95% confidence intervals (CIs) were reported. ORs for distance were reported per 100-mile increase, and ORs for age were reported for each 4-week increase (∼1 month). We interpreted our results to be significant when the ORs for the limits of the adherence range (ie, when not assuming outside care and when assuming outside care) were both significant at the 5% level and in the same direction. SAS statistical software version 9.4 (SAS Institute, Inc, Cary, NC) was used for these analyses. We obtained approval for this study from the Institutional Review Board of Virginia Commonwealth University.
Results
Overall WCV Attendance
There were a total of 152 418 patients from 752 individual practices (746 OCHIN practices and 6 VCUHS practices; Table 1) who met inclusion criteria. The overall proportion of total WCVs attended was 52% when assuming no subsequent outside care and 77% when assuming subsequent outside care (Table 2). ORs and 95% CIs for overall adherence by race and/or ethnicity and insurance type can be found in Table 3. Asian American and Hispanic patients had higher odds of attending WCVs compared with non-Hispanic white patients. African American (versus non-Hispanic white) patients had higher odds of attendance when not assuming outside subsequent care but lower odds of attendance when assuming outside subsequent care. There were no significant differences in attendance between white patients and patients of "other" races. Those who were publicly insured and uninsured (versus privately insured) also had lower odds of overall visit attendance. Attendance for girls was slightly lower than for boys, although it was statistically significant. Distance to the clinic was not significantly associated with overall WCV attendance.
Patient Characteristics (N = 152 418)
Number of WCVs Attended Within Each Age Range, Followed by the Proportion of WCVs Attended in Each Age Range When Assuming No Outside Care After the Last WCV and Outside Care After the Last WCV
Overall Visit Attendance Across All Included WCVs
Age-Specific WCV Attendance
The visit that was most frequently attended was the 2-month visit, followed by the 6- and 4-month visits (Table 2). The WCV that was least frequently attended was the 4-year visit, followed by the 15- and 18-month visits. However, the proportion of 15- or 18-month visits attended was 60% when assuming no subsequent outside care and 89% when assuming subsequent outside care.
Although the patterns of age-specific attendance were similar to the overall patterns of attendance between those of different racial groups and those with different insurance types, there were some notable exceptions (Figs 1 and 2). Patients who were publicly insured and uninsured had lower odds of WCV attendance between birth and 15 months of age compared with those who were privately insured, whereas those who were uninsured had higher odds of attending the 4-year WCV compared with those who were privately insured. In addition, African American (versus non-Hispanic white) patients had significantly higher odds of attending at least 1 of the 15- or 18-month WCVs.
Percentage of age-specific WCVs attended by those in various categories of race and/or ethnicity. Each line spans from assuming no outside care after the last WCV to assuming complete outside care after the last WCV.
Percentage of age-specific WCVs attended by those with various insurance types. Each line spans from assuming no outside care after the last WCV to assuming complete outside care after the last WCV.
Discussion
To our knowledge, this is the first study to use longitudinal EHR data to determine exactly at what ages WCVs are being missed. In this large and diverse cohort of children who are mostly publicly insured or uninsured, we found that attendance was high in early and middle infancy (2, 4, and 6 months of age) then declined and rebounded at 5 years of age. When examined individually, the 15-month, 18-month, and 4-year WCVs were the least frequently attended. Using the optimistic assumption that subsequent WCVs are attended elsewhere, we estimate that ∼1 in 5 WCVs are missed. Without this assumption, we estimate that approximately half of WCVs are being missed.
These results are concordant with the previous nationally representative MEPS study, in which researchers reported that the lowest WCV adherence in young children is between 1 and 2 years of age and between 3 and 5 years of age.4 In addition, a more recent study using 2007 MEPS data revealed that 40% of children ages 3 to 5 years who were insured had no WCV recorded.17 Our findings are also consistent with national trends of vaccination compliance in which a noted dip occurs in coverage of the fourth diphtheria-tetanus-acellular pertussis, pneumococcal conjugate, and Haemophilus influenzae type B vaccines that are recommended between 12 and 18 months of age.18,19
One proposed explanation for lesser attendance at the 15- and 18-month WCVs is that fewer vaccinations are required at those visits compared with at visits in early infancy.20 Relatedly, children between 1 and 2 years of age are frequently seen for sick visits21 and may be given vaccinations at that time; children who are vaccinated at sick visits have been shown to be less likely to attend subsequent WCVs.22,23 Also, although the AAP recommends WCVs at both ages, it is possible that providers recommend their patients attend only 1 given the immunization schedule.
Yet, most children in this cohort attended either the 15- or 18-month visit. The implications of missing 1 of these 2 visits are unclear. On one hand, because 15- to 18-month-old children begin to expand their vocabulary and become increasingly ambulatory, attending both WCVs at these ages may be critical in identifying speech and motor delay, the prevalence of which tends to be higher in low-income populations. Indeed, 1 study revealed that children who attended AAP-recommended WCVs were diagnosed with autism spectrum disorder 1.6 months earlier than children who received no well-child care.24 On the contrary, 2 recent US Preventive Service Task Force reviews included “I” recommendations (suggesting insufficient evidence) for language and autism screening in children who are asymptomatic.25,26 Therefore, to what degree attendance of these toddler WCVs affects these conditions remains unclear.
Low attendance at WCVs between 3 and 5 years of age may reflect the fact that parents are waiting until 5 years of age to bring their children in for their “kindergarten” visit for completion of requirements needed for school entry (eg, immunizations). Like the toddler visits, the 4-year visit is uniquely positioned for providers to assess key developmental milestones. For instance, this visit may be the best opportunity for a provider to assess school readiness and address any emerging behavioral problems that may impact school performance.
There also may be external forces, such as insurance coverage and quality measures, by which certain WCVs are emphasized and therefore affect adherence rates. One study revealed that each uninsured month is associated with a 3% decrease in the number of WCVs attended.27 The National Committee for Quality Assurance has included WCVs in 2 Healthcare Effectiveness Data and Information Set measures that are used to assess and compare the quality of care across health care institutions: (1) whether a child completes 6 WCVs before 15 months of age and (2) the number of WCVs between 3 and 6 years of age.28 There is also a third relevant Healthcare Effectiveness Data and Information Set measure that is used to assess the number of recommended vaccinations by a child’s second birthday.29 A health system’s quality rating often depends on how well it is performing on these measures8; for some, there are also implications for financial reimbursement related to the impact on value-based payments. To improve WCV attendance, it may be worth considering increasing the target number of WCVs. Most of the clinics included in OCHIN are FQHCs. The Health Resources and Services Administration has specific reporting requirements for FQHCs that include immunization targets but not expectations for the number of WCVs.30
The range of overall attended WCVs was 52% to 77%, spanning the 61% compliance ratio found in the previous MEPS study.4 The relatively large range (when assuming no subsequent outside care and when assuming complete subsequent outside care) reflects a limitation in our study. Using EHR data, we could not distinguish between patients who simply missed a WCV from those who missed a WCV because they received their health care elsewhere. However, our approach of calculating adherence using 2 assumptions (all versus none of the subsequent WCVs were attended outside of the network after the last recorded WCV) is also a strength of our study because true adherence likely falls within this range. Furthermore, the relatively high WCV attendance at 5 years suggests that patients within this cohort did not in fact move out of the health care system. Previous studies have also revealed that families that are publicly insured or uninsured have limited choices about where they can receive care, so there is a lower likelihood of them receiving care outside of these safety net networks.31–33
Another limitation is that we used unique eligibility periods to describe patterns of attendance for each WCV, which may yield an underestimation of WCV attendance for particular ages. It is possible that certain groups may be more likely to attend WCVs early or late, and we may have differentially underestimated attendance in these groups. Other studies have accounted for this problem by using average compliance ratios over larger time frames,4 but this methodology cannot be used to address the unique question that we attempted to answer: exactly which WCVs children are missing.
Conclusions
In this large cohort of children who were primarily publicly insured or uninsured, we found that visits at 15 months, 18 months, and 4 years were the most frequently missed. Deficiencies in these visits may negatively impact the provider’s ability to detect motor and speech delay, assess school readiness, and address early behavioral problems. There may be opportunities to improve these WCVs at the level of the family (emphasizing the importance of WCVs that include fewer vaccinations and assisting with transportation and child care), the health system (appointment reminders and case management), and nationally (refining quality metrics to incentivize alternative WCVs or full rather than partial fulfillment of WCVs). Given that 42% of children nationally have public insurance and 5% are uninsured,33 more research is needed to understand how to best promote WCV attendance among children who are publicly insured and uninsured.34
Acknowledgments
We thank the patients and their families for participating in the study. We also thank Rebecca Etz, Martha Gonzalez, Paulette Kashiri, Teresa Day, Julia Rozman, and Stephen Rothemich for their contributions to the project.
Footnotes
- Accepted August 9, 2018.
- Address correspondence to Elizabeth R. Wolf, MD, MPH, Department of Pediatrics, Children’s Hospital of Richmond at Virginia Commonwealth University, 1000 E. Broad St, Richmond, VA 23219. E-mail: elizabeth.wolf{at}vcuhealth.org
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Funded by a Bright Futures Young Investigators Award from the Academic Pediatric Association and Maternal and Child Health Bureau. The research consortium and data for this study were made available through funding from the National Cancer Institute (1R01 CA166375-01A1). Dr Krist was funded through the National Center for Advancing Translational Sciences (UL1TR000058). Dr DeVoe received funding from the Agency for Healthcare Research and Quality (R01 HS024270).
POTENTIAL CONFLICT OF INTEREST: Dr Krist is a member of the US Preventive Services Task Force, but this article does not necessarily represent the views and policies of the US Preventive Services Task Force; the other authors have indicated they have no potential conflicts of interest to disclose.
References
- Copyright © 2018 by the American Academy of Pediatrics
In this issue
Jump to section
Related Articles
Cited By...
- Reach Out and Read and developmental screening: using federal dollars through a health services initiative
- Life Course Health Development in Pediatric Practice
- Well-Child Care Attendance and Risk of Asthma Exacerbations
- Well-Child Care Adherence After Intrauterine Opioid Exposure
- Caregiver and Clinician Perspectives on Missed Well-Child Visits