 |
|
 |
|
 |
 |
|
 |
 |
 |
 |
 |
 |
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
From the National Immunization Program, Centers for Disease Control and Prevention, Atlanta, Georgia
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Objective. To evaluate the validity of the proposed simplified method for assessing immunization status in a nationally representative population of infants and children who had ever been enrolled in WIC before 35 months old.
Methods. This was a cross-sectional analysis of the 2000 National Immunization Survey representing children ages 3 to 24 months who had ever been enrolled in WIC. For the 6277 children in the study population, we compared personal records of completion status for DTaP with personal records of completion status for all immunizations appropriate for age in the combination 4:3:1:3 schedule to see which of the 2 (single vs multiple screening) methods would better predict the child’s true (provider-reported) status for the 4:3:1:3 series. The main outcome measures were the comparative sensitivity, specificity, and overall test efficiency of the 2 methods in correctly identifying underimmunized WIC children.
Results. Completion status for DTaP was less sensitive than completion status for all vaccinations in correctly identifying truly underimmunized children (sensitivity = 70% and 77%, respectively). However, it was more specific in correctly identifying children who were truly UTD for age (specificity = 86% and 82%, respectively). The 2 methods were essentially identical with respect to overall test efficiency (82% and 81% for DTaP assessment and assessment of all vaccines, respectively).
Conclusions. Given limited resources to do immunization screening and referral in nonmedical settings such as WIC, simplifying the process by using DTaP from the personal vaccination record as a proxy for the 4:3:1:3 series is a viable option. Loss in sensitivity may well be offset by gains in the capacity of WIC clinics to screen more children.
Key Words: immunization • Women • Infants and Children • evaluation • underserved • sensitivity • specificity • policy • evaluation • preventive services
Abbreviations: WIC, Special Supplemental Nutrition Program for Women, Infants, and Children • DTaP, diphtheria-tetanus-acellular-pertussis • MMR, measles-mumps-rubella • Hib, Haemophilus influenzae type b • NIS, National Immunization Survey • UTD, up-to-date
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Despite increases in national childhood immunization coverage since the measles resurgence of 1989–1991, preschool children in poverty continue to be significantly underimmunized.2 To reach these children, state immunization programs collaborate with the US Department of Agriculture’s Special Supplemental Nutrition Program for Women, Infants, and Children (WIC), which is the single largest point of access to economically disadvantaged preschoolers.3
Vaccination-promoting initiatives in WIC clinics have improved completion rates for the "4:3:1:3" series of childhood immunizations, which includes 4 doses of diphtheria-tetanus-acellular-pertussis (DTaP), 3 of polio, 1 of measles-mumps-rubella (MMR), and 3 of Haemophilus influenzae type b (Hib) vaccines administered within recommended age ranges and completed by 19 months old.4–6 WIC clinics assess children’s vaccination progress by reviewing the handheld record brought in by the parent. Children with missing doses (immunization delays) are referred to a vaccine provider, ideally to the child’s medical home.
This complex and time-consuming process is 1 of many challenges that WIC programs have faced in providing nutrition services and, at the same time and without additional funding, functioning as the gateway to a variety of health and welfare programs that, over the past decade, have undergone considerable change.7 In 2001, realizing that taking the time to review all vaccines listed on each child’s record may exceed the capacity of many WIC clinics, an interagency team of experts from national immunization and WIC programs proposed a less time-consuming method for assessing vaccination status. By this method, WIC staff will assess completion, or up-to-date (UTD) status for the DTaP vaccine as a proxy for UTD status of all vaccines in the 4:3:1:3 schedule. That is, instead of reviewing all documented vaccinations in the child’s record, WIC clinic staff may elect to review only the number of DTaP doses received. The child would be classified as being UTD for age for the entire 4:3:1:3 series if the record shows that he or she is UTD for age for DTaP. Only those children late for a DTaP dose would be referred to an immunization provider.
It is not known whether these time-saving benefits outweigh the risks of failing to identify children who lack doses, or of inappropriately referring children for vaccines who are already UTD. A condition of the proposed new procedure is that all assessments be based on a printed document of the child’s vaccination history rather than on parental recall. However, because the accuracy of handheld records presented by parents has not been consistently demonstrated, relying on these records may also introduce error.
In this study, we used data from the 2000 National Immunization Survey (NIS), which includes children enrolled in WIC as well as those not enrolled. For enrolled WIC children, we determined whether age-specific UTD status as shown on the parent-held vaccination record accurately reflects physician-verified UTD status at 6 milestone ages for the 4:3:1:3 combination schedule. Furthermore, we evaluated the extent to which using a single vaccine (DTaP) to represent multiple vaccine completion status would correctly identify children in WIC who were not UTD as well as those who were on schedule. Finally, we compared the Hib and polio vaccines with the DTaP vaccine as potential proxy predictors of true status.
|
METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
We included in our analysis the subset of year 2000 NIS records for which 1) the child’s WIC participation status was determinable; 2) parental response to the immunization history component of the interview was based on a documented vaccination record rather than on recall alone; and 3) complete provider-reported vaccination history data existed as part of the record.
Classification of Age-Specific UTD Status
We classified each study participant’s immunization completion status using guidelines established by the Advisory Committee on Immunization Practices (Table 1). These guidelines describe the types of vaccines to be given, the number of doses required, and the age recommendations for each dose in the 4:3:1:3 series.11
|
According to the proposed new method, if the parent does not provide a documented record of the child’s vaccination history, WIC staff members will not attempt to classify the child’s immunization completion status at that visit. Correspondingly, in our study when the data necessary to calculate age-specific UTD status were missing from the NIS household record, we classified household-reported status as indeterminate at that age.
Statistical Analyses
We calculated the sensitivity, specificity, and test efficiency of UTD status from parent-held records in predicting true immunization status as reported by providers. In keeping with a risk-assessment approach, the condition screened for was immunization delay. Therefore, the sensitivity of each of the 2 household-based tests was defined as the percentage of truly immunization-delayed children according to provider records who were correctly classified as delayed by the test. Specificity was defined as the percentage of children who were truly UTD and were classified as such by the test. Test efficiency was defined as the percentage of times the screening test gave the correct answer, whether positive or negative, out of all times the test was applied. Nationally representative population estimates of immunization completion rates as well as sensitivity, specificity, and test efficiency were analyzed with the SAS-callable SUDAAN crosstab procedure for complex survey designs (Research Triangle Institute, Research Triangle Park, NC).
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Figure 1 shows differences in coverage rates at milestone ages between WIC and non-WIC children. These differences were statistically significant at every age (P < .0001), and were consistently lower for WIC participants. Timely completion for both groups is highest at age 3 months, decreased steadily across the milestone ages through the first year, then increased through the second year so that by the beginning of the third year coverage levels were almost as high as they were at the 3-month milestone.
|
|
Like specificity, test efficiency tends to be higher for the DTaP predictor. On average, use of either the DTaP series or the entire 4:3:1:3 series for screening will result in accurate classification of UTD status 
81% to 82% of the time. Test efficiency, like specificity, is fairly uniform across the milestone ages but tends to be highest for assessments of children <13 months of age. Disparity in the efficiency of the 2 household tests is greatest at 24 months old.
The accuracy of the 2 household-based tests in predicting true UTD status varies widely across the milestone ages. Sensitivity is highest for both tests at 7 months old, and lowest at 24 months old. Specificity is highest at 19 months old for the DTaP-based test and at 24 months old for the 4:3:1:3-based test. Specificity is lowest for both tests at 13 months.
Comparison of DTaP With Other Antigens
The Hib series was a more sensitive predictor of provider-reported UTD status than both DTaP and polio. However, its specificity and overall test efficiency were poor relative to both of the other vaccines. Polio had very high specificity, but poor sensitivity (Table 2).
|
15% at both ages when the DTaP-MMR combination is used, and overall test efficiency declined by 8% as well. |
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Although the finding is not new that children participating in WIC remain less well-immunized than children never participating in WIC, our results show that this finding holds true at all milestone ages from 3 to 36 months and is most pronounced at 5 and 7 months of age. This, together with our observation that timely completion is highest at 3 months old and then decreases steadily across the milestone ages through the first year before increasing during the second year, has special significance for WIC clinics, where 88% of infants are <3 months old at the initial visit and >33% of enrolled children are <1 year old.12 It suggests that opportunities for identifying underimmunized children will be greatest between the ages of 7 and 19 months. WIC thus remains a good capture point for underimmunized poor children in this country, and is especially important for vaccination promoting interventions being conducted in WIC during the first year of life, when immunization status is even more discrepant between WIC and non-WIC, and before the large dropout (30%) from the WIC program, which occurs between the first and second year of life.3
Deliberations about the use of a single vaccine to represent true UTD status for all antigens included considerations about which 1 of the 4 might have the best predictive validity. Studies had not yet been performed to provide empirical evidence for this decision, but the DTaP series seemed a good candidate because it has been a long-established component of the 4:3:1:3 series; it is typically initiated at the same time for all infants regardless of birth weight, disease history, or other potential contraindications; and the spacing of the sequential components of the 4-dose series affords an opportunity to check immunization status periodically throughout the months of infancy and early childhood when WIC services are most often sought. However, recent shortages in the supply of DTaP vaccine highlight a potential problem for this method. Although these shortages occurred after the 2000 NIS survey and thus did not affect the results of this study, the possibility of future shortages suggests that other vaccines should be evaluated in terms of their usefulness as markers for UTD status.
We found that Hib is a more sensitive predictor of true UTD status than DTaP, but its specificity and test efficiency are lower. Moreover, assessments based on this antigen would be complicated by the fact that the dosage schedule depends on the type of Hib vaccine given: some products require a 3-dose series and others, a 4-dose series. For these reasons we believe that Hib is not a viable candidate vaccine to use in a simplified assessment procedure.
The DTaP-MMR combination for children ages 19 and 24 months certainly deserves further consideration. Most children have dropped out of WIC participation by these ages; the addition of a single vaccine to the screening procedure for the small number of children remaining in WIC may make a slightly more complex assessment feasible.
We did not include hepatitis B vaccine in our assessments because the timing of the initial dose differs depending on the physician’s preference and hospital convention. For most infants, the initial dose is given shortly after birth while the infant is still in the hospital nursery; but for others, the first dose is delayed. Consequently, although some of these initial hepatitis B vaccinations become part of the outpatient immunization providers’ records, many of them are retained in the hospital record and not readily available for routine outpatient chart review. The pneumococcal conjugate vaccines were not part of the routine immunization schedule for our study population, but given their importance in the current schedule, we will monitor the extent to which they track with the DTaP series and will evaluate their suitability as a potential alternative representative of age-specific completion status for all vaccines in the routine schedule. Meanwhile, for all ages combined and considering the 3 statistical measures of sensitivity, specificity, and efficiency together, DTaP appears, at least for now, to be the most reliable of the 3 candidate vaccines as a proxy measure of true UTD status.
The finding that both the DTaP count and the 4:3:1:3 count were imperfect predictors of true 4:3:1:3 status has important implications for clinical practice. It most likely results from administered vaccines not being recorded appropriately on the child’s personal vaccination record. From our analysis we know that the personal record will be insensitive to true immunization delay whenever the child has completed the DTaP series but missed 1 or more of any of the other 3 vaccines, ie, either polio, Hib, or MMR. Conversely, it will be nonspecific to true 4:3:1:3 status whenever the child is UTD for all 4 vaccines and any of them are missing from the child’s personal vaccination record. As the child progresses through the schedule of immunizations during the first 18 months of life, problems associated with incomplete updating of the personal vaccination record become more pronounced. Therefore, professional organizations who contemplate using single vaccine screening methods such as the one we describe will need to place more emphasis on accuracy of these records and to encourage both parents and providers to make sure that they are updated with the receipt of each vaccine.
Comparisons With Previous Studies
For preschool children 19 to 35 months of age, 1999 NIS analysis found that when the personal vaccination record shows the child to be UTD then there is agreement with the provider records 
90% of the time; if the personal vaccination record indicates that the child is not UTD, then agreement with the provider is 
29%.13 Our findings are similar for the subset of children that we studied. However, ours is the first study to evaluate the correspondence between household and provider reports at milestone ages and the accuracy of counting DTaP as a proxy for the combination 4:3:1:3 series.
Limitations
The study is limited by the process used by the NIS to collect immunization histories, which relies not on inspection of the personal vaccination record but rather on parents’ "reading" of the immunization dates over the telephone, which may result in some loss of accuracy. This highlights the fact that in the applied setting, possession of a personal vaccination record does not necessarily mean that this record will be brought to all provider visits, especially visits to providers of nonmedical services such as WIC. Accuracy of the single-vaccine model that we evaluated is dependent on the extent to which the parent or caregiver presents a complete and UTD record for review. As the childhood vaccination schedule becomes more complex with the addition of new vaccines, future research in single-vaccine screening methods might assess sensitivity and specificity associated with links to immunization registries in comparison to those associated with personal vaccination records.
Policy Implications
Our results suggest that given limited resources to conduct immunization screening and referral in nonmedical settings such as WIC, simplifying the process by using DTaP as a proxy for the combination series is a viable option. When compared with screening methods that involve assessing all doses of all antigens, this method is less sensitive in identifying children who are in need of vaccinations but more specific in that fewer children would be referred to a provider for vaccinations already received. It is important to recognize the need to balance sensitivity and specificity in weighing the merits of this method. Optimizing sensitivity may lead to the identification of more children with vaccine deficiencies, but if this is achieved at the cost of reduced specificity, other problems ensure. In particular, as the link between WIC clinics and the health care system has become weakened, the need to maintain good communications with private-sector immunization providers has become more important. Inappropriately referring fully immunized children to these providers compromises the credibilty of the WIC clinic staff and places the child at risk for receiving duplicate vaccinations.
This assessment model may be useful for programs serving populations that are at risk not only for delayed immunizations but also other preventive services, such as vision and hearing screening. Reports that underimmunized inner-city preschoolers are significantly more likely to be anemic and to have elevated lead levels than their UTD counterparts confirm that these children are at risk for more than just vaccine-preventable diseases.14 Programs such as Medicaid and Housing and Urban Development that, like WIC, are in regular contact with low income, inner-city children, may have particular interest in this streamlined approach to immunization screening. Because it is a practice that can lead to the identification of other pediatric primary care conditions, it offers the potential for improving the overall health status of these children.
We endorse the use of the single vaccine screening method only by nonmedical providers. Although their contact with children at risk for immunization delay is often more frequent and predictable than that of medical service providers, their objectives are different. In nonmedical settings such as WIC clinics, immunization screening activities are seen as ancillary to the primary mission of the organization; the focus of these activities is to make appropriate referrals to vaccine providers. By contrast, because immunization screening activities in medical settings are directly related to the organization’s mission to provide preventive health care services, the objective is assessment and administration of needed vaccines. Comprehensive immunization screening more likely represents standard operating procedure in medical settings, where staff are equipped with the knowledge and the resources to conduct professional assessments. For these reasons, in our study we modeled the medical service provider as the "gold standard" of knowledge of true immunization status and of responsibility for assuring appropriate standards of medical care.
WIC programs are currently directed to conduct immunization screening and referral at all WIC sites nationwide (White House Executive Memorandum, December 11, 2000). Effective October 1, 2002 (US Department of Agriculture policy memorandum, June 2001), use of DTaP alone to ascertain vaccination status has been recommended as a viable option for those WIC sites not able to conduct full antigen screening. The loss in sensitivity associated with this simplified method may well be offset by increases in the number of WIC clinics that can integrate immunization assessment as a component of routine WIC certification, with consequent gains in the number of children assessed. The findings that we report here have implications for the feasibility and the potential cost effectiveness of this simplified immunization assessment procedure, not only in the WIC clinic setting but also in other nonmedical settings such as federal and state family health and social service programs, child care centers, and schools.
|
FOOTNOTES |
|---|
Address correspondence to Donna L. Rickert, MA, DrPH, Centers for Disease Control and Prevention, National Immunization Program, 1600 Clifton Rd NE, Mailstop E-52, Atlanta, GA 30333. E-mail: DRickert{at}cdc.gov
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
||
|
||
Home | My Pediatrics | Journal Information | Current Issue | Past Issues | Subscriptions & Services | Contact Us Click here for faster international access © 2003 American Academy of Pediatrics. All rights reserved. |
||
|
||
