PEDIATRICS Vol. 99 No. 5 May 1997, pp. e3 (doi:10.1542/peds.99.5.e3)
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow P3Rs: Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when P3Rs are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow E-mail this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My File Cabinet
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via CrossRef
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Suarez, L.
Right arrow Articles by Smith, D. R.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Suarez, L.
Right arrow Articles by Smith, D. R.
Related Collections
Right arrow Infectious Disease & Immunity

PEDIATRICS Vol. 99 No. 5 May 1997, p. e3
Copyright ©1997 by the American Academy of Pediatrics

ELECTRONIC ARTICLE:
Errors and Correlates in Parental Recall of Child Immunizations: Effects on Vaccination Coverage Estimates

Lucina Suarez*, Diane M. Simpson*, and David R. SmithDagger

From the * Associateship for Disease Control and Prevention, Texas Department of Health, Austin, Texas; and the Dagger  Office of the Commissioner of Health, Texas Department of Health, Austin, Texas.

ABSTRACT
INTRODUCTION
METHODS
RESULTS
DISCUSSION
ACKNOWLEDGMENTS
ABBREVIATIONS
REFERENCES

ABSTRACT

Objective.  We evaluated the accuracy of parental recall of children's immunization histories as compared with provider records and examined how errors in parental recall correlate with sociodemographic characteristics.

Design.  The validation study was part of a population-based household survey designed to assess immunization levels among Texas children under age 2 years. For 72% (n = 3278), interviewers used vaccination records from the parent to copy dates for the diphtheria and tetanus toxoids and pertussis vaccine (DTP), oral polio vaccine (OPV), and measles, mumps, and rubella (MMR) shots. For parents without shot records (n = 1216), interviewers asked about each vaccine, whether the child had received the shot, how many, and at what age. Of these, 85% (n = 1029) were validated with health provider records.

Results.  Measured against provider records, only 34% of parents accurately recalled the number of DTP shots a child had. More often (42%) parents underestimated the number of DTP shots than overestimated (24%). Agreement between parental recall and provider records was high (83%) for the single dose of MMR. Accuracy of parents' recall did not differ by race/ethnicity, education level, or type of health insurance coverage, but decreased as child's age increased. Having a vaccination record at home was associated with a higher immunization status. Hispanic, lower educated, and uninsured parents were more likely to have a vaccination record than non-Hispanic, higher educated, and privately insured parents.

Discussion.  Validity of parental recall of children's immunization histories depends on the vaccine and the age of the child, which is highly correlated with the number of shots parents must recollect. Results suggest that inclusion of parent recall information from vaccination surveys underestimates DTP:OPV:MMR coverage. This underestimation is consistent across economic and race/ethnic groups. Thus, community surveys based on cards and recall should provide reliable conclusions about which groups need intensive program efforts. For the routine monitoring of vaccination coverage, reasonable estimates can be obtained by combining parent-held record and parent recall data. Caution is required when comparing coverage estimates from different surveys since the source of information and method of derivation will produce widely varying coverage rates. vaccinations, immunization, validity, recall bias, infant.

INTRODUCTION

The public health initiative to get 90% of the nation's children adequately immunized before their third birthday1 has focused much attention on measurement issues.2 Although federal funding to states for immunizations is tied to the achievement of prescribed levels, state and local immunization programs have few options to gauge community vaccination coverage. Computerized tracking systems in pediatric clinics are expensive and do not include all children in a community. Retrospective surveys of the school records of children are less costly and are population-based, but their weakness is that estimates of coverage are several years out-of-date. Thus, population surveys of immunization coverage may be the best means for assessing needs, setting goals, and measuring progress of immunization initiatives.2 One problem with community immunization surveys is that parental responses may be inaccurate. Concern about the validity of parental recall is reasonable since childhood immunization schedules are complicated; children routinely receive 9 to 10 different antigens during four or more visits.3 Consequently, verification of parental recall with provider records is desirable, but makes community surveys even more costly and time consuming.

Few studies have evaluated the accuracy of parental recall of their child's immunization history.4 Most studies are from other countries4,11 and only one examined factors related to accuracy.8 Some studies6,7 reported errors of overestimation in recalling measles vaccine; others report that mothers underestimated their child's immunizations8 or that mothers generally gave accurate assessments.4 Two studies of children presenting to United States emergency rooms reported that parental information was too inaccurate for making decisions on whether to immunize.9,10

Using a statewide household survey, we evaluated the degree of accuracy of parental recall of children's immunization histories as compared with provider records. In addition, we report how errors in parental recall correlate with sociodemographic characteristics. Also, we discuss the effect of errors on vaccination coverage estimates and the usefulness of information from parent-held vaccination records, parental recall, and provider records.

METHODS

Survey

The validation study was part of a population-based household survey designed to assess immunization levels among Texas children under age 2 years. The statewide survey consisted of samples from 30 counties; 10 counties were preselected because of their population size or race/ethnic composition. To represent the remainder of the state, 20 counties were randomly chosen with selection probabilities proportional to the number of births. For all but four counties, we used a three-stage sampling process involving census block groups, blocks, and housing units. In the first stage, a simple random sample (SRS) of 40 census block groups was obtained for each county. In the second stage, a SRS of at least 50% of the blocks was selected from each block group; and in the third stage, a SRS of all housing units up to 20 was chosen from each block. In four sparsely populated counties, we changed the design to reduce costs and increase efficiency by randomly selecting children from county birth records.

Each selected housing unit was screened to find a child age 2 to 24 months living in the household. At least three attempts were made to contact a member of the selected household. If there was no eligible child in the household, the interviewer asked if an adjacent household had a child 2 to 24 months of age. If there was, an interview was conducted in the adjacent housing unit; if not, a replacement housing unit was randomly drawn. In households with more than one child under 2 years, one child was randomly selected for study.

Trained interviewers used a standardized English- or Spanish-language instrument to solicit information from a parent or legal guardian. The instrument was modeled after the Centers for Disease Control and Prevention State and Local Area Immunization Coverage and Health Survey questionnaire.11 To determine a child's immunization history, interviewers obtained shot records from the parent and copied dates for each vaccine including diphtheria and tetanus toxoids and pertussis vaccine (DTP), oral polio vaccine (OPV), and measles, mumps, and rubella (MMR) shots. For parents without shot records, interviewers asked about each vaccine, whether the child had received the shot, how many, and at what age.

Over 116 000 households were screened to find children of the appropriate age for inclusion in the survey. A total of 4832 interviews with parents/guardians was completed during June through November 1994. Respondents were paid $10 for a completed face-to-face interview. There were 311 refusals from households with eligible children and 1038 refusals in which it was unknown whether an eligible child resided in the household. A conservative estimate of the response rate is 78% (4832/4832 + 1038 + 311).

Consistent with the recommendations of the Advisory Committee on Immunization Practices of the Public Health Service,3 we considered children up-to-date if they were: 1) age 3 to 4 months and had received one DTP and one OPV vaccine; 2) age 5 to 6 months and had received two DTP and two OPV vaccines; 3) age 7 to 15 months and had received three DTP and two OPV vaccines; and 4) age 16 to 24 months and had four DTP, three OPV, and one MMR vaccine. For study of vaccination coverage, we restricted the sample to children age 3 to 24 months (n = 4552).

Validation Study

Among children 3 to 24 months, 72% (n = 3278) of parents had a vaccination record in the home. Interviewers had to obtain vaccination histories by parental recall for 27% of the respondents (n = 1216). To validate immunization histories taken by parental recall, interviewers asked where vaccinations were obtained, the health provider name, and health care facility. Parents who said their children had never been vaccinated were not asked for their provider's name and were excluded from the validation process (n = 58). Mailed requests for the child's vaccination records were sent to health providers, after obtaining a signed medical records release from the parent. Written requests were followed up with telephone calls. Providers were located and contacted for 1029 (85%) parents who recalled their child's immunization histories. In the remaining 187 cases, most could not be located based on information given by the respondent, or in a few instances providers refused requests for information.

Analysis

In this article, we report data on the validity of parental recall when compared with provider records using the 1029 cases in which a provider was successfully contacted. The number of shots for DTP, OPV, and MMR reported by the parent is compared with the number documented in the provider record assumed to be accurate. We measured the degree of accuracy by the percent of parents who were concordant with the number of shots in the provider record. Statistical differences in the proportion who were concordant across race/ethnic, education, and health insurance groups were assessed with a chi 2 test or chi 2 test for trends. For those in the validation sample, we also report the degree of agreement on up-to-date immunization status between recall and provider record. For this purpose, we calculated the positive predictive value as the percentage of children reported as up-to-date that were verified with provider records. Negative predictive value is the percentage of children reported as underimmunized verified as such in the provider records.

Second, to examine the variation in vaccination coverage estimates by the source of information, we will refer to three information groups formed from all survey respondents. First is the "parent-held record" group, or those with vaccination records at home; second is the "recall" group, those without home records who could recall the number of shots; and third, a "don't know" group, those without home records who were unable to recall whether their child had a shot or the number.

RESULTS

Validity of Parental Recall

Table 1 shows the agreement on the number of DTP shots between parental recall and provider records for the 1029 children age 3 to 24 months in the validation study. For 11% (n = 117) of children with no home record, the provider had no vaccination record on the child. In addition, when parents were specifically asked about their child's DTP shots, 19% of parents did not know if the child had received the shot (6%) or how many (13%). Provider records indicated that 89% of the "don't know" group had at least one shot. Excluding those without provider records and responses of "don't know", only 34% of parents accurately recalled the number of DTP shots based on provider records. More often (42%) a parent underestimated the number of DTP shots a child had than overestimated (24%). Agreement on the number of OPV shots (not shown) mirrored results on the DTP shot; the two vaccines have similar multidose schedules.

Table 1. Agreement Between Parental Recall and Provider Record on the Number of DTP Shots, Children Age 3 to 24 Months

[View Table]

Table 2 shows the agreement between parental recall and provider records for the single MMR shot for children age 16 to 24 months, when the MMR should have been received. Agreement between parental recall and provider records was high (83%). Parents more often overreported than underreported whether the child had received an MMR shot. The proportion of "don't know" responses was higher (31%) than for the DTP shot.

Table 2. Agreement Between Parental Recall and Provider Record on the Number of MMR Shots, Children Age 16 to 24 Months

[View Table]

Accuracy of parents' recall, measured by the concordance on the number of DTP shots, did not differ among race/ethnic groups, by education level, or type of health insurance coverage (Fig 1). Accuracy did decrease as the age of the child increased (chi 2 for linear trend = 44.1, P < .00001), and this was directly related to the number of shots that parents had to remember.

Fig. 1. Concordance on the number of DTP shots by selected demographic factors.
[View Larger Version of this Image (54K GIF file)]

Table 3 shows the positive and negative predictive value of parental recall. Of greater interest is the comparison of the immunization levels measured by parental recall with levels recorded by the provider. For DTP and OPV, the percent up-to-date is higher when based on provider records. For the single MMR shot, parents overreported whether their child received a dose; thus, parental recall shows a higher level of immunization. For the series, the up-to-date level based on provider records is 20 percentage points higher than what would have been obtained by parental recall.

Table 3. Percent Up-to-date* (%UTD) on Immunizations: Parental Recall vs Provider Records

[View Table]

Vaccination Cards and Immunization Status

Among all survey respondents, the proportion with vaccination records in the home varied with race/ethnicity, education level, and health insurance coverage but not with the child's age. Eighty percent of Hispanic parents held a vaccination record, far more than Anglos (65%) or African-American parents (61%), (chi 2 = 144.0, df = 2, P < .000001). The higher the education level, the less likely the parent had a vaccination record in the home (chi 2 for linear trend = 67.8, P < .00001). The uninsured were more likely to have a vaccination record (78%) than parents with private health insurance (66%) or Medicaid (73%), (chi 2 = 59.3, df = 2, P < .000001). In multivariate analysis, these factors (Hispanic ethnicity, education, uninsured status) remained independent and significant predictors of having a vaccination record, with ethnicity as the strongest predictor. Having a vaccination record in the home was associated with a higher immunization status (66% up-to-date). Provider record information showed that the parental recall and the "don't know" group had lower immunization levels (53% and 44% up-to-date, respectively).

DISCUSSION

The importance of accurate assessments of vaccination coverage in meeting national goals cannot be overemphasized. Under Centers for Disease Control and Prevention monitoring guidelines, state immunization programs must obtain annual estimates of vaccination coverage. To this end, states are strongly encouraged to conduct population-based telephone, household, or birth certificate follow-back surveys. These population surveys of vaccination coverage, whether in person or by telephone, will need to rely on vaccination records in the home or parents' reports or both. How data from parent-held records and parental recall are eventually used to calculate coverage estimates will have a great effect on estimates obtained. Observations from our study may offer some insight into the validity of immunization coverage estimates from survey information. We can comment on the direction of parental recall bias and the effect of this bias on coverage estimates.

Direction of Bias

We found that the validity of a parent's recall of their child's immunization history depended on the vaccine and the age of the child, which is highly correlated with the number of shots parents must recollect. Parents overreported if their child had received the one recommended dose of MMR, but tended to underestimate the total number of doses of DTP and OPV. This error in recall has been demonstrated in two studies outside the United States.5,8 In a population study of 1171 Costa Rican mothers, accuracy also was related to age of the child; that is, the larger the number of doses, the more the mother underestimated.8 A study among children seen at a New South Wales, Australia, hospital also found that parents recalled fewer doses of DTP or OPV than were recorded in the medical records.5 As in the Costa Rican study, we were reassured that this bias was nondifferential; that is, the accuracy of parental recall was not related to demographic or economic characteristics of the mother. Thus, surveys based on parental recall will reliably assess the relative immunization status of various population groups, an important aspect for targeting community efforts.

Effect of "Don't Know" Responses on Coverage Estimates

When relying on parental recall, the treatment of the inevitable missing information is an important issue. A study of parents attending a pediatric emergency room showed that the frequency of "don't know" responses to immunization questions depended on the wording.9 General questions such as "has your child received all, some, or none of their immunizations" generated fewer "don't knows" than specific questions about the number of shots for specific vaccines. We found this to be true in our survey. Asking parents without vaccination records about each vaccine, the number of doses, and age at each dose resulted in missing information for 29%. When asked a general question (has your child received all shots needed?), 98% of parents gave a definite yes or no answer. But on further examination, we found that this general query was no better in its positive predictive value (65%) than the specific queries (68%). Moreover, parents overreported an up-to-date immunization status (75%) in response to this question when compared with provider records (57%). Within a clinic setting, parental recall is not a satisfactory method of determining if a child presenting for medical care is up-to-date.

With regard to measuring population coverage, assumptions about the immunization status of the "don't know" group will have some effect on coverage estimates. The proportion of missing information in our survey was not large (8% for DTP:OPV:MMR series) when information from cards and recall was combined. In a previous study,12 we calculated DTP:OPV:MMR coverage rates based on all information collected, including parent-held vaccination records and parental recall. Classifying children whose parents cannot recall their immunization history as underimmunized resulted in a state coverage estimate of 55%. Our validation data showed that about half of the unknown status group is up-to-date when medical records were examined. If we had excluded this unknown group, the coverage estimate would have been 60%, much closer to the true rate. Thus, in surveys where validation with provider records is not possible, excluding this group from calculation should result in more accurate coverage estimates.

Effect of Source of Information on Coverage Estimates

Goldman's careful study of immunization coverage in Guatemala showed that vastly different estimates could be derived by varying the method of calculation and the source of data.13 The use of vaccination records alone, discarding responses without a written record, seriously overestimates coverage in the general population.13 Our study shows that having a vaccination record in the home is related to a higher immunization status and is strongly associated with race/ethnicity, education, and health insurance. Thus, coverage studies using written records alone would result in biased estimates for subpopulation groups.

Our validity results show that the effect of using parental recall information will be to underestimate DTP:OPV:MMR coverage. But does supplemental provider record information significantly improve the coverage estimate from surveys of parents? In this state survey of vaccination coverage, the additional cost of retrieving provider records for parents who did not have vaccination records in the home was considerable. Replacing parental recall with provider record information increased the estimate from 60% to 62%. The lack of any significant change in the DTP:OPV:MMR coverage estimate is probably due to the high proportion of subjects with written records. Since provider record verification was a costly endeavor, the improvement in accuracy may not have been worth the effort. For the routine monitoring of vaccination coverage, we believe that reasonable estimates can be obtained by combining parent-held record and parent recall data.

In conclusion, our results suggest that inclusion of parent recall information on child vaccinations slightly underestimates DTP:OPV:MMR coverage. This underestimation is consistent across economic and race/ethnic groups. Thus, community surveys based on cards and recall should provide reliable conclusions about which groups need intensive program efforts. Caution is required, however, when comparing coverage estimates from different surveys since the source of information and method of derivation will produce widely varying coverage rates.

FOOTNOTES

Received for publication Mar 4, 1996; accepted May 23, 1996.

Reprint requests to (L.S.) Associateship for Disease Control and Prevention T-406, Texas Department of Health, 1100 West 49th St, Austin, TX 78756.

ACKNOWLEDGMENTS

We thank James Dyer, PhD, Craig Blakely, PhD, and their staff at the Public Policy Research Institute at Texas A&M University who conducted the survey and provided consultation on the survey design. We also thank, in particular, Radmila Prislin, PhD, who reviewed the manuscript and offered valuable suggestions.

ABBREVIATIONS

SRS, simple random sample. DTP, diphtheria and tetanus toxoids and pertussis vaccine. OPV, oral polio vaccine. MMR, measles, mumps, and rubella.

REFERENCES

  1. Robinson CA, Sepe SJ, Lin DFY The president's child immunization initiative---a summary of the problem and the response. Public Health Rep. 1993; 108:419-425 [Medline][Medline]
  2. Cutts FT, Zell ER, Mason D, Bernier RH, Dini EF, Orenstein WA Monitoring progress toward US preschool immunization goals. JAMA. 1992; 267:1952-1955 [Medline][Abstract]
  3. Centers for Disease Control and Prevention. General recommendations on immunization: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR. 1994;43(No. RR-1):1-38
  4. Gareaballah ET, Loevinsohn BP The accuracy of mothers' reports about their children's vaccination status. Bull World Health Organ. 1989; 67:669-674 [Medline][Medline]
  5. Ferson MJ Immunisation state and its documentation in hospital patients. Arch Dis Child. 1990; 65:763-767 [Medline][Abstract]
  6. Hawe P, Wilson A, Fahey P, The validity of parental report of vaccination as a measure of a child's measles immunisation status. Med J Aust. 1991; 155:681-686 [Medline][Medline]
  7. McKinney PA, Alexander FE, Nicholson C, Cartwright RA, Carrette J Mothers' reports of childhood vaccinations and infections and their concordance with general practitioner records. J Public Health Med. 1991; 13:13-22 [Medline][Abstract/Free Full Text]
  8. Valadez JJ, Weld LH Maternal recall error of child vaccination status in a developing nation. Am J Public Health. 1991; 82:120-122[Abstract/Free Full Text]
  9. Goldstein KP, Kviz FJ, Daum RS Accuracy of immunization histories provided by adults accompanying preschool children to a pediatric emergency department. JAMA. 1993; 270:2190-2194 [Medline][Abstract]
  10. Joffe MD, Luberti A Effect of emergency department immunization on compliance with primary care. Pediatr Emerg Care. 1994; 10:317-319 [Medline][Medline]
  11. Centers for Disease Control and Prevention. State and Local Area Immunization Coverage and Health Survey. SLICHS Questionnaire. Atlanta, GA; Centers for Disease Control and Prevention; April-June, 1994. Paper version Q2-94 Final
  12. Simpson D, Suarez L The immunization status of Texas children age 3-24 months: results of the 1994 Texas immunization survey. Tex Med. 1996; 92:60-65 [Medline][Medline]
  13. Goldman N, Pebley AR Health cards, maternal reports and the measurement of immunization coverage: the example of Guatemala. Soc Sci Med. 1994; 38:1075-1089 [Medline]
Pediatrics (ISSN 0031 4005). Copyright ©1997 by the American Academy of Pediatrics



This article has been cited by other articles:


Home page
 Arch Pediatr Adolesc MedHome page
D. D'Souza-Vazirani, C. S. Minkovitz, and D. M. Strobino
Validity of Maternal Report of Acute Health Care Use for Children Younger Than 3 Years
Arch Pediatr Adolesc Med, February 1, 2005; 159(2): 167 - 172.
[Abstract] [Full Text] [PDF]

Home page
 Arch DermatolHome page
M. E. Ming, R. M. Levy, O. J. Hoffstad, J. Filip, P. A. Gimotty, and D. J. Margolis
Validity of Patient Self-reported History of Skin Cancer
Arch Dermatol, June 1, 2004; 140(6): 730 - 735.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow P3Rs: Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when P3Rs are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow E-mail this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My File Cabinet
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via CrossRef
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Suarez, L.
Right arrow Articles by Smith, D. R.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Suarez, L.
Right arrow Articles by Smith, D. R.
Related Collections
Right arrow Infectious Disease & Immunity