This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow E-mail this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My File Cabinet
Right arrow Download to citation manager
Right arrowRequest Permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via CrossRef
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Daley, M. F.
Right arrow Articles by Kempe, A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Daley, M. F.
Right arrow Articles by Kempe, A.
Related Collections
Right arrow Infectious Disease & Immunity
Right arrowRelated AAP Red Book topics:
Influenza
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Facebook   Add to Reddit   Add to Technorati   Add to Twitter  
What's this?
PEDIATRICS Vol. 113 No. 1 January 2004, pp. e26-e33

Identification and Recall of Children With Chronic Medical Conditions for Influenza Vaccination

Matthew F. Daley, MD*,§, Jennifer Barrow, MSPH§, Kellyn Pearson, MSN§, Lori A. Crane, PhD{ddagger},§, Dexiang Gao, PhD*, John M. Stevenson, MA||, Stephen Berman, MD*,§ and Allison Kempe, MD, MPH*,§

* Departments of Pediatrics
{ddagger} Preventive Medicine and Biometrics
§ Children’s Outcomes Research Program, University of Colorado Health Sciences Center and Children’s Hospital, Denver, Colorado
|| National Immunization Program, Centers for Disease Control and Prevention, Atlanta, Georgia


   ABSTRACT
TOP
 ABSTRACT
METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 
Objectives. Despite long-standing recommendations to provide annual influenza vaccination to children with chronic medical conditions, immunization rates are <10% in most primary care settings. Many obstacles impede implementation of these recommendations, including the challenge of identifying targeted children and the need to immunize yearly in a short time interval. The objective of this study was to assess the accuracy of billing data for identifying children who have high-risk conditions (HRCs) and need influenza vaccination and 2) to evaluate the efficacy of reminder/recall for children with HRCs.

Methods. The study was conducted in 4 private pediatric practices in metropolitan Denver, Colorado, that share a computerized billing system and also participate in an immunization registry. For all children aged 6 to 72 months, registry records were linked with the billing database. Patients with ≥1 encounters for an HRC in the previous 24 months were selected, with HRCs identified from International Classification of Diseases, Ninth Revision, Clinical Modification diagnostic codes. Using medical records as the "gold standard," we reviewed 327 randomly selected records to determine the sensitivity, specificity, and accuracy of billing data for identifying HRCs. For children with an HRC, we then conducted a randomized, controlled trial of reminder/recall for influenza vaccination. The primary outcome of the recall trial was receipt of influenza vaccine.

Results. Billing data had a sensitivity of 72% (95% confidence interval [CI]: 48%–95%), specificity of 95% (95% CI: 90%–100%), and overall accuracy of 90% (95% CI: 84%–96%) in determining which children had an HRC. Of the 17 273 patients aged 6 to 72 months, 2007 had ≥1 HRCs (12% overall; range: 9%–14% per practice). Asthma/reactive airways disease accounted for 87% of all HRCs. Reminder/recall significantly increased influenza immunization in children with HRCs, with a vaccination rate of 42% in those recalled, compared with 25% in control subjects. Recalled subjects were more likely to have an office visit (68% vs 60%) and less likely to have a missed opportunity to immunize (28% vs 37%) compared with control subjects.

Conclusions. Diagnosis-based billing data accurately identified children who had HRCs and needed annual influenza vaccination, and registry-driven reminder/recall significantly increased influenza immunization in targeted children.

Key Words: immunization • influenza vaccine • chronic illness • reminder/recall • immunization registries

Abbreviations: ACIP, Advisory Committee on Immunization Practices • ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification • HRC, high-risk condition • CI, confidence interval • I, intervention group • C, control group

The influenza viruses place a significant health burden on the US population, causing an average of 34 000 deaths and 114 000 influenza-related hospitalizations nationally each winter.1,2 Young children, older adults, and individuals of any age with certain chronic medical conditions are particularly at risk for complications, hospitalizations, and death during annual influenza epidemics.3–7 In a retrospective study of 20 years of Tennessee Medicaid enrollees, children with high-risk conditions (HRCs) were 2 to 4 times more likely to have an influenza-associated hospitalization than were healthy children in the same population.5,6 For children 6 months to 17 years of age, the prevalence of these conditions has been estimated at between 7% and 14%, with ~8 million children nationally at increased risk of complications from influenza infection every year.8

Vaccination is the primary means of preventing influenza infection and its associated morbidity.9 For many years, the Advisory Committee on Immunization Practices (ACIP), the American Academy of Pediatrics, and the American Academy of Family Physicians have strongly recommended influenza vaccination for anyone 6 months or older who is at increased risk of complications from influenza.9–11 Despite these long-standing recommendations, the influenza vaccination rate in children with chronic medical conditions is poor. For children with asthma in primary care settings, influenza immunization rates are typically 7% to 10%.12,13 The rate achieved in medical specialty clinics may be higher, with 25% of children with moderate to severe asthma seen in an allergy/immunology clinic and 79% of pediatric cystic fibrosis patients followed by a regional cystic fibrosis center vaccinated.14,15 A limited number of studies have tested reminder/recall in academic primary care settings for children with asthma, typically achieving influenza immunization rates of 30% to 32%.16,17 Recall of children for influenza vaccination has not been reported in private practice settings or for those with conditions other than asthma.

Efficiently and accurately identifying children with a wide array of chronic medical conditions remains a critical challenge in implementing ACIP influenza recommendations. Information gathered for billing purposes, despite documented limitations in clinical precision,18 has been used successfully to study a variety of chronic health conditions in pediatric populations.19–23 Billing databases, built on the diagnostic codes of the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM),24 have been used to determine influenza-associated hospitalization rates in children with chronic medical conditions.6,25 However, no published studies have used a comprehensive set of ICD-9-CM codes to identify and recall children with chronic medical conditions for annual influenza vaccination. The objectives of the current study were 1) to assess the accuracy of billing data for identifying children who have HRCs and need influenza vaccination and 2) to investigate the efficacy of reminder/recall for influenza vaccination of children with HRCs. Our major hypothesis was that a reminder/recall intervention would significantly increase the rate of influenza immunization in children with HRCs.


   METHODS
TOP
 ABSTRACT
 METHODS
RESULTS
 DISCUSSION
 REFERENCES
 
Study Setting and Population
The study was conducted in 4 private pediatric practices in metropolitan Denver, Colorado, from July 2002 through May 2003. These practices shared a computerized billing system and also participated in a regional immunization registry, with the registry containing records for all children <72 months of age. The study population included all children who were age 6 to 72 months, had a record in both the registry and billing databases, and had at least 1 clinic visit in the preceding 18 months. Excluded were children who had moved or gone elsewhere for care (as noted by the clinic or by billing data), had died, or had requested in writing not to participate in the immunization registry. This investigation was conducted with the approval of the Colorado Multiple Institutional Review Board, which oversees human subjects research at our institution.

The 4 study practices were selected because of their use of a common billing system and registry and their willingness to participate in immunization-related research. The pediatric sites had a range of 5 to 8 pediatricians and 2 to 6 mid-level providers per practice, and a range of 3678 to 5111 patients aged 6 to 72 months. Before this investigation, the practices did not have any formal mechanism for identifying and recalling high-risk children for annual influenza vaccination. Some providers had kept their own lists of ~10 to 25 high-risk individuals to use as a personal reminder, but the lists were not used by the practices in any systematic way.

Data Sources
Administrative/Billing Data
The billing system used by the study practices contained demographic information, including patient address and telephone numbers, for all clinic patients and was updated at the time of each patient visit. A bill was generated for each clinical encounter, with diagnoses coded using the ICD-9-CM classification system24 and immunizations recorded using Current Procedural Terminology codes.26 Insurance information was also obtained from this database, with insurance status assessed as of October 1, 2002.

Immunization Registry
In the year preceding the current investigation, all children <72 months of age in the study practices were entered into an existing regional immunization registry, using immunization data gathered from medical records and the billing database. The registry, which operates in accordance with national standards for immunization registries,27 is Internet based and compiles immunization data from several rural Colorado areas as well as other urban Denver sites. Clinic staff were instructed to enter all immunizations given, including influenza vaccine, into the registry within 24 hours of administration. On the basis of a medical record review conducted in November to December 2002, the study practices’ registry error rate, defined as the percentage of all vaccines in medical records not entered or entered incorrectly into the registry, was 14%.

Selection of ICD-9-CM Codes
To identify children with HRCs from billing data, we sought to translate accurately the published influenza immunization recommendations28 into specific diagnostic codes while also incorporating input from providers in the study practices. After developing a list of ICD-9-CM codes based on ACIP guidelines,28 we compared our codes with previously published sets of chronic disease codes, such as those developed by Neuzil et al6 and Feudtner et al.20,21 We identified several additional HRC codes by using the 3-digit "root" codes from our preliminary list to search for matching codes in a database of all clinic encounters in the previous 2 years. Once a set of codes was created, we met with providers from each study practice to develop a consensus about which conditions to include in a final list. Several practices desired to immunize children with muscular dystrophy, cerebral palsy, and Down syndrome, diseases that are not designated as high-risk in ACIP recommendations.28 However, these conditions can be associated with recurrent pneumonias or chronic pulmonary insufficiency (for muscular dystrophy and cerebral palsy)29,30 or with ill-defined immunodeficiencies (for Down syndrome)31 that may increase the likelihood of influenza-related complications; therefore, these 3 conditions were added to the HRC list.

On the basis of our discussions with study providers and a review of ACIP recommendations,28 we decided not to include the ICD-9-CM code for wheezing in the list of selected HRC codes. Because not all wheezing episodes are caused by asthma and because of the relatively frequent occurrence of wheezing associated with viral respiratory infections in young children, it can be difficult to differentiate an acute episode of wheezing from asthma in young children.32,33 Study providers stated that they primarily used the code for wheezing for children who did not meet their criteria for asthma. Providers did not consider children who had wheezed but had not received a diagnosis of asthma as high-risk patients needing annual influenza immunization, a view that is consistent with ACIP recommendations.28

Finally, to search for any systematic clinic-based coding errors, we examined the frequency of all HRC codes stratified by clinic. For 2 ICD-9-CM codes (160.1, middle ear malignancy; and 270.1, phenylketonuria), individual study sites had an apparent disease prevalence that greatly exceeded published national rates,34,35 whereas the remaining sites had no cases. We reviewed medical records for a random sample of children with these diagnostic codes: phenylketonuria (PKU) was incorrectly coded at 1 practice when providers described the 2-week-old well-child visit as a "PKU check"; in another practice, the code for middle ear malignancy was erroneously placed on preprinted billing sheets with the diagnosis "seborrheic dermatitis." These 2 codes were excluded from the final list used to identify HRCs (Table 1).


View this table:
[in this window]
[in a new window]
  		TABLE 1. ICD-9-CM Codes of HRCs Targeted for Annual Influenza Vaccination and Numbers of Children Identified

 
Medical Record Review
To determine the accuracy of billing data for HRC designation, we randomly selected 373 records of patients aged 6 to 72 months. Diagnoses written on encounter sheets, clinic flow sheets, or problem lists were extracted, along with the associated dates. Blinded to the billing code designation, 2 investigators (A.K. and M.F.D.) independently reviewed data from these chart extractions to determine whether, according to medical records, the patient had an HRC. Conditions that were diagnosed >2 years before, with no subsequent visits for those diseases, were not considered HRCs. Using medical records as the "gold standard," we calculated the sensitivity, specificity, and accuracy of the billing-based HRC designation for each practice. Because the study clinics varied in size, we then adjusted the practice-specific percentages by the relative size of each clinic, to calculate a summary sensitivity, specificity, and accuracy for the entire study population.

Reminder/Recall Intervention
Using the selected codes, we searched the billing database to identify all children aged 6 to 72 months with an encounter for an HRC in the previous 2 years. When ≥2 siblings within the same household had an HRC, 1 child was randomly selected for the recall trial and the others were excluded. Within each study practice, subjects with HRCs were assigned to intervention versus control groups by simple random allocation using SAS software (version 8.0; SAS, Cary, NC). We did not inform providers about which of their patients had been identified or recalled. However, intervention subjects may have subsequently informed providers that they had received a recall letter. We did not perform any provider-based immunization interventions (eg, education, assessment, feedback) before or during the reminder/recall trial.

The intervention group received a staged letter and postcard recall. During the second week of October 2002, all intervention subjects received a letter strongly encouraging influenza vaccination for their child, with a telephone number provided to schedule an appointment. Four weeks later, another reminder was mailed to those who had not yet been vaccinated, as determined by reviewing their influenza immunization status in the registry. The second letter emphasized that clinic records indicated that their child may have a condition that increased the risk associated with influenza infection, and vaccination would provide some protection. Four weeks after the second reminder, a postcard was sent to all unimmunized intervention subjects, stating "There is still time to protect your child against the flu!" These mailings used practice letterhead and were addressed to the parents of the study subjects. Missed opportunities were assessed by determining the proportion of subjects with ≥1 clinic visits between October 1, 2002, and January 31, 2003, who were eligible for but did not receive an influenza vaccine.

In March 2003, the influenza immunization status of each study subject was determined using the immunization registry and billing data. A randomly selected subset of those who were not immunized were telephoned in April to May 2003 to inquire whether they had received influenza vaccination elsewhere than at their primary care provider’s office.

Data Analyses
The primary study outcome was receipt of influenza vaccination as documented by registry or billing data. We followed the principle of intention to treat, not excluding subjects even when they did not successfully receive the intended intervention (eg, their reminder letter was returned without a forwarding address). {chi}2 tests were used to explore the association between categorical variables. When comparing 2 groups, continuous variables were analyzed using the Student’s t test, or the Wilcoxon rank sum test when normality could not be assumed. All statistical tests were 2-tailed, and significance was assessed at the 5% level, with 95% confidence intervals (CIs) determined for the main study outcomes. All statistical calculations were performed using SAS software (SAS 8.0).


   RESULTS
TOP
 ABSTRACT
 METHODS
 RESULTS
DISCUSSION
 REFERENCES
 
Accuracy of Billing Data
Of the 373 medical records randomly selected for review, 6 were missing and 40 were purged because patients had moved; therefore, 327 records were available for analysis. An HRC was present in 48 of the reviewed medical records, 32 of which were identified correctly by billing data. For 9 of the 16 subjects with HRCs not captured by billing data, asthma was written in the medical record, but wheezing, cough, or bronchiolitis was the diagnosis billed. Additional reasons that HRCs were missed by billing data included 1) diagnoses were made before the study practices began using this billing system or after the billing data were searched for HRCs, 2) HRCs were diagnosed at well-child visits but only well-child care was billed, 3) diagnoses were made in other sites (eg, emergency departments), or 4) diagnoses were written on problem lists but never billed. No other systematic errors in coding were discovered. After adjustment for differences in clinic size, billing data had a sensitivity of 72% (95% CI: 48%–95%), specificity of 95% (95% CI: 90%–100%), and overall accuracy of 90% (95% CI: 84%–96%) in determining which children had an HRC.

Prevalence of HRCs
Figure 1 illustrates the steps undertaken to identify and recall children for influenza vaccination. Of the 17 273 active patients in the study practices, 2007 (12%) were determined by billing data to have an HRC, with the prevalence of HRCs ranging from 9% to 14% in the 4 study practices.


Figure 1
View larger version (21K):
[in this window]
[in a new window]
  		Fig 1. Study flow diagram.

 
Table 1 presents the ICD-9-CM codes used to identify children with HRCs and the corresponding number of subjects with each condition. Seventy-four (4%) subjects had >1 HRC. The 3 most common HRCs were asthma (n = 1755; 87% of total), congenital heart disease (n = 39; 2%), and bronchopulmonary dysplasia (n = 35; 2%).

Efficacy of Reminder/Recall for Influenza Vaccination
The intervention and control groups were similar at baseline with respect to age, insurance status, and percentage up-to-date with routine immunizations (Table 2). By chance, slightly more boys were allocated to the control group than the intervention group. The predominance of boys in the overall HRC population (1147 of 1851 HRC subjects [62%]) reflects the higher prevalence of asthma in prepubertal boys compared with girls, as has been previously documented.36


View this table:
[in this window]
[in a new window]
  		TABLE 2. Comparison of the Study Groups for the Trial of Immunization Reminder/Recall

 
Figure 2 illustrates the influenza immunization rates in the reminder/recall group compared with those receiving usual care. Reminder/recall significantly improved immunization rates for children with HRCs in each study practice, including in a practice (site 2) that had a relatively high influenza immunization rate in control subjects. The impact of recall was similar in children with asthma (42% intervention [I] vs 25% control [C]; P < .001) and those with HRCs other than asthma (43% I vs 26% C; P = .02). The influenza immunization rate in recalled subjects was higher than the rate in control subjects for those with private insurance (45% I vs 28% C; P < .001; 61% relative increase) and those publicly insured (34% I vs 12% C; P < .001; 183% relative increase), but reminder/recall did not improve rates for uninsured children (9% I vs 9% C; P = 1.0; no increase).


Figure 2
View larger version (34K):
[in this window]
[in a new window]
  		Fig 2. Influenza immunization rates: intervention versus control groups.

 
Recalled subjects were significantly more likely to have an office visit during October to January compared with control subjects (68% I vs 60% C; P < .001). However, those recalled had a significantly lower rate of missed opportunities to immunize compared with control subjects (28% I vs 37% C; P < .001). The study site that achieved the highest influenza immunization rate in recalled subjects (site 2) had a lower rate of missed opportunities in those recalled compared with the other 3 study sites (22% vs 29%; P = .06). The influenza immunization rate in healthy children aged 6 to 72 months was also higher in site 2 in comparison with other participating clinics (34% vs 13%; P < .001).

In April to May 2003, we administered a telephone survey to 295 randomly selected parents of HRC children who were not immunized at the study sites (response rate: 64%). Among those surveyed, 14% of intervention subjects versus 11% of control subjects reported that their child was immunized against influenza (not significant), with vaccination occurring in the following settings: at a different primary care office, at a school or church, at a hospital, or at a study practice (despite no record of this in registry or billing data). None reported immunization at a medical subspecialty office. Compared with control subjects, intervention subjects did not have an increased likelihood of being vaccinated in settings other than their primary care office (9% I vs 8% C; not significant). Extrapolating survey data to all HRC subjects who were considered unimmunized based on registry and billing data, we estimate that 50% of those recalled (95% CI: 46%–54%) and 34% of control subjects (95% CI: 29%–39%) were immunized against influenza during the study period.


   DISCUSSION
TOP
 ABSTRACT
 METHODS
 RESULTS
 DISCUSSION
REFERENCES
 
This study demonstrates that integrating diagnosis-based billing data with registry-driven reminder/recall creates a powerful tool for boosting influenza immunization rates in children who are at high risk of influenza-related complications. Computerized billing data in this setting was an accurate method for identifying children with HRCs, with an average of 12% of 6- to 72-month-olds having ≥1 HRCs. Reminder/recall, occurring in a staged manner based on timely influenza vaccination data available from a registry, raised influenza immunization rates higher than was achieved in the 2 largest previous studies of influenza recall in primary care settings.16,17 The reminder/recall process increased the likelihood that children with HRCs were seen by their care providers during influenza season and decreased missed opportunities for immunization. Nonetheless, the 42% influenza immunization rate achieved is well below the current vaccination rates for all other recommended pediatric immunizations.37

Because influenza immunization is recommended for individuals with a wide array of chronic medical conditions,9 efficiently and accurately identifying high-risk patients poses a substantial challenge for many primary care providers.38 Billing data provided us with a means of searching for children with any 1 of several hundred conditions. The ICD-9-CM codes that we used represented a broad interpretation of ACIP definitions of HRCs, because study practices were committed to an inclusive influenza immunization policy. Among children with certain types of diseases, the ACIP has defined HRCs as those requiring "regular medical follow-up or hospitalization during the preceding year."9 These criteria may be difficult to apply, particularly when patients with HRCs receive care in multiple clinical sites, and therefore children for whom influenza vaccination is indicated may be missed. In the current study, we purposefully cast a wide net when identifying children with HRCs, in the hopes of providing influenza immunization to any who might benefit from it. The 12% prevalence of HRCs that we documented using this method is similar to the 7% to 14% range determined from population-based survey data.8

In this setting, billing data identified >70% of all clinic patients aged 6 to 72 months with HRCs. Providers in the study practices estimated that our method identified many more high-risk children than had been targeted for influenza immunization in previous years. However, that almost one third of children with HRCs were not detected by billing data highlights a limitation of practice-based billing systems: conditions not billed are not captured. When providers noted asthma in the medical record but billed for wheezing, this may reflect the diagnostic uncertainty surrounding new-onset wheezing, but diagnoses made during well-child care or at another site of care were also missed. In addition, through our data quality analyses, we uncovered several systematic coding errors. Although these errors were detected and remedied before reminder/recall, these problems can compromise the utility of billing data and highlight the need for data quality processes whenever similar databases are used. Despite these limitations, billing data proved to be a powerful and efficient tool for identifying children who need influenza immunization.

Once high-risk children were identified, a registry-based recall successfully boosted influenza immunization rates in targeted children. Patient reminder/recall interventions have proved effective in multiple settings for children who were not up to date with routine immunizations39–43 but have been less well explored for children who need influenza immunization. In 2 randomized, controlled trials of reminder/recall for children with asthma, influenza immunization rates of 30%16 and 47%44 were achieved in recalled subjects. Similarly, Gaglani et al17 documented an increase in influenza immunization rates from 5% before to 32% after implementing a computerized reminder/recall system. The current study, to our knowledge, is the largest and most comprehensive trial of reminder/recall for influenza immunization of all targeted children, and our results demonstrate the efficacy of reminder/recall in patients with asthma and other chronic medical conditions in private practice settings.

One practice in the current study achieved an influenza immunization rate of 56% in recalled children, a rate almost twice as high as has been typically found in recalled children with asthma.16,17 This site gave influenza immunizations to a higher proportion of healthy 6- to 72-month-olds than did the remaining study sites, and the influenza immunization rate in control children with chronic medical conditions was also higher. Recalled subjects at this site were less likely to have a missed opportunity to immunize compared with those at the other participating sites. Two strategies in this office that may contribute to high influenza immunization rates are that patients can present to clinic without an appointment for influenza vaccination and that the entire family, including parents, can be immunized at the office. These data suggest that higher influenza immunization rates in high-risk children can be achieved in a setting where 1) influenza immunization is encouraged in healthy children, 2) missed opportunities are minimized, and 3) permissive office policies allow immunization without an appointment and for the entire family. Even when these conditions are present, reminder/recall may nonetheless increase influenza immunization rates, as it did in this site.

Among HRC children with different types of insurance coverage, the highest influenza immunization rate was achieved in privately insured patients who were recalled. However, the relative impact of reminder/recall was greatest for publicly insured children and least for those without insurance. The low immunization rate in publicly insured children who were not recalled suggests that barriers to immunization, such as poverty, underutilization of services, and lack of parental knowledge about vaccine recommendations, continue to have a negative impact on vaccination delivery.45 Nonetheless, reminder/recall substantially increased influenza immunization in this group. For the small number of patients who were uninsured in this setting, influenza immunization rates were poor regardless of recall. Uninsured patients would be expected to bear the entire cost of a visit for influenza vaccination, which likely represented an insurmountable financial barrier.

This investigation has several limitations. The study was conducted in 4 urban private pediatric practices with a relatively low Medicaid population, and the results may not be generalizable to other clinical settings. Because we used retrospective billing data to identify children with HRCs, the investigation relied on provider diagnoses of chronic medical conditions. As a consequence, there may have been differences in the criteria that individual providers used to make certain diagnoses, such as asthma. In addition, whereas influenza immunization is indicated for any child ≥6 months of age with certain chronic medical conditions,9 our intervention was conducted only for children aged 6 to 72 months. The age of study participants was restricted because the reminder/recall was conducted through an existing immunization registry, which contained complete records only for children <6 years of age. However, the methods developed to identify children should apply equally well to children older than 6 years, and as immunization registries such as this one mature, recall should be feasible in older children. Probable missed opportunities to immunize were assessed using billing data rather than medical record review, which limited our ability to examine reasons for nonimmunization among subjects with clinic visits. Finally, the reminder/recall intervention may have heightened provider awareness regarding influenza immunization, leading to higher rates in control subjects. If so, then we may have underestimated the impact of reminder/recall on influenza immunization in this setting.

In conclusion, this investigation provides strong support for existing ACIP recommendations advocating the use of reminder/recall to enhance influenza immunization in populations at high risk of influenza-related morbidity.9 Because of the wide use of diagnosis-based billing systems, the method and specific billing codes presented here can be used in a number of clinical settings to identify children who are at increased risk from influenza. Once targeted children are identified, a reminder/recall plan can be implemented, and this process can be successful in boosting immunization rates with or without an immunization registry.16,17 There are important benefits, however, of linking billing data to immunization registries whenever this is feasible. Registries can track influenza immunization rates in recalled children and can increase the efficiency of reminder/recall by restricting subsequent mailings only to those patients who are not immunized after an initial recall letter. Finally, the techniques described here likely can be used to improve coverage rates for other targeted immunizations, such as meningococcal and pneumococcal polysaccharide vaccines,46,47 because linking billing data to immunization registries creates an efficient and effective mechanism for identification and recall of children with specific chronic medical conditions.


   ACKNOWLEDGMENTS
 
Portions of this work were presented at the 37th National Immunization Conference, March 2003, Chicago, Illinois; and the Pediatric Academic Societies’ Annual Meeting, May 2003, Seattle, Washington.

This project was supported under a cooperative agreement with the Centers for Disease Control and Prevention (CDC) through the Association of American Medical Colleges (AAMC), grant MM-0067-02/02, and by a Research Scholar Award from the Children’s Hospital Research Institute (Denver, CO). The contents of this report are solely the responsibility of the authors and do not necessarily represent the official views of the CDC or the AAMC.


   FOOTNOTES
 
Received for publication Jul 18, 2003; Accepted Sep 30, 2003.

Reprint requests to (M.F.D.) 1056 E 19th Ave, B032, Denver, CO 80218. E-mail: daley.matthew{at}tchden.org


   REFERENCES
TOP
 ABSTRACT
 METHODS
 RESULTS
 DISCUSSION
 REFERENCES
 

  1. Thompson WW, Shay DK, Weintraub E, et al. Mortality associated with influenza and respiratory syncytial virus in the United States. JAMA.2003; 289 :179 –186[Abstract/Free Full Text]
  2. Simonsen L, Fukuda K, Schonberger LB, Cox NJ. The impact of influenza epidemics on hospitalizations. J Infect Dis.2000; 181 :831 –837[CrossRef][Web of Science][Medline]
  3. Barker WH, Mullooly JP. Impact of epidemic type A influenza in a defined adult population. Am J Epidemiol.1980; 112 :798 –811[Abstract/Free Full Text]
  4. Glezen WP, Decker M, Perrotta DM. Survey of underlying conditions of persons hospitalized with acute respiratory disease during influenza epidemics in Houston, 1978–1981. Am Rev Respir Dis.1987; 136 :550 –555[Web of Science][Medline]
  5. Neuzil KM, Mellen BG, Wright PF, et al. The effect of influenza on hospitalizations, outpatient visits, and courses of antibiotics in children. N Engl J Med.2000; 342 :225 –231[Abstract/Free Full Text]
  6. Neuzil KM, Wright PF, Mitchel EF Jr, Griffin MR. The burden of influenza illness in children with asthma and other chronic medical conditions. J Pediatr.2000; 137 :856 –864[CrossRef][Web of Science][Medline]
  7. Izurieta HS, Thompson WW, Kramarz P, et al. Influenza and the rates of hospitalization for respiratory disease among infants and young children. N Engl J Med.2000; 342 :232 –239[Abstract/Free Full Text]
  8. Fukuda K, O’Mara D, Singleton JA. Drug shortages, part 4: how the delayed distribution of influenza vaccine created shortages in 2000 and 2001. Pharm Therapeutics.2002; 27 :235 –242
  9. Centers for Disease Control and Prevention. Prevention and control of influenza: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Morbid Mortal Wkly Rep.2003; 52(RR-8) :1 –34
  10. American Academy of Pediatrics, Committee on Infectious Diseases. Reduction of the influenza burden in children. Pediatrics.2002; 110 :1246 –1252[Abstract/Free Full Text]
  11. Zimmerman RK. Recommended childhood and adolescent immunization schedule, United States, 2003 and update on childhood immunizations. Am Fam Physician.2003; 67 :188 , 190, 195–196[Medline]
  12. Kramarz P, DeStefano F, Gargiullo PM, et al. Influenza vaccination in children with asthma in health maintenance organizations. Vaccine Safety Datalink Team. Vaccine.2000; 18 :2288 –2294[CrossRef][Web of Science][Medline]
  13. Szilagyi PG, Holl JL, Rodewald LE, et al. Evaluation of New York State’s Child Health Plus: children who have asthma. Pediatrics.2000; 105 :719 –727
  14. Chung EK, Casey R, Pinto-Martin JA, et al. Routine and influenza vaccination rates in children with asthma. Ann Allergy Asthma Immunol.1998; 80 :318 –322[Web of Science][Medline]
  15. Marshall BC, Henshaw C, Evans DA, et al. Influenza vaccination coverage level at a cystic fibrosis center. Pediatrics.2002; 109(5) . Available at: http://www.pediatrics.org/cgi/content/full/109/5/e80
  16. Szilagyi PG, Rodewald LE, Savageau J, et al. Improving influenza vaccination rates in children with asthma: a test of a computerized reminder system and an analysis of factors predicting vaccination compliance. Pediatrics.1992; 90 :871 –875[Abstract/Free Full Text]
  17. Gaglani M, Riggs M, Kamenicky C, Glezen WP. A computerized reminder strategy is effective for annual influenza immunization of children with asthma or reactive airway disease. Pediatr Infect Dis J.2001; 20 :1155 –1160[CrossRef][Web of Science][Medline]
  18. Muldoon JH, Neff JM, Gay JC. Profiling the health service needs of populations using diagnosis-based classification systems. J Ambul Care Manage.1997; 20 :1 –18[Medline]
  19. Perrin JM, Kuhlthau K, McLaughlin TJ, et al. Changing patterns of conditions among children receiving Supplemental Security Income disability benefits. Arch Pediatr Adolesc Med.1999; 153 :80 –84[Abstract/Free Full Text]
  20. Feudtner C, Christakis DA, Connell FA. Pediatric deaths attributable to complex chronic conditions: a population-based study of Washington State, 1980–1997. Pediatrics.2000; 106 :205 –209[Abstract/Free Full Text]
  21. Feudtner C, Hays RM, Haynes G, et al. Deaths attributed to pediatric complex chronic conditions: national trends and implications for supportive care services. Pediatrics.2001; 107(6) . Available at: http://www.pediatrics.org/cgi/content/full/107/6/e99
  22. Kuhlthau K, Ferris TG, Beal AC, et al. Who cares for Medicaid-enrolled children with chronic conditions? Pediatrics.2001; 108 :906 –912[Abstract/Free Full Text]
  23. Hwang W, Ireys HT, Anderson GF. Comparison of risk adjusters for Medicaid-enrolled children with and without chronic health conditions. Ambul Pediatr.2001; 1 :217 –224[CrossRef][Web of Science][Medline]
  24. US Department of Health and Human Services. The International Classification of Diseases, 9th Revision, Clinical Modification. Washington, DC: US Government Printing Office; 2002
  25. Chiu SS, Lau YL, Chan KH, et al. Influenza-related hospitalizations among children in Hong Kong. N Engl J Med.2002; 347 :2097 –2103[Abstract/Free Full Text]
  26. American Medical Association. Current Procedural Terminology: CPT 2002 Standard Edition. Chicago, IL: American Medical Association; 2002
  27. National Immunization Program. Immunization registry minimum functional standards. Available at: http://www.cdc.gov/nip/registry/mfs2001.htm. Accessed May 29, 2002
  28. Centers for Disease Control and Prevention. Prevention and control of influenza: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Morbid Mortal Wkly Rep.2002; 51(RR-3) :1 –31
  29. Leger P, Leger SS. Respiratory concerns in Duchenne muscular dystrophy (DMD). Pediatr Pulmonol.1997; 16(suppl) :137 –139[CrossRef]
  30. Reddihough DS, Baikie G, Walstab JE. Cerebral palsy in Victoria, Australia: mortality and causes of death. J Paediatr Child Health.2001; 37 :183 –186[CrossRef][Web of Science][Medline]
  31. Cuadrado E, Barrena MJ. Immune dysfunction in Down’s syndrome: primary immune deficiency or early senescence of the immune system? Clin Immunol Immunopathol.1996; 78 :209 –214[CrossRef][Web of Science][Medline]
  32. Martinez FD, Wright AL, Taussig LM, et al. Asthma and wheezing in the first six years of life. N Engl J Med.1995; 332 :133 –138[Abstract/Free Full Text]
  33. Wright AL. Epidemiology of asthma and recurrent wheeze in childhood. Clin Rev Allergy Immunol.2002; 22 :33 –44[CrossRef][Web of Science][Medline]
  34. Albright JT, Topham AK, Reilly JS. Pediatric head and neck malignancies: US incidence and trends over 2 decades. Arch Otolaryngol Head Neck Surg.2002; 128 :655 –659[Abstract/Free Full Text]
  35. National Newborn Screening and Genetics Resource Center (NNSGRC). National newborn screening report–2000. Austin, TX: NNSGCR; 2003
  36. Osborne ML, Vollmer WM, Linton KL, Buist AS. Characteristics of patients with asthma within a large HMO: a comparison by age and gender. Am J Respir Crit Care Med.1998; 157 :123 –128[Free Full Text]
  37. Centers for Disease Control and Prevention. National, state, and urban area vaccination levels among children aged 19–35 months–United States, 2002. MMWR Morbid Mortal Wkly Rep.2003; 52 :728 –732[Medline]
  38. Davis MM, McMahon SR, Santoli JM, et al. A national survey of physician practices regarding influenza vaccine. J Gen Intern Med.2002; 17 :670 –676[CrossRef][Web of Science][Medline]
  39. Young SA, Halpin TJ, Johnson DA, et al. Effectiveness of a mailed reminder on the immunization levels of infants at high risk of failure to complete immunizations. Am J Public Health.1980; 70 :422 –424[Abstract/Free Full Text]
  40. Tollestrup K, Hubbard BB. Evaluation of a follow-up system in a county health department’s immunization clinic. Am J Prev Med.1991; 7 :24 –28[Web of Science][Medline]
  41. Linkins RW, Dini EF, Watson G, Patriarca PA. A randomized trial of the effectiveness of computer-generated telephone messages in increasing immunization visits among preschool children. Arch Pediatr Adolesc Med.1994; 148 :908 –914[Abstract/Free Full Text]
  42. Lieu TA, Capra AM, Makol J, et al. Effectiveness and cost-effectiveness of letters, automated telephone messages, or both for underimmunized children in a health maintenance organization. Pediatrics.1998; 101(4) . Available at: http://www.pediatrics.org/cgi/content/full/101/4/e3
  43. Rodewald LE, Szilagyi PG, Humiston SG, et al. A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care. Pediatrics.1999; 103 :31 –38[Abstract/Free Full Text]
  44. Kemper KJ, Goldberg H. Do computer-generated reminder letters improve the rate of influenza immunization in an urban pediatric clinic? Am J Dis Child.1993; 147 :717 –718[Abstract/Free Full Text]
  45. Santoli JM, Setia S, Rodewald LE, et al. Immunization pockets of need: science and practice. Am J Prev Med.2000; 19 :89 –98[CrossRef][Web of Science][Medline]
  46. Centers for Disease Control and Prevention. Preventing pneumococcal disease among infants and young children. Recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Morbid Mortal Wkly Rep.2000; 49(RR-9) :1 –35
  47. Centers for Disease Control and Prevention. Prevention and control of meningococcal disease. Recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Morb Mortal Wkly Rep.2000; 49(RR-7) :1 –10
PEDIATRICS (ISSN 1098-4275). ©2004 by the American Academy of Pediatrics

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Facebook Facebook   Add to Reddit Reddit   Add to Technorati Technorati   Add to Twitter Twitter    What's this?


This article has been cited by other articles:


Home page
 Arch Pediatr Adolesc MedHome page
M. A. Allison, M. F. Daley, J. Barrow, L. A. Crane, B. L. Beaty, N. Allred, and A. Kempe
High Influenza Vaccination Coverage in Children With High-Risk Conditions During a Vaccine Shortage
Arch Pediatr Adolesc Med, May 1, 2009; 163(5): 426 - 431.
[Abstract] [Full Text] [PDF]

Home page
 Arch Pediatr Adolesc MedHome page
C. M. Rand, P. G. Szilagyi, B.-K. Yoo, P. Auinger, C. Albertin, and M. S. Coleman
Additional Visit Burden for Universal Influenza Vaccination of US School-Aged Children and Adolescents
Arch Pediatr Adolesc Med, November 1, 2008; 162(11): 1048 - 1055.
[Abstract] [Full Text] [PDF]

Home page
 Qual Saf Health CareHome page
M. T Britto, P. J Schoettker, G. M Pandzik, J. Weiland, and K. E Mandel
Improving influenza immunisation for high-risk children and adolescents
Qual. Saf. Health Care, October 1, 2007; 16(5): 363 - 368.
[Abstract] [Full Text] [PDF]

Home page
 CLIN PEDIATRHome page
M. F. Daley, L. A. Crane, V. Chandramouli, B. L. Beaty, J. Barrow, N. Allred, S. Berman, and A. Kempe
Misperceptions About Influenza Vaccination Among Parents of Healthy Young Children
Clinical Pediatrics, June 1, 2007; 46(5): 408 - 417.
[Abstract] [PDF]

Home page
 Ann Fam MedHome page
R. K. Zimmerman, A. Hoberman, M. P. Nowalk, C. J. Lin, D. P. Greenberg, S. T. Weinberg, F. S. Ko, and D. E. Fox
Improving Influenza Vaccination Rates of High-Risk Inner-City Children Over 2 Intervention Years
Ann. Fam. Med, November 1, 2006; 4(6): 534 - 540.
[Abstract] [Full Text] [PDF]

Home page
 Arch Pediatr Adolesc MedHome page
K. J. Dombkowski, M. M. Davis, L. M. Cohn, and S. J. Clark
Effect of missed opportunities on influenza vaccination rates among children with asthma.
Arch Pediatr Adolesc Med, September 1, 2006; 160(9): 966 - 971.
[Abstract] [Full Text] [PDF]

Home page
 J. Am. Med. Inform. Assoc.Home page
D. L. Sanders and D. Aronsky
Biomedical Informatics Applications for Asthma Care: A Systematic Review
J. Am. Med. Inform. Assoc., July 1, 2006; 13(4): 418 - 427.
[Abstract] [Full Text] [PDF]

Home page
 Arch Pediatr Adolesc MedHome page
P. G. Szilagyi, S. Schaffer, R. Barth, L. P. Shone, S. G. Humiston, S. Ambrose, and F. Averhoff
Effect of Telephone Reminder/Recall on Adolescent Immunization and Preventive Visits: Results From a Randomized Clinical Trial
Arch Pediatr Adolesc Med, February 1, 2006; 160(2): 157 - 163.
[Abstract] [Full Text] [PDF]

Home page
 Arch Pediatr Adolesc MedHome page
M. F. Daley, B. L. Beaty, J. Barrow, K. Pearson, L. A. Crane, S. Berman, and A. Kempe
Missed Opportunities for Influenza Vaccination in Children With Chronic Medical Conditions
Arch Pediatr Adolesc Med, October 1, 2005; 159(10): 986 - 991.
[Abstract] [Full Text] [PDF]

Home page
 Arch Pediatr Adolesc MedHome page
G. De Marco, D. Ummarino, E. Giannetti, T. Magurno, and A. Guarino
Impact of Mass Communication in the Implementation of Influenza Vaccination for Infants
Arch Pediatr Adolesc Med, June 1, 2005; 159(6): 596 - 596.
[Full Text] [PDF]

Home page
 Arch Pediatr Adolesc MedHome page
S. G. Humiston, P. G. Szilagyi, M. K. Iwane, S. J. Schaffer, J. Santoli, L. Shone, R. Barth, T. McInerny, and B. Schwartz
The Feasibility of Universal Influenza Vaccination for Infants and Toddlers
Arch Pediatr Adolesc Med, September 1, 2004; 158(9): 867 - 874.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow E-mail this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My File Cabinet
Right arrow Download to citation manager
Right arrowRequest Permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via CrossRef
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Daley, M. F.
Right arrow Articles by Kempe, A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Daley, M. F.
Right arrow Articles by Kempe, A.
Related Collections
Right arrow Infectious Disease & Immunity
Right arrowRelated AAP Red Book topics:
Influenza
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Facebook   Add to Reddit   Add to Technorati   Add to Twitter  
What's this?