This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow P3Rs: Submit a response
Right arrow P3Rs: View responses
Right arrow Alert me when this article is cited
Right arrow Alert me when P3Rs are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow E-mail this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My File Cabinet
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via CrossRef
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Szilagyi, P. G.
Right arrow Articles by Rodewald, L. E.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Szilagyi, P. G.
Right arrow Articles by Rodewald, L. E.
Related Collections
Right arrow Infectious Disease & Immunity
Right arrowRelated AAP Red Book topics:
Haemophilus influenzae Infections
Poliovirus Infections

PEDIATRICS Vol. 110 No. 5 November 2002, pp. e58


ELECTRONIC ARTICLE

Reducing Geographic, Racial, and Ethnic Disparities in Childhood Immunization Rates by Using Reminder/Recall Interventions in Urban Primary Care Practices

Peter G. Szilagyi, MD, MPH*, Stanley Schaffer, MD, MS*, Laura Shone, MSW*, Richard Barth, BS*, Sharon G. Humiston, MD, MPH{ddagger}, Mardy Sandler, MSW§ and Lance E. Rodewald, MD||

* Department of Pediatrics and Strong Children’s Research Center, University of Rochester School of Medicine and Dentistry, Rochester, New York
{ddagger} Department of Emergency Medicine and Strong Children’s Research Center, University of Rochester School of Medicine and Dentistry, Rochester, New York
§ Department of Social Work, Strong Memorial Hospital, University of Rochester School of Medicine and Dentistry, Rochester, New York
|| National Immunization Program, Centers for Disease Control and Prevention

-->
    ABSTRACT
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 CONCLUSION
 REFERENCES
 
Context. An overarching national health goal of Healthy People 2010 is to eliminate disparities in leading health care indicators including immunizations. Disparities in US childhood immunization rates persist, with inner-city, black, and Hispanic children having lower rates. Although practice or clinic-based interventions, such as patient reminder/recall systems, have been found to improve immunization rates in specific settings, there is little evidence that those site-based interventions can reduce disparities in immunization rates at the community level.

Objective. To assess the effect of a community-wide reminder, recall, and outreach (RRO) system for childhood immunizations on known disparities in immunization rates between inner-city versus suburban populations and among white, black, and Hispanic children within an entire county.

Setting. Monroe County, New York (birth cohort: 10 000, total population: 750 000), which includes the city of Rochester. Three geographic regions within the county were compared: the inner city of Rochester, which contains the greatest concentration of poverty (among 2-year-old children, 64% have Medicaid); the rest of the city of Rochester (38% have Medicaid); and the suburbs of the county (8% have Medicaid).

Interventions. An RRO system was implemented in 8 city practices in 1995 (covering 64% of inner-city children) and was expanded to 10 city practices by 1999 (covering 74% of inner-city children, 61% of rest-of-city children, and 9% of suburban children). The RRO intervention involved lay community-based outreach workers who were assigned to city practices to track immunization rates of all 0- to 2-year-olds, and to provide a staged intervention with increasing intensity depending on the degree to which children were behind in immunizations (tracking for all children, mail, or telephone reminders for most children, assistance with transportation or scheduling for some children, and home visits for 5% of children who were most behind in immunizations and who faced complex barriers).

Study Participants. Three separate cohorts of 0- to 2-year-old children were assessed—those residing in the county in 1993, 1996, and 1999.

Study Design. Immunization rates were measured for each geographic region in Monroe County at 3 time periods: before the implementation of a systematic RRO system (1993), during early phases of implementation of the RRO system (1996), and after implementation of the RRO system in 10 city practices (1999). Immunization rates were compared for children living in the 3 geographic regions, and for white, black, and Hispanic children.

Immunization rates were measured by the same methodology in each of the 3 time periods. A denominator of children was obtained by merging patient lists from the practice files of most pediatric and family medicine practices in the county (covering 85% to 89% of county children). A random sample of children (>500 from the suburbs and >1200 from the city for each sampling period) was then selected for medical chart review at practices to determine demographic characteristics (including race and ethnicity) and immunization rates. City children were oversampled to allow detection of effects by geographic region and race. Rates for the 3 geographic regions and for the entire county were determined using Stata to adjust for the clustered sampling.

Main Outcome Measures. Immunization rates at 12 and 24 months for recommended vaccines (4 diphtheria-tetanus-pertussis:3 polio:1 measles-mumps-rubella: >=1 Haemophilus influenzae type b on or after 12 months of age).

Results. Disparities by Geographic Region: Baseline immunization rates (1993) for 24-month-olds were as follows: inner city (55%), rest of city (64%), and suburbs (73%), with an 18% difference in rates between the inner city and suburbs. By 1996, immunization rates rose faster in the inner city (+21% points) than in the suburbs (+14% points) so that the difference in rates between the inner city and suburbs had narrowed to 11%. In 1999, rates were similar across geographic regions: inner city (84%), rest of city (81%), and suburbs (88%), with a 4% difference between the inner city and suburbs. Disparities by Race and Ethnicity: Immunization rates were available in 1996 and 1999 by race and ethnicity. Twenty-four-month immunization rates in 1996 showed disparities: white (89%), black (76%), and Hispanic (74%), with a 13% difference between rates for white and black children and a 15% difference between white and Hispanic children. In 1999, rates were similar across the groups: white (88%), black (81%), and Hispanic (87%), with a 7% difference between rates for white and black children, and a 1% difference between white and Hispanic children.

Conclusions. A community-wide intervention of patient RRO raised childhood immunization rates in the inner city of Rochester and was associated with marked reductions in disparities in immunization rates between inner-city and suburban children and among racial and ethnic minority populations. By targeting a relatively manageable number of primary care practices that serve city children and using an effective strategy to increase immunization rates in each practice, it is possible to eliminate disparities in immunizations for vulnerable children.

Key Words: immunization rates • reminders • recall • disparities

Abbreviations: RRO, reminder, recall and outreach • VFC, Vaccines for Children


    INTRODUCTION
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 CONCLUSION
 REFERENCES
 
An overarching national health goal under Healthy People 2010 is to eliminate disparities in care for leading health indicators.1 Vaccination coverage rate is a leading health indicator,2 and childhood immunization has been recognized as one of our most important public health achievements.3 Although US childhood immunization rates have risen over the past 2 decades,4 disparities continue,5 with lower rates among children living in poverty,68 among urban children,9 and among black and Hispanic children.1013

Studies have described barriers to immunizations among the populations of children with lowest rates.6,1416 Suggestions for overcoming these barriers generally involve targeting of high-risk populations with specific interventions that have demonstrated effectiveness. During the past decade, new studies,1720 systematic reviews,2126 and guidelines from advisory groups7,15,27,28 have published evidence-based strategies designed to improve childhood immunization rates; yet disparities in rates persist.7,8

Little evidence exists that interventions that are effective in specific clinical settings can eliminate disparities across entire populations. For example, although patient reminder/recall systems have been found to improve immunization rates in a variety of clinical settings18,19,2124,26 and are a major potential activity of community-wide immunization registries,29 such systems have not been shown to reduce disparities in immunization rates at the community level.

The objective of this study was to assess the impact of a community-wide reminder, recall, and outreach (RRO) system for childhood immunizations on known disparities in immunization rates between inner-city versus suburban populations and among white, black, and Hispanic children within an entire county.


    METHODS
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 CONCLUSION
 REFERENCES
 
The study was approved by the University of Rochester Research Subjects Review Board.

Setting
The study setting was Monroe County, New York, an upstate metropolitan county with a population of 750 000 surrounding the city of Rochester. For study purposes, the county was divided into 3 regions30—the inner city of Rochester, in which >60% of young children qualify for Medicaid; the rest of the city of Rochester, comprised of the remaining census tracts in the city; and the suburbs surrounding the city. These regions were previously analyzed in health services studies in the early 1990s30 and retained for comparisons. Like many regions in the United States,31 both poverty and minority populations are concentrated in the inner city of Rochester, whereas the suburbs have low rates of poverty and consist mostly of white populations. For all of Monroe County, the poverty rate for children 0 to 4 years of age increased from 16% to 21% between 1990 and 1995, with poverty rates in the city remaining higher than 40%.32

In Monroe County, <1% of immunizations are provided by the health department, and these are primarily for school-aged children. The vast majority of preschool children are vaccinated within primary care practices. Previous studies have noted that nearly 100% of Monroe County’s young children receive medical care at primary care practices, and nearly all make at least 1 visit to a primary care provider within the first 2 years of life.33,34

The population for assessment of immunization rates was all 0- to 2-year-old children living in Monroe County. Rates were calculated at the standard reference ages of 12 and 24 months.35 Children living in outlying counties who were served by Monroe County practices were excluded because the goal was to assess childhood immunization rates for county residents.

The intervention targeted the 10 largest practices that serve mainly the inner-city or rest-of-city population.

Intervention
A randomized, controlled trial evaluating the effectiveness of an RRO for immunizations for children 0 to 2 years of age was performed in 6 city practices between March 1994 and August 1995; the methods have been reported elsewhere.19 Because this trial noted a 20% improvement in immunization rates of 2-year-olds compared with standard-of-care controls, the intervention was expanded in late 1995 to include the controls (ie, all 0- to 2-year-olds within these 6 practices) so that by 1996, 4100 children received the intervention (64% of inner city, 31% of rest of city, 8% of suburban, and 18% of all Monroe County). Two additional city practices were added in 1996 and 2 more were added in 1997. By mid-1999, 0- to 2-year-olds at 10 practices (6300 children; 74% of inner city, 61% of rest of city, 9% of suburb, and 32% of all Monroe County) were receiving the RRO. The intervention was at the practice level for city practices; thus, a small number of children who live in the suburbs but are served by these practices received the intervention as well.

The RRO intervention recruited lay outreach workers from the neighborhoods around the practices and assigned them to one or more city practices. Outreach workers 1) were trained to follow a strict reminder/recall protocol, 2) were provided with a list of age-eligible children for whom they were responsible, 3) set up a tickler-file system to track immunizations within their primary care practice and used medical charts to assess and monitor the immunization status of their caseload, and 4) applied the intervention protocol when children were behind in immunizations. The intervention was staged, with increasing intensity for children who were further behind in immunizations—all children were tracked; three-quarters received some type of reminder (telephone, postcard, or letter); many received multiple reminders; and a small number of children for whom all previous strategies failed (5%) received home visits to address barriers to care.19 The average caseload was 400 children. Practice-level immunization rates were determined and reviewed by the project leaders bimonthly between 1995 and 1999.

County-wide Assessment of Immunization Rates
Rates were assessed for the entire county and separately for the 3 geographic regions in 1993 (before any intervention), 1996 (when 46% of inner-city plus rest-of-city children received the intervention), and 1999 (when 68% of inner-city plus rest-of-city children received the intervention). Identical methods were used to assess rates for the 3 time periods except that the 1993 assessment did not include race or ethnicity, and used slightly different sampling percentages (8.3% instead of 10% for suburban practices).

Developing the Denominator
We created a listing of all primary care practices in the county, sent a letter to each practice requesting collaboration with chart reviews to measure immunization rates, and obtained a practice denominator for specific birth cohorts who would have been 2 years old at the time of chart review and had made at least 1 visit to the practice.

During each of the 3 assessment years, 83 to 85 primary care practices were eligible and 63 to 70 practices were assessed, including 100% of city-based practices, 77% to 86% of pediatric practices (mostly suburban), and 71% to 75% of family medicine practices. We were able to obtain denominator files or estimates of birth cohorts from most of the nonparticipating practices and calculated that 85% to 89% of the county’s birth cohort was included in the practices that participated. Approximately 90% of participating practices were able to provide computer-generated listings of their patients (lower percentage in 1993). For practices that could not supply a computerized list, we reviewed every medical chart at the practice to identify the practice’s population of 0- to 2-year-olds. The individual practice patient lists were merged and duplicate records (children seen at multiple practices) were identified using matching techniques based on name, date of birth, and gender. Because patient mobility and changing of primary care practices can complicate population-wide measurement of immunization rates, the denominator files from all practices were carefully analyzed to match children who were seen at multiple practices, and overall >10% of children were identified at multiple sites. In addition to date of birth and gender, the denominator file contained the most recent street address for mapping subjects into the 3 geographic regions. For the 1996 and 1999 assessments, race and ethnicity were obtained; these measures were not available for the 1993 assessment.

Sample Selection
We used a stratified, clustered sampling design with the primary sampling unit being the practice, then sampling from practices stratified by city versus suburban location. Patient records were assigned a random number; a 10% random sample was selected from practices located in the suburbs, and a 25% random sample was selected from practices located in the city. This sampling strategy ensured sufficient sample size (>500 from suburbs and >1200 from city for each sampling period) to estimate immunization rates within geographic areas with 95% confidence intervals of ± 5 percentage points at baseline rates of 80%.

Medical Chart Reviews
Individual medical charts were reviewed with data recorded on a standardized abstraction form. Demographic information (date of birth, gender, race, ethnicity, most recent street address, most recent insurance, number of primary care practices) and specific immunization dates were recorded.35 Progress notes, summary pages, and interpractice communication records were all reviewed to locate, verify, and cross-check immunizations received. Interrater reliability checks were performed on ~1% of charts with agreement for >99% of the subjects. A second chart review at the practice was performed for children seen at multiple practices (to accurately merge records) and for all children determined to lack any immunizations on the first review (to confirm that immunizations were truly lacking and not just missed because of abstraction error).

Measures
Race and ethnicity (for the 1996 and 1999 assessments) were classified as white, black, or Hispanic, consistent with Office of Management and Budget criteria.36 Because of small numbers, Asian race (when available) was included in the calculations as "white."36 Up-to-date measures for recommended vaccinations were calculated for individual vaccinations and for standard combinations of vaccinations.4,5,35

Analysis
Statistical adjustments were made (using Stata software37) to account for the following: 1) the probability weights of 10% or 25% sampling, 2) clustering (including a practice term as the primary sampling unit), and 3) stratification (city or suburban practice). Immunization rates were calculated for the entire county and the 3 geographic regions, as well as for the 3 racial/ethnic groups.


    RESULTS
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 CONCLUSION
 REFERENCES
 
Table 1 shows demographic characteristics of children within each of the 3 geographic regions of Monroe County. The inner city of Rochester has about one fifth of the county’s population and the highest concentration of black and Hispanic children and Medicaid recipients, with the majority of children served by hospital clinics or neighborhood health centers. The suburbs have predominantly white children covered by commercial insurance and served by private pediatric practices. The "rest of the city" is an intermediate zone in terms of these characteristics. Because the target of the outreach intervention was the primary care practice, targeting city practices that serve a combination of inner-city and rest-of-city children (and a small number of suburban children), the vast majority of the county’s children who received the intervention resided in the inner city or the rest of the city of Rochester, although 9% of suburban children attended these city practices and therefore were also exposed to the intervention.


View this table:
[in this window]
[in a new window]
 
TABLE 1. Demographic Characteristics of 2-Year-Old Children Within the 3 Geographic Study Regions, From the 1999 County-Wide Assessment*

 
Immunization rates rose steadily throughout the entire county (Table 2), with the greatest rise in the inner city and rest of the city where the intervention took place. Disparities in immunization rates between the inner city and suburbs were reduced from 18 to 21 percentage points in 1993 to 4 to 5 percentage points in 1999, eliminating previously statistically significant differences by 1999.


View this table:
[in this window]
[in a new window]
 
TABLE 2. Immunization Rates by Geographic Region for Recommended Vaccines, at 12 Months and 24 Months of Age

 
Table 3 shows disparities in immunization rates by race and ethnicity for 1996 and 1999, the 2 years for which race and ethnicity data were available. In 1996, immunization rates for white children were 11% to 15% higher than immunization rates for black or Hispanic children. These disparities were reduced significantly by 1999, with racial or ethnic differences in immunization rates at 24 months no longer reaching statistical significance in 1999. Additional analyses (not shown in table) of 1999 immunization rates for only children living in the city of Rochester, or for only children living in the suburbs, found no statistically significant differences among white, black, or Hispanic children.


View this table:
[in this window]
[in a new window]
 
TABLE 3. Immunization Rates by Race and Ethnicity for Recommended Vaccines, at 12 Months and 24 Months of Age

 
By 1999, immunization rates in Monroe County surpassed state and national rates. Up-to-date rates for 24-month-olds for diphtheria, tetanus, and pertussis4/polio3/measles-mumps-rubella1/Haemophilus influenzae type b3/Hepatitis B3 (4:3:1:3:3) were 87% in the city of Rochester (inner city and rest-of-city combined), 89% in the suburbs, and 88% for all of Monroe County, compared with rates of 69% for New York City,38 78% for the rest of New York State excluding New York City,38 74% for all of New York State,38 and 71% for the United States.38 Coverage rates for children in the city of Rochester were substantially higher than statewide rates excluding New York City, and higher than overall national rates.

Comment
This study found that extending a reminder, recall, and outreach intervention for childhood immunizations to the majority of children residing in a large city was associated with a marked reduction of disparities in immunization rates between city children and suburban children, and between white children and black or Hispanic children. A "dose response effect" was noted as the intervention expanded between 1996 and 1999 to more inner-city children, with disparities in immunization rates narrowing even further. A "dose response effect" also appeared with greater rise in immunization rates in the inner city (where three quarters of children received the intervention) than the rest of the city (where fewer received the intervention).

Disparities in health measures represent one of our nation’s most vexing problems. Multiple factors have been identified, but no easy solutions have been found.39,40 For childhood immunizations, it does not seem that lack of parental motivation or concerns about vaccine safety are major causes of the lower vaccination rates among impoverished and minority children.10,16,41 In fact, there is some evidence that parents of these children are less likely to be concerned about vaccine safety or efficacy than parents of nonpoor or white children.42 Similarly, there is little evidence that provider bias or ineffectiveness account for these disparities; in fact, some data suggest that providers serving high-risk populations are more aggressive in their immunization practices than providers serving populations of higher socioeconomic status.4345 The major cause of lower immunization rates16,46 seems to be "the immunization information gap"47—a combination of patient factors such as barriers to care, missed appointments, and unfamiliarity with the vaccination schedule; provider factors including failure to identify children who are behind and to apply effective interventions such as reminder and recall; and health care system factors including lack of community-wide immunization registries or interventions, access barriers, and cost issues. Often multiple factors exist, resulting in children being behind in immunizations without parents or providers recognizing it; hence, the "immunization information gap." Focusing on elimination of only 1 barrier may not reduce disparities. For example, racial disparities in receipt of immunizations have been found to persist in settings where free vaccine is available9 and in managed care settings.48

Some experts have suggested that the solution to racial and ethnic disparities is to target minority populations with special interventions or services.49 Our study suggests that targeting primary care practices that serve geographic areas where minority children reside may reduce or eliminate racial and ethnic disparities without the need for differential interventions according to race or ethnicity.

The essence of this community-wide intervention is that we extended a recommended practice-level intervention7,15,2224,27—patient reminder and recall—to the largest primary care practices that serve the most vulnerable children. There are several advantages to intervening on a practice-level. First, focusing on primary care practices is a more efficient way to identify children and to target an entire population than other potential interventions such as door-to-door campaigns,50 census-level outreach,51 or targeting children according to risk factors.23 Second, collaborating with primary care practices results in a natural entrée for patient-level interventions because most families are closely linked with their medical homes but many may be suspicious of governmental or external interventions.39 Third, interventions performed in primary care practices can result in other practice-level changes that have demonstrated effectiveness,2226 such as provider prompts, audit and feedback, and standing orders.

One of the hallmarks of recent quality-improvement initiatives is to implement strategies that have demonstrated effectiveness systematically, rather than searching for novel interventions.5254 Our intervention adopted a successful practice-level strategy to the community-wide level. Recent advances make this type of community-wide intervention more feasible. Community and state immunization registries are expanding29 and will be able to provide efficient denominator-based tracking and reminder systems (we used paper files and created a manual tracking system). A challenge for practices is to determine which children are behind in immunizations. Scattering of immunization records exacerbates this problem (and was noted in our study),55 and immunization registries that span practices have the potential to facilitate this tracking and identification process. Although practices’ electronic billing systems could accomplish this as well, few providers utilize these billing systems to track and recall children for immunizations, they do not contain algorithms for immunization status and do not link records across practices.16,26,40 Although their potential remains to be seen, immunization registries may help to reduce immunization disparities on a community level if they are combined with systematic reminder, recall, and outreach interventions.

One concern about our intervention involves cost. In the original randomized clinical trial19 the cost of the intervention was $5.27 per child per month and the cost per additional child brought up to date was $316 over a 1-year period. Because the intervention was subsequently applied to all patients within the 10 practices, it is not possible to calculate the cost-effectiveness of the current intervention. The current costs of the entire program are $240 000 per year— 70% for outreach worker salaries and the remainder for supervisory personnel, computer and administrative support, and local travel, phone, and other expenses for the outreach workers. Because the 10 practices have a birth cohort of 3189 and the intervention is applied for the first 2 years of life and beyond for children still behind, >6400 children are served, for a cost of $38 per child per year, or $3.12 per child per month. The lower costs after the randomized trial are attributable to expanding the outreach caseload as immunization rates increased. It is important to note that the intervention described in this study not only increased immunization rates but also had a beneficial spillover effect by increasing health supervision visits and preventive screening for anemia and lead poisoning, thus providing additional benefits beyond immunization delivery.19

This study has several limitations. Regarding external validity, our intervention relied on the finding that nearly all children in Rochester make at least 1 contact with a primary care practice during their first 2 years of life, a situation typical of children (including poor children) across the United States.56 This allowed for the intervention to be based in practices. Practice-level interventions may be unsuccessful in communities in which a large proportion of children never visit a primary care provider. Also, in some communities in which the majority of urban children are served by a very large number of primary care practices (each having few children), it may be less feasible to focus interventions on a few large practices. However, many urban centers mirror the situation in Rochester in which a relatively few primary care practices and clinics serve a high proportion of urban children.

A second potential limitation is that this study was conducted in Rochester, New York, a community with recognized high levels of pediatric care. However, the 1993 baseline county-wide immunization rates of 66% at 24 months (55% in the inner city) were similar to national immunization rates and to low rates reported in large urban centers.57,58

Third, our intervention was implemented at primary care sites and not centrally; this was necessary to track records and charts in the period before computerized registries and also resulted in high acceptance by practices and families. It is possible that our study findings would not be applicable to a centrally operated tracking and outreach program such as a centralized immunization registry. Furthermore, our study does not provide evidence in either direction of the effectiveness of reminder/recall in suburban settings.

Notably, the reduction in racial disparities in Rochester was made possible by the high racial and ethnic clustering in the city of Rochester. In our setting, 80% of the county’s black children, 83% of Hispanic children, and 31% of white children were served by the city practices receiving the RRO intervention. Conversely, 68% of children in the city practices receiving the RRO intervention were either black or Hispanic, while 25% of children in practices not receiving the intervention were black or Hispanic. Because the inner-city practices served most of the county’s minority children, interventions directed at those practices were able to reduce racial and ethnic disparities at the county level. Although this demographic pattern exists in many urban areas and across the United States,59 a community with less clustering of minority populations in the city might have different results if only city practices were targeted by interventions.

This study also has limitations to internal validity. Immunization rates were assessed using medical chart reviews at participating practices, and it is possible that the 11% to 15% of children who were not included had immunization rates different from those included in the assessments. However, because all city-based practices participated, children who were not included were more likely to reside in the suburbs; their lack of inclusion would not have affected estimates of immunization rates for city children but could possibly have affected estimates of disparities. In addition, children who were never seen in a primary care practice would have been missed by the immunization assessments and also by the intervention. We believe this group is small based on findings from previous studies in Rochester emergency departments34,60 that noted that almost 100% of infants and toddlers seen in the pediatric emergency departments had made at least 1 visit to their primary care practice during their life (and therefore would have been included in the denominators for assessment), and by random digit dial surveys noting that few children living in Rochester lacked a medical home.61,62 Also, the birth cohorts determined by the immunization assessments (adjusted for the 11%–15% not assessed) were virtually identical to the birth cohorts of Monroe County from the US census. For example, our estimate of a birth cohort of 10 066 in 1999 was close to the US Census count of 10 488 Monroe County persons under 1 year of age in 1990.63

A second possible concern about internal validity is that the assessments were conducted by medical chart review, with no contact of patients, and some immunizations received at nonparticipating practices might have been missed by the chart reviews. This would be a problem if nearly all primary care practices had not participated in the immunization assessments, but should not pose a major bias, because most county children were included. Also, some immunizations received outside of the Rochester area may not be recorded in chart reviews. Although there is no perfect method to measure immunization rates, the National Immunization Survey currently relies on parents to indicate the providers who vaccinated their children, and then conducts medical chart reviews for determining immunization dates and coverage rates.64 Thus medical chart review is currently the "gold standard" for assessment of immunization dates.

Most importantly, it is possible that the reduction in disparities and the greater rise in immunization rates among inner-city and minority children were not the result of the intervention but other secular trends occurring in the inner city of Rochester. There were no other major immunization interventions occurring during this time period in the city of Rochester. Although overall communications about immunizations increased during the 1990s, communications from managed care plans, the Department of Health, or family or community organizations did not specifically target the city population or city practices during this period. The introduction of the Vaccines for Children (VFC) Program for children covered by Medicaid or not having insurance coverage for vaccines in early 1994 did disproportionately affect city children, but immunization rates continued to rise well after 1994 without significant changes in the number of children qualifying for VFC vaccines. Because all city practices enrolled in the VFC program early in 1994, any potential impact of the VFC program would not have affected the reduction in disparities in immunization rates between 1996 and 1999. Similarly, Medicaid managed care increased during the 1990s, but none of these plans instituted special immunization interventions because our city-wide RRO program existed.


    CONCLUSION
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 CONCLUSION
 REFERENCES
 
A community-wide intervention of patient RRO raised childhood immunization rates in the inner city of Rochester and was associated with marked reductions in disparities in immunization rates between inner-city and suburban children and among racial and ethnic minority populations. By targeting a relatively manageable number of primary care practices that serve city children, it may be possible to eliminate disparities in immunizations and achieve improvements in overall health care for vulnerable children.


    ACKNOWLEDGMENTS
 
This study was supported by grant U38-CCU217960 from the Centers for Disease Control and Prevention, the Monroe County Department of Health, and the Daisy Marquis Jones Foundation.

We appreciate the assistance of Drs Klaus Roghmann, Andrew Doniger, and Nancy Bennett on various phases of this work.


    FOOTNOTES
 
Received for publication May 13, 2002; Accepted Jul 11, 2002.

Address correspondence to Peter G. Szilagyi, MD, MPH, University of Rochester School of Medicine, Box 632, Strong Memorial Hospital, 601 Elmwood Ave, Rochester, New York 14642. E-mail: peter_szilagyi{at}urmc.rochester.edu

This work was presented in part at the 1998 Annual Meeting of the Pediatric Academic Societies, New Orleans, LA, May 4, 1998 (presidential plenary session); the 1998 National Immunization Conference, Atlanta, GA, July, 1998; the 2002 National Immunization Conference, Denver, CO, April 30, 2002 (plenary session); and the 2002 Annual Meeting of the Pediatric Academic Societies, Baltimore, MD, May 5, 2002.


    REFERENCES
 TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 CONCLUSION
 REFERENCES
 

  1. US Department of Health and Human Services. Healthy People 2010, Second Edition. With Understanding and Improving Health and Objectives For Improving Health. Washington, DC: US Department of Health and Human Services; 2000
  2. Office of Disease Prevention and Health Promotion. Healthy People 2010. Available at: http://www.health.gov/healthypeople/about/hpfact.htm. Accessed December 26, 2001
  3. Centers for Disease Control and Prevention. Ten great public health achievements—United States, 1900–1999. MMWR Morb Mortal Wkly Rep.1999; 48 :241 –243[Medline]
  4. Centers for Disease Control and Prevention. National, state, and urban area vaccination coverage levels among children aged 19–35 months—United States, 2000. MMWR Morb Mortal Wkly Rep.2001; 50 :637 –641[Medline]
  5. Luman ET, Barker LE, Simpson DM, Rodewald LE, Szilagyi PG, Zhao Z. National, state, and urban-area vaccination-coverage levels among children aged 19–35 months, United States, 1999. Am J Prev Med.2001; 20 :88 –153[ISI][Medline]
  6. Orenstein WA, Atkinson W, Mason D, Bernier RH. Barriers to vaccinating pre-school children. J Health Care Poor Underserved.1990; 1 :315 –330[Medline]
  7. National Vaccine Advisory Committee. Strategies to sustain success in childhood immunizations. JAMA.1999; 282 :363 –370[Abstract/Free Full Text]
  8. Klevens RM, Luman ET. US children living in and near poverty. Risk of vaccine-preventable diseases. Am J Prev Med.2001; 20 :41 –46[CrossRef][ISI][Medline]
  9. Kenyon TA, Matuck MA, Stroh G. Persistent low immunization coverage among inner-city preschool children despite access to free vaccine. Pediatrics.1998; 101 :612 –616[Abstract/Free Full Text]
  10. Bates AS, Fitzgerald JF, Dittus RS, Wolinksy FD. Risk factors for underimmunization in poor urban infants. JAMA.1994; 272 :1105 –1110[Abstract]
  11. Wood D, Donald-Sherbourne C, Halfon N, et al. Factors related to immunization status among inner-city Latino and African American preschoolers. Pediatrics.1995; 96 :295 –301[Abstract/Free Full Text]
  12. Daniels D, Jiles RB, Klevens RM, Herrera GA. Undervaccinated African-American preschoolers. A case of missed opportunities. Am J Prev Med.2001; 20 :61 –68[ISI][Medline]
  13. Herrera GA, Zhao Z, Klevens M. Variation in vaccination coverage among children of Hispanic ancestry. Am J Prev Med.2001; 20 :69 –74[CrossRef][ISI][Medline]
  14. Cutts FT, Orenstein WA, Bernier RH. Causes of low preschool immunization coverage in the United States. Annu Rev Publ Health.1992; 13 :385 –398[CrossRef][ISI][Medline]
  15. National Vaccine Advisory Committee. The measles epidemic: the problems, barriers, and recommendations. JAMA.1991; 266 :1547 –1552[Abstract]
  16. Santoli JM, Szilagyi PG, Rodewald LE. Barriers to immunization and missed opportunities. Pediatr Ann.1998; 27 :366 –374[ISI][Medline]
  17. Pierce C, Goldstein M, Suozzi K, Gallaher M, Dietz V, Stevenson J. The impact of the standards for pediatric immunization practices on vaccination coverage levels. JAMA.1996; 276 :626 –630[Abstract]
  18. Lieu TA, Capra AM, Makol J, Black SB, Shinefield HR. Effectiveness and cost-effectiveness of letters, automated telephone messages, or both for underimmunized children in a health maintenance organization. Pediatrics.1998; 101(4) . Available at: http://www.pediatrics.org/cgi/content/full/101/4/e3
  19. Rodewald LE, Szilagyi PG, Humiston SG, et al. The effects of an outreach-based recall system and a policy to reduce missed immunization opportunities on immunization coverage and primary care. Pediatrics.1999; 103 :31 –38[Abstract/Free Full Text]
  20. Bordley WC, Margolis PA, Stuart J, Lannon C, Keyes L. Improving preventive service delivery through office systems. Pediatrics.2001; 108(3) . Available at: http://www.pediatrics.org/cgi/content/full/108/3/e41
  21. Udovic S, Lieu TA. Evidence on office-based interventions to improve childhood immunization delivery. Pediatr Ann.1998; 27 :355 –361[ISI][Medline]
  22. Task Force on Community Preventive Services. Vaccine-preventable diseases: improving vaccination coverage in children, adolescents, and adults. MMWR Morb Mortal Wkly Rep.1999; 48 :1 –15
  23. Shefer A, Briss P, Rodewald LE, et al. Improving immunization coverage rates: an evidence-based review of the literature. Epidemiol Rev.1999; 21 :96 –142[Free Full Text]
  24. Briss PA, Rodewald LE, Hinman AR, et al. Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. Am J Prev Med.2000; 18(suppl 1) :97 –140[ISI][Medline]
  25. Bordley WC, Chelminski A, Margolis PA, Kraus R, Szilagyi PG, Vann, JJ. The effect of audit and feedback on immunization delivery: a systematic review. Am J Prev Med.2000; 18 :343 –350[CrossRef][ISI][Medline]
  26. Szilagyi PG, Bordley C, Vann JC, et al. The effect of patient reminder/recall interventions on immunization rates: a critical review. JAMA.2000; 284 :1820 –1827[Abstract/Free Full Text]
  27. Ad Hoc Working Group for the Development of the Standards for Pediatric Immunization Practices. Standards for pediatric immunization practices. JAMA.1993; 269 :1817 –1822[CrossRef][ISI][Medline]
  28. Institute of Medicine. Calling the Shots: Immunization Finance Policies And Practices. Washington, DC: National Academy Press; 2000
  29. Centers for Disease Control and Prevention. Development of community- and state-based immunization registries. MMWR Morb Mortal Wkly Rep.2001; 50 :1 –17[Medline]
  30. Haggerty RJ, Roghmann KJ, Pless IB. Child Health and the Community. New Brunswick: Transaction Publishers; 1993
  31. The Children’s Defense Fund. The State of America’s Children. Yearbook 1998. Washington, DC: The Children’s Defense Fund; 1998
  32. Monroe County Department of Health. Maternal/Child Health Report Card Update. February 2000. Data from 1990 US Census
  33. Rodewald LE, Szilagyi PG, Humiston SG, et al. Factors associated with undervaccination of preschool-age children: a case-control study. Abstracts from the 33rd Annual Meeting of the Ambulatory Pediatric Association. Pediatr Res.1993; 719 :122A
  34. Humiston SG, Rodewald LE, Szilagyi PG, et al. Decision rules for predicting undervaccination of preschool-age emergency department patients. J Pediatr.1993; 123 :887 –892[CrossRef][ISI][Medline]
  35. Rodewald LE, Maes E, Stevenson J, Lyons B, Stokley S, Szilagyi PG. Immunization performance measurement in a changing immunization environment. Pediatrics.1999; 103 :889 –897[Abstract/Free Full Text]
  36. Standards for maintaining, collecting and presenting federal data on race and ethnicity. Federal Register. October 30, 1997
  37. StataCorp. Stata Statistical Software: Release 6.0. College Station, TX: Stata Corporation; 1999
  38. Estimated vaccination coverage with individual vaccines and selected vaccination series, by 24 months of age by state and immunization action plan area—US, National Immunization Survey, 1999. Available at: http://www.cdc.gov/nip/coverage/nis/99/24months_iap.xls. Accessed January 9, 2002.
  39. Fiscella K, Franks P, Gold MR, Clancy CM. Inequality in quality: addressing socioeconomic, racial, and ethnic disparities in health care. JAMA.2000; 283 :2579 –2584[Abstract/Free Full Text]
  40. Katz JN. Patient preferences and health disparities. JAMA.2001; 286 :1506 –1509[Free Full Text]
  41. Miller LA, Hoffman RE, Baron AE, Marine WA, Melinkovich P. Risk factors for delayed immunization against measles, mumps and rubella in Colorado two-year-olds. Pediatrics.1994; 94 :213 –219[Abstract/Free Full Text]
  42. Strobino D, Keane V, Holt E, Hughart N, Guyer B. Parental attitudes do not explain underimmunization. Pediatrics.1996; 98 :1076 –1083[Abstract/Free Full Text]
  43. Szilagyi PG, Roghmann KJ, Rodewald LE, Winter NL, Humiston SG, Campbell JR. Immunization practices of primary care practitioners and their relation to immunization levels. Arch Pediatr Adolesc Med.1993; 148 :158 –166
  44. Szilagyi PG, Rodewald LE, Humiston SG, et al. Immunization practices of pediatricians and family physicians in the United States. Pediatrics.1994; 94 :517 –523[Abstract/Free Full Text]
  45. Taylor JA, Cufley D. The association between parental health beliefs and immunization status among children followed by private pediatricians. Clin Pediatr.1996; 35 :18 –22[Abstract/Free Full Text]
  46. Institute of Medicine. Overcoming Barriers to Immunization: A Workshop Summary. Washington, DC: National Academy Press; 1994
  47. Rodewald L. Every medical home needs an immunization recall system. AAP News. February 2001:89
  48. Schneider EC, Cleary PD, Zaslavsky AM, Epstein AM. Racial disparity in influenza vaccination: does managed care narrow the gap between African Americans and whites? JAMA.2001; 286 :1455 –1460[Abstract/Free Full Text]
  49. Smedley BD, Stith AY, Nelson AR, eds. Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care, Board on Health Sciences Policy. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care. Washington, DC: National Academy Press; 2002
  50. LeBaron CW, Starnes D, Dini EF, Chambliss JW, Chaney M. The impact of interventions by a community-based organization on inner-city vaccination coverage: Fulton County, Georgia, 1992–1993. Arch Pediatr Adolesc Med.1998; 152 :327 –332[Abstract/Free Full Text]
  51. Wood D, Halfon N, Donald-Sherbourne C, et al. Increasing immunization rates among inner-city, African American children. a randomized trial of case management. JAMA.1998; 279 :29 –34[Abstract/Free Full Text]
  52. Kenagy JW, Berwick DM, Shore MF. Service quality in health care. JAMA.1999; 281 :661 –665[Abstract/Free Full Text]
  53. Bauchner H, Homer C, Salem-Schatz SR, Adams W. The status of pediatric practice guidelines. Pediatrics.1997; 99 :876 –881[Free Full Text]
  54. Institute of Medicine. Crossing The Quality Chasm. Washington, DC: National Academy Press; 2001
  55. Stokley S, Rodewald LE, Maes EF. The impact of record scattering on the measurement of immunization coverage. Pediatrics.2001; 107 :91 –96[Abstract/Free Full Text]
  56. Newacheck PW, Stoddard JJ, Hughes DC, Pearl M. Health insurance and access to primary care for children. N Engl J Med.1998; 338 :513 –519[Abstract/Free Full Text]
  57. Centers for Disease Control and Prevention. Retrospective assessment of vaccination coverage among school-aged children—selected US cities, 1991. MMWR Morb Mort Wkly Rep.1992; 41 :103 –107[Medline]
  58. Centers for Disease Control and Prevention. Vaccination coverage of 2-year-old children—United States, 1993. MMWR Morb Mortal Wkly Rep.1994; 43 :705 –709[Medline]
  59. US Department of Commerce, Bureau of Census. 1990 Census of Population: General Population Characteristics, United States. Available at: http://www.census.gov/prod/cen1990/cp1/cp-1-1.pdf. Accessed March 8, 2002.
  60. Rodewald LE, Szilagyi PG, Humiston SG, et al. Is an emergency department visit a marker of undervaccination? Pediatrics.1993; 91 :605 –611[Abstract/Free Full Text]
  61. Klein JD, McNulty M, Flatau C. Adolescent’s access to care: teenagers’ self. Arch Pediatr Adolesc Med.1998; 152 :676 –682[Abstract/Free Full Text]
  62. Harris Interactive. Survey of Health Care Coverage and Access in Monroe County. Rochester, NY: Harris Interactive, Inc; 2001
  63. US Census Bureau. USA Counties, 1998. General Profile: Monroe County, NY Available at: http://tier2.census.gov/cgi-win/usac/table.exe. Accessed March 8, 2002
  64. Bartlett DL, Ezzati-Rice TM, Stokley S, Zhao Z. Comparison of NIS and NHIS/NIPRCS vaccination coverage estimates. Am J Prev Med.2001 :20 :25 –27[CrossRef][ISI][Medline]

PEDIATRICS (ISSN 1098-4275). ©2002 by the American Academy of Pediatrics



This article has been cited by other articles:


Home page
Arch Pediatr Adolesc MedHome page
P. G. Szilagyi, G. Fairbrother, M. R. Griffin, R. W. Hornung, S. Donauer, A. Morrow, M. Altaye, Y. Zhu, S. Ambrose, K. M. Edwards, et al.
Influenza Vaccine Effectiveness Among Children 6 to 59 Months of Age During 2 Influenza Seasons: A Case-Cohort Study
Arch Pediatr Adolesc Med, October 1, 2008; 162(10): 943 - 951.
[Abstract] [Full Text] [PDF]


Home page
Arch Intern MedHome page
K. Fiscella and R. M. Epstein
So Much to Do, So Little Time: Care for the Socially Disadvantaged and the 15-Minute Visit
Arch Intern Med, September 22, 2008; 168(17): 1843 - 1852.
[Abstract] [Full Text] [PDF]


Home page
Med Care Res RevHome page
M. Wasserman, D. Bender, and S.-Y. D. Lee
Use of Preventive Maternal and Child Health Services by Latina Women: A Review of Published Intervention Studies
Med Care Res Rev, February 1, 2007; 64(1): 4 - 45.
[Abstract] [PDF]


Home page
Arch Pediatr Adolesc MedHome page
M. E. O'Connor, B. S. Matthews, and D. Gao
Effect of open access scheduling on missed appointments, immunizations, and continuity of care for infant well-child care visits.
Arch Pediatr Adolesc Med, September 1, 2006; 160(9): 889 - 893.
[Abstract] [Full Text] [PDF]


Home page
Arch Pediatr Adolesc MedHome page
K. J. Dombkowski, M. M. Davis, L. M. Cohn, and S. J. Clark
Effect of missed opportunities on influenza vaccination rates among children with asthma.
Arch Pediatr Adolesc Med, September 1, 2006; 160(9): 966 - 971.
[Abstract] [Full Text] [PDF]


Home page
Health Promot PractHome page
S. E. Findley, M. Irigoyen, M. Sanchez, L. Guzman, M. Mejia, M. Sajous, D. A. Levine, S. Chen, and F. Chimkin
Community-Based Strategies to Reduce Childhood Immunization Disparities
Health Promot Pract, July 1, 2006; 7(3_suppl): 191S - 200S.
[Abstract] [PDF]


Home page
Arch Pediatr Adolesc MedHome page
P. G. Szilagyi, S. Schaffer, R. Barth, L. P. Shone, S. G. Humiston, S. Ambrose, and F. Averhoff
Effect of Telephone Reminder/Recall on Adolescent Immunization and Preventive Visits: Results From a Randomized Clinical Trial
Arch Pediatr Adolesc Med, February 1, 2006; 160(2): 157 - 163.
[Abstract] [Full Text] [PDF]


Home page
Am. J. Public HealthHome page
J. C. Blanchard, Y. Haywood, B. D. Stein, T. L. Tanielian, M. Stoto, and N. Lurie
In Their Own Words: Lessons Learned From Those Exposed to Anthrax
Am J Public Health, March 1, 2005; 95(3): 489 - 495.
[Abstract] [Full Text] [PDF]


Home page
Health Aff (Millwood)Home page
D. R. Nerenz
Health Care Organizations' Use Of Race/Ethnicity Data To Address Quality Disparities
Health Aff., March 1, 2005; 24(2): 409 - 416.
[Abstract] [Full Text] [PDF]


Home page
Arch Pediatr Adolesc MedHome page
D. L. Kirschke, A. S. Craig, W. Schaffner, J. R. Daugherty, J. Narramore, and M. R. Griffin
Childhood Immunization Rates Before and After the Implementation of Medicaid Managed Care
Arch Pediatr Adolesc Med, March 1, 2004; 158(3): 230 - 235.
[Abstract] [Full Text] [PDF]


Home page
Arch Pediatr Adolesc MedHome page
S. J. Hambidge, A. J. Davidson, S. L. Phibbs, V. Chandramouli, G. Zerbe, C. W. LeBaron, and J. F. Steiner
Strategies to Improve Immunization Rates and Well-Child Care in a Disadvantaged Population: A Cluster Randomized Controlled Trial
Arch Pediatr Adolesc Med, February 1, 2004; 158(2): 162 - 169.
[Abstract] [Full Text] [PDF]


Home page
Ann Fam MedHome page
M. A. Schillaci, H. Waitzkin, E. A. Carson, C. M. Lopez, D. A. Boehm, L. A. Lopez, and S. F. Mahoney
Immunization Coverage and Medicaid Managed Care in New Mexico: A Multimethod Assessment
Ann. Fam. Med, January 1, 2004; 2(1): 13 - 21.
[Abstract] [Full Text] [PDF]

P3Rs:

Read all P3Rs

Reducing inequalities in immunisation
Richard Reading
Pediatrics Online, 30 Dec 2002 [Full text]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow P3Rs: Submit a response
Right arrow P3Rs: View responses
Right arrow