Objective. To determine rates of immunization coverage among children 3 to 72 months of age in a large public housing development, to develop a community-based outreach program to increase coverage, and to evaluate the effect of the program.
Design. A door-to-door canvass of the development by specially trained emergency medical technicians to enroll families, to determine immunization status from written records, and to follow-up to encourage immunizations and well-child care. The program was evaluated, comparing rates of immunization by age with an expectation based on the immunization histories before enrollment.
Setting. A Chicago public housing development, October 1993 through December 1996.
Outcome Variables. Antigen-specific and series-specific coverage based on written records.
Results. Of the caregivers, 92% were able to identify a primary care provider. At the time of enrollment, 37% of 1075 children were up-to-date, but that proportion varied by age with 27% of children 19 to 35 months of age being up-to-date. The program increased rates of immunization compared with the expectation from the preenrollment rates. At their final assessment, 50% of the children were up-to-date. For individual vaccines, there was a positive program effect. For example, before enrollment, 22% of children 15 months of age had received measles, mumps, and rubella vaccine. However, 39% of children who were enrolled in the program before they were 12 months of age had received their first immunizations by 15 months of age.
Conclusions. Children in the housing development had very low rates of immunization before enrollment. An in-person intervention was effective in reaching families and determining immunization status. In the 3-year enrollment and observation period, rates of immunization increased.
The National Immunization Survey has found that nationally over three quarters of children 19 to 35 months of age are up-to-date for 4 doses of diphtheria, tetanus, and pertussis (DTP) vaccine, 3 doses of polio-containing vaccine (PCV), and 1 dose of measles, mumps, and rubella vaccine (MMR) (4:3:1).1 However, aggregate rates such as these mask considerable heterogeneity. In particular, there are some communities with concentrations of underimmunized children, communities that are therefore at increased risk for outbreaks of vaccine-preventable illnesses. Such communities have been termed pockets of need. The National Immunization Survey focuses on overall immunization coverage and does not identify pockets of need, leaving that identification to state and local agencies.2
Strong evidence that the city of Chicago contains pockets of need comes from a cluster survey conducted by the Chicago Department of Public Health in 1994, which found that among black children 19 to 35 months of age residing in public housing developments, the rate of 4:3:1 receipt was 29%. Moreover, 11% had received no immunizations.3
The project we describe here represents an intensive effort to document and track immunization rates in the largest public housing development in Chicago and to develop a strategy to improve those rates. We explain why a clinic-based reminder–recall program is not feasible in this community and instead propose an outreach program that develops its own community-based vaccine registry and provides reminder–recall service in person. We use the outreach program to reinforce families' ties to their medical home, and we present evidence of the program's impact.
The Robert Taylor Housing Development (RTH) is a Chicago Housing Authority public housing development originally consisting of 28 16-story apartment buildings with 10 apartments per floor. Of these buildings, 5 were demolished in 1997. The buildings are located along a 2-mile corridor adjacent to a major expressway on the south side of Chicago. The Chicago Housing Authority estimated in 1996 that 12 000 people resided in the RTH. Thus, RTH is the most densely populated public housing in the country. Of the residents, 51% are <15 years of age, compared with 22% of the city's population. An estimated 2800 children <6 years of age live in the housing development: 1800 of them in the 16 buildings eligible for the initial phase of this program.4
The 1990 census and Chicago Department of Public Health demographic data indicate that the community that includes the RTH development, Grand Boulevard, is one of the poorest and most violent in the city. The population of Grand Boulevard (36 000) is 99% black with a median household income of $7146. The unemployment rate is 34%; females head 80% of the households. Of the households, 78% receive public assistance (including Aid to Families with Dependent Children, Aid to the Aged, Blind and Disabled, General Assistance, Medicaid, and Food Stamps). Of the adults, 45% are high school graduates.5Of 77 Chicago community areas, Grand Boulevard ranks highest in birth rate, second highest in rates of low birth weight and death by homicide and fifth highest in infant mortality rate.5 The police district that includes RTH comprised 2.3% of the city's population in 1993 but accounted for 7.4% of the city's violent crimes in that year.6
Both mail and telephone communications with residents of the RTH are unreliable. Each building has a mailbox room; mailboxes are vandalized frequently and locks are broken. Gang violence has led to temporary suspension of mail delivery by the US Postal Service in some buildings. We have found that ∼60% of apartments do not have working telephones, a rate confirmed by other service providers (M. Gayle Hart, Chicago Department of Public Health, personal communication, February 4, 1999).
The Pediatric Immunization Program (PIP) began in 1993. The program employs community-based outreach workers to canvass RTH residents to identify children ≤71 months of age, determine their immunization status from written records, explain the immunization status and importance of well-child care to the caregiver, and revisit families as needed to reach and maintain up-to-date immunization status.
PIP hires and trains outreach workers who are high school graduates and current or former residents of Chicago Housing Authority developments. Workers receive extensive training in components and scheduling of American Academy of Pediatrics (AAP) and Advisory Committee for Immunization Practices (ACIP) recommendations for well-child care visits. The training curriculum includes for each recommended immunization information regarding the disease prevented, schedule for routine and catch-up immunizations, any recent changes in schedule recommendations, and the development of new combination vaccines and trade names that might be encountered on immunization records. Practice exercises were developed to assure proficiency in assessing immunization status and making recommendations for subsequent doses. Outreach workers were instructed to report any evidence of abuse or neglect that they encountered to the project manger, who would discuss the situation with the physician directing the program.
Before acquiring its own offices and outreach workers, PIP was nested within an existing community program, the First Aid Care Team (FACT). FACT emergency medical technicians provide immediate first aid response to residents of 16 buildings at RTH who call 911 for emergency services. FACT offices are on the first floor of 2 of the high rise buildings. The FACT program is sponsored by the Chicago Fire Department and the Jane Addams Hull House Association and medically supervised by the University of Chicago Hospitals' Department of Emergency Medicine. Selected FACT workers, generally current or former residents of public housing, were assigned exclusively to PIP. The initial PIP target area included the 16 RTH buildings served by FACT.
Wearing distinctive uniforms, PIP workers knock on every door in program buildings to identify children <6 years of age and pregnant women. All apartments are recanvassed periodically to identify additional age-eligible children who were not enrolled previously, who recently moved into the buildings, or who were born in the interim. Because of the limitations of mail and telephone service, door-to-door outreach is considered essential to enroll and maintain contact with families.
When a child is identified through the door-to-door canvass, the PIP worker records the parent or legal guardian's name, address, phone number if available, and the child's date of birth. The caregiver is asked to identify a primary health care provider for his or her child. If the caregiver cannot identify a provider, PIP workers distribute a list of neighborhood health care facilities. The PIP worker also asks for the parental opinion of the child's immunization status at the time of enrollment and whether there are immunization records in the home. If available these are used to assess the child's immunization status on the spot. Unverified parental recall of administered immunizations is not used for assessment. If there are no records in the home and the caretaker believes that the child has received any immunizations, the PIP worker obtains a signed release form to request the immunization record(s) from the designated clinic(s). If the records are not received within 2 weeks, PIP workers telephone the clinic/doctor's office; efforts are repeated to obtain records for 6 months, at which time PIP assumes immunization records are unavailable and that the child received no immunizations, in accordance with ACIP/AAP recommendations.
As soon as PIP is able to assess the child's immunization status, the date of each immunization is recorded on a form that provides a visual representation of the child's immunization history and ages for recommended immunizations and well-child care. This form is used to explain the child's present status and upcoming needs for immunizations and well-child care. Parents also are given copies of records received from clinics and urged to have them updated at clinic visits. Further, the program attempts to revisit families before due dates for future immunizations and well-child care (or leaves reminders when parents are not at home) and to revisit families to verify that immunizations were received (obtaining records from providers if not available in the home).
For personal safety and to avoid theft, the outreach specialists do not use computerized equipment. Data from handwritten forms are entered into a computer database at the University of Chicago Children's Hospital.
Recommendations of the AAP and the ACIP7–12 were used to define the age-appropriate immunization criteria for DTP, PCV, MMR, and Haemophilus influenzae type b (Hib). These are shown in Table 1. Initial immunization status was determined for each enrollee.
The length of time enrolled was calculated from the date of agreement to participate in the program until the last assessment date (either the last face-to-face parental contact, last receipt of records, or the end of this analysis period, December 31, 1996). Final immunization status was based on immunization records reviewed as of the last assessment date. Each child is categorized as being up-to-date for immunizations (all), having received no immunizations (none), or having received some of the recommended immunizations (some).
We also used a second method to evaluate program effect. Because PIP enrolls children from birth to 6 years of age, it follows that there are up to 6 years of immunization history for each new PIP enrollee. We used those previous immunization histories to construct rates of immunization coverage, were there no PIP. We compared the rate of immunization coverage by age before enrollment in PIP with the rate of immunization coverage by age for children after enrollment in PIP. This comparison allowed us to estimate program effect. We used the Kaplan-Meier product-limit estimator to determine the proportion who had received each individual immunization (eg, MMR and first DTP) by age, a method that allows adjustment for loss to follow-up.
Finally, we tested whether a secular trend in immunization status during the study period could explain our findings. To do this, we focused on MMR by 15 months. In a logistic regression model, we tested whether birth year significantly predicted the odds of receiving MMR by 15 months, after adjusting for month of PIP enrollment.
Data were entered into EpiInfo Version 6.0 (Centers for Disease Control and Prevention, Atlanta, GA) initially. These data were later transferred to, and new data entered into, EXCEL (Microsoft, Inc, Redmond, CA). STATA 5 (STAT Corp, College Station, TX) was used for the analysis.
Enrollment and Retention Data
A total of 510 families enrolled 1075 children <6 years of age in PIP from October 1993 through April 1996. Of the children, 11% were <7 months of age at the time of enrollment, 21% were between 7 and 18 months of age, 26% were between 19 and 35 months of age, and 43% were 36 to 71 months of age. Because enrollment and disenrollment occurred throughout the study period, the length of time a child was enrolled and retained in PIP varied. The mean length of enrollment was 13.2 months. Of the children in the program, 54% were enrolled for ≥1 year during the study period. Disenrollment occurred for several reasons: 72 enrollees (7%) aged-out at 72 months; 256 (24%) moved from the 16 buildings; 12 (1%) changed custody/guardianship; 11 (1%) were voluntarily withdrawn by the parent/legal guardian; 2 (<1%) were lost to follow-up; and 1 (<1%) died. An additional 34 (3%) moved within the 16 buildings and maintained enrollment in the program.
When asked about the usual source of care for their children, 92% of caregivers were able to name a clinic. Health care was dispersed widely; caregivers of the 1075 enrollees identified 83 clinics. Of these clinics, 5 accounted for 52% of the enrolled children's designated health care sites. One Chicago Department of Public Health clinic physically located in the RTH complex was designated by 22% of enrollees.
Overall, immunization records were obtained for 791 (73.6%) of the 1075 children. Of the children, 549 (51%) had immunization records in the home. Caregivers of 319 of the remaining 526 children (61%) signed a release of information form for the outreach workers to obtain immunization records from health care providers. Those not signing release forms said that the child had never received any immunizations, that they wanted to obtain the records themselves, or simply refused to sign the form. Records from clinics were received for 242 children (75.8% of the 319 for whom records were requested).
At initial assessment, PIP determined that 37% of children 3 to 71 months of age had received all age-recommended immunizations. Figure 1 demonstrates that the percentage of children up-to-date decreases with increasing age from 3 to 4 months through 16 to 18 months, as the number of immunizations required for up-to-date status increases. Only 12% of children 16 to 18 months of age are up-to-date. In this analysis, up-to-date status for children ≥19 months of age is 4:3:1 plus Hib. Because additional immunizations are not required to maintain up-to-date status once 4:3:1 plus Hib has been received, immunization status for each child can remain the same or improve after 19 months but cannot decline from up-to-date to some. This is reflected in the baseline data, because the proportion up-to-date increases with age for each age category from 19 to 35 months to 60 to 72 months. The effect of the school requirement for documented immunizations may explain the higher rates in the 2 oldest age groups.
Among those categorized as having received no immunizations are both those thus classified by parental report (ie, the parent could suggest no clinic from which records might be requested; n = 43) and those for whom the program could obtain no evidence of immunization (n = 139), but the parents thought the child had received some immunizations.
As we showed in a different setting,13 parental opinion is not a good predictor of verified immunization status. Of the children whose parents thought they were up-to-date, 33% were not, although 15% of children thought by their parents to have received some but not all their immunizations were in fact up-to-date.
Change in Status
More children were up-to-date at their final assessment, than at their initial assessment: 50% versus 37% (P < .001). However, some of the improvement would be attributable to the older age distribution at the final assessment, with more children potentially influenced by school entry requirements for kindergarten. Therefore, we also compared children 19 to 35 months of age at their final assessment, of whom 35% were up-to-date, with children 19 to 35 months of age at the time of enrollment, of whom 27% were up-to-date (P = .04).
Comparisons of the rate of receipt for individual vaccine antigens also demonstrate an effect of the program. Figure 2 compares the expected age distribution for receipt of a single MMR in the absence of PIP with the actual date of MMR receipt for children enrolled in the program by 12 months of age. The difference between the curves is the estimated gain in immunization from PIP. By 15 months of age, the expected rate of MMR receipt without PIP was 22%, whereas among PIP enrollees 39% had received MMR (P = .002). Figure 3 compares the expected age distribution for receipt of the first DTP dose (DTP1) in the absence of PIP with the actual dates of first dose receipt for children enrolled early by 60 days of age. At 3 months, the expected rate of DTP1 without PIP is 40%. However, among infants enrolled in PIP before 2 months of age, the rate of DTP1 receipt at 3 months was 56% (P = .08). The low number of infants enrolled in the first 2 months of age limits the ability to detect a significant difference in the percentage immunized for DTP1 compared with MMR, although the effect-estimates are similar.
To assess whether a secular trend toward higher rates of immunization could be contributing to the program effect we found, we tested whether birth year predicted the odds of receiving MMR by 15 months, after adjusting for month of PIP enrollment. There was no evidence that birth year explained the findings (P = .921).
Our data demonstrate that RTH is a pocket of need, with very low childhood immunization rates. Children are less and less likely to be up-to-date for immunization coverage as they age from 3 to 19 months. Only 27% of newly enrolled children 19 to 35 months of age are up-to-date for 4:3:1 plus Hib, a figure similar to the 29% up-to-date found by the Chicago Department of Public Health for black children in public housing using a cluster survey.3 A clinic-based reminder–recall program would be extremely difficult to implement in this community because of the unreliable mail service, low rates of telephone service, and a highly fragmented network of >80 primary care clinics. Nonetheless, a very high percentage of caregivers (92%) were able to name a clinic or primary care provider.
We use the previous immunization histories of newly enrolled children to allow us to assess the program effect. The cross-sectional enrollment data demonstrate a strong nonlinear association between age and immunization status. If we were to assess program effect by simply comparing immunization status for each child at last versus first assessment, our findings would be confounded by age. We demonstrated that rates of immunization improved for children enrolled in the program, although they were still low. In particular, the program was relatively successful at increasing the likelihood that an enrolled child would receive the next immunization on-schedule. Infants enrolled before their first immunization was due were more likely to have received that immunization on time: on-time receipt of the first immunization has been shown to be associated with improved immunization coverage among inner-city children.14
Because there is no control group, the design of the project does not allow for attribution of improvement to this intervention alone. Other efforts by public and private providers of health care and area-wide educational campaigns were occurring simultaneously. However, we do not find that a secular trend by birth year explains our finding of a program effect with respect to MMR receipt. However, that does not preclude the possibility that some of the effect we find is not attributable to PIP.
The program's priority was to gather the information needed to make the initial immunization status assessment and educate caregivers about that status. That initial intervention had a measurable effect on improving immunization coverage. However, the program was less successful in bringing children requiring several catch-up visits completely up-to-date, either because more persistent follow-up was needed or because the length of enrollment was inadequate.
Families were referred to their chosen primary health care site for several reasons. We wish to strengthen rather than weaken the role of the primary care provider and to encourage the maintenance of comprehensive records in 1 site. We did not wish to provide door-to-door immunization service, an approach that separates immunizations from comprehensive pediatric care, although the immunization rates themselves might have shown greater improvement. Children who are delayed in immunization receipt have been shown to be at risk for not receiving other components of primary pediatric care, such as lead and anemia screening.15 We cannot quantify PIP's impact on other components of well-child care, however, because our initial consent form only allowed us to request immunization records from clinics. The cost of the program is ∼$170 000 per year to follow ∼500 young children per year. Almost all the expense is salary for the outreach workers, project manager, data entry clerk, and compensated time for the physician director of the program.
The door-to-door approach is not without problems. Personal safety is a daily concern in a community that endures gang violence. At times, the movements of health care workers and community residents are limited by actual or threatened violence between competing gangs. PIP purchased bulletproof vests and walkie–talkies for the outreach workers. However, the outreach workers have rarely chosen to use them. Outreach workers work in pairs and may cancel home visits if they sense heightened tensions in the community. Gang members have not interfered directly with access to the buildings and apartments.
The weather also has been a hindrance to outreach efforts. The buildings have external walkways rather than internal hallways. Extremes of temperature have made it excessively uncomfortable and even dangerous for workers to spend hours outdoors during both winter cold snaps and summer heat waves.
Our findings have helped direct the next phase of PIP. We are placing more emphasis on enrolling newborns, before they have a chance to fall behind; and we are directing more of our effort to follow-up visits for children requiring multiple clinic visits to attain up-to-date status. Finally, we are exploring new strategies for coordinating our outreach with other service programs.
There are limitations to our ascertainment of initial immunization status and hence evaluation of program effects. We may have undercounted the previous immunizations children in the program received, both because some caregivers may have stated incorrectly that there were no clinics with immunization records for their children and also because we required written records. The large number of children assessed as none because no records were received may be attributable to parents incorrectly identifying the location where immunizations were received or to inadequate record keeping and response from clinics. Our documentation requirement, however, is consistent with survey strategies used by the Chicago Department of Public Health and national recommendations for providers. We also may have overestimated initial up-to-date status. The high percentage of families with records in the home may represent a biased enrollment of families who use preventive health care and cooperate with community health efforts.
The Kaplan-Meier hazard functions for those enrolled in the program assume that persons who are lost to follow-up are not different with respect to rate of immunization receipt after we cease observing them from those with whom follow-up is maintained. Whether this is correct cannot be determined from the data at hand. However, when we compared the hazard functions for those whom we ceased to observe because the study period ended with those whom we ceased to observe because they were lost to follow-up, we saw no consistent differences.
The Pediatric Immunization Program has demonstrated an ability to locate and track high-risk children in a community where communication and access are very difficult. We demonstrate a great need for improvements in the receipt of primary health care services, and we show that direct contact with the family can affect immunization rates.
This study was supported by the University of Chicago Hospitals, the Joyce Foundation, the R.R. McCormick Chicago Tribune Foundation (Cubs Care), People's Gas, Light and Coke Company, the Lloyd A. Fry Foundation, Chicago Community Trust (Major League Baseball Player's Association), the W.P. and H.B. White Foundation, City of Chicago Public Health Department, the Wyler Children's Hospital Annual Golf Classic, Polk Brothers Foundation, University of Chicago Women's Board, Pasteur Merieux Connaught, and the Grant Health Care Foundation.
We thank the late Jo Anne Merrit of the Jane Addams Hull House and Sunny Fisher of the Joyce Foundation, who provided the initial inspiration for this project. Laura Colon and Maria Withrow provided coordination between the outreach effort and the University of Chicago. Ralph Muller and the University of Chicago Hospitals provided annual financial support and emergency supplementation as needed. Our outreach workers braved adverse working environments because they believe in this program. Sandra Potts and Lisa Verber currently carry the banner forth each day. Cheryl Byers of the City of Chicago Public Health Department has provided daily encouragement for our effort, as have Centers for Disease Control and Prevention staff members Ed Mihalek and Mike Mattuck.
- Received November 10, 1998.
- Accepted June 14, 1999.
Reprint requests to (R.S.D) Department of Pediatrics, University of Chicago, 5841 S Maryland Ave, MC 6054, Chicago, IL 60637-1470. E-mail:
- DTP =
- diphtheria, tetanus, and pertussis vaccine •
- PCV =
- polio-containing vaccine •
- MMR =
- measles, mumps, and rubella vaccine •
- 4:3:1 =
- 4 doses of DTP, 3 doses of PCV, and 1 dose of MMR •
- RTH =
- Robert Taylor Housing Development •
- PIP =
- Pediatric Immunization Program •
- AAP =
- American Academy of Pediatrics •
- ACIP =
- Advisory Committee for Immunization Practices •
- FACT =
- First Aid Care Team •
- Hib =
- Haemophilus influenzae type b •
- DTP1 =
- first DTP dose
- Centers for Disease Control and Prevention
- ↵Centers for Disease Control and Prevention. National Immunization Program: Methods Used To Identify Pockets of Underimmunized Children Not Evaluated. Atlanta, GA: Centers for Disease Control and Prevention; October 1, 1997. GAO/HEHS-97-136R
- Kenyon TA,
- Matuck M,
- Stroh G
- ↵Chicago Housing Authority. Residential Accounting Detail Report: Summary of Young Children. Chicago, IL: Chicago Housing Authority; March 11, 1996
- ↵Department of Public Health. City of Chicago. Community Area Health Inventory, 1992–1994: Demographic and Health Profiles. Vol 1. Chicago, IL: Department of Public Health; April 1996:86–87
- ↵Chicago Department of Police. Monthly Summary—Incidence of Crime-District Analysis. Chicago, IL: Chicago Department of Police; 1993. DPOL 7352-01
- Centers for Disease Control and Prevention
- Advisory Committee on Immunization Practices
- Centers for Disease Control and Prevention
- Mitchell CK,
- Franco SM
- Copyright © 1999 American Academy of Pediatrics