PEDIATRICS Vol. 107 No. 4 April 2001, p. e49

ELECTRONIC ARTICLE:
Variation in Clinician Recommendations for Multiple Injections During Adoption of Inactivated Polio Vaccine

Tracy A. Lieu, MD, MPH^{*, §}, Robert L. Davis, MD, MPH, Angela M. Capra, MA^*, Loren K. Mell, BS, Charles P. Quesenberry, PhD^*, Kathleen E. Martin^*, Ann Zavitkovsky, MPH, MPA, Steven B. Black, MD^*, Henry R. Shinefield, MD^*, Robert S. Thompson, MD, and Lance E. Rodewald, MD, MS

From the ^* Division of Research and the Vaccine Study Center, Kaiser Permanente, Oakland, California;  Center for Health Studies, Group Health Cooperative, Seattle, Washington; ^§ Department of Ambulatory Care and Prevention, Harvard Pilgrim Health Care and Harvard Medical School, Boston, Massachusetts, Pediatric Clinical Effectiveness Program, Children's Hospital, Boston, Massachusetts, and Department of Health Policy and Management, Harvard School of Public Health, Boston, Massachusetts; and the  National Immunization Program, Centers for Disease Control and Prevention, Atlanta, Georgia.

	ABSTRACT

Top Abstract Methods Results Discussion Conclusion References

Objectives.  To describe variation in clinician recommendations for multiple injections during the adoption of inactivated poliovirus vaccine (IPV) in 2 large health maintenance organizations (HMOs), and to test the hypothesis that variation in recommendations would be associated with variation in immunization coverage rates.

Design.  Cross-sectional study based on a survey of clinician practices 1 year after IPV was recommended and computerized immunization data from these clinicians' patients.

Study Settings.  Two large West Coast HMOs: Kaiser Permanente in Northern California and Group Health Cooperative of Puget Sound.

Outcome Measures.  Immunization status of 8-month-olds and 24-month-olds cared for by the clinicians during the study.

Results.  More clinicians at Group Health (82%), where a central guideline was issued, had adopted the IPV/oral poliovirus vaccine (OPV) sequential schedule than at Kaiser (65%), where no central guideline was issued. Clinicians at both HMOs said that if multiple injections fell due at a visit and they elected to defer some vaccines, they would be most likely to defer the hepatitis B vaccine (HBV) for infants (40%). At Kaiser, IPV users were more likely than OPV users to recommend the first HBV at birth (64% vs 28%) or if they did not, to defer the third HBV to 8 months or later (62% vs 39%). In multivariate analyses, patients whose clinicians used IPV were as likely to be fully immunized at 8 months old as those whose clinicians used all OPV. At Kaiser, where there was variability in the maximum number of injections clinicians recommended at infant visits, providers who routinely recommended 3 or 4 injections at a visit had similar immunization coverage rates as those who recommended 1 or 2. At both HMOs, clinicians who strongly recommended all possible injections at a visit had higher immunization coverage rates at 8 months than those who offered parents the choice of deferring some vaccines to a subsequent visit (at Kaiser, odds ratio [OR]: 1.2; 95% confidence interval [CI]: 1.0-1.5; at Group Health, OR: 1.8; 95% CI: 1.1-2.8).

Conclusions.  Neither IPV adoption nor the use of multiple injections at infant visits were associated with reductions in immunization coverage. However, at the HMO without centralized immunization guidelines, IPV adoption was associated with changes in the timing of the first and third HBV. Clinical policymakers should continue to monitor practice variation as future vaccines are added to the infant immunization schedule.  Key words:  immunizations, vaccines, provider practices, practice variation, inactivated polio vaccine.

In the past decade, the number of injections recommended between birth and 18 months old has increased from 6 to a minimum of 11 and a maximum of 16.^1-3 Recent recommendations to use inactivated poliovirus (IPV)⁴ and acellular diphtheria-pertussis-tetanus (DTaP)⁵ vaccines have increased the immunization scheduling options open to clinicians and parents, as well as the required number of injections. Pneumococcal conjugate vaccination for infants, which the Advisory Committee on Immunization Practices has recommended, will add 4 more injections to this schedule.

When a child is due for multiple injections, parents and clinicians can elect to give them all simultaneously or to defer some vaccines until another visit.^6-9 Retrospective studies have suggested that missed opportunities attributable to deferred injections reduce immunization rates.^10-13 To date, there is little evidence that describes what clinicians recommend to parents when multiple injections are due or evaluates whether variation in such recommendations results in varying immunization coverage rates.

This study's aims were to: 1) describe the immunization practices of clinicians in 2 large health maintenance organizations (HMOs) after IPV and DTaP were recommended nationally; and 2) test the hypothesis that clinicians who recommend all vaccines simultaneously will have higher immunization coverage rates than those who offer parents the option of deferring some vaccines to later visits.

	METHODS

Top Abstract Methods Results Discussion Conclusion References

Design

This cross-sectional study described the immunization practices of clinicians in 2 HMOs in spring 1998, ~1 year after the Advisory Committee on Immunization Practices recommended adoption of a sequential IPV/oral poliovirus vaccine (OPV) schedule. We evaluated associations between self-reported clinician practices and the immunization coverage rates of their patients identified using computerized databases.

Study Populations

We studied clinicians and children in 2 large nonprofit HMOs: Kaiser Permanente (KP), which has ~2.8 million members in Northern California, and Group Health Cooperative of Puget Sound (GHC), which has ~530 000 members in the Seattle area. Most members of these HMOs receive health insurance as an employment benefit. KP and GHC are both group-model HMOs in which a large provider group contracts exclusively with the health insurance plan to provide care to members for capitated (predetermined) fees.

At each health plan, we selected a survey population of ~300 clinicians, prioritizing those who were most active in providing immunizations to children. At KP, we first identified all staff clinicians (pediatricians and nurse practitioners) with patient panels that included children younger than 1 year old at any time during 1997. We included in the survey sample all clinicians whose panel included 100 or more children who met these criteria; this represented 76% of the initially identified clinicians. At GHC, we included in the survey sample all staff clinicians (pediatricians, family practitioners, nurse practitioners, and physicians' assistants) who conducted at least 1 well-child visit in 1997. In both HMOs, temporary clinicians who provided mainly urgent care, residents, and other clinicians known to have retired or to have left the health plans were ineligible for the survey.

For the patient samples, we identified all children in each HMO who: 1) turned 8 months old between January 1 and June 30, 1998; or 2) turned 24 months old between July 1 and December 31, 1998. These children were chosen because they were the ones whose immunization status was most likely to have been affected by clinician practices at the time of our survey in the spring of 1998. To study associations between clinician practices and immunization coverage, we needed to link each patient to his or her provider. We did this by first identifying the clinician at each vaccine-associated visit, defined as the outpatient clinic visit closest before, but not >7 days before, each vaccine date. Then, each patient was assigned to a clinician by the following hierarchical rules: 1) the clinician to whom the most vaccine-associated visits were made; 2) in case of ties, the clinician listed as the personal provider on the computerized database of the HMO; 3) in case of ties and no personal provider listed, the clinician to whom the most recent vaccine-associated visit was made; and 4) in case of no vaccine-associated visits, the one listed as the personal provider. If there were no vaccine-associated visits and no personal provider listed, the patient went unmatched.

HMO Immunization Guidelines

Within KP, each of the 17 hospital-based medical centers and 16 freestanding medical clinics could make a local decision about whether and when to adopt IPV and DTaP. Within each medical center, individual clinicians could make differing immunization recommendations based on their own preferences. The HMO had not developed regional or national guidelines for immunizations. All recommended immunizations were fully covered without copayment. KP did not make the combination Haemophilus influenzae type b-hepatitis B vaccine (Hib-HBV) available for use.

At GHC, a committee of clinicians published and regularly updated immunization recommendations for all clinicians in the HMO. At the time of this study, GHC recommended DTaP at 2, 4, 6, and 15 months and 4 to 6 years old, Hib-HBV at 2, 4, and 15 months old, IPV at 2 and 4 months old, OPV at 15 months and 4 to 6 years old, and measles-mumps-rubella (MMR) at 15 months old. All nationally recommended immunizations were fully covered without copayment except varicella vaccination. The guidelines recommended clinicians give parents a pamphlet about varicella vaccine at 15 months and gave clinicians the option of administering the vaccine.

Data Collection

In spring 1998, we mailed each clinician a self-administered survey that included 13 closed-ended questions about their immunization practices. The introduction to the survey said that we were interested in what each clinician actually recommended to parents, whether these recommendations differed from those of his or her department, and that responses would be used to help improve immunization practices. The survey asked about practices with regard to IPV/OPV and DTaP or H influenzae type b-diphtheria-tetanus-pertussis vaccine (H-DTP), as well as how strongly the clinician recommended having all shots due at 1 visit when 3, 4, and 5 shots were due. The survey also asked about provider demographics. Providers were given a small monetary incentive or a gift certificate, and 2 waves of mailings were sent. The study protocols were approved by the institutional review boards of each health plan.

At the time of this study, most parents of infants making their 2-month clinic visit at KP were offered enrollment in a study of pneumococcal conjugate vaccine, which would add an additional injection at 2, 4, 6, and 15 months of age. The KP clinician survey asked respondents to report what they would recommend about multiple shots for routine patients, ie, those who were not getting extra shots because of being in a vaccine study.

Definitions of Immunization Delay

We chose the 8-month and 24-month birthdates as the ages at which to categorize children as either up-to-date or delayed for immunizations. To be up-to-date at 8 months, children needed 3 DTP-type vaccines (either DTaP or H-DTP), appropriate Hib vaccines (at KP, 3 Hib and/or H-DTP; at GHC, 2 Hib-HBV or 3 Hib), 2 polio vaccines (either IPV or OPV), and appropriate HBV (defined at KP as 2 HBV and at GHC as 1 monovalent HBV after 3 weeks of age or 2 Hib-HBV vaccines). To be up-to-date at 24 months, children needed 4 DTP-type vaccines, appropriate Hib vaccines (4 Hib and/or H-DTP at KP; 3 Hib-HBV or 4 HIB at GHC), 3 polio vaccines, appropriate HBV (3 HBV at KP; 3 Hib-HBV or 2 HBV after 3 weeks at GHC), and 1 MMR.

Statistical Methods

Preliminary analyses used univariate and bivariate statistical methods, including cross-tabulations, to identify provider practice variables and other predictor variables that were candidates for inclusion as independent variables in subsequent analyses. In multivariate analyses, we used logistic regression techniques to evaluate associations between predictor variables and the outcomes (immunizations up-to-date vs delayed at 8 months or 24 months). Parameter estimation was conducted with generalized estimating equations,¹⁴ (using the SAS Macro, Version 2.03 for generalized estimating equations; SAS, Cary, NC) to account for the fact that patients of a single provider are clustered and are not statistically independent observations with regard to the dependent variable, their immunization status.

We constructed models separately for KP and GHC because immunization schedules differed between the 2 HMOs. For KP, we conducted separate models evaluating the effects on immunization status (the dependent variable) of different independent variables, including recommendations for 1) polio vaccination; 2) the maximum number of injections; and 3) options for deferring multiple injections. At KP, approximately one half of patients had enrolled in a trial of pneumococcal conjugate vaccine. For these patients, routine immunizations were ordered by the primary care clinicians but were administered by research nurses along with the experimental vaccine. To address the possibility that the research process might have influenced vaccine delivery, we conducted a secondary analysis that excluded patients in the pneumococcal vaccine study. For GHC, because there was very little variability in polio vaccination practices and the maximum number of injections, we only modeled the effects of options for deferring multiple injections.

	RESULTS

Top Abstract Methods Results Discussion Conclusion References

Clinician Survey

Overall, 525 of the 585 eligible clinicians (90%) responded to the mailed survey (266/289 at KP and 259/296 at GHC). Of respondents at both HMOs, 63% had completed residency or professional training >10 years ago, while 25% had completed it 4 to 10 years ago, and 11% had completed it 3 or less years ago.

As Table 1 shows, clinicians reported greater variability in their polio vaccine recommendations at KP, where no guideline existed, than at GHC, where a central guideline recommended the IPV/OPV sequential schedule. The proportion of clinicians using the IPV/OPV sequential schedule was higher at GHC (82%) than at KP (65%). Among clinicians who recommended IPV at the 2-month visit, 66% of the combined sample said that they recommended it very strongly or somewhat strongly, whereas 32% said that they recommended it somewhat weakly or offered parents the choice between IPV and OPV.

Table 2 shows clinician recommendations when multiple injections were due. When 3 shots were due at a visit, most clinicians (76% overall) usually did not offer parents the option of deferring any shots to a follow-up visit. When 4 shots were due at a visit, 69% of KP clinicians usually did not offer parents the option of deferring any shots, whereas 63% of GHC clinicians said that they either offered the option of deferring shots or that this situation did not occur in their practice.

Clinicians reported that if they deferred any shots to a subsequent visit, they were most likely to defer hepatitis B vaccine for 0- to 11-month-olds (211 of the 525 respondents; 40%). At KP, IPV users were more likely to recommend the first HBV at birth (120/187 or 64%) than OPV users (13/46 or 28%; P < .01). However, among KP clinicians who were not recommending the first HBV at birth, IPV users (48/67 or 72%) were more likely than were OPV users (13/33 or 39%) to schedule the third HBV at 8 months or later (P < .01). Overall at KP, 124 of 266 clinicians (47%) reported recommending the third HBV at 8 months or later. GHC clinicians were not surveyed about their HBV practices because other analyses that we have conducted show minimal variability in these practices.¹⁵

Immunization Coverage

During the study, 11 582 infants at KP and 1411 infants at GHC turned 8 months old and were continuously enrolled from birth. Some patients were ineligible for study because they were not matched to any physician (n = 369 at KP and 191 at GHC) or because they were matched to a physician who was ineligible for the survey because of serving low numbers of infants (n = 2996 at KP and 2 at GHC). Among the 8217 eligible KP patients, 7655 (93%) were matched to a physician who had returned the survey. Among the 1218 eligible GHC patients, 1124 (92%) were matched to a physician who had returned the survey.

Of the infants in our final sample, 86% at KP and 77% at GHC were fully immunized by their 8-month birthdate. The DTP, Hib, and hepatitis B vaccines were the most frequently missed vaccines at KP; 7% lacked DTP and Hib only, 2% lacked hepatitis B only, and 1% lacked both. At GHC, the most frequent patterns among children underimmunized at 8 months old were DTP only (7%), DTP and Hib (4%), and hepatitis B only (2%).

Table 3 shows associations between clinicians' immunization recommendations and infants' immunization status at 8 months old. At KP, clinicians who used IPV or who recommended 3 or 4 injections at infant visits did not have different immunization rates compared with those who used OPV or who recommended 1 or 2 injections at infant visits. At both HMOs, infants were more likely to be fully immunized at 8 months old if their clinician reported strongly recommending all possible shots simultaneously, compared with always offering the parent the option of deferring 1 or more shots to a follow-up visit. This difference was statistically significant in the GHC model (odds ratio [OR]: 1.78; 95% confidence interval [CI]: 1.11-2.85) but not in the KP model (OR: 1.20; 95% CI: 0.99-1.45).

In both HMOs, Medicaid patients were less likely to be fully immunized at 8 months old compared with commercially insured patients (at KP, OR: 0.34; 95% CI: 0.12-0.94; at GHC, OR: 0.42; 95% CI: 0.30-0.61). Patients of nonwhite clinicians at KP had lower immunization rates (OR: 0.80; 95% CI: 0.68-0.95). Patient gender and years since the clinician completed residency training were not associated with immunization status in either HMO.

At KP, among the 8-month-olds in this study population, 3405 (44%) had participated in a randomized, controlled trial of pneumococcal conjugate vaccine. Because patients in this trial sometimes received their routine vaccinations from research nurses, we conducted a secondary analysis that excluded these patients. The general results of the analytic models for the remaining 4250 infants did not change. However, patients whose clinicians recommended an all-IPV schedule were significantly more likely to be fully immunized at 8 months old than those whose clinicians recommended all-OPV (OR: 1.61; 95% CI: 1.02-2.56).

We also analyzed the association of provider practices with the immunization status of children who turned 24 months old during the study. At 24 months of age, there were no differences in immunization coverage associated with how strongly clinicians said that they recommended all possible injections simultaneously. However, at KP, patients whose clinicians recommended a maximum of 4 injections at 1 of the visits between 12 and 18 months old had higher odds of being fully immunized at 24 months old than those whose clinicians recommended only 1 or 2 injections (OR: 1.41; 95% CI: 1.08-1.85). At KP, we conducted a secondary analysis restricted to 24-month-olds who had not participated in the pneumococcal conjugate vaccine trial (n = 2866 or 48% of the original 5973). Results were similar, but the association between clinicians' recommending a maximum of 4 injections at a visit between 12 and 18 months old and patients being fully immunized was not significant (OR: 1.10; 95% CI: 0.80-1.52).

	DISCUSSION

Top Abstract Methods Results Discussion Conclusion References

Major Findings

This study found that neither IPV adoption nor the use of multiple injections at infant visits were associated with reductions in immunization coverage. At Kaiser, where no central guideline for IPV use was issued, IPV adoption was slower and was associated with deferral of the third hepatitis B vaccine to later infant visits. At both HMOs, clinicians who strongly recommended all possible injections simultaneously had higher coverage rates for 8-month-olds.

Comparisons With Other Studies

The current research is unique in that we linked the self-reported practices of clinicians with the immunization outcomes of their patients. Previous immunization studies have used either patient records^10-13 or provider surveys,^6-9,16-20 but we are aware of only 1 other study that has attempted to directly evaluate associations between these. In the previous study, Taylor et al²¹ found that individual clinicians were the variable most strongly associated with immunization status in statistical models but could not pinpoint which clinician recommendations were important because the study included only 15 pediatricians. The current study, which included 525 clinicians and nearly 9000 eight-month-olds, had the statistical power and variability to identify specific clinician recommendations associated with immunization coverage.

Our findings differ from those of physician surveys in the mid-1990s in which providers expressed reluctancewhen presented with hypothetical situations of 3 or 4 injections being due at a visitto administer multiple injections.^7-9 A 1997 survey found that most providers would not defer injections when 3 were due at a visit.¹⁹ Currently, 4 injections falling due at infant visits is routine rather than hypothetical in the larger HMO that we studied. In our study, most providers said that they would recommend 4 injections at a given visit, and most of these said that they usually did not offer parents the option of deferring any injections to a follow-up visit.

Our finding that IPV adoption was not associated with delayed immunization agrees with our analysis of computerized immunization tracking data.¹⁵ At KP, IPV users tended to give the first hepatitis B at birth, a practice associated with timely receipt of other vaccines in another study.²² However, IPV introduction at KP seemed to cause clinicians who were not giving the first HBV at birth to schedule the third HBV later (at 8 months or later). Because the third HBV can be given any time between 6 and 18 months of age, follow-up studies will be needed to determine whether scheduling the third HBV after the 6-month visit will cause this vaccine to be missed. Our analyses to date have not found decreased immunization coverage of 24-month-olds overall.

This work has broad implications for children's health care delivery because it identifies an area of practice variation that potentially affects all US infants. The possibility that reducing practice variation can improve outcomes has been studied in adult conditions with varying results.^23-25 Few studies in pediatrics have evaluated whether practice variation results in differences in outcomes.^26-28 Some flexibility in vaccine scheduling seems warranted based on the variation in available combination vaccines and parent and clinician preferences. However, continued monitoring is warranted to ensure that such variation does not lead to reduced immunization coverage rates.

Limitations

This study was conducted in 2 closed-panel HMOs, 1 a staff-model and 1 a group-model, which have relatively cohesive provider groups. Other settings with more loosely organized provider networks and variable forms of reimbursement for vaccines are likely to have even greater variation in vaccine scheduling. Such variation will probably increase in the near future, when pneumococcal conjugate vaccination is introduced.

Our findings are most generalizable to insured populations with access to primary care providers. Relative to other factors, clinician recommendations might be a less potent influence on immunization status for children who lack either financial coverage for immunizations or primary care providers. At the larger HMO in this study, we excluded from the survey the 24% of clinicians who were least active in providing care to infants, who were often specialists. The findings, thus, may underrepresent the practices of physicians who see small numbers of infants for immunizations. However, there is little reason to believe that the associations that we observed between clinician recommendations and immunization coverage would not generalize to this group.

The associations observed in this study cannot be interpreted as causal relationships. For example, the fact that providers who strongly recommended all injections simultaneously had higher immunization coverage rates among 8-month-olds could have been caused by factors that this study did not evaluate, such as more adherent patient populations or local clinical interventions. We did not observe a similar association between strongly recommending all injections simultaneously and immunization coverage rates among 24-month-olds. However, clinicians who recommended a maximum of 4 or 5 injections at a visit between 12 and 18 months old had higher coverage rates for 24-month-olds than did those who recommended fewer injections simultaneously.

	CONCLUSION

Top Abstract Methods Results Discussion Conclusion References

Neither IPV adoption nor the use of multiple injections at infant visits were associated with reductions in immunization coverage. However, at the HMO without centralized immunization guidelines, IPV adoption was associated with changes in the timing of the first and third HBV. Clinical policymakers should continue to monitor the effects of multiple injections and practice variation as future vaccines are added to the infant immunization schedule.

	ACKNOWLEDGMENTS

This work was supported by the Vaccine Safety Datalink Program, National Immunization Program, Centers for Disease Control and Prevention.

We are grateful to Maureen Kolasa, MPH, for her advice on study design and survey questions. We thank the many clinicians of Kaiser Permanente and Group Health Cooperative who reported on their immunization practices for this study.

	FOOTNOTES

Received for publication Feb 1, 2000; accepted Aug 2, 2000.

Reprint requests to (T.A.L.) Department of Ambulatory Care and Prevention, Harvard Pilgrim Health Care and Harvard Medical School, 126 Brookline Ave, Suite 200, Boston, MA 02215. E-mail: tracy_lieu{at}hphc.org

	ABBREVIATIONS

IPV, inactivated polio vaccine; DTaP, diphtheria-tetanus-acellular pertussis vaccine; HMO, health maintenance organization; OPV, oral poliovirus vaccine; KP, Kaiser Permanente; GHC, Group Health Cooperative; Hib, Haemophilus influenzae type b vaccine; HBV, hepatitis B vaccine; MMR, measles-mumps-rubella; H-DTP, Haemophilus influenzae type b diphtheria-tetanus-acellular pertussis vaccine; OR, odds ratio; CI, confidence interval.

	REFERENCES

Top Abstract Methods Results Discussion Conclusion References

1. Centers for Disease Control and Prevention Recommendations of the Immunization Practices Advisory Committee (ACIP): general recommendations on immunization. MMWR Morb Mortal Wkly Rep 1989; 38:205-227 [Medline]
2. Centers for Disease Control and Prevention Recommended childhood immunization scheduleUnited States, 1999. MMWR Morb Mortal Wkly Rep 1999; 48:12-16 [Medline]
3. Committee on Infectious Diseases Recommended childhood immunization scheduleUnited States, January-December 1999. Pediatrics 1999; 103:182-185 [Free Full Text]
4. Centers for Disease Control and Prevention. Poliomyelitis prevention in the United States: introduction of a sequential vaccination schedule of inactivated poliovirus vaccine followed by oral poliovirus vaccine; recommendations of the Advisory Committee on Immunization Practice (ACIP). MMWR Morb Mortal Wkly Rep. 1997;46:No. RR-3
5. Centers for Disease Control and Prevention. Pertussis vaccination: use of acellular pertussis vaccines among infants and young children. MMWR Morb Mortal Wkly Rep. 1997;46:No. RR-7
6. Askew GL, Finelli L, Lutz J, Beliefs and practices regarding childhood vaccination among urban pediatric providers in New Jersey. Pediatrics 1995; 96:889-892 [Abstract/Free Full Text]
7. Woodin KA, Rodewald LE, Humiston SG, Physician and parent opinions: are children becoming pincushions from immunizations? Arch Pediatr Adolesc Med 1995; 149:845-849 [Abstract]
8. Melman ST, Chawla T, Kaplan JM, Multiple immunizations. Arch Fam Med 1994; 3:615-618 [Abstract]
9. Madlon-Kay DJ, Harper PG Too many shots? Parent, nurse, and physician attitudes toward multiple simultaneous childhood immunizations. Arch Fam Med 1994; 3:610-613 [Abstract]
10. Szilagyi PG, Rodewald LE, Humiston SG, Missed opportunities for childhood vaccinations in office practices and the effect on vaccination status. Pediatrics 1993; 91:1-7 [Abstract/Free Full Text]
11. Dietz VJ, Stevenson J, Zell ER, Potential impact on vaccination coverage levels by administering vaccines simultaneously and reducing dropout rates. Arch Pediatr Adolesc Med 1994; 148:943-949 [Abstract]
12. Lieu TA, Black SB, Sorel M, Would better adherence to guidelines improve childhood immunization rates? Pediatrics 1996; 98:1062-1068 [Abstract/Free Full Text]
13. Sabnis SS, Pomeranz AJ, Lye PS, et al. Do missed opportunities stay missed? A 6-month follow-up of missed vaccine opportunities in inner city Milwaukee children. Pediatrics. 1998;101(5). URL: http://www.pediatrics.org/cgi/content/full/101/5/e5
14. Liang K-Y, Zeger S Longitudinal data analysis using generalized linear models. Biometrika 1986; 73:13-22 [Abstract/Free Full Text]
15. Davis RL, Mell LK, Zavitkovsky A, Impact of the sequential IPV/OPV schedule on vaccination coverage levelsUnited States, 1997. MMWR Morb Mortal Wkly Rep 1998; 47:1017-1019 [Medline]
16. Freed GL, Clark SJ, Pathman DE, Impact of a new universal purchase vaccine program in North Carolina. Arch Pediatr Adolesc Med 1997; 151:1117-1124 [Abstract]
17. Ruch-Ross HS, O'Connor KG Immunization referral practices of pediatricians in the United States. Pediatrics 1994; 94:508-514 [Abstract/Free Full Text]
18. Szilagyi PG, Rodewald LE, Humiston SG, Immunization practices of pediatricians and family physicians in the United States. Pediatrics 1994; 94:517-523 [Abstract/Free Full Text]
19. Zimmerman RK, Schlesselman JJ, Baird AL, A national survey to understand why physicians defer childhood immunizations. Arch Pediatr Adolesc Med 1997; 151:657-664 [Abstract]
20. Centers for Disease Control and Prevention Use of a data-based approach by a health maintenance organization to identify and address physician barriers to pediatric vaccinationCalifornia, 1995. MMWR Morb Mortal Wkly Rep 1996; 45:188-192 [Medline]
21. Taylor JA, Darden PM, Slora E, The influence of provider behavior, parental characteristics, and a public policy initiative on the immunization status of children followed by private pediatricians. Pediatrics 1997; 99:209-215 [Abstract/Free Full Text]
22. Lauderdale DS, Oram RJ, Goldstein KP, Hepatitis B vaccination among children in inner-city public housing, 1991-1997. JAMA 1999; 282:1725-1730 [Abstract/Free Full Text]
23. Guadagnoli E, Hauptman PJ, Ayanian JZ, Variation in the use of cardiac procedures after acute myocardial infarction. N Engl J Med 1995; 333:573-578 [Abstract/Free Full Text]
24. Selby JV, Fireman BH, Lundstrom RJ, Variation among hospitals in coronary-angiography practices and outcomes after myocardial infarction in a large health maintenance organization. N Engl J Med 1996; 335:1888-1896 [Abstract/Free Full Text]
25. McCulloch DK, Price MJ, Hindmarsh M, A population-based approach to diabetes management in a primary care setting: early results and lessons learned. Eff Clin Pract 1998; 1:12-22 [Medline]
26. Ringer SA, Richardson DK, Sacher RA, Variations in transfusion practice in neonatal intensive care. Pediatrics 1998; 101:194-200 [Abstract/Free Full Text]
27. Horbar JD, Badger GJ, Lewit EM, Hospital and patient characteristics associated with variation in 28-day mortality rates for very low birth weight infants. Vermont Oxford Network. Pediatrics 1997; 99:149-156 [Abstract/Free Full Text]
28. Homer CJ, Szilagyi P, Rodewald L Does quality of care affect rates of hospitalization for childhood asthma? Pediatrics 1996; 98:18-23 [Abstract/Free Full Text]