OBJECTIVE. Incorporating Healthy Steps for Young Children into pediatric practice has been shown to have positive effects for children and families. Although this model of care has also been integrated into several pediatric and family medicine training programs, published reports to date have focused only on residents’ perceptions of their interactions with the model of care. In this study, we report the impact on primary care outcomes after integrating Healthy Steps for Young Children into residency training.
METHODS. Continuity of care, longitudinal care in the practice, quality of primary care services, and rates of behavioral, developmental, and psychosocial diagnoses were measured for 3 cohorts: (1) Healthy Steps–enrolled children, (2) non–Healthy Steps–enrolled children who were followed at the same site of care, and (3) non–Healthy Steps–enrolled children who were receiving primary care at a similar residency training site within the same training program. All data were extracted from patient charts at the 2 practice sites.
RESULTS. Continuity of care was significantly better for Healthy Steps–enrolled children compared with non–Healthy Steps–enrolled children at the Healthy Steps site for both total visits and health maintenance visits. Longitudinal care and quality of primary care services did not differ within or between sites. The rates of documentation of behavioral, developmental, or psychosocial diagnoses did not differ between Healthy Steps–enrolled and non–Healthy Steps–enrolled children at the Healthy Steps for Young Children site but were significantly different between the Healthy Steps and the non–Healthy Steps for Young Children sites; the effect was driven wholly by differences in psychosocial diagnoses.
CONCLUSIONS. Multiple indices that measure health service outcomes suggest benefits of incorporating Healthy Steps for Young Children into pediatric residency training. Most important, continuity of care in residents’ practices significantly improved, as did the residents’ documentation of psychosocial issues in children.
Continuity clinics are an important and required pediatric residency training experience. The Accreditation Council for Graduate Medical Education's Pediatric Residency Review Committee explicitly defines requirements for this experience and a complementary curriculum.1 In particular, continuity clinic training should replicate a generalist, pediatric, real-world primary care experience. It should model a longitudinal patient-care experience with continuity, coordination, and comprehensiveness of care. Furthermore, the Residency Review Committee incorporates training requirements in developmental and behavioral pediatrics for both subspecialty and general pediatrics. These include training in normal and abnormal behavior, physical and emotional development, interviewing skills, developmental and psychosocial screening, and developing appropriate skills to meet the needs of children and families who are at risk from factors such as poverty and fragmented relationships.1 These training issues are also reflected in the strategic pediatric workforce concerns expressed in the Future of Pediatric Education II, particularly the critical need for enhanced training in neurodevelopmental and behavioral pediatrics.2 Although widely accepted as important goals, many, if not most, residency training programs struggle to implement these recommendations and requirements.3,4
Hospital-based clinics are frequently the site for residents’ continuity clinic training and have been shown to provide a clinical spectrum of pediatric primary care experiences similar to what is seen in office-based pediatric practice.5 As well, residents who are trained in these settings are exposed to children with behavioral, developmental, and psychosocial issues, although these conditions may not be fully recognized by residents.6 One challenge facing residency training is to define the structural changes that will more fully use available resident learning experiences.7
Healthy Steps for Young Children (HS) is a practice model based partly on findings of a national survey of parents. HS addresses issues identified by parents as important for managing the stresses of raising children and for fostering the physical, emotional, and intellectual growth and development of their children.8
The key element of HS is the introduction of the Healthy Steps Specialist (HSS), a co-practitioner with training in early childhood development, into the practice setting to focus on behavioral, developmental, and psychosocial issues. HS and its genesis have been described in detail and have been and continue to be evaluated.9–17 Integration of an HS program into residency training, with its emphasis on continuity of care (COC) and early childhood development and behavior, seems to be a promising approach to address the training needs and recommendations identified.
In July 1997, the Department of Pediatrics at the University of Illinois at Chicago established a Pediatric Primary Care Residency that incorporated HS from the outset. Residents in the primary care residency track had their continuity clinic experience at Ravenswood Hospital and Medical Center (RH) on the north side of Chicago. In this setting, an HSS and a pediatric resident jointly conducted well-child visits, with both providers present in the examination room for most of the visit. This enabled residents to see the HSS create “teachable moments,” an effective communication skill that is used to address parents’ issues; initiate discussions of common behavioral and developmental concerns; and broach sensitive topics such as maternal depression, parental disagreement regarding childrearing, and domestic violence.18 The co-practice activities with the HSS also enabled residents to participate in a systematic assessment of infant and toddler development and behavior and to gather information about child–parent relationships. In addition, exposure to assessment tools such as the Infant Temperament Scale and to concepts such as “goodness of fit” afforded residents opportunities to understand behavioral issues better.19,20 Residents were expected to provide care with the HSS for as many HS families as possible, with the minimum acceptable number being 5 families during residency.
Home visiting, a component of HS, has been shown to have both clinical and resident educational value.21,22 Residents, during the first year of their training, made at least 3 home visits with the HSS to families whom they followed in their continuity clinics and were encouraged but not required to make additional visits during the second and third years of their training. During home visits, residents, with the HSS, gathered additional information about parent–child and family relationships, conducted developmental screenings, conducted home safety assessments, and built rapport with families outside the office setting.
The HSS also helped residents become familiar with community-based services for families, both by linking families to such services and by arranging visits for residents to venues such as homeless and domestic violence shelters and teen parenting programs. The HSS additionally modeled a liaison role by making developmental and mental health referrals for families. Finally, the HSS and senior residents conducted monthly team conferences during which they discussed cases from the practice that involved behavioral, developmental, and psychosocial issues, such as feeding difficulties, discipline, toilet training, sleep problems, maternal depression, or domestic violence.
Except for their HS experiences, the training in behavioral and developmental pediatrics was the same for residents in the primary care track as that of their colleagues in the traditional track. Residents in the traditional track had little or no exposure to HS activities. The study reported here was undertaken to examine the impact on primary care outcomes after integrating HS into the residents’ continuity clinic experience.
Participants and Data Sources
A total of 363 children in 3 cohorts were enrolled in this study. All children who were born at RH between July 1, 1997, and March 31, 1998; met the enrollment criteria of the national HS program (regardless of whether they were actually enrolled); were followed in the residents’ continuity clinics; and had retrievable medical charts were included when (1) the first visit to the resident was made by 28 days of age for HS-enrolled children or 61 days of age for non–HS-enrolled children, (2) the parents spoke English or Spanish, and (3) there were no medical problems that precluded the participation of the non–HS-enrolled children by 28 days of age (as defined by the national HS exclusion criteria).10 The HS-enrolled and non–HS-enrolled children at RH composed the first 2 cohorts; these children received their care from residents who participated in the HS curriculum. The third cohort consisted of 100 consecutive children who met the same selection criteria as the non–HS-enrolled children at RH and were born during the same period. These children received their care from residents who had their continuity care experience at University of Illinois Hospital (UIH) and who had little or no contact with HS. Six faculty physicians supervised residents in their continuity clinics; 4 of these physicians supervised at both RH and UIH, 1 supervised only at RH, and 1 supervised only at UIH. This study was approved with a waiver for informed consent by the institutional review boards at both institutions. Eighty-nine families were enrolled in HS during the study period, and during a 3-year period, first-, second-, and third-year pediatric residents co-practiced with an HSS to provide these families with primary care services.
Data Extraction and Coding
A total of 263 charts were retrieved at RH, the HS site. Seventy-one (80%) of 89 HS-enrolled children had medical charts that could be retrieved. The charts of 192 non–HS-enrolled children who received primary care at RH were retrieved, and data were extracted. One hundred charts were retrieved and data were extracted for children who were seen at UIH, the non-HS site. Data at RH were extracted from birth through 36 months of age. The charts at both sites contained only physician-visit notes, and no data were extracted from notes made by HSS.
Because of conversion to an electronic medical chart, resulting in charts that, for clinical study purposes, were not fully retrievable, only data through 18 months of age were extracted from the medical charts at UIH. Table 1 illustrates the numbers and characteristics of participants in each cohort.
A trained research assistant and attending faculty physicians performed the data extraction from the study patients’ medical charts. The data were obtained from the original medical chart and were entered for all general pediatric outpatient visits. The data extraction form included the date of visit, visit type, visit physician, problems/diagnoses (up to 4 per visit), medications, referrals, immunizations, developmental assessment, and the dates of hemoglobin and lead screening tests. No information outside the medical chart was available for analysis. Chart extractors were not blinded to the site of care or the HS status of the child.
Each problem or diagnosis was further categorized as behavioral, developmental, psychosocial, or none of these. Because there had been no a priori categorization of behavioral, developmental, or psychosocial diagnoses or conditions, a modified Delphi process was used to identify such categories. After chart extraction was completed, all diagnoses were listed in alphabetical order in the words used in the medical chart. This list contained no information regarding site of care or HS status of child. The list of diagnoses was given to 7 attending physicians and general academic pediatric fellows in the Division of General and Emergency Pediatrics at the University of Illinois at Chicago. They classified each diagnosis into 1 of 7 mutually exclusive categories: (1) definitely behavioral; (2) definitely developmental; (3) definitely psychosocial; (4) possibly behavioral; (5) possibly developmental; (6) possibly psychosocial; or (7) definitely not behavioral, developmental, or psychosocial. Using a least restrictive paradigm, all diagnoses or problems that were considered to be definitely or possibly developmental, behavioral, or psychosocial by at least 4 of the 7 physicians were included in the analysis.
Main Outcome Measures
Continuity of Care
COC for all clinic visits to RH was calculated for each patient using the COC index of Bice and Boxerman.23 This index measures COC from a patient-centered perspective and is calculated for each patient according to the formula where N is the total number of visits, ni is the number of visits to provider I, and s is the number of providers.
This index varies from 0 to 1, where 0 indicates that all visits were made with a different provider and 1 indicates that all visits were made with a single provider. COC was calculated for each child twice, first for all visits to the clinical site, regardless of visit type, and separately for health maintenance visits.
Table 2 provides an example of the possible values of the COC index applied to a patient who has 6 clinical visits with up to 4 providers (A, B, C, and D). Regardless of the sequence of visits, the COC index changes with the number of providers and number of visits to each provider. A qualitative rating reflects an assessment of excellent, good, and poor COC, where excellent indicates (in this example) that at least two thirds of the 6 visits were made to 1 provider (COC index >0.40), good indicates that at least half of the visits were made to 1 provider (COC index ≤0.40 and >0.20), and poor indicates greater dispersion among providers (COC index ≤0.20). A similar qualitative scoring system has been used previously.24
Longitudinal care was measured as the duration of care from birth to the last recorded visit up to a maximum of 3 years of age (1095 days). A maximum of 3 years was chosen because the HS program is a birth–to–3-years intervention.
Quality of Primary Care Services
Quality of primary care services was assessed by measurement of the percentage of children who were aged 19 months and completed their immunizations on time, according to the then-current recommendations of the Advisory Committee on Immunization Practice (4 diphtheria-tetanus-acellular pertussis, 3 inactivated poliovirus, 1 measles-mumps-rubella, 1 varicella, 3 hepatitis B, and at least 1 Haemophilus influenzae type B after 12 or 15 months of age), and the percentage of children who completed screenings for hemoglobin/hematocrit and lead level by 15 months of age. These percentages were calculated as the number of children with these criteria (complete immunizations or recorded hemoglobin) divided by the number of children who remained in the practice at the age surveyed. Although not a comprehensive measure of quality of primary care, these indices are routine measures of the quality of preventive primary care services.25
Rates of Behavioral, Developmental, and Psychosocial Problems
Rates of behavioral, developmental, and psychosocial problems were calculated as simple percentages, with number of children as the denominator.
Comparisons between the cohorts were conducted primarily using nonparametric statistics. The Wilcoxon rank-sum test was used to make pairwise comparisons of COC between the 2 RH cohorts. Proportions of children who received primary care within the practice through different ages (the longitudinality of care measure); proportions who received complete immunizations or screenings (the quality of primary care measure); and proportions with a recorded behavioral, developmental, or psychosocial diagnosis were compared with Fisher's exact tests. Mean numbers of visits and numbers of diagnoses were compared using t tests.
Continuity of Care
The COC indices for total visits and for health maintenance visits were greater at RH for the HS-enrolled children compared with non–HS-enrolled children (Wilcoxon rank-sum: z = 4.56 [P < .05] for total visits and z = 4.39 [P < .05] for health maintenance visits). The median COC index scores for all visits were 0.24 for HS-enrolled children and 0.11 for non–HS-enrolled children and for health maintenance visits only were 0.36 and 0.20, respectively. Figure 1 displays the distribution of COC index scores grouped qualitatively, where poor represents children with a COC index from 0.0 to 0.2, good represents COC index from >0.2 to 0.4, and excellent represents COC index >0.4. Figure 1A shows the distribution for all visits and Fig 1B for only health maintenance visits. Fifty-two percent of HS-enrolled children had a COC index for health maintenance visits >0.4, compared with 28% of children who did not receive HS interventions.
HS enrollees also had significantly more health maintenance visits (mean: 7.96 vs 7.15, t261 = 1.99, P < .05) but not significantly more total visits (mean: 13.31 vs 11.98, t261 = 1.41, P = .16) than non-HS enrollees. Comparable COC could not be evaluated for children at UIH.
Longitudinality of Care
Longitudinal care did not differ significantly between HS and non-HS enrollees at RH for any period. Table 3 shows the percentage of children who continued to receive primary care within their practice site at various ages. The quality of the UIH medical charts limited our ability to make comparisons after the first year of life. Longitudinal care at RH for HS-enrolled and non-HS–enrolled children was similar and, through the first year of life, also was similar for children who were seen at UIH.
Quality of Primary Care
There were no differences in quality of preventive services care outcomes between HS-enrolled and non–HS-enrolled children at RH. Seventy-four percent of HS-enrolled children had complete immunizations by 19 months of age compared with 70% of non–HS-enrolled children. There were similar rates for anemia and lead screening between cohorts: 73% and 67%, respectively, for HS-enrolled children and 77% and 64% for non–HS-enrolled children.
Behavioral, Development, and Psychosocial Diagnoses
Twenty-four (33.8%) of 71 HS-enrolled children had documentation of 55 behavioral, developmental, and psychosocial diagnoses during the study period; 76 (40.2%) of 192 non–HS-enrolled children had 120 diagnoses recognized. Neither the number of diagnoses per child nor the number of diagnoses per child-year differed statistically between these cohorts by Wilcoxon rank-sum test.
Because longitudinal care through 12 months of age was similar at RH and UIH, we compared the number of behavioral, developmental, and psychosocial diagnoses during the first year of life at UIH with the number during the first year of life at RH for the HS-enrolled and non–HS-enrolled groups combined. At UIH, 14 (14%) children had 17 behavioral, developmental, and psychosocial diagnoses during the first year of life. At RH, 44 (23%) non–HS-enrolled children had 55 such diagnoses, and 16 (21%) HS-enrolled children with 27 had diagnoses. The percentage of children with at least 1 behavioral, developmental, or psychosocial diagnosis in their first year of life was significantly lower at UIH compared with RH (combining HS-enrolled and non–HS-enrolled children; Fisher's exact test, 1-tailed: P = .04); the effect is driven wholly by psychosocial diagnoses in year 1 (Fisher's exact test, 1-tailed: P = .024). The 1-tailed percentages did not differ between HS-enrolled and non–HS-enrolled children at RH (Fisher's exact test, 1-tailed: P = .53; Fig 2).
Multiple indices suggest important benefits of incorporating an HS curriculum into pediatric residency training. COC was significantly better for HS-enrolled children than for non–HS-enrolled children at RH. There were numerically, although not statistically significant, more developmental, behavioral, or psychosocial diagnoses in the HS-enrolled cohort of children at RH than in the non–HS-enrolled cohort. Furthermore, the percentage of children with at least 1 behavioral, developmental, or psychosocial diagnosis in their first year of life was significantly lower at UIH, where no residents were exposed to the HS program. The significantly improved COC at RH for HS-enrolled children can reasonably be attributed to the HS program, because non–HS-enrolled children received care at the same site from the same residents and attending physicians. We postulate that inclusion of the HSS in clinic and home visits was perceived by families as adding sufficient value to their care to improve continuity, a concept supported by similar findings in 2 other evaluations of HS in pediatric practice where parents expressed higher satisfaction with care and were more likely to remain in the practice.11,12 Worthy of note is that the COC for HS residents’ patients was greater than was reported in another study of pediatric residents’ COC practices and was similar in quality to that reported in a university hospital–based practice, where faculty, not residents, provided the majority of care.4,26
That documentation of developmental, behavioral, or psychosocial issues was similar for the HS-enrolled and non–HS-enrolled children at RH suggests that residents’ experiences with the HSS, when providing care for the HS cohort of children, had a substantial impact on their ability to uncover and document such issues regardless of whether the HSS was present. That RH residents, compared with UIH residents, documented a significantly greater percentage of children with psychosocial issues during the first year of life further supports this interpretation. Alternatively, the results may be explained by true differences in the rates of psychosocial (although not behavioral or developmental) issues in the populations that attended the 2 clinics. Both clinic sites provided care for children who were predominantly insured by Medicaid, but other differences such as ethnicity and parental education were not available for analysis. The classification of developmental, behavioral, and psychosocial diagnoses and issues as used in this study, although carefully developed, should be tested in other settings to determine suitability for broader use.
There were no statistically notable differences between the HS-enrolled and non–HS-enrolled cohorts of children regarding on-time immunizations at 19 months of age or for percentage of children who remained in the practice at 24 and 36 months of age, both outcomes that were improved in the larger national evaluation of HS.11 However, the absolute differences in that study were similar to the trended differences between our cohorts. Statistical significance for differences this small would not be expected in this study, with its much smaller cohort sizes.
The small sample size and retrospective nature of this study limit more in-depth exploration of additional educational impact, as well as generalization to other settings. However, evidence of significant impact on residents’ COC of HS-enrolled children while in training and their increased documentation of psychosocial issues in families are noteworthy outcomes of this study of the impact of HS in pediatric residency education. Ideally, future educational intervention studies will address some of the inherent shortcomings of a retrospective cohort design. More specific, random assignment of residents to an HS or non-HS curriculum at both sites of care would have removed confounding factors related to differences in both practice and population variables between RH and UIH and to differences between residents in the primary care and traditional residency tracks. These prospective studies should be conducted in other pediatric as well as family medicine training programs, most importantly to assess whether experience with an HSS during residency will influence postgraduates’ everyday pediatric practice. Data from follow-up surveys of our residency program graduates suggest just this (unpublished data). If true, then it might mitigate the necessity of incorporating the HSS into pediatric practices, thereby addressing a major cost deterrent to the widespread integration of HS in real-world practice.27 Alternatively, enhanced residency training of general pediatricians in these areas may accelerate the introduction of both practice-based and community-based programs that focus on optimizing child health and development, as graduating residents advocate for family-focused services that are comparable to those that they experienced during their training.
This project was supported by grant 20020427 from the Commonwealth Foundation.
We acknowledge the support of the Commonwealth Fund, particularly Steven Schoenbaum and Ed Schor, for encouragement and advice. Also, we are grateful to Margaret Mahoney for fully embracing the expansion of the HS model into pediatric residency training. We also acknowledge the support of Mike Barth, Healthy Steps National Director. In addition, we are grateful to Jennifer Dubrow-Prozement and Rose Ruiz, HSS; to the pediatric residents who participated in HS; and to Dominic Starosta, who assisted with data collection
- Accepted January 30, 2007.
- Address correspondence to Leo G. Niederman, Department of Pediatrics (mc 856), 840 S Wood St, Chicago, IL 60612. E-mail:
The authors have indicated they have no financial relationships relevant to this article to disclose.
This work was presented in part at the annual meeting of the Pediatric Academic Societies; May 1–4, 2004; San Francisco, CA.
- ↵Accreditation Council of Graduate Medical Education. Program Requirements for Residency Education in Pediatrics. Chicago, IL: Accreditation Council of Graduate Medical Education; 2006:1– 44
- ↵Leslie L, Rappo P, Abelson H, et al. Final Report of the FOPE II Pediatric Generalists of the Future Workgroup. Pediatrics.2000;106(suppl) :1199– 1223
- ↵Dumont-Driscoll MC, Barbian LT, Pollock BH. Pediatric residents’ continuity clinics: how are we really doing? Pediatrics.1995;96 :616– 621
- ↵Garfunkel LC, Byrd RS, McConnochie KM, Auinger P. Resident and family continuity in pediatric continuity clinic: nine years of observation. Pediatrics.1998;101 :37– 42
- ↵Wissow LS, Larson S, Anderson J, Hadjiisky E. Pediatric residents’ responses that discourage discussion of psychosocial problems in primary care. Pediatrics.2005;115 :1569– 1578
- ↵Zuckerman B, Kaplan-Sarnoff M, Parker S, Young KT. The Healthy Steps for Young Children Program. Zero to Three.1997;June/July:20–25
- ↵Zuckerman B, Parker S, Kaplan-Sanoff M, Augustyn M, Barth MC. Healthy Steps: a case study of innovation in pediatric practice. Pediatrics.2004;114 :820– 826
- ↵Zuckerman B, Parker S. Teachable moments: assessment as intervention. Contemp Pediatr.1997;14 :103– 118
- ↵Lester BM, Zachariah B, Garcia-Coll CT, et al. Developmental outcomes as a function of the goodness of fit between the infant's cry characteristics and the mother's perception of her infant's cry. Pediatrics.1995;95 :516– 521
- ↵Starfield B. Primary Care: Balancing Health Needs, Services and Technology. New York, NY: Oxford University Press; 1998
- Copyright © 2007 by the American Academy of Pediatrics