PEDIATRICS Vol. 108 No. 3 September 2001, p. e42

ELECTRONIC ARTICLE:
From Concept to Application: The Impact of a Community-Wide Intervention to Improve the Delivery of Preventive Services to Children

Peter A. Margolis, MD, PhD^*, Rachel Stevens, EdD, RN, W. Clayton Bordley, MD, MPH^*, Jayne Stuart, MPH^*, Christina Harlan, RN, MA, Lynette Keyes-Elstein, Dr PH^§, and Steve Wisseh, MS, Dr PH^§

From the ^* Department of Pediatrics, University of North Carolina Children's Primary Care Research Group, University of North Carolina School of Medicine;  North Carolina Institute of Public Health, University of North Carolina School of Public Health; and the ^§ Frank Porter Graham Child Development Center, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina.

	ABSTRACT

Top Abstract Methods Results Discussion References

Objective.  To improve health outcomes of children, the US Maternal and Child Health Bureau has recommended more effective organization of preventive services within primary care practices and more coordination between practices and community-based agencies. However, applying these recommendations in communities is challenging because they require both more complex systems of care delivery within organizations and more complex interactions between them. To improve the way that preventive health care services are organized and delivered in 1 community, we designed, implemented, and assessed the impact of a health care system-level approach, which involved addressing multiple care delivery processes, at multiple levels in the community, the practice, and the family. Our objective was to improve the processes of preventive services delivery to all children in a defined geographic community, with particular attention to health outcomes for low-income mothers and infants.

Design.  Observational intervention study in 1 North Carolina county (population 182 000) involving low- income pregnant mothers and their infants, primary care practices, and departments of health and mental health. An interrupted time-series design was used to assess rates of preventive services in office practices before and after the intervention, and a historical cohort design was used to compare maternal and child health outcomes for women enrolled in an intensive home visiting program with women who sought prenatal care during the 9 months before the program's initiation. Outcomes were assessed when the infants reached 12 months of age.

Interventions.  Our primary objective was to achieve changes in the process of care delivery at the level of the clinical interaction between care providers and patients that would lead to improved health and developmental outcomes for families. We selected interventions that were directed toward major risk factors (eg, poverty, ineffective care systems for preventive care in office practices) and for which there was existing evidence of efficacy. The interventions involved community-, practice-, and family-level strategies to improve processes of care delivery to families and children. The objectives of the community-level intervention were: 1) to achieve policy level changes that would result in changes in resources available at the level of clinical care, 2) to engage multiple practice organizations in the intervention to achieve an effect on most, if not all, families in the community, and 3) to enhance communication between, among, and within public and private practice organizations to improve coordination and avoid duplication of services. The objective of the practice-level interventions was to overcome specific barriers in the process of care delivery so that preventive services could be effectively delivered. To assist the health department in implementing the family-level intervention, we provided assistance in hiring and training staff and ongoing consultation on staff supervision, including the use of structured protocols for care delivery, and regular feedback data about implementation of the program. Interventions with primary care practices focused on the design of the delivery system within the office and the use of teamwork and data in an "office systems" approach to improving clinical preventive care. All practices (N = 8) that enrolled at least 5 infants/month received help in assessing performance and developing systems (eg, preventive services flow sheets) for preventive services delivery. Family-level interventions addressed the process of care delivery to high-risk pregnant women (<100% poverty) and their infants. Mothers were recruited for the home visiting intervention when they first sought prenatal care at the community health center, the county's largest provider of prenatal care to underserved women. The home visiting intervention involved teams of nurses and educators and involved 2 to 4 visits per month through the infant's first year of life to provide parental education on fetal and infant health and development, enhance parents' informal support systems, and link parents with needed health and human services. We included training in injury prevention and discipline, and home visitors assisted mothers in obtaining care from one of the primary care offices.

Results.  There were high levels of participation, changes in the organization of the delivery system, and improvements in preventive health outcomes. Agencies cooperated in joint contracting, staff training, and defining program eligibility. All 8 eligible practices agreed to participate and 7/8 implemented at least 1 new office system element. Of eligible women, 89% agreed to participate, and outcome data were available on 80% (180/225). After adjusting for differences in baseline characteristics, intervention group women were significantly more likely than comparison group women to use contraceptives (69% vs 47%), not smoke tobacco (27% vs 54%) and have a safe and stimulating home environment for their children. Intervention group children were more likely to have had an appropriate number of well-child care visits (57% vs 37%) and less likely to be injured (2% vs 7%). Intervention mothers also received Aid to Families with Dependent Children for fewer months after the birth of their child (7.7 months vs 11.3 months).

Conclusions.  We observed a number of positive effects at all 3 levels of intervention. Policy-level changes at the state and community led to lasting changes in the organization and financing of care, which enabled changes in clinical services to take place. These changes have now been expanded beyond this community to other communities in the state. We were also able to engage multiple practice organizations, reduce duplication, and improve the coordination of care. Changes in the process of preventive services delivery were noted in participating practices. Finally, the outcomes of the family-level intervention were comparable in direction and magnitude to the outcomes of previous randomized trials of the intervention. All the changes were achieved over a relatively brief 3-year study period, and many have been sustained since the project was completed. Tiered, interrelated interventions directed at an entire population of mothers and children hold promise to improve the effectiveness and outcomes of health care for families and children.  Key words:  children, preventive services, quality improvement.

Despite increasing emphasis on prevention and abundant evidence of ways to improve the delivery of preventive services to children^1-3 many Healthy People 2010 goals for children have not been achieved. Traditional approaches to moving evidence into practice have not worked. Therefore, current recommendations for improvement concentrate on changing the system of delivering these services. For example, the US Bureau of Maternal and Child Health and other policy analysts are recommending efforts to improve child health through better integration of clinical and population-based preventive services for children.⁴ The Bureau suggests that primary care practices should strive for "vertical" continuity among physicians, nurses, and office staff, as well as "horizontal" continuity of practices with "community-based health and human service programs such as day care centers, early intervention programs, mental health services, and well-developed public health programs."

Applying these recommendations in communities is challenging because they require both more complex systems of care delivery within organizations and more complex interactions between them. Currently, the health care system is fragmented at the community level. Population-based approaches to the delivery of preventive services have been successful primarily in organizations with responsibility for defined populations, such as closed model health maintenance organizations.⁵ In most communities, managed care plans concentrate on their own subscribers rather than on the health of all children, and the responsibility for coordinating services to families is unclear. There are also problems at the level of care provision. Health department programs are organized by type of service or disease category (eg, Women, Infant's, and Children [WIC], immunizations, tuberculosis), making it difficult to combine services for families with multiple needs. In primary care offices, a lack of data on performance and outcomes (eg, home safety, child development, practice-wide preventive service rates) makes it difficult to identify gaps in services and plan improvements. In addition, busy practitioners and health departments may not be aware of emerging evidence on effective interventions or have the support needed to incorporate effective interventions into their settings.^5-7 These are problems of the delivery system, not lack of knowledge or technology.

To improve the way that preventive health care services are organized and delivered in 1 community, we developed a health care system-level approach that involved addressing multiple care delivery processes, at multiple levels in the community, the practice, and the family. Our objective was to improve the processes of preventive services delivery to all children in a defined geographic community, with particular attention to health outcomes for low-income mothers and infants. We have reported on the feasibility of the interventions previously.⁸ Here we report on the effects of the interventions.

	METHODS

Top Abstract Methods Results Discussion References

Study Site and Participants

The Linkages for Prevention project was a community-wide health system intervention to improve the delivery of preventive services to all children under 2 years of age in Durham, North Carolina, (population 182 000) between July 1994 and July 1997.⁹ We selected preventive services as a target because they are of central importance to the care of young families. We hoped to accelerate the improvement of health outcomes by addressing specific care delivery processes at the level of the clinical interaction between care providers and patients. Because our primary objective was to achieve process change that would lead to clinically relevant changes in outcomes, we selected interventions that were directed toward major risk factors (eg, poverty, ineffective care systems for preventive care in office practices) and for which there was existing evidence of efficacy.

Community-Level Intervention

The objectives of the community level intervention were: 1) to achieve policy level changes that would result in changes in resources available at the level of clinical care, 2) engage multiple practice organizations in the intervention to achieve an effect on most, if not all, families in the community, and 3) to enhance communication between, among, and within public and private practice organizations to improve coordination and avoid duplication of services.

We formed an advisory board chaired by the director of North Carolina Division of Medical Assistance (Medicaid), with representatives from community agencies, primary care practices, and county government, to provide advice about how the project could fit in among existing community health improvement projects. We sought leadership from state health policy makers to develop needed resources and achieve the cooperation between the health and mental health departments necessary to implement the home visiting program. Because practices and agencies face common barriers to improved care delivery, we sought to increase communication among them about how to overcome specific problems in care delivery. Once the practice-level intervention had begun, meetings of practices were held so that physicians and staff could share new approaches to organizing preventive care. To enhance coordination among groups conducting immunization outreach, we formed a task force of all the groups to identify and close gaps in immunization outreach activities. Finally, we sought to link practice and family interventions by encouraging home visitors to identify a nurse in each practice with whom they could communicate about the families both served. Our aim was to help build lasting change in the organization of health care in the community.

Practice-Level Interventions

The objective of the practice level interventions was to overcome specific barriers in the process of care delivery so that preventive services could be effectively delivered. To assist the health department in implementing the family-level intervention described below, we provided assistance in hiring and training staff and ongoing consultation on staff supervision, including the use of structured protocols for care delivery, and regular feedback data about implementation of the program. We also assisted the health department in the development of strategies and processes to screen, recruit, and enroll families. Interventions with primary care practices focused on the design of the delivery system within the office and the use of teamwork and data in an "office systems" approach to improving clinical care.¹⁰ We sought to involve all primary care practices in Durham that enrolled at least 5 newborns per month. Eight of the 14 practices in the city met this criterion, and together they cared for >80% of the children in the community. The office systems intervention we used with these practices is described in a companion paper.¹¹

Family-Level Intervention

Family-level interventions provided "intensive" home visiting to poor pregnant women and their infants to address risk factors for adverse health outcomes. The home visiting intervention was selected because it had been shown in randomized trials to have immediate and enduring effects in improving the quality of the care giving environment, improving maternal life course development, reducing dependence on government programs, and decreasing childhood injury.^12,13 Women (regardless of parity) whose incomes were 100% or less of the federal poverty level and who presented consecutively for prenatal care at the community health center were eligible for the intervention. At the time the study was conducted, this clinic served approximately 50% of the low-income pregnant women in the county.

Mothers were recruited for the home visiting intervention when they first sought prenatal care at the community health center, the county's largest provider of prenatal care to underserved women. The home visiting intervention began when the mother presented for prenatal care and involved 2 to 4 visits per month through the infant's first year of life to provide parental education on fetal and infant health and development, enhance parents' informal support systems, and link parents with needed health and human services. We included training in injury prevention and discipline,¹⁴ and home visitors assisted mothers in obtaining care from one of the primary care offices. The home visiting intervention was provided by a public health nurse from the health department and an early childhood educator from the mental health department. Based on our experience in a feasibility study, an early childhood educator was added to the nurse visitor to enhance the emphasis on early childhood development.⁸ Educators were hired through the county Department of Mental Health, which allowed us to take advantage of additional sources of governmental funding.

Study Design and Data Collection

Outcomes of the program were evaluated at all 3 levels of the intervention to assess the degree to which we achieved our objectives. Because there is already evidence that the preventive services we targeted are effective in improving the health of children, we focused on changes in the rates of delivery of care. We were interested in evidence of change in the organization of the delivery system and outcomes of care.

We measured the community-level impact by documenting changes in the participation, financing, and organization of services delivered by participating organizations. We assessed the practice-level intervention by measuring rates of preventive services before and after the intervention, using a random sample of charts abstracted from each practice. (This is described in detail in the companion paper.¹¹) We assessed the family-level intervention by comparing the extent to which we achieved outcomes comparable to those achieved in earlier randomized trials. We used a historical cohort design that compared maternal and child health outcomes in 2 groups: women enrolled in the intensive home visiting program and women who had sought prenatal care during the 9 months before the program's initiation. Enrollment in the comparison group took place between November 1994 and March 1995. Enrollment in the intervention group occurred between April 1995 and March 1996. Funding for home visiting competent at the intervention was provided by the North Carolina Division of Medical Assistance, but only through the infant's first year. Therefore, we focused on outcomes that could be influenced during this period.

Our measurement approach was based on the work of Kitzman et al¹⁵ In brief, participating women were interviewed by trained interviewers at the time of enrollment and when the child reached 12 months of age. The enrollment interview collected data on demographic,⁵ obstetrical, and medical histories, and data on health-related behaviors (eg, smoking, alcohol use), stress, self-efficacy, and social support. Twelve-month interviews with mothers and/or guardians were used to reassess socioeconomic conditions, medical and reproductive history, health-related behaviors, parenting knowledge and attitudes, and social support. This interview also involved an assessment of the home environment and home safety.

Prenatal, obstetrical, and newborn records were abstracted to measure use of prenatal care and pregnancy outcomes. Birth certificates were obtained from the North Carolina Center for Health Statistics. Social services records were searched for reported child abuse and neglect. Medical records of all hospitals in the region were searched to identify emergency department visits and hospital admissions, and the total number of visits for injuries and ingestions was recorded. The North Carolina Immunization Registry was searched for any record of immunization among children found to be underimmunized. Abstracters and interviewers were unaware of the mothers' treatment assignment.

Prenatal care and pregnancy outcomes included the number of prenatal visits, pregnancy weight gain, gestational age, and complications of pregnancy. The adequacy of prenatal care was measured using the Kotelchuck Index.¹⁶ Hospital records were used to create the index and to measure gestational weight gain, prenatal care, and neonatal problems when birth certificates were unavailable.

Behavioral outcomes included reported use of birth control, and tobacco, alcohol, and illicit drugs. Alcoholic abuse was measured with the CAGE questions.¹⁷ Maternal depression was measured with the Beck Depression Inventory.¹⁸ Rates of preventive services, including immunizations and screening for lead, anemia, tuberculosis, blood pressure, and vision, were measured by chart abstractions in the primary care practices in which mothers sought care. Children were determined to be up-to-date for immunizations if they had received 3 diphtheria, pertussis, and tetanus vaccines; 2 oral poliovirus vaccines; 2 Haemophilus influenzae type b vaccines; and 3 hepatitis B vaccines by 12 months of age. The total number of visits and the number of well-child visits were also recorded. Because 4 or 5 well-child visits are recommended by 12 months of age, the number of well-child visits was dichotomized at 4 or more visits. Access to care and satisfaction with care were measured with questions from the National Health Interview Survey.¹⁹ Mothers were also asked if they had had a prenatal visit with the infant's physician and if they knew the date when the child's next immunization was due.

Home safety outcomes were assessed by direct observation using a home safety checklist.¹⁴ The characteristics of the home environment were measured with the infant/toddler version of Home Observation for Measurement of the Environment.²⁰ Mothers' beliefs associated with child abuse were measured by the Adult Adolescent Parenting Inventory.²¹ Parenting knowledge and skills were assessed using a modified 19-item version of the Knowledge of Infant Development Index.²² Satisfaction with parenting was measured by the Parenting Satisfaction Scale.²³ Social Support was measured using the Maternal Social Support Inventory.²⁴

In the 12-month interview, mothers were asked about their postnatal work and educational history, as well as the type of child-care arrangements and the number of hours per week the child used them. Use of human services was measured by asking mothers about their current receipt of WIC and Aid to Families With Dependent Children (AFDC), an approach found comparable to secondary data collection.¹⁵

Statistical Analysis

To assess the effects of the family-level intervention, we compared the intervention and comparison groups on maternal, parenting, and child outcomes. The analysis of the practice-level intervention is reported in the companion paper. The community level intervention was not amenable to statistical testing.

All family-level analyses used an "intention-to-treat" approach, in which all women enrolled in the project were included regardless of how long they participated in the intervention. We assessed the potential confounding effects for a number of independent variables, selected based on differences between treatment and comparison groups at the time of enrollment, previous evidence, and theory. We used logistic regression to model dichotomous variables and analysis of variance models (analysis of variance) for continuous variables. The following covariates were included in every model: mother's age, education, ethnicity, child order, marital status, baseline smoking status, and alcohol use. In modeling outcomes for alcohol use, drug use, postnatal work outside the home, and school attendance, baseline values were included in the model as well. For instance, to model alcohol use at the end of the intervention, we adjusted for reported alcohol use at enrollment in addition to the basic set of covariates. Models for outcomes that were time-dependent, including parental knowledge, number of well-care visits, and use of WIC and AFDC, were adjusted for the child's age at follow-up. For dichotomous outcomes, we present the adjusted proportion, as well as the measures of effect. For proportions, the adjusted value is the predicted probability of an event when the intervention and control groups both had the same mean values for each of the covariates.

Because of concern that the intervention might have variable effects depending on whether the mother was bearing her first child, a treatment-by-birth order (first child vs not first child) interaction term was included in all adjusted analyses. An interaction term was included if this was significant at the 0.10 level. When significance was detected, group comparisons were done separately for each level of child order.

We collected substantial amounts of data to be able to compare our results to previous randomized trials of the intervention model. However, because of the shorter duration of the intervention (1 year as opposed to 2 years in the randomized trials) and because our emphasis was on translating evidence into practice, our analyses focused on the direction, pattern, and clinical significance of changes observed, in comparison to the randomized trials.

	RESULTS

Top Abstract Methods Results Discussion References

Implementation

All state and county organizations agreed to participate. All the practices proceeded through the 3 phases of the intervention: measuring current performance and understanding the current office system, designing and implementing office system elements, and monitoring implementation. However, practices varied in the number and types of materials designed and implemented.

When all women who agreed to participate in the program were included, families received an average of 32.5 (standard deviation: 21.2) home visits between the time of enrollment and the time the infants reached 12 months of age. The number of visits completed represented 62.3% of the anticipated number of visits, based on our protocols, and is similar to other reported results.¹² Of women in the comparison group, 34.6% (28/81) reported having had at least 1 home visit during their pregnancy by a public health nurse or social worker.

Community-Level Outcomes

Table 1 lists the organizations that participated in the project at state, county, and local levels and changes that took place that may have been related to the interventions. The North Carolina Division of Medical Assistance and the North Carolina Division of Women and Children's Health collaborated to provide start-up funding and Medicaid reimbursement for the intensive home visiting services. With the assistance of the North Carolina Divisions of Medical Assistance and Mental Health, the county departments of health and mental health developed a joint contract that enabled them to hire and fund the home visitors. During the course of the study, the intensive home visiting program was adopted as an ongoing service of the health department. The health department also worked with other community home visiting projects to define eligibility criteria for each program and share staff training. These home visiting programs have continued to coordinate their services to maximize the number of children in the community who can be served. As a result of the practice level intervention, the primary care practices involved in the study met twice to share improvement ideas with each other and are continuing work to measure and improve the quality of their care. Finally, the health department, community health center, volunteers, and the North Carolina Immunization Section participated in joint planning of immunization outreach services. However, despite uncovering ways in which outreach could be improved, the planning effort did not result in changes in outreach services during the study period.

Practice-Level Outcomes

The 8 pediatric and family practices that met the eligibility criterion for the study included an academic pediatric practice, community health center, staff model health maintenance organization, 3 private pediatric practices, and 2 private family practices. All agreed to participate and 7 of 8 practices implemented new office systems. The effects of the practice-level intervention are described in detail in the companion paper on the effectiveness of the office systems intervention in improving preventive services delivery.

Family-Level Outcomes

During the recruitment period for the comparison group, a total of 274 women sought prenatal care. Of these women, 249 (91%) were screened for eligibility (screening was not done when no research assistant was present in the clinic). Of the 121 eligible women, 116 (96%) agreed to participate. Of the 557 women who sought prenatal care during the recruitment period for the intervention group, 449 (81%) were screened for program eligibility. Of the 225 women who were eligible, 201 (90%) agreed to participate. At the end of the study, 110 women had children old enough for the 12-month follow-up interview. Mothers who refused to participate said the program was too time consuming (5%), they already had parenting skills (2%), or they gave other unrelated reasons (3.5%). Of women in the intervention group, 5% moved out of Durham County and 17% had fewer than 5 home visits.

A high proportion of follow-up assessments were completed among eligible participants: 94% of newborn and delivery records were abstracted, and 96% of birth certificates were matched. Among those women enrolled who experienced no fetal or child death, 83% completed the 12-month home interviews, and 89% of primary care records and 97% of emergency department and hospital records were abstracted.

Table 2 shows the characteristics of mothers in the intervention and comparison groups at enrollment. Differences between the intervention and control in the groups were small. Intervention mothers were younger, more often black, less educated (nearly half had not completed high school), more likely to use alcohol, and more likely to be bearing their first child than comparison group mothers. As noted earlier, we adjusted for baseline differences in these and other potentially confounding factors in the analyses.

Prenatal and Birth Outcomes

There were no overall differences between the intervention and comparison groups in utilization of prenatal care, as measured by the Kotelchuck index, or in urinary infections or sexually transmitted diseases (Table 3). No differences were observed in gestational weight gain, birth weight, or gestational age.

Child Abuse and Injuries and Use of
Preventive Services

Although there were no substantiated cases of abuse in either group, the proportion of infants seen in the emergency department or hospitalized for injuries or ingestions was smaller in the intervention group than in the comparison group (2% vs 7%; crude P = .11; Table 4). Among the 103 intervention infants, there were 7 (7%) cases of substantiated neglect, compared with 4 cases among 105 (4%) comparison infants (adjusted P = .34). This difference may have reflected home visitors' referrals of at risk infants to family preservation services.

Among intervention group children, we observed a favorable pattern of preventive services outcomes. Intervention mothers were more likely than comparison group mothers to have made 4 or more well-child visits by the time the child was 12 months of age (57% vs 37%; P = .02). In addition, more intervention mothers (33% vs 20%) had an introductory visit regarding the infant with a physician (P = .09). Although 10% more intervention infants were up-to-date on immunizations (73% vs 62%), this difference was not statistically significant (P = .16). We did not observe any difference in mothers' satisfaction with their child's health care, such as whether the physician answered questions, provided enough information, or treated parents with respect. We also did not observe differences in the proportion of families visiting only 1 site of care (80% vs 82%; P = .86).

Home Environment: Safety and Parenting

Three of 5 measures of the safety of the child's home environment favored the intervention group (Table 5). Intervention infants' homes were more likely to have a poison control number and syrup of ipecac. Although they also seemed more likely to have the home's hot water temperature set below 120°, this outcome could not be consistently measured because a number of 12-month interviews took place outside the home. There were no differences in the proportion of homes with smoke detectors. Slightly fewer intervention mothers reported that the child always rode in a car seat.

Measurement of parental knowledge of development, parenting satisfaction, and child rearing views at 12 months of age did not reveal differences between the intervention and comparison groups. This lack of effect of the intervention by 1 year of age may reflect the limited ability of measures to discriminate among infants at an age when developmental differences are small and the behavioral challenges of the second year of life have not yet taken place. However, the home environment in which children were raised seemed more stimulating in the intervention group. The proportion of families scoring in the high range of the HOME scale favored the intervention group, as did all but 1 of the subscales. The magnitude of the differences was similar to that reported previously.¹²

Maternal Health, Life Course, and Use of
Human Services

We observed a favorable pattern of effects on a wide range of maternal outcomes (Table 6). Intervention mothers were more likely to use effective contraception (69% vs 47%; P = .007), and less likely to smoke (27% vs 54%; P = .04). They were also more likely to report higher levels of social support. There was no intervention effect on the proportion of women who attempted to breastfeed, the duration of breastfeeding, or the proportion using drugs or alcohol. More intervention group women were working or in school. Although a comparable proportion of mothers received Medicaid and WIC for themselves and their children after the birth of the child, fewer mothers in the intervention group reported receiving WIC and food stamps at the time when their infant reached 1 year of age. In addition, the mean duration of the receipt of AFDC was shorter for intervention women (7.7 months vs 11.3 months, P = .01).

	DISCUSSION

Top Abstract Methods Results Discussion References

The results of this study suggest that tiered, interrelated interventions directed at an entire population of children in a community are feasible. We observed a number of positive effects at all 3 levels of intervention. Policy-level changes at the state and community led to lasting changes in the organization and financing of care, which enabled changes in clinical services to take place. These changes have now been expanded beyond this community to other communities in the state. We were also able to engage multiple practice organizations, reduce duplication, and improve the coordination of care. Changes in the process of preventive services delivery were noted in participating practices. Finally, the outcomes of the family-level intervention were comparable in direction and magnitude to the outcomes of previous randomized trials of the intervention. All the changes were achieved over a relatively brief 3-year study period, and many have been sustained since the project was completed. Although these results are preliminary, they suggest that system-level interventions hold promise to improve the effectiveness and outcomes of care for children.

The interventions were complex and involved introducing multiple changes into the health care delivery system. Because it was not appropriate to use a measurement approach focusing on the effects of individual interventions, we took a systems analysis approach. Although this typically involves the use of an interrupted time series design, we strengthened the approach by including comparison groups at different levels of intervention.

The use of multiple simultaneous interventions also required the use of different evaluation strategies at different levels of intervention. There was no comparison group for the community-level outcomes. However, many community-level changes took place in a community that has had a history of challenges in attempting community collaboration.²⁵ We are encouraged by the amount of collaboration that occurred and by the willingness of community agencies to sustain efforts begun during the project. Furthermore, although the work reported here represents efforts undertaken in a single community, the interventions and measurement approaches may be useful to others as they undertake similar efforts to improve child health outcomes.

The family-level intervention was evaluated with a cohort study design rather than a randomized trial. Although this was a limitation, it enabled us to provide the home visiting intervention to all eligible mothers while still having a comparison group. We observed effects that were similar to those seen in previous randomized trials of the intervention.¹³ Although some effects were more limited, others were greater than observed previously, or were achieved in a shorter period of time (eg, reductions in tobacco use and supplemental assistance). The reductions in the use of AFDC and food stamps may have been attributable to other trends; however, the brief 9-month difference between measurement of the 2 groups makes the effect of secular trends less likely.

There are few examples today of comprehensive, population-based strategies to improve the health and development of all young children in a community (not just those enrolled in particular managed care plans or other closed systems of care). We focused on pregnancy and early childhood because this period provides an opportunity to reach all mothers and children.

Physicians who provide primary pediatric care are in a unique position to improve children's health and development because they encounter virtually every child in a community and are capable of connecting children to needed services. There is also a long tradition of physicians and nurses acting as advocates for children in communities. Linking clinical interventions in physicians' offices and home visits by public health nurses was a concept first developed nearly 100 years ago by Josephine Baker, MD.²⁶ Working in the Hell's Kitchen area of New York in 1907, she implemented a model of well-child care that involved infant examinations in health stations and home visits by nurses to provide education to mothers. The intervention was credited with dramatic reductions in infant mortality attributable to diarrhea, which ravaged the poor sections of the city every summer. Baker had been influenced by the work of Lillian Wald, RN, who pioneered the field of public health nursing by sending nurses into the home. Although Baker's approach was later expanded to other parts of the country, it disappeared with the advent of the private practice of pediatrics.

The model we developed differs from an approach advocated by Zuckerman and Parker,²⁷ which seeks to broaden the scope of care pediatricians provide for families by adding other services (eg, social work, legal assistance) in the medical office. Our approach is based on systems theory, which views care delivery as a series of processes extending from the home to the primary care practice and other community health and social services.²⁸ Systems theory suggests that many opportunities for improvement exist in the interactions between elements of a system (ie, practices, agencies).²⁹ We sought to build capacity and effectiveness at several different levels (the community, the primary care office, and the family) and avoid the need to add resources in any single site by increasing the coordination of existing resources. As illustrated by the cooperative arrangement that developed between the county departments of health and mental health, enhanced cooperation had multiple effects. At a programmatic level, it increased the focus on early childhood development. At a policy level, it has led state policy makers to combine funding streams to support the family-level intervention and expand this model to additional communities in the state.

Many opportunities remain to improve the health and developmental outcomes of young children, especially the socially disadvantaged. Future research should seek to understand how to accelerate and expand the impact of promising interventions like this one. At a community level, we need to understand how factors such as leadership, coordination, and financing can affect the speed and magnitude of change. At a practice level, we need to understand how to implement, spread, and sustain innovative strategies. At a family level, we need to continue to evaluate how enhancing the training and skills of home visitors in particular content areas (eg, home safety, smoking) can magnify their impact. In addition, we need to explore the "dose-response" effects of this intervention and understand how to link in-home and community-based interventions for children.

Future programs must be measured by their impact on the health of all children in a community, not just those who are enrolled in a particular practice or medical care plan. If system-level interventions are feasible and useful, as suggested by this project, the next steps are to determine who is responsible for system-wide improvement, and what resources are required to achieve the gains in child health and development that are possible.

	ACKNOWLEDGMENTS

Funding for the project was provided by the Kate B. Reynolds Charitable Trust, the Duke Endowment, Durham Regional Hospital, Connaught Laboratories, and the North Carolina Division of Maternal and Child Health. Dr Margolis received support as a Robert Wood Johnson Generalist Faculty Scholar.

We thank the families, practices, and agencies in Durham, North Carolina, who gave their time and effort to this project. We deeply appreciate the Linkages For Prevention staff and staff at community agencies who made this work possible, including Laura Dominguez; Maria Reiss; Chris Brooks; Laura Brown, MPH, Debbie Sears, Robert LeTourneau, MPH, Sandy Fuller, Katrina Teachy, Kim Bartholowmew, RN; Kerry Smith, RN; Camellia Seabrook, RN; Julia Gamble, RN; Jan Stratton MD, MPH; Linda Alsberry. We appreciate the participation of Durham Pediatrics, Chapel Hill Pediatrics, Kaiser Permanente, Lincoln Community Health Center, Duke General Pediatrics, Regional Pediatrics, Lakewood on Broad Family Practice, and Triangle Family Practice.

We also thank Barbara Matula, Dennis Williams, Carolyn Sexton, Susan Robinson, Thomas Vitaglione, Mary Beth Lister, and Lisa Orringer from the North Carolina Divisions of Medical Assistance, Maternal and Child Health, and Mental Health. Their foresight made this project possible. Finally, we are indebted to David Olds and Harriet Kitzman for sharing their protocols with us, and to Milton Kotelchuck, PhD, David Ransohoff, MD, Joanne Earp, ScD, and Tim Carey, MD, MPH, for their advice in the design and execution of the project.

	FOOTNOTES

Received for publication Dec 27, 2000; accepted May 7, 2001.

Address correspondence to Peter A. Margolis, MD, PhD, Associate Professor of Pediatrics and Epidemiology, University of North Carolina Children's Primary Care Research Group, 1700 Airport Road, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599-7226. E-mail: margolis{at}med.unc.edu

	ABBREVIATIONS

WIC, Women, Infant's and Children; AFDC, Aid to Families With Dependent Children.

	REFERENCES

Top Abstract Methods Results Discussion References

1. Szilagyi PG, Bordley WC, Vann JC, Effect of patient reminder/recall interventions on immunization rates. JAMA 2000; 284:1820-1827 [Abstract/Free Full Text]
2. Olds DL, Eckenrode J, Henderson CR, Long term effects of home visitation on maternal life course, child abuse and neglect, and children's antisocial and adaptive behavior: 15-year follow-up of a randomized trial. JAMA 1999; 278:637-643 [Abstract]
3. DiGuiseppi C, Roberts IG Unintentional injuries in childhood. Future Child. 2000; 10:53-70 [CrossRef][Medline]
4. Kessel S, Green M Diagnosing and treating health: bright futures. Pediatrics 1993; 91:998-1000 [Abstract/Free Full Text]
5. Thompson R, Taplin S, McAfee T, Mandelson M, Smith A Primary and secondary prevention services in clinical practice. Twenty years' experience in development, implementation, and evaluation. JAMA 1995; 273:1130-1135 [Abstract]
6. Welton W, Kantner T, Katz S Developing tomorrow's integrated community health systems: A leadership challenge for public health and primary care. Milbank Q 1997; 75:261-288 [CrossRef][Medline]
7. Margolis P, Carey T, Lannon C, Earp J, Leininger L The rest of the access to care puzzle: addressing structural and personal barriers to health care for socially disadvantaged children. Arch Pediatr Adolesc Med 1995; 149:541-545 [Abstract]
8. Margolis P, Lannon C, Stevens R, Linking clinical and public health approaches to improve access to health care for socially disadvantaged mothers and children. Arch Pediatr Adolesc Med 1996; 150:815-821 [Abstract]
9. 1990 Census of Population and Housing Guide. Washington, DC: US Department of Commerce, Economics and Statistics Administration, Bureau of the Census; 1992
10. Leininger L, Finn L, Dickey L, An office system for organizing preventive services: a report by the American Cancer society Advisory Group on Preventive Health Care Reminder Systems. Arch Fam Med 1996; 5:108-115 [Abstract]
11. Bordley WC, Margolis PA, Stuart J, Lannon C, Keyes L. Improving preventive service delivery through office systems. Pediatrics. 2001;108(3). Available at: http://www.pediatrics.org/cgi/content/full/108/3/e41
12. Enhancing the outcomes of low-birth-weight, premature infants: a multisite, randomized trial. JAMA. 1990;263:3035-3042
13. Kitzman H, Olds D, Henderson JJ, Effect of prenatal and infancy home visitation by nurses on pregnancy outcomes, childhood injuries, and repeated childbearing. A randomized controlled trial. JAMA 1997; 278:644-652 [Abstract]
14. Sheaffer LG, Cohen LR, Gordon AN, Baird TL. Child Development, Health, and Safety: Educational Materials for Home Visitors and Parents. Gaithersburg, MD: Aspen Publishers; 1996
15. Kitzman H, Olds DL, Sidora K, Enduring effects of nurse home visitation on maternal life course: a 3-year follow-up of a randomized trial. JAMA 2000; 283:1983-1989 [Abstract/Free Full Text]
16. Kotelchuck M An evaluation of the Kessner adequacy of prenatal care index and a proposed adequacy of prenatal care utilization index. Am J Public Health 1994; 84:1414-1420 [Abstract/Free Full Text]
17. Allen J, Eckardt M, Wallen J. Screening for Alcoholism (CAGE). Public Health Reports. 1988
18. Beck C The effects of postpartum depression on child development: a meta-analysis. Arch Psychiatric Nurs 1998; 12:12-20
19. National Health Interview Survey. Hyattsville, MD: National Center for Health Statistics; 1994
20. Caldwell B, Bradley R. Home observation for measurement of the environment. Little Rock, AR: University of Arkansas at Little Rock; 1979
21. Bavolek S. Handbook for the AAPI: Adult-Adolescent Parenting Inventory. Eau Claire, WI: Family Development Resources Inc; 1984
22. MacPhee D. Knowledge of Infant Development Inventory. Princeton, NJ: Educational Testing Service; 1981
23. Guidubaldi J, Cleminshaw H. Parenting Satisfaction Scale. San Antonio, TX: Harcourt Brace & Company; 1994
24. Pascoe J, Earp J The effect of mother social support and life changes on the stimulation of their children in the home. Am J Public Health 1984; 74:358-360 [Abstract/Free Full Text]
25. Davidson O. The Best of Enemies: Race and Redemption in the New South. New York, NY: Simon and Schuster; 1996
26. Baker J Women and the invention of well child care. Pediatrics 1994; 94:527-531 [Abstract/Free Full Text]
27. Zuckerman B, Parker S Preventive pediatricsnew models of providing needed health services. Pediatrics 1995; 95:758-762 [Abstract/Free Full Text]
28. Womack J, Jones D, Roos D. The Machine That Changed the World. New York, NY: Harper Trades; 1991
29. Nolan T, Knapp M Community-wide health improvement: lessons from the IHI-GOAL/QPC Learning Collaborative. Qual Letter Healthcare Leaders 1996; 8:13-20