June 2016, VOLUME137 /ISSUE Supplement 4

The National Children’s Study: Recruitment Outcomes Using an Enhanced Household-Based Approach

  1. Laura L. Blaisdell, MD, MPHa,
  2. Jennifer A. Zellner, PhDb,
  3. Alison A. King, MD, MPH, PhDc,
  4. Elaine Faustman, PhDd,
  5. Mari Wilhelm, PhDe,
  6. Mark L. Hudak, MDf, and
  7. Robert D. Annett, PhDg
  1. aMaine Medical Center, Portland, Maine;
  2. bUniversity of California San Diego, San Diego, California;
  3. cWashington University School of Medicine, St Louis, Missouri;
  4. dUniversity of Washington, Seattle, Washington;
  5. eUniversity of Arizona, Tucson, Arizona;
  6. fUniversity of Florida College of Medicine–Jacksonville, Jacksonville, Florida; and
  7. gDepartment of Pediatrics, University of Mississippi Medical Center, Jackson, Mississippi
  1. Drs Blaisdell and Zellner conceptualized the manuscript; acquired, analyzed, and interpreted the data; drafted the initial manuscript; and critically reviewed and revised the draft and subsequent manuscripts; Drs King and Wilhelm conceptualized the manuscript, acquired the data, drafted the initial manuscript, and critically reviewed and revised the draft and subsequent manuscripts; Dr Faustman conceptualized the manuscript and critically reviewed and revised subsequent manuscripts; Dr Hudak conceptualized the manuscript; acquired, analyzed, and interpreted the data; drafted the initial manuscript; and critically reviewed and revised the draft and subsequent manuscripts; Dr Annett conceptualized the manuscript; acquired, analyzed, and interpreted the data; drafted the initial manuscript; critically reviewed and revised the draft and subsequent manuscripts; and coordinated incorporation of the critical review and suggested revisions of all authors; and all authors approved the final manuscript as submitted.


OBJECTIVES: Ten National Children’s Study (NCS) study locations with diverse demographic characteristics used an enhanced household-based recruitment (EHBR) approach to enroll preconceptional and pregnant women. Study centers used different types and dosages of community outreach and engagement (COE) activities and supplemental strategies. The goal of the study was to determine whether variability in enumeration and recruitment outcomes correlated with study location characteristics or types and dosages of COE activities (number of COE events, number of advance household mailings, total media expenditures, and total COE expenditures).

METHODS: Each of the sites provided data on COE activities, protocol implementation, supplemental recruitment activities, location demographic characteristics, and enumeration/recruitment outcomes.

RESULTS: COE activities varied across sites in breadth and scope. Numerous strategies were used, including media advertising, social media, participation in community-wide events, presentations to stakeholders, and creation of advisory boards. Some sites included supplemental recruitment efforts. EHBR sites enrolled 1404 women at the initial pregnancy screening. No significant relationships were found between study location demographic characteristics or between the types and dosages of COE activities and recruitment outcomes.

CONCLUSIONS: Probability sampling for a long-term study requires a positive image with stakeholders and within communities; this requirement may be especially true for door-to-door recruitment. EHBR sites successfully recruited a representative sample of preconceptional and pregnant women. Sites reported implementing similar COE activities but with varying dosage and cost; however, analyses did not support a benefit of COE strategies on study recruitment.

  • Abbreviations:
    Alternative Recruitment Strategy
    community outreach and engagement
    dwelling unit
    enhanced household-based recruitment
    enumeration, pregnancy screening, consent
    initial Vanguard Study
    National Children’s Study
    pregnancy probability group
    study center
  • The present report describes the variable best practices used by the 10 study centers (SCs) in the National Children’s Study (NCS)1 tasked to create a comprehensive enhanced household-based recruitment (EHBR) strategy; it also describes the range of innovations used to prepare communities and households for research participation through community outreach and engagement (COE), media, and direct mail. Two hypotheses were tested: (1) that the urban/rural character of a county had an independent effect on the number of dwelling units (DUs) approached, enumerations completed, pregnancy screening questionnaires completed, and consents obtained; and (2) that increased dosages of 4 community outreach procedures would correlate with increased rates of enumeration and recruitment. Finally, differences in study protocols among SCs that may have contributed to variation in enumeration and recruitment outcomes were considered.


    Core Recruitment Procedures

    The NCS Program Office at the Eunice Kennedy Shriver National Institute of Child Health and Human Development developed the basic protocol and general guidelines for approaching local communities before enumeration as well as the specific methodology for household recruitment procedures. Each SC selected either a national federated institutional review board or its local institutional review boards to review all study activities.

    Before recruitment, EHBR SCs selected a procedure to identify and listed all eligible DUs within each randomly selected geographical study segment.2 Although the 7 initial Vanguard Study (IVS) sites were required to perform a manual listing in the field before initiating recruitment, EHBR SCs could purchase lists from commercial or public entities (eg, United States Postal Service), perform fieldwork within the segments to record or verify addresses of DUs either before or during recruitment, or use some combination of the 2 methods. Identification of DUs that did not appear on purchased lists was typically resolved by visual identification of missing or hidden DUs, such as a second residence on a property (eg, basement apartment), newly constructed residences, or multifamily dwellings. This listing procedure also removed seasonal DUs or otherwise uninhabited units.

    Household canvassing consisted of 3 stages: enumeration, pregnancy screening, and consent (EPSC). Households within randomly selected geographic study segments were approached at the doorstep to determine if the household qualified (ie, it was not a retirement or senior living facility, temporary residence, group home, or correctional facility). If an adult residing within the household verbally consented to the interview, the interviewer obtained a household census (enumeration). Eligible women ages 18 to 49 years identified during enumeration were then asked to participate in further eligibility screening. Due to the probability sampling recruitment model, volunteers were not accepted. Women who were either pregnant (pregnancy probability group 1 [PPG1]) or actively trying to become pregnant (ie, preconceptional [PPG2]) were immediately eligible and were asked to enroll/consent to participate in the NCS. Women who had experienced a recent pregnancy loss (PPG3) or women who were not actively trying to become pregnant (PPG4) could be enrolled/consented into the study at a later time if a subsequent follow-up contact ascertained an incident pregnancy.

    EHBR Community and Household Preparation Activities

    Enhanced preparatory activities for door-to-door recruitment focused on 2 areas: (1) local COE; and (2) targeted household outreach through mailings.

    Local COE

    Each EHBR SC determined COE activities that were suited to the local communities and health care providers who would support the study through prerecruitment and recruitment. COE activities sought to establish credibility of the NCS and garner enthusiasm and support within the county as well as with potential participants. Although the approaches varied greatly, SCs used and evaluated 3 major domains of enhanced COE activities: (1) media and branding; (2) COE staffing and events; and (3) development of community advisory boards and engagement of key stakeholders.

    1. Media and branding. The IVS used media outreach variably; thus, the use of mass media to coordinate outreach to the community in the Alternative Recruitment Strategy (ARS) substudy was considered an enhancement. The NCS Program Office purchased time-limited television, radio, and/or print advertisements for the ARS sites, but the media buys varied across sites. The national media buys occurred over a 4-week period in October and/or November 2010 and were not timed to coincide with any individual SC recruitment launch dates. To supplement the national advertising purchases, SCs targeted the timing of locally arranged media campaigns to the launch of their specific field operations. Local media campaigns used various combinations of television, radio, billboards, print, online, and social media advertising to support recruitment. SCs spent a considerable amount of time branding the recognition of NCS logos in their communities to increase study visibility and legitimacy. For example, many SCs required EPSC staff to wear official NCS-branded gear, including but not limited to uniforms (eg shirts, hats, vests), identification badges, and NCS logos on the bags in which staff carried paperwork or other materials. Preparatory communications often advised household members to anticipate that NCS staff would be wearing branded shirts and badges. Some SCs used prominent NCS magnetic logos mounted on field staff vehicles to increase the visibility and legitimacy of staff during enumeration of study segments. Media communications often embraced branding; for example, the Grant County, Washington, SC tied a “Say Yes to the Vest” media campaign to the NCS uniform vests worn by EPSC staff that contained the same slogan.

    2. COE staff and events. Each SC’s academic or medical lead institution had at least 10 years of experience with local community-based partnerships and programs. Before initiating recruitment, SCs devoted up to 2 years to identify broader community stakeholders and to engage in active outreach efforts that built support for and communicated the benefits of individual and community participation in the NCS. SCs employed experienced COE coordinators with native community knowledge and established local credibility to plan and implement outreach activities. COE staff attended or organized local community events such as neighborhood gatherings, baby showers, fairs, or rodeos. In addition, COE staff provided presentations to local health care providers (eg, obstetricians, birthing hospitals, pediatricians, nursing staff) and local community organizations and/or leaders (eg, public health workers, school nurses, parent–teacher associations, neighborhood organizations, religious officials). Engaging trusted providers, organizations, and individuals further prepared community members and potential participants by enlisting these groups to voice study support and encourage their patients/clientele to learn more about the NCS.

    3. Engagement of stakeholders. Early COE efforts centered around engaging key stakeholders (individuals, organizations, and health care providers engaged with mothers and children) to guide the broader community engagement process, provide feedback and critical insights to local NCS investigators, and serve as ambassadors for the NCS to their community. Community advisory boards were locally assembled at all 10 EHBR study locations and comprised instrumental community stakeholders with varied expertise to assist the local NCS SC team by facilitating community- and neighborhood-specific study preparation and participation. The composition of the community advisory board and the frequency of the meetings were at the discretion of the local SC. These typically involved representatives from local public health agencies, schools, child health advocacy programs, community members, and health care providers.

    Targeted Household Outreach

    Because advance mailings have increased response rates to postal and electronic surveys,36 NCS protocol dictated that each eligible household receive a single form letter in advance of enumeration procedures. In contrast to the IVS sites, EHBR locations were allowed to send additional mailings to households. These supplemental mailings varied from an enveloped letter to glossy postcards and mailings with enclosed incentives. For example, 1 location included a magnetic “coupon” redeemable for a $5 enumeration incentive with the advance mailing. In addition, study personnel targeted households that could not be enumerated on a field visit with printed materials such as “sorry we missed you” cards, handwritten personalized notes that were left at the household doorstep, and individualized follow-up mailings that encouraged participation in repeat household screening.

    Variation in Study Protocol Implementation

    The NCS Program Office allowed the 10 EHBR SCs some discretion in the implementation of the core NCS protocol. First, the EHBR SCs were allowed to employ a subcontractor to assist with selected elements of the recruitment procedure. Six EHBR SCs used the services of a subcontractor to complete EPSC, while 4 accomplished these activities through their academic institutions (Table 1). Second, SCs determined whether to implement EPSC procedures simultaneously or sequentially across all study location segments. Third, SCs decided on the appropriate number, qualifications, and training of field staff.

    TABLE 1

    EHBR SC Characteristics

    EHBR study locations also experienced chronological variations in protocol implementation. Federal information security requirements stipulated that each SC complete a detailed data security plan, which resulted in varied dates on which SCs received authority to operate from the NCS Program Office. Thus, the EHBR SCs launched recruitment at different points in time (ranging from December 2010 to February 2011). The NCS Program Office determined that the study had reached a steady state of enrollment, and active door-to-door recruitment ceased at all locations in November 2011, with passive recruitment ending in February 2012. The combination of different durations and mechanics of field operations led to substantial variability among SCs with respect to the percentage of households enumerated and the number of consents achieved by close of recruitment activities.

    Two additional and important sanctioned study enhancements included the provision of small incentives for enumeration and the ability to supplement recruitment of participants through community and provider venues (Table 1). Four EHBR locations chose to offer small incentives for enumeration. In some cases, incentives were provided in the form of gift cards to the respondent; in others, the incentive for each enumerated household was donated to the local school district or to another local community organization. Some locations offered incentives only to hard-to-reach populations, such as occupants of rental properties that refused to allow EPSC staff on-site; in this case, incentives were offered to encourage residents of these properties to call the local NCS SC office to complete enumeration by telephone.

    Supplemental recruitment of participants was allowed in health care settings, community events, or when physical access to eligible DUs was a barrier (Table 1). Some SCs actively recruited through providers by embedding NCS SC staff within provider offices and subsequently contacting these women for EPSC. In more passive forms of provider recruitment, some EHBR SCs placed NCS materials that contained participant interest cards in provider offices. Some SCs also screened women at community events such as rodeos, balloon fiestas, community fairs, high school sports events, or at tables within housing complexes when access to individual household units was denied. At these events, screening eligibility was verified by on-site address look-up, and arrangements for further recruitment procedures were established. In some cases, SCs created tailored recruitment events such as baby showers for pregnant women who resided in eligible neighborhoods; NCS study information was provided at these events, and women were asked to provide their contact information if they were interested in joining or learning more about the study.

    Data Collection

    The present article provides results of the basic recruitment data from the aforementioned protocol, but the study also conducted analyses to determine if and to what extent community engagement processes altered recruitment. EHBR study recruitment demographic characteristics were obtained from each study location. Each study location maintained a standardized biweekly report of recruitment for submission to the NCS program office. Data were collected directly from and verified by each center. US Census data from the American Community Survey were used for comparison of study location populations and analysis of recruitment of representative populations.

    A recruitment goal of the EHBR arm of the NCS was to obtain study data on infants born into the study and their mothers’ pregnancies. At the time of this analysis, 6 SCs were available to report location data on women pregnant at the time of initial screening and/or the ratio of the number of live-born study infants to either the number of enrolled/consented women (PPG1 and PPG2) or the number of nonenrolled/nonconsented women (PPG3 and PPG4) based on their pregnancy status at the time of the initial screening.

    Community and household preparation activities were evaluated by using the following metrics: (1) media use and expenses; (2) COE metrics (including full-time equivalents), number of contacts made in the community, number of community events attended, and overall COE expenses; and (3) number of advance mailings. Each SC submitted the following information via spreadsheet questionnaires.

    COE Full-Time Equivalents

    The personnel effort required to plan, coordinate, implement, and sustain the COE and media plans in full-time equivalents over the maximum 25-month period of prerecruitment and recruitment activities was recorded.

    Outreach Activities

    The number of community events attended and the number of individual and organizational contacts that had been made in the course of outreach efforts were reported. Contacts were defined as individuals or organizations with whom meaningful or purposeful connections were made to build enthusiasm for the NCS.

    Outreach Expenses

    Each SC reported expense of media purchases, COE operational expenditures (eg, printing, mailing, supply expenses), and COE staff expenses as documented from invoices.

    Advance Household Mailings.

    The number of mailings sent by the United States Postal Service directly to the household of potential recruits was reported.

    By consensus, the authors a priori selected 4 COE variables to test whether any of these correlated with recruitment success. These variables included the number of COE events, the number of advance household mailings, total media expenditures, and total COE expenditures. The success of recruitment preparation activities was measured by using rates of enumeration and enrollment/consent. We hypothesized that increased numbers of COE events, advance household mailings, total expenditures on media campaigns, and cumulative COE expenditures would yield increased enumeration and enrollment/consent rates. For the purposes of this analysis, the 4 variables were assigned discrete values as follows: COE events, <100 events, 100 to 200 events, and ≥201 events; preparatory mailings, 1 mailing, 2 to 3 mailings, and ≥4 mailings; total expenditures on media campaigns, $0 to $99 000, $100 000 to $249 000, and ≥$250 000; and total COE expenditures (including media campaigns), $0 to $99 000, $100 000 to $249 000, and ≥$250 000.

    Data Analysis

    The initial approach to data analysis was to develop a descriptive presentation of the counties participating in the EHBR recruitment strategy. Comparisons with US Census data from the American Community Survey were identified as a basis with which to contrast descriptive recruitment information. SC enrollment data were collated from data compiled from each SC. Rural (n = 5) and urban (n = 5) centers were defined on the basis of county population density (<500 people vs ≥500 people per square mile). Centers were also characterized based on the use of subcontractors (n = 6) or if they were independent (n = 4).

    To examine the initial hypothesis regarding the relationship between urban and rural center enrollment, t tests were used to examine summary data for DUs approached, enumerations completed, pregnancy screening questionnaires completed, and consents obtained. This approach was repeated for the subcontractor versus independent centers. Power was calculated with the use of G*Power software (Version 3.0.10, Department of Cognitive and Industrial Psychology, Heinrich Heine Universität, Düsseldorf, Germany) for these exploratory analyses by using a 1-tailed t test for matched pairs and assuming a substantial effect size (0.8) and exploratory α of .1. With these assumptions, the 10 SCs provide sufficient sample size to detect a significant difference between urban and rural centers (power, 0.85).

    The second hypothesis sought to determine how preparation activities were associated with the proportion of households enumerated and the proportion of women enrolled/consented by using metrics that were obtained from SCs. Critical COE metrics were determined a priori and included: (1) number of community events attended by NCS staff; (2) number of advance mailings; (3) media expenditures; and (4) outreach expenditures (absent media expenditures). We first inspected the raw data for each metric for continuity to determine whether the data met the criteria for parametric analysis (Levene’s statistic). Second, continuity was assessed with respect to enumerated households and women enrolled/consented by using 1-way analysis of variance. Power for these exploratory analyses assumed the use of an F statistic and a substantial effect size (0.85) and exploratory α of .1. The 10 SCs provide a sufficient sample size to detect a difference between groups (power, 0.7).


    EHBR Study Locations and Enrollment Demographic Characteristics

    Table 2 summarizes the demographic diversity of the EHBR study locations. Study location populations ranged from 27 115 persons in Baker County, Florida, to 3 095 313 persons in San Diego County, California. The geographical sizes of study locations ranged from 62 square miles (St Louis, MO) to 5366 square miles (Pinal County, AZ). US Census data for the female population ages 18 to 49 years also demonstrated significant variations in racial and ethnic diversity across the 10 centers (ranges, 19%–91% non-Hispanic white, <1%–48% black, 2%–56% Hispanic, <1%–42% Asian). In general, EHBR SCs succeeded in recruiting a sample of women that represented the racial and ethnic diversity of the study locations. Among enrolled/consented participants, younger women (aged 18–34 years) were overrepresented in the EHBR sample relative to older women (aged 35–49 years), which was expected given the study’s focus on pregnancy.

    TABLE 2

    EHBR Study Locations: Comparing Census Data and Enrollment Results for Key Demographic Indicators

    Table 3 lists the results of EPSC procedures according to urban (n = 5) and rural (n = 5) SC subgroups. The EHBR sites enrolled a total of 1404 women at the initial pregnancy screening. Overall, although we found some variability in outcomes across the SCs, no significant differences were identified between the urban and rural study locations in the percentages of DUs approached, households enumerated, pregnancy screenings completed, women identified as eligible for enrollment from the pregnancy screening, and women (both pregnant and prepregnant) consented. EPSC outcomes also did not differ between SCs that used subcontractors compared with those that used the SC’s institutional resources. There was considerable variability among SCs with respect to the percentage of women within EHBR locations who were identified as pregnant through population screening (range, 2.7%–13.7% of women) and the success of the SC to obtain consent for study participation from these women (range, 56.1%–86.8%).

    TABLE 3

    EPSC Results From Household-Based Recruitment Methodology

    We also examined the contribution of the various cohorts of women identified at the initial pregnancy screening to the NCS infants ultimately delivered at a selection of the EHBR sites. As shown in Table 4, women who were pregnant at the time of initial screening contributed between 59.2% and 90% of the infants born at 6 EHBR SCs; women who were actively trying to conceive at the time of initial screening ultimately contributed 2.0% to 19.5% of the infants born; and women who were not trying to conceive at the time of initial screening contributed <1% to 24.5% of the infants born. Table 5 summarizes what we have termed the “study fecundity index,” or the ratio of the number of live-born study infants to either the number of enrolled/consented women (PPG1 and PPG2) or the number of nonenrolled/nonconsented women (PPG3 and PPG4) based on their pregnancy status at the time of initial screening. As would be expected, the pregnant cohort contributed between 89 and 99 live-born study infants per 100 consented women; however, the nonpregnant cohorts also made significant contributions at select EHBR SCs (eg, 27 infants per 100 consented “high triers” at SC4; 14 infants per 100 consented “low triers” at SC6; and 29 infants per 100 consented women who reported a recent pregnancy loss at SC5).

    TABLE 4

    Data on Total Infants Delivered to Women Screened at Selected EHBR SCs and EHBR SC Operational Variables

    TABLE 5

    Study Fecundity Index for Infant Enrollment According to PPG Status at Time of Initial Pregnancy Screening

    EHBR Community and Household Preparation Activities

    Media Use and Expenses

    Sixty percent of EHBR SCs reported local purchase and use of advertisements via television, 80% radio, 90% print, 60% online, and 50% social media (Facebook being the most popular). In addition, NCS Program Office–purchased advertising in the form of television, print, and radio announcements ran for a 4-week period in each study location in the months preceding recruitment launch. The penetration of media campaigns was assessed by asking enrolled/consented women how they had heard about the NCS; 6% of women reported hearing about the study via television, 11.6% from print media, 10.6% from radio, and 1% from social media (data not shown). Fifty percent of SCs spent less than $100 000 on media campaigns before and during active enrollment, 20% spent $100 000 to $250 000, and 30% spent $250 000 to $500 000. There were no differences in media use and expenditures between urban and rural study locations.

    COE Full-Time Equivalents

    The number of staff involved in COE activities at each study location ranged from 7 to 54 (mean, 14) and percentage of effort ranged from 1.23 to 4.22 (mean, 2) full-time equivalents. SCs deployed staff for different amounts of effort across the time intervals before and during active recruitment.

    Outreach Activities

    The number of SC-staffed events in which ≥5 community members were in attendance ranged from 21 to 388 (mean, 145; median, 101). Individual contacts ranged from 12 to 1434 (mean, 413). Organizational contacts ranged from 34 to 967 (mean, 234), perhaps reflecting the differences in size (population and square miles) and diversity of the communities involved.

    Outreach Expenses

    When the expense of media purchases, COE operational expenditures (eg, printing, mailing, supply expenses), and COE staff expenses were combined, 10% of centers had total COE expenses between $100 000 and $250 000; 70% spent between $250 000 and $500 000; and 20% had spent more than $500 000. Many COE staff worked part-time, and local university students were often recruited to supplement COE staffing efforts.

    Advance Household Mailings

    Among the EHBR SCs, the number of advance mailings sent to eligible households ranged from 1 to 7 (mean, 2.3; median, 2). More than one-quarter (25.7%) of enrolled/consented women reported hearing about the study from these mailings (data provided by 9 of 10 centers). There were no statistical differences between urban and rural SCs.

    Comparative Enumeration and Enrollment/Consent Ratio by Research Preparation Activity

    Figures 1, 2, 3, and 4 provide percentages for households enumerated and women enrolled/consented for the 10 EHBR locations as a function of our predictive variables. The number of COE events did not significantly affect the percentage of households enumerated across the 10 EHBR centers (F = 0.6; P = NS) (Fig 1). Similarly, the number of advance mailings (F = 1.4; P = NS), media campaign expenditures (F = 1.5; P = NS), and total COE expenditures (F = 0.7; P = NS) did not significantly alter the percentage of households enumerated (Figs 2, 3, 4, respectively). Similarly, the percentage of eligible women enrolled/consented was not associated with the number of COE events (F = 1.5; P = NS), number of advance mailings (F = 0.1; P = NS), media campaign expenditures (F = 0.5; P = NS), or total COE expenditures (F = 3.1; P = NS).

    FIGURE 1

    Impact of COE events on SC recruitment. The percentage of households enumerated and the percentage of women consented into the study as a function of the number of community events attended by COE staff. Percent households enumerated comprises all households that were eligible. Percent women consented comprises all eligible study participants identified in screening.

    FIGURE 2

    Impact of advance mailings on SC recruitment. The percentage of households enumerated and the percentage of women consented into the study as a function of the number of mailings sent by SCs directly to households as part of the targeted household outreach. Percent households enumerated comprises all households that were eligible. Percent women consented comprises all eligible study participants identified in screening.

    FIGURE 3

    Impact of media expenditures on SC recruitment. The percentage of households enumerated and the percentage of women consented into the study as a function of media expenditures. Percent households enumerated comprises all households that were eligible. Percent women consented comprises all eligible study participants identified in screening.

    FIGURE 4

    Impact of outreach expenditures on SC recruitment. The percentage of households enumerated and the percentage of women consented into the study as a function of outreach expenditures, including outreach and engagement activities, staffing, and media purchases. Percent households enumerated comprises all households that were eligible. Percent women consented comprises all eligible study participants identified in screening.


    We summarized community and household preparation practices implemented in 10 diverse NCS study locations across the United States to examine the effect of selected practices on EHBR recruitment and enrollment success. Study locations varied widely with respect to population, area, and the racial and ethnic composition of women aged 18 to 49 years. We found no systematic differences among SCs in the enumeration and enrollment processes, but there was considerable variability in the percentage of women who self-identified as pregnant during population screening. We believe that this variability was the result of both the enhancements (described previously) used by some SCs to identify pregnant women and the unmeasured population differences (eg, fertility rates).

    Variations in population, geography, and sociodemographic characteristics of the study locations proved challenging in implementing a standardized protocol. All SCs agreed that the use of local institutions and local personnel contributed to the success of this protocol for several reasons. An understanding of many nuances within communities proved fundamental to initiating engagement and to overcoming barriers to EPSC. In addition, local management of the study allowed SCs to anticipate and respond nimbly to barriers by adapting procedures to achieve greater success.

    Neither the IVS nor the EHBR protocol used robust variables to measure the role of COE in recruitment. However, all study investigators have indicated that COE contributed to the acceptability of the NCS at their study location. COE by its very nature requires local, grassroots efforts to ensure the credibility and trust that yields participation. SCs generally used similar COE techniques that built on the experience of the 7 IVS SCs, and hence it is not surprising that we could not discriminate differential success across the SCs. The importance of community engagement and knowledge of community context has been emphasized across many studies but is especially relevant for children’s health cohorts. This topic was recently discussed in a series of lessons learned from papers of the National Institute of Environmental Health Sciences– and Environmental Protection Agency–supported Children’s Health Centers.5

    The present analysis found that the number of COE events and dollars spent on COE and media did not significantly alter enumeration or enrollment/consent rates. Although we noted a nonsignificant positive correlation between the number of advance mailings and increasing enumeration rates (Fig 2), advance mailings were not related to enrollment/consent rates. It is possible that the advance mailings better prepared households for the initial contact, thereby increasing willingness for screening, but they did not affect the level of trust and credibility required for consent. In addition, operational variables such as duration of recruitment and follow-up attempts linearly affected enrollment outcomes. However, the use of a subcontractor did not demonstrate consistent effect, either in the data or in anecdotal observations from the study principal investigators.

    EHBR SCs were very successful at recruiting a hard-to-identify population: women actively trying to conceive. Considerable debate surrounds the importance of a preconception cohort in the NCS.79 The argument for the importance of such a cohort recognizes the critical role of prepregnancy environmental exposures10; however, the best methodology for recruitment of this cohort remains unknown. Buck et al8 reviewed the published experience on the utility and feasibility of prospective pregnancy studies. Among the studies reviewed, 4% to 97% of eligible individuals were successfully contacted, with enrollment rates ranging from 42% to 100%. The results of the NCS EHBR recruitment strategy demonstrate similar feasibility of recruiting women who are not yet pregnant. A total of 568 preconception women at the initial pregnancy screener enrolled in the study, representing 40.5% of all EHBR consented women at the initial pregnancy screening. Given the difficulty in identifying preconceptional women, this outcome can be considered one of the strengths of a household-based recruitment approach, as the EHBR strategy was more successful at recruiting prepregnant women compared with provider-based recruitment11 and recruitment using direct-to-consumer marketing.12

    Targeted follow-up of the population of women who screened as actively trying to conceive ultimately led to a greater number of children born into the study and provided an opportunity to measure important preconception and early pregnancy exposures. In addition, some EHBR SCs were able to recruit a substantial proportion of infants who were born to mothers who were not actively attempting to become pregnant (PPG3 and PPG4) at the time of initial screening. Clearly, women who were pregnant at the time of screening demonstrated the highest study fecundity index. This index was not 1.0 for several reasons (most importantly, miscarriages and loss to follow-up [eg, moved out of county or unable to re-contact]). The “high trying” cohort delivered 11 to 27 study infants per 100 consented women, and variability was related in part to the duration and frequency of follow-up at individual SCs. As might be anticipated, the study fecundity index was lower among PPG4 women (who were followed up at SCs 3–6). These women contributed only 1 to 3 study infants per 100 screened women. Nonetheless, because the relative numbers of these women identified during enumeration were so large, these women did contribute substantially to the birth cohort. At SCs 3 through 6, births to PPG4 mothers represented 45% to 60% of infants born to the preconceptional cohort and contributed between 8.0% and 24.5% to the entire birth cohort.

    We acknowledge that the relatively small sample size (ie, 10 SCs) with limited power affected our ability to discern an effect of COE activities on our enumeration and enrollment outcomes. The lack of an observed effect could, alternatively, be due to factors such as limited variation across EHBR centers in enhancement efforts, the lack of a reference group in which no COE enhancements occurred, and the lack of robustness in the COE metrics. In addition, variability in the demographic characteristics of the study locations and more subtle differences in the COE and recruitment processes and procedures adopted by the 10 SCs challenged our ability to detect possible effects. All SCs embraced a process for engaging community stakeholders, with the goal of optimizing standard procedures to respond best to the specific nuances in local communities. This collaboration took a variety of forms based on locally determined community needs, which might have influenced EPSC outcomes. In addition, SC infrastructure differences (eg, choice of subcontractor for enumeration) may have contributed to the intercenter variability of some outcomes, such as the percentage of households recruited. We do not yet know the impact of the different recruitment procedures on the ability of the SC to retain consented women in the NCS for 21 years, a performance parameter critical for valid epigenetic research. Nonetheless, the EHBR SCs achieved a high degree of success in that they enrolled 68.1% and 60.8%, respectively, of identified PPG1 and PPG2 women into the study.


    This presentation provides detailed information on the implementation of enhanced COE procedures used to support household-based recruitment in 10 diverse study locations engaged in the NCS. SCs used best practice procedures to increase community engagement in the recruitment process. We detailed the overall success of these procedures in recruiting 836 pregnant women and 568 preconception women at the initial pregnancy screening. Engaging these diverse communities resulted in the deployment of creative processes and procedures both before and during the enumeration and screening processes. Our report highlights the importance of multilevel community relations, including media outreach, as well as focused effort by the study staff. Despite the lack of quantitative significance, the EHBR investigators strongly believe that the effort made by NCS SC staff to collaborate directly with community stakeholders and organizations made qualitatively significant contributions to the recruitment and consent of eligible women in both the urban and rural communities. Advance efforts to prepare diverse communities for research by generating trusted relationships between community members, participants, and researchers require considerable effort and expense. Ultimately, these efforts yielded the high degree of success achieved by the EHBR SCs. The role of maintaining these trusted relationships in successful long-term participant retention remains an important area of continuing research.


    This article was developed by a writing team assembled by the NCS Publications Committee (and is thus a “primary NCS publication”) for the purpose of timely sharing of centrally collected NCS data. The analysis was conducted as part of the NCS, supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development, and funded, through its appropriation, by the Office of the Director of the National Institutes of Health.

    The writing team thanks the families and community members who participated in or contributed to the NCS EHBR ARS. Their generous contribution of time and information was the essence of the IVS. The team also thanks the many talented NCS staff and consultants who helped implement the EHBR ARS and whose efforts informed the article, who include the following: Case Western Reserve University (Cleveland, OH): Dorr Dearborn, MD, Michele Walsh, MD, MS, Sarah Fulton, MS, Jessica Surdam, MPH, Leslie Clarke, RN, Jeno Moses, BA, Mark Beno, BA; Maine Medical Center (Portland, ME): John Bancroft, MD, Laura L. Blaisdell, MPH, MD, Lee Lucas, RN, MS, PhD; St Louis University (St Louis, MO): Louise Flick, DrPH, Alison A. King, MD, MPH, PhD, Karen Tucker, MA, Amanda Harrod, MPH, Grant Farmer, PhD, Lisa John, PhD, Elaina Murray, MPH, Kevin Stillman, MBA, Akihiko Noguchi, MD, MPH, Maggie Callon, MSW, LCSW, Nikki Weinstein, MSW, Vetta Sanders Thompson, PhD, Courtney Beers, MPH, Terry Leet, PhD; University of Arizona (Tucson, AZ): Mari Wilhelm, MA, PhD, Fernando Martinez, MD, James Roebuck, PhD, Kathryn Rae Engle, MD, Debra Cunningham, RN, Alexandra Armenta, BA, Nikita Lane, BA; University of California San Diego and San Diego State University (San Diego, CA): Christina Chambers, PhD, MPH, Melbourne F. Hovell, PhD, MPH, Jennifer A. Zellner, PhD, Suzanne Hughes, PhD, Diana Simmes, MPH, Norma Kelly, BA; University of Florida College of Medicine–Jacksonville (Jacksonville, FL): Mark Hudak, MD, Natalie Freeman, PhD, Elizabeth Piantadosi, RN, Trace Getzlaff, BA, Shelly Crawford, RN, Sue Combs, RN, Lori Hodges, RN, Jonathan Gaskins, BA, Kimberly Barnette, RRT, Vanessa Martin, BA, Steven Lipshulrz, MD, Tracie Miller, MD, Daniel Armstrong, PhD; University of Hawaii at Manoa (Honolulu, HI): Lynnae Millar Sauvage, MD, Beatriz L. Rodriguez, MD, PhD, Elizabeth McFarlane, PhD, MPH, Evelyn Hein, BA; University of Iowa (Iowa City, IA): Jeffrey Murray, MD, Rizwan Shah, MD, Sarah Brooks, BA; University of New Mexico (Albuquerque, NM): Robert D. Annett, PhD, Renate D. Savich, MD, Elizabeth Tigges, RN, PhD, Kristine Tollestrup, PhD, Josala Fetherolf, MA, Rebecca Smith; University of Washington (Seattle, WA): Elaine Faustman, PhD, Shirley Beresford, PhD, Thomas Burbacher, PhD, Kimberly Grant, PhD, Lisa Younglove, MPH; Grant County Field Office: Phillip Butterfield, PhD, Patricia Butterfield, PhD, Julie Postma, PhD, Tamara Odom-Maryon, PhD; Battelle Memorial Institute: Michelle Anderson, MA; Westat: Chao Zhou, MS; Eunice Kennedy Shriver National Institute of Child Health and Human Development NCS Program Office: Marion J. Balsam, MD, Christina Park, PhD, Carl Hill, PhD, MPH, and Maria Lopez-Class, PhD.


      • Accepted March 1, 2016.
    • Address correspondence to Robert D. Annett, PhD, Department of Pediatrics, University of Mississippi Medical Center, Jackson, MS 39216. E-mail: rannett{at}
    • FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

    • FUNDING: Supported in part by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (HHSN267200700021C, HHSN267200700023C, HHSN267200700025C, HHSN267200700031C, HHSN267200700033C, HHSN275200800002C, HHSN275200800009C, HHSN275200800011C, HHSN275200800025C, HHSN275200800034C, HHSN275201200013C, HHSN275201200020C, and HHSN275201200004I). Funded by the National Institutes of Health (NIH).

    • POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.