The National Children’s Study: Early Recruitment Outcomes Using the Direct Outreach Approach
OBJECTIVE: In 2009, the National Children’s Study (NCS) Vanguard Study tested the feasibility of household-based recruitment and participant enrollment by using a birth rate probability sample. In 2010, the NCS Program Office launched 3 alternative recruitment methods. We tested whether direct outreach (DO) recruitment could be a more efficient strategy to recruit women of child-bearing age.
METHODS: The NCS DO recruitment approach recruited women, 18 to 49 years, who were pregnant or trying to conceive using passive recruitment methods emphasizing broad community outreach and engagement to create study awareness. Study mailings to listed households included a pregnancy screening questionnaire to identify potentially eligible women from selected neighborhoods to contact the study center. Unique features of this recruitment approach included the following: (1) expansion of selected neighborhoods to maximize potential participant recruitment and enrollment while minimizing in-person participant contact and (2) offering 2 levels of study participation distinguished by data collection intensity.
RESULTS: Ten study centers listed 255 475 geographically eligible households for contact representing, on average, 3.3% of households per Primary Sampling Unit. A total of 19 354 women were identified for screening, and 17 421 completed a pregnancy screener representing 6.8% of eligible households. Study-eligible pregnant women were older, more educated, and less likely to be Hispanic than the general population. Only 16% (2786) of 17 421 screened women were study-eligible, and 81.1% of these 2786 women consented to participate.
CONCLUSIONS: Although feasible, the DO approach recruited a sample of study-eligible pregnant women significantly different from the population. This recruitment approach was labor intensive for the yield of enrolled women.
- ARS —
- Alternate Recruitment Strategy
- DO —
- direct outreach
- DU —
- dwelling unit
- NCS —
- National Children’s Study
- PS —
- pregnancy screener
- PSU —
- Primary Sampling Unit
- SC —
- study center
- SSU —
- Secondary Sampling Unit
- TSU —
- Tertiary Sampling Unit
This article describes the implementation and outcomes related to identification, recruitment, and enrollment of eligible women into the National Children’s Study (NCS)1,2 using the Direct Outreach (DO) approach. The DO approach was unique relative to other NCS recruitment strategies in key aspects, including the following: (1) expansion of selected neighborhoods up to 3 times the size of other recruitment approaches to maximize community outreach efforts without increasing in-person participant contact; (2) passive recruitment of participants through community outreach and engagement and mailings to households listed by address within selected neighborhoods but not identified by names or family composition; and (3) offering participants 2 levels of data collection intensity to test for differences in participant enrollment rates as a potential means to evaluate participant fatigue and study attrition. Study findings provide unique information on the feasibility and enrollment outcomes associated with the approach to recruit prebirth women for longitudinal child and life-course health research.
Identification of Segments and Dwelling Units for Recruitment
Ten study centers (SCs), each representing 1 US county, were assigned to conduct the DO strategy. Counties varied by population density and demographics (Table 1). SCs comprised 10 rural and urban Primary Sampling Units (PSUs) where a population density greater than 500 persons per square mile defined an urban location. Population densities across PSUs varied from 97 persons per square mile in Cache County, Utah to 5495 per square mile in Cook County, Illinois. Demographics also differed including populations that were predominantly non-Hispanic white (95.3%) in Westmoreland County, Pennsylvania and African American (60.2%) in New Orleans, whereas Los Angeles had the largest concentration of Hispanic persons (47.7%).
The NCS sampling strategy for the Alternate Recruitment Strategy (ARS) divided PSUs (generally an entire county) into Secondary Sampling Units (SSUs) composed of 10 to 15 neighborhoods (clusters of census blocks with dwelling units [DUs] or households), normalized to yield 250 live births per year per SC.3 Participant recruitment was restricted to women, ages 18 to 49 years, residing within these selected neighborhoods.
One unique feature of the DO approach entailed expanding the original neighborhood recruitment areas by adding adjacent neighborhoods to each SSU (except for Baldwin County, which included its entire county). Sampling units designated as Tertiary Sampling Units (TSUs) were sampled directly within SSUs in a multistage sample creating 3 levels of sampling stages. (TSUs formed the sampled segments equivalent to SSUs in other ARS approaches).
Another unique feature was the use of TSUs to test whether an initial invitation of low intensity data collection (and less participant time) followed by an invitation into a high intensity data collection protocol was a more attractive means to engage women who might be unwilling to initially enroll in a high intensity data collection protocol. Low intensity data collection involved only self-report survey data without in-person contact. In contrast, high intensity data collection included home visits where NCS staff obtained a written consent for collection of biological and environmental samples and in addition conducted a questionnaire. Figure 1 depicts a hypothetical county (PSU) and illustrates the relation between SSUs and TSUs.
The study protocol required all geographically eligible households, termed DUs, such as houses or apartments, located within selected neighborhoods to be “listed.” However, no in-person household visits to identify families’ composition (ie, no household enumeration) were allowed. NCS staff obtained lists of DUs from commercial vendors. Each SC employed additional strategies to enhance the precision of the list by using Google Earth to verify a vendor’s list, validating addresses relative to returned mailings, or visually inspecting a sample of DUs in potentially problematic areas including apartment complexes or rural areas lacking street addresses.
The Institutional Review Board of the Eunice Kennedy Shriver National Institute for Health and Human Development and each SC’s institutional review board approved the study for verbal consent into the low intensity protocol and written consent into the high intensity protocol.
The DO recruitment strategy included general outreach to the public, targeted outreach and engagement to selected neighborhoods (ie, SSUs and TSUs), including community agencies and healthcare providers, and study-specific mailings to listed households. Outreach and engagement methods were designed to create study awareness before households received study mailings.4
Potential participants were introduced to the NCS through mailed communications (Fig 2). The mailings included NCS information and invited household members to contact NCS staff to complete a pregnancy screener (PS) by telephone or mail.
The PS questionnaire asked if any women in the household were ages 18 to 49 years and, if yes, were they pregnant or planning to become pregnant or had conditions that precluded pregnancy (eg, hysterectomy). If a woman was pregnant or planning to conceive, and without condition(s) that precluded pregnancy, the household representative was invited to contact the SC; alternatively, researchers contacted the household in response to telephone messages or mailed communications. Follow-up postcards and repeat mailings thanked respondents and requested nonresponding households to contact NCS staff.
Participant Eligibility, Consent, and Data Collection Protocol
Women, ages 18 to 49, residing within listed DUs, who were pregnant or planning to conceive, were eligible. All study-eligible women who consented to participate were enrolled into the low intensity data collection protocol that entailed answering a telephone questionnaire. Questionnaire items assessed pregnancy information, general health and sources of health care, medical history, health insurance coverage, environmental and housing characteristics, and tobacco and alcohol use. After a 30-day waiting period, women residing in neighborhoods selected for high intensity data collection (ie, the TSUs) were invited to convert from low to high intensity data collection and were reconsented.
Approach to Data Analysis
Analyses are descriptive and address how participants learned of the NCS, the number and percentages of listed DUs, the frequency of mailed PSs returned to SCs, and the proportion of eligible and consented women over a 10-month (average) active recruitment period (Table 2). Recruitment was also evaluated by comparing initial enrollment into low intensity data collection to subsequent enrollment into the high intensity data collection using descriptive statistics. Data collected from the study sites were transmitted to a central data repository, processed, and analyzed for this article.
Publicly available county natality data were aggregated to assess potential enrollment bias5 by comparing the average percent of pregnant women residing in the TSUs and screened study-eligible to the average percent of births in the county by maternal age, race/ethnicity, and marital status. Eight of the 10 SCs also had comparison data on maternal education. Average percentages were compared because each study site was selected with an expected number of 250 births per year (under sample design assumptions). In theory, each site was expected to recruit women who would resemble the demographic profile of each PSU. When the study-eligible women are combined across the sites, the most appropriate demographic profile of the reference population is the average percent distribution for each demographic characteristic so that each site contributes equally to the reference population statistics. Additionally, since each site screened different numbers of women the average percentage of the sample distributions is the most comparable statistic to the population average percentage.
Population level information was provided by the Program Office using the Centers for Disease Control and Prevention’s Natality data 2009–2010.6 The demographic distribution of pregnant women screened for study eligibility was tested relative to the population’s distribution, using a Wilcoxon signed-rank test for each demographic variable. The null hypothesis was that the observed sample distribution is consistent with the population distribution. Findings show results only for sample participants residing in the TSUs for consistency with other ARS articles.
Expanded Geographic Area and Listed DUs
Among all SCs, 255 475 households or DUs were included in SSUs and eligible to be listed to receive study mailings (Table 2). On average, only 3.3% of all DUs in PSUs were listed. The number of eligible DUs varied across SCs (540–49 854 DUs). Of these, 103 980 DUs were located within the TSUs (representing 1.3% of all DUs across all PSUs).
How Participants Learned About the Study
Among geographically and age-eligible women who completed a PS to determine study eligibility, 60% reported learning about the NCS before completing the PS; 57% of these women recalled a study mailing as their primary information source.4
PSs Returned and Women Screened Eligible
Among 255 475 listed DUs, 6.8% (17 421) of households completed a PS; specific to the TSUs, 6.9% (7141) of households returned screeners (Table 2). Overall, 16% (2786) of the 17 421 women screened were study-eligible. Comparable results were seen in the TSUs.
Comparison of Pregnant Women Screened Study Eligible to County Natality Data
A subset of 564 pregnant women residing in the TSUs and identified at initial NCS screening as study-eligible were compared with new mothers by using county-level natality data to assess potential enrollment bias (Table 3). Significant differences between groups were seen for age, ethnicity, and education. Relative to the population, pregnant women screened eligible for the NCS were significantly older, less frequently Hispanic, and more educated.
Women Consented Into the Low and High Intensity Protocols
Among all women determined study-eligible by PS, the overall consent rate into the low intensity protocol was 81.1% (2259) of 2786. Among eligible pregnant women 85.3% (1134) of 1330 consented to study participation at initial screening, and 75.3% (809) of 1074 eligible women trying to conceive also consented.
After the initial screening, postscreening activities (eg, a follow-up telephone call 3–6 months later) identified an additional 316 (82.7%) of 382 women who subsequently became pregnant and consented to participate. Similar results were seen in the TSUs.
Among 1164 eligible women residing in the TSUs, 80.8% (941) women enrolled in the low intensity protocol, and 76.3% (718) of these study participants subsequently consented to high intensity data collection (data not shown on Table 2).
Among the total 2259 study participants, most were ages 30 to 34 (30.8%) or 25 to 29 (30.6%); fewer were 35 years and older (20.3%), or <25 years (18.2%). Women’s racial/ethnic identity was primarily non-Hispanic white (72.9%), African American (12.4%), or Asian (3.3%); Hispanic ethnicity was reported by 8.3%. Most women spoke English (97.9%) and were married (79.7%) or partnered (9.8%). The majority had earned a college or graduate degree (57.2%), or an associate degree/some college credits (26.6%). Annual household incomes revealed a bimodal distribution peaking at <$30 000 (30.8%) and $50 000–$99 999 (31.1%) with fewer households reporting incomes at $30 000–$49 999 (19.5%), and $100 000 or greater (18.5%).
A major finding of this article is that identifying study-eligible women using the passive recruitment methods of DO to the community was feasible. However, the pregnant women recruited were significantly different (older, less frequently Hispanic, and more educated) than the underlying population of new mothers. Additionally, the DO recruitment approach was labor intensive as recruitment began with 10 SCs contacting 255 475 selected households from which only 6.8% (17 421) of household members completed PSs, and only 16% (2786) of women screened were study-eligible. Nevertheless, 2259 women (81.1% of those screened-eligible) consented to participate. Consent rates among women identified at an initial screening were relatively high for pregnant women (85.3%), and less so for preconceptional women (75.3%). The low number of eligible women enrolled is sobering given the extensive recruitment efforts.
Findings from Canadian7 and US8 prebirth recruitment studies suggest a combination of active and passive recruitment strategies are most successful. Prebirth cohort studies in Norway, the Netherlands, and the United Kingdom identified the importance of personal contact at enrollment for effective recruitment,9 although these studies recruited women through hospitals and clinics, a strategy tested by the provider-based recruitment methodology.10 Even so, enrollment rates varied substantially including 45% of women recruited during routine ultrasounds in Norway11 and pilot study estimates of 60% associated with general practitioners inviting women into a Danish study at 6 to 12 weeks of pregnancy.12
Community outreach and targeted household mailings were key strategies for women to learn about the NCS. However, only 60% of eligible and enrolled women heard of the NCS before completing the PS and approximately half of these women learned via letters. Creating study awareness among potential participants was particularly challenging in densely populated counties as selected neighborhoods represented, on average, only 3% of the counties’ total DUs. Individualized rather than generic mailings are recommended for enhancing response rates, an approach consistent with the social exchange theory, which posits that personalized approaches to recruitment increase perceived rewards for responding and promote trust in beneficial study outcomes.13
Study findings on the impact of inviting participants into varying intensity levels of data collection revealed a decrease of only 4.5 percentage points from women’s initial consent rates into the low intensity protocol and subsequent conversion to the high intensity protocol, a modest difference. The findings suggest that providing women the opportunity to develop a relationship, and possibly some trust, with the research team may have decreased the potential for greater attrition in association with the higher intensity data collection protocol. The bias associated with volunteer participants in this study is not unexpected given the passive recruitment approach. Early evidence of NCS recruitment by using a provider-based approach in Wayne County, Michigan suggests greater success in recruiting women at-risk of adverse pregnancy outcomes.10 Enrollment varied across SCs and multivariate models are needed to evaluate the importance of factors such as the SC’s approach to implementing recruitment protocols in addition to community characteristics.
Although feasible, the DO approach to recruitment was labor intensive for the yield of enrolled women given the short period of active recruitment. It generated a sample of study- eligible pregnant women who were significantly different from the general population. Future recruitment approaches for a prebirth cohort study planning to enroll a nationally representative sample should ensure the ability to target representative samples of prebirth women and employ recruitment practices that motivate families’ contributions to the study and realize the potential for study findings to generalize to the nation’s children and ultimately influence health policy and clinical practice.
This article, a “primary NCS publication,” was developed by a Writing Team assembled by the NCS Publications Committee for the purpose of timely sharing of centrally collected NCS data. The Writing Team thanks the families and community members who participated in or contributed to the NCS Direct Outreach Alternative Recruitment Study. Their generous contribution of time and information was the essence of the Vanguard Study. We also thank the many talented NCS staff and consultants who helped implement the Direct Outreach ARS and whose efforts informed this article. Identified in alphabetical order they include the following: Mischka Garel, MPH, Jill Karr, PhD, Joslyn Levy, MPH, Nikki McKoy, MPH, Will Nicholas, MA, MPH, PhD, Bonika Peters, MPH, Lynette Lau Schumann, PhD, Judith Kadosh Solomon, RN, BSN, LuAnn White, PhD, and the St. Paul-Ramsey County Department of Health.
- Accepted March 1, 2016.
- Address correspondence to Patricia M. McGovern, PhD, MPH, RN, School of Public Health, University of Minnesota, Mayo MMC 807, 420 Delaware St SE, Minneapolis, MN 55455. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDED: The analysis was conducted as part of the National Children’s Study, supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development, and funded, through its appropriation, by the Office of the Director of the National Institutes of Health. Supported in part by the National Institute of Child Health and Human Development contracts numbers HHSN275200800024C, HHSN27520080033C, HHSN267200700027C, HHSN275200800010C, HHSN267200700017C, HHSN275200800018C, HHSN267200700022C, HHSN267200700029C, HHSN267200700015C, and HHSN275200800004C. Funded by the National Institutes of Health (NIH).
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- National Research Council and Institute of Medicine
- Trasande L,
- Andrews HF,
- Goranson C, et al
- Kaar JL
- Boughner RL
- Centers for Disease Control and Prevention (CDC)
- Velott DL,
- Baker SA,
- Hillemeier MM,
- Weisman CS
- Magnus P,
- Irgens LM,
- Haug K,
- Nystad W,
- Skjaerven R,
- Stoltenberg C; MoBa Study Group
- Olsen J,
- Melbye M,
- Olsen SF, et al
- Dillman DA,
- Smyth JD,
- Christian LM
- Copyright © 2016 by the American Academy of Pediatrics