Maintenance of Certification Part 4 Credit and Recruitment for Practice-Based Research
BACKGROUND: Competing priorities in pediatric practice have created challenges for practice-based research. To increase recruitment success, researchers must design studies that provide added value to participants. This study evaluates recruitment of pediatricians into a study, before and after the development and addition of a quality improvement (QI) curriculum approved for American Board of Pediatrics Maintenance of Certification (MOC) Part 4 Credit as an enrollment incentive.
METHODS: Researchers implemented multiple outreach methods to enroll pediatric practices over 28 months. Field note review revealed that many physicians declined enrollment, stating that they prioritized MOC Part 4 projects over research studies. A QI curriculum meeting standards for MOC Part 4 Credit was developed and added to the study protocol as an enrollment incentive. Enrollment rates and characteristics of practitioners enrolled pre- and post-MOC were compared.
RESULTS: Pre-MOC enrollment contributed 48% of practices in 22 months; post-MOC enrollment contributed 49% of practices in 6 months. An average of 3.5 practices enrolled per month pre-MOC, compared with 13.1 per month post-MOC (P < .001). Clinicians in pre- and post-MOC groups were similar in age, gender, race, and time spent on patient care; practices enrolled post-MOC were more likely to be located in federally designated Medically Underserved Areas than those enrolled pre-MOC (28.6% vs 12%, P = .03).
CONCLUSIONS: Addition of MOC Part 4 Credit increased recruitment success and increased enrollment of pediatricians working in underserved areas. Including QI initiatives meeting MOC Part 4 criteria in practice-based research protocols may enhance participation and aid in recruiting diverse practice and patient populations.
- AAP —
- American Academy of Pediatrics
- ABP —
- American Board of Pediatrics
- AHIPP —
- Adolescent Health in Pediatric Practice
- EHR —
- electronic health records
- MOC —
- Maintenance of Certification
- MUA —
- medically underserved area
- PBRN —
- practice-based research network
- PDSA —
- Plan, Do, Study, Act
- PROS —
- Pediatric Research in Office Settings
- QI —
- quality improvement
What’s Known on This Subject:
Pediatric primary care has undergone a cultural shift. Changes in electronic health records, certification requirements, and practice structure have left many physicians feeling too busy to participate in research. Practice-based research networks must adapt to fit the current climate.
What This Study Adds:
Adding quality improvement activities that meet Maintenance of Certification Part 4 criteria to research study design adds value to a practice-based research protocol. This incentive meets the needs of busy physicians, and may help researchers meet study recruitment goals.
Practice-based research plays a critical role in advancing the field of general pediatrics, allowing researchers to expand the evidence base and translate findings into practice. Practice-based research networks (PBRNs) are uniquely positioned to perform clinically relevant research, as their structure provides both a population to study and a network of clinicians with a vested interest in developing clinical evidence for practice. Recent shifts in pediatric primary care have created new challenges for PBRNs as they recruit clinicians for research studies from increasingly busy practice settings.1 To successfully recruit physicians, PBRN investigators must adapt to this current climate by designing research studies that provide meaningful incentives to participants.
The American Academy of Pediatrics (AAP) Pediatric Research in Office Settings (PROS) network is the largest pediatric PBRN in the United States,2 and has successfully conducted a wide range of primary care research studies on a variety of health topics.3–11 However, the traditional recruitment methods used by PROS proved insufficient to recruit enough pediatricians to participate in the Adolescent Health in Pediatric Practice (AHIPP) PROS study, a randomized controlled trial of adolescent tobacco cessation counseling. The AHIPP protocol was approved by the institutional review board at AAP and 30 institutional review boards at participating practice sites across the country. In this article, we describe 1 solution to study recruitment challenges: the development and addition of a quality improvement (QI) curriculum meeting standards for American Board of Pediatrics (ABP) Maintenance of Certification (MOC) Part 4 Credit. This QI curriculum was integrated into the existing research protocol after 22 months of physician recruitment, and MOC Part 4 Credit was advertised as an enrollment incentive. Recruitment then continued for 6 months. In the following analyses, we examine the effectiveness of this strategy for increasing recruitment success in PBRN studies, and compare enrollment and practitioner characteristics before and after MOC Part 4 Credit was available.
We evaluated the success of recruitment efforts for the AHIPP study before and after the addition of a QI curriculum meeting standards for MOC Part 4 Credit, and compared the demographic factors of practices/practitioners enrolled pre- and post-MOC. AHIPP was a randomized controlled trial of the effectiveness of a tobacco screening and brief counseling intervention for adolescents, delivered in the pediatric primary care setting. Control arm participants screened and counseled adolescents about social media use. To ensure sufficient power for planned statistical analyses, AHIPP set a goal of enrolling 160 community-based pediatric practices in the study. Eligible practices had to have a patient flow of at least 1 adolescent (age 14+) per week and a clinician-estimated smoking rate of at least 10% among adolescent patients. Eligible clinicians were pediatricians, nurse practitioners, and physician assistants; however, with the exception of a handful of nonphysician staff in PROS practices, only pediatricians were recruited directly. Each participating AHIPP practice was asked to screen all adolescent patients arriving for well-child visits or nonurgent sick visits for study eligibility and to enroll 100 adolescents into the study. Practitioners were trained to use guidelines-based screening with all adolescents, regardless of study enrollment, for the AHIPP tobacco intervention or the control arm social media intervention, and to deliver these counseling interventions as appropriate.
Recruitment Procedures and Challenges
Pre-MOC recruitment began in 2011. We used multiple outreach strategies to meet our enrollment goal. Practices that were already enrolled in the PROS network were recruited via a combination of direct-recruitment fax, E-mail, and phone contacts. These recruitment strategies were standard protocol for PROS and had been successful in previous studies.4–11 To expand the pool of AHIPP-eligible practices, we recruited new practices to the PROS network. New practices were recruited via direct outreach, physician networking, and in-person recruitment at continuing medical education events and national meetings. To enhance the likelihood of enrolling a sufficient number of smokers to power the study analyses, we focused our outreach efforts for new PROS practices in geographic areas known to have high population smoking rates.
After 1 year of recruitment by using these methods, enrollment goals were not being met. Many pediatricians expressed initial interest in the study but did not complete enrollment. A review of project staff field notes from recruitment contacts revealed that the most common reason that interested physicians ultimately declined to participate was that they were “too busy.” We examined cited reasons for this response (nonparticipation) and identified 4 factors most frequently mentioned as responsible for the physicians not enrolling in AHIPP (Table 1): daily practice concerns, perceived burden of administrative tasks, planned or recent implementation of electronic health records (EHR) systems, and planned or current participation in activities necessary to meet ABP MOC Part 4 Performance in Practice requirements. We considered options for adapting the study protocol to meet the needs of potential enrollees and concluded that only 1 of these barriers could be addressed while maintaining study integrity: physician prioritization of MOC Part 4 projects.
Development of a QI Curriculum
In hopes of better meeting the needs of potential participants and boosting recruitment success, AHIPP investigators designed a QI curriculum to work in tandem with the existing study protocol (for a listing of study protocol tasks and QI curriculum tasks, please see Table 2). The curriculum focused on an area of pediatric care that has been cited in the literature as needing improvement: systematic screening for delivery of preventive services. It is known that successful implementation of preventive services in practice requires systematic screening,12 yet many clinicians do not use such systematic approaches, which potentially prevents patients from receiving recommended services.13–15 The QI curriculum added to AHIPP was designed to encourage a disciplined approach to systematic screening through the experience of implementing the AHIPP research protocol. This ability to screen systematically could then be applied to other areas of clinical practice.
Proper implementation of the study protocol required the use of 2 systematic screening processes, one that identified eligible patients for study enrollment and another that assessed whether patients should receive the preventive services intervention. The QI curriculum supplemented this protocol by adding 3 new elements: (1) 5 QI measures, designed by AHIPP investigators and approved by the AAP Quality Cabinet, which oversees AAP QI initiatives and the AAP MOC portfolio; (2) a series of Plan-Do-Study-Act (PDSA) cycles that collected and analyzed data on performance in each measure over time; and (3) a QI posttest. Over the course of the PDSA cycles, physicians developed, implemented, and improved the screening systems required for the research protocol, with the goal of fully integrating them into routine patient flow. Participant data were compared with project benchmarks, progress was tracked in run charts, and participants received structured feedback and coaching as appropriate. Successful completion of the QI curriculum allowed participating pediatricians to simultaneously reach 3 goals: (1) completing the AHIPP research study, (2) improving their ability to systematically screen in practice, and (3) meeting AAP and ABP requirements for meaningful participation in this MOC Part 4 project.
Addition of MOC Part 4 Credit as an Enrollment Incentive
The QI curriculum described previously was reviewed by the AAP Quality Cabinet as part of the AAP portfolio of MOC Part 4 projects, and was approved for 25 points of credit by ABP in April 2013. Study recruitment materials were updated to reflect this added benefit, and we continued to use the standard recruitment strategies described previously. Recruitment continued for 6 months after the addition of MOC Part 4 Credit as an enrollment incentive. It should be noted that pediatricians who enrolled pre-MOC were offered the opportunity to earn MOC Part 4 Credit if they chose to participate in the additional QI measures and PDSA cycles. All pediatricians who elected to earn MOC Part 4 Credit received training in both the QI curriculum and in the AAP and ABP meaningful participation requirements required to claim credit for completing the QI curriculum.
We used nonparametric Mann-Whitney tests to evaluate the relative success of practice recruitment efforts before and after the addition of MOC Part 4 Credit as an enrollment incentive. To assess the utility of this incentive for future research efforts, we wanted to determine whether the MOC Part 4 Credit incentive affected the type of practitioners/practices that joined the study. To do this, we used nonparametric Mann-Whitney tests, χ2, and t tests to examine the demographic factors of the practices and practitioners enrolled pre- and post-MOC.
Differences in Enrollment Rates
During the first 22 months of study recruitment (Pre-MOC Group), we approached a total of 9534 pediatric clinicians. These efforts yielded 152 clinicians in 76 practices joining the study (47.5% of our enrollment target). After this period, MOC Part 4 Credit was added as an enrollment incentive and recruitment continued for 6 months. During those 6 months (MOC Group), we approached 8311 pediatric clinicians and enrolled 200 clinicians in 79 practices in the study (49.4% of enrollment goal). An average of 6.9 clinicians in 3.5 practices were enrolled each month during Pre-MOC recruitment, compared with 33.3 clinicians in 13.1 practices per month for the MOC group (P < .001 for both clinicians and practices). It should be noted that of the 352 enrolled clinicians described previously; 295 (84%) were pediatricians and 57 (16%) were nurse practitioners or physician assistants. Because ABP MOC Part 4 Credit is only a recruitment incentive for board-certified pediatricians, it is unlikely that this benefit directly affected nonphysician participants’ decision to enroll. However, we included nonphysician clinicians in enrollment comparisons because they joined the study with their physician colleagues.
To assess the impact of seasonality on recruitment, the 6-month period of MOC recruitment (May–October 2013) was compared with the same 6-month period in 2012, when MOC was not offered. The mean number of practices recruited per month during this period in 2013 was 13.2, significantly higher (P = .002) than the 3.3 practices per month recruited during this period in 2012.
Differences in Pediatrician-level Factors
Table 3 compares demographic and practice factors between the 122 pediatricians recruited pre-MOC and the 173 pediatricians recruited post-MOC. (We excluded nurse practitioners and physician assistants from these analyses because MOC Part 4 Credit pertains only to board-certified pediatricians. Thus, the addition of this incentive was unlikely to affect the demographics of nonphysician participants.) No significant differences were found in pediatrician age, gender, ethnicity, or in the hours and percentage of time spent in direct patient care.
As noted previously, pediatricians recruited in the Pre-MOC Group were also offered an opportunity to complete the QI curriculum and earn MOC Part 4 Credit after they had already begun study participation: 55.7% chose to seek this credit. In comparison, 83.2% of pediatricians who were recruited and enrolled in the MOC Group sought to earn the available MOC Part 4 credit.
Differences in Practice-level Factors
We compared the 76 practice sites enrolled during pre-MOC recruitment to the 79 practice sites enrolled post-MOC (of these, 75 and 77, respectively, provided demographic information). These groups did not differ significantly by geographic setting or patient panel description (Table 4). However, pediatricians recruited with MOC Part 4 Credit were more likely to be practicing at sites that were located in federally designated Medically Underserved Areas (MUA; P = .03).
Pediatric primary care has undergone a significant cultural change. Shifting market forces have increased the number of practices owned by integrated health systems.16 Implementation of the Health Information Technology for Economic and Clinical Health Act and meaningful use requirements have accelerated adoption of EHR systems,17 temporarily disrupting office workflow and practice efficiency.18 In addition, changes in mandatory professional development requirements have significantly altered the work required to maintain professional certification.19 These changes have left physicians with a variety of daily practice concerns that may prevent them from joining PBRNs or participating in research.16,19–21 These concerns were echoed by physicians recruited for the AHIPP study before MOC Part 4 Credit was added as an enrollment incentive.
PBRNs have been called to adapt to the current culture by offering research opportunities that provide real value to clinicians.19–22 Research networks that design feasible, clinically relevant studies may be more likely to succeed in recruiting physicians.20,23 Appealing studies should combine minimal burden with real-time improvements to patient care.21,22 One way to accomplish this is to merge traditional research methods with QI initiatives.19,21,22
The addition of QI activities meeting standards for MOC Part 4 Credit increases the value of a research project in 2 ways: (1) it trains practitioners in a clinically valuable skill (QI), and (2) it provides pediatricians with an opportunity to earn credit hours toward board certification. QI initiatives have been linked to clinical practice benefits, including improved health outcomes,24–26 increased efficiency of clinical processes,24 and cost-savings inherent in preventing clinical errors.24,27,28 In addition, the opportunity to enact QI processes in practice has been cited as a key incentive for physician participation in practice-based research.21 Thus, pediatricians may prioritize participation in research studies with an MOC-approved QI curriculum over traditional research studies. After the addition of MOC Part 4, enrollment increased from an average of 3.5 to 13.1 practices per month, suggesting that MOC was a significant motivator for physicians.
Although the addition of an MOC-approved QI curriculum was a successful enrollment incentive for AHIPP, it may be challenging to adapt this approach to other projects. ABP does not award MOC Part 4 Credit to standard research projects; a separate QI curriculum was developed and approved to create this incentive. QI development was possible because the AHIPP research protocol required systematic screening for preventive services: this is an area of clinical care that the literature has identified as needing QI.12–15 Other research protocols may not lend themselves as easily to QI activities; investigators should determine whether their individual projects are appropriate to this approach by reviewing standards for MOC Part 4 Credit on the ABP Web site (www.abp.org) or consulting with colleagues who are experts in QI. Additionally, future projects may be better served by including a QI component from inception instead of adding this curriculum after several months of study implementation.
Demographic comparisons of pediatricians enrolled in the pre- and post-MOC groups may reveal an additional benefit of using an MOC-approved QI curriculum as an enrollment incentive: this incentive may increase participation from physicians practicing in federally designated MUAs. Although individual motivations for participating were not assessed, it is possible that these physicians were particularly affected by time and resource constraints, making AHIPP impractical to take on without MOC. A similar trend was identified in a study of family physicians. Those practicing in underserved areas were more likely to have lapsed board certification than their colleagues in other communities; authors credited this difference to time and resource demands.29 Although this relationship requires future study, it has potential benefits for research. MUA sites are more likely to serve low socioeconomic status, rural, and minority populations, all of which are often underrepresented in research participation.30,31 If the addition of a QI curriculum meeting MOC Part 4 standards allows pediatricians practicing in MUAs to participate in research, it could increase the overall representativeness of research results.30,31 Inclusion of traditionally underrepresented groups is particularly important in tobacco control research, as individuals with low socioeconomic status are disproportionately more likely to smoke.32
There were several limitations to this study. First, the list of barriers to study participation named by physicians was not comprehensive. More than 17 000 physicians were recruited, so it was not feasible to collect follow-up data on all reasons for nonparticipation. Second, there was no planned, systematic collection of refusal reasons from all recruited pediatricians. These data were collected only from participants who expressed initial interest in the study; it is possible that physicians who never responded to AHIPP recruitment outreach had other reasons for nonparticipation. In addition, because we did not systematically assess the relative value of MOC Part 4 Credit compared with other incentives that could have been added to the AHIPP study, we cannot know whether another incentive could have produced greater results. Finally, the comparison of recruitment efforts with and without MOC was not planned as an empirical test. The MOC-approved QI curriculum was added as an additional incentive that helped us meet our study goals, and the effect of this incentive was assessed retrospectively. Thus, the study design may have created biases for which we did not adequately control.
Merging QI activities meeting standards for MOC Part 4 Credit with practice-based research studies serves 3 important functions: (1) provides an incentive for pediatricians to participate in research, (2) provides training in a clinically relevant skill, and (3) may increase participation of pediatricians serving traditionally underserved populations. In a culture in which primary care clinicians are often being asked to do more with less, research studies that include a QI component meeting standards for MOC Part 4 Credit have added value for pediatricians. This added value can help PBRNs achieve their recruitment goals and continue the important work of advancing the science of primary care and evidence-based interventions.
- Accepted July 21, 2014.
- Address correspondence to Julie A. Gorzkowski, MSW, American Academy of Pediatrics, 141 Northwest Point Blvd, Elk Grove Village, IL 60007. E-mail:
Ms Gorzkowski drafted the initial manuscript and incorporated all edits from the authorship team, drafted the final manuscript, carried out study recruitment, supervised data analysis, and supervised data collection at the American Academy of Pediatrics (AAP) and all project sites; Dr Klein conceptualized and designed the study, critically reviewed and revised the manuscript, and oversaw data analysis, recruitment, and data collection; Ms Harris carried out initial analyses, reviewed and revised the manuscript, coordinated study recruitment, and carried out data collection at AAP and all project sites; Ms Kaseeska reviewed and revised the manuscript, carried out study recruitment, and coordinated data collection at AAP and all project sites; Ms Shaefer critically reviewed and revised the manuscript, and oversaw study recruitment and data collection at AAP; Ms Bocian reviewed and revised the manuscript, and oversaw data analysis, recruitment, and data collection; Drs Davis and Gotlieb critically reviewed and revised the manuscript, conceptualized and participated in study recruitment strategies, and oversaw study implementation; Dr Wasserman critically reviewed and revised the manuscript, and conceptualized and oversaw study recruitment strategies, data collection, and data analysis; and all authors approved the final manuscript as submitted.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Supported by National Institutes of Health National Cancer Institute grant 5R01CA140576, a grant from the Flight Attendant Medical Research Institute to the American Academy of Pediatrics Julius B. Richmond Center of Excellence, and by the Pediatric Research in Office Settings Network, which receives core funding from the Health Resources and Services Administration Maternal and Child Health Bureau and by the American Academy of Pediatrics. The funders had no role in the design or conduct of the study; collection, management, analysis, and interpretation of the data; or in preparation, review, and approval of the manuscript. Funded by the National Institutes of Health (NIH).
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- ↵Pediatric Research in Office Settings Network. PROS Network Web site: About PROS. 2013. Available at: www2.aap.org/pros/abtpros.htm. Accessed December 16, 2013
- Wasserman RC,
- Slora EJ,
- Bocian AB,
- et al
- Wasserman RC,
- Croft CA,
- Brotherton SE,
- American Academy of Pediatrics
- Taylor JA,
- Darden PM,
- Brooks DA,
- et al
- Taylor JA,
- Darden PM,
- Slora E,
- Hasemeier CM,
- Asmussen L,
- Wasserman R
- Forrest CB,
- Glade GB,
- Starfield B,
- Baker AE,
- Kang M,
- Reid RJ
- Herman-Giddens ME,
- Slora EJ,
- Wasserman RC,
- et al
- Hagan J,
- Shaw J,
- Duncan P
- Committee on Adolescence American Academy of Pediatrics
- Hilgers D
- King J,
- Patel W,
- Furukawa M
- Agency for Healthcare Research and Quality
- Calmbach WL,
- Ryan JG,
- Baldwin LM,
- Knox L
- Gibson K,
- Szilagyi P,
- Swanger CM,
- et al
- Mold JW,
- Peterson KA
- Fagnan LJ,
- Handley MA,
- Rollins N,
- Mold J
- ↵US Department of Health and Human Services. Health Resources and Services Administration. Quality improvement. 2011. Available at: www.hrsa.gov/quality/toolbox/methodology/qualityimprovement/index.html. Accessed July 3, 2014
- Xierali IM,
- Rinaldo JC,
- Green LA,
- et al
- National Institutes of Health
- Copyright © 2014 by the American Academy of Pediatrics