Development and Evaluation of Global Child Health Educational Modules
OBJECTIVES: To determine if a standardized global child health (GCH) modular course for pediatric residents leads to satisfaction, learning, and behavior change.
METHODS: Four 1-hour interactive GCH modules were developed addressing priority GCH topics. “Site champions” from 4 Canadian institutions delivered modules to pediatric residents from their respective programs during academic half-days. A pre–post, mixed methods evaluation incorporated satisfaction surveys, multiple-choice knowledge tests, and focus group discussions involving residents and satisfaction surveys from program directors.
RESULTS: A total of 125 trainees participated in ≥1 module. Satisfaction levels were high. Focus group participants reported high satisfaction with the concepts taught and the dynamic, participatory approach used, which incorporated multimedia resources. Mean scores on knowledge tests increased significantly postintervention for 3 of the 4 modules (P < .001), and residents cited increases in their practical knowledge, global health awareness, and motivation to learn about global health. Program directors unanimously agreed that the modules were relevant, interesting, and could be integrated within existing formal training time.
CONCLUSIONS: A relatively short, participatory, foundational GCH modular curriculum facilitated knowledge acquisition and attitude change. It could be scaled up and serve as a model for other standardized North American curricula.
- GCH —
- global child health
- PGY —
- postgraduate year
What’s Known on This Subject:
Global health is of increasing interest and relevance to North American pediatric trainees. Opportunities for resident global health training and exposure are most often limited to electives or trainees in dedicated global health tracks.
What This Study Adds:
A series of short, structured, participatory global child health modules improved knowledge and were well received and integrated within academic programs. Such modules enable global health learning for all residents, including those who never intend to practice overseas.
North American pediatricians are caring for growing numbers of new immigrants and refugees, international adoptees, and pediatric travelers. Pediatric training needs to reflect these changing demographic characteristics. North American medical educators have become increasingly sensitized to the importance of global child health (GCH) and are advocating for the integration of these topics into mainstream pediatric training.1,2 Licensing bodies for Canadian and US pediatric training programs have identified competencies related to caring for vulnerable and underserviced populations as important for all pediatricians, and global pediatric priorities are aligned with these objectives. There is growing recognition that domestically focused pediatric training is no longer sufficient. As was recently stated by Garfunkel and Howard, 21st century pediatricians must be globally competent.2
To address increasing demand and enthusiasm for global health training, North American pediatric postgraduate programs have introduced GCH through electives,3,4 GCH-specific tracks,5 and formal (competency-based) and informal courses in global health.1,6,7 However, most pediatric GCH residency opportunities are currently only accessed by residents with a strong global health interest and/or intention to practice abroad.6,8 In addition, the evaluation of such programs has been limited.5,6
A previous needs assessment of Canadian pediatric postgraduate training programs identified heterogeneity in the coverage of core GCH topics and program interest in addressing these gaps.9 A working group of Canadian Paediatric Society members with GCH interest was established; their goal was to develop and evaluate a national foundational GCH program for inclusion in the training of all pediatric trainees. The current article presents findings from the mixed methods evaluation of these modules, assessing reaction (satisfaction), learning, and perceived behavior change after their delivery.
After the aforementioned needs assessment, a GCH working group, composed of GCH-interested practitioners from 8 academic centers located in 5 Canadian provinces, convened to develop GCH modules suitable for integration into existing formal training programs across Canada. Members were recruited from the Canadian Paediatric Society Global Child and Youth Health Section and included pediatricians (hospital-based generalists, infectious disease, and emergency medicine specialists), an adult infectious disease internist, and a medical education researcher. A number of our contributors also had extensive field experience in nutrition in low-resource settings.
After reviewing the needs assessment findings and consulting with program directors about logistics and priority needs, a unified format for presentation design and module delivery was established. Objectives were then developed in alignment with Canadian national pediatric resident training objectives (Royal College of Physicians and Surgeons of Canada, Canadian Medical Education Directives for Specialists).10 Priority topics were then divided between modules, and content was mapped to encourage consistency and flow between modules. Small teams comprising 3 to 4 working group members developed 4 draft modules, which were subsequently piloted by 2 of the authors at 2 nonstudy sites to 94 residents at all stages of training. Feedback from the pilot presentations was used to make revisions to the modules’ content, length, and format. Comments from pilot presenters informed the creation of a trainer manual. Modules were revised and edited for consistency, then translated (into French). A training session was conducted for “site champions” who served as preceptors to resident groups at study sites. Modules were delivered and evaluated by participants and by program directors. After the study, modules were revised based on evaluation results and made available electronically for national rollout. The modules and accompanying trainer manual are available for free to all Canadian Paediatric Society members at http://www.cps.ca/en/curriculum. Nonmembers can purchase a copy for the cost of printing and postage.
Four modules, each 1 hour in length, were administered to Canadian pediatric residents (postgraduate years [PGYs] 1–4 of training) from programs at urban academic health science centers in 4 different provinces (3 English, 1 French) during core curricular academic half-days. Attending the sessions was an academic program requirement, but participation in evaluation was voluntary. The study sites were purposefully selected as the home sites of our core team members to facilitate session organization and ethics approval for evaluation purposes. Co-presenters at each of the 4 study sites were authors and/or editors on 1 of the modules, but they had not contributed to the development of the other 3 modules, for which they used the trainer manual. The modules were presented over 2 academic half-days, 2 at each session.
Module topics included: (1) Global Child Mortality; (2) Undernutrition; (3) Fever in the Returning Child Traveler; and (4) Children and Youth New to Canada (Refugee and Immigrant Health). Each module included a series of 37 to 45 PowerPoint slides, case examples, and interactive self-assessment exercises (eg, practice Objective Standardized Clinical Examination questions). Two of the modules included video clips of real patients. Modules were delivered by 2 site champion global health enthusiasts from each study site who were guided by using a comprehensive trainer manual describing proposed delivery and background.
The GCH working group prospectively developed an evaluation framework to assess the GCH module intervention. The evaluation used a mixed method approach and was based on the first 3 levels of the Kirkpatrick 4-level model for evaluating training programs11 as follows.
Participating residents were invited to complete an 8-item reaction questionnaire based on a modified version of the Client Satisfaction Questionnaire12 immediately after each GCH module. The questionnaire includes a 4-point scale ranging from positive to negative reactions. After the delivery of all modules, the residents also participated in a 1-hour focus group which included 3 specific questions on aspects of the GCH modules that were most or least helpful. The program directors at 3 of the 4 sites also completed a separate 10-item reaction questionnaire after the delivery of all modules.
Participating residents completed closed-book knowledge tests, each consisting of 10 multiple-choice questions, immediately before and after each module. The study team created all knowledge tests. To ensure equal difficulty, the same test was used before and after each module. All test items corresponded to specific learning objectives from the GCH modules and were developed to align with core pediatric training objectives. Two external content experts assessed the final content of the tests for clarity and relevance. In the aforementioned focus groups, residents also reflected on their perceived knowledge acquisition.
Lastly, residents in the focus group were asked 3 specific questions about their recent application of the skills and knowledge that they obtained from the modules and how they might use the information and resources provided in their future clinical practice.
Ethics approval was granted by all participating study sites.
Descriptive statistics (SPSS version 18.0, IBM SPSS Statistics, IBM Corporation, Armonk, NY) were used to summarize demographic characteristics and reaction questionnaire scores, as well as the multiple choice knowledge test scores. Average tool scores (range: 8–32) were also calculated for the resident reaction questionnaires. Pre- and postmodule knowledge scores were compared by using paired t tests. Two-sided P values <.05 were considered significant.
All focus group discussions were audiorecorded and transcribed verbatim. Two members of the working group (who were not involved in the module delivery) independently analyzed the focus group transcripts by hand. First, they created a starter coding system based on the evaluation framework discussed earlier. They then read each focus group transcript several times, annotated phrases within the text, and coded the data by using the starter coding system. At this point, they also allowed for codes not identified a priori to emerge from the data. Upon completion of their coding, these members shared and compared their findings, discussed the accuracy of their analyses, and decided on the key themes to report.
The intervention consisted of four 1-hour modules and was delivered at 4 separate study sites during the 2009–2010 academic year.
A total of 125 of a potential 152 residents from 4 study sites participated in evaluation of at least 1 module: 6 (5%) participated in only 1 module, 61 (49%) participated in 2 modules, 5 (4%) participated in 3 modules, and 53 (42%) participated in all 4 modules (Table 1). Most sites presented the modules coupled in 2 sessions with 2 modules. Demographic characteristics were available for 122 participants. Thirty percent were residents in PGY-1, 25% were in PGY-2, 29% were in PGY-3, 13% were in PGY-4, and 3% were another level (fellow or clinical clerk). Participation was high across all 4 sites. Almost 60% of participants self-reported “no previous GCH experience,” whereas 40% reported at least some GCH experience.
Based on responses to the reaction questionnaire, residents were satisfied with the modules. The highest satisfaction was reported for the Undernutrition module (Table 2).
Resident focus group participants also described a high satisfaction level with the intervention. Three satisfaction themes emerged.
Module Detail and Contextualization
Residents were particularly satisfied with the figures and statistics presented and utilization of the World Health Organization publications and guidelines. They also appreciated site champion contextualization of information surrounding the realities of conditions in lower-income countries, explaining how this setting rendered the knowledge practical and accessible for use. Residents enjoyed considering how to make a difference worldwide and valued content regarding global social issues.
Technology and Resources
Participants had positive reactions toward the use of technology and multimedia resources. Many appreciated the use of simple technology (ie, mid/upper arm circumference paper tapes) to diagnose severe malnutrition. Residents also spoke highly of the visual slide layout, video clips, and the presenters’ experiential knowledge: “I appreciated the video tape… actually taken in Uganda and actually see a girl from Uganda who suffered from malnutrition, [it was] very helpful to hear what people there had to say, their cultural perspective on illness, on social issues, like family issues....” Residents appreciated that the site champions could appropriately answer questions based on personal experiences.
Residents reported satisfaction with session dynamics and design. They enjoyed the interactive module nature, including the combination of didactic teaching, role-playing, and question/answer opportunities. One Undernutrition activity, which assigns each resident to a child nutrition profile for plotting on growth charts, was popular: “[following this activity], I went home all excited; I was like I am going to run a WHO [World Health Organization] re-feeding program.” Some specific suggestions related to timing and structure of group exercises were raised.
The 3 responding program directors agreed that the GCH modules were relevant, effective, helpful, and appropriate for their residents. Specific elements cited as being beneficial included Objective Standardized Clinical Examination–style clinical vignettes, the practical GCH experience of site champions, and the emphasis on global, social determinants of health.
Pre- and postknowledge test scores demonstrated statistically significant resident improvement after 3 of the 4 modules: Global Child Mortality, Undernutrition, and Fever in Returned Child Traveler. Some knowledge gain was noted after the Children and Youth New to Canada (Refugee and Immigrant Health) module (Table 3).
Some focus group participants suggested they had already applied knowledge and skills from the modules; others anticipated future use of learning during their clinical training. Two major behavior change themes emerged.
GCH Awareness and Motivation
Residents reported improved motivation toward their own involvement in GCH issues. One junior resident stated, “It’s a motivation to get involved in international child health,” while another validated: “I had an interest in international health before this session…for me it is not so much behavior that will change but I found it encouraging that [the Canadian Paediatric Society] actually has a sub group of international child health…it gives more motivation and encouragement to try to get involved....”
Influence on Clinical Practice
Residents discussed how their newly acquired knowledge and skills could be used in their short- and long-term clinical practice, highlighting improved history-taking content and integration of social factors into differential diagnosis development. As 1 resident noted: “I have been working in the [emergency department] and there was a patient who came not with fever but with vomiting and had recently returned from Mexico…I think I was able to take a better travel history than I would have previously…” And according to another, “I think we look at failure to thrive as something that we all know as a script; [the Undernutrition module] added to that.”
What if all pediatric residents had the benefit of a global perspective of child health in their training, even those not planning to do electives or to work outside of North America? Would there be space available in an already-packed curriculum? Would such training align with existing pediatric core competencies? The results of our evaluation suggest that a short, interactive, foundational set of GCH modules is highly valued and can be integrated into pediatric residency training. While enhancing residents’ knowledge, these modules also demonstrated the potential to influence residents’ clinical practices. Increases in self-reported confidence in the ability to manage and advocate for GCH issues illustrate the potential for motivating future pediatricians, including those less likely to pursue other global health–related opportunities.
Our study and intervention are timely. In principle, our intervention addresses calls by North American educators2,13 and professional organizations14 for increased pediatric training and GCH programming.1 In practice, our program addresses identified specific gaps in Canadian national programs,9 national training core competencies (Canadian Medical Education Directives for Specialists),15 several major topics proposed by the US Accreditation Council for Graduate Medical Education,6 and the GCH competencies proposed by the American Academy of Pediatrics Section on International Child Health resident education working group.13 Use of a competency-based design and emphasis on relevant skills practice by using real-life scenarios is consistent with the “ideal” GCH training approaches proposed by expert groups.3,7,14
Results showing positive reaction and learning with modules suggest that basic GCH training for all pediatric residents can be effective and well received. In 2008, Oettgen et al5 observed increased pediatric resident knowledge after elective participation in a modular GCH program. Our findings complement these and extend to all pediatric trainees attending regular academic half-days. Measurable knowledge gain, as well as positive satisfaction scores, was observed in a variety of residents, including those with and without previous global health experience/interest. Also encouraging in our study, as mentioned in the focus groups, was the actual application of skills introduced shortly after the sessions, suggesting early behavior change resulting from the intervention. Our study design used a structured and prospective program assessment approach; the lack of such an approach had been identified as a deficit in previous GCH training program development.2,6,14 Results were strengthened by successful program uptake at 4 separate geographical and linguistic sites spread across the country.
Our GCH intervention is logistically feasible, and program director feedback was positive. Use of a relatively short, well-planned, standardized set of modules made the delivery of GCH during protected teaching time possible in the postgraduate training setting. The practicalities of our intervention included packaging of standardized, professional quality slides, audiovisual materials, handouts, and tutor guides in a format that could be delivered as four 1-hour modules, enabling flexible integration into academic half-days. The package was designed to equip any facilitator with GCH interest (but with varying levels of expertise) to facilitate sessions on relatively short notice and with minimal preparation.
However, several important study limitations require consideration. First, reaction (satisfaction) scores may have been falsely elevated due to participation bias. Less-interested residents may have made less effort to attend sessions; participating program directors were likely those considering GCH as important; and focus group participation was voluntary, and systematic differences between those who participated and those who did not were not documented and therefore may have been positively biased. Potentially, residents who participated in this evaluation component may have been more vocal, active, and interested in GCH, and they may have expressed more positive views toward the modules than those who did not participate. In hindsight, additional demographic information regarding the level of global health experience of the participants would also have been of interest. Second, learning scores reflect only short-term knowledge gain; knowledge retention over time was not tested. Finally, assessment of behavior change postintervention was limited. Self-perceived practice change provides a limited proxy for behavioral impact; larger longitudinal studies with cohort controls tracking residents’ future attitudes and activities would be needed to estimate actual behavioral outcomes.
The intervention provides an opportunity for all North American pediatric residents in GCH. The 4 foundational GCH modules developed and tested in the current study have undergone minor revisions based on study feedback and are currently being rolled out throughout Canada and at certain sites in the United States. The Canadian Paediatric Society Global Child and Youth Health Section will maintain and update the curriculum. Upcoming steps include the creation of a user follow-up survey and the promotion of best practices for GCH elective pre- and postdeparture training.
Several questions remain for future research. Does brief GCH training during residency enhance knowledge and interest longer-term, and do measurable practice changes ensue? How will the usability and response to the modules vary with different facilitators and in different practice settings? Would the modules and trainer manual still be effective if the facilitator did not have GCH experience or attend a training workshop? An important next step will be obtaining feedback from users of the modules by those who attended the facilitator training workshop as well as those outside of the Canadian context. A follow-up utilization survey of these groups is planned. Other potential future directions include modification for online use. We are aware of several other interest areas in the Canadian setting in which a similar modular approach is being used,16 and we see the opportunity for using this model for other pediatric subspecialties and interest groups who are considering developing high-quality, standardized, national educational materials.
We would advocate that core GCH content can and should be integrated within all mainstream pediatric training. Our carefully designed, interactive, targeted GCH modules could serve this purpose. Modules developed for this study may be complementary but should not replace established GCH streams, GCH electives, or predeparture training programs currently in use or being developed.
With GCH inclusion, soon-to-be pediatricians can have improved interest and ability to care for their young immigrant/refugee, international adoptee, and traveling patients. They also have the potential to be global advocates, even those pediatricians who will never leave North America.
The authors thank the following individuals who significantly contributed as advisors to module development: Drs Manuel Mah, Christine Greenaway, Susan Kuhn, Jennifer Druker, Hasu Rajani, Charles Larson, and Lindy Samson. The authors would like to thank our funders (see below) and the Canadian Pediatric Society for their ongoing support and assistance with copyediting of module materials and module translation and Kristina Rohde for her assistance with data analysis.
- Accepted September 19, 2013.
- Address correspondence to Tobey A. Audcent, MD, Children’s Hospital of Eastern Ontario, 401 Smyth Rd, Ottawa, ON, K1H 8L1. E-mail:
Dr Audcent served as project coordinator and participated in the conceptualization of the study, literature review, proposal writing, co-authoring 1 of the modules, editing of the modules, study development and data analysis, drafting the initial manuscript, and reviewing, revising, and submitting the manuscript; Dr MacDonnell served as curriculum lead and participated in the conceptualization of the study, proposal writing, presenting the modules and supervising data collection at 1 of the 4 study sites, editing of the educational modules, and reviewing and revising the manuscript; Dr Moreau was the data coordinator and designed the data collection instruments, coordinated the data collection from all sites, coordinated and chaired the focus groups, oversaw the study development and data analysis, and participated in drafting the initial manuscript and reviewing and revising the manuscript; Dr Hawkes participated in the conceptualization of the study, co-authoring 1 of the modules, proposal writing, study development and data analysis, and drafting the initial manuscript, reviewing, and revising the manuscript; Dr Sauve participated in conceptualization and design of the study, co-authoring 1 of the modules, presenting the modules and supervising data collection at 1 of 2 pilot sites, study development and data analysis, and reviewing and revising the manuscript; Dr Crockett participated in co-authoring 1 of the modules, study development and data analysis, presenting the modules and supervising data collection at 1 of the 4 study sites, and reviewing and revising the manuscript; Dr Fisher participated in co-authoring 1 of the modules, presenting the modules and supervising data collection at 1 of the 4 study sites, study development and data analysis, and reviewing and revising the manuscript; Dr Goldfarb participated in co-authoring 1 of the educational modules and reviewing module content, study development and data analysis, and reviewing and revising the manuscript; Dr Hunter participated in co-authoring 1 of the modules, presenting the modules and supervising data collection at 1 of 2 pilot sites, study development and data analysis, and reviewing and revising the manuscript; Dr McCarthy participated in the conceptualization of the study, critique of the funding proposal, study development and data analysis, review of the module content, and reviewing and revising the manuscript; Dr Pernica participated in conceptualization and design of the study, design of the data collection instruments, study development and data analysis, co-authoring of 1 of the modules, and reviewing and revising the manuscript; Dr Liu participated in the literature review, proposal writing, co-presenting the modules and supervising data collection at 1 of the 4 study sites, study development and data analysis, translation of the modules, and reviewing and revising the manuscript; Dr Luong participated in the literature review, proposal writing, co-presenting the modules and supervising data collection at 1 of the 4 study sites, study development and data analysis, translation of the modules and editing of the translated version, and reviewing and revising the manuscript; Drs Sandhu and Rashed participated in co-authoring of the modules, editing of the modules, co-presenting the modules and supervising data collection at 1 of the 4 study sites, study development and data analysis, and reviewing and revising the manuscript; Dr Levy participated in the literature review, proposal writing, co-presenting the modules and supervising data collection at 1 of the 4 study sites, study development and data analysis, translation of the modules, and reviewing and revising the manuscript; and Dr Brenner was the principal investigator and participated in conceptualization of the study, proposal writing, co-authoring 1 of the educational modules, study development and data analysis, editing the modules, and reviewing and revising the manuscript.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Financial support for this project and study was provided by the Royal College of Physicians and Surgeons of Canada and Associated Medical Services Inc, the University of Montreal, Department of Education, Sainte-Justine Hospital, Department of Paediatrics, Maisonneuve-Rosemont Hospital, and the Children’s Hospital of Eastern Ontario Research Institute.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Nelson BD,
- Lee AC,
- Newby PK,
- Chamberlin MR,
- Huang CC
- Oettgen B,
- Harahsheh A,
- Suresh S,
- Kamat D
- ↵Audcent T, MacDonnell H, Brenner J, Samson L. International Child Health (ICH) Education in Canadian paediatric residency programs. Clin Invest Med. 2007. Available at: http://cimonline.ca/index.php/cim/article/view/2825. Accessed October 15, 2013
- ↵Royal College of Physicians and Surgeons of Canada Objectives of Training in Pediatrics. Available at: http://rcpsc.medical.org/residency/certification/objectives/pediat_e.pdf. Accessed October 2, 2012
- Kirkpatrick DL,
- Kirkpatrick JD
- AAP Section on International Child Health Resident Education Working Group
- ↵Royal College of Physicians and Surgeons of Canada CanMEDS Framework. Available at: www.royalcollege.ca/portal/page/portal/rc/canmeds. Accessed October 2, 2012
- Copyright © 2013 by the American Academy of Pediatrics