Implementation of “Helping Babies Breathe”: A 3-Year Experience in Tanzania

Discussion

This study adds valuable information on implementing and sustaining the HBB program, because it is one of the first and most extensive assessments of HBB at scale. We analyzed 3 years of monitoring and evaluation data from >13 000 health care providers in 15 of Tanzania’s mainland regions. Although the evaluation used several methods to assess the success of the program, including a cost analysis and qualitative interviews, the current study focuses largely on the assessment of providers’ resuscitation skills and equipment availability.

The initial success of HBB on developing provider-level resuscitation skills is evidenced by 87.1% of providers passing the HBB OSCE after training. However, to determine skill retention, we also assessed HBB skills over time. During the follow-up visits led by the implementing partner, the percentage of providers with passing OSCE scores decreased by 7.7% at 4 to 6 weeks and by 31.3% at 4 to 6 months. Early identification of this decline in skills prompted the development of interim interventions, which included structured tools to facilitate on-the-job HBB practice and supportive supervision by trained maternity ward leaders and district-level trainers. Similar to findings in a recent review of newborn resuscitation training approaches, we saw an improvement in HBB skill retention with the implementation of these interventions. Another remarkable accomplishment of the program was that the percentage of providers who passed the OSCE improved as the program matured (Fig 2). However, the odds of a provider passing the OSCE differed by the facility level in which they worked and their health cadre. The odds of passing the OSCE were 15% lower among providers working at dispensaries compared with hospitals. Furthermore, compared with medical doctors, medical assistants were less likely to pass the OSCE. However, medical assistants are usually unskilled birth attendants and perform very few deliveries. Skilled birth attendants such as nurse-midwives and clinical officers, who perform most of the deliveries, showed no significant difference in odds of passing the OSCE compared with medical doctors.

In addition to improving providers’ resuscitation skills, other successes of the HBB program in Tanzania were creating critical attention for newborn health and achieving widespread program coverage and implementation in an affordable way. In each region, by 4 to 6 weeks after implementation of the program, 68.7% of providers working in the maternity wards and obstetric theaters had received the HBB course. Coverage dropped to 57.1% at 4 to 6 months, probably because of attrition, new hires, and the common practice of interdepartmental staff rotation. Although the majority of providers were still trained in HBB at long-term follow-up visits, the decline in trained providers indicates the need for continued trainings. Coverage of resuscitation equipment, such as bag-mask devices and penguin suction devices, was high at >90% of facilities. Equipment also proved to be durable, in that almost all devices were still functional at the longer-term follow-up visits. The cost to train providers and equip the facilities was ∼$600 USD per health facility, equating to a national rollout cost of ∼$4 million, making HBB affordable in low-resource regions.¹⁰ A recent study examining HBB costs per lives saved in a rural missionary hospital in Tanzania found implementation costs of $233 USD per life saved ($4.21 USD per life year gained) and program maintenance costs of $80 USD per life saved ($1.44 USD per life year gained).¹²

There were challenges to at-scale implementation of the HBB program. Sustaining high coverage of trained providers was difficult, as discussed. Although attempts were made to measure program impact on neonatal mortality by using historic and existing health information systems, reliably quantifying this impact was not possible. Measuring mortality in resource-limited settings is challenging because deaths often go undocumented, especially before a program is implemented. Subsequently, with program implementation comes better documentation, which can initially lead to a perceived increase in mortality. HBB training also corrects providers’ often-misunderstood definitions of perinatal outcomes, which can lead to an apparent increase in newborn mortality (eg, previously miscategorized “stillbirths” are correctly categorized as newborn deaths). Options to help overcome some of these data challenges include parallel data collection systems, sentinel panel of districts or facilities, active surveillance, and composite indicators (eg, perinatal death). But these options come with significant financial and logistical costs for country-level programs.

Recognizing the challenges of resources and clinical coverage during trainings, the American Academy of Pediatrics supports 1- or 2-day HBB trainings for providers.⁹ The Tanzania MOHSW chose a 1-day training approach because of the desire to expeditiously and cost-effectively train a large number of providers across large geographic regions with finite resources. This 1-day training was complemented by the mentioned interim interventions, specifically facility-based supportive supervision visits and structured on-the-job training intended to mitigate any negative impact from the shorter training course. The ideal duration of training remains unclear because data show that frequent, short trainings are more effective than long, infrequent trainings.¹³ Therefore, the ideal neonatal resuscitation training program for large-scale implementation in resource-limited settings may permit 1-day courses if they are followed by frequent refresher or formal in-facility mentorship programs. Consistent with this approach, Jhpiego is currently implementing a new facility-based clinical mentorship program to address concerns with skill retention. Nevertheless, it is still possible that the shorter, pragmatic 1-day training approach reduced the potential impact of the HBB program on provider skills. The program was implemented largely as a stand-alone vertical training program; more effectively embedding HBB into comprehensive newborn programming and preservice training would probably help ensure cost-effectiveness and sustainability.

The current study does have limitations. First, this country-level HBB implementation in Tanzania may not necessarily be generalizable to other resource-limited settings. However, its 3-year design, large and diverse sample, and implementation within 15 large regions significantly increase the study’s representativeness. Second, social desirability bias could have occurred during focus group discussions. The use of an independent and experienced evaluation team and the confidential nature of the discussions probably mitigated this bias. Third, it is unknown whether good performance on OSCEs translates into improved care in the clinical setting.^14,15 Fourth, some providers were lost to follow-up through attrition or rotation to other departments. Lastly, as discussed, we were not able to accurately measure the program’s impact on neonatal mortality because of the poor quality and availability of data in historic health information systems. Additional program focus and resources will probably be necessary to accurately measure mortality impact in these resource- and data-limited settings.