Persistent Pulmonary Hypertension of the Newborn in the Era Before Nitric Oxide: Practice Variation and Outcomes
Objectives. In the era before widespread use of inhaled nitric oxide, to determine the prevalence of persistent pulmonary hypertension (PPHN) in a multicenter cohort, demographic descriptors of the population, treatments used, the outcomes of those treatments, and variation in practice among centers.
Study Design. A total of 385 neonates who received ≥50% inspired oxygen and/or mechanical ventilation and had documented evidence of PPHN (2D echocardiogram or preductal or postductal oxygen difference) were tracked from admission at 12 Level III neonatal intensive care units. Demographics, treatments, and outcomes were documented.
Results. The prevalence of PPHN was 1.9 per 1000 live births (based on 71 558 inborns) with a wide variation observed among centers (.43–6.82 per 1000 live births). Neonates with PPHN were admitted to the Level III neonatal intensive care units at a mean of 12 hours of age (standard deviation: 19 hours). Wide variations in the use of all treatments studied were found at the centers. Hyperventilation was used in 65% overall but centers ranged from 33% to 92%, and continuous infusion of alkali was used in 75% overall, with a range of 27% to 93% of neonates. Other frequently used treatments included sedation (94%; range: 77%–100%), paralysis (73%; range: 33%–98%), and inotrope administration (84%; range: 46%–100%). Vasodilator drugs, primarily tolazoline, were used in 39% (range: 13%–81%) of neonates. Despite the wide variation in practice, there was no significant difference in mortality among centers. Mortality was 11% (range: 4%–33%). No specific therapy was clearly associated with a reduction in mortality.
To determine whether the therapies were equivalent, neonates treated with hyperventilation were compared with those treated with alkali infusion. Hyperventilation reduced the risk of extracorporeal membrane oxygenation without increasing the use of oxygen at 28 days of age. In contrast, the use of alkali infusion was associated with increased use of extracorporeal membrane oxygenation (odds ratio: 5.03, compared with those treated with hyperventilation) and an increased use of oxygen at 28 days of age.
Conclusions. Hyperventilation and alkali infusion are not equivalent in their outcomes in neonates with PPHN. Randomized trials are needed to evaluate the role of these common therapies.
- PPHN =
- persistent pulmonary hypertension •
- NICHD =
- National Institute of Child Health and Human Development •
- NICU =
- neonatal intensive care unit •
- HFV =
- high frequency ventilation •
- ECMO =
- extracorporeal membrane oxygenation •
- CDH =
- congenital diaphragmatic hernia •
- OR =
- odds ratio •
- CI =
- confidence interval
Persistent pulmonary hypertension of the newborn (PPHN) contributes to neonatal hypoxemia, which is often refractory and associated with a high mortality (11%–48%).1–3However, these data are derived from reports that are over a decade old and represent experience from single centers. There has been no attempt to prospectively describe the neonatal population with PPHN, and thus information critical to the design of future interventional trials regarding prevalence and demographic descriptors such as racial or gender characteristics is lacking. Furthermore, there is no recent description of the natural history of PPHN.
The optimal approach to the treatment of PPHN remains controversial. In 1978, hyperventilation was advocated for the treatment of PPHN.4 ,5 It continues to be the cornerstone of therapy despite the lack of controlled trials and the challenge from Wung and colleagues.6 Newer therapies such as alkali infusion have been introduced without the support of randomized trial. Some neonatologists have assumed that alkali infusion is equivalent in its effects to hyperventilation and use it in their management of PPHN in addition to or instead of hyperventilation. There is a singular lack of knowledge about how these diverse management styles impact patient outcome.7–10
In 1993, large controlled trials of inhaled nitric oxide were in the planning stages. Given the large number of treatments used in neonates with PPHN before a child is considered for treatment with inhaled nitric oxide, we wished to document treatment of PPHN and outcomes of those treatments before the widespread dissemination of inhaled nitric oxide. Therefore, we performed a prospective, observational study to document prevalence of PPHN, the treatments and outcomes in a large population of infants treated at the 12 centers of the National Institute of Child Health and Human Development (NICHD) Neonatal Research Network. Further, we sought to compare treatment with hyperventilation to treatment with alkali infusion.
The study was conducted between October 1993 and December 1994 at 12 neonatal intensive care units (NICUs) participating in the NICHD Neonatal Research Network (designated here as centers A–L). All centers are university affiliated with active training programs for pediatric residents and neonatal fellows. A total of 71 558 neonates were inborn at the centers during the study. At 11 of the 12 Level III centers, additional newborns were transported for care. Advanced treatments for PPHN were available at all centers: high frequency ventilation (HFV) was available at all center and extracorporeal membrane oxygenation (ECMO) was available within the institution at 9 centers and by transport to other institutions at 3 centers (B, D, and L). Inhaled nitric oxide was a new treatment at the time of this study and was available at only 6 centers (A, B, C, F, I, and H).
Neonates with gestational ages ≥34 weeks who were <7 postnatal days of age were included if they met the following criteria: 1) mechanical ventilation and/or fraction of inspired oxygen >.50; and 2) documented pulmonary artery hypertension as defined by either two-dimensional echocardiographic evidence of elevated pulmonary pressure (judged by right to left or bidirectional shunt), or a preductal to postductal oxygen gradient >20 mm Hg. The infants were screened within the first 24 hours and were followed until enrollment criteria were met or until they reached 7 days of age, whichever occurred first. The ventilatory criterion of Fio 2 ≥.50 were deliberately chosen to be lower than the Fio 2 ≥.90 typically reported in works of PPHN, because we wished to detect all neonates with disease, although we postulated that few neonates receiving Fio 2 .50–.89 would be enrolled. Infants with a major congenital anomaly, with the exception of congenital diaphragmatic hernia (CDH) or patent ductus arteriosus, atrial and/or ventricular septal defect, were excluded.
Data Collection and Study Endpoints
Trained neonatal nurses abstracted the hospital record to collect demographic, perinatal, and neonatal information. Method of diagnosis and time of diagnosis were documented as well as treatments used and outcomes of these treatments.
Respiratory diagnoses were coded as respiratory distress syndrome, pneumonia, meconium aspiration syndrome, idiopathic pulmonary hypertension, pulmonary hypoplasia, CDH, or other. The specific therapies recorded included ventilator practices (hyperventilation defined as CO2 <35 mm Hg for >12 hours, maximum peak inspiratory pressure, highest mean airway pressure, maximum ventilatory rate), alkali infusion (sodium bicarbonate, tris(hydrixymethyl)-aminomethane, or both), inotrope administration (dopamine, dobutamine, isuprel, epinephrine, or other), paralysis, sedation, intravenous vasodilators (tolazoline, sodium nitroprusside, adenosine, prostaglandin E1, or other), surfactant administration, HFV, inhaled nitric oxide, and ECMO. Severity of illness was assessed as the lowest oxygen level in the first 7 days of life. The primary study endpoints were death, prevalence of oxygen use at 28 days of age, duration of intubation, and duration of oxygen administration. Treatment outcomes were designated at discharge, death, or 120 days of age, whichever occurred first.
The primary purpose of the study was to accurately describe the population of PPHN and to derive an accurate estimate of mortality. Therefore, the a priori sample size of 400 cases was calculated to estimate mortality within 5% with 95% confidence based on a mortality estimate of 20%. Prevalence was calculated using only inborn patients, because no accurate estimate of the total births among outborn patients was available. All other analyses included both inborn and outborn patients.
Continuous variables were summarized as means with standard deviation and range when the variables were normally distributed, and with medians when non-normally distributed. Categorical variables were analyzed with Student's t tests or analysis of variance. Outcome risks were estimated using dichotomized variables with odds ratios (0Rs) and 95% confidence intervals (CIs). Significance was set at α = .05 in all analyses.
The centers were ranked in order of survival and coded A (center with the lowest survival rate) through L (center with the highest survival rate). Treatment practices were compared as a function of this rank.
During analysis, we found that survival was significantly reduced in the neonates with CDH (n = 38), compared with all other diagnoses. Therefore, all analyses on treatment outcome were performed excluding neonates with CDH.
The study design and protocol were approved at each institution. Because the study was observational rather than experimental, with composite anonymous results, parental consent for participation was not sought.
A total of 71 558 neonates were born at the centers during the study. A total of 1351 neonates were screened for study of whom 385 (139 inborn) met study criteria of PPHN. The prevalence of PPHN (based on inborn births) was 1.9 per 1000 live births with a wide variation in prevalence among centers (.43–6.82 per 1000 live births).
Of the neonates enrolled in the study, 36% (range: 13–100) were born in the Network centers, and 64% were outborn. Mean birth weight was 3.3 ± .6 kg (range: 1.8–6.7 kg) and mean gestational age was 39 ± 2 weeks (range: 34–43). The cohort was 58% male, 33% black, 49% white, and 19% other racial group. Sixty-four percent had either public insurance or no insurance.
The characteristics of the cohort include a number of markers of high-risk status. Of 285 neonates with antenatal fetal heart rate monitoring, 139 (49%) had fetal bradycardia (38%), fetal tachycardia (7%), or both (4%). One hundred eighty-nine (50%) had meconium-stained amniotic fluid, of whom 70% received oropharyngeal suctioning by the obstetrician before delivery of the body, and 80% were intubated after birth to remove meconium from the airway. Forty-nine percent of the study infants were delivered by the vaginal route, and 51% by cesarean section. Of note, 32% of mothers, who delivered by cesarean section, did not have labor at the time of delivery.
Abnormal Apgar scores were common: 54% of the cohort had a 1-minute Apgar score of ≤5, and 21% had a 5-minute Apgar score ≤5. The enrolled population frequently received interventions in the delivery room. Of the population, 91% received oxygen, 55% were ventilated by bag and mask, 42% had an endotracheal intubation for resuscitation, 9% received chest compressions, and 12% were given 1 or more doses of emergency medications (epinephrine, sodium bicarbonate, calcium, atropine, narcan, or volume expander).
Diagnosis of PPHN
Neonates diagnosed with PPHN were admitted to the Level III NICUs at a mean of 12 ± 19 hours of age (range: 0–167 hours). Of the neonates, 77% were diagnosed with PPHN in the first 24 hours, 93% within the first 48 hours, and 97% by 72 hours of age. The diagnosis of PPHN was confirmed by 2 dimensional echocardiogram in 72%, by pre-postductal oxygen difference in 17%, and by both in 11%. The primary respiratory diagnoses of those enrolled were meconium aspiration syndrome (41%), idiopathic pulmonary hypertension (17%), pneumonia (14%), respiratory distress syndrome (13%), pneumonia and/or respiratory distress syndrome (classified as such when the 2 could not be distinguished; 14%), CDH (10%), and pulmonary hypoplasia (4%). Although all infants receiving Fio 2 ≥.5 without mechanical ventilation were screened for the diagnosis of PPHN, most infants (365/384; 95%) were ventilated and receiving Fio 2 ≥.9.
Overall survival was 88%. Center A (n = 6) had the lowest survival rate (67%), and center L (n = 23) had the highest survival rate (96%). These differences were not statistically significant. There was no correlation between survival rates and the prevalence of PPHN at any individual center (Table 1).
Survival varied significantly by diagnostic category. Survival was 94% among those with meconium aspiration syndrome, 91% among those with RDS or pneumonia, and 61% among those with CDH (P < .0002). Because of this difference, infants with CDH were removed in all subsequent analyses of the impact of treatments.
Treatments and Variation in Use
A large number of treatments were used in PPHN with considerable variation in practice between the 12 centers. Hyperventilation (defined as a Paco 2<35 mm Hg for >12 hours) was used in the treatment of 66% of neonates with wide variation between centers (range: 33%–92%) (Fig 1). When hyperventilation was used, the mean pH achieved was 7.63 ± .09 (range: 7.58–7.85). At least 1 infant was treated with a pH exceeding 7.7 in all 12 centers. A continuous infusion of alkali was used in 75% of neonates diagnosed with PPHN (92% sodium bicarbonate, 2% tris(hydrixymethyl)-aminomethane, and 6% both). Again, there was marked variation in use among centers (range: 27%–93%) (Fig 1). Neonates treated with hyperventilation were 2.8 times more likely to receive alkali infusion than those managed without hyperventilation.
Similar variation was seen in the use of inotropic agents (mean: 84%; range: 46%–100%) and intravenous vasodilators (mean: 39%; range: 13%–81%) (Table 2). Of those who received intravenous vasodilators, 53% received tolazoline. Use of sedation was common and relatively more uniform among centers (mean: 94%; range: 77%–100%). Use of paralysis was highly variable (73%; range: 33%–98%). Surfactant was used in 36% (range: 12%–71%) and HFV (39%; range: 0%–763%). Of those treated with HFV, 72% received oscillatory ventilation and 26% received jet ventilation. Inhaled nitric oxide, a new treatment at the time of this study, was used at 6 centers and in 8% of the cohort. ECMO was used in 34% of neonates (range: 0%–85%).
Impact of Treatments on Mortality and Pulmonary Outcome
Of all therapies evaluated, no individual therapy was associated with a reduced risk of death (OR: .97–2.84). In the full cohort 2 therapies were associated with increased mortality risk: neuromuscular paralysis (OR: 2.84; 95% CI: 1.12,7.16) and ECMO (OR: 2.12; 95% CI: 1.46,3.94). An additional therapy approached significance: HFV (OR: 1.83; 95% CI: .98,3.40). However, in subgroup analyses excluding patients with CDH, the mortality risks associated with these therapies were reduced substantially (Table 3). This suggests that the elevated mortality risk initially seen was largely explained by the increased severity of illness among those with CDH. Because of the differential impact on the subgroup with CDH, all subsequent analyses of treatment effect were performed excluding those with CDH (n = 38) leaving 347 neonates in the cohort.
In the cohort with CDH excluded, neonates were intubated for 10 ± 12 days. Seven percent were still treated with oxygen at 28 days of age. The average duration of hospitalization was 19 ± 16 days with 18% staying >28 days.
Hyperventilation was used in 66% of the cohort excluding CDH, and alkali was used in 75%. Of the cohort, 13% was treated with hyperventilation alone, 21% with alkali alone, 53% with both hyperventilation and alkali, and 13% with neither hyperventilation nor alkali (Table 4). Birth weight, gestational age, and respiratory diagnosis were similarly distributed across the 4 groups. As anticipated, patients treated without hyperventilation or alkali were modestly less ill than those treated with either hyperventilation alone or alkali alone, judged by lowest oxygen level in the first 7 days of life (45 ± 22 mm Hg, vs 43 ± 11, vs 39 ± 21; P < .01). Similarly, patients treated with both hyperventilation and alkali were modestly sicker than those receiving only 1 of the treatments (37 ± 11 mm Hg). Although these results were statistically significant, it is not clear that oxygenation differences in this range are clinically meaningful. Mortality was similar among the 4 groups. Those treated with hyperventilation alone were less likely to be treated with ECMO compared with the group who received neither hyperventilation nor alkali, and demonstrated the lowest use of oxygen at 28 days (Table 4). In contrast, the group treated with alkali infusion had a greatly increased chance of treatment with ECMO (OR: 5.03, compared with those treated with hyperventilation), and an increased rate of oxygen at 28 days (OR: 2.4; P < .01). Thus, treatment with alkali was not equivalent to treatment with hyperventilation.
This study is the first prospective, multicenter epidemiologic evaluation of the natural history of PPHN in a large cohort of neonates and defines the types of treatments used in the era before the introduction of inhaled nitric oxide. Several demographic features are of interest. First, 64% of those enrolled were born outside the network centers. Second, mothers with public or no insurance are represented heavily (64% of this cohort). This association has not been previously reported and may represent an unrecognized risk factor. Alternatively, it may be representative of the populations at the NICUs in the NICHD network, which are referral centers for high-risk populations. Third, meconium aspiration syndrome continues to be the largest diagnostic category seen. Preventive perinatal interventions would be most productive if focused on this category. Fourth, patients ultimately diagnosed with PPHN present early in life. Future studies can limit screening efforts to the first 72 hours of life and enroll 97% of all available patients.
Although the epidemiologic data collected are important, the diversity of practice documented at these 12 academic NICUs is perhaps the most intriguing aspect of this cohort. Despite the tremendous diversity of practice, there is no statistically significant difference in survival rates among the centers (67%–96%). We speculate that treatment variation did not impact mortality because most neonates who failed any combination of treatments were generally offered ECMO as a treatment of last option, and thus mortality was minimized (11% in this cohort). Therefore, studies that evaluate therapies for PPHN together with ECMO must use outcome measures other than mortality to assess the efficacy of these treatments. Candidates for appropriate outcome measures include intermediate and longer term measures of pulmonary and neurodevelopmental morbidity. In addition, the impact of new therapies on the use of ECMO and its associated risks is a legitimate outcome measure.
We further focused our analysis on hyperventilation and alkali infusion: the 2 therapies widely accepted as standard practice. Neither has been studied in a randomized controlled trial. Both were frequently utilized (hyperventilation in 65%; range: 33–88) and alkali infusion in 74% (range: 30–93). These analyses were exploratory in nature and designed to generate hypotheses to be tested in future trials, rather than to draw definitive conclusions about efficacy. Although we considered the possibility that neonatologists might use these therapies in different patient populations, we found the populations to be similar in birth weight, gestational age, and distribution of respiratory diagnoses. The OR of mortality associated with both of these therapies approach 1.0, which when considered alone suggests that both therapies are neither helpful nor harmful with respect to mortality. However, closer analysis suggests that hyperventilation may decrease the use of ECMO without increasing pulmonary morbidity as measured by the use of oxygen at 28 days of age (Table 4).
The outcome data on the impact of alkali infusion are, to our knowledge, the first analysis to estimate the association between the use of alkali and subsequent outcomes in neonates with PPHN. Many neonatologists have inferred that alkalosis induced by the metabolic route is equivalent to hyperventilation. These data suggest that the effects of metabolic and respiratory alkalosis are not equivalent. Alkali may not reduce mortality and may be associated with increased risk for the use of ECMO and prolonged oxygen dependency. We speculate that the use of alkali in neonates with parenchymal lung disease may raise Paco 2, prompting increases in peak inspiratory pressure, and thus aggravating barotrauma. These data suggest a need to further evaluate the use of metabolic alkalosis in PPHN.
Two therapies were associated with a statistically increased risk of death in the analysis that included the entire cohort: paralysis and ECMO. In addition, HFV was also associated with an elevated mortality risk that approached statistical significance (OR: 1.83; 95% CI: .98,3.40). When the neonates with diaphragmatic hernia were removed from the cohort, the elevated mortality risk was no longer seen. We believe that the elevated risk seen is attributable to the increased severity of illness in the CDH cohort. These data provide additional support for the practice of analyzing outcomes of PPHN treatments by diagnostic subclasses.
The increased risk of death associated with paralysis is perplexing and of concern. The OR of death associated with paralysis (2.84) exceeded that associated with either HFV (1.8) or ECMO (2.1). Although the differences were no longer significant when the CDH cohort was removed, we believe that the subject warrants additional investigation because the confidence limits were so broad in this cohort (OR: 1.95; 95% CI: .74,5.18) and because the therapy was associated with a mortality risk exceeding any other therapy. The mechanism of a potentially harmful effect of paralysis is unknown. Runkle and Bancalari11 investigated the acute cardiopulmonary effects of pancuronium bromide in 49 mechanically ventilated neonates. In 51% of infants, oxygenation improved by ≥10 mm Hg, whereas 49% had no change or deteriorated. The association of paralysis with an increased risk of death in our study should lead to a reexamination of this common practice.
By design, this study was performed before the randomized evaluation of inhaled nitric oxide at the NICHD Neonatal Research Network Centers12 to allow assessment of standard therapies before the introduction of the new treatment. We believe that our findings are important even in an era of wide spread availability of inhaled nitric oxide. Neonates with PPHN do not currently receive inhaled nitric oxide as their first treatment; it is reserved for the treatment of neonates who have failed other therapies such as hyperventilation and/or alkali infusion. Thus, the new therapy inhaled nitric oxide is given in addition to, not instead of, other therapies. The neonates ultimate outcome will be determined by the sum of the risks of all the therapies that he or she receives.
An analysis of the variations in practice among centers is intriguing and highlights possible areas for future trials. Only the frequency of the use of paralytic agents approached a statistically significant difference among centers; the 4 centers with survival exceeding the mean of 88% used paralysis less frequently (67 ± 28%) than at the other centers. (74% ± 18%; P = .06). Other practice trends at the 4 centers with the highest survival included less use of hyperventilation and alkali infusion and increased use of HFV.
There are several possible explanations for the practice variation seen. One interpretation might be that the diversity of practice reflects the heterogeneity of diagnoses or severity of illness in PPHN. However, meconium aspiration syndrome was the largest diagnostic category at every institution, and the distribution of diagnoses did not differ among centers. Therefore, diagnosis did not seem to explain the practice variation seen. A second possible interpretation is that a portion of treatment variation also is determined by individual physician preference.
Practice variation is not unique to PPHN. Wennberg and colleagues13 ,14 were the first to draw attention to the diversity of practice in benign prostatic hypertrophy. Subsequently, similar practice variation has been documented in numerous other disease states including coronary artery bypass surgery and carotid endarterectomy.15 ,16 Recent work has drawn attention to practice variation in neonatology.
Horbar and co-workers17 have focused attention on the differences in mortality in very low birth weight infants among different NICUs, which were unexplained by demographic factors or severity of illness. Similar work has shown wide practice differences between major academic perinatal centers in 2 adjacent New England states in the use of transfusions and narcotic administration.18 ,19 Our understanding of the factors that lead equally qualified physicians to select opposite treatments for similar patients is incomplete. O'Conner et al20has argued that one of the major drivers of practice variation is the inherent isolation in which most physicians practice: 1 doctor and 1 patient. Thus, particularly with less common diseases, like PPHN, 1 physician is unable to develop a large enough population to have an accurate view of the spectrum of outcomes among a patient population. Another factor contributing to practice variation in PPHN is the paucity of randomized trials of treatments.
Our study has limitations that we wish to acknowledge. Our study was conducted in level III centers and is generalizable only to patients at similar centers. The entire population of neonates with PPHN includes infants who are less ill and are never transported to level III centers as well as critically ill infants who die before transport.10 Thus, the most well and the most ill infants are not represented in this cohort. It is important to recognize that this study is observational. As such, treatment assignments are not randomly generated and thus may be biased by many factors including severity of illness and physician preference. An important variable in this study is severity of illness. We analyzed severity of illness using the lowest oxygen value recorded in the first 7 days. Other measures of severity of illness, such as oxygenation index are more robust and may have strengthened the design.
The intent of this study was to generate hypotheses for exploration in future studies. By design we evaluated PPHN treatments immediately before the widespread introduction of inhaled nictric oxide therapy. It is not known whether inhaled nictric oxide will supplant older treatments used in PPHN or whether it will be used in addition to those treatments. This study documents practice before inhaled nictric oxide and can be used in part of judge the impact of inhaled nictric oxide on the use of other treatment modalities.
Our study, the only multicentered epidemiologic investigation of a cohort of neonates with PPHN ever analyzed, suggests that the optimal treatment of neonates with PPHN before the use of inhaled nitric oxide is unknown. It strongly suggests that infusion of alkali to induce metabolic alkalosis is not equivalent in its effects to hyperventilation. This investigation highlights the difficulty of evaluating treatments introduced into practice without randomized clinical trials. We must examine untested therapies for evidence of efficacy and to question the use of therapies failing this examination. Our study suggests that in PPHN, a reevaluation of the roles of hyperventilation, alkali infusion, and paralysis is warranted.
This work was supported by the NICHD Cooperative Agreements U10 HD27881, U10 HD21373, U10 HD27851, U10 HD27853, U10 HD21397, U10 HD19897, U10 HD21415, U10 HD27856, U10 HD21364, U10 HD27880, U10 HD27904, U10 HD27871, and U10 HD21385 with the National Institute of Child Health and Human Development, and by General Clinical Research Center Grants MO1 RR00997, MO1 RR08084, MO1 RR00750, MO1 RR00070, and MO1 RR06022.
- Received May 21, 1997.
- Accepted February 24, 1998.
Reprint requests to (M.C.W.) Rainbow Babies and Childrens Hospital, Room 3020, 11100 Euclid Ave, Cleveland, OH 44106-6010. E-mail:
- Drummond WH,
- Gregory GA,
- Heymann MA,
- et al.
- Wung JT,
- James LS,
- Kilchevsky E,
- et al.
- Walsh-Sukys MC,
- Cornell DJ,
- Houston LN,
- et al.
- Winslow CM,
- Kosecoff JB,
- Chassin MR,
- Knouse DE,
- Brook RH
- Winslow CM,
- Solomon DH,
- Chassin MR,
- et al.
- Horbar JD,
- Badger GJ,
- Lewit EM,
- et al.
- Ringer SA,
- Richardson DK,
- Sacher RA,
- et al.
- Porter JE
- Manus DA,
- Werner TR,
- Strub RJ
- Alba T,
- Souders J,
- Silverman WA. Doing more good than harm. In: Warren KS, Mosteller F, eds. Doing More Good Than Harm: The Evaluation of Health Care Interventions. Ann N Y Acad Sci. 1993;703:5–11
- Copyright © 2000 American Academy of Pediatrics