OBJECTIVES: Determine the performance of the American Academy of Pediatrics (AAP) critical congenital heart disease (CCHD) newborn screening algorithm and the impact of an alternative algorithm.
METHODS: Screening was performed on term infants without a known CCHD diagnosis at or near 24 hours of age at a tertiary birth hospital by using the AAP algorithm from 2013 to 2016. Retrospective review from the birth hospital and the area’s sole pediatric cardiac center identified true- and false-positives and true- and false-negatives. A simulation study modeled the results of a modified screening algorithm with a single repeat pulse oximetry test instead of 2.
RESULTS: Screening results were collected on 77 148 newborns . By using the current AAP algorithm, 77 114 (99.96%) infants passed screening, 18 infants failed for an initial saturation of <90%, and 16 failed after not attaining a passing pulse oximetry level after 3 tests. There was 1 true-positive (total anomalous pulmonary venous return), 33 false-positives, and 6 false-negatives, yielding an overall specificity of 99.96%, a sensitivity of 14.3%, and a false-positive rate of 0.043%. Among false-positives, 10 (31.3%) had significant non-CCHD disease. Simulating the modified algorithm, sensitivity remained at 14.3%, and the false-positive rate increased to 0.054%.
CONCLUSIONS: Although CCHD screening in a tertiary care birth hospital may not detect many new cases of CCHD, it can detect other important diseases in newborns. Modifying the screening algorithm to 1 repeat pulse oximetry test instead of 2 may detect additional infants with significant disease without a substantial increase in the false-positive rate.
- AAP —
- American Academy of Pediatrics
- ASD —
- atrial septal defect
- CAVC —
- complete atrioventricular canal
- CCHD —
- critical congenital heart disease
- CHD —
- congenital heart disease
- CHOA —
- Children’s Healthcare of Atlanta
- HLHS —
- hypoplastic left heart syndrome
- TAPVR —
- total anomalous pulmonary venous return
- TOF —
- tetralogy of Fallot
- VSD —
- ventricular septal defect
What’s Known on This Subject:
Current critical congenital heart disease (CCHD) screening recommendations involve the use of pulse oximetry to detect hypoxemia in newborns. No single algorithm has proven to be superior. Infants with failed CCHD pulse oximetry screenings have been found to have non-CCHD disease.
What This Study Adds:
Modifying the algorithm for CCHD screening such that there is only 1 repeat screening instead of 2 could improve detection of hypoxemic disease in the newborn with minimal impact on the CCHD screening false-positive rate.
Congenital heart disease (CHD) occurs in ∼8 out of every 1000 live births each year, affecting ∼40 000 infants per year in the United States.1–3 Among infants with CHD, ∼25% have lesions considered to be critical congenital heart disease (CCHD), requiring surgical or catheterization intervention within the first year of life.1 Although the 1-year survival rate for infants with CCHD has improved over the decades,4 it remains a leading cause of infant morbidity and mortality.5 To aid in the early detection of CCHD, screening for CCHD by using pulse oximetry was added to the US Recommended Uniform Screening Panel in 2011 and has been endorsed by multiple professional societies, including the American Academy of Pediatrics (AAP), the American Heart Association, and the American College of Cardiology, to improve early identification of infants with CCHD.6,7 Recently, CCHD screening has been shown to decrease infant mortality from CCHD in states with mandatory screening policies.8 CCHD pulse oximetry screening was added to the Georgia newborn screening panel in 2015,9 although many hospitals in Georgia began using CCHD pulse oximetry screening as early as 2012.10
Current CCHD screening recommendations involve the use of pulse oximetry to detect levels of hypoxemia in the newborn infant. On the basis of the pulse oximetry values, an algorithm is used to determine if a child has passed or failed screening; however, there is no single algorithm that has been proven to be superior.6,11 In most states in the United States, screening is performed by using the algorithm endorsed by the AAP (Fig 1),12 although other states such as New Jersey13 and Tennessee14 have adopted modifications of that algorithm. The AAP algorithm is based off of previous work performed in Sweden.15
The purposes of this study were to (1) evaluate the experience of CCHD screening in a large tertiary birth hospital as part of a collaborative quality improvement initiative and (2) to determine the impact of CCHD screening by using pulse oximetry if modification was made to the existing AAP algorithm to have a single repeat screen instead of 2.
Through a collaborative quality improvement project between the cardiology department at Children’s Healthcare of Atlanta (CHOA) and the neonatology department at Northside Hospital, a retrospective review was conducted on newborn CCHD screening results collected on term infants born between January 1, 2013, and December 31, 2016, at Northside Hospital, a large tertiary birth hospital with 2 delivery campuses in metropolitan Atlanta, Georgia. Full-time pediatric cardiology coverage is provided daily at Northside Hospital by physicians from CHOA. Approximately 80% of infants with CCHD were diagnosed prenatally. Infants with a prenatal diagnosis of CCHD and infants transferred to the NICU before screening were excluded from CCHD pulse oximetry screening. The study was submitted and approved by the Northside Hospital Institutional Review Board. Aggregate de-identified data were used for analysis. Informed consent was not required because of the retrospective observational nature of the study.
Data collected from birth records at Northside Hospital included (1) saturation levels from the right hand and either foot for the initial and any repeat screenings, (2) the results of any echocardiographic evaluations, and (3) other significant noncardiac diagnoses identified during the newborn hospitalization for infants who failed CCHD screening. Provider interpretation of results was also included, although this was noted to be missing or incorrectly interpreted for 320 infants (0.41%). To correct for these errors, we completed an independent analysis of pulse oximetry results using the AAP CCHD screening algorithm to identify infants as passing with a saturation ≥95% in either the hand or the foot and a ≤3% difference, failing because of a saturation <90%, and those failing after completing 3 indeterminate screens.
Identifying True-Positives, False-Positives, True-Negatives, and False-Negatives
True-positives and false-positives were identified from the screening records at Northside Hospital. To ascertain true-negatives and false-negatives, surgical records were reviewed at CHOA, the sole pediatric cardiac surgical center in the area, between January 1, 2013, and June 30, 2017, to identify infants with documentation of Northside Hospital as the birth hospital and who underwent surgical intervention for CCHD within the first 6 months of life. We considered CCHD lesions to include hypoplastic left heart syndrome (HLHS), pulmonary atresia, tetralogy of Fallot (TOF), total anomalous pulmonary venous return (TAPVR), transposition of the great arteries, tricuspid atresia, truncus arteriosus, coarctation of the aorta, double outlet right ventricle, Ebstein anomaly, interrupted aortic arch, and any other single ventricle physiology.6 Infants born at Northside without a prenatal diagnosis of CCHD who underwent surgical intervention at CHOA for 1 of these CCHDs and who were not detected by screening were considered false-negatives; all other infants who had negative screenings were considered true-negatives.
A simulation study was performed to model how the results would have differed if the algorithm were modified to have only 1 repeat pulse oximetry test instead of 2 for those infants with indeterminate results on their initial screen (Fig 2). Birth records were reviewed for the additional infants then identified as failed screenings for any significant diagnoses.
Summary statistics were calculated on the basis of the screening experience and the simulation study by using SAS version 9.4 (SAS Institute Inc, Cary, NC). Sensitivity, specificity, and false-positive rate were calculated by using Excel for the traditional AAP algorithm and the modified algorithm.
Pulse oximetry results were collected for a total of 77 184 newborns. Hospital providers identified 76 850 infants as passing screening (99.6%) and 168 infants as failing screening (0.22%). There were 166 infants with no interpretation given (0.22%). Independent analysis of pulse oximetry results revealed provider misinterpretation of results in 154 screenings (0.20%), including 20 misinterpreted as passing (0.026%) (Supplemental Table 4). After correcting for errors in interpretation and exclusion of infants with insufficient screening data, 77 148 infants were included in analysis. We identified 77 114 infants (99.96%) who passed screening, 18 infants who failed screening on the basis of a saturation <90% in the hand or foot, and 16 who failed screening on the basis of indeterminate results after completion of 3 tests (Fig 3). All infants who failed because of a saturation <90% (n = 18) were identified on the initial screening test.
There was 1 true-positive, 33 false-positives, and 6 false-negatives, yielding an overall specificity of 99.96%, sensitivity of 14.3%, and false-positive rate of 0.043% (1 per ∼2500 live births). Positive screen results are shown in Table 1. The lone true-positive was an infant with TAPVR identified by a failed screening due to an initial saturation <90%. Among the false-positives, 10 (31.3%) were identified with non-CCHD disease considered to be significant because of the need for intervention or cardiology follow-up to prevent significant morbidity or mortality. Diagnoses for these 10 infants included pulmonary hypertension (n = 5), neonatal abstinence syndrome (n = 1), and noncyanotic CHD, including atrial septal defect (ASD) (n = 1), ventricular septal defect (VSD) (n = 1), complete atrioventricular canal (CAVC) defect (n = 1), and hypertrophic cardiomyopathy in an infant of a diabetic mother (n = 1). False-negatives included HLHS (n = 1), TOF (n = 1), and coarctation of the aorta (n = 4) (Table 2).
In the simulation study in which the algorithm was modified to have only 1 repeat pulse oximetry test instead of 2, an additional 9 infants would have failed the test (Fig 4). Among these, 2 had echocardiograms obtained that were normal (1 obtained for evaluation of a murmur, 1 obtained after being misinterpreted as a failed screen), and none had a CCHD or significant non-CCHD diagnosis at the time of discharge (Table 3). The overall sensitivity remained at 14.3%, and the false-positive rate increased to 0.054% (1 per ∼2000 live births).
In this, the largest single hospital report of screening thus far in the United States, we found that CCHD screening by using pulse oximetry has limited ability to detect additional cases of CCHD in a tertiary birth center with a high prenatal detection rate. However, we demonstrate that screening can detect other important non-CCHD diseases in newborns, and we would propose that modifying the algorithm to have only 1 repeat screen instead of 2 may identify additional newborns with significant hypoxemic disease, both cardiac and noncardiac, with a minimal impact on the CCHD screening false-positive rate. Given these results, we advocate that algorithms used for CCHD screening by using pulse oximetry be modified such that a single repeat screen is recommended for an indeterminate result instead of 2.
CCHD screening by using pulse oximetry identified only 1 infant with CCHD, similar to recent studies16–18 in which researchers found that few infants with CCHD were identified by pulse oximetry screening. The low detection rate by using pulse oximetry screening in our study is likely explained by the ∼80% prenatal detection rate of infants with CCHD born at Northside Hospital identified by fetal imaging and subsequently excluded from pulse oximetry screening. Indeed, CCHD screening is expected to have a greater detection rate in hospitals with level I or II nurseries than in a hospital with a level III nursery such as Northside.19 We found pulse oximetry screening to have an overall low sensitivity (14.3%), much lower than that seen in the 2012 meta-analysis by Thangaratinam, et al20 (76.5%), although this difference may be in the definition of what defects should be classified as a CCHD, namely coarctation of the aorta. Pulse oximetry screening failed to identify 6 children with CCHD, including 1 infant with HLHS, 1 with TOF, and 4 with coarctation of the aorta. It is well established that detection of coarctation of the aorta and TOF is low when using pulse oximetry screening20–22 because these infants do not always present in the initial newborn period with hypoxemia. All of the infants with false-negative screening results eventually presented with respiratory symptoms or abnormal examination findings, highlighting the importance of complete evaluation and not relying solely on pulse oximetry screening and prenatal imaging to identify infants with CCHD.
Although screening with pulse oximetry did not detect many infants with CCHD, it did help identify infants with other significant diseases that require intervention to prevent significant morbidity and mortality in a newborn. Among the infants with failed screenings, we identified multiple infants with pulmonary hypertension, an infant with neonatal abstinence syndrome, hypertrophic cardiomyopathy in an infant of a diabetic mother, and non-critical CHD cases that required additional evaluation and follow-up by a pediatric cardiologist, including ASD, VSD, and CAVC. These results are similar to findings by Singh, et al22 who reviewed infants admitted to the NICU for failed CCHD pulse oximetry screenings and identified more infants with non-CCHD diseases, including pulmonary hypertension, pneumonia, sepsis, and non-critical CHD, than those with CCHD. Although the primary purpose of CCHD pulse oximetry screening has been to target and identify infants with CCHD, using pulse oximetry screening can additionally target newborns with other significant diseases, leading to earlier evaluation, diagnosis, and potentially lifesaving intervention.
Although screening was easily performed on nearly 20 000 newborns born each year at a large tertiary birth hospital, the interpretation of pulse oximetry results was at times inaccurate. Authors of previous studies17,23 have similarly shown that providers may often misinterpret the AAP algorithm and results of CCHD pulse oximetry. In our study, there were 320 instances (0.41%, roughly 1 per ∼250 births) when screening was incomplete or interpreted inaccurately. In some cases, screening was incomplete because of procedural processes with pulse oximetry obtained on only the right hand. Although screening the right hand only may not miss an infant with systemic hypoxemia, it may miss an infant with lesions that cause postductal hypoxemia without affecting the preductal saturation, like in the case of a critical coarctation of the aorta. The remaining cases were misinterpretations of results. Most of these involved misinterpretation of a passing result as a failure or need for repeat screening. Although this did not affect the final outcome for the infants, it did result in additional evaluation and potential anxiety for parents. More concerning were cases in which results were incorrectly interpreted as passing when the infant should have had repeat screening performed. In many of these cases, an infant was deemed as passing, although there was a saturation differential of 4% between the right arm and a leg, whereas in the remaining cases, the infant did not attain a passing saturation level ≥95%. Although the rate of misinterpretation as passing was only 1 in ∼4000 live births and such instances were brought to a provider's attention, it does present a quality improvement opportunity.
Although the alternative algorithms currently used in New Jersey13 (requiring saturation of ≥95% in both the right hand and either foot) and Tennessee14 (initial screening of either foot with saturation ≥97% resulting in a “pass”) differ slightly from the AAP algorithm, we sought to evaluate the impact of modifying the AAP algorithm to have only 1 repeat screen instead of 2. Conceivably, infants with a saturation level <90% are most likely to have a significant disease that requires emergent evaluation and treatment. All of the infants who failed pulse oximetry screening on the basis of a saturation level <90%, including the true-positive case (TAPVR), were identified by the initial pulse oximetry screening. No infant with a saturation level <90% would have been missed by eliminating the third screening, and there would be no decrease in the overall sensitivity of screening (14.3%). Eliminating the third screening would have increased the false-positive rate from 0.043% (1 per ∼2500 births) to 0.054% (1 per ∼2000 births), resulting in an additional 2 to 3 infants undergoing cardiology evaluations each year in a hospital system with nearly 20 000 deliveries per year. Although this would increase the overall number of infants needing further evaluation, modifying the CCHD screening algorithm to have only 1 repeat pulse oximetry test instead of 2 may help detect additional infants with significant diseases, as discussed above, without a substantial increase in the false-positive rate. Simplifying the algorithm may also improve compliance and interpretation of screening.
This is the largest single-hospital report of CCHD pulse oximetry screening thus far in the United States. In addition, right hand and foot saturation levels were available for all infants included in the analysis, allowing for direct comparison of provider interpretation of CCHD pulse oximetry screening to an independent analysis of pulse oximetry results by using the current AAP algorithm. Finally, with 1 sole provider of pediatric cardiac surgery in the area, we were able to capture any false-negatives who would have presented for surgical intervention after discharge from the newborn hospitalization.
Our study is not without limitations. First, this was a retrospective review of screening results and hospital records. Errors in documentation may have occurred that resulted in inaccurate recording of saturation levels. Additionally, in the first year of pulse oximetry screening, staff reported challenges documenting the repeat screening results, which may have affected the number of apparent incomplete screenings. Second, information regarding diagnoses of infants who passed screening per the AAP algorithm but would have failed with our modified algorithm was limited to those diagnoses listed in the newborn record. These infants did not undergo cardiology evaluation, except for 2 infants who had echocardiograms performed because of abnormal physical examination findings and misinterpretation of screening results, respectively. Finally, infants with CCHD who passed pulse oximetry screening but later died after discharge from the nursery were not identifiable because death records were not reviewable, although any such cases would further decrease screening sensitivity.
Although CCHD screening by using pulse oximetry in a tertiary care birth hospital with a high CCHD prenatal detection rate may not detect many new cases of CCHD, it can detect other important causes of both cardiac and noncardiac disease. Modifying the CCHD screening algorithm to have only 1 repeat pulse oximetry test instead of 2 may increase the detection of infants with significant disease without a substantial increase in the false-positive rate and may improve compliance and interpretation of screening. Further efforts to improve the sensitivity of screening are also warranted.
- Accepted February 21, 2018.
- Address correspondence to Matthew E. Oster, MD, MPH, Sibley Heart Center Cardiology, 2835 Brandywine Rd, Atlanta, GA 30341. E-mail:
FINANCIAL DISCLOSURE: Dr Kochilas is a consultant for the Novartis Pharmaceutical Corporation and site investigator for the Novartis PANORAMA-HF trial on pharmacologic management of pediatric heart failure and receives research funds from the National Institutes of Health National Heart, Lung, and Blood Institute; the other authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2018-0576.
- Mahle WT,
- Newburger JW,
- Matherne GP, et al; American Heart Association Congenital Heart Defects Committee of the Council on Cardiovascular Disease in the Young, Council on Cardiovascular Nursing, and Interdisciplinary Council on Quality of Care and Outcomes Research; American Academy of Pediatrics Section on Cardiology and Cardiac Surgery; Committee on Fetus and Newborn
- Hoffman JI,
- Kaplan S
- Oster ME,
- Aucott SW,
- Glidewell J, et al
- Tennessee Department of Health
- de-Wahl Granelli A,
- Wennergren M,
- Sandberg K, et al
- Klausner R,
- Shapiro ED,
- Elder RW,
- Colson E,
- Loyal J
- Ailes EC,
- Gilboa SM,
- Honein MA,
- Oster ME
- Singh A,
- Rasiah SV,
- Ewer AK
- Copyright © 2018 by the American Academy of Pediatrics