pediatrics
June 2003, VOLUME111 /ISSUE 6

Evaluating the American Academy of Pediatrics Diagnostic Standard for Streptococcus pyogenes Pharyngitis: Backup Culture Versus Repeat Rapid Antigen Testing

  1. Karen E. Gieseker, PhD*,§,
  2. Martha H. Roe, SM (ASCP),
  3. Todd MacKenzie, PhD#, and
  4. James K. Todd, MD*,,§,,
  1. *Departments of Epidemiology
  2. Pathology, Children’s Hospital
  3. §Departments of Preventive Medicine
  4. Microbiology
  5. Pediatrics, University of Colorado Health Sciences Center, Denver, Colorado
  6. #Department of Medicine, Dartmouth Medical School, Hanover, New Hampshire

Abstract

Objective. The American Academy of Pediatrics recommends that all negative rapid diagnostic tests for Streptococcus pyogenes pharyngitis be backed up by culture, which creates a dilemma for clinicians who must make treatment decisions without complete diagnostic information at the time of visit. The use of a follow-up serial rapid antigen test instead of a follow-up culture would provide a more timely result.

Methods. Two swabs were collected from children who were suspected of having S pyogenes pharyngitis. Each swab was used for a culture and an OSOM Ultra Strep A Test rapid antigen test. The gold standard of comparison was defined as the identification of S pyogenes on either of the 2 culture plates. Three diagnostic strategies were evaluated: a single rapid antigen test, a rapid antigen test with follow-up rapid antigen test (rapid-rapid), and a rapid antigen test with follow-up culture (rapid-culture).

Results. A total of 210 (23.7%) of 887 throat cultures with matched data were identified with S pyogenes. A single rapid antigen test had a sensitivity of 87.6% (95% confidence interval [CI]: 83.2%–92.1%), the sensitivity of the rapid-rapid follow-up was 91.4% (95% CI: 87.6%–95.2%), and the sensitivity of the rapid-culture follow-up was 95.7% (95% CI: 93.0%–98.5%), which was significantly higher than the others. As shown in Fig 1, when these test strategies were evaluated on a subgroup with clinical symptoms commonly associated with S pyogenes pharyngitis, the sensitivities all increased and were no longer significantly different. None of the strategies reliably exceeded a 95% sensitivity threshold.

Conclusions. The American Academy of Pediatrics strategy for S pyogenes detection in children with pharyngitis, requiring a backup culture for those with negative antigen tests, was not exceeded by any other test strategy; however, a rapid-rapid diagnostic strategy may approximate it with the use of judicious clinical selection of patients.

  • group A Streptococcus
  • Streptococcus pyogenes
  • pharyngitis
  • rapid
  • antigen test
  • diagnosis
  • throat culture

The diagnosis of acute pharyngitis is made >10 million times annually in physicians’ offices.1 Although there are many causes of pharyngitis, Streptococcus pyogenes accounts for approximately 30% of cases with the remainder being mostly viral.2

The office laboratory diagnosis of pharyngitis is complicated by the imprecise signs and symptoms of S pyogenes pharyngitis, by the false-negative rates and turnaround times of various testing strategies, and by laboratory regulations of the Clinical Laboratory Improvement Act (CLIA). The American Academy of Family Physicians Pediatric URI Consensus Team states that the diagnosis of S pyogenes pharyngitis should be made on the basis of results of appropriate laboratory tests in conjunction with clinical and epidemiologic findings.3 The 2000 American Academy of Pediatrics (AAP) Red Book states that laboratory confirmation of S pyogenes is recommended for children with pharyngitis because reliable clinical differentiation of viral and S pyogenes pharyngitis is not possible.4 In addition, the Red Book notes that when a patient, suspected on clinical grounds of having S pyogenes pharyngitis, has a negative rapid streptococcal test, a follow-up throat culture should be obtained to ensure that the patient does not have S pyogenes pharyngitis. Cultures that are negative for S pyogenes after 24 hours should be incubated for a second day to optimize its recovery. When the rapid antigen test is negative and a culture is done as recommended, results may take an additional 24 to 48 hours, leaving clinicians to make treatment decisions without complete diagnostic results. Patients either receive antibiotics empirically, which may or may not be warranted, or must wait 1 to 2 days, during which they may have untreated symptoms and may not be allowed to attend child care or school. This diagnostic dilemma is complicated by the physician’s desire for accuracy in diagnostic testing. Although rigorous laboratory diagnosis of S pyogenes pharyngitis is recommended, difficulties engendered by compliance with the CLIA have decreased the number of offices that do any diagnostic testing57 and, specifically, have decreased the number of offices that do rapid antigen tests and cultures for S pyogenes.5,8

The objective of this study was to evaluate the sensitivity and specificity of 3 different diagnostic strategies: a single rapid antigen test, a rapid antigen test with a follow-up rapid antigen test if negative (rapid-rapid diagnostic strategy), and a rapid antigen test with follow-up culture if negative (rapid-culture)—the AAP diagnostic strategy—all compared with a 2-plate culture gold standard.9 In addition, because a recent convenience sample survey (unpublished data) of pediatricians showed that 67 (80%) of 84 were willing to miss no more than 5% of streptococcal infections, we also compared the ability of these strategies to achieve an absolute diagnostic test sensitivity of >95%.

METHODS

Study Design and Population

Throat cultures were collected from children who were suspected of having S pyogenes pharyngitis and presented at the Children’s Medical Center, a large pediatric office in downtown Denver, Colorado, from February 15 through April 2001. These cultures were collected on Culturette II Swabs (Beckton Dickinson Microbiology Systems, Cockeysville, MD) by vigorously rubbing 2 swabs together simultaneously over the posterior pharynx and tonsils. The swabs were then randomly labeled swab 1 and swab 2 (Fig 2). Swab 1 was used by the office staff to inoculate a Strep Selective (BBL) media plate (culture 1) and was then used to test for S pyogenes antigen using Genzyme’s OSOM Ultra Strep A Test (OSOM Ultra #1). Swab 2 was retained in the culturette tube with the transport media vial broken and batched at room temperature. Twice daily, the batched swab 2s and culture 1s were taken to the Children’s Hospital, where a microbiologist, blinded to the previous results, used swab 2 to inoculate a second Strep Selective (BBL) media plate (culture 2). The second sample was then tested for S pyogenes antigen by using the Genzyme’s OSOM Ultra Strep A Test (OSOM Ultra #2). A sample size calculation based on results of our previous studies suggested the need for at least 302 matched testing pairs.

Fig 1.

Sensitivity and 95% CI of throat culture diagnostic strategies. Asterisk indicates subgroup analysis of children with strong clinical criteria for S pyogenes pharyngitis.

Fig 2.

Study design for the comparison of S pyogenes pharyngitis diagnostic strategies.

Rapid Antigen Tests

The Genzyme OSOM Ultra Strep A Test was performed by specifically trained personnel in accordance with package insert instructions.

Diagnostic Standard

The 2-plate culture gold standard for the presence of S pyogenes was considered to be the isolation and confirmation of S pyogenes on either of the 2 culture plates.9

Cultures

Both culture plates were incubated at 35°C in 6%–8% CO2 and were read at 24 and 48 hours. Microbiologists who read the plates were blinded to the rapid test results. Beta-hemolytic streptococci were confirmed as S pyogenes by the BBL Streptocard Acid Latex Test.

Diagnostic Strategies

The single rapid antigen strategy consisted of the result of the OSOM Ultra #1 test. The rapid-rapid diagnostic strategy included the OSOM Ultra #1 and the OSOM Ultra #2 results, and the rapid-culture (AAP diagnostic strategy) combined the results of the OSOM Ultra #1 and culture 2.

Clinical Criteria

Data were collected prospectively by the examining physician on the signs and symptoms of each individual. Signs and symptoms included sore throat, cough, runny nose, known streptococcal contact, antibiotics within the previous 7 days, tonsillar exudates, lower respiratory infection, otitis media, scarlatina rash, and anterior cervical adenitis.

IRB Approval

This study was approved by the Colorado Multiple Institutional Review Board as an exempt study.

Statistics

Data were analyzed using SAS 8.1.10 Paired data were compared using the McNemar test for correlated proportions. Exact 95% confidence intervals (CIs) were calculated for proportions.

RESULTS

During the study period—February 15 through April 20, 2001—891 throat cultures were obtained from all children who were suspected of having S pyogenes pharyngitis and were seen at the Children’s Medical Center. Four cultures did not have the OSOM Ultra #2 test run; therefore, 887 of the 891 cultures had matched results for 2 rapid antigen tests. Of those 887, 210 tested positive for S pyogenes using a 2-plate culture gold standard and latex agglutination confirmation test. This represents an S pyogenes prevalence rate of 23.7%.

Table 1 shows the sensitivity and specificity of the single rapid antigen, rapid-rapid, and rapid-culture diagnostic strategies as compared with a 2-plate culture gold standard. The individual tests (OSOM Ultra #1 and #2) had sensitivities of 87.6% and 86.2%, respectively. The sensitivity of the rapid-rapid diagnostic strategy was 91.4%, whereas the sensitivity of the rapid-culture AAP diagnostic strategy was 95.7%. The rapid-culture diagnostic strategy had a significantly higher sensitivity than the rapid-rapid diagnostic strategy (P = .01, McNemar test for correlated proportions), as compared with the 2-plate culture gold standard.

TABLE 1.

Diagnostic Strategies Versus a 2-Plate Culture Gold Standard

Table 2 shows the clinical symptoms that were significantly associated with S pyogenes pharyngitis. These included anterior cervical adenitis (P = .0006), scarlatina rash (P = .0001), and presence of tonsillar exudates (P = .0001).

TABLE 2.

Association of Clinical Findings With S pyogenes Isolation for Children With Suspected S pyogenes Pharyngitis

The sensitivity and specificity of each diagnostic strategy was evaluated on a subgroup of patients who had strong clinical criteria (anterior cervical adenitis or scarlatina rash or tonsillar exudates) and did not have cough or runny nose. Table 3 shows the sensitivity and specificity of the various diagnostic strategies for this subgroup. The single rapid antigen test had a sensitivity of 90.5%, the rapid-rapid diagnostic strategy had a sensitivity of 93.7%, and the rapid-culture diagnostic strategy had the highest sensitivity at 96.8%. Comparison of the rapid-rapid and rapid-culture diagnostic strategies in this subgroup showed no significant difference (P = .16).

TABLE 3.

Subgroup* of Clinically Significant Individuals: Single Rapid, Rapid-Rapid, and Rapid-Culture Strategies Compared With a 2-Plate Culture Gold Standard

Figure 3 compares each of the diagnostic strategies to a sensitivity threshold of 95%, which was the minimum sensitivity considered clinically acceptable by the majority of pediatricians in the convenience sample survey. To meet this threshold and to say with certainty that the strategy has a 95% sensitivity, the lower limit of the CI should be 95% or greater. None of the diagnostic strategies meets this threshold.

Fig 3.

Sensitivity and 95% CI of all diagnostic tests and strategies. Asterisk indicates subgroup analysis of children with strong clinical criteria for S pyogenes pharyngitis.

DISCUSSION

The accurate and timely diagnosis of S pyogenes pharyngitis is a reasonable clinical goal, yet many factors affect a clinician’s ability to achieve it. Guidelines from the AAP and American Academy of Family Physicians recommend that diagnosis only follow laboratory confirmation of S pyogenes and that for any rapid antigen test that is negative, a follow-up culture should be done.3,4 CLIA regulations have had a significant impact on the percentages of offices that do any testing and have added several requirements for those offices that continue to do in-office testing such that waived tests are preferred.58

It has been suggested that a single rapid antigen test without follow-up culture may be sufficient. Gerber et al11 showed that a single optical immunoassay (OIA) had a sensitivity of 84% when compared with a 2-plate criterion standard, which consisted of a single blood agar plate and a Todd Hewitt broth-enhanced culture. In this same study, the sensitivity of the single-plate culture as compared with the criterion standard was 78%. They concluded that because the sensitivity of the OIA was greater than the single blood agar plate and because a single culture is considered sufficient for use alone as diagnosis for S pyogenes pharyngitis in an office setting, the OIA alone should be sufficient.11 This would mean that 22% of pharyngitis patients with streptococcal pharyngitis would have been missed, a much higher false-negative rate than the 5% maximum considered acceptable in our convenience sample survey of pediatricians.

In a previous study,9 we documented the need for cultures from at least 2 samples to approximate a true gold standard for S pyogenes diagnosis. In that study, which was a head-to-head comparison, the waived OSOM Ultra Strep A Test was significantly more sensitive than the nonwaived Strep A OIA Max Test. In the current study, the sensitivity of a single OSOM Ultra Strep A Test was shown to be 87% with specificities >96%. This study also showed that even in a population with clinical symptoms that are significantly associated with S pyogenes pharyngitis, the sensitivity of a single rapid antigen test is only 90.5%. Although the culture standard that we used is rigorous and likely to decrease the apparent sensitivity of any comparative test, it does approximate the true prevalence of S pyogenes in our pharyngitis population.9 In addition, there is good evidence that a significant percentage of patients with negative antigen tests and positive cultures or cultures with low numbers of S pyogenes still may have active infection.12 If the clinical expectation for these diagnostic tests is a sensitivity >95% compared with such a rigorous standard, then the result of a single rapid antigen test is not yet sensitive enough to stand alone.

One of the concerns noted by clinicians in doing follow-up cultures when the initial rapid antigen test is negative is determining treatment at the time of initial visit. One treatment plan, which follows AAP guidelines, does not treat the patient with a negative rapid antigen test at the time of visit but waits until backup culture results are available, thus leaving the patient untreated for 1 to 2 additional days. Another approach is to treat all patients empirically at the time of visit and then to either discontinue or continue treatment once culture results are available, 1 or 2 days later.1315 In either case, once backup culture results are available, they must be relayed to the patient so that appropriate treatment adjustments can be made. Some patient populations such as those seen at emergency departments or urgent care centers may not be reachable after they leave the point of care and create a management challenge. Having a correct diagnosis at the time of visit could have a significant impact on this potential overuse of antimicrobial therapy for pharyngitis, although the alternative of waiting for a culture result to determine the need for treatment would not be likely to alter the generally low rates of rheumatic fever in the United States.

The overall sensitivity and specificity of the AAP diagnostic strategy for the diagnosis of S pyogenes pharyngitis, consisting of an S pyogenes rapid antigen test with a backup culture if the rapid antigen test was negative, was shown in this study to be 95.7% (95% CI: 93.0%–98.5%) and 96.2% (95% CI: 94.7%–97.6%), respectively. Although the rapid-culture strategy sensitivity is significantly higher than the rapid-rapid strategy sensitivity, the magnitude of the difference may not outweigh the added patent care complexity of a 1- to 2-day delay in final results. In addition, the rapid-culture sensitivity may be falsely elevated because of the bias of the second culture, which was a part of the 2-plate culture gold standard as well as serving as the follow-up culture in the AAP diagnostic strategy. The rapid-rapid antigen diagnostic strategy had a sensitivity and specificity of 91.4% (95% CI: 87.6%–95.2%) and 95.0% (95% CI: 93.3%–96.6%), respectively, and although overall statistically less than the AAP diagnostic strategy, results could be obtained at the time of the initial visit.

When a high-risk clinical subgroup was evaluated, the sensitivities of all of the diagnostic strategies increased. In addition, the 95% CIs widened, which is directly related to the smaller sample size of this subgroup. In this group, there was no significant difference in the sensitivity of the rapid-rapid strategy and the rapid-culture strategy. Thus, the rapid-rapid strategy has logistic advantages over the current AAP diagnostic strategy recommendations and similar sensitivities in children who are most likely to have S pyogenes pharyngitis.

None of the proposed diagnostic strategies met the 95% sensitivity threshold. Even the AAP diagnostic strategy falls below the 95% threshold with the lower confidence limit of 93.0%. Either clinicians have unrealistic expectations of current diagnostic strategies, or the current test strategies are not sufficient to meet clinicians’ expectations.

It is interesting that another diagnostic strategy has been proposed in the literature. A study by Kurtz et al16 described the importance of inoculum size in rapid antigen testing of S pyogenes pharyngitis. The use of a combined, 2-swab, extraction method significantly increased the sensitivity of a single rapid antigen test (P < .05). It is possible that current antigen tests might reach the 95% threshold and/or prove to be equal to the AAP diagnostic strategy if a larger sample were obtained for antigen extraction. Clearly, a strategy that provides accuracy in diagnosis at the time of visit, preferably with a single test, is of importance and is a challenge that invites additional research on the diagnosis of S pyogenes pharyngitis.

Acknowledgments

The physicians and staff of the Children’s Medical Center provided essential assistance and expertise.

This study was funded by Genzyme, the manufacturer of OSOM Ultra Strep A Test.

Footnotes

    • Received October 30, 2002.
    • Accepted February 19, 2003.
  • Reprint requests to (J.K.T.) Department of Epidemiology, Children’s Hospital, 1056 E. 19th Ave, Denver, CO 80218. E-mail: todd.james{at}tchden.org
  • The design and conduct of the study and contents of this manuscript were determined solely by the investigators.

CLIA, Clinical Laboratory Improvement Act, AAP, American Academy of Pediatrics, CI, confidence interval, OIA, optical immunoassay

REFERENCES