ARTICLE |
a Preventive Medicine
b Pediatrics
f Medicine, Vanderbilt University School of Medicine, Nashville, Tennessee
c Department of Pediatrics, University of Rochester School of Medicine and Dentistry, Rochester, New York
d Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, Ohio
e National Immunization Program, Centers for Disease Control and Prevention, Atlanta, Georgia
| ABSTRACT |
|---|
|
|
|---|
OBJECTIVE. We sought to assess the times at which rapid tests are most predictive of influenza infection.
METHODS. The New Vaccine Surveillance Network enrolled children aged <5 years who were hospitalized with respiratory symptoms or fever from October 2000 through September 2004. Nasal and throat swabs were obtained, and influenza virus was detected by culture and reverse-transcription polymerase chain reaction. Provider-ordered rapid influenza tests were compared with the criterion standard (culture and reverse-transcription polymerase chain reaction) to determine their sensitivity and specificity. The New Vaccine Surveillance Network also enrolled children in outpatient settings during the 20022003 and 20032004 influenza seasons and determined the weekly influenza prevalence among symptomatic children. Trends in weekly predictive values of the rapid tests were estimated over the influenza seasons.
RESULTS. Rapid influenza tests had an overall sensitivity of 63% and specificity of 97%. In 20022003, the prevalence of influenza in symptomatic outpatient children peaked at 21% and stayed above 10% for
4 weeks. In contrast, in 20032004, influenza prevalence peaked at 60% and remained above 20% for
6 weeks. The positive predictive value of the rapid tests approached 80% when influenza prevalence was
15% but decreased to <70% when influenza prevalence was <10%.
CONCLUSIONS. Influenza prevalence varies between and within seasons. On the basis of our estimates, rapid tests are of limited use when prevalence is <10%. The appropriate interpretation of rapid influenza tests requires local influenza surveillance and timely communication of this information to the practitioners.
Key Words: human influenza predictive value rapid diagnostic tests
Abbreviations: NVSNNew Vaccine Surveillance Network RT-PCRreverse-transcription polymerase chain reaction CIconfidence interval PPVpositive predictive value NPVnegative predictive value LRlikelihood ratio
Every year influenza virus infections result in excess hospitalizations, outpatient and emergency department visits, and increased antibiotic use in the United States.1 Compared with adults, young children have higher attack rates and more prolonged viral shedding resulting in the spread of influenza in the community.2,3 Intensive population-based influenza surveillance performed by the New Vaccine Surveillance Network (NVSN) in 3 US counties over 4 consecutive years found average annual influenza hospitalization rates to be 4.5, 0.9, and 0.3 per 1000 children 0 to 5 months, 6 to 23 months, and 24 to 59 months of age, respectively. In the outpatient setting, influenza-attributable visits to clinics or emergency departments were 56 and 122 per 1000 children <5 years during the 20022003 and 20032004 influenza seasons, respectively.4
Because the clinical manifestations of influenza overlap with those attributable to other common respiratory illnesses of childhood, establishing a diagnosis of influenza requires confirmatory testing.59 The positive aspects of a timely diagnosis of influenza include the opportunity to provide antiviral therapy, allow implementation of measures to limit virus transmission, avoid unnecessary antibacterial prescriptions and testing in the emergency department, reduce length of stay in the hospital, and provide a timely opportunity to educate patients and families about influenza vaccine.817 Despite the potential benefits of accurately identifying influenza-related illness, practice guidelines regarding the optimal use of influenza diagnostic tests are limited. Recent evidence indicates that fewer than half of hospitalized children with fever and respiratory symptoms during influenza seasons have a clinical diagnostic test for influenza performed.4,18
Viral culture has been considered the gold standard for influenza diagnosis, but the delay in obtaining results makes it impractical for clinical decision-making.9 Reverse-transcription polymerase chain reaction (RT-PCR) is more sensitive than standard viral culture in the detection of influenza. The ratio of influenza-positive results by RT-PCR to those by viral culture has been reported to be 1.6.19 However, RT-PCR is not widely available for clinical use and is expensive.19,20 As an alternative, rapid influenza antigen detection tests (rapid tests) are relatively inexpensive and can provide timely information for clinical decisions.8,21
The correct interpretation of rapid-test results depends not only on the clinical presentation of the disease but also on the test characteristics and the prevalence of influenza in the community. Defining a time at which rapid influenza tests are more likely to correctly predict influenza infections is important to improve diagnostic accuracy and to reduce disease misclassification and its potential consequences. We estimated rapid influenza test validity indices and calculated predictive values for these tests during consecutive influenza seasons to identify times at which the tests were more likely to predict influenza infections.
| METHODS |
|---|
|
|
|---|
In addition to inpatient surveillance, NVSN outpatient surveillance was implemented in Davidson County and Monroe County in 2002 and Hamilton County in 2003. In each site, surveillance activities during the respiratory seasons involved
1 pediatric emergency department and 1 to 4 outpatient practices. Detailed descriptions of NVSN methodology and results have been published elsewhere.4,19,22,23 Influenza seasons represented a subset of the respiratory seasons and were defined as those county-specific 13 consecutive weeks that contained 95% of all influenza cases in 20022003 and 99% in 20032004, respectively.4
Several types of rapid influenza tests were available during the study period. Although their reported sensitivities and specificities were relatively similar, influenza rapid tests were, in general, reported to be more specific than sensitive.21,2428 In Davidson and Monroe Counties, the most commonly used rapid influenza test during the study period was Directigen A+B (Becton Dickinson, Sparks, MD). Starting in 20032004, Quick Vue A/B (Quidel, San Diego, CA) also became available in the Davidson County surveillance site. In Hamilton County, the most commonly used rapid test was Directigen A (Becton Dickinson) through 2002; and starting in 2003, the most commonly used test was NOW Flu A/B (Binax, Portland, ME). In this study, we termed any of these types of rapid influenza test as clinical rapid tests. This study was approved by the institutional review boards at Vanderbilt University Medical Center, Cincinnati Children's Hospital Medical Center, the University of Rochester, and the Centers for Disease Control and Prevention.
Statistical Analysis
Using data from 4 years of inpatient surveillance, clinical rapid-test results were compared with viral culture and RT-PCR results, and the sensitivity, specificity, and their respective binomial 95% confidence intervals (CIs) were calculated.
Weekly prevalence of influenza in the outpatient clinical setting during individual influenza seasons was estimated by dividing the number of enrolled children with influenza (by viral culture or RT-PCR) by the total number of enrolled children (all with fever or respiratory symptoms). Using the sensitivity and specificity for clinical rapid tests calculated from the inpatient surveillance data and the prevalence estimated from the outpatient surveillance data, predictive values for influenza rapid-test results were estimated and plotted for each week of the influenza seasons.
The positive predictive value (PPV) of the clinical rapid test equaled the proportion of subjects with confirmed influenza among children who had a positive rapid test, whereas the negative predictive value (NPV) equaled the proportion of subjects without influenza among children who had negative rapid tests.29,30 All of the calculations were performed by using Stata 8.2 (Stata Corp, College Station, TX).
| RESULTS |
|---|
|
|
|---|
Of the 41 children (15%) with influenza detected by the criterion standard, viral culture, or RT-PCR, 26 were influenza positive by a clinical rapid test (sensitivity: 63%; 95% CI: 47%78%). Among 229 children who tested negative for influenza by the criterion standard, 223 had a negative clinical rapid-test result (specificity: 97%; 95% CI: 94%99%). Rapid tests should be performed in samples collected within 4 days of disease onset31,32; and, when calculations were stratified by reported disease duration, sensitivity and specificity of rapid influenza tests among hospitalized children with disease duration
4 days were 64% (95% CI: 44%81%) and 98% (95% CI: 93%99%), respectively. Rapid tests performed in children with longer disease duration at the time of admission had an estimated sensitivity and specificity of 62% (95% CI: 32%86%) and 97% (95% CI: 92%99%), respectively.
Weekly Prevalence of Influenza Among Children With Respiratory Symptoms or Fever During 2 Influenza Seasons
During the mild 20022003 influenza season, weekly prevalence of influenza virus infection in the 767 children tested in the outpatient setting ranged from 0% to 21% (Fig 1). In contrast, the 20032004 influenza season was characterized by an early onset with moderately severe activity.33 During this season, the weekly prevalence of influenza among 975 children enrolled was >30% for 5 consecutive weeks and
60% at the peak of the season.
|
At the beginning of the 20022003 season, when the prevalence of influenza in children with acute respiratory symptoms or fever was 5%, the predictive value of a positive rapid test was
50%. In other words, with a 5% influenza prevalence, a positive result was equally likely to represent a true influenza infection or a false-positive result. On the other hand, a negative rapid-test result represented a true-negative
98% of the time. As the season progressed, the prevalence of influenza among children with compatible symptoms peaked at 21%. At this relatively high prevalence, the PPV was 85%. Conversely,
9% of true influenza cases had a negative rapid-test result (false-negatives). After the peak in activity, the prevalence of influenza stayed above 10% for 3 more weeks and then declined gradually. This resulted in a reduction in the PPV and a rise in the NPV. For example, when influenza prevalence was 1.6%, only 25% of those children with a positive rapid-test result had confirmed influenza infections, but 99% of those children with a negative rapid-test result did not have influenza. Throughout the mild 20022003 season, the PPV of the clinical rapid tests was
70% for only 4 weeks.
In the 20032004 season, when the prevalence of influenza among children with compatible symptoms in the outpatient surveillance sites peaked at 60%, nearly 97% of positive rapid tests were true-positives. In contrast,
37% of children with a negative rapid-test result were false-negative. During this season, the prevalence of influenza was high throughout the season, and the PPV of the clinical rapid tests was
80% during the 7 weeks when influenza prevalence was
15%.
| DISCUSSION |
|---|
|
|
|---|
We calculated the overall sensitivity and specificity of rapid influenza tests that were conducted on inpatients and compared them with RT-PCR and viral culture using information from an active prospective surveillance system. These test results were not available until after discharge and, thus, the treating physicians did not have this information at the time of ordering or interpreting the rapid tests.
Diagnostic tests rarely provide a perfect distinction between those with and without disease, but by applying careful interpretation of the results, the likelihood of diagnostic errors can be diminished. Before ordering an influenza rapid test and in anticipation of providing appropriate test interpretation, it is important to have an estimate of influenza prevalence in the community. The knowledge of disease prevalence will provide an estimate of the baseline probability of infection, and the result of the rapid test will modify this probability on the basis of the known test characteristics. Instead of taking a positive or negative test result for granted, the consideration of predictive values as conditional probabilities could prevent inappropriate interpretations of test results. In practice, the PPVs and NPVs provide the most useful clinical information. This necessary integration of the laboratory test characteristics (sensitivity and specificity) with knowledge of the prevalence of the disease for which testing is being performed is a concrete "real-world" application of Bayes theorem, put forth by the Reverend Thomas Bayes in the 18th century.35 Unfortunately, the prevalence of influenza among children presenting with fever or respiratory symptoms is usually not known at the time of testing; therefore, it is often difficult to derive appropriate interpretations of rapid-test results.
Although sensitivity and specificity are considered fixed values, in routine practice, symptomatic children attend clinic at different stages of their disease, potentially affecting the estimates of test characteristics. Most rapid influenza tests detect viral antigens present in the samples.8 Because viral shedding is higher in young children and during the first days of disease,2,3 infected young children having a test performed during the first days of disease will be more likely to yield a positive result compared with infected older children presenting after the initial days of disease. This would increase the estimated test sensitivity. Accordingly, samples should be collected within 4 days of disease onset or
5 days for young children.31,32 We assessed the potential effect of disease duration in our calculations of test characteristics. Our estimates were similar when they were stratified by disease duration, but small numbers precluded further stratification. In addition, determination of test characteristics requires careful sample collection and handling.32 In the present article, we applied test characteristic estimates obtained from the inpatient setting to outpatient children. If children treated in the ambulatory settings were systematically younger and/or presented at an early stage of their disease, then our overall test sensitivity estimates would be conservative. Information on rapid influenza tests performed in the ambulatory settings was not available for all of the children, and we applied the overall estimates derived from the inpatient settings for illustration purposes.
Test characteristics were estimated overall and included different types of rapid influenza antigen detection tests. Similar estimates can be obtained for specific rapid influenza tests using clinical data from reported studies or the information provided by the manufacturer. Because RT-PCR enhanced the detection of influenza cases by the NVSN, reported sensitivities for rapid influenza tests based on viral cultures alone (without RT-PCR) are expected to be higher than our overall sensitivity estimates. Nonetheless, our estimates of sensitivity and specificity for rapid influenza tests were within the range of previously reported values, in which sensitivities ranged from 44% to 95% and specificities from 76% to 100%.21,2428,31
An alternative way to apply test results to clinical practice is through the calculation of likelihood ratios (LR). LRs represent the magnitude by which the probability of a diagnosis in a patient is modified by the result of a diagnostic test.36 Although LR calculations for a test do not require information on disease prevalence, its application to clinical practice does. As our study shows, the knowledge of influenza circulation in the community is fundamental for the interpretation of rapid influenza test results. Many urban areas have large academic medical centers and public health laboratories that perform routine influenza surveillance. In addition, the Centers for Disease Control and Prevention provides information on regional influenza circulation on their Web site (www.cdc.gov/flu/weekly/fluactivity.htm). The Pandemic Influenza Plan provides specific recommendations for surveillance,37 and its implementation represents an opportunity to strengthen current systems and to enhance the timely dissemination of information generated by them.
During the peak of influenza epidemics, a positive result is highly likely to represent true influenza illness. Furthermore, a negative test suggests a low likelihood of influenza. In this scenario, most children presenting with respiratory symptoms and fever with a positive test would have influenza.8 On the other hand, when the prevalence is low, positive rapid-test results are more likely to be false-positive.29,3840 When influenza prevalence is
5%, a positive rapid-test result will be correct only 50% of the time, providing no useful information for clinical decision-making. Additional confirmatory testing would be necessary for these children. In contrast, when influenza prevalence is
10%, a positive rapid-test result will predict influenza infection correctly
70% of the time.
The appropriate interpretation of rapid influenza tests can provide useful information for decision-making for both clinical practitioners and public health officials. Appropriate communication of information about local influenza circulation to practitioners could help optimize the interpretation of rapid influenza tests.
| ACKNOWLEDGMENTS |
|---|
| FOOTNOTES |
|---|
Address correspondence to Marie R. Griffin, MD, MPH, A-1110 Medical Center North, Preventive Medicine Department, Vanderbilt University Medical Center, Nashville, TN 37232-2637. E-mail: marie.griffin{at}vanderbilt.edu
The authors have indicated they have no financial relationships relevant to this article to disclose.
The findings and conclusions in this article are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.
| REFERENCES |
|---|
|
|
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||