Predictive Validity of a 2-Question Alcohol Screen at 1-, 2-, and 3-Year Follow-up

Discussion

Both medical and federal organizations have recommended screening and behavioral counseling interventions to reduce alcohol misuse and have developed resources to support the implementation of these services. In a recent study of US hospitals that treat injured adolescents, only 18% reported providing universal alcohol screening for their adolescent emergency department patients.²⁰ This same study revealed that 31% of the US hospitals that were surveyed did not use a standardized screening instrument but rather incorporated alcohol questions into their general hospital assessments. A brief screen, such as the NIAAA 2-question screen, might significantly contribute to a more widespread adoption of screening.

This multisite assessment study of the predictive validity of the NIAAA 2-question screen revealed that the screen had acceptable sensitivity and specificity with respect to a DSM-5 diagnosis of AUD 1, 2, and 3 years after its original administration. The unique contribution of this study was the 3-year follow-up period and our ability to examine long-term prediction in this sample. The best combined sensitivity and specificity scores were achieved by using “lower risk” and higher as a cutoff for the prediction of a DSM-5 diagnosis at 1 year (sensitivity = 86.2%; specificity = 78.1%), 2 years (sensitivity = 75.6%; specificity = 79.2%), and 3 years (sensitivity = 60.0%; specificity = 80.0%). The areas under the curve of the DSM-5 diagnosis ROC at 1- and 2-year follow-up compared favorably to those we found for an AUD diagnosis cross-sectionally¹⁸; the ROC findings were weaker by 3 years. Although prediction at 1 year might be of most interest to the primary care physician because screening is typically repeated at annual visits, the fact that the NIAAA 2-question screener has comparable relevance for prediction over 2 years adds to our understanding of the use of alcohol-use screening in the PED.

Adolescents who initially screened as being at highest risk on the NIAAA 2-question screen were shown to be more likely to have a higher number of drinking days and to be at higher risk for AUD at 1, 2, and 3 years. However, it is noteworthy that the predictive ability of the 2-question screen does not differentiate between those who initially screen as being at lower risk and those who screen as being at moderate risk. It appears that at lower numbers of drinking days, the screen is less effective in estimating future risk level. This may be related to the fact that the risk assessment algorithm for the NIAAA 2-question screen¹ is based on age, and for 12- to 15-year-old adolescents, any drinking is categorized as at least moderate risk (ie, there is no possibility of being categorized as lower risk in this age group). Because a significant portion of our sample fell into this younger age group, this reduced the likelihood of discerning differences between the lower- and moderate-risk groups even if they existed.

Research has revealed that over the course of adolescence, the proportion of individuals who report past-year alcohol use increases dramatically, from 17.6% of eighth-graders to 56% of 12th-graders nationwide.²¹ In our study, at 3-year follow-up, many participants in the lower-risk baseline group would have been at least 18 years of age and might “normally” be expected to be engaged in riskier drinking. This also likely explains the finding that the number of drinking days increased from 1- to 3-year follow-up for all of the baseline risk groups because this appears to represent the “normal” age-based trajectory (ie, prevalence increases). Because we followed the same participants from the time of enrollment through the 3-year follow-up contact, it would be expected that drinking rates would have increased over that time, and hence these data also contribute to our understanding of the natural history of adolescent alcohol use.

Our findings suggest that the NIAAA 2-question screen may offer a viable first step for screening adolescents who may be at risk for the development of an AUD. In this study (and typically in clinical practice) most adolescents who are screened in this manner will screen with negative results, and, at least in the PED setting, further action would be unnecessary. By using our results as a starting point, an assessment of the consequences of screening can be estimated. For example, using the optimal cutoff from the ROC curve, we would expect about 20 false alarms (patients who have positive results on the screen but do not go on to develop an AUD within 3 years) out of every 100 patients who were screened. Stated another way, because 23% of our participants screened with positive results on the NIAAA screen and 11.6% of those participants exhibited an AUD at the 3-year follow-up, it would be expected that 38 adolescents would need to be screened to detect 1 participant with positive results. Although on first glance this appears to be a relatively modest return, the brevity of the screening process and the potential impact may argue in favor of screening.

Our findings differ somewhat from those of a recent cross-sectional study²² of adolescents who were screened in primary care clinics, which found age differences in the cutoff score for risk of an AUD. The differential findings might be related to the different settings and/or the cross-sectional versus longitudinal nature of the 2 studies. Nonetheless, the ideal approach for responding to positive results on a screen (lower-risk category or higher) remains to be determined and is likely to vary from 1 setting or 1 PED to another, depending on a number of factors, including the availability of resources. In some resource-intensive settings, a positive response might initiate a more in-depth investigation into the adolescent’s alcohol use either through direct questioning or a more thorough self-report assessment. Although this approach may vary by setting, the ultimate purpose would be to aid decisions about further evaluation and referral to treatment. In fact, researchers in a number of studies have evaluated brief interventions that could potentially be administered in the PED setting.^23–30 Additional studies will be required to determine the most effective means for screening, the best process for delivering a brief intervention in the PED when called for, and the circumstances in which referral to further treatment is appropriate.

There are a number of limitations to this study. First, although the sample was large and diverse, PECARN PEDs are generally based in urban medical centers and may not be representative of US PEDs. Similarly, it is conceivable that those who refused participation in the study differed in some systematic way from those who agreed to participate. Follow-up rates were in the 70% range, thereby allowing for the possibility that those who did not respond to follow-up may systematically differ from the group reported here. Also, measures in the current study were self-administered, and thus we do not know whether the screen would perform differently if administered directly by a health care provider. Importantly, however, self-administered screening for sensitive questions has been shown to be more accurate than face-to-face screening^31,32 and should be considered in clinical practice. In addition, because the study required parental permission, adolescent reports of substance use may have been affected despite our reassurances that responses were confidential. Finally (as noted above), the 2-question screen is used to risk stratify the number of drinks in the past year differently for younger adolescents than for older adolescents. This presents some challenges when adolescents are followed longitudinally.