OBJECTIVE: The goal of this study was to compare the performance of 4 operational definitions of sexual activity by using data electronically abstracted from electronic health records (EHRs) and examine how documentation of Chlamydia screening and positivity vary according to definition of sexual activity.
METHODS: Extracts were created from EHRs of adolescent females 12 to 19 years old who had ≥1 visit to a primary care practice during 2011 at 4 US pediatric health care organizations. We created 4 definitions of sexual activity derived from electronically abstracted indicator variables. Percent sexually active, documentation of Chlamydia screening, and rate of positive Chlamydia test results per 1000 adolescent females according to the sexual activity definition were calculated.
RESULTS: The most commonly documented individual indicator of sexual activity was “patient report of being sexually active” (mean across 4 sites: 19.2%). The percentage of adolescent females classified as sexually active varied by site and increased as more indicator variables were included. As the definition of sexual activity expanded, the percentage of sexually active females who received at least 1 Chlamydia test decreased. Using a broader definition of sexual activity resulted in improved identification of adolescent females with Chlamydia infection. For each sexual activity definition and performance item, the difference was statistically significant (P < .0001).
CONCLUSIONS: Information about sexual activity may be gathered from a variety of data sources, and changing the configurations of these indicators results in differences in the percentage of adolescent females classified as sexually active, screened for Chlamydia infection, and Chlamydia infection rates.
- EHR —
- electronic health record
- HEDIS —
- Healthcare Effectiveness Data and Information Set
- STI —
- sexually transmitted infection
What’s Known on This Subject:
The use of electronic health record systems to measure adolescent health care quality requires an operational definition of sexual activity for measuring recommended health promotion activities such as Chlamydia screening and others related to reproductive health.
What This Study Adds:
This study is the first to compare operational definitions of sexual activity by using information electronically abstracted from electronic health records of adolescent females. Our research supports the use of broader operational definitions of sexual activity for health quality measurement.
Performance measurement plays an increasingly important role in health care quality improvement efforts. Traditionally, performance measures have been calculated from administrative claims data, medical record review, or a combination of the 2, with varying results.1–4 The introduction of electronic health record (EHR) systems to manage clinical information has spurred a national movement toward using these systems for performance measurement.5–13 Similarly, federal government efforts are encouraging quality measurement and use of health information technology in the area of pediatric and adolescent health care.14,15 For example, the Pediatric Quality Measures Program assesses responsiveness to health information technology as a criterion for quality measures development.16–18
High-quality health services for adolescents include assessment of sexual activity status and risk factors for adverse outcomes.19,20 These services include: assessing risk for sexually transmitted infections (STIs) and risk reduction counseling, counseling on correct and consistent use of condoms, counseling on abstinence, Chlamydia screening in adolescent females Chlamydia screening in males at increased risk, HIV testing, high-intensity behavioral counseling to prevent STIs, and contraception counseling.21–27 However, developing measures to assess the quality of these services is constrained because there is no agreed upon definition of sexual activity within the medical community (ie, which types of sexual encounters qualify, how recent does a sexual encounter need to be in relation to a medical visit). Moreover, there are no professional standards regarding documentation of sexual activity, EHR products vary in how sexual activity information is captured, and clinician documentation practices may not match those intended by the developers of EHRs. Progress in the evaluation of adolescent health care quality necessarily includes being able to accurately measure sexual activity status via EHRs because sexual activity status defines the population of youth who are eligible for specific services.
Chlamydia trachomatis is the most common bacterial STI in the United States and disproportionately affects adolescent and young adult women.28 The sequelae of Chlamydia infection include pelvic inflammatory disease, infertility, chronic pelvic pain, and ectopic pregnancy; these sequelae are costly.29 Because most Chlamydia infections are silent, screening for sexually active adolescent and young adult females should occur at least annually.23,24
Current administrative-based measures that rely on sexual activity status to identify the denominator population use proxy information to infer sexual activity. Specifically, the Chlamydia Screening in Women measure in the National Committee for Quality Assurance’s Healthcare Effectiveness Data and Information Set (HEDIS) uses administrative data sources to identify sexually active females.30 First, the measure uses codes from Current Procedural Terminology and International Classification of Diseases, as well as other sources, that indicate sexual activity or receipt of services indicative of sexual activity (eg, pregnancy services). Second, the measure uses pharmacy data to identify receipt of contraceptives. Identification through either of these methods will classify a female as eligible for the Chlamydia Screening in Women measure. Use of claims and pharmacy data to populate the HEDIS Chlamydia screening measure may underestimate the population of sexually active young women.31,32 In addition, a potential limitation to this claims-based approach is the misclassification of females for the denominator (ie, that nonsexually active women using hormonal contraceptives for gynecologic indications are identified as sexually active according to the HEDIS measure, potentially resulting in lower Chlamydia screening rates).
Using EHR data to populate performance measures is attractive due to ease of use and their potential to provide richer information than administrative claims data.3,5,33 For example, claims data cannot supply patient-reported information of care provided through other funding programs and reported to clinicians (eg, screening for STIs at a health fair or public health department). Moreover, claims data cannot provide clinical context such as health risk behaviors. The use of structured and coded data elements (ie, data that are easily retrieved and transferred) may provide this supplemental information through the logging of clinician–patient discussion topics, problems lists, and medication lists. Recent studies have shown some promise toward this end.4,33,34 Gold et al4 demonstrated that using EHR-derived social history information improved the accuracy of a Chlamydia screening measure compared with claims-based information. The study, however, did not assess the validity of the definition of sexual activity used in the denominator of their measure, which has prompted some criticism about the application of their approach to health care quality measurement.35
The present study adds to the literature on this topic by comparing the performance of 4 operational definitions of sexual activity by using data electronically abstracted from 2 EHRs at 4 US pediatric health centers and examines how documentation of Chlamydia screening and positivity vary according to sexual activity definition. We hypothesized that the percentage of adolescent females identified as sexually active, documentation of Chlamydia screening, and rates of Chlamydia infection would vary according to sexual activity definition.
This study examined the impact of varying an EHR-derived definition of sexual activity on the percentage of adolescent females identified as sexually active, documentation of Chlamydia screening, and rates of Chlamydia infection. Data were gathered during the National Collaborative for Innovation in Quality Measurement’s field testing of EHR-derived adolescent well-care measures.17 The study did not meet criteria for human subject research according to Nationwide Children’s Hospital’s institutional review board. Data collection occurred from April to July 2012.
Investigators recruited 6 pediatric health centers (hereafter referred to as sites) throughout the United States to participate in field testing. Data from 2 sites were excluded because we were unable to reliably abstract EHR information on Chlamydia screening or Chlamydia test results.
Female adolescents were eligible for inclusion in the sample if they were between the ages of 12 and 19 years by December 31, 2010, and had at least 1 primary care visit at the site in 2011.
Sites were instructed to electronically abstract the following indicator variables from the EHR for eligible participants: (1) clinician documentation of current and past sexual activity status reported by the patient; (2) clinician-diagnosed STIs, including bacterial vaginosis, Chlamydia, gonorrhea, hepatitis, herpes, human papillomavirus, pelvic inflammatory disease, syphilis, trichomoniasis, chancroid, Lymphogranuloma venereum, pubic lice infestation, scabies, and HIV/AIDS (only if sexually transmitted); (3) pregnancies (all pregnancy-related codes were included, including termination and miscarriage); (4) number of sexual partners reported by patient; (5) orders for hormonal contraceptives, with medical indication if available; (6) reported use of nonhormonal contraceptives such as a condom, diaphragm, intrauterine device, surgical option, spermicide, rhythm method, and sponge; (7) documentation of Chlamydia screening; (8) documentation of Chlamydia screening results; (9) race/ethnicity; and (10) insurance status. Sites were also instructed to electronically abstract documentation of clinician-diagnosed STIs and pregnancies before the measurement period (Supplemental Appendix). Site personnel submitted data extracts as .csv files to the National Collaborative for Innovation in Quality Measurement’s team via a secure, Web-based portal. The measurement period was the 18-month period before and including the 2011 primary care visit. If an adolescent completed >1 visit in 2011, the last visit was the index visit.
Operationalization of Sexual Activity Definitions
Four definitions were created to classify adolescent females in the sample as sexually active by using EHR-derived indicator variables. Table 1 reports the EHR-derived indicator variables associated with each definition. Sexual activity was examined in the following ways.
Current Status Only
Adolescents were classified as sexually active if there was clinician documentation of the adolescent’s self-report of currently engaging in sexual activity. This approach represented the most simple and straightforward means of obtaining sexual activity status. The indicator “patient report of being currently sexually active” was determined differently across the sites.
At sites 1 and 3, this was any response of Yes in the structured field for sexual activity status documentation in the Substance and Sexuality History tab of the Social History Activity section. Response options for the sexual activity structured field included No, Not Currently, Yes, and Not Asked.
At site 2, sexual activity was documented as a structured field in the Sexual History tab of the Social History page. Response options were Yes and No. All responses of Yes were coded as current sexual activity.
At site 4, templated portions of the Progress Notes section were used to document sexual activity. Progress notes were scanned for the term “present partners #,” after preliminary analysis that “present partners 0” was not being used.
Adolescents were classified as sexually active if they either met the Current Status Only definition, had documentation of reporting past sexual activity at a clinical encounter during the measurement period, had a clinician-confirmed STI during or before the measurement period, had a self-reported or clinician-confirmed pregnancy during or before the measurement period, reported the use of nonhormonal contraceptives, or reported having ≥1 total sexual partner. The indicator “patient report of past sexual activity” was determined as follows: at sites 1 and 3, this was any response of Not Currently in the structured field for sexual activity status documentation in the Substance and Sexuality History tab of the Social History Activity section.
To assess the effect of contraceptive use on the total number of females categorized as sexually active, we collected information on indications for contraceptive use (ie, contraceptive versus noncontraceptive use). Two additional definitions were developed to distinguish between these indications.
Expanded + Contraceptives Excluding Noncontraceptive Indication
Adolescents were classified as sexually active if they met any of the criteria listed in Table 1, excluding orders for hormonal contraceptives with noncontraceptive indications.
Expanded + Any Contraceptive:
Adolescents were classified as sexually active if they met any of the criteria listed in Table 1, including any order for hormonal contraceptives. This definition most closely resembles the HEDIS claims-based approach for determining sexual activity, although the HEDIS measure excludes women who have a prescription for retinoids (ie, a noncontraceptive use).
Percentages of female adolescents categorized as sexually active were calculated for each indicator variable. We then calculated the percentage of female adolescents who met criteria for each definition of sexual activity, the percentage of female adolescents having documentation of a Chlamydia test performed, and means and ranges for the rate of positive Chlamydia test results per 1000 adolescent females, according to sexual activity definition and by site. This value was calculated as the number of positive test results among adolescent females meeting criteria for a definition of sexual activity per total number of adolescent females in the overall sample. Due to differences in the number of adolescents per site, reports of arithmetic arithmetic means of the site percentages were weighted so that each site contributed equally. A Pearson χ2 test was used to test for differences across sites in each of the performance items (eg, documented as sexually active, performance of Chlamydia test, and positive Chlamydia test results per 1000 female adolescents) for each sexual activity definition. We calculated sensitivity of each definition of sexual activity by using “Expanded + Any Contraceptive,” the broadest definition, as the gold standard.
There were 35 063 female patients in the study, with the number per site ranging from 978 to 27 631 (Table 2). The mean age was 15.3 years; site means ranged from 14.3 to 16.6 years. More than 32% of female adolescents at each site were classified as other than non-Hispanic white. The most common insurance type for this sample was Medicaid (40%), followed by commercial insurance (34.7%), and self-pay or other insurance (17.6%).
The most commonly documented individual indicator of sexual activity was “patient report of being sexually active” (mean across 4 sites: 19.2%) (Table 3). Orders for hormonal contraceptives, when indication status was disregarded, identified the largest percentage of female adolescents as sexually active, capturing 25.8% of patients on average. Not all of the indicators of sexual activity were available at all of the sites, including the indication for contraceptive orders. Other individual indicators of sexual activity were less frequently documented.
The percentage of adolescent females classified as sexually active varied by definition and by site and increased as more indicator variables were included in the definition (Table 4). In addition, as the definition of sexual activity expanded and the denominator size increased, the percentage of sexually active females who received at least 1 Chlamydia test decreased. Seventy-nine percent of adolescent females defined as sexually active by using the Current Status Only definition had documentation of Chlamydia screening compared with 55.9% of those meeting the Comprehensive definition.
Using a broader definition of sexual activity resulted in greater identification of positive Chlamydia infections. By focusing on 1 indicator only (Current Status Only), the rate of positive infections was 28.6 per 1000 adolescent females (Table 4). Using a broader definition of sexual activity that included pregnancy and other STIs or contraceptive use increased the observed rate of Chlamydia infection to a range of 42.7 to 45.7 per 1000 adolescent females. For each sexual activity definition and performance item (eg, documented as sexually active, performance of Chlamydia test, and positive Chlamydia test results per 1000 female adolescents), the difference was statistically significant (P < .0001). Our sensitivity calculations demonstrated that the Current Status Only definition detected approximately one-half of adolescent females documented as sexually active, Chlamydia screenings, and positive Chlamydia tests compared with the broadest definition (Table 5).
In this study of the medical record extractions of >30 000 adolescent females, we examined several operational definitions of sexual activity by using EHR-derived data. Our research demonstrates many challenges in the initial steps to translate a traditional health quality measure by using claims-based data into an EHR-derived measure. All of the information related to sexual activity was not available at all of the sites. In fact, 2 of our 6 original sites were not included in our final analysis because we were unable to access information about Chlamydia screening. Orders for contraception and report of being sexually active were the most common indicators of sexual activity. Not surprisingly, the percentage of adolescent females identified as sexually active increased as more indicator variables were included in the definition. The percentage of adolescent females tested for Chlamydia and the Chlamydia positive rate varied according to definition of sexual activity, with the 2 broadest definitions of sexual activity being superior at identifying infections. Our results showed that removing noncontraceptive use of hormonal contraceptives led to a slight improvement in calculated rates of Chlamydia screening. This improvement could be a result of removing noneligible females from the denominator population of those who should be screened. Thus, EHRs have the potential to improve our ability to correctly identify females for screening by providing additional information such as how contraceptives are being used.
Operationalizing a definition of sexual activity, admittedly, is challenging. Clinician, adolescent, and parent definitions of sexual activity are embedded in social and cultural contexts and are likely to vary by region and over time.36 In addition, for clinicians who care for minor adolescents, concerns about confidentiality may compromise their use of structured fields to identify patients as sexually active.37,38 These unresolved issues and others influence how adolescents’ sexual activity status is documented (or not documented) in the EHR. Use of more comprehensive operational definitions of sexual activity may minimize the impact of these and other user-related issues with EHR data entry by offering more ways to capture the concept.
To our knowledge, this study is the first consideration of EHR use for quality measurement focused specifically on adolescent reproductive health care. The strengths of this study are the large sample size and inclusion of multiple sites and EHR vendors. Several limitations should be acknowledged. Our findings may not be generalizable to adolescent females who do not seek health care in the primary care setting. Interestingly, however, the 2006–2010 National Survey of Family Growth estimates that 42.6% of adolescent females ages 15 to 19 years have ever had sexual intercourse.39 Alternatively, our findings may underestimate the true prevalence of sexual activity and Chlamydia screening in this population of female adolescents because adolescents may obtain reproductive health services at family planning centers, school-based clinics, or community health centers.40,41 Sites were asked to electronically abstract “clinician documentation of current and past sexual activity status reported by patient.” We cannot be certain that this information actually represented patient self-report and does not indicate physician knowledge of patient sexual activity status derived from information other than patient report. As well, adolescents may not trust the health care system to maintain confidentiality and so withhold sensitive information.42
Information about sexual activity can be gathered from a variety of data fields, and changing the configurations of these indicators results in differences in the percentage of adolescent females classified as sexually active, screened for Chlamydia infection, and Chlamydia infection rates. Our study also shows that relying on clinicians to indicate a patient’s sexual activity status within a specific field in current EHR products does not accurately and reliably identify sexually active teenagers. We believe that the use of EHR for large-scale quality measurement in adolescent health care in nuanced and/or sensitive areas, such as sexual health, may be premature. In this large study, challenges in abstracting information combined with large site-to-site variation produced many difficulties in the first steps toward using the EHR to measure adolescent reproductive health care quality. In light of the growing interest in pay-for-performance systems, providers themselves may be directly affected by the variations in quality measurement described in the present article. Given the imperative to develop EHR-derived health care quality measures and the challenges in initial efforts to define sexual activity described here, it is important that clinicians who care for adolescents participate in efforts to improve usability of and data collection in EHRs as well as adolescent health care quality measure development.
This study demonstrated the variability and quality of information related to adolescent female sexual activity status that is currently available for electronic abstract within EHRs. Our research supports the use of broader definitions of sexual activity for health quality measurement. Contextual information available in EHRs may allow the exclusion of adolescent females using hormonal contraceptives for noncontraceptive reasons and may consequently improve the accuracy of Chlamydia screening measures. Collaborations between EHR vendors, health informatics specialists, and adolescent health experts will be critical if we are to keep adolescent health quality measurement on pace with other fields.
- Accepted February 20, 2014.
- Address correspondence to Elise D. Berlan, MD, MPH, Section of Adolescent Medicine, Nationwide Children's Hospital, 700 Children’s Way, Columbus, OH 43205. E-mail:
Dr Berlan conceptualized and designed the study, interpreted the data, drafted the initial manuscript, and critically reviewed and revised the manuscript; Dr Ireland coordinated and supervised data collection, interpreted the data, wrote part of the first draft, and critically reviewed and revised the manuscript; Ms Morton coordinated and supervised data collection, conducted statistical analyses, interpreted the data, wrote part of the first draft, and reviewed and revised the manuscript; Ms Byron and Dr Kelleher conceptualized and designed the study, interpreted the data, wrote part of the first draft, and critically reviewed and revised the manuscript; Mr Canan conceptualized and designed the study, coordinated and supervised data collection, wrote part of the first draft, and reviewed and revised the manuscript; and all authors approved the final manuscript as submitted.
The content of this article is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This project was supported by grant U18HS020503 from the Agency for Healthcare Research and Quality.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Tieder JS,
- Hall M,
- Auger KA,
- et al
- Gold R,
- Angier H,
- Mangione-Smith R,
- et al
- Thompson WK, Rasmussen LV, Pacheco JA, et al. An evaluation of the NQF quality data model for representing electronic health record driven phenotyping algorithms. AMIA Annu Symp Proc. 2012;2012:911–920
- Benin AL,
- Fenick A,
- Herrin J,
- Vitkauskas G,
- Chen J,
- Brandt C
- Casciato A,
- Angier H,
- Milano C,
- Gideonse N,
- Gold R,
- DeVoe J
- ↵Centers for Medicare & Medicaid Services. Medicare and Medicaid EHR incentive programs. Available at: www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/index.html. Accessed April 29, 2013
- ↵American Recovery and Reinvestment Act (ARRA) of 2009, Pub. L. No. 111-5, 123 Stat. 115, 516 (Feb. 19, 2009).
- ↵Agency for Healthcare Research and Quality. CHIPRA Pediatric Quality Measures Program. Available at: www.ahrq.gov/policymakers/chipra/pqmpback.html/. Accessed April 17, 2013
- ↵Children's Health Insurance Program Reauthorization Act of 2009, Pub. L. no. 111-3, 123 Stat 8 (2009).
- Hagan JF,
- Shaw JS,
- Duncan PM
- Zuckerman B,
- Nathan S,
- Mate K
- ↵Pickering LK, ed. American Academy of Pediatrics. Red Book: 2012 Report of the Committee on Infectious Diseases. 29th ed. Elk Grove, IL: American Academy of Pediatrics; 2012
- US Preventive Services Task Force. Behavioral counseling to prevent sexually transmitted infections, recommendation statement. Available at: www.uspreventiveservicestaskforce.org/uspstf08/sti/stirs.htm. Accessed May 13, 2013
- Centers for Disease Control and Prevention
- ↵National Committee for Quality Assurance. HEDIS 2014 Volume 2: Technical Specifications for Health Plans. Washington, DC: National Committee for Quality Assurance; 2013
- Tang PC,
- Ralston M,
- Arrigotti MF,
- Qureshi L,
- Graham J
- ↵Horowitz DA. Quality of care measures require collection of quality data. [E-letter], Pediatrics (July 23, 2012), Available at: http://pediatrics.aappublications.org/content/130/1/139.full/reply#pediatrics_el_54063. Accessed February 12, 2014
- Martinez G,
- Copen CE,
- Abma JC
- Klein JD,
- Allan MJ,
- Elster AB,
- et al
- Copyright © 2014 by the American Academy of Pediatrics