BACKGROUND AND OBJECTIVE: Chlamydia trachomatis infections are common among sexually active young women. We developed a practice-based quality improvement (QI) collaborative to increase Chlamydia screening in at-risk young women.
METHODS: Structured data fields were integrated into the electronic record for practices affiliated with Boston Children’s Hospital. A learning community (LC) was developed. Content included the adolescent well visit, assessment of sexual/risk behaviors, epidemiology of sexually transmitted diseases, and screening methods. The QI initiative effectiveness was assessed by comparing preintervention and postintervention rates of Chlamydia screening by using statistical process control analyses and logistic regressions.
RESULTS: LC participants demonstrated significant increases in recommended Chlamydia screening, as illustrated by using Healthcare Effectiveness Data and Information Set (HEDIS) screening rates (LC1: 52.8% preintervention vs 66.7% postintervention [P < .0001]; LC2: 57.8% preintervention vs 69.3% postintervention [P < .0001]). Participating practices reported total improvements larger than nonparticipating practices (13.9% LC1, 11.5% LC2, and 7.8% nonparticipants). QI and LC efforts also led to increased documentation of sexual activity status in the record (LC1: 61.2% preintervention to 91.2% postintervention [P < .0001]; LC2: 43.3% preintervention to 61.2% postintervention [P < .0001]). Nonparticipating practices were more likely to perform indiscriminate screening.
CONCLUSIONS: Through our QI and LC efforts, statistically and clinically meaningful improvements in Chlamydia screening rates were attained. Differences in rates of improvement indicate that LC participation likely had effects beyond electronic medical record changes alone. During the project time frame, national HEDIS screening rates remained unchanged, suggesting that the observed improvements were related to the interventions and not to a national trend. As a result of QI tools provided through the LCs, HEDIS screening goals were achieved in a primary care setting.
- CI —
- confidence interval
- EMR —
- electronic medical record
- HEDIS —
- Healthcare Effectiveness Data and Information Set
- HMO —
- health maintenance organization
- LC —
- learning community
- PPOC —
- Pediatric Physicians’ Organization at Children’s
- QI —
- quality improvement
- SPC —
- statistical process control
- STD —
- sexually transmitted disease
Sexually transmitted diseases (STDs) caused by Chlamydia trachomatis are common in the United States,1 particularly among sexually active women <25 years old.2 In Massachusetts women ages 15 to 19 years and 20 to 24 years, Chlamydia rates in 2013 were 1731.5 per 100 000 and 2968.5 per 100 000, respectively.3 Screening is essential because the majority of infected women are asymptomatic. Laboratory screening of at-risk individuals is a key strategy for identifying infections that would otherwise go undetected. The main objectives of Chlamydia screening are to prevent pelvic inflammatory disease, infertility, and ectopic pregnancy; women with untreated Chlamydia infection are at higher risk for these negative outcomes. Annual screening would prevent 61% of Chlamydia-related pelvic inflammatory disease in women who became infected.4 Primary care providers are in a unique position to screen patients for risky behaviors and prevent associated morbidity and mortality.
Routine annual screening of sexually active women aged <25 years is a Top 10 Recommendation from the United States Preventive Services Task Force.5 Despite these recommendations, Chlamydia screening remains suboptimal.6,7 Screening practices vary by pay or; in 2009, screening rates were 43% among eligible female patients enrolled in commercial health care plans and 57% among the Medicaid population.7 Education can affect changes in clinician performance and improve practice-related outcomes.8 Providers have cited a lack of training as the primary barrier to delivery of recommended preventive health care to adolescents.9,10 Various educational interventions attempting to improve preventive services have varied in results, and they have been limited by subjective outcome measures and assessment of short-term outcomes only.11–13 Few investigations have used objective outcomes, measures of actual behavioral change, or long-term assessments of clinician behavior. We developed a community-based, quality improvement (QI) intervention in the form of a longitudinal learning community (LC) with the goal of increasing Chlamydia screening in at-risk adolescent and young adult females. Our primary hypothesis was that clinician screening for Chlamydia would improve after LC participation.
The Pediatric Physicians’ Organization at Children’s (PPOC) is an independent practice association comprised of nearly 300 pediatricians and 100 associated health care providers in 85 pediatric primary care offices affiliated with Boston Children’s Hospital. As part of the PPOC’s QI agenda, a menu of LC opportunities is offered to providers each year. Each annual LC focuses on a specific area of practice improvement and requires participating practices to perform practice-level QI interventions. Coursework sessions are held for 9 months; over the last 3 months, practices take the lessons learned from the final session, implement them into their improvement projects, and measure the effects of their test of change. Each practice has to participate in at least 1 LC per year, but some enroll in multiple LCs simultaneously. LCs focused on adolescent health and Chlamydia screening were offered from June 2012 to March 2013 (LC1) and from June 2013 to March 2014 (LC2).
In January 2012, a Chlamydia screening measure was negotiated into the organization’s largest managed care contract. In preparation for that contract measure, all PPOC practices received information regarding the importance of Chlamydia screening and began to receive quarterly reports of their Healthcare Effectiveness Data and Information Set (HEDIS) Chlamydia screening rates. These reports use claims data to track the monthly progress of Chlamydia screening rates at the practice level. They also include a registry of patients who meet the HEDIS criteria for Chlamydia screening but have not been screened in that calendar year. In addition, structured data fields (Supplemental Figure 5) were integrated into the ambulatory note portion of the organization’s electronic medical record (EMR) to remind providers to collect relevant STD risk history during adolescent/young adult checkups and to allow the organization to collect data on measures of Chlamydia screening processes.
The PPOC developed the adolescent medicine LC to assist providers in improving adolescent preventative care and Chlamydia screening rates for at-risk female subjects. This project was organized and run by 2 physician medical directors who developed educational materials and by 2 QI consultants who assisted participating practices with performance improvement.
The LC consisted of four in-person 2-hour sessions and four 1-hour interactive webinars with diverse content that were held for pediatric practice clinicians and staff members (Table 1). Social history–taking techniques for adolescent patients were reviewed, as well as confidentiality issues relating to teenagers/young adults. Session content included the adolescent well visit, assessment of sexual/risk behaviors, epidemiology of STDs, and appropriate screening methods. Skills-building was incorporated into each of the sessions, including mock interviews with trained peer educators, role-play of parent–provider scenarios, and discussion of video scenarios. Review of QI tools assisted practices in evaluating their current process for Chlamydia screening and identifying inefficiencies and opportunities for improvement. The webinars promoted shared learning among practices through the presentation of their QI cycle-of-change projects related to Chlamydia screening in their office.
As our primary control, HEDIS Chlamydia screening rates reported by the National Committee on Quality Assurance were used to provide a comparison of our data versus national trends. We calculated a weighted average of the rates of the 2 age groups from the national HEDIS data to match the age distribution of our own data, in which 72.9% of patients were aged 16 to 20 years and 27.1% were aged 21 to 24 years. These weighted averages for the national HEDIS health maintenance organization (HMO) data were 42.1% for 2010, 43.4% for 2011, 42.4% for 2012, and 43.8% for 2013 (Fig 1). For a secondary control, the performance of PPOC practices who did not participate in either LC was evaluated. However, these nonparticipating practices were “contaminated” by the LC intervention. We purposefully attempted to distribute the work conducted in the LCs to nonparticipating practices through our quality consultants (who worked with practices who did and did not participate in the LCs) and through mass communications including “Tips of the Week” that are regularly distributed to the organization. As such, we used the national HEDIS data from the same time period as the intervention as a form of a control group.
The impact of the LCs was measured in 2 ways. For the primary outcome, insurance claims data of female patients aged 16 to 24 years at the time of visit over the preintervention period (January 1, 2010–December 31, 2011) and the postintervention period (January 1, 2012–October 31, 2014) were used to measure performance on the HEDIS Chlamydia screening measures. The 2014 HEDIS specification14 (designed to calculate an annual rate of Chlamydia screening) was modified to provide a monthly measure of change. Our denominator was comprised of “at-risk” patients who were sexually active according to HEDIS criteria (≥1 insurance claim for Papanicolaou tests/pelvic examinations, contraceptive services/prescriptions, pregnancy testing or other pregnancy-related services, and/or screening and treatment of STDs) with ≥1 relevant claim within each month, rather than within the calendar year. Continuous enrollment (≥11 months of the previous 12 months) was also required for a patient to be included in the denominator. Our numerator was comprised of those at-risk female subjects who had a paid claim for a Chlamydia screening test performed in the same month as the eligibility claim or within the previous 11 months. Claims data came from 4 commercial insurers and include all paid insurance claims for PPOC patients covered by HMO-type products from these insurers. This data set includes 25% of all PPOC patients and 34% of all commercially insured patients. All of the patients included were insured with commercial HMO products.
Second, we measured the impact of our interventions on various process measures using chart review from the PPOC’s centralized EMR system. Clinicians’ performance was measured at all checkups for female subjects aged 16 to 24 years on process measures, including documenting patients’ sexual activity status, ordering Chlamydia testing on those deemed to be sexually active, and recording the results of testing in the record. The EMR study data were derived from a subset of 59 PPOC practices that use the centralized EMR. All eligible patients from those practices were included in the EMR review, regardless of insurance type.
Statistical process control (SPC) analyses were conducted to analyze the effect of LC participation on the outcome measures at the level of the practice. Practice-level analysis was used (as opposed to provider-level) because the purpose of the LC is to have participating providers implement cycles of change within their practice as a whole. A practice was considered a participant in the LC if at least 1 practice member received a completion certificate for the project (ie, attended at least three-fourths of the in-person sessions). SPC analyses were performed by using QI Macros v2003.12 (KnowWare International, Denver, CO).
To distinguish between the effects of the EMR and the LC interventions separately, 3 time periods were defined. The “preintervention” time period was defined as the time before any specific interventions related to Chlamydia screening; it ranged from January 2010 until December 2011 for all practices. The second time period (post-EMR/pre-LC) was defined as the time beginning with the implementation of structured EMR fields to capture sexual activity and external Chlamydia testing at all adolescent well visits (January 2012 for all practices) until the start of LC participation. The last time period (post-LC) was defined as the end of LC participation through December 2014. For practices that did not participate in either of the LCs (non-LC participants), the second time period extends from January 2012 (post-EMR/pre-LC) through December 2014 (the end of data collection).
To evaluate the effect of the EMR and LC interventions independently, the monthly HEDIS Chlamydia screening rates were modeled by using logistic regression according to the time periods described earlier for all PPOC practices, including those who participated in LC1, those who participated in LC2, and nonparticipants. The analysis was performed by using SAS version 9.3 (SAS Institute, Inc, Cary, NC). This work met our institution’s definition of QI and was therefore exempt from institutional review board review.
Twenty-four practices participated in LC1 (Table 2). Thirty-six practices in total participated in LC2; however, 15 of the 36 had at least 1 clinician participate in LC1. For purposes of our analyses, the LC2 cohort was limited to the 21 practices that participated in the LC for the first time during LC2. Forty PPOC practices not involved in either LC constituted the control group. Practices were similar in terms of size and geographic region, although LC1 participants were somewhat more likely to be from larger practices and from the PPOC’s west region.
At baseline, participants in both LCs and control practices demonstrated screening rates above the national HEDIS rate. LC1 participants reported significant increases in recommended Chlamydia screening of at-risk female patients, from 52.8% preintervention to 54.5% postimplementation of the EMR changes, and 66.7% after completion of LC participation. Special cause change was identified by using SPC analysis starting just before beginning the LC and accelerating throughout the period of LC participation and leveling off after completion of the LC experience (Fig 1A, Table 3). For LC1 participants, a 1.7% increase (95% confidence interval [CI], 0.1–3.2) from baseline was attributable to the EMR changes alone, and a 13.9% increase (95% CI, 13.0–14.8) was attributable to the combination of EMR changes and LC participation. Improvements were seen in both HEDIS age groups. Screening rates increased from 50.3% to 63.6% for 16- to 20-year-olds (13.4%) and from 60.1% to 74.4% for 21- to 24-year-olds (14.3%).
Participants in LC2 had a higher baseline Chlamydia screening rate (57.8%) than practices included in LC1. Improvements in screening rates were reported among LC2 practices, from 57.8% preintervention to 61.5% after the EMR changes. Rates increased to 69.3% after LC participation, with special cause effect demonstrated beginning a few months after the EMR changes and rising further with LC participation (Fig 1B, Table 3). For LC2, a 3.7% increase (95% CI, 2.0–5.3) was attributable to the EMR changes alone, with an 11.5% increase (95% CI, 9.8–13.2) attributable to the EMR changes plus LC participation. Improvements were again seen in both age groups; screening rates increased from 54.9% to 67.8% for 16- to 20-year-olds (12.9%) and from 64.4% to 73.1% for 21- to 24-year-olds (8.7%).
We also sought to determine whether these improvements in performance were limited to those practices that participated in the LCs or whether they had spread throughout the PPOC. Non-LC participating practices started with the highest baseline screening rate (58.3%). Nonparticipating practices experienced an increase to 66.1% after the EMR changes (7.8% increase [95% CI, 6.8–8.9]) (Table 3), with special cause identified starting a few months after the EMR changes (Fig 1C). After the EMR changes, screening rates increased from 55.8% to 63.0% for the 16- to 20-year-olds (7.2%) and from 64.9% to 74.1% for the 21-to 24-year-olds (9.2%).
As a further control, we compared our data with national trends. During the period of the study analysis, national HEDIS Chlamydia screening rates among HMO-insured patients remained essentially unchanged for female subjects aged 16 to 20 years (41%), and they increased somewhat, from 46% in 2010 to 50% in 2013, for female subjects aged 21 to 24 years.14 Comparing our data versus weighted averages for the national HEDIS HMO data, the observed increase in screening rates was not reflective of a national trend toward increased Chlamydia screening (Fig 1).
Secondary analyses were conducted to evaluate process measures that may have contributed to increased Chlamydia screening, such as whether the provider documented sexual activity status in the EMR at the time of the routine checkup. The prevalence of provider documentation of the patient’s sexual activity status at checkup visits increased over time in all 3 groups but by the greatest magnitude in the LC1 cohort (LC1: 61.2% preintervention to 91.2% postintervention [P < .0001]; LC2: 43.3% preintervention to 61.2% postintervention [P < .0001]; non-LC 47.2% preintervention to 73.9% postintervention [P < .0001]) (Fig 2).
We next explored whether providers changed their frequency of ordering Chlamydia screening tests at the time of the checkup or of documenting that appropriate screening had been performed within the last 12 months on those female patients who they themselves reported as sexually active within the EMR. All 3 cohorts similarly improved EMR documentation of Chlamydia screening for female subjects identified as sexually active (LC1: 61.0% preintervention to 84.5% postintervention [P < .0001]; LC2: 57.8% preintervention to 86.4% postintervention [P < .0001]; non-LC: 63.1% preintervention to 81.0% postintervention [P < .0001]) (Fig 3).
Last, we evaluated whether Chlamydia testing was performed in patients for whom risk screening was not documented at the checkup visit. In this analysis, non-LC participants had patterns of screening markedly different from LC participants (Fig 4). Within the cohort of female patients without any EMR documentation of sexual activity status, non-LC practices sent screening tests for Chlamydia on 29.6% of patients preintervention and 59.7% of these patients postintervention (P < .0001). LC participants screened these female subjects without documentation of sexual activity status less frequently: LC1, 21.8% pre-LC to 25.0% post-LC [P = .002]; LC2, 12.2% pre-LC to 27.2% post-LC [P < .0001]. These findings suggest that nonparticipating practices performed more indiscriminate screening (ie, performing laboratory testing on patients who were not documented as sexually active) than did LC participants.
Through our QI efforts involving changes in EMR documentation fields and LC educational and practice efforts, clinically meaningful improvements in Chlamydia screening rates for adolescent and young adult women were attained within our pediatric primary care organization. Practices involved in the LCs achieved screening rates significantly above their own preintervention levels and nearly 30 percentage points above national rates. Observed improvements were maintained over time. Improvements were also seen among PPOC practices that did not directly participate in the adolescent medicine LCs.
What could account for this pattern of observed effects? The improvements in screening practices were likely multifactorial. Initial increases in Chlamydia screening were seen with the introduction of structured data fields in the EMR and the start of a new risk-based contract with quarterly performance reports. These strategies were adopted by all PPOC members, not just those groups participating in the LC. The largest improvements in screening overall were seen among participants in the first LC. These practices had the lowest preintervention levels of Chlamydia screening among the PPOC, suggesting that the practices which signed up for LC1 may have self-selected to participate because of a perceived need to improve care in this area. Participants in LC2 and non-LC participants had higher screening levels at baseline and thus had less room for improvement.
When comparing the improvements in the 3 groups included in the present evaluation, we attempted to disentangle the effects of the EMR changes and LC participation. This process proved difficult to accomplish, given the design of our intervention. Because this project was designed as a QI intervention, and not a controlled clinical trial, 2 critical factors make it difficult to isolate the effects of the various parts of the intervention. First, practices were not randomized to the 3 cohorts but rather self-selected to participate in the LC (or not) based on unmeasured factors such as provider perception of efficacy in caring for adolescents or willingness to change. Second, we deliberately distributed the lessons learned within each LC throughout the organization via mass communications and through QI consultants who worked with both LC participating practices and nonparticipating practices alike. Thus, we did not have the opportunity to perform a clean comparison with an internal control group matched to LC participants and unaffected by the LC experience, as would be performed in a controlled clinical trial.
Our analysis nonetheless allows us to draw certain likely conclusions. First, practices that participated in the LCs had total improvements larger than nonparticipating practices (13.9% for LC1, 11.5% for LC2, and 7.8% for nonparticipants). Based on these results, it is likely that LC participation produced effects beyond the EMR changes alone. Second, we attempted to separate the effect of the EMR changes and the LCs through statistical modeling. In this analysis, we hypothesized that LC2 provides the most accurate representation of the effect of the 2 interventions. For LC1 participants, there was little time between the EMR changes and the start of the LC for us to observe the effects of the EMR changes alone. For nonparticipants, there was clearly contamination of their practices by intentional dissemination of lessons learned in the LCs. In contrast, LC2 practices had ample time for the EMR changes to take hold before the start of their LC participation, as well as sufficient time after the end of the LC to observe post-LC effects (Fig 1B). We observed a 3.7% increase for LC2 after the EMR changes but before the start of the LC, and an additional 7.8% increase after LC participation. We believe that this scenario is the most accurate representation of the individual effect of the 2 interventions that we are able to provide given the project’s design.
In addition to our primary goal of increasing Chlamydia screening rates, we also examined process measures to help us understand how practices achieved improvements. In all 3 groups, the observed improvements in documentation of sexual activity status within the EMR seemed to be more closely related in timing to the onset of the PPOC’s risk-based contract than to the LCs. In addition, a differential pattern between LC participants and non-LC participants became evident. Although all 3 groups markedly improved provider documentation of patients’ sexual activity status in the EMR during checkup visits, as well as testing of those noted to be sexually active, differences became apparent when looking at those patients for whom sexual activity was not charted. LC participants demonstrated documentation and screening best-practices in line with the purpose of the HEDIS measures. Non-LC participants seem to have been much more likely to test patients indiscriminately without documenting their sexual activity status. This result suggests that LC participants learned not only the importance of Chlamydia screening but were also able to determine when it was appropriate to screen patients. In contrast, non-LC participants seemed to have started testing all patients without soliciting a sexual history and making guided screening decisions. The goals of screening are to protect patients from harm, not to bolster numbers. Haphazard screening defeats the goals of such a preventative test due to the harms associated with false-positive test results, which include emotional trauma for the patient. In addition, the inappropriate screening of nonsexually active female subjects unnecessarily increases the costs of care. Because ∼46.8% of high school students in the United States have ever had sexual intercourse,15 screening all patients within that age group for Chlamydia indiscriminately would double the numbers of patients screened and substantially increase the costs associated with follow-up testing. Within the LCs, we recommended against such indiscriminate testing and instead taught a targeted approach using confidential, sensitive interview techniques as a first-line screen, with the performance of appropriate Chlamydia testing for those patients deemed to be at risk.
Lessons were also learned from the project itself. In our collaborative LC, which took a very “hands-on” approach to developing QI initiatives, practices were supported by a staff of physicians and QI consultants. Observed changes in practice may not have occurred without this high level of participant support. LC sessions reviewed the skills required to take an appropriate sexual history. A recent article from Philadelphia notes that only 25% of primary care providers documented a sexual history within the patient medical record16; authors hypothesized that this lack of documentation may be related to discomfort with asking these questions. The LC helped providers to overcome this obstacle by normalizing the history taking.
It should be acknowledged that the LC we describe requires a substantial investment of time, effort, and money to replicate. Although we do not know the precise cost of the two LCs described in this report, we estimate that the total direct cost for fielding a typical 9-month LC in a metropolitan area such as ours is approximately $50 000, which includes costs for webinar services, speaker travel, facilities expenses, and salary for the QI consultants. This estimate does not include additional costs of travel to meetings or loss of clinical revenue, which should also be considered. We were able to demonstrate some substantial improvements with as simple an intervention as structured changes to the EMR, as well as feedback to practices regarding performance in this specific QI area. An important question for further investigation is whether this type of high-touch, longitudinal LC is more effective in changing provider practices and improving care than other potential approaches, such as relying on point-of-care EMR support, regular performance reporting, individual provider financial incentives for higher levels of performance, or some combination of these efforts. We hypothesize that our LC approach, which strives to build a community of providers committed to shared learning and continuous QI, will lead to sustainable improvement and set the stage for other successful QI projects within our organization. This hypothesis remains to be proven, however.
Limitations of our project should be acknowledged. Our population had a high level of Chlamydia screening at baseline; it is possible that they were more likely to be comfortable with screening at baseline. Participation in the LC was voluntary. Selection bias may have occurred if the LC participants were already interested in adolescent health and more savvy in their practices. However, some LC practices had low or no Chlamydia screening rates at the time of LC initiation, which argues against this hypothesis. In addition, advertisement of our LC opportunity and discussions among LC participants and nonparticipants may have heightened awareness of Chlamydia and screening practices among all PPOC members (a “cross-contamination” effect). Last, HEDIS claims data are problematic as a proxy for sexual activity status because insurance claims for Papanicolaou tests/pelvic examinations (not performed routinely in sexually active women aged <21 years) may result in falsely low estimates of the number of sexually active women, whereas contraceptive services/prescriptions may falsely elevate numbers (because not all users of contraceptive medications are sexually active).
Through our QI efforts primarily involving EMR changes and an LC model, we achieved statistically and clinically meaningful improvements in Chlamydia screening rates in adolescent and young adult women within the primary care setting that were well above national trends. Analyses showed that although EMR changes led to initial increases in screening, participation in an LC was associated with more substantial improvements in screening rates. The improvement was generally disseminated throughout our organization, although it is important to note that our initial LC participants generally had lower baseline performance than nonparticipants, suggesting that our efforts to recruit and improve the performance of lower performing practices were successful. Our project shows that QI efforts coupled with educational outreach through an LC helped to advance appropriate screening practices. Next steps will include expansion of these efforts to other adolescent health risk areas such as mental health screening and screening for substance use.
This QI research was funded with internal funds of the PPOC. Permission to use medical claims for this work was granted by the health plans that provided the claims data.
- Accepted November 16, 2015.
- Address correspondence to Amy D. DiVasta, MD, MMSc, Division of Adolescent Medicine, Boston Children’s Hospital, 333 Longwood Ave, Boston, MA 02115. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Funded with internal funds of the Pediatric Physicians’ Organization at Children’s Hospital.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- CDC National Center for HIV/AIDS
- Price MJ,
- Ades AE,
- De Angelis D, et al
- Davis D,
- Davis N
- Sanci LA,
- Coffey CM,
- Veit FC, et al
- Lustig JL,
- Ozer EM,
- Adams SH, et al
- Ozer EM,
- Adams SH,
- Lustig JL, et al
- Division of STD Prevention
- Kann L,
- Kinchen S,
- Shanklin SL, et al; Centers for Disease Control and Prevention (CDC)
- Copyright © 2016 by the American Academy of Pediatrics