BACKGROUND: High-quality preventive services for youth aged 11 to 18 include assessment and counseling regarding health behavior risks and developmental tasks/strengths of adolescence. Nationally, primary care health behavior risk screening and counseling rates lag consid-erably behind other preventive health services. The purpose of this project was to assist pediatric and family medicine practices to make office systems–based changes that promote comprehensive screening and counseling for risks and developmental tasks/strengths during adolescent preventive services visits.
METHODS: Over a 9-month period, 7 pediatric and 1 family medicine primary care practices (13 physicians and 3 nurse practitioners) participated in a modified Breakthrough Series Collaborative. This project was designed to support primary care practitioner efforts to implement comprehensive screening and counseling for risk behaviors and developmental tasks/strengths for their adolescent patients and increase the rate of brief office intervention and referral. Composite variables were designed to reflect whether screening and counseling were documented for risks and developmental tasks. Statistical comparisons were made by using the nonparametric Wilcoxon matched-pairs signed rank test.
RESULTS: There were increases in the composite measures of screening and counseling for risk behaviors (all 6 risks: 26%–50%, P = .01) and 3 of 4 developmental tasks/strengths (32%–66%, P = .01). Documentation of office interventions for identified risks and out-of office referral rates did not change.
CONCLUSIONS: With the use of an office systems–based approach, screening and counseling for all critical risk behaviors and developmental tasks/strengths during adolescent preventive services visits can be improved in primary care practices.
- BTS —
- Breakthrough Series
- CRAFFT —
- EMR —
- electronic medical record
- HEEADSSS —
- Home environment, Education/Employment, Eating, Activities, Drugs, Sexuality, Suicide, and Safety
- PCP —
- primary care practitioner
High-quality preventive services for adolescent youth in the primary care setting include health behavior risk assessment (inadequate physical activity, poor nutrition, sexuality-related behaviors, substance abuse, and unintentional and intentional injury-related behaviors1) and the youth’s progress on adolescent developmental tasks. The Bright Futures third edition guidelines for pediatric preventive services describe the following significant developmental tasks/strengths: competencies, healthy behaviors, relationships, community engagement, self-confidence, resiliency, and decision-making.2,3 These developmental tasks, also referenced as strengths or protective factors, correlate with psychosocial thriving, physical health, and lower likelihood of negative or risky behaviors during adolescence.4–8
In a 2005 national study by Ma et al,9 anticipatory guidance rates for health promotion (nutrition and physical activity) and risky behaviors related to sexual activity, substance use, and safety (defined as contraception use, being screened for HIV, tobacco use, and injury) were <30%. Similarly, in a study by Irwin et al10 that used 2001–2004 Medical Expenditure Panel Survey data, counseling rates of adolescents were 49% for healthy eating, 40% for exercise, and 31% for seat belt and helmet use. To our knowledge, no national data set measures the rate of developmental screening and counseling during adolescent preventive services visits.
Several studies suggest that office-based systems changes can improve delivery of preventive services.11–13 Klein et al14 implemented Guidelines for Adolescent Preventive Services training programs at community and migrant health centers. Youth reported receiving increased anticipatory guidance in 51 of 79 Guidelines for Adolescent Preventive Services–related areas. In a managed care organization, Ozer et al15 demonstrated improved screening and counseling for adolescent health risks, attributable to a combination of interventions, including practitioner education, office system tools implementation, and health educator availability. In another study, Ozer and colleagues16 found that practitioner training significantly increased 3 targeted areas of adolescent preventive health screening and counseling: safety belts/helmet use, substance abuse, and sexuality.
The aim of this quality improvement project was to assist pediatric and family medicine practices to make office systems–based changes that promote comprehensive screening and counseling for risks and developmental tasks/strengths for 95% of adolescents during adolescent preventive services visits.
Thirteen physicians (12 pediatricians and 1 family medicine physician) and 3 nurse practitioners (2 pediatric and 1 family medicine) participated from 7 pediatric and 1 family medicine primary care practices. These private and hospital-owned practices represented a convenience sample from 3 geographic areas in Vermont. Two practices are located in the first geographic area, Chittenden County, with one-fourth of the state’s population, and are the sites for the University of Vermont pediatric residency teaching program. In one of these practices, 4 of 5 primary care practitioners (PCPs) participated in the study, and, in the second practice, 1 of 3 PCPs participated. The other 6 practices are clustered in 2 rural counties, and, although the number of PCPs per practice is small, they provide care to a large number of children and youth in these counties. One of these sites includes a school-based health center. All PCPs from 5 rural practices participated, as did 2 of 3 in the other rural practice. No practice had an electronic health record.
An intervention was designed to (1) support PCP efforts to implement comprehensive screening and counseling for risk behaviors and developmental tasks/strengths for their adolescent patients, and (2) increase the rate of brief office intervention and/or referral resulting from those risks being screened, identified, and discussed with adolescent patients during preventive services visits.
Project staff educated PCPs and their office staff to make changes to office systems by using a modified Breakthrough Series (BTS) Collaborative model.17 The BTS Collaborative intervention model includes 3-day-long conferences (learning sessions), with “action periods” between sessions during which participants improve their practice settings by using Plan-Do-Study-Act cycles and self-measurement.17 Participants also take part in monthly coaching phone calls with project staff to review data, discuss progress, and brainstorm solutions to challenges (see Fig 1). This approach is anchored on the work of a practice team, which includes a physician or nurse practitioner, 1 office nurse, and 1 practice support staff person, who attend all the learning sessions. Each team brings the information back to each of their practices and then leads the participation for their practice in the monthly data collection, review of run charts, and all practice phone calls (see Fig 2). This team works with the rest of the practice to plan, initiate, and study the practicewide efforts in systems change. This project lasted 9 months, differing from the BTS model only in that learning session 2 was replaced with a community meeting in each geographic area.
Learning session 1 was attended by teams from all 8 practices. The educational content included review of the evidence base for adolescent preventive services, Plan-Do-Study-Act cycles, and office system improvement strategies to support adoption of new preventive services or increased screening and counseling rates for existing services. PCPs were trained to screen for and counsel about 6 health risk behaviors (Table 1).
Training was provided in the use of the Car-Relax-Alone-Forget-Friends-Trouble (CRAFFT) tool, a validated screening instrument designed to help practitioners determine the severity of a youth’s substance use problem.18 Risk assessment screens such as the HEEADSSS19(Home environment, Education/Employment, Eating, Activities, Drugs, Sexuality, Suicide, and Safety) were also offered. Risk behavior information was coupled with strategies for managing appropriate referrals and conducting brief office-based intervention (eg, shared decision-making/motivational interviewing).
This learning session also included education and training in screening for and counseling about developmental tasks/strengths with adolescent patients. Unlike developmental milestones of physical changes like puberty and menarche, adolescent developmental tasks mark healthy social and emotional development. Bright Futures identifies these tasks as competencies, healthy behaviors, relationships, community engagement, self-confidence, resiliency, and decision-making.2,3
Several frameworks summarize these developmental tasks/strengths.20–22 The practices chose Brendtro’s Circle of Courage for this project.21 This framework uses 4 terms (generosity, independence, mastery, and belonging) to describe factors important for healthy adolescent development that correlate with lower risk behavior rates (Table 1). The developmental tasks/strengths can be assessed by expanding the HEEADSSS interview by using focused questions.23 Prompting stickers were provided to practices as a suggested temporary strategy until appropriate prompts could populate their permanent paper health record or eventually their electronic health record. All practices used some formal or informal reminder to assess 6 risks and 4 developmental tasks/strengths.
Learning session 2 took the form of community meetings in each geographic area. These were organized by the practice teams with support from the project staff. In each community meeting, local mental health, substance abuse treatment, public health, and school professionals joined the practice project participants to discuss community resources and referrals, especially targeting the difficult area of substance abuse. The 3-person team provided leadership at the community meetings, but the entire practice was invited to attend. The fact that these meetings were local and took place in the early morning or during the lunch hour allowed many practice members to attend and be more engaged in the project. The staff in the rural practices was small: often just 1 or 2 nurses and 1 or 2 practice support staff or a business manager who all participated in the community meetings.
Learning session 3 was attended by the original teams from each practice. Each team presented their most successful accomplishments, including detailed systems changes that led to these accomplishments. All teams discussed challenges and next steps to maintain the gains they made and continue to make progress on goals not yet accomplished.
Measurement was designed to assess practice change in risk and developmental tasks/strengths screening and counseling during preventive services visits. Random samples of adolescent preventive services visits by participating practitioners during 12 months pre- and 5 months postintervention were identified by 2 private Vermont health insurers and Vermont Medicaid. The purpose was to represent patients from all project insurer partners, including Medicaid. During the chart audit periods (pre and post), some practices had <10 adolescent preventive visits covered by one of the private insurers. In those practices, additional Medicaid-insured patient charts were sampled. Medical records grouped by age (11–14 and 15–18 years) were reviewed by an independent nurse auditor.
An average of 42 charts was reviewed per practice (range, 15–60). The goal was to obtain 20 to 30 charts per practice at each audit (pre and post); however, 2 of the practices did not have a sufficient number of adolescent preventive services visits when the health plans examined their claims data. The main outcome measures were 4 composite variables determining if screening and counseling were documented for all (6 of 6 risks and 4 of 4 developmental tasks/strengths) or most (5 of 6 and 3 of 4, respectively) screening and counseling topics. We also assessed screening and counseling for individual risks and developmental tasks/strengths. Additional data included whether a CRAFFT screening had been performed. Data were also collected on whether a brief office intervention and/or referral occurred when any risk behavior was identified.
Because not all practitioners from each practice participated, only data from patients of the participating PCPs within a single practice contributed to the practice-level estimate for each practice. The unit of analysis was the practice. The distributions of practice-level screening rates (averages) were compared pre- versus post- intervention by using the nonparametric Wilcoxon matched-pairs signed rank test24 (α = .05, 2-tailed). PASW 18.0 statistical software was used for data analysis. The project had University of Vermont Institutional Review Board approval.
A total of 184 preintervention and 155 postintervention charts were reviewed. Data quality was ensured by 100% duplicate data entry by a different person, and all discrepancies were resolved. Table 2 presents demographic data for the pre- and postintervention samples; Table 3 presents pre- versus postintervention statistical findings.
Outcomes: Risk Behaviors and CRAFFT Screening and Counseling
The composite measure of all 6 risk behaviors screened and counseled nearly doubled (from 26% to 50%, P = .01), whereas 5 or more risk behaviors increased by 46% (from 28% to 74%, P = .001). Comparing individual risk screening and counseling rates pre- versus postintervention, screening and counseling for emotional health/depression achieved statistical significance (from 43% to 74%, P = .01). There was no increase in CRAFFT screening rates.
Outcomes: Developmental Tasks/Strengths
The composite measure of 4 developmental tasks/strengths screened and counseled measured 16% in the preintervention time period and 29% in the postintervention time period (not significant). Screening and counseling for 3 or more developmental tasks/strengths doubled (from 32% to 66%, P = .01). With the exception of mastery (increase from 58% to 72%, P = .02), there were no statistically significant increases in the screening and counseling rates for individual developmental tasks.
Outcomes: Brief Office Interventions and Referrals
All the charts were reviewed for referrals and office interventions. Office interventions did not increase (24%–38%), nor did referrals for substance abuse or mental health issues (2%–5%).
Bright Futures has identified evidence-informed priorities for yearly adolescent preventive services visits, recommending that practitioners address 6 risk behaviors and the developmental tasks/strengths of adolescence.2 The participating practices improved in providing this comprehensive assessment and discussion for their adolescent patients.
This project demonstrates for the first time that PCPs are willing to incorporate routine adolescent developmental task assessment and discussion into preventive care, demonstrating that these components of 2008 Bright Futures guidelines can be implemented and measured.2
Nationally, primary care adolescent health risks screening and counseling rates trail other adolescent preventive health services (eg, vision, hearing, and immunizations).10 In comparison with national rates from the Ma and Irwin reports,9,10 our rates of 66% to 91% for health promotion and risk-related behavior items are higher. The 2005 health maintenance organization intervention provides comparison data: postintervention screening and counseling rates for substance abuse, sexuality, and safety belt/helmet use ranged from 78% to 88%,16 whereas we found a 66% to 86% range on those same topics. Practitioners identified the predominant causes of not reaching the 95% goal as (1) lack of time and (2) focus on a patient- or practitioner-identified problem area. Some practitioners mentioned use of prompting stickers as reminders to address remaining issues at subsequent visits. Our team, including the practitioners, believes in the importance of addressing all risks and developmental tasks in the annual preventive services visit, with the goal of accomplishing this in 95% of the visits, with the use of office systems change strategies such as previsit questionnaires and handouts.
Although many of the individual increases were not statistically significant, they merit further study. Use of the CRAFFT tool was very low in preintervention data collection. Although practitioners did demonstrate 75% inquiry about substance abuse, the majority were not familiar with the CRAFFT tool. Use of the CRAFFT screening tool did not increase significantly, possibly because only 3 practices were responsible for almost all the improvements. For this project, practitioners were advised that the CRAFFT tool can be used in 1 of 2 ways: universally or only when a first-level question about substance use is answered in the affirmative. In this project, the CRAFFT tool was administered universally by 2 practitioners, whereas others used it only upon receiving a positive answer during first-level substance use screening. When used as a secondary screen in this project, the practice goal was not 95%, unlike other preventive screening and counseling. Because we did not collect data on the percentage of screenings that were positive for substance use, we cannot determine the appropriateness of the use of the CRAFFT tool as a secondary screen.
Our method of data collection limits our ability to link positive screens to referrals. A practice audit of 60 charts did not have the power to demonstrate an increase in a referral that few youth need. Because there was no increase in referrals, we can only depend on conversations with the practitioners at the learning sessions and coaching calls. They reported more office intervention, sometimes because youth and/or parents refused to accept a referral in the community.
One pressing need the practitioners expressed before this project began was timely access to substance abuse treatment and mental health services for patients identified as needing such services. This led to the project’s strategy of attempting to engage all the practices providing pediatric primary care in a region to work with local substance abuse and mental health providers through a community meeting. In the community meetings, local representatives from state mental health, substance abuse treatment, public health and area health education centers, as well as middle and high school nurses and counseling staff discussed improvements in referrals for youth with the practice clinicians and staff. Anecdotal reports of improved communication in these communities were encouraging. Two practices and several mental health professionals established a monthly joint meeting to discuss and plan care for youth with complex mental health needs, which is still ongoing. A third practice colocated 2 mental health professionals and invited another practice to use these professionals for their patients as well. In the more urban/suburban county, the most important result of the community meeting was identification of the existence of student assistance counselors in the middle and high schools who could accept practice referrals for initial discussion of youth substance abuse problems and assist youth and their families with further referrals. Since this project, this urban/suburban practice has also colocated a mental health provider.
Although most practitioners asked informally about a youth’s general functioning at home and in school, the preintervention chart audit reflected lower rates for developmental assessment than for risk screening and counseling. The implementation of a more structured developmental tasks/strengths screening and counseling during preventive services visits presented some challenges initially. Learning session 1 included significant education about the relationship between risk-taking behavior and a young person’s strengths as well as opportunities to practice screening questions and counseling interactions with youth and parent educators.
In practitioner presentations and group discussions during learning session 3 at the end of the project, PCPs indicated a preference for approaching risk behavior assessment in a strength-based fashion that builds relationships and alliances rather than the traditional approach of just assessing risks. Their reports and the data reflect that discussing the concept of generosity (helping out at home, community service, volunteering with a faith-based group) was the most difficult to incorporate. These issues were discussed in several coaching calls, and practitioners were particularly interested in hearing what others were actually saying to youth and parents.
It was clear during discussions at this learning session 3 that the prompts were a major ingredient for successful change. Having already adopted a screening and counseling prompting system during this study should make incorporation into the electronic medical record (EMR) and sustainability of the gains much easier. Although none of the practices had an EMR during this project, incorporation into the EMR would probably be the most effective strategy and has been the focus of subsequent efforts in our work on adolescent preventive services. Since the end of this project, the 2 urban/suburban practices have added an EMR and used the adolescent preventive services flow sheet, with prompts for risks and strengths developed during the project, as the EMR template for these visits.
Our study suggests that use of the modified BTS collaborative approach with PCPs and office staff leads to comprehensive screening and counseling of risk and developmental tasks/strengths, consistent with the 2008 Bright Futures Guidelines.
The high preintervention screening and counseling rates for 4 risk factors may reflect practitioner participants who are already motivated to improve adolescent preventive services. The achievable improvements could be lower in a general population of practitioners. The project attempted to minimize sample bias by seeking different representative practice types (academic, school-based, private) and varied geographic locations of the practices and patient population.
The outcome measures were based on independent chart review, and PCPs were encouraged to use checklist stickers serving both as prompts and documentation. This may have caused substantial improvement in the documentation of the care being delivered. Although we were not able to use patient report or tape-recorded visits,25 it was apparent from participating PCPs’ discussions during the coaching calls and learning session 3 that they were making changes in interviewing and interacting with adolescents. Our methods examined only adolescent preventive services visit screening and counseling, thus not identifying risk or strength screening and counseling done at acute or other “nonwell” visits, potentially underestimating the totality of actual screening and counseling.
With no comparison group, adolescent risk behavior and strength screening and counseling increases may be due to factors other than the intervention. There were no recognized similar interventions occurring in Vermont, nor indications nationally of such spontaneous improvements. The Ozer study comparison group demonstrated no change in screening and counseling rates over the intervention period.16 Our improvements were unlikely to be due to other factors.
Finally, can this intervention in Vermont practices be generalized nationally? This quality improvement project tested the feasibility of applying BTS methodology to adolescent preventive services and demonstrated a positive outcome in Vermont. BTS methodology has been successful nationwide, so it would be expected to have similar results elsewhere. Although this intervention showed improvements over baseline, many rates of screening and counseling remain at less than optimal levels. Further research would benefit from use of randomized trials or comparison groups and should address spread strategies and impact on adolescent risk behaviors.
This study demonstrates that comprehensive screening and counseling for risk behaviors and developmental tasks/strengths during adolescent preventive services visits can be improved in primary care practices by using an office systems–based approach. Assessment of adolescent risk behaviors in the context of a strength-based approach may play a role in the success of this approach.
- Accepted May 16, 2012.
- Address correspondence to Paula Duncan, MD, UVM/VCHIP, 1 South Prospect St, 7th Floor, Burlington, VT 05401. E-mail:
FINANCIAL DISCLOSURE: Dr Duncan received funding from the American Academy of Pediatrics for the Bright Futures Performing Preventive Services Implementation Project; the other authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Provided by the Vermont Agency of Human Services including the Vermont Department of Health and the Office of Vermont Health Access (Medicaid); Banking, Insurance, Securities, & Health Care Administration (BISHCA); Blue Cross and Blue Shield of Vermont; MVP Health Care; The Vermont Health Plan; and the University of Vermont College of Medicine.
- ↵CDC. Healthy People 2010, Leading Health Indicators. Available at: www.healthypeople.gov/. Accessed June 26, 2009
- ↵Hagan J, Shaw J, Duncan P. Bright Futures: Guidelines for Health Supervision of Infants, Children and Adolescents. 3rd ed. Elk Grove Village, IL: American Academy of Pediatrics; 2008:520–521
- ↵Association of Maternal and Child Health Programs. A Conceptual Framework for Adolescent Health. A Collaborative Project of the Association of Maternal and Child Health Programs and the National Network of State Adolescent Health Coordinators. Available at: www.aecf.org/upload/publicationfiles/ar3622h421.pdf. Accessed August 10, 2012
- Irwin CE Jr,
- Adams SH,
- Park MJ,
- Newacheck PW
- Lannon CM,
- Flower K,
- Duncan P,
- Moore KS,
- Stuart J,
- Bassewitz J
- Bordley WC,
- Margolis PA,
- Stuart J,
- Lannon C,
- Keyes L
- Shaw JS,
- Wasserman RC,
- Barry S,
- et al
- Klein JD,
- Allan MJ,
- Elster AB,
- et al
- Ozer EM,
- Adams SH,
- Lustig JL,
- et al
- Langley J,
- Nolan K,
- Nolan T,
- Norman C,
- Provost L
- Goldenring JM,
- Rosen DS
- Benson P,
- Galbraith J,
- Espeland P
- Brendtro LK,
- Brokenleg M,
- Van Bockern S
- ↵Pittman K, Irby M, Tolman J, et al. Preventing Problems, Promoting Development, Encouraging Engagement: Competing Priorities or Inseparable Goals? Washington, DC: The Forum for Youth Investment, Impact Strategies, Inc; 2001, revised 2003. Available at: www.forumfyi.org/files/Preventing%20Problems,%20Promoting%20Development,%20Encouraging%20Engagement.pdf. Accessed August 10, 2012
- ↵Ott R, Longnecker M. An Introduction to Statistical Methods and Data Analysis. 5th ed. Pacific Grove, CA: Duxbury; 2001
- Bethell C
- Copyright © 2012 by the American Academy of Pediatrics