PEDIATRICS Vol. 107 No. 5 May 2001, pp. 1100-1107

Improving the Delivery of Adolescent Clinical Preventive Services Through Skills-Based Training

Julie L. Lustig, PhD^*, Elizabeth M. Ozer, PhD^*, Sally H. Adams, PhD^*, Charles J. Wibbelsman, MD, C. Daniel Fuster, MD^§, Robert W. Bonar, MD, and Charles E. Irwin Jr, MD^*

From the ^* Division of Adolescent Medicine, Department of Pediatrics, School of Medicine, University of California, San Francisco, San Francisco, California;  Kaiser Permanente, Northern California, Department of Pediatrics; and ^§ Kaiser Permanente, Southern California, Department of Pediatrics.

	ABSTRACT

Top Abstract Methods Results Discussion Conclusion References

Objective.  To examine the efficacy of skills-based training workshops on primary care providers' screening and counseling practices with adolescents during routine outpatient well visits.

Design.  Sixty-three primary care providers in outpatient pediatric departments within a managed health care organization participated in two 4-hour workshops on clinical preventive services for adolescents. The workshops focused on adolescent health, confidentiality, screening, and anticipatory guidance/brief counseling for 5 risk behaviors including: helmet and seatbelt use, tobacco use, alcohol use, and sexual behavior. A pre/posttest design was used to assess clinicians' screening and counseling practices during the pretraining and posttraining periods. Independent adolescent reports of clinicians' practices were obtained from 2 samples of 14- to 16-year-old adolescents immediately after their routine well visit in the outpatient clinics. One sample of adolescents reported during a pretraining period and a separate sample reported during a period after the training.

Results.  Adolescent reports indicated that after the training workshops, the average percentage of adolescents screened by their primary care providers increased significantly for seatbelt use (from mean 38% to 56%), helmet use (from mean 27% to 45%), tobacco use (from mean 64% to 76%), alcohol use (from mean 59% to 76%), and sexual behavior (from mean 61% to 75%). Additionally, the average percentage of adolescents offered brief counseling by their clinicians increased significantly after training in the areas of seatbelt use (from mean 36% to 51%), helmet use (from mean 25% to 43%), and sexual behavior (from mean 42% to 58%). Improvement after the training in brief counseling for tobacco use was marginally significant (from mean 60% to 69%) and for alcohol use was not significant, although there was an increase. Clinicians also significantly increased their discussion of the limits of confidentiality with their adolescent patients after the training workshops (from mean 32% to 45%).

Conclusions.  This study offers strong support for the efficacy of skills-based training for primary care providers as a method for increasing screening and counseling practices with adolescents. The present findings suggest that with appropriate skills-based training, practicing clinicians can implement several of the national guidelines that direct them to provide preventive services for multiple behaviors in a routine outpatient visit. Screening and counseling in these visits are important in the early identification, detection, and prevention of behaviors associated with the primary adolescent morbidities and mortalities. Thus, enhancing the delivery of clinical preventive services is an important step in the prevention of untoward health outcomes for youth.  Key words:  primary care, provider training, preventive services, adolescents, risk behavior.

The majority of adolescent morbidity and mortality can be attributed to preventable risk factors.¹ These include unhealthy behaviors such as substance use and abuse, unsafe sexual behavior practices, and risky vehicle use.^2-4 Often, these risky behaviors remain unidentified until an adolescent develops health problems, such as a sexually transmitted disease, that require acute medical care.^1,5-7 Primary care providers for adolescents are in a unique position to screen for risky behaviors and to provide anticipatory guidance and brief counseling.

To facilitate the provision of preventive services to adolescents, national recommendations have been developed and disseminated as a guide for physicians, including the American Medical Association's Guidelines for Adolescent Preventive Services,⁸ the Maternal and Child Health Bureau's Bright Futures,⁹ the American Academy of Pediatrics' Health Supervision Guidelines,¹⁰ and the Department of Health and Human Services' The Clinician's Handbook, Put Prevention Into Practice.¹¹ In general, these guidelines recommend that all adolescents have an annual, confidential preventive services visit during which time primary care providers screen, educate, and counsel their adolescent patients on a range of issues that affect adolescent health. However, despite the dissemination of guidelines, evidence that adolescents trust health providers, and findings that adolescents are willing to talk with providers about topics outlined in the guidelines,^12-16 implementation of clinical preventive services remains far below recommended levels.^12,17-20

Green and colleagues'^21,22 Precede-Proceed model provides a framework that organizes many factors likely to contribute to the low rates of implementation of clinical preventive services. The framework includes predisposing factors (eg, attitudes, knowledge), enabling factors (eg, perceived self-efficacy, skills), and reinforcing factors (eg, positive reinforcement and support) that influence providers' behavior.^21-23 Predisposing factors include the necessary attitudes and motivation to perform a behavior; enabling factors include the competence, skills, and resources necessary to perform the behavior; and reinforcing factors are those that support or reward the behavior.^22,23 Training clinicians in the delivery of preventive services for adolescents may address many of these factors through the provision of education, skills, and opportunities to increase their perceived competence.^24,25 Nevertheless, primary care providers who see adolescent patients have reported insufficient training as the most significant barrier to their delivery of preventive health care to adolescents.²⁶ Furthermore, there is a paucity of studies to determine efficacious training intervention approaches, particularly with adolescents.

In a review of interventions to improve practices of health professionals, Oxman and colleagues²⁴ reported that comprehensive approaches to training that use intrasession practice and rehearsal are more effective than is distribution of materials or didactic sessions alone. However, training interventions often rely on lectures and distribution of materials.^25,27

Few investigators have evaluated training interventions that use both didactic and interactive methods and focus on clinicians who serve pediatric patients. The limited studies that have been conducted focus on training medical students and residents and typically address only a single category of risky behavior. For example, Klein and colleagues²⁸ implemented a smoking cessation curriculum for pediatric residents that included didactic seminars and small group exercises. Both residents' self-reports and exit interviews with patients' parents revealed few significant differences in screening and counseling practices between residents who participated in the training and those who did not participate.

Kokotailo and colleagues demonstrated the efficacy of a training program focused on pediatric residents' screening and counseling of adolescent patients regarding alcohol and other drug use. The training included education and role-plays about assessment and interviewing skills. Analysis of videotapes of clinician interviews with confederate patients yielded significant improvement for the intervention group compared with the control group in use of screening techniques and interviewing skills.²⁹ This study suggests that a more comprehensive approach to training that includes skill enhancement may be an effective method for improving practices of clinicians seeing adolescents. However, this study included only pediatric residents, was limited to a focus on substance use, and relied on analyses of clinician interviews with confederate patients to evaluate clinicians' performance.

As illustrated by these examples, another important limitation to most studies on clinician training is that they focus on a single category of behavior. Given the high rates of covariation among risk behaviors,^5,30-33 and that guidelines recommend screening and counseling in multiple risk areas, it is important to train clinicians to discuss a range of risk behaviors. In addition, it has been shown that discussion of confidentiality and the limits of confidentiality increase disclosure to clinicians about risky behaviors in a medical visit.³⁴ However, discussion of confidentiality rarely is emphasized in training interventions nor evaluated as an outcome of training.

In 1 recent study, some of these limitations were addressed. Sanci and colleagues³⁵ evaluated the impact of a 12-hour training program for family physicians on delivery of health care to adolescent patients. Physician self-report and videotapes of simulated patients revealed that physicians improved in their discussion of confidentiality, comfort, and self-perceived knowledge and skill. An important next step is to evaluate the efficacy of training practicing clinicians to screen and counsel on multiple risky behaviors and to use reports from their actual adolescent patients as a measure of clinician behavior in a pediatric visit.

The present study is an evaluation of skills-based training workshops for primary care providers in outpatient pediatric clinics within a health maintenance organization (HMO). The training workshops were a central component of a larger model for implementing clinical preventive services with adolescents in the health care setting, drawing on the Precede-Proceed framework.^21,22,36 The goal of the training workshops was to enhance clinicians' screening and brief counseling of adolescents regarding a set of important risky behaviors and to improve their discussion of confidentiality. Bandura's Social Cognitive Theory ^37,38 guided the development of the training workshops, which emphasized knowledge, self-efficacy, and skills. Our primary hypothesis was that clinician screening, counseling, and discussion of confidentiality with adolescents during routine well visits would increase after the provision of skills-based training.

	METHODS

Top Abstract Methods Results Discussion Conclusion References

Design

The present study is a pre-post evaluation of the effectiveness of training workshops to increase clinician screening, counseling, and discussion of confidentiality practices with adolescent patients during routine physical examination visits. Independent adolescent reports of clinician screening and counseling practices were obtained from 2 separate groups of adolescents who attended well-visits with their clinicians: 1 group during a pretraining baseline period and the second after the training during the posttraining period. The independent reports completed by adolescents during each of the time periods served as the basis of the analyses of change in clinician practices from the pretraining to the posttraining period. All procedures were approved by the internal review boards at the University of California, San Francisco and the participating HMO.

Clinician Participants

We conducted training workshops in 3 outpatient pediatric clinics within a large group model HMO. These clinics were selected based on their provision of care to large numbers of adolescents (eg, 3000 visits per year of 14-year-old patients) and their agreement to be part of a longitudinal study on clinical preventive services with adolescents. In addition, because of an interest in studying general pediatric care, we selected general pediatric clinics that did not have separate active teen clinics within them. Seventy-nine clinicians from the 3 outpatient pediatric clinics were invited to attend the training workshops. All 79 clinicians signed informed consent forms indicating their agreement to attend the training sessions and participate in the study. The chiefs of pediatrics for each clinic encouraged and facilitated clinician attendance. The clinicians were released from their regular clinic schedules, and the HMO was reimbursed for the clinicians' release time by study funds. The clinicians also received continuing medical education credits. Seventy-five (95%) clinicians participated in the workshops, and the remaining 4 had previous commitments (eg, vacations, maternity leave) and were trained at a later date.

The analyses presented here include clinicians who attended the scheduled training workshops and who saw at least 1 adolescent study patient before the training and 1 adolescent study patient after the training, resulting in a final N of 63 clinicians. Eighty-four percent of the 63 clinicians were pediatricians, and 16% were nurse practitioners. Sixty-two percent of all clinicians were female. The average number of years since training was 17 (standard deviation [SD] 6.2; range: 2-37) and none of the clinicians were board-certified in adolescent medicine.

Assessment of Clinician Practices

Adolescent Reporters Reports were obtained from 2 samples of adolescents regarding their clinician's behavior after their physical examinations. The first sample had seen their clinicians before the training workshops and the second sample had seen their clinicians after the workshops. Participants were recruited using lists obtained from the clinics of adolescents with scheduled physical examination or sports physical visits. Parents and adolescents were first sent a letter describing the study that was followed up by a telephone recruitment call. Approximately 92% of adolescents (and parents) contacted agreed to participate in the study. A total of ~71% of those who agreed to participate completed their study visits and the questionnaire. Informed consent was obtained from all adolescents and their parent or legal guardian.

Measure of Clinician Practices The assessment measure of clinician screening and counseling practices is based on independent individual reports from adolescents. This type of measure yields a unique appraisal of clinician practices that is free of the confounding influences of clinician self-report and social desirability biases. Few studies have included independent adolescent reports of clinician practices. Klein and colleagues³⁹ found that adolescent reports were valid indicators of actual clinicians' behavior with their adolescent patients.

Training Workshops

The training workshops were developed using Bandura's Social Cognitive Theory^37,38 to address multiple factors that influence implementation of preventive services for adolescents. Social Cognitive Theory emphasizes the importance of knowledge, attitudes, self-efficacy, and skills in creating behavior change. The training workshops instructed primary care providers in a brief office-based intervention and addressed these variables as mediators of clinician behavior change.^22,40

We assembled an advisory panel of adolescent medicine specialists from the study HMO to review our plan for the workshops. Workshops were conducted by a panel comprised of all authors on this study. In addition, we used educational theater actors from the HMO to portray adolescents, and to participate in demonstration and practice role-plays. Training manuals were developed to include information on each topic, lists of related resources in the clinic and local community, relevant empirical articles, and a bibliography.

The workshops were divided into two 4-hour sessions. Each workshop was repeated to facilitate the attendance of as many clinicians as possible. As suggested by the review of effective interventions for health care professionals,²⁴ the workshops contained 4 components: 1) didactic; 2) discussion; 3) demonstration role plays; and 4) interactive role plays. The first didactic component included presentations on statistics for adolescent risk behaviors and health in the United States, adolescent development, the role of primary care clinicians in reducing risky behaviors leading to adolescent morbidities and mortalities, a general framework for interviewing adolescents, confidentiality, useful screening questions and brief counseling messages both for adolescents engaging in risky behaviors and those not engaging, and prioritizing in the clinical visit. The second component was comprised of interactive question and answer sessions and discussion after the presentations. The third component involved demonstration role-plays with adolescent patients. Role-plays demonstrated a clinical visit with an adolescent patient and began with a discussion of confidentiality, then moved to brief screening, and counseling on the 5 target risk behaviors. The fourth component involved group practice wherein an actor used case scenarios to role-play an adolescent patient with a clinician, and 2 facilitators along with the other clinicians in the group offered immediate feedback. The training was videotaped for clinicians who were unable to attend.

Analysis Plan

We conducted paired t test analyses for each set of mean scores, comparing pretraining to posttraining changes in behavior. There were 3 sets of outcome variables: 1) screening for helmet and seatbelt use, tobacco use, alcohol use, and sexual behavior, and an aggregate screening score across target areas; 2) counseling on helmet and seatbelt use, tobacco use, alcohol use, and sexual behavior, and an aggregate counseling score across areas; and 3) general discussion of confidentiality and discussion of the limits of confidentiality.

Three sets of analyses were conducted. The first set includes paired t tests evaluating change in clinician screening from pretraining to posttraining, both for the specific risk areas and the aggregated screening score. The second set of analyses includes paired t tests evaluating change in clinician brief counseling from pretraining to posttraining, both for the specific risk areas and the aggregated counseling score. The third set includes paired t tests evaluating pretraining to posttraining changes in the 2 aspects of clinician discussion of confidentiality.

	RESULTS

Top Abstract Methods Results Discussion Conclusion References

Screening

Paired t tests indicated that, based on adolescent reports, clinician screening practices increased significantly from pretraining to posttraining in each of the 5 risk areas: helmet use (from mean 27% to 45%); seatbelt use (from mean 38% to 56%); tobacco use (from mean 64% to 76%); alcohol use (from mean 59% to 76%); and sexual behavior (from mean 61% to 75%) (Table 1). The paired t test also indicated that the aggregated screening score across all 5 risk areas increased significantly from pretraining to posttraining (mean pretraining 50%; SD 0.27 to mean posttraining 65%; SD 0.23; t = 4.59; P < .000).

Brief Counseling

Clinicians' brief counseling increased from the pretraining to posttraining period. Specifically, t tests of change in clinicians' counseling practices across all adolescents showed significant increases for clinicians' encouragement to wear a helmet (from mean 25% to 43%) and encouragement to use a seatbelt (from mean 36% to 51%). Counseling adolescents who were not sexually active to delay sexual activity increased significantly (from mean 42% to 58%) (Table 2). Counseling nontobacco users to continue not to use tobacco increased marginally (from mean 60% to 69%) and counseling nondrinkers to not begin drinking increased, but the difference from pretraining to posttraining was not significant (51% to 61%). The aggregate score for counseling across all 5 risk areas increased significantly from pretraining to posttraining (mean pretraining: 43%; SD: 0.26 to mean posttraining: 56%; SD: 0.26; t = 3.95; P < .000).

Confidentiality

The 2 components of discussion of confidentiality were analyzed separately: discussion of general confidentiality (ie, information that would not be disclosed to parents) and discussion of the limits of confidentiality (ie, information that would be disclosed to parents if necessary). Pretraining to posttraining increase in clinician discussion of general confidentiality occurred in the expected direction, but was not significant (from 44% to 52%). However, clinician discussion of the limits of confidentiality increased significantly (mean pretraining: 32%; SD: 0.31 to mean posttraining: 45%; SD: 0.32; t = 2.29; P < .03).^a

	DISCUSSION

Top Abstract Methods Results Discussion Conclusion References

In this study, we examined the efficacy of skills-based training workshops designed to enhance clinicians' delivery of preventive services to their adolescent patients in a group model HMO. These findings support the hypothesis that clinicians' screening, counseling, and discussion of confidentiality with their adolescent patients increase significantly after skills-based training workshops. These training workshops offer a promising approach to facilitate the implementation of national guidelines that recommend screening and counseling adolescents on a range of risky behaviors.

Training Practicing Clinicians

The training workshops in the present study were provided to practicing clinicians in a busy health care environment. Studies of education and training for clinicians typically focus on medical students and residents,^27,28,41,42 despite the need for studies of training of clinicians already in practice. In the present study, although clinicians had been trained an average of 17 years ago, they attended the trainings and modified their practices significantly after our trainings. Indeed, we trained 95% of eligible clinicians, and posttraining clinician evaluations revealed positive responses to the training and retention of the material covered (eg, clinician reports averaged 4.7 out of 5 on meeting objectives of the trainings). Clinicians therefore were exposed to the full "dose" of trainings and played an active role in the workshops.

This high rate of attendance and involvement may have been attributable, in part, to our efforts to collaborate with the clinicians, chiefs of the departments, and administrators in each of the clinics throughout the process of planning, scheduling, and implementing the trainings. In addition, clinicians seemed highly motivated to learn new skills to facilitate their work with their adolescent patients, perhaps because of the introduction of this HMO's preventive services guidelines, along with increased awareness of high rates of adolescent morbidity and mortality attributable to risky behavior. Thus, the present findings offer support for the integration of continuing education, such as skills-based trainings, for clinicians practicing within the health care system.

The trainings were conducted in a group model HMO where such trainings were feasible and the screening and counseling methods could be incorporated into clinicians' practices. In other health care environments, modifications may be necessary in the logistics of conducting the trainings and in the methods of incorporating prevention into the clinical settings. It is noteworthy that national guidelines are recommending the integration of preventive services and it will be important to explore methods of effectively conducting trainings to implement these services in a wide range of practice settings.

Multiple Behavior Approach

In this study, we targeted multiple behaviors rather than focusing solely on 1 target risk behavior. Studies of clinician trainings typically target a single risk area such as tobacco use,^28,43-45 alcohol and drug use,²⁹ or sexually transmitted diseases,⁴⁶ despite the literature suggesting that there is high covariation among various risk behaviors.^5,30-33 Given the challenges imposed by the national guidelines to screen and counsel on many behaviors in clinicians' visits with adolescents, we are encouraged by findings from the present study that clinicians increased their queries to teens in all risky behaviors during their brief office visits. In our trainings, we emphasized methods to efficiently address multiple areas of risk in a routine physical examination visit. These results suggest that the trainings facilitated clinicians' efficacy to incorporate preventive services into their actual visits with adolescents and to give attention to a range of risky behaviors, including sensitive topics such as sexual behavior and substance use.

Variation Across Risky Behaviors

The workshops were associated with significant increases in overall clinician screening and counseling of adolescent patients for risky behaviors. When examined individually by target risk behaviors, clinicians' screening practices improved significantly from the pretraining to posttraining periods for all 5 behaviors. Brief counseling, however, increased significantly for safety and sexual behavior, but not for substance use. Independent adolescent reports of their clinician visits is considered a stringent measure of clinician provision of services to their adolescent patients in an actual patient encounter,²⁸ suggesting there is variation in clinicians' counseling practices.

The most marked improvement in clinicians' screening and brief counseling was in the target areas of helmet and seatbelt use. There is a paucity of research on clinician practices regarding safety, and the limited research suggests that it is difficult to alter clinicians' practices in the area of injury prevention.⁴⁷ However, relative to our other target areas, helmet and seatbelt use screening and counseling are fairly straightforward issues that do not require extensive assessment or clinical sensitivity.

We obtained significant improvement in clinicians' screening for tobacco and alcohol use; marginally significant changes in counseling for tobacco use, and an increase in counseling for alcohol use that was not statistically significant. The counseling variable reflected brief counseling for teens who were not currently smoking or drinking and emphasized encouragement to continue refraining from those behaviors. It is plausible that some clinicians did not introduce a discussion of the complex topic of substance use because of lack of time and resources. Although we provided lists of resources in the clinic and local community, additional tools and resources such as health risk screening questionnaires, counseling prompts, and support staff may be needed to increase clinicians' comfort initiating a discussion about substance use.^24,36,48 Thus, training alone was sufficient to improve screening for substance use, but additional system-level interventions may be needed for consistent, ongoing improvement in counseling.

In the target area of sexual behavior, clinicians' screening of all adolescents and brief counseling of nonsexually active adolescents to delay onset of sexual activity improved significantly after the training workshops. Little research has been conducted to improve clinicians' screening and counseling regarding adolescents' sexual behavior. The limited studies that address screening and counseling on sexual behavior typically focus on at-risk or high-risk patients.^46,49 The present study is unique in its primary prevention focus on adolescent patients, including those who had not begun to engage in risky sexual behavior. We recruited our study patients from prescheduled well-visits, and the majority were not sexually active. The workshops therefore emphasized training on preventive counseling, such as delaying onset of sexual activity for adolescents as a method to prevent sexually transmitted diseases, human immunodeficiency virus infection, and pregnancy, as well as counseling for sexually active adolescents that emphasized options to maximize their health and safety. Preventive and early interventions and health promotion are both important in reducing negative behavioral and health outcomes⁴¹ and the present findings suggest that training is 1 useful method to improve such interventions.

Finally, discussion of the limits of confidentiality increased significantly after the trainings, whereas general discussion of confidentiality did not show a significant increase. Preventive guidelines recommend that clinicians discuss confidentiality in an adolescent visit,^8,9 yet clinicians generally are not instructed in how to discuss this topic with their adolescent patients. In the present study, clinicians increased their discussions about the particular circumstances under which they would need to disclose information to an adolescents' parent. Perceiving that a visit is confidential has been found to be a significant correlate of adolescents' willingness to disclose personal information.³⁴ Informing adolescents about the limited circumstances under which information would be disclosed is an important component of a confidentiality discussion. Ford and colleagues³⁴ demonstrated that discussion of the limits of confidentiality was not associated with reductions in adolescent patients' willingness to disclose information.

In the current study, although there was a positive increase in discussion about issues that would not be disclosed, it is interesting that the improvement was not significant, whereas, the increase in discussion of the limits of confidentiality was significant. This may in part have resulted because the pretraining rate of discussion of general confidentiality was substantially higher than the rate of discussion of the limits of confidentiality, thus creating a ceiling effect. Alternatively, our training workshops emphasized mandatory reporting and clarification of the limits of confidentiality. Future trainings should equally emphasize the importance of explaining to patients the type of information that will remain confidential, as well as the type of information that will be disclosed.

Limitations and Future Directions

Although these findings are promising, it is important to note limitations to the methodology. Although we used an internal comparison, we did not use a randomized, controlled design to evaluate our training workshops. Also, the clinics we selected to participate in our study of the implementation of preventive services may not be representative of all clinics within the HMO, thus limiting generalizability. The chiefs of pediatrics and clinicians in the clinics were interested in improving their services for adolescents and had agreed to participate in the training workshops and longitudinal study.

Finally, because this study was part of a larger study in which additional system interventions were later added, the data are limited to immediate follow-up within 3 months posttraining. Future studies are needed to examine the longitudinal effects of the trainings, using ongoing feedback, reinforcement, and booster sessions that are important in maintaining effects of training.²⁴

	CONCLUSION

Top Abstract Methods Results Discussion Conclusion References

This study offers strong support for the efficacy of skills-based workshops for primary care clinicians to improve screening and counseling practices with adolescents in a health care setting. The enhancement of screening across all 5 target risk behaviors suggests that training workshops alone significantly improve screening across a broad array of behaviors. Screening for risky behaviors is a critical component of an outpatient pediatric visit for adolescents wherein clinicians may identify and detect risky behaviors early. Provision of counseling in complex areas such as substance use may require additional system-level interventions to address organizational/system factors that influence service delivery, such as a need for charting tools and additional staff resources. However, skills-based workshops alone are associated with improvement in screening and counseling practices and may represent an important component of a larger system intervention to enhance overall delivery of clinical preventive services. Ongoing research is being conducted to examine a larger system model, including adolescent screening questionnaires, clinician prompts, forms, and additional staff resources, to improve the delivery of preventive services. Improving such services is an essential step in the prevention of untoward behavioral and health outcomes for youth.

	ACKNOWLEDGMENTS

This research was supported primarily by Grant Number 96-42 from The California Wellness Foundation. Additional support was provided through grants from the Maternal and Child Health Bureau, Health Resources and Services Administration, Department of Health and Human Services, the Interdisciplinary Leadership Training in Adolescent Health (Project #MCJ-000978), the Policy Information and Analysis Center for Middle Childhood and Adolescence (Project #MC-00023), and the National Adolescent Health Information Center (Project #MCJ-06A80).

We thank Susan Millstein, PhD, for her contributions to the design of the study and of the training workshops, and Carol Bandura Cowley, MSN, CPNP, for her assistance in conceptualizing the training workshops, contributions to the development of materials, and her helpful insights. We also appreciate the technical assistance and consultation in developing the trainings of Art Elster, MD, Missy Fleming, PhD, Janet Gans-Epner, PhD, and Pat Levenberg, PhD, of the Department of Adolescent Health at the American Medical Association. We thank Anne Claiborne for her skillful assistance in developing the materials for the trainings, organizing the workshops, and preparing this manuscript. We also appreciate the assistance of Scott Burg in preparing materials for the trainings and Denise Mailloux in the final preparation of the manuscript. Finally, we are grateful to the clinicians in the 3 Kaiser Permanente Northern California Clinics who participated in the training workshops and demonstrated an interest in improving preventive health care for adolescents.

	FOOTNOTES

^a We also conducted analyses using adolescent reports across clinician visits, not associating visits with individual clinicians, to obtain the rates of screening and counseling practices across the 3 clinics. ² analyses showed that the majority of results were the same as results using mean scores for each clinician, suggesting a similar pattern of findings.

Received for publication Jul 13, 2000; accepted Oct 4, 2000.

This work was presented in part in a plenary session at the annual meeting of the Society for Adolescent Medicine; March 23, 2000; Washington, DC, and at the annual meeting of the Society for Prevention Research; June 1, 2000; Montreal, Quebec.

Address correspondence to Julie L. Lustig, PhD, University of California-San Francisco, School of Medicine, Department of Pediatrics, Division of Adolescent Medicine, 3333 California St, Suite 245, Box 0503, San Francisco, CA 94143-0503. E-mail: jlustig{at}itsa.ucsf.edu

	ABBREVIATIONS

HMO, health maintenance organization; SD, standard deviation; AROV, Adolescent Report of the Visit.

	REFERENCES

Top Abstract Methods Results Discussion Conclusion References

1. Ozer EM, Brindis CD, Millstein SG, Knopf DK, Irwin CE Jr. America's Adolescents: Are They Healthy? San Francisco, CA: University of California, San Francisco, National Adolescent Health Information Center; 1998
2. Centers for Disease Control and Prevention. CDC Wonder Online Database, Mortality Data Set (Compressed). Atlanta, GA: Centers for Disease Control and Prevention. Available at: http://wonder.cdc.gov/mortsql.shtml. Accessed July 2000
3. Kann L, Warren CW, Harris WA, et al. Youth Risk Behavior Surveillance-United States, 1993. Atlanta, GA: Surveillance and Evaluation Research Branch, Division of Adolescent and School Health. Centers for Disease Control and Prevention; 1996
4. National Health Interview Survey. Research for the 1995-2004 Redesign. Series 2: Data Evaluation and Methods Research. Publ. No. 126. Hyattsville, MD: US Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics; 1999
5. Elliott DS. Health-enhancing and health-compromising lifestyles. In: Millstein SG, Petersen AC, Nightingale EO, eds. Promoting the Health of Adolescents. New York, NY: Oxford University Press; 1993:119-145
6. Jessor R, Donovan JE, Costa FM. Beyond Adolescence: Problem Behavior and Young Adult Development. Vol. 15. New York, NY: Cambridge University Press; 1991:312
7. Riley A, Leaf P, Horwitz S, Lustig J, Vaden-Kiernan M. Detection of psychosocial problems among children with significant medical problems. Presented at: Annual International Conference on Mental Disorders in the General Health Care Sector; October 20, 1994; McLean, VA
8. Elster AB, Kuznets NJ, American Medical Association. AMA Guidelines for Adolescent Preventive Services (GAPS): Recommendations and Rationale. Baltimore, MD: Williams & Wilkins; 1994:191
9. Green M, Palfrey JS. Bright futures: guidelines for health supervision of infants, children and adolescents. Arlington, VA: National Center for Education in Maternal and Child Health; 2000
10. Stein ME. Health Supervision Guidelines. 3rd ed. Elk Grove Village, IL: American Academy of Pediatrics; 1997
11. The Challenge and Potential For Assuring Quality Health Care for the 21st Century. Washington, DC: Domestic Policy Council (US), United States Department of Health and Human Services; 1999
12. Blum RW, Beuhring T, Wunderlich M, Resnick MD Don't ask, they won't tell: the quality of adolescent health screening in five practice settings. Am J Public Health 1996; 86:1767-1772 [Abstract/Free Full Text]
13. Joffe A, Radius S, Gall M Health counseling for adolescents: what they want, what they get, and who gives it. Pediatrics 1988; 82:481-485 [Abstract/Free Full Text]
14. Levenson PM, Morrow JR, Morgan WC, Pfefferbaum BJ Health information sources and preferences as perceived by adolescents, pediatricians, teachers and school nurses. J Early Adolesc 1986; 6:183-195
15. Malus M, LaChance PA, Lamy L, Macaulay A, Vanasse M Priorities in adolescent health care: the teenager's viewpoint. J Fam Pract 1987; 25:159-162 [Medline]
16. Steiner BD, Gest, KL Do adolescents want to hear preventive counseling messages in outpatient settings? J Fam Pract 1996; 43:375-381 [Medline]
17. Ellen JM, Franzgrote M, Irwin CE Jr, Millstein SG Primary care physicians' screening of adolescent patients: a survey of California physicians. J Adolesc Health 1998; 22:433-438 [CrossRef][Medline]
18. Franzgrote M, Ellen JM, Millstein SG, Irwin CE Jr Screening for adolescent smoking among primary care physicians in California. Am J Public Health 1997; 87:1341-1345 [Abstract/Free Full Text]
19. Igra V, Millstein S Current status and approaches to improving preventive services for adolescents. JAMA 1993; 269:1408-1412 [Abstract/Free Full Text]
20. Halpern-Felsher BL, Ozer EM, Millstein SG, Preventive services in a health maintenance organization: how well do pediatricians screen and educate adolescent patients? Arch Pediatr Adolesc Med 2000; 154:173-179 [Abstract/Free Full Text]
21. Green LW, Eriksen MP, Schor EL Preventive practices by physicians: behavioral determinants and potential interventions. Am J Prev Med 1988; 4:101-110 [Medline]
22. Green LW, Kreuter MW. Health Promotion Planning: An Educational and Ecological Approach. 3rd ed. Mountain View, CA: Mayfield Publishing Co; 1999
23. Walsh JM, McPhee SJ A systems model of clinical preventive care: an analysis of factors influencing patient and physician . Health Educ Q 1992; 19:157-175 [Medline]
24. Oxman AD, Thomson MA, Davis DA, Haynes RB No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ 1995; 153:1423-1431 [Abstract]
25. Davis DA, Taylor-Vaisey A Translating guidelines into practice: a systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical practice guidelines. Can Med Assoc J 1997; 157:408-416 [Abstract]
26. Blum RW, Bearinger LH Knowledge and attitudes of health professionals toward adolescent health care. J Adolesc Health 1990; 11:289-294
27. Gemson DH, Ashford AR, Dickey LL, Putting prevention into practice: impact of a multifaceted physician education program on preventive services in the inner city. Arch Intern Med 1995; 155:2210-2216 [Abstract/Free Full Text]
28. Klein JD, Portilla M, Goldstein A, Leininger L Training pediatric residents to prevent tobacco use. Pediatrics 1995; 96:326-330 [Abstract/Free Full Text]
29. Kokotailo PK, Langhough R, Neary EJ, Matson SC, Fleming MF Improving pediatric residents' alcohol and other drug use clinical skills: use of an experiential curriculum. Pediatrics 1995; 96:99-104 [Abstract/Free Full Text]
30. Irwin CE Jr, Igra V, Eyre S, Millstein S Risk-taking behavior in adolescents: the paradigm. Ann N Y Acad Sci 1997; 817:1-35
31. Hawkins JD, Catalano RF, Miller JY Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: implications for substance abuse prevention. Psychol Bull 1992; 112:64-105 [CrossRef][Medline]
32. Lindberg LD, Boggess S, Porter L, Williams S. Teen Risk Taking: A Statistical Portrait. Washington, DC: The Urban Institute; 2000
33. Millstein SG, Irwin CE Jr, Adler NE, Cohn LD, Kegeles SM, Dolcini MM Health-risk behaviors and health concerns among young adolescents. Pediatrics 1992; 89:422-428 [Abstract/Free Full Text]
34. Ford CA, Millstein SG, Halpern-Felsher BL, Irwin CE Jr Influence of physician confidentiality assurances on adolescents' willingness to disclose information and seek future health care: a randomized controlled trial. JAMA 1997; 278:1029-1034 [Abstract/Free Full Text]
35. Sanci LA, Coffey CM, Veit FC, Evaluation of the effectiveness of an educational intervention for general practitioners in adolescent health care: randomised controlled trial. BMJ 2000; 320:224-30 [Abstract/Free Full Text]
36. Lustig JL, Ozer EM, Adams SH, et al. Addressing adolescent risk behaviors in health care settings: developing a clinic based preventive intervention. Presented at: Annual Conference of the Society for Prevention Research; June 22, 1999; New Orleans, LA
37. Bandura A. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs, NJ: Prentice Hall; 1986
38. Bandura A. Self-efficacy: The Exercise of Control. New York, NY: WH Freeman; 1997
39. Klein JD, Graff CA, Santelli JS, Hedberg VA, Allan MJ, Elster AB Developing quality measures for adolescent care: validity of adolescents' self-reported receipt of preventive services. Health Serv Res 1999; 34:391-404 [Medline]
40. Lawrence RS Diffusion of the US preventive services task force recommendations into practice. J Gen Intern Med 1990; 5:S99-S103 [Medline]
41. Millstein SG, Petersen AC, Nightingale EO. Promoting the Health of Adolescents: New Directions for the Twenty-first Century. New York, NY: Oxford University Press; 1993
42. Cohen SJ, Halvorson HW, Gosselink CA Changing physician behavior to improve disease prevention. Prev Med 1994; 23:284-291 [CrossRef][Medline]
43. Cummings SR, Coates TJ, Richard RJ, Training physicians in counseling about smoking cessation: a randomized trial of the "Quit for Life" program. Ann Intern Med 1989; 110:640-647
44. Kottke TE, Solberg LI, Brekke ML, Conn SA, Maxwell P, Brekke MJ A controlled trial to integrate smoking cessation advice into primary care practice: doctors helping smokers, Round III. J Fam Pract 1992; 34:701-708 [Medline]
45. Strecher VJ, O'Malley MS, Villagra VG, Can residents be trained to counsel patients about quitting smoking? Results from a randomized trial. J Gen Intern Med 1991; 6:9-17 [Medline]
46. Rabin DL, Boekeloo BO, Marx ES, Bowman MA, Russell NK, Willis AG Improving office-based physicians' prevention practices for sexually transmitted diseases. Ann Intern Med 1994; 21:513-519
47. Hansen K, Wong D, Young PC Do the Framingham Safety Surveys improve injury prevention counseling during pediatric health supervision visits? Pediatrics 1996; 129:494-498 [CrossRef]
48. Cabana MD, Rand CS, Powe NR, Why don't physicians follow clinical practice guidelines? A framework for improvement . JAMA 1999; 82:1458-1465
49. Ward J, Sanson-Fisher R Does a 3-day workshop for family medicine trainees improve preventive care? A randomized control trial. Prev Med 1996; 25:741-747 [CrossRef][Medline]