 |
|
 |
|
 |
 |
|
 |
 |
 |
 |
 |
 |
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ARTICLE |
a Pediatric and Adolescent Health Research Center, Floating Hospital for Children at Tufts-New England Medical Center, Boston, Massachusetts
b American Academy of Pediatrics, Elk Grove Village, Illinois
 |
ABSTRACT |
|---|
 TOP ABSTRACT METHODSRESULTSDISCUSSIONREFERENCES |
|---|
DESIGN. A modified electronic Delphi process was used to gain consensus among 50 national multidisciplinary violence-prevention experts. Participants were unaware of other participants' identities.
METHODS. The process consisted of 4 serial rounds of inquiry beginning with a broad open-ended format for the generation of anticipatory guidance and screening topics across 5 age groups (infant, toddler, school age, adolescent, and all ages). Each subsequent round narrowed the list of topics toward the development of a manageable set of essential topics for screening and counseling about positive youth development and violence prevention.
RESULTS. Forty-seven unique topics were identified, spanning birth to age 21 years. Topics cover 4 broad categories (building blocks): physical safety, parent centered, child centered, and community connection. Participants placed topics into their developmentally appropriate visit-based schedule and made suggestions for an appropriate topic reinforcement schedule. The resulting schedule provides topics for introduction and reinforcement at each visit.
CONCLUSIONS. The Delphi technique proved a useful approach for accessing expert opinion, for analyzing and synthesizing results, for achieving consensus, and for setting priorities among the numerous anticipatory guidance and assessment topics relevant for raising resilient, violence-free youth.
Key Words: anticipatory guidance • child development • Delphi technique • pediatrics • violence prevention
Abbreviations: AAP—American Academy of Pediatrics • TFOV—Task Force on Violence • VIPP—Violence Intervention and Prevention Program
The American Academy of Pediatrics (AAP) logo proclaims the commitment: "dedicated to the health of all children." Increasingly, the greatest threats to child health are part of what has been called the "new morbidity."1 More than a decade ago, the AAP's Committee on Psychosocial Aspects of Child and Family Health approved a policy statement advocating better preparation of pediatricians to address the behavioral and psychosocial factors that threatened child health with increasing prevalence.1 More recently, the AAP's commitment has been reaffirmed.2 In recognition of the newer morbidities, such as violence, suicide, substance abuse, and school problems, the committee concluded, "These are the morbidities that place our patients at risk. In other words, after infancy, children in the United States are more likely to die from injuries or violence and suicide than from infectious disease."2
Increasingly, pediatricians are called on to address these new morbidities, effectively and efficiently, not only through diagnosis, treatment, and referral, but through prevention as well. Preventive care is a cornerstone of pediatric practice, but the new morbidities require new approaches from physicians: "To effectively address these new morbidities, pediatricians will need a model that encompasses expanded areas of competence in child behavior, development, and family function."2
Violence ranks highly among the clear threats to child health. The World Health Organization operationally defined violence as "the intentional use of physical force or power, threatened or actual, against oneself, another person, or against a group or community, that either results in or has a high likelihood of resulting in injury, death, psychological harm, maldevelopment or deprivation."3 Numerous statistics speak to the severity and prevalence of the problem, particularly in the United States.4 The US Surgeon General identified youth violence as a major public health issue and a top priority.5
In response to this threat to child health, the AAP's Task Force on Violence (TFOV) was charged with developing policy for the AAP regarding the pediatrician's role in the prevention of youth violence. The 10-member group developed a set of recommendations regarding clinical practice, advocacy, education, and research. Among the recommendations, clinical practice should include "violence-prevention counseling and screening as early as the pediatric prenatal visit and continuing into adulthood."6
The TFOV commissioned an AAP periodic survey (No. 38) to solicit information on youth violence from the membership7; many questions from this survey were repeated in 2003.8 The results from these nationally representative samples underscored both pediatricians' awareness of the issue, as well as barriers to addressing it. More than 40% of pediatricians (44%) involved in direct patient care reported treating children with injuries resulting from community violence during the preceding 12 months. More than 70% of pediatricians believed that community-based programs could be effective, that pediatricians should be involved, and that they should screen for community violence (76%, 72%, and 71%, respectively).9
Yet, recognition of the problem and beliefs in efficacy were not enough. Most respondents did not feel confident in their abilities to identify children at risk for community violence (65%), did not feel they had adequate training in community violence (79%), and were not comfortable discussing these issues with parents (58%). Between 1998 and 2003, physician interest in incorporating violence prevention into routine care grew. The task force outlined ideas for the components to be included in a Violence Intervention and Prevention Program (VIPP), an anticipatory guidance program, if funding is secured for its development, implementation, and evaluation.6
Funded by the US Department of Justice Office of Juvenile Justice and Delinquency Prevention, VIPP was a 3-year project designed to develop systematically a program for pediatricians to incorporate the prevention of youth violence into routine health care. VIPP sought to develop a violence-prevention protocol for use in pediatric practices based on the knowledge, attitudes, and beliefs of 3 distinct groups of stakeholders: patients and their families, pediatricians, and public health experts. The central hypothesis underlying VIPP is that physicians, through assessment, counseling, and referral during routine health maintenance visits, can both promote positive youth development while they reduce injuries caused by interpersonal violence.
Although there is general acceptance that pediatricians must address injury prevention topics during health maintenance visits, it is also well established that there are numerous topics to cover, and not all of the topics are equally relevant at all stages of a child's development. Although the intent of the AAP directive is easy to accept, how best to translate that directive into practice is far more difficult. Priorities needed to be set. Similarly, developmentally appropriate guidance as to which topics are best discussed during which visits also needed to be established.
Previous work has demonstrated that written materials, reinforced by physician counseling, can be powerful tools for change in parent home literacy activities and childhood injury prevention behaviors.9,10 However, if the topics covered and the materials developed are not sufficiently compelling, then even demonstrated changes in practice do not lead to changes in family or child outcomes.11 In this particular field, earlier efforts at skills-based anticipatory guidance have demonstrated effects, particularly including the promotion of alternatives to corporal punishment.12 In a recent review, Glascoe et al13 demonstrated that the strategic use of physician counseling in the context of health care visits may be effective in leading to behavior change, particularly if the counseling addresses issues of concern to parents. Verbal counseling accompanied by personalized written information seemed to be most effective.
This article describes the results of a systematic approach used to obtain expert opinion to support the development of anticipatory guidance for the VIPP. The expert consensus described here was combined with other information14 to create the VIPP program titled "Connected Kids: Safe, Strong and Secure."15 The complete package consists of parent and adolescent educational brochures, a clinical guide for pediatricians, a Web site with supporting literature, and supporting training materials available on the Internet.
|
METHODS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
The Delphi technique has been used extensively in a variety of areas. With special relevance for the VIPP effort, a similar approach was used successfully by Cohen et al11 to assess consensus regarding the prioritization of prevention strategies for office-based injury prevention counseling for parents of children from birth to 2 years.
The Delphi method is a structured process for collecting and distilling knowledge from a group of experts through a series of questionnaires.16 As noted earlier, the process is particularly helpful in facilitating interactions among experts and for avoiding the barriers observed frequently in more traditional group discussions.17 The method allows discussion without the influence of social status, permits participants to change their minds without "losing face," and provides an efficient and parsimonious approach to combine many views into a single set of criteria in a reasonable amount of time.
The VIPP development effort used a modified Delphi process. The first round consisted of a Web-based survey administered to a carefully selected group of experts, described below. The initial survey used an open-ended format to elicit a wide range of anticipatory guidance and assessment topics across 5 age groups (infant, toddler, school age, adolescent, and all ages). The results of each round were compiled, summarized, and distributed back to the original contributors. Participants were invited to refine the topics by adding clarification, as well as by identifying strengths and weaknesses of the offerings. Through a series of 3 subsequent rounds, the list of topics was pared down to a manageable set of core topics, by consensus, by combining some topics and eliminating others.
Participant Selection
Participant selection is a critical element of successful Delphi processes. The VIPP project team developed an initial list of organizations and individuals based on expertise, the diversity of the target audiences, and the earlier work of the AAP's TFOV. Through this process, a list of 20 national groups was generated along with key contacts in each. Senior VIPP staff made direct contact and explained the purpose of the effort, as well as what involvement would entail. They asked each contact if they would be available to participate or to identify a suitable representative for their group or organization.
In addition to representatives of key stakeholder organizations and knowledgeable experts, each relevant AAP committee was asked to appoint a participant to represent their area of expertise and interest. In all, 50 experts agreed to participate.
Delphi Process
Round 1
During the first round, each expert was asked to suggest topics to be included in the new AAP violence-prevention program, topics that would become part of routine anticipatory guidance and health screening during well-child visits. No formal definition of "violence" was offered so as not to limit unnecessarily the topics generated for initial consideration. Participants were asked to list topics based on their appropriateness at different ages: infant (newborn to 1 year), toddler (1–4 years), school age (5–11 years), and adolescence (
12 years). In addition, participants were asked to list topics that would be suitable at every visit, regardless of the child's age. Responses were compiled, and only clearly duplicate topics for each age group were eliminated. If there was any doubt as to meaning, all of the original wordings were maintained to avoid subjective judgments and possible misinterpretations.
Round 2
During the second round, the VIPP team organized the results to provide more structure, consistent with the intent of the process. Within each age group, lists of anticipatory guidance and screening topics were further categorized as (1) topics pertaining to the child, (2) topics pertaining to the parent/caregiver, and (3) topics pertaining to the child's environment. Although the environment rubric was useful for the conceptual exercise, it was not formally incorporated into the final results, which were practically categorized by intended audience. Participants were asked to rate each topic on 4 dimensions using a subjective 4-point ordinal scale: (1) evidence: how strong is the evidence base demonstrating that screening and/or anticipatory guidance on this topic will have positive outcomes for children?; (2) effect size: what is the likely effect size associated with the outcome?; (3) feasibility: how feasible would it be for pediatricians to address this issue with their patients during health maintenance visits?; and (4) importance: how important is it to include this topic in this protocol?
For each topic, a rough but useful overall score was calculated by summing the ratings for evidence, effect size, and feasibility. The resulting score was then weighted by the level of importance that was assigned to the topic. The top scoring items were presented in the subsequent round.
Round 3
For the third round, the top scoring items were broken down by age group and whether they constituted "anticipatory guidance" or "screening." For this round, participants were asked to rank each item in order of importance for inclusion. Responses were again compiled, and the top scoring items were then presented in the subsequent and final round.
Round 4
The top scoring items from round 3 were placed into a sample protocol modeled after the protocol used in the Injury Prevention Program, a well-known and widely used program for unintentional injury prevention. Participants were asked to comment on the flow of topics and to note where items needed to be rearranged. The experts were also asked to review items from round 3 that did not make the final cut and which they believed should be restored to the list, along with a justification for making the change. Finally, participants were asked to review and comment on the potential effectiveness of an "existing" adolescent protocol, the American Medical Association's Guidelines for Adolescent Preventive Services.18 In the section that follows, we report the results of this iterative process.
As will become apparent in the next section, the number of participants changed from round to round. The overall success of the consensus process relied in part on participants' willingness to remain engaged. Given logistic and time constraints, not all of the participants participated in all of the Delphi rounds. However, even if they missed a round, all of the participants were invited to review and contribute to subsequent rounds. In this way, the fundamental integrity of the process was preserved, because each participant had the opportunity to challenge or augment previous contributions, even if they themselves missed a round.
|
RESULTS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
|
During the earlier Delphi rounds, we erred on the side of inclusion, retaining some topics that might seem to be redundant, lest too early synthesis eliminate shades of meaning and distinctions that might prove important later on. Over time, however, once the preliminary results were shared with our expert panel, we were more confident in combining similar topics and in making more parsimonious categorizations. These recategorizations, coupled with expert ratings of the proposed topics, resulted in the final set of topics.
During the first round, 34 respondents generated a total of 233 anticipatory guidance and 211 screening topics. The results from round 1 were compiled, categorized, and distributed to the Delphi experts. During this second round, 28 participants rated each of the anticipatory guidance topics and the screening topics. As noted earlier, the topics were grouped by vector (child, parent/caregiver, or environment), and participants were asked to rate each topic on 4 dimensions: evidence base, effect size, feasibility, and overall importance for inclusion. For each topic, an overall score was calculated and then weighted by the level of importance it was assigned. Table 2 summarizes the topics that made it to Round 3.
|
The Delphi results were vetted with an expert advisory committee that also considered results from parent and practitioner focus groups15 and were used to develop a preliminary counseling schedule. Parent and adolescent education materials and clinic guides were developed to introduce and reinforce various topics developmentally.15
|
DISCUSSION |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Any structured communication method has limitations, and the Delphi method is no exception. Perhaps chief among these limitations is how dependent the process is on the composition and active engagement of the expert panel. Although we made considerable effort to be inclusive and diverse in the identification of key stakeholder groups and in the initial identification and recruitment of expert participants, in the end, the results of the iterative process depend on the initial pool of topics generated. To the extent that important perspectives may have been overlooked, either for systematic or less obvious reasons, then the end result of the process may be incomplete.
Another critique of the method is whether experts were willing to engage actively over controversial topics to reach consensus or whether the more controversial topics dropped out because agreement could not be reached. With respect to violence prevention, we observed that the controversy on topics such as suicide, firearms, bullying, and corporal punishment is more publicly apparent than it is among the experts. Even under the veil of anonymity, our panel had no difficulty grappling with these topics head on. Subsequent vetting of the clinical guidelines and materials with physician focus groups revealed no general disagreement with even the most publicly controversial content.14
As intended, the method enabled the VIPP team to work with professionals from across the United States in an efficient time-limited manner, reducing the risk of bias introduced by intrapersonal and interpersonal dynamics. Subsequent focus group results obtained from physicians, parents, and teens, as well as preliminary results from the field tests of the final educational materials, all support the efficacy of this approach toward development (unpublished data, 2005).14
In this article we have described the systematic development of one component required to develop effective anticipatory guidance: an expert consensus on the content most likely to effectively reduce violence. This information, combined with input from practitioners and families, was used to develop a new comprehensive approach to violence prevention. Well-meaning advice developed without these key inputs may be less likely to be effective. Although it has been unusual to expend the time and resources to gather these inputs before developing a new product, the sensitivity of the topic and the extraordinary potential implicit in the relationships between families and their children's health care providers suggest that this investment will be worthwhile. Future research may help determine the relationship between the predicted efficacy of these materials and the actual experience of providers and families who will use them.
|
ACKNOWLEDGMENTS |
|---|
We want to acknowledge the following individuals who agreed to participate in the Delphi process (the listing of participants' names does not imply their endorsement of the data or conclusions): Shari Barkin, MD, FAAP; Miriam Bar-on, MD, FAAP; Lisa Cohen Barrios, DPH; Carl Bell, MD; Wyndolyn Bell, MD, FAAP; Iris Wagman Borowsky, MD, FAAP; Diane Bridgeman, PhD; Stephanie Bryn, MPH; Katherine Kaufer Christoffel, MD, MPH, FAAP; George Cohen, MD, FAAP; Larry Cohen, MSW; Angela Diaz, MD, FAAP; Denise Dowd, MD, MPH, FAAP; Jacqueline Gentry, PhD; Edward Goldson, MD, FAAP; Peter Gorski, MDA, MPA, FAAP; Rodney Hammond, PhD; Gilbert Handal, MD, FAAP; Christian Hanna, MPH; Steven Kairys, MD, MPH, FAAP; Murray Katcher, MD, PhD, FAAP, Jane Knapp, MD, FAAP; Lyndee Knox, PhD; Victor LaCerva, MD, FAAP; Phillip Leaf, PhD; William Modzeleski, MPA; Luis Montes, MD, FAAP; Robert Pallay, MD; Deborah Prothrow-Stith, MD; Frederick Rivara, MD, MPH, FAAP; Susan Rosenthal, MD, FAAP; Peter Scheidt, MD, MPH, FAAP; Roland Smart; Peter Stringham, MD; Jack Swanson, MD, FAAP; Howard Taras, MD, FAAP; Patrick Tolan, PhD; Evelyn Tomaszevski, ACSW; Daniel Webster, ScD, MPH; Constance Wieman, PhD; Elaine Williams, RN, NP; Janet Williams, MD, FAAP; Linda Wolfe, RN, BSN, MEd, CSN; Joseph Wright, MD, MPH, FAAP; and Thomas Young, MD, FAAP.
|
FOOTNOTES |
|---|
Address correspondence to Edward De Vos, EdD, Pediatric and Adolescent Health Research Center, Tufts-New England Medical Center, 750 Washington St, NEMC Box 531, Boston, MA 02111. E-mail: edevos{at}tufts-nemc.org
The authors have indicated they have no financial relationships relevant to this article to disclose.
|
REFERENCES |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
||
|
||
Home | My Pediatrics | Journal Information | Current Issue | Past Issues | Subscriptions & Services | Contact Us Click here for faster international access © 2006 American Academy of Pediatrics. All rights reserved. |
||
|
||
