OBJECTIVE: To examine whether the Safe Environment for Every Kid (SEEK) model of enhanced primary care would improve the attitudes, knowledge, comfort, competence, and behavior of child health care professionals (HPs) regarding addressing major risk factors for child maltreatment (CM).
METHODS: In a cluster randomized controlled trial, 18 private practices were assigned to intervention (SEEK) or control groups. SEEK HPs received training on CM risk factors (eg, maternal depression). The SEEK model included the parent screening questionnaire and the participation of a social worker. SEEK's impact was evaluated in 3 ways: (1) the health professional questionnaire (HPQ), which assessed HPs' attitudes and practice regarding the targeted problems; (2) observations of HPs conducting checkups; and (3) review of children's medical records.
RESULTS: The 102 HPs averaged 45 years of age; 68% were female, and 74% were in suburban practices. Comparing baseline scores with 6-, 18-, and 36-month follow-up data, the HPQ revealed significant (P < .05) improvement in the SEEK group compared with controls on addressing depression (6 months), substance abuse (18 months), intimate partner violence (6 and 18 months), and stress (6, 18, and 36 months), and in their comfort level and perceived competence (both at 6, 18, and 36 months). SEEK HPs screened for targeted problems more often than did controls based on observations 24 months after the initial training and the medical records (P < .001).
CONCLUSIONS: The SEEK model led to significant and sustained improvement in several areas. This is a crucial first step in helping HPs address major psychosocial problems that confront many families. SEEK offers a modest yet promising enhancement of primary care.
WHAT'S KNOWN ON THIS SUBJECT:
It is well established that risk factors such as maternal depression are prevalent and jeopardize children's health and development. Pediatric primary care offers an opportunity for helping address such psychosocial problems that are connected with child abuse and neglect.
WHAT THIS STUDY ADDS:
Results of this study indicate that the Safe Environment for Every Kid model helps pediatric health professionals address targeted psychosocial problems. The study is one of the first to examine change in pediatric private practices concerning the management of psychosocial problems.
Child maltreatment (CM) remains a pervasive problem, with 772 000 US children substantiated as abused or neglected in 2008.1 CM may lead to short- and long-term medical, psychological, and developmental problems, and occasionally death.1,–,3 Therefore, preventing CM is essential to optimize children's health, development, and safety.
The pediatrician's role is important in helping prevent CM.4,5 The American Academy of Pediatrics recommends that pediatricians (or child health professionals [HPs]) address CM by addressing key psychosocial risk factors, including family stress, intimate partner violence (IPV), maternal depression, and substance abuse.6,–,8Bright Futures also recommends addressing family psychosocial concerns.9 Regular checkups offer an excellent opportunity for HPs to help address such risk factors.10
One barrier to HP involvement in these sensitive areas has been a lack of training and tools.11,12 To intervene effectively, pediatricians must become knowledgeable, competent, and comfortable to address these problems.13 The SEEK (Safe Environment for Every Kid) model of pediatric primary care was developed to help HPs do so and thereby promote children's health, development, and safety.
In this study we focused on examining SEEK's impact on HPs. We hypothesized that the SEEK model would improve HPs' attitudes, knowledge, perceived competence, sense of comfort, and practice behavior pertaining to risk factors for CM. The second hypothesis was that SEEK's impact on HPs' screening behavior would be mediated via improved attitudes, knowledge, competence, and comfort. Finally, we hypothesized that female HPs would benefit more from the model than male HPs. Female physicians are reportedly more likely to engage families regarding psychosocial concerns.14
The SEEK Model
HPs in SEEK practices attended a 4-hour, small-group training session in the early evening or on a Saturday morning. The creation of SEEK was influenced by Bright Futures in recognition of the importance of viewing children's health broadly and in the context of their environment.9 The focus was on the significance of targeted problems (parental depression, major stress, substance abuse, and IPV) for children's health, development and safety, how to briefly assess identified problems, and how to initially address them, including principles of motivational interviewing. The training was conducted by our interdisciplinary team of pediatricians, a social worker, and a psychologist.
The model recognizes the need for ongoing training. Approximately every 6 months, the SEEK group received a “booster” focused on the targeted problems. Attendance and the “training dose” were variable. In addition, the project sent out periodic newsletters every 9 months. The intervention group newsletter focused on the targeted problems; the control newsletter included only project updates.
The Parent Screening Questionnaire
The parent screening questionnaire (PSQ) is a 20-item yes/no screen for the targeted psychosocial risk factors: substance abuse in the family, maternal depression, major stress, and IPV. This modified version built on our previous work, which had demonstrated adequate stability and validity.12,–,15 The PSQ was to be given to all parents bringing their child (0–5 years) for a checkup at 2, 9, 15, 24, 36, 48, and 60 months at a SEEK practice. Completing it was optional.
Parent handouts were developed, customized for each practice (ie, local resources). The model also included a Web-based directory of community resources.
A project social worker spent a half or full day per week in each SEEK practice. She was available by telephone to HPs and parents during the regular work week. HPs together with parents had flexibility regarding when to use the social worker.
Overview of the Study Design to Evaluate the SEEK Model
An important goal of the study reported in this article was to evaluate how the SEEK model influenced HPs' thinking and practice regarding the risk factors. After recruitment, practices were randomly assigned to SEEK or control groups. SEEK's impact was evaluated via multiple HP questionnaires, review of children's medical records, and direct observation of the HPs (Fig 1).
Twenty-three practices loosely associated with the University of Maryland were originally approached; 17 initially agreed to participate. They were informed that practices would be randomly assigned to SEEK or control groups, stratifying for size (small, medium, and large). The practices ranged from solo to 1 with 32 HPs. Individual HPs within practices could opt out of the study; none did. One intervention practice dropped out early in the project, leaving 7 in this group. Because of the 1 very large practice in the intervention group we added 2 control practices to have a better balanced number of HPs in each group (Fig 2). Ultimately, 18 private practices participated in the study.
Characteristics of the practices and HPs are shown in Table 1. Approximately 76% were pediatricians; the remainder were pediatric nurse practitioners. Most HPs were women and had little experience addressing the targeted problems. They served a primarily white, middle-class, well-educated population.
Our Institutional Review Board approved the study; informed consent was obtained from HPs. The SEEK model was evaluated from June 2006 through January 2009. HPs in both groups completed the health professional questionnaire (HPQ) to assess their baseline attitudes, knowledge, comfort, competence, and practice behavior concerning the targeted problems. In addition, a student observed HPs conducting 3 checkups, rating whether they addressed the targeted problems. Subsequently, HPs in SEEK practices attended a 4-hour training session to prepare them to address the problems. HPs in control practices received no special training or SEEK materials; they continued to provide standard pediatric care. HPs completed the same HPQ after 6, 18, and 36 months. Observations of HPs were repeated 24 months after the start of the SEEK model, at which time the students also reviewed the children's medical records.
Health Professional Questionnaire
The HPQ was developed to evaluate the effect of SEEK on the HPs. The HPQ has 5 vignettes, each with 7 to 12 statements assessing HPs' knowledge, attitudes, comfort level, perceived competence, and practice concerning the targeted problems. For example: “You're seeing 3-month-old SK for a checkup. He's quite fussy, and his mom seems a bit irritated with him. She says ‘He's not easy like my other two.’ You suggest how she could care for him and she responds, ‘We'll be fine!’ ” This was followed by statements such as “It's understandable that she's irritated; I would not interfere.” For each statement, HPs responded on a 5-point Likert scale (strongly agree to strongly disagree).
HPQ items were grouped conceptually into 4 topical scales (eg, substance abuse) that covered their attitudes, knowledge, comfort, competence, and practice regarding each problem. Cronbach's16 α values were adequate for most scales: α = 0.76 (depression), α = 0.80 (IPV), α = 0.80 (major stress), and α = 0.58 (substance abuse). In addition, HPQ items were grouped into themes with adequate internal consistencies: attitudes (α = 0.77), knowledge (α = 0.55), comfort (α = 0.68), competence (α = 0.74), and practice (α = 0.70). Note that an item could pertain to both the depression and attitudes scales, for example.
The HPQ included demographic information on the HP's age, gender, years in practice, and previous training and experience regarding the risk factors, as well as the proportion of their practice receiving Medicaid and who were minority.
Children's Medical Record Review
Medical records of all index children of families participating in the evaluation were reviewed to assess screening for the targeted problems and whether identified problems were addressed. The medical students followed clear guidelines for record abstraction, entering the data on computerized, standardized forms. Questions were resolved with a project pediatrician.
Observation of HPs Conducting Child Health Supervision Visits
The students observed HPs conducting checkups, 3 at baseline, and 3 toward the study end. Our goal was to make the observations as objective as possible; clear guidelines were developed for rating HP actions. For example, “How are you doing?” was not considered a screen for depression. Here too we coded whether screening occurred for targeted problems and how HPs responded to positive screens. Ratings were entered on a standardized form.
We used mixed-effects regression models (proc mixed in SAS17) to examine changes in the HPQ from baseline to 6, 18, and 36 months later, and changes in the medical records and observed behavior from baseline to 24 months later. Outcomes in the HPQ models were difference scores from baseline to 6, 18, and 36 months, respectively. Analyses controlled for the percentage of patients on Medical Assistance in each practice, the number of years each HP had been in practice, and a random effect for clinical practice, to account for the clustering of HPs within practices. In the HPQ models, a random effect for participant was included, to account for the correlation between repeated measures from the same HP.
We examined whether the impact of SEEK on screening for targeted problems was explained by its impact on intervening (“mediating”) variables.18 We considered a variable (eg, HP comfort levels) as a possible mediator if the variable was affected by the intervention, and if it was independently associated with the outcome (screening rates). To assess the degree to which it was a mediator, we examined the extent to which the association between SEEK and screening rates was reduced after controlling for the mediating variable.
Separate from mediation, we examined if key variables influenced whether, not how, SEEK affected outcomes (eg, was SEEK more effective with female versus male HPs?) This is termed “moderation.”18 We examined whether HP gender, years in practice, and discipline (pediatrician or pediatric nurse practitioner) moderated the relationship of SEEK to screening on the basis of observations and the medical records. Potential moderation was examined by including interaction terms (eg, SEEK x HP gender) in the models.
Health Professional Questionnaire
Fifty-two SEEK and 43 control HPs completed the baseline HPQ, and 43 SEEK and 40 control HPs did so at 6 months. Response rates were similar at 18 months. At 36 months, 17 SEEK and 30 control HPs completed the HPQ; these HPs had sociodemographic characteristics similar to those who did not complete 36-month HPQs, other than having fewer minority patients.
Table 2 lists the mean changes in HPQ scores between baseline and follow-up assessments. In the SEEK group, HPQ scores increased from baseline to follow-up for every domain and at every time point. The changes in controls were mostly very small or nonexistent. The impact of SEEK on change in HPs' self-reported thinking and practice is summarized in Tables 3 and 4. HPQ scale scores were standardized, making β estimates interpretable in SD units. SEEK HPs reported more improvement than controls in overall competence and comfort in addressing the targeted problems 6, 18, and 36 months after baseline. A similar pattern was observed with regard to addressing stress. Similar improvements that resulted from SEEK were found in topic scale scores, including evidence of a small effect at 36 months for addressing depression. Although the means for practice behavior suggested an impact of SEEK, small numbers limited our power.
Medical Record Review and Observed Checkups
Table 5 lists the mean percentage of times that HPs screened for targeted problems on the basis of the medical records and through direct observation, both before and during the study.
Before the study, SEEK and control HPs rarely screened for the problems. By medical record data, SEEK HPs improved by >20 percentage points in screening for each risk factor. Controls barely changed. Similarly, the observations showed increased SEEK practice screening for depression, IPV, and substance abuse, with controls barely changing. Of note, screening could have been via a PSQ or in other ways, including clinical assessment.
Mediation Between SEEK and Screening
After adjusting for families on Medical Assistance, HP years in practice, and the random effect of practice, SEEK increased screening for IPV by 18 percentage points (Fig 3). This was partially mediated by HP comfort level; the impact of SEEK dropped to 16 percentage points when comfort level was added to the model (Fig 3). Thus, increased HP comfort was responsible for some of the increased IPV screening. The effect of SEEK on screening for IPV was similarly partially mediated by the HPQ IPV scale; the impact of SEEK dropped from 18% to 16% with the HPQ IPV scale added to the model (Fig 4). We did not identify mediators for the other screening outcomes.
Moderation According to HP Gender, Years in Practice, and Discipline
SEEK HPs with >10 years in practice improved more in screening for depression (mean percentage-point change: 25%) than those with <11 years (mean percentage-point change: 20%; P = .04). A similar pattern was found for substance abuse (P = .02). Screening among controls changed very little. There was no significant difference in screening between male and female HPs.
HP discipline moderated SEEK's relationship with substance abuse (β = −0.10; P = .035) and stress screening (β = 0.14, P = .024), on the basis of medical records. SEEK pediatricians improved their screening for substance abuse (mean percentage-point change: 24%) and stress (mean percentage-point change: 26%) more than nurse practitioners (substance abuse mean percentage-point change: 20%; stress mean percentage-point change: 24%).
HPs in pediatric primary care can play an important role in helping to identify and address prevalent psychosocial problems that impair parental and family functioning and constitute risk factors for CM.15,16,19 The SEEK model of pediatric primary care was developed to help HPs play this preventive role. To our knowledge, it is the first such randomized controlled trial regarding this issue.
We hypothesized that SEEK would improve HPs' attitudes, knowledge, comfort, competence, and practice behavior in addressing the targeted risk factors. Long after the initial training, SEEK HPs reported greater improvement than controls in their overall comfort and competence concerning all the risk factors. They similarly reported improved attitudes and behavior concerning IPV, substance abuse, and major parental stress. The enduring improvement up to 36 months is especially encouraging, as early improvements found in demonstration projects are often not sustained.4
We examined the HPs' practice behavior in 2 additional ways: (1) by reviewing the children's medical records; and (2) observing the HPs conducting routine checkups. Both revealed that mothers in the SEEK group were more likely to be screened compared with controls. It is noteworthy how seldom screening occurred without the SEEK model, including while being observed. Again, it is encouraging that the improved screening was evident 2 years after the initial training.
The PSQ contributed substantially to the improved screening. This attests to the value of having a practical, brief tool as part of SEEK. It was automatically included in selected well-child visits and HPs did not need to make additional efforts.
We probed what explained the change in HPs' attitudes and behavior. Changes in HPs' comfort level, perceived competence, attitudes, or knowledge did not mediate SEEK's relation with improved screening for depression, substance abuse, or major stress. Perhaps the PSQs were the primary reason for the improved screening, which also suggests the usefulness of a practical screening tool.
However, for IPV screening, we found that improved attitudes and comfort level specific to IPV partially explained improved IPV screening. This supports earlier research suggesting that a lack of comfort impeded screening for IPV in private pediatric practice.20 It is clear that HPs need to be prepared to tackle this challenging problem.
We also examined whether SEEK training would affect male and female HPs differently. Previously, female physicians were found to engage more in psychosocial issues.21,22 However, we found that male and female HPs improved similarly in their screening behavior. Perhaps gender differences are becoming less significant as more male HPs accept the importance of psychosocial issues in health care. Gender differences may also have been minimized by most SEEK HPs being highly motivated by the model.
To further probe what influenced screening, we examined the impact of HPs' duration in practice. Previous literature has not identified whether duration in practice is associated with screening for psychosocial problems.23,24 Baseline screening percentages did not differ, but at follow-up, screening for depression and substance abuse was more frequently documented in the medical records by more experienced SEEK HPs. Possibly, more experienced HPs felt more comfortable in their practice and open to incorporate something new compared with younger HPs.
There are several limitations to this study. The relatively small sample might limit the generalizability of our findings. However, we think that our sample of HPs is probably representative of those in pediatric private practice. In addition, the findings are comparable to those in a similar study done in resident continuity clinics serving a high-risk urban population.5 SEEK HPs may have been more aware of the study goals, modifying their responses to the HPQ and their behavior when observed. A limitation of self-report data is the possibility of respondents providing socially desirable information.
Improvement in screening is an important first step. However, identification of risk factors alone does prevent CM. We did find less psychological aggression (eg, swore or cursed) and minor physical assault (eg, shook or slapped) reported by mothers toward their children in the SEEK practices (data not shown); these findings support those of the previous SEEK study in a high-risk population.5 Our data (not shown) also indicate that after a positive screen, HPs did generally assess and take some action.
Despite recommendations to screen for psychosocial problems, physicians often state that they lack the knowledge, tools, or time.25,–,27 SEEK provides the necessary tools, enabling HPs to meet the goals of Bright Futures.9 This study shows how the SEEK model of pediatric primary care offers a practical approach to helping address serious and prevalent psychosocial problems that jeopardize children's health, development and safety. The training, the PSQ, and parent handouts, and the availability of a social worker are all likely to have contributed to the HPs' gains in comfort level, competence, and increased screening.
These findings offer promise that pediatric primary care HPs can be effectively equipped to help address psychosocial problems confronting many families and children. It is especially encouraging as the SEEK model has been associated with reductions in child maltreatment.5 Thus, the SEEK model may help move pediatric practice beyond a focus on the identification and reporting of CM into the realm of prevention.
The SEEK model led to significant and sustained improvement in several areas, which is a crucial first step in helping HPs address major psychosocial problems that confront many families. SEEK offers a modest yet promising enhancement of primary care.
This research was supported by the Centers for Disease Control and Prevention (grant 1R49CE000588 01R49C) and the Doris Duke Charitable Foundation.
We are very grateful to the primary care pediatricians and pediatric nurse practitioners as well as Nicolette Minas, MS, Ann Pemberton, LCSW-C, Angela Grodack, MA, and Matthew Loftus for their contributions to this study.
- Accepted January 3, 2011.
- Address correspondence to Howard Dubowitz, MD, MS, Department of Pediatrics, University of Maryland School of Medicine, 520 W Lombard St, Baltimore, MD 21201. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
- CM =
- child maltreatment •
- HP =
- health professional •
- IPV =
- intimate partner violence •
- PSQ =
- parent screening questionnaire •
- HPQ =
- health professional questionnaire •
- SEEK =
- Safe Environment for Every Kid
- 1.↵US Department of Health and Human Services, Administration for Children, Youth, and Families. Child Maltreatment, 2009: Reports from the National Child Abuse and Neglect Data Systems. Washington, DC: US Government Printing Office, 2010
- Hussey JM,
- Chang JJ,
- Kotch JB
- Stirling J Jr.
- Amaya-Jackson L
- Dubowitz H
- Dubowitz H,
- Feigelman S,
- Lane W,
- Kim J
- 6.↵American Academy of Pediatrics. Practicing safety program. Available at: www.aap.org/practicingsafety. Accessed February 28, 2011
- Borowsky IW,
- Ireland M
- 9.↵American Academy of Pediatrics. Bright Futures. 3rd ed.Elk Grove Village, IL: American Academy of Pediatrics; 2008
- Narayan AP,
- Socolar RR,
- St Claire K
- Flaherty EG,
- Stirling J Jr.
- Trowbridge MJ,
- Sege RD,
- Olson L,
- O'Connor K,
- Flaherty E,
- Spivak H
- Dubowitz H,
- Black MM,
- Kerr MA,
- Hussey JM,
- Morrel TM,
- Everson MD,
- Starr RH Jr.
- 17.↵SAS Institute, Inc. SAS [computer program]. Version 9.2. Cary, NC: SAS Institute, Inc; 2008
- Erickson MJ,
- Hill TD,
- Siegel RM
- Aira M,
- Kauhanen J,
- Larivaara P,
- Rautio P
- Olson AL,
- Kemper KJ,
- Kelleher KJ,
- Hammond CS,
- Zuckerman BS,
- Dietrich AJ
- Copyright © 2011 by the American Academy of Pediatrics