
,
* Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland
Department of Medicine, Sinai Hospital, Baltimore, Maryland
Société Psychanalytique de Paris, Paris, France
| ABSTRACT |
|---|
|
|
|---|
Methods. Reanalysis of 167 audiotapes from 2 studies of parent-doctor communication in a pediatric residents' continuity clinic was performed. Tapes included visits by 100 mothers or female guardians to 55 residents who were the children's primary care providers. Coders identified mentions of psychosocial topics and noted the content and the doctor's response. Responses were classified with an adaptation of a previously described, psychoanalytically derived typology of avoidant or discouraging responses.
Results. Discouraging responses occurred in 34 (77%) of 44 discussions that involved corporal punishment and 51 (34%) of 64 discussions that involved other psychosocial topics. The particular topic (parent/family versus routine parenting issue) and how the topic was framed (as a problem versus simply mentioned) were associated with doctors' discouraging responses (OR: 3.07; 95% confidence interval: 1.566.05; and OR: 7.57; 95% confidence interval: 3.5016.44; respectively). Discouraging responses were not related to the doctor's gender, parent's ethnicity, length of the parent-doctor relationship, or doctor's overall interview style (patient-centeredness). Discouraging responses to routine problems tended to be dismissive, but 41% of discouraging responses to parent/family problems were failed attempts to provide advice.
Conclusions. Discouraging responses seem to be related less to doctor or patient characteristics than to the type and acuity of the psychosocial topic. These responses may originate with doctors' discomfort with particular subject areas and thus might be approached with training that combines communication and emotion-handling skills with clinical tools such as Bright Futures in Practice: Mental Health or the International Classification of Diseases, 10th Revision, Primary Care.
Key Words: mental health primary care resident training
One persistent difficulty in pediatric primary care is the contrast between the high prevalence of parents' concerns about child behavior or development and the limited extent to which such concerns are discussed with doctors. The analysis by Horwitz et al 1 of pediatric visits in the New Haven, Connecticut, area found that
80% of parents of 4- to 8-year-old children thought that they had a psychosocial concern appropriate to discuss with their pediatricians but only approximately one half actually did so. In adult primary care visits, patients often approach psychosocial issues tentatively, starting with hints or feelers that they hope doctors will recognize. 2
Doctors may contribute to these hesitations consciously or unconsciously. Studies in both adult and pediatric primary care, spanning several decades, found that doctors ignore or dismiss the majority of patients' hints and disclosures of emotional distress. 29 Some patients may take these responses as a message that, in the future, the doctor would rather not hear about similar issues. 10,11
Many things may lead doctors to avoid discussion of patients' distress. Doctors may be troubled by the emotions involved and by feeling that they lack the skills, time, or referral resources required to manage emotional problems. 1216 Clinicians also may chose to distance themselves from patients whose needs they see as going beyond what is expected or reasonable for their specialty or type of interaction. 17
Although previous studies documented the extent to which primary care providers avoid discussion of patient psychosocial concerns or engage in discussions that fail to provide advice, we lack an understanding of the kinds of concerns that evoke avoidance and of which doctors are most likely to be avoidant. Understanding when and why clinicians become avoidant could facilitate development of training, structural alterations to practice settings, and incentives that might promote better care of emotional issues. To that end, Hadjiisky et al 18 interviewed clinicians from a variety of backgrounds (social workers, day care center workers and directors, nurses, and physicians) who had encountered a situation widely acknowledged to promote discomfort among professionals, ie, interaction with a family for which there was suspicion of child abuse. Those investigators asked clinicians to describe their feelings and responses and then used psychoanalytic theory to classify and to interpret the reactions the clinicians reported. Psychoanalytic theory posits that clinicians develop emotional responses to patients and their life situations, just as patients develop emotional responses to their clinicians and to what is discussed during treatment. When these responses are uncomfortable, clinicians, like their patients, can manage them in ways that are more or less adaptive. 19 Hadjiisky et al 18 identified 3 main mechanisms clinicians used to distance themselves from the unpleasant emotions associated with cases of suspected abuse. First, they might reframe or reexplain what they had heard, to make it seem as if it should not be labeled as abuse or to make it seem less severe ("doubt" or denial of reality). Second, they might acknowledge the severity of the problem but propose an overly simplistic solution, as if their recommendation was sufficient to make everything better ("all-powerfulness"). Finally, they might rationalize inaction or silence by saying to themselves that the problem was insoluble ("helplessness").
We developed a system to identify apparent manifestations of the distress responses described by Hadjiisky et al 18 in audiotape recordings of primary care visits between pediatric residents and their patients' mothers. We hypothesized that these responses would be more common when mothers presented residents with psychosocial issues that were outside the expected content of a pediatric visit or that were presented as problems, that is, associated with potential or overt emotional distress. We also wanted to explore the influence of the length of the mother-doctor relationship, the doctor's gender and "patient-centered" style of communicating, and the mother's ethnicity (Fig 1). Female physicians are noted to be more concerned with psychosocial issues, 20 and some studies find that longer patient-doctor relationships increase trust and satisfaction, 21 as does doctors' use of a patient-centered style of communication. 22 In the United States, black patients report difficulty communicating with doctors of different ethnicities. 23,24 Finally, we hoped that the classification of responses described by Hadjiisky et al 18 would provide us with clues about the mechanisms behind residents' responses and thus lead us to hypotheses about possible interventions.
|
| METHODS |
|---|
|
|
|---|
Initial Coding
Tapes were initially coded with the Roter Interaction Analysis System, 26 a scheme for classifying talk in clinical conversations that identifies the purpose of the talk (eg, question or information giving), the speaker, and the intended recipient. Doctors' patient-centeredness was calculated as the sum of all doctor talk classified as giving information (medical and psychosocial), asking psychosocial questions, showing empathy, giving reassurance, and building a partnership. In our previous work, this formulation of patient centeredness was associated with higher levels of parents' psychosocial talk. 27
Visits Included in the Analysis
In the first study, 25 coders identified 44 of 234 visits as including discussion of physical punishment. In 26 of the 44 visits, the parents stated that they used physical punishment; they mentioned its use as a possibility in 13 visits, and there was some other discussion of discipline that raised concerns about its use in the remaining 5 visits. Twenty-nine doctors provided care in these visits; 20 (69%) were female, 27 were white, 1 was black, and 1 was Asian American. Eleven doctors were in their third year of training, 10 were in their second year, and 8 were at the end of their first year. In these visits, the majority of children (33 of 44 children [75%]) were
3 years of age, and the oldest was 10 years of age. Twenty (45%) of the 44 mothers had a high school education or some college, and all except 4 were black. The mothers' average age was 26.6 years (range: 1668 years). Mothers were asked to estimate how many times they had seen the same doctor previously. The average estimate was 11, with a range of 0 to 16.
In the second study, 11 coders identified 164 visit segments in 123 visits (of a total of 692 visits) containing discussion of a psychosocial topic (Table 1). Coders grouped the topics into categories (routine parenting, parents' relationships with partners or parents, stresses related to a child's medical condition, parents' concerns about their own schooling, job, or child care arrangements, and parents' substance abuse or legal problems) and noted whether the topic was simply mentioned or was framed as a problem for the parent. Simple mentions included parents' passive responses to doctors' questions and explicit statements that there was no problem. An example of an interchange with a passive parent answer is as follows.
|
Parent: Mm.
Doctor: Good, glad to hear it.
An explicit statement of "no problem" is as follows.
Doctor: And how are you guys getting along [now that the infant is here]?
Mother: Good.
Doctor: You getting help from dad?
Mother: [pause] Yeah.
Framing as a problem included hints that there might be a problem, statements of parents' increased vigilance, explicit disclosures of parents' distress or concern, and explicit requests for help by parents. A hint from a parent might be as follows.
Doctor: Are you having fun with him?
Parent: He's a handful.
Doctor: What's that?
Parent: He's a handful.
Doctor: Yeah, he is.
A parent's disclosure of distress or a problem would be as follows.
Parent: I'm just having a lot of problems 'cause I graduated from truck driver school and got a job offer.
Doctor: Oh, really?
Parent: But I don't have anybody to leave the kids with.
An example of an explicit request for help is as follows.
Parent: Is there a family counselor here? Because I have this problem. Right? With her.
We approached reliability of coding for topic and problem type by having initial discussions about groupings and definitions, openly discussing several examples to develop a coding manual, and then having 2 investigators (S.L. and L.S.W.) independently code 25 parent-doctor psychosocial interchanges. Agreement between the 2 raters ranged from 88% to 100% for the 7 topic categories and from 84% to 100% for the 6 problem/distress categories.
All of the children in visits in the second study were infants (<1 year of age). Because this was a longitudinal study, the 123 visits involved only 56 mothers, of whom 50 (89%) were black and 6 were white. Mothers ranged in age from 17 to 64 years (mean: 26.1 years; range: 1764 years). Most (39 of 56 [70%]) had finished high school or had taken some college-level courses. Twenty-six doctors were represented; 23 (88%) were white, 3 identified themselves as Asian, and 20 (76%) were female. Fifteen doctors were in their first year of training, and 11 were in their second year. In this study, the original goal was to recruit parent-doctor dyads as close to the first newborn visit as possible and then to record subsequent visits over the course of a year. Therefore, the visits in this sample ranged from the first to the eighth meeting between the doctor and patient (median: second visit).
Coding of Doctors' Discouraging Responses to Psychosocial Topics
We developed an initial set of codes and definitions by transcribing excerpts from visits with discussions of physical punishment. We chose these excerpts because of their similarity to the situations in the original work by Hadjiisky et al 18 and because physical punishment is a subject toward which attitudes are known to vary among doctors. 28 Two of the authors (E.H. and L.S.W.) reviewed the transcripts independently and then together, matching clinicians' responses to the broad categories of reactions noted in the interviews with clinicians by Hadjiisky et al. 18 On the basis of these discussions, we developed 9 categories of responses, 4 discouraging psychosocial discussion and 5 encouraging discussion (Table 2). The 4 discouraging responses corresponded to the 3 broad categories described by Hadjiisky et al 18 plus a category of seemingly ignored parent remarks. The 5 categories of responses that seemed to encourage disclosure particularly well (openly seeking help from other sources, building alliance with empathy, reducing the crisis without minimizing, bolstering credibility without omnipotence, and seeking additional information from the parent before giving advice) corresponded to elements of doctors' problem-identification and emotion-handling skills that were shown to help reduce psychosocial distress among adult primary care patients. 29
|
The coding process involved going to the place in a recording where the target discussion took place and listening to it
1 times, paying attention to tone of voice as well as the words spoken. Tone and context were particularly useful in labeling doctor responses that in isolation might have seemed ambiguous. For example, "I wouldn't worry about it," might be labeled as dismissive and discouraging of discussion if it was said curtly and without any accompanying explanation. In contrast, it might be labeled as empathetic and positive if it was said warmly and after an explanation and an opportunity for the parent to make clear her concerns.
Not all responses were classified as discouraging or particularly encouraging. Responses classified as neutral were those that gave appropriate information but did not make additional attempts to explore the parent's feelings or to provide empathy. A neutral response to a disclosure of fatigue related to child sleep problems might be as follows.
Parent: I'm feeling really wiped out because I'm going in to check every time I hear him cry at night.
Doctor: At this age it's OK to let them cry for a few minutes before going in; they need to learn how to wake up and settle themselves down. How's he feeding?
Coders were allowed to make their own determination of when a discussion ended and were asked to assign the single response code that seemed to fit best. The final coding process thus involved (1) initial coding of tapes to mark locations of psychosocial discussion, (2) study of the coding manual (descriptions of the codes, with examples), (3) training to agreement with someone already familiar with the system, and (4) independent coding, with quality checks.
Statistical Methods
Issues of statistical independence complicated analysis of the 2 studies. In both studies, doctors had visits by >1 parent; in study 2, there could be >1 visit between unique doctor-parent pairs and >1 psychosocial segment within each visit. We computed final results with generalized estimating equation-based procedures in the statistical software program Stata 6.0 (Stata Corp, College Station, TX). These procedures compute population-averaged statistics that take into account the nonindependence of observations and also require few assumptions about the distributions of the variables explored. 30
| RESULTS |
|---|
|
|
|---|
|
Types of Discouraging Responses
Among the 89 visits or segments with discouraging responses, ignoring or dropping the subject was the most common (32 segments [36%]), followed by seeming powerless (25 segments [28%]), acting omnipotent (19 segments [21%]), and changing reality (13 segments [15%]). The type of discouraging response was associated with the type of problem mothers presented. "Routine" psychosocial issues, when they elicited discouraging responses, tended to be ignored, reframed, or dismissed with simple omnipotent formulations (Table 3). In contrast, when parents presented nonroutine topics as problems, 41% of doctors' discouraging responses were of the powerless type, suggesting that doctors had tried to give a substantive response but ultimately found themselves unable to do so successfully. For example,
Mother: He [her son] said, "Man, I hate you so much." My sister and I just laughed. I mean, we really didn't have any other choice. I could have whipped his [...]. If I put my hands on him, I am going to hurt him and they are going to put me in jail.... There've been times I have held him down, but I don't beat him.... I try to scare the [...] out of him.
Doctor: Sounds like you are struggling between wanting to hit him and wanting to be his parent. So we need to figure out how to make that better.
Mother: I made an appointment. He had an appointment for last week but I missed it.
Doctor: So his next appointment is next week? And are you going to let me know what happens?
Mother: I can get you copies if they get ...
Doctor: I would be glad to meet with [breaks off]. Do you have anyone to help you?
Mother: He never listens to anyone. He is bad to everybody. Nobody can really handle him.
Doctor: Sounds like you have a lot to deal with. Make sure you let me know what happens.
Parent Factors Associated With Doctors' Discouraging Responses
Discouraging responses were not associated with the mother's age (mothers receiving a discouraging response were on average 0.6 years younger; 95% CI: 0.2.2 to 1.12), the mother's ethnicity (odds for response to be elicited by white mothers versus black mothers: 1.02; 95% CI: 0.362.9), the mother's distress as reflected on the General Health Questionnaire 31 (OR: 1.07; 95% CI: 0.581.96), or the length of the mother-doctor relationship (discouraging responses were associated with visit numbers that were on average 0.12 greater; 95% CI: 1.2 to 1.4). In visits with discouraging responses, mothers tended to give more medical information (average of 8.4 more information statements; 95% CI: 0.4316.4) and more psychosocial information (average of 15.2 more information statements; 95% CI: 0.52 to 30.83), but they did not ask significantly more questions.
Doctor Factors Associated With Discouraging Responses
Female doctors were no more or less likely than male doctors to make discouraging responses (OR for discouraging response if doctor was female: 1.68; 95% CI: 0.584.8). Visits in which doctors made discouraging responses in psychosocial interactions were marked by doctors asking fewer psychosocial questions over the course of the entire visit. On average, in visits with discouraging responses, doctors asked
8 fewer psychosocial questions (95% CI: 13.1 to 2.8), a decrease of
24% in the number of psychosocial questions asked in an average visit. Discouraging responses were not related significantly to differences in doctors' patient-centeredness (mean difference in number of patient-centered utterances: 17; 95% CI: 53 to +19).
Of the 55 doctors included in the study, 10 had no psychosocial segments with discouraging responses and 22 had some segments with discouraging responses; for 23, all of the psychosocial segments had discouraging responses. There was no difference in gender among these 3 groups of doctors. The proportion of a doctor's visits containing discouraging responses was correlated with the proportion of visits in which parents raised nonroutine topics (r = 0.61; P < .001).
Responses That Encouraged Discussion of Psychosocial Topics
Among the 164 instances of psychosocial topics in the longitudinal study (encouraging responses were not coded in the first, cross-sectional study), doctors made responses that actively encouraged discussion 24 times (15%). The most common encouraging responses were building an alliance with empathetic statements (10 of 164 segments) and seeking additional information (13 of 164 segments). An example of an encouraging response, in which the doctor actively sought more information, is paraphrased here, because it included
2 minutes of interaction.
During the visit, a young child becomes insistent that he has to go to the bathroom. The parent remarks to the doctor that the child is "hard headed" as she tells the child to wait. The doctor asks the mother how she disciplines; she remarks that she "just makes him do things." The doctor asks specifically about spanking and the mother says that she does spank sometimes. Rather than going into a lecture about spanking, the doctor asks for more detail about how she spanks and how often. She replies but also asks several questions about specific behavior problems, which the doctor is able then to address.
Encouraging responses were no more common in visits with white versus black parents (OR: 1.2; 95% CI: 0.295.0) and were not related to the parent's age, the length of the parent-doctor relationship, or the doctor's gender. As with discouraging responses, encouraging responses were associated with the nature of the psychosocial topic and whether parents presented the topic as a problem (Table 3). In contrast to discouraging responses, however, encouraging responses were driven more by the nature of the topic (routine versus not) than by its being posed as a problem. Routine topics rarely received encouraging responses, whether or not they were presented as problems, whereas nonroutine topics, even if not presented as problems, had a 10-fold increased odds of receiving an encouraging response (OR: 10.9; 95% CI: 1.963.4). In contrast to discouraging responses, encouraging responses were related to the doctor's degree of patient-centered interview style. In visits with encouraging responses, doctors made on average 47 more patient-centered utterances (95% CI: 093), compared with visits with discouraging responses.
| DISCUSSION |
|---|
|
|
|---|
In addition, the type of problem presented influenced the type of discouraging response. Discouraging responses to routine problems tended to be dismissive or avoidant. That is, they seemed intended to provide quick reassurance and easy solutions or to avoid exploration of the topic. In contrast, when mothers posed nonroutine issues as problems, 41% of the discouraging responses took the form of failed attempts to offer more extensive advice. This pattern is consistent with findings observed in the treatment of patients with unexplained physical symptoms. In that setting, clinicians often manage patients' initial problem disclosures with quick simple advice, followed by a change of subject. If patients persist, giving additional perplexing details and revealing more distress, then doctors seem to feel compelled to respond (in the case of unexplained symptoms, by ordering tests they think the patient does not really need). 33 In the case of the residents in our study, failed attempts seemed to indicate an acknowledgment that the problem being presented by the parent was legitimate and should not be ignored but the resident lacked the skills or knowledge to be of help.
Another new insight comes from the contrast between predictors of discouraging and encouraging responses. Encouraging responses seem to occur when unusual problems are presented in the context of visits in which residents use a patient-centered interview style. This makes sense, given how patient-centeredness is defined, as a set of clinician behaviors that includes empathetically eliciting and understanding the patient's concerns. 34 Discouraging responses tend to occur when parents pose problems in the context of visits in which residents have been asking a smaller than average number of psychosocial questions. This also seems to make sense; in an atmosphere in which the conversational style seems aimed at limiting the disclosure of psychosocial information, statements about distress are likely to elicit additional avoidance.
We do not know why these different interactional contexts occur. There is some evidence that both parents and residents play a role. Of the 18 residents in the longitudinal study who had
1 visit with a discouraging response and for whom we had >1 visit in the data set, 8 had no visits in which they gave a psychosocial topic an encouraging response. These 8 might be residents who, for reasons we do not know, have a particularly difficult time with psychosocial distress. In contrast, the other 10 residents also had
1 visit in which they made encouraging responses, which suggests that some other factor or factors played a role in determining what they would do in any given visit. Parents' communication style might be one driving force here. Visits with discouraging responses were associated with parents giving more medical and psychosocial information, possibly increasing the chance that residents would feel overwhelmed and react defensively. Other factors that we did not measure could also play a role. Studies in a variety of clinical settings have found relationships between aspects of organizational climate and culture and patient care. Doctors who reported a greater sense of control over their work environment were seen as more participatory by their patients. 35 In community-based child and youth mental health services, a supportive and motivating work climate was associated with better clinician adherence to treatment guidelines 36 and with improved functioning among treated children. 37
These findings have implications for resident training in managing mental health problems in primary care. Focusing on skills that promote a participatory or patient-centered interchange 38 may be important but not sufficient in situations that can evoke doctors' emotional discomfort Additional training may need to focus on the discomfort itself and its causes, whether they are related to the substance of a parent's concerns or the volume of questions and information that the parent presents. This additional training might include communication skills that address explicitly residents' anxiety related to time management (such as skills for the empathetic redirection of rambling and the setting of an agenda for the visit), plus guidance on initial responses to common mental health and psychosocial problems. Practicing pediatricians say that they would tackle both parent and child problems more readily if they had better skills. 16,39 Tool kits such as Bright Futures in Practice: Mental Health 40 and the International Classification of Diseases, 10th Revision, Primary Care (recently revised with an expanded child mental health section) 41 offer efficient ways to improve knowledge of mental health and behavioral problems. Training programs have also been developed to address difficult emotional issues, such as helping families cope with domestic violence, 42 and to increase clinicians' willingness to elicit concerns from patients during cancer treatment. 43 These programs incorporate problem-specific knowledge, problem-specific communication skills, and opportunities for trainees to share their own feelings and ways of managing them. Finally, inevitably all clinicians find themselves confronted with patient problems for which they lack knowledge of what to do. Training in how to deal with such situations could be incorporated into the "difficult patient" training programs offered at many medical schools.
Another Coding System for Doctors' Responses to Patients' Concerns
Ours is one of several coding systems used to examine doctors' responses to psychosocial questions (Table 4). 2,3,7,8,43,44 The previous systems were empirically based. Two, those reported by Sharp et al 7 and Lynch et al, 44 examined doctors' actions, ie, whether they respond at all to parents' concerns and, if so, what concrete actions they take. Maguire et al, 43 Suchman et al, 2 and Levinson et al 3 developed systems that focus more explicitly on the affective and communication value of what doctors do, ie, whether they are responding in ways that facilitate or inhibit the discussion of concerns and emotions. Although our classification of responses contains categories that are similar to those developed by Levinson et al 3 and Maguire et al, 43 its foundation in both theory and clinicians' reflections allows us to generate hypotheses about the origins of pediatricians' behaviors. By differentiating among less-than-optimal responses, we arrive at a more optimistic conclusion than earlier studies; we see that an important proportion of inadequate responses represent not so much avoidance or dismissal of psychosocial issues but rather failed attempts to provide a meaningful response. In fact, the probability of both failed and potentially helpful responses increased as the problems mothers presented became more difficult.
|
Limitations
Our study has several limitations. Perhaps most importantly, we do not know how results might have differed with more experienced doctors or in visits that included fathers or older children. Resident visits are very different from those of practicing community providers; they are longer, residents have less experience, and practices in the low-income urban communities where many residents train often involve patients with more medical and psychosocial problems. 45 Children's medical status was an unknown element in our analyses. All of the children enrolled in our studies came for health maintenance visits, but it is well recognized that medical and psychosocial issues compete for attention in primary care visits. 15,46 However, Levinson et al 3 found similarly high rates of inadequate responses to psychosocial issues in their study of community physicians seeing middle-class adult patients. We also would have liked to explore not only the parents' ethnicity but also the influence of ethnic concordance between mothers and residents. 24 Unfortunately, the data available to us included visits with only 1 black resident, and the large majority of parents in our study were black.
Another limitation is that our splitting of psychosocial problems into what we considered "routine" and what might be considered part of an expanded pediatric mission of responding to parent and family issues might have different meaning in other settings. 47 In our study, routine parenting issues fared relatively poorly; when presented as problems they tended to evoke dismissive responses, and when not presented as problems they received few encouraging ones. This might not be the case in training programs that place a greater emphasis on normal developmental and behavioral issues or in those that explicitly take a more family-centered approach to pediatric care.
| CONCLUSIONS |
|---|
|
|
|---|
| ACKNOWLEDGMENTS |
|---|
We are grateful to Dr Peter Salmon and 2 anonymous reviewers for thoughtful comments on earlier drafts of this manuscript.
| FOOTNOTES |
|---|
Reprint requests to (L.S.W.) School of Public Health, 749 Hampton House, 624 N Broadway, Baltimore, MD 21205. E-mail: lwissow{at}jhsph.edu
This work was presented in part at the annual meeting of the Pediatric Academic Societies; May 14, 2004; San Francisco, CA.
No conflict of interest declared.
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
L. G. Niederman, A. Schwartz, K. J. Connell, and K. Silverman Healthy Steps for Young Children Program in Pediatric Residency Training: Impact on Primary Care Outcomes Pediatrics, September 1, 2007; 120(3): e596 - e603. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. M. Dumont-Mathieu, B. A. Bernstein, P. H. Dworkin, and L. M. Pachter Role of Pediatric Health Care Professionals in the Provision of Parenting Advice: A Qualitative Study With Mothers From 4 Minority Ethnocultural Groups Pediatrics, September 1, 2006; 118(3): e839 - e848. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||