ELECTRONIC ARTICLE |


* Lilly Research Laboratories, Indianapolis, Indiana
Behavioral Health Center, Charlotte, North Carolina
Children's Hospital, Pediatric Clinical Trials International, Columbus, Ohio
|| Louisiana State University Health Science Center, New Orleans, Louisiana
¶ NeuroScience, Inc, Bethesda, Maryland
| ABSTRACT |
|---|
|
|
|---|
Methods. This study was a randomized, multicenter, double-blind, placebo-controlled trial conducted at 12 outpatient sites in the United States. A total of 197 children, 6 to 12 years of age, who had been diagnosed as having ADHD, on the basis of the Diagnostic and Statistical Manual of Mental Disorders (4th ed.) criteria, were randomized to receive 8 weeks of treatment with atomoxetine or placebo, dosed once daily in the mornings. ADHD symptoms were assessed with parent and investigator rating scales. The primary outcome measure was the Attention-Deficit/Hyperactivity Disorder Rating Scale-IV-Parent Version: Investigator-Administered and Scored total score. Daily parent assessments of children's home behaviors in the evening and early morning were recorded with an electronic data entry system. This instrument measures 11 specific morning or evening activities, including getting up and out of bed, doing or completing homework, and sitting through dinner.
Results. Seventy-one percent of the children enrolled were male, 69% met criteria for the combined subtype (both inattentive and hyperactive/impulsive symptoms), and the most common psychiatric comorbidity was oppositional defiant disorder (35%). Once-daily atomoxetine (final mean daily dose of 1.3 mg/kg) was significantly more effective than placebo in treating core symptoms of ADHD. Mean reductions in the Attention-Deficit/Hyperactivity Disorder Rating Scale-IV-Parent Version: Investigator-Administered and Scored total score were significantly greater for patients randomized to atomoxetine, beginning at the first visit after the initiation of treatment and continuing at all subsequent visits. Both inattentive and hyperactive/impulsive symptom clusters were significantly reduced with atomoxetine, compared with placebo. With continued treatment and dose titrations, core symptoms of ADHD continued to decrease throughout the 8-week study. Mean reductions in the daily parent assessment total scores for patients randomized to atomoxetine were superior during the first week, beginning with the first day of dosing, and were also superior at endpoint. Efficacy outcomes for the evening hours for atomoxetine-treated patients were superior to those for placebo-treated patients, as assessed with 2 different assessment scales. Decreases in the daily parent assessment morning subscores at endpoint showed a significant reduction in symptoms that lasted into the mornings. Rates of discontinuations attributable to adverse events were <5% for both groups. Adverse events reported significantly more frequently with atomoxetine were decreased appetite, somnolence, and fatigue.
Conclusions. Among children 6 to 12 of age who had been diagnosed as having ADHD, once-daily administration of atomoxetine in the morning provided safe, rapid, continuous, symptom relief that lasted not only into the evening hours but also into the morning hours. Atomoxetine treatment was safe and well tolerated.
Key Words: atomoxetine ADHD children continuous relief once-daily rapid onset of action
Abbreviations: ADHD, attention-deficit/hyperactivity disorder GIPE, Global Index: Parent-Evening ADHD RS, Attention-Deficit/Hyperactivity Disorder Rating Scale-IV-Parent Version: Investigator-Administered and Scored ECG, electrocardiogram LOCF, last observation carried forward CGI-ADHD-S, Clinical Global Impressions-Attention-Deficit/Hyperactivity Disorder-Severity CI, confidence interval ANCOVA, analysis of covariance DPREMB-R, Daily Parent Ratings of Evening and Morning Behavior-Revised
Attention-deficit/hyperactivity disorder (ADHD) is an early-onset, neurobehavioral childhood disorder that affects 3% to 7% of school-aged children in the United States.1 The disorder is associated with high levels of comorbidities, such as learning disabilities, mood and anxiety disorders, impairment of academic and social functioning,2 and poorer outcomes later in life.39
The National Institute of Mental Health-funded Multimodal Treatment Study of Children with ADHD recently demonstrated the superiority of careful, standardized, drug therapy over psychosocial interventions alone in ADHD symptom reduction for most children.10 Stimulant treatment outside school (thrice-daily methylphenidate treatment) has also been suggested to be superior to therapy restricted to school hours (twice-daily methylphenidate treatment).11,12 Currently, the most commonly used pharmacologic interventions for ADHD involve various formulations of and delivery systems for the psychostimulants methylphenidate and amphetamine.
Atomoxetine is a potent inhibitor of the presynaptic norepinephrine transporter (Ki: 4.5 nM) that was recently approved for the treatment of ADHD in the United States. (Atomoxetine was originally called tomoxetine; the name was recently changed to avoid potential confusion with tamoxifen, which might lead to errors in drug dispensing.) The effectiveness of atomoxetine in the treatment of ADHD has been demonstrated among children and adolescents when administered once or twice daily1316 and among adults when administered twice daily.17 In a recently published study,15 atomoxetine appeared to be as effective when the daily dose was administered once in the morning as when the dose was divided and administered in the morning and evening, as in previous studies. This finding is compelling, given that the half-life of this drug is
5 hours for most patients, and suggests that the effects of atomoxetine on ADHD symptoms persist beyond its direct pharmacologic effects. If true, this is of particular interest because it suggests a marked difference from the stimulants, the effects of which are generally closely correlated with plasma drug levels.18 The study reported here was designed to confirm and extend the observations made in the initial study of once-daily atomoxetine therapy,15 by assessing, in a more systematic way, efficacy during the evening and early morning hours. Another goal was to determine how early in treatment it was possible to discern a specific effect of the drug on ADHD symptoms.
| METHODS |
|---|
|
|
|---|
After description of the procedures and purpose of the study and before administration of any study procedure or dispensing of study medication, written informed consent was obtained from each patient's parent or guardian and written assent was obtained from each patient. This study was approved by each site's institutional review board and was conducted in accordance with the ethical principles that have their origin in the Declaration of Helsinki and are consistent with good clinical practices and applicable laws and regulations.
Study Design
This study was a randomized, multicenter, double-blind, placebo-controlled trial conducted at 12 outpatient sites in the United States. All patients underwent a minimum 5-day, medication-free, evaluation period, followed by randomization to atomoxetine or placebo, in a double-blinded manner, for approximately 8 weeks. To keep to a minimum the number of patients assigned to receive placebo, randomization was unbalanced, with a ratio of 2 patients assigned to atomoxetine for each patient assigned to placebo. After randomization at the second study visit, patients underwent evaluation at 1, 4, and 8 weeks (visits 35).
The study drug was administered as a single daily dose. Parents were instructed to give the study drug in the morning at a time of their own choosing, with or without food. Patients in the atomoxetine treatment arm began treatment at 0.8 mg/kg per day for 3 days, with the dose increasing to 1.2 mg/kg per day for the remainder of the first week. The patients continued to receive atomoxetine at 1.2 mg/kg per day unless there were safety or tolerability problems that precluded them from continuing with this dose. After 4 weeks, patients receiving 1.2 mg/kg per day who demonstrated significant residual symptoms (scores of
3 on the Clinical Global Impressions-Attention-Deficit/Hyperactivity Disorder-Severity [CGI-ADHD-S]22) and for whom there were no safety or tolerability contraindications could have their dose increased to 1.8 mg/kg per day. The maximal dose for any subject was 1.8 mg/kg per day and could not exceed 120 mg per day, regardless of weight. Compliance was monitored with direct questioning and pill counting at each physician visit.
Assessments
Efficacy
The primary efficacy measure was the ADHD RS, a semistructured interview with the patient's parent (or primary caregiver) that is rated by the investigator. It consists of 18 items, with 1 item for each of the 18 symptoms contained in the Diagnostic and Statistical Manual of Mental Disorders (4th ed.) diagnosis.20,21 Each item is scored on a 4-point scale (0 = never or rarely; 1 = sometimes; 2 = often; 3 = very often).
Other assessments included a parent-completed daily questionnaire evaluating the child's behavior during the early morning and late afternoon/evening (Daily Parent Ratings of Evening and Morning Behavior-Revised [DPREMB-R]), beginning 5 days before randomization and continuing for
4 weeks, using an electronic entry system (PHT Esendant LogPad technology; PHT Corp, Charlestown, MA). This instrument, which was pilot-tested previously15 and was revised for use in this study, measures 11 specific morning or evening activities, including getting up and out of bed, doing or completing homework, and sitting through dinner. Each item is rated from 0 (no difficulty) to 3 (a lot of difficulty). Parents were instructed to complete the diary each evening between 6 PM and 12 midnight and to complete the morning items based on the time before medication was administered. Data for a particular date could be recorded only on that date.
The Conners' Global Index: Parent is a 10-item questionnaire that assesses characteristics common to children with ADHD.23 In this study, it was used to collect parent observations of these behaviors in the evening (hence, Conners' Global Index: Parent-Evening [GIPE]). Each characteristic was rated on a 4-point scale (0 = not true at all/never; 1 = just a little true/occasionally; 2 = pretty much true/often, quite a bit; 3 = very much true/very often). Parents completed the assessment by responding to questions on the Conners' GIPE on the basis of their children's behavior in the evenings since the last visit. The total score, a restless-impulsive subscore (sum of items 14 and 68), and an emotional lability score (sum of items 5, 9, and 10) were calculated.
CGI-ADHD-S is a single-item rating based on the clinician's total experience with subjects with ADHD.22,23 Severity is rated on a 7-point scale (1 = normal, not at all ill; 7 = among the most extremely ill subjects).
Safety
Safety and tolerability measures included vital signs, electrocardiograms (ECGs), assessments of adverse events collected with open-ended questioning, and clinical laboratory tests.
Statistical Analyses
Patient data were analyzed on an intent-to-treat basis. All statistical tests were performed with a 2-sided, .05 significance level. The primary outcome measure, the ADHD RS total score, was analyzed with a mixed-model, repeated-measures analysis, using the MIXED procedure in SAS,24 to compare the treatment groups. The dependent variables were the scores from visit 3 through visit 5 (all postrandomization data). The model also contained fixed-class effect terms for treatment, investigator, and visit and an interaction term for the interaction between treatment and visit. The model included a random patient effect and baseline (visit 2) scores as covariates and used an unstructured covariate structure. In addition, visit-wise efficacy was assessed by using contrasts from this model at each postbaseline visit. To illustrate change-from-baseline values for ADHD RS scores, the same model was implemented with change scores instead of observed scores as the dependent variable. The P values for treatment differences overall and by visit were not affected by replacing observed scores with change-from-baseline scores as the dependent variable. Change-from-baseline scores on the DPREMB-R scale during the first week of treatment were also analyzed by using the repeated-measures technique, with day of the week replacing visit in the model.
Analyses were also performed by using a last observation carried forward (LOCF) approach. For analysis of the LOCF mean change from baseline to endpoint, patients with a baseline value and
1 postbaseline measurement were included in the analysis. Treatment comparisons for efficacy measures of the mean change from baseline between groups were assessed by using a fixed-effects analysis of covariance (ANCOVA) model that contained terms for baseline score, treatment, and investigator. Because DPREMB-R findings were recorded daily, change scores were calculated by averaging total scores or subscores on a weekly basis, to minimize the effects of day-to-day variations. To estimate the change to endpoint, each patient's average baseline score was subtracted from the last weekly average score. To be included in the DPREMB-R weekly calculation, at least 4 of the 5 baseline records and 6 of 7 records in 1 of the 4 weeks after baseline must have been completed. This inclusion criterion was used to ensure that an accurate assessment of the child's daily behavior was captured in the analysis.
Effect size for the primary outcome measure was calculated as the least-squares mean treatment difference for atomoxetine versus placebo divided by the square root of the mean square error, based on the ANCOVA of LOCF change-from-baseline scores. Protocol-specified definitions of response (
25% decrease in ADHD symptoms, as indicated by the ADHD RS total score, or an endpoint CGI-ADHD-S score of 1 or 2) were analyzed by using Fisher's exact test.
Correlations between the DPREMB-R evening subscore and the Conners' GIPE total score were computed to assess the sensitivity achieved when using the Conners' GIPE to measure behaviors during a specific time of day. To assess symptom severity at endpoint, relative to an unaffected population, scores on the primary outcome measure were also analyzed as t scores. This is a transformation based on normative data adjusted for gender and age. For a normal population, 50 represents the mean score and 10 points represents 1 SD from the mean.
For efficacy scales, if >1 item of a subscale was missing, then the score for the subscale was also considered missing. If a single item was missing, then the mean score for all other items in the subscale was imputed as the score for the missing item. Total scores were computed as the sum of imputed subscores.
Treatment differences in binary measures between groups were assessed by using Fisher's exact test. Treatment comparisons for safety measures in mean changes from baseline between groups were assessed by using a fixed-effects analysis of variance model that contained terms for treatment and investigator.
| RESULTS |
|---|
|
|
|---|
|
|
25% reduction from baseline in total ADHD RS score: atomoxetine: 62.7%; placebo: 33.3%; 95% confidence interval [CI]: 15.243.6%; P < .001; endpoint CGI-ADHD-S score of 1 or 2 [normal or minimally ill]: atomoxetine: 27.0%; placebo: 5.0%; 95% CI: 12.831.2%; P < .001). Mean reductions in the CGI-ADHD-S scores at endpoint were also significantly greater for patients randomized to atomoxetine (with LOCF ANCOVA) (Table 2).
|
|
|
|
The clinical laboratory results indicated that the change-from-baseline values for atomoxetine were statistically significantly different from placebo for alkaline phosphatase levels (atomoxetine: 0.4 ± 28.8 U/L; placebo: 18.2 ± 36.7 U/L; P < .001), inorganic phosphorous levels (atomoxetine: 0.04 ± 0.21 mmol/L; placebo: 0.06 ± 0.22 mmol/L; P = .021), total protein levels (atomoxetine: 1.1 ± 3.8 g/L; placebo: 1.4 ± 3.9 g/L; P < .001), and albumin levels (atomoxetine: 0.8 ± 2.4 g/L; placebo: 0.9 ± 2.9 g/L; P < .001). In each case, the values for the placebo-treated subjects deviated from baseline to a greater extent than did the values of the atomoxetine-treated patients.
The rates of discontinuation as a result of adverse events were low and similar between treatment groups. Six atomoxetine-treated patients (4.5%) discontinued because of 4 adverse events (somnolence: 3; aggression: 1; fatigue: 1; syncope: 1), and 1 placebo-treated patient (1.6%) discontinued because of nausea. Eight atomoxetine-treated patients (6.0%) and 3 placebo-treated patients (4.7%) left the study because of protocol violations, mainly noncompliance with electronic diaries. In addition, 6 atomoxetine-treated patients (4.5%) and 2 placebo-treated patients (3.1%) left for reasons categorized as personal conflict, primarily specified as withdrawal of consent. The personal conflict category was typically used for situations in which the patient or parent refused to complete a study requirement (eg, the patient did not want to swallow capsules or the parent did not want to keep clinic appointments). Overall, 107 of 133 patients randomized to atomoxetine (80.5%) and 47 of 64 patients randomized to placebo (73.4%) completed the study.
| DISCUSSION |
|---|
|
|
|---|
The most striking finding of this study is the confirmatory evidence that once-daily dosing in the morning is associated with significant symptom reduction that persists into the evening and into the morning hours. This was assessed with a daily diary (DPREMB-R) completed by parents beginning 5 days before randomization and continuing for an additional
4 weeks. Parents used an electronic entry system (PHT Esendant LogPad technology) to record the DPREMB-R in the evenings between 6 PM and 12 midnight and to transmit data daily to the PHT server through a modem. In a previous study,15 an individual-item analysis of the original DPREMB, collected daily for a 1-week period via telephone, with an interactive voice-response system, suggested a drug-specific benefit late in the day. In the present study, with DPREMB-R collected daily, the DPREMB-R total score and the evening and morning subscales showed statistically significant improvements from baseline to endpoint, demonstrating effectiveness in behavior control during both evening and morning hours. Comparisons of mean changes in the individual items of the DPREMB-R demonstrated significant atomoxetine-specific reductions for 5 of the 8 evening items and 2 of the 3 morning items. The results should be interpreted cautiously, because the instrument is new and its psychometric characteristics have not been studied. However, results obtained with the Conners' GIPE,23 a validated scale assessing evening behaviors, also showed statistically significant improvements in the total score for evening behaviors from baseline to endpoint. A statistical correlation of .62 was found between the DPREMB-R evening items and the Conners' GIPE items, supporting the DPREMB-R results and confirming the long-lasting effects of atomoxetine into the late evening hours. A more detailed examination of the psychometric properties of this scale will be the subject of another publication.
This is the third study15,16 reporting the superior efficacy of once-daily dosing of atomoxetine versus placebo in reducing the severity of symptoms among children with ADHD. Patients treated with atomoxetine demonstrated statistically significant reductions in the ADHD RS total score and inattentive and hyperactive/impulsive subscale scores at endpoint. Changes associated with atomoxetine were clinically significant; the mean t scores at the endpoint were
1.5 SDs above age and gender normative values for the atomoxetine group, compared with 2.5 SDs above normative values for the placebo group. In addition, improvements of 1.6 points on the CGI-ADHD-S scale indicated a clinically meaningful response. In this study, a more vigorous, early dose titration schedule was used, with patients receiving a starting atomoxetine dose of 0.8 mg/kg per day for the first 3 days and then a dose of 1.2 mg/kg per day for the remainder of the first week. This made it possible to more accurately assess the onset of treatment effects. A visit-wise analysis of the ADHD RS total score showed statistically significant improvement after 1 week of treatment and additional improvement at all subsequent visits. Evidence for superior efficacy, compared with placebo, as soon as the first day of treatment with the more aggressive dose titration schedule was shown in daily analyses of the DPREMB-R during the first week of treatment.
The relative efficacy of once- versus twice-daily atomoxetine could not be definitively determined, because this study did not include a twice-daily dosing arm. However, preliminary comparisons could be made because the primary outcome measure and most study design elements in this study were the same as those used in 3 previous placebo-controlled studies that used twice-daily dosing and 2 that used once-daily dosing. In addition, all of the investigators in this trial had participated in at least one of the previous studies. The treatment effect size in this study for the primary outcome measure was 0.7, compared with 0.8, 0.7, and 0.6 in the studies of twice-daily atomoxetine13,14 and 0.715 and 0.625 in the studies of once-daily atomoxetine. Although definitive conclusions require an adequately sized, direct comparison, these data suggest that the magnitude of symptom reduction with once-daily dosing is similar to that with twice-daily dosing. These results are also comparable to effect sizes reported for methylphenidate with parent-rated measures,11,26 although the comparison is limited by differences in study designs, measures, investigators, and study populations.
Acute treatment with atomoxetine was well tolerated in this study population, despite the use of a titration schedule that was shorter than those used in previous atomoxetine clinical trials. This titration schedule closely approximated the dosing recommendations in the Food and Drug Administration-approved package insert. The majority of atomoxetine-treated patients (80.5%) completed treatment, and the rate of discontinuations as a result of adverse events was low (4.5%). In the present study, the only adverse events reported significantly more frequently than with placebo were decreased appetite, somnolence, and fatigue. As reported in previous once-daily administration studies, these events (excluding decreased appetite and diarrhea) and the other gastrointestinal events were self-limiting, peaking at 1 to 3 days of treatment and decreasing thereafter.15,27 When data from the 3 once-daily atomoxetine studies were pooled, a greater incidence of events (primarily gastrointestinal events) was observed, compared with placebo. However, similar incidences of events were observed when once-daily administration studies were compared with studies with twice-daily atomoxetine administration.27 Consistent with earlier studies, patients treated with atomoxetine exhibited modest but statistically significant weight loss, compared with those treated with placebo. Modest increases in ECG heart rate and supine pulse are consistent with vital sign changes reported in previous studies.1315,27
Several factors limit the interpretation of the results presented here. As noted, no comparisons with other atomoxetine dosing paradigms or other drugs were included. Also, the short duration of this study limits the ability to make assumptions regarding the long-term safety or efficacy of atomoxetine once patients have achieved a satisfactory initial response. However, data for patients who participated in earlier studies and were treated with atomoxetine for
1 year suggest that there are no long-term safety concerns.28 Specifically, reports of decreased appetite and weight loss, which were reported statistically significantly more often with atomoxetine than with placebo in acute trials, continued to decline during long-term treatment, as did reports of other adverse events. Modest increases in blood pressure and heart rate were observed initially but stabilized with long-term treatment. After
1 year of treatment, atomoxetine increased the mean heart rate 6.4 beats per minute and increased the mean diastolic blood pressure 2.8 mm Hg. Few comorbid psychiatric conditions other than oppositional defiant disorder were observed in the study population. This is consistent with previous atomoxetine studies but different from some studies in the literature.10,29 This difference is likely related to the requirement that comorbid psychiatric diagnoses be confirmed with the Kiddie Schedule for Affective Disorders and Schizophrenia for School-Aged Children, which is highly specific but also highly stringent.
The data suggesting that atomoxetine continued to produce statistically significant efficacy in the morning hours merit additional clinical investigations. Parents were instructed to administer the study medication in the mornings. Although parents completed the diary questions in the evenings, they were instructed to answer the morning questions on the basis of the child's behavior before medication was administered. It is possible that the parents' answers were influenced by behavior observed after medication administration; however, this is unlikely to some extent. Because peak plasma concentrations do not occur until
2 to 3 hours after atomoxetine administration, it is unlikely that parents would administer the study medication early enough to influence questions addressing difficulty getting out of bed and difficulty getting ready in the morning.
Finally, although the demonstration of efficacy in the morning and evening hours suggests symptom control during school hours, this study did not include teacher ratings to measure the effects of atomoxetine on school behavior. The effectiveness of atomoxetine at school, however, was demonstrated in 2 atomoxetine trials reported elsewhere.15,16
| CONCLUSIONS |
|---|
|
|
|---|
| ACKNOWLEDGMENTS |
|---|
| FOOTNOTES |
|---|
Reprint requests to (D.K.) Lilly Technology Center South, DC4135, Indianapolis, IN 46285. E-mail: kelsey_douglas_k{at}lilly.com
This research was funded by Eli Lilly and Company. Drs Kelsey, Sumner, Sutton, Schuh, and Allen; Ms Gonzales; and Ms Malcolm are employees and shareholders of Eli Lilly and Company. Dr Casat serves on speaker panels and advisory boards for Eli Lilly and Company and has research grants from the company. Drs Coury, Quintana, and Saylor are paid consultants and investigators for studies sponsored by Eli Lilly and Company.
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
W. Weber, A. Vander Stoep, R. L. McCarty, N. S. Weiss, J. Biederman, and J. McClellan Hypericum perforatum (St John's Wort) for Attention-Deficit/Hyperactivity Disorder in Children and Adolescents: A Randomized Controlled Trial JAMA, June 11, 2008; 299(22): 2633 - 2641. [Abstract] [Full Text] [PDF] |
||||
![]() |
V. Sung, H. Hiscock, E. Sciberras, and D. Efron Sleep Problems in Children With Attention-Deficit/Hyperactivity Disorder: Prevalence and the Effect on the Child and Family Arch Pediatr Adolesc Med, April 1, 2008; 162(4): 336 - 342. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. J. Spencer, F. R. Sallee, D. L. Gilbert, D. W. Dunn, J. T. McCracken, B. J. Coffey, C. L. Budman, R. K. Ricardi, H. L. Leonard, A. J. Allen, et al. Atomoxetine Treatment of ADHD in Children With Comorbid Tourette Syndrome J Atten Disord, January 1, 2008; 11(4): 470 - 481. [Abstract] [PDF] |
||||
![]() |
S. V. Faraone, S. B. Wigal, and P. Hodgkins Forecasting Three-Month Outcomes in a Laboratory School Comparison of Mixed Amphetamine Salts Extended Release (Adderall XR) and Atomoxetine (Strattera) in School-Aged Children With ADHD J Atten Disord, July 1, 2007; 11(1): 74 - 82. [Abstract] [PDF] |
||||
![]() |
A. P Gibson, T. L Bettinger, N. C Patel, and M L. Crismon Atomoxetine Versus Stimulants for Treatment of Attention Deficit/Hyperactivity Disorder Ann. Pharmacother., June 1, 2006; 40(6): 1134 - 1142. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. D. Rappley Attention Deficit-Hyperactivity Disorder N. Engl. J. Med., January 13, 2005; 352(2): 165 - 173. [Full Text] [PDF] |
||||
![]() |
J. G. Millichap Effectiveness of Atomoxetine for ADHD AAP Grand Rounds, October 1, 2004; 12(4): 43 - 44. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||