PEDIATRICS Vol. 110 No. 3 September 2002, pp. e34
ELECTRONIC ARTICLE |
Long-Term Safety and Efficacy of Risperidone for the Treatment of Disruptive Behavior Disorders in Children With Subaverage IQs


* Department of Family and Community Medicine, University of Toronto, Toronto, Ontario, Canada
Department of Clinical Affairs, Janssen-Ortho Inc, Toronto, Ontario, Canada
Department of Pediatrics, Royal University Hospital, Saskatoon, Saskatchewan, Canada
|| Department of Psychiatry, London Health Sciences Centre, London, Ontario, Canada and the Conduct Study Group
| ABSTRACT |
|---|
|
|
|---|
Objective. The objective of this study was to investigate the long-term safety and efficacy of risperidone in disruptive behavior disorders in children with subaverage IQs. Disruptive behavior disorders were defined as oppositional defiant disorder, disruptive behavior disorder, and conduct disorder as per the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) criteria.
Methods. This was a 48-week open-label (OL) extension study of risperidone in 77 children diagnosed with a disruptive behavior disorder, and either borderline intellectual function or mild or moderate mental retardation who had participated in a previous 6-week, double-blind (DB) study and completed at least 2 weeks of DB therapy. Children, aged 5 to 12 years inclusive, who had: 1) a DSM-IV Axis I diagnosis of conduct disorder, oppositional defiant disorder, or disruptive behavior disorder- not otherwise specified; 2) a parent-assessed rating of
24 in the Conduct Problem Subscale of the Nisonger-Child Behavior Rating Form28; 3) a DSM-IV Axis II diagnosis of mild or moderate mental retardation or borderline intellectual functioning with an IQ
36 and
84; and 4) a score of
84 on the Vineland Adaptive Behavior Scale. Participants received oral solution risperidone given at a once daily dose of between 0.02 and 0.06 mg/kg for a maximum of 48 weeks. Participants in the DB study who had been randomized would have had a maximum of 54 weeks of risperidone therapy. Study visits were scheduled at entry, weekly for the first month, and monthly for the remaining 11 months.
Results. Baseline scores on the conduct problem subscale at the start of the previous DB study were similar for both treatment groups: mean values of 33.5 and 33.3 were recorded for placebo- and risperidone-treated participants, respectively. At the time of the OL baseline visit, mean Conduct Problem Subscale scores were lower in those who had been treated with risperidone than in those who remained risperidone-naïve (17.5 and 26.1, respectively). Within 1 week of receiving daily risperidone therapy (mean daily dose: 1.38 mg), those participants who had been risperidone-naïve at OL entry showed a rapid improvement in the Conduct Problem Subscale score. At the week 1 assessment, the mean change from baseline for those who had been risperidone-naïve at OL entry was similar in magnitude to the change from DB baseline recorded for participants who had received risperidone in the DB study. This mean improvement was sustained in both groups throughout the remainder of the OL study.
At study endpoint, those participants who had been risperidone-naïve at OL entry experienced a highly significant mean decrease from OL baseline in the mean Conduct Problem Subscale score of 10.6 ± 2.18. The response to risperidone in the OL trial remained stable in those participants who had been treated with risperidone in the previous DB trial; in this group, the mean change at study endpoint from OL baseline was a nonsignificant decrease of 1.26 ± 1.45. At DB baseline, 68% of participants had a Clinical Global Impression assessment rated as marked, severe, or extremely severe. By DB study endpoint, only 17% of participants (15% of whom had received placebo and 19% of whom had been treated with risperidone in the previous study) had this severe an assessment; 63% of participants had symptoms rated as either none, very mild, or mild. Similarly, highly significant decreases from baseline in the Vineland Adaptive Behavior Scale rating of the most troublesome symptom (often identified as either aggression (hitting, fighting, or temper tantrums) were observed by study endpoint after 48 weeks of risperidone therapy. For those participants who had received placebo in the previous study, a mean decrease of 47.1 ± 4.87 mm from a DB baseline of 79.4 ± 2.69 mm was observed. In those who had received risperidone, a mean decrease of 43.5 ± 4.57 mm from a DB baseline of 79.3 ± 3.66 mm was observed. Five subgroup analyses of the primary efficacy outcome were performed. These included analysis by diagnosis (conduct disorder, oppositional defiant disorder, and disruptive behavior disorder-not otherwise specified), degree of mental retardation (borderline, mild, moderate), and presence or absence of somnolence, attention-deficit/hyperactivity disorder, and psychostimulants.
The results showed that the efficacy of risperidone was not affected by type of disorder, level of retardation, presence/absence of somnolence or attention-deficit/hyperactivity disorder, or use of psychostimulants. Adverse events were reported for 76 participants; none were serious and most were mild/moderate in severity. Somnolence (52%), headache (38%), and weight gain (36%) were the most common adverse events. The degree of sedation was mild and not associated with cognitive deterioration. In fact, for most parameters assessed on the modified California Verbal Learning Test (a test for verbal learning and memory), there were statistically significant improvements relative to both OL and DB baselines in the mean scores. In addition, statistically significant improvements over baseline were also seen for some Continous Performance Task (which is a test for attention and impulsivity) parameters. Overall, no deterioration of cognitive function was observed while participants were treated with risperidone.
Almost half of the 8.5 kg gained was attributable to normal growth. Asymptomatic peak prolactin levels were observed within 4 weeks of beginning risperidone treatment and declined over time to within normal range. At study endpoint, mean prolactin levels were statistically significantly greater than baseline only in male participants but still <20 ng/mL, which is within the normal range. Twenty participants experienced mild or moderate extrapyramidal symptoms, although none withdrew for this reason.
Conclusions. Risperidone, administered as an oral solution at a mean dose of 1.38 mg/d (range: 0.020.06 mg/kg/d) for 1 year, was well tolerated, safe, and showed maintenance of effect in the treatment of disruptive behavior disorders in children aged 5 to 12 years with subaverage IQs.
Key Words: disruptive behavior disorders conduct disorder oppositional defiant disorder disruptive behavior disorder not otherwise specified mental retardation borderline intellectual function risperidone
Abbreviations: DBD, disruptive behavior disorders ADHD, attention-deficit/hyperactivity disorder DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition CD, conduct disorder ODD, oppositional defiant disorder DBD-NOS, disruptive behavior disorder-not otherwise specified EPS, extrapyramidal symptoms TD, tardive dyskinesia DB, double-blind N-CBRF, Nisonger Child Behavior Rating Form OL, open-label ESRS, Extrapyramidal Symptoms Rating Scale VAS, visual analog scale ECG, electrocardiogram CPT, Continuous Performance Task MCVLT, Modified California Verbal Learning Test CGI, Clinical Global Impression
| INTRODUCTION |
|---|
|
|
|---|
Disruptive behavior disorders (DBDs) are some of the most common forms of psychopathology in children; the overall prevalence is
6%, with boys more commonly affected than girls and those with lower intellectual function more commonly affected than the general population.1,2 Prevalence rates as high as 64% have been reported in severely mentally retarded subjects.3,4 A low IQ has not only proven to be predictive of the presence of DBDs, it has also been associated with the persistence of symptoms and the resistance of those symptoms to treatment.37 Important comorbidities include attention-deficit/hyperactivity disorder (ADHD), depression, anxiety disorders, and learning disabilities.8 As defined in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV), DBD includes conduct disorder (CD), oppositional defiant disorder (ODD), and disruptive behavior disorder-not otherwise specified (DBD-NOS).9 The main characteristic of these disorders is a repetitive and persistent pattern of antisocial, aggressive, or defiant behavior that involves major violations of age-appropriate norms. CD, which is considered the most severe subcategory, is characterized by the predominance of specific conduct-type symptoms such as severe destructiveness and violence, whereas ODD is characterized by the predominance of milder oppositional-type symptoms such as tantrums. DBD-NOS does not fully meet the DSM-IV criteria for either classification.
DBDs have a considerable impact on both the individual affected and their family. In children with subaverage IQs, persistent symptoms of these disorders are the most important reason for breakdown of mainstreaming and community placement, expulsion from school, and permanent hospitalization.10,11 These children are also at later risk for antisocial personality disorder and other psychiatric disorders, marital break-up, employment difficulties, criminality, substance abuse, and premature death.12,13 Safe and effective interventions are needed for this population of high-risk children.
The management of DBDs in children is typically multidisciplinary and may involve a number of different psychosocial and pharmacotherapies.
Although conventional antipsychotics have been commonly used in clinical practice, the risk of debilitating side effects, such as extrapyramidal symptoms (EPSs), tardive dyskinesia (TD), excessive sedation, and cognitive blunting limit their use.14,15 Lithium and psychostimulants, such as methylphenidate, have also been used to manage the symptoms of these disorders in children, yet the clinical trial data published to date have not been consistent or compelling. Lithium was associated with behavioral improvements in 3 studies, but it was not significantly different from placebo in 2 others.1620 And, although methylphenidate was modestly effective for the treatment of disruptive behaviors in 1 study of children with above-average intelligence,21 in those with subaverage IQs, psychostimulants were not only less effective but were associated with excessive side effects.22
In the 1990s, reports were published on the benefits of using the atypical antipsychotic agent, risperidone, to treat aggressive behaviors in children and adults.23,24 Based on these preliminary data, and on those of a small pilot study conducted in children with borderline intellectual functioning,25 two 6-week studies were undertaken to investigate the use of risperidone for the treatment of DBDs in a total of 217 children with subaverage IQs.26,27 Data from these randomized, double-blind (DB), placebo-controlled studies1 conducted in Canada, the United States, and South Africa; and 1 solely in the United Statesprovides convincing evidence that risperidone is an effective treatment in this population. In both studies, efficacy was assessed using the Nisonger-Child Behavior Rating Form (N-CBRF), a validated rating form for children with disabilities.28 Using this and other tools, risperidone use (average daily doses of 0.98 mg and 1.16 mg, respectively) was associated with highly significant improvements in measures of disruptive behaviors from weeks 1 through 6 reflected by a reduction (improvement) in Conduct Problem Subscale scores and by an increase (improvement) in prosocial subscale scores.26,27 In addition, risperidone was safe and well tolerated with few patients discontinuing treatment.
Although these short-term data are promising, DBDs are chronic conditions that may require long periods of treatment. Hence, the reconsideration of appropriate therapy and psychosocial intervention is important. This 48-week, open-label (OL) extension of the Canadian study was undertaken to gather data primarily on the long-term safety of risperidone. The secondary objective was to investigate the long-term efficacy of risperidone for the treatment of DBDs in this population.
| METHODS |
|---|
|
|
|---|
Study Design
This study was a 48-week, OL extension study designed to gather safety and efficacy data on the long-term use of risperidone in children who had participated in the 6-week, multicenter, randomized, DB, placebo-controlled study. Only those participants who had completed at least 2 weeks of the DB study were eligible to enter the OL extension study. Participants received oral solution risperidone at a daily dose of between 0.02 and 0.06 mg/kg for a maximum of 48 weeks. Participants on risperidone during the DB study would have had a maximum of 54 weeks (6 plus 48) of risperidone therapy. Study visits were scheduled at entry, weekly for the first month, and monthly for the remaining 11 months. For those participants who entered the OL extension study within 10 days of completing the DB study, the final safety and efficacy assessments made during the earlier trial served as baseline data at entry for this study. Otherwise, participants were reassessed. This follow-up study was conducted in accordance with the Declaration of Helsinki as revised in 1983 and was approved by the institutional review boards at each participating center and by the Health Protection Branch of Canada.
Participants
Participants who had completed at least 2 weeks of treatment in the DB study and continued to meet all selection criteria of that study were eligible to participate in this OL follow-up study. Children, aged 5 to 12 years inclusive, who had: 1) a DSM-IV Axis I diagnosis of CD, ODD, or DBD-NOS; 2) a parent-assessed rating of
24 in the Conduct Problem Subscale of the N-CBRF28; 3) a DSM-IV Axis II diagnosis of mild or moderate mental retardation or borderline intellectual functioning with an IQ
36 and
84; and 4) a score of
84 on the Vineland Adaptive Behavior Scale.30 In addition, participants had to be outpatients who were physically healthy and have a behavioral problem sufficiently severe that the investigator believed antipsychotic treatment was warranted at entry to the DB trial. Individuals with ADHD were eligible provided they met all other selection criteria. A responsible person was required to accompany the participant at clinic visits, provide reliable assessments, and dispense medications.
Subjects were excluded who had a diagnosis of pervasive development disorder, schizophrenia, other psychotic disorder, head injury or seizure disorder, history of TD, neuroleptic neuropathy, known hypersensitivity to neuroleptics or risperidone, tested positive for human immunodeficiency virus, abnormal laboratory values, or who were using a prohibited medication. Subjects were also excluded from this extension study if >3 weeks had elapsed since their participation in the previous DB trial or if, during that trial, they experienced a hypersensitivity reaction to trial medication, EPS not controlled by medication, an adverse event possibly related to risperidone, or one for which they were withdrawn. Participants provided verbal and, if capable, written informed consent; signed consent was also obtained from the participants legal representative.
Study and Other Medications
Risperidone was provided by Janssen Research Foundation (Toronto, Canada) as an oral solution of 1.0 mg/mL to be administered once daily in the morning at an initial dose of 0.01 mg/kg on days 1 and 2, and increased to 0.02 mg/kg on day 3. Thereafter, the dose could be adjusted by the investigator at weekly intervals to a maximal allowable dose of 0.06 mg/kg/d; increments were not to exceed 0.02 mg/kg/d. For those with breakthrough symptoms, the dosing schedule could be changed to a twice-daily regimen.
Medications used to treat EPS were to be discontinued at DB trial entry. For those with emergent EPS during the trial, the dose of risperidone could be reduced; the rate of dose reduction was not limited. Anticholingeric agents were permitted only in cases where dose reduction resulted in deterioration of behavioral symptoms or failed to improve EPS and the Extrapyramidal Symptoms Rating Scale (ESRS) had been completed.31
Prohibited medications included any antipsychotics other than the study medication, anticonvulsants, antidepressants, lithium, clonidine, guanfacine, carbamazepine, valproic acid, or cholinesterase inhibitors. Psychostimulants, including methlyphenidate, pemoline, and dexedrine, were allowed for the treatment of ADHD provided the participant was already taking them at a stable dosage 30 days before participation in the previous DB study, and every attempt was made to keep the dose constant throughout the DB and OL extension trials. Sedative/hypnotic medications were allowed provided that the dose and frequency of use were kept to a minimum. Behavior intervention therapies were also allowed during the OL extension trial.
Safety Outcome Measures
Adverse event data and vital signs (pulse, blood pressure, respiration, temperature measured sitting after 5 minutes rest) were collected at each visit. EPSs were assessed using the ESRS at each visit for the first 6 months and again at month 9 and at the beginning of month 12.31 Body weight (wearing similar clothing) was measured and laboratory tests (hematology: complete blood count with differential and platelets; biochemistry: including electrolytes, liver function tests; urinalysis by dipstick, followed by microscopic examination if positive) including prolactin levels, were conducted at baseline, week 4, and at months 3, 6, 9 and at the beginning of month 12; sedation was also assessed at these visits and at the week 1 visit, using a caregiver completed visual analog scale (VAS). Cognitive tests and an electrocardiogram (ECG) were performed at OL baseline, and at months 6 and at the beginning of month 12. Cognition was assessed using the Continuous Performance Task (CPT) and a modification of the California Verbal Learning Test-Childrens Version (mCVLT-CV).32 A pediatric cardiologist reread all ECGs and examined them for any abnormality (heart rate, QRS complex, QT interval, QTc interval, JT interval). Fridericias formula (which is more appropriate for higher heart rates seen in children) was used to calculate the QTc interval.
Secondary Outcome Measures
Efficacy assessments were conducted at OL baseline, at weeks 1 and 4, at months 3, 6, and 9, and at the beginning of month 12. The primary efficacy outcome was the change from baseline score at OL study endpoint in the Conduct Problem Subscale of the parent/caregiver rated N-CBRF.28,33 Secondary efficacy outcomes included change from baseline score at OL study endpoint in: the Positive Social Behavior and Problem Behavior subscales of the N-CBRF; the investigators Clinical Global Impression (CGI) of the severity of the participants condition; and a VAS rating by the parent/caregiver of the most troublesome symptom.
Statistical Analyses
Participants baseline patient and disease characteristics were extracted from the DB database. All participants who took at least 1 dose of study medication were included in the safety analysis; those participants for whom any postbaseline assessment data were also available were included in the primary efficacy analysis. Changes from baseline (± SE) were calculated using both the DB baseline when all patients were risperidone-naïve, and the OL baseline at the start of OL extension study. Outcome measures were compared based on whether patients had received placebo or active treatment in the previous DB trialie, between those who were risperidone-naïve at entry and those who had been treated with risperidone before the start of the OL extension study. All statistical tests were interpreted at the 5% significance level (2-tailed).
All adverse events were tabulated by type and incidence and included those that were newly emergent or reemergent events in comparison with the previous DB study and those that had worsened since the start of the OL trial. An event was considered serious if it was fatal; life-threatening; significantly, persistently or permanently disabling; required intervention to prevent permanent impairment; or required hospitalization. Descriptive statistics were provided for vital signs, cognitive function tests, and for clinical laboratory data. In the latter case, pretreatment and posttreatment frequencies, including those for important abnormalities, were also calculated. Total ESRS scores were summarized using descriptive statistics; changes from baseline for each visit were compared using Wilcoxon signed rank tests. Two-sided paired t tests were used to compare changes from baseline for CGI score frequency counts, within group ECG parameters, and for sedation VAS scores. Two-sided paired t tests were also used to compare changes from baseline for both primary and secondary efficacy outcomes. In cases of nonnormality, Wilcoxon signed rank tests were performed.
| RESULTS |
|---|
|
|
|---|
Participants
A total of 77 participants, at 9 Canadian investigational sites, entered this OL extension study. Of these, 39 had received placebo (ie, were risperidone-naïve at entry), and 38 received risperidone in the previous DB trial. Both groups had similar baseline demographic and disease characteristics (Table 1). Participants, three-quarters of whom were male, ranged in age from 5 to 12 years with a mean age of 8.7 years. Approximately 60% of participants had a DSM-IV diagnosis of ODD. Most (79%) suffered from ADHD. Half (50.6%) of all participants had borderline intellectual function; the remainder were either mildly or moderately mentally retarded. Most (85.7%) lived with their parents.
|
Most participants (78%) completed the 1-year study. Seventeen participants (22%) withdrew before completion; of these, 9 received placebo and 8 received risperidone in the previous DB trial (Table 2). The most common reasons for withdrawal were insufficient response, lost to follow-up, closure of the investigational site, and noncompliance.
|
Study and Other Medications
Participants received treatment with risperidone for an average of 321.8 ± 8.3 days (range: 36417 days). The mean dose of risperidone, including days off drug, was 0.041 ± 0.001 mg/kg/d or 1.38 ± 0.057 mg/d; the most frequently used dose was 0.02 mg/kg/d or 1.5 mg/d. Eighteen participants (23.4%) took <1.0 mg/kg/d, 32 participants (41.6%) took 1.0 or more but <1.5 mg/kg/d, and 27 (35.1%) took 1.5 mg/kg/d or more. The most common concomitant medications, taken by 62.3% of participants, were psychostimulants to treat ADHD. Analgesics (48.1%) and antibiotics (42.9%) were also commonly used. No behavior interventions were instituted during this time period.
Safety Outcomes
Adverse Events
Seventy-six (98.7%) of the 77 subjects participating in this year-long, OL extension trial experienced a newly emergent, reemergent, or worsening adverse event (Table 3). The most commonly reported events were somnolence (51.9%), headache (37.7%), and weight gain (36.4%). Adverse events were generally considered by the investigator to be either mild or moderate in severity. Twenty-three participants (29.9%) experienced a total of 17 different adverse events that were judged by the investigator to be severe. Thirteen of these events were uniqueeach occurring in only 1 patient. Only 4 events judged to be severe occurred in >1 patient: weight gain (n = 4, 5.2%), emotional lability (n = 2, 2.6%), aggressive reaction (n = 2, 2.6%), and agitation (n = 2, 2.6%). Each of these events was deemed to be either possibly or definitely related to the study medication. Two participants (2.6%) discontinued treatment because of an adverse event: 1 case of mild headache that persisted for 4 days after 256 days of risperidone therapy and 1 case of dyspnea and headache after 235 days of therapy. No participant experienced a serious adverse event during this trial.
|
Prolactin Levels
Elevated prolactin levels were reported as a newly emergent, reemergent, or worsening adverse events for a total of 15 participants (19.5%; Table 3). There were few physical signs of the manifestations of this event: 2 participants experienced transient amenorrhea (1 mild, 1 moderate), neither of whom had coincident abnormal prolactin levels; no participant experienced gynecomastia.
In those participants who were risperidone-naïve at entry, mean prolactin levels measured at the OL baseline visit were not significantly different from those measured at the DB baseline (Fig 1, A and B). In the risperidone-naïve group, levels for both male and female participants increased after the initiation of risperidone treatment, peaked at week 4 and declined over the remaining months of treatment. OL mean baseline prolactin levels were 5.85 ng/mL and 8.33 ng/mL for males and females, respectively; at week 4, levels of 28.4 ng/mL and 39.6 ng/mL were reached. At the final assessment visit, prolactin levels had decreased to a mean of 18.6 ng/mL for boys and 21.3 ng/mL for girls (P < .001 and P > .05, respectively, in the change from OL baseline). These levels were just slightly above the normal range for males of 2 to 18 ng/mL and within the normal range for nonpregnant females of 3 to 30 ng/mL.
|
Participants who had been treated with risperidone in the previous DB study had significantly higher OL mean baseline prolactin levels than those who were risperidone-naïve at entry (P
.001; Fig 1, A and B). These OL levels were also significantly higher than those recorded at the DB baseline when all participants were risperidone-naive (P
.001). For both male and female participants who had been previously treated with risperidone, mean prolactin levels declined from OL baseline levels throughout the 11 months of treatment. At the final OL assessment visit, levels were reduced to 17.7 ng/mL (from 29.8 ng/mL) for boys and 15.4 ng/mL (from 31.0 ng/mL) for girls (P = .005 and P = .013, respectively, in the change from OL baseline). There were no discontinuations nor any physical symptoms noted attributable to hyperprolactinemia.
Body Weight
Increased appetite was reported as an adverse event in 21 (27.3%) participants; 10 reported increased appetite alone without reporting increased weight, whereas 11 reported both increased appetite and weight increase. Increased weight was reported in 31 (40.3%) participants; 20 of whom reported weight increase only and 11 reported both increased appetite and weight increase.
These 2 events were reported in 14 (18.2%) participants and 28 (36.4%) participants, respectively (Table 3). Mean body weight increased significantly for all participants (P < .001); overall increases were similar regardless of whether participants had been risperidone-naïve at entry or had been treated with risperidone in the previous trial. At the end of the 11-month OL study, mean weight had increased by 7.1 kg from an OL baseline weight of 31.8 kg and by 8.5 kg from a DB baseline weight of 30.7 kg. The summary of weight change from DB baseline to OL endpoint is presented in Table 6. Weight change occurring from OL baseline for each visit during the OL trial is presented in Table 7. Weight increased at a faster rate during the first 3 months of treatment and leveled off as treatment progressed.
|
|
Other Laboratory and Physical Findings
There were no laboratory abnormalities considered to be clinically relevant. In addition, there were no clinically relevant mean changes from baseline in any vital signs, other physical findings, or ECG recordings.
EPSs
Twenty participants (26.0%) experienced some type of EPS during this study; none withdrew for this reason. Those symptoms most frequently reported included hypertonia (11.7%), involuntary muscle contractions (7.8%), hyperkinesia (3.9%), and tremor (3.9%). No participant experienced severe EPSs. Symptoms were judged to be either mild or moderate in severity. No cases of TD were reported.
The total mean ESRS score for all participant at DB baseline was 0.41 ± 0.13 (range: 0.08.0). During the OL study, statistically significant differences in the change from DB baseline were evident only at the week 4 and month 3 assessments: +0.80 ± 0.16 and + 0.86 ± 0.20, respectively, both P < .05. These changes were not considered clinically significant by the authors. Thereafter, no statistically significant changes in total ESRS from DB baseline were evident. At study endpoint, the mean change from baseline in total ESRS for all patients was 0.53 ± 0.11. As rated on the investigators CGI of ESRS, few changes from baseline were recorded in the rates of dyskinesia, Parkinsonism, or dystonia.
Sedation and Cognition
Participants who were risperidone-naïve at entry experienced an increase in the level of sedation. Those who received risperidone during DB treatment showed no significant change at endpoint.
For most parameters assessed, there were statistically significant increases (indicating improvement) relative to both OL and DB baselines in the mean scores of the mCVLT, which is a test of verbal learning and memory regardless of the treatment, including psychostimulants, participants received during the previous DB study. In addition, statistically significant improvements over baseline were also seen for some CPT (which is a test for attention and impulsivity) parameters. Overall, no deterioration of cognitive function was observed while participants were treated with risperidone.
Efficacy Outcomes
Conduct Problem Subscale of the N-CBRF
Baseline scores on the Conduct Problem Subscale at the start of the previous DB study were similar for both treatment groups: mean values of 33.5 and 33.3 were recorded for placebo- and risperidone-treated participants, respectively (Table 4). By the time of the OL baseline visit, mean Conduct Problem Subscale scores were lower in those who had been treated with risperidone than in those who remained risperidone-naïve (17.5 and 26.1, respectively; Table 5). Within 1 week of receiving daily risperidone therapy, those participants who had been risperidone-naïve at OL entry showed a rapid improvement in the Conduct Problem Subscale score (Fig 2). At the week 1 assessment, the mean change from baseline for those who had been risperidone-naïve at OL entry was similar in magnitude to the change from DB baseline recorded for participants who had received risperidone in the DB study. This mean improvement was sustained in both groups throughout the remainder of the OL study.
|
|
At study endpoint, those participants who had been risperidone-naïve at OL entry experienced a highly significant mean decrease from OL baseline in the mean conduct problem subscale score of 10.6 ± 2.18 (P < .001; Table 6). This decrease was comparable to the decrease from DB baseline experienced by participants who were treated with risperidone during the DB period (Table 4). The response to risperidone in the OL trial remained stable in those participants who had been treated with risperidone in the previous DB trial; in this group, the mean change at study endpoint from OL baseline was a nonsignificant decrease of 1.26 ± 1.45 (Table 5).
Other Subscales of the N-CBRF
Similar patterns of responses were observed when other subscales of the N-CBRF were examined (Tables 4 and 5). Baseline scores for each of the other 7 N-CBRF subscale were similar between groups at the start of the previous DB study. By the start of the OL extension study, risperidone-naïve participants had similar baseline scores to those recorded at the DB baseline, whereas participants who had received risperidone in the previous trial showed improvements in each score. At the end of the OL trial, all participants experienced significant improvements over DB baseline in each subscale.
Other Efficacy Measures
At DB baseline, 68% of participants had a CGI assessment rated as marked, severe, or extremely severe. By study endpoint, only 17% of participants (15% of whom had received placebo and 19% of whom had been treated with risperidone in the previous study) had this severe an assessment; 63% of participants had symptoms rated as either none, very mild, or mild. Similarly, highly significant decreases from baseline in the VAS rating of the most troublesome symptom (often identified as either aggression [hitting, fighting, or temper tantrums]) were observed by study endpoint after 12 months of risperidone therapy. For those participants who had received placebo in the previous study, a mean decrease of 47.1 ± 4.87 mm from a DB baseline of 79.4 ± 2.69 mm was observed (P < .001). In those who had received risperidone, a mean decrease of 43.5 ± 4.57 mm from a DB baseline of 79.3 ± 3.66 mm was observed (P < .001).
Subgroup Analyses of the Primary Efficacy Outcome
Five subgroup analyses of the primary efficacy outcome were performed. These included analysis by diagnosis (CD, ODD, and DBD-NOS), degree of mental retardation (borderline, mild, moderate), and presence or absence of somnolence, ADHD, and psychostimulants. The results showed that the efficacy of risperidone was not affected by type of disorder, level of retardation, presence/absence of somnolence or ADHD, or use of psychostimulants.
| DISCUSSION |
|---|
|
|
|---|
DBDs are typically chronic conditions that require long-term management. Particularly in children with subaverage IQs who are at high risk for various difficulties in their adult years, a therapeutic intervention that proved safe and effective over the long-term could form an important component of the management plan for these children.
Data from short-term, DB studies indicated that risperidone was both safe and effective in the treatment of disruptive behaviors in children with subaverage IQs.2527 Data collected from this OL extension study with daily risperidone treatment for an average of 321 days, and that of a separate, parallel study conducted in the United States,29 demonstrate that this population remained free of the more serious long-term safety risks associated with the use of conventional neuroleptics. Furthermore, the efficacy of oral solution risperidone in this population, which was evident within 1 week of treatment, was sustained over the long-term.
Patients
All participants in this study population had below normal intellectual function although, in keeping with the study design, none were severely retarded. Disruptive behavior and CD-type symptoms of aggression, self-injury, and destructiveness were evinced by the children enrolled in the DB trial.26 Typical of this family of disorders, the study population was primarily male, and most (61/77) suffered from comorbid ADHD. Various studies indicate that the comorbidity distribution in our sample was similar to other clinical studies.34,35 Comorbidity among DBDs is the rule rather than exception, and the results of this study may be generalizable to many patient samples with DBDs.
Safety Outcomes
Risperidone, at a mean dose of 1.38 mg/d (0.04 mg/kg/d), was well tolerated throughout this 11-month, OL extension study. The most common adverse events were somnolence, headache, and weight gain, which were rated as mild or moderate in severity in the opinion of the investigator. Approximately 80% of participants completed the full course of treatment. Only 2 participants discontinued treatment because an adverse event: headache in 1 case, and headache and dyspnea in the other, and both after >200 days of risperidone therapy. Although one quarter of participants experienced EPSs at some point during this trial, no participants experienced severe EPSs, none withdrew for this reason, and none experienced TD.
There is limited published literature on the incidence of TD in a pediatric population with atypical antipsychotics. Long-term treatment with haloperidol and other traditional (typical) antipsychotics are associated with high incidences of TD in subjects with mental retardation. The incidence of TD is estimated to be between 7% and 12% in subjects with mental retardation receiving long-term treatment with conventional neuroleptics for <1.5 years.14
In addition, no significant changes from DB baseline in total ESRS scores were evident at study endpoint. EPS seemed to be easily managed and not unduly distressful to the subject.
In this study, risperidone therapy was associated with 3 adverse outcomes: elevated prolactin levels, weight gain, and sedation. None of these events met the definition of a serious adverse event. In fact, no serious adverse events were reported at any point during this year-long trial. Mean prolactin levels became elevated after initiation of risperidone treatment, peaked (mean: <40 ng/mL) after 4 weeks of treatment, and then declined throughout the remaining 11 months of the study. By study endpoint, mean prolactin levels in female participants were not statistically different from baseline values; only in male participants was the difference still statistically significant. However, mean prolactin levels at OL endpoint for both males and females were <20 ng/mL, which are within the normal laboratory ranges. Importantly, although prolactin levels remained elevated at OL endpoint compared with baseline, no physical manifestations of this laboratory abnormality were evident at any point during the trial. It is hypothesized that had the OL extension continued, these prolactin levels may have decreased further. There was no evidence of gynecomastia, and the 2 cases of transient amenorrhea that were reported occurred in participants with normal prolactin levels. It is not unusual to see abnormal periods of menses in pubertal girls in whom the menstrual cycle is not fully established.
Weight gain was more pronounced during the first months of risperidone therapy. By study endpoint compared with DB baseline, participants had gained an average of 8.5 kg. Group mean weight change from OL baseline to endpoint was 7.1 kg (Table 7). Not all of this weight gain should be attributed to risperidone usage. According to the National Center for Health Statistics, 3.85 kg or slightly less than half of the weight gained could be attributed to normal growth over this period.36 It should also be noted that there were no abnormal glucose values or any side effect that could be attributed to abnormal glucose metabolism experienced by participants over the 11-month trial. Clinicians considering a trial with risperidone to treat symptoms of disruptive behaviors should encourage children and caregivers to institute a dietary regimen and plan of physical activity for the child as a way of potentially preventing or diminishing the weight gain that may occur with risperidone. Although physicians need to be aware of possible weight changes that may occur in patients receiving risperidone, it should be noted that, in a meta-analysis of weight changes in adults, risperidone was at the lower end of the spectrum of antipsychotic agents.37 In a systematic review of the atypical agents, risperidone was found to be associated with less weight gain than clozapine, olanzapine, or quetiapine.38
Somnolence was the most frequently reported adverse event; however, no participants discontinued treatment for this reason. One could speculate that sedation could have an impact on cognitive function, yet based on results obtained on the mCVLT and CPT, there was no evidence of cognitive deterioration associated with risperidone use. On the contrary, marked improvements in cognitive function were observed on both tests of cognitive function. The mCVLT is a measure of verbal learning and memory and the CPT is a test for attention, vigilance and concentration. Although this study was not designed to specifically study the effects of risperidone on cognitive function, these results cannot rule out the positive cognitive changes identified with risperidone treatment may likely have a clinically meaningful impact. Because the key target symptoms to be treated in this trial included aggression, hitting, agitation, and temper tantrums, some clinicians would find sedation a desired effect provided cognition is not affected. Subgroup analyses examining the effect of risperidone in children with and without sedation on the primary behavioral scales showed comparable efficacy irrespective of sedation. This indicates that risperidone is effective in treating the symptoms of DBDs independent of the side effect of sedation. No clinically relevant changes from baseline were observed in any other safety outcomes measured. A similar safety profile was evident in the 1-year risperidone extension study conducted in the United States.29
Sixty-one of 77 participants in the OL trial had a preexisting diagnosis of ADHD. The results of this OL trial indicate that there were no deleterious side effects seen when combining risperidone with psychostimulants for an extended period of time (up to 52 weeks for children on risperidone during the DB trial). No noted drug interactions with psychostimulants were reported, and drug reactions were not expected. Risperidone is extensively being metabolized in the liver by cytochrome P450IID6. Methylphenidate, the most commonly taken psychostimulant (>80% of patients taking psychostimulants) in the trials, is not a known inhibitor of CYP enzymes, and as such does not raise a particular concern for pharmacokinetic interactions. In the Aman et al27 and present trials, there was no difference observed in the plasma active moiety levels in participants taking methylphenidate and those not taking this concomitant medication.
Efficacy Parameters
Based on the primary outcome of change from baseline in the Conduct Problem Subscale of the N-CBRF, the efficacy of oral risperidone for the treatment of disruptive behaviors in children with subaverage IQs was evident within 1 week of treatment, and remained undiminished over the duration of the trial.
In those participants who had been risperidone-naïve at OL entry, changes from OL baseline to study endpoint were highly significant for both the primary efficacy outcome and all secondary outcomes examined. Improvements in symptoms of disruptive behavior for participants on placebo during the DB trial were readily apparent within 1 week of beginning risperidone treatment. Between OL weeks 1 to 4, the former placebo patients had the same magnitude of improvement in Conduct Problem Subscale scores as those participants who had received risperidone in the DB trial. Also, this improvement was sustainable over a total time period of 11 months.
In those participants who had been treated with risperidone in the previous DB study, improvements observed in the DB study were sustained throughout the 11-month OL extension study. In either case, after 1 year of risperidone therapy, participants had significantly less severe and fewer symptoms of disruptive behavior such as aggression, impulsivity, hyperactivity, and self-injurious behavior. They had significantly improved outcomes on subscales measuring self-isolated/ritualistic and overly sensitive behavior than they had before receiving treatment. They were also significantly more compliant and sociable. By study endpoint, VAS ratings of the most troublesome symptom (usually hitting, aggression, temper tantrums) had decreased by more than half of baseline values, and 4 times fewer participants had a CGI rating of severe. These results are in agreement with those reported for the parallel OL study conducted in the United States.29
The control of aggressive, self-injurious behavior and the symptoms of ODD and CD may contribute to the prevention of the development of antisocial personality disorder, whereas the institutionalization of children with disruptive behaviors may compromise their social and environmental adjustment and growth.13,3941 Once symptoms are controlled, these children may be able to better function at home and in a regular school environment. Certainly the risk/benefit ratio regarding the use of risperidone to control disruptive/aggressive behavior should be weighed before commencing treatment. This long-term OL trial shows positive effects on behavior with minimal side effects compared with reports in the literature regarding the use of haloperidol, clonidine, and lithium.16,24,4248
Based on various a posteriori subgroup analyses of the primary behavioral outcome variable, low-dose risperidone was equally effective for the treatment of symptoms of CD, ODD, and DBD-NOS, and in participants who had borderline intellectual function, or were mildly or moderately mentally retarded. Furthermore, the efficacy of risperidone was not diminished by the presence of somnolence or ADHD, or by the use of psychostimulants.
| CONCLUSION |
|---|
|
|
|---|
Risperidone, administered as an oral solution at a mean dose of 1.38 mg/d (range: 0.020.06 mg/kg/d) for 1 year, was well tolerated, safe, and showed maintenance of effect in the treatment of DBDs in children aged 5 to 12 years with subaverage IQs.
|
| ACKNOWLEDGMENTS |
|---|
This trial was supported by the Janssen Research Foundation.
We thank Margaret Steele, Deborah Ellison, Jennifer Williams (Ontario), Declan Quinn, Maureen Bingham (Saskatchewan), Alan Carroll, Suneeta Monga, Elizabeth Chomin (Alberta), David Ng, Herman Gelber, Keith Cameron (Ontario), Margaret Secord, Ruth Barton, Kathy Renee (Ontario), and Cathie Keeler (Nova Scotia). Special thanks go to Doreen Stern, Jean Twiner, Fiona Dunbar, and Ben Lyons.
| FOOTNOTES |
|---|
Received for publication Feb 19, 2002; Accepted May 17, 2002.
Reprint requests to (A.T.) University of Toronto, Scarborough Hospital, 3050 Lawrence Ave E, Scarborough Ontario Canada M1P 2V5. E-mail: aturgay{at}tsh.to
Members of the Conduct Study group are Allan Carroll, MD, Jovan Simeon, MD, William Mahoney, MD, Umesh Jain, MD, John Strang, PhD, and Rhodri Evans, MD.
Drs Turgay, Snyder, and Fisman have received compensation from Janssen-Ortho Inc for participating in this trial and have presented for Janssen-Ortho Inc and participated in its advisory boards.
| REFERENCES |
|---|
|
|
|---|
- Steiner H. Practice parameters for the assessment and treatment of children and adolescents with conduct disorder. J Am Acad Child Adolesc Psychiatry.1997; 36(suppl) :122S 139S[CrossRef][Medline]
- Bauermeister JJ, Canino G, Bird H. Epidemiology of disruptive behavior disorders. Child Adolesc Psychiatry Clin North Am.1994; 3 :177 194
- Dosen A. Diagnosis and treatment of psychiatric and behavioral disorders in mentally retarded individuals: the state of the art. J Intellect Disabil Res.1993; 37(suppl) :1 7
- Gillberg C, Persson E, Grufman N, Themner U. Psychiatric disorders in mildly and severely mentally retarded urban children and adolescents: epidemiological aspects.
Br J Psychiatry.1986; 149
:68
74
[Abstract/Free Full Text] - Campbell SB, Ewing LJ. Follow-up of hard-to-manage preschoolers: adjustment at age 9 and predictors of continuing symptoms. J Child Psychol Psychiatry.1990; 31 :871 889[Web of Science][Medline]
- Moffitt TE. Juvenile delinquency and attention deficit disorder: boys developmental trajectories from age 3 to age 15. Child Dev.1990; 61 :893 910[CrossRef][Web of Science][Medline]
- Kazdin AE. Conduct Disorder in Childhood and Adolescence. 2nd ed. Thousand Oaks, CA: Sage Publications; 1995
- Angold A, Costello EJ, Erkanli A. Comorbidity. J Child Psychol Psychiatry.1999; 40 :57 87[CrossRef][Web of Science][Medline]
- American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th ed. Washington DC: American Psychiatric Association; 1994
- Bardone AM, Moffitt TE, Caspi A, Dickson N, Stanton WR, Silva PA. Adult physical health outcomes of adolescent girls with conduct disorder, depression, and anxiety. J Am Acad Child Adolesc Psychiatry.1998; 37 :594 601[CrossRef][Web of Science][Medline]
- Satterfield JH, Schell A. A prospective study of hyperactive boys with conduct problems and normal boys: adolescent and adult criminality. J Am Acad Child Adolesc Psychiatry.1997; 36 :1726 1735[CrossRef][Web of Science][Medline]
- Robins LN. Sturdy childhood predictors of adult antisocial behaviour: replications from longitudinal studies. Psychol Med.1978; 8 :611 622[Web of Science][Medline]
- Robins LN. Conduct disorder. J Child Psychol Psychiatry.1991; 32 :193 212[Web of Science][Medline]
- Kaplan SL, Simms RM, Busner J. Prescribing practices of outpatient child psychiatrists. J Am Acad Child Adolesc Psychiatry.1994; 33 :35 44[CrossRef][Web of Science][Medline]
- Aman MG, Singh NN. Patterns of drug use, methodological considerations, measurement techniques, and future trends. In: Aman MG, Singh NN, eds. Psychopharmacology of the Developmental Disabilities. New York, NY: Springer Verlag; 1988:128
- Campbell M, Small AM, Green WH, et al. Behavioral efficacy of haloperidol and lithium carbonate. A comparison in hospitalized aggressive children with conduct disorder.
Arch Gen Psychiatry.1984; 41
:650
656
[Abstract/Free Full Text] - Campbell M, Adams PB, Small AM, et al. Lithium in hospitalized aggressive children with conduct disorder: a double-blind and placebo-controlled study. J Am Acad Child Adolesc Psychiatry.1995; 34 :445 453[CrossRef][Web of Science][Medline]
- Malone RP, Delaney MA, Luebbert JF, Cater J, Campbell M. A double-blind placebo-controlled study of lithium in hospitalized aggressive children and adolescents with conduct disorder.
Arch Gen Psychiatry.2000; 57
:649
654
[Abstract/Free Full Text] - Klein RG. Preliminary results: lithium effects in conduct disorders. Proceedings 144th Annual Meeting of the APA;1991 :119 120
- Rifkin A, Karajgi B, Dicker R, et al. Lithium treatment of conduct disorders in adolescents. Am J Psychiatry.1997; 154 :554 555[Abstract]
- Klein RG, Abikoff H, Klass E, Ganeles D, Seese LM, Pollack S. Clinical efficacy of methylphenidate in conduct disorder with and without attention deficit hyperactivity disorder.
Arch Gen Psychiatry1997; 54
:1073
1080
[Abstract/Free Full Text] - Aman MG, Marks RE, Turbott SH, Wilsher CP, Merry SN. Clinical effects of methylphenidate and thioridazine in intellectually subaverage children. J Am Acad Child Adolesc Psychiatry.1991; 30 :246 256[Web of Science][Medline]
- Aman MG, Madrid A. Atypical antipsychotics in persons with developmental disabilities. Mental Retardation Dev Disabil Res Rev.1999; 5 :253 263
- Findling RL, McNamara NK, Branicky LA, Schluchter MD, Lemon E, Blumer JL. A double-blind pilot study of risperidone in the treatment of conduct disorder. J Am Acad Child Adolesc Psychiatry.2000; 39 :509 516[CrossRef][Web of Science][Medline]
- Van Bellinghen M, De Troch C. Risperidone in the treatment of behavioral disturbances in children and adolescents with borderline intellectual functioning: a double-blind, placebo-controlled pilot trial. J Child Adolesc Pyschopharmacol.2001; 11 :5 13
- Snyder R, Turgay A, Aman MG, Binder C, Fisman S, Carroll A, and the Risperidone Conduct Study Group. Effects of risperidone in conduct and disruptive behavior disorder in children with subaverage IQs. J Am Acad Child Adolesc Psychiatry. In press
- Aman MG, De Smedt G, Derivan A, Lyons B, Findling RL, and the Risperidone Disruptive Behavior Study Group. Risperidone treatment of children with disruptive behavior symptoms and subaverage IQs: a double-blind, placebo-controlled study. Am J Psychiatry. In press
- Aman MG, Tassé MJ, Rojahn J, Hammer D. The Nisonger CBRF: a child behavior rating form for children with developmental disabilities. Res Dev Disabil.1996; 17 :41 57[CrossRef][Web of Science][Medline]
- Findling RL, Aman MG, De Smedt G, Derivan MD, the Risperidone Disruptive Behavior Study Group. A long-term open-label study of risperidone in children with severe disruptive behaviors and subaverage IQs. Am J Psychiatry. In press
- Sparrow SS, Balla DA, Cicchetti CV. Vineland Adaptive Behavior Scales. Circle Pines, MN: American Guidance Service Inc; 1984
- Chouinard G, Ross-Chouinard A, Annable L, Jones BD. The extrapyramidal symptom rating scale. Can J Neurol Sci.1980; 7 :233
- Delis DC, Kramer JH, Kaplan E, Ober BA. California Verbal Learning Test, Childrens Version, Manual. Orlando, FL: Psychological Corp/Harcourt, Brace & Co; 1998
- Tassé MJ, Aman MG, Hammer D, Rojahn J. The Nisonger Child Behavior Rating Form: age and gender effects and norms. Res Dev Disabil.1996; 17 :59 75[CrossRef][Web of Science][Medline]
- Kuhne M, Schachar R, Tannock R. Impact of comorbid oppositional or conduct problems on attention-deficit hyperactivity disorder. J Am Acad Child Adolesc Psychiatry.1997; 36 :1715 1725[CrossRef][Web of Science][Medline]
- Lalonde J, Turgay A, Hudson JI. Attention-deficit hyperactivity disorder subtypes and comorbid disruptive behavior disorders in a child and adolescent mental health clinic. Can J Psychiatry.1998; 43 :623 628[Web of Science][Medline]
- Hamill PV, Drizd TA, Johnson CL, Reed RB, Roche AF, Moore WM. Physical growth: National Center for Health Statistics percentiles.
Am J Clin Nutr.1979; 32
:607
629
[Abstract/Free Full Text] - Allison DB, Mentore JL, Heo M, et al. Antipsychotic-induced weight gain: a comprehensive research synthesis.
Am J Psychiatry.1999; 156
:1686
1696
[Abstract/Free Full Text] - Taylor DM, McAskill R. Atypical antipsychotics and weight gaina systematic review. Acta Psychiatr Scand.2000; 101 :416 432[CrossRef][Web of Science][Medline]
- Hechtman L. Attention deficit hyperactivity disorder. In: Hechtman L, ed. Do They Grow Out of it? Long-Term Outcome of Childhood Disorders. Washington, DC: American Psychiatric Press; 1996:1738
- Robins LN. Deviant Children Grown Up. Baltimore, MD: Williams and Wilkins; 1966
- Weis G, Hechtman L. Hyperactive Children Grown Up. New York, NY: Guilford Press; 1993
- Buitelaar JK. Open-label treatment with risperidone of 26 psychiatrically-hospitalized children and adolescents with mixed diagnoses and aggressive behavior. J Child Adolesc Psychopharmacol.2000; 10 :19 26[Web of Science][Medline]
- Campbell M, Gonzalez NM, Silva RR. The pharmacologic treatment of conduct disorders and rage outbursts. Psychiatry Clin North Am1992; 15 :69 85
- Campbell M, Rapoport JL, Simpson GM. Antipsychotics in children and adolescents. J Am Acad Child Adolesc Psychiatry.1999; 38 :537 545[CrossRef][Web of Science][Medline]
- Kazdin AE. Treatment of antisocial behavior in children: current status and future directions. Psychol Bull.1987; 102 :187 203[CrossRef][Web of Science][Medline]
- Kazdin A, Wassell G. Therapeutic changes in children, parents, and families resulting from treatment of children with conduct problems. J Am Acad Child Adolesc Psychiatry.2000; 39 :414 420[CrossRef][Web of Science][Medline]
- Kemph JP, DeVane CL, Levin GM, Jarecke R, Miller RL. Treatment of aggressive children with clonidine: results of an open pilot study. J Am Acad Child Adolesc Psychiatry.1993; 32 :577 581[Web of Science][Medline]
- Kewley G. Risperidone in comorbid ADHD and ODD/CD. J Am Acad Child Adolesc Psychiatry.1999; 38 :1327 1328
PEDIATRICS (ISSN 1098-4275). ©2002 by the American Academy of Pediatrics
This article has been cited by other articles:
![]() |
C. U. Correll Antipsychotic Use in Children and Adolescents: Minimizing Adverse Effects to Maximize Outcomes Focus, June 1, 2008; 6(3): 368 - 378. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Kumra, J. V. Oberstar, L. Sikich, R. L. Findling, J. M. McClellan, S. Vinogradov, and S. Charles Schulz Efficacy and Tolerability of Second-Generation Antipsychotics in Children and Adolescents With Schizophrenia Schizophr Bull, January 1, 2008; 34(1): 60 - 71. [Abstract] [Full Text] [PDF] |
||||
![]() |
O. Sabuncuoglu Risperidone-to-methylphenidate switch reaction in children: three cases J Psychopharmacol, March 1, 2007; 21(2): 216 - 219. [Abstract] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||









