PEDIATRICS Vol. 111 No. 3 March 2003, pp. 573-578
Intravenous Bisphosphonate Therapy in Children With Osteogenesis Imperfecta
* Department of Genetics and Center for Human Genetics
Department of Pediatrics
General Clinical Research Center
|| Department of Rehabilitation
¶ University Suburban Health Center, University Hospitals of Cleveland and Case Western Reserve University School of Medicine, Cleveland, Ohio
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Objective. Several studies have reported beneficial effects of bisphosphonates in children with osteogenesis imperfecta (OI); however, these studies have differed in the protocols they used, and none has been independently replicated. We intended to confirm the efficacy of a specific intravenous bisphosphonate protocol in children with moderate to severe OI.
Methods. We used the protocol described by Glorieux et al and performed a prospective clinical trial in 6 children who were aged 22 months to 14 years. Each patient received intravenous pamidronate therapy for a minimum of 2 years in cycles of 1 mg/kg daily over 3 consecutive days at a mean cycle interval of 3.8 months. Outcome measures included lumbar spine areal bone mineral density (BMD) and z score, fracture rate, and occupational therapy functional assessment with serial Pediatric Evaluation of Disability Inventory.
Results. While on therapy, the average annual increase in areal BMD was 48% and the average annual increase in BMD z score was 1.0. This increase in z score is statistically significant. There was no clear correlation between changes in BMD and fracture rate. All patients experienced functional improvement in mobility.
Conclusions. Our results support the findings of Glorieux et al that cyclic administration of intravenous pamidronate in children with OI has beneficial effects with respect to BMD z scores and physical disability. Long-term follow-up will be required to determine whether bisphosphonate therapy will decrease fracture rates and increase mobility in children with moderate to severe OI.
Key Words: osteogenesis imperfecta • bisphosphonate • pamidronate • bone density
Abbreviations: OI, osteogenesis imperfecta • BMD, bone mineral density • PEDI, Pediatric Evaluation of Disability Inventory
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Osteogenesis imperfecta (OI) is a disease of "brittle bones" that is primarily caused by genetic defects in the production of type 1 collagen. More than 200 disease-causing mutations in the genes encoding the component chains of type 1 collagen have been identified.1 OI is a clinically variable disorder that has been classified into 4 major types on the basis of clinical and radiographic findings.2 In addition to bone fragility, other component features of OI can include short stature, scoliosis, progressive long bone deformity, hearing loss, and abnormal dentition.1
Greater clinical severity in OI is associated with an increased frequency of fractures, progressive deformity, chronic bone pain, and loss of mobility.3 No cure for OI has been found. Historically, treatment options have been limited to vitamin or hormonal supplementation, analgesics to alleviate chronic bone pain, prompt fracture detection and casting, and orthopedic bracing or intramedullary rodding to prevent progressive deformity.1,3,4 Recently, several independent studies have suggested that bisphosphonate therapy can reduce the morbidity associated with moderate to severe forms of OI.5–14 Bisphosphonate compounds are synthetic analogues of pyrophosphate, an endogenous inhibitor of bone resorption. In OI, a disease that has high bone turnover,13 bisphosphonates may restore a more normal balance between osteoblast-mediated bone synthesis and osteoclast-mediated bone resorption.15
Among the reported beneficial effects of bisphosphonate therapy in OI are increased bone mineral density (BMD),5–14 decreased fracture rates,5,8,9,13,14 normalization of bone metabolism,6,7,10,13 and subjective reductions in bone pain and improvements in quality of life.6,7,13 Glorieux et al,13 for example, reported on the effect of cyclic administration of intravenous pamidronate (aminohydroxypropylidene bisphosphonate) in 30 children who had severe OI and were aged 3 to 16 years. These same investigators later reported on an additional 9 patients who were treated when they were younger than 3 years.14 Both studies indicated that bisphosphonate therapy benefited the patients. However, neither the Glorieux et al13 protocol nor other published protocols have been independently confirmed, and interstudy comparisons are limited by the differences in the class of bisphosphonate used, mode of drug delivery, dosage, or treatment cycle interval. To provide independent confirmation of the specific protocol reported by Glorieux et al,13 we used their protocol to treat 6 children with moderate to severe OI.
|
METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Participant Recruitment
Subjects who were followed in the Bone Disorders Clinic at University Hospitals of Cleveland were invited to participate in this study. The clinical diagnosis of moderate to severe OI was based on the occurrence of 2 or more fractures within the previous 12 months, evidence of progressive long bone deformity, or lumbar spine BMD >3 standard deviations below the mean for age-matched controls. This study received approval from the University Hospitals of Cleveland Institutional Review Board before its implementation. Informed consent was obtained from the study participants’ parents.
Treatment Protocol
We used the protocol described by Glorieux et al.13 Briefly, patients received cycles of intravenous pamidronate (Aredia; Novartis Pharmaceuticals, East Hanover, NJ) at approximately 4-month intervals. Each cycle consisted of a 3-hour intravenous infusion of pamidronate (1 mg/kg/d dissolved in 5% dextrose, 25% normal saline to achieve a concentration <0.12 mg/mL) given on 3 consecutive days. During day 1 of the first cycle only, the dose was reduced to 0.5 mg/kg/d. For accommodating nonmedical patient and family needs, cycle intervals occasionally varied between 2 and 5 months, but the mean interval length was 3.8 months. The total annual dose of pamidronate was maintained at 9 mg/kg. All patients received dietary counseling to ensure that they received the age-appropriate recommended daily allowance for calcium and vitamin D. Oral calcium carbonate supplementation, to achieve the recommended daily allowance, was given when calcium intake was insufficient or when the ionized calcium level was below the normal range.
Clinical Studies and Outcome Measures
Parents were asked to keep a diary about their child’s general health and fracture history between treatment cycles. Annual fracture rates were calculated on the basis of parental and medical record reports of postnatal fractures; fractures present at birth were not included in determining this rate. Lumbar spine BMD studies were obtained at the beginning of the study and then at approximately yearly intervals. All BMD determinations were performed using a Hologic QDR4500 dual-energy x-ray absorptiometer with pediatric scan and analysis functions.16 Z scores, the number of standard deviations for BMD above or below the mean for age-matched controls, were derived on the basis of the manufacturer’s data. Statistical analyses were performed using the Wilcoxon signed rank test. Annualized increases in BMD were determined by dividing the percentage change in BMD by the number of years on therapy. Laboratory studies, including a complete blood count, total and ionized serum calcium, phosphorous, alkaline phosphatase, and creatinine, as well as spot urine calcium and creatinine were obtained before and after each cycle. Functional assessment was performed using the Pediatric Evaluation and Disability Inventory (PEDI)17 during each cycle by the same pediatric occupational therapist. This standardized scaling tool measures subjective and objective changes in the domains of life skills, mobility, and social function.
Study Participants
Table 1 summarizes each participant’s age, gender, type of OI, presence of bone pain, family history of OI, initial BMD z score, and fracture rate before entry into the study.
|
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
BMD Improves While on Therapy
Increases in lumbar spine BMD occurred in all patients after the initiation of intravenous bisphosphonate therapy (Fig 1). The mean annualized increase in lumbar spine areal BMD was 48%. More important, the rate of increase exceeded that of age-matched unaffected controls,18 as indicated by the improvement in z scores (Fig 2). Our patients’ areal BMD z scores increased by an annual average of 1.0 during the study (range: 0.5–1.4). This improvement in z score is statistically significant (P < .03).
|
|
The deposition of new bone indicative of linear growth is evident in the posttreatment radiographs of long bones in prepubertal children (Fig 3). Transverse opacities delineate treatment cycles, with the distance between these transverse lines representing new bone growth. However, precise quantitative measures of growth rates were not possible because of intervening intramedullary rodding and casting procedures.
|
New Fractures Occurred While on Therapy in All Participants
All 6 patients experienced new fractures while receiving bisphosphonate therapy (Table 2). There was no statistically significant reduction in fracture rates. However, many new fractures that occurred on therapy were associated with increased activity and moderate to severe trauma (eg, patient 3 while punching a wall and lifting heavy cartons; patient 4 after tripping while skipping rope; patient 6 after falling on steps). It is interesting that patient 2 tumbled down a flight of stairs after 10 months on therapy and sustained no fractures. After 19 months on therapy, patient 6 also tripped on stairs with force considerable enough to bend her left femoral intramedullary rod; however, she sustained only a single fracture that occurred at the site where the rod bent.
|
Functional Evaluations Demonstrated Improvements in All Patients
Serial occupational therapy evaluations documented improvements in mobility in all patients during the course of therapy. The 2 oldest patients, patients 3 and 4, progressed from being wheelchair dependent to completely ambulatory. The PEDI scale was administered over the course of treatment, but it did not allow for a clear assessment of patient rate of gain in self-care and mobility skills. Intervening fractures and/or surgical procedures were associated with significant drops in PEDI score. Despite these setbacks, no child experienced permanent loss of mobility or strength. Bone pain, causing nighttime awakening, was reported in patients 3, 4, and 5 before treatment initiation; this resolved while on therapy. All parents perceived improved patient quality of life during the treatment period.
Clinically Significant Laboratory Abnormalities Did Not Occur While on Therapy
Total serum calcium levels dropped during each cycle. However, total ionized calcium dropped below normal in only 2 of 57 total cycles and in no instance caused clinical signs of hypocalcemia. Alkaline phosphatase levels, which were elevated in 3 patients before therapy, normalized while on therapy. No abnormality in complete blood count, serum creatinine, blood urea nitrogen, or urinary calcium/creatinine ratio was observed while on therapy.
Adverse Occurrences During Treatment Protocol
Five of 6 patients developed self-limited flu-like symptoms during the first treatment cycle that consisted of decreased appetite, low-grade fever (<39.0°C), myalgia, irritability, and decreased energy. Symptoms were well controlled with standard doses of acetaminophen. No patient developed symptoms during subsequent cycles. Peripheral intravenous line infiltration occurred in 2 patients without any adverse effect. Patient 3 reported a transient metallic taste in his mouth for several hours after his first cycle of therapy. Patient 6 experienced transient tachycardia (heart rate 119) during cycle 4, although she was noted to have a concurrent mild upper respiratory infection. Finally, patient 6 sustained a tibial fracture 1 month after beginning therapy at a site that had preexisting anterior bowing. The fracture was treated by intramedullary rodding, but an apparent nonunion at the site remained 18 months later.
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
In a prospective, clinical trial using intravenous pamidronate in children with moderate to severe OI, we observed increased lumbar spine areal BMD z scores, decreased propensity to fracture with minimal trauma, improved mobility and function, decreased bone pain, and improved parental perceptions regarding patient quality of life. Although several oral and intravenous bisphosphonate protocols for OI have been published, we had 3 reasons for choosing to use the protocol described by Glorieux et al.13 First, Glorieux et al had published results using their protocol to treat 30 children with severe OI, including 9 children who had received therapy for >2 years; all other studies reported on smaller numbers of patients. Second, intravenous therapy allowed for a more consistent control of drug dosing and drug bioavailability than would oral therapy. Last, we thought that 4-month intervals between cycles would lessen patient anxiety and be better able to accommodate family needs than would the shorter intervals used in other studies.
The results in this study are comparable to those reported by Glorieux et al13 and therefore serve to support independently a beneficial effect for intravenous pamidronate in children with OI. The average annualized increase in lumbar spine areal BMD was 48%, comparable to the 42% increase reported by Glorieux et al. Our patients’ areal BMD z scores significantly increased by an average of 1.0 annually during the study. Rates of BMD z score change were not reported by Glorieux et al; however, the 30 patients in their study received an average of 2.1 years of treatment and had a mean BMD z score improvement of 1.9, suggesting a mean annual increase in z score close to 0.9. A limitation of our study and of several others is that we were unable to include an untreated, affected control group. However, Glorieux et al reported serial BMD measures in several patients before initiating therapy that suggests BMD does not increase as a result of age alone. Furthermore, other studies of BMD in untreated patients with moderate to severe OI also suggest that BMD z score is unlikely to normalize with age.19–21 Therefore, we conclude that the increased BMD z scores that we observed are related to bisphosphonate therapy.
One third of the patients reported by Glorieux et al13 remained free of fractures while on therapy. All patients in our study developed new fractures while receiving therapy. Most new fractures occurred in the context of activities that children were previously unable to perform, such as running up stairs or skipping rope. Increased mobility while on therapy is further reflected by serial assessments. We recognize that factors besides the bisphosphonate therapy may contribute to the increased mobility and decreased fracture rates in our patients. These factors include age-related acquisition of motor skills; orthopedic interventions, such as bracing and intramedullary rodding; and, in the cases of patients 3 and 4, progression through or entrance into puberty.
We did not observe any significant adverse effects of therapy during this study. Similar to other studies,5,13 our patients developed a "flu-like" acute-phase reaction when first exposed to intravenous pamidronate that responded to acetaminophen and did not recur in subsequent cycles. Theoretical adverse effects, some of which have occurred in patients who did not have OI and received intravenous pamidronate, include increased bone pain, growth plate abnormalities, altered fracture healing, hypocalcemia, and anemia.15 It is not possible to determine the precise effect of therapy on our patients’ growth because several patients had orthopedic interventions while receiving bisphosphonate therapy. However, bisphosphonate deposition lines are evident on long bone radiographs, indicating that continued linear bone growth occurred. Normalization of elevated alkaline phosphatase occurred in 3 patients. This normalization is likely to be a direct consequence of bisphosphonate therapy as similar reductions have been reported with bisphosphonate use in osteoporosis.22 With 1 exception, all fractures sustained while on therapy did not have impaired healing. The exception was an apparent tibial nonunion in patient 6. Because nonunions can occur in children with OI23 and have not been reported in any other study of bisphosphonates in children with OI, we do not know whether the nonunion and therapy are related. Total serum calcium levels decreased during each cycle, and ionized calcium levels fell below the normal range without clinical consequence at the end of 2 of 57 treatment cycles. We did not observe alterations in complete blood counts or renal function panels.
Although we chose to use the intravenous protocol described by Glorieux et al,13 other intravenous and oral bisphosphonate protocols have also reported significant improvements in areal BMD, chronic bone pain, and fracture rates. For example, Bembi et al11 used single-day intravenous pamidronate infusions at intervals of 10 to 20 days to treat 3 prepubertal girls with OI; 2 of the 3 children had substantial increases in areal BMD and z score, whereas the third had minimal improvement. Brumson et al12 treated 4 postpubertal patients with daily oral pamidronate or olpadronate for over 5 years and demonstrated an average annual BMD z score improvement of 0.3. Åström and Söderhäll6 treated 3 girls, aged 13, 16, and 20 years, with monthly infusions of intravenous pamidronate for at least 2 years; although not calculated in their publication, their data demonstrated average annualized BMD increases of 40%, 11%, and 0%, respectively. These same authors recently reported increased BMD, increased mobility, increased well-being, and decreased bone pain in a cohort of 28 children (aged 0.6–18 years) with OI who received intravenous pamidronate for 2 to 9 years.7 Although the investigators did not look at changes in fracture rate or z score, they did observe a substantial annual increase in BMD, which averaged nearly 100%.7 The authors gave intravenous pamidronate monthly and at doses of >18 mg/kg/y, suggesting that greater benefit may occur in patients with OI who receive more frequent and/or higher doses of bisphosphonates.
At present, there is no consensus regarding clinical indications and age to initiate treatment or optimal bisphosphonate compound, dosing regimen, and route of administration for treating children with OI. Several studies have similarly concluded that bisphosphonates seem to be safe and beneficial in reducing morbidity associated with this disorder. Follow-up of treated individuals will be needed to confirm that bisphosphonate therapy has no unanticipated complications and that increases in BMD, gains in mobility, reductions in bone pain, and decreases in fracture rates will translate to long-term improvements in quality of life. Until other bisphosphonate protocols are independently confirmed as being effective7 or new therapeutic agents or modalities supersede the use of bisphosphonates (see www.oif.org), our results suggest that intravenous bisphosphonate therapy be considered for all children with moderate to severe OI.
|
ACKNOWLEDGMENTS |
|---|
This work was supported by the General Clinical Research Center at University Hospitals of Cleveland and National Institutes of Health grant M01RR00080. Dr Warman is a recipient of a Clinical Scientist Award for Translational Research from the Burroughs Wellcome Fund and is an assistant investigator with the Howard Hughes Medical Institute.
We thank K.Y. Warman, J. Komisarz, M. O’Riordan, S. Morrison, and the staff of the GCRC for sharing clinical expertise
Note Added In Proof—Subsequent to this article’s submission, Zacharin and Bateman24 published similar results when using the same bisphosphonate protocol.
|
FOOTNOTES |
|---|
Received for publication Feb 4, 2002; Accepted Jul 23, 2002.
Reprint requests to (M.L.W.) Department of Genetics, Rm BRB-719, 2109 Adelbert Rd, Cleveland, OH 44106. E-mail: mlw14{at}po.cwru.edu
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
- Marini JC, Chernoff EJ. Osteogenesis imperfecta. In: Cassidy SB, Allanson JE, eds. Management of Genetic Syndromes. Philadelphia, PA: Wiley-Liss Inc; 2001:281–300
- Sillence DO, Senn A, Danks DM. Genetic heterogeneity in osteogenesis imperfecta.
J Med Genet.1979; 16
:101
–116
[Abstract/Free Full Text] - Marini JC. Osteogenesis imperfecta: managing brittle bones [editorial].
N Engl J Med.1998; 339
:986
–987
[Free Full Text] - Antoniazzi F, Bergoldo F, Mottes M, et al. Growth hormone treatment in osteogenesis imperfecta with quantitative defect of type I collagen synthesis. J Pediatr.1996; 129 :432 –439[CrossRef][Web of Science][Medline]
- Fujiwara I, Ogawa E, Igarashi Y, Ohba M, Asanuma A. Intravenous pamidronate treatment in osteogenesis imperfecta. Eur J Pediatr.1998; 157 :261 –262
- Åström E, Söderhäll S. Beneficial effect of bisphosphonate during five years of treatment of severe osteogenesis imperfecta. Acta Paediatr.1998; 87 :64 –68[CrossRef][Web of Science][Medline]
- Åström E, Söderhäll S. Beneficial effect of long term intravenous bisphosphonate treatment of osteogenesis imperfecta.
Arch Dis Child.2002; 86
:356
–364
[Abstract/Free Full Text] - Landsmeer-Beker EA, Massa GG, Maaswinkel-Mooy PD, Van de Kamp JJP, Papapoulos SE. Treatment of osteogenesis imperfecta with the bisphosphonate olpadronate (dimethylaminohydroxypropylidene bisphosphonate). Eur J Pediatr.1997; 156 :792 –794[CrossRef][Web of Science][Medline]
- González E, Pavia C, Ros J, Villaronga M, Valls C, Escolá J. Efficacy of low dose schedule pamidronate infusion in children with osteogenesis imperfecta. J Pediatr Endocrinol Metab.2001; 14 :529 –533[Web of Science][Medline]
- Roldán E, Pasqualini T, Plantalech L. Bisphosphonates in children with osteogenesis imperfecta may improve bone mineralization but not strength. Report of two patients. J Pediatr Endocrinol Metab.1999; 12 :555 –559[Web of Science][Medline]
- Bembi B, Parma A, Bottega M, et al. Intravenous pamidronate treatment in osteogenesis imperfecta. J Pediatr.1997; 131 :622 –625[CrossRef][Web of Science][Medline]
- Brumsen C, Hamdy N, Papapoulos SE. Long-term effects of bisphosphonates on the growing skeleton: studies of young patients with severe osteoporosis. Medicine.1997; 76 :266 –283[CrossRef][Medline]
- Glorieux FH, Bishop NJ, Plotkin H, Chabot G, Lanoue G, Travers R. Cyclic administration of pamidronate in children with severe osteogenesis imperfecta.
N Engl J Med.1998; 339
:947
–952
[Abstract/Free Full Text] - Plotkin H, Rauch F, Bishop NJ, et al. Pamidronate treatment of severe osteogenesis imperfecta in children under 3 years of age.
J Clin Endocrinol Metab.2000; 85
:1846
–1850
[Abstract/Free Full Text] - Licata AA. Bisphosphonate therapy. Am J Med Sci.1997; 313 :17 –22[CrossRef][Web of Science][Medline]
- Njeh CF, Jergas M, Genant HK. Bone density and imaging of osteoporosis. In: Degroot LJ, Jameson LJ, eds. Endocrinology. 4th ed. New York, NY: WB Saunders Co; 2001:1187–1190
- Haley SM, Coster WJ, Ludlow LH, Haltiwanger JT, Andrellos PJ. Pediatric Evaluation of Disability Inventory (PEDI), Version 1.0: Development, Standardization, and Administration Manual. Boston, MA: New England Medical Center Hospitals Inc, and PEDI Research Group; 1992
- Zanchetta JR, Plotkin H, Alvarez Filgueira ML. Bone mass in children: normative values for the 2–20-year-old population. Bone.1995; 16 :393S –399S[Medline]
- Davie MWJ, Haddaway MJ. Bone mineral content and density in healthy subjects and in osteogenesis imperfecta.
Arch Dis Child.1994; 70
:331
–334
[Abstract/Free Full Text] - Lund AM, Mølgaard C, Müller J, Skovby F. Bone mineral content and collagen defects in osteogenesis imperfecta. Acta Pediatr.1999; 88 :1083 –1088[CrossRef][Web of Science][Medline]
- Cepollaro C, Gonnelli S, Pondrelli C, et al. Osteogenesis imperfecta: bone turnover, bone density, and ultrasound parameters. Calcif Tissue Int.1999; 65 :129 –132[CrossRef][Web of Science][Medline]
- Landman JO, Hamdy NAT, Pauwels EKJ, Papapoulos SE. Skeletal metabolism in patients with osteoporosis after discontinuation of long term treatment with oral pamidronate. J Clin Endocrinol Metab.1995; 80 :3465 –3468[Abstract]
- Gamble JG, Rinsky LA, Strudwick J, Bleck EE. Non-union of fractures in children who have osteogenesis imperfecta.
J Bone Joint Surg (Am).1988; 70-A
:439
–443
[Abstract/Free Full Text] - Zacharin M, Bateman J. Pamidronate treatment of osteogenesis imperfecta—lack of correlation between clinical severity, age at onset of treatment, predicted collagen mutation and treatment response. J Pediatr Endocrinol Metab.2002; 15 :163 –174[Web of Science][Medline]
PEDIATRICS (ISSN 1098-4275). ©2003 by the American Academy of Pediatrics
CiteULike 
Connotea 
Del.icio.us 
Digg 
Facebook 
Reddit 
Technorati 
Twitter What's this?
This article has been cited by other articles:
|
|
 |
|
  M. Al Muderis, T. Azzopardi, and P. Cundy Zebra Lines of Pamidronate Therapy in Children J. Bone Joint Surg. Am., July 1, 2007; 89(7): 1511 - 1516. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J Thornton, D M Ashcroft, M Z Mughal, R A Elliott, T W O'Neill, and D Symmons Systematic review of effectiveness of bisphosphonates in treatment of low bone mineral density and fragility fractures in juvenile idiopathic arthritis Arch. Dis. Child., September 1, 2006; 91(9): 753 - 761. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. P. Huang, C. G. Ambrose, E. Sullivan, and R. J. Haynes Functional Significance of Bone Density Measurements in Children with Osteogenesis Imperfecta J. Bone Joint Surg. Am., June 1, 2006; 88(6): 1324 - 1330. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. P. Mashru, M. J. Herman, and P. D. Pizzutillo Tibial Shaft Fractures in Children and Adolescents J. Am. Acad. Ortho. Surg., September 1, 2005; 13(5): 345 - 352. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
