PEDIATRICS Vol. 121 No. 5 May 2008, pp. 988-993 (doi:10.1542/peds.2007-1871)
ARTICLE |
Variability in Pediatric Brain Death Determination and Documentation in Southern California
Divisions of a Pediatric Critical Care
d Pediatric Neurology, Children's Hospital
c Department of Epidemiology and Biostatistics, School of Public Health, Loma Linda University, Loma Linda, California
b OneLegacy, Los Angeles, California
| ABSTRACT |
|---|
|
|
|---|
OBJECTIVES. Because the concept of brain death is difficult to define and to apply, we hypothesized that significant variability exists in pediatric brain death determination and documentation.
METHODS. Children (0–18 years of age) for whom death was determined with neurologic criteria between January 2000 and December 2004, in southern California, were included. Medical charts were reviewed for documented performance of 14 specific elements derived from the 1987 brain death guidelines and confirmatory testing.
RESULTS. A total of 51.2% of children (142 of 277 children) referred to OneLegacy became organ donors. Care locations varied, including PICUs (68%), adult ICUs (29%), and other (3%). One patient was <7 days, 6 were 7 days to 2 months, 22 were 2 months to 1 year, and 113 were >1 year of age. The number of brain death examinations performed was 0 (4 patients), 2 (122 patients), 3 (14 patients), or 4 (2 patients). Recommended intervals between examinations were followed for 18% of patients >1 year of age and for no younger patients. A mean of only 5.5 of 14 examination elements were completed by neurologists and pediatric intensivists and 5.8 by neurosurgeons. No apnea testing was recorded in 60% of cases, and inadequate PaCO2 increase occurred in more than one half. Cerebral blood flow determination was performed as a confirmatory test 74% of the time (83 of 112 cases), compared with 26% (29 of 112 cases) for electroencephalography alone.
CONCLUSIONS. Children suffering brain death are cared for in various locations by a diverse group of specialists. Clinical practice varies greatly from established guidelines, and documentation is incomplete for most patients. Physicians rely on cerebral blood flow measurements more than electroencephalography for confirmatory testing. Codifying clinical and testing criteria into a checklist could lend uniformity and enhance the quality and rigor of this crucial determination.
Key Words: organ donation brain death guidelines pediatric intensive care
Abbreviations: EEG—electroencephalography CBF—cerebral blood flow OPO—organ procurement organization
The Uniform Determination of Death Act (1981) sets the general legal standard for determining brain death in the United States but is silent on acceptable diagnostic tests and medical procedures. As it was originally conceived by the National Conference of Commissioners of Uniform State Laws, the medical profession remains free to formulate acceptable medical practices and to use new biomedical knowledge, diagnostic tests, and equipment.1 Clinical guidelines for the determination of brain death were promulgated the same year, but these were limited to adult patients and affirmatively excluded children <6 years of age.2 In 1987, the American Academy of Pediatrics and several other professional organizations established guidelines for determining brain death in children of all ages, which were published simultaneously in various professional journals and widely disseminated.3 These guidelines called for 2 examinations and confirmatory testing, separated by varying intervals based on the patient's age. The interexamination interval required was
48 hours for patients between 7 days and 2 months of age,
24 hours for patients between 2 months and 1 year of age, and
12 hours for patients >1 year of age. Confirmatory testing with electroencephalography (EEG) to document electrocerebral silence was recommended for all children <1 year of age. Despite establishment of these guidelines, the declaration of "death" on the basis of the cessation of brain function remains complex and controversial. Moreover, these guidelines have not been updated, despite the wider availability and application of several testing modalities for confirmation of brain death.4,5
Previous studies related to brain death assessment by pediatric intensivists and subspecialists showed that these groups have difficulty defining and applying the concept of brain death. Actual practices used to define brain death in PICUs are extremely variable.6,7 In addition, intensive care for children is poorly regionalized in communities across the United States.8 Critically ill children may be cared for at various types of institutions (adult hospitals, children's hospitals, community hospitals, or county hospitals) and by various specialists, many of whom may not be pediatricians or intensivists. Depending on location, access to pediatric intensive care and neurologic or neurosurgical consultation may be limited. Hospital policies regarding determination of brain death vary significantly as well.9 All of these factors add to the potential for confusion and variability in determining death through neurologic criteria in children. There are no published data on actual practices used in declaring brain death in children in settings other than PICUs.
The purpose of this study was to describe the determination and documentation of brain death in a 7-county region in southern California. Our hypothesis was that physician practices to determine brain death in children are variable, in the context of the 1987 guidelines, nonregionalized care locations, changing technology, and variable hospital policies.
| METHODS |
|---|
|
|
|---|
OneLegacy is the regional organ procurement organization (OPO) for 7 counties in southern California, inhabited by
20 million people. As mandated by law, OneLegacy receives referrals from
220 hospitals in the region for all patients who have sustained a severe neurologic insult resulting in a Glasgow Coma Scale score of <6. Each patient is then monitored serially, and the family is approached with a request for organ donation if clinical progression to brain death occurs. OneLegacy maintains archived copies of medical charts for all cases referred for evaluation. The institutional review boards for OneLegacy and Loma Linda University approved this study. We screened all pediatric (newborn to 18 years of age) referrals to OneLegacy over a 5-year period from January 2000 to December 2004. In situations where the family or coroner declined organ donation, the patient charts were not archived in their entirety by OneLegacy; these patients were excluded from analysis. Physician documentation of brain function determined through clinical examination in children who progressed to brain death and became organ donors was then reviewed in greater detail. Three investigators (Ms Petersen, Ms Stadtler, and Ms Rose), who are nurses by background and have 10, 5, and 3 years of experience, respectively, as clinical procurement transplant coordinators for OneLegacy, scrutinized these records for performance of 14 specific elements derived from the 1987 brain death guidelines and confirmatory testing (Table 1). To ensure that legibility of physician notes did not affect the assessment of adequacy of documentation, these 3 investigators reviewed all archived charts together. The presence or absence of documentation for each examination element for each patient was recorded when all 3 investigators concurred. A generally used medicolegal standard (ie, "If it's not documented, it wasn't done") was used in assessing whether each element was performed. Whether an apnea test was performed and the final PaCO2 reached at the conclusion of apnea testing were recorded. We also noted donation-related issues, including the time of referral to OneLegacy and the type of personnel initiating the donation discussion with the family. Data were entered into a Microsoft Access (Microsoft, Redmond, WA) database at the point of data collection.
|
Deidentified data were analyzed statistically by using SPSS 12 software (SPSS, Chicago, IL). The
2 test was used to compare performance of examination or confirmatory testing between groups. In cases in which >2 examinations were performed, the final examination used to pronounce brain death was classified as examination 2 for data analysis. Interexamination interval data were not distributed normally and were reported as median and range. Analysis of variance was used to determine differences between hospital types for the average number of testing elements completed during the brain death examination. Scheffé's multiple-comparison test was used as a posthoc test to establish significant differences. A P value of <.05 was considered statistically significant. | RESULTS |
|---|
|
|
|---|
Study Group
A total of 277 children were declared dead by using clinical neurologic criteria during the study period. Of those children, 51.2% proceeded to become organ donors and had complete medical charts available for review.
Location of Care
Children received care at diverse locations, including freestanding children's hospitals (34%), children's hospitals within adult hospitals (8%), community hospitals (31%), county hospitals (13%), and university-affiliated hospitals (13%). Most patients were cared for in dedicated PICUs (68%) or adult ICUs (29%), but 3% received care in other locations (such as emergency departments). Pediatric critical care specialists were involved in these patients' care in 70% of cases. Almost all of the patients (41 of 42 patients) for whom pediatric intensivists were not involved were >1 year of age.
Variability in the Determination of Brain Death
Two brain death examinations at specific intervals, depending on age, were recommended in the 1987 criteria; in our review, however, the number of documented brain death examinations ranged from 0 to 4. No examinations were documented for 4 patients (3%), 2 for 122 patients (86%), 3 for 14 patients (10%), and 4 for 2 patients (1%). Of the patients without any documented examinations, 2 were being cared for at children's hospitals, 1 at an adult/children's hospital, and 1 at a community hospital. The age-specific interval between examinations (a minimum of 48, 24, or 12 hours, depending on age) recommended in the 1987 criteria was adhered to for only 18.5% of patients >1 year of age. None of the patients in the younger age groups received examinations separated by the recommended intervals. Interexamination intervals for patients <2 months of age ranged from 75 minutes to 27 hours, with the median interval being 10.7 hours. For patients between 2 months and 1 year of age, intervals ranged from 10 minutes to 14.4 hours, with the median being 1.6 hours. Four patients >1 year of age had simultaneous examinations recorded.
Completeness
A median of 6 of 14 elements overall were completed for examinations 1 and 2. In fact, >10 elements were completed only 8 times during the first brain death examination and only 3 times during the second brain death examination. Only 1 examination had all 14 elements completed. Overall, analysis of variance detected significant differences in completion of examination elements according to hospital type for examination 1 but not for examination 2. Posthoc tests showed that examination 1 was more complete at university (P < .05), children's (P < .001), and community (P < .05) hospitals, compared with adult/children's hospitals. There were no differences detected for examination 2 in the analysis of variance. There was no difference in the completeness of the examination according to physician subspecialty expertise (pediatric intensive care, neurology, neurosurgery, or internal medicine) for examination 1 or 2 (P = not significant) (Table 2).
|
Apnea Test
There were no significant differences in whether an apnea test was or was not performed according to hospital type. Overall, apnea testing was recorded in association with the first brain death examination in 57 instances (40%) but, of those tests, the final PaCO2 documented was >60 mmHg in just 65%. With the second examination, 61 patients (43%) had an apnea test performed, and a final PaCO2 of >60 mmHg was reached in 57% of those tests. Overall, the final PaCO2 ranged from 40 mmHg to 58 mmHg for patients for whom the apnea test was performed suboptimally.
Confirmatory Testing
There were 112 instances in association with either the first or second examination in which confirmatory testing was performed. Physicians used cerebral blood flow (CBF) studies much more often than EEG (83 of 112 cases [74%] vs 29 of 112 cases [26%]; P < .001). EEG and CBF studies were performed together only 6 times (5.6%). Of the 29 EEG studies performed, 7 were for children <1 year of age and 22 were for older patients. None of the EEG studies was read as nonspecific or had too much artifact. EEG and CBF tests were performed concurrently with the first examination for 2 patients; however, both subsequently underwent a second set of confirmatory tests. Therefore, for no patients were CBF studies used to shorten the observation period or to eliminate the need for repeat examination and EEG, although this would have been permissible according to the guidelines (Table 3).
|
Donation-Related Issues
The OPO was contacted at varying times in relation to the clinical progression of brain death. The majority (84%) of calls were appropriately made before the first neurologic examination. A minority of contacts (12%) were between the first and second examinations, and even fewer (3.5%) were made after the second examination and pronouncement of death. In 0.7% of cases, we could not find any documentation of OneLegacy being notified about the potential referral. Donation conversations were initiated mainly by OneLegacy (44%) or the family (23%). However, some discussions were initiated by physicians (11%), nurses (3%), other hospital personnel (3%), chaplains (0.7%), or social workers (1.4%).
| DISCUSSION |
|---|
|
|
|---|
This is the first study to examine brain death determination practices in diverse settings where critically ill children are cared for, and it is the largest examination of pediatric brain death determination to date. Several aspects of the data we report are troubling and should raise concern in the medical community.
We found that approximately one third of pediatric patients suffering brain death are cared for in locations other than a PICU and a similar proportion do not have access to specialists in pediatric intensive care or pediatric neurology/pediatric neurosurgery. Variability in brain death determination practices does not seem to stem from the care location or specialist involvement, because documentation of brain function examinations was universally highly incomplete, with a median of only 6 of 14 elements completed. Neurologic specialists, whether neurosurgeons or neurologists, were as deficient in their documentation as pediatric intensivists. In fact, only 1 examination was fully complete, with all 14 elements recorded. Apnea testing, which should be an integral part of every brain death examination, was documented for less than one half of the patients. Of those, only one third of the patients had a sufficient increase in PaCO2 and reached an acceptable threshold during the apnea test; therefore, the majority of tests, even when performed, were inadequate. This represents deterioration from previously reported rates of apnea testing for three fourths of studied PICU patients and adequacy of the performed test in 88% of those cases.7 This observation may be partly explained by the variety of care locations at the regional level for almost one third of the patients included in our study, compared with the previous study, which focused only on PICUs in various parts of the country. In addition, our study covered a region governed by California state law, thus eliminating variability related to state jurisdictions. Apnea testing cannot be performed meaningfully for patients with cervical spinal cord injuries and may be unsafe for patients with severe acute lung injuries or those undergoing high-frequency oscillatory ventilation. It cannot be performed to completion if the patient develops oxygen desaturation because of an underlying lung injury or hypotension resulting from hypercarbia causing respiratory acidosis. However, we did not see any physician notation of apnea testing being limited in our patients for these reasons.
Our study clearly reflects an increased reliance on CBF testing as a preferred confirmatory test; it was the sole test confirming brain death for the vast majority (78%) of patients. The 1987 guidelines require
1 EEG study as a confirmatory test for children <1 year of age, and the sole performance of CBF studies is at odds with a strict interpretation of those guidelines. Our results are in contrast to a survey that reported that PICU physician preferences for confirmatory testing were almost evenly divided, with 41% using EEG and 43% CBF testing.10 In another study, EEG was performed to confirm brain death for 33% of patients, CBF studies were performed for 25%, and 25% underwent both tests. Only 6% of our patients underwent both EEG and CBF studies. Although no recent studies compared CBF testing directly with EEG, CBF testing is thought to be more robust, as well as highly sensitive and specific.4,5 Given these factors, the greater use of CBF testing shows the current physician preference regarding confirmatory testing. This choice may be related to practical issues (ie, CBF testing and interpretation may be more readily available than EEG at some hospitals) or may simply be a matter of physicians' personal choice.
Our study design does not allow us to explore the reasons behind the incompleteness of the brain death examinations and variations in confirmatory testing. It could be speculated that the variability stems from a lack of familiarity with the established guidelines, a lack of experience in performing the complex multistep process, knowledge deficits, variable hospital policies, lack of access to pediatric specialists, or physicians willfully ignoring published brain death guidelines. Because we were reviewing archived medical charts, it is possible that examination and testing were performed but not documented. In our review, we included physician documentation where either the conclusion or the title of the note related to brain death. It is also possible that some elements of the examination were performed separately but not designated properly. Nevertheless, our findings highlight the need for a means of verifiable documentation. In addition, strictly interpreted, our findings are applicable only to the southern California region studied. Additional studies should be conducted in other areas of the country, to confirm our findings.
The retrospective nature of our study enabled us to review actual physician documentation of brain death but limited data collection in another respect. Regulatory requirements (Joint Commission on Accreditation of Healthcare Organizations and Centers for Medicare and Medicaid Services regulations and federal law) have mandated that hospital personnel refer all patients with severe neurologic devastation meeting clinical triggers to the designated OPO for evaluation. Patients not recognized as meeting referral criteria and those not referred to OneLegacy for any reason were not included in our analysis. However, review of OneLegacy quality assurance data revealed that there were only 7 such missed referrals during the study period; it is unlikely that this would have introduced significant bias in our data.
Laws providing for brain death currently exist in all 50 states. Although the legal community defines brain death as "irreversible cessation of all functions of the entire brain," no specific standards for the actual determination are provided. All cases that we reviewed would be consistent with California law governing brain death determination, which simply states, "When an individual is pronounced dead by determining that the individual has sustained an irreversible cessation of all functions of the entire brain, including the brainstem, there shall be independent confirmation by another physician."11 However, the observed variability in documentation by physicians of how they determined something as fundamental as death is of great concern to us. These results are similar to previous studies in the United States and the United Kingdom that focused on adult patients.12,13 Clarity, consistency, and uniformity should be minimal requirements in brain death determination and may minimize existing variability or deficits in the recognition, diagnosis, and documentation of neurologic death.14
Indeed, the goals of the presidential commission were to develop guidelines that are clear, consistent, uniform, and reliable for diagnosis, declaration, documentation, and reporting of brain death. The 1987 pediatric guidelines were also intended to fit those needs but in practice have not proven to do so. This may be attributable in part to age-related differences in these guidelines; clinical examination and testing intervals differ on the basis of the patient's age, and confirmatory testing is mandatory for children <1 year of age but is optional for older children. Additional confirmatory testing may be used to shorten the observation interval before determination of brain death, but only for children >1 year of age. However, we take the view that the variability in determination of brain death for children stems from a vital flaw, that is, guidelines are guidelines and are open to subjective interpretation and application. Because specific standards of medical practice for brain death determination in children remain merely guidelines, neurologic death is diagnosed according to criteria determined by individual hospitals or physicians. This decentralized and nonstandardized approach to determining brain death introduces unnecessary variability. Ultimately, we suspect that our data represent the failure of the medical profession to realize the shortcomings that occur when individual providers translate complex guidelines into clinical practice and to codify the guidelines into a simpler format for consistent implementation.
Formulation and use of a checklist to improve compliance with guidelines to determine neurologic death in children would be a simple effective approach to overcoming some of the issues identified in our study. In 2 pediatric referral centers in the United Kingdom, use of a checklist improved rates of adherence to local guidelines from
50% to >90%.15 In Canada, a national process of reviewing neurologic determination of death involved various stakeholders (intensivists, neurologists, neurosurgeons, ethicists, and laypersons) and culminated in a national consensus checklist for pediatric brain death determination.16 A similar tool might well help to improve uniformity in the application of the 1987 brain death guidelines in the United States (a sample checklist based on current guidelines can be accessed at: www.pediatrics.org/cgi/content/full/988.) A checklist would demystify the determination of brain death, cut across the various specialties involved in pediatric brain death declaration, and make the process more explicit, adaptable, and verifiable. It would improve and standardize the way we currently pronounce brain death in children. With wide dissemination of the 1987 guidelines (or any future revisions) in this simpler format, the diverse group of physicians involved in brain death determination could use them with equal consistency. In addition, OPOs could provide an additional check by implementing processes to ensure that declaration of brain death has been performed according to the guidelines before procurement proceeds. However, we agree with the current nationwide legal standards that preclude transplant surgeons' involvement with a potential donor in any way, including the determination of brain death or its verification.
A brain death checklist also can serve as a reminder about important end-of-life issues such as organ donation and autopsy. The Canadian consensus document includes items about whether the patient is medically eligible for organ and tissue donation, whether this option has been offered, and whether consent for donation has been obtained. Delays in referring potential donors to the OPO and families being approached by untrained hospital staff members have been shown to affect consent rates for organ donation unfavorably.17 More than 15% of calls in our study were made after the first brain death examination, clearly constituting a delay in referral to OneLegacy, although those families did provide consent for donation. Inclusion of a reminder about the referral for organ donation in the checklist would potentially facilitate timely notification of the OPO about potential donors. Talking to OPO personnel before being asked to make a donation decision and spending more time with the personnel are both strongly associated with decisions to donate.18
| CONCLUSIONS |
|---|
|
|
|---|
Clinical practice varies greatly from established pediatric brain death determination guidelines. Children suffering brain death are being cared for in PICUs as well as other locations, by a diverse group of specialists. Among clinicians, there is greater reliance on CBF testing than on EEG for confirmatory testing. Documentation of clinical criteria used to determine brain death is highly incomplete, and apnea testing is inadequate in most cases. There is a need for a review of brain death determination guidelines, to determine whether they should be revised. An assessment also should be made to improve methods so that these guidelines can be disseminated more effectively. Codifying clinical and testing criteria into a specific checklist would lend uniformity and enhance the quality and rigor of this crucial determination. This not only is important for optimal patient care but also should help retain the confidence of our communities in the integrity of brain death determination and enhance opportunities for organ and tissue donation.
| ACKNOWLEDGMENTS |
|---|
We acknowledge Tom Mone and Renee Hawthorne at OneLegacy for their support of this study. We also thank all of the organ and tissue donors and their families for giving the gift of life and the gift of knowledge through their generous donations.
| FOOTNOTES |
|---|
Accepted Sep 7, 2007.
Address correspondence to Mudit Mathur, MD, Department of Pediatrics, Loma Linda University Children's Hospital, 11175 Campus St, CP A1117, Loma Linda, CA 92350. E-mail: mmathur{at}ahs.llumc.edu
The authors have indicated they have no financial relationships relevant to this article to disclose.
| What's Known on this Subject Guidelines for determining brain death in children are complex, differ with patient age, and have remained unchanged since 1987. Physicians have difficulty defining and applying the concept of brain death.
|
| What This Study Adds Clinical practice varies from established guidelines, and documentation is highly incomplete. Physicians rely on cerebral blood flow studies over electroencephalography for confirmatory testing. Codifying clinical and testing criteria into a checklist could enhance the quality of brain death determination.
|
| REFERENCES |
|---|
|
|
|---|
1. National Conference of Commissioners on Uniform State Laws. Uniform Brain Death Act: Uniform Laws annot 12:63 (West 1993; West suppl 1997). Presented at the National Conference of Commissioners of Uniform State Laws meeting; July 26–August 1, 1980; Kauai, HI
2. President's Commission for the Study of Ethical Problems in Medicine and Biomedical and Behavioral Research. Guidelines for the determination of death: report of the medical consultants on the diagnosis of death to the President's Commission for the Study of Ethical Problems in Medicine and Biomedical and Behavioral Research.
JAMA. 1981;246
(19):2184
–2186
3. American Academy of Pediatrics, Task Force on Brain Death in Children. Report of special task force: guidelines for the determination of brain death in children.
Pediatrics. 1987;80
(2):298
–300
4. Singh NC, Reid RH, Loft JA, Frewen TC, Parker BL, Dhillon JS. Usefulness of Tc99m HM-PAO scan in supporting clinical brain death in children: uncoupling flow and function. Clin Intensive Care. 1994;5 (2):71 –74[Medline]
5. Conrad G, Sinha P. Scintigraphy as a confirmatory test of brain death. Semin Nuclear Med. 2003;33 (4):213 –323
6. Harrison AM, Botkin JR. Can pediatricians define and apply the concept of brain death? Pediatrics. 1999;103 (6). Available at: www.pediatrics.org/cgi/content/full/103/6/e82
7. Mejia RE, Pollack MM. Variability in brain death determination practices in children.
JAMA. 1995;274
(7):550
–553
8. American College of Critical Care Medicine, Society of Critical Care Medicine, Pediatric Task Force on Regionalization of Pediatric Critical Care, American Academy of Pediatrics, Committee on Pediatric Emergency Medicine. Consensus report for regionalization of services for critically ill or injured children. Crit Care Med. 2000;28 (1):236 –239[CrossRef][Web of Science][Medline]
9. Powner DJ, Hernandez M, Rives TE. Variability among hospital policies for determining brain death in adults. Crit Care Med. 2004;32 (6):1284 –1288[CrossRef][Web of Science][Medline]
10. Lynch J, Eldadah MK. Brain-death criteria currently used by pediatric intensivists.
Clin Pediatr (Phila). 1992;31
(8):457
–460
11. State of California. California health and safety code, section 7181. Available at: www.leginfo.ca.gov/cgi-bin/displaycode?section=hsc&group=07001-08000&file=7181-7184.5. Accessed March 17, 2008
12. Wang MY, Wallace P, Gruen JP. Brain death documentation: analysis and issues. Neurosurgery. 2002;51 (3):731 –736[CrossRef][Web of Science][Medline]
13. Keogh AT, Akhtar TM. Diagnosing brain death: the importance of documenting clinical test results. Anaesthesia. 1999;54 (1):81 –85[CrossRef][Web of Science][Medline]
14. Shemie SD, Doig C, Belitsky P. Advancing toward a modern death: the path from severe brain injury to neurological determination of death.
CMAJ. 2003;168
(8):993
–995
15. Kafrawy U, Stewart D. An evaluation of brainstem death documentation: the importance of full documentation. Paediatr Anaesth. 2004;14 (7):584 –588[CrossRef][Medline]
16. Shemie SD, Doig C, Dickens B, et al. Severe brain injury to neurological determination of death: Canadian forum recommendations.
CMAJ. 2006;174
(6):S1
–S13
17. Gortmaker SL, Beasley CL, Sheehy E, et al. Improving the request process to increase family consent for organ donation. J Transpl Coord. 1998;8 (4):210 –217[Medline]
18. Siminoff LA, Gordon N, Hewlett J, Arnold RM. Factors influencing families' consent for donation of solid organs for transplantation.
JAMA. 2001;286
(1):71
–77
PEDIATRICS (ISSN 1098-4275). ©2008 by the American Academy of Pediatrics
This article has been cited by other articles:
![]() |
A. R. Joffe, L. Lequier, and D. Cave Specificity of radionuclide brain blood flow testing in brain death: case report and review. J Intensive Care Med, January 1, 2010; 25(1): 53 - 64. [Abstract] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||






