 |
|
 |
|
 |
 |
|
 |
 |
 |
 |
 |
 |
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
SPECIAL ARTICLE |
a Center for Pediatric Clinical Epidemiology
b Department of Neonatology, Emma Children's Hospital, Academic Medical Centre, University of Amsterdam, Netherlands
 |
ABSTRACT |
|---|
 TOP ABSTRACT METHODSRESULTSDISCUSSIONCONCLUSIONSAPPENDIX: OPTIMAL STUDY DESIGN...REFERENCES |
|---|
METHODS. We searched Medline and Embase for cohort studies on subsequent neurodevelopment after episodes of hypoglycemia in the first week of life. Reference lists of available studies were reviewed, and content experts were contacted for additional studies. Included studies were selected and appraised for methodologic quality by 2 reviewers. Methodologic quality was assessed according to well-accepted criteria for prognostic studies. Eventually, all studies were given an overall quality score: poor, moderate, or high quality. Studies in the latter 2 categories were considered for quantitative data analysis.
RESULTS. Eighteen eligible studies were identified. The overall methodologic quality of the included studies was considered poor in 16 studies and high in 2 studies. Pooling of results of the 2 high-quality studies was deemed inappropriate because of major clinical and methodologic heterogeneity. None of the studies provided a valid estimate of the effect of neonatal hypoglycemia on neurodevelopment. Building on the strengths and weaknesses of existing studies, we developed a proposal for an "optimal" future study design.
CONCLUSIONS. Recommendations for clinical practice cannot be based on valid scientific evidence in this field. To assess the effect of neonatal hypoglycemia on subsequent neurodevelopment, a well-designed prospective study should be undertaken. We submit a design for a study that may answer the still-open questions.
Key Words: hypoglycemia • prognosis • developmental disabilities • neonates • systematic review • research design
Abbreviations: LGA—large for gestational age • SGA—small for gestational age • AGA—appropriate for gestational age • DDS—Denver Developmental Scale • CI—confidence interval
Glucose is the essential substrate for brain function. Although important at all ages, it is particularly so in childhood because a normal supply is necessary to protect neural development.1 Hypoglycemia is the most common metabolic problem in neonatal medicine.2 Still, there is much controversy about the definition of a "safe" blood glucose concentration (ie, a value above which there is no risk of long-term neurodevelopmental impairment).
In 2000, a group of investigators in the field of hypoglycemia provided a consensus statement for the operational thresholds of blood glucose concentrations in the neonate.3 They deliberately agreed on a high operational threshold (ie, a safe plasma glucose concentration [>2.5 mmol/L] that is applicable to any infant whether term or preterm). However, although several review articles on this topic have emerged,1,2,4,5 no systematic review of the available studies on the prognosis after neonatal hypoglycemia exists. Also, existing reviews generally conclude that well-designed studies are scarce and that "more research is needed" but fail to provide suggestions for an optimal design of these desired studies.
Our aim was to answer the following question: What is the neurodevelopmental outcome after neonatal hypoglycemia in the first week of life? To this end, we set out to find all relevant empirical studies on the prognosis after neonatal hypoglycemia in humans, appraise them for methodologic quality, and summarize their results in a way that informs both clinical practice and subsequent research. The focus is on the available evidence with regard to the longer-term prognosis, that is, neurodevelopment, late neurophysiology or neuroradiology. Finally, building on the strengths and weaknesses of existing studies, we developed a proposal for an "optimal" future study design.
|
METHODS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONSAPPENDIX: OPTIMAL STUDY DESIGN...REFERENCES |
|---|
Inclusion Criteria
To be eligible for inclusion in this review, a study had to meet the following 3 criteria: (1) It had prospectively or retrospectively studied neonates <1 week after birth with hypoglycemia using any definition and any assay method. Retrospective studies were only eligible for inclusion when a prospective protocol was used to collect patient data. (2) Hypoglycemic neonates were compared with a control group or symptomatic versus asymptomatic hypoglycemia, or the studies should have provided information about other risk factors for adverse outcome. (3) At least 1 of the following outcome measures had been used: neurodevelopment, neurophysiology, or neuroradiology at 
1 year of age. Studies written in English, French, German, or Dutch were eligible for inclusion. Excluded were case reports, studies with <3 patients, and "abstracts only."
Study Selection
Two investigators (N.B. and A.v.K.) independently selected studies as being potentially relevant on the basis of the titles and abstracts. Potentially relevant citations were retrieved in full text and checked for the presence of our eligibility criteria independently by 2 reviewers (N.B. and M.O.). Simple interobserver agreement for the latter step was calculated. Differences between the reviewers over which studies should be included were resolved by consensus.
Data Abstraction
Once a study met the inclusion criteria, 2 members of the research team (N.B. and M.O.) independently abstracted data by using structured data-abstraction forms. We captured information on the language of the report, study population, definition of hypoglycemia, glucose-assay method, treatment given for hypoglycemia, additional risk factors for adverse neurodevelopment, neurodevelopmental outcome measurement, duration of follow-up, and main results. Disagreements were resolved by consensus.
Methodologic Validity of Included Studies
To determine the methodologic validity of the selected studies, 2 investigators assessed the design and execution of each study. Validity was assessed according to criteria for prognostic studies described by the Evidence-Based Medicine Working Group6 with additional items described by Altman.7 We specified each methodologic item adjusted to the subject of our review. To maximize the validity of these items we used a pilot sample of 5 articles to test the interrater agreement and came up with a final list of important items:
Studies were considered to be of (1) high methodologic quality if the study group was well defined, follow-up was >80%, or dropouts were described and shown to be comparable to the root population, objective outcome measures were used or, if not using objective outcome measures, outcome assessment should have been blinded, and if all important risk factors were assessed with adequate adjustment; of (2) moderate methodologic quality if the study group was not well defined (–), follow-up was at least 50%, no objective outcome measures were used and no blinding of outcome assessment, and 3 of the 4 important risk factors were assessed with adequate adjustment; and of (3) poor methodologic quality if <3 important risk factors were assessed with descriptive data only or follow-up was <50%. Only the studies judged to be of high or moderate methodologic quality were considered for quantitative data analysis and are described in detail below.
A Priori Hypothesis Regarding Sources of Heterogeneity
Sources of heterogeneity in this systematic review were considered to be differences in study population, definition of hypoglycemia, types of outcomes assessed, follow-up time, and assessment and adjustment for other prognostic factors for adverse neurodevelopment.
Statistical Analysis
Pooling of data from individual studies to yield summary statistics of cumulative incidence of adverse outcome and relative risk measures for the various different prognostic factors was considered to be adequate in the absence of heterogeneity only.
|
RESULTS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONSAPPENDIX: OPTIMAL STUDY DESIGN...REFERENCES |
|---|
4 were retrospective without a prospective protocol,11,33,41,42 4 were cross-sectional studies,15,36,44,51 2 were case reports,9,12 1 was abstract-only,39 1 did not study neonates,43 1 outcome was brain section,26 and 1 study had a follow-up of only 1 week for neurodevelopment.48 Checking reference lists did not yield additional studies. Contacting content experts in this field yielded 1 additional study that met our inclusion criteria, but it had not been published at the time of this review.53 Thus, a total of 18 studies was included in our systematic review. The interobserver agreement was 87% for this selection. All differences were resolved by consensus.
Description of the Selected Studies
Details on included studies are summarized in Table 1. Eighteen studies comprising 1583 infants were identified.
Three studies were clearly prospective studies by design,28,32,46 1 study was a combined prospective and retrospective design,8 in 10 studies it was not clear whether the design was prospective or retrospective,
and 4 studies were definitely retrospective but used a "prospective data collection protocol."17,18,40,53
|
Fourteen studies provided a definition of hypoglycemia: 2 studies with term infants32,53 (<2.5 or 
2.5 mmol/L), 7 studies with both premature and term infants28,37,38,45,46,50,52 (<1.7 or 1.7–1.9 mmol/L), 6 studies with both premature and term infants16,24,25,27,28,38 (<1.1 mmol/L), and 1 study using 3 categories40 (<0.3, <1.6, and <2.6 mmol/L). Two studies used different definitions for different birth weights.28,38
The assay method used to measure blood glucose was reported in 13 studies,# 8 of which used the glucose-oxidase method**; the other 5 used various other methods.16,25,28,32,38 Thirteen studies reported what time the blood glucose was measured: routinely at preset times in 6 studies,28,40,45,46,50,53 routinely but not at preset times in 3 studies,22,24,37 and in case of signs and symptoms of hypoglycemia in 6 studies.16,27,37,38,46,50 Two studies measured glucose both routinely and in case of symptoms.46,50 Mean blood glucose levels were reported in 5 studies28,32,40,50,53 and varied between <0.6 and 2.6 mmol/L.
Follow-up outcome variables that were used were (1) neurodevelopment, using various different scales, questionnaires, and neurologic examination (all 18 studies
), (2) neuroradiology (1 study32), and (3) neurophysiology, in all cases using the electroencephalogram (4 studies17,22,24,46). Mean follow-up time ranged from 2 months to 20 years across studies. Completeness of follow-up ranged from 21% to 100%. Seven studies assessed all 4 defined adverse-outcome risk factors.24,25,28,38,40,46,53 Three studies assessed 3 risk factors,16,32,45 5 studies assessed 2 risk factors,22,27,37,50,52 1 study assessed 1 risk factor,8 and 2 studies assessed none of the defined risk factors.17,18
Validity
Data on the validity of the studies are shown in Table 2. The study population was "well defined," representative, and included in the study at a similar point in the disease in 7 of 18 studies.25,28,37,40,45,46,53 Follow-up was complete in 8 of 18 studies.16–18,32,37,40,45,50 The outcome variable for at least 1 outcome variable was "objective" and blinded in 6 of 18 studies.25,28,40,46,50,53 Four studies adequately adjusted for our defined other risk factors for impaired neurodevelopment.40,53 Treatment of cases of neonatal hypoglycemia was fully described in 9 studies.
Only 2 studies were considered to be of high methodologic quality,40,53 none were of moderate quality, and 16 (89%) were of poor quality.
|
|
The study by Lucas et al40 reported the incidence of hypoglycemia (ie, plasma glucose concentration <2.6 mmol/L) in 661 premature infants and related the occurrence and persistence of hypoglycemia to the neurodevelopmental outcome at 18 months of age. In the 31 infants with hypoglycemia recorded on 5 separate days, the authors found reduced mental score (–14 points; 95% CI: –22 to –6) and motor score (–13 points; 95% CI: –20 to –5) on the Bayley developmental scale at 18 months of corrected age, compared with 177 nonhypoglycemic infants, even after adjustment for a wide range of factors known to influence development. Also, the incidence of cerebral palsy or developmental delay (ie, mental or motor score of 70) was increased by a factor 3.5 (95% CI: 1.3 to 9.4). These data suggest that moderate hypoglycemia, if prolonged, is associated with an increased risk of impaired neurodevelopment.
|
DISCUSSION |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONSAPPENDIX: OPTIMAL STUDY DESIGN...REFERENCES |
|---|
Our findings are the result of a systematic search for all relevant studies on the neurodevelopmental outcome after neonatal hypoglycemia and a critical appraisal of the methodologic quality of these studies. The results of the individual studies are quite conflicting: some studies found no differences between neonates with and without episodes of hypoglycemia on neurodevelopment, whereas others show serious brain damage after episodes of neonatal hypoglycemia.
Several sources for this variability were identified. First, there is wide agreement that a reliable prognostic study requires a well-defined cohort of patients at the same stage of their disease course. In our review, the majority of included studies failed to define the study population and the timing and number of hypoglycemic episodes adequately.
Second, a well-known problem in retrospective studies is the fact that they largely depend on the memory of patients or their parents or on the accuracy and completeness of data in medical charts. In our review only 3 studies were clearly prospective in design and execution; in 55% of the studies, it was not clear whether it was a prospective or retrospective design.
Third, the accuracy of the measurement of blood glucose is another important source of variability. In our review, 72% of the studies reported which assay method was used, but only 44% of these studies used the reliable glucose-oxidase method.
Next, assessment of outcomes should be blinded for prognostic information. This is especially important for outcomes requiring a great deal of judgment, as is the case for some neurodevelopmental tests and neurologic examination. In our review, only 33% of the studies used blinded outcome assessment.
Then, follow-up should be long enough to detect the outcomes of interest. Several studies have shown that cognitive delay, particularly in the mildest forms, cannot be predicted accurately from tests in early infancy.54 Ideally, in this field of research, an intelligence test should be performed at, for example, 7 years of age. In our review, only 11% of the studies had a follow-up of 5 to 7 years of age, and potential underestimation of the risk of impairment of cognitive performance may have occurred.
Next, to get a valid picture of the relative impact of the primary prognostic variable (ie, a low neonatal blood sugar level), it is important to account for the presence of other prognostic variables that may confound the observed relationship with impaired neurodevelopment. Adjustment by multiple-regression analysis or by stratification according to the presence or absence of other prognostic factors for adverse neurodevelopment should be performed. Because the choice of prognostic factors in any analysis is arbitrary, we chose the 4 prognostic factors that should be addressed anyway in this field. We are aware that there are other prognostic factors that affect neurodevelopment, such as "chronic lung disease" and "time spent on the ventilator." Still, only 22% of the studies assessed the defined "minimum" of other prognostic factors for adverse neurodevelopment and adjusted the results accordingly. In this line of thought, the acute treatment of neonatal hypoglycemia is another important issue. If the infants' treatment varies with the presence of other prognostic variables, then a study cannot provide an unbiased and meaningful assessment of the prognostic significance of neonatal hypoglycemia episodes per se. Neonatal hypoglycemia treatment was fully described in only 50% of the included studies. Ideally, prognostic variables should be evaluated in a cohort of patients treated according to a protocol, such as in a randomized trial.
As in other varieties of retrospective empirical research, reviewing the medical literature is prone to several types of bias. The one most known is publication bias, a form of selection bias in systematic reviews in which "positive" results have a better chance of being published and included in the review than "negative" results. This holds especially for intervention research, where a study showing a beneficial treatment effect is more likely to be published than a study showing no treatment effect or an adverse effect. In our sample of prognostic studies, we found heterogeneous results, suggesting that publication bias is probably not a great problem. Second, it is possible that we missed some studies. However, a comprehensive search of the published literature was conducted, and experts in this field were asked for additional studies. We therefore believe that no important studies were missed. Another potential limitation of our review is the fact that we only included studies written in English, French, German, or Dutch. Several studies have shown that language restrictions do not change the results of systematic reviews.55,56 Another type of bias is information bias; the available information in the articles may be incomplete or may be generated by using methods that are of substandard quality. In this case, reviewers need to be cautious with both the validity of the information they include in their review and the completeness of the material. In our review, data were always abstracted independently by 2 investigators using structured data-abstraction forms. In addition, the methodologic quality was also assessed by 2 investigators. We used a widely quoted and internationally used checklist for the methodologic quality of prognostic studies described by the Evidence-Based Medicine Working Group,6 with some important additional items.7 One difficulty with any quality-assessment tool is that answering the question for each item often requires judgment. To overcome this problem, we prespecified each general methodologic question from the list for our review and used a pilot sample to test the interrater agreement. Using these context-specific questions, it appeared that little discussion about differences in the answers to the methodologic questions was needed. As a result, there was 100% agreement about the "overall methodologic quality" of the individual studies.
We conclude that the methodologic quality of the majority of the included studies is poor. Even the 2 "high-quality" studies have limitations. The study by Brand et al53 is a retrospective study, underpowered to detect differences in IQ smaller than 15 points. Also, the motivations for starting or withholding glucose treatment are unavailable. A minor point is the fact that the DDS that was used is less comprehensive and sensitive than the Bayley Scales of Infant Development. The study by Lucas et al40 was not a prospective study to investigate the long-term effects of neonatal hypoglycemia. Rather, retrospective data were obtained for a group of preterm infants with birth weights <1850 g who had been enrolled in a multicenter feeding study. The investigators adjusted the results for cerebral hemorrhage, but only 2 of 5 participating centers conducted routine ultrasound investigations. At the time of the study in 1988, it was probably not possible to reliably detect small cerebral hemorrhages on ultrasound, which may, on the other hand, not be major confounders of the observed relationship between neonatal hypoglycemia and neurodevelopment. Furthermore, in an observational study design it is hard to prove that the association between modest hypoglycemia and poor neurodevelopment is causal or just reflects failure to adjust for all potential (known and unknown) confounders. The clinical importance of a reduction in mental and motor development scores of 14 and 13, respectively, at 18 months may be questioned. Although never formerly published in an article, in his response to a letter to the editor,57 Lucas produced long-term follow-up results of this study: at 7.5 to 8 years, evidence of persisting associations between neonatal hypoglycemia and lower test scores were found in 2 of the 4 outcomes, arithmetic and motor tests, with 
0.5 SD reduction in scores (adjusted for respiratory support, birth weight, and gestation) after the neonatal concentration of blood glucose was <2.6 mmol/L for >3 days (P < .005).57
We considered statistical pooling of the results of the included studies to get an estimate of the overall effect of episodes of neonatal hypoglycemia on subsequent neurodevelopment. Heterogeneity is a major threat to the validity of such meta-analyses and can be ascribed to differences in study methods, study populations, interventions, outcomes, or chance.58 In our study we found, as stated earlier, major clinical heterogeneity in patient characteristics, definitions of hypoglycemia, assay methods, treatment protocols, length of follow-up, assessed outcomes, and methodologic quality (Table 1). Therefore, we decided that the statistical pooling of results was inappropriate.
None of the studies that we reviewed could validly quantify the effect of episodes of neonatal hypoglycemia on subsequent neurodevelopment. In the last 15 years, several authors have urgently requested methodologically sound studies.3,59–62 To our knowledge, however, no one has come up with a design for such a future study that overcomes the wide range of problems that we encountered, and as of yet, these studies have not been performed, probably because of the perceived complex nature of the study of hypoglycemia in the neonate. Yet, we do think that this problem deserves further study and that it can be studied using valid methods. On the basis of what we learned about the studies included in our review and pathophysiological concepts, we propose an optimal study design to answer the still open questions. We provide a guide for the design and execution of such a future study in the Appendix.
|
CONCLUSIONS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONSAPPENDIX: OPTIMAL STUDY DESIGN...REFERENCES |
|---|
|
APPENDIX: OPTIMAL STUDY DESIGN TO ESTABLISH THE RELATIONSHIP BETWEEN NEONATAL HYPOGLYCEMIA AND SUBSEQUENT NEURODEVELOPMENT |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONSAPPENDIX: OPTIMAL STUDY DESIGN...REFERENCES |
|---|
|
Design
The design should be that of a prospective cohort study to answer question 1, with a nested randomized, controlled clinical trial to answer question 2.
Patient (Sub)groups
High-risk patient groups for hypoglycemia with subsequent brain damage have not been defined in the available studies. Therefore, definition of high-risk groups can currently be based on pathophysiological reasoning only. Hypoglycemia can cause brain damage, because glucose is the major fuel for brain cells. Alternative fuels for the perinatal brain are lactate and ketone bodies.63,64 The immature brain is capable of using other organic metabolites such as free fatty acids and amino acids.63,64 High-risk groups for developing hypoglycemia are those infants with diminished glucose-production capacity (ie, with impaired glycogenolysis or gluconeogenesis) and those with increased glucose utilization. Using this approach, infants who cannot rely on sufficient alternative fuel supply are at particular risk for subsequent brain damage. Thus, the most common risk groups for hypoglycemia with subsequent brain damage are term infants (SGA, LGA), infants born preterm (SGA, AGA, LGA), and infants from diabetic mothers. These children are the prime target of future studies. Infants with symptomatic hypoglycemia must be analyzed separately. To attribute signs and symptoms to hypoglycemia, Whipple's triad must be fulfilled (ie, a reliable low–blood glucose measurement, signs and symptoms consistent with hypoglycemia, and resolution of signs and symptoms after restoring blood glucose to normal values).65
Measurements
Ideally, continuous measurements of glucose, alternative fuels (main: ketone bodies, lactate; additional: free fatty acids, amino acids), glucoregulatory hormones (insulin, glucagon, cortisol, growth hormone, catecholamines), and gluconeogenic precursors (glycerol, amino acids, lactate) should be collected. This approach will also reveal information on the pathogenesis of brain damage resulting from hypoglycemia and on the pathophysiology of hypoglycemia itself and could be helpful in identifying subgroups of infants with a higher risk for hypoglycemia-induced brain damage. Because it will not be possible to measure all these variables in large groups of infants on a regular basis and, more importantly, these measurements are not available on short notice in daily clinical practice, the decision to treat hypoglycemia as yet is based on the blood glucose concentration only. Therefore, we propose the measurement of glucose, and the main alternative fuels ketone bodies and lactate every 3 hours before feeding for the first 72 hours of life. Number of episodes and depth and duration of hypoglycemia should all be recorded.
Assay Method
Faultless blood-sampling and handling techniques are critical, because small mistakes can cause major errors in the results. Therefore, the protocol should contain detailed instructions on blood-sampling, sample-handling, and analysis methods.
Treatment
Treatment for hypoglycemia should be standardized. Routine blood glucose measurements can result in: (1) very low glucose concentrations (eg, <1.8 mmol/L), (2) moderately low glucose concentrations (eg, 1.8–2.6 mmol/L), or (3) glucose concentrations considered "safe" (eg, >2.6 mmol/L). With very low glucose concentrations, and also in symptomatic hypoglycemia, treatment in the form of carbohydrate supplementation will always be instituted, and with glucose concentrations considered safe, no such treatment is necessary. Moderately low glucose concentrations are in the "gray zone," and here the controversy about the necessity to treat these infants is maximal. Therefore, with moderately low glucose concentrations, the allocation to a carbohydrate-supplementation group and a nonintervention/close-monitoring group could best be randomized. The short-term aim of the intervention is to keep the plasma glucose level above a very low glucose level for all children. An example of a standardized treatment intervention could be to raise the carbohydrate intake with a predefined amount of glucose (expressed in mg/kg per minute).
Other Risk Factors for Neurodevelopmental Impairment
Cerebral hemorrhage, asphyxia, sepsis, respiratory distress syndrome, chronic lung disease, time spent on the ventilator, other neurologic diseases, etc, should be recorded prospectively according to accepted definitions and be explored as both confounders and effect modifiers of the central determinant-outcome relationship, preferably using multivariate-regression techniques.
Outcome Measurement Instruments
Animal studies have shown that severe hypoglycemia causes neuronal loss in the superficial cerebral cortex, the dentate gyrus, the hippocampus, and the caudate nucleus. The brainstem and the posterior fossa structures are apparently spared.66 Therefore, we may expect severe hypoglycemia to be associated with both intellectual and motor deficits in survivors. Therefore, neurologic function, neurodevelopment, and motor development should be assessed at predefined ages by using standardized and validated tests. Several studies have shown that cognitive delay, particularly in the mildest forms, is difficult to detect during early infancy. Therefore a validated intelligence test should be performed at a suitable age. In any case, outcome assessors have to be blinded to the perinatal data.
Sample Size
For the intervention part of this prospective study (ie, the comparison of more or less invasive treatment of children with moderately low glucose concentrations), the sample size should be based on the exclusion of a clinically important difference in neurodevelopmental outcome for the treatment groups. For example, for Bayley scales and intelligence tests, a difference of 1 SD (15 IQ points) is generally considered to be clinically important. Yet, because a 15-IQ point difference is rather large, we suggest to not tolerate an average IQ >5 IQ points lower in the nonintervention/close-monitoring group compared with the intravenous carbohydrate-supplementation group. This 1-sided question could be answered in a noninferiority trial design. With 80% power and 1-tailed 
of .05, 112 patients would need to be included in each group. To anticipate a loss to follow-up of, for example, 30%, at least 150 patients in each group should be included, which means 300 patients in total for each subgroup.
|
ACKNOWLEDGMENTS |
|---|
|
FOOTNOTES |
|---|
Address correspondence to Nicole Boluyt, MD, Emma Children's Hospital/Center for Pediatric Clinical Epidemiology, Room H3-145, Academic Medical Centre, PO Box 22700, 1100 DD Amsterdam, Netherlands. E-mail: n.boluyt{at}amc.uva.nl
The authors have indicated they have no financial relationships relevant to this article to disclose.
All authors are responsible for the reported research. Dr Boluyt is the guarantor of and had the idea for this article. All authors were involved in the design of this review and the interpretation of data. Dr Boluyt performed the literature search. Drs Boluyt and van Kempen selected studies for inclusion. Drs Boluyt and Offringa independently abstracted data from all included studies using structured data-abstraction forms and assessed the design and execution of each study. All authors were involved in the development of the "optimal study design." Dr Boluyt wrote the concept article and Drs van Kempen and Offringa commented on subsequent drafts of this article. All authors approved the manuscript as submitted.
* Refs 8, 16–18, 22, 24, 25, 27, 28, 32, 37, 38, 40, 45, 46, 50, and 52. 
Refs 10, 13, 14, 19–21, 23, 29–31, 34, 35, 47, and 49.
Refs 8,16–18, 22, 24, 25, 27, 28, 32, 37, 38, 40, 45, 46, 50, 52, and 53.
Refs 16, 22, 24, 25, 27, 37, 38, 45, 50, and 52.
|| Refs 8, 16, 22, 25, 27, 37, 38, 46, 50, and 52.
¶ Refs 8, 16–18, 22, 24, 25, 27, 32, 37, 38, 46, 50, and 52.
# Refs 16, 17, 24, 25, 27, 28, 32, 37, 38, 40, 46, 50, and 53.
** Refs 17, 24, 27, 37, 40, 46, 50, and 53.
Refs 16–18, 22, 24, 25, 27, 28, 32, 37, 38, 40, 45, 46, 50, 52, and 53.
Refs 16, 24, 27, 32, 37, 38, 40, 46, and 53.
|
REFERENCES |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONSAPPENDIX: OPTIMAL STUDY DESIGN...REFERENCES |
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
