PEDIATRICS Vol. 122 No. 1 July 2008, pp. 52-57 (doi:10.1542/peds.2007-2849)
ARTICLE |
Child Versus Adult Research: The Gap in High-Quality Study Design
Division of General Pediatrics, Boston University School of Medicine/Boston Medical Center, Boston, Massachusetts
 |
ABSTRACT |
|---|
 TOP ABSTRACT METHODSRESULTSDISCUSSIONREFERENCES |
|---|
OBJECTIVE. The objective of this study was to determine whether there were differences in study design and purpose between published child- and adult-focused clinical research.
METHODS. We reviewed all articles published in the New England Journal of Medicine, Journal of the American Medical Association, Annals of Internal Medicine, Pediatrics, Archives of Internal Medicine, and Archives of Adolescent and Pediatric Medicine during the first 3 months of 2005 and assessed each study's design and purpose. We compared articles focused on adults with those focused on children.
RESULTS. We included 370 original research reports in our analysis (New England Journal of Medicine, n = 46; Journal of the American Medical Association, n = 60; Annals of Internal Medicine, n = 27; Pediatrics, n = 130; Archives of Internal Medicine, n = 73; Archives of Adolescent and Pediatric Medicine, n = 34), of which 189 included only adults as subjects and 181 only children. Among adult studies, compared with child studies, there were more randomized, controlled trials (23.8% vs 8.8%) and systematic reviews (10.6% vs 1.7%) and fewer cross sectional studies (16.9% vs 40.9%). Study purposes also varied, with studies of therapies constituting 38.1% of adult studies, compared with 17.7% of child studies. In contrast, epidemiological studies, defined as studies describing the prevalence or incidence of diseases or risk factors or showing associations between risk factors and diseases, constituted 6.4% of adult studies, compared with 26.5% of child studies.
CONCLUSIONS. In 6 leading generalist and specialist journals, studies involving adults were significantly more likely than child studies to be randomized, controlled trials, systematic reviews, or studies of therapies. If such studies are to be viewed as the highest possible quality of evidence, then this difference has implications for quality of care for children and for funding and future directions in clinical research involving children.
Key Words: pediatric research • research design • randomized • controlled trials • adult • child • therapeutics
Abbreviations: RCT—randomized, controlled trial
Clinical investigators face numerous challenges when conducting research involving children. Challenges include difficulties enrolling children,1 small market share for therapeutic agents aimed at children,2 and low prevalence of many childhood diseases.3,4 These barriers are less problematic for research involving adults, which may explain why the evidence for much health practice for children is thought to be less robust than that for adults.5,6 The lack of evidence for health care practices involving children is particularly striking when medications are considered; approximately two thirds of all medications commonly prescribed for children have insufficient evidence regarding dosing, safety, and efficacy.7,8
It has been argued that it may be appropriate to recommend specific health care practices for which high-quality evidence, such as that from randomized, controlled trials (RCTs), is not available but for which there is a strong history of perceived usefulness based on clinical expertise.9 There is good rationale for this approach in some situations, especially in the application of low-risk and low-cost services. In addition, classic health services and well-designed observational studies can be useful in guiding care when RCTs are cost-prohibitive or too difficult to perform or when the topic of interest does not lend itself to a randomized design.10 To counter the lack of evidence for therapeutic agents for children, however, many have argued that additional high-quality RCTs are required.1,3,4
Despite the widespread perception that child health research lags behind adult research, this has not been studied systematically. Therefore, we sought to compare recently published adult and child health research studies. Our primary hypothesis was that RCTs, and by association studies of therapies, would be less frequent among child studies than among adult studies. A secondary hypothesis was that child RCTs would be most frequent where access to large numbers of children is greatest, namely, general pediatrics. We reviewed all reports published during a 3-month period in 6 leading journals, and we compared the study design and study purpose across adult and child studies.
|
METHODS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Journals Studied
All original research articles published during the first 3 months of 2005 in 6 journals were reviewed; 2 journals were generalist journals, publishing in all fields (New England Journal of Medicine and Journal of the American Medical Association), 2 journals focused on children (Pediatrics and Archives of Pediatrics and Adolescent Medicine), and 2 journals focused on adults (Annals of Internal Medicine and Archives of Internal Medicine). These journals were chosen because they have among the highest impact factors in their specific fields.11 Considering that
60 original articles relevant to children and an equal number relevant to adults are published in these journals each month and that 10% of the child studies tend to be RCTs, we estimated that 3 months’ worth of publications would be necessary to detect a 10% difference in the proportions of child versus adult RCTs and studies of therapies, with 80% power and a 2-tailed 
level of .05.
Article Review and Classification
Review Procedure
One author (Dr Martinez-Castaldi) read each article abstract, with review of the full journal article as needed for clarification of the variables of interest. A second investigator (Dr Silverstein) independently reviewed a random subset of 50 articles. The interrater reliability for classification of study design and study purpose was determined.
Study Population: Adult Versus Child Studies
We defined study populations according to the age of the study subjects, regardless of the journal in which the study was published. Adult studies were defined as those that included only subjects >21 years of age or that studied health care systems pertaining only to that age group. Child studies were defined as those that included only subjects between 0 and 21 years of age or that studied health care systems pertaining only to that age group. If there was any overlap in ages, then the study was placed in the "both adult and child" category. All 3 authors later reviewed these categorizations and moved studies out of the both adult and child category if a clear majority of patients were either children or adults. We excluded studies that remained in the both adult and child group, as well as studies of other populations (including studies of fetal tissue, animal studies, studies of child risk factors for adult disease, and studies of maternal risk factors for child disease).
Study Design
We defined study design on the basis of the traditional epidemiological classification,12 that is, RCTs, systematic reviews or meta-analyses, retrospective cohort studies, prospective cohort studies, case-control studies, cross-sectional studies, clinical trials (nonrandomized), diagnostic test studies, case series, case reports, and ecologic studies (Table 1). Case series and cohort studies were distinguished as follows. A case series was defined as a study of a series of <100 subjects with a defined disorder, with or without a therapeutic intervention, without a concurrent control group. Cohort studies were those that defined a group of subjects according to their exposure to a risk factor (defined a priori) and monitored the subjects over time for the outcome of interest. Any design that did not fall into one of these categories was defined as "other."
|
Study Purpose
We used 6 categories for study purpose, namely, therapy, diagnosis, prognosis, harm, epidemiology, and health services (Table 1). We adapted the first 4 from Users’ Guide to the Medical Literature: Essentials of Evidence-Based Clinical Practice13 and created the latter 2 on the basis of author consensus on the literature published in these journals (Table 2). Epidemiological studies demonstrated the prevalence or incidence of a disease or risk factor or studied associations between risk factors and diseases. Health services studies investigated health care providers and systems (including studies of issues of cost and quality and health services research itself).
|
Study Location
We defined study location as the setting in which the study was conducted (Table 1). We compared studies located in inpatient settings (medical wards, ICUs, and labor and delivery wards) versus outpatient settings (general practices, subspecialty practices, and clinical research centers). A secondary analysis of outpatient practices compared general versus subspecialty practice locations.
Data Analysis
Unadjusted comparisons of adult and child studies were performed by using the 
2 test. P values of 
.05 were taken to be statistically significant. It was not possible to adjust for journal in a multivariate analysis because there would be journal cells with 0 observations, that is, there were no adult studies published in child journals or child studies published in adult journals. Therefore, we performed subanalyses with only the generalist journals, which publish both adult and child studies. All analyses were performed by using SAS 9.1 (SAS Institute, Cary, NC).
Institutional Review Board Approval
Our study did not fit the definition of human subjects research.14 Therefore, institutional review board approval was not sought.
|
RESULTS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Studies Reviewed
We reviewed a total of 405 original research reports (New England Journal of Medicine, n = 53; Journal of the American Medical Association, n = 72; Annals of Internal Medicine, n = 28; Pediatrics, n = 134; Archives of Internal Medicine, n = 82; Archives of Adolescent and Pediatric Medicine, n = 36). Of these, 189 included only adults as subjects, 181 only children, and 26 both adults and children. Studies pertaining to other study populations totaled 9; 4 used fetal tissue, 1 was an animal study, 1 studied child risk factors for adult disease, 2 studied maternal risk factors for child diseases, and 1 was unidentifiable. We excluded all studies that did not fall into the categories of adult or of child studies, leaving 370 studies.
Interrater Reliability
For study design, the overall interrater reliability15 was substantial (
= 0.65). Agreement was almost perfect for RCTs ( = 0.91), and it was substantial for systematic reviews/meta-analyses ( = 0.79) and cross-sectional studies ( = 0.63). For study purpose, the overall agreement was moderate ( = 0.56), but it was substantial for therapeutic studies ( = 0.77) and epidemiological studies ( = 0.65).
Study Design
Among the 370 abstracts, the distribution of study designs differed significantly according to study population. In adult studies, compared with child studies, there were more RCTs (23.8% vs 8.8%; P < .001) and systematic reviews (10.6% vs 1.7%; P < .001). However, there were fewer cross-sectional studies (16.9% vs 40.9%; P < .001). There were no significant differences between adult and child studies in the proportions of any other study design (Table 3). When we restricted our analysis to studies of therapies, there were no differences between adult and child studies in the proportions of RCTs (54.2% vs 50%; P = .69) or systematic reviews (15.3% vs 6.3%; P = .34).
|
Study Purpose
In adult studies, compared with child studies, there were more studies of therapies (38.1% vs 17.7%; P < .001) and fewer epidemiological reports (6.4% vs 26.5%; P < .001). Other study purposes were not significantly different between adult and child studies (Table 4).
|
Generalist Journal Subanalyses
We performed subanalyses by using only the generalist journals, which publish both child and adult studies. In the New England Journal of Medicine and the Journal of the American Medical Association, there were 89 adult studies and 17 child studies. For study design, there were no statistically significant differences between adult and child studies, but trends were similar to the aforementioned results. In adult studies, compared with child studies, there tended to be more RCTs (37.1% vs 17.7%; P = .17), more systematic reviews (10.1% vs 0%; P = .35), and fewer cross-sectional studies (10.1% vs 23.5; P = .22). For study purpose, in adult studies, compared with child studies, there were more studies of therapies (51.7% vs 23.5%; P = .04) and fewer epidemiological studies (4.5% vs 29.4%; P = .005).
Study Location
There were no significant differences in study location between adult and child studies. The proportions of outpatient (compared with inpatient) studies did not differ between adult and child studies (68% vs 59.8%; P = .24), and neither did the proportion of outpatient studies conducted in general practice (compared with subspecialty practice) (21.9% vs 32.7%; P = .18).
When our location analysis was restricted to RCTs, the proportions of outpatient (compared with inpatient) adult and child RCTs did not differ (65% vs 63.6%; P = 1). However, adult outpatient RCTs were less likely to be conducted in general practice (compared with subspecialty practice) than were child outpatient RCTs (15.4% vs 71.4%; P = .009). The most common location of RCTs among child studies was general practice, whereas that among adult studies was subspecialty practice.
|
DISCUSSION |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
In 6 leading journals, we found that studies involving adults, compared with children, were more likely to be RCTs or systematic reviews and were more likely to focus on therapy. Of studies involving therapies, however, child and adult studies were equally likely to be RCTs.
To our knowledge, no study to date has directly compared adult and child studies with respect to study design and purpose. One recent study compared adult and pediatric RCTs in 5 generalist journals and found that the rate of increase of adult RCTs over a 20-year period was significantly greater than that of pediatric RCTs and that the quality of RCTs was similar between adult and child studies.16 Our study parallels those findings and extends them to other study designs. We found that a greater proportion of child studies, compared with adult studies, used a cross-sectional design, but we found no significant differences between adult and child studies for other designs. However, we were not specifically powered to assess those other designs. Although we did not examine the quality of individual studies, we found that, when faced with the challenge of studying a therapy, adult and child health researchers were equally likely to choose high-quality study designs. Overall, however, child health researchers carry out these high-quality designs and studies of therapies far less often than their adult research counterparts.
Our study quantifies the potential gap in high-quality study design. This gap is reflected in the available evidence for child and adult health practices. For example, in a review of the US Preventive Services Task Force recommendations, we found that child health recommendations for preventive care were more likely to be graded "I" for insufficient evidence than were adult health recommendations (C.M.C., M.S., H.B., unpublished data, 2006). The 7 most-recent US Preventive Services Task Force child health recommendations involved screening for developmental hip dysplasia, elevated lead levels in high-risk and average-risk children, speech and language delays in young children, and iron deficiency anemia in children 6 to 12 months of age, as well as treatment with iron supplementation for children at average and increased risk for iron deficiency anemia. For 5 of these 7, there was insufficient evidence to make a recommendation. In every case, the US Preventive Services Task Force stated that the grading of "I" was driven at least in part by lack of evidence for effective interventions.
It is quite possible, however, that our finding of a discrepancy in the proportions of high-quality study designs in child literature, compared with adult literature, does not represent a deficiency in child research. It could be that the determinants of child health are best assessed through nonrandomized studies or that clinical therapies are not as important for children as they are for adults. Kleinman9 argued that child well-being must be seen in the greater context of family, community, and society. A recent report from the United Nations Children's Fund on child well-being supports this position.17 Very few of the measures used in this report represent specific diseases or traditional medical outcomes. For example, rates of poverty, school achievement, and family structures are included in the 6 dimensions assessed. Of note, the United States ranked in the bottom third for 5 of the 6 dimensions. To help understand these important aspects of child health, classic epidemiological studies may be necessary and relatively more important than such studies in the adult population.
There are numerous other reasons why there are fewer RCTs involving children. First, investigators and institutional review boards are faced with the dual challenge of ensuring that children are not denied access to clinical trials and ensuring that research risks are minimized,18 which has been difficult to implement consistently.19 Second, enrollment of children in clinical trials is difficult, because of both low disease prevalence and parental reluctance.1,3,4,20 Lastly, securing funding has always been difficult for child health researchers, with small numbers of National Institutes of Health grants going to pediatrics departments and limited support for training of future child health researchers.21–23 This could be driven by the decreasing proportion of National Institutes of Health dollars focused on child health research (12.3% in 1998 and 11.2% in 2003; Lynne M. Haverkos, MD, MPH, written personal communication, 2005).
There have been attempts to increase the number of child RCTs. Notable among these is the Best Pharmaceuticals for Children Act of 2002,24 which encourages drug companies to test drugs in children by giving them 6 months of market exclusivity in exchange for doing so. This benefit has been controversial25 but, overall, information has been gained, with 37% of these studies resulting in a labeling change.26 Another area of success in promoting child RCTs has been increased collaboration between institutions to overcome the problem of low disease prevalence. This is exemplified by national cancer research groups, which have been conducting multicenter trials for decades.27 More recently established collaborative research networks, including the Pediatric Emergency Care Applied Research Network28 and the Medicines for Children Research Network in the United Kingdom,29 hold great promise for the future of child RCTs. However, multicenter trials present challenges of their own.3
Our study was limited by several factors. First was the choice of journals. We chose the 6 journals because of their high impact factors and because they are considered to publish the highest-quality research within their respective fields, but they may not be representative of all journals. Specifically, the quality of all published child and adult studies may not be as high as those published in these 6 journals. Second, our data represent a 3-month snapshot in time, and there is a possibility that these 3 months do not reflect the published literature. Third, although RCTs are judged to represent the highest quality of research, there can be variability in the quality of RCTs. We did not formally assess the quality of individual RCTs. Lastly, we acknowledge that our interrater reliability showed moderate agreement for all study purposes combined. However, our primary aim was to compare high-quality trials and studies of therapies, for which we obtained almost perfect ( = 0.91) and substantial ( = 0.77) agreement, respectively.
These limitations notwithstanding, our data show that RCTs, systematic reviews, and studies of therapies are less common among child studies, compared with adult studies. Although the barriers to carrying out these studies are real, the lack of high-quality study designs has repercussions for the quality of care for children, especially in the realm of effective therapies. Given the unique challenges of conducting research with children, future efforts will likely entail optimizing observational studies while striving to reduce barriers and to prioritize efforts to conduct RCTs in children.
|
ACKNOWLEDGMENTS |
|---|
Funding was provided by National Institutes of Health grant T32 HP1001414.
|
FOOTNOTES |
|---|
Accepted Nov 1, 2007.
Address correspondence to Carolina Martinez-Castaldi, MD, Division of General Pediatrics, Boston University School of Medicine/Boston Medical Center, 88 E. Newton St, Vose 3, Boston, MA 02118. E-mail: carolina.martinez-castaldi{at}bmc.org
The authors have indicated they have no financial relationships relevant to this article to disclose.
| What's Known on This Subject There is a lack of evidence for many child health practices, including medication use. The highest quality of evidence is thought to come from RCTs. However, there are several barriers to conducting RCTs in children.
|
| What This Study Adds In 6 leading journals, studies involving adults were significantly more likely than those involving children to be RCTs, systematic reviews, or studies of therapies. Among studies of therapies, however, adult and child studies were equally likely to be RCTs.
|
|
REFERENCES |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
- Caldwell PH, Murphy SB, Butow PN, Craig JC. Clinical trials in children. Lancet.2004; 364 (9436):803 –811[CrossRef][Web of Science][Medline]
- Kauffman RE. Essential drugs for infants and children: North American perspective.
Pediatrics.1999; 104
(3):603
–605
[Free Full Text] - Bauchner H. Clinical Trials in Children. Hoboken, NJ: Wiley; In press.
- Smyth RL, Weindling AM. Research in children: ethical and scientific aspects. Lancet.1999; 354 (suppl 2):SII21 –SII24[CrossRef][Medline]
- Leatherman S, McCarthy D. Quality of Care for Children and Adolescents: A Chartbook. New York, NY: Commonwealth Fund;2004
- Mangione-Smith R, Onstad K, Wong L, Roski J. Deciding not to measure performance: the case of acute otitis media. Jt Comm J Qual Saf.2003; 29 (1):27 –36[Medline]
- Steinbrook R. Testing medications in children.
N Engl J Med.2002; 347
(18):1462
–1470
[Free Full Text] - Coté CJ, Kauffman RE, Troendle GJ, Lambert GH. Is the "therapeutic orphan" about to be adopted?
Pediatrics.1996; 98
(1):118
–123
[Abstract/Free Full Text] - Kleinman LC. Prevention and primary care research for children: the need for evidence to precede "evidence-based." Am J Prev Med.1998; 14 (4):345 –351[CrossRef][Web of Science][Medline]
- Kramer MS. A comparison of laboratory and epidemiologic approaches to pediatric research. Pediatr Res.1998; 43 (6):711 –718[CrossRef][Web of Science][Medline]
- Thomson Scientific. ISI Web of Knowledge. Available at: http://scientific.thomson.com/isi/. Accessed October 25,2007
- Aschengrau A, Seage G. Essentials of Epidemiology in Public Health. Sudbury, MA: Jones & Bartlett;2003
- Guyatt G, Rennie D, eds. Users’ Guide to the Medical Literature: Essentials of Evidence-Based Clinical Practice. 3rd ed. Chicago, IL: AMA Press;2004
- US Department of Health and Human Services. Code of Federal Regulations, Title 45: Public Welfare, Part 46: Protection of Human Subjects. Washington, DC: US Department of Health and Human Services;2005 . Available at: www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.htm. Accessed October 25, 2007
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics.1977; 33 (1):159 –174[CrossRef][Web of Science][Medline]
- Cohen E, Uleryk E, Jasuja M, Parkin PC. An absence of pediatric randomized controlled trials in general medical journals, 1985–2004. J Clin Epidemiol.2007; 60 (2):118 –123[CrossRef][Web of Science][Medline]
- Adamson P. Child Poverty in Perspective: An Overview of Child Well-being in Rich Countries: Report Card 7. Florence, Italy: UNICEF Innocenti Research Centre;2007
- Federman D, Hanna K, Roderiguez L, eds. Responsible Research: A Systems Approach to Protecting Research Participants. Washington, DC: National Academies Press;2002
- Shah S, Whittle A, Wilfond B, Gensler G, Wendler D. How do institutional review boards apply the federal risk and benefit standards for pediatric research?
JAMA.2004; 291
(4):476
–482
[Abstract/Free Full Text] - Caldwell PH, Butow PN, Craig JC. Parents’ attitudes to children's participation in randomized controlled trials. J Pediatr.2003; 142 (5):554 –559[CrossRef][Web of Science][Medline]
- Stiehm ER. Growth and development of pediatric research. Pediatr Res.1985; 19 (6):593 –598[Web of Science][Medline]
- Stiehm ER. Pediatric research: National Institutes of Health support in the "steady-state" decade, 1983–1993.
Arch Pediatr Adolesc Med.1996; 150
(9):971
–974
[Abstract/Free Full Text] - McCabe LL. National Institutes of Health support for research and training: future of pediatrician scientists.
Arch Pediatr Adolesc Med.1998; 152
(9):839
–842
[Free Full Text] - Best Pharmaceuticals for Children Act, Pub L 107–109, 107th Cong (January 4, 2002)
- Zimmerman R. Drug makers find a windfall testing adult drugs on kids. Wall Street Journal. February 5,2001 :1
- Benjamin DK Jr, Smith PB, Murphy MD, et al. Peer-reviewed publication of clinical trials completed for pediatric exclusivity.
JAMA.2006; 296
(10):1266
–1273
[Abstract/Free Full Text] - Reaman GH. Pediatric cancer research from past successes through collaboration to future transdisciplinary research.
J Pediatr Oncol Nurs.2004; 21
(3):123
–127
[Abstract/Free Full Text] - Corneli HM, Zorc JJ, Majahan P, et al. A multicenter, randomized, controlled trial of dexamethasone for bronchiolitis.
N Engl J Med.2007; 357
(4):331
–339
[Abstract/Free Full Text] - Medicines for Children Research Network. Available at: www.mcrn.org.uk. Accessed October 25,2007
PEDIATRICS (ISSN 1098-4275). ©2008 by the American Academy of Pediatrics
CiteULike 
Connotea 
Del.icio.us 
Digg 
Facebook 
Reddit 
Technorati 
Twitter What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
