 |
|
 |
|
 |
 |
|
 |
 |
 |
 |
 |
 |
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ARTICLE |
a Department of Pediatrics and Medicine
c Department of Medicine and Feist-Weiller Cancer Center, Louisiana State University Health Sciences Center, Shreveport, Louisiana
b Institute for Healthcare Studies, Northwestern University, Chicago, Illinois
d Department of Pediatrics, University of California, Davis, California
e Department of Psychology, Louisiana Tech University, Ruston, Louisiana
 |
ABSTRACT |
|---|
 TOP ABSTRACT METHODSRESULTSDISCUSSIONLIMITATIONSCONCLUSIONSREFERENCES |
|---|
PATIENTS AND METHODS. A total of 1533 adolescents aged 10 to 19 years attending 1 of 5 middle schools, 3 high schools, 1 pediatric clinic, or 2 summer programs in Louisiana and North Carolina participated in face-to-face interviews. Demographic information was solicited, and participants were administered a battery of reading tests, including the REALM-Teen, Wide Range Achievement Test–Revised (WRAT-3), and Slosson Oral Reading Test–Revised (SORT-R). Internal consistency for the REALM-Teen was determined using Cronbach’s 
, and criterion validity was established through correlations with both the WRAT-R and SORT-R. Using reading below grade level (according to SORT-R scores) as an outcome, instrument accuracy and corresponding cutoff scores were calculated by plotting receiver operating characteristic curves and stratum-specific likelihood ratios.
RESULTS. Participants were 50% black and 53% female; 34% were enrolled in middle school and 66% in high school. The average time required to administer the REALM-Teen was 3 minutes. Internal consistency was excellent, as was test-retest reliability. The REALM-Teen is strongly correlated with both the WRAT-R and SORT-R. Five reading level categories were identified: 3rd grade and below, 4th to 5th grade, 6th to 7th grade, 8th to 9th grade, and 10th grade and above. Forty-six percent of participants were reading below grade level according to the SORT-R and 28% had repeated at least 1 grade.
CONCLUSION. The REALM-Teen is a brief, reliable instrument for assessing adolescent literacy skills and reading below grade level.
Key Words: literacy • health literacy • adolescents • literacy tests • below-grade reading
Abbreviations: WRAT-3—Wide Range Achievement Test–Revised • SORT-R—Slosson Oral Reading Test–Revised • REALM—Rapid Estimate of Adult Literacy in Medicine • REALM-Teen—Rapid Estimate of Adolescent Literacy in Medicine • SSLR—stratum-specific likelihood ratio
Low literacy is a prevalent social problem in the United States.1,2 Almost half (43%) of American adults have basic or below-basic literacy levels according to the 2003 National Assessment of Adult Literacy, and 66% of high school students have similarly low levels on the National Assessment of Educational Progress.1–4 Limited ability to read, understand, and act on health information in particular translates into poorer health and poor health outcomes in adults.5–11 The magnitude and consequences of low literacy are being recognized as important factors not only in education and employment but also in health and health care.8–12 However, the extent to which literacy affects health, health behaviors, and health care during adolescence has been inadequately studied.
Identifying low literacy in adolescents could be helpful to health professionals as a screen for academic problems, a potential marker for health risk behaviors, and to know when to tailor health information. We know that low literacy is a risk factor for school failure and school drop out,3,4 both of which are associated with increased health risk behaviors in teens.13–16 Approximately one quarter of American adolescents are reading well below grade level.4 These students do not have the reading skills to comprehend information found in their text books and are at risk for falling further behind and eventually dropping out of school. Currently, almost one third of 9th-grade students (one half among minority students) do not finish high school.17
Below-grade reading has been shown to impact adolescent health risk behavior and health care.18,19 Previous studies by the authors found that low-income middle school students who were reading at least 2 grades below their expected level were more likely to engage in fights with injuries needing medical attention, use weapons, and report being afraid at school compared with peers reading on or above grade level.18 Fortenberry and colleagues19 found that lower literacy was a barrier in adolescents seeking care for sexually transmitted diseases.
Academic failure has also been used as a marker for potential health risk behaviors. Although assessing adolescents’ educational progress, either by noting grade retention or failing grades, elicits important information,13–16 it does not assess literacy. Testing is the only way to accurately assess literacy level,20 yet there is no instrument in the literature specifically developed and standardized to screen adolescents for low literacy in health care settings.20
The few adolescent health studies that assessed literacy used the Wide Range Achievement Test–Revised21 (WRAT-3) or the Slosson Oral Reading Test–Revised (SORT-R).22 Both are standardized reading tests commonly used in educational settings. However, neither is ideal for health care research. The major limitation of the WRAT-3 is that it rapidly becomes difficult for low-level readers, and many patients give up quickly.20 Almost one third of the words are above a 9th-grade reading level, and even health care providers struggle with words such as "assuage," "terpsichorean," and "epithalamion." The test may cause anxiety, because it requires subjects to continue saying words out loud until they have missed 10 consecutive items. The major limitation of the SORT-R for health care settings or research is that it takes 
10 minutes to administer and score.20
We have developed a brief literacy screening test for use with adolescents in health care settings. The test is modeled on the Rapid Estimate of Adult Literacy in Medicine (REALM),20,23,24 the most commonly cited literacy test in adult health care settings.20 The purpose of this study was to validate an adolescent version of the REALM, named the Rapid Estimate of Adolescent Literacy or REALM-Teen. This test will allow health professionals to screen youth in grades 6 through 12 for below-grade reading.
|
METHODS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONLIMITATIONSCONCLUSIONSREFERENCES |
|---|
In each setting, parents were first notified of the study and were provided a form to sign if they did not want their child to participate. This implied consent is routinely used in public schools and was recommended by administrators from the school systems involved. Two weeks later, students in mainstream classes in grades 6 through 12 whose parents had not objected were recruited. Grade level was emphasized, and students were not excluded by age. A small incentive consisting of a candy bar was offered. Adolescents were taken to a separate classroom where testing could occur privately. Those who chose not to participate were offered an alternative activity. Patients and their parents in the pediatric clinic were also asked whether they or their child would be willing to participate, and parents were asked to sign a consent form. Participants were then tested in a private room in the clinic. Less than 2% of the adolescents or their parents in any setting refused participation.
A total of 1533 adolescents across all sites participated in structured interviews that included a general demographic survey. The REALM-Teen and SORT-R were administered to 1145 adolescents, of whom 953 also received the WRAT-3 as a second assessment of criterion validity. Internal consistency was determined using a separate sample of 388 students who received the REALM-Teen only.
The interviews with reading tests were administered by former school teachers, research assistants who were college graduates, and a clinic nurse. All received training on how to administer literacy assessments among this population in a standardized manner with sensitivity. Training for the REALM-Teen alone took approximately 10 minutes. The Louisiana State University Health Sciences Center–Shreveport Institutional Review Board, Caddo Parish School Board (Louisiana), and Guilford County School Board (North Carolina) all approved the study methods and instruments used.
Instruments
The REALM-Teen is a reading recognition instrument, modeled after the REALM, which measures an individual’s ability to pronounce words in ascending order of difficulty (Fig 1). 20,23 Although not designed to measure comprehension, word-recognition tests are useful predictors of general reading ability in English.20,25 If an individual has difficulty pronouncing words in isolation, which is a beginning-level reading skill, he or she is likely to have difficulty with comprehension, a higher-order skill.20,25
|
Neither the REALM, REALM-Teen, or any other formal standardized instrument is designed to comprehensively test health literacy or an individual’s capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions.20 However, several instruments assess reading comprehension and/or numeracy in a health care context. All of these instruments have been used as a proxy measure in the increasing body of health literacy research.20 Therefore, we refer to the construct measured by the REALM and REALM-Teen as literacy and not health literacy.
Like the REALM, the REALM-Teen is a 1-page instrument consisting of 66 health words arranged in increasing order of difficulty on 3 widely spaced columns on lime-green paper. Adolescents taking the REALM-Teen are asked to say the words out loud beginning with the first word in the left-hand column. When they encounter a word they cannot read, patients are told to do the best they can or say "skip" and go on to the next word. If patients stop, they are asked to look down the list and pronounce as many of the remaining words as they can; there is no time limit. The examiner marks patient responses on a separate form (Fig 1), which is easily scored by counting the number of incorrect pronunciations and skipped words and subtracting from the total. Dictionary pronunciation is the scoring standard.20,25 (A dictionary is the recognized guide for people seeking help in pronouncing unfamiliar words, regardless of their culture or the region of the country in which they reside.28) An adolescent’s raw score is the total number of correctly pronounced words. A chart for converting raw scores into grade-range estimates is printed on the back of the examiner’s copy, along with a brief description of the test and standardized directions for administering and scoring. Test administration and scoring routinely take 2 to 3 minutes.
Item Development and Pilot Testing
In developing the REALM-Teen, a preliminary list of 116 words such as "stress," "risks," "asthma," "prevention," and "hepatitis" was selected from American Academy of Pediatrics’ adolescent patient education pamphlets. The list was pilot tested on 200 students in 6th through 12th grade in a southern city. Adolescents were tested individually by a trained research assistant at 1 of 3 sites (a city-wide swim team, a Job Training Partnership Act program, and a university pediatric clinic). Participants were given a brief structured interview to elicit their age, upcoming grade in school, the number of grades they had repeated, and their race and gender. They were then given the preliminary list of words and asked to pronounce each individually.
A panel of doctors, nurses, social workers, psychologists, and educators reviewed the word list, indicating words they used most often with adolescent patients. Retention of items was based on psychometric estimate of item difficulty, item discrimination, and the panel’s judgment. The list was reduced to 66 words to be consistent with the REALM.
Criterion Validity
Criterion validity was based on correlations between REALM-Teen raw scores and the raw scores of the most current versions of 2 standardized reading tests commonly administered to adolescents, the SORT-R22 and the WRAT-3.21
The SORT-R22 is a standardized reading recognition test that evaluates an individual’s ability to pronounce words at various levels of difficulty. Two hundred words are divided into 10 columns corresponding with kindergarten to 9th-grade levels. Words are read out loud. The number of words correctly pronounced determines the raw score, which is converted into a grade level stated in years and months. Administration and scoring take 
10 minutes. The SORT-R is highly correlated with the Woodcock-Johnson Test of Achievement–Letter Word Recognition subtest and the Peabody Individual Achievement Test–Revised Reading Recognition subtest.20
The WRAT-321 is a nationally standardized achievement test consisting of letter reading (naming 15 letters of the alphabet) and word reading (pronouncing 42 words). Raw scores, which range from 1 to 57, can be converted to grade-equivalent reading levels ranging from preschool to post–high school. WRAT-3 has been extensively tested for validity and reliability and is highly correlated with the Total Reading score of the California Test of Basic Skills (4th ed) and the Total Reading score of the Stanford Achievement Test. The WRAT-3 takes 3 to 5 minutes to administer and score.21,20
Analysis Plan
Descriptive statistics (mean and standard deviation, frequency and percentage) were calculated to summarize student demographic characteristics and performance on the various literacy tests (WRAT-3, SORT-R, and REALM-Teen). Reliability of the REALM-Teen was determined by measuring internal consistency with Cronbach’s . Criterion validity was determined by calculating the correlation (Pearson r) between scores on the REALM-Teen and those on the SORT-R and WRAT-3 literacy assessments. Test-retest reliability was determined by calculating the Pearson r correlation between scores on the REALM-Teen at baseline and at 1-week follow-up. Accuracy of the REALM-Teen to predict reading skills below grade level (1 grade below, 3 grades below) according to the SORT-R was calculated using receiver operating characteristic curves. Stratum-specific likelihood ratios (SSLRs) were then calculated for each scoring category on the REALM-Teen; these ratios provide additional confirmation of the score categories by offering estimates of the likelihood of reading below grade level within each grade grouping.
|
RESULTS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONLIMITATIONSCONCLUSIONSREFERENCES |
|---|
|
|
: .94). Scores on the REALM-Teen demonstrated strong test-retest reliability (r = 0.98) and high criterion validity, as tested by correlation with the WRAT-3 (r = 0.83) and SORT-R (r = 0.93). REALM-Teen cutoff scores were identified using the SORT-R established cutoffs using frequency distributions (Table 2) . The area under the receiver operating characteristics curve for predicting reading skills (as determined by the SORT-R) below grade level was 0.84 (95% confidence interval: 0.82–0.87).
|
3rd-grade level) had nearly a fivefold greater likelihood of having reading skills below grade level compared with the next lowest category of REALM-Teen scores (4th- to 5th-grade level, scores: 38–47). Similarly, those students scoring 0 to 37 on the REALM-Teen had more than a fivefold greater likelihood of reading 3 grades below compared with students in the highest 2 categories of scores (8th- to 9th-grade level [scores: 59–62]; 10th grade and above [scores: 63–66]). Based on these findings raw REALM-Teen scores (0–66) can be converted into 1 of 5 reading grade levels (Table 2). |
DISCUSSION |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONLIMITATIONSCONCLUSIONSREFERENCES |
|---|
Among adults, low levels of literacy have been repeatedly linked to lower socioeconomic status and poorer health outcomes.5–11 Teens who do not master reading will eventually transition into adulthood without the prerequisite literacy skills needed to obtain, process, and understand basic information and services needed to make appropriate health decisions.8 The endemic nature of this problem is apparent: High school student reading scores have not improved over the last 3 decades.2–4,12 Today, two thirds of students entering 9th grade lack proficient literacy skills to keep up with an increasingly complex high school curriculum.2–4,12 Literacy screening in health care settings linked with effective intervention strategies in clinics and schools may improve the life trajectory of teens reading below their current grade level.
The REALM-Teen provides researchers the ability to examine the impact of lower-than-expected literacy on an adolescent’s health and health behavior and to determine to what extent below-grade literacy performance contributes to the initiation of health risk behaviors. Furthermore, such a test could also help researchers and clinicians assess the appropriateness of written materials and health surveys, alerting them that age-appropriate oral and written patient education and communication may need to be simplified. In addition, the REALM-Teen may assist clinical assessments of school problems and help direct the youth and families toward more specific resources aimed at improving basic academic skills and school outcomes. As a result, academic failure and future literacy problems may be prevented.
Before deciding to screen adolescents for below-grade level literacy, health professionals need to consider where patients will be tested, who will do the testing, and how they will be trained. If testing is conducted for clinical purposes, providers need to consider how test results will be used and documented.20 For some adolescents, particularly those with low literacy, test-taking may be an unpleasant experience in school: Being given a literacy test in a health care setting, no matter how it is presented, can be stressful. Previous studies in adult medicine found patients with low literacy are often ashamed and try to hide their problem.20,29 Clinicians and research assistants must be sensitive to these possibilities in screening for low literacy in adolescents.
|
LIMITATIONS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONLIMITATIONSCONCLUSIONSREFERENCES |
|---|
|
CONCLUSIONS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONLIMITATIONSCONCLUSIONSREFERENCES |
|---|
|
ACKNOWLEDGMENTS |
|---|
|
FOOTNOTES |
|---|
Address correspondence Terry C. Davis, PhD, Department of Pediatrics, Louisiana State University Health Sciences Center, Shreveport, PO Box 33932, Shreveport, LA 71130
The authors have indicated they have no financial relationships relevant to this article to disclose.
|
REFERENCES |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONLIMITATIONSCONCLUSIONSREFERENCES |
|---|
This article has been cited by other articles:
 |
|
 |
|
  L K. Hironaka and M. K Paasche-Orlow The implications of health literacy on patient-provider communication Arch. Dis. Child., May 1, 2008; 93(5): 428 - 432. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. A. Manganello Health literacy and adolescents: a framework and agenda for future research Health Educ. Res., November 17, 2007; (2007) cym069v1. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
||
|
||
Home | My Pediatrics | Journal Information | Current Issue | Past Issues | Subscriptions & Services | Contact Us Click here for faster international access © 2006 American Academy of Pediatrics. All rights reserved. |
||
|
||
