Readability, Suitability, and Characteristics of Asthma Action Plans: Examination of Factors That May Impair Understanding
OBJECTIVE: Recognition of the complexity of asthma management has led to the development of asthma treatment guidelines that include the recommendation that all pediatric asthma patients receive a written asthma action plan. We assessed the readability, suitability, and characteristics of asthma action plans, elements that contribute to the effectiveness of action plan use, particularly for those with limited literacy.
METHODS: This was a descriptive study of 30 asthma action plans (27 state Department of Health (DOH)–endorsed, 3 national action plans endorsed by 6 states). Outcome measures: (1) readability (as assessed by Flesch Reading Ease, Flesch-Kincaid, Gunning Fog, Simple Measure of Gobbledygook, Forcast), (2) suitability (Suitability Assessment of Materials [SAM], adequate: ≥0.4; unsuitable: <0.4), (3) action plan characteristics (peak flow vs symptom-based, symptoms, recommended actions).
RESULTS: Mean (SD) overall readability grade level was 7.2 (1.1) (range = 5.7–9.8); 70.0% were above a sixth-grade level. Mean (SD) suitability score was 0.74 (0.14). Overall, all action plans were found to be adequate, although 40.0% had an unsuitable score in at least 1 factor. The highest percent of unsuitable scores were found in the categories of layout/typography (30.0%), learning stimulation/motivation (26.7%), and graphics (13.3%). There were no statistically significant differences between the average grade level or SAM score of state DOH developed action plans and those from or adapted from national organizations. Plans varied with respect to terms used, symptoms included, and recommended actions.
CONCLUSIONS: Specific improvements in asthma action plans could maximize patient and parent understanding of appropriate asthma management and could particularly benefit individuals with limited literacy skills.
- disease management
- health literacy
- patient-doctor communication
- patient education
- practice guidelines
- DHHS —
- Department of Health and Human Services
- DOH —
- Department of Health
- F-K —
- FOG —
- Gunning Fog
- FRE —
- Flesch Reading Ease
- SAM —
- Suitability Assessment of Materials
- SMOG —
- Simple Measure of Gobbledygook
What’s Known on This Subject:
National asthma treatment guidelines include the recommendation that all asthma patients receive a written asthma action plan. No previous study has sought to examine the readability, suitability, and content of asthma action plans within a nationally representative sample.
What This Study Adds:
Although variability was found across written asthma action plans, and improvements in readability, suitability, and content are needed, there were also many common elements that would support a move to a single universal standard action plan.
Asthma is one of the most common childhood chronic diseases,1 affecting nearly 10% of US children.1–3 Recognition of the complexity of asthma management has led to the development of national and international asthma treatment guidelines which include the recommendation that all adult and pediatric asthma patients receive a written asthma action plan.4–6 Although use of action plans has been associated with a reduction in asthma-related hospitalizations and emergency department visits,7–12 there are concerns about their effectiveness when used for individuals with low health literacy, who represent more than one-third of US adults.13 Individuals with low health literacy are disproportionately from low socioeconomic status backgrounds and racial/ethnic minority groups, the same populations at greatest risk for the development of asthma and worse asthma-related morbidity.1,3,14–16
Readability, or grade level, and suitability, which comprise additional aspects of content, literacy demand, graphics, layout/typography, learning stimulation, and cultural appropriateness, contribute to the effectiveness of written materials.17–20 To date, although readability has been assessed for other types of patient education materials, there has been limited analysis of asthma action plans.21,22 One study performed a decade ago found that national asthma action plans had an eighth-grade level on average, higher than state-developed action plans,21 despite the recommendation that patient education materials for the general population be written at the sixth-grade level or lower.17,23,24 No previous study has systematically examined the suitability of asthma action plans or documented levels of consistency in content and format across plans.
Optimization of readability and suitability within the context of asthma action plans requires close examination of characteristics across a range of plans, because no single standard plan exists.21 In this study, we therefore sought to examine systematically the readability, suitability, and characteristics of asthma action plans used across the United States to better understand standard practices and to inform recommendations to maximize their effectiveness. Such an evaluation is consistent with the goals of national initiatives related to health literacy, including the US Department of Health and Human Services (DHHS) National Action Plan to Improve Health Literacy25,26 and meaningful use guidelines,27 which support the development and dissemination of patient care information that is understandable and actionable.
Identification of the Sample of Written Asthma Action Plans
Written asthma action plans endorsed by each state’s Department of Health (DOH) were independently sought by 2 investigators (DS, CL). Inclusion criteria were as follows: (1) English language and (2) endorsement by state DOH. Availability of a state DOH-endorsed plan was first assessed by using the Internet, with phone calls made to each state DOH when an online state DOH-endorsed plan could not be found. In the few cases of disagreement between the 2 investigators, 2 additional investigators (SY, RG) reviewed the cases and came to a consensus regarding the final set of action plans. Thirty action plans were ultimately identified for inclusion in analyses: 27 state DOH-developed and 3 national action plans (American Academy of Allergy, Asthma and Immunology; American Lung Association; National Institutes of Health/National Heart, Lung and Blood Institute). Six state DOHs endorsed 1 of 3 national action plans. Seventeen states provided no DOH-specified or national action plan. There is no existing mandate for the provision of a DOH-endorsed asthma action plan, and the rationale for why a state did not have a DOH-endorsed plan was not assessed.
Asthma Action Plan Assessments.
Three types of assessments were performed to examine (1) readability, (2) suitability, and (3) action plan features.
Five readability formulas were used to determine the grade level of each plan (Readability Plus software, Micro Power & Light Co, Dallas, TX): Flesch Reading Ease (FRE), Flesch-Kincaid (F-K), Gunning Fog (FOG), Simple Measure of Gobbledygook (SMOG), and Forcast.21,28–32 A composite score was calculated, in which the average reading level across the 5 formulas was determined to balance the strengths and weaknesses of each formula and increase reliability. On the basis of the recommendation that patient education materials for the general population be written at a sixth-grade level or lower,17,23 including state-level standards that exist33,34 (eg, Medicaid forms, in which the majority of states recommend a sixth-grade level or lower), each plan was also categorized on the basis of whether the composite reading level was sixth grade level or lower.
The FRE, F-K, FOG, and SMOG each use sentence length and number of syllables per word to assess difficulty, weighting these in different ways.28–31 The F-K is often thought to underestimate reading level,35 because it is based on a requirement of 50% comprehension (ie, 50% of individuals at the calculated grade level can comprehend the material), whereas the SMOG uses a cutoff of 100%.36 The Forcast is based on the number of monosyllabic words and is considered to be particularly useful for nonprose documents.32 Because the FRE provides grade-level ranges above the eighth grade, we used the mean grade within the range (ie, 8.5 for eighth- to ninth-grade range) to represent the FRE grade level included in the calculated average grade level.
Before using the readability software, each document was converted to text and prepared systematically. Only text that a parent was expected to read and understand was included in the text to be analyzed. The following was removed: organization name/slogan, volume/issue, page numbers, copyright, adapted from information, funding. Nonsentences, such as section headers, labels, and table headings were also removed. Bulleted phrases describing asthma symptoms were converted into sentences, because symptom recognition is necessary for proper asthma management.
We used an adapted version of the Suitability Assessment of Materials (SAM) instrument,17,37 a common tool used to analyze documents systematically based on elements that may affect readability and ease of use (content, literacy demand, graphics, layout/typography, learning simulation, cultural appropriateness).38 A modified SAM scoring system (adequate = 1 vs unsuitable = 0) was used to grade each individual factor.39 For this analysis, cultural appropriateness was also not assessed because action plans were created for general audiences rather than a specific target population. Total SAM score was calculated by dividing the sum of the scores for each factor by the total number of applicable items; a ≥40% rating was considered to be adequate.
Action Plan Features
Action plan–specific features were examined to determine the extent to which similarities and differences exist across plans. Determination of the features to include for analysis was guided by an examination of a preliminary sample of action plans, as well as previous literature on aspects of action plans considered to be important from medical/health literacy perspectives.4,40,41 The final list was created with input from 2 clinicians/health literacy experts (HSY, BPD), a cognitive scientist/health literacy expert (MSW), and 2 clinicians with expertise in asthma care (RG, ST).
Each action plan was assessed to determine whether it (1) was symptom- or peak-flow based, (2) was intended to be filled in by hand or electronically, (3) included a severity classification, (4) identified patient-specific triggers, (5) requested physician contact information, and (6) included a place for a physician signature. The format of medication charts was also examined (eg, fill in, check boxes).
Use of color coding was assessed to determine whether the standard traffic-light motif was supported with the use of green, yellow, and red8 and whether additional colors were used. The use of pictorial illustrations to distinguish between the 3 zones of the typical action plan was also assessed and image type determined (ie, realistic/lifelike drawings, smiley face icons, traffic light icons).
Wording used to refer to everyday (or controller) and rescue medications was examined. For each of the 3 zones, we also assessed symptoms listed and recommended actions.
Abstraction of Data
Readability was assessed by using software as previously described. With respect to suitability, 2 reviewers independently rated each of the 30 action plans after a 2-hour training session and calibration. Interrater reliability was high (κ = 0.71), based on agreement or disagreement for the scoring of each individual feature as being adequate, unsuitable, or not applicable. A third reviewer provided an independent review in cases of discordant ratings, with majority rule implemented in all cases of discordance.
Similarly, for the abstraction of action plan features, 2 reviewers independently examined each plan. Interrater reliability was high (κ = 0.87), based on agreement or disagreement for scoring each individual characteristic as being present or absent. As with the suitability assessment, a third rater reviewed any instances in which there was not agreement between raters, with final results reflecting agreement of 2 of the 3 investigators.
Descriptive analyses were performed for each variable of interest. χ2 or Fisher’s exact tests were performed to examine associations between readability and suitability and action plan origin (being from or adapted from a national organization, or developed by the state). SPSS version 18.0 statistical software (SPSS Inc, Chicago, IL) was used for data analyses. A 2-tailed P value ≤ .05 was considered to be the threshold for statistical significance.
Thirty action plans were included in analyses, representing 33 states (27 state DOH-endorsed plans, 6 adapted from national sources) and 3 national plans (endorsed by 6 states).
Mean (SD) overall grade level was 7.2 (1.1; range = 5.7–9.8; Table 1). Seventy percent of action plans were written at higher than a sixth-grade level. There was no statistically significant difference by origin (7.2 [state] vs 7.1 [national], P = .7).
Mean (SD) suitability score was 0.74 (0.14; Table 2). Overall, all action plans in our sample were found to be adequate. Forty percent of action plans had a score of unsuitable in ≥1 factor. There was no statistically significant difference between the overall suitability score by origin (0.73 [state] vs 0.77 [national], P = .4). Issues included lack of explicitly stated purpose (73.3%), limited use of a problem/question-based format to encourage interactive learning (63.3%), and suboptimal subheadings (50.0%; Fig 1).
Action Plan Features
The majority of action plans were both symptom and peak-flow based, with only 1 being exclusively symptom-based (Table 3). More than 90% were intended to be filled in by hand (93.3%). The majority of medication charts were fill-in (86.7%); 43.3% had check-box options. The mean (SD) number of check box items within zones was 8.6 (17.6); range = 0 to 92. A green, yellow, red color motif was used in almost all plans (93.3%). More than 75% included illustrations.
Eight terms were used for controller medications, most commonly “green zone,” “control”/“controller,” “daily” medication. Six terms were used for rescue medications, with the most common being “quick relief”/“quick reliever” medication. Symptoms included in each zone were variable, with respiratory signs/symptoms (eg, cough) and functional abilities (eg, ability to play/exercise) most common. Almost all plans had overlap in the top few recommendations associated with the individual zones, but variability existed for topics such as spacer use and oral steroids.
To our knowledge, this study is the first to assess systematically both the readability and suitability of written asthma action plans, within the context of the range of action plans used across the United States. Of the 30 state DOH-endorsed action plans examined, we found that a majority was written at higher than a sixth-grade level. In addition, we found that although overall suitability was adequate, layout, learning stimulation, and graphics were particular aspects of action plans that could be improved. Finally, we found variability in the format and content of action plans, including symptoms and recommended actions.
Of the plans studied, fewer than 1 in 3 were written at a sixth-grade level or lower, as recommended for the general population17,23 (eg, consistent with state standards for Medicaid forms33). Although federal agencies and professional/scientific organizations have recognized the need to improve the readability of patient health-related documents (eg, DHHS,25,26 Institute of Medicine,42 Centers for Disease Control,43 American Medical Association44), no specific federal cutoff for readability currently exists, and few organizations have issued readability standards. None of the action plans in our sample had an overall readability lower than fifth grade. A third- to fifth-grade level is recommended for low-literate populations.17,23 These findings are consistent with a previous study of asthma action plans from a decade ago21; however, whereas that study found that local plans had an average grade below that of national plans, we did not find this difference.
Although all of the action plans we assessed were considered to have adequate suitability overall, 40% had ≥1 “not suitable” factor. More than 70% were lacking the presence of an explicit purpose. We recognize, however, that action plan use is typically accompanied by provider counseling, during which document purpose may be more clearly highlighted. Layout/typography was one of the suitability categories with the lowest mean scores, with 1 in 3 documents considered to be unsuitable. Layout problems included poor use of visual cuing (eg, boxes, arrows) to direct attention to key content, and lack of adequate white space. One contributor to lack of white space was the number of check-box options. We found that action plans had an average of 9, but up to 57 check-box options in a single zone, listing numerous possible medications and doses. Typographical problems included small font size (<12 point), use of all capital letters, and suboptimal use of cues (eg, bolding, size, color). Half of the plans examined were found to have too many concepts present below headings. To maximize understanding, it is recommended that concepts be separated into “chunks,” with ≤7 independent items.17 For those with low literacy, 3 to 5 items are preferred.17
Learning stimulation and motivation were additional elements in need of improvement; complex topics were often not subdivided to improve reader likelihood of being able to achieve small successes in understanding, which is thought to enhance sense of self-efficacy and motivation to learn.17
With respect to literacy demand, use of common words, and better use of headers or topic captions, would enhance comprehension. We found that 1 in 4 documents were considered unsuitable with respect to not using common words and suboptimal use of “road signs” to help readers know what topic is to be covered next. Wording such as “daily preventive anti-inflammatory medicine,” “exacerbation,” and “modifications” add unnecessary complexity.
Inclusion of graphics is a key strategy to enhance patient understanding of complex concepts.23,45 Although graphics were considered to be adequate overall in our sample, more than a third of the plans we studied would benefit from improvements in how lists, tables, and graphics are explained.
A number of studies have examined general features of asthma action plans, noting that these plans include concise, detailed, individualized recommendations,46,47 as well as information on daily disease management and worsening symptoms.48 Our study has similarly identified these general features but provides additional information on specific characteristics. In our sample, almost all plans were both symptom and peak flow-based, despite a Cochrane systematic review noting that action plans that are symptom-based are preferred by families over peak flow-based plans and that children who receive symptom-based plans have fewer asthma exacerbations.7 Removal of peak flow information may be one strategy to simplify action plans and make them more usable.
We also found that action plans differed substantially in the wording used for controller and rescue medications, asthma symptoms included, and recommended actions. Simplifying wording and increasing consistency across plans could improve action plan effectiveness and allow for a standardized provider approach. Addressing these issues may help change physician perception that existing action plans are too difficult to understand and could increase the likelihood that plans are used21,49; studies have shown that fewer than half of physicians routinely give out these written plans.50–52
Although variability was found across asthma action plans and improvements in readability, suitability, and content are needed, there were also many common elements found across action plans. Color and certain thematic framing (ie, green/yellow/red zones) were nearly consistent across plans, supporting the proposition to move toward a single universal standard written asthma action plan. Development of such a universal plan should also incorporate principles of existing evidence-based low-literacy patient action plans (eg, diabetes,53 congestive heart failure54).
We recognize that our study focuses on an examination of asthma action plan features and does not include a test of actual patient/parent understanding. Patient-focused studies are clearly needed to examine which particular features will enhance understanding, retention, and adherence; our study findings can be used to inform the design of such studies. For example, patient studies could help in the selection of which symptoms would be most meaningful to include, as well as which and how many recommendations should be given, such that the information provided maximizes patient ability to act on instructions while considering cognitive load factors.
There are limitations to our study. Although we focused on examining current standard practices by assessing 30 DOH-endorsed asthma action plans, there exist numerous other action plans developed by clinics, hospital systems, and national/international organizations, including those linked to electronic medical records. Because many of these plans are proprietary and not publicly available, we did not include these in our analyses; future examination of these plans would be beneficial. We also limited the scope of our sample to English-language plans; we recognize that future study of action plans in other languages is needed. We note that in preparing documents for assessment using the readability software, we did follow standard practices in removing headers and table headings, but we did not follow standard practices in removing bulleted information, because we considered bulleted symptom information to be essential for patient understanding. This may have led to an underestimation of the reading level. Finally, we only assessed the written structure of asthma action plans, independent of verbal counseling, which limits the conclusions we can draw. Nevertheless, this is the first study, to our knowledge, to examine systematically the readability, suitability, and features of written asthma action plans across a national sample.
Improvements in the readability, suitability, and features of asthma action plans are needed to maximize patient and parent understanding of appropriate asthma management. Such improvements align with national initiatives to address health literacy issues, including the DHHS National Action Plan to Improve Health Literacy26 and Healthy People2020.49 Study findings should be used to inform the development of guidelines to enhance the effectiveness of existing asthma action plans, as well as contribute to the development of a standardized approach to care, with the goal of improving pediatric asthma management and health outcomes.
We thank Emily P. Ellison, MS, Nicole M. Fortuna, BA, Jennifer P. King, MPH, Jennifer L. Lenahan, BS, Allison L. Russell, BA, and Eleanor Small, BA, for their assistance in suitability assessments. We thank Perry Nagin, BA for her assistance in reviewing this article.
- Accepted August 14, 2012.
- Address correspondence to: H. Shonna Yin, MD, Department of Pediatrics, New York University School of Medicine, 550 First Ave, NBV8S4-11, New York, NY, 10016. E-mail:
Dr Yin conceptualized and designed the study, led the analysis and interpretation of data, drafted the manuscript, and approved the final manuscript as submitted; Dr Gupta conceptualized and designed the study, assisted in the analysis and interpretation of data, provided critical revision of the manuscript for important intellectual content, and approved the final manuscript as submitted; Dr Tomopoulos participated in the concept and design of the study, assisted in the analysis and interpretation of data, provided critical revision of the manuscript for important intellectual content, and approved the final manuscript as submitted; Dr Wolf participated in the concept and design of the study, assisted in the analysis and interpretation of data, provided critical revision of the manuscript for important intellectual content, and approved the final manuscript as submitted; Dr Mendelsohn participated in the concept and design of the study, assisted in the analysis and interpretation of data, provided critical revision of the manuscript for important intellectual content, and approved the final manuscript as submitted; Ms Antler participated in the concept and design of the study, participated in the acquisition of data, assisted in the analysis and interpretation of data, assisted in the drafting of the manuscript, and approved the final manuscript as submitted; Ms Sanchez participated in the concept and design of the study, participated in the acquisition of data, assisted in the analysis and interpretation of data, assisted in the drafting of the manuscript, and approved the final manuscript as submitted; Ms Lau participated in the concept and design of the study, participated in the acquisition of data, assisted in the analysis and interpretation of data, assisted in the revision of the manuscript, and approved the final manuscript as submitted; and Dr Dreyer participated in the concept and design of the study, assisted in the analysis and interpretation of data, provided critical revision of the manuscript for important intellectual content, and approved the final manuscript as submitted.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Drs Yin and Gupta are funded by career development grants through the Robert Wood Johnson Physician Faculty Scholars Program. The Robert Wood Johnson Foundation had no role in the design and conduct of the study, in the collection, management, analysis, or interpretation of the data, or in the preparation, review, or approval of the manuscript. Funding for this study was also provided by the KiDS of NYU Foundation.
- ↵Bloom B, Cohen RA, Freeman G. Summary health statistics for U.S. children: National Health Interview Survey, 2007. Vital and Health Statistics. Series 10, Data From the National Health Survey. 2009(239):1–80
- Akinbami LJ,
- Moorman JE,
- Garbe PL,
- Sondik EJ
- Lara M,
- Akinbami L,
- Flores G,
- Morgenstern H
- ↵National Asthma Education and Prevention Program Expert Panel Report 3. Guidelines for the Diagnosis and Management of Asthma (NIH Publication Number 08-5864). Bethesda, MD: National Heart, Lung, and Blood Institute; 2007
- National Asthma Council Australia
- ↵British Thoracic Society, Scottish Intercollegiate Guidelines Network. British Guideline on the Management of Asthma: A national clinical guideline. Edinburgh, Scotland: NHS Quality Improvement Scotland; 2009
- Lieu TA,
- Quesenberry CP Jr,
- Capra AM,
- Sorel ME,
- Martin KE,
- Mendoza GR
- Kutner MA,
- Greenberg E,
- Jin Y,
- Paulsen C
- Akinbami LJ,
- Moorman JE,
- Liu X
- Moorman JE. Current Asthma Prevalence—United States, 2006–2008. Atlanta, GA: Centers for Disease Control and Prevention; 2011
- Doak CC,
- Doak LG,
- Root JH
- ↵Weiss BD. Health Literacy and Patient Safety: Help Patients Understand. Chicago, IL: American Medical Association Foundation; 2007
- ↵US Department of Health and Human Services. Healthy People 2020. 2012. Available at: www.healthypeople.gov/2020/default.aspx. Accessed February 13, 2012
- ↵US Department of Health and Human Services. National Action Plan to Improve Health Literacy. Washington, DC: US Department of Health and Human Services; 2010
- Kincaid JP,
- Fishburne RP,
- Rogers RL,
- Chissom BS
- Gunning R
- ↵Caylor JS, Stitch TG, Fox LC, Ford JP. Methodologies for Determining Reading Requirements of Military Occupational Specialties. Alexandria, VA: Human Resources Research Organization; 1973
- Health Literacy Innovations
- Friedman DB,
- Hoffman-Goetz L
- ↵Finnie RK, Felder TM, Linder SK, Mullen PD. Beyond reading level: a systematic review of the suitability of cancer education print and Web-based materials. J Cancer Educ. 2010;25(4):497–505
- Chapman KR,
- Boulet LP,
- Rea RM,
- Franssen E
- ↵Committee on Health Literacy. Health Literacy: A Prescription to End Confusion. Washington, DC: The National Academies Press; 2004
- ↵Center for Disease Control and Prevention. Health Literacy for Public Health Professionals. Atlanta, GA: Center for Disease Control and Prevention; 2009
- ↵Weiss BD. Health Literacy and Patient Safety: Help Patients Understand. Chicago, IL: American Medical Association Foundation and American Medical Association; 2007
- Gibson PG,
- Powell H
- ↵Beauchesne MF, Levert V, El Tawil M, Labrecque M, Blais L. Action plans in asthma. Can Respir J 2006;13(6):306–310
- Sulaiman N, Aroni R, Thien F, et al. Written Asthma Action Plans (WAAPs) in Melbourne general practices: a sequential mixed methods study. Prim Care RespirJ. 2011;20(2):161–169
- Gupta RS,
- Weiss KB
- Wolff K,
- Cavanaugh K,
- Malone R,
- et al
- Copyright © 2013 by the American Academy of Pediatrics