Literacy is defined as “the ability to read and use written information and to write appropriately in a range of contexts.”1 Health literacy is a more specialized aspect of literacy and is “the capacity to obtain, process and understand basic health information and services needed to make appropriate health decisions.”2
The National Assessment of Adult Literacy (NAAL) in the United States (US) found that 5% of 19,000 participants were nonliterate and 43% of participants had either basic or below basic English literacy.3,4 The Adult Literacy and Life Skills Survey found that up to 60% of Australians ages 15 to 74 years achieved scores below Level 3 for the health literacy domain, where Level 3 is the “minimum required for individuals to meet the complex demands of everyday life and work in the emerging knowledge-based economy.”5 Similar results were reported from Canada.6
Low literacy is associated with poorer health outcomes, for example, increased asthma morbidity,7,8 poorer diabetic control,9,10 less stable anticoagulation,11 and increased mortality.12,13 Those accessing health care require adequate health literacy and numeracy (the ability to use and understand numbers in daily life)10 skills to understand written instructions regarding medication, appointments with health care professionals and to calculate correct medication doses.14,15 Patient self-reported reading skills correlated poorly with actual reading scores.16 Limited health literacy is associated with medication noncompliance and misunderstanding of instructions on medication prescription labels.17–19 Patients with poorer health literacy were less likely to keep appointments with health care professionals, participate in health screening programs or seek medical assistance.20 Poor health literacy has also been linked to less health knowledge and fewer self-care behaviors.21,22
Limited health literacy affects use of health care resources and expenditure.23–28 Poor health literacy also raises questions regarding informed consent, the right to quality care, and antidiscrimination.29 Limited health literacy can be a significant source of shame and embarrassment.30,31
Large rural populations exist in countries such as the United States, Canada, and Australia. Thirty percent of people in Australia (population 22 million) reside outside a capital city.32 Rural residents have poorer health outcomes for many conditions, such as coronary heart disease,33 colorectal cancer,34,35 stroke,36 and HIV.37 An important factor affecting health outcomes may be health literacy, yet there are limited data regarding health literacy in rural residents. The largest published study addressing this examined 3850 rural residents (population centre <50,000 people) and 14,260 urban dwellers from the NAAL database.38 Rural residents performed worse in all domains of literacy and health literacy. However, there was no difference in health literacy between the 2 groups once age, sex, ethnicity, education, and income were corrected for.38
Ten percent of patients with rheumatoid arthritis attending a community-based Australian Rheumatology practice had inadequate or marginal functional health literacy or a reading age at or below the US high school grade equivalent of seventh–eighth grade.39 However, as that practice was located in an affluent suburb of a major capital city, these findings may not be generalizable to other demographic areas. Up to 24% of rheumatology patients at an academic US medical centre had a reading level of eighth grade or less.40 One in 6 rheumatology patients at a Scottish hospital were illiterate and struggled to understand education materials and prescription labels.23 These findings are concerning, as rheumatologists often use medications such as methotrexate (MTX) or biologic therapies with severe adverse effects if taken incorrectly.
Given the lack of data regarding health literacy in rural patients we sought to determine the level of health literacy (word recognition, comprehension, and numeracy) in outpatients attending a rural rheumatology practice; compare the health literacy of these patients to those attending an urban rheumatology practice; and determine whether patients could follow written dosing instructions for common medications used in rheumatology practice.
PATIENTS AND METHODS
This was a cross-sectional study involving 2 community-based rheumatology practices, a rural one in Coffs Harbour and the other in Kogarah, Sydney, both in New South Wales (NSW), Australia.
Setting and Study Participants
Coffs Harbour (population 70,990 people, land area 117,374 ha, population density 60.5 persons/km2)41 is located halfway between the major cities of Sydney and Brisbane but provides specialist medical services to another 50,000 people in the surrounding area. Summary characteristics of Coffs Harbor residents are as follows: median age 42.5 years, 17.9% born overseas (mainly in north-western Europe), 5.7% spoke a language other than English at home, main source of employment was health care and social assistance, 55% had postschool qualifications, and the average annual personal income was AUD 40,300. 41 Rheumatology services are provided by 2 resident rheumatologists (HB and PKKW) at the private Mid-North Coast Arthritis Clinic (MNCAC).
Kogarah (population 59,782 people, land area 1555 ha, population density 3845 persons/km2)42 is a suburb located approximately 10 km southwest of Sydney CBD. Summary characteristics of residents are as follows: median age 37.6 years, 45.8% born overseas (mainly in Asia), 49.8% spoke a language other than English at home, main source of employment was health care and social assistance, 61% had postschool qualifications, and the average annual personal income was AUD 53,357. The Combined Rheumatology Practice (CRP) is a private group practice (participating rheumatologists, FJ and PB) located in Kogarah who provides rheumatology services to the surrounding area.
Every fifth patient attending either the rural (MNCAC, n = 161) or urban (CRP, n = 130) practice was contacted by a combination of mail and telephone during a 5-month period using a standard “proforma.” Patients were offered study participation at a time of their convenience, usually before or after the next scheduled appointment with a rheumatologist. As knowledge of the purpose of this study may have resulted in those with poor health literacy declining participation, patients were blinded to the exact study aim. Instead, they were told the aim was to assess what they understood from reading material used by the practice. This would assist with design of better patient educational brochures.
Exclusion criteria were as follows: age <18 years, low vision preventing reliable reading of assessment tools, poor hearing limiting ability to reliably follow verbal instructions, inability to speak English, or severe ill-health.
All structured interviews were performed under no significant time constraints in a quiet well-lit room by 1 observer (LC). Study participants wore hearing and visual aids if these were usually worn. Corrected vision was tested using a nonalphabet Snellen chart. Hearing was assessed by asking the participant whether they could hear speech at normal conversational levels. The following patient demographics were recorded: age, sex, ethnicity, marital status, occupation, country of birth, primary language spoken at home, Aboriginal or Torres Strait Islander heritage, years completed at school, further education, and Internet use (“Do you use the Internet at least once per week?”). Participant occupations were classified into 8 major categories according to the Australia and New Zealand Standard Classification of Occupations (ANSCO).43
Word recognition was assessed using the Rapid Estimate of Adult Literacy in Medicine (REALM), a standardized test widely used as a health literacy screening tool (Table 1).44–46 This tool (maximum possible score 66) requires <5 minutes to administer and assesses recognition of common medical words. For this study, American-English spellings were changed to Australian-English spellings, for example, “behavior” to “behaviour.” Testing involved presenting participants with a laminated sheet containing 3 lists of 22 words each, arranged in ascending order of number of syllables and pronunciation difficulty.44 Patients were asked to read aloud as many words as possible beginning with the first word in the first column. If they were unable to pronounce several consecutive words, they were asked to scan down the list and pronounce as many of the remaining words as possible. The scoring standard was dictionary pronunciation.44 The final score was used to derive US high school grade range estimates (equivalent to Australian school grades 1–12)47 as an approximation of health literacy.
Functional health literacy and numeracy was assessed using the Test of Functional Health Literacy in Adults (TOFHLA).48 This is a well-validated instrument developed to assess patient functional health literacy using material from health care settings such as prescription labels and appointment slips.49 The final score (range 0–100) allows allocation into categories of functional health literacy (Table 1).48 As the TOFHLA was designed for a US population, minor modifications were made to some words to improve cultural appropriateness for an Australian setting.50,51
Although a critical appraisal of 19 health literacy assessment instruments found the REALM and TOFHLA had the strongest psychometric properties, these do not contain specific items relevant to routine rheumatological practice.51 Hence, to assess practical health literacy and numeracy skills, participants were asked to follow standard prescribing instructions for 5 commonly used rheumatology medications (Table 2). The 5 questions were drafted by 2 rheumatologists (PKKW, HB), a rehabilitation physician (KC), and a rheumatology nurse (DF) and reviewed by a literacy expert (JJ). Clarity of language was assessed using a focus group of 10 randomly selected patients from the rural practice. These instructions were on average, “readable” for those with the equivalent of the upper level of a sixth-grade education (6.954) by Flesch-Kincaid Reading Ease analysis (63.7/100).52 However, as these were not a validated health literacy assessment tool this instrument will be referred to as the Rheumatology Literacy Guide (RLG).
Descriptive summary statistics (means and medians, as appropriate) were used to summarize participant demographic characteristics. Student t test was used to compare means of normally distributed parameters. As many of the variables were skewed, the Mann–Whitney U test was used to compare medians of the 2 groups. Frequency data were analyzed using χ2 testing. For all statistical tests, P < 0.05 was considered significant. Spearman correlation coefficients were used to examine associations between nonnormally distributed variables. Data analysis was undertaken using IBM SPSS Statistics version 19 (Armonk, NY).
Approval was obtained from the North Coast Area Health Service Human Research Ethics Committee (HREC) for the Coffs Harbour site and the University of New South Wales HREC for the Sydney site.
There were 223 participants in the study, n = 124 from the rural practice (MNCAC, Coffs Harbour) and n = 99 from the urban practice (CRP, Kogarah). One patient at each practice was excluded because of poor vision and 10 patients at the urban practice were excluded, because their primary language was not English.
Characteristics of study participants are shown in Table 3. The mean age of participants at both sites was 60 years, while approximately two-thirds of participants were female. A higher proportion of participants from the urban practice was born overseas (P = 0.007) and spoke a primary language other than English at home (P = 0.006). Participants from the rural practice (MNCAC) had lower levels of education, were more likely to be unemployed, and, if employed, were less likely to be managers or professionals.
Thirty-six out of 160 patients (22.5%) approached at the rural practice (MNCAC) declined participation, compared with 20 out of 119 participants (16.8%) at the urban site (CRP, P > 0.05). Overall, 56 out of 279 (20%) patients approached declined study participation. There was no difference between the 2 practices in mean age, sex, or proportion born overseas in those declining participation (data not shown). There was also no difference in mean age or sex between those who declined participation compared with study participants (data not shown). Reasons for declining participation are outlined in Table 4. Three patients at the rural site (MNCAC) admitted they had poor literacy and declined study participation because of embarrassment. These were excluded from the analysis.
Health Literacy Scores by Practice
Results of health literacy assessment (REALM, TOFHLA, and RLG) are shown in Table 5. The REALM scores indicated more participants from the rural practice compared with the urban site had a Grade 8 or lower word recognition level (23/124 [19%] vs 10/97 [10.3%], respectively). However this difference was not statistically significant (P
= 0.09 by χ2 analysis).
The TOFHLA scores indicated that 12/124 (9.7%) of rural patients had inadequate or marginal functional health literacy compared with 6/97 (6.2%) of urban participants (Table 5). This difference was not statistically significant (P = 0.35).
Approximately one-third of participants answered Question 2 (ibuprofen) and up to one-quarter of participants answered Question 4 (MTX) incorrectly (Table 5). Questions dealing with tramadol, prednisone, and alendronate were answered correctly by most participants.
As rural and urban participants achieved similar scores on all 3 health literacy assessment tools, data were pooled to provide a more meaningful assessment of health literacy in rheumatology outpatients.
Correlations Between Health Literacy Scores and Demographic Variables
Spearman correlation coefficients examining the relationship between relevant study variables are shown in Table 6. There was a moderately strong positive correlation (r = 0.39, P < 0.01) between REALM and TOFHLA scores. The RLG scores correlated weakly (r = 0.27, P < 0.01) with REALM scores and moderately strongly with TOFHLA scores (r = 0.43, P < 0.01). There was a weak negative correlation between TOFHLA scores and increasing age (r = −0.32, P < 0.01) but a moderately strong positive correlation with school years completed (r = 0.42, P < 0.01) and Internet use (r = 0.45, P < 0.01). Scores on the RLG correlated weakly with school years completed (r = 0.34, P < 0.01) and the Internet use (r = 0.39, P < 0.01). REALM scores also correlated weakly with the Internet use (r = 0.32, P < 0.01).
Table 7 shows median health literacy scores from the REALM, TOFHLA, and RLG stratified by demographic variables. Females scored better on the TOFHLA (P = 0.036) and REALM (P
= 0.009) compared with males. As expected, those whose primary language spoken at home was English, were university educated, or currently employed performed better on the REALM, TOFHLA, and RLG than those who spoke another primary language at home, had not attended university, or were currently unemployed. Internet users also performed better on all 3 measures than those who did not use the Internet.
Previous studies of health literacy in rheumatology patients have examined urban populations39 in tertiary referral centres.23,40,53 We extend these findings to show comparable levels of low health literacy in rural residents. There was no significant difference in word recognition, comprehension, and understanding of common rheumatology medication dosing instructions between rural and urban rheumatology patients. Despite a higher proportion of rural compared with urban participants having Grade 8 or lower word recognition ability (18.5% at MNCAC vs 10.3% at CRP, respectively) using the REALM and having marginal or inadequate functional health literacy using the TOFHLA (9.7% at MNCAC vs 6.2% at CRP, respectively), these differences were not statistically significant. This may have been because of Type II error (lack of power). However, our data suggest that clinicians should consider poor health literacy in their patients, regardless of urban or rural location. Importantly, we also found up to one-third of patients were unable to correctly follow written dosing instructions for commonly prescribed potent rheumatologic medications. These results should be generalizable to other rural and urban centers in Australia, and probably to other English-speaking countries.
Given the rural centre had higher unemployment (8.3% for Coffs Harbour vs 5.5% for Kogarah), lower mean annual income ($40,300 for Coffs Harbour vs $53,357 for Kogarah), and fewer managers/professionals (71% for Coffs Harbour vs 78% for Kogarah),41,42 it is surprising there was no significant difference in health literacy between rural and urban patients. Although the NAAL study found rural residents performed worse in all domains of literacy and health literacy, this difference disappeared once age, sex, race/ethnicity, education, and income were controlled for.38 Both urban and rural sites were private practices where patients were charged consultation fees. This may have resulted in patients at the rural site not being representative of a poorer rural population. However, as expected, rural participants had a lower level of school and higher education completion and were more likely to be unemployed than their urban counterparts (Table 3). The urban study sample had a higher proportion of overseas-born patients and more patients who spoke a non-English primary language at home. These factors may have counteracted the disadvantage associated with poorer education and higher unemployment status of the rural study sample.
The higher proportion of patients declining participation at the rural site (22.5% at MNCAC vs 16.8% at CRP, respectively) may have contributed to the similar REALM, TOFHLA, and RLG scores at the 2 sites. Three rural patients declined study participation because of embarrassment from poor literacy and were excluded from the analysis. None at CRP did so for this reason. Others declined participation with no reason offered, or with reasons such as “I’m feeling unwell” or “I don’t have enough time” (Table 4). Those at risk for poor literacy may decline study participation citing reasons other than embarrassment.30,31 This is not surprising as formal literacy assessment can be threatening, with a fear of decreased self-esteem and social acceptance on the part of the participant.16,54 Health practitioners should be alert to these issues because many patients are unwilling to admit that they have literacy problems.30,31 The proportion of patients with poor health literacy is therefore probably underreported.55
The limitations of the REALM and TOFHLA, with particular emphasis on validity, reliability, and feasibility have been well described.51 However, as these have the strongest psychometric properties of the currently used literacy assessment tools, they remain the 2 most widely used measures. One limitation of the TOFHLA is that while it allows classification of respondents into “inadequate,” “marginal,” or “adequate” health literacy, it does not provide functional definitions of what these categories mean in clinical practice. Even the REALM, which provides a school grade estimate of reading ability, does not outline which individuals may require low-literacy materials. Although the REALM purports to assess literacy, it really assesses reading and pronunciation.56 Neither the REALM nor the TOFHLA assesses writing ability. Although the REALM and TOFHLA have been shown to correlate highly (r = 0.81–0.84),48,49 in our hands the correlation was more modest (r = 0.39). A lower correlation between the REALM and TOFHLA in clinical practice has also been noted by others.39,50
Despite these limitations, our data suggest that regardless of geographic location, up to 15% of rheumatology patients would have difficulty reading and understanding most patient education materials. This may even be an underestimate as 20% (n = 56/279) of our pooled study sample declined study participation (Table 4). Some of those declining may have been at risk for poor literacy.
Although participants generally scored highly on both the REALM (median score 65 from a possible maximum score of 66) and TOFHLA (median score 95 from a possible maximum score of 100), up to one-third of patients could not correctly follow dosing instructions for ibuprofen or MTX (Table 5). This may have been because of poor numeracy skills. It is unlikely the length of the written instructions for ibuprofen or MTX was responsible, as the instructions for alendronate were longer and yet correctly understood by 95% of respondents. Our findings are concerning as these medications are commonly used in rheumatology practice, and if taken incorrectly can cause serious complications—even death. This suggests health literacy assessment tools such as the REALM and TOFHLA may not necessarily be predictive of a patient's ability to follow medication dosing instructions, possibly because of a “ceiling effect.”
This study identified several possible risk factors for poor health literacy: male sex, non-English primary language, lack of university education, lack of employment, and failure to use the Internet (Table 6). Although many of these have been previously identified,3–5,20 no single factor is a robust predictor of poor health literacy. However, all these factors are easily elicited during clinical assessment, and when taken together may assist in identifying those at risk for low health literacy. A novel finding of our study was that Internet use correlated with all 3 health literacy assessment tools. Although this requires further analysis in larger studies, poor information/technological literacy as exemplified by limited computer and Internet use has previously been associated with poor overall literacy.57 Despite the benefits and increasing use of eHealth applications for patient education, these may be of limited utility in those with poor health literacy.57
Improving health care professionals’ awareness of health literacy is important as patients are unlikely to disclose difficulty understanding medication instructions31 and also overestimate their reading ability.16 Helpful strategies to address poor health literacy include assessing baseline patient understanding of their condition before providing information, use of plain language rather than medical jargon, emphasizing <3 main points that are repeated several times during the consultation and use of the “teach back” technique, which involves asking patients to explain or demonstrate what they have been told.58,59 As most rheumatology patient education materials are written at readability levels above the recommended sixth-grade reading level,60 assessment of design and readability of such material is recommended with the use of pictures and videos instead of written text.31,61 Resources such as the Health Literacy Universal Precautions Toolkit are readily available to assist clinicians to reduce the complexity of medical care and ensure patients successfully navigate the health care system.62
Our results show that up to 15% of patients from either a rural or urban location have poor health literacy. It is concerning that up to one-third of patients in this study were unable to correctly follow written dosing instructions for commonly prescribed rheumatologic medications. This may not be well predicted by traditional health literacy assessment tools such as the REALM and TOFHLA. Although poor health literacy is a sensitive and challenging issue for patients and clinicians, it needs to be addressed. Risk factors for poor health literacy that can be easily elicited during a consultation may be male sex, overseas birth, a non-English primary spoken language at home, lack of university education, lack of current employment, and lack of Internet use.
1. Australian Government, Department of Education Employment and Training. The Australian Language and Literacy Policy. Canberra: Australian Government; 1991.
2. Simonds SK. Health education as social policy. Health Educ Monogr
3. Kutner M, Greenberg E, Jin Y, et al. Literacy in everyday Life: Results From the 2003 National Assessment of Adult Literacy; NCES 2007–480. Washington, DC: National Center for Education Statistics; U.S. Department of Education; 2007.
4. Cutilli CC, Bennett IM. Understanding the health literacy of America. Results of the National Assessment of Adult Literacy. Orthop Nurs
6. Human Resources and Skills Development Canada (HRDC) and Statistics Canada. Building on Our Competencies: Canadian Results of the International Adult Literacy and Skills Survey 2003. Ottawa: Human Resources and Skills Development Canada (HRDC) and Statistics Canada; 2005.
7. Adams RJ, Appleton SL, Hill CL, et al. Inadequate health literacy is associated with increased asthma morbidity in a population sample. J Allergy Clin Immunol
8. Rosas-Salazar C, Apter AJ, Canino G, et al. Health literacy and asthma. J Allergy Clin Immunol
9. Schillinger D, Grumbach K, Piette J, et al. Association of health literacy with diabetes outcomes. JAMA
10. Cavanaugh K, Huizinga MM, Wallston KA, et al. Association of numeracy and diabetes control. Ann Intern Med
11. Estrada CA, Martin-Hryniewica M, Peek BT, et al. Literacy and numeracy skills and anticoagulation control. Am J Med Sci
12. Baker DW, Wolf MS, Feinglass J, et al. Health literacy and mortality among elderly persons. Arch Intern Med
13. Sudore RL, Yaffe K, Satterfield S, et al. Limited literacy and mortality in the elderly: the health, aging, and body composition study. J Gen Intern Med
14. Miles S, Davis TC. Patients who can’t read. Implications for the health care system. JAMA
15. American Medical Association, Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs, Health literacy: report of the Council on Scientific Affairs. JAMA
16. Ferguson B, Lowman SG, DeWalt DA. Assessing literacy in clinical and community settings: the patient perspective. J Health Com
17. Davis TC, Wolf MS, Bass PF, et al. Literacy and misunderstanding prescription drug labels. Ann Intern Med
18. Wolf MS, Davis TC, Osborn CY, et al. Literacy, self-efficacy, and HIV medication adherence. Patient Educ Couns
19. Pandit AU, Tang JW, Bailey SC, et al. Education, literacy, and health: mediating effects on hypertension knowledge and control. Patient Educ Couns
20. Williams MV, Parker RM, Baker DW, et al. Inadequate functional health literacy among patients at two public hospitals. JAMA
21. Gazmararian JA, Williams MV, Peel J, et al. Health literacy and knowledge of chronic disease. Patient Educ Couns
22. Berkman ND, Sheridan SL, Donahue KE, et al. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med
23. Gordon MM, Hampson R, Capell HA, et al. Illiteracy in rheumatoid arthritis patients as determined by the Rapid Estimate of Adult Literacy in Medicine (REALM) score. Rheumatology (Oxf)
24. American Medical Association, AMA Press, Schillinger D, Davis T. Schwartzberg JG, VanGeest JB, Wang CC. A conceptual framework for the relationship between health literacy and health care outcomes: the chronic disease exemplar. Understanding Health Literacy: Implications for Medicine and Public Health
25. Levy M. The impact of consumers’ health literacy on public health. J Cons Affairs
26. Baker DW, Gazmararian JA, Williams MV, et al. Functional health literacy and the risk of hospital admission among Medicare managed care enrollees. Am J Public Health
27. Eichler K, Wieser S, Brügger U. The costs of limited health literacy: a systematic review. Int J Public Health
28. Parker RM, Ratzan SC, Lurie N. Health literacy: a policy challenge for advancing high-quality health care. Health Affairs
29. Clark B. Using law to fight a silent epidemic: the role of health literacy in health care access, quality, & cost. Ann Health Law
30. Parikh NS, Parker RM, Nurss JR, et al. Shame and health literacy: the unspoken connection. Patient Educ Couns
31. Safeer RS, Keenan J. Health literacy: the gap between physicians and patients. Am Fam Physician
33. Sexton PT, Sexton TL. Excess coronary mortality among Australian men and women living outside the capital city statistical divisions. Med J Aust
34. Helewa RM, Turner D, Wirtzfeld D, et al. Geographical disparities of rectal cancer local recurrence and outcomes: a population-based analysis. Dis Colon Rectum
35. Hines R, Markossian T, Johnson A, et al. Geographic residency status and census tract socioeconomic status as determinants of colorectal cancer outcomes. Am J Public Health
36. Desai A, Bekelis K, Zhao W, et al. Association of a higher density of specialist neuroscience providers with fewer deaths from stroke in the United States population. J Neurosurg
37. Weis KE, Liese AD, Hussey J, et al. Associations of rural residence with timing of HIV diagnosis and stage of disease at diagnosis, South Carolina 2001–2005. J Rural Health
38. Zahnd WE, Scaife SL, Francis ML. Health literacy skills in rural and urban populations. Am J Health Behav
39. Buchbinder R, Hall S, Youd JM. Functional health literacy of patients with rheumatoid arthritis attending a community-based rheumatology practice. J Rheumatol
40. Swearingen CJ, McCollum BA, Daltroy LH, et al. Screening for low literacy in a rheumatology setting: more than 10% of patients cannot read “cartilage,” “diagnosis,” “rheumatologist,” or “symptom”. J Clin Rheumatol
44. Davis TC, Long SW, Jackson RH, et al. Rapid Estimate of Adult Literacy in Medicine: a shortened screening instrument. Fam Med
45. Davis TC, Michielutte R, Askov EN, et al. Practical assessment of adult literacy in health care. Health Educ Behav
46. Buchbinder R, Batterham R, Ciciriello S, et al. Health literacy: what is it and why is it important to measure? J Rheumatol
48. Parker RM, Baker DW, Williams MV, et al. The test of functional health literacy in adults: a new instrument for measuring patients’ literacy skills. J Gen Int Med
49. Baker DW, Williams MV, Parker RM, et al. Development of a brief test to measure functional health literacy. Patient Educ Couns
50. Barber MN, Staples M, Osborne RH, et al. Up to a quarter of the Australian population may have suboptimal health literacy depending upon the measurement tool: results from a population-based survey. Health Promot Int
51. Jordan JE, Osborne RH, Buchbinder R. Critical appraisal of health literacy indices revealed variable underlying constructs, narrow content and psychometric weaknesses. J Clin Epidemiol
52. Kincaid JP, Fishburne RP, Rogers RL, Chissom BS. Derivation of New Readability Formulas (Automated Readability Index, Fog Count, and Flesch Reading Ease formula) for Navy Enlisted Personnel. Research Branch Report 1975; 8–75. Chief of Naval Technical Training: Naval Air Station Memphis.
53. Hirsh JM, Boyle DJ, Collier DH, et al. Limited health literacy is a common finding in a public health hospital's rheumatology clinic and is predictive of disease severity. J Clin Rheumatol
54. Brez SM, Taylor M. Assessing literacy for patient teaching: perspectives of adults with low literacy skills. J Adv Nurs
55. Griffin JM, Partin MR, Noorbaloochi S, et al. Variation in estimates of limited health literacy by assessment instruments and non-response bias. J Gen Int Med
56. Dumenci L, Matsuyama RK, Kuhn L, Perera RA, Siminoff LA. On the validity of the Rapid Estimate of Adult Literacy in Medicine (REALM) scale as a measure of health literacy. Commun Methods Meas
57. Collins SA, Currie LM, Bakken SM, et al. Health literacy screening instruments for eHealth applications: a systematic review. J Biomed Inform
58. Kripalani S, Weiss B. Teaching about health literacy and clear communication. J Gen Int Med
59. Clement S, Ibrahim S, Crichton N, et al. Complex interventions to improve the health of people with limited literacy: a systematic review. Patient Educ Couns
60. Rhee RL, Von Feldt JM, Schumacher HR, Merkel PA. Readability and suitability assessment of patient education materials in rheumatic diseases. Arthritis Care Res (Hoboken)
61. Stossel LM, Segar N, Gliatto P, et al. Readability of patient education materials available at the point of care. J Gen Intern Med
62. DeWalt DA, Callahan LF, Hawk VH, et al. Health Literacy Universal Precautions Toolkit. (Prepared by North Carolina Network Consortium, The Cecil G. Sheps Center for Health Services Research, The University of North Carolina at Chapel Hill, under Contract No. HHSA290200710014.) AHRQ Publication No. 10-0046-EF) Rockville, MD. Agency for Healthcare Research and Quality. April 2010.