Obstetrics & Gynecology:
Evaluation of the Readability of ACOG Patient Education Pamphlets
FREDA, MARGARET COMERFORD EdD, RN; DAMUS, KARLA PhD, RN; MERKATZ, IRWIN R. MD
Albert Einstein College of Medicine, Montefiore Medical Center, Bronx, New York.
Address reprint requests to: Margaret C. Freda, EdD, RN, 1695 Eastchester Road, Suite 301 Bronx, NY 10461. E-mail: firstname.lastname@example.org
Received August 21, 1998. Received in revised form October 23, 1998. Accepted November 5, 1998.
Objective: To evaluate whether ACOG's patient education pamphlets comply with the recommended readability level for health education materials intended for the general public.
Methods: All 100 English-language pamphlets available during 1997 (created or revised between 1988 and 1997) were evaluated using four standard readability formulas.
Results: Mean readability levels of ACOG's pamphlets were between grade 7.0 to grade 9.3, depending on the formula used. Analysis of readability over the 10 years showed a trend toward lower readability levels. Analysis by category of pamphlet found that the lowest readability levels were in “Especially for teens” pamphlets.
Conclusion: Our data suggested that most of ACOG's patient education pamphlets currently available are written at a higher readability level than recommended for the general public. The readability of those pamphlets improved in the 10 years since the organization published its first pamphlet, but the goal of sixth-grade readability level has not been reached.
A major component of primary health care for women, including prenatal and preconceptional care, is effective patient education.1 There are many methods of educating patients, but a common strategy is to give women pamphlets produced by reputable organizations that address specific topics to reinforce verbal teaching by health care providers. Many factors influence the effectiveness of written health education materials, including patients' age and education, cultural and linguistic relevance of pamphlets, layout and organization of material, illustrations used, and readability of pamphlets.2 One of the most frequently studied factors is readability, defined as the ease of reading and understanding a document. Readability was shown to be essential in appropriate use of written health educational materials by patients.2
When considering the literacy of their patients, health care providers mistakenly might assume that patient educational attainment correlates with reading ability, which has been studied frequently.2 Davis et al3 found that literacy levels in a sample of adults in primary care settings ranged from fifth to tenth grade, with 60% of patients reading at least three grade levels below their last grade attended. Miller and Bodie4 found that patients read at an average of six grade levels below their last grade completed. Similar results were found by others.5–7 The average reading level of United States citizens is eighth grade, and one in five adults reads at the fifth-grade level or below2; therefore, the general recommendation is that health education materials should be written at sixth-grade reading level to be readable to the widest audience possible.2,8
Readability is evaluated with mathematical formulas that most often measure combinations of frequency of multisyllabic words and sentence length. Certain formulas are more appropriate for specific audiences, such as children, textbook readers, or adults. It is recommended that more than one formula by used to evaluate readability, and that they match the audience intended for the document.9–12 Although different formulas produce slightly different results, Meade and Smith13 showed that results correlate highly with each other. The purpose of this study was to evaluate the readability of ACOG's patient education pamphlets, comparing their readability level with that recommended by health education experts. The last published evaluation of ACOG pamphlets was a decade ago.14
Materials and Methods
Each of the 100 English-language ACOG patient education pamphlets available in 1997 was evaluated for readability by using four different formulas. The pamphlets covered pregnancy, gynecology, and women's health issues, and were grouped into eight categories by the ACOG. Every pamphlet was imprinted with its original publication or revision date. All pamphlets analyzed were written or revised between 1988 and 1997.
Calculation of readability and grade level was done by typing the entire pamphlet into Readability Calculator software (Micro Power and Light, Dallas TX). Because the literature suggested using multiple formulas for evaluation of readability, we chose the four formulas most appropriate for the audience likely to receive those pamphlets, adult women presenting for obstetric or gynecologic care. The formulas were the Fry graph, the Flesch formula, Gunning's Fog formula, and McLaughlin's SMOG formula. The Fry graph (named for its developer) determines grade level by the number of sentences and syllables in each passage. Introduced in 1965, the Fry graph is widely accepted for evaluating a broad range of grade levels and is used often in assessing functional literacy materials.15,16 The Flesch formula (named for its developer) evaluates readability as the average number of words per sentence and the average number of syllables per word, and assigns a grade level. This formula was first devised to test readability of military training manuals, which necessarily contain some technical language and unfamiliar terms.12,17 Gunning's Fog formula (named for taking the “fog” out of reading) counts numbers of words, polysyllabic words, and sentences, and places emphasis on word length rather than number of words in a sentence, making the assumption that multisyllabic words are more difficult to read. It is useful for fourth grade through college reading levels.8,12,18 McLaughlin's SMOG formula (no universally accepted meaning for the acronym) counts a single variable, the number of words in the sample containing three or more syllables, is frequently used because it can be easily calculated by hand, and is useful for fifth grade through college reading levels.2,8,19 McLaughlin's SMOG formula was the only one used in the previously published evaluation of ACOG patient education pamphlets.14
Mean readability levels were calculated by each formula, category of pamphlet, and the year of publication or revision using the NCSS program version 6.0.3 (Number Crunching Statistical Systems, Kaysville, UT). Mean readability scores were compared using analysis of variance. Statistical significance was P < .05.
Table 1 shows the mean readability levels of all 100 ACOG pamphlets by the readability formulas used. The Flesch formula found the lowest mean and median levels of readability. All four readability formulas found mean and median readability levels above the recommended sixth-grade reading level.
Table 2 presents readability levels by formula and year of pamphlet development or revision. For the first pamphlet developed in 1988, the readability level was grade 12.0–14.5, depending on the formula. Readability levels decreased between 1988 and 1997; For example in 1992, readability levels ranged from 7.0–9.4, and in 1997 from 6.4–8.7. There was a statistically significant decrease in the level of readability over time, based on three (SMOG, Flesch, Fog) of the four formulas.
Table 3 provides the mean readability levels by formula and category of pamphlet. Differences were found by category, with the highest levels in the “Physiology and sexuality” pamphlets (range 9.3–10.0) and lowest in the “Especially for teens” category (range 4.5–8.0). Differences by category were statistically significant using Fry and Flesch formulas, but not significantly different based on SMOG or Fog formulas. The progress made toward lower readability levels for ACOG pamphlets over time is shown by aggregated years of publication or revision in Table 4. A downward trend was found for all four readability formulas.
Our data suggest that most of ACOG's patient education pamphlets are written at a higher readability level than recommended for the general public. The readability of those pamphlets has improved in the 10 years since the organization published its first pamphlet (from grade 13.0 in 1988 to grade 8.7 in 1997), but the goal of a sixth-grade readability level has not been reached. When examined by category, only “Especially for teens” came close to a sixth-grade reading level (Table 3).
If health care providers want to effectively educate patients using written materials, they must be aware of adult literacy. Although reading level correlates poorly with last grade completed in school,3,4 maternal education information from birth certificate data provided some estimates of upper limits of literacy for women giving birth in the United States. For example, in 1994 23% of the 3,952,267 women who gave birth had less than a high school education, and 20% had a college or higher degree.20 Other national data indicated that, in most age groups, birth rates are higher for women who have the lowest level of education, the exception being women aged 30–39 years, in which birth rates were highest in those who have more than 16 years' education.20 If reading levels are assumed to be on average 3 years lower than level of education, 60% of women giving birth annually are reading below a tenth-grade level.
Written information that accompanies oral teaching enhances understanding of complicated topics,2,8,21 and health education experts have recommended that patient education sessions end with the provision of written information for the patient to take home. The written information, however, must be readable by the patient because material written at an inappropriately high reading level does not educate patients.22–26 Although some of the patients we serve might have advanced degrees and high reading levels, that does not mean that health education materials should be written at different levels for different reading abilities. There is evidence that adults with advanced reading levels learn complex health information better when the materials are easy to read.2 The sixth-grade reading level for health education materials can be a level recommended for all adults.2
It is common for health education materials to be written at readability levels far above the average reader's eighth-grade reading level and farther above the recommended reading level of sixth grade.22–26 Estey and colleagues22 studied comprehension of health education materials developed at different reading levels and found that 77% of patients were able to comprehend health education materials that were prepared at the fifth-grade level, but only 30% could read it when prepared at the ninth grade level. Davis et al3 found that materials produced by the American Academy of Pediatrics, the Centers of Disease Control, and pharmaceutical companies were written at the tenth-grade reading level. Analysis of consent forms from the National Cancer Institute found readability levels of grades 12–17.27 Ott and Hardie23 analyzed the readability of advance directive documents and found an average readability of grades 11.3–18.2. Sarna and Ganley25 analyzed the readability of written materials for patients with lung cancer and found that all had a readability of tenth grade or higher. Cooley and colleagues24 analyzed the readability of American Cancer Society patient education materials, and found an average readability of grade 9.8. In obstetrics and gynecology, patient information booklets from ACOG were analyzed by Zion and Aiman,14 and readability was 11th grade and higher.
Although readability is essential for basic comprehension, there are other concepts that contribute to complete evaluation of health education materials. One of the methods suggested for analysis of many of those concepts is suitability assessment of materials,2 which evaluates content, literacy, graphics, layout and typography, learning stimulation, and cultural appropriateness, with scores from 0–100. That scoring system can be used by organizations that develop materials or by individuals who use patient education materials.
All providers of written patient education materials should work toward producing materials that are written at appropriate levels for the general public. Organizations that publish health education pamphlets need to ensure that their materials can be read by the widest audience possible and must continue to evaluate the materials they develop.
1. U.S. Public Health Service. Caring for our future: The content of prenatal care. Washington DC: U.S. Government Publication Office, 1989.
2. Doak C, Doak L, Root J. Teaching patients with low literacy skills, 2nd ed. New York, New York: Lippincott-Raven Publishers, 1996.
3. Davis TC, Crouch MA, Wills G, Miller S, Abdehou DM. The gap between patient reading comprehension and the readability of patient education materials. J Fam Pract 1990;31:533–8.
4. Miller B, Bodie M. Determination of reading comprehension levels for effective patient education materials. Nurs Res 1994;43:118–9.
5. Jackson RH, Davis TC, Bairnsfather LE, George RB, Crouch MA, Gault H. Patient reading ability: An overlooked problem in health care. South Med J 1991;84:1172–5.
6. Foltz A, Sullivan J. Reading level, learning presentation preference, and desire for information among cancer patients. J Cancer Educ 1996;11:32–8.
7. Williams MV, Parker RM, Baker DW, Parikh NS, Pitkin K, Coates WC, et al. Inadequate functional health literacy among patients at two public hospitals. JAMA 1995;274:1677–82.
8. Redman BK. The process of patient education, 8th ed. St. Louis, Missouri: Mosby Year Book Inc., 1997.
9. Meade CD, Byrd JC, Lee M. Improving patient comprehension of literature on smoking. Am J Public Health 1989;79:1411–2.
10. Olson AV. A question of reading validity. J Res Dev Educ 1986;19:33–40.
11. Calabro K, Taylor WC, Kapadia A. Pregnancy, alcohol use and the effectiveness of written health education materials. Patient Educ Counseling 1996;2:301–9.
12. Klare GR. Readability. In: Person D, ed. Handbook of reading research. New York: Longman, 1984:681–744.
13. Meade CD, Smith CF. Readability formulas: Cautions and criteria. Patient Educ Counseling 1991;17:153–8.
14. Zion AB, Aiman J. Level of reading difficulty in the American College of Obstetricians and Gynecologists patient education pamphlets. Obstet Gynecol 1989;74:955–60.
15. Fry E. Fry readability graph: Clarifications, validity, and extensions to level 17. J Reading 1977;12:242–52.
16. Fusaro JA. Applying statistical rigor to a validation study of the Fry readability graph. Reading Res Instruction 1988;28:44–8.
17. Flesch R. The art of readable writing. New York, New York: Harper & Row, 1974.
18. Gunning R, Kallan R. How to take the fog out of business writing. Chicago, Illinois: Dartnell, 1994.
19. McLaughlin GH. SMOG grading: a new readability formula. J Reading 1969;12:639–46.
20. Mathews MS, Ventura AM. Birth and fertility rates by educational attainment: United States, 1994. Monthly Vital Statistics Report NCHS 45 (10) Supplement April 24, 1997.
21. Arthur VA. Written patient information: A review of the literature. J Adv Nurs 1995;21:1081–6.
22. Estey A, Musseau A, Keehn L. Patient's understanding of health information: A multihospital comparison. Patient Educ Counseling 1994;24:73–8.
23. Ott BB, Hardie TL. Readability of advance directive documents. Image J Nurs Sch 1997;29:53–7.
24. Cooley ME, Moriarty H, Berger MS, Selm-Orr D, Coyle B, Short T. Patient literacy and the readability of written cancer educational materials. Oncol Nurs Forum 1995;22:1345–51.
25. Sarna L, Ganley BJ. A survey of lung cancer patient education materials. Oncol Nurs Forum 1995;22:1545–50.
26. Dowe MC, Lawrence PA, Carlson J, Keyserling TC. Patients' use of health-teaching materials at three readability levels. Appl Nurs Res 1997;10:86–93.
27. Meade CD, Howser DM. Consent forms: How to determine and improve readability. Oncol Nurs Forum 1992;19:1523–8.
© 1999 The American College of Obstetricians and Gynecologists