Secondary Logo

Journal Logo

AAPA Members can view Full text articles for FREE. Not a Member? Join today!

Quality and readability of online information about type 2 diabetes and nutrition

Bernard, Stephanie, PA-C, RD; Cooke, Tiffany, PA-C, RD; Cole, Tascha, MBA; Hachani, Laura, PA-C, RD; Bernard, Johnathan, MD, MPH

Journal of the American Academy of PAs: November 2018 - Volume 31 - Issue 11 - p 41–44
doi: 10.1097/01.JAA.0000546481.02560.4e
Original Research

Objective: The internet has become a vital resource through which patients learn about medical conditions. The aim of this study was to assess the quality and readability of online information about nutrition and diabetes management.

Methods: An internet search was conducted using three search terms of varying sophistication (how to eat with diabetes, diabetes diet, and medical nutrition therapy for diabetes) and the three most popular search engines (Yahoo, Bing, and Google). Forty-two websites were prospectively analyzed for quality of information and assessed for readability using the Flesch-Kincaid score.

Results: The 42 websites reviewed demonstrated wide variability in quality, regardless of the search term entered. The reading level required to understand the materials varied based on sophistication of the search term and ranged from the 6th- to the 11th-grade level.

Conclusions: The quality of online information on nutrition education for patients with diabetes was extremely variable and readability often was higher than the average American reading level (8th grade). An awareness of quality and readability of the materials found on the internet can strengthen the patient-provider relationship.

Stephanie Bernard is an assistant professor in the PA program at Shenandoah University in Leesburg, Va. Tiffany Cooke practices at Valley GI Consultants in Glendale, Ariz. Tascha Cole is a data manager for the PA program at Shenandoah University. Laura Hachani practices in the cardiac unit at Good Samaritan Hospital in Baltimore, Md. Johnathan Bernard is an orthopedic surgeon at the National Sports Medicine Institute in Leesburg, Va. Dr. Bernard discloses research/grant support from the Arthroscopy Association of North America, Smith and Nephew, Inc., and the American Board of Orthopaedic Surgery, and financial or material support from Supreme Orthopedic Systems, LLC, and Zimmer Biomet Holdings, Inc. The authors have disclosed no potential conflicts of interest, financial or otherwise.

The effect of the internet on healthcare has been explosive, making medical information dramatically more available. Online information has become a primary source for lay public medical knowledge.1-3 Although the internet can be a valuable and important resource, patients may have difficulty finding professionally regulated or peer-reviewed content.4,5 Clinical information on the internet across many medical subspecialties varies in content and readability.2,4-8 Many websites with medical information are written above the average US reading level (8th grade) and may not be understood by most of the public.7,9-11

For the millions of patients striving to manage their diabetes through diet, online health information has become a common resource for nutrition information. In the United States today, more than 30 million patients have been diagnosed with diabetes, and another 84 million have been diagnosed with prediabetes.12 Medical nutrition therapy is essential to prevent diabetes, prevent complications associated with uncontrolled diabetes, and for patients to manage the disease.13-16

Physician assistants (PAs) and advanced practice registered nurses (APRNs) are at the forefront of primary care and are critical to patient education, especially for patients with chronic conditions such as diabetes.17,18 Guidelines for diabetes management that translate healthcare policy into clinical practice remain a challenge.13-16 Increasing clinician awareness of what patients read on the internet may improve diabetes education and care, because it gives clinicians an opportunity to educate patients on acceptable internet resources and recommendations.

This study assessed the quality and readability of online information about nutrition education and diabetes management. Three common search terms of differing sophistication were entered into three of the most common search engines.19 The websites found by these searches were independently reviewed for quality by three PAs with dual training as registered dietitian nutritionists (RDNs). One of the reviewers was a former certified diabetes educator.

Websites also were assessed for readability. Our hypothesis was that the quality of information on nutrition and diabetes would vary widely and that more complex medical terminology would yield higher quality but lower readability. With this study, we hoped to achieve a better understanding of quality and readability of information available on the internet about nutrition and type 2 diabetes, creating an opportunity to improve patient education for clinicians treating patients with type 2 diabetes.

Back to Top | Article Outline


Web search

We used the same methodology as previous studies that assessed websites for patient-oriented medical information.8,20 Three web search strings about nutrition and diabetes diet were used—diabetes diet, how to eat with diabetes, and medical nutrition therapy for diabetes. These search strings, which varied in sophistication, were entered into three of the most popular search engines: Google, Yahoo, and Bing, which represented more than 90% of all global internet searches at the time of the study.19 The term medical nutrition therapy was chosen as demonstrated by the American Diabetes Association (ADA), the American Association of Diabetes Educators, and the Academy of Nutrition and Dietetics, as the medical terminology used by professional societies to define optimal nutrition guidelines.13,14

All of the web searches were performed on May 9, 2016, resulting in nine unique searches (three search strings for three search terms). The top 25 search results for each term were reviewed for a total of 225 websites. Of those websites, 116 were excluded as duplicates, 8 as websites not focused on nutrition or diabetes, 29 as dictionary definitions, 14 as news sites or blogs, and 16 as websites with no content based on officially accepted definitions. The websites were captured electronically on the day of the search and search terms were blinded to remove bias.

Back to Top | Article Outline

Quality assessment of websites

Previous methodology was used to create the quality assessment, with the joint position statement from the ADA, American Academy of Diabetes Educators, and the Academy of Nutrition and Dietetics providing the gold standard for nutrition guidelines.8,13,20 Valuable patient education on diabetes and nutrition was defined as understanding nutrition principles and goals for prevention and management. These specific domains from the position statement were used to construct the quality assessment (Table 1).



Three PAs with dual training as RDNs served as the review panel. The quality assessment construct was determined before the web searches and scoring by the reviewers. A total possible quality assessment score of 30 points was based on two main areas: understanding nutrition principles (top score, 14 points) and nutrition goals for prevention and management (top score, 16 points).

Back to Top | Article Outline

Readability of websites

The Flesch-Kincaid Reading Level Score was used to assess reading level of the websites. This analysis method was designed and used by the US Navy in 1975 to assess word length and sentence length as related to reading comprehension.21,22 The Flesch-Kincaid score has been used to evaluate printed information about medical conditions.8,20 Reading level scores were calculated using an online calculator.22

Back to Top | Article Outline


SPSS version 25 was used for statistical analysis. Groups were formed based on the search term used: how to eat with diabetes, diabetes diet, and medical nutrition therapy. ANOVA tests were used to compare means on normally distributed data. When applicable, independent student t-tests were used to compare groups. An intraclass correlation coefficient was calculated using a two-way mixed effects model to evaluate agreement between the three independent expert raters because the quality score was based on their average ratings. The threshold for statistical significance was P < .05 for all analyses with confidence intervals displaced at the 95% level. No human testing was needed for this study, so institutional review board approval was not required. A qualitative subgroup analysis was completed on the quality assessment breakdown between nutrition principles versus prevention and management.

Back to Top | Article Outline


The final analyses were performed on 42 unique websites. The mean quality assessment score for all websites was 11.4 points (±7.75 SD) with a range of 2 to 29 points. The PA/RDN review panel was in near-perfect agreement, with an intraclass correlation coefficient of 0.994 (CI 0.986-0.997, 95%). Evaluation of each specific search term demonstrated a quality score of 10 for how to eat with diabetes, 10.7 for diabetes diet, and 13 for medical nutrition therapy for diabetes, although these differences were not statistically significant. A subgroup analysis found variation in the distribution of earned quality points based on the search term used. Of the three search terms, medical nutrition therapy for diabetes earned a greater amount of points from the more sophisticated quality measures in the prevention and treatment subgroup. The less sophisticated search terms, diabetes diet and how to eat with diabetes resulted in a greater proportion of information from the basic nutrition principles (Figure 1).



Differences in reading levels were based on the search terms used. Flesch-Kincaid reading level was highest, at the 11th-grade level (11.5 [9.8-13.2, CI 95%]), when the search term medical nutrition therapy for diabetes was used. How to eat with diabetes demonstrated a reading level of 6.43 grade (5.9-7, CI 95%), and diabetes diet demonstrated a reading level of 6.29 grade (5-7.6, CI 95%).

Back to Top | Article Outline


Demand for primary care services has increased significantly in recent years.23 A creative approach is needed to manage millions of patients who may have multiple chronic conditions, and to do so in a manner that demonstrates meaningful use criteria patient education.24 The financial effect of patient knowledge and adherence also is paramount.25 One approach to this challenge, as demonstrated in this study, has been to better understand the resources most accessed to supplement patient medical education, such as the internet.

When a patient or family member wants more information about a medical condition, such as type 2 diabetes, the internet is one of the most commonly accessed resources.3 This has remained a trend despite concerns that online information is not always correct. Type 2 diabetes is a common and complicated disease to manage, requiring daily attention to diet, exercise, and medications to control and prevent complications.12,13 The internet, however, does not provide consistent quality resources and many websites are at a reading level too advanced for most Americans. Website content also is skewed toward nutrition principles, with less information about prevention and management, which are the critical tools needed for adherence.

Knowing this, clinicians may decide to accumulate a list of approved sites that have demonstrated quality information at an appropriate reading level. Screening websites may strengthen the patient-provider relationship and maximize education during each encounter. Clinicians also may consider working with national associations, such as the ADA, the American Association of Diabetes Educators, and the Academy of Nutrition and Dietetics, to publish patient education materials at an appropriate reading level. Further study is needed to identify preferred search terms that patients may use when seeking online nutrition and diabetes management education resources.

Back to Top | Article Outline


This was a novel study on diabetes education. Because no standard quality assessment existed, a review panel of PAs and RDNs, as well as the position statement from the ADA, the American Association of Diabetes Educators, and the Academy of Nutrition and Dietetics, were used as the gold standard for current guidelines. We did this to make the results generalizable to PAs. Our three independent clinicians demonstrated near-perfect agreement in assessment of the websites, serving as an internal control for assessing the quality of the websites generated by the search terms. In addition, we were limited in the number of websites to review. The internet is ubiquitous and countless other websites may be accessed by patients that were not evaluated. However, our chosen search engines accounted for more than 90% of the global searches on the internet.9 Future studies may want to expand consideration to patients who speak languages other than English. Studying which websites most clinicians prefer and use most often, and why, also may be beneficial.

Back to Top | Article Outline


Diabetes management and nutrition education remain critical components to patient success and well-being. As demonstrated by this study, clinicians should consider prescreening websites or have a list of approved online resources ready to share with patients to enhance education. Simple tips such as using sites ending in .org or .edu as opposed to those ending in .com may help patients find better-quality information. Our PA and RDN panel agreed that two acceptable online resources for patients seeking information on managing diabetes through nutrition are from the ADA and (use the keyword diabetes to search this site) from the Academy of Nutrition and Dietetics.

The trend for patients to use the internet as a resource for medical information will likely only continue to increase. Therefore, embracing this trend may help clinicians guide patients to quality resources and improve patient care. With a coordinated effort from patients and clinicians alike, readable, high-quality patient education material on type 2 diabetes may be more readily provided.

Back to Top | Article Outline


1. Wald HS, Dube CE, Anthony DC. Untangling the web—the impact of internet use on health care and the physician-patient relationship. Patient Educ Couns. 2007;68(3):218–224.
2. McMullan M. Patients using the Internet to obtain health information: how this affects the patient-health professional relationship. Patient Educ Couns. 2006;63(1-2):24–28.
3. Wang L, Wang J, Wang M, et al Using internet search engines to obtain medical information: a comparative study. J Med Internet Res. 2012;14(3):e74.
4. Badarudeen S, Sabharwal S. Assessing readability of patient education materials: current role in orthopaedics. Clin Orthop Relat Res. 2010;468(10):2572–2580.
5. Bruce-Brand RA, Baker JF, Byrne DP, et al Assessment of the quality and content of information on anterior cruciate ligament reconstruction on the internet. Arthroscopy. 2013;29(6):1095–1100.
6. Berg GM, Hervey AM, Atterbury D, et al Evaluating the quality of online information about concussions. JAAPA. 2014;27(2):1–8.
7. Lambert K, Mullan J, Mansfield K, et al Evaluation of the quality and health literacy demand of online renal diet information. J Hum Nutr Diet. 2017;30(5):634–645.
8. Fabricant PD, Dy CJ, Patel RM, et al Internet search term affects the quality and accuracy of online information about developmental hip dysplasia. J Pediatr Orthop. 2013;33(4):361–365.
9. Centers for Disease Control and Prevention. Health literacy. Accessed August 13, 2018.
11. US Department of Health and Human Services. Office of Disease Prevention and Health. National action plan to improve health literacy. Washington, DC. 2010:1-73.
12. Centers for Disease Control and Prevention. National Diabetes Statistics Report, 2017. Estimates of diabetes and its burden in the United States. Accessed August 13, 2018.
13. Powers MA, Bardsley J, Cypress M, et al Diabetes self-management education and support in type 2 diabetes: a joint position statement of the American Diabetes Association, the American Association of Diabetes Educators, and the Academy of Nutrition and Dietetics. J Acad Nutr Diet. 2015;1–11.
14. American Diabetes Association. Standards of medical care in diabetes—2017. Diabetes Care. 2017;40(suppl 1):S1–S135.
15. Nutrition recommendations and interventions for diabetes: a position statement of the American Diabetes Association. Diabetes Care. 2008;31(suppl 1):S61–S78.
16. Academy of Nutrition and Dietetics. Treating diabetes with medical nutrition therapy. Accessed August 13, 2018.
17. National Commission on Certification of Physician Assistants. 2017 statistical profile of certified physician assistants: an annual report of the National Commission on Certification of Physician Assistants. Accessed September 5, 2018.
18. Gardner G, Duffield C, Doubrovsky A, Adams M. Identifying advanced practice: a national survey of a nursing workforce. Int J Nurs Stud. 2016;55:60–70.
19. Netmarketshare. Market share reports. Accessed August 13, 2018.
20. Dy CJ, Taylor SA, Patel RM, et al Does the quality, accuracy, and readability of information about lateral epicondylitis on the internet vary with the search term used. Hand (N Y). 2012;7(4):420–425.
21. Kincaid JP, Fishburne RP, Rogers RL, Chissom BS. Derivation of new readability formulas (automated readability index, fog count, and Flesch reading ease formula) for Navy enlisted personnel. Chief of Naval Technical Training: Naval Air Station Memphis. Research Branch Report. 1975:8–75.
22. Flesch-Kincaid. FK reading score. Accessed August 13, 2018.
23. Konchak JN, Moran MR, O'Brien MJ, et al The state of diabetes prevention policy in the USA following the Affordable Care Act. Curr Diab Rep. 2016;16(6):55.
24. Proser M, Bysshe T, Weaver D, Yee R. Community health centers at the crossroads: growth and staffing needs. JAAPA. 2015;28(4):49–53.
25. Cohen RA, Zammitti EP, Martinez ME. Health insurance coverage: early release of estimates from the National Health Interview Survey, 2016. Accessed August 13, 2018.

internet; nutrition; diabetes; quality; readability; information

Copyright © 2018 American Academy of Physician Assistants