The effect of the internet on healthcare has been explosive, making medical information dramatically more available. Online information has become a primary source for lay public medical knowledge.1-3 Although the internet can be a valuable and important resource, patients may have difficulty finding professionally regulated or peer-reviewed content.4,5 Clinical information on the internet across many medical subspecialties varies in content and readability.2,4-8 Many websites with medical information are written above the average US reading level (8th grade) and may not be understood by most of the public.7,9-11
For the millions of patients striving to manage their diabetes through diet, online health information has become a common resource for nutrition information. In the United States today, more than 30 million patients have been diagnosed with diabetes, and another 84 million have been diagnosed with prediabetes.12 Medical nutrition therapy is essential to prevent diabetes, prevent complications associated with uncontrolled diabetes, and for patients to manage the disease.13-16
Physician assistants (PAs) and advanced practice registered nurses (APRNs) are at the forefront of primary care and are critical to patient education, especially for patients with chronic conditions such as diabetes.17,18 Guidelines for diabetes management that translate healthcare policy into clinical practice remain a challenge.13-16 Increasing clinician awareness of what patients read on the internet may improve diabetes education and care, because it gives clinicians an opportunity to educate patients on acceptable internet resources and recommendations.
This study assessed the quality and readability of online information about nutrition education and diabetes management. Three common search terms of differing sophistication were entered into three of the most common search engines.19 The websites found by these searches were independently reviewed for quality by three PAs with dual training as registered dietitian nutritionists (RDNs). One of the reviewers was a former certified diabetes educator.
Websites also were assessed for readability. Our hypothesis was that the quality of information on nutrition and diabetes would vary widely and that more complex medical terminology would yield higher quality but lower readability. With this study, we hoped to achieve a better understanding of quality and readability of information available on the internet about nutrition and type 2 diabetes, creating an opportunity to improve patient education for clinicians treating patients with type 2 diabetes.
We used the same methodology as previous studies that assessed websites for patient-oriented medical information.8,20 Three web search strings about nutrition and diabetes diet were used—diabetes diet, how to eat with diabetes, and medical nutrition therapy for diabetes. These search strings, which varied in sophistication, were entered into three of the most popular search engines: Google, Yahoo, and Bing, which represented more than 90% of all global internet searches at the time of the study.19 The term medical nutrition therapy was chosen as demonstrated by the American Diabetes Association (ADA), the American Association of Diabetes Educators, and the Academy of Nutrition and Dietetics, as the medical terminology used by professional societies to define optimal nutrition guidelines.13,14
All of the web searches were performed on May 9, 2016, resulting in nine unique searches (three search strings for three search terms). The top 25 search results for each term were reviewed for a total of 225 websites. Of those websites, 116 were excluded as duplicates, 8 as websites not focused on nutrition or diabetes, 29 as dictionary definitions, 14 as news sites or blogs, and 16 as websites with no content based on officially accepted definitions. The websites were captured electronically on the day of the search and search terms were blinded to remove bias.
Quality assessment of websites
Previous methodology was used to create the quality assessment, with the joint position statement from the ADA, American Academy of Diabetes Educators, and the Academy of Nutrition and Dietetics providing the gold standard for nutrition guidelines.8,13,20 Valuable patient education on diabetes and nutrition was defined as understanding nutrition principles and goals for prevention and management. These specific domains from the position statement were used to construct the quality assessment (Table 1).
Three PAs with dual training as RDNs served as the review panel. The quality assessment construct was determined before the web searches and scoring by the reviewers. A total possible quality assessment score of 30 points was based on two main areas: understanding nutrition principles (top score, 14 points) and nutrition goals for prevention and management (top score, 16 points).
Readability of websites
The Flesch-Kincaid Reading Level Score was used to assess reading level of the websites. This analysis method was designed and used by the US Navy in 1975 to assess word length and sentence length as related to reading comprehension.21,22 The Flesch-Kincaid score has been used to evaluate printed information about medical conditions.8,20 Reading level scores were calculated using an online calculator.22
SPSS version 25 was used for statistical analysis. Groups were formed based on the search term used: how to eat with diabetes, diabetes diet, and medical nutrition therapy. ANOVA tests were used to compare means on normally distributed data. When applicable, independent student t-tests were used to compare groups. An intraclass correlation coefficient was calculated using a two-way mixed effects model to evaluate agreement between the three independent expert raters because the quality score was based on their average ratings. The threshold for statistical significance was P < .05 for all analyses with confidence intervals displaced at the 95% level. No human testing was needed for this study, so institutional review board approval was not required. A qualitative subgroup analysis was completed on the quality assessment breakdown between nutrition principles versus prevention and management.
The final analyses were performed on 42 unique websites. The mean quality assessment score for all websites was 11.4 points (±7.75 SD) with a range of 2 to 29 points. The PA/RDN review panel was in near-perfect agreement, with an intraclass correlation coefficient of 0.994 (CI 0.986-0.997, 95%). Evaluation of each specific search term demonstrated a quality score of 10 for how to eat with diabetes, 10.7 for diabetes diet, and 13 for medical nutrition therapy for diabetes, although these differences were not statistically significant. A subgroup analysis found variation in the distribution of earned quality points based on the search term used. Of the three search terms, medical nutrition therapy for diabetes earned a greater amount of points from the more sophisticated quality measures in the prevention and treatment subgroup. The less sophisticated search terms, diabetes diet and how to eat with diabetes resulted in a greater proportion of information from the basic nutrition principles (Figure 1).
Differences in reading levels were based on the search terms used. Flesch-Kincaid reading level was highest, at the 11th-grade level (11.5 [9.8-13.2, CI 95%]), when the search term medical nutrition therapy for diabetes was used. How to eat with diabetes demonstrated a reading level of 6.43 grade (5.9-7, CI 95%), and diabetes diet demonstrated a reading level of 6.29 grade (5-7.6, CI 95%).
Demand for primary care services has increased significantly in recent years.23 A creative approach is needed to manage millions of patients who may have multiple chronic conditions, and to do so in a manner that demonstrates meaningful use criteria patient education.24 The financial effect of patient knowledge and adherence also is paramount.25 One approach to this challenge, as demonstrated in this study, has been to better understand the resources most accessed to supplement patient medical education, such as the internet.
When a patient or family member wants more information about a medical condition, such as type 2 diabetes, the internet is one of the most commonly accessed resources.3 This has remained a trend despite concerns that online information is not always correct. Type 2 diabetes is a common and complicated disease to manage, requiring daily attention to diet, exercise, and medications to control and prevent complications.12,13 The internet, however, does not provide consistent quality resources and many websites are at a reading level too advanced for most Americans. Website content also is skewed toward nutrition principles, with less information about prevention and management, which are the critical tools needed for adherence.
Knowing this, clinicians may decide to accumulate a list of approved sites that have demonstrated quality information at an appropriate reading level. Screening websites may strengthen the patient-provider relationship and maximize education during each encounter. Clinicians also may consider working with national associations, such as the ADA, the American Association of Diabetes Educators, and the Academy of Nutrition and Dietetics, to publish patient education materials at an appropriate reading level. Further study is needed to identify preferred search terms that patients may use when seeking online nutrition and diabetes management education resources.
This was a novel study on diabetes education. Because no standard quality assessment existed, a review panel of PAs and RDNs, as well as the position statement from the ADA, the American Association of Diabetes Educators, and the Academy of Nutrition and Dietetics, were used as the gold standard for current guidelines. We did this to make the results generalizable to PAs. Our three independent clinicians demonstrated near-perfect agreement in assessment of the websites, serving as an internal control for assessing the quality of the websites generated by the search terms. In addition, we were limited in the number of websites to review. The internet is ubiquitous and countless other websites may be accessed by patients that were not evaluated. However, our chosen search engines accounted for more than 90% of the global searches on the internet.9 Future studies may want to expand consideration to patients who speak languages other than English. Studying which websites most clinicians prefer and use most often, and why, also may be beneficial.
Diabetes management and nutrition education remain critical components to patient success and well-being. As demonstrated by this study, clinicians should consider prescreening websites or have a list of approved online resources ready to share with patients to enhance education. Simple tips such as using sites ending in .org or .edu as opposed to those ending in .com may help patients find better-quality information. Our PA and RDN panel agreed that two acceptable online resources for patients seeking information on managing diabetes through nutrition are www.diabetes.org from the ADA and www.eatright.org (use the keyword diabetes to search this site) from the Academy of Nutrition and Dietetics.
The trend for patients to use the internet as a resource for medical information will likely only continue to increase. Therefore, embracing this trend may help clinicians guide patients to quality resources and improve patient care. With a coordinated effort from patients and clinicians alike, readable, high-quality patient education material on type 2 diabetes may be more readily provided.
1. Wald HS, Dube CE, Anthony DC. Untangling the web—the impact of internet
use on health care and the physician-patient relationship. Patient Educ Couns
2. McMullan M. Patients using the Internet
to obtain health information
: how this affects the patient-health professional relationship. Patient Educ Couns
3. Wang L, Wang J, Wang M, et al Using internet
search engines to obtain medical information
: a comparative study. J Med Internet Res
4. Badarudeen S, Sabharwal S. Assessing readability
of patient education materials: current role in orthopaedics. Clin Orthop Relat Res
5. Bruce-Brand RA, Baker JF, Byrne DP, et al Assessment of the quality
and content of information
on anterior cruciate ligament reconstruction on the internet
6. Berg GM, Hervey AM, Atterbury D, et al Evaluating the quality
of online information
about concussions. JAAPA
7. Lambert K, Mullan J, Mansfield K, et al Evaluation of the quality
and health literacy demand of online renal diet information
. J Hum Nutr Diet
8. Fabricant PD, Dy CJ, Patel RM, et al Internet
search term affects the quality
and accuracy of online information
about developmental hip dysplasia. J Pediatr Orthop
11. US Department of Health and Human Services. Office of Disease Prevention and Health. National action plan to improve health literacy. Washington, DC. 2010:1-73.
13. Powers MA, Bardsley J, Cypress M, et al Diabetes
self-management education and support in type 2 diabetes
: a joint position statement of the American Diabetes
Association, the American Association of Diabetes
Educators, and the Academy of Nutrition
and Dietetics. J Acad Nutr Diet
14. American Diabetes
Association. Standards of medical care in diabetes
—2017. Diabetes Care
. 2017;40(suppl 1):S1–S135.
recommendations and interventions for diabetes
: a position statement of the American Diabetes
Association. Diabetes Care
. 2008;31(suppl 1):S61–S78.
17. National Commission on Certification of Physician Assistants. 2017 statistical profile of certified physician assistants: an annual report of the National Commission on Certification of Physician Assistants. http://www.nccpa.net
. Accessed September 5, 2018.
18. Gardner G, Duffield C, Doubrovsky A, Adams M. Identifying advanced practice: a national survey of a nursing workforce. Int J Nurs Stud
20. Dy CJ, Taylor SA, Patel RM, et al Does the quality
, accuracy, and readability
about lateral epicondylitis on the internet
vary with the search term used. Hand (N Y)
21. Kincaid JP, Fishburne RP, Rogers RL, Chissom BS. Derivation of new readability
formulas (automated readability
index, fog count, and Flesch reading ease formula) for Navy enlisted personnel. Chief of Naval Technical Training: Naval Air Station Memphis. Research Branch Report. 1975:8–75.
23. Konchak JN, Moran MR, O'Brien MJ, et al The state of diabetes
prevention policy in the USA following the Affordable Care Act. Curr Diab Rep
24. Proser M, Bysshe T, Weaver D, Yee R. Community health centers at the crossroads: growth and staffing needs. JAAPA