Journal Logo

Original Clinical Science—Liver

Health Literacy Burden Is Associated With Access to Liver Transplantation

Bababekov, Yanik J. MD, MPH1; Hung, Ya-Ching MD, MPH1; Rickert, Charles G. MD1; Njoku, Faith C. BA2; Cao, Bonnie BA3; Adler, Joel T. MD, MPH4; Brega, Angela G. PhD5; Pomposelli, James J. MD6; Chang, David C. PhD, MPH, MBA1; Yeh, Heidi MD1

Author Information
doi: 10.1097/TP.0000000000002536
  • Free



Accessibility of the healthcare system for vulnerable populations is an over-arching population health problem in the United States.1,2 Transplantation is an ideal test case to study healthcare access given that it is a federally monitored system. Only 20% of all patients with end-stage liver disease (ESLD) are listed for transplantation, with even fewer actually being transplanted.3 Decreased access to kidney transplantation has been associated with lower educational attainment (EA).4 Lower EA is thought to contribute directly to poor access to care through mechanisms such as insurance coverage, income, and employment status and indirectly through suboptimal self-care (eg, poor utilization of preventive services).5 Low EA is likely to be a significant problem among ESLD patients, as well, as the prevalence of cirrhosis leading to ESLD increases with decreasing levels of education.6

To improve access to care for patients with low EA, current efforts have focused on patient-directed interventions. However, EA of patients is not readily modifiable, and its improvement is beyond the reach of the healthcare system. Developing healthcare systems that do not require patients to have advanced EA may have an equally powerful impact on access to and delivery of patient-centered care and is more feasible.7,8

Organizational health literacy (OHL) refers to the degree to which a healthcare organization is actively engaged in implementing system-level changes to ensure that all patients are able to understand health information, navigate the healthcare system, engage in the healthcare process, and successfully manage their health.9-11 The role that OHL plays in access to transplantation is not known, but much as a physician who is fluent in Spanish will be more accessible to patients whose preferred language is Spanish rather than English, a center with high OHL will likely be more accessible to patients with low EA. Websites have been shown to be an accurate indicator of organizational culture and to affect perception of organizational culture.12-14 Therefore, we used website readability as a barometer of OHL in this study, as patient education websites are easily accessible, and validated measures for quantifying their understandability and actionability exist. We hypothesized that centers with difficult-to-understand websites would have fewer low EA patients on their liver transplant waitlists.


We conducted an observational study of the understandability of liver transplant center education websites and assessed whether the understandability of these websites was associated with disparities in access to the waitlist for a vulnerable patient population, as operationalized as the percent of patients with low EA waitlisted at that center. This study was approved under an exemption by the Institutional Review Board of Partners HealthCare. The role that OHL plays in access to transplantation is not known, but much as a physician who is fluent in Spanish will be more accessible to patients whose preferred language is Spanish rather than English, a center with high OHL will likely be more accessible to patients with low EA. Websites have been shown to be an accurate indicator of organizational culture and to affect perception of organizational culture.12-14 Therefore, we used website readability as a barometer of OHL in this study, as patient education websites are easily accessible, and validated measures for quantifying their understandability and actionability exist. We hypothesized that centers with difficult-to-understand websites would have fewer low EA patients on their liver transplant waitlists.

Data Sources

The data reported here have been supplied by the Minneapolis Medical Research Foundation, the contractor for the Scientific Registry of Transplant Recipients (SRTR). The interpretation and reporting of these data are the responsibility of the authors and in no way should be seen as an official policy of or interpretation by the SRTR or the US Government. This study used data from the SRTR. The SRTR data system includes data on all donor, waitlisted candidates, and transplant recipients in the United States, submitted by the members of the Organ Procurement and Transplantation Network (OPTN). The Health Resources and Services Administration, US Department of Health and Human Services, provides oversight to the activities of the OPTN and SRTR contractors. Per OPTN/United Network for Organ Sharing (UNOS) policy, the United States is geographically divided into 11 UNOS regions, with each region subdivided into several designated groups of counties, or 58 Donor Service Areas (DSAs).

The SRTR database was used to identify all patients ≥18 years old on the liver transplant waitlist from January 1, 2007, to December 31, 2016. Patients waitlisted as status 1 (risk of imminent death without transplant), or those requiring emergent liver transplant, were excluded as these patients had minimal time on the waitlist given disease severity. DSAs that did not list patients for liver transplant during the study period were excluded. New York State was grouped into a single DSA due to previously described practice patterns.15 The DSAs of Puerto Rico and the US Virgin Islands were excluded as data for some variables were not available.


Accessibility of Liver Transplant Centers

We operationalized access to care at liver transplant centers as the percentage of patients on the waitlist at each center who have a limited EA during the study period.1,2 Low EA was defined as having less than a high school education. For each center-year, we computed the percentage of patients on the liver transplant waitlist with low EA by taking the number of waitlisted patients with less than a high school education from any given year divided by the total number of waitlisted patients in that year.

Understandability of Educational Information

We assessed the understandability of liver transplant education information available from each transplant center’s website as an indicator of OHL. The SRTR website was searched for adult liver transplant centers active in the year 2016 (n = 114).16 Liver transplant patient education information available on the websites of the 114 liver transplant centers was evaluated. Materials reviewed were limited to those written in English and located directly on the website of each liver transplant center. External links to online videos and education websites not affiliated with the transplant center were excluded. Although we assessed information available on the websites of 114 centers, data from 112 centers were included in the final analysis, given aforementioned exclusion criteria.

Understandability was evaluated using the Clear Communication Index (CCI). In response to federal legislation and national action plans to make patient information easy to understand, the Centers for Disease Control and Prevention (CDC) developed the CCI to assess the clarity of print material.17 Prior assessment tools have evaluated the reading level of print materials by accounting for word and sentence length. The CCI goes beyond traditional readability formulas and assesses understandability of text-based materials. A CCI Score Sheet is used to evaluate 20 characteristics of written educational materials. Absolute CCI scores are converted to a 100-point scale, with a score ≥90 indicating that the print material is easy to understand. Two reviewers (B.C., F.C.N.) independently completed the CCI for all online liver transplant patient education materials. Discrepancies in CCI scores were resolved by an adjudication panel (Y.J.B., D.C.C., H.Y.). The preadjudication inter-rater reliability for CCI scoring was 97.4% (disagreement on 3 out of 114 total websites).


Access to care is multifactorial, and thus, we created a robust model to adjust for center, health system, and general population characteristics.

Liver Transplant Center Characteristics

Covariates representing liver transplant center case-mix were calculated across all study years for all waitlisted patients. Patients on the liver transplant waitlist are prioritized for transplant by the Model for End-Stage Liver Disease (MELD) score.18 From SRTR, we utilized the last recorded MELD for each patient before liver transplant or removal from the waitlist. This SRTR variable accounts for the match MELD for patients listed from 2007 to 2015 and for the match Na-MELD for patients listed in 2016. For simplicity, we will refer to Na-MELD as MELD in this article.

For each center, we calculated mean MELD score at transplant, mean MELD score at removal from the waitlist (ie, for patients determined to be too sick to undergo transplant), and mean age at removal from the waitlist. In addition, the percentage of waitlisted males, whites, candidates with private insurance, candidates with hepatitis C virus (HCV), candidates with alcoholic liver disease (EtOH), and candidates with HCV and EtOH were calculated for each center. Disease causes such as EtOH and HCV were included as these are the most common causes for liver failure requiring transplantation.19

We calculated case-mix variables at the center level as our primary outcome of interest is differences in accessibility to the waitlist for each transplant center. Therefore, both of our independent and dependent variables were calculated at a center level. Performing the analysis at the patient level would not be appropriate because all patients in the same center would have the same measure of center accessibility to the waitlist. In other words, the methods require adjustment for differences between patient groups, instead of differences between individual patients.

EA of General Population

EA and health literacy, or the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions,20 are highly correlated.21 EA is thus an appropriate proxy for patients who would be vulnerable to accessibility barriers of transplant center education websites. The American Community Survey is considered the main source of data on EA in the United States.22 EA of the general population of each DSA was obtained from the county-level US data from the American Community Survey for adults ≥25 years, utilizing 5-year estimates for the time periods 2007–2011 and 2012–2016 for patients from 2007 to 2011 and 2012 to 2016, respectively.23 For each 5-year estimate period, we grouped US counties into DSAs and calculated the percentage of adults with low EA, defined as the percentage of people with less than a high school education, for each DSA. Low EA was defined as having less than a high school education as education is the strongest parameter of socioeconomic status, and it is suggested in the literature that high school completion is a meaningful metric of EA.24-26

Practice Patterns

Market competition within DSAs is known to impact center practice patterns, including aggressiveness in patient recruitment, and resultant outcomes in transplantation.15,27 Market competition can be precisely calculated in transplantation given known designated areas of practice for each transplant center. Market competition within DSAs in this study was quantified via the Herfindahl-Hirschman Index (HHI).27 Market share for each liver transplant center was defined as the percentage of transplants performed by a given center within a DSA and expressed as a decimal. The HHI, a statistic calculated for each DSA, was calculated by taking the sum of the square of market shares across all study years. An HHI of 1.00 represents a monopoly in which 1 transplant center performs all of the liver transplants in a DSA. Conversely, an HHI approaching zero indicates a market with maximized competition, with many transplant centers performing a relatively equal number of liver transplants within the DSA. Share 35 was implemented in June of 2013 to facilitate allocation of donor livers to patients with MELD ≥35.28 As Share 35 is process change meant to impact organ allocation at the UNOS region and not access to the waitlist, we did not account for it in our analysis of market competition within DSAs.

Statistical Analysis

In our descriptive analysis, after obtaining the mean for each covariate for each center, we reported the median value of the covariate for all centers. Unadjusted comparisons were performed using Mann-Whitney U test. Inter-rater reliability on CCI assessment of transplant center websites was calculated. To assess the association between the percentage of the general population with low EA and liver transplant website CCI score, we utilized the Spearman correlation test. In adjusted analyses, multivariable linear regression was performed, adjusting for general population, DSA, and transplant center covariates while clustering data by DSA. We did not report standardized coefficients, as our current coefficients allow for interpretation of each variable in its native scale. All analyses were 2-tailed, and statistical significance was accepted at the P < 0.05 level. Statistical analyses were performed using Intercooled Stata software, version 13.1 (StataCorp, College Station, TX).


Center Characteristics

The final analysis included 84 774 waitlist patients at 112 liver transplant centers over 10 years. Across all centers, the median age at which patients were placed on the waitlist was 54.9 years (IQR, 54.3–55.4). In addition, 66.1% (IQR, 63.4–68.8) of patients were male, 73.1% (IQR, 62.6–82.4) of patients were white, and 55.6% (IQR, 48.7–62.8) of patients had private insurance. The 2 most common primary diagnoses for transplant were cirrhosis related to HCV and alcohol, 28.3% (IQR, 23.5–33.5) and 17.9% (IQR, 14.9–21.7), respectively, with another 4.5% (IQR, 2.3–7.9) suffering from both. The median MELD at transplant was 21.4 (IQR, 20.6–22.7) and 20.3 (IQR, 19.5–21.2) at listing (Table 1).

Liver transplant center and Donor Service Area characteristics

Waitlisted patients (11.0%; IQR. 6.6–16.8) had less than a high school education. The median CCI score for liver transplant centers was 73.6 (IQR, 67.6–75.0; Table 1), with all centers scoring <90, the cutoff considered easy to understand and the goal of the CDC (Figure 1).

Website Clear Communication Index (CCI) Score of adult liver transplant Centers by United Network for Organ Sharing (UNOS) region. Each vertical line represents a liver transplant center.

DSA Characteristics

Fifty-one DSAs with active liver transplant centers were included in the analysis. General population (13.6%; IQR, 11.5–18.8) had low EA. The median HHI of the DSAs was 50.1 (IQR, 26.7–69.3.3; Table 1). There were no differences between centers when stratified by 50th percentile CCI (Table 2).

Unadjusted comparison of liver transplant center and Donor Service Area characteristics stratified by Clear Communication Index score

Percentage of the General Population With Low EA in the DSA Is Not Associated With Liver Transplant Website CCI Score

To determine if CCI scores simply reflected an underlying association with the EA of the general population, we performed an analysis and found that there was no association between liver transplant website CCI scores and the percent of the general population in the DSA with low EA (P = 0.93; Figure 2).

Percentage of the general population with low educational attainment (EA) in the Donor Service Area (DSA) is not associated with liver transplant website Clear Communication Index (CCI) score.

Easier to Understand Website Educational Materials Associated With Greater Percent of Low EA Patients on the Waitlist

Multivariate linear regression demonstrated that EA of the general population correlated with percent of low EA of the waitlisted patients at each center (P < 0.05). Independent of that effect, CCI was also significantly associated with the proportion of waitlisted patients with low EA (P < 0.05). For every 1-point increase in CCI, the percent of patients on the waitlist with low EA increased by 0.2. If transplant centers were able to attain the easy to understand threshold on the CCI, the proportion of waitlisted patients with low EA would have increased by 3.6%. This translates to >3000 additional low EA patients could have been listed during the study period or nearly 3% the number of patients waitlisted for liver transplantation during that time.29 Increased market competition (lower HHI) did not correlate with a larger percentage of patients on the waitlist with low EA (P > 0.05; Table 3).

Adjusted analysis of factors independently associated with access to the liver transplant waitlist for patients with low educational attainment


We found that liver transplant centers with strong OHL, as measured by their website readability, have a larger percentage of patients with low EA on their waitlists, even after accounting for center, health system, and general population characteristics. The measurement tool we used to assess website readability goes beyond traditional readability formulas that assign a 1-dimensional reading level to patient education material.30-32 The CDC has reasoned such formulas to be inadequate to assess effectiveness of communication. Unlike readability formulas, the CCI was developed by the CDC to consider the audience, purpose, and communication characteristics that contribute to clarity and comprehension of print material.17

It is possible that our findings reflect centers serving areas with more vulnerable populations developing easier to understand websites in response to the needs of the population. However, we assessed the EA of the general population in each DSA and found no association between the population EA and website CCIs in the respective DSAs (Figure 2). In addition, website CCI impacted waitlist accessibility for low EA patients even after adjusting for the EA of the general population. This further suggests that it is not primarily the general population composition driving waitlist composition or website understandability.

Our study has multiple limitations. A major limitation of our study is the use of website CCI as a measure of OHL, rather than a comprehensive review of the institution’s written materials, in-person education, and navigability. On the patient side, our study is similarly limited by the use of EA as a measure of patient health literacy, rather than an actual examination of patients by interview and validated survey, together with patient perceptions and subsequent reasons for failing to be listed. While this could be done at a single center, followed by a concerted improvement in the components of OHL and observation of whether patient access changed in response to institutional change, the results would have limited generalizability. To perform such a study on a national level is not feasible, and so the use of surrogate measures and metrics was a necessary trade-off. An additional limitation is that our regression model includes data from 2007 to 2016, while OHL was measured using the websites available in 2016. However, we are using website CCI not as a discreet event having direct impact on patient access, but rather as a barometer of institutional culture and its health literacy. Furthermore, there was no change in the percentage of low EA patients in >95% of the centers over the 10-year study period (data not shown). In addition, there is some contribution to differences in the waitlist EA from regional differences in population education that are likely not related to center practice. However, we adjusted for baseline population differences in EA and found CCI score of transplant center websites still to be a significant independent predictor. Finally, we did not explore other barriers to access, such as website components of cultural literacy, or distance.33

Previous work in transplantation has focused on the negative impact of limited patient health literacy on health outcomes, modifying which can be beyond the scope of a transplant program.34-36 This is the first study investigating the potential impact of health system literacy, which is more easily modifiable by the transplant center. Our system-level findings support a national movement to improve the quality of health communication as none of the 112 liver transplant center websites assessed met the CDC threshold for easy to understand. The concept of system design has been previously championed by the Institute for Healthcare Improvement in that every system is perfectly designed to get the results it gets.37 Ideally, if systems were redesigned to enhance OHL, all patients—irrespective of their health literacy level or EA—would have improved access to care.

In addition, providing educational materials that are easier to understand has been demonstrated to improve quality of care and patient satisfaction in multiple healthcare settings. In the oncology setting, a randomized trial demonstrated that easier to read clinical trial consent forms reduce patient anxiety and yield higher patient satisfaction compared with standard consent forms.38 Extrapolating from other fields beyond healthcare, we can see that system-level improvements tailored to the end user can improve the likelihood of favorable outcomes. For example, brands that present technical information on websites in nontechnical terms secure more business.39 It is well known that simpler online purchase decision journeys and perceived ease in website usability are associated with increased consumer satisfaction, trust, and loyalty to the product.39,40 System redesigns can improve the quality of care, and systems are more easily modified than patients. Our data from this current study demonstrate that a natural and important next step will be to prospectively assess how transplant center website improvements impact access for low health literacy individuals and whether these improvements increase waitlist enrollment.

In conclusion, reducing the health literacy burden of a health system is associated with improved access for vulnerable patients. Health systems should recognize patient health literacy limitations and work to improve the understandability of written patient education materials.


1. Agency for Healthcare Research and Quality. United States Department of Health and Human Services. Available at Published 2005. Accessed September 29, 2017.
2. Glance LG, Kellermann AL, Osler TM, et al. Impact of risk adjustment for socioeconomic status on risk-adjusted surgical readmission rates. Ann Surg. 2016;263:698–704.
3. Julapalli VR, Kramer JR, El-Serag HB. Evaluation for liver transplantation: adherence to AASLD referral guidelines in a large Veterans Affairs Center. Liver Transplant. 2005;11:1370–1378.
4. Goldfarb-Rumyantzev AS, Sandhu GS, Baird BC, et al. Social Adaptability Index predicts access to kidney transplantation. Clin Transplant. 2011;25:834–842.
5. Burgard SA, Hawkins JM. Race/ethnicity, educational attainment, and foregone health care in the United States in the 2007-2009 recession. Am J Public Health. 2014;104:E134–E140.
6. Scaglione S, Kliethermes S, Cao G, et al. The epidemiology of cirrhosis in the United States: a population-based study. J Clin Gastroenterol. 2015;49:690–696.
7. Bababekov YJ, Ven Fong Z, Chang DC, et al. Is liver transplant education patient-centered? Liver Transpl. 2017;23:1070–1072.
8. The Joint Commission. “What Did the Doctor Say?”: Improving Health Literacy to Protect Patient Safety. 2007.
9. Brach C, Keller D, Hernandez LM, et al. Attributes of a Health Literate Organization. 2012Institute of Medicine of the National Academies.
10. Brega AG, Freedman MA, LeBlanc WG, et al. Using the Health Literacy Universal Precautions Toolkit to improve the quality of patient materials. J Health Commun. 2015;20Suppl 269–76.
11. Mabachi NM, Cifuentes M, Barnard J, et al. Demonstration of the Health Literacy Universal Precautions Toolkit: lessons for quality improvement. J Ambul Care Manage. 2016;39:199–208.
12. Cober RT, Brown DJ, Levy PE, et al. Organizational web sites: web site content and style as determinants of organizational attraction. Int J Sel Assess. 2003;11:158–169.
13. Braddy PW, Meade AW, Kroustalis CM. Organizational recruitment website effects on viewers’ perceptions of organizational culture. J Bus Psychol. 2006;20:525–543.
14. Catanzaro D, Moore H, Marshall TR. The impact of organizational culture on attraction and recruitment of job applicants. J Bus Psychol. 2010;25:649–662.
15. Adler JT, Yeh H, Markmann JF, et al. Market competition and density in liver transplantation: relationship to volume and outcomes. J Am Coll Surg. 2015;221:524–531.
16. Scientific Registry of Transplant Recipients. Program-specific reports. Available at Published 2015. Accessed August 1, 2016.
17. Centers for Disease Control and Prevention. The CDC Clear Communication Index. U.S. Department of Health and Human Services. Available at Published 2016. Accessed April 24, 2017.
18. Wiesner R, Edwards E, Freeman R, et al. Model for End-Stage Liver Disease (MELD) and allocation of donor livers. Gastroenterology. 2003;124:91–96.
19. Kim WR, Lake JR, Smith JM, et al. OPTN/SRTR 2015 Annual Data Report: liver. Am J Transplant. 2017;17Suppl 1174–251.
20. Ratzan SC, Parker RM. Selden CR, Zorn M, Ratzan SC, Parker RM. Introduction. National Library of Medicine Current Bibliographies in Medicine: Health Literacy. NLM Pub. No. CBM 2000-1. 2000. Bethesda, MD:National Institutes of Health, United States Department of Health and Human Services.
21. Kutner M GE, Jin Y, Paulsen C. The Health Literacy of America’s Adults: Results From the 2003 National Assessment of Adult Literacy (NCES 2006-483). 2006. Washington, DC:National Center for Education Statistics.
22. Ryan CSJ. Educational Attainment in the United States:2009. 2012. Washington, DC:U.S. Census Bureau.
23. United States Census Bureau. Educational attainment in the United States: 2016. United States Department of Commerce. Available at Published 2017. Updated March 31, 2017. Accessed August 7, 2017.
24. Winkleby MA, Jatulis DE, Frank E, et al. Socioeconomic status and health: how education, income, and occupation contribute to risk factors for cardiovascular disease. Am J Public Health. 1992;82:816–820.
25. Molla MT, Madans JH, Wagener DK. Differentials in adult mortality and activity limitation by years of education in the United States at the end of the 1990s. Popul Dev Rev. 2004;30:625–646.
26. Freudenberg N, Ruglis J. Reframing school dropout as a public health issue. Prev Chronic Dis. 2007;44A107.
27. Adler JT, Yeh H, Markmann JF, et al. Is donor service area market competition associated with organ procurement organization performance? Transplantation. 2016;100:1349–1355.
28. Edwards EB, Harper AM, Hirose R, et al. The impact of broader regional sharing of livers: 2-year results of “Share 35”. Liver Transpl. 2016;22:399–409.
29. Organ Procurement and Transplantation Network. National data. Waiting list additions age by listing year. Available at Accessed September 7, 2018.
30. Storino A, Castillo-Angeles M, Watkins AA, et al. Assessing the accuracy and readability of online health information for patients with pancreatic cancer. JAMA Surg. 2016;151:831–837.
31. Morony S, Flynn M, McCaffery KJ, et al. Readability of written materials for CKD patients: a systematic review. Am J Kidney Dis. 2015;65:842–850.
32. Rodrigue JR, Feranil M, Lang J, et al. Readability, content analysis, and racial/ethnic diversity of online living kidney donation information. Clin Transplant. 2017;319.
33. Goldberg DS, French B, Forde KA, et al. Association of distance from a transplant center with access to waitlist placement, receipt of liver transplantation, and survival among US veterans. JAMA. 2014;311:1234–1243.
34. Grubbs V, Gregorich SE, Perez-Stable EJ, et al. Health literacy and access to kidney transplantation. Clin J Am Soc Nephrol. 2009;4:195–200.
35. Miller-Matero LR, Hyde-Nolan ME, Eshelman A, et al. Health literacy in patients referred for transplant: do patients have the capacity to understand? Clin Transplant. 2015;29:336–342.
36. Cavanaugh KL, Wingard RL, Hakim RM, et al. Low health literacy associates with increased mortality in ESRD. J Am Soc Nephrol. 2010;21:1979–1985.
37. Mitchell K. Like magic? (“Every system is perfectly designed…”). Institute for Healthcare Improvement. Improvement Blog Web site. Available at Accessed July 27, 2017.
38. Coyne CA, Xu R, Raich P, et al. Randomized, controlled trial of an easy-to-read informed consent statement for clinical trial participation: a study of the Eastern Cooperative Oncology Group. J Clin Oncol. 2003;21:836–842.
39. Spenner P, Freeman K. To Keep Your Customers, Keep It Simple (Harvard Business Review). 2012Harvard Business School Publishing.
40. Flavian C, Guinaliu M, Gurrea R. The role played by perceived usability, satisfaction and consumer trust on website loyalty. Inform Manage. 2006;43:1–14.
Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.