A key target of quality improvement in health care is patient experience. Indeed, federal regulations (42 CFR §422.152(c) and §422.152(d)) require a Medicare Advantage (MA) organization’s quality improvement program to include a quality improvement project that measures and demonstrates improvement in beneficiary satisfaction. But what specific aspects of patient experience should such a program target in order to improve overall patient experience? A substantial body of research suggests that patient experiences with various aspects of care are relevant, including doctor communication, access to care, and coordination of care.1–4
Although all of these experiences are important, understanding what matters most to patients is critical to efficient and effective quality improvement. One way to do so is to test for associations between self-reported experience and overall ratings of care. In the handful of studies that have done so, doctor communication emerges as the most important predictor.2,5
However, focusing on a patient population as a whole may obscure key differences among subpopulations. For example, one prior study5 observed substantial variation in the predictors of rating of care according to type of hospitalization (defined by diagnostic category and service line). Patient demographic characteristics may also be associated with variation in the experiences most closely associated with overall patient experience,6 including race/ethnicity.
Racial, ethnic, and language subgroups differ in their health care-related beliefs, practices, and values.7 Some of these subgroups experience substantial disparities in health care access, utilization, and quality.8 Comparison of patient experience measures also reveals key differences by race/ethnicity. African Americans and Hispanics report different experiences with care than whites in some studies.9–14 Differences in patient experience according to language preference have also been found.14–16 Addressing these differences has become a central goal of US health policy.17
If, however, these subgroups differentially value various aspects of the care experience, improvement efforts must take this into account. For example, recent strides have been made in improving the experience most important to patients overall: provider-patient communication.8 But if communication is not of central importance to a minority group this change will have had little impact on that group, and increased disparities could occur even as it improved quality overall. Cultural differences between racial, ethnic, and language-preference groups in their beliefs about care and its delivery make such a scenario plausible, suggesting that some subgroups may weigh particular experiences more heavily than others in assessing their care overall.18
Recently, Paddison et al19 tested for racial and ethnic differences in the English General Practice Patient Survey. Similar to prior work, a measure reflecting provider communication was the strongest predictor of overall satisfaction with care among 6 patient experience measures as well as the most important predictor of overall satisfaction within each racial/ethnic subgroup studied. Although Paddison and colleagues did not find evidence of differential weighing of experiences by race/ethnicity, their data were collected in England and provide limited information about the largest minority groups in the United States, whose make-up, background and experiences are likely to differ markedly from British minorities. The differential organization and financing of health care in England may also have affected findings.
If there are racial, ethnic or language differences in the domains of care that drive overall patient experiences, it would be particularly important to identify them among older adults. Older adults have high levels of health care need and utilization. Thus, meeting their needs is of particular interest to overall quality improvement. Moreover, older adults may be reluctant to express their preferences to health care providers and prone toward deference to these providers, relative to younger patients.20,21 Thus, it may be necessary to illuminate the preferences of older adults through research in order to improve their experiences of care.
In this study, we examine data from Medicare beneficiaries 65 and older, testing for strength of associations between overall care ratings and measures tapping 5 major domains of patients’ experiences with their health care4,22 and testing for differences in the strength of the association of each domain by race, ethnicity, and survey language. Patient experience is central to health care quality.23 Identifying any racial/ethnic or language-based differences in the importance of particular domains would enable the tailoring of service improvements to a particular patient mix, potentially reducing disparities and more efficiently enhancing the quality of care overall.
Data come from the 2013 implementation of the Medicare Consumer Assessment of Healthcare Providers and Systems (MCAHPS) survey, a large-scale cross-sectional survey that collects information from Medicare beneficiaries on their health care experiences. Four survey versions specific to coverage type were used in 2013. Here, we used data from the 274,517 beneficiaries, age 65 or older and residing in the 50 states or DC, who responded to surveys asking about experiences with medical care in an MA health plan or in Medicare Fee-for-Service (FFS), with a response rate of 46.7%. Analytic weights that adjust for probability of selection, propensity to respond, and poststratification to match the Medicare population are used in all analyses.
Data collection was by mail with phone follow-up for nonrespondents. Spanish-language and Chinese-language surveys were sent to beneficiaries who had expressed to their MA plan a preference for Spanish-language or Chinese-language material, respectively; all other beneficiaries were sent English-language surveys. Language of telephone follow-up was similarly targeted. Any MA beneficiary could request a survey in any of the 3 available languages; only Spanish and English were available for FFS beneficiaries. Chinese surveys were offered by <1% of participating MA plans.
Overall Rating and Other Measures of Experience of Care
Beneficiaries evaluated their health care in the last 6 months with a single-item global rating: “Using any number from 0 to 10, where 0 is the worse health care possible and 10 is the best health care possible, what number would you use to rate all your health care in the last 6 months?” Five composite scales tapped patient experiences with care in the last 6 months in these domains: doctor communication (4 items), getting needed care (2 items), getting care quickly (3 items), customer service (3 items), and care coordination (6 items) (see Table A1 for full text of component survey items and response options). For example, patients report how often wait times were 15 minutes or less on a 4-point scale, with options ranging from “never” to “always.” The development, reliability, and validity2,22 of these composite scales have been described elsewhere.
We created a measure of race, ethnicity, and language (hereafter “REL”) using self-reported race, Hispanic ethnicity, and language of survey response. We defined 5 mutually exclusive subgroups: Hispanics responding in English, Hispanics responding in Spanish, non-Hispanic whites (hereafter “whites”), African Americans, Asians/Pacific Islanders (hereafter Asian/PI). Three other groups were included in the models for completeness but due to limited samples sizes, differential associations between care domains and overall ratings of care are not tested for these other groups: Native Americans/Alaska Natives, those of multiracial background, as well as a group not reporting their racial/ethnic background. Because of small numbers of non-Hispanic beneficiaries completing the CAHPS survey in Spanish and small numbers of participants of any race completing CAHPS in a Chinese language, all groups other than Hispanics were restricted to those responding to the survey in English.
We restricted our analytic sample to those who self-reported key variables, including age, at least one of the composite patient-experience measures, and the rating of care outcome. This restriction omitted 11.3% of the total sample. Further omitting cases that did not fall into one of the 5 main or additional 3 REL subgroups dropped an additional 0.3% of the sample for an analytic N of 242,782.
To deal with other missing data, we followed the approach of O’Malley et al24 and used observed data to make inferences about the expected experiences of those who did not answer all items, including items for which the respondent did not have the experience that would have triggered a particular item during the reference period. This approach avoids the bias that would otherwise result from restricting analyses to the highest utilizers who triggered all items. For instance, respondents who had not tried to get information from customer service in the last 6 months appropriately skipped the items in the customer service composite. Less frequently, some respondents skipped items that they were eligible to answer. We imputed missing values of the scales using SAS PROC MI under the missing at random assumption.24
We estimated a linear model where the outcome was rating of care and regressors were indicators of REL group (using white as the reference category), the 5 experience-composite domain measures, and the REL by experience-composite domain interaction terms. We included standard CAHPS case-mix adjusters25 (age, highest level of education, self-reported general and mental health rating, proxy status, and dual eligibility for Medicaid (a low-income indicator)), as well as health plan (MA contract, treating FFS as one level) and location (Health Referral Region).
We assessed evidence of heterogeneity of associations between experience-composite domains and rating of care across REL groups using a joint test of interactions. We limited this test to the 5 largest REL groups (ie, we jointly tested the 20 interaction terms between each of the 5 experience-composites domains and 4 largest minority groups Hispanic-English, Hispanic-Spanish, African American, and Asian/PI with white as the reference group). We assessed evidence of heterogeneity among the 5 REL groups of interest for each of the 5 domains with a joint test of the 4 REL interaction terms with that domain.
Before use in modeling, all patient experience measures were standardized to have a mean of 0 and a standard deviation of 1. We summed corresponding coefficients from the composite indictors and their interactions with the REL indicators to create standardized partial correlation coefficients between overall rating and each composite for each of the 5 major REL groups. Here, main effects of composites represent the coefficients for the reference non-Hispanic white group.
Table 1 shows the characteristics of the sample and the size of the REL groups. Whites made up 80% of the sample, African Americans 7%, Asian/PIs 3%, Hispanic English-preferring participants 5%, and Hispanic Spanish-preferring participants 1%. More than half the sample was under age 75. Self-rated health was “very good” or “excellent” among 38% of the sample.
A summary of results from the regression model appears in Table 2. The 5 experience composite domains account for a considerable amount of the variance in patient overall ratings of care. For non-Hispanic whites, the domain with the strongest association with overall quality was doctor communication. The getting care quickly and getting needed care domains also had substantial associations with overall ratings of care and these associations were of similar strength. The association between the care coordination and customer service domains and overall rating of care was also significant, but substantially weaker than the associations found with the other 3 domains.
The joint test of the 20 interactions between the composite scores and the 5 largest REL groups was significant (P<0.0001), indicating that the importance of these domains of patient experience (strength of the associations) varied across the REL groups. The joint tests between the 5 largest REL groups and 2 of the 5 composites were not statistically significant (getting needed care P=0.68 and customer service P=0.17), indicating that there is no evidence of heterogeneity in associations of these 2 composites with overall rating of care across subgroups. However, there was evidence of differential associations between the other 3 composites and care-ratings across groups: doctor communication (P=0.001), getting care quickly (P=0.008), and care coordination (P=0.009). The parameter estimates and significance tests in Table 2 indicate that all domains had statistically significant associations with care ratings in all REL groups with 2 exceptions: doctor communication and care coordination did not have statistically significant associations with the overall care ratings of Hispanics surveyed in Spanish.
There were other variations across patient subgroups, as well. Figure 1 depicts these variations for each experience domain within each of the 5 largest REL subgroups. The profiles for whites and Hispanics surveyed in English are fairly similar—doctor communication had the strongest association with care ratings for both of these groups. But for African Americans, doctor communication did not stand out in importance, instead associations with overall care were similar for doctor communication and all other domains except customer service (which had a much weaker relationship to overall care rating). And among Asians/PI respondents, doctor communication was less important than other domains. Among Hispanics surveyed in Spanish, getting needed care and getting care quickly were the most important domains and neither doctor communication nor care coordination had a statistically significant association with care ratings for this group. Looking across all REL groups, the 2 domains that had the most consistent and strong relationships with overall ratings of care were getting needed care and getting care quickly. The importance of customer service was consistent as well, but fairly modest relative to other factors.
Figure 2 shows these same associations but maps differences by REL in the strength of the association of each composite experience domain with overall rating of care. Although the relative importance of different domains appears similar for whites and Hispanics surveyed in English, the association between overall care ratings and getting care quickly is somewhat weaker among Hispanics surveyed in English. Doctor communication was less strongly related to rating of care for Hispanics surveyed in Spanish, African Americans, and Asians/PIs, relative to whites. For Asians/PIs, the getting care quickly domain had a weaker association with overall rating of care than for whites. For African Americans, the care coordination and customer service domains had a stronger association with overall rating of care than for non-Hispanic whites.
We find substantial differences in the domains most strongly associated with patients’ ratings of their health care depending on their REL. These diverse patterns have important implications for quality improvement efforts and suggest that addressing any one aspect of patient experience will not improve overall patient experience to the same extent across all REL subgroups. The domain with the most consistent relationship with overall ratings was customer service, but its association was consistently weak, making it a poor candidate for intervention. Improvements in doctor communication have the greatest potential to improve patient experience among whites, English-preferring Hispanics, and African Americans. However, targeting this aspect of patient experience may have a modest effect on patients from Asian/PI backgrounds and may have none at all for Hispanics whose language preference is Spanish, given the relatively weak associations found here for these groups. Quality improvement efforts will need to pay attention to this diversity. Where this is not possible, our data suggest that the greatest improvement for the broadest set of patients is likely to be achieved by addressing access issues.
Getting needed care and getting care quickly were among the domains with the strongest associations with care ratings for all patient subgroups. Increases in the percentage of individuals covered by insurance under the Affordable Care Act may facilitate in getting needed care and help to address this aspect of patient experience. For those who have insurance but still have difficulty getting the care they need, physician shortages may need to be addressed. Where provider shortages are not a problem, more innovative solutions may be needed, such as telehealth visits26 or systems engineering solutions27 to decrease waiting times for appointments.
Although across-the-board interventions may be helpful in improving patient experience, the varying associations we observed between composite measures and overall care ratings by REL suggest that there are cultural differences in beliefs or expectations regarding aspects of care,28 and that tailoring quality improvement interventions based on patient characteristics may have greater utility than a one-size-fits-all approach. For example, for whites and Hispanics surveyed in English, existing efforts to improve communication such as communication training for providers may be most effective.29 For Asian/PI patient populations and Hispanics with a Spanish language preference, some of the interventions to improve access noted above may be more likely to improve their overall patient experiences. For African Americans, a multi-pronged approach addressing communication, access, and coordination of care may be needed.
It is important to place our findings in the context of the large body of literature addressing the importance of improving doctor communication for REL minorities. Several studies suggest that nonwhite patients are less active participants in conversations with their doctors.30–34 One study found that African Americans asked fewer questions, expressed fewer concerns, and made fewer assertions than whites, and doctors, in response, offered African Americans less information related to their care.35 Concordance between patient REL and that of physicians, improving the respect doctors show patients, and increasing patients’ willingness to disclose important details of their lives may be central to optimizing technical quality of care as well as the experience of all patient subgroups. It is also important to note that the specific aspects of doctor-patient communication that are important to patient experience also appear to differ across REL subgroups.36 Efforts that focus on improving communication may benefit from further tailoring to address this difference.
Our finding regarding language of survey administration is consistent with results from Seid et al.37 These authors found that differences in Latinos and whites care experiences could be accounted for by language spoken. Our results show strong differences in the importance of doctor communication between Hispanics surveyed in Spanish and whites, but we found no such differences between Hispanics surveyed in English and whites.
Our overall results are quite different from those of another major study that has examined associations between specific measures of patient experience and overall patient assessments of care. Paddison et al19 found few differences in features associated with satisfaction across several racial/ethnic groups in England. There are numerous differences between our study and theirs that might account for this, including differences in the racial/ethnic subgroups and cultural context of race in the United States versus England, differences in the systems of care in the 2 countries, and differences in age.
This study has several limitations. Response rates were moderate and it is unknown whether the same associations would be observed among nonrespondents. We cannot attribute a causal role to the patient experiences measured, given the cross-sectional and observational nature of the design. Thus, tailored quality improvement interventions based on these data may not improve overall ratings of care. The use of the term “doctor” in the 2013 CAHPS survey limits our results regarding communication to this subset of healthcare providers. Also, we were unable to reliably test for associations between composite experience measures and ratings of care in the smaller REL groups that are part of the Medicare population—those who took the survey in a language other than English or Spanish, Native Americans/Alaska Natives, and those who identify as multiracial.
Nonetheless, our results provide insights into the manner in which health care is experienced by the largest REL groups in the Medicare population.
Finally, the mechanism for some patterns we observed is not clear, particularly differences by language preference. It is unclear why communication and care coordination were not strongly associated with overall care ratings among Spanish preferring Hispanics. It may be that the presence or absence of a language concordant provider or a translator may dominate doctor communication for this group. Spanish language preference is probably also a marker for unmeasured factors such as concordance between the doctor and patient in language or ethnicity, use of a translator during appointments, nativity, or acculturation. For example, expectation regarding care coordination may vary by nativity, which might explain its weak association with overall experiences for Spanish-preferring Hispanics. Unfortunately, these factors are not included in the CAHPS survey, limiting our ability to determine which, if any, of them, account for differential patterns of associations for the Spanish-preferring subgroup.
Our findings indicate that policies to improve patient experience may be more effective if tailored to the patient population at a given practice or hospital. Efforts based on findings for the US population as a whole will primarily address the needs of the majority (whites) and lead to inefficiencies in practices with predominantly minority patients. Moreover, such efforts may have the unintended effect of increasing disparities in overall quality of care.
The authors would like to thank Fergal McCarthy, MPhil, and Biayna Darabidian, BA. for preparation of the manuscript.
1. Doty MM, Fryer AK, Audet AM. The role of care coordinators in improving care coordination: the patient's perspective. Arch Intern Med. 2012;172:587–588.
2. Hays RD, Martino S, Brown JA, et al. Evaluation of a care coordination measure for the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Medicare survey. Med Care Res Rev. 2014;71:192–202.
3. Peikes D, Chen A, Schore J, et al. Effects of care coordination on hospitalization, quality of care, and health care expenditures among Medicare beneficiaries: 15 randomized trials. JAMA. 2009;301:603–618.
4. Kontopantelis E, Roland M, Reeves D. Patient experience
of access to primary care: identification of predictors in a national patient survey. BMC Fam Pract. 2010;11:61.
5. Martino SC, Elliott MN, Cleary PD, et al. Psychometric properties of an instrument to assess Medicare beneficiaries' prescription drug plan experiences. Health Care Financ Rev. 2009;30:41–53.
6. Elliott MN, Lehrman WG, Beckett MK, et al. Gender differences in patients' perceptions of inpatient care. Health Serv Res. 2012;47:1482–1501.
7. Betancourt JRG, Alexander R, Carrillo JE, et al. Defining cultural competence: a practical framework for addressing racial/ethnic disparities
in health and health care. Public Health Rep. 2003;118:294–302.
9. Dayton E, Zhan C, Sangl J, et al. Racial and ethnic differences in patient assessments of interactions with providers: disparities or measurement biases? Am J Med Qual. 2006;21:109–114.
10. Lurie N, Zhan C, Sangl J, et al. Variation in racial and ethnic differences in consumer assessments of health care. Am J Manag Care. 2003;9:502–509.
11. Morales LS, Elliott MN, Weech-Maldonado R, et al. Differences in CAHPS adult survey reports and ratings by race and ethnicity: an analysis of the National CAHPS benchmarking data 1.0. Health Serv Res. 2001;36:595–617.
12. Weech-Maldonado R, Elliott MN, Morales LS, et al. Health plan effects on patient assessments of Medicaid managed care among racial/ethnic minorities. J Gen Intern Med. 2004;19:136–145.
13. Weech-Maldonado R, Elliott MN, Oluwole A, et al. Survey response style and differential use of CAHPS rating scales by Hispanics. Med Care. 2008;46:963–968.
14. Weech-Maldonado R, Morales LS, Elliott M, et al. Race/ethnicity, language, and patients' assessments of care in Medicaid managed care. Health Serv Res. 2003;38:789–808.
15. Elliott MN, Edwards WS, Klein DJ, et al. Differences by Survey Language
and Mode among Chinese Respondents to a CAHPS Health Plan Survey. Public Opin Q. 2012;76:238–264.
16. Weech-Maldonado R, Fongwa MN, Gutierrez P, et al. Language and regional differences in evaluations of Medicare managed care by Hispanics. Health Serv Res. 2008;43:552–568.
17. US Department of Health and Human Services. HHS action plan to reduce racial and ethnic health disparities: a nation free of disparities in health and health care. Washington, DC: US Department of Health and Human Services. 2011.
18. Weech-Maldonado R, Elliott M, Pradhan R, et al. Can hospital cultural competency reduce disparities in patient experiences with care? Medical Care. 2012;50 (suppl):S48–S55.
19. Paddison C, Abel G, Roland M, et al. Drivers of overall satisfaction with primary care: evidence from the English General Practice Patient Survey. Health Expect. 2015;18:1081–1092.
20. Belcher VN, Fried TR, Agostini JV, et al. Views of older adults on patient participation in medication-related decision making. J Gen Intern Med. 2006;21:298–303.
21. Levinson W, Kao A, Kuby A, et al. Not all patients want to participate in decision making: a national study of public preferences. J Gen Intern Med. 2005;20:531–535.
22. Hargraves JL, Hays RD, Cleary PD. Psychometric properties of the Consumer Assessment of Health Plans Study (CAHPS) 2.0 adult core survey. Health Serv Res. 2003;38:1509–1527.
23. Anhang Price R, Elliott MN, Zaslavsky AM, et al. Examining the role of patient experience
surveys in measuring health care quality. Med Care Res Rev. 2014;71:522–554.
24. O’Malley AJ, Zaslavsky AM, Hays RD, et al. Exploratory factor analyses of the CAHPS Hospital Pilot Survey responses across and within medical, surgical, and obstetric services. Health Serv Res. 2005;40:2078–2095.
25. Elliott MN, Zaslavsky AM, Goldstein E, et al. Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores. Health Serv Res. 2009;44:501–518.
26. Ashwood JSM, Mehrotra A, Cowling D, et al. Direct-to-consumer telehealth may increase access to care but does not decrease spending. Health Aff. 2017;36:485–491.
27. Gabow PAG, Philip L. The Lean Prescription: Powerful Medicine for Our Ailing Healthcare System. London, UK: CRC Press; 2014.
28. Sofaer S, Firminger K. Patient perceptions of the quality of health services. Ann Rev Public Health. 2005;26:513–559.
29. Kennedy DMF, John P, Gullen DJ. Improving the patient experience
through provider communication skills building. Patient Exp J. 2014;1:56–60.
30. Gordon HS, Street RL Jr, Kelly PA, et al. Physician-patient communication following invasive procedures: an analysis of post-angiogram consultations. Soc Sci Med. 2005;61:1015–1025.
31. Johnson RL, Roter D, Powe NR, et al. Patient race/ethnicity and quality of patient-physician communication during medical visits. Am J Public Health. 2004;94:2084–2090.
32. Street RL Jr, Gordon HS, Ward MM, et al. Patient participation in medical consultations: why some patients are more involved than others. Med Care. 2005;43:960–969.
33. Ward MM, Sundaramurthy S, Lotstein D, et al. Participatory patient-physician communication and morbidity in patients with systemic lupus erythematosus. Arthritis Rheum. 2003;49:810–818.
34. Young M, Klingle RS. Silent partners in medical care: a cross-cultural study of patient participation. Health Commun. 1996;8:29–53.
35. Gordon HS, Street RL Jr, Sharf BF, et al. Racial differences in doctors' information-giving and patients' participation. Cancer. 2006;107:1313–1320.
36. Quigley DD, Elliott MN, Farley DO, et al. Specialties differ in which aspects of doctor communication
predict overall physician ratings. J Gen Intern Med. 2014;29:447–454.
37. Seid M, Stevens GD, Varni JW. Parents' perceptions of pediatric primary care quality: effects of race/ethnicity, language, and access. Health Serv Res. 2003;38:1009–1031.