Rudolph, Linda MD, MPH; Dervin, Kathy MPH; Cheadle, Allen PhD; Maizlish, Neil PhD, MPH; Wickizer, Tom PhD
Patient satisfaction and patient perceptions of outcomes have become important components in the assessment of the quality of health care. 1 Patients are uniquely able to provide information about their ease or difficulty of obtaining care, the interpersonal dimensions of the patient-physician relationship, the patient’s view of the technical quality of care provided, and the patient’s functional status and perceived well-being. Patients can provide both subjective ratings of care and more factual reports about what happened in their medical encounters. 2
State and federal government agencies (eg, Centers for Medicare and Medicaid Services), widely recognized accrediting agencies (eg, National Committee for Quality Assurance), and private and public purchasers (eg, California Public Employees Retirement System, Pacific Business Group on Health) now all require the collection of patient satisfaction data using the standardized Consumer Assessments of Health Plans Study surveys developed by the federal Agency for HealthCare Research and Quality. 3
There are no comparable requirements for assessing patient satisfaction among injured workers receiving care in any state workers’ compensation system. At least one instrument for assessing workers’ compensation care has now been tested for reliability and validity. 4 The American Accreditation HealthCare Commission/Utilization Review Accreditation Committee has also developed a patient satisfaction survey intended for use by workers’ compensation managed care organizations, but it remains untested. 5
Information about the experience of injured workers with health care after work injury is limited. There are few published reports of patient satisfaction in workers’ compensation, and several of these are evaluations of specific, limited-term managed care programs, rather than more general assessments of injured workers’ perceptions of care. 6,7 Recently, several state workers’ compensation agencies have also conducted surveys of injured workers as part of a Robert Wood Johnson Foundation-funded effort to test the feasibility of collecting comparable data across states to assess the performance of workers’ compensation systems. 8,9 This study reports on the results of a survey of more than 800 injured workers in California’s workers’ compensation system, to assess patient satisfaction with medical care and patient perceptions of health and functional outcome after work injury.
The California Division of Workers’ Compensation (DWC) contracted with the University of California, Berkeley, Survey Research Center (SRC) to develop a standardized self-administered questionnaire that could be used to collect data on patient satisfaction and outcomes in injured workers receiving care in California’s workers’ compensation system, and to conduct a pilot test of the survey instrument and mail-out procedure. Full details about survey development can be found in Wiley et al. 10
The SRC and DWC reviewed available patient questionnaires used in other settings. New questions were developed to assess key aspects of the treatment of occupational injury and illness, such as physician’s effort to elicit information about the worker’s job, physician understanding of the impact of the injury on ability to perform the job, and the extent of physician–patient communication about the patient’s return to work.
An ad hoc advisory committee of physicians and other health professionals, union representatives, academics, and injured workers advocates reviewed a draft of the survey. A revised draft of the instrument was evaluated in a focus group with injured workers.
The pilot questionnaire was mailed to a sample of 800 workers, randomly selected from the claims files or medical records of six cooperating organizations, which included three self-insured employers, a large workers’ compensation carrier, a health maintenance organization, and a large occupational health practice. Nearly 30% of workers returned the pilot survey. The SRC then conducted an intensive telephone follow-up to those workers who did not return the mail survey, obtaining a response rate of over 60%.
An analysis of characteristics and responses of both mail and phone respondents was conducted. Female and older workers were over-represented among mail respondents. There were no significant differences between mail and phone respondents with respect to subjective ratings of satisfaction with medical care or reports on the medical care experience. Phone respondents were slightly more likely to report better self-rated functional outcomes than mail respondents; this finding was largely, but not entirely, accounted for by the longer lag time between date of injury and date of response in phone respondents. SRC also assessed item nonresponse, validity of response, response variability, and item consistency. 10
On the basis of the results of the pilot survey, the questionnaire was again revised. Final survey domains included (1) postinjury health and functional status, (2) patient reports and evaluation of care, (3) utilization of medical services, (4) return to work, and (5) demographic and occupational characteristics of injured workers (Table 1).
The sampling frame was a convenience sample, including injured workers who (1) were enrolled in a state-certified workers’ compensation health care organization, (2) were enrolled in the state’s 24-hour pilot program, (3) obtained care at two large managed care organizations that contract with employers to provide fee-for-service workers’ compensation care, or (4) were employed at time of injury by five large self-insured employers.
Initial criteria for eligibility in the survey sample were (1) injured between July 1, 1997, and December 30, 1997;and (2a) had 3 or more days of lost time or received payment for temporary disability, or (2b) used medical services with total costs of more than $2500. These criteria were intended to allow for a uniform time frame from date of injury to date of survey and to ensure that respondents had experienced more than casual contact with the workers’ compensation medical system. Samples from each organization were selected at random, with sampling fractions varying based on the total number of cases contributed by each organization. Date criteria were subsequently relaxed to ensure adequate numbers of cases from all organizations.
The survey was administered by phone by the University of California Survey Research Center from February through July 1998, at an average length of 8 months after the date of injury. Interviewers used CASES (Computer Assisted Survey Execution System; University of California Berkeley. http://www.cases.berkeley.edu), a computerized automated telephone interview software system. Up to 10 attempts were made to contact each individual. Methods used to locate respondents included directory assistance, on-line reverse directories, and on-line searches with address inputs and phone number outputs. Monolingual Spanish-speaking respondents were referred for Spanish-language interviews; other non-English speaking respondents were excluded.
Simple descriptive and univariate analyses were performed. For the purpose of statistical analyses of overall satisfaction and choice of provider, the four-point satisfaction scales were dichotomized. A response of very satisfied or somewhat satisfied was classified as “satisfied” and a response of somewhat dissatisfied or very dissatisfied was classified as “dissatisfied.” Similarly, responses to questions with five-point scales were dichotomized so that responses of “excellent” or “very good” were classified as “very good,” and those of “good, fair, poor” were classified as “not very good.” An exploratory factor analysis, using orthogonal rotation, identified four factors (Table 2).
Scales for doctor–patient interaction, satisfaction, occupational medicine orientation, and physical function were constructed by adding up the variables in each scale coded as Likert scales (eg, 1 = strongly disagree to 5 = strongly agree). The codings were reversed as appropriate so that all variables loaded in the same direction on the factors and scales. Cut points were established to produce dichotomous variables (eg, satisfied or not) from each scale.
In addition to univariate analyses, general satisfaction with care was modeled using stepwise forward logistic regression. Variables for three different models were specified from the following lists of “satisfaction variables” and “nonsatisfaction variables.” Satisfaction-related variables included satisfaction with choice of provider, high doctor/patient interaction, satisfaction with claims handling, good functioning, satisfaction with job preinjury, good/excellent relations with supervisor, hired an attorney, and covered by insurance when injured. Nonsatisfaction variables included patient age, sex, education, race, marital status, language, income, injury type, occupation, time to survey interview, and employer helpful in assisting return to work. Model I considered for forward entry all 20 variables (satisfaction variables + nonsatisfaction variables). Model II considered only the group of 9 satisfaction variables for forward entry, and Model III considered only the 11 nonsatisfaction variables. The P value for inclusion in any model was P ≤ 0.05.
Response Rates and Respondent Characteristics
The overall response rate for the survey was 61.3%. According to limited available administrative data, respondents (mean, 41.2 years) were more likely to be older than nonrespondents (mean, 38.6 years). Both respondents and nonrespondents were approximately 60% women. Respondent characteristics are shown in Table 3.
Access and utilization.
A total of 13.3% of workers reported “some or a lot of trouble getting medical care” when they were first injured; 77% reported no trouble at all. Eighty-six percent reported that they told their employer about the injury before seeking medical care. Few respondents first saw a doctor at a private doctor’s office (Table 4).
Less than 20% of patients saw only one doctor for treatment of their injury, whereas one quarter of the patients saw five or more different doctors. Over 22% of injured workers made 25 or more doctor visits; 45% had fewer than 10 doctor visits for the work injury. Sixty-three percent of respondents said a medical doctor had provided most of the care for the injury. Fifteen percent had most of their care provided by a physical therapist, 6.5% by a chiropractor, 2% by a physician assistant or nurse practitioner, and the remaining by “another type of professional” or “unknown.” Workers were asked whether they had been told to see the one doctor who was most involved in treating the work injury, and by whom. Only 46% reported that someone had told them to see this doctor; of these, just over 50% said they had been referred to the doctor by the insurer or employer.
Noncompensated medical costs and use of other benefits.
A total of 127 (15.7%) respondents had paid $1 to $99 in nonreimbursed medical costs for their work injury; 46 (5.7%) reported paying $100 to $499, and 15 (1.9%) reported nonreimbursed medical costs of greater than $500. Over 42% of patients reported that they had used sick leave or vacation leave to cover time lost at work because of the injury.
Satisfaction and patient ratings of care.
Nearly 25% of injured workers reported dissatisfaction overall with the medical care received for their work injury or with the number of doctors they could choose from to treat their work injury (Table 5). Many patients rated as only fair or poor several aspects of communication, interpersonal interaction, and perceived technical competence of their primary provider (Table 6).
Patient reports on provider behaviors.
One third of respondents indicated that they were involved very little or not at all in decisions about their medical care. Many workers reported that physicians did not engage a lot in behaviors considered important in occupational medicine, such as eliciting a job description or talking about return to work or prevention of reinjury (Table 7).
The survey explored several aspects of postinjury work experience. Nearly all respondents (94%) had worked for pay at some time since the injury; 70% had missed no work. Over 44% said they had returned to work “too soon” after the injury; 23% of workers said their employers were “not at all” or “not too helpful” in helping them to return to work after the injury. Thirty-eight percent had job changes to help return to work after injury, and most (79%) of those with job changes were satisfied with the changes. Eighty-six percent of respondents stated that they were working at a regular job for pay at the time of interview, and 85% of these were working for the at-injury employer.
Pain and functional outcomes after injury.
Respondents were asked a series of questions about perceptions of current health status and pain levels and functional outcomes. A large number of workers reported significant continuing impacts of the injury: self-rated health worse now than before the injury, the injury continues to affect life today, and workers not feeling fully recovered from the injury (Table 8). Similarly, many workers reported significant levels of pain that interfere with their life or work (Table 9). Forty percent of workers with pain had seen a doctor in the past 4 weeks, and most of these (81%) reported that the doctors had been somewhat or very helpful with pain management.
Many workers also reported difficulty with ordinary activities such as lifting, climbing a flight of stairs, or handling objects; interestingly, workers did distinguish between functional difficulties they thought were due to the work injury versus those attributable to other reasons (Table 10). Workers also reported considerable impacts of injury on job performance (Table 11). However, less than 10% of current workers had cut down on their number of hours of work.
Satisfaction and patient characteristics.
Patient satisfaction with specific aspects of care and patient reports of physician behavior varied substantially among patients with different characteristics, although there were few significant differences among groups in overall satisfaction with care (Table 12). Respondents who were younger, Spanish-speaking, non-white, lower income, less educated, or laborers reported significantly lower satisfaction with doctor–patient interaction. Workers with back injury or upper extremity nerve damage were more likely to report physician behaviors consistent with an occupational medicine orientation, whereas male workers and monolingual Spanish-speakers were less likely to report occupational medicine orientation. Relationship with supervisor and attorney representation were not significantly associated with satisfaction with medical care in this study.
There were significant differences among organizations from which the sample was drawn (sample subgroups) with regard to reports of occupational medicine orientation and satisfaction with doctor–patient interaction; these differences became insignificant when demographic characteristics were controlled for, and there were no significant differences among sample subgroups with overall satisfaction with care or with choice of provider.
Outcomes and patient characteristics.
There were also significant self-reported differences in physical and emotional function, and amount of missed work, among workers with different characteristics (Table 13). Younger workers were more likely to report good physical functional outcomes. Caucasian respondents were more likely to report good emotional outcomes. Workers with upper extremity nerve damage were far less likely to report good functional outcomes; however, these workers were also less likely to miss any work as a result of their injury. Service workers and laborers, older workers, male workers, and Hispanic and monolingual Spanish-speakers were more likely to miss work.
Differences in sample populations.
There were very significant demographic differences among respondents drawn from different sample subgroups within the sampling frame (Table 14). For example, group C had a markedly higher proportion of Hispanic, Spanish-speaking, and “blue-collar” workers. Group D had higher education levels and more clerical workers. Injury types also varied greatly among groups, with group D reporting far more upper extremity nerve damage.
The stepwise forward regression models selected satisfaction-subscale variables as strong, independent predictors of general satisfaction with care (Table 15). In the model that considered both satisfaction and nonsatisfaction variables for forward entry, satisfaction with provider choice was very highly associated (odds ratio [OR], 15.7;P < 0.01) with overall satisfaction, as were high doctor/patient interaction (OR, 5.1;P <0.01) and satisfaction with claims handling (OR, 5.1;P < 0.01). To a lesser degree, good functional outcome (OR, 2.9;P < 0.01) and employer assistance in return to work (OR = 1.6;P = 0.03) were associated with general satisfaction. The results were very similar for the model that considered only satisfaction variables. In the third model, which considered nonsatisfaction variables, back injury type was negatively associated with general satisfaction (OR, 0.6, P = 0.03); that is, low back patients were less satisfied compared with other injury types.
An open-ended question (“Is there anything else you think we should know about the medical care you received after your work injury?”) elicited an outpouring of response, with nearly half of respondents providing additional comments (Table 16). Recurrent themes emerged in these comments, including desire for more choice of provider, particularly if dissatisfied with the treating physician; lack of continuity of care in clinic settings; lack of adequate time with the physician; frustration and anger about the claims handling process (especially delays or denials in authorizations of care, other benefit delays, lack of information about rules and rights, and being treated “like a criminal”); distrust of the “company doctor”; frustration with continued pain and functional limitations; concerns about lengthy treatments that did not produce improvement; and lack of availability of modified work. On the other hand, many workers also expressed great appreciation for the care they had received, often singling out particular individuals who had demonstrated care for them as a person or who had taken time to explain details. Many respondents also thanked the interviewer for the opportunity to talk about their experience after work injury.
The respondents represented a spectrum of workers, injury types, geographic locations, employers, and care delivery systems; however, the sample was not randomly drawn and may not be truly representative of injured workers or the California workers’ compensation health care system. Although our response rate was acceptable, the nature of response bias in surveys of injured workers is unknown. Previous analysis of the pilot test of the survey instrument suggests that there is no significant bias with respect to satisfaction with care but that respondents might be slightly more likely to report poorer recovery and functional outcomes than nonrespondents. 10 Further research on this issue in larger populations of injured workers would be helpful, particularly because an intensive effort was required to obtain a response rate over 60%.
There is little published information about the availability of or patterns of care for workers’ compensation patients. Nearly one quarter of respondents indicated at least a little difficulty in obtaining care, a finding consistent with anecdotal reports of access problems after work injury. 11 The large proportion of workers first receiving care in an emergency department suggests that improvements in referral to and use of acute care clinics might reduce the high costs of care associated with emergency department use. 12,13
Approximately one quarter of injured workers reported dissatisfaction with overall care and with the choice of providers. These findings are consistent with recent findings in surveys of injured workers in Minnesota and Florida. 8,9 Injured workers tended to report somewhat less overall satisfaction with care than enrollees in commercial or Medicaid managed care plans. 14 The high levels of dissatisfaction with care and with physician-patient communication are cause for concern. Satisfaction with care may impact care-seeking behavior, compliance with prescribed care, and changing of physicians. 15,16 The physician–patient relationship may be an important influence on patients’ health outcomes. 17
Nearly 40% of workers believed that their treating physician did not understand the impact of their injury on their ability to do their job; large proportions of workers also reported that the treating physician did not discuss return to work or prevention of reinjury. The importance of a focus on functional recovery and return to work in the treatment of common work-related injuries is now well recognized. 18 The survey results suggest, however, that there is substantial room for improvement in this aspect of workers’ compensation health care.
This introductory exploration suggests that injured worker satisfaction with care is rooted in the experience of care, interactions with health professionals, and perceived outcomes. However, factors not directly related to the quality of medical care (eg, satisfaction with claims handling) also seem to significantly affect workers’ satisfaction with care.
Injured workers have a unique perspective on the medical care they receive and on their recovery after injury. Patient satisfaction surveys are a key component in efforts to assess the value of health care services. Insight into workers’ perspective on quality of care can provide important information for health care providers and organizations, employers, claims administrators, purchasers, and policymakers. Such information can be used for health care quality improvement, negotiation of health care contracts, consumer and provider education, choice of providers, or provider network enhancement. For employers, workers’ perceptions of care and outcomes after injury may provide important insights into issues that affect workplace morale and productivity. For claims administrators, satisfaction with medical care may affect control over medical treatment and litigation rates. Managed care organizations can use survey information to target quality improvement efforts; patient surveys can identify areas in which organizational resources are needed to augment physician services (eg, enhanced case management services) or when more aggressive physician education and support may be warranted. Labor organizations can use patient survey results to advocate for focused improvements in care.
Assessment of injured worker satisfaction with care and outcomes after injury provides an important tool for monitoring and improving quality; it should be more routinely implemented in the workers’ compensation system.
The authors gratefully acknowledge the many individuals who helped in the development and administration of the survey, including Jim Wiley, PhD, Public Health Institute, and Madonna Camel, University of California, Berkeley, Survey Research Center. We also thank Jim Bellows, CIH, MPH, Kaiser-Permanente Care Management Institute, and Glenn Shor, PhD, Division of Workers’ Compensation, for technical assistance and support. We remember the late Karen Garrett with profound appreciation; this work could not have been done without her.
1. Agency for Health Care Policy and Research. Consumer Assessments of Health Plans Study (CAHPS®): Overview. Rockville, MD: AHCPR; 1998.
2. Ware JEJr, Snyder MK, Wright WR, Davies AR. Defining and measuring pa-tient satisfaction with medical care. Eval Program Plann. 1983; 6: 247–263.
3. Agency for Health Care Research and Quality. Consumer Assessment of Health Plans (CAHPS®). Fact Sheet. Rockville, MD: AHRQ; 2000.AHRQ publication no. 00-PO47.
4. Kyes KB, Franklin G, Weaver MR. Reliability and validity of medical outcome and patient satisfaction measures among injured workers in Washington State: a pretest. Am J Ind Med. 1997; 31: 427–434.
5. Utilization Review Accreditation Committee/American Accreditation HealthCare Commission. Measuring Quality in Workers’ Compensation Managed Care Organizations. Technical Manual of Performance Measures. Washington, DC: URAC/AAHCC; 2000.
6. Kyes KB, Wickizer TM, Franklin G, et al. Evaluation of the Washington State Workers’ Compensation Managed Care Pilot Project I: medical outcomes and patient satisfaction lsqb;see comments]. Med Care. 1999; 37: 972–981.
7. Borba PS, Appel D, Fung M. Workers’ Compensation Managed Care Pilot Project: Final Report to the Florida Legislature. Tallahassee, FL: Florida Department of Insurance: 1994.
8. MacDonald C. Measuring the Performance of the Minnesota Workers’ Compensation System. Summary Report of a Survey of Workers Injured in Minnesota between November 1998 and May 1999. Minneapolis, MN: Minnesota Department of Labor and Industry; 2000.
9. Garrett-James B. An Assessment of Florida’s Workers’ Compensation System through a Survey of Injured Workers. Tallahassee, FL: Florida Division of Workers’ Compensation; 2000.
10. Wiley J, Rudolph L, Garrett K, et al. What Do Injured Workers Think of Their Medical Care? A Report on the Development of a Survey. San Francisco, CA: Division of Workers’ Compensation; 1998.
11. Sum J. Navigating the California Workers’ Compensation System. The Injured Worker’s Experience.Prepared for the Commission on Health and Safety and Workers’ Compensation. Berkeley, CA: Labor Occupational Health Program, University of California; 1996.
12. Elam KC, Cherkin D, Deyo RA. How emergency physicians approach low back pain: choosing costly options. J Emerg Med 1995; 13: 143–150.
13. Tacci JA, Webster BS, Hashemi L, et al. Healthcare utilization and referral patterns in the initial management of new-onset, uncomplicated, low back workers’ compensation disability claims. J Occup Environ Med. 1998; 40: 958–963.
14. Agency for Health Care Research and Quality. Annual Report of the National CAHPS Benchmarking Database. 2000. What Consumers Say About the Quality of Their Health Plans and Medical Care. Rockville, MD: AHRQ; 2000.AHRQ publication no. 01-0005.
15. Ware JE, Davies AR. Behavioral consequences of consumer dissatisfaction with medical care. Eval Program Plann. 1983; 6: 291–297.
16. Keith RA. Patient satisfaction and rehabilitation services. Arch Phys Med Rehabil. 1998; 79: 1122–1128.
17. Kaplan SH, Greenfield S, Ware JE, Assessing the effects of physician-patient interactions on the outcomes of chronic disease. Med Care 1989; 27 (suppl): S110–127.
18. Frank J, Sinclair S, Hogg-Johnson S, et al. Preventing disability from work-related low-back pain. New evidence gives new hope: if we can just get all the players onside. Can Med Assoc J. 1998; 158: 1625–1631.