Secondary Logo

Journal Logo

Research Reports

Job Resources, Physician Work Engagement, and Patient Care Experience in an Academic Medical Setting

Scheepers, Renée A. MSc, PhD; Lases, Lenny S.S. MD; Arah, Onyebuchi A. MD, PhD; Heineman, Maas Jan MD, PhD; Lombarts, Kiki M.J.M.H. MSc, PhD

Author Information
doi: 10.1097/ACM.0000000000001719
  • Free

Abstract

The medical community is increasingly acknowledging that physician well-being is an important ingredient for optimal patient care.1–3 Research has found that poor physician well-being could lead to suboptimally performing health care systems.1,4 Accordingly, current hospital accreditation standards encourage hospitals to manage matters of physician well-being.5 This call also resonates in modern policies of the Accreditation Council for Graduate Medical Education6 and in the renewed Canadian Medical Education Directions for Specialists (CanMEDS).7 According to CanMEDS, a competent professional commits to his or her personal well-being to foster optimal patient care and sustainable medical practice.7

Studies on physician well-being have focused mostly on physician burnout.8–10 These studies show that many physicians experience low energy and suboptimal well-being in their work.8,11 Optimal work-related well-being has been widely studied as “work engagement.”12–14 Work engagement is marked by a motivational state of positive well-being involving high levels of energy, enthusiasm, and dedication to one’s work. Energetic, dedicated, work-engaged professionals are proactive in attaining work goals and striving for excellence.12,15 Research has shown that work-engaged professionals ultimately perform better in their work.13,16 Similarly, work-engaged physicians report that they experience a better work ability17 and commit fewer medical errors.18 In addition, work engagement is associated with more adequate patient-safety-related behaviors and attitudes.19

Performance of work-engaged physicians has not yet been studied from the patient perspective. Patients’ reported experiences of care, important measures of the patient perspective, have provided unique information on physicians’ performance.20 Research shows that patient care experience is positively associated with patient outcomes such as clinical effectiveness and patient safety.21 Yet, whether work-engaged physicians also perform better in the eyes of patients is unknown. We hypothesized that patients’ experiences may be more positive if patients receive care from physicians who are more engaged in their work—and who may, therefore, be more likely to put in more effort into providing optimal care.

In addition to the reported beneficial impact on physician performance,17–19 work engagement has been associated with less burnout,22 with decreased (sickness) absenteeism,23 and with greater life satisfaction.24 Work engagement appears to be higher in work environments with adequate job resources that (1) help employees achieve work goals, (2) reduce work demands, and (3) stimulate personal growth, learning, and development.25 Job resources include autonomy, colleague support, participation in decision making, and opportunities to learn and develop.26 Research shows that job resources increase work engagement especially when demands are high,27 and can thus facilitate physicians’ coping with the multiple demands of modern practice.1,28 However, few investigators have explored whether job resources should be leveraged to enhance physician work engagement specifically—and if so, which ones. Insight into this topic may support policies for optimizing physician work engagement, which is associated with better work performance.13

The authors of this study, therefore, investigated (1) whether physician work engagement is associated with patient care experience, and (2) whether job resources are related to physician work engagement—and if so, which ones.

Method

Study population and setting

From April 2014 to April 2015, we collected patient care experience evaluations at 10 outpatient clinics of two academic hospitals in The Netherlands. These outpatient clinics comprised cardiology, gastroenterology, internal medicine, otorhinolaryngology, obstetrics–gynecology, neurology, neurosurgery, nuclear medicine, pediatrics, and pulmonology. At each outpatient clinic, we collected evaluations on a daily basis for approximately two months to ensure inclusion of a representative and large sample of patients for each physician. All patients except those who did not speak Dutch were eligible for participation (research shows that language differences can complicate both patient–physician communication and survey assessment29,30). Participation was voluntary for patients and physicians alike, and we took measures to safeguard the anonymity and confidentiality of all participants. In total, we registered 9,802 Dutch-speaking patients and 238 of their consulting physicians during the study.

For the multilevel setting of our study whereby patients were clustered within physicians, we estimated that if 5% to 15% of the outcome variance (5% to 15% of the intraclass correlation coefficient or ICC was comparable to that seen in the literature) was attributable to the physician, then including between 10 and 30 patients per physician would require sampling between 57 and 165 physicians to adequately power the study at 80%, such that the type I error rate was no more than 5%.31–33 The institutional ethical review board of the Academic Medical Center of the University of Amsterdam waived ethical approval for this study.

Procedure

We informed all physicians—both attending and resident physicians—who provided patient care at the participating outpatient clinics about the study by e-mail and invited them to complete a Web-based survey. Medical secretaries at the outpatient clinics informed the patients about the study and provided an information letter, affirming their anonymity if they agreed to participate. After the consultation, academic students (studying medicine, the health sciences, and/or psychology) guided patients through the questionnaire following a structured protocol. First, students asked patients if they were prepared to provide anonymous and confidential feedback on their physicians’ care on a tablet (Prestigio, Multipad 4 Quantum 10.1, 2014; Apple, Ipad 2/Ipad Mini 2, 2014). After patients gave their oral consent, students opened the questionnaire—provided via the tablet—whereby patients could again read and approve the terms and conditions of the study. Then, patients answered multiple-choice questions intended to garner information about their demographics and experience during the just-completed outpatient consultation.

Measures

To measure patient care experience, we used the Dutch Consumer Quality (CQ) index.34 The CQ index is a standardized patient survey tool developed by the Dutch Center for Consumer Experience in Health Care,34 inspired by both the U.S. Consumer Assessment of Health Care Providers and Systems35 and the Dutch Quote (or QUality Of care Through the patients’ Eyes) tools.36 The final tool was developed through a systematic literature review, consumer focus groups, stakeholder input, and pilot testing.34 We used nine items in total to measure physician care behaviors, which we selected on the basis of their suitability to cover, as needed for our study, multiple specialties as well as various medical consultations at the outpatient clinic (both new and follow-up consultations). The first six items queried the following physician care behaviors: (1) listening attentively, (2) spending enough time, (3) explaining things clearly, (4) asking about medication use, (5) sharing the decision making, and (6) paying attention to health complaints.37 In agreement with the survey developer (O.A.A.), we rephrased one item slightly to make it broad enough to cover patients’ physical and mental health. Rather than “The doctor spent enough time and attention on physical complaints,” Item 6 read, “The doctor spent enough time and attention on health complaints.” We added two more items from the CQ index to cover physicians’ (7) paying attention to patients’ personal situation and (8) providing an opportunity for questions.38 We also decided to add an item on whether the physician provided (9) information on continuing treatment, since such information can be considered necessary for adequately informing patients on decisions in their care process.39 Patients answered all items using a five-point scale that ranged from 1 (“Totally disagree”) to 5 (“Totally agree”). We also provided the option “Not applicable.” In addition, we included questions about patient demographics—self-reported gender, self-rated health, number of previous consultations (with this physician), educational level, and country of birth—since research shows they are associated with patient experience.40

Using the Web-based survey, we asked physicians for their age, gender, country of birth, trainee level (resident or attending physician), year of graduation from medical school, specialty, and years of experience in clinical practice. We also asked about their perception of job resources and work engagement. The four job resources we inquired about were (1) job autonomy, (2) colleague support, (3) participation in decision making, and (4) opportunity to learn and develop. The measures originated from the validated Questionnaire on the Experience and Evaluation of Work.41 Shortened versions of the job resources in this questionnaire are widely studied, including in physician-focused research.42 The topics of job autonomy,42 colleague support,43 and opportunity to learn and develop42 each comprised three items, and participation in decision making42 comprised four items. Participants answered the questions measuring job autonomy and colleague support through a five-point scale ranging from 1 (“Never”) to 5 (“Always”), and they answered the questions about participation in decision making and opportunity to learn and develop using a five-point scale ranging from 1 (“Totally disagree”) to 5 (“Totally agree”). All scales were identical for residents and specialists; however, for colleague support, residents could rate the support of their resident colleagues and specialists could rate this for their specialist colleagues.

We measured work engagement using the nine-item Utrecht Work Engagement Scale,44 which has been extensively validated in various occupational groups, including physicians.16 Physicians could self-report their engagement in their work on a seven-point scale from 0 (“Never”) to 6 (“Always/Daily”).

Statistical analyses

We calculated the overall mean for the study measures including job resources, physician work engagement, and patient care experience. To study psychometric properties of the study measures, we performed principal components analysis (PCA) with varimax rotation, item-to-scale correlations, and reliability analysis for internal consistency. For internal consistency, we considered a Cronbach alpha of 0.7 or higher as acceptable.

To study associations between physician work engagement and patient care experience, we conducted mixed linear model analysis using random intercept to account for clustering of patients within physicians. Hence, we used patient evaluations as clustering variables. We included only patients who were 16 years and older in the analysis.45 We adjusted the analysis for patient variables (age, sex, self-rated health, number of appointments with evaluated doctor, country of birth, educational level) as well as physician variables (age, sex, and training level) by including these as covariates in the analysis. To adjust for variance within outpatient clinics, we considered outpatient clinic to be a fixed effect in the analysis.

Lastly, we performed linear regression analyses to study the associations between the four job resources (independent variables) and physician work engagement (dependent variable). We adjusted these analyses for physician gender, years of experience, training level (resident or attending physician), and specialty by treating them as covariates.

For both the mixed linear models and the linear regression analysis, we tested whether training level moderated the association between the predictor (work engagement in the mixed linear model and job resources in the linear regression analysis) and the outcome (patient care experience for the mixed linear model and work engagement for the linear regression analysis). To that end, we added to each respective analysis a product term of training level and the concerning predictor. In case the product term showed a nonzero association with work engagement, we reported results separately for attending physicians and residents. We performed all analyses using SPSS 20.0 (IBM, Armonk, New York).

Results

During the study period, we registered visits by 9,802 patients at the participating outpatient clinics, and of these, we collected evaluations for 4,573 (47%); 238 physicians conducted these visits, and of these 185 (78%) completed the survey (Table 1). Of the 185 physicians, 103 were attending physicians, and the remaining 82 were residents. For the 185 participating physicians, 4,130 patient evaluations were available (the 443 remaining patient evaluations concerned the physicians who did not participate in the study). The 4,130 patient evaluations resulted in an average of 22.3 patient evaluations per participating physician.

Table 1
Table 1:
Demographics of Physicians and Patients Participating in a Study of Job Resources, Work Engagement of Physicians, and Patient Experience of Care, 2014–2015a

On the basis of the PCA on patient care experience, the nine separate items emerged as one factor. This single factor reflected a satisfactory Cronbach alpha of 0.83 for patient care experience measurement (Table 2). The four job resources—autonomy, colleague support, participation in decision making, and opportunity to learn and develop—and work engagement showed acceptable to good rates for internal consistency, and all items showed adequate factor loadings (Table 2).

Table 2
Table 2:
Psychometric Properties of Scales Used in a Study of Job Resources, Work Engagement of Physicians, and Patient Experience of Care, 2014–2015a

As detailed in Table 3, we found no association between physician work engagement and patient care experience (B = 0.01; 95% confidence interval [CI] = −0.02 to 0.03; P = .669). This finding remained unchanged in further analyses accounting for different levels of training (attending physicians vs. residents).

Table 3
Table 3:
Unstandardized Regression Coefficients (B) and 95% CIs for the Association Between Physician Work Engagement and Overall Patient Experience in a Study of Job Resources, Work Engagement of Physicians, and Patient Experience of Care, 2014–2015a

Autonomy was positively associated with physician work engagement (B = 0.31; 95% CI = 0.10 to 0.51; P = .004)—as was the opportunity to learn and develop (B = 0.28; 95% CI = 0.05 to 0.52; P = .019). Additional analyses showed that the association between autonomy and work engagement was especially applicable to attending physicians (B = 0.35; 95% CI = 0.08 to 0.61; P = .010), while the association between opportunity to learn and develop and work engagement was especially applicable to residents (B = 0.69; 95% CI = 0.28 to 1.12; P = .001). Colleague support and participation in decision making did not show associations with work engagement (respectively, B = 0.15; 95% CI = −0.04 to 0.33; P = .114 and B = 0.22; 95% CI = −0.02 to 0.46; P = .071). See Table 4 for further details.

Table 4
Table 4:
Unstandardized Regression Coefficients (B) and 95% CIs for the Associations Between the Four Job Resources Scales and Physician Work Engagement in a Study of Job Resources, Work Engagement of Physicians, and Patient Care Experience, 2014–2015a

Discussion

Main findings

This study in an outpatient setting in The Netherlands indicates that physicians’ work engagement is not associated with their performance as perceived by patients and that physicians perceive autonomy and opportunities to learn and develop as job resources for their work engagement. Specifically, autonomy was identified as an especially valuable job resource for attending physicians, while opportunities to learn and develop were especially valuable for resident work engagement.

Possible explanation of findings

According to previous research, work engagement facilitated high performance.16,18 Interestingly, the findings of this current study show that physician work engagement may not necessarily be associated with better performance as perceived by patients. The patients in this current study reported performance specifically regarding physicians’ interpersonal behaviors, such as listening attentively.35 As documented in a systematic review, inappropriate interpersonal behaviors of physicians are infrequently observed.46 In general, physicians receive extensive training—throughout their formal education and beyond—regarding professional interpersonal behaviors with patients.46 Possibly, this extensive training and experience may act as a safeguard to uphold standards for professional interpersonal behaviors—even in the face of low engagement or disengagement at work.

However, as this is, to our knowledge, the first study of work engagement and patient care experience, we accordingly encourage caution in interpreting the results. Whereas previous work engagement research in health care has focused on supervisor-, resident-, or self-evaluated performance,16,18,19,47 the current study specifically involved patient-evaluated performance. Like other studies of patient-evaluated performance,48–51 our patient care experience ratings are highly positive, and scores vary only minimally. This low variability and positive valence may have affected potential between-physician differences.52 Future research could include narrative patient feedback of physicians to obtain more detailed insight into the role of physician work engagement in the patient care experience.

Our findings—along with findings from previous research16,44,53 showing that levels of physician work engagement are not higher than average—indicate an opportunity to better enable physician engagement. For physicians, particularly attending physicians, optimizing autonomy may facilitate greater engagement. Although autonomy has traditionally been one of the cornerstones of the medical profession, it has gradually decreased with the rise of standardization of care and the increased requirements demanded by quality management, accountability, and bureaucracy in modern health care.54,55 In light of these modern developments, our findings, which do not necessarily point to high levels of physician autonomy (Table 2), may be expected. Increasing autonomy levels may support physician work engagement; adjusting physicians’ work conditions accordingly may be worth considering.

Our results showed that learning and development opportunities stimulate physician work engagement, especially among residents. The ongoing pursuit of training—that is, lifelong learning—is a value of the medical profession. Our finding may indicate that residents embrace this value as they are more engaged when offered more learning possibilities. Research shows that compared with extrinsic incentives (e.g., financial rewards), intrinsic incentives for learning (e.g., feedback that permits physicians to independently improve their practice) lead to better improvement56; therefore, we advocate formative performance measures that stimulate intrinsic motivation for learning instead of external quality control.57 Besides being potentially fruitful for physicians’ performance growth, formative measures in service of learning may also ultimately enhance physician engagement and well-being.

Study strengths and limitations

The use of on-site tablet computers loaded with the survey tool enabled the large sample size in this study (4,573 patients). Similarly, the assistance of students facilitated survey completion (a meta-analysis58 has shown that self-completion and assisted survey completion result in overall equivalent scores). The response rate of the patient sample in this study (47%) was in the range of common response rates for this type of research (30%–60%).59–61 Still, we acknowledge that higher response rates would benefit patient-experience-focused research. The (on average) 22.3 patient evaluations per physician in the current study resulted in robust and reliable patient care experience measurement.49 The 78% physician response rate contributed to a representative display of job resources and work engagement as perceived by the physicians under study. Given the cross-sectional nature of the study data, we cannot guarantee causality regarding the association between job resources and work engagement; nonetheless, previous research in health care using longitudinal data has indicated that job resources were antecedents of work engagement in the long term.23,53 Our study included only quantitative measures of job resources, yet other measures of job resources (e.g., the amount of learning opportunities) and qualitative measures (such as the nature of autonomy) could add to the understanding of this topic.

Implications

Results of this pioneer study on physician work engagement and patient care experience contrast with previously reported results indicating that work engagement tends to positively affect patient care quality.18,19 Because the current study is, to our knowledge, the first to link physician work engagement to performance as evaluated by patients specifically, the findings should be considered preliminary; more research is needed. Other investigations could examine additional indicators and measures of patient-evaluated performance in various care settings (outpatient versus inpatient or academic versus nonacademic). Fostering optimal physician work engagement could benefit from enabling more autonomy, whereby physicians could participate in specific interventions that successfully promote feeling increasing control over the highly dynamic work environment in health care.62 Team-based peer support could offer additional possibilities for promoting physician autonomy.63 Research shows that such programs eventually decrease health care providers’ burnout levels63; we expect they might also enhance engagement. Finally, our findings point to the need for optimizing job resources to the specific training level of physicians: Attending physicians especially value autonomy, and residents especially value learning opportunities.

Conclusions

At a time when physicians’ work is strained, the average levels of physician work engagement presented in this study represent a positive contrast to reported concerns regarding physicians’ well-being.1 Although research suggests that work engagement positively affects patient care,18,19 this study indicates that work engagement is not necessarily associated with physicians’ performance as perceived by patients. According to our physician respondents, greater autonomy and increased learning opportunities could safeguard or strengthen engagement in physicians’ work. Ultimately, a highly engaged physician workforce requires stimulating and healthy conditions that allow physicians to grow, to work autonomously, and to provide outstanding patient care even within the challenging context of modern health care.

Acknowledgments:

The authors would especially like to thank all the patients, attending physicians, and residents who participated in this study. In addition, they want to thank all the students who helped in the process of data collection at the outpatient clinics. The authors also want to acknowledge the outpatient clinic managers and medical secretaries for their cooperation in this study. The authors would like to thank the Patient Perception Program project team for their commitment to and their contribution to the design, implementation, and conduct of the study. Finally, the authors would like to thank M.Sc. Milou Silkens for her helpful support in supervising students and data cleaning.

References

1. Wallace JE, Lemaire JB, Ghali WA. Physician wellness: A missing quality indicator. Lancet. 2009;374:17141721.
2. Epstein RM, Krasner MS. Physician resilience: What it means, why it matters, and how to promote it. Acad Med. 2013;88:301303.
3. Zwack J, Schweitzer J. If every fifth physician is affected by burnout, what about the other four? Resilience strategies of experienced physicians. Acad Med. 2013;88:382389.
4. Scheepers RA, Boerebach BC, Arah OA, Heineman MJ, Lombarts KM. A systematic review of the impact of physicians’ occupational well-being on the quality of patient care. Int J Behav Med. 2015;22:683698.
5. Joint Commission. Comprehensive Accreditation Manual for Hospitals: The Official Handbook. 2017.Oakbrook Terrace, IL: Joint Commission Resources.
6. Jennings ML, Slavin SJ. Resident wellness matters: Optimizing resident education and wellness through the learning environment. Acad Med. 2015;90:12461250.
7. Frank JR, Snell L, Sherbino J. The Draft CanMEDS 2015 Physician Competency Framework—Series IV. Ottawa, Ontario, Canada: Royal College of Physicians and Surgeons of Canada; March 2015. http://www.royalcollege.ca/portal/page/portal/rc/common/documents/canmeds/framework/canmeds2015_framework_series_IV_e.pdf. Accessed March 6, 2017.
8. Dyrbye LN, West CP, Satele D, et al. Burnout among U.S. medical students, residents, and early career physicians relative to the general U.S. population. Acad Med. 2014;89:443451.
9. Gundersen L. Physician burnout. Ann Intern Med. 2001;135:145148.
10. Linzer M, Visser MR, Oort FJ, Smets EM, McMurray JE, de Haes HC; Society of General Internal Medicine (SGIM) Career Satisfaction Study Group (CSSG). Predicting and preventing physician burnout: Results from the United States and the Netherlands. Am J Med. 2001;111:170175.
11. Shanafelt TD, Boone S, Tan L, et al. Burnout and satisfaction with work–life balance among US physicians relative to the general US population. Arch Intern Med. 2012;172:13771385.
12. Prins JT, Hoekstra-Weebers JE, Gazendam-Donofrio SM, et al. Burnout and engagement among resident doctors in the Netherlands: A national study. Med Educ. 2010;44:236247.
13. Christian MS, Garza AS, Slaughter JE. Work engagement: A quantitative review and test of its relations with task and contextual performance. Pers Psychol. 2011;64:89136.
14. Bakker AB. An evidence-based model of work engagement. Curr Dir Psychol Sci. 2011;20:265269.
15. Salanova M, Schaufeli WB. A cross-national study of work engagement as a mediator between job resources and proactive behaviour. Int J Hum Res Manag. 2008;19:116131.
16. Scheepers RA, Arah OA, Heineman MJ, Lombarts KM. In the eyes of residents good supervisors need to be more than engaged physicians: The relevance of teacher work engagement in residency training. Adv Health Sci Educ Theory Pract. 2015;20:441455.
17. Mache S, Danzer G, Klapp BF, Groneberg DA. Surgeons’ work ability and performance in surgical care: Relations between organisational predictors, work engagement and work ability. Langenbecks Arch Surg. 2013;398:317325.
18. Prins JT, van der Heijden FM, Hoekstra-Weebers JE, et al. Burnout, engagement and resident physicians’ self-reported errors. Psychol Health Med. 2009;14:654666.
19. Daugherty Biddison EL, Paine L, Murakami P, Herzke C, Weaver SJ. Associations between safety culture and employee engagement over time: A retrospective analysis. BMJ Qual Saf. 2016;25:3137.
20. Black N, Jenkinson C. Measuring patients’ experiences and outcomes. BMJ. 2009;339:b2495.
21. Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3:e001570.
22. Tijdink JK, Vergouwen AC, Smulders YM. Emotional exhaustion and burnout among medical professors; a nationwide survey. BMC Med Educ. 2014;14:183.
23. Schaufeli WB, Bakker AB, Van Rhenen W. How changes in job demands and resources predict burnout, work engagement, and sickness absenteeism. J Organ Behav. 2009;30:893.
24. Hakanen JJ, Schaufeli WB. Do burnout and work engagement predict depressive symptoms and life satisfaction? A three-wave seven-year prospective study. J Affect Disord. 2012;141:415424.
25. Hakanen JJ, Schaufeli WB, Ahola K. The job demands–resources model: A three-year cross-lagged study of burnout, depression, commitment, and work engagement. Work Stress. 2008;22:224241.
26. Xanthopoulou D, Bakker AB, Demerouti E, Schaufeli WB. Reciprocal relationships between job resources, personal resources, and work engagement. J Vocat Behav. 2009;74:235244.
27. Bakker AB, Hakanen JJ, Demerouti E, Xanthopoulou D. Job resources boost work engagement, particularly when job demands are high. J Educ Psychol. 2007;99:274.
28. Williams ES, Rondeau KV, Xiao Q, Francescutti LH. Heavy physician workloads: Impact on physician attitudes and outcomes. Health Serv Manage Res. 2007;20:261269.
29. Weech-Maldonado R, Morales LS, Elliott M, Spritzer K, Marshall G, Hays RD. Race/ethnicity, language, and patients’ assessments of care in Medicaid managed care. Health Serv Res. 2003;38:789808.
30. Murray-García JL, Selby JV, Schmittdiel J, Grumbach K, Quesenberry CP Jr.. Racial and ethnic differences in a patient survey: Patients’ values, ratings, and reports regarding physician primary care performance in a large health maintenance organization. Med Care. 2000;38:300310.
31. Moineddin R, Matheson FI, Glazier RH. A simulation study of sample size for multilevel logistic regression models. BMC Med Res Methodol. 2007;7:34.
32. Maas CJM, Hox JJ. Sufficient sample sizes for multilevel modeling. Methodology. 2005;1:8692.
33. Gelman A, Hill J. Data Analysis Using Regression and Multilevel/Hierarchical Models. 2006. Cambridge, UK: Cambridge University Press; http://www.ievbras.ru/ecostat/Kiril/R/Biblio/R_eng/Gelman,%20Hill-Data%20Analysis%20Using%20Regression%20(2007).pdf. Accessed March 6, 2017.
34. Delnoij DM, Rademakers JJ, Groenewegen PP. The Dutch consumer quality index: An example of stakeholder involvement in indicator development. BMC Health Serv Res. 2010;10:88.
35. Sixma HJ, Kerssens JJ, Campen CV, Peters L. Quality of care from the patients’ perspective: From theoretical concept to a new measuring instrument. Health Expect. 1998;1:8295.
36. Hekkink CF, Sixma HJ, Wigersma L, et al. QUOTE-HIV: An instrument for assessing quality of HIV care from the patients’ perspective. Qual Saf Health Care. 2003;12:188193.
37. van der Veer SN, Jager KJ, Visserman E, et al. Development and validation of the Consumer Quality index instrument to measure the experience and priority of chronic dialysis patients. Nephrol Dial Transplant. 2012;27:32843291.
38. van Linschoten CP, Barf HA, Moorer P, Spoorenberg S. CQ-index Poliklinische Ziekenhuiszorg: meetinstrumentontwikkeling. [CQ-index Outpatient Hospital Care: measurement tool development.] 2011. https://www.zorginzicht.nl/bibliotheek/Poliklinische%20zorg/MeetinstrumentenDocumenten/CQ-index%20Poliklinische%20Ziekenhuiszorg%20-%20meetinstrumentontwikkeling.pdf. Accessed April 29, 2017.
39. Coulter A, Ellins J. Effectiveness of strategies for informing, educating, and involving patients. BMJ. 2007;335:2427.
40. van der Veer SN, Arah OA, Visserman E, et al. Exploring the relationships between patient characteristics and their dialysis care experience. Nephrol Dial Transplant. 2012;27:41884196.
41. van Veldhoven M, de Jonge J, Broersen S, Kompier M, Meijman T. Specific relationships between psychosocial job conditions and job-related stress: A three-level analytic approach. Work Stress. 2002;16:207228.
42. Bakker AB, ten Brummelhuis Lieke L, Prins JT, van der Heijden FMMA. Applying the job demands–resources model to the work–home interface: A study among medical residents and their partners. J Vocat Behav. 2011;79:170180.
43. Veldhoven MJPM, Meijman TF. Measuring Psychosocial Workload With a Questionnaire: The Questionnaire on the Experience and Evaluation of Work (QEEW) [in Dutch]. 1994. Amsterdam, Netherlands: Nederlands Instituut voor Arbeidsomstandigheden NIA.
44. Schaufeli WB, Bakker AB. Utrecht Work Engagement Scale: Preliminary Manual. Utrecht, Netherlands: Occupational Health Psychology Unit, Utrecht University; November 2003. http://www.beanmanaged.com/doc/pdf/arnoldbakker/articles/articles_arnold_bakker_87.pdf. March 6, 2017.
45. Law on medical treatment agreement [in Dutch]. https://www.kindenziekenhuis.nl/professionals/modelrichtlijn-wgbo/. Published 2015. Accessed March 6, 2017.
46. Boerebach BC, Scheepers RA, van der Leeuw RM, Heineman MJ, Arah OA, Lombarts KM. The impact of clinicians’ personality and their interpersonal behaviors on the quality of patient care: A systematic review. Int J Qual Health Care. 2014;26:426481.
47. Salanova M, Lorente L, Chambel MJ, Martínez IM. Linking transformational leadership to nurses’ extra-role performance: The mediating role of self-efficacy and work engagement. J Adv Nurs. 2011;67:22562266.
48. Lipner RS, Blank LL, Leas BF, Fortna GS. The value of patient and peer ratings in recertification. Acad Med. 2002;77(10 suppl):S64S66.
49. Campbell JL, Richards SH, Dickens A, Greco M, Narayanan A, Brearley S. Assessing the professional performance of UK doctors: An evaluation of the utility of the General Medical Council patient and colleague questionnaires. Qual Saf Health Care. 2008;17:187193.
50. Hall W, Violato C, Lewkonia R, et al. Assessment of physician performance in Alberta: The physician achievement review. CMAJ. 1999;161:5257.
51. Wood J, Collins J, Burnside ES, et al. Patient, faculty, and self-assessment of radiology resident performance: A 360-degree method of measuring professionalism and interpersonal/communication skills. Acad Radiol. 2004;11:931939.
52. Boerebach BC, Arah OA, Heineman MJ, Lombarts KM. Embracing the complexity of valid assessments of clinicians’ performance: A call for in-depth examination of methodological and statistical contexts that affect the measurement of change. Acad Med. 2016;91:215220.
53. Mauno S, Kinnunen U, Ruokolainen M. Job demands and resources as antecedents of work engagement: A longitudinal study. J Vocat Behav. 2007;70:149171.
54. Levey NN. Medical professionalism and the future of public trust in physicians. JAMA. 2015;313:18271828.
55. Grol R. Improving the quality of medical care: Building bridges among professional pride, payer profit, and patient satisfaction. JAMA. 2001;286:25782585.
56. Kolstad JT. Information and Quality When Motivation Is Intrinsic: Evidence From Surgeon Report Cards. Cambridge, MA: National Bureau of Economic Research; February 2013. http://www.nber.org/papers/w18804. Accessed February 2, 2017.
57. Overeem K, Faber MJ, Arah OA, et al. Doctor performance assessment in daily practise: Does it help doctors or not? A systematic review. Med Educ. 2007;41:10391049.
58. Rutherford C, Costa D, Mercieca-Bebber R, Rice H, Gabb L, King M. Mode of administration does not cause bias in patient-reported outcome results: A meta-analysis. Qual Life Res. 2016;25:559574.
59. Rodriguez HP, von Glahn T, Rogers WH, Chang H, Fanjiang G, Safran DG. Evaluating patients’ experiences with individual physicians: A randomized trial of mail, Internet, and interactive voice response telephone administration of surveys. Med Care. 2006;44:167174.
60. Boscardin CK, Gonzales R. The impact of demographic characteristics on nonresponse in an ambulatory patient satisfaction survey. Jt Comm J Qual Patient Saf. 2013;39:123128.
61. Lasek RJ, Barkley W, Harper DL, Rosenthal GE. An evaluation of the impact of nonresponse bias on patient satisfaction surveys. Med Care. 1997;35:646652.
62. Dunn PM, Arnetz BB, Christensen JF, Homer L. Meeting the imperative to improve physician well-being: Assessment of an innovative program. J Gen Intern Med. 2007;22:15441552.
63. Le Blanc PM, Hox JJ, Schaufeli WB, Taris TW, Peeters MC. Take care! The evaluation of a team-based burnout intervention program for oncology care providers. J Appl Psychol. 2007;92:213227.
Copyright © 2017 by the Association of American Medical Colleges