Secondary Logo

Journal Logo


Feedback to Patients About Patient-reported Outcomes Does Not Improve Empowerment or Satisfaction

Ackermans, Lisette MD; Hageman, Michiel G. MD; Bos, A. H. MD; Haverkamp, Daniel MD, PhD; Scholtes, Vanessa A. B. MD, PhD; Poolman, Rudolf W. MD, PhD

Author Information
Clinical Orthopaedics and Related Research: April 2018 - Volume 476 - Issue 4 - p 716-722
doi: 10.1007/s11999.0000000000000069
  • Free



A patient-reported outcome measure (PROM) is defined as any report of the status of a patient’s health condition that comes directly from the patient without interpretation of the patient’s response by a clinician or anyone else [5]. PROMs typically are validated questionnaires that focus on symptoms or other aspects of well-being, functioning (or disability), and general health status or perceptions. They are generally used to measure a given patient’s response to an intervention, report on the overall effectiveness of a treatment, or to evaluate the quality of a healthcare provider. However, several authors have posited that PROMs could be used as instruments to give patients a better understanding about their physical function and to support communication [1,17,26].

Several studies have assessed the role of feedback on health-related quality-of-life questionnaires to patients and clinicians in the oncologic setting and found small improvements in patient-physician communication and discussion, although patient satisfaction was unaffected [4,11,24,25]. However, there are few data on whether this might apply in patients with musculoskeletal disorders. In addition, although clinicians intuitively believe that active involvement or empowerment on the part of patients will result if their PROMs are discussed, no studies that we know of have explored this assumption. Patient empowerment, defined as a process through which people gain great control over decisions and action affecting their health, can be measured with validated scoring tools such as the Patient Activation Measure (PAM) [9]. In addition, less self-efficacy or ineffective coping strategies explain to a great extent the variation in musculoskeletal illness. Patients with less self-efficacy might have less desire about an active role in their health process and might prefer to fall back to a more resigned and passive role.

We asked the following: (1) Does giving patients feedback on their PROM scores improve a patient's sense of empowerment or satisfaction? (2) Do demographic factors, duration of disability, or self-efficacy correlate with patient empowerment?

Patients and Methods

This prospective cohort study was conducted after approval of the institutional review board at the outpatient clinic of the orthopaedic department at the OLVG, a secondary referral clinic in Amsterdam, The Netherlands (WO15.042). The study was performed from June to September 2015.

All adult (18 years or older) patients with Dutch fluency and literacy and the ability to provide informed consent were asked to consider participation before consultation with the treating orthopaedic surgeon. Inclusion criteria were patients with osteoarthritis of the hip or knee. Exclusion criteria were patients not able to fill out questionnaires because of cognitive disabilities. Patients were informed that this study evaluated the information process without referring to its specific objective or hypotheses. Informed consent was obtained from each patient before enrollment. Patients also were informed that their participation was voluntary and that they may refuse participation at any time. The patient sample was a consecutive series of new patients and those seen for followup who fit the inclusion criteria.

Patients enrolled in the control group received standard care.

Patients in the intervention cohort completed the PROMs Hip disability and Osteoarthritis Outcome Score–Physical function short form (HOOS-PS) [3] or Knee disability and Osteoarthritis Outcome Score–Physical function short form (KOOS-PS) [2] and Numerical Pain Rating Scale, as part of the initial assessment, before the first visit with the treating physician [7]. The HOOS-PS is a five-item measure of a patient’s opinion about the function of the hip derived from the Hip disability and Osteoarthritis Outcome Score (HOOS) subscales concerning physical function, daily living, sports, and recreational activity [3,10]. Scores range between 0 and 100 points with a higher score indicating worse hip-specific disability and pain. The KOOS-PS is a similar seven-item measure of a patient’s opinion about the function of the knee derived from Knee disability and Osteoarthritis Outcome Score subscales concerning function, daily living, sports, and recreational activity [2,16]. Scores range between 0 and 100 points with a higher score indicating worse knee-specific disability and pain. Pain was assessed with the Numeric Rating Scale [7]. Patients were asked to rate their pain during activity and rest for the last week on a scale from 0 to 10, where 0 represents “no pain” and 10 represents “the worst pain possible” using whole numbers (11 integers including zero) [7]. The patients’ PROM total scores were calculated and graphically displayed using colored scales ranging from green (no disability), light green (mild disability), yellow (moderate disability), orange (severe disability), and red (extreme disability) and supported by text explaining the results to the patients. Afterward the reports were printed and given to the patients to discuss the results with their treating physician.

Before enrollment of the intervention cohort, a workshop was given to physicians on how to interpret the scores and how to discuss it with their patients. Physicians were asked to discuss the outcome of the PROM scores with the patient and how treatment may influence the outcome.

Immediately after the consultation, patients from both cohorts completed a demographic survey and the following questionnaires: PAM [9], patient satisfaction, the Pain Self-efficacy Questionnaire (PSEQ) [13].

The PAM is a validated 13-item instrument measuring patient empowerment [9]. The PAM score, which requires 10 of 13 questions to be answered, is converted to a 0 to 100 scale with higher scores indicating that the patient is likely to participate more actively in healthcare processes and to take more responsibility for his or her health [9].

The patient-physician relationship is a validated nine-item instrument measuring the patient-doctor relationship [23]. It consists of a nine-item questionnaire with a scale ranging from 1 (not at all) through 5 (totally) [23].

The PSEQ is a questionnaire designed to assess a patient’s confidence that they can achieve their goals despite pain. It involves 10 items, which can be scored by the patient on a 7-point Likert scale, ranging from 0 (not at all confident) to 6 (completely confident) [13]. The outcome score is calculated by adding the items on a scale ranging from 0 to 70. A higher score indicates greater confidence [13].

Patient satisfaction was measured with an 11-point ordinal scale assessing the overall management of a condition for which the patient sought help ranging from 0 (completely dissatisfied) to 10 (completely satisfied).

Although no minimum clinically important differences (MCIDs) have been defined for the patient empowerment or satisfaction scales, we believe that differences smaller than 0.5 of the SD are unlikely to be clinically important, even if they are statistically significant.

Among the 169 patients who were eligible for the study, 15 declined to participate because of insufficient interest or lack of time, five were excluded as a result of lack of Dutch proficiency, five did not have osteoarthritis, two were excluded because of technical difficulties, the physicians did not discuss the first questionnaire in the intervention group with two patients, and eight patients did not complete the whole questionnaire after their medical appointment, mostly claiming lack of time. The physician sample consisted of three orthopaedic surgeons (RWP, DJFM, ALvdZ) specializing in hip and knee surgery and an other category comprised of two residents (MJ, JvdL).

We enrolled 142 participants in this trial, 56 men and 86 women. The average age was 66 years (range, 27–88 years) (Table 1). More of the patients in the intervention cohort were single (40 of 72) than in the control group (20 of 70). More patients were treated by one of the three study physicians.

Table 1.
Table 1.:

A minimum sample size of 128 patients (64 in each group) was chosen for patients to detect a medium difference (0.50) in patient empowerment between those who did or did not receive the intervention with a power of 80% and an α of 0.05 on a two-tailed t-test.

We blinded the participants to the hypotheses of our study. Because the attending surgeons were part of the intervention, it was not possible to blind them for the intervention. A pre- and postimplementation design allowed us to make a comparison between the control group and the intervention group.

Fisher’s exact tests or chi square tests were conducted to determine any baseline differences between the intervention and control groups for dichotomous and categorical demographic variables (ie, sex, diagnosis, marital status, work status, physician), and unpaired Student’s t-tests or Mann-Whitney U-test were used for continuous demographic variables (ie, age, education, duration of complaints, HOOS and KOOS-PS, pain self-efficacy).

Unpaired Student’s t-tests or Mann-Whitney U-tests were performed to determine the differences between the intervention and control groups for the primary and secondary continuous outcome variables (empowerment, satisfaction, and patient-doctor relationship).

In case the univariate correlation was significant (p < 0.05), the variable was entered in multiple linear regression analyses using the backward stepwise regression method.


Although the group that received PROM feedback had higher scores for empowerment than the group that did not, the effect size was small and likely not to have been clinically important (2.8 ± 0.85 versus 3.1 ± 0.811; mean difference -0.31; 95% confidence interval, 2.8-3.1; p = 0.037; Table 2).

Table 2
Table 2:
Bivariate analyses

With the numbers available, there were no differences between the group given PROM feedback and the group that did not receive such feedback in terms of satisfaction scores (8.6 ± 1.4 versus 8.8 ± 1.2; mean difference -0.19; p = 0.39) and patient-physician relationship (4.3 ± 0.69 versus 4.4 ± 0.56; mean difference -0.09; p = 0.41; Table 2).

A higher level of self-efficacy was associated with a higher level of patient empowerment (coefficient = 0.40, p = 0.001) and satisfaction (coefficient = 0.34, p = 0.001), but age, education level, marital status, etc, were not associated (Table 2).

After controlling for demographic factors such as level of education and duration of complaint, we found that the group that received PROM feedback and who had a higher level of self-efficacy had higher scores for empowerment than the group that did not, although the effect size was small (Table 3).

Table 3
Table 3:
Multivariate analyses


Although PROMs are helpful to measure whether an intervention has improved health from the patient’s point of view, they may also serve an empowering educational role [1,17]. We found that counseling based on PROMs had a limited effect on empowerment (activation) and no effect on satisfaction. The most important positive finding is that more effective coping strategies are associated with greater patient empowerment.

The results of this study should be evaluated in light of its shortcomings. First, the study was not randomized and other changes in care over the study period could have influenced the findings. On the other hand, the study period was relatively short, and so this seems unlikely. Second, there are no defined MCIDs for the PAM satisfaction scales we used. An effect size of 0.3 of 10 is almost certainly clinically unimportant. Third, satisfaction was measured with a simplistic 11-point ordinal scale, whereas more robust outcome tools are recommended [15]. Fourth, the study was performed in one orthopaedic department and might not be generalized to other conditions or practice settings.

The finding that our intervention did not have a clinically important impact on empowerment, satisfaction, or the patient-doctor relationship might indicate that counseling based on PROMs during a single visit has little or no impact. It is possible that counseling based on PROMs during multiple visits would be more effective. Our finding was not in concordance with other studies that found a positive association between the role of feedback on health-related quality-of-life questionnaires to patients and patient empowerment in an oncologic setting [4,24] and mental health care setting [22], although patient satisfaction was unaffected [21]. Patient involvement in decision-making was associated with a more satisfying patient-doctor relationship in many prior studies [8,12,20]. In this study providing feedback based on PROM scores, potentially supporting the decision-making process, did not have a positive influence on the patient-doctor relationship. Future research might address whether specific structured formats for providing feedback are more impactful.

The correlation between effective coping strategy and empowerment reinforces that patients who have effective coping strategies are also more engaged in their own recovery [6,14,18,19]. Future research might address whether evidence-based treatments to improve coping strategies also improve empowerment.

Effective communication is an early step to involve patients in their health recovery; however, unstructured counseling based on PROMs does not seem to improve empowerment, satisfaction, or the patient-doctor relationship during a single visit. There may be a way to structure the feedback that helps patients think critically, identify their true preferences based on their values and not on misconceptions, ask more questions, and find working on resiliency more appealing. Future research might focus on such structured feedback and feedback over time.


We thank Dirk Jan F. Moojen and Arnout L. van der Zwan, orthopaedic surgeons, and Just A. van der Linden and Michiel Janssen, residents of the Department of Orthopedic Surgery from OLVG-Amsterdam, The Netherlands, for participating in this study.


1. Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res. 2013;13:211.
2. de Groot IB, Favejee MM, Reijman M, Verhaar JA, Terwee CB. The Dutch version of the Knee Injury and Osteoarthritis Outcome Score: a validation study. Health Qual Life Outcomes. 2008;6:16.
3. de Groot IB, Reijman M, Terwee CB, Bierma-Zeinstra S, Favejee MM, Roos E, Verhaar JA. Validation of the Dutch version of the Hip disability and Osteoarthritis Outcome Score. Osteoarthritis Cartilage. 2009;17:132.
4. Detmar SB, Muller MJ, Schornagel JH, Wever LD, Aaronson NK. Health-related quality-of-life assessments and patient-physician communication: a randomized controlled trial. JAMA. 2002;288:3027–3034.
5. Fayers T, Strouthidis NG, Garway-Heath DF. Monitoring glaucomatous progression using a novel Heidelberg Retina Tomograph event analysis. Ophthalmology. 2007;114:1973–1980.
6. Gruber JS, Hageman M, Neuhaus V, Mudgal CS, Jupiter JB, Ring D. Patient activation and disability in upper extremity illness. J Hand Surg Am. 2014;39:1378–1383.e1373.
7. Hartrick CT, Kovan JP, Shapiro S. The numeric rating scale for clinical pain measurement: a ratio measure? Pain Pract. 2003;3:310–316.
8. Hess EP, Knoedler MA, Shah ND, Kline JA, Breslin M, Branda ME, Pencille LJ, Asplin BR, Nestler DM, Sadosty AT, Stiell IG, Ting HH, Montori VM. The chest pain choice decision aid: a randomized trial. Circ Cardiovasc Qual Outcomes. 2012;5:251–259.
9. Hibbard JH, Mahoney ER, Stockard J, Tusler M. Development and testing of a short form of the Patient Activation Measure. Health Serv Res. 2005;40:1918–1930.
10. Klassbo M, Larsson E, Mannevik E. Hip disability, Score Osteoarthritis Outcome. An extension of the Western Ontario and McMaster Universities Osteoarthritis Index. Scand J Rheumatol. 2003;32:46–51.
11. McLachlan SA, Allenby A, Matthews J, Wirth A, Kissane D, Bishop M, Beresford J, Zalcberg J. Randomized trial of coordinated psychosocial interventions based on patient self-assessments versus standard care to improve the psychosocial functioning of patients with cancer. J Clin Oncol. 2001;19:4117–4125.
12. Montori VM, Shah ND, Pencille LJ, Branda ME, Van Houten HK, Swiglo BA, Kesman RL, Tulledge-Scheitel SM, Jaeger TM, Johnson RE, Bartel GA, Melton LJ 3rd, Wermers RA. Use of a decision aid to improve treatment decisions in osteoporosis: the osteoporosis choice randomized trial. Am J Med. 2011;124:549–556.
13. Nicholas MK. The pain self-efficacy questionnaire: taking pain into account. Eur J Pain. 2007;11:153–163.
14. Remmers C, Hibbard J, Mosen DM, Wagenfield M, Hoye RE, Jones C. Is patient activation associated with future health outcomes and healthcare utilization among patients with diabetes? J Ambul Care Manage. 2009;32:320–327.
15. Ring D, Leopold SS. Editorial—Measuring satisfaction: can it be done? Clin Orthop Relat Res. 2015;473:3071–3073.
16. Roos EM, Toksvig-Larsen S. Knee injury and Osteoarthritis Outcome Score (KOOS)—validation and comparison to the WOMAC in total knee replacement. Health Qual Life Outcomes. 2003;1:17.
17. Santana MJ, Feeny D. Framework to assess the effects of using patient-reported outcome measures in chronic care management. Qual Life Res. 2014;23:1505–1513.
18. Skolasky RL, Mackenzie EJ, Wegener ST, Riley LH. Patient activation and functional recovery in persons undergoing spine surgery. Orthopedics. 2011;34:888.
19. Skolasky RL, Mackenzie EJ, Wegener ST, Riley LH 3rd. Patient activation and adherence to physical therapy in persons undergoing spine surgery. Spine (Phila Pa 1976). 2008;33:E784–791.
20. Stacey D, Legare F, Col NF, Bennett CL, Barry MJ, Eden KB, Holmes-Rovner M, Llewellyn-Thomas H, Lyddiatt A, Thomson R, Trevena L, Wu JH. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2014;1:CD001431.
21. Taenzer P, Bultz BD, Carlson LE, Speca M, DeGagne T, Olson K, Doll R, Rosberger Z. Impact of computerized quality of life screening on physician behaviour and patient satisfaction in lung cancer outpatients. Psychooncology. 2000;9:203–213.
22. Tambuyzer E, Van Audenhove C. Is perceived patient involvement in mental health care associated with satisfaction and empowerment? Health Expect. 2015;18:516–526.
23. Van der Feltz-Cornelis CM, Van Oppen P, Van Marwijk HW, De Beurs E, Van Dyck R. A patient-doctor relationship questionnaire (PDRQ-9) in primary care: development and psychometric evaluation. Gen Hosp Psychiatry. 2004;26:115–120.
24. Velikova G, Booth L, Smith AB, Brown PM, Lynch P, Brown JM, Selby PJ. Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial. J Clin Oncol. 2004;22:714–724.
25. Velikova G, Brown JM, Smith AB, Selby PJ. Computer-based quality of life questionnaires may contribute to doctor-patient interactions in oncology. Br J Cancer. 2002;86:51–59.
26. Wasson JH, Stukel TA, Weiss JE, Hays RD, Jette AM, Nelson EC. A randomized trial of the use of patient self-assessment data to improve community practices. Eff Clin Pract. 1999;2:1–10.
© 2018 by the Association of Bone and Joint Surgeons