When a patient is admitted to the hospital, their quality of care is based not only on clinical outcomes but also on their perception of care. Recently, the field of medicine has evolved into the electronic and social media era, enabling the patient to more thoroughly evaluate their in-hospital experience. In addition to positive clinical outcomes, the concept of a patient-centered experience has been increasingly a key focus and strategy of many hospital systems. Patient satisfaction is measured using a survey known as the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS), which helps standardize care in hospitals at a national level.1 On discharge from the hospital, patients are sent a survey to be completed within 6 weeks. The survey consists of 32 questions divided among eight domains: Communication with Nurses, Communication with Doctors, Staff Responsiveness, Pain Management, Communication About Medications, Discharge Information, Cleanliness and/or Quietness, and Overall Hospital Rating (Figure 1). Credit is then awarded when a “top box” response is marked, which is the top selection for the respective question. Once completed, the information is then made public and shared to a national online database (www.medicare.gov/hospitalcompare). This method gives a platform to empower the patient, with overall goals of hospital accountability and improved patient care.
Along with providing an objective clinical reputation, the HCAHPS surveys affect financial reimbursement from state and federal governments. The HCAHPS survey is the main tool used for value-based purchasing, which is an initiative required under the Patient Protection and Affordable Care Act of 2010 to allocate funds for Medicare/Medicaid spending based on the total performance score (TPS).2 The TPS is broken down into divisions, where 70% come from basic core measures in clinical care and 30% come from patients' perception of their hospital stay, measured using the HCAHPS survey. In October 2012, the Center for Medicare and Medicaid Services reduced the base operating diagnosis-related group payments by 1% to create an incentive fund, estimated at $850 million, which increased to 2% in 2017.1,3 These incentive funds are then allocated to top-performing hospitals and withheld from underperforming hospitals based on their TPS.
Mixed results have been reported with the application of the HCAHPS surveys and their relation to clinical outcomes for the surgical patients.4-9 Some studies have found patient satisfaction to be independent of hospital compliance with surgical processes of quality of care, hospital safety culture, surgical complications, and readmissions,7-9 whereas other studies have supported the use of patient satisfaction surveys.4-6 Critics of the HCAHPS argue that most patients are medically undereducated, and whereas aiming to address deficiencies of the HCAHPS/TPS may temporarily allay superficial deficiencies, fixing larger, underlying problems may not be incentivized. There is a paucity of literature regarding the use of the HCAHPS in surgical patients, and because of this, we evaluated the implementation of an orthopaedic surgery resident rounding initiative on all orthopaedic surgery inpatients at one community hospital and its effect on the HCAHPS surveys.
We sought to find a simple and direct intervention to enhance our patients' perception of their hospital stay and in turn increase the HCAHPS survey scores. At our institution, similar to many teaching hospitals, orthopaedic surgery patients are seen every day by orthopaedic surgery residents between the hours of 5:00 and 6:30 AM. During this time, the patient is often woken up, asked to participate in a simple physical examination, informed of the treatment plan for the day, and given the opportunity to discuss any questions or concerns. Understandably, this interaction can be somewhat limited due to the early hour, patient fatigue, and time constraints of the rounding physician. Patients may not have recollection of their morning examination, and at discharge, they may leave the hospital feeling dissatisfied with their postoperative care. In an effort to ameliorate possible dissatisfaction, we instituted the measured intervention of this study, an evening rounding regimen that included specific questions created by the authors.
We hypothesize that an additional and consistent regimented rounding intervention will positively affect a patient's overall perception of care as measured through the HCAHPS scores at our institution, most notably in the Communication with Doctors and Overall Hospital Rating domains.
This study was approved by the Institutional Review Board as a quality initiative. Beginning May 1, 2017, a structured rounding protocol was instituted. A retrospective review of the HCAHPS surveys for all orthopaedic surgical patients designated to a specific unit at one community hospital (Long Island Jewish Valley Stream) was evaluated 6 months before (November 1, 2016, to April 30, 2017) and 6 months after (May 1, 2017, to November 30, 2017) implementation of the study protocol. The pre-intervention group included 60 patients and the post-intervention group included 94 patients for a total of 154 completed patient surveys (Table 1). Demographic data of the intervention group are shown in Table 2.
All rounding were undertaken by orthopaedic surgery residents instructed to the specific protocol, consisting of traditional daily morning rounds and an additional evening round between 5:00 and 9:00 PM. During the rounds, each patient was asked the following questions:
- (1) How are you doing overall?
- (2) How is your pain?
- (3) How is physical therapy going?
- (4) Is there anything else I can help you with?
The resident physician then addressed any events from the day and confirmed any changes in the plan with the patient and nursing staff. No formal documentation was made during the rounds in the electronic medical record, such that the physician-patient interaction was not interrupted by documentation of the encounter.
The authors chose five questions from the HCAHPS survey for statistical analysis to capture the effect of an additional rounding session (Figure 2). For each question, three sets of data were evaluated using the top box score, mean score, and a specific study devised measure labeled the “positive score.” The top box score is a measure of the top option selected for each question in the survey. For example, if a question had four choices (Never, Sometimes, Usually, Always), only the option “Always” would be recorded as a positive response. The mean score is the numerical average of the corresponding question of the HCAHPS survey. Again, using the same sample as mentioned earlier, if a question had four choices (Never, Sometimes, Usually, Always), the choice is given a numerical value1-4 and is divided by the total number of points possible.4 The “positive score” is a measure devised by the authors, which looked at whether the top two options for a question are selected; that is, for a question with four choices (Never, Sometimes, Usually, Always), the third or fourth choices (Usually, Always) are recorded as a positive response. In this way, favorable responses are included, which otherwise would not be captured by the top box score.
As a control, the HCAHPS survey data were also collected from two similar community hospitals (Plainview Hospital and Huntington Hospital). These hospitals have a dedicated orthopaedic surgery unit, similar surgical case diversity, and the same orthopaedic surgery residents from the tested institution. At the control hospitals, the traditional early morning rounding regimen was maintained. The intervention hospital results were then compared with both hospital controls and statistical analysis was undertaken. Inclusion criteria for the study were based on diagnosis-related group codes (460, 468, 470, 472 to 473, 480 to 482, 488, 494, 501, 502, 517) for both elective and nonelective orthopaedic surgical procedures that required inpatient admission. All adult orthopaedic surgery patients admitted to the hospital were eligible for the study. Patients deemed not to have medical capacity were excluded. An interpreter service was used for the rounding initiative for all non–English-speaking patients. Patients who did not return their HCAHPS surveys were excluded from the analysis.
Last, to evaluate changes in a larger and more relevant context, we compared our top box scores with those of the national database to obtain a percentile rank before and after our intervention. These types of measurements are how the HCAHPS surveys are used by the government and other institutions to assess hospital quality, rankings, and financial reimbursement.
A chi-square test was used to detect statistical significance for the top box and positive scores. A Mann-Whitney U test was used to assess the mean scores. Statistical significance was defined as P<0.05. A power analysis was also performed to detect the effect size.
After reviewing the HCAHPS surveys before and after implementation of our rounding protocol, the positive score measure for the question “How well staff worked together to care for you” was the only statistically significant result (P = 0.046). There was no statistically significant difference in the other HCAHPS questions we chose for the analysis. Table 3 displays the pre- and post-intervention HCAHPS response results and statistical analysis.
The top box results for all three physician domain questions (questions 5, 6, and 7) were averaged and compared with the national database. Our study population went from 85.9% pre-intervention to 87.9% post-intervention, which correlated to the change in percentile ranking from 91st to 96th (Table 4). The national average for the Communication with Doctors top box domain was 80.2%, with SD of 4.18.10 The top box responses for question 22, “Would you recommend this hospital to your friends and family?”, went from 65% pre-intervention to 75.5% post-intervention, which was a national percentile ranking increase from 21st to 63rd (Table 4). The national average was 72%, with SD of 9.5.10 Graphical representation of the results for each question is shown in Figures 3-7 (http://links.lww.com/JG9/A48).
Power analysis performed on the study population to detect small, medium, and large effect sizes was found to be 0.237, 0.961, and 1.0, respectively.
Our study targeted orthopaedic surgical inpatients at a community hospital to determine whether a consistent rounding protocol would have any influence on their perception of care and result in changes in the HCAHPS survey scores. In our study, we found statistical significance in the positive score for the question “How well staff worked together to care for you?”. The positive score for “Would you recommend this hospital to your friends and family?” and the mean score for “How well the staff worked together?” approached statistical significance at 0.057 and 0.056, respectively. Positive trends toward statistical significance were demonstrated in almost all questions evaluated, as well as corresponding decreases at the control hospitals. Important results of note, when comparing our change in the top box score with the national database average for the question “How likely are you to recommend this hospital to friends and family?”, we showed a substantial increase in national percentile ranking going from 21st percentile pre-intervention to 63rd percentile post-intervention. This significant improvement suggests that simple interventions can improve rankings, which, in turn, could lead to increased financial reimbursement for the hospital.
Across the United States, hospitals are investing great time and expenses on different strategies to improve patient care such as interdisciplinary rounding, but there is little published evidence suggesting that these types of interventions result in improvements in patient satisfaction.11,12 It is difficult to detect changes in specific domains of the scoring system and correlate these changes to targeted interventions due to many confounding factors. This leads to hospitals investing significant resources to improve the HCAHPS scoring with the hope of generating reimbursements from the Center for Medicare and Medicaid Services, likely based primarily on anecdotal evidence.13
In the literature, studies have focused on modifiable interventions and their influence on inpatients. Mistry et al14 provided meta-analyses of factors influencing patients' hospital rating after total joint arthroplasty. They determined that the Overall Hospital Rating was significantly influenced by the Communication with Nurses, Responsiveness of Hospital Staff, Communication with Doctors, and Hospital Environment. Interestingly, they found no difference in the survey scores when comparing patients with and without postoperative complications. These specific domains have the most influence and weight on the overall HCAHPS score, and because of this, most published studies have looked at modifiable interventions that target these specific domains. Harper et al15 looked at the use of therapy dogs before physical therapy sessions on patients who underwent hip or knee replacement surgery. They found significant improvement in the Pain Management (P = 0.024), Communication with Nurses (P = 0.035), and Overall Hospital Rating (P < 0.001) domains. Other studies looked at the use of communication skill training for physicians and the use of face cards and found positive results but with no statistical significance.16,17
The doctor-physician interaction has primarily been the main focus in patient satisfaction studies. With a similar aim, Gross et al18 found that longer visits with patients led to higher patient satisfaction and a better perception of their overall care. A study by Lin et al19 reiterated the concept of patient-centered medicine and found that patient satisfaction was affected by the patient’s perceived duration of the physician visit, along with if their visit expectations were met. Similarly, the intervention in our study increased the doctor-patient interaction and lead to positive trends in the HCAHPS survey question responses.
Our study demonstrates that with a simple intervention, positive trends in the HCAHPS scores within our study population were achieved. However, despite these trends, most scores did not show statistically significant improvement. This study has several important limitations. Assessing specific interventions is difficult due to many confounding variables, such as simultaneous quality improvement measures. At our institution, quality improvements are often made at the system level, and two similarly sized hospitals in the same system were therefore used as controls. Compared with the control hospitals, the difference in improvement was greater in the study population in all questions assessed. Another limitation was our smaller cohort of patients. In the study population, our HCAHPS surveys response rate was 26.6%, which is below the New York state and national average for response rate, which is 31% and 33%, respectively. A power analysis was performed for the study hospital, and the power to detect a large effect was 1.0, meaning a 100% chance to detect any large effect in the study population if it truly existed, which was not found. Similarly, the power to detect a small effect for the study population was 0.23. This indicates that small effects would only be detected 23% of the time, and it is possible that this effect was missed due to the small sample size. Last, another limitation is the use of the HCAHPS survey as a surrogate for patient satisfaction, which has shown to be variable within the literature.20,21
The concept of patient-centered medicine can be applied across all medical fields, and the use of the HCAHPS survey is gaining popularity as a useful tool to assess satisfaction. Despite some variability in the literature for its efficacy, the use of the HCAHPS survey has been shown to have many benefits, such as increased financial reimbursement, decreased readmission rates, and shorter length of stay,21 which will undoubtedly be useful for surgeons in the future.
This study shows that simple interventions can influence positive trends in the HCAHPS surveys for orthopaedic surgical inpatients. Further research is needed to better define the impact of physician-patient interaction on patient satisfaction scores and on the overall quality of patient care.
The authors thank Larry Lutsky and Kevin Masick of the Krasnoff Quality Management Institute for the statistical analysis and study design and Joseph Narvaez for additional help in data collection and study design. The study would not have been possible without help from the Plainview Hospital orthopaedic surgery residents.
2. The Patient Protection and Affordable Care Act [HR 3590]. 111th Congress. Public Law 111-148. March 2010; 23.
4. Sacks GD, Lawson EH, Dawes AJ, et al: Relationship between hospital performance on a patient satisfaction survey and surgical quality. JAMA Surg 2015;150:858–864.
5. Kemp KA, Santana MJ, Southern DA, McCormack B, Quan H: Association of inpatient hospital experience with patient safety indicators: A cross-sectional, Canadian study. BMJ Open 2016;6:e011242.
6. Anhang Price R, Elliott MN, Zaslavsky AM, et al: Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev 2014;71:522–554.
7. Tevis SE, Kennedy GD, Kent KC: Is there a relationship between patient satisfaction and favorable surgical outcomes? Adv Surg 2015;49:221–233.
8. Lyu H, Wick EC, Housman M, Freischlag JA, Makary MA: Patient satisfaction as a possible indicator of quality surgical care. JAMA Surg 2013;148:362–367.
9. Kennedy GD, Tevis SE, Kent KC: Is there a relationship between patient satisfaction and favorable outcomes? Ann Surg 2014;260:592–598.
11. Mitchell MD, Lavenberg JG, Trotta RL, Umscheid CA: Hourly rounding to improve nursing responsiveness: A systematic review. J Nurs Adm 2014;44:462–472.
12. Townsend-Gervis M, Cornell P, Vardaman JM: Interdisciplinary rounds and structured communication reduce re-admissions and improve some patient outcomes. West J Nurs Res 2014;36:917–928.
13. Davidson KW, Shaffer J, Ye S, et al: Interventions to improve hospital patient satisfaction with healthcare providers and systems: A systematic review. BMJ Qual Saf 2017;26:596–606.
14. Mistry JB, Gwam CU, Chughtai M, et al: Factors influencing patients' hospital rating after total joint arthroplasty. Orthopedics 2017;40:377–380.
15. Harper CM, Dong Y, Thornhill TS, et al: Can therapy dogs improve pain and satisfaction after total joint arthroplasty? A randomized controlled trial. Clin Orthop Relat Res 2015;473:372–379.
16. Boissy A, Windover AK, Bokar D, et al: Communication skills training for Physicians improves patient satisfaction. J Gen Intern Med 2016;31:755–761.
17. Simons Y, Caprio T, Furiasse N, Kriss M, Williams MV, O'Leary KJ: The impact of face cards on patients' knowledge, satisfaction, trust, and agreement with hospital physicians: A pilot study. J Hosp Med 2014;9:137–141.
18. Gross DA, Zyzanski SJ, Borawski EA, Cebul RD, Stange KC: Patient satisfaction with time spent with their physician. J Fam Pract 1998;47:133–137.
19. Lin CT, Albertson GA, Schilling LM, et al: Is patients' perception of time spent with the physician a determinant of ambulatory patient satisfaction? Arch Intern Med 2001;161:1437–1442.
20. Farley H, Enguidanos ER, Coletti CM, et al: Patient satisfaction surveys and quality of care: An information paper. Ann Emerg Med 2014;64:351–357.
21. Tsai TC, Orav EJ, Jha AK: Patient satisfaction and quality of surgical care in US hospitals. Ann Surg 2015;261:2–8.
Supplemental Digital Content
Copyright © 2019 The Authors. Published by Wolters Kluwer Health, Inc. on behalf of the American Academy of Orthopaedic Surgeons