Secondary Logo

Journal Logo

Viewpoint

Press Ganey is a Worthless Tool for the ED

Mosley, Mark MD

doi: 10.1097/01.EEM.0000616512.68475.69
Viewpoint
Figure

Figure

Patient satisfaction is important in emergency care, but patient satisfaction scores are an unreliable measure of that in the ED. Surveys are poor scientific tools with errors and low response rates. The national average response rate for the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) surveys is around 32 percent, according to the Centers for Medicare and Medicaid Services. (http://bit.ly/2o0mMht.) Emergency department survey response rates are even lower at 3.6-16 percent. (Ann Emerg Med. 2018;71[5]:545.)

The time lag between the ED visit and when patients take the survey affects the validity of the results because emotions color the memory of the visit. The patient's knowledge also plays a significant role; if he believed an MRI was needed to evaluate his back pain, he would be dissatisfied if he did not get that test. The patient's motives for filling out the survey also affect the validity of the results. If the patient is angry because of the medical outcome or the bill or because the security guard made him move his car, he may decide to reflect that by giving every category a low number without really caring if he is grading cleanliness, nurse courtesy, or physician communication.

Those surveyed in the ED are patients who were not admitted and have not been treated in the past 90 days, who are available, and who choose to answer the survey. The hospital is allowed to choose these patients, meaning this is not a random sample of ED patients. The Press Ganey scale is also subjective and ill-defined. One patient's 5 may be another patient's 3, and the difference between a 4 and 5 may be quite narrow and the difference between 3 and 4 quite broad. The patient is also not informed that only a 5 counts. A better scale would be dichotomous, using yes or no, because people have the same understanding of yes and no, and the response clearly represents a positive or negative value.

ED patients are not consumers. Most tertiary EDs have a large population of patients who have a mental illness, struggle with alcohol and drugs, or may not have adequate financial, medical, or social resources to make better decisions. People who want to compare the ED with the airline industry or a retail establishment have never worked in an ED.

Physicians who work in a tertiary inner-city ED compared with those who work in a more affluent suburban ED get different survey results. (Psychiatr Serv. 2014;65[12]:1474, http://bit.ly/2pDhR6h; West J Emerg Med. 2019;20[3]:454, http://bit.ly/2obbYwW; Ann Emerg Med. 2016;68[5]:531, http://bit.ly/2pHY6uv.) The ED even scores differently from the hospital because the ED survey includes only low-acuity patients who weren't hospitalized. (Ann Emerg Med. 2018;71[5]:545.) Higher-acuity patients tend to score Press Ganey higher because patients who think their lives were saved tend to be more grateful for their care. This group, however, is cut out of the ED survey and given to the hospitalists and specialists.

One can find studies showing that worse mortality is correlated with better patient satisfaction scores, and vice versa. (Neurosurgery. 2018;65[CN_suppl_1]:34; http://bit.ly/2nYQZ0l; Arch Intern Med. 2012;172[5]:405; http://bit.ly/2oahtfj.) Correlations and their conclusions are only as good as the quality of the data. To draw any meaningful conclusions from such poor-quality data using Press Ganey in the ED is suspect regardless of the conclusion.

The federal government became active in patient satisfaction scores in 2002 with the 27-question HCAHPS survey. The Affordable Care Act in 2010 then tied hospitalized Medicare pay-for-performance with HCAHPS. A majority of hospitals use Press Ganey to capture the HCAHPS payment. Hospitals can pick the way they want to achieve the score, which allows them to game the system. Most senior health care executives' compensation is tied to patient satisfaction scores. (Virtual Mentor. 2013;15[11]:982; http://bit.ly/32YVjv9.) Hospital executives also have various strategies to incentivize health care providers to push for higher scores, like posting leaders on public boards, distributing unblinded comparisons among physicians, and providing financial bonuses for achieving certain scores.

CMS and hospitals don't really care if Press Ganey is reliable or accurate in its design or implementation, and administrators and physicians who have aligned with administration are mostly disinterested in its myriad faults. The responses vary from the softer “we know that no survey is perfect” to the starker “get over it—there is nothing you or I can do about it.”

This dilemma is about money, not science or patients. Some EPs believe they must play the game if they want to keep their contract. (We should all be reminded that this is exactly the mindset we took with pain scores and opioids in the ED.) Getting over the lack of science and just playing the game is professional cowardice, and it harms patients. Physicians whose compensation is more strongly linked to patient satisfaction are more likely to deliver discretionary services like advanced imaging for acute low back pain. (Arch Intern Med. 2009;169[10]:972; http://bit.ly/2M5gR2w.) Changing professional medical behavior strictly to make more money for the hospital or ourselves erodes the moral foundation of what it means to be a physician.

Press Ganey requires a minimum of 30 survey responses to draw meaningful conclusions, and will not stand by statistical analysis with fewer. The company also prefers to have 50 responses based on a theory questioned by many experts in statistics. Other statisticians base the minimum number for validity upon the number of people surveyed. A minimum of 375 survey responses is necessary for every 1000 patients seen in a quarter. In an ED where more than 30,000 patients are seen each quarter, an individual physician may receive 15-20 survey responses, including patients seen by advanced practice providers and residents. The hospital doesn't purchase more surveys because of expense. Anyone half-educated in statistics would realize this is moronic, but no one has said a word.

Displaying invalid scores to compel physicians to compete with each other to win a financial crown induces burnout, especially among those who do not win. (Neurology. May 12, 2017; http://bit.ly/2Mdmiwb.) Burnout is more aptly called moral injury in this case (STAT. July 26, 2018; http://bit.ly/2Me6ObD), which is “a deep soul wound that pierces a person's identity, sense of morality, and relationship to society.” (Pacific Standard. Jan. 22, 2015; http://bit.ly/350EWjC.) Healthy companies don't build performance appraisal systems that pit employees against each other in performance. (Forbes. Sept. 2, 2016, http://bit.ly/330LnRX; Forbes. April 7, 2017; http://bit.ly/353i6Yy.)

Furthermore, does creating competition improve scores? I imagine that it could, especially for those near the top. Perhaps a more important question is whether individual competition pulls up the bottom-dwellers or changes the aggregate mean. The most important question is does individual competition actually improve behavior in the ED and lead to patients who are truly cared for? The answer appears to be no. (Acad Emerg Med. 2017;24[9]:1051; http://bit.ly/2ocYCA9.)

Can this same change in behavior be produced in a way that does not force competition or lead to burnout or moral injury among physicians? And are there system changes unrelated to physicians that could be implemented for the cost of buying Press Ganey surveys such as pillows, warm blankets, and warm IV fluid or an extra full-time employee to clean rooms better and faster. These would unquestionably improve patient satisfaction. Of course, all of these questions are predicated on the idea that you truly want to provide a meaningful improvement in a healing experience and are willing to invest money into that goal. Most of these questions will remain irrelevant if the primary short-term goal is simply to get more money from a better score.

A response to Press Ganey:

  • We need to keep agreeing that patient satisfaction is important, find valid ways to measure it, and more importantly, develop education and systems that foster it.
  • We must keep reminding each other and administration that the science behind Press Ganey as used in the ED is invalid, that these scores do not accurately measure true patient satisfaction, and that profit is the primary reason these are used. We must reject responses like “no survey is perfect” as evasive and absurd.
  • We can accept the brutal fact that we will not change CMS, the ACA, our hospital, or the use of Press Ganey regardless of our arguments about its unreliability. We can do this, however, without succumbing to the unprofessional attitude of playing the game.
  • All ED directors should have to divulge whether they receive any financial incentive for better patient satisfaction scores and what the amount is. This is no different from giving a lecture or publishing a paper while having significant financial conflicts of interest. Reporting this financial relationship (or lack of) should be required every time patient satisfaction scores are reported.
  • Demand to know the n and response rate when reports are distributed. Educate others on why this is essential.
  • In a medical executive session, there should be a vote that survey responses with any n less than 30 (or maybe less than 50) for an individual physician cannot be reported because Press Ganey has stated that this number lacks validity. There could be legal ramifications for a hospital that imposes penalties or threatens a contract based on numbers that Press Ganey stated are inaccurate.
  • Even better, vote that no individual comparisons of physicians will be reported because of their inaccuracy and contribution to burnout. Press Ganey scores will only be reported in their aggregate for the department (along with the n and response rate).
  • At a national level, EPs could consider a class action suit against Press Ganey for promoting the 30-50 minimum as an accurate standard for ED patient satisfaction scores.

It is time for EPs to embrace the moral base of our profession—to stand up for patients and stand against anyone (even in our own profession) who would sacrifice our science and ethics purely for profit.

Dr. Mosleyis the medical director for student and resident education at Wesley Medical Center in Wichita, KS.

Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.