Emergency department statistics are inadequate; many times they even lie. At best, our attempts to apply meaningful numerical descriptions to department processes are terribly misleading and only make sense within a numerical construct. How we measure what we do has a narrow scope and limited applicability. What we need are numerical constructs that have the power of language to better convey meaning.
We tend to put numbers to processes and to pretend the numbers mean something in real life, but the metrics lack realistic clinical significance. National standards of ED metrics make sense only within this numerical construct and fall far short of the eloquence of language.
Measures such as length of stay, wait time, and patients seen per hour are supposed to tell us something about our departments. We often simply count what is easiest to count when it comes to management metrics, despite intellectual advances in evidence-based medicine and statistical modeling. Baseball metrics in that sense are more advanced than medicine.
Baseball author Bill James coined the term sabermetrics, which he defined as the search for objective knowledge about baseball. He recognized a deficiency in baseball metrics, not dissimilar to shortfalls in medical data mining. Mr. James exposed this deficiency in traditional baseball recordkeeping and developed formulas based on regression and correlational statistics to describe processes in more meaningful ways that became clearly more quantifiable.
He then could ask in-depth questions of scale. How great was Babe Ruth and did he really help win games? He was, after all, simultaneously a homerun king and a strike-out king. Mr. James, using expanded formulas, could quantify this qualitative question with greater precision. Rather than simply noting what Babe Ruth did, his asked about the significance of performance data. Relativity and normalization arguments help make comparisons and put interpreted results in context.
Baseball is about winning games. You must produce runs to win games. Players must get on base to produce runs. The further players advance on base, the more business that is produced to this end. Baseball statisticians now have many formulas for measuring this business.
Hoban's Effectiveness Quotient (HEQ) is one such measure: total bases + runs scored + runs batted in + stolen bases + 0.5 × bases on balls. Total bases is defined by the formula 1 (1 base) + 2 (2 bases) + 3 (3 bases) + 4 (homerun), which weight and reward base advancement toward creating a run. This is added to the runs scored plus the runs batted in plus stolen bases, adding 0.5 multiplied by bases on balls (or walks) to create the formula, which is typical of sabermetric methodology. This measures how much business was produced. It is a brilliant construct.
Mathematics professor Michael Hoban said this consistently measures a player's effectiveness at generating runs. A player with an HEQ of 600 had an excellent year, and that score provides much more meaningful and comparable data than the one-dimensional measures of homeruns or RBIs alone.
Sabermetrics was a radical development to understanding the sport. Bestselling author Michael Lewis called this method "eloquent" in describing the work of Bill James and others in his book, Moneyball, which itself has become a bestseller that now guides Fortune 500 companies.
Sabermetrics even has a park factor that quantifies the effect a field has on a team. The numerator of the park factor formula is quite simply the average runs scored at home games, and the denominator is the average of runs scored on the road.
What does this have to do with emergency department metrics? What does it mean if an emergency physician sees an average of two patients an hour and the average wait time is three hours? These are terribly one-dimensional measures of efficiency.
These may be great numbers in a busy department, but these numbers are terrible in a department with low volume. No park factor exists in medicine. How large is the department? How much staff does it take to conduct this amount of business? It has no context.
Why not consider more meaningful formulas? Formulas such as V/S × R = throughput capacity where daily patient volume (V) is divided by staffing (S) multiplied by room capacity of the department (R). Say we have 100 patients a day. We divide that number of patients by 10 nurses multiplied by a 15-bed department to get a score of 0.67. Now let's change one variable. Consider another 15-bed department with 100 patients but nine nurses. Now the throughput score equals 0.74.
Throughput now has more quantifiable meaning in context. The number has eloquence: the higher the number, the greater the department efficiency with available resources, which are factored into the equation. This type of measurement allows comparative contextual analysis.
How about patient satisfaction scores? A department strives for 100 percent satisfaction, but is this meaningful or realistic in context? Let's say 10 percent of patients who present to an ED are drug seekers feigning disease to obtain drugs. Obtaining a narcotic illegally is the only goal that will satisfy these individuals pretending to be patients, and they complain of not being satisfied if they do not succeed in this illegal pursuit.
If the department achieves a satisfaction score of 95 percent, this means that drug-seeking patients succeeded in accomplishing their illicit goal half the time. Shouldn't the satisfaction goal in this construct be closer to 90 percent? Where is the context?
When do lengths of stay and wait times become negative-sum pursuits that sacrifice quality care at the expense of quality metrics? Clearly, the patient held for more time to conduct a CT for abdominal pain should be weighted within the statistical construct as advancing bases are weighted in baseball and case-mix adjusted. Sending an abdominal pain patient home quickly with an unclear diagnosis may produce a better metric but sacrifices quality. Where is the case-mix adjustment?
Baseball managers no longer measure players simply by homeruns or RBIs without sophisticated context. That costs money. They have learned how to hire an unknown player that really helps win games rather than pay an exorbitant amount for another Babe Ruth who does not. The game has been revolutionized, and teams are hiring statisticians as well as jocks.
But we do not measure emergency physicians that way even though the stakes are life and death. Should not the measure of an emergency physician be just as sophisticated, meaningful, and contextual?
What does it mean if one physician sees 2.5 patients an hour and another physician sees two an hour? Is the ED volume or acuity level of patients they saw factored into the equation? Baseball is more efficient in factoring these types of mitigating elements.
What about pain scores? Does anyone really trust the 10-point Likert scale? Doesn't a patient who rates his pain as a nine while sipping a soda and another who rates his pain as a five while bent over grimacing prove that more in-depth formulas are needed to quantify a qualitative measure?
What about having the patient rate his pain on a nine-point scale and allow a medical provider to add a modifier? A nurse could supply the first number in the formula, with one being no obvious pain, two being mild objective pain, and three being severe objective pain; the patient would supply the second subjective number. This method would give our soda-sipping patient a pain score of 1–9 and our grimacing patient a score of 2–5, which is much more useful.
A merger between reason and medicine must occur to create a new value system that evaluates ED efficiency. Emergency medicine can be revolutionized. We simply must consider a new way of thinking about how we measure things. We need eloquence of process measurement by developing data in meaningful fragments that ultimately provide depth to our understanding of ED efficiency.
Rather than reacting to ED crowding by a methodology akin to placing one-dimensional bets on a table hoping to win, we should be more like sophisticated card counters, gaming the system in multiple dimensions to better understand the statistical information.
We need a new language to talk about what we do in emergency medicine. What we need are numbers with the power of language. We need fewer reactive policy makers and more proactive card counters. We need to discard one-dimensional constructs. We need to do the same thing for medicine that sabermetrics did for baseball — it revolutionized the game.
