Study called a highly misleading, deeply flawed analysis based on three small studies
;)
Figure: AHRQ, Agency for Healthcare Research and Quality, ED deaths, emergency room, diagnostic errors, emergency medicine
FigureHeadlines this winter screamed the bad news: Emergency physicians are literally killing people!
“As many as 250,000 people die every year because they are misdiagnosed in the emergency room, with doctors failing to identify serious medical conditions like stroke, sepsis and pneumonia,” reported The New York Times. (Dec. 15, 2022; http://bit.ly/3H5bE7l.)
It's a bloodbath in there, chimed in CNN, warning that the ED gets it significantly wrong for every 18th patient. As a result, the news channel continued, a staggering 2.6 million people suffer a preventable harm every year. (Dec. 16, 2022; http://bit.ly/3XAwliB.)
That's more people than the population of New Mexico harmed every year—by us.
The source of these startling assertions was “Diagnostic Errors in the Emergency Department: A Systematic Review,” a 744-page (!) report published by the U.S. government. (Agency for Healthcare Research and Quality. Dec. 15, 2022; http://bit.ly/3H6rRJr.)
“We estimate,” the authors stated, “that among 130 million emergency department (ED) visits per year in the United States that 7.4 million (5.7%) patients are misdiagnosed, 2.6 million (2.0%) suffer an adverse event as a result, and about 370,000 (0.3%) suffer serious harms [death or severe disability].”
The authors said an average 25,000-visit ED might witness 50 misdiagnosis-associated deaths every year. That's a death every week!
Magic Calculations
The review was commissioned by the Agency for Healthcare Research and Quality, which describes itself as the lead federal agency charged with improving the safety and quality of health care for all Americans. Unfortunately, it's a deeply flawed analysis.
CNN reported that the authors “reviewed nearly 300 studies” from the past 20 years. But the fine print—or at least the second bullet point in the abstract!—specifies that “overall error and harm rates are derived from three smaller studies conducted outside the United States (in Canada, Spain, and Switzerland, with combined n=1,758).”
That's right. The claim that we are killing a patient every other day with misdiagnoses is based on three small studies from other countries.
Two of those studies were used to calculate the error rates. Each involved a single hospital, one in Spain, the other in Switzerland. Neither country at that time had emergency medicine residency-trained physicians.
The 2004 Spanish study looked at 250 bouncebacks within 72 hours to a hospital in the Canary Islands. (Qual Saf Health Care. 2006;15[2]:102.) The authors reported that physicians there would see three patients per hour, which is “within the international standards.” Maybe so, but it's also bangingly busy. All data were collected during a three-month period “with the highest patient census.”
Using the magic of comparing 250 bouncebacks with 250 controls, the authors calculated a diagnostic error rate of 4.1 percent. How? Well, there are Kruskal-Wallis nonparametric comparisons, there is linear regression, there are Kendall tau-b linear rank correlation coefficients. Really the only thing missing is Vinz Clortho, Keymaster of Gozer wearing an EEG made of a spaghetti colander and telling us, “During the rectification of the Vuldronaii, the Traveler came as a very large and moving Torb!” (“Ghostbusters,” 1984.)
So, yes, a 4.1 percent miss rate, calculated magically, during the worst time of the year in a single Canary Islands ED where clinicians without EM residency training cranked through three patients per hour 19 years ago.
The Swiss study looked at 755 consecutive admitted patients and found the discharge diagnosis “differed substantially” from the admittance diagnosis in 12.3 percent of cases. (Scand J Trauma Resusc Emerg Med. 2019;27[1]:54; http://bit.ly/3XTlLmn.)
Interestingly, the Swiss authors found only one thing that predicted this looming diagnostic discrepancy: when the emergency physician reported at admission that he was not sure what was going on with the patient. Are we describing errors in this study or just an ongoing collaborative process?
‘Highly Misleading’
The Swiss study combines this 12.3 percent diagnostic discrepancy in Switzerland with the Canary Islands 4.1 percent misdiagnosis rate, and calculates that U.S. emergency physicians make a misdiagnosis 5.7 percent of the time.
It sounds like voodoo. Or as our professional societies put it in a joint communique of rage: “[W]e strongly believe that it is scientifically invalid....” (Dec. 14, 2022; https://bit.ly/3XEazKM.)
What about those 250,000 U.S. deaths? Those were extrapolated from the third study—503 patients seen in the high-acuity side of two Canadian emergency departments in which one patient died. (CJEM. 2010;12[5]:421; http://bit.ly/3XA3gUx.)
One missed aortic dissection divided by 503 high-acuity patients equals 0.199 percent. When that is multiplied by 130 million undifferentiated (high- and low-acuity) U.S. emergency department visits, the result is “more than 250,000” misdiagnosis deaths.
It's pretty incredible to say that U.S. emergency physicians kill a quarter of a million people every year based on one death in Ottawa in 2004. “Highly misleading,” fumed our professional societies, “if not outright unconscionable.”
What about systems-level interventions to prevent diagnostic errors? Why aren't hospitals incentivized to build surge capacity, for example, for the utterly predictable “crisis” of an influenza-and-RSV season known as winter?
The AHRQ report goes in a different direction: It targets the physician. “ED diagnostic errors were mostly cognitive errors linked to the process of bedside diagnosis,” the report claimed. As evidence, it cites malpractice claims!
The authors asserted that 89 percent of misdiagnosis involves “failures of clinical decision-making or judgment.” The report continued: “Most often these were attributed to inadequate knowledge, skills, or reasoning.”
Ah, I see. People are dying because I'm deficient in knowledge, skills, and reasoning. This drove our professional societies to new levels of fury: “[T]o obtain payouts from insurance, unadjudicated malpractice claims intentionally target individual failures even when most experts understand that about 90% of adverse events are due to systemic failure at multiple levels,” they wrote. “Malpractice cases are highly skewed to making such assertions, without any known scientific validity.”
Dr. Bivensworks at emergency departments in Massachusetts, including St. Luke's in New Bedford and Beth Israel Deaconess Medical Center in Boston. Follow him on Twitter@matt_bivens.
