Estimating Hospital-Related Deaths Due to Medical Error: A Perspective From Patient Advocates

Abstract

The authors present a viewpoint regarding the quality of data used in estimating the number of preventable hospital deaths in the United States. Data derived from countries with a nationalized healthcare system with well-defined and near uniform implementation of standards may not be applicable to the fragmented noncentralized delivery system found in the United States. Although U.S. studies evaluating preventable mortality have based their projections on a small sample size, it is unlikely that this observation is due to chance, because other studies evaluating adverse events, a precursor to preventable mortality, have a much larger sample size and also report an unacceptably high number of events. In addition, although these estimates involved adult and Medicare-eligible patients who may have a higher incidence of events and create a bias, but they also did not capture all events, taken into account of mortality, which occurs after hospitalization or from misdiagnoses. It is also important not to mitigate adverse events in patients whose death is imminent. Medicine does not have the moral authority to place differing values on days, weeks, or years of life. The contention that there are approximately 200,000 preventable hospital-related deaths each year in the United States is not unreasonable. Not all hospital systems in the United States make the same investment in patient safety. Recently, the Agency for Healthcare Research & Quality has demonstrated a decline in adverse events in hospitals, but until uniform implementation of safety standards takes place, our healthcare system as a whole may well lag behind other industrialized nations.

A debate has emerged for the last several decades about the safety of the United States' healthcare system and the number of patients who die each year in hospitals because of medical errors. The most recent estimate was by Makary and Daniel¹ and published in The British Medical Journal, and the estimated number is more 200,000 patients annually.

Why is estimating error-related mortality so important? It obviously represents the most severe outcome that can be caused by a wide variety of preventable complications including the following: bed ulcers, infections, embolism, surgical error, misdiagnosis, etc. Unlike other adverse outcomes, mortality can be unequivocally defined. Granted, the overall metric that determines risk and preventability adds layers of complexity, at least everyone agrees with the definition of the event. Many argue over when an infection occurs or when a fall should be a reportable event but death is a definitive and nondebatable outcome. Prevention mandates that institutions invest in patient safety and foster a culture of safety among staff and administration.

Many in the healthcare industry assert that the estimations of preventable mortality are too high.^1,2 However, Makary and Daniel wrote just 1 of 3 recent reports that used varying methodologies and gave estimates of preventable deaths of well more than 100,000. These include the following:

• September 2013: John James in the Journal of Patient Safety estimated that there were between 210,000 and 440,000 preventable hospital deaths.³ This study has been highly quoted in the literature and reopened the discussion regarding preventable mortality from medical errors. What is most striking is that this report did not have its genesis within the healthcare industry but was from a National Aeronautics and Space Administration scientist, John James, who lost his son because of a medical error.
• April 2016: The Leapfrog Group estimated 206,201 avoidable deaths in hospitals.⁴
• May 2016: The report by Makary and Daniel from Johns Hopkins University who estimated the deaths at greater than 250,000.¹

The analysis by Makary and Daniel has been criticized^1,2 to a large extent on the basis of its use of poor metrics, such as the Agency for Healthcare Research and Quality (AHRQ) patient safety indicators (PSIs) and lack of preventability determination of the Global Trigger Tool.⁵ The accuracy of PSIs was questioned by Winters et al, who compared PSIs from administrative data with those from an independent chart review.⁶ Patient safety indicators from administrative data both under (sensitivity) and overestimated (positive predictive value) the presence of adverse events; however, the former seemed to be of greater magnitude.⁶ Overall, PSIs from an independent chart review may miss a substantial number of adverse events in a facility, including those not readily defined as a PSI.⁷

Makary and Daniel analyzed 4 different studies and all 4 projected preventable mortality well over the 44,0000 to 98,000 projected deaths in the 1999 Institute of Medicine Report.⁸ The Healthgrades study⁹ used PSIs, but only for a portion of its data. The other 3 studies used the Global Trigger Tool to identify events. In 2 of these studies, preventable harm was determined by a additional chart review.^10,11

The LeapFrog Group measures hospital safety using a total of 30 publicly reported metrics including laboratory reporting of Methicillin-resistant Staphylococcus aureus and Clostridia Difficile and 5 patient survey measurements (Hospital Consumer Assessment of Healthcare Providers and Systems metrics).¹² Of the 30 metrics, Winters et al⁶ question 2 which comprise approximately 5% of the composite score. This argues in favor of the overall validity of the Leapfrog estimate, not against it. The Johns Hopkins researchers, who performed the Leapfrog study, also discussed how their findings are likely conservative, because they applied mortality rates only to a subset of known safety problems.

James³ reviewed 4 studies, all of which used the Global Trigger Tool. Two of the studies also determined preventability^10,11 and comprised 75% of the records analyzed in James-weighted estimate.³

Performing an analysis using the methodology described by James³ on the 2 studies that used the Global Trigger Tool and also determined preventability^10,11 yields an estimated preventable annual hospital mortality of 163,156. If one adds to this figure the deaths from unnecessary surgery, along with the understanding that the Global Trigger Tool does not capture all events,² including those that are delayed and occur on an outpatient basis, the estimated preventable hospital-related mortality can be projected above 200,000. Taken into consideration of the subject population, these studies are based on is skewed toward older and sicker patients who would be expected to have a higher incidence of adverse events, but this estimate also does not take into account the deaths from diagnostic errors.^13,14 The latter is largely preventable and may account for up to 5% of hospital deaths in the United States.¹⁵ It has been estimated that there are between 715,000 and 776,000 hospital deaths each year in the United States,¹⁶ and this would add up to an additional 35,750 to 38,800 preventable deaths.

Several arguments have been put forth to question the validity of this estimate and to assert the safety of our healthcare system. The major ones are as follows:

• 1. Causality of death: The contention has been made that researchers did not demonstrate that the observed preventable adverse events caused the deaths, which have been attributed to them. Proving causality of each incident has not been a standard used by the healthcare industry for other important public health matters. For instance, causality was not initially proven by researchers for deaths related to smoking, but the research was compelling enough to drive important policy decisions that served many lives.
• A single causative event is often difficult to define. For example, if a patient dies of an acute myocardial infarction, was this the cause of death? Or was it high cholesterol, obesity, diabetes, or smoking? Or perhaps was it because the patient waited in emergency triage for 2 hours and did not receive thrombolytic agent in time. One could make a case that any one of these could be the cause of death.
• The causality of death is not well established by current reporting systems. Information on cause of death is currently derived by the Centers for Disease Control and Prevention from death certificates. Many consumer advocates would like a multitude of other diseases listed as checkbox items, including smoking, drug-resistant organisms, and medical errors. Because of the complexity and multitude of problems in the determination of causality and the myriad of provider types, including funeral directors,¹⁷ filling out certificates with varying accuracy, this source is unlikely to produce comprehensive data.
• In comparison, when the National Transportation Safety Board (NTSB) investigates a crash, it lists all primary and contributing factors. These factors are not treated any differently from one another, and the complexities of proving causality is not used to mitigate the impact of an action taken regarding a contributing factor. The NTSB goes to extraordinary lengths to understand the probable cause and contributing factors of an accident to formulate future interventions, which are designed to prevent future events. Until healthcare can produce similar reliable data, we will need to rely on self-reported events, chart reviews, trigger tools, and claims analysis to estimate causes. Recording devices analogous to aviation's black boxes, such as operative room cameras, have not been embraced by the industry.
• 2. Many “preventable” deaths involve very sick or near terminal patients: The fact that some of these deaths occur in terminally ill patients does not diminish the importance of the numbers. Even if a terminal patient dies prematurely from a preventable adverse event, this is still an unacceptable occurrence. Medicine does not have the moral authority to discount or disregard days, weeks, or months of life. Even if the unintended event only shortens life by a few days, it does not mitigate the severity of the event. One should not confuse the ethical debates surrounding end-of-life care and decision making with the undisputable tragedy of prematurely precipitating death from a preventable event.
• When reporting airline crash deaths, the NTSB reports the total deaths. If 350 people die, the NTSB reports that the crash caused the death in 350 people, not 348 because two of them had terminal disease and would have died anyways.
• 3. It is the patient's fault: An error, which occurs in administration of the treatment, cannot be mitigated because of poor patient life choices, which caused the need for the treatment. Similarly, many patient advocates suggest that the best way to avoid preventable adverse events and error-related mortality is to stay healthy and avoid going to the hospital. However, such arguments, based on the patient's lifestyle choice, are similar to the NTSB, mitigating the deaths from a preventable train accident, because passengers did not choose to travel by airline, which has a better safety record. Thus, it is not the patient's fault.
• 4. The United States uses the same technology as other higher performing countries: Some have used international studies to bolster their contention that the United States' system is providing safe and effective patient care. However, using studies derived from countries with an integrated nationalized healthcare system to estimate the quality of the U.S. healthcare system is not valid, because the United States has a fragmented system with little centralized or national control. Operational structure and philosophy vary widely between hospital systems, making uniform adoption of healthcare policies problematic.
• For example is a study published by Hogan et al¹⁸ who found that 3.6% of hospital deaths were likely avoidable. Using this finding, Shojania and Dixon-Woods¹⁵ projected that the United States would have 25,000 preventable hospital deaths. However, Hogan et al¹⁸ studied hospitals in the United Kingdom's National Health Service (NHS), a nationalized integrated healthcare system. We do not feel that data from NHS hospitals should be used to estimate the performance of hospitals in the United States,
• The NTSB calculates statistics and performance of the airline industry only on the basis of airline carriers in the United States. Even if a foreign plane crashes in the United States, it is not included. The NTSB realizes that safety oversight and administration vary widely between countries.
• 5. Better data are needed: All are in agreement that better data are needed. Some have claimed that the data collection and reporting problems result in an exaggeration of numbers of preventable deaths. On the contrary, an abundance of literature suggests the data problem results in an underestimation of events. There is great concern among patient advocates that adverse events, preventable harm, and deaths are underreported in voluntary and nonaudited reporting mechanisms. These concerns have been fueled by a 2010 Office of Inspector General Report that found diagnostic codes “absent or inaccurate” in 7 of 11 Medicare hospital-acquired conditions (HACs), and in 93% of the time, incident reports were not submitted.¹⁹ In addition, since 2005, an average of less than 1000 sentinel events have been voluntarily reported to The Joint Commission per year from the hospitals that have undergone their accreditation process. Of these hospitals, there was an average nationwide total of less than 50 reports of medication errors per year.²⁰ Of the sentinel events reported, more than 5000 resulted in patient deaths. Officials at The Joint Commission estimate that less than 0.1% of events have been reported.²¹ The AHRQ has reported that almost half of the more than 600 surveyed facilities (196,462 respondents) have not reported any adverse events at their institution for the previous 12 months.²²
• Sjojania and Dixon-Woods¹⁵ have also criticized the data because the observed “N” for preventable deaths was small,^10,21 and thus, the observation may be based on random effects. These concerns should be mitigated by the finding that the observed rates of adverse events, which are the precursors of preventable deaths, were based on a larger “N” and were also at an unacceptably high level.^7,10,11,23 In addition, one study from the United States Department of Health and Human Services' Office of Inspector General followed a smaller pilot study, and both studies had similar high rates of adverse events and deaths.^11,23
• Even the submission of incident reports has been brought into question some. At least 1 patient safety expert has called incident reporting “the single biggest waste of time in the last 15 years.”² However, in actuality, incident reporting has served as the cornerstone of safety in many industrial sectors, and in the noncentralized dissimilar healthcare systems in the United States, the occurrence of seemingly disparate metrics and events may be linked to the facility's willingness to foster a culture of safety and invest in patient safety.
• Completeness of the medical record is also a problem that has been exacerbated by cut-and-paste functions contained in electronic medical records²⁴ Thus, chart reviews would not be expected to yield complete data. There are strong indications that voluntary and nonaudited systems have significant problems with the integrity of the data, both in underdocumentation and underreporting of events. All of these concerns would be expected to bias the data toward an underestimation of event occurrence.
• The NSTB investigates all aircraft accidents, near-misses, and certain incidents in which there was not an accident. All accidents and incidents—not just a sample—are tracked. In the U.S. healthcare system, only a small subset of events are reported or investigated. For example, the United States does not have accurate numbers for healthcare-acquired infections, such as Methicillin-resistant Staphylococcus aureus. With the current functioning of our healthcare system, it would not be possible to investigate every adverse event, but universal reporting and identifying error-related mortality needing investigation should be possible to offer a better platform for estimating the full extent of the problem and to perform root analyses of the causes leading to more effective preventive interventions.

NATIONAL TRANSPORTATION SAFETY BOARD

We have used the NTSB to illustrate shortcomings in the United States' patient safety initiatives. Many healthcare advocates are calling for the formation of a body similar to the NTSB to at least investigate clusters of adverse events.²⁵ The Joint Commission, an organization born out of the healthcare industry to promote the quality and safety of care, can also be called on to perform NTSB-like functions, such as investigating adverse events. They provide an onsite review that relies not only on the medical records but also on interviews and observations. However, the accuracy of The Joint Commission surveys has been brought into question by a recent Centers for Medicare & Medicaid Services report, which found more than 40% of their accreditation surveys to be out of compliance. Today, even industry leaders recognize that the self-policing policy of hospitals investigating themselves is not optimal to assure quality.²

WHY ARE THESE NUMBERS AND MEASURING MORTALITY IMPORTANT?

Some have asserted that the “problem in patient safety has been unrealistic expectations that hospitals can tackle problems, which we do not yet have a solution,”² is contradicted by decades of established research. Advocates are not calling for the prevention of problems for which solutions are not known, but calling to implement known solutions. Failure to adopt best practices system-wide is not just in the healthcare industry. The GM-Toyota Joint Venture in the mid 1980s produced a safe and superior car, but GM was unable to implement the culture and techniques in their other plants.²⁶ The U.S. healthcare system is even more resistant to change with its fragmented nonuniform systems without centralized control.

Despite having the knowledge to prevent adverse events, many health systems do not adequately invest in patient safety to put well-known safety improvement strategies in place. For example, there is a large patient safety initiative calling for mandatory nurse-to-patient ratios. The most common HACs observed by the AHRQ are adverse drug events, pressure ulcers, falls, and catheter-associated urinary tract infections.²⁷ Adequate nursing is a prerequisite to preventing falls, pressure sores, and urinary tract infections. In addition, nurses also intercept many prescription errors. Nonetheless, too many institutions are not adequately investing in nursing. Many becoming cost-driven with the largest portions of a facility's operating budget being staffing.²⁸ Having adequate numbers of trained staff and a culture of safety is expensive and is the common denominator that cuts across many different types of adverse events and explains why patient safety surveys have been found to correlate with many different types of safety metrics and a culture of safety.^29–31

WE CAN DO BETTER

The United States healthcare system as a whole can substantially decrease the incidence of adverse events and associated deaths. The AHRQ has estimated using a 2010/2011 baseline that 44% of HACs are preventable.²⁷ The AHRQ has reported that preliminary 2015 data indicate that the Partnership for Patient Program has resulted in a 21% decrease in HACs between 2010 and 2015, allowing a projection of 125,000 deaths prevented in hospitals, more than 37,000 fewer deaths in 2015 alone.²⁷ In addition, a Leapfrog Group study estimated 33,429 lives would be saved if all hospitals performed at a similar level to hospitals, which achieved a safety score of “A.”³² Deaths from adverse events are all too common and preventable. Much more work needs to be conducted, because we are not yet to the halfway point in the AHRQ reduction goals.

CONCLUSIONS

Advocates are not calling to prevent problems for which solutions are not known but calling to implement known solutions to prevent all too common problems. What ties the occurrence of preventable adverse events and mortality together is the willingness and determination of facilities to adopt a culture of safety and invest in patient safety. The adoption of preventive protocols is further hindered by the United States' fragmented, nonuniform healthcare systems composed of facilities with differing philosophies and administrative structure.

Some in the healthcare industry suggest the data are inadequate, and therefore, the estimates of death and injury are wrong. As a result, we should ignore current studies and wait for more perfect data. However, the onus should not be on consumers but on the healthcare industry to generate comprehensive data to demonstrate that their product is safe.

Even if the death rate from medical errors is not the 163,156 that we have projected but is as low as the 25,000 per year based on the United Kingdom's NHS data,¹⁵ that equates to approximately 5 potentially preventable deaths per year per hospital in the United States or 1 every 2 to 3 months. In addition, one could argue that this figure should be doubled by accounting for deaths from diagnostic errors. In what other industry would such a record be tolerated, let alone defended? Would the airline industry and public ever tolerate even a single preventable airline crash? We can and must do better.