It is notoriously difficult—some say impossible—to conduct good clinical studies in medical toxicology. The acutely poisoned patient most often presents unexpectedly and needs immediate interventions, which can vary from practitioner to practitioner and confound the results of any trial. Seriously poisoned patients are relatively rare, meaning that a large number of subjects would be required to detect any meaningful change in clinical outcome.
Many major studies in medical toxicology—even ones that are widely read and frequently cited—have biases and methodological flaws because of this, and readers should be aware of those when analyzing reported results. Let's review some very well-known toxicology papers to determine if the authors' methods hold up to scrutiny.
Management of Acutely Poisoned Patients Without Gastric Emptying
Kulig K, et al.
Ann Emerg Med.
Gastric lavage and induced vomiting with syrup of ipecac were still considered standard of care for many, if not most, patients who presented after an overdose when this paper was published in the mid-1980s.
These researchers enrolled patients with an initial diagnosis of oral drug overdose. Exclusion criteria included prior emesis, ingestion of a hydrocarbon or corrosive, iron, or strychnine, and sole ingestions of ethanol or acetaminophen.
Patients were assigned based on their mental status and the date they presented to one of four treatment groups: ipecac plus activated charcoal, oral activated charcoal only, gastric lavage plus activated charcoal, or activated charcoal only given via nasogastric tube. Patients who presented on odd-numbered days went into one arm of the study; those presenting on even-numbered days went into the other.
The authors claimed that this was a randomized study, and almost always when this study is referenced, it is cited as a randomized, controlled trial. It is listed as such in the current edition of Goldfrank's Toxicologic Emergencies. Even the reviewers who assign a publication type to papers in MEDLINE have classified this an RCT.
But this is distinctly not an RCT.
As K. P. Suresh, PhD, pointed out, randomization “ensures that each patient has an equal chance of receiving any of the treatments under study.” (J Hum Reprod Sci. 2011;4:8; https://bit.ly/3UvUQfG.) Its purpose is to avoid bias before a patient is assigned to a treatment group, just as blinding is intended to avoid bias after a patient is assigned to a treatment group.
But that is not what happens when a patient is assigned to a group based on the day he presented. The researchers and clinicians know from the get-go whether the patient presented on an odd or even day, and it is clear to which group the patient will be assigned before enrollment. This leaves the door open to all sorts of bias.
Given the general pro-lavage mindset that existed four decades ago, a clinician who believed that lavage offered real clinical benefit might decide not to enroll a sick patient because it was already known that he would be lavaged as part of the study. Significantly, the paper stated, “Not included were seven patients deliberately removed from the study by the attending physician; these patients were critically ill and were lavaged outside of the randomized protocol.” This alone should invalidate any of the study's conclusions.
The even-odd day method is not actually randomization; it is pseudo-randomization. Other examples would be assignment based on the patient's birthday, last number of a social security number, or any other factor that could be known before enrollment. This does not avoid bias, and considering this a valid randomization technique is a crucial cognitive error.
Hyperbaric Oxygen for Acute Carbon Monoxide Poisoning
Weaver LK, et al.
N Engl J Med.
This paper is often cited by the pro-hyperbaric oxygen faction in the debate about whether HBO reduces the risk of delayed neurological sequelae in acute carbon monoxide poisoning. It has been referenced by 726 other publications, according to SCOPUS, putting it in the upper one percent of all publications.
This large study—152 patients—concluded that its results “support the use of hyperbaric oxygen in patients with acute carbon monoxide poisoning.” Impressive, but as usual the devil is in the details.
The two arms of the study compared a treatment group that received three sessions of hyperbaric oxygen with controls receiving three sessions of normobaric room air. This seems to be a completely unfair comparison. Even if the treatment group indeed had statistically significant improved outcomes, it very well might have been because of their increased exposure to oxygen, not necessarily the fact that the oxygen was hyperbaric.
To my mind, this renders the rest of the study moot.
Hyperbaric or Normobaric Oxygen for Acute Carbon Monoxide Poisoning: A Randomized Controlled Clinical Trial
Scheinkestel CD, et al.
Med J Aust.
This is another large study of HBO treatment in CO poisoning, comprising 191 patients in all. It concluded that HBO provided no benefit and may in fact have been associated with worse outcomes. It is often cited by those who do not believe that HBO is beneficial for this condition.
Again, the details are important. The treatment group received three sessions of HBO over three days as well as 100 percent normobaric oxygen between sessions. The placebo group received three days of normobaric 100 percent oxygen. Because 72 hours of 100 percent oxygen administration is not even vaguely similar to any treatment protocol I know, the results of the study, even if valid, cannot be generalized to real-world clinical care.
All three of these papers have been frequently misinterpreted by those who do not delve carefully into the details of their methods and potential biases. They address important issues but use methods that cannot really provide convincing answers to the questions they pose. Caveat lector!
Dr. Gussowis a voluntary attending physician at the John H. Stroger Hospital of Cook County in Chicago, an assistant professor of emergency medicine at Rush Medical College, a consultant to the Illinois Poison Center, and a lecturer in emergency medicine at the University of Illinois Medical Center in Chicago. Read his blog atwww.thepoisonreview.com, follow him on Twitter@poisonreview, and read his past columns athttp://bit.ly/EMN-ToxRounds.