How often have we read claims like, “After controlling for confounding variables, patients with Medicaid were more likely to develop infection after surgery than were patients with commercial insurance,” or its evil twin, “Even after controlling for all relevant confounding patient characteristics and risk factors, patient insurance status predicts the risk of infection, and should be considered when assessing patient risk factors prior to surgery”?
Some details have been changed to protect the guilty, but if you spend much time with your nose in orthopaedic journals, you have seen claims just like those.
As analytic approaches become more robust, it is easy to become more trusting. We know that the old high-school statistics approach—using a chi-squared test to compare the proportion of patients with one kind of insurance who develop an infection and compare it to the proportion of patients with another kind of insurance—is insufficient. Researchers and readers alike know that we need to control for confounding variables and so clinician scientists throw additional factors into the mix, like smoking or BMI, and run some sort of multivariable analysis. After the experiment is over, they dragoon an unsuspecting statistician, ask that poor soul whether the whole exercise was kosher, and assert a claim of “independent association.”
Here’s why this is a problem. The corporate logo on a check—whether Medicaid, Medicare, or some commercial payor—is a social factor that has no biological component to it. It cannot possibly increase the risk of a biological complication like infection. And with rare exceptions , neither can race. That being so, any study claiming that insurance status or race predict, influence, or even are “independently associated” with medical or surgical complications—and many studies have done so—almost certainly has not controlled for all relevant variables. The best it is likely to be able to claim is that the list of factors considered in the statistical model, which almost never includes the complete roster of potentially confounding variables that matter, was unlikely to contain the culprit.
Social deprivation often goes hand-in-hand with poorer health and inadequate social support, as well as with poorer payor status. That being so, if a study evaluates endpoints that might be influenced by poor health or inadequate social support, poverty is, by definition, a confounding variable. Imagine a study that evaluates whether patients with yellow fingers are more likely to develop lung cancer. That study would need to do more than control for patient age, sex, and hair color; if that study were to fail to include cigarette smoking in its statistical model, it will draw the misleading conclusion that yellow fingers are independently associated with malignancy. Poor health may result in poverty (or vice versa), and poverty may result in a patient being insured by Medicaid, but Medicaid itself cannot cause surgical complications. Claiming otherwise may exacerbate access-to-care problems, and even nudge surgeons who accept patients with Medicaid to note the increased risk (and perhaps even counsel patients differently; “patients with Medicaid like you are at higher risk…”) without searching zealously for its actual cause, which in some patients is likely to be modifiable.
The kinds of statistical approaches I’m writing about are in widespread use, and using them is better than not trying to account for confounding variables. But as readers, it’s important that we not be lulled into a false sense of security. Assuming we share the goal of ensuring that all patients have access to good care, findings like those in the examples at the top of this essay risk having the opposite of the desired effect, especially in an era of bundled payments and institutional responsibility for costs associated with unpreventable complications. Suggesting that insurance status—or worse, race—are “independently associated” with these complications is as close to declaring causality as journals usually get. I fear that many readers then make the leap, along with the odious inferences that go along with doing so, such as imagining that patients of one or another race (or that poor patients) must be unclean or weak based on poorly reported studies suggesting they are more likely to develop infections or complicating illnesses . This, of course, has cascading effects. Hospitals and practices may screen patients based on insurance status (many already do), which has the perverse effect of resulting in delayed care, and worsening health. This biased and unfair cycle then self-sustains.
For these reasons and others, journals shouldn’t publish papers making such biologically implausible claims, and when they do, readers should turn the page. In addition to misleading readers and decreasing access of deserving patients to good care, such claims may also cause future researchers to halt the search for the real causes. After all, once we’ve found the key factors that are “independently associated” with an outcome of interest, there is little incentive to go deeper; indeed, we’ve all but said there is nothing deeper. Believing that, in that instance, would be both incorrect and harmful.
There might be some exceptions, though I think they are few, and their impact generally limited. Patients covered under worker’s compensation sometimes must contend with sets of incentives that can be at odds with uneventful recovery [3, 7], as might those patients with complicating social factors like pending litigation . Some kinds of insurance impose limits on the amount of physical therapy they will pay for after surgery. It is conceivable to me that those limits could make it more likely that a patient might develop stiffness after joint surgery, though I’ve never read any studies showing that this actually occurs. Perhaps most severely, patients with certain life-threatening orthopaedic conditions (such as malignant musculoskeletal tumors ) whose insurance coverage results in limited access to health care may have their diagnoses delayed, which certainly can reduce the likelihood of limb salvage and perhaps even compromise survival. Thankfully, given the rarity of musculoskeletal sarcomas, this sad story is as uncommon as it is serious.
Some social factors certainly cause biological harms. But in this regard, Medicaid insurance and race are different from variables like cigarette smoking, intravenous drug use, and excessive drinking; those factors are biologically linked to complications. Insurance and race (in all but exceptional circumstances ) are not. And of course, many other social factors likely influence important endpoints after surgery. For example, the absence of family support may be associated with an increased risk of readmission, the list of covariates that might affect an individual’s ability to return to work after injury or surgery is long, and social factors like race or insurance status may well be associated with the likelihood that a surgeon will recommend surgery after controlling for relevant confounding variables . Studying these things is worthy and appropriate. Finally, it is true that there are genetic factors that are linked to the probability that an individual will get a disease or recover differently from it, and some of those factors may be more common in one race than another. It is reasonable to study these questions. But studies that do so should be explicit in their suggestions that the differences in question can plausibly be genetically encoded, they should make clear the means of gathering data on race and the limitations of the approaches used , and if those studies don’t themselves evaluate the genetics of the topic in question, they should make specific suggestions as to how future studies might do so.
In general, though, orthopaedic surgeons should be skeptical of papers that link social factors that have little or no physiology to them (such as insurance status and race) with biological endpoints like infection or medical complications after surgery. Accepting such claims potentiates prejudices, decreases access to care for those patients who need it most, and may prematurely terminate the hunt for the actual causes of the complications in question—both in the exam room and in future research—resulting in preventable harm.
I am grateful to Mark C. Gebhardt MD, and Terence J. Gioe MD, for working through these issues with me in the context of papers we’ve edited together at Clinical Orthopaedics and Related Research ®. I also thank Frederick A. Matsen MD, whose shoulder arthritis blog discusses this topic . Finally, I am grateful to Raphaël Porcher PhD, whose suggestions improved this essay.
1. Congiusta DV, Amer KM, Merchant AM, Vosbikian MM, Ahmed IH. Is insurance status associated with the likelihood of operative treatment of clavicle fractures? Clin Orthop Relat Res. [Published online ahead of print June 6, 2019]. DOI 10.1097/CORR.0000000000000836.
2. Hou CH, Lazarides AL, Speicher PJ, Nussbaum DP, Blazer DG 3rd, Kirsch DG, Brigman BE, Eward WC. The use of radiation therapy in localized high-grade soft tissue sarcoma and potential impact on survival. Ann Surg Oncol. 2015;22:2831–2838.
3. Kadzielski JJ, Bot AGJ, Ring D. The influence of job satisfaction, burnout, pain, and worker’s compensation status on disability after finger injuries. J Hand Surg. 2012;37:1812–1819.
4. Leopold SS, Beadling L, Calabro AM, Dobbs MB, Gebhardt MC, Gioe TJ, Manner PA, Porcher R, Rimnac CM, Wongworawat MD. Editorial: The complexity of reporting race and ethnicity in orthopaedic research. Clin Orthop Relat Rest. 2018;476:917–920.
5. MacKenzie E, Bosse M, Kellam J, Pollak A, Webb L, Swiontkowski M, Smith D, Sanders R, Jones A, Starr A, McAndrew M, Patterson B, Burgess A, Travison T, Castillo R. Early predictors of long-term work disability after major limb trauma. J Trauma. 2006;61:688–694.
7. Styron JF, Barsoum WK, Smyth KA, Singer ME. Preoperative predictors of returning to work following primary total knee arthroplasty. J Bone Joint Surg Am. 2011;93A:2–10.