“A good retrospective case series.” An oxymoron? Not necessarily.
The retrospective case series, or Level IV study, has always been, and remains today, the most common research design in our literature. When I first looked at this topic more than 10 years ago, nearly four in five orthopaedic studies were retrospective . Although it is trendy in evidence-based medicine circles to turn one’s nose up at the humble case series, there is no avoiding them, and, in fact, a majority of the 100 most-commonly cited orthopaedic articles are Level IV studies .
As the Editor-in-Chief of Clinical Orthopaedics and Related Research®, I would love to see the journal populated with randomized trials and meta-analyses. But I also know that —at least for the foreseeable future—I will see many Level IV studies for every randomized trial that crosses my desk. The retrospective reports that rise to the top likely will share several common elements.
They Will Start with Interesting, Specific Questions
Retrospective studies can be good conversation starters. They can serve as “proof of concept” before embarking on more convincing (and more expensive) prospective studies. New topics almost always make their first appearances in the literature as retrospective reports. Focusing on (or introducing) a controversy is another potential use of these reports, as has been suggested before . However the questions must be interesting and the authors must articulate very specific endpoints to start a good conversation. The paper’s methods then must be tooled specifically to answer those questions. Statements like “We report outcomes on…” or “We present the results of…” are too general to be useful.
They Will Recognize that Level IV Research Cannot Drive the Adoption of an Approach or an Implant, but a Retrospective Series of Failures Might Lead Surgeons to Abandon One
“We did 10 of these, and seven worked” is not convincing in the absence of a control group. “We did 10 of these, and seven failed” might very well be, especially if the high risk of failure is a new finding.
They Will Articulate Clearly What, and Who, was Treated
How did the treating physician decide to use the approach in question? What were the diagnostic criteria, and which patients with that diagnosis got the treatment in question? As important, which patients did not? Selection bias can be a critical weakness in Level IV therapeutic studies. Clear diagnoses and indications for treatment are critical to help readers understand whether selection bias influenced the findings.
They Will Deal with the Three Biggest Problems of Level IV Studies: Bias, Bias, and Bias
Selection bias is not the only problem with these papers. Other frequent offenders include:
- Transfer bias—loss to follow-up
- Co-treatment bias—application of additional treatments, such as other drugs or therapies, to the patients getting the new approach, and
- Assessor bias—having an interested party evaluate the patients (such as the operating surgeon), or failing to use a suitably validated outcomes tool to quantify the results.
Level IV reports can be useful. Many, if not most, of the important advances in our specialty began as retrospective case series. However, they must be written, read, and published with care.
1. Brand, RA. and Lotke, PA. Editorial - Scientific reporting: how to focus the “good ole boy paper”. Clin Orthop Relat Res.
2011; 469: 607-608. 10.1007/s11999-010-1734-5
2. Lefaivre, KA., Shadgan, B. and O’Brien, PJ.100 most cited articles in orthopaedic surgery. Clin Orthop Relat Res.
2011; 469: 1487-1497. 10.1007/s11999-010-1604-1
3. Leopold, SS., Warme, WJ., Fritz Braunlich, E. and Shott, S. Association between funding source and study outcome in orthopaedic research. Clin Orthop Relat Res.
2003; 415: 293-301. 10.1097/01.blo.0000093888.12372.d9