There is great agreement that randomized controlled trials (RCTs) produce the most reliable evidence about the benefits and risks of newly developed or already existing clinical interventions, ultimately leading to better care for patients.1 The quality of an RCT depends on several methodological issues that need to be considered. Blinding of practitioners, participants, and outcome assessors are crucial to avoid performance and detection bias.2 Compared to drug trials, surgical RCTs face several challenges. For example, surgeons can normally not be blinded. Nevertheless, patients and outcomes assessors can usually be blinded and investigators of RCTs should undertake every possible effort to incorporate blinding, especially when “soft” endpoints are evaluated.2–4 A systematic review from 2008 found that only 10% of outcome assessors in orthopedic trauma RCTs were blinded.5 This number is worrying especially when having in mind that most guidelines and clinicians decisions are based on RCTs.6 Therefore, this mini-review assessed the blinding and reporting of blinding in the most prestigious medical and surgical journals in 2015.
A search in PubMed was conducted from January 1, 2015 until December 31, 2015 for any surgical RCTs in the 10 general medical journals with the highest impact factor and in the 10 surgical journals with the highest impact factor (according to the Thomson Reuters InCites Journal Citation Reports; search strategy within the Appendix, https://links.lww.com/SLA/B209). Included were surgical RCTs, which compared different surgical interventions or 1 surgical intervention versus a conservative treatment. Excluded were studies, which assessed different teaching methods, treatments before surgery (eg, chemotherapy vs. chemoradiotherapy), different drugs during surgery (eg, anesthesia trials), and different interventions after surgery (eg, physical training, different pain management programs). The following informations were extracted from all relevant publications: population, intervention and control, description of the blinding status of the surgeon, patient and the outcome assessor, and the primary outcome. All methods of blinding were accepted without any classification. Primary outcomes were further stratified into (i) patient-reported outcomes, (ii) “hard” outcomes that are easy to measure and not readily subject to observer bias (no room for interpretation; eg, mortality, body mass index)7; (iii) “soft” outcomes that are more subjective (influenced at least to some degree by the judgment or effort of the outcome assessor; eg, complications, lymph node detection rate)7; and (iv) no specification of primary outcome.
The computer-aided search yielded 301 publications from PubMed. Based on titles, abstracts, and full text 99 publications were classified as relevant. The most common reasons for exclusion were that the intervention tested in the RCT was not specifically surgery related (eg, different drugs given; n = 72), that the intervention took place before or after surgery (n = 44), or that the study was not an RCT (n = 30; Fig. 1A). In 57 RCTs it was explicitly stated that the surgeon was not blinded (Fig. 1B). In 39 studies no specific statement regarding the surgeon was given; however, 5 out of these 39 studies mentioned that the trial was “open-labeled” or “single-blind” together with the information who was blinded (eg, outcome assessor). Within 3 of the 99 RCTs blinding of the surgeon was possible and also reported. Patients were blinded in 37 out of the 99 RCTs. In 40 studies it was clearly stated that patients were not blinded, whereas it remained unclear in 31 RCTs. In 8 out of these 31 studies it was, however, either highly unlikely that patients were blinded (ie, surgical intervention vs alternative treatment) or they were stated as “open-labeled” or “single-blind” together with the information who was blinded (eg, outcome assessor). The outcome assessor was clearly stated as blinded in 51 RCTs, whereas the assessment was not blinded in 24 RCTs. Furthermore, 24 studies remained unspecific about the blinding of the outcome assessor (including 3 “open-labeled” studies and 1 “single-blind” study with blinded patients). The primary outcomes of the 48 studies that had unblinded (n = 24) outcome assessors or were unspecific about the status of the outcome assessor were the following (Fig. 1C): (i) patient-reported outcomes (eg, pain, disease-specific score; n = 12); (ii) easy to measure “hard” clinical outcome (eg, mortality, change in body mass index; n = 19); (iii) had a “soft” clinical endpoint that could potentially be influenced (consciously or subconsciously) by the outcome assessor (eg, complications, lymph node detection rate; n = 15); (iv) did not specify the primary outcome (n = 3).
This mini-review indicates that 52% (51 out of 99 RCTs) of outcome assessors of surgical RCTs that were published in high-impact journals were blinded to treatment allocation. This number is considerably higher than the 10% blinded outcome assessors of all orthopedic RCTs in peer-reviewed journals from 1995 until 2004.5 The systematic research in the present study was, however, only conducted in high-impact journal, because they were deemed to have the highest effect on decision making. Hence, these results cannot be transferred to all peer-reviewed surgical RCTs. The majority of the remaining 48 RCTs in which outcome assessors were not blinded or the status was not specified, had a patient-reported primary outcome or a “hard” clinical outcome (n = 12, n = 19, respectively; total n = 31); hence, there was no urgent need in these studies to blind the outcome assessors. A total of 15 studies, however, had a “soft” outcome in which blinding of outcome assessors would have been possible (eg, complications, lymph node counts).
In addition, the analysis presented here shows that the blinding status of surgeon, patient, and outcome assessors remains often unclear even in prestigious journals. It is clear that surgeons can usually not be blinded and that it is often complicated to blind patients in surgical studies (eg, open vs laparoscopic operation). Nevertheless, the status of blinding should be reported for each of those involved subjects. The commonly used terms such as “double-blind” or “single-blind” were often used without further specification, or even worse, were misleading. For example, a study, which was labeled as “single blind” but stated that “the patients, investigators who completed follow-up, vascular-laboratory personnel, core laboratory evaluators, and members of the clinical-events committee were unaware of the treatment received”8 or as an “open label” and specified later that outcome assessors were blinded.9,10 Hence, authors and journals publishing RCTs should pay attention that ambiguous wording such as “double-blind” without a clear specification is avoided.4
1. Duley L, Antman K, Arena J, et al. Specific barriers to the conduct of randomized trials. Clin Trials
2. Higgins JPT, Sally G, Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. 2011. Available at: www.handbook.cochrane.org
. Accessed February 24, 2017.
3. Evaniew N, Carrasco-Labra A, Devereaux PJ, et al. How to use a randomized clinical trial addressing a surgical procedure users’ guide to the medical literature. Jama Surg
4. Farrokhyar F, Karanicolas PJ, Thoma A, et al. Randomized controlled trials of surgical interventions. Ann Surg
5. Karanicolas PJ, Bhandari M, Taromi B, et al. Blinding of outcomes in trials of orthopaedic trauma: an opportunity to enhance the validity of clinical trials. J Bone Joint Surg Am
6. Kennedy HL. The importance of randomized clinical trials and evidence-based medicine: a clinician's perspective. Clin Cardiol
7. Zanolla L, Zardini P. Selection of endpoints for heart failure clinical trials. Eur J Heart Fail
8. Rosenfield K, Jaff MR, White CJ, et al. Trial of a paclitaxel-coated balloon for femoropopliteal artery disease. N Engl J Med
9. Mimoz O, Lucet JC, Kerforne T, et al. Skin antisepsis with chlorhexidine-alcohol versus povidone iodine-alcohol, with and without skin scrubbing, for prevention of intravascular-catheter-related infection (CLEAN): an open-label, multicentre, randomised, controlled, two-by-two factorial trial. Lancet
10. Park SJ, Ahn JM, Kim YH, et al. Trial of everolimus-eluting stents or bypass surgery
for coronary disease. N Engl J Med