Did Experimenter Bias Conceal the Efficacy of Spinal Opioids in Previous Studies with the Spinal Nerve Ligation Model of Neuropathic Pain?
Eisenach, James C. M.D.*; Lindner, Mark D. Ph.D.†
THE current issue of Anesthesiology contains a report of a laboratory study 1
that addresses the question, “Is it appropriate to treat neuropathic pain with spinal opioids?” This editorial examines the data that address this question, and the possibility that the differences between laboratory studies reporting positive and negative effects may be attributed, at least in part, to experimenter bias. We recognize that efficacy with opioids, and perhaps other analgesics, may be lost over time with chronic treatment, and that the results of opioids may also differ according to patient population, laboratory model, and route of administration. Therefore, we focus this discussion on whether spinal delivery of opioids has analgesic effects, regardless of the duration of action, in the commonly used spinal nerve ligation model of neuropathic pain. 2
Initial clinical reports of chronic intrathecal morphine treatment, published 20 yr ago, showed both acute and sustained efficacy in patients with chronic pain, including neuropathic pain, although dose escalation occurred with time. 3,4
At the same time, others reported little or no analgesic effects of spinal morphine in neuropathic pain patients. 5
Later reports suggested that spinal opioids produce partial pain relief in neuropathic pain patients, 6,7
and other studies reported that intrathecal opioids produce substantial or good analgesic effects in patients with neuropathic pain, even after long-term administration. 8,9
Thus, the clinical literature regarding efficacy of spinal opioids for neuropathic pain has evolved from initial good efficacy, to poor efficacy, and, more recently, back to good efficacy in many patients.
A laboratory model of neuropathic pain in rats in which ligation of low lumbar spinal nerves results in reduced withdrawal threshold to tactile stimulation of the paw, akin to mechanical allodynia in patients with neuropathic pain, was first described over a decade ago. 2
Initial studies using this model showed it to be sympathetic-dependent, with hypersensitivity resolving after chemical or surgical sympathectomy. 10
Several laboratories, including ours, showed that the spinal nerve ligation model was also resistant to treatment with intrathecal morphine. 11–13
These were considered important validation studies because neuropathic pain was at that time thought to frequently be sympathetic-dependent and, as noted above, resistant to treatment with intrathecal morphine.
This Editorial View accompanies the following article: Zhao C, Tall JM, Meyer RA, Raja SN: Antiallodynic effects of systemic and intrathecal morphine in the spared nerve injury model of neuropathic pain in rats. Anesthesiology 2004; 100:905–11.
Neuropathic pain is now considered to be only infrequently sympathetic-dependent and often responds to spinal opioids, and these changes in understanding of clinical neuropathic pain have been mirrored by changes in results obtained in this spinal nerve ligation model. For example, the current authors more recently noted that the effect of sympathectomy was smaller than originally described and more variable, depending on rat strain, 14
and we 15
and others 16
failed to observe an effect of sympathectomy on hypersensitivity using this model. In addition, the current report 1
demonstrates full efficacy of intrathecal morphine to reduce hypersensitivity to mechanical stimulation in rats with spinal nerve ligation or spared nerve injury, another model of neuropathic pain. The dose of intrathecal morphine found to be effective was small, and, as noted by the authors, was similar to that needed to treat nonneuropathic acute and chronic pain in these species.
It is somewhat reassuring that the data from the spinal nerve ligation model now seem to be consistent with the growing consensus that neuropathic pain is not very sympathetic-dependent and that spinal opioids are often effective in patients with neuropathic pain. However, these findings also raise the concern that the results obtained in studies with the spinal nerve ligation model of neuropathic pain may be affected by the expectations of the experimenters at the time the studies are conducted. In fact, Zhao et al.1
were careful to use blinding procedures to prevent experimenter bias from affecting their results, and they suggest that experimenter bias could account for the negative results of earlier studies. Results of studies conducted in our laboratory support this suggestion. We previously reported that intrathecal morphine was ineffective after spinal nerve ligation, 12
but in a recent, rigorously blinded replication of our initial nonblinded study, we detected analgesic effects of intrathecal opioids (unpublished data) at doses similar to those reported by Zhao et al.1
In addition to the differences in blinding procedures, many other factors could also contribute to the differences between the results of these studies. For example, even seasonal changes in the source of the protein included in commercial rodent chow—with no change in the total protein or calorie content—can significantly affect the results of studies with these laboratory pain models. 17
It is possible that such subtle differences in diet or some other unidentified and uncontrolled factor could also contribute to the differences in the results between these studies, so it is not appropriate to conclude that all of the differences in the results of the studies with the spinal nerve ligation model discussed above can be attributed to experimenter bias in the nonblinded studies.
However, experimenter bias is a real phenomenon that has been clearly demonstrated in clinical research. For example, in a clinical trial for multiple sclerosis, patient evaluations performed by nonblinded clinicians detected statistically significant therapeutic effects, but patient evaluations performed in the same study by blinded clinicians revealed that the treatment did not produce beneficial effects. 18
Apparent therapeutic effects are often observed in initial, small, nonblinded clinical trials, but not in larger, blinded trials. For example, in a series of clinical studies conducted to test the therapeutic potential of a monoclonal antibody for rheumatoid arthritis, all of the initial, small, nonblinded trials reported therapeutic effects of the antibody. 8
Subsequently, three large, blinded experiments, some of them conducted by the same investigators who had conducted some of the initial nonblinded studies, all failed to detect any beneficial effect of the treatment. 19
This scenario is so common in clinical trials that it is not considered at all surprising. In these cases, the beneficial results of the treatment reported in the nonblinded trials are typically attributed to experimenter bias. For this reason, decisions about whether clinical treatments are truly effective are often based only on the most well controlled studies. For example, systemic opioids are now considered one of the first-line treatments for neuropathic pain, but that decision was based on only five recent studies, all of which were large, randomized, double-blind, placebo-controlled clinical trials (for review see Dworkin et al.20
). Clearly, there is a need for strict controls to prevent experimenter bias, which we recognize when testing the potential efficacy of treatments in humans.
Classic studies published more than 40 yr ago demonstrated that experimenter bias could also affect the results of laboratory studies. For example, in a conditioning study, planaria were given a 3-s light cue followed by a 1-s shock. Observers recorded whether the planaria made contractions or head turns during the light cue, which were the dependent measures used to determine if the planaria were learning to associate the light with the shock. Increasing numbers of contractions and head turns in response to the light would be evidence of classic conditioning. Observers who had been told to expect rapid conditioning recorded significantly increased numbers of anticipatory movements, compared with observers who had been told to expect little or no evidence of conditioning. 21
In a learning study in rats, experimenters trained rats to run to the dark arm of a T-maze for a food reward. Rats were given 10 trials each day for 5 days. Experimenters who were told to expect their rats to demonstrate the task quickly did in fact record faster acquisition and better performance in their rats than experimenters who had been told to expect that their rats would be poor learners. 22
In another learning study, rats were trained to perform a series of tasks for a food reward in an operant or “Skinner” box. If the experimenter had been told that the rats had been bred for good performance in these tasks, better performance was recorded for the rats than if the experimenter had been told that the rats had been bred for poor performance in these tasks. 23
Anyone who has conducted tail flick or formalin testing in rodents recognizes that the expectations of the investigator can affect the results of laboratory pain tests as well. In the case of paw withdrawal in the relatively unrestrained animal, which often raises the paw to walk, groom itself, or make postural adjustments (e.g.
, in the Hargreave or von Frey test), the influence of investigator bias can be even greater.
Experimenter bias is not limited to the time when the behavioral observations are recorded; it can also affect nonbehavioral measurements. For example, standard laboratory procedures used to count blood cells were shown to require a degree of consistency that was not possible, given the equipment and procedures used, yet laboratory results typically conformed to these unrealistic requirements for consistency; this could only have occurred as a result of the bias of laboratory technicians. 24
In addition to affecting initial measurements, experimenter bias can also play a role in the way the data are managed and analyzed. Experimenters can include or exclude data based in part on experimenter expectations, and they can perform additional analyses, analyzing the data as raw scores or differences from baselines, and dividing subjects into any number of subgroups based on a range of different criteria. 25–27
All of these procedures serve to drastically inflate the probability of detecting an expected effect by increasing the probability of producing a false-positive result.
We suggest that experimenter bias should receive more attention in laboratory research, and that blinding procedures to guard against experimenter bias should be a more common practice in laboratory investigations. Blinding procedures should be used when recording any measurements, not just in behavioral testing, and all decisions about inclusion/exclusion of data should be made before the blind is broken. We would also caution that some discussion or plan should be considered as to how the data will be analyzed before studies are completed, to reduce the risk that the rate of false-positive results will be inflated through the use of numerous unplanned analyses. Furthermore, we propose that the precise procedures used to prevent experimenter bias should be an essential component of the methods described in laboratory publications. Investigators should explicitly report the details of the procedures they used. A general statement that “the experimenter was blinded” is not sufficient. Finally, we also support the suggestion that laboratory studies should be evaluated in the same way that clinical trials are, with more emphasis on the difference between blinded and nonblinded studies and more weight given to studies that have carefully controlled for potential bias. 28
In summary, recent blinded experiments detected clear analgesic effects of spinal opioids in the spinal nerve ligation model. Previous, nonblinded experiments (one of them from our own laboratory) reported that spinal opioids were not effective for neuropathic pain, which was the expected result at that time. The differences in the results between these studies may have been attributed to experimenter bias in the earlier nonblinded experiments, a suggestion that is supported by the results of carefully blinded experiments conducted recently in our own laboratory. Experimenter bias is a well-recognized phenomenon in clinical trials, and although it does not receive as much attention in laboratory research, there should be no doubt that laboratory studies are just as vulnerable to experimenter bias as clinical studies. Thus, the very narrow focus of this editorial, on studies of spinal opioids in the spinal nerve ligation model of neuropathic pain, leads to consideration of a much more general issue—experimenter bias in laboratory research—and the suggestion that the use of rigorous blinding procedures may be just as important in laboratory experiments as they are in clinical trials.
1. Zhao C, Tall JM, Meyer RA, Raja SN: Antiallodynic effects of systemic and intrathecal morphine in the spared nerve injury model of neuropathic pain in rats. A nesthesiology 2004; 100: 905–11
2. Kim SH, Chung JM: An experimental model for peripheral neuropathy produced by segmental spinal nerve ligation in the rat. Pain 1992; 50: 355–63
3. Coombs DW, Maurer LH, Saunders RL, Gaylor M: Outcomes and complications of continuous intraspinal narcotic analgesia for cancer pain control. J Clin Oncol 1984; 2: 1414–20
4. Penn RD, Paice JA: Chronic intrathecal morphine for intractable pain. J Neurosurg 1987; 67: 182–6
5. Arner S, Arner B: Differential effects of epidural morphine in the treatment of cancer-related pain. Acta Anaesthesiol Scand 1985; 29: 32–6
6. Krames ES: Intraspinal opioid therapy for chronic nonmalignant pain: Current practice and clinical guidelines. J Pain Symptom Manage 1996; 11: 333–52
7. Samuelsson H, Malmberg F, Eriksson M, Hedner T: Outcomes of epidural morphine treatment in cancer pain: Nine years of clinical experience. J Pain Symptom Manage 1995; 10: 105–12
8. Winkelmuller M, Winkelmuller W: Long-term effects of continuous intrathecal opioid treatment in chronic pain of nonmalignant etiology. J Neurosurg 1996; 85: 458–67
9. Hassenbusch SJ, Stanton-Hicks M, Covington EC, Walsh JG, Guthrey DS: Long-term intraspinal infusions of opioids in the treatment of neuropathic pain. J Pain Symptom Manage 1995; 10: 527–43
10. Kim SH, Chung JM: Sympathectomy alleviates mechanical allodynia in an experimental animal model for neuropathy in the rat. Neurosci Lett 1991; 134: 131–4
11. Bian D, Nichols ML, Ossipov MH, Lai J, Porreca F: Characterization of the antiallodynic efficacy of morphine in a model of neuropathic pain in rats. NeuroReport 1995; 6: 1981–4
12. Lavand’homme PM, Eisenach JC: Exogenous and endogenous adenosine enhance the spinal antiallodynic effects of morphine in a rat model of neuropathic pain. Pain 1999; 80: 31–6
13. Lee YW, Chaplan SR, Yaksh TL: Systemic and supraspinal, but not spinal, opioids suppress allodynia in a rat neuropathic pain model. Neurosci Lett 1995; 199: 111–4
14. Xie J, Park SK, Chung K, Chung JM: The effect of lumbar sympathectomy in the spinal nerve ligation model of neuropathic pain. J Pain 2001; 2: 270–8
15. Lavand’homme P, Pan HL, Eisenach JC: Intrathecal neostigmine, but not sympathectomy, relieves mechanical allodynia in a rat model of neuropathic pain. A nesthesiology 1998; 89: 493–9
16. Ringkamp M, Eschenfelder S, Grethel EJ, Habler HJ, Meyer RA, Janig W, Raja SN: Lumbar sympathectomy failed to reverse mechanical allodynia- and hyperalgesia-like behavior in rats with L5 spinal nerve injury. Pain 1999; 79: 143–53
17. Shir Y, Ratner A, Raja SN, Campbell JN, Seltzer Z: Neuropathic pain following partial nerve injury in rats is suppressed by dietary soy. Neurosci Lett 1998; 240( 2): 73–6
18. Noseworthy JH, Ebers GC, Vandervoort MK, Farquhar RE, Yetisir E, Roberts R: The impact of blinding on the results of a randomized, placebo-controlled multiple sclerosis clinical trial. Neurology 1994; 44: 16–20
19. Epstein WV: Expectation bias in rheumatoid arthritis clinical trials: The anti-CD4 monoclonal antibody experience. Arthritis Rheum 1996; 39: 1773–80
20. Dworkin RH, Backonja M, Rowbotham MC, Allen RR, Argoff CR, Bennett GJ, Bushnell MC, Farrar JT, Galer BS, Haythornthwaite JA, Hewitt DJ, Loeser JD, Max MB, Saltarelli M, Schmader KE, Stein C, Thompson D, Turk DC, Wallace MS, Watkins LR, Weinstein SM: Advances in neuropathic pain: Diagnosis, mechanisms, and treatment recommendations. Arch Neurol 2003; 60: 1524–34
21. Cordaro L, Ison JR: Psychology of the scientist: X. Observer bias in classical conditioning of the planarian. Psychol Rep 1963; 13: 787–9
22. Rosenthal R, Fode KL: The effect of experimenter bias on the performance of the albino rat. Behav Sci 1963; 8: 183–9
23. Rosenthal R, Lawson R: A longitudinal study of the effects of experimenter bias on the operant learning of laboratory rats. J Psychiatr Res 1963; 2: 61–72
24. Berkson J, Magath TB, Hurn M: The error of estimate of the blood cell count as made with the hemocytometer. Am J Physiol 1940; 128: 309–23
25. Assmann SF, Pocock SJ, Enos LE, Kasten LE: Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet 2000; 355: 1064–9
26. Pocock SJ, Hughes MD, Lee RJ: Statistical problems in the reporting of clinical trials: A survey of three medical journals. N Engl J Med 1987; 317: 426–32
27. Pocock SJ, Assmann SE, Enos LE, Kasten LE: Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: Current practice and problems. Stat Med 2002; 21: 2917–30
28. Lindner MD, Frydel BR, Francis JM, Cain CK: Analgesic effects of adrenal chromaffin allografts: Contingent on special procedures or due to experimenter bias? J Pain 2003; 4: 64–73
This article has been cited 5 time(s).
Bosnian Journal of Basic Medical Sciences
Hyperalgesia-type response reveals no difference in pain-related behavior between Wistar and Sprague-Dawley rats
Bosnian Journal of Basic Medical Sciences, 7(2):
Emerging Strategies for the Treatment of Neuropathic Pain
Opioids and neuropathic pain
Emerging Strategies for the Treatment of Neuropathic Pain, ():
PainAnimal models and the prediction of efficacy in clinical trials of analgesic drugs: A critical appraisal and call for uniform reporting standardsPain
PainAnti-allodynic efficacy of the chi-conopeptide, Xen2174, in rats with neuropathic painPain
© 2004 American Society of Anesthesiologists, Inc.
Publication of an advertisement in Anesthesiology Online does not constitute endorsement by the American Society of Anesthesiologists, Inc. or Lippincott Williams & Wilkins, Inc. of the product or service being advertised.