Secondary Logo

Journal Logo

Special Theme: Complementary, Alternative, and Integrative Medicine: SPECIAL THEME COMMENTARIES

Complementary and Alternative Medicine and the Need for Evidence-based Criticism

Astin, John A., PhD

Author Information
  • Free

After reading the recent set of papers entitled “Alternative Medicine: The Importance of Evidence in Medicine and Medical Education,” published in the March 2001 issue of Academic Medicine, I feel compelled to respond to a number of the points raised by the authors.

With the exception of the paper by Frenkel and Ben-Arye, which was quite fair and even-handed,1 throughout the set the claim is frequently made that complementary and alternative medicine (CAM) is not evidence-based. However, the great irony with this and many other statements contained in this set of articles is that such claims of “lack of evidence” are typically not backed up with any supporting evidence themselves (what one might term “non—evidence-based criticism”). To give but one example, Grollman states that “the list of demonstrably effective herbals today would consist of opium poppy, cinchona, ergot, ipecacuanha, foxglove, cocoa, belladonna, and ma huang.”2 However, the author either appears unaware of, or (worse) completely overlooks or ignores the results of positive (or at least highly suggestive) meta-analyses for a number of other herbals and nutraceuticals, including St. John's wort for mild to moderate depression,3,4 saw palmetto for benign prostatic hyperplasia,5,6Gingko biloba for dementia,7 Alzheimer's disease,8 and intermittent claudication,9 kava kava for anxiety,10 yohimbine for erectile dysfunction,11 and glucosamine—chondroitin for osteoarthritis.12 The failure to cite such evidence contributes to a very misleading picture of the state of the scientific evidence base underlying CAM.

Sampson13 and several others in the March set of papers frequently make reference to how many if not most CAM therapies have been shown to be useless or even dangerous. However, no specifics are provided. Which CAM therapies have been “disproven?” Which specific therapies are “dangerous”? The authors mention therapeutic touch and homeopathy as examples of therapies that have been scientifically disproven, but again fail to cite evidence to support this claim. How can Sampson claim to be a proponent of evidence-based medicine when he both fails to cite any research to back up such claims of these therapies' non-effectiveness and overlooks the results of meta-analyses in both homeopathy14 and therapeutic touch15 that suggest these therapies may, in fact, be effective despite the lack of clear mechanisms to explain their efficacy? It would be one thing to debate these meta-analytic findings and publish their critiques in peer-reviewed journals, but these authors have not done this. Instead, Sampson's approach is to simply dismiss all of the evidence for CAM by citing one editorial16 that concluded there was little credible or convincing evidence in these areas. This approach of selectively citing one negative article while failing to cite any of the positive systematic reviews or meta-analyses is the antithesis of evidence-based medicine. It is, in short, opinion-based medicine.

As an example of the lack of evenhandedness in how these authors review the literature, Sampson cites the review by Sloan et al.17,18 challenging the validity of the data linking spiritual/religious beliefs and practices to health outcomes as if it were the final word. However, he fails to mention that Sloan et al.'s review is by no means systematic, and likewise fails to mention the recently published meta-analyses19 that found a “robust” effect size across 42 study samples suggesting that religious involvement may in fact positively influence health outcomes even after controlling for important moderating variables. It is hard not to conclude from this selective reading of the literature (which is precisely what systematic reviews and evidence-based medicine are designed to guard against) that the authors in this series are somehow more interested in promoting their particular ideologic agendas than they are in fairly determining the state of the evidence in CAM.

IS CAM USE A SIGN OF FAULTY REASONING?

The paper by Beyerstein20 is a clear example of this approach. For example, he proposes numerous theories to explain patients' and practitioners' decisions to use CAM therapies but fails to consider the significant body of research that has already examined, in considerable detail, the question of why patients turn to CAM.21,22,23,24,25 The claim that the public and some practitioners' desires to use CAM therapies represent irrational or faulty reasoning is in no way substantiated (i.e., it is basically offered as an opinion). In cases where people are using demonstrably inefficacious or unsafe therapies, one could persuasively argue that such choices do, in fact, represent unsound, irrational, or uncritical thinking. I don't debate that human beings frequently fall prey to fantastic claims of healing, weight loss, etc., based upon little if any credible evidence. I would personally love to see the public become more thoughtful, reflective, and critical of what they read, what their friends, family (and even doctors) advise them to do, and what they hear in the media about how to care for their health. However, while it is true that the safety and efficacy of many if not most CAM therapies have yet to be fully determined, very few of these approaches have been disproved or proven dangerous as the authors in the March set claim (again with no supporting evidence).

Given that most CAM therapies have not been disproven, is it irrational or a sign of faulty thinking when a patient—who has some chronic condition that has not responded well to more conventional biomedical/pharmacologic therapies, or who is trying to avoid more invasive procedures such as surgery—continues using some nutritional supplement, manipulative therapy, or CAM approach such as acupuncture, massage, or meditation if he or she finds these approaches are effectively reducing pain? Of course, it is true that without well-designed clinical trials with appropriate controls, it is often difficult to determine whether a given condition would have resolved itself over time (i.e., without the aid of a particular therapy) or whether the observed changes simply represent regression to the mean. However, in cases where patients have suffered for years and had little success reducing such suffering, it is difficult to explain away their subjective sense (or practitioners' clinical observations) of improvement in terms of the “natural variability” of the disease or simple regression to the mean (or even the placebo effect, since presumably other approaches would have produced equally successful placebo effects). In these instances, such explanations would not constitute reasonable or plausible rival hypotheses, and therefore a patient's or clinician's continued use of some “unproven” CAM therapy that he or she finds has improved health cannot be fairly characterized as irrational or uncritical. In fact, it could be seen as a highly rational choice under certain circumstances.

I am a scientist. If I had some chronic medical condition (which I fortunately do not) that had not responded well to conventional therapies, or if I was not wanting to experience the side effects of the drugs being used, wouldn't it represent a rational, intelligent choice to experiment in the laboratory of my own body with various alternatives that I felt relatively confident were not dangerous and see whether I experienced any relief? To label such a choice as irrational is in my mind terribly dismissive, judgmental, and condescending. It is certainly true—as the authors in this set point out—that various herbs and nutraceuticals can be associated with adverse effects and potentially harmful interactions with pharmaceuticals. To be sure, “natural” does not necessarily mean benign. However, decisions in medicine, irrespective of how much objective evidence we gather, always involve the weighing of probabilities. What is the probability that “intervention x” will produce such and such an effect in “patient x” and what are the potential risks associated with this intervention in relation to its potential benefits? For example, while an herb such as St. John's wort may be associated with some side effects, we must assess its potential benefits—based on randomized clinical trials (RCTs) and meta-analyses—in the light of the comparative cost—benefit ratios of conventional therapies such as selective serotonin reuptake inhibitor (SSRI) drugs, which, at least based on present scientific understanding, appear to have significantly greater side effects than does St John's wort and comparable efficacy.26

I would like to make one final point as it relates to the need for evidence-based answers to the question of whether a given therapy is or is not effective. I agree with the authors that this is of vital importance whether we are investigating conventional or CAM approaches. Our medicine needs to be based on as much evidence and as high a quality of evidence as is practically and financially feasible. And while it is true that the randomized controlled experiment makes it easier to rule out alternative or rival explanations for any effects we observe in our patients, conventional and CAM practitioners are always having to make subjective assessments as to whether a given treatment or prescription's success or failure can be attributed to the treatment per se (and not some other factor). It would obviously not be possible to conduct an RCT with every patient seen in clinical care to determine whether our therapeutic interventions are “working” with those particular patients. Instead, we are always faced with having to make educated guesses (again based on the available evidence) regarding which approaches to use with which patients, and then we must rely on our subjective observations and assessments to determine efficacy. To suggest that randomized controlled trials, meta-analyses, and clinical practice guidelines will somehow eliminate this need for subjective clinical judgment is to misrepresent the realities of clinical medicine (both CAM and conventional). If medicine could be purely evidence-based (which is highly debatable, both practically and financially), then in theory medical care (including diagnoses and treatment protocols) could essentially be administered by computers and computer algorithms.

DOES MEDICAL EDUCATION ADDRESS MIND-BODY INTERACTIONS?

Marcus27 is critical of CAM proponents' assertion that conventional medical education ignores mind—body interactions and feels that it is unfair and inaccurate to say that the biomedical paradigm views the body merely as a machine. To counter these negative characterizations of conventional medicine by so-called CAM proponents, he states that “training physicians to consider the whole person and to identify and address emotional and social problems (the biopsychosocial model) is a central tenet of medical education.” I agree with Dr. Marcus that medical education has made important strides in terms of addressing the non-physical dimensions of health and in focusing greater attention upon the humanistic development of physicians. However, in his article we again see the familiar pattern of citing little if any evidence to support the stated claims. For example, what do we actually know from the literature regarding how well or poorly the biopsychosocial model is being integrated into medical training? As part of a National Institutes of Health grant proposal, I recently reviewed much of the literature in this area. I concluded that evidence from several lines of research suggests that despite what appears to be relatively widespread acknowledgement of and support for the biopsychosocial model in medicine, psychosocial factors continue to be overlooked or missed in clinical encounters and are frequently underemphasized in medical education. For example, Roter et al.28 in their analysis of doctor—patient communication patterns among primary care providers found that the majority (66%) of visits (with at least 60% of the study participants) could be characterized as physician-dominated and narrowly focused on biomedical issues. Overall, only 20% of the visits they analyzed were coded as “biopsychosocial” (representing a balance of biomedical and psychosocial issues). Based on videotaped office visits, Marvel et al.29 similarly found that among residents, psychosocial issues (e.g., family, career) were discussed in approximately 25% of visits, while emotional reactions of patients and family members were addressed even less frequently.

In a study of physicians' responses to patients' emotional clues, Levinson et al.30 found that 62% of surgeons and 79% of primary care providers failed to respond “positively” (either adequately or appropriately), frequently missing opportunities to acknowledge and address patients' feelings. Gulbrandsen et al.31 found that of those psychosocial problems predefined by patients as affecting their physical health, physicians correctly identified between 19% and 53%, depending on the natures of the problems. Suchman et al.'s32 findings suggest that in most instances, doctors fail to acknowledge either clues to or direct expressions of affect or emotional concerns. They hypothesized that this observed lack of “empathic response” may stem from (1) natural variability in such sensitivity, (2) fears of tapping into patients' suffering and emotional pain, and (3) attitudes acquired during medical training that “emphasize objective data and the quest for control over subjective experience and the cultivation of relationships.”32

While it is the case that most medical schools offer some instruction at the undergraduate and graduate levels in the role of behavioral factors in health and the importance of the biopsychosocial model, it has been suggested that these areas have not been fully integrated into the medical curriculum.33,34 In fact, there is some evidence to suggest that students may actually become less aware of and sensitive to paychosocial/humanistic issues as they progress through their medical training.35,36,37,38

As discussed by Schmidt,33 an unsolved challenge in terms of realizing such integration is that basic science continues to dominate medical training. She notes that “at best, biopsychosocial issues are treated as separate but equal—and often as separate and not equal.” Tresolini and Shugars,34 in their qualitative analysis of medical school faculty and administrators' perspectives on developing more integrated curricula, identified as a major barrier the “deeply-rooted biases and cultural beliefs” of faculty and administrators. This included the belief that “attention to psychosocial concerns is inappropriate for physicians and that an integrated approach is for low-status generalists.” An additional barrier to integration they identified was physicians' continued abuse of and disrespect for both medical students and patients, which, as the researchers note, contradicts the basic philosophy and spirit underlying a more whole-person, patient-centered, integrated approach to education and clinical care. As an example of the above attitudes, a fourth-year medical student recently shared, in a course I was teaching, that after she had spent some time talking to a patient about some emotional concerns the patient was having, her attending physician informed her that she was “not a nurse” and basically should attend to her assigned duties.

While I acknowledge the above is not a systematic review of the literature examining physicians' skills in addressing the psychosocial dimensions of patients' lives, these findings certainly raise doubts as to how successful we have been in training future physicians in these areas. Therefore, the jury is most certainly still out regarding whether mind—body interactions are being sufficiently addressed in medical school. While some CAM proponents may be implying that conventional medical education “denies” the importance of mind—body interactions, most CAM educators and researchers with whom I work are instead arguing that it is a matter of degree, and that by and large, it is felt (and the evidence cited above lends strong credence to the notion) that this continues to be an area that is underemphasized and given inadequate attention in medical training. The CAM community may ultimately be doing all of medicine an enormous service by advocating that there be greater attention given to these historically neglected domains, helping to make medicine both more effective and more humane.

NEED FOR PLAUSIBLE EXPLANATORY MECHANISMS

The last point I want to make concerns the issue raised primarily in the article by Sampson13—but echoed frequently by other CAM critics and skeptics—of the need to have plausible explanatory mechanisms before we can accept a given therapy as part of evidence-based medicine. There are several problems associated with this belief. First, we have numerous examples within conventional medicine of our using therapies with demonstrated efficacy (e.g., aspirin) without our fully understanding the drug's mechanisms of action. In gravity, we have an analogous example of a “force” that despite its never having been directly seen or measured (nor its mechanism understood) is still regarded as “real” based on inferring its presence from natural observations. Similarly, most scientists would agree that there is such a thing as a “placebo effect” (in fact much of evidence-based medicine rests largely on controlling for this phenomenon), yet a “placebo” has never been seen (only inferred from observations) nor are its mechanisms of action understood.

Second, it is the antithesis of good science to presume, a priori, that something (e.g., prayer, therapeutic touch, homeopathy) cannot work simply because its purported mechanisms do not fit with the current or prevailing understandings of science. If we always operated from this premise, science and scientific paradigms would never evolve.

As I read the set of articles discussed above, I found myself asking the question: “What is the source of this powerful resistance to CAM? Why such skepticism, and why are the CAM detractors and critics so vehement in their arguments (and, I believe, unfair in their characterizations of CAM as lacking in any evidence)? In many respects, I think Beyerstein,20 without intending to, answered these questions perfectly when he offered up the following explanation for why CAM proponents cling so fiercely to their doctrines even in the face of non-confirming evidence. He states: “Because one's concept of health is entwined with one's fundamental assumptions about reality, an attack on someone's belief in unorthodox healing becomes a threat to his or her entire metaphysical outlook. Understandably, this will be resisted fervently.” However, if one simply substitutes “orthodox” for “unorthodox” in the above quote, one has a reasonable explanation for the biased and largely non-evidence-based tone and approach taken by most of the authors in the March set. With these points in mind, I close with a quote from a recent response to critics that my co-authors and I published in the Annals of Internal Medicine39 following the publication of our somewhat controversial meta-analysis of research in prayer and distant healing.

In many cases, no amount of empirical evidence is sufficient to change one's prior beliefs, particularly if such beliefs are held to strongly. This appears to be the case whether such evidence refutes a layperson's belief based, say, on faith (e.g., “my religion is true”) or a scientist's skepticism that something (such as distant healing or homeopathy) is not possible. Ironically, although scientists frequently argue that their lack of belief in certain phenomena is based on reason and rationality, such skepticism shares much in common with religious dogma in that it is based largely on a set of untested assumptions and is not easily refuted by contradictory evidence. Understanding the complex reasons underlying people's unwillingness to alter their perspectives even in the face of evidence is of paramount importance because oftentimes (whether in medical science or in our personal lives) only by letting go of previously held beliefs can new learning and discovery ever take place.

REFERENCES

1. Frenkel M, Ben-Arye E. The growing need to teach about complementary and alternative medicine: questions and challenges. Acad Med. 2001;76:251–4.
2. Grollman AP. Is there wheat among the chaff? Acad Med. 2001;76:221–3.
3. Stevinson C, Ernst E. Hypericum for depression. An update of the clinical evidence. Eur Neuropsychopharmacol. 1999;9:501–5.
4. Linde K, Ramirez G, Mulrow CD, Pauls A, Weidenhammer W, Melchart D. St John's wort for depression—an overview and meta-analysis of randomised clinical trials. BMJ. 1996;313:253–8.
5. Wilt TJ, Ishani A, Rutks I, MacDonald R. Phytotherapy for benign prostatic hyperplasia. Public Health Nutr. 2000;3:459–72.
6. Wilt TJ, Ishani A, Stark G, MacDonald R, Lau J, Mulrow C. Saw palmetto extracts for treatment of benign prostatic hyperplasia: a systematic review. JAMA. 1998;280:1604–9.
7. Ernst E, Pittler MH. Ginkgo biloba for dementia. a systematic review of double-blind, placebo-controlled trials. Clin Drug Invest. 1999;17:301–8.
8. Oken BS, Storzbach DM, Kaye JA. The efficacy of Ginkgo biloba on cognitive function in Alzheimer disease. Arch Neurol. 1998;55:1409–15.
9. Pittler MH, Ernst E. Ginkgo biloba extract for the treatment of intermittent claudication: a meta-analysis of randomized trials. Am J Med. 2000;108:276–81.
10. Pittler MH, Ernst E. Efficacy of kava extract for treating anxiety: systematic review and meta-analysis. J Clin Psychopharmacol. 2000;20:84–9.
11. Ernst E, Pittler MH. Yohimbine for erectile dysfunction: a systematic review and meta-analysis of randomized clinical trials. J Urol. 1998;159:433–6.
12. McAlindon TE, LaValley MP, Gulin JP, Felson DT. Glucosamine and chondroitin for treatment of osteoarthritis: a systematic quality assessment and meta-analysis. JAMA. 2000;283:1469–75.
13. Sampson W. The need for educational reform in teaching about alternative therapies. Acad Med. 2001;76:248–50.
14. Linde K, Clausius N, Ramirez G, et al. Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trials. Lancet. 1997;350:834–43.
15. Astin JA, Harkness E, Ernst E. The efficacy of “distant healing”: a systematic review of randomized trials. Ann Intern Med. 2000;132:903–10.
16. Knipschild P. Alternative treatments: do they work? Lancet. 2000;356(12 suppl):S4.
17. Sloan RP, Bagiella E, Powell T. Religion, spirituality, and medicine. Lancet. 1999;353:664–7.
18. Sloan RP, Bagiella E, VandeCreek L, et al. Should physicians prescribe religious activities? N Engl J Med. 2000;342:1913–6.
19. McCullough ME, Hoyt WT, Larson DB, Koenig HG, Thoresen C. Religious involvement and mortality: a meta-analytic review. Health Psychol. 2000;19:211–22.
20. Beyerstein BL. Alternative medicine and common errors of reasoning. Acad Med. 2001;76:230–7.
21. Astin JA. Why patients use alternative medicine: results of a national study. JAMA. 1998;279:1548–53.
22. Furnham A, Kirkcaldy B. The health beliefs and behaviours of orthodox and complementary medicine clients. Br J Clin Psychol. 1996;35:49–61.
23. Furnham A, Beard R. Health, just world beliefs and coping style preferences in patients of complementary and orthodox medicine. Soc Sci Med. 1995;40:1425–32.
24. Furnham A, Forey J. The attitudes, behaviors and beliefs of patients of conventional vs. complementary (alternative) medicine. J Clin Psychol. 1994;50:458–69.
25. Furnham A, Bhagrath R. A comparison of health beliefs and behaviours of clients of orthodox and complementary medicine. Br J Clin Psychol. 1993;32:237–46.
26. Ernst E, Rand JL, Barnes J, Stevinson C. Adverse effects of profile of the herbal antidepressant St. John's wort (Hypericum perforatum L.). Eur J Clin Pharmacol. 1998;54:589–94.
27. Marcus DM. How should alternative medicine be taught to medical students and physicians? Acad Med. 2001;76:224–9.
28. Roter DL, Stewart M, Putnam SM, Lipkin M Jr, Stiles W, Inui TS. Communication patterns of primary care physicians. JAMA. 1997;277:350–6.
29. Marvel MK. Involvement with the psychosocial concerns of patients. observations of practicing family physicians on a university faculty. Arch Fam Med. 1993;2:629–33.
30. Levinson W, Gorawara-Bhat R, Lamb J. A study of patient clues and physician responses in primary care and surgical settings. JAMA. 2000;284:1021–7.
31. Gulbrandsen P, Hjortdahl P, Fugelli P. General practitioners' knowledge of their patients' psychosocial problems: multipractice questionnaire survey. BMJ. 1997;314:1014–8.
32. Suchman AL, markakis K, Beckman HB, Frankel R. A model of empathic communication in the medical interview. JAMA. 1997;277:678–82.
33. Schmidt H. Integrating the teaching of basic sciences, clinical sciences, and biopsychosocial issues. Acad Med. 1998;73(9 suppl):S24–S31.
34. Tresolini CP, Shugars DA. An integrated health care model in medical education: interviews with faculty and administrators. Acad Med. 1994;69:231–6.
35. Newton BW, Savidge MA, Barber L, et al. Differences in medical students' empathy. Acad Med. 2000;75:1215.
36. Eisenthal S, Stoeckle JD, Ehrlich CM. Orientation of medical residents to the psychosocial aspects of primary care: influence of training program. Acad Med. 1994;69:48–54.
37. Coles R. The moral education of medical students. Acad Med. 1998;73:55–7.
38. Dornbush RL, Richman S, Singer P, Brownstein EJ. Medical school, psychosocial attitudes, and gender. J Am Med Wom Assoc. 1991;46:150–2.
39. Astin J, Harkness E, Ernst E. Distant healing. Ann Intern Med. 2001;134:533.
© 2002 Association of American Medical Colleges