At the turn of the 20th century, the relationship between orthodox Western medicine—which is variously called “conventional,” “scientific,” or “allopathic” medicine—and alternative schools of medical thought, currently referred to collectively as complementary and alternative medicine (CAM), is the subject of significant debate. Since the recorded inception of the healing arts, numerous theories of disease and collections of treatments have competed for the attention and acceptance of those who would be patients. This heterogeneity of medicine is not culture-specific. Although the various schools of medical thought in ancient Greece, the Roman Empire, or 18th-century America may be more familiar to Western-trained physicians, the healing traditions of other cultures, including Asian, African, and Native American societies, have also developed in the context of competing and often incongruous theories of disease and cure. Often one particular group of practitioners, sharing a common understanding and possessing similar tools, becomes more widely accepted than the others and thus comes to represent the healing orthodoxy within a culture, with attendant social and political advantages.1 Other schools of thought may continue to coexist, often in an openly antagonistic fashion, providing patients with an alternative to orthodox practitioners and therapies. In addition, these alternative schools of thought may also present a challenge to the philosophical underpinnings of the orthodox medicine, providing an impetus for self-examination and change.
The popularity of CAM in the United States2 and other Western nations3,4 is undeniable. Practitioners of orthodox medicine, however, continue to assert that they provide patients with demonstrably superior treatments.5 The popularity of CAM is generally attributed to the scientific illiteracy and logical inconsistency of “otherwise intelligent patients”6 who do not know what is good for them. Claims for the superiority of many orthodox therapies assert that such interventions have demonstrated efficacy through the process of sound clinical research, utilizing control groups, placebos, and statistical analysis. The emphasis on incorporating the results of such research into the practice of medicine has become embodied in the notion of evidence-based medicine (EBM).7 As it has embraced EBM, orthodox medicine has also begun to demand that alternative therapies be subjected to the same types of controlled clinical trials that form the basis for evidence-based practice.8–10
Under the construct of EBM, knowledge in orthodox medicine is increasingly becoming defined by the results of clinical research, rather than by the simple observations or pathophysiologic rationales of physicians. That is, EBM explicitly prefers one form of medical knowledge over others. This is not to say that EBM denies the value of clinical experience and inductive reasoning for clinical practice, only that it defines the best available evidence for making clinical decisions in terms of the results from systematic research.11 As such, EBM represents an explicit attempt to define a medical epistemology, distinguishing between medical knowledge and belief based on the availability and quality of empirical, statistical evidence.
At its core, the call for an evidence-based CAM is a call for alternative practitioners (and the public) to accept this epistemic framework of orthodox medicine. The call for an evidence-based CAM, however, does not represent a scientific necessity, but rather is a philosophical demand. The reasonableness of such a demand appears to hinge on two main premises. First, the epistemology of EBM must have clear and comprehensive value, at the least with regard to orthodox medicine. Second, the argument must be made that the epistemology of EBM is applicable to any of the healing arts, regardless of the theory of disease or healing that underlies a particular practice or modality. The evidence-based epistemology, then, must be defensible as the preferred method for obtaining knowledge in any form of medicine, not simply one of many equally acceptable methods. By demanding that alternative medicine become evidence-based, EBM seeks to define itself not only as orthodoxy in Western medicine, but also as the primary arbiter of all medical knowledge.
In this article we examine the fundamental epistemologic differences between orthodox medicine and many of its alternatives, nothing the historical shift in orthodox epistemology and the consequences of adopting that epistemology for alternative schools of thought. Although proponents of EBM seem to assert that a medical epistemology based on the results of clinical research is preferable regardless of the underlying theory of disease and healing, we argue that this position is untenable. First, the epistemology of EBM is problematic even for orthodox medicine. That is, other forms of knowledge in medicine, such as clinical experience, expert opinion, or a pathophysiologic rationale, may at times be more compelling than evidence from even large well-designed clinical trials, meta-analyses, or systematic reviews. EBM has, thus far, provided no instruction as to when such alternative kinds of knowledge can take precedence over the best available scientific evidence, although it has begun to recognize that such knowledge should play some role in medical decision making. Second, medical epistemology cannot be separated from medical metaphysics. The epidemiologic epistemology of EBM fits relatively well with the biophysiologic theory of disease associated with orthodox medicine, but may not be coherent with other theories of disease and healing. Finally, we discuss the requirements of potential alternative methods of generating medical knowledge in CAM.
ORTHODOX MEDICAL EPISTEMOLOGY
Success in modern clinical medicine, according to proponents of EBM, depends not upon the production and explication of the most coherent or convincing theory of disease, but upon the completion and utilization of systematic studies designed to demonstrate the effectiveness of particular treatments for particular conditions.12 Some have gone so far as to argue that the rationale and basis for a particular treatment, whether derived from scientific investigation or magical thinking, is “largely irrelevant except for historical purposes and cultural interest.”5 Under such a view of EBM, there need be no reliance on a theory of disease or healing to determine the value of medical therapies, simply reliance on the results of well-designed and well-controlled clinical trials. With this understanding, “there is no alternative medicine. There is only scientifically proven evidence-based medicine supported by solid data or unproven medicine, for which scientific evidence is lacking.”5 Proof, and knowledge of medical truth, for evidence-based orthodox medicine lies in the statistical results of controlled clinical trials,13 presumed to generate knowledge independently of any theory of disease. The evidence-based practitioner would then simply base clinical decisions on this “best available scientific evidence.”14 The highest-quality scientific evidence, according to the hierarchies of EBM, derives from large, randomized, controlled clinical trials, with lower-quality evidence flowing from less rigorous study designs.15 It remains important to note that EBM does not deny the value of additional forms of medical knowledge, such as clinical experience and pathophysiologic rationale, but explicitly prefers the knowledge gained from clinical research. Virtually the entire corpus of EBM literature deals only with the performance, analysis, critical appraisal, and incorporation into practice of empirical clinical research.
Although now considered a fundamental tenet of orthodox medical practice, this reliance on empirical evidence represents a fairly recent and major shift in Western medicine. Prior to the 19th century, orthodox physicians distinguished themselves from other purported healers by adherence to a specific theory of disease and healing rather than reliance on observation and trial-and-error. Quackery was characterized by such reliance on the observed; the true physician possessed a theory regarding the nature of illness, one that provided a rationalistic framework to legitimize clinical practices.16 Rationalism, the reasoning from predetermined principles and theories to individual cases, reigned over empiricism.
But reliance on a theory of disease alone to guide prognosis and therapeutics was problematic. Although recognition that individual observations are not always reliable in determining the effects of medical therapies and provide tenuous grounds on which to base medical decisions was well entrenched in orthodox medicine, the complete rejection of empiricism could not last forever. In a tradition traced back to the Paris School in the early 19th century, systematic observation of large numbers of patients became a tool to develop medical knowledge.17 This “new empiricism” continued to reject the simple observation of individual cases and the trial-and-error approach to therapeutics as untrust-worthy, instead searching for truth and certainty among large populations and by utilizing control groups. Combined with statistical expertise and sophisticated study design, a standard method has developed from that approach to answer specific questions regarding the accuracy of diagnostic tests and efficacy of particular therapies. But only over the last several decades has this specific kind of medical knowledge ascended to the preeminence assigned to it by EBM. Rationalism has been summarily rejected as an appropriate form of medical reasoning, replaced with reasoning from the “evidence,” where “the best evidence” comes from large, controlled clinical trials.
ALTERNATIVES TO THE ORTHODOX EPISTEMOLOGY
Several alternatives exist to the orthodox reliance on empirical evidence derived from clinical research. Recognizing the limitations of knowledge obtained from simple, non-systematic observations of individual patients certainly does not mean that no meaningful knowledge can be gained in this way. Nor is this kind of experiential knowledge always superseded by the knowledge obtained from well-designed clinical trials in clinical practice. For example, a recent randomized controlled trial demonstrated that patients with symptomatic reflux had more heartburn pain and more acid reflux associated with a meal of Wendy's chili and red wine than with a McDonald's Quarter Pounder, fries, and a chocolate shake.18 If a patient presents with frequent and consistent symptoms associated with hamburgers and shakes but no problems with chili and wine, however, it seems doubtful that even the most strident supporter of EBM would, despite the “evidence,” attempt to persuade the patient that he is in error and recommend abstaining from the chili and sticking to hamburgers. The patient's consistent symptoms associated with hamburgers appears to establish causality; the results of the trial are irrelevant in this particular case.
In therapeutics as well, there are numerous examples where the causal relationship of treatment and effect is convincing without appeal to anything other than simple observation of a single case. For instance, patients with cystic fibrosis generally produce large amounts of sputum, and many methods have been developed to aid in airway clearance. Studies comparing these methods show equivalency, yet there is a tremendous variability among individual patients in their responses, both objective and subjective, to a specific modality. In practice, a patient's response to one mode of airway clearance may be impressive and immediate, while a “scientifically” equivalent therapy will have no effect. Demonstrating causality in a single individual is most convincing when long-standing signs or symptoms abate shortly after the institution of a therapy and recur in the absence of that therapy, only to disappear once again when therapy is re-instituted.19 The recognition that the causal relationship between treatment and effect can be determined in such a fashion forms the epistemic basis of support for the n-of-1 trial.20,21
That knowledge in medicine can come not only from clinical trials but also from simple clinical observation, then, does not appear to be in doubt. Empirical evidence comes in many forms, from the results of large clinical trials to the simple observation of individual patients. But knowledge from divergent sources may appear to conflict. EBM gives general preference to knowledge obtained from controlled clinical trials; this preference, however, may not be warranted.22 The claim of EBM that certain types of evidence are better guides for medical decision making is not scientifically demonstrable, but rather represents an epistemologic preference. Relying primarily on evidence from clinical trials purporting to demonstrate statistically significant differences is itself problematic.23 What EBM has determined to be “the best available evidence” may, in fact, be ill-suited for guiding many clinical decisions,24,25 while remaining essential for decisions relating to public health or health care economics. The alternative claim that direct observation of the individual patient remains preferable to reliance on the results of clinical trials, then, cannot simply be dismissed.
Rationalism, or the reasoning from theory, offers another potential epistemic framework that may be embraced by alternative medicine. Despite falling out of favor over the last century, this kind of reasoning still exists in orthodox medicine when physicians apply physiologic principles to the patient at hand. In CAM, where a particular theory of illness and healing may play a central role, rationalism may be predominant. The strength of reasoning in rationalism depends, in large degree, on the ultimate truth of the underlying theory. Developing new knowledge, even knowledge that challenges the theory, remains important under this construct. The relationship between medical metaphysics, that is, those theories of disease and healing, and medical epistemology, that is, the development and understanding of medical knowledge, is examined in more detail below.
EVIDENCE FOR INEFFECTIVENESS
In the traditional understanding of clinical medicine, a medical intervention need only be effective in one individual in order to be considered an effective therapy. Despite the emphasis in modern health care economics of demonstrating effectiveness across a population, clinical medicine remains a personal undertaking, relating one individual in need of healing with another who professes and promises to heal.26 To label an intervention clinically “ineffective,” then, is to say that it will not benefit any individual patient, or at least not any individual with a particular disorder.
Unfortunately, EBM lacks the ability to determine that any particular intervention is ineffective. Randomized, controlled trials that fail to demonstrate efficacy of an intervention across a population do not tell us that the intervention was never effective in a particular individual. Some individuals in any large clinical trial may have causally benefited from an intervention that failed to demonstrate efficacy across the population as a whole. It is easy, for instance, to imagine the dismissal as ineffective of an intervention that helps some and harms others in an equal proportion when no difference is found in outcomes compared with a control group. Clinical researchers often attempt to avoid this pitfall by analyzing subgroups of study patients looking for benefit. But even the lack of an apparent sub-group where benefit is demonstrated still does not rule out the possibility of causal effectiveness of the intervention in particular individuals.
Clinical trials that do not show a benefit of a particular intervention are also limited in their claims to demonstrate ineffectiveness by the particulars of the study design. That is, ineffectiveness, at best, can be claimed only for the use of the particular intervention, in a particular dose or format, on a particular schedule, by a particular group of practitioners, in a particular patient population. Presumptions of ineffectiveness can be appropriately challenged by arguments that the intervention was used incorrectly, that the practitioners were unskilled, that the population studied was the wrong one. A seemingly infinite number of studies would be required to demonstrate true ineffectiveness of any particular intervention.
When an individual patient appears to have benefited from an intervention deemed “ineffective” by a clinical trial, alternative explanations other than a direct causal pathway are often offered. But although placebo effects and spontaneous improvement certainly occur, claims that they are the reason for a particular patient's recovery remain speculative. If a patient demonstrates improvement after a session of therapeutic touch, for instance, the possibility that the intervention, and not placebo effect or chance, caused that improvement in that individual still remains, regardless of the results of clinical trials said to demonstrate the ineffectiveness of the practice. Denial of benefit can certainly be argued from a rationalist perspective, that the treatment is incongruous with the biophysiologic model of disease, but it cannot be proven simply by the accumulation of empiric evidence. In addition, it can be argued that the reason for the benefit, whether direct cause or placebo or other indirect effect, is irrelevant to the notion of therapeutic value. The fact that an individual patient feels better, regardless of the reason for that perception, represents a claim of efficacy. Many in CAM require only that patients feel better, while EBM appears to require that they feel better in a certain way, as the result of an intervention that has been shown to be effective in a clinical trial.
The method of EBM provides a means by which a particular therapy can be deemed effective for treating a particular disorder (or more properly, a group of patients with similar manifestations), but cannot alone be used to dismiss interventions that purport to be beneficial for particular patients. That is, EBM lacks the ability to “falsify” claims of benefit made by individual patients and practitioners. The individual patient's perception of improvement may constitute direct evidence of benefit based on primary experience. To prefer indirect evidence, such as that obtained from clinical trials, over primary experience represents an epistemic choice, not a scientific necessity.27 CAM and CAM practitioners, therefore, can continue to emphasize individual outcomes without inconsistency even when the therapies they utilize have failed to demonstrate efficacy in controlled clinical trials.
MEDICAL EPISTEMOLOGY AND MEDICAL METAPHYSICS
The insistence that CAM adopt the epistemic framework of EBM implies that the tools and methods of EBM are sufficient for generating medical knowledge regardless of the underlying theory of disease—the metaphysics of medicine. This position is not philosophically tenable.
The controlled clinical trial may be rejected as the primary method for obtaining medical knowledge in any medical metaphysics that incorporates the following tenet: There are recognizable, but non-measurable or non-quantifiable, differences in the ways disease manifests in individuals that are important for determining accurate diagnosis, prognosis, or treatment. As a corollary, if illness can be viewed only in the context of the individual who is ill rather than as a distinct and disembodied concept of disease, then population-based studies are methodologically inappropriate. In particular, if understanding perceptible but non-measurable aspects of illness is of primary importance in healing, then the focus of research must remain on the individual patient and practitioner.
It is important here to recognize that many of the factors that clinical researchers believe they are measuring are, in fact, unmeasurable in a scientific sense. For instance, pain may be quantified by individuals (e.g., 8-on-a-scale-of-10 chest pain), but it is not measurable in the same way as something like body temperature. With the latter, we can compare the patient who has a temperature of 38.5°C with the one whose temperature is 37.0°C and conclude the former is hotter. But two patients reporting 8-out-of-10 pain are not necessarily experiencing the same thing. One can argue without demonstrable conclusion about whether a woman's 8/10 pain associated with childbirth is worse than her husband's 8/10 pain associated with renal lithiasis. Pain is perceptible, often even to an observer, but it is not measurable in a scientific sense. Our attempt to quantify phenomena such as pain should not be confused with measuring those phenomena.
Schools of CAM may center on the belief in important, perceptible, but non-measurable features and forces, such as Qi. Qi has been defined as “an invisible force which gives life to all living matter … and could be understood as a life or vital energy.”28 Although invisible and non-measurable, Qi is said to be perceptible by both patient and healer. The belief that such spiritual, emotional, psychological, or other non-measurable aspects of the individual patient's presentation are important for healing does not require one to reject evidence obtained from clinical trials, but it does require the recognition that knowledge gained from such methods will be insufficient to guide optimal clinical practice. The emphasis placed on different forms of empirical evidence, either clinical trials or direct observation, will be dependent upon the perceived importance of those non-measurable factors in the healing process. That is, the theory of disease and healing underlying a particular school of medical practice will determine the appropriate epistemic framework for that discipline.29 The EBM framework fits relatively well with the biophysiologic model of disease that underlies orthodox medicine; it does not fit equally well with other theories of disease and healing. The importance of Qi in traditional Chinese medicine (TCM) means that research that cannot and does not account for the force will never be compelling for a TCM practitioner.
Coherence with theory remains an important aspect of not only CAM but orthodox medicine as well. The effectiveness of a particular therapy, demonstrated through a randomized, controlled trial, does not alone lead to the acceptance of that particular therapy even in this age of EBM. Rather, skepticism on the part of orthodox practitioners generally remains until an explanation, preferably supported by additional scientific evidence, is provided as to how that therapy alters human genetics, biochemistry, or physiology. That is, evidence of efficacy alone is not sufficient, coherence within the biophysiologic model is still required.
The most obvious examples of this dependence on theory in orthodox medicine come from CAM therapies that demonstrate efficacy in controlled trials. For instance, acupuncture had been consistently dismissed by orthodox medicine as having no scientific basis,30 yet clinical studies began to suggest effectiveness. Gradual acceptance of acupuncture within the orthodox medical community followed not from the simple accumulation of good clinical evidence, but from a scientifically supported hypothesis that the effects of acupuncture come “not by manipulating Qi but rather by neuroelectric stimulation for the gene expression of neuropeptides.”31 Acceptance is contingent not simply on the weight of empirical evidence, but on coherence with the dominant theory of disease.
When scientific evidence appears that seems to undermine the conventional biophysiologic theory, as happened with studies of the biologic effects of ultra-high dilutions similar to those used in homeopathy, orthodox practitioners are forced to either reject the evidence (a problem when the evidence is of high quality) or else revise or reject the conventional theory.32 Early on, it is easier to do the former rather than the latter.33
Coherence with theory, it appears, remains of vital importance even in an era of EBM. Recognizing their own dependence on theory, orthodox practitioners should respect the same need for coherence in CAM. Instead, the theories and beliefs upon which these therapies have traditionally rested are routinely dismissed as unimportant.34 The theory of disease and healing underlying a particular school of alternative practice remains vital to understanding the relative weight such a school will assign to knowledge gained from controlled clinical trials. For theories that emphasize the importance of non-measurable or non-quantifiable factors in disease and healing, such evidence will be generally dismissed as unhelpful. Acceptable methods of obtaining medical knowledge will depend on coherence with the theory. It follows that CAM practitioners should be able to defend such methods in a rational fashion.
EXPECTATIONS FOR CAM RESEARCH
Recognition of the limitations of evidence-based methods for assessing CAM therapies should not be viewed as an excuse for CAM to avoid seeking meaningful knowledge regarding the effects of its treatments and the claims of its practitioners. The problems of bias and proof of efficacy that are raised by orthodox practitioners regarding CAM are real and must be addressed. CAM, if it is to remain viable and truly alternative, must embrace and develop an alternative epistemic framework for elucidating knowledge valuable for providing clinical care. Still, simply adopting a strictly evidence-based epistemology would likely undermine CAM rather than lead to increased credibility, as some have maintained,5,35 as evidence-based epistemology may not be coherent with the medical metaphysic of the alternative discipline. This would result in an internal inconsistency between a theory of disease and the method for obtaining knowledge within the discipline. Such incoherence would be corrupting.
It is worth noting here the pluralism of CAM, which comprises many different schools of thought. Within this diverse grouping, many important questions that arise may be answerable by appeals to orthodox forms of clinical research. When CAM makes a claim of benefit across a population, the performance of a controlled clinical trial is certainly appropriate. For instance, if St. John's wort is said to be as beneficial as conventional therapy for a certain class of people with depression, then a controlled trial should be able to demonstrate that benefit.36 If, however, the claim is that manipulation of Qi through one of a variety of mechanisms chosen according to the unique presenting features of the particular individual will benefit that individual, controlled clinical trials are inappropriate for attempting to examine that claim.
The question of how CAM can and should be evaluated while remaining internally coherent is currently being debated, and practical methods are being proposed and developed.37 For example, development of a specific method for single-case assessment of causality would fit well with many alternative theories of illness and also could be rigorous enough to satisfy orthodox practitioners. An example of such an approach would be a figural correspondence model, taking advantage of direct observation of certain predefined types of relationships between phenomena generally recognized as being causative.19 The strength of the figural experiment is actually increased when the observer is the same person who performed the intervention, rather than a blinded observer. (Just as I am more certain that a fist caused the hole in the wall when it was my fist.) Such methods are subject to their own epistemic limitations, but are applicable to many interventions and approaches that cannot be subjected to controlled clinical trials. What is important to note about any research focused on the outcome in a particular individual is that it cannot produce knowledge that is generalizable. The value of any form of the n-of-1 trial is that it answers directly the clinical question of whether a particular intervention is beneficial to the patient at hand; what it does not do is tell us anything about whether we should be using the same intervention in other patients or whether it makes sense as a public health measure.
When both the individuality of the patient and the individuality of the practitioner are seen as vital to the healing process, the tools of EBM designed to obscure these effects (e.g., blinding, randomization, placebos) cannot be employed. Instead, alternative methods of measuring outcomes and attempting to avoid bias have begun to be developed. This early work has often been of high quality, but it has yet to find its way into the orthodox medical literature.29,38–41
Improving the dialog between orthodox medicine and CAM practitioners has several potential benefits. First, the development and ultimate acceptance of novel methods for the development of knowledge within CAM will be improved by the vigorous critique of orthodox medicine. CAM needs to acknowledge and overcome the limitations of its current methods for assessing interventions if it desires to convince patients, payers, and physicians of its value. Likewise, the limits of EBM are made most clear when well-reasoned and well-articulated alternatives exist. Arguably, periods where the orthodox medicine went mostly unchallenged, for instance, the Middle Ages in Europe, have been the times of greatest intellectual and clinical stagnation. While EBM has begun to be critiqued internally,22,24,42–44 perhaps the greatest impetus for self-examination comes from the recognition of the continued popularity of CAM despite the apparent scientific success of orthodox medicine. Orthodox medicine stands to gain from the presence of thoughtful and articulate alternatives. Criticism is as important for the advancement of a science or art as method.
The call for an evidence-based CAM is problematic. It assumes, perhaps erroneously, a preference for empirical evidence gained from controlled clinical trials in guiding clinical decision making. It also assumes that the evidence-based epistemic framework generates the best evidence regardless of the underlying theory of disease and healing. But EBM will remain an incomplete guide to optimal clinical practice for disciplines that assert that non-measurable but detectable differences between individuals are important to the diagnosis and treatment of illness. CAM, on the other hand, must more fully develop a rigorous alternative epistemic framework and method if it hopes to gain acceptance among orthodox practitioners.
Ultimately, it is likely that neither evidence-based epistemology nor any particular alternative framework will result in a full medical exegesis. Thoughtful practitioners of orthodox medicine recognize that not all medical questions are best answered by appeal to or performance of a controlled clinical trial. Nor have CAM practitioners rejected EBM out of hand, but instead many seek to define the questions within their realm that are answerable by appeal to evidence.45–47 For now, orthodox medicine should consider abandoning demands that CAM become evidence-based, at least as “evidence” is currently narrowly defined, insisting instead upon a more complete description and defense of the alternative epistemic methods and tools of these disciplines.
1. Fuller R. Alternative therapies: social history. In: Reich W (ed). Encyclopedia of Bioethics. 2nd ed, vol. 1. New York: Macmillan, 1995:126–35.
2. Eisenberg DM, Kessler RC, Foster C, Norlock FE, Calkins DR, Delbanco TL. Unconventional medicine in the United States. Prevalence, costs, and patterns of use. N Engl J Med. 1993;328:246–52.
3. MacLennan A, Wilson D, Taylor A. Prevalence and cost of alternative medicine in Australia. Lancet. 1996;347:569–73.
4. Fulder S, Munroe R. Complementary medicine in the United Kingdom: patients, practitioners and consultations. Lancet. 1985;2(8454):542–5.
5. Fontanarosa PB, Lundberg GD. Alternative medicine meets science. JAMA. 1998;280:1618–9.
6. Beyerstein B. Alternative medicine and common errors of reasoning. Acad Med. 2001;76:230–7.
7. The Evidence-Based Medicine Working Group. Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA. 1992;268:2420–5.
8. Margolin A, Avants SK, Kleber HD. Investigating alternative medicine therapies in randomized controlled trials. JAMA. 1998;280:1626–8.
9. Levin JS, Glass TA, Kushi LH, Schuck JR, Steele L, Jonas WB. Quantitative methods in research on complementary and alternative therapies. A methodological manifesto. Med Care. 1997;35:1079–94.
10. Hensley MJ, Gibson PG. Promoting evidence-based alternative medicine. Med J Aust. 1998;169:573–4.
11. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312:71–2.
12. Sackett DL, Rosenberg WM. The need for evidence-based medicine. J R Soc Med. 1995;88:620–4.
13. Davidoff F, Case K, Fried PW. Evidence-based medicine: why all the fuss? Ann Intern Med. 1995;122:727.
14. Davidoff F, Haynes B, Sackett D, Smith R. Evidence based medicine. BMJ. 1995;310:1085–6.
15. Hadorn DC, Baker D, Hodges JS, Hicks N. Rating the quality of evidence for clinical practice guidelines. J Clin Epidemiol. 1996;49:749–54.
16. Rosenberg C. The therapeutic revolution: medicine, meaning and social change in nineteenth-century America. Perspect Biol Med. 1977;20:485–506.
17. Ackerknecht E. A Short History of Medicine. Baltimore, MD: Johns Hopkins, 1982:145–51.
18. Rodriguez S, Miner P, Robinson M, Greenwood B, Maton PN, Pappa K. Meal type affects heartburn severity. Dig Dis Sci. 1998;43:485–90.
19. Kiene H, von Schon-Angerer T. Single-case causality assessment as a basis for clinical judgment. Altern Ther Health Med. 1998;4:41–7.
20. Guyatt G, Sackett D, Taylor DW, Chong J, Roberts R, Pugsley S. Determining optimal therapy—randomized trials in individual patients. N Engl J Med. 1986;314:889–92.
21. Larson E. N-of-1 trials: a technique for improving medical therapeutics. West J Med. 1990;152:52–6.
22. Tonelli MR. The philosophical limits of evidence-based medicine. Acad Med. 1998;73:234–40.
23. Goodman SN. Toward evidence-based medical statistics. 1: The P value fallacy. Ann Intern Med. 1999;130:995–1004.
24. Feinstein AR, Horwitz RI. Problems in the “evidence” of “evidence-based medicine.” Am J Med. 1997;103:529–35.
25. Herman J. Shortcomings of the randomized controlled trial: a view from the boondocks. J Eval Clin Pract. 1998;4:283–6.
26. Pellegrino ED, Thomasma DC. A Philosophical Basis of Medical Practice: Toward a Philosophy and Ethic of the Healing Professions. New York: Oxford University Press, 1981:209–10.
27. Reed E. The Necessity of Experience. New Haven, CT: Yale University Press, 1996.
28. Hiep N. The Dictionary of Acupuncture and Moxibustion. Wellingborough, U.K.: Thorsons, 1987:112.
29. Edwards RA. Our research approaches must meet the goal of improving patient care. Altern Ther Health Med. 1997;3:99.
30. Reports of the Council on Scientific Affairs of the American Medical Association. Chicago, IL: American Medical Association, 1981.
31. Ulett G, Han J, Han S. Traditional and evidence-based acupuncture: history, mechanisms, and present status. South Med J. 1998;91:1115–20.
32. Vallance A. Can biological activity be maintained at ultra-high dilution? An overview of homeopathy, evidence and Bayesian philosophy. J Altern Complement Med. 1998;4:49–76.
33. Kuhn T. The Structure of Scientific Revolutions. Chicago, IL: University of Chicago Press, 1970.
34. Thomasma D. Toward a 21st century bioethic. Alt Ther Health Med. 1995;1:74–5.
35. Levin JS, Glass TA, Kushi LH, Schuck JR, Steele L, Jonas WB. Quantitative methods in research on complementary and alternative medicine. Med Care. 1997;35:1079–94.
36. Woelk H. Comparison of St John's wort and imipramine for treating depression. BMJ. 2000;321:536–9.
37. Dossey L. How should alternative therapies be evaluated? An examination of fundamentals. Altern Ther Health Med. 1995;1:6–10.
38. Kiene H. A critique of the double-blind clinical trial. Part 1. Altern Ther Health Med. 1996;2:74–80.
39. Kiene H. A critique of the double-blind clinical trial. Part 2. Altern Ther Health Med. 1996;2:59–64.
40. Kienle GS, Kiene H. Placebo effect and placebo concept: a critical methodological and conceptual analysis of reports on the magnitude of the placebo effect. Altern Ther Health Med. 1996;2:39–54.
41. Lukoff D, Edwards D, Miller M. The case study as a scientific method for researching alternative therapies. Altern Ther Health Med. 1998;4:44–52.
42. Naylor CD. Grey zones of clinical practice: some limits to evidence-based medicine. Lancet. 1995;345:840–2.
43. Maynard A. Evidence-based medicine: an incomplete method for informing treatment choices. Lancet. 1997;349:126–8.
44. Horwitz RI. The dark side of evidence-based medicine. Cleve Clin J Med. 1996;63:320–3.
45. Gatchel RJ, Maddrey AM. Clinical outcome research in complementary and alternative medicine: an overview of experimental design and analysis. Altern Ther Health Med. 1998;4:36–42.
46. Moye LA, Richardson MA, Post-White J, Justice B. Research methodology in psychoneuroimmunology: rationale and design of the IMAGES-P clinical trial. Altern Ther Health med. 1995;1:34–9.
47. Tap V, Goppel M. Summary of a report by the Health Council of The Netherlands on alternative modes of treatment under scientific investigation. Altern Ther Health Med. 1995;1:58–63.