Share this article on:

An Evaluation of the Evidence in “Evidence-Based” Integrative Medicine Programs

Marcus, Donald M. MD; McCullough, Laurence PhD

doi: 10.1097/ACM.0b013e3181b185f4
Integrative Medicine Programs

Alternative therapies are popular, and information about them should be included in the curricula of health profession schools. During 2000 to 2003, the National Institutes of Health National Center for Complementary and Alternative Medicine awarded five-year education grants to 14 health professions schools in the United States and to the American Medical Students Association Foundation. The purpose of the grants was to integrate evidence-based information about complementary and alternative medicine (CAM) into the curriculum. The authors reviewed the educational material concerning four popular CAM therapies—herbal remedies, chiropractic, acupuncture, and homeopathy—posted on the integrative medicine Web sites of the grant recipients and compared it with the best evidence available. The curricula on the integrative medicine sites were strongly biased in favor of CAM, many of the references were to poor-quality clinical trials, and they were five to six years out of date. These “evidence-based CAM” curricula, which are used all over the country, fail to meet the generally accepted standards of evidence-based medicine. By tolerating this situation, health professions schools are not meeting their educational and ethical obligations to learners, patients, or society. Because integrative medicine programs have failed to uphold educational standards, medical and nursing schools need to assume responsibility for their oversight. The authors suggest (1) appointing faculty committees to review the educational materials and therapies provided by integrative medicine programs, (2) holding integrative medicine programs’ education about CAM to the same standard of evidence used for conventional treatments, and (3) providing ongoing oversight of integrative medicine education programs.

Dr. Marcus is professor of medicine and immunology emeritus, Baylor College of Medicine, Houston, Texas.

Dr. McCullough is professor, Center for Ethics and Health Policy, Baylor College of Medicine, Houston, Texas.

Correspondence should be addressed to Dr. Marcus, Department of Medicine, MS 285, Baylor College of Medicine, 1 Baylor Plaza, Houston, TX 77030; telephone: (713) 798-6014; e-mail: (

Alternative therapies are popular, and information about them should be included in the curricula of health professions schools so that patients can be given sound information and guidance. During 2000 to 2003, the National Institutes of Health National Center for Complementary and Alternative Medicine (NCCAM) awarded five-year education grants to 14 health professions schools in the United States and to the American Medical Students Association Foundation (AMSA).1 Many publications supporting the efficacy of alternative therapies were published during 1980 to 2000. The premise of the NCCAM program was that there was strong evidence supporting the efficacy of some complementary and alternative (CAM) therapies and that information about those therapies was lacking in the curricula of health professions schools. The primary purpose of the grants was to facilitate the incorporation of evidence-based CAM into the medical curriculum. A secondary goal was to accelerate the integration of CAM and conventional medicine by developing integrative medicine programs. The strategies employed by the grant recipients, and their challenges and achievements, were summarized in nine articles in the October 2007 issue of Academic Medicine. Although the need to provide conventional health professionals with authoritative information about CAM was identified as one of the challenges, the emphasis of the articles was on the strategies used to integrate evidence-based CAM into the curriculum. The quality of the evidence was not considered.

Our sole purpose in writing this article was to evaluate the quality of the evidence cited in the evidence-based CAM curricula of the 15 organizations that received NCCAM grants. We do not consider the large, complex issues concerning the role of CAM in health care. Evidence-based medicine (EBM) requires a critical analysis of the quality of research, use of an up-to-date database, and continuing revision of evaluations as new data become available.2 It also requires the integration of the best research evidence with clinical expertise and each patient’s unique values and circumstances.

We examined the Web sites of the 15 NCCAM grantees to ascertain what evidence is presented to students and physicians and how that evidence meets EBM guidelines. Three sites—those of AMSA,3 the Integrative Health Program of the Department of Family Medicine of the University of Washington School of Medicine,4 and the Center for Spirituality and Healing of the University of Minnesota5—contain comprehensive syllabi. The most comprehensive information is in the AMSA Educational Development for Complementary and Alternative Medicine (EDCAM) course modules that include a core curriculum for 10 alternative therapies/healing systems. The curricula were created in 2002, and the curriculum on homeopathy and flower essences was updated in 2003. The EDCAM course modules are listed as a resource on many integrative medicine Web sites, and many sites contain links to the modules. Most of the Web sites did not present detailed curricula, but they contained general monographs about CAM therapies, reading lists, and links to national resources.

We summarize below the best evidence concerning several popular therapies and compare that evidence with the information provided by the integrative medicine centers programs funded by NCCAM. Our evaluation of the quality of the evidence is based on meta-analyses and systematic reviews; randomized, controlled clinical trials published in leading medical journals; our independent evaluation of publications; and recent monographs by Bausell6 and by Singh and Ernst.7 R. Barker Bausell, PhD, a research methodologist who was the research director of an NCCAM-funded center for five years, recently performed an extensive analysis of the quality of CAM trials and, as part of that process, surveyed the Cochrane Database of Systematic Reviews. Edzard Ernst, MD, is director of the Department of Complementary Medicine at the Universities of Exeter and Plymouth in Great Britain, and is an international authority in this field.

Back to Top | Article Outline

Herbal Medicines

Evidence-based medicine

Herbal medicines are plant extracts that have been used for medicinal purposes for thousands of years. As purified medications became available during the 20th century, herbal use in the United States declined until the passage of the Dietary Supplement and Health Education Act (DSHEA) in 1994.8 By designating herbal medicines as “dietary supplements”—that is, as neither food nor drugs—the DSHEA made it impossible for the Food and Drug Administration (FDA) to effectively regulate the use and advertising of these products. The annual sales of dietary supplements amounted to approximately $4 billion in 1994 but rose to $18 billion during the next decade.9

What is the evidence for the efficacy and safety of herbals, what is their risk/benefit ratio, and what advantages do they offer compared with conventional medications? A fundamental problem inherent in the use of botanical extracts is the lack of quality control of the products.10–14 Safe and effective use of a medication requires consistency in composition and biological activity. The active ingredients of most herbals have not been identified, and they are not standardized for their biological activities. Claims for standardization relate only to extraction procedures or arbitrary chromatographic markers. Moreover, adulteration by heavy metals and over-the-counter or prescription medications is common, especially in “all-natural” remedies for sexual dysfunction and weight loss.15–17

With regard to efficacy, until recently most trials of herbal medicines and supplements were funded by manufacturers, and their positive outcomes conflict with negative findings by independent studies. For example, an analysis of trials of glucosamine for osteoarthritis of the knee found that all trials funded by manufacturers were positive and that all independently funded trials were negative.18 The positive effect size of the industry-funded trials was 0.55, a very strong clinical benefit, compared with an effect size of zero for independent trials. Recent, independently funded trials of black cohosh, echinacea, saw palmetto, glucosamine, and ginkgo biloba have been negative,19–24 as was a review of glucosamine and chondroitin by the Agency for Healthcare Research and Quality.25 As pointed out in recent reviews and monographs,6,7,10,14,26 until recently most of the data available for reviews were from poor-quality trials. The methodological defects of these trials include lack of information about random allocation of subjects, blinding of subjects and evaluators, and lack of disclosure concerning the sources of funding for the studies and who analyzed the data. These defects were not present in the recent trials funded by NCCAM and other independent sources. Inclusion of the earlier flawed trials in many reviews obscured the results of the less numerous good trials.

The belief that herbals are generally safe is based mostly on tradition and not on scientifically obtained data. Clinical trials of herbals have not included laboratory monitoring to detect adverse effects, and manufacturers were not required to report severe adverse effects to the FDA before December 2007. Despite these limitations, numerous reports of severe adverse events and deaths have been published in recent years.10,11,14,17 The most serious problem identified is the nephrotoxicity and carcinogenicity of herbals containing extracts of Aristolochia plants, which have been used all over the world for more than 1,000 years. More than 100 women in Brussels developed nephropathy and renal failure caused by an herbal weight loss product that contained Aristolochia fangchi , and carcinomas of the urothelial tract developed in 40% of them.27,28

The use of herbal medicines is plausible because they may have beneficial active ingredients. However, uncertainty about their composition, activity, purity, and safety makes it very difficult to evaluate their risk:benefit ratio, and their use should not be recommended.

Back to Top | Article Outline

Integrative medicine programs

The AMSA EDCAM module entitled “Herbal Medicines”3 was written by Mark Blumenthal, PhD, the director of the American Botanical Council. The council is a nonprofit organization that disseminates information about herbals and promotes their use. The module supports the use of herbals in a primary or adjunct role in the treatment of circulatory, endocrine, gastrointestinal, genitourinary, and neurological conditions. The research summary states that “there are numerous systematic reviews and meta-analyses on herbs and phytomedicines, many supporting the relative safety and efficacy of a particular herb or herb product over placebo for a particular indication.”3 Seven landmark studies and reviews, published between 1996 and 2002, are cited as supporting evidence. The important issues of quality control and safety are not discussed at all. There are no references to recent high-quality clinical trials of herbals19–24 that found no efficacy for many of the herbals recommended in the module. The module is biased, out of date, and misrepresents the evidence for the risks and benefits of herbal remedies.

The University of Minnesota Center for Spirituality and Healing monograph entitled “Botanical Medicines”5 supports the use of botanicals for health maintenance, disease prevention, and treatment of disorders that affect all major organ systems. Claims for treatment of diseases with herbal products are illegal without the approval of the FDA, but the monograph contains a disclaimer that the “Web pages are for general health science purposes only and are not intended to provide medical advice.” Brief reviews of popular herbals are provided and are accompanied by references to reports published in 2003 and earlier. The issues of quality control and safety are not discussed, but a comment is made about the “inherently large margin of safety for the majority of botanicals.” The monograph contains links to the Web sites of the American Botanical Council, the Herbal Products Association, and other organizations that promote the use of herbals.

The University of Washington School of Medicine’s Web site4 expresses concern about the lack of quality control and standardization of herbals and other dietary supplements, and summarizes evidence concerning 18 supplements and herbals. The publications cited are from 2003 and earlier and lack data from more recent, high-quality, independent trials.

None of the other Web sites of the organizations that received the NCCAM grants contain specific information about herbal medicines, but, with the exception of the Boston Children’s Hospital site, they all either offer courses in herbal medicine or offer clinical services that include the use of herbal therapy.

Back to Top | Article Outline


Evidence-based medicine

In its initial formulation, chiropractic manipulation of the spine to correct subluxation of vertebrae was recommended for treatment of all medical disorders. The use of medications was prohibited because they might interfere with the natural healing promoted by manipulation.7 (pp145–189), 29 The profession is now divided into a minority of “straights” and a majority of “mixers.” Mixers employ a variety of physical therapy techniques, herbal medicines, and even acupuncture, in addition to manipulation. Chiropractic care is used primarily for treatment of musculoskeletal problems, especially low-back and neck pain. Straights use only manipulation, oppose standard childhood immunizations, and offer to provide primary care for children.

After exclusion of serious underlying conditions, current clinical guidelines for treatment of acute low-back pain include reassurance about its favorable outcome and the patient’s ability to maintain activity, using acetaminophen if needed.30–34 There is no consistent, convincing evidence that spinal manipulation or other therapies accelerate recovery, although patients who receive “active” therapies are more satisfied with their care.29–35 Manipulation of the spine appears to be safe, but manipulation of the neck has been associated with vertebral artery dissections and cerebrovascular accidents.29,36 An editorial in Neurology commented that “in the absence of randomized-controlled evidence demonstrating the efficacy of cervical manipulation, the best current evidence suggests that the small risk of dissection and stroke outweighs the benefit of this treatment modality for patients with acute neck pain.”37 The opposition of some chiropractors to childhood vaccinations puts children and the community at risk.

Chiropractic practice is very eclectic: Some practitioners use manipulation only, whereas others employ a variety of modalities, including herbal medications. The evidence supporting chiropractic treatment of musculoskeletal disorders is, at best, borderline, and manipulation of the cervical spine poses a small risk of serious adverse events.

Back to Top | Article Outline

Integrative medicine centers

The AMSA EDCAM curriculum,3 written by a professor at the Philadelphia College of Osteopathic Medicine and a doctor of osteopathy, contains a monograph entitled “Chiropractic and Osteopathy” that focuses on manipulative treatment. The monograph supports the efficacy of manipulation for musculoskeletal disorders and suggests possible benefits for other complaints such as asthma, infantile colic, chronic obstructive pulmonary disease, and pneumonia. Most of the landmark studies cited concern osteopathic manipulation; they are all from 1998 and 1999, and there is no mention of possible adverse effects.

The University of Minnesota Medical School monograph about chiropractic states that evidence supports the use of chiropractic treatment for headaches and for neck and back pain.5 The articles and books cited include references from 2005, but negative studies and reports of adverse events were not included. The brief monograph about chiropractic on the Web site of the University of Washington School of Medicine states that there is little evidence to support any benefit of chiropractic treatment for neck or back pain.4 In contrast, the Web site of the School of Nursing of the University of Washington,38 which also received an education grant from NCCAM, supports the use of chiropractic and contains a PowerPoint presentation that presents its benefits.

Back to Top | Article Outline


Evidence-based medicine

Homeopathy is based on the principle that an illness can be cured by administration of a highly diluted medication that, in higher concentration, produces symptoms similar to the ailment: “Like cures like.”7 (pp91–143) Most homeopathic remedies are so diluted that they contain no molecules of the original medication.7 (pp98–100), 39

Homeopathic drugs are regulated differently by the FDA than conventional medications or dietary supplements. Under the provisions of the Food, Drug, and Cosmetic Act of 1938,40 all homeopathic remedies listed in the homeopathic pharmacopoeia of that time were exempted from tests for efficacy or safety. As stated on the FDA Web site, “Manufacturers of homeopathic drugs are deferred from submitting new drug applications to the FDA. Their products are exempt from good manufacturing requirements related to expiration dating and from finished product testing for identity and strength.” Although these remedies were initially prepared only by physicians, they can now be purchased over the counter and on the Internet as long as they do not claim to treat a serious disease.

What is the evidence for the efficacy of homeopathic remedies? A widely cited 1997 publication concluded that “the results of our meta-analysis are not compatible with the hypothesis that the clinical effects of homeopathy are completely due to placebo.”41 After critics pointed out that 68 of the 89 trials included in the meta-analysis were of poor quality, the authors reexamined the same data two years later and concluded that “studies with better methodological quality tended to yield less positive results.”42 A later meta-analysis, which identified only eight trials that met higher standards for quality, concluded that “homeopathy was only very marginally more effective than placebo.”43 That report was accompanied by an editorial in The Lancet entitled “The end of homeopathy.”44 Despite these analyses demonstrating no clear benefit for homeopathy beyond a placebo effect,7 (pp91–143), 45 advocates continue to cite the 1997 publication as evidence for homeopathy’s efficacy.

Homeopathy has been more popular in Europe than in the United States. In 2007, the West Kent Primary Care Trust, a unit of the British National Health Service, commissioned a review of studies of homeopathy. On the basis of findings that “there was insufficient evidence of effectiveness and very little evidence about cost-effectiveness,” the trust phased out funding for homeopathy.46 Other trusts have also either stopped referring patients to homeopathic hospitals or have strictly limited referrals.47

Back to Top | Article Outline

Integrative medicine centers

The AMSA EDCAM monograph,3 written by a physician who is past president of the American Institute of Homeopathy, was created in 2002 and updated in 2003. The favorable 1997 meta-analysis by Linde et al41 is cited by EDCAM, but the later publications that negated its conclusions were not cited by EDCAM or the sites discussed below. EDCAM lists the Linde report as one of six landmark studies that suggest the value of homeopathic treatments. The summary states that clinical research suggests efficacy for homeopathy in many clinical conditions, and it questions the value of randomized controlled trials of homeopathy because they “fail to change philosophical beliefs.” Links to the EDCAM syllabi are present on most of the integrative medicine Web sites.

The University of Minnesota Medical School’s Web site doesn’t contain a monograph on homeopathy, but the site’s overview section states that homeopathy is more than a placebo and cites as evidence the 1997 Linde publication.5 The University of Washington School of Medicine’s brief summary, revised in 2004,4 concludes that although the evidence supporting homeopathy is inconclusive, “physicians should be open-minded about homeopathy’s possible value.”

There is no specific curriculum content about homeopathy on other integrative medicine Web sites, but the clinical services offered in many integrative medicine clinics include homeopathy and naturopathy, which include homeopathic therapy.

Back to Top | Article Outline


Evidence-based medicine

As developed in China and other Asian countries more than 2,000 years ago, acupuncture needles are inserted into the skin at specific points along meridians, or channels.7 (pp39–88) Selected groups of points are used to treat different diseases. In traditional Chinese medicine, acupuncture is part of a comprehensive treatment plan that may also include herbal therapy, diet, and exercise, but it is increasingly used as a stand-alone treatment by Western health care providers.

Advocates recommend acupuncture for treatment of numerous diseases, but the only indications supported by some credible evidence are for pain and nausea. Trials conducted in the 1970s and 1980s demonstrated that patients treated with acupuncture experienced less pain than untreated patients.7 (pp39–88) However, no treatment is an inadequate control, and starting in the 1990s a variety of “sham” acupuncture controls were used, including insertion of needles at nonacupuncture points, noninsertion of needles, and use of telescoping needles that appeared to be inserted into the skin. These studies compared no treatment against sham and traditional acupuncture for treatment of headaches, back pain, osteoarthritis, and relief of pain following dental surgery. Sham and conventional acupuncture provided significant analgesia compared with no treatment. However, there was little or no difference between the relief experienced by the two acupuncture groups.48–52 In these studies, most participants could not distinguish between the sham and traditional acupuncture treatments. The roles of belief in the procedure and expectations of relief were demonstrated in several trials.

The conclusion of these studies is that the relief of pain observed resulted from placebo effects and not from insertion of needles at specific points prescribed by tradition.52–54 Because the acupuncture treatments, conventional and sham, provided analgesia, some investigators believe that the treatment should be recommended to patients without revealing its placebo nature.

In summary, a rigorous evidence-based evaluation of acupuncture is that there is conflicting evidence about its use for treating nausea, and there is no evidence for a specific benefit beyond the placebo effect for relief of pain.7 (pp39–88), 48–55 There is no credible evidence at present supporting the use of acupuncture to treat any other condition.

Back to Top | Article Outline

Integrative medicine centers

A brief discussion of acupuncture is included in the AMSA EDCAM curriculum monograph entitled “Traditional Chinese Medicine, Kampo, Tibetan Medicine and Acupuncture.”3 The monograph, written by three doctors of Oriental medicine and a Western physician, includes summaries of 10 landmark studies published between 1998 and 2002 that support the use of acupuncture.

The University of Minnesota Medical School’s monograph entitled “Traditional Chinese Medicine” contains a section on acupuncture.5 It supports the use of acupuncture for pain management, treatment of injury, and a variety of conditions including nausea, dysmenorrhea, asthma, stroke rehabilitation, and substance abuse addiction. The University of Washington School of Medicine’s monograph entitled “Acupuncture and Oriental Medicine” indicates that there is “some evidence suggesting benefit for treating low back pain, chronic headaches, chemotherapy and pregnancy-related nausea and vomiting and irritable bowel syndrome.”4 No efficacy was found for treatment of asthma, tobacco addiction, or abuse of cocaine, alcohol, or heroin.

None of these monographs consider recent studies indicating that acupuncture analgesia is mediated by placebo effects. A course of acupuncture treatments is time-consuming and expensive; individual sessions may cost $50–$125 each. It may be ethically questionable to recommend acupuncture treatment to patients without revealing what is known about its mechanism of action. Moreover, acupuncturists who are trained in traditional Chinese medicine also frequently prescribe herbal remedies, which may cause renal failure and other severe adverse events.56

Back to Top | Article Outline


The quality of evidence in integrative medicine curricula

In general, the evidence base used by the integrative medicine programs whose Web sites contain relevant information includes only studies reporting positive outcomes, regardless of the studies’ quality. Few publications after 2003 are cited, which include most of the best-quality trials of CAM therapies, and the sites have not been revised to take into account new data. The evidence base of these programs fails to meet the generally accepted standards of EBM, and its representation as “evidence based” is misleading. The AMSA EDCAM modules, which are used widely by integrative medicine programs, are uncritical endorsements that were written by practitioners and advocates of CAM. The University of Washington School of Medicine’s site4 is more critical in evaluating the efficacy of alternative therapies, but its references have not been updated since 2003. Because our analysis focused only on institutions that received education grants from NCCAM, curricula used by other integrative medicine programs should also be reviewed.

In retrospect, the premise of the NCCAM education grant program, that there was solid evidence supporting the efficacy of alternative therapies, was incorrect. Larger, more rigorous, independently funded clinical trials performed during the last decade have not confirmed the positive results of earlier trials, and systematic reviews have pointed out the methodological shortcomings of those trials. The failure of integrative medicine programs to update their database and revise their evaluations of therapies suggests a lack of genuine commitment to evidence-based medicine. Moreover, some integrative medicine programs question the validity of randomized, controlled trials for studying alternative therapies, and they promote the acceptance of traditional beliefs. The University of North Carolina at Chapel Hill School of Medicine’s Program in Integrative Medicine Web site contains a general monograph entitled “Assessing the Effectiveness of Complementary and Alternative Medicine.”57 The monograph is skeptical of the value of randomized, controlled trials in evaluating healing systems. It makes a distinction between the efficacy of a treatment, its “internal validity” as analyzed by a randomized, controlled trials, and its effectiveness or “external validity,” that is, whether it is perceived to work in clinical practice. In other words, it discounts evidence in favor of anecdotes and clinical impressions. This issue was discussed by Dr. Stephen Straus, the former director of NCCAM, and his colleagues who concluded, “We argue that public health and safety demand rigorous research evaluating CAM therapies.”58

The remarkable improvement in the practice of medicine during the last 50 years was made possible by advances in biomedical science and evidence-based therapeutics. Readers of the British Medical Journal voted evidence-based medicine as one of the 15 most important medical milestones since 1840.59 A brief overview stated that “the systematic synthesis of evidence is the foundation of all medical discoveries and of good clinical practice.”60 Although the evidence base is always provisional and subject to revision, rejection of evidence in favor of traditional beliefs and clinical anecdotes compromises educational standards and clinical practice.

Back to Top | Article Outline

Educational and ethical issues

The flawed curricula presented by integrative medicine programs constitute an educational failure on the part of health professions schools and AMSA. Evidence supporting the efficacy and safety of CAM should be evaluated by the same standards used to analyze conventional therapies. Failure to do so undermines the problem-based educational initiative that emphasizes rigorous review of current literature; it tolerates a double standard of evidence, and it promotes a poor standard of medical practice.

By tolerating this situation, health professions schools are not meeting their ethical obligations to learners, patients, or society. Academic health centers should provide exemplary care to the communities that they serve. An essential component of that mission is provision of the best information available to foster informed decisions about health care. Because CAM is promoted uncritically by the media,7 (pp250–266), 61 academic health centers should be a source of sound information that is not otherwise available. Failure to meet this obligation confers undeserved credibility on alternative therapies and promotes their use.

These problems have arisen because, to the best of our knowledge, there is no formal oversight of curriculum content by medical schools. Oversight of curriculum content takes place in an informal manner by faculty colleagues who share responsibility for courses and clerkships. For example, a teacher who presented an outdated, unsound therapy for hypertension would be taken to task by her or his colleagues. This corrective process clearly doesn’t occur in integrative medicine programs, as one of us (D.M.M.) cautioned several years ago.62

This situation presents a unique challenge to health professions schools. We suggest the following steps to deal with the problem:

  • appoint a faculty panel to review educational material and therapies provided by integrative medicine centers;
  • provide education about CAM based on the same standard of evidence used to develop guidelines for treatments for cardiovascular diseases, diabetes, etc.; and
  • carry out ongoing oversight of integrative medicine education programs.

Implementation of these procedures will face resistance. Integrative medicine faculty will raise the issue of academic freedom, but academic freedom requires accountability, and it doesn’t justify ignoring educational standards. Leaders of academic institutions, and faculty who teach and practice genuine evidence-based medicine, need to acknowledge the problem and take steps to remedy it.

Back to Top | Article Outline


1 Pearson NJ, Chesney MA. The CAM Education Program of the National Center for Complementary and Alternative Medicine: An overview. Acad Med. 2007;82:921–926.
2 Straus SE, Richardson WS, Glasziou P, Haynes RB. Evidence-Based Medicine. How to Practice and Teach EBM. 3rd ed. New York, NY: Elsevier; 2005.
3 AMSA EDCAM Initiative: A National Curriculum for Medical Students. Available at: ( Accessed June 3, 2009.
4 University of Washington Integrative Health Program (UHW-IHP). Available at: ( Accessed June 4, 2009.
5 University of Minnesota Center for Spirituality and Healing. Available at: ( Accessed June 3, 2009.
6 Bausell RB. Snake Oil Science. The Truth About Complementary and Alternative Medicine. New York, NY: Oxford University Press; 2007.
7 Singh S, Ernst E. Trick or Treatment? The Undeniable Facts About Alternative Medicine. New York, NY: W.W. Norton; 2008.
8 Dietary Supplement Health and Education Act (DSHEA, Public Law 103-417, 25 October 1994; Codified at 42 USC 287C-11).
9 Committee on the Use of Complementary and Alternative Medicine by the American Public. Dietary supplements. In: Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press; 2005.
10 De Smet PA. Herbal remedies. N Engl J Med. 2002;347:2046–2056.
11 Marcus DM, Grollman AP. Botanical medicines—The need for new regulations. N Engl J Med. 2002;347:2073–2076.
12 Straus SE. Herbal medicines—What’s in the bottle? N Engl J Med. 2002;347:1997–1998.
13 Garrard J, Harms S, Eberly LE, Matiak A. Variations in product choices of frequently purchased herbs. Caveat emptor. Arch Intern Med. 2003;163:2290–2295.
14 De Smet PA. Health risks of herbal remedies: An update. Clin Pharmacol Ther. 2004;76:1–17.
15 Cole MR, Fetrow CW. Adulteration of dietary supplements. Am J Health Syst Pharm. 2003;60:1576–1580.
16 Saper RB, Kales SN, Paquin J, et al. Heavy metal content of ayurvedic medicine products. JAMA. 2004;292:2868–2873.
17 Smolinske SC. Herbal product contamination and toxicity. J Pharm Pract. 2005;18:189–208.
18 Vlad VC, LaValley MP, McAlindon TE, Felson DT. Glucosamine for pain in osteoarthritis. Why do trial results differ? Arth Rheum. 2007;56:2267–2277.
19 Pockaj BA, Gallagher J, Loprinzi CL, et al. Phase III double-blind, randomized, placebo-controlled crossover trial of Black Cohosh in the management of hot flashes: NCCTG trial NO1CC. J Clin Oncol. 2006;24:2836–2841.
20 Newton KM, Reed SD, LaCroix AZ, Grothaus LC, Ehrlich K, Guiltinan J. Treatment of vasomotor symptoms of menopause with black cohosh, mutlibotanicals, soy, hormone therapy or placebo: A randomized trial. Ann Intern Med. 2006;145:869–879.
21 Turner RB, Bauer R, Woelkart K, Hulsey TC, Gangemi JD. An evaluation of Echinacea augustifolia in experimental rhinovirus infections. N Engl J Med. 2005;353:341–348.
22 Bent S, Kane C, Shinohara K, et al. Saw palmetto for benign prostatic hyperplasia. N Engl J Med. 2006;354:557–566.
23 Clegg DO, Reda DJ, Harris CL, et al. Glucosamine, chondroitin sulfate, and the two in combination for painful knee osteoarthritis. N Engl J Med. 2006;354:795–808.
24 DeKosky ST, Willamson JD, Fitzpatrick AL, et al. Ginkgo biloba for prevention of dementia. A randomized controlled trial. JAMA. 2008;300:2253–2262.
25 Agency for Healthcare Research and Quality. Treatment of Primary and Secondary Osteoarthritis of the Knee. Rockville, Md: Agency for Healthcare Research and Quality; September 2007. AHRQ Publication No. 07-E012.
26 Guo P, Canter PH, Ernst E. A systematic review of randomized clinical trials of individualized herbal medicines for any indication. Postgrad Med J. 2007;83:633–637.
27 Nortier JL, Martinez MM, Schmeiser HH, et al. Urothelial carcinoma associated with the use of a Chinese herb (Aristolochia fangchi). N Engl J Med. 2000;342:1686–1692.
28 Vanherweghem J-L, Bebelle F, Muniz-Martinez M-C, Nortier J. Aristolochic acid nephropathy after Chinese herbal remedies. In: De Broe ME, Porter GA, Bennett WB, Verpooten GA, eds. Clinical Nephrotoxins. 2nd ed. Dordrecht, Netherlands: Kluwer Academic Publishers; 2003:579–586.
29 Ernst E. Chiropractic: A critical evaluation. J Pain Symptom Manage. 2008;35:544–562.
30 Koes BW. Evidence-based management of acute low back pain. Lancet. 2007;370:1595–1596.
31 Hancock MJ, Maher CG, Latimer J, et al. Assessment of diclofenac or spinal manipulative therapy, or both, in addition to recommended first-line treatment for acute low back pain. Lancet. 2007;370:1638–1643.
32 Assendelft WJJ, Morton SC, Yu EI, Suttorp MJ, Shekelle PG. Spinal manipulative therapy for low back pain. Ann Intern Med. 2003;138:871–881.
33 Cherkin DC, Sherman KJ, Deyo RA, Shekelle PG. A review of the evidence for effectiveness, safety and cost of acupuncture, massage therapy, and spinal manipulation for back pain. Ann Intern Med. 2003;138:898–906.
34 Deyo RA. Treatments for back pain: Can we get past trivial effects? Ann Intern Med. 2004;141:957–958.
35 Ernst E, Canter PH. A systematic review of systematic reviews of spinal manipulation. J R Soc Med. 2006;99:192–196.
36 Smith WS, Johnston SC, Skalabrin EJ, et al. Spinal manipulative therapy is an independent risk factor for vertebral artery dissection. Neurology. 2003;60:1424–1428.
37 Williams LS, Biller J. Vertebrobasilar dissection and cervical spine manipulation. A complex pain in the neck. Neurology. 2003;60:1408–1409.
38 University of Washington School of Nursing. Integrating CAM: A nursing emphasis. Available at: ( Accessed June 4, 2009.
39 Bunker MM. Therapeutic homeopathy. In: Rakel D, ed. Integrative Medicine. 1st ed. New York, NY: Saunders; 2003:781–787.
40 FDA Compliance Policy Guide. Conditions Under Which Homeopathic Drugs May Be Marketed (CPG 7132.15) Sec 400.400. Available at: ( Accessed June 4, 2009.
41 Linde K, Clausius N, Ramirez G, et al. Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trials. Lancet. 1997;350:834–843.
42 Linde K, Scholz M, Ramirez G, Clausius N, Melchart D, Jonas WB. Impact of study quality on outcome in placebo-controlled trials of homeopathy. J Clin Epidemiol. 1999;52:631–636.
43 Shang A, Huwiler-Muntener K, Nartey L, et al. Are the clinical effects of homeopathy placebo effects? Comparative study of placebo-controlled trials of homeopathy and allopathy. Lancet. 2005;366:726–732.
44 The end of homeopathy. Lancet. 2005; 366:690.
45 Ernst E. Homeopathy: The effective promotion of ineffective remedies. Br J Clin Pharmacol. 2006;62:645–646.
46 Samarasekera U. Pressure grows against homoeopathy in the UK. Lancet. 2007;370:1677–1678.
47 Barrett S. Another UK health service agency ends homeopathic funding. Available at: ( Accessed June 3, 2009.
48 White P, Lewith G, Prescott P, Conway J. Acupuncture versus placebo for the treatment of chronic mechanical neck pain. Ann Intern Med. 2004;141:911–919.
49 Assefi NP, Sherman KJ, Jacobsen C, Goldberg J, Smith WR, Buchwald D. A randomized clinical trial of acupuncture compared with sham acupuncture in fibromyalgia. Ann Intern Med. 2005;143:10–19.
50 Scharf HP, Mansmann U, Streitberger K, et al. Acupuncture and knee osteoarthritis. Ann Intern Med. 2006;145:12–20.
51 Haake M, Muller H-H, Schade-Brittinger C, et al. German acupuncture trials (GERAC) for chronic low back pain. Arch Intern Med. 2007;167:1892–1898.
52 Foster NE, Thomas E, Barlas P, et al. Acupuncture as an adjunct to exercise based physiotherapy for osteoarthritis of the knee: Randomized controlled trial. BMJ. 2007; 335:436.
53 Manheimer E, Ezzo J, Hadhazy V, Berman B. Published reports of acupuncture trials showed important limitations. J Clin Epidemiol. 2006;59:107–113.
54 Manheimer E, Linde K, Lao L, Bouter LM, Berman BM. Meta-analysis: Acupuncture for osteoarthritis of the knee. Ann Intern Med. 2007;146:868–877.
55 Ernst E. Acupuncture—A critical analysis. J Intern Med. 2006;259:125–137.
56 Meyer MM, Chen T-P, Bennett WM. Chinese herb nephropathy. Proc (Bayl Univ Med Cent). 2000;13:334–337.
57 Program on Integrative Medicine, Department of Physical Medicine and Rehabilitation, UNC Chapel Hill School of Medicine. Available at: ( Accessed June 2, 2009.
58 Miller FG, Emanuel EJ, Rosenstein DL, Straus SE. Ethical issues concerning research in complementary and alternative medicine. JAMA. 2004;291:599–604.
59 Godlee F. Milestones on the long road to knowledge. BMJ. Available at: ( Click on “Archives,” then “volume 334, supplement 1, January 2007.”
60 Dickersin K, Straus SE, Bero LA. Evidence based medicine: Increasing, not dictating, choice. BMJ. Available at: ( Click on “Archives,” then “volume 334, supplement 1, January 2007.”
61 Weeks L, Verhoef M, Scott, C. Presenting the alternative: Cancer and complementary and alternative medicine in the Canadian print media. Support Care Cancer. July 12, 2007. Available at: ( Accessed June 3, 2009.
62 Marcus DM. How should alternative medicine be taught to medical students and physicians? Acad Med. 2001;76:224–229.
© 2009 Association of American Medical Colleges