In this month’s issue of Anesthesia & Analgesia, the editors provide the reader with an exceptional service: 2 differing perspectives regarding the risks (safety) and, to a lesser extent, the benefits (efficacy) of hydroxyethyl starches (HES).1,2 There has been substantial deliberation (perhaps too agreeable a word) in the past 2 or 3 years, some of it providing more heat than light, some with unbalanced analyses, some of it unnecessarily vitriolic and vituperative, and some needlessly confounding the discussion by reference to withdrawn publications owing to apparent scientific, moral, and ethical misconduct.3–5
The Open Mind articles by Raghunathan et al.2 and by Irwin and Gan1 are a welcome change. Each carefully details the authors’ somewhat disparate views about what we know (and should know, but do not) about HES safety. A recent editorial6 pointed out the critical importance of context and need not be repeated in detail here other than to emphasize its importance, as it also has been recognized in the 2 current publications. It is simultaneously curious and instructive that the senior author of each of the 2 The Open Mind pieces of disparate views is from the same institution. The 2 sets of authors rely on the same information, but differ in their interpretations, perhaps owing to differing clinical perspectives: one centered in the operating room and the other in intensive care units (ICUs). These articles also demonstrate the value of the ordinarily unseen editorial and peer review process. Opaque to the reader, these publications are more centered on the data (still allowing for interpretation), with lesser conjecture than the initial submissions. The focus on the data, as mandated by the reviewers, resulted in views that are closer to each other than were the original submissions. Both views are valuable to the readership.
That substantial differences of opinion exist is an indication that we do not have sufficient data/knowledge to fully answer all questions about HES safety. I do not intend to discuss the issue of efficacy, as that is deeply embedded in the “crystalloid versus colloid” controversy. That topic has been at the center of much research, many symposia, countless discussions, and likely hundreds of thousands of written words, without clear resolution. I would take special note of the recently reported CRISTAL (Colloids versus Crystalloids for the Resuscitation of the Critically Ill) trial, comparing colloids with crystalloids in the ICU, but not intraoperative environment.7 Rather, my focus will be an attempt to evaluate the safety data and how we might apply them.
Many factors characterize the studies that address HES safety. They include differing: HES molecules and preparations, study populations (including disorders(s) and severity), presence or absence of randomization, open or blinded conditions, criteria for initiating and terminating therapy, volume administered, duration of therapy, comparator fluid, attention to and impact of pharmacokinetic differences, criteria or absence thereof for various diagnoses and therapies, end points and their timing relative to fluid administration, sample size (number of patients studied), statistical analyses and their transparency, and author interpretation and portrayal of the data. There is no fully adequate way to parse these; any attempt to do so reflects some bias.
First, it is of substantial importance to recognize that there are several different HES molecules available for IV administration, differing in source material, average in vitro molecular weight, range of in vitro molecular weights, degree and ratio of locations of substitution, extent and rate of metabolism and resultant in vivo molecular weight distribution over time, and nature of the electrolyte solution. These issues have been reviewed in a number of other publications, and need not be repeated here other than to note that these factors do result in differing physiologic consequences, such as intravascular half-life,8 plasma accumulation after multiple doses,9,10 tissue accumulation8,11 (some regard this as a possible etiology of any putative HES “toxicity”), and effects on coagulation.12–14 In the United States, there are 3 HES molecules available: 2 hetastarches, 600/0.75 and 670/0.75 (in reality these are quite similar: while the mean molecular weights differ by approximately 9%, the substitution is identical, and the molecular weight ranges are so large as to question any clinical significance of that difference); and a more modern tetrastarch, 130/0.4. Elsewhere there is generally a greater variety. In the United States, hetastarch is used predominantly, while in much of the rest of the world, hetastarches are not available and a tetrastarch is favored. Clinical trials, reviews, meta-analyses, and regulatory assessments not accounting for the molecular, pharmacokinetic, and pharmacodynamic differences among the various HES molecules are sufficiently flawed to be of little scientific value and can be misleading rather than enlightening (see below).
It is perhaps best to sort the discussion according to clinical context. This appears to be the center of the controversy, as some wish to apply data obtained in critically ill patients to a healthier surgical population. There is legitimate physiologic and pathophysiologic rationale to support parsing the two.
The patient populations discussed in the two The Open Mind articles,1,2 as well as by regulators can be divided into 3 categories: (1) critically ill in ICUs, (2) surgical, and (3) cardiac surgery. A physiologically important difference between the critically ill and others lies in the state of the endovascular glycocalyx (Ref. 6 for a discussion as related to this issue).
Two randomized trials in critically ill patients published in 2012 are largely responsible for the current controversy. “6S”15 compared HES 130/0.42 and “CHEST”16 compared HES 130/0.4 to crystalloid with primary end points of 90-day mortality. 6S studied severely septic patients, finding a statistically significant increased 90-day mortality (50.5% vs 43.0%; P = 0.040), but no difference in 28-day mortality (38.7% vs 36.0%; P = 0.46), in the analysis of the full Kaplan-Meier survival plots (P = 0.07 at 90 days, P = 0.14 at 2 years) or in either of 2 per protocol analyses (excluding those receiving the wrong or no intervention). I have expressed my view elsewhere, and reiterate it here, that in many, if not most trials a more sensible statistical approach to endpoint analysis is that of a modified intention-to-treat analysis. I find it difficult to accept inclusion of patients receiving neither or the wrong therapy. CHEST studied a more broad ICU population, with a significant fraction having been septic, but not necessarily as severely as those in 6S, finding no mortality difference at 90 days (18.0% vs 17.0%; P = 0.28), 28 days (13.8% vs 13.1%; P = 0.42), in the analysis of the full 90-day Kaplan-Meier survival plot (P = 0.27), or in any of the several analyzed subgroups. These differences between the 2 trials cannot be resolved easily and, thus, have led to much speculation. Possibilities include different HES molecules having been studied, or different populations, with a hypothesis that HES might be detrimental in severe sepsis, but not otherwise. A possible pathophysiologic mechanism that would support such a hypothesis is the degradation of the endothelial glycocalyx in sepsis, shock, and septic shock (see Ref. 6 for elaboration). There are limited data regarding the effects of HES on the glycocalyx in such circumstances; however, surprisingly, HES 670/0.75 inhibits endothelial cell permeability in vitro, as well as does plasma, and significantly better than lactated Ringer’s solution,17 but plasma restores endothelial glycocalyx thickness after hemorrhagic shock in rats, while LR and HES 670/0.75 do not.18
Both trials were “pragmatic,” meaning that only random allocation of test article group assignment was controlled; all else was according to physician judgment. In these trials, the test article (HES or crystalloid) was given after initial resuscitation, according to physician preference, many hours after ICU admission, and in CHEST at a time when mean central venous pressure was 10 mm Hg (13.6 cm H2O). Thus, in these trials, it does not appear that the test was one of the volume resuscitation, but rather that of ongoing repetitive volume therapy over many days. This may have impacted the results related to renal, cardiovascular, and hepatic safety. CHEST found a lesser incidence of new cardiovascular failure (36.5% vs 39.9%; P = 0.03), but a higher fraction of patients with increased serum bilirubin (1.9% vs 1.2%; P = 0.046) in those patients randomly allocated to the HES arm of the trial. Neither CHEST nor 6S reported more usual data referable to hepatic injury, such as serum alanine aminotransferase activity. The results of both trials are confounded by the substantial fraction of patients with major protocol violations that exceeded the magnitudes of any differences between groups.
The renal effects in these populations are even less clear. 6S reported an increased incidence of subjective use (no study criteria) of renal replacement therapy (RRT; 21.9% vs 16.3%; P = 0.047), but not when combined with a renal Sequential Organ Failure Assessment score of ≥3. The more objective Risk, Injury, Failure, Loss, and End-stage Kidney (RIFLE) data were not supportive of the RRT data, with RIFLE-R having a higher incidence in the crystalloid group (P = 0.044 by my calculation using the Fisher exact test, rather than the authors’ use of the χ2 approximation), and all other RIFLE criteria showing no difference between groups. CHEST reported an unadjusted significant difference in RRT use, again without objective criteria for its institution (7.0% vs 5.8%, P = 0.0495) that became not statistically significant after adjustment for cofactors. However, I would not quibble over whether the value is just above or just below the arbitrary, although generally accepted, criterion of requiring a probability of >95% to reject the null hypothesis of no difference between groups. Either way it is close and at least highly suggestive. Similar to 6S, however, the data of the more objective RIFLE criteria conflicted with the use of RRT in that RIFLE-R and RIFLE-I had significantly fewer patients meeting those criteria when given HES than crystalloid (P = 0.007 and P = 0.005, respectively), while there was no difference for RIFLE-F (RIFLE-L was not reported). These discrepancies are not easily explained, but one possibility is that in these blinded pragmatic trials, where fluids were given without knowledge of the specific pharmacokinetics of the test article, that the HES persisted intravascularly for longer periods, with possible higher resultant physiologic measures of intravascular “overload” (e.g., central venous pressure or pulmonary arterial wedge pressure) prompting physicians to institute RRT more frequently, independent of the more objective RIFLE criteria of renal injury.
The results of a meta-analysis assessing HES trials in critically ill patients,19 published in February 2013, do not provide additional enlightenment, as all HES molecules and preparations were combined, and the 2 trials discussed above (CHEST and 6S) contributed to >75% of the patients in the meta-analysis, thus driving the results. Patient population subanalyses failed to note a mortality difference for the 714 assessed nonseptic, nontrauma patients (HES 23.0% versus comparators 23.0%).
More recently, the results of the randomized, but not blinded, CRISTAL trial have been reported.7 This “ pragmatic” trial, also conducted in ICUs, was a more general test of colloids versus crystalloids used for initial volume resuscitation (mean systolic blood pressure at baseline approximately 93 mm Hg, mean heart rate 105), not fluid maintenance. The results of those patients given only 1 type of fluid were reported and analyzed separately, allowing comparison of HES with crystalloid when used for volume resuscitation. For the primary end point of 28-day mortality (note that this trial was initiated in 2003, when 28 days was the standard time point, rather than the more recent 90 days in this ICU population), HES mortality did not differ from that of NaCl or lactated Ringer’s solution (HES 23.1% versus crystalloid 26.8%; P = 0.088). However, at 90 days this difference reached significance (HES 28.1% versus crystalloid 33.6%; P = 0.0165). The HES preparations used in this trial were those in use at each institution. However, judging from the site locations, it would appear likely that the vast majority of HES used were tetrastarches. Colloid use had some better clinical outcomes (including lesser use of vasopressors and mechanical ventilation) but these were not analyzed separately by type of colloid (HES, human serum albumin, gelatin), and thus, without availability of additional data, no further conclusions can be drawn.
The pattern of mortality is similar in the 2 ICU trials that showed mortality difference (6S and CRISTAL): not statistically significant at 28 days, but statistically significant at 90 days. Despite some views to the contrary, I would judge these results similarly, with equal validity at the 2 time points, even though one had a primary end point at 28 days and the other at 90 days: that is, both had a statistically nonsignificant result at 28 days with the differences widening with time so that they became significant at 90 days.
GENERAL SURGICAL POPULATION
With few exceptions, nearly all trials have been of small sample size, prompting a recent review20 and 2 recent meta-analyses.21,22 The largest of these encompassed 4529 surgical patients in 59 randomized trials.20 However, not all were blinded, and few followed patients for >1 week. That review, examining tetrastarches exclusively, found no hint of any difference in mortality, blood loss, use of RRT, or peak postoperative serum creatinine, and a suggestion of decreased fraction of patients transfused. Similarly, Martin et al.21 found no evidence for HES 130/0.4-induced renal dysfunction and Gillies et al.22 found no difference in mortality or acute kidney injury (AKI) when comparing all 6% HES solutions together against control fluids or when assessing only tetrastarches against control fluids (Gillies et al.,22 Supplemental Online Information). In the latter report, mortality and RRT use favored (not statistically significantly) tetrastarches. However, as correctly noted by those authors, a trial to demonstrate such improved outcomes for tetrastarches would require large sample sizes (I calculate with a power of 90%, approximately 19,000 and 26,000 patents, respectively). That is not practical, or warranted, especially considering the lack of adverse signal seen in the 3 reviews. Other analyses have been performed, but as indicated by Gillies et al.,22 there are valid reasons to exclude them from consideration.22
It is useful to examine the surgical subcategory of cardiac surgery, in as much as these patients have a higher incidence of AKI and mortality than do general surgical population, and cardiopulmonary bypass can cause some degradation of the endothelial glycocalyx, perhaps secondary to an inflammatory process.23 The meta-analysis of Gillies et al.22 analyzed cardiac surgery trials, finding no difference in mortality (P = 0.91), AKI (P = 0.34) or use of RRT (P = 0.56). Van der Linden et al.20 did not analyze separately the cardiac surgery studies included in their review. I have done so for those studies. There were 1974 patients approximately equally divided between receiving a tetrastarch or some comparator in 21 randomized trials. There was no difference, or a trend, or hint of increase in mortality, blood loss, fraction of patients transfused, or peak postoperative serum creatinine (there were too few reports on RRT use to permit a meaningful analysis) in those patients given a tetrastarch.
Some have argued that postoperative follow-up encompassing just the period of hospitalization is inadequate. However, given the low risks (event rates) of these populations for mortality, AKI, and RRT, and that nearly all such events would be in the very immediate postoperative period, it would seem very unlikely that longer follow-up would produce different results. To propose a trial with the large sample sizes required, especially in the light of the lack of an adverse signal suggesting harm, would not seem warranted.
Adding to the turbidity, rather than clarity, 2 regulatory agencies, the Food and Drug Administration (FDA) in the United States and the European Medicines Agency in the European Union, have weighed in, again, with differing conclusions based on the extant data. As described by Raghunathan et al.,2 the European Medicines Agency revised their conclusion after just 4 months, implying an unsatisfactory initial process, but without much assurance that their second attempt was devoid of similar deficits.24 Their more recent declaration (which is currently being implemented in the European Union) is that all HES solutions “must no longer be used in patients with sepsis or burn injuries or in critically ill patients... but may be used to treat hypovolaemia caused by acute blood loss where treatment with... crystalloids alone are not considered to be sufficient. ... and that HES solutions should not be used for more than 24 hours and patient’s kidney function should be monitored after HES administration.” The FDA provided a “safety update” (with a revision of the product labels [so-called package insert]) with a “boxed warning” (so-called “black-box warning”) “do not use HES solutions in critically ill adult patients, including those with sepsis; avoid use in patients with preexisiting renal dysfunction” and that the “FDA considers increased mortality and renal injury ... to be a class effect.” and to “... monitor the coagulation status of patients undergoing open heart surgery in association with cardiopulmonary bypass as excess bleeding has been reported with HES solutions in this population.”25 The prohibition in all ICU patients is neither supported by the data nor is the concept of a “class effect.” The CHEST trial found no mortality difference, and the conflicting renal data are at best, unclear. In their analysis, the FDA disregarded the differences in HES preparations as described above. It is relevant to note that there is 1 clinical trial, in major orthopedic surgery, comparing 2 HES products available in the United States, HES 130/0.4 and HES 670/0.75.14 That randomized, blinded trial, with the test fluid administered to achieve physiologic goals (in contrast to the pragmatic trials) found lower postoperative coagulation factor VIII activity and von Willebrand factor antigen concentration and greater volume of red cells transfused in those patients given HES 670/0.75 compared with those given HES 130/0.4. Other trials have found differences among other HES preparations. These results highlight the hazard of evaluating these molecules as if they were equivalent and render such analyses and resultant recommendations suspect. Furthermore, as delineated above, the data do not support the warning regarding bleeding with cardiac surgery.
The subtitle of this editorial is taken from A. J. Ayer’s masterpiece of the same name.26 Philosophers have delved into epistemology for many centuries. In general, they have dealt with what it means “to know” in absolute terms: that is, if there is some doubt, then one cannot claim to know something.26 In medical science, we generally deal with probabilities emanating from results of experiments or clinical trials, and the probability of one therapy being superior, equivalent, or noninferior to another, or the untreated disorder, usually accepting a result if there is 95% or more confidence (based on mathematical probability). However, as the 95% “rule” is arbitrary, it would seem that one ought to bring some medical sense to results (see comment above regarding RRT in CHEST). However, that alone does not circumscribe the matter. The “problem” in the matter of HES is confounded further by many critical issues. The inconsistent findings of published studies may be attributable to them having been conducted in differing contexts: differing populations, with different protocols (patient populations, patients enrolled in differing states of their conditions, blinding, criteria for initiating, continuing and terminating therapy, criteria for diagnosis, and treatment of morbidities) and with different HES products. Given the above, philosophers would say that we do not “know” the safety of HES preparations for IV use.
In terms of medical probability, I would suggest that we have enough information, although not “knowledge” as defined above, to guide us. It would seem prudent to avoid prolonged, nonacute resuscitative use in severely septic patients, despite the important methodologic issues of the trials, until better data are available. The 2 contexts of use of HES (ICU and surgery) have physiologic rationales justifying different clinical approaches, as outlined here, by the two The Open Mind pieces, and in a previous editorial.6 There appears to be data strongly suggesting lack of harm in short-term perioperative use for blood volume replacement, albeit with the uncertain importance of the caveat of the usual, but limited length of postoperative follow-up. A decision to institute any therapy should follow consideration of the benefits and risks in comparison with other therapies and the risks of the untreated disorder. The aggregate data, as presented by Van Der Linden et al.20 and Gillies et al.,22 point perhaps mostly in the direction of a potential benefit in the surgical environment; however, sufficiently robust individual trials are lacking. As discussion continues, let it do so based on valid data rather than passion or self- protection, applying the following to all (including me): “...error of opinion may be tolerated where reason is left free to combat it.”27
Name: Richard B. Weiskopf, MD.
Contribution: The author wrote the manuscript.
Attestation: Richard B. Weiskopf approved the final manuscript.
Conflicts of Interest: The author has a relationship with or consults for the following companies and organizations that have an interest in fluid therapy: U.S. Food and Drug Administration; U.S. National Heart, Lung, and Blood Institute/National Institutes of Health; U.S. Department of Defense; CSL Behring; and Terumo BCT. The author was project/corporate Vice President, Chief Medical Officer Biopharmaceuticals, and Executive Scientific Advisor at Novo Nordisk A/S 2005–2007. No one from any of these organizations influenced or participated in the writing or had any knowledge of this editorial.
This manuscript was handled by: Steven L. Shafer, MD.
The author is grateful to many colleagues for many conversations and expositions of their views, during many years prior to writing this editorial, related to some of the topics discussed.
1. Irwin MG, Gan TJ. Volume therapy with hydroxyethyl starches: are we throwing the anesthesia baby out with the intensive care unit bathwater? Anesth Analg. 2014;119:737–9
2. Raghunathan K, Miller TE, Shaw AD. Intravenous starches: is suspension the best solution? Anesth Analg. 2014;119:731–6
3. Shafer SL. Notice of retraction. Anesth Analg. 2010;111:1567
4. Shafer SL. Editor’s note: notices of retraction. Anesth Analg. 2011;112:1246–7
5. Shafer SL. Shadow of doubt. Anesth Analg. 2011;112:498–500
6. Weiskopf RB. Equivalent efficacy of hydroxyethyl starch 130/0.4 and human serum albumin: if nothing is the same, is everything different? The importance of context in clinical trials and statistics. Anesthesiology. 2013;119:1249–54
7. Annane D, Siami S, Jaber S, Martin C, Elatrous S, Declere AD, Preiser JC, Outin H, Troche G, Charpentier C, Trouillet JL, Kimmoun A, Forceville X, Darmon M, Lesur O, Regnier J, Abroug F, Berger P, Clec’h C, Cousson J, Thibault L, Chevret S. Effects of fluid resuscitation with colloids vs crystalloids on mortality in critically ill patients presenting with hypovolemic shock: the CRISTAL randomized trial. JAMA. 2013;310:1809–17
8. Jungheinrich C, Neff TA. Pharmacokinetics of hydroxyethyl starch. Clin Pharmacokinet. 2005;44:681–99
9. Waitzinger J, Bepperling F, Pabst G, Opitz J. Hydroxyethyl starch (HES) [130/0.4], a new HES specification: pharmacokinetics and safety after multiple infusions of 10% solution in healthy volunteers. Drugs R D. 2003;4:149–57
10. Lehmann GB, Asskali F, Boll M, Burmeister MA, Marx G, Hilgers R, Förster H. HES 130/0.42 shows less alteration of pharmacokinetics than HES 200/0.5 when dosed repeatedly. Br J Anaesth. 2007;98:635–44
11. Leuschner J, Opitz J, Winkler A, Scharpf R, Bepperling F. Tissue storage of 14C-labelled hydroxyethyl starch (HES) 130/0.4 and HES 200/0.5 after repeated intravenous administration to rats. Drugs R D. 2003;4:331–8
12. Kozek-Langenecker SA, Jungheinrich C, Sauermann W, Van der Linden P. The effects of hydroxyethyl starch 130/0.4 (6%) on blood loss and use of blood products in major surgery: a pooled analysis of randomized clinical trials. Anesth Analg. 2008;107:382–90
13. Kozek-Langenecker SA. Effects of hydroxyethyl starch solutions on hemostasis. Anesthesiology. 2005;103:654–60
14. Gandhi SD, Weiskopf RB, Jungheinrich C, Koorn R, Miller D, Shangraw RE, Prough DS, Baus D, Bepperling F, Warltier DC. Volume replacement therapy during major orthopedic surgery using Voluven (hydroxyethyl starch 130/0.4) or hetastarch. Anesthesiology. 2007;106:1120–7
15. Perner A, Haase N, Guttormsen AB, Tenhunen J, Klemenzson G, Åneman A, Madsen KR, Møller MH, Elkjær JM, Poulsen LM, Bendtsen A, Winding R, Steensen M, Berezowicz P, Søe-Jensen P, Bestle M, Strand K, Wiis J, White JO, Thornberg KJ, Quist L, Nielsen J, Andersen LH, Holst LB, Thormar K, Kjældgaard AL, Fabritius ML, Mondrup F, Pott FC, Møller TP, Winkel P, Wetterslev J6S Trial Group; Scandinavian Critical Care Trials Group. . Hydroxyethyl starch 130/0.42 versus Ringer’s acetate in severe sepsis. N Engl J Med. 2012;367:124–34
16. Myburgh JA, Finfer S, Bellomo R, Billot L, Cass A, Gattas D, Glass P, Lipman J, Liu B, McArthur C, McGuinness S, Rajbhandari D, Taylor CB, Webb SACHEST Investigators; Australian and New Zealand Intensive Care Society Clinical Trials Group. . Hydroxyethyl starch or saline for fluid resuscitation in intensive care. N Engl J Med. 2012;367:1901–11
17. Wataha K, Menge T, Deng X, Shah A, Bode A, Holcomb JB, Potter D, Kozar R, Spinella PC, Pati S. Spray-dried plasma and fresh frozen plasma modulate permeability and inflammation in vitro in vascular endothelial cells. Transfusion. 2013;53(Suppl 1):80S–90S
18. Torres LN, Sondeen JL, Ji L, Dubick MA, Torres Filho I.. Evaluation of resuscitation fluids on endothelial glycocalyx, venular blood flow, and coagulation function after hemorrhagic shock in rats. J Trauma Acute Care Surg. 2013;75:759–66
19. Zarychanski R, Abou-Setta AM, Turgeon AF, Houston BL, McIntyre L, Marshall JC, Fergusson DA. Association of hydroxyethyl starch administration with mortality and acute kidney injury in critically ill patients requiring volume resuscitation: a systematic review and meta-analysis. JAMA. 2013;309:678–88
20. Van Der Linden P, James M, Mythen M, Weiskopf RB. Safety of modern starches used during surgery. Anesth Analg. 2013;116:35–48
21. Martin C, Jacob M, Vicaut E, Guidet B, Van Aken H, Kurz A. Effect of waxy maize-derived hydroxyethyl starch 130/0.4 on renal function in surgical patients. Anesthesiology. 2013;118:387–94
22. Gillies MA, Habicher M, Jhanji S, Sander M, Mythen M, Hamilton M, Pearse RM. Incidence of postoperative death and acute kidney injury associated with i.v. 6% hydroxyethyl starch use: systematic review and meta-analysis. Br J Anaesth. 2014;112:25–34
23. Bruegger D, Schwartz L, Chappell D, Jacob M, Rehm M, Vogeser M, Christ F, Reichart B, Becker BF. Release of atrial natriuretic peptide precedes shedding of the endothelial glycocalyx equally in patients undergoing on- and off-pump coronary artery bypass surgery. Basic Res Cardiol. 2011;106:1111–21
25. FDA. FDA Safety Communication: Boxed Warning on Increased Mortality and Severe Renal Injury, and Additional Warning on Risk of Bleeding, for Use of Hydroxyethyl Starch Solutions in Some Settings. Revised 25 November 2013. 2013 Available at: http://www.fda.gov/biologicsbloodvaccines/safetyavailability/ucm358271.htm
. Accessed March 18, 2014
26. Ayer AJ The Problem of Knowledge. 1956 London Macmillan
27. Jefferson T First Inaugural Address. March 4, 1801