Secondary Logo

Journal Logo

Review Article

Preclinical Research Reporting in Shock: Room for Improvement

Reynolds, Penny S.; Garvan, Cynthia W.

Author Information
doi: 10.1097/SHK.0000000000001544
  • Free
  • Editor's Choice

Abstract

INTRODUCTION

The ARRIVE (Animal Research: Reporting of In Vivo Experiments) guidelines [http://www.nc3rs.org.uk/arriveguidelines] comprise a checklist of the most essential items of information to be included in animal-based research reports. This checklist enables investigators to identify and report key information necessary for independent scientific appraisal of the research. Transparent reporting also improves communication of research methods and findings, facilitates extraction of key information for systematic reviews (1), and increases potential for translation (2, 3). It is not necessary that investigators perform every item listed, only that reporting of each item is complete, and if key methodological procedures necessary for best scientific practice (such as randomization and blinding) are not performed, then their omission should be mentioned and justified if possible.

The Shock Society endorsed ARRIVE guidelines in 2012 (4). Since then, their application has been promoted repeatedly in review articles and scientific best practice recommendations published in Shock(5–8) and elsewhere (9–12). Shock provides a statement of expectation of use on the Instructions for Authors page, and reviewers must indicate that manuscripts conform to ARRIVE reporting standards. Unfortunately, statements made in research articles indicate that the purpose and intent of ARRIVE appear to be widely misunderstood. Out of 39 research articles in Shock since 2010 that made explicit mention of ARRIVE, 32 referred to them as a statement of investigator compliance with ethical care and use standards (e.g., “All the animal experiments were performed in accordance with the ARRIVE guidelines””; “The animal experimental protocol was in accord with the ARRIVE guidelines””; “All animals were maintained in accordance with the ARRIVE Guidelines”). This misconception may be the root cause of the lack of overall reporting in accordance with ARRIVE as noted for other journals (11, 13–16).

The purpose of this article was to present results of a systematic survey of animal-based research articles published in Shock over 5 years subsequent to ARRIVE endorsement. Although the ARRIVE guidelines are intended to be the minimum set of items to be reported in their entirety for each study, this survey was not intended to be a comprehensive assessment of compliance, e.g., (13, 16). Instead, we prioritized reporting of information most relevant to study generalizability, animal welfare, and best statistical practice. Our goals were to quantify reporting in these areas before and after ARRIVE endorsement to determine if ARRIVE endorsement had resulted in any change in reporting standards, and to assess transparency of welfare-related reporting, namely anesthesia, analgesia, euthanasia, and humane endpoints. Studies were not assessed for efficacy of test interventions.

METHODS

We performed a structured review of original animal-based research published in Shock. Study inclusion criteria were full-text research articles reporting primary data and measured effects in live animals. Clinical, cell- or cadaver-based studies, articles describing data compiled from one or more previously reported studies, and nonexperimental reports (letters, editorials, abstracts, reviews) were excluded. Research articles meeting inclusion criteria (n = 698) were listed and numbered. A random sample of 100 articles published between 2014 and 2018 was selected for appraisal, stratified by year to ensure balance. Forty randomly selected articles from 2005 and 2010 were used as a pre-ARRIVE comparator group (n = 20 per year). To detect at least a 50% absolute difference between pre-ARRIVE and post-ARRIVE reporting with α = 0.05 and β = 0.2, a group sample size of n = 20 was the estimated minimum number required for an increase of 10% to 50%, and n = 40 for an increase of 30% to 60%. Stratified random sampling without replacement was performed in SAS proc surveyselect (SAS v 9.4, Cary, NC). Adequacy of the stratified random sample was assessed for model representation by calculating percent of each major group of animals (rats, mice, swine, other) for all studies reporting animal use and compared with percentage representation in the stratified sample.

Assessment domains

We assessed ARRIVE items in four domains: study identifiers and ethical oversight assurances, animal demographics, welfare-related methods and procedures, and key statistical design elements (Table 1).

Table 1 - Description of reporting items assessed in animal-based research articles published in Shock 2005 and 2010 (pre-ARRIVE; n = 40) and 2014 to 2018 (post-ARRIVE; n = 100)
Domain Reporting item ARRIVE item
1. Study identifiers First author Title Reference (volume, issue, pages) N/A
Animal model Disease/injury model Animal identified in title Animal identified in Abstract 1, 2, 8
Ethical oversight assurance Institutional oversight assurance statement Institutional protocol number Assurance of animal care and use in accordance with nationally recognised and specifically identified guidelines 5
2. Animal demographics Species/strain Age Sex reported? (M, F, both, unknown) Weight Vendor, source Caging, housing Acclimation; duration Environment (temperature, photoperiod, humidity) Access to food and water (ad lib vs restricted; diet, food, feeding, diet manufacturer) 8, 9
Group baseline data reported: weight, characteristics, physiological measures, etc. 14
3. Welfare-related procedures
Presurgical food deprivation, duration Anesthetic drugs (agent, dose, route, frequency) Pre-emptive/perioperative analgesia (agent, dose, route, frequency) Post-procedural analgesia (agent, dose, route, frequency) Opioids used? Mortality endpoint (survival study) Humane endpoints Method of euthanasia: inhalant, drug, physical; not specified; not reported 7, 9
4. Study design and statistics
 (a) Time-line diagram or flow chart 6
 (b) Numbers Total exact sample size (n = all animals used in study) Exact group sample sizes (n per group) 10, 12
Sample size justification (e.g., power calculation) Primary outcome identified
 (c) Risk of bias Random allocation to group (selection bias) Method of random allocation (selection bias) 11
Blinding: allocation (selection, performance bias) Blinding: outcome assessment (detection bias) 6
 (d) Outcomes reporting Effect size with measure of precision 16
Specific ARRIVE reporting items are identified by numerals.

Study-specific identifiers enable assessment of model applicability and ethical oversight compliance, and facilitate flagging in database searches. Included in this survey were identification of the disease/injury model and animal model (item 3), institutional and ethical oversight assurances, and guidelines for animal care and use (item 5).

Animal demographic information (species, strain/breed, age, sex, weight, vendor, or source; item 8), and descriptions of housing and husbandry conditions (item 9) and baseline physiology (item 14) are required to assess pre-intervention similarity of groups within a study (internal validity), and generalization of research results (external validity).

ARRIVE does not describe specific welfare-related procedures to be reported. However, the guidelines do state that “precise details of all procedures carried out” are to be given (item 7), and details are to be provided for all “welfare-related assessments and interventions that were carried out prior to, during, or after the experiment” (item 9.c). Descriptions of welfare-related procedures are required for assessment of ethical compliance and method adequacy, validity, and potential for confounding of experimental results. Rigorous documentation of drug-related or other nonexperimental procedures (anesthesia, analgesia, and euthanasia; item 7.a) is essential to evaluate efficacy and potential effect on experimental endpoints (17). Onset, duration, and sufficiency for purpose are affected by choice of agent (specific drug or drug combinations, manufacturer, route of administration, dose, volume, frequency of administration), the animal model (species, strain, age, sex, body weight), and the procedures performed (18). Procedures that are not directly related to the experimental intervention but affect responses (and are therefore potential confounders) include acclimation time (19), presurgical food deprivation (20, 21), and post-surgical prebaseline physiological stabilization (22). Euthanasia methods must be reported (item 7), approved by the institutional oversight committee, and follow internationally-accepted guidelines (23).

Statistical quality is determined by performance of best-practice procedures that ensure results are robust, reliable, and reproducible (11, 24). Detailed standards for statistical reporting have been available elsewhere for many years, e.g., (25–27). ARRIVE stipulates that statistical reporting should include exact sample sizes and numbers analyzed (items 10, 15), sample size justification (item 10.b), clearly defined primary and secondary outcomes (item 12), and any bias minimization procedures performed (randomization, blinding, appropriate controls; items 6.b, 11). Results of an analysis (item 16) are to be reported as effect size (an estimate of the difference between groups) summarized with the appropriate estimator and measure of precision (e.g., mean and confidence intervals). P values are not mentioned in ARRIVE, because they do not measure the size of an effect or the biological importance of the difference between groups and cannot be compared between studies (28).

Data analyses

Reporting frequencies for all items were summarized as counts and percentages for pre- and post-ARRIVE cohorts. Changes in reporting were quantified as absolute proportions and risk ratios (RR) with 95% confidence intervals (CI). The expected value of RR under the null hypothesis of no difference is 1.0; if the 95% CI includes 1.0, then there is insufficient evidence to conclude that the groups are statistically different (29, 30). RRs were not calculated if counts for a cohort item were zero.

RESULTS

The stratified random sample (n = 140) provided adequate representation of animal models, as assessed by comparison with all animal studies published in Shock during the survey period (n = 698). Mice, rats, and swine comprised 49% (221/698), 34% (154/698), and 12% (52/698) respectively of animal models in all studies, compared with 48%, 36%, and 12% for the 140 sample articles. The majority of studies were disease models of sepsis and inflammation (58/140; 41%), and trauma/hemorrhagic shock (28/140; 20%) (Table 2).

Table 2 - Disease models and study identifiers reported in Shock pre-ARRIVE (2005, 2010) and post ARRIVE (2014–2018)
N = 40 N = 100
Pre-ARRIVE n (%) Post- ARRIVE n (%) RR 95% CI
Disease models
 Sepsis, inflammation 18 (45) 43 (43) 0.96 0.63 1.44
 Trauma/hemorrhagic shock 8 (20) 23 (23) 1.15 0.56 2.35
 I/R injury 2 (5) 5 (5) 1.00 0.20 4.94
 Other 9 (23) 25 (25) 1.11 0.57 2.17
Animal models
 Mouse 13 (33) 46 (46) 1.42 0.86 2.32
 Rat 22 (55) 35 (35) 0.64 0.43 0.94
 Swine 3 (12) 14 (14) 1.87 0.57 6.15
 Other 3 (8) 3 (3) 0.40 0.08 1.90
General identification, oversight
 Animal in title? 18 (45) 54 (54) 1.20 0.81 1.77
 In abstract? 40 (100) 98 (98) 0.98 0.95 1.01
 Institutional oversight assurance 32 (80) 98 (98) 1.23 1.05 1.43
 Oversight protocol number 0 (0) 23 (23) . . .
 Guidelines for care 14 (35) 41 (41) 1.17 0.72 1.90
N is total number of articles sampled in each cohort; n is number of studies reporting the specific item. The expected value of the risk ratio RR under the null hypothesis of no difference is 1.0; if the 95% CI includes 1.0, then there is insufficient evidence to conclude that the groups are statistically different. RRs were not calculated if counts for a cohort item were zero.

Identifiers

One-half of surveyed studies (72/140, 51%) explicitly identified the research as animal-based in the title and abstract (Table 2). Three papers did not identify use of an animal model in either title or abstract. Statements of institutional oversight increased from 80% to 98% reporting post-ARRIVE (RR 1.23; 95% 1.05, 1.43). Reporting of specific verifiable protocol numbers was 23% post-ARRIVE, increased from zero pre-ARRIVE. Thirty-four percent (51/140) stated compliance with recognized and verifiable national guidelines for animal care and use, 30% of these referred to the National Research Council Guide for the Care and Use of Laboratory Animals(31). Forty-one percent (59/140) cited vague and unverifiable standards of care (e.g., “NIH guidelines”; “institutional standards for use of animal laboratory animals”), and 25% (32/140) did not mention any.

Animal demographics

There was little change in reporting completeness post-ARRIVE (Table 3). Reporting was highest for reporting strain, primarily because most studies used rodents (123/140; 88%). Breed information was unavailable for 9/24 large animal studies (10%). Most studies reported the sex of animals (127/140; 91%). The majority of studies (106/140, 76%) used only male animals. In 2015, NIH recommended that sex as a biological variable was to be factored into research designs and investigators are supposed to provide strong scientific justification for including only one sex (32). However, the number of studies incorporating both sexes was low, six prior to 2015, and four post-2015. Seventy-six percent (107/140) reported body weights. Group-specific weights were reported for only eight studies. Less than half (49%; 68/140) reported age. Two-thirds (66%; 92/140) reported the vendor or source of animals. Fewer than 25% reported any relevant information on housing and husbandry (caging, enrichment, social environment, bedding, and housing environment (temperature, photoperiod)). Over one-half (78/140, 56%) stated that animals had free access to food and water prior to experimentation. However, descriptions were general (e.g., “food and water ad lib”), and feed specifics (type, manufacturer) were usually not described. Group-specific baseline data for two or more variables was reported for only 27 studies, the majority of which (n = 15) were for large animal models (swine, sheep).

Table 3 - Animal demographic reporting pre-ARRIVE and post-ARRIVE
N = 40 N = 100
pre-ARRIVE n (%) post-ARRIVE n (%) RR 95% CI
Strain 37 (93) 90 97) 0.97 0.87 1.09
Age 15 (38) 53 (53) 1.43 0.90 2.28
Sex reported 36 (90) 91 (91) 1.05 0.92 1.20
M only 28 (70) 77 (77) 1.08 0.87 1.34
F only 3 (8) 9 (9) 1.20 0.34 4.21
Both 5 (13) 5 (5) 0.40 0.12 1.31
Not reported 4 (10) 9 (9) 0.90 0.29 2.76
Weight 33 (83) 75 (75) 0.91 0.76 1.09
Baseline group data 4 (10) 27 (27) 2.70 1.01 7.22
Vendor, source 29 (73) 65 (65) 0.90 2.00 0.73
Housing; caging 9 (23) 13 (13) 0.58 0.27 1.24
Environmental
 Photoperiod only 5 (13) 17 (17) 1.36 0.54 3.44
 Temperature only 1 (3) 1 (1) 0.40 0.03 6.24
 Both 7 (18) 24 (24) 1.37 0.64 2.93
 None reported 28 (70) 59 (57) 0.84 0.65 1.09
 Access to food and water 23 (58) 57 (57) 0.99 0.72 1.36
 Acclimation 15 (38) 24 (24) 0.64 0.38 1.09
N is total number of articles sampled in each cohort; n is number of studies reporting the specific item. The expected value of the risk ratio RR under the null hypothesis of no difference is 1.0. If the 95% CI includes 1.0, then there is insufficient evidence to conclude that the groups are statistically different. RRs were not calculated if counts for a cohort item were zero

Welfare-related reporting

Food deprivation

Presurgical food deprivation is usually routine for large animal studies; however, 7 of 20 large animal studies made no mention of fasting, and the remainder reported that animals were fasted either “overnight” (n = 5) or between 12 and 24 h before experimentation (n = 8) (Table 4). Presurgical fasts of 12 to 24 h were reported for 22 rodent studies.

Table 4 - Welfare-related reporting: presurgical food deprivation, recovery procedures, humane endpoints, anesthesia, analgesia, euthanasia
Pre-ARRIVE Post-ARRIVE
n (%) n (%) RR 95% CI
Welfare-related procedures
 Conscious/recovery procedures 29 (73) 61 (61) 0.84 0.66 1.08
 Presurgical food deprivation 12 (28) 24 (24) 0.87 0.47 1.61
 Fasted rodents 8 (20) 14 (14) 0.70 0.32 1.54
 Mortality endpoint 13 (33) 36 (36) 1.11 0.66 1.86
 Humane endpoints specified 2 (5) 6 (6) 1.20 0.25 5.70
Postprocedural monitoring period
 < 24 h 13 (33) 28 (28) 0.86 0.50 1.49
 1–3 d 12 (30) 32 (32) 1.07 0.61 1.85
 4–7 d 7 (18) 22 (22) 1.26 0.58 2.71
 > 1 wk 3 (8) 7 (7) 0.93 0.25 3.43
 Not specified 1 (3) 7 (7) 2.80 0.36 22.03
Anesthesia: injectables
 Studies reporting use 21 (53) 56 (56) 1.07 0.76 1.50
 Route only reported 2/21 (5) 4/56 (4) 0.75 0.15 3.80
 Concentration only reported 3/21 (8) 13/56 (13) 1.63 0.51 5.14
 Both reported 16/21 (40) 35/56 (35) 0.82 0.60 1.12
Anesthesia: inhalants
 Studies reporting use 14 (35) 46 (46) 1.31 0.82 2.11
 Concentration only reported 5/14 (36) 14/46 (30) 0.85 0.37 1.95
 Concentration and carrier gas reported 2/14 (14) 17/46 (37) 2.59 0.68 9.85
 Neither reported 7/14 (50) 16/46 (35) 0.70 0.36 1.34
 Other (flow rates) 2/14 (14) 3/46 (7) 0.46 0.08 2.46
Drug classes
 Barbiturates 12 (30) 25 (25) 0.83 0.47 1.49
 Ketamine, ketamine-xylazine 4 (10) 17 (17) 1.70 0.61 4.74
 Isoflurane, sevoflurane 9 (25) 44 (44) 1.96 1.06 3.62
 Other 9 (25) 16 (16) 0.71 0.34 1.48
 Not reported 2 (5) 7 (7) 1.40 0.30 6.45
Analgesia
 Studies reporting use 9 (23) 55 (55) 2.44 1.34 4.46
 Preemptive 5 (13) 23 (23) 1.84 0.75 4.50
 Postintervention 5 (13) 29 (29) 2.32 0.97 5.57
 Opioids 7 (18) 28 (28) 1.60 0.76 3.36
Euthanasia
 Anesthetic overdose 10 (25) 18 (18) 0.72 0.36 1.42
 Anesthetic plus drug or physical method 12 (30) 34 (34) 1.13 0.66 1.96
 Physical method only 4 (10) 10 (10) 1.00 0.33 3.00
 Not specified 4 (10) 13 (13) 1.30 0.45 3.75
 Not reported 12 (30) 33 (33) 1.10 0.63 1.91
N is total number of articles sampled in each cohort; n is number of studies reporting the specific item. The expected value of the risk ratio RR under the null hypothesis of no difference is 1.0; if the 95% CI includes 1.0, then there is insufficient evidence to conclude that the groups are statistically different. RRs were not calculated if counts for a cohort item were zero.

Postoperative monitoring, humane endpoints

Recovery experiments (when animals were brought out of anesthesia for further experimentation or monitoring) were reported by 64% (90/140) of studies. Post-procedural monitoring occurred over hours (46%), days (53/140, 38%), or weeks (30/140, 21%). Mortality endpoints (i.e., survival studies) were reported by 36%. However, only eight studies reported unambiguous humane endpoint criteria. No study reported side effects or unexpected deaths.

Anesthesia, analgesia

Use of injectable agents (e.g., barbiturates, ketamine-xylazine) was reported by 49% (69/140) of studies (Table 4). Of these, 50 (72%) reported both route and concentration. Thirteen reports described use of anesthetic agents prohibited or discouraged by US institutions (e.g., IP chloral hydrate (33), ketamine alone, ether). The proportion of articles reporting use of inhalant anesthetic agents (isoflurane, sevoflurane) doubled post-ARRIVE (from 25 to 44%; RR 1.96, 95% CI [1.06, 3.62]). Agent concentrations and carrier gas information were reported by 20/59 (34%), and 7/59 (12%) reported flow rates. Information other than agent was not reported for 19/59 (32%). Reported analgesia use increased approximately 60% post-ARRIVE. Use of opioids was reported by 36/140 (26%), pre-emptive analgesia by 32/140 (23%), and postintervention analgesia by 36/140 (26%), with reported use more than doubled post-ARRIVE (RR 2.40, 95% CI [1.00, 5.75]).

Euthanasia

One-third (41/140, 29%) of studies did not report whether or not euthanasia was performed, and 19% (26/140) made a general statement without specifying a specific method (e.g., “Animals were sacrificed”; “tissue harvest under anesthesia”; “Animals were anesthetized and killed”). Reported methods included anesthetic overdose (28/140; 20%), overdose of either a single agent or combination of agents coupled with physical methods (exsanguination, thoracotomy, cervical dislocation, decapitation; 45/140, 33%), or physical method only (11/140; 8%). Three papers reported cervical dislocation of unanesthetized adult rats, a method prohibited by US institutions.

Statistical reporting

Twenty-two percent (22/100) of studies post-ARRIVE provided an experimental work flow diagram or experimental timeline, compared with 10% (4/40) pre-ARRIVE (Table 5). All studies were hypothesis tests of efficacy, and used inferential statistics. Nearly all (94%; 132/140) were positive, claiming statistically significant benefit of test interventions over controls. Only 10 studies post-ARRIVE reported a clearly defined primary outcome. Only one study reported effect sizes with 95% confidence intervals. No study provided information on statistical context or the biologically relevant difference to be detected.

Table 5 - Study design and statistics reporting
Pre-ARRIVE Post-ARRIVE
n (%) n (%) RR 95% CI
Experimental timeline, workflow 4 (10) 22 (22) 2.20 0.81 5.98
Animal numbers
 Total sample size 10 (25) 38 (38) 1.52 0.84 2.75
 Sample size per group 19 (54) 54 (54) 1.14 0.78 1.65
 Power calculations, sample size justification 0 (0) 10 (10) . . .
 Defined primary outcomes 0 (0) 11 (11) . . .
Bias minimization methods
 Randomization: allocation 13 (33) 43 (43) 1.32 0.80 2.18
 Randomization: method specified 0 (0) 7 (7) . . .
 Blinding: allocation concealment 1 (2.5) 3 (3) 1.20 0.13 11.20
 Blinding: outcome assessment 11 (28) 23 (23) 0.84 0.45 1.55
 Randomized and blinded allocation 1 (3) 0 (0) . . .
 Randomized allocation, blinded assessment 4 (10) 4 (4) 0.40 0.11 1.52
 Controls 40 (100) 97 (97) 0.97 0.94 1.00
 Sham 16 (40) 65 (65) 1.63 1.08 2.44
N is total number of articles sampled in each cohort; n is number of studies reporting the specific item. The expected value of the risk ratio RR under the null hypothesis of no difference is 1.0; if the 95% CI includes 1.0, then there is insufficient evidence to conclude that the groups are statistically different. RRs were not calculated if counts for a cohort item were zero.

Animal numbers

Sample size justification with power calculations and a defined primary outcome was reported for 10% (10/100) studies post-ARRIVE, compared with 0% pre-ARRIVE. Exact numbers of animals could not be determined for the majority of studies (91/140, 65%) because sample size information was either missing or inadequately reported (e.g., sample sizes per group not reported, group sample sizes presented as a range, not all groups reported, numbers differed between Methods and Results).

Randomization, blinding, controls

Thirty-eight percent (53/140) studies reported performing randomization, although only seven studies described how randomization was performed. Eight studies reported adding control groups after “randomization” of experimental interventions. Four studies reported blinding of personnel to group allocation, 17% (24/140) reporting blinding for outcome assessment (usually personnel assigned to read and interpret histological specimens), and only one study reported both randomized and blinded treatment allocation. No study provided information on how blinding was achieved. Nearly all studies (98%) reported one or more control groups. Over half (81/140; 58%) reported using a “sham” group (non-surgery and/or nonintervention controls), with the proportion of studies reporting sham use increasing after 2014 (RR 1.63, 95% CI [1.08, 2.44]).

DISCUSSION

Nearly a decade after endorsement of ARRIVE by the Shock Society, reporting standards in Shock have shown minimal improvement. This situation is, unfortunately, not unique to Shock. Similar problems and deficiencies have been described for animal-based research in general (12, 16), as well as specific research focus areas, e.g., (2, 34–37). Most investigators understand why it is necessary to provide detailed descriptions of the biochemical, histological, molecular, and other miscellaneous techniques used in their study. It is certainly acceptable for authors to provide references to previously published research if space limitations preclude detailed description, and if methods in the original publication are completely and transparently described, and freely available to the scientific community. However, even if experimental techniques are the same, new studies use different animals and the analysis methods must reflect the goals and the design of the specific study. Therefore, descriptions of animal demographics, surgical and welfare-related procedures, and statistical methodology must be reported in study-specific detail. This information is required to assess method adequacy and validity of the results, and to ensure specific experiments were performed ethically. These reporting principles also apply to in vitro experiments on tissues harvested from animals killed for purpose (38). In this survey, nearly all studies omitted mention of one or more key items required for assessing reproducibility and validity, and most items from all four domains were omitted by studies using animals killed primarily for tissue harvest.

The principal methods used to ensure reproducibility are randomization, concealed allocation (blinding) of treatment assignments, blinding of outcome assessments, and sample size justification (11). Randomization and blinding reduce bias from undetected systematic differences between treatment groups, or bias from other sources (12, 13, 39). Randomization in particular is required for valid statistical tests (40). Appropriate sample sizes are required to avoid wasting animals in noninformative experiments (11). Therefore, it is of concern that relatively few studies reported these details. Even if such methods were not performed, discussion and justification of the rationale for omission are still required to enable objective assessment of study quality and rigor.

Because of the emphasis on bias minimization methods as reproducibility criteria, less attention has been given to other items such as animal demographics, potential confounders, and welfare-related procedures. These items are important for similar reasons: to allow independent assessment of factors contributing to variation in response and generalizability of research findings (11, 24).

Descriptions of experimental animals and baseline characteristics are analogous to the reporting of patient demographics for a human clinical trial (41). Species differ in size, physiological function and metabolism, and therefore will differ in response to ischemia, hypoxia, pharmacological interventions, and other physiological injuries. The necessity for accurate reporting should be evident. Even within the same species or strain, age, sex, and individual differences can contribute to profoundly different hemodynamic, neurological, and immunological responses (38, 42). Of note is the research gap identified by this survey, namely the preponderance of single-sex male-only studies, with no change in research practice since the 2015 NIH recommendations (32).

The surgical or pre-intervention environment can be a major influence on animal response, and can confound interpretation of experimental outcomes. Relevant information includes methods related to presurgical induction, anesthesia, ventilation, and oxygenation, thermoregulatory control, pharmacological interventions, and the injury- or disease-induction process. Variation in procedures between labs may contribute to contradictions and inconsistencies observed for results in the literature (17). The Utstein guidelines for animal-based CPR research (43) were among the earliest to recommend standardized reporting for these items.

If performed, presurgical fasting must be scientifically justified and reported accurately.

Overly-prolonged food deprivation results in weight loss, metabolic, hormonal, and cardiovascular disruption, hypothermia, hypoglycemia, and profound distress, especially in small mammals (rodents, lagomorphs) (21, 44). Effects of food restriction also vary with age and size, as adults usually can tolerate more severe restriction than young growing animals. Unless scientifically justifiable, fasting of small mammals is strongly discouraged. Small mammals cannot vomit, so preoperative fasts to prevent reflux are unnecessary. Even in large animals such as swine, prolonged fasts are usually unnecessary; gastric passage times are relatively rapid, and preoperative fasts of 6 to 8 h can be sufficient to empty the stomach and prevent reflux (45). If fasting is required to clear digestive contents for the purpose of the study, then fast duration must be proportionate to animal size and gut clearance rates, and scientifically justified.

Of concern was the incomplete reporting of methods related to anesthesia, analgesia, humane endpoints, and euthanasia. Animals that are otherwise similar can respond quite differently to different anesthetic protocols, pharmacological interventions, and methods and timing of euthanasia, so relevant details must be reported to allow assessment of study applicability and validity (18, 38, 46, 47). Pre-emptive, concurrent, and follow-up use of appropriate pain relief medications (such as opioids and/or non-steroidal anti-inflammatories) are required for major and/or multiple survival surgeries, experiments with mortality as an endpoint, and prolonged postintervention monitoring periods (48), unless there is strong, well-documented scientific justification against their use. Mortality endpoints and experiments resulting in substantial adverse effects on the animals involved must always be justified, and specific descriptions provided of what was done to minimize pain and distress (18). It is encouraging that reported analgesia use has approximately doubled over the past 5 years. However, descriptions of anesthesia and analgesia frequently omitted basic information on specific agents or methods, justification for use, and specific dose, route, concentration, administration schedules, and specialist equipment (item 7; (18). Three studies reported using IP chloral hydrate for rodent anesthesia, although its use is inhumane and without scientific justification (23, 33). Descriptions of postoperative and palliative care and humane endpoints (49) were frequently missing or inadequately reported, even though many studies reported monitoring experimental subjects for hours to days postinjury and before euthanasia. Euthanasia methods were not reported in over one-third of surveyed studies, and several reported using methods of euthanasia prohibited by the American Veterinary Medical Association (23).

CONCLUSIONS

It is clear that completeness and accuracy of reporting of animal-based research could be improved. The quality and usability of research evidence is closely allied with the quality and transparency of reporting (12). It is incumbent on all investigators to conduct well-designed experiments, analyze data with appropriate statistical methods, and report methods and results in sufficient detail so that the work is reproducible and replicable (50, 51). Revised and updated versions of the ARRIVE guidelines and companion Explanation and Elaboration document are scheduled for release this year, and are designed to facilitate increased use of best practice reproducibility methods by clarifying key concepts and providing examples of good reporting. Greater rigor in study conduct and reporting will lead to overall improvement in experimental quality, and single out Shock and the Shock Society as leaders and innovators in shock-allied basic research.

REFERENCES

1. Henderson VC, Kimmelman J, Fergusson D, Grimshaw JM, Hackam DG. Threats to validity in the design and conduct of preclinical efficacy studies: a systematic review of guidelines for in vivo animal experiments. PLoS Med 2013; 10:e1001489.
2. Ramirez FD, Motazedian P, Jung RG, Di Santo P, MacDonald ZD, Moreland R, Simard T, Clancy AA, Russo JJ, Welch VA, et al. Methodological rigor in preclinical cardiovascular studies: targets to enhance reproducibility and promote research translation. Circ Res 2017; 120 (12):1916–1926.
3. Denayer T, Stöhr T, Van Roy M. Animal models in translational medicine: validation and prediction. New Horizons Transl Med 2014; 2:5–11.
4. Reynolds PS, Wall P, van Griensven M, McConnell K, Lang C, Buchman T. Shock supports the use of animal research reporting guidelines. Shock 2012; 38 (1):1–3.
5. Osuchowski MF, Remick DG, Lederer JA, Lang CH, Aasen AO, Aibiki M, Azevedo LC, Bahrami S, Boros M, Cooney R, et al. Abandon the mouse research ship? Not just yet!. Shock 2014; 41 (6):463–475.
6. Osuchowski MF, Thiemermann C, Remick DG. Sepsis-3 on the block: what does it mean for pre-clinical sepsis modeling? Shock 2017; 47 (5):658–660.
7. Tremoleda JL, Watts SA, Reynolds PS, Thiemermann C, Brohi K. Modeling acute traumatic hemorrhagic shock injury: challenges and guidelines for preclinical studies. Shock 2017; 48 (6):610–623.
8. Zingarelli B, Coopersmith CM, Drechsler S, Efron P, Marshall JC, Moldawer LL, Wiersinga WJ, Xiao X, Osuchowski MF, Thiemermann C. Part I: Minimum Quality Threshold in Preclinical Sepsis Studies (MQTiPSS) for study design and humane modeling endpoints. Shock 2019; 51 (1):10–22.
9. Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research. PLoS Biol 2010; 8 (6):e1000412.
10. Laman JD, Koisrta SM, Clausen BE. Reproducibility issues: avoiding pitfalls in animal inflammation models. Methods Mol Biol 2017; 1559:1–17.
11. Landis SC, Amara SG, Asadullah K, Austin CP, Blumenstein R, Bradley EW, Crystal RG, Darnell RB, Ferrante RJ, Fillit H, et al. A call for transparent reporting to optimize the predictive value of preclinical research. Nature 2012; 490:187–191.
12. Macleod MR, Lawson McLean A, Kyriakopoulou A, Serghiou S, de Wilde A, Sherratt N, Hirst T, Hemblade R, Bahor Z, Nunes-Fonseca C, et al. Risk of bias in reports of in vivo research: a focus for improvement. PLoS Biol 2015; 13 (11):e1002301.
13. Avey MT, Moher D, Sullivan KJ, Fergusson D, Griffin G, Grimshaw JM, Hutton B, Lalu MM, Macleod MR, Marshall J, et al. The devil is in the details: Incomplete reporting in preclinical animal research. PLoS One 2016; 11 (11):e0166733.
14. Hair K, Macleod MR, Sena E. IICARus Collaboration: A randomised controlled trial of an Intervention to Improve Compliance with the ARRIVE guidelines (IICARus). Res Integr Peer Rev 2019; 4:12.
15. Baker D, Lidster K, Sottomayor A. Amor S: two years later: journals are not yet enforcing the ARRIVE guidelines on reporting standards for pre-clinical animal studies. PLoS Biol 2014; 12:e1001756.
16. Leung V, Rousseau-Blass F, Beauchamp G, Pang DSJ. ARRIVE has not ARRIVEd: Support for the ARRIVE (Animal Research: Reporting of In Vivo Experiments) guidelines does not improve the reporting quality of papers in animal welfare, analgesia or anesthesia. PLoS One 2018; 13 (5):e0197882.
17. Haensel JX, Spain A, Martin C. A systematic review of physiological methods in rodent pharmacological MRI studies. Psychopharmacology (Berl) 2015; 232:489–499.
18. Carbone L, Austin J. Pain and laboratory animals: publication practices for better data reproducibility and better animal welfare. PLoS One 2016; 11 (5):e0155001.
19. Obernier JA, Baldwin RL. Establishing an appropriate period of acclimatization following transportation of laboratory animals. ILAR J 2006; 47 (4):364–369.
20. Bradbury AG, Clutton RE. Review of practices reported for preoperative food and water restriction of laboratory pigs (Sus scrofa). J Am Assoc Lab Anim Sci 2016; 55 (1):35–40.
21. Jensen TL, Kiersgaard MK, Sørensen DB, Mikkelsen LF. Fasting of mice: a review. Lab Anim 2013; 47 (4):225–240.
22. Reynolds PS, Garvan CW. Gap analysis of swine-based hemostasis research: “Houses of brick or mansions of straw? Mil Med 2020; 185 (S1):88–95.
23. Leary S, Underwood W, Anthony R, Cartner S, Corey D, Grandin T, Greenacre C, Gwaltney-Brant S, McCrackin MA, Meyer R, et al. AVMA Guidelines for the Euthanasia of Animals: 2013 Edition. Schaumburg, IL: American Veterinary Medical Association; 2013.
24. National Institutes of Health: Principles and Guidelines for Reporting Preclinical Research. Available at: www.nih.gov/research-training/rigor-reproducibility/principles-guidelinesreporting-preclinical-research (2014). Accessed October 19, 2019.
25. Altman DG, Gore SM, Gardner MJ, Pocock SJ. Statistical guidelines for contributors to medical journals. BMJ 1983; 286:1489–1493.
26. Curran-Everett D, Benos DJ. American Physiological Society: Guidelines for reporting statistics in journals published by the American Physiological Society. Am J Physiol Endocrinol Metab 2004; 287:E189–E191.
27. Lang TA, Altman DG: Basic statistical reporting for articles published in clinical medical journals: the SAMPL Guidelines. Available at: https://wwwequator-networkorg/reporting-guidelines/sampl/ (2013). Accessed January 11, 2019.
28. Wasserstein RL, Lazar NA. The ASA's statement on p-values: context, process, and purpose. Amer Stat 2016; 70 (2):129–133.
29. Hopewell S, Dutton S, Yu L-M, Chan AW, Altman DG. The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. BMJ 2010; 340:c723.
30. Deeks JJ, Altman DG, Bradburn MJ. Egger M, Smith GD, Altman DG. Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. Systematic Reviews in Health Care: Meta-Analysis in Context. BMJ Books: London, UK; 2001; 285-310.
31. National Research Council: Guide for the Care and Use of Laboratory Animals. National Academies Press, Washington DC, 2011:220.
32. Hewitt JA, Brown LL, Murphy SJ, Grieder F, Silberberg SD. Accelerating biomedical discoveries through rigor and transparency. ILAR J 2017; 58 (1):115–128.
33. Baxter MG, Murphy KL, Taylor PM, Wolfensohn SE. Chloral hydrate is not acceptable for anesthesia or euthanasia of small animals. Anesthesiology 2009; 111 (1):209.
34. Bara M, Joffe AR. The methodological quality of animal research in critical care: the public face of science. Ann Intensive Care 2014; 4:26.
35. Merkow JS, Hoerauf JM, Moss AF, Brainard J, Mayes LM, Fernandez-Bustamante A, Mikulich-Gilbertson SK, Bartels K. Animal experimental research design in critical care. BMC Med Res Methodol 2018; 18 (1):71.
36. Minnerup J, Wersching H, Diederich K, Schilling M, Ringelstein EB, Wellmann J, Schäbitz WR. Methodological quality of preclinical stroke studies is not required for publication in high-impact journals. J Cereb Blood Flow Metab 2010; 30 (9):1619–1624.
37. Vesterinen HM, Egan K, Deister A, Schlattmann P, Macleod MR, Dirnagl U. Systematic survey of the design, statistical analysis, and reporting of studies published in the 2008 volume of the Journal of Cerebral Blood Flow and Metabolism. J Cereb Blood Flow Metab 2011; 31 (4):1064–1072.
38. McGrath J, Drummond G, McLachlan E, Kilkenny C, Wainwright C. Guidelines for reporting experiments involving animals: the ARRIVE guidelines. Br J Pharmacol 2010; 160:1573–1576.
39. Kilkenny C, Parsons N, Kadyszewski E, Festing MF, Cuthill IC, Fry D, Hutton J, Altman DG. Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PLoS One 2009; 4 (11):e0007824.
40. Altman DG, Bland JM. Treatment allocation in controlled trials: why randomise? BMJ 1999; 318:1209.
41. Sifers SK, Puddy RW, Warren JS, Roberts MC. Reporting of demographics, methodology, and ethical procedures in journals in pediatric and child psychology. J Pediatr Psychol 2002; 27 (1):19–25.
42. Wenzel V, Padosch SA, Voelckel WG, Idris AH, Krismer AC, Bettschart-Wolfensberger R, Lindner KH. Survey of effects of anesthesia protocols on hemodynamic variables in porcine cardiopulmonary resuscitation laboratory models before induction of cardiac arrest. Comp Med 2000; 50 (6):644–648.
43. Idris AH, Becker LB, Ornato JP, Hedges JR, Bircher NG, Chandra NC, Cummins RO, Dick W, Ehmeyer U, Halperin HR, et al. Utstein-Style Guidelines for Uniform Reporting of Laboratory CPR Research: A Statement for Health Care Professionals From a Task Force of the American Heart Association, the American College of Emergency Physicians, the American College of Cardiology, the European Resuscitation Council, the Heart and Stroke Foundation of Canada, the Institute of Critical Care Medicine, the Safar Center for Resuscitation Research, and the Society for Academic Emergency Medicine. Ann Emerg Med 1996; 28 (5):527–541.
44. Norecopa: Fasting in Rodents. In A. Smith (ed): Vol. 3R Guide/10807. Oslo, Norway: Norecopa Veterinaerinstituttet, 2009, p. 11.
45. Swindle MM, Sistino JJ: Anesthesia, analgesia, and perioperative care. In M. M. Swindle and A. C. Smith (eds): Swine in the Laboratory: Surgery, Anesthesia, Imaging, and Experimental Techniques. Boca Raton, FL: Chapman and Hall/CRC, 2015, p. 593.
46. Al-Mousawi AM, Kulp GA, Branski LK, Kraft R, Mecott GA, Williams FN, Herndon DN. Jeschke MG: Impact of anesthesia, analgesia, and euthanasia technique on the inflammatory cytokine profile in a rodent model of severe burn injury. Shock 2010; 34 (3):261–268.
47. Rice ASC, Cimino-Brown D, Eisenach JC, Kontinen VK, Lacroix-Fralish ML, Machin I, Mogil JS, Stöhr T. on behalf of the Preclinical Pain Consortium. Animal models and the prediction of efficacy in clinical trials of analgesic drugs: A critical appraisal and call for uniform reporting standards. Pain 2009; 139:243–247.
48. Foley P, Hitt N, Leblanc M, McCrackin M, Sexton S, Villano J. ACLAM position statement on rodent surgery. J Am Assoc Lab Anim Sci 2016; 55 (6):822–823.
49. Nemzek JA, Xiao HY, Minard AE, Bolgos GL, Remick DG. Humane endpoints in shock research. Shock 2004; 21 (1):17–25.
50. Altman DG. Statistics and ethics in medical research. Misuse of statistics is unethical. BMJ 1980; 281 (6249):1182–1184.
51. MacCallum CJ. Reporting animal studies: Good science and a duty of care. PLoS Biol 2010; 8 (6):e1000413.
Keywords:

Animal welfare; ARRIVE (Animal Research: Reporting of In Vivo Experiments); reproducibility

Copyright © 2020 by the Shock Society