Journal Logo

Observational Studies in Orthopaedic Surgery: The STROBE Statement as a Tool for Transparent Reporting

Sheffler, Lindsey C., MD, MAS1; Yoo, Brad, MD2; Bhandari, Mohit, MD, MSc, FRCSC3; Ferguson, Tania, MD2

doi: 10.2106/JBJS.L.00484
The Orthopaedic Forum
Free
Disclosures

1Department of Orthopaedic Surgery, University of California, San Francisco, 500 Parnassus Avenue, MU 320W, San Francisco, CA 94143-0728. E-mail address for L.C. Sheffler: shefflerl@orthosurg.ucsf.edu

2University of California, Davis Medical Center, 4860 Y Street, ACC 3800, Sacramento, CA 95817

3293 Wellington Street North, Suite 110, Hamilton, ON L8L 8E7, Canada

Evidence-based medicine in orthopaedic surgery comprises predominantly observational studies. While the gold standard of study methodology is considered to be randomized controlled trials (RCTs), observational studies provide valuable information regarding disease prevalence and etiology, rare outcomes, and adverse treatment effects. Orthopaedic surgeons care for many diseases and injuries that are rare and will likely never be the subject of an RCT. Given the bias to which observational studies are prone, however, transparent reporting is imperative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement is a checklist of items that can help clinician-scientists to improve the transparency with which observational studies are reported. We offer the following guidelines and examples for how the STROBE statement can be applied to reporting observational studies in orthopaedic surgery.

Back to Top | Article Outline

Observational Studies

Observational studies inform clinicians about disease etiology, natural history, prognostic factors, and treatment effectiveness1,2. The most common observational study designs include cohort, case-control, and cross-sectional studies. In a cohort study, subjects are divided into two groups, or cohorts: those with an exposure of interest and those without. The groups are then followed prospectively and are observed for an outcome of interest. In a case-control study, subjects who have experienced an outcome (cases) are matched with subjects who have not experienced an outcome (controls). The two groups are then studied retrospectively to determine a causal relationship between unmatched risk factors and the outcome of interest. In a cross-sectional study, each subject in a population is evaluated at a single point in time, often to calculate the prevalence of disease or to establish an association between risk factors and outcome.

Observational studies, specifically, case series, predominate the surgical literature in both general surgery (46%) and orthopaedic surgery (88%)2-4. One reason for the high prevalence of observational studies is that, unlike in other fields of medicine, many questions in surgical subspecialties cannot feasibly or ethically be answered with RCTs. A candidate for surgery may not wish to be randomized to operative or nonoperative care, and a surgeon cannot feasibly be blinded to an operative procedure. Furthermore, surgical experience may affect the outcome of a new procedure and randomizing patients to surgeons with varying levels of experience may pose an ethical dilemma. While observational studies are more suitable than RCTs for studying rare outcomes, adverse treatment effects, and disease etiology, the clinician-scientist must recognize the limitations and biases of nonrandomized study designs.

Observational studies are prone to bias, most notably confounding, sampling bias, and recall bias (Table I)5. Bias is any systematic error that causes an incorrect estimate of the association between an exposure and an outcome of interest. Unlike RCTs, observational studies do not benefit from the process of randomization, which balances confounders between groups. In case-control studies, unevenly balanced groups also may be the result of inherent differences between each group due to sampling bias. Cohort study designs are also susceptible to sampling bias in cases in which the sample does not represent the population from which it was drawn or the population at large. Retrospective observational studies are further prone to recall bias, which increases the potential for incomplete or biased data collection. Checklists that provide recommendations for how to report observational studies ensure that confounding, sampling bias, and recall bias are clearly recognized so that the strengths of nonrandomized studies can be appreciated.

TABLE I - Confounding, Sampling Bias, Recall Bias5
Confounding factors: variables associated with both the exposure and outcome of interest
 Effect: May misrepresent the relationship between exposure and outcome
 Example: A clinician-scientist aims to study the effects of alcohol use (exposure) on time to fracture-healing (outcome). She includes fifty individuals who drink an average of more than two alcoholic beverages a day (cases) and fifty individuals who do not drink alcohol (controls). She finds that subjects who drink alcohol have an increased time to fracture-healing than the controls do. However, the subjects who drank alcohol were also more likely to smoke cigarettes. Smoking, which has been reported to increase fracture-healing time, is a confounding factor associated with both the exposure (alcohol use) and the outcome of interest (fracture-healing).
Sampling bias: systematic error due to selection of study participants that do not represent the population from which they were drawn or the population at large
 Effect: Groups may have inherent differences due to how subjects were recruited; results may not be applicable to other populations
 Example 1: A clinician-scientist aims to study the association between activity level and osteoarthritis. He recruits high-activity-level subjects (cases) from a local fitness club and low-activity-level subjects (controls) by advertising in the local newspaper. During data analysis, he learns that the high-activity-level group was, on average, younger in age and had fewer comorbidities than the low-activity-level group. These inherent differences between groups may affect the results of his study.
 Example 2: An investigator reports the incidence of reverse shoulder arthroplasty with use of a database from one academic hospital. To estimate the incidence of reverse shoulder arthroplasty in the United States, a multicenter database with information from both private and academic hospitals in rural and urban locations throughout the country would more accurately represent the population at large.
Recall bias: systematic error due to differences in the accuracy of or extent to which subjects recall a previous event
 Effect: May overestimate or underestimate the effect size of a risk factor
 Example: An investigator is interested in studying potential risk factors of osteosarcoma. He recruits 100 subjects with osteosarcoma and asks them to complete a detailed questionnaire about prior health habits. Some of the subjects cannot recall of the details of their prior health habits; others provide biased answers because they are worried that their actions were responsible for the development of osteosarcoma.

Back to Top | Article Outline

Checklist for Observational Studies: The STROBE Statement

In order to practice evidence-based orthopaedic surgery, a reader must critically appraise the published research, understand the strengths and weaknesses of the study, and determine if the reported results and conclusions apply to other clinical situations. The research paper must clearly describe the research question, the study population, how the study was designed to answer the question, and whether the research methodology and analysis were appropriate to reach the author’s conclusions. As described above, observational studies are prone to bias, and it is often challenging for readers to understand if a study population (and therefore the study’s conclusion) is truly representative of clinical practice. Authors of observational studies not only must clearly report their research design and conduct but also must examine and disclose potential factors that may threaten the validity and applicability of their findings.

The STROBE initiative was established in 2004 in an effort to improve the quality of reporting of observational research (Table II). A multinational group of methodologists, researchers, and journal editors met to develop recommendations intended to help investigators write “a clear presentation of what was planned, done, and found in an observational study.”6 The STROBE initiative further aimed to assist reviewers and editors when evaluating such studies for publication and to guide readers when interpreting and applying a published study’s results and conclusions. The resulting checklist has been adopted by many of the current orthopaedic journals, including The Journal of Bone and Joint Surgery, which included the STROBE statement in its instructions to authors in 20077.

TABLE II - The STROBE Statement—Checklist of Items That Should be Included in Reports of Observational Studies
Item No Recommendation
Title and abstract 1 (a) Indicate the study’s design with a commonly used term in the title or the abstract
(b) Provide in the abstract an informative and balanced summary of what was done and what was found
Introduction
Background/rationale 2 Explain the scientific background and rationale for the investigation being reported
Objectives 3 State specific objectives, including any prespecified hypotheses
Methods
Study design 4 Present key elements of study design early in the paper
Setting 5 Describe the setting, locations, and relevant dates, including periods of recruitment, exposure, follow-up, and data collection
Participants 6 (a) Cohort study—Give the eligibility criteria, and the sources and methods of selection of participants. Describe methods of follow-upCase-control study—Give the eligibility criteria, and the sources and methods of case ascertainment and control selection. Give the rationale for the choice of cases and controlsCross-sectional study—Give the eligibility criteria, and the sources and methods of selection of participants
(b) Cohort study—For matched studies, give matching criteria and number of exposed and unexposedCase-control study—For matched studies, give matching criteria and the number of controls per case
Variables 7 Clearly define all outcomes, exposures, predictors, potential confounders, and effect modifiers. Give diagnostic criteria, if applicable
Data sources/measurement 8* For each variable of interest, give sources of data and details of methods of assessment (measurement). Describe comparability of assessment methods if there is more than one group
Bias 9 Describe any efforts to address potential sources of bias
Study size 10 Explain how the study size was arrived at
Quantitative variables 11 Explain how quantitative variables were handled in the analyses. If applicable, describe which groupings were chosen and why
Statistical methods 12 (a) Describe all statistical methods, including those used to control for confounding
(b) Describe any methods used to examine subgroups and interactions
(c) Explain how missing data were addressed
(d) Cohort study—If applicable, explain how loss to follow-up was addressedCase-control study—If applicable, explain how matching of cases and controls was addressedCross-sectional study—If applicable, describe analytical methods taking account of sampling strategy
(e) Describe any sensitivity analyses
Results
Participants 13* (a) Report numbers of individuals at each stage of study—eg numbers potentially eligible, examined for eligibility, confirmed eligible, included in the study, completing follow-up, and analyzed
(b) Give reasons for non-participation at each stage
(c) Consider use of a flow diagram
Descriptive data 14* (a) Give characteristics of study participants (eg demographic, clinical, social) and information on exposures and potential confounders
(b) Indicate number of participants with missing data for each variable of interest
(c) Cohort study—Summarize follow-up time (eg average and total amount)
Outcome data 15* Cohort study—Report numbers of outcome events or summary measures over time
Case-control study—Report numbers in each exposure category, or summary measures of exposure
Cross-sectional study—Report numbers of outcome events or summary measures
Main results 16 (a) Give unadjusted estimates and, if applicable, confounder-adjusted estimates and their precision (eg, 95% confidence interval). Make clear which confounders were adjusted for and why they were included
(b) Report category boundaries when continuous variables were categorized
(c) If relevant, consider translating estimates of relative risk into absolute risk for a meaningful time period
Other analyses 17 Report other analyses done—eg analyses of subgroups and interactions, and sensitivity analyses
Discussion
Key results 18 Summarize key results with reference to study objectives
Limitations 19 Discuss limitations of the study, taking into account sources of potential bias or imprecision. Discuss both direction and magnitude of any potential bias
Interpretation 20 Give a cautious overall interpretation of results considering objectives, limitations, multiplicity of analyses, results from similar studies, and other relevant evidence
Generalisability 21 Discuss the generalizability (external validity) of the study results
Other information
Funding 22 Give the source of funding and the role of the funders for the present study and, if applicable, for the original study on which the present article is based
*
Give information separately for cases and controls in case-control studies and, if applicable, for exposed and unexposed groups in cohort and cross-sectional studies.
Note:
An Explanation and Elaboration article discusses each checklist item and gives methodological background and published examples of transparent reporting. The STROBE checklist is best used in conjunction with this article (freely available on the web sites of PLoS Medicine at http://www.plosmedicine.org/, Annals of Internal Medicine at http://www.annals.org/, and Epidemiology at http://www.epidem.com/). Information on the STROBE Initiative is available at www.strobe-statement.org.
(Reproduced, with permission of Elsevier, from: von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP; STROBE Initiative. J Clin Epidemiol. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. 2008 Apr;61(4):344-9. Copyright [2008].

The STROBE statement is an itemized checklist of twenty-two recommendations that are considered essential for transparent reporting of cohort, case-control, and cross-sectional studies. Of the twenty-two items, eighteen are general to all three categories of observational studies and four are study-design-specific. Recommendations include a description of study aims, identification of potential confounders and bias, and guidelines for reporting study limitations and generalizability.

Back to Top | Article Outline

Guide to Investigators: How to Use the STROBE Statement

The STROBE statement is not intended as a tool for investigators aiming to improve study design; rather it is a guideline for authors aiming to improve the quality and clarity of presenting research findings. Adherence to the STROBE statement assists the author in disclosing potential bias inherent to the study design and allows the reader to recognize the limitations of the study’s results and conclusions. The following checklist is a guide intended to assist investigators in preparing manuscripts of observational studies.

Back to Top | Article Outline

Timing

Although the checklist is aimed at the development of a well-written manuscript, we recommend using the checklist during the preparation of the study proposal and in funding applications. Specifying categorical variables and subgroup analyses after data have been collected and reviewed can increase Type-I (alpha) error. Public disclosure of hypotheses and statistical methods prior to data examination may increase the validity of observational studies. Adherence to the STROBE checklist in the design of a study will aid in the development of a suitable working title and will help authors to define objectives, hypotheses, study and target populations, variables, research methodology, and statistical methods.

Back to Top | Article Outline

Applying the Items

We include an explanation of STROBE statement items, including examples from recently published observational studies in The Journal of Bone and Joint Surgery that we consider to be suitable examples of select checklist items. The articles referenced below may not, in their entirety, adhere to the STROBE checklist; however, we believe that the excerpts provided demonstrate transparent reporting. The following sections provide explanations and examples of checklist items.

Back to Top | Article Outline

Title and Abstract

Item 1: Title and Abstract

(a) Title

Indicate the study’s design with a commonly used term in the title or abstract.

Explanation: The research title should tell the reader exactly what research method was used to investigate a topic (cohort, case-control, or cross-sectional).

Example: “Surgical Compared with Nonoperative Treatment for Lumbar Degenerative Spondylolisthesis. Four-Year Results in the Spine Patient Outcomes Research Trial (SPORT) Randomized and Observational Cohorts.”8

Back to Top | Article Outline

(b) Abstract

Provide in the abstract an informative and balanced summary of what was done and what was found.

Explanation: The abstract is commonly a reader’s first exposure to the study and must clearly and succinctly depict the key details of the study. Sections of the abstract should be demarcated with subheadings. The Introduction section should tell the reader what the study is about and why it is important, the Methods section should briefly explain the study approach, the Results section should report the findings of the study in numerical format, and the Conclusion section should restate the pertinent findings and relevance of the study.

Back to Top | Article Outline

Introduction Section

Item 2. Background/Rationale

Explain the scientific background and rationale for the investigation being reported.

Explanation: The introduction of the paper should be limited to the pertinent information: Why did you do this study? What question did you hope to answer, and why is this question important? When you set out on this investigation, what was your hypothesis? This should not be a review of the literature or a citation of your previous work, but rather an explanation of the genesis of your investigation.

Back to Top | Article Outline

Item 3. Objectives

State specific objectives, including any prespecified hypotheses.

Explanation: A study’s specific aims and/or hypotheses identify the goals and expectations of the study. Clearly stating the study’s objectives allows the reader to judge if the study methods were suitable to test the hypotheses.

Example: “The purpose of this study was to investigate further the correlation between brachial plexus birth palsy and glenohumeral deformity…We hypothesized that ratios between internal and external rotator muscle cross-sectional areas would differ when the affected shoulder was compared with the unaffected shoulder and that the magnitude of these differences would correlate with greater glenohumeral deformity.”9

Back to Top | Article Outline

Methods Section

Item 4. Study Design

Present key elements of study design early in the paper.

Explanation: According to the STROBE statement, key study design elements should be presented at the end of the introduction or at the beginning of the Methods section. The type of observational study (cohort, case-control, or cross-sectional) should be clearly stated. Cohort studies should include the exposure status of the subjects, case-control studies should include information regarding the population from which the subjects were drawn, and cross-sectional studies should include the point in time during which the study took place.

Example 1: “The aim of the present study was to determine whether patients with a diagnosis of diabetes mellitus have an increased rate of infection following foot and ankle surgery compared with a cohort of patients without diabetes…Furthermore, we sought to demonstrate whether patients with complicated diabetes (associated with the presence of neuropathy, a history of ulcers, Charcot neuroarthropathy, or vascular disease) are at a greater risk for postoperative wound infection than are patients with uncomplicated diabetes or patients without diabetes.”10

Example 2: “Databases of patients who underwent operation at either of two specialized centers were used to identify all primary total knee arthroplasties that were performed for end-stage degenerative joint disease…All opioid medications and dosages were converted to a morphine-equivalent dose with use of ratios published by Labby et al, with only those patients using a minimum equivalent dose of 20 mg/day included in the preoperative opioid group (equivalent to approximately four hydrocodone or three oxycodone tablets per day)…This opioid cohort group was matched to a group of patients who had primary total knee arthroplasty at the same centers over a similar time period, but who were not treated with chronic opioids prior to surgery.”11

Back to Top | Article Outline

Item 5. Setting

Describe the setting, locations, and relevant dates, including periods of recruitment, exposure, follow-up, and data collection.

Explanation: A detailed report of where, when, and how data were collected provides a historical context so that readers can determine if the results are applicable to other settings and populations.

Example: “The study was conducted in the Departments of Orthopaedic Surgery and Traumatology at the University Hospital Basel in Switzerland, an 800-bed tertiary health-care center. Between May 2006 and October 2007, we prospectively included consecutive hospitalized patients who were eighteen years of age or older with a new onset of fever…Patient records were prospectively abstracted with use of a standardized data-collection case report form to retrieve demographic, clinical, microbiological, radiographic, and laboratory data.”12

Back to Top | Article Outline

Item 6. Participants

(a) Eligibility Criteria

Explanation: Include information regarding eligibility criteria (both inclusion and exclusion criteria), the group from which the population was selected (general population versus subgroup or location), and the methods of recruitment and follow-up. Case-control studies should include the rationale for choosing case and control groups.

Example: “A computerized database search for paraplegic patients was performed in the Department of Orthopaedic Surgery and Rehabilitation Medicine…An advertisement was placed in our regional newspaper in order to include a representative sample from the normal population as controls for the study. Because of cost constraints, we only included one control per case…A patient was included in the study if he or she (1) had been wheelchair-dependent for a minimum of thirty years; (2) was not morbidly obese (body mass index <40)…and (9) did not present with any contraindications for undergoing magnetic resonance imaging studies…With the exception of wheelchair dependence, the inclusion and exclusion criteria were the same for the spinal cord injury group and the control cohort.”13

Back to Top | Article Outline

(b) Matching Criteria

Explanation: Matching ensures that the case and control groups and the exposed and unexposed groups are equal and comparable. Matching criteria should be explicitly stated, and the number of subjects in each group should be reported.

Example: “Between October 2005 and May 2007, 100 paraplegic patients (200 shoulders) who had been paraplegic and wheelchair-dependent for a mean of thirty-three years entered the study and were matched with a cohort of 100 able-bodied volunteers (200 shoulders). The matching was done on the basis of age within five years and sex. The control cohort of able-bodied volunteers was selected in the order in which they responded to our invitation and advertisement. By virtue of the inclusion and exclusion criteria, the matching criteria seemed to be adequate for establishing comparable groups.”13

Back to Top | Article Outline

Item 7. Variables

Clearly define all outcomes, exposures, predictors, potential confounders, and effect modifiers. Give diagnostic criteria, if applicable.

Explanation: A clear definition of variables such as outcomes, exposures, and confounding variables allows the reader to understand the author’s intentions and to determine if the results of the study are applicable to other populations.

Example: “A postoperative infection was defined as an infection that occurred within thirty days after surgery…Mild infection was defined as purulent drainage with <2 cm of peri-incisional erythema and outpatient treatment with oral antibiotics. Severe infection was defined as purulent drainage with ≥2 cm of peri-incisional erythema and/or treatment by inpatient hospitalization or surgical intervention…Postoperative infection was chosen as the primary dependent variable, and various medical risk factors (age, Charcot neuroarthropathy, diabetes…) were considered potential independent variables.”10

Back to Top | Article Outline

Item 8. Data Sources/Measurement

For each variable of interest, give sources of data and details of methods of assessment (measurement). Describe the comparability of assessment methods if there is more than one group.

Explanation: The way in which a variable is collected and measured will affect the validity of the study. The source of the data and the method used to evaluate the data must be clearly described, such that a reader could repeat your method and reach the same result. The potential for confounding and bias is considerable in observational studies, and the author should aim to expose and mitigate these effects. For example, radiographic outcome measures (length, alignment, rotation) will vary with the precision of the film acquisition, and small changes in the orientation of the x-ray beam will change the measurement of the outcome variables. Thus, authors should describe the methods that were used to standardize image acquisition and measurement. The data itself may be flawed if the source (patient chart, database, or registry) contains inaccurate data. The authors should describe the estimated validity or reliability of the data source and measurement techniques and should describe any efforts to cross-validate their measures or adjust their analysis for errors.

Example: “The effective dose was calculated for standard computerized tomographic examinations of the upper and lower extremities as well as the cervical, thoracic, and lumbar spines…In order to validate our methodology, the effective dose of computerized tomographic studies of the chest, abdomen, and pelvis were also estimated and compared with the corresponding values previously reported in the literature.”14

Back to Top | Article Outline

Item 9. Bias

Describe any efforts to address potential sources of bias.

Explanation: As stated above, observational studies are prone to bias due to features inherent in the methodology. Biased studies produce results that differ from truth in a systematic way. Methodological variation in the enrollment of patients and errors in data entry may introduce bias when data extracted from registries or databases are used. Inappropriate assignment of a patient into a cohort or control group can lead to inaccurate assessment of risk. For example, if a patient’s weight is entered into a total joint registry as 350 lb when indeed the patient weighed only 150 lb, this low-risk patient may be inappropriately included in a cohort of morbidly obese patients selected on the basis of weight and decrease the observed risk of the cohort. Authors should anticipate potential sources of bias and should estimate the probable direction and magnitude of the effect on the results.

Example 1: “All studies that had been ordered were evaluated by the staff radiologists at the time of the injury…Results were systematically logged by the author (M.W.S.). A second radiologist (J.D.R.), who was not the original interpreter…, reviewed and reinterpreted all radiographic examinations. Each imaging modality was reviewed separately to minimize bias in interpretation.”15

Example 2: “Finally, as all physical examinations were performed by the treating orthopaedic surgeon…a potential observer bias was introduced that could have been avoided had an independent blinded examiner performed the examinations.”16

Back to Top | Article Outline

Item 10. Study Size

Explain how the study size was established.

Explanation: The sample size must be large enough to detect differences or associations if they exist. If sample size calculations are performed, they should be included. If other factors dictated sample size, such as a fixed available sample, this should be elucidated.

Example 1: “The sample size could not be calculated as the preexisting evidence was insufficient. However, assuming the incidence of rotator cuff tear in the controls to be 15% and the estimate of the relative risk (odds ratio) for the patients to be increased threefold, a sample size of seventy-five individuals per group would be sufficient to detect these risk increases.”13

Example 2: “A power analysis revealed that a minimum sample size of nine patients for each of the five glenoid types would provide 80% power (two-tailed α = 0.05, β = 0.20) to detect a mean difference of 20% in the PM/ER ratio among the types.”9

Back to Top | Article Outline

Item 11. Quantitative Variables

Explain how quantitative variables were handled in the analyses. If applicable, describe which groupings were chosen and why.

Explanation: For continuous variables, linearity or nonlinearity of the data must be described. If the continuous variable is divided into groups, these groups must be defined, as grouping may have implications in the statistical analysis of the data.

Example: “The body mass index was calculated, and patients were categorized according to the system of the National Institutes of Health (NIH)…as normal weight if the body mass index was <25 kg/m2, overweight if it was 25 to 30 kg/m2, and obese if it was >30 kg/m2.”17

Back to Top | Article Outline

Item 12. Statistical Methods

Explanation: The STROBE statement includes a sub-itemized list of seven recommendations for reporting the statistical methods used to collect and analyze the data. These recommendations are dependent on the investigational plan and study design and must be considered at the point of planning the data acquisition and analysis. Planning the statistical approach of a study is outside the scope of this guide, and we urge investigators to thoroughly understand the probable sources of confounding, bias, and threats to their study’s validity (internal and external) before beginning the study itself18. Authors should intricately describe all statistical methods, and the following items are directed at addressing the most common problems with the collection and analysis of observational research data.

Back to Top | Article Outline

(a) Control for Confounding

Explanation: Generally, confounding occurs when two groups of subjects are compared as if they are similar, when in fact there is an underlying fundamental difference between the groups. The observed relationship between the predictive variable (exposure) and the outcome variable can be due to extraneous “confounding” factors that influence both the exposure and the outcome and thus obscure the observed relationship of interest and threaten the validity of any conclusions drawn. The investigator should think beforehand about any potential confounding factors. Statistical adjustments such as stratification, regression analyses, or propensity score analysis should be made to expose and control for these relationships. All efforts taken to identify and control for confounding should be clearly reported.

Back to Top | Article Outline

(b) Subgroup/Interaction Analysis

Explanation: Subgroup analysis refers to looking for a pattern in a subset of the study subjects. The reader must be able to understand when the subgroup was identified (a priori or post hoc) as post hoc subgroup analysis may lead to compromised, biased results. Furthermore, the results from a post hoc analysis must be explicitly labeled as such in the report to avoid misleading readers.

Back to Top | Article Outline

(c) Handling Missing Data

Explanation: Incomplete data (common to retrospective studies and any study involving registry or database sources) predispose a study to selection bias. While it is tempting to simply include only patients with complete data on all variables of interest, this strategy (complete-case analysis) is generally insufficient to exclude the probability of bias. Authors should evaluate the data source and should understand the manner in which data points are missing (at random, completely random, not random), should employ appropriate methods to deal with the missing data (likelihood-based approaches, weighted estimation, multiple imputation analyses)19, and should describe these strategies clearly in their report.

Back to Top | Article Outline

(d) Case-Specific Statistical Methods

Explanation: This item is further subdivided by the type of study and is focused on ensuring that the study subjects are as comparable with each other or as representative of a target population as possible.

1. Describe how “loss to follow-up” was addressed. Loss to follow-up is rarely a truly random event and is a major source of bias in cohort studies. Patients with complete follow-up may be systematically different from those with incomplete follow-up. If loss to follow-up occurs more frequently among higher-risk patients than among lower-risk patients, the analysis of patients with complete follow-up will underestimate the relationship of interest. For example, imagine that a cohort is followed for the development of deep-vein thrombosis (DVT) after trauma and that only patients with one year of follow-up are analyzed. Any patient who died of a pulmonary embolus after a DVT after less than one year might not be identified for the final analysis, and thus the overall observed rate of DVT would be an underestimation of the true rate of DVT. It is important to distinguish features of patients who reach the end of the study from those who have not and to report how loss to follow-up was handled statistically (e.g., sensitivity analysis, exclusion of patients, censoring strategies).

2. Explain how cases and controls were matched. In order to draw meaningful conclusions from a comparison of two groups, the two groups need to be as similar as possible. Matching is a process by which potential confounding variables are identified and cases and controls are “matched” on the basis of these factors such that the groups are the same with respect to these factors. Matching is associated with important limitations and is not always advised. In order for the reader to judge if the matched design was appropriate, the author should clearly describe if, how, and why matching was performed and should describe the variables. An evaluation of the risks of matching and a prediction of the bias that may have resulted from matching should be addressed.

3. Describe the sampling strategy used. The goal of most clinical studies is to analyze the findings of a representative sample of subjects in order to generalize a conclusion about a larger “target” population of similar individuals. Most cross-sectional studies use a sampling strategy to select subjects from a source population. These strategies are often complex and include mathematical methods of estimating the probability that the sample’s characteristics are representative of a target population. Authors should clearly state the methods that were used to select subjects so that readers can understand how the chosen sampling method might influence the precision and generalizability of the results.

Back to Top | Article Outline

(e) Describe Any Sensitivity Analysis

Explanation: Sensitivity analyses are done to determine if the main results of an analysis are consistent with those obtained with alternative statistical or analytical strategies or assumptions. These techniques systematically change the parameters in a mathematical model to determine the effects of the changes on the results and can be used to identify confounding, selection bias, or informational bias required to distort an observed association. Authors should describe any sensitivity analyses performed, should explain why the analyses were done, and should describe what alternative findings were uncovered.

Example: “Sensitivity analysis tested the stability of the conclusions over a range of structural assumptions, probability estimates, and outcome values…In deterministic sensitivity analyses, the values of one or multiple variables in question were varied while the other probability and outcome values remained constant. All variables were tested with one-way sensitivity analysis over a range of plausible estimates.”20

Back to Top | Article Outline

Results Section

Item 13. Participants

(a) Numbers of Participants at Each Stage

Report the numbers of individuals at each stage of the study—e.g., the numbers of patients who were potentially eligible, who were examined for eligibility, who were confirmed as eligible, who were included in the study, who had complete follow-up, and who were analyzed.

Back to Top | Article Outline

(b) Reasons for Nonparticipation

Give the reasons for nonparticipation at each stage.

Back to Top | Article Outline

(c) Flow Diagram

Consider the use of a flow diagram.

Explanation: Transparent reporting allows the reader to recognize potential areas of selection bias. Authors should report the number of participants at each stage of the study (eligibility, recruitment, enrollment, and follow-up) and should include information about subjects who were deemed to be ineligible or who were lost to follow-up.

Example: A flow diagram is an effective method for presenting data regarding participants. In Figure 1, a detailed flow diagram from the SPORT study presents participant information at each step of the study from screening to four years of follow-up8.

Fig. 1

Fig. 1

Back to Top | Article Outline

Item 14. Descriptive Data

(a) Study Participants

Provide the characteristics of the study participants (e.g., demographic, clinical, social) and information on exposures and potential confounders.

Back to Top | Article Outline

(b) Missing Data

Indicate the number of participants with missing data for each variable of interest.

Explanation: See item 12(c) above for a description of the problem of missing data.

Example: In the study by Morshed et al., the authors retrospectively evaluated the National Trauma Database to identify a cohort of multiply injured patients who underwent fixation of femoral shaft fracture in order to evaluate the correlation of time to treatment with in-hospital mortality21. A clear illustration in the article demonstrates that 106 patients did not have a data point for the “time of surgery” predictive variable and that 336 patients did not have data entered for at least one of the covariates anticipated (by the investigators) to be associated with mortality. For these reasons, the authors chose to exclude these patients from the analysis. The authors stated in the Discussion: “Moreover, missing information on measured covariates for subjects who otherwise would have been eligible for inclusion in the study sample…may have introduced selection bias.”21

Back to Top | Article Outline

(c) Cohort Study: Summarize Follow-up Time

Explanation: Item 12(d) identifies the bias resulting from loss to follow-up. Readers need to know the duration of follow-up for the outcome data and thus need to know how many patients did not reach final follow-up and why.

Example: “Of the remaining thirty-nine patients, two had died of unrelated causes, three refused to return for clinical evaluation, and eight patients had been lost to follow-up. The remaining twenty-six patients had a complete preoperative clinical evaluation, operative records, and a minimum duration of follow-up of fifteen years (average, twenty-four years and nine months; range, fifteen to thirty-nine years) and were included in the study.”22

Back to Top | Article Outline

Item 15. Outcome Data

Explanation: In the Results section, descriptive data about the study population should be reported. For cohort and cross-sectional studies, outcome data should include the number of events or the rate of events over time for each outcome of interest. For case-control studies, information regarding the frequency of exposure for case and control groups should be included.

Example: Tables can effectively and clearly present descriptive outcome data. Table III depicts the outcome data for a cohort study in which the number of events (infections) is reported for each variable (group) for the whole population and a population of subjects with diabetes10.

TABLE III - Distribution of Infections in Entire Cohort and in the Diabetic Study Group
Entire Group
Diabetic Group (N = 190)
Variable (Group) No. of Patients No. (%) of Infections per Group No. (%) of Infections
All patients 1000 48 (4.8) 25 (52)
Hospitalized postoperatively 17 17 (100)* 9 (53)
Charcot neuroarthropathy 75 17 (23) 16 (94)
External fixation 110 19 (17) 10 (53)
Impatient surgery 356 21 (5.9) 10 (48)
Outpatient surgery 638 27 (4.2) 15 (56)
Peripheral neuropathy 163 29 (18) 23 (79)
Peripheral arterial disease 16 3 (19) 3 (100)
Previous ulcer 99 20 (20) 16 (80)
Rheumatoid arthritis 54 6 (11) 2 (33)
Tobacco use 155 7 (5) 3 (43)
Transplant recipient 15 2 (13) 2 (100)
*
The rate was 35% of all infections. (Reproduced from: Wukich DK, Lowery NJ, McMillen RL, Frykberg RG. Postoperative infection rates in foot and ankle surgery: a comparison of patients with and without diabetes mellitus. J Bone Joint Surg Am. 2010;92[2]:287-95.)

Back to Top | Article Outline

Item 16. Main Results

(a) Unadjusted and Adjusted Estimates

Give unadjusted estimates and, if applicable, confounder-adjusted estimates and their precision (e.g., 95% confidence interval). Make clear which confounders were adjusted for and why they were included.

Explanation: Including both unadjusted and adjusted estimates allows the reader to determine how confounding factors influence the direction and degree of association. The authors should explain which confounding factors were included or excluded and why.

Example: In a study of postoperative infection rates, Wukich et al. describe how the variables were selected in the Materials and Methods section: “The charts were reviewed and the following data were extracted for each subject: patient age, sex, history of diabetes mellitus…diagnosis of rheumatoid arthritis, length of surgery, and follow-up time in weeks. We chose these variables since other studies have associated them with postoperative infections.”10 In the Results section, the authors present data from both a univariate and multivariate analysis: “Stepwise multivariate logistic regression included all those variables with significant associations (p ≤ 0.05) found on univariate analysis. Multivariate logistic regression demonstrated that, when controlling for age, Charcot neuroarthropathy, diabetes status, peripheral arterial disease, and rheumatoid arthritis, only peripheral neuropathy (odds ratio, 3.98 [95% confidence interval, 1.52 to 10.45]; p < 0.05), a history of an ulcer (odds ratio, 2.42 [95% confidence interval, 1.14 to 5.16]; p < 0.05), and the use of external fixation (odds ratio, 2.80 [95% confidence interval, 1.39 to 5.66]; p < 0.05) were significantly associated with increased rates of infection.”10

Back to Top | Article Outline

(b) Category Boundaries

Report category boundaries when continuous variables were categorized.

Explanation: Item 11 emphasizes the importance of defining category boundaries if a continuous variable is divided into groups. In the Results section, authors should further define the range and mean or median for each variable group and, if relevant, statistical analysis as a continuous variable should also be included.

Example: “In the cemented study arm, however, the patients in the 73 to 82-kg weight group displayed a significant risk reduction of about 37% (odds ratio = 0.63; 95% confidence interval, 0.4 to 0.98) compared with the reference group of patients who weighed <64 kg. When weight was analyzed as a continuous variable, risks were increased per additional kilogram of body weight, but the difference was not significant (p = 0.62 for cemented cups and p = 0.52 for uncemented cups).”17

Back to Top | Article Outline

(c) Relative Risk and Absolute Risk

If relevant, consider translating estimates of relative risk into absolute risk for a meaningful time period.

Explanation: Relative risk is often more translatable to other populations than absolute risk, but in specific cases, such as determining the risk of an adverse drug effect, absolute risk may provide more pertinent information to the reader than relative risk does. If appropriate, the author may appeal to a broader audience and increase the generalizability of the results by including estimates of both relative and absolute risk.

Back to Top | Article Outline

Item 17. Other Analyses

Report other analyses that were done (e.g., analyses of subgroups and interactions and sensitivity analyses).

Explanation: See item 12(b) for a description and warning regarding subgroup analysis. Item 12(e) describes sensitivity analysis.

Example: The example in 12(e) is an excerpt from the Methods section of the manuscript, describing the indications for sensitivity analysis. In the Results section of the same paper, the authors write: “Deterministic sensitivity analysis revealed that a coordinator led to cost savings in comparison with no coordinator under four conservative conditions: (1) if the cost per hip fracture was as low as C$8000, (2) if only 60% of patients initiated treatment and only 40% complied, (3) if treatment efficacy reduced the incidence of future hip fractures by no more than 10%, and (4) if as few as 350 patients were seen annually.”20

Back to Top | Article Outline

Discussion Section

Item 18. Key Results

Summarize key results with reference to study objectives.

Explanation: By restating the study aims, authors remind readers of the primary objective of the study and convey whether or not the key results support or disclaim their hypotheses.

Example: “In this study, we aimed to investigate the correlation between the magnetic resonance imaging findings and the chronicity of an anterior cruciate ligament injury. We used one direct finding (anterior cruciate ligament morphology) and three indirect findings (joint effusion, posterior cruciate ligament angle, and bone bruise) on magnetic resonance imaging of a large number of subjects with varying chronicity of injury of the anterior cruciate ligament…The principal finding of this study is that there is a distinct correlation between the four magnetic resonance imaging findings and the chronicity of the anterior cruciate ligament tear.”23

Back to Top | Article Outline

Item 19. Limitations

Discuss limitations of the study, taking into account sources of potential bias or imprecision. Discuss both the direction and the magnitude of any potential bias.

Explanation: The Discussion section of the paper should include a detailed disclosure of the limitations of the paper. Simply stating that the retrospective nature of a study predisposes to potential bias and confounding is insufficient. Each element of the study design and the type of potential bias should be described. Efforts that were made to minimize bias and confounding should be outlined, as should the probable effects of these factors on the observed results.

Back to Top | Article Outline

Item 20. Interpretation

Give a cautious overall interpretation of the results while considering objectives, limitations, multiplicity of analyses, results from similar studies, and other relevant evidence.

Explanation: The Discussion section should inform readers where the study fits into the broader literature and how the results can be interpreted and thus applied to other populations.

Example: “In summary, the present study failed to demonstrate any important clinical benefit in association with the use of oral iron treatment for patients who had a decrease in the hemoglobin level from >110 g/L on admission to <110 g/L after surgery for the treatment of a hip fracture. In conjunction with previous studies on this topic, we conclude that the practice of using oral iron supplementation to treat anemia after orthopaedic surgery in patients who are not anemic before surgery is of doubtful value.”24

Back to Top | Article Outline

Item 21. Generalizability

Discuss the generalizability (external validity) of the study results.

Explanation: The investigators should explain any possible discrepancies between the study population and a more general target population that may threaten the generalizability of the results to a reader’s patients.

Example: “Despite its multicenter design, this study cohort may not be representative of the population of patients who undergo revision total knee arthroplasty. The seventeen centers are all high-volume joint replacement centers, the functional outcomes of which may be superior to those of the average community hospital.”25

Back to Top | Article Outline

Other Information

Item 22. Funding

Give the source of funding and the role of the funders for the present study and, if applicable, for the original study on which the present article is based.

Explanation: There is a strong association between the source of funding and the conclusions of many research articles, regardless of funding agency. Any funding, regardless of source, should be clearly described so that the reader can assess the possibility or probability of conflicts and undue influence inherent to the financial support of the project.

Example 1: “This study was funded by the Orthopaedic Research and Education Foundation, American Geriatrics Society, and The Knee Society. Dr. Ghomrawi was also supported in part by the Weill Cornell Medical College Center for Education and Research on Therapeutics (CERT) Program from the Agency for Healthcare Research and Quality, Grant Number U18 HS016075.”25

Example 2: “In support of this research for or preparation of this work, one or more of the authors received outside funding or grants for >$10,000 from ArthroCare.”26

Back to Top | Article Outline

Conclusions

Observational studies, including case-control, cohort, and cross-sectional study designs, are prevalent in the orthopaedic literature and provide meaningful information about disease etiology, natural history, prognostic factors, and treatment effectiveness. Nonrandomized study designs, however, are inherently prone to bias, most notably confounding, sampling bias, and recall bias. Authors must recognize the limitations of observational studies and expose potential biases that threaten the validity and applicability of their findings. The STROBE statement is a well-validated checklist that is designed to assist clinician-scientists in reporting observational studies. Using examples from The Journal of Bone and Joint Surgery, we have expanded on the STROBE statement to provide a reference tool for orthopaedic surgeons reporting observational studies. We have highlighted STROBE checklist items that improve transparent reporting by addressing potential bias and providing a context for the results of an observational study.

Investigation performed at the University of California, Davis Medical Center, Sacramento, California

Back to Top | Article Outline

References

1. Hoppe DJ, Schemitsch EH, Morshed S, Tornetta P 3rd, Bhandari M . Hierarchy of evidence: where observational studies fit in and why we need them. J Bone Joint Surg Am. 2009;91(Suppl 3):2-9.
2. Mundi R, Chaudhry H, Singh I, Bhandari M . Checklists to improve the quality of the orthopaedic literature. Indian J Orthop. 2008;42(2):150-64.
3. Carr AJ. Evidence-based orthopaedic surgery: what type of research will best improve clinical practice? J Bone Joint Surg Br. 2005;87(12):1593-4.
4. Horton R. Surgical research or comic opera: questions, but few answers. Lancet. 1996;347(9007):984-5.
5. Rosner B. Fundamentals of biostatistics. 6th ed. Boston: Brooks/Cole; 2005. p 632-48.
6. Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, Poole C, Schlesselman JJ, Egger M ; STROBE Initiative. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. Epidemiology. 2007;18(6):805-35.
7. The Journal of Bone and Joint Surgery. Instructions for authors. http://jbjs.org/public/instructionsauthors.aspx. Accessed 2012 Nov 20.
8. Weinstein JN, Lurie JD, Tosteson TD, Zhao W, Blood EA, Tosteson AN, Birkmeyer N, Herkowitz H, Longley M, Lenke L, Emery S, Hu SS . Surgical compared with nonoperative treatment for lumbar degenerative spondylolisthesis. Four-year results in the Spine Patient Outcomes Research Trial (SPORT) randomized and observational cohorts. J Bone Joint Surg Am. 2009;91(6):1295-304.
9. Waters PM, Monica JT, Earp BE, Zurakowski D, Bae DS . Correlation of radiographic muscle cross-sectional area with glenohumeral deformity in children with brachial plexus birth palsy. J Bone Joint Surg Am. 2009;91(10):2367-75.
10. Wukich DK, Lowery NJ, McMillen RL, Frykberg RG . Postoperative infection rates in foot and ankle surgery: a comparison of patients with and without diabetes mellitus. J Bone Joint Surg Am. 2010;92(2):287-95.
11. Zywiel MG, Stroh DA, Lee SY, Bonutti PM, Mont MA . Chronic opioid use prior to total knee arthroplasty. J Bone Joint Surg Am. 2011 Nov 2;93(21):1988-93.
12. Hunziker S, Hügle T, Schuchardt K, Groeschl I, Schuetz P, Mueller B, Dick W, Eriksson U, Trampuz A . The value of serum procalcitonin level for differentiation of infectious from noninfectious causes of fever after orthopaedic surgery. J Bone Joint Surg Am. 2010;92(1):138-48.
13. Akbar M, Balean G, Brunner M, Seyler TM, Bruckner T, Munzinger J, Grieser T, Gerner HJ, Loew M . Prevalence of rotator cuff tear in paraplegic patients compared with controls. J Bone Joint Surg Am. 2010;92(1):23-30.
14. Biswas D, Bible JE, Bohan M, Simpson AK, Whang PG, Grauer JN . Radiation exposure from musculoskeletal computerized tomographic scans. J Bone Joint Surg Am. 2009;91(8):1882-9.
15. Smith MW, Reed JD, Facco R, Hlaing T, McGee A, Hicks BM, Aaland M . The reliability of nonreconstructed computerized tomographic scans of the abdomen and pelvis in detecting thoracolumbar spine injuries in blunt trauma patients with altered mental status. J Bone Joint Surg Am. 2009;91(10):2342-9.
16. Tokish JM, McBratney CM, Solomon DJ, Leclere L, Dewing CB, Provencher MT . Arthroscopic repair of circumferential lesions of the glenoid labrum. J Bone Joint Surg Am. 2009;91(12):2795-802.
17. Röder C, Bach B, Berry DJ, Eggli S, Langenhahn R, Busato A . Obesity, age, sex, diagnosis, and fixation mode differently affect early cup failure in total hip arthroplasty: a matched case-control study of 4420 patients. J Bone Joint Surg Am. 2010;92(10):1954-63.
18. Morshed S, Tornetta P 3rd, Bhandari M . Analysis of observational studies: a guide to understanding statistical methods. J Bone Joint Surg Am. 2009;91(Suppl 3):50-60.
19. Little RJA, Rubin DB . Statistical analysis with missing data. 2nd ed. New York: Wiley; 2002. A taxonomy of missing-data methods; p 19-23.
20. Sander B, Elliot-Gibson V, Beaton DE, Bogoch ER, Maetzel A . A coordinator program in post-fracture osteoporosis management improves outcomes and saves costs. J Bone Joint Surg Am. 2008;90(6):1197-205.
21. Morshed S, Miclau T 3rd, Bembom O, Cohen M, Knudson MM, Colford JM Jr . Delayed internal fixation of femoral shaft fracture reduces mortality among patients with multisystem trauma. J Bone Joint Surg Am. 2009;91(1):3-13.
22. Antuña SA, Sánchez-Márquez JM, Barco R . Long-term results of radial head resection following isolated radial head fractures in patients younger than forty years old. J Bone Joint Surg Am. 2010;92(3):558-66.
23. Yoon JP, Chang CB, Yoo JH, Kim SJ, Choi JY, Choi JA, Seong SC, Kim TK . Correlation of magnetic resonance imaging findings with the chronicity of an anterior cruciate ligament tear. J Bone Joint Surg Am. 2010;92(2):353-60.
24. Parker MJ. Iron supplementation for anemia after hip fracture surgery: a randomized trial of 300 patients. J Bone Joint Surg Am. 2010;92(2):265-9.
25. Ghomrawi HM, Kane RL, Eberly LE, Bershadsky B, Saleh KJ ; North American Knee Arthroplasty Revision (NAKAR) Study Group. Patterns of functional improvement after revision knee arthroplasty. J Bone Joint Surg Am. 2009;91(12):2838-45.
26. Zoric BB, Horn N, Braun S, Millett PJ . Factors influencing intra-articular fluid temperature profiles with radiofrequency ablation. J Bone Joint Surg Am. 2009;91(10):2448-54.

Disclosure: None of the authors received payments or services, either directly or indirectly (i.e., via his or her institution), from a third party in support of any aspect of this work. One or more of the authors, or his or her institution, has had a financial relationship, in the thirty-six months prior to submission of this work, with an entity in the biomedical arena that could be perceived to influence or have the potential to influence what is written in this work. No author has had any other relationships, or has engaged in any other activities, that could be perceived to influence or have the potential to influence what is written in this work. The complete Disclosures of Potential Conflicts of Interest submitted by authors are always provided with the online version of the article.

Copyright © 2013 by The Journal of Bone and Joint Surgery, Incorporated