The approval and marketing of prescription medications is contingent on randomized controlled trial (RCT) evidence that demonstrates efficacy and safety, usually in comparison to a placebo; however, RCTs have limitations that can limit real-world generalizability. Most RCTs require strict patient monitoring protocols and often exclude participants with comorbidities. In effect, results commonly overestimate benefits and underestimate adverse effects on patients in the “real-world,” where patients are more likely to have comorbidities and treatment with other drugs.1 Randomized controlled trials are also costly, limiting the size of enrollment with an aim to detect efficacy, and thus potentially missing rare side effects.1 Observational, population-based studies, otherwise known as pharmacoepidemiologic or “real-world evidence” studies,1,2 are employed to study the effectiveness and safety of medications in large populations under routine care. Evidence generated from observational studies is used in conjunction with trials to inform regulatory and clinical policy. The Food and Drug Administration,3 Health Canada,4 and the European Medicines Agency5 have all published guidance on using real-world evidence to guide decision-making.
Many observational studies use routinely collected, administrative health care data, such as pharmacy claims, hospital discharge records, physician billings, and patient registration or enrollment, to examine drug effects in large populations.6 Observational studies that leverage these data to study drug effects, however, are subject to biases because data are not collected explicitly for research purposes and are therefore often lacking clinical detail. Thus, study design must be considered carefully to mitigate the effects of potential biases on results. Many design considerations are also specific to the population, drug, outcome of interest, and local health care system data context; researchers must be aware of all these factors during the study design process.
Several new classes of medications to treat type 2 diabetes mellitus (T2DM) have been approved since January 1, 2000, including dipeptidyl peptidase 4 inhibitors, glucagon-like peptide-1 (GLP-1) agonists, and sodium-glucose co-transporter 2 inhibitors. These medications have been rapidly adapted as second-line T2DM therapies; sitagliptin, liraglutide, and canagliflozin are among the top 200 drugs dispensed in the United States.7 Observational studies have identified safety signals that suggest these agents may be associated with rare adverse effects, including increased fracture risk.8-10 Several observational studies have been recently published that examine T2DM medication classes and fracture outcomes but use different methodologies.11,12
Drugs can affect fracture risk through reduced bone mineral density or increased falls risk.13 Different drugs, therefore, have different onsets of effect on fracture risk based on the specific drug (eg, drug metabolism) and the mechanism of fracture risk. Bone mineral density changes and falls also affect fracture risk of sites in the body differently; for example, the fragility of weight-bearing sites such as the spine is largely affected by changes in bone mineral density, while forearm fractures are most common after a fall.14 Fracture sites may also be captured in different administrative data sources with variable validity (eg, validity of hip fractures [positive predictive value, PPV, 13% to 60%] versus wrist fractures [PPV 71% to 86%] via physician visits without procedural codes).15,16 Better understanding of current practices is needed for observational studies of medications and fracture risk.
A case study that examines a narrow exposure group, such as medication classes for one indication, can illustrate study design concepts with examples and facilitate comparisons of methodology that incorporate exposure-specific considerations (eg, pharmacokinetics of various drugs within a class). We believe a scoping review of T2DM medications and fracture outcomes could form an ideal case study to illustrate current practices in the application of observational research methods to study drug effects on fracture risk that will inform future systematic reviews with quality appraisal to create best practice guidelines. A search of Open Science Framework, PROSPERO, MEDLINE, the Cochrane Database of Systematic Reviews, and JBI Evidence Synthesis was conducted, and no current scoping or systematic review is underway that evaluates the use of observational research methodology to study fracture risk. Three reviews examining empiric effects of T2DM or related medications and fracture risk in both RCTs and observational studies were identified, yet these reviews focused on drug effects and not methods used to estimate drug effects.17-19
The objective of this scoping review is to summarize research methods employed in observational studies leveraging administrative health care data to examine fracture risk of medications used to treat patients with T2DM.
What observational research methods are applied to study fracture risk effects of T2DM pharmacotherapies? Specifically, among studies that utilize administrative health care data to examine T2DM medications (with any comparator) and fracture risk in adults, what populations are included or excluded, how are major variables (exposure, outcome, covariates) defined, and what study design, follow-up, and statistical methods are employed?
This scoping review will consider studies that include adult populations (≥18 years) that examine at least one medication used to treat T2DM on a fracture outcome as the primary research objective (see eligible medications in Table 1). Studies that examine any fracture outcome will be eligible. Exposure to diabetes medications and fracture outcomes must be defined using administrative data sources, such as that from dispensation claims, prescriptions, or hospitalization discharge codes. Other variables may be defined using any measurement technique (eg, patient surveys).
Table 1 -
Eligible type 2 diabetes mellitus
||Drugs within that class
GLP-1 receptor agonists
• Exenatide extended-release
|| • Canagliflozin
|| • Acarbose
||Bolus (prandial insulins)
• Aspart (Novolog)
• Glulisine (Apidra)
• Lispro U-100 (Humalog U-100)
• Lispro U-200 (Humalog U-200)
• Regular (Humulin R, Novolin R)
• NPH (isophane insulin, Humulin N, Novolin N)
• Degludec U-100 (Tresiba U-100)
• Degludec U-200 (Tresiba U-200)
• Detemir (Levemir)
• Glargine U-100 (Lantus U-100)
• Glargine U-300 (Lantus U-300)
• Biphasic insulin aspart
• Lispro/lispro protamine suspension
• Premixed regular-NPH
• Gliclazide modified-release
|| • Glitazone
|| • Amylin (pramlintide)
The concept of interest in this scoping review is the methodology applied by various observational studies to study fracture risk. The following characteristics will be collected for each selected study to summarize methodology: the populations (including inclusions and exclusions) and time periods studied; exposure and outcome definitions (including follow-up length and exposure-outcome risk windows); comparator groups; potential covariates and mediators considered; and statistical methods (model types, use of propensity scores, etc.).
Studies will be included regardless of geographic location, specific setting, or cultural factors. There are no specific racial or gender-based interests; however, we will examine whether sex-specific data are presented to describe populations studied. Type 2 diabetes mellitus medications will be used as a case study due to known applications of observational methodology to evaluate fracture risk associated with their use.
Types of sources
Inclusion will be limited to studies published or made available since January 1, 2000. The year 2000 is used as the lower end of the date range because a previous systematic review of pharmacoepidemiologic literature identified that the vast majority of observational drug effects studies were published in or after the year 200120; restriction of papers published after 2000 therefore captures modern observational study methodology and data sources and enhances feasibility of the search.
Non-human studies, studies in pediatric populations, and studies without the full text will be excluded. Non-English studies will also be excluded due to limited available resources to the study team to translate or contract translating services for foreign-language articles. Only empirical observational studies that conduct analysis on an individual (versus ecological) level, such as a cohort, case control, case crossover, self-controlled case series, or case cohort design, will be included. Time-series, ecological/quasi-experimental designs, and cross-sectional studies are not eligible. Systematic and scoping reviews are not eligible for inclusion, yet the reference lists of these articles will be screened by reviewers to identify any potential eligible articles missed by the search.
Conference abstracts and theses will not be searched. Gray literature, which varies in purpose and reporting completeness, will not be included, as the purpose of this review is to describe current practices used in peer-reviewed articles in the field of diabetes medications and fracture risk.
The proposed scoping review will be conducted in accordance with the JBI methodology for scoping reviews.21 This review has been registered under the Open Science Framework (DOI: 10.17605/OSF.IO/M3CFB).
The search strategy will aim to locate published and unpublished (pre-prints, electronic articles published ahead of print, and accepted [in-press]) studies that satisfy three concepts: i) medications used to treat T2DM, ii) fractures, and iii) observational research methods (study design and data source). The preliminary search strategy was developed in consultation with a science liaison librarian. Search terms for T2DM medications were identified from the 2018 Diabetes Canada Clinical Guidelines as well as a recently published Cochrane review focused on glucose-lowering agents.22,23 Our search strategy was additionally informed by a validation study that yielded over 95% sensitivity for epidemiologic publications and from text mining epidemiologic studies published by the University of Texas that identified terms associated with observational studies and administrative health care data.24,25
Search terms were adapted following two rounds of validation conducted on MEDLINE, each with five to 10 articles, after which our MEDLINE search strategy was finalized (Appendix I). This search strategy was then translated into preliminary search strategies for Embase and CINAHL. The reference lists of articles selected for full-text review will be screened by two reviewers for additional studies. The databases to be searched include peer-reviewed scientific journal databases MEDLINE (Ovid), Embase (Ovid) and CINAHL (EBSCO), from January 1, 2000, to present.
The reference lists for articles selected for full-text review and any reviews identified by the search will be hand searched by one reviewer. An expert in the topic area of interest will be identified and consulted to identify any potentially relevant authors and articles missed by the search. Reviewers may contact study authors for further information if required.
Following the search, all identified records will be collated and uploaded into Zotero v.5.0.87 (Corporation for Digital Scholarship and Roy Rozenweig Center for History and New Media, VA, USA) and duplicates removed using EndNote v.X9 (Clarivate Analytics, PA, USA). Titles and abstracts will then be screened by two independent reviewers for assessment against the inclusion criteria for the review using Covidence (Veritas Health Innovation, Melbourne, Australia). Potentially relevant papers remaining after screening will be retrieved in full and their citation details imported into Covidence. The full text of selected citations will be assessed in detail against the inclusion criteria by two independent reviewers. Reasons for exclusion of full-text papers that do not meet the inclusion criteria will be recorded and reported in the scoping review. The order of exclusions applied will be: i) non-English study; ii) study not in humans; iii) published or made available before January 1, 2000; iv) review article; v) experimental study; vi) abstract-only, gray literature, or no full text available; vii) only pediatric population studied; viii) eligible T2DM medications are not primary exposure; ix) fracture is not primary outcome; x) non-eligible observational study design (eg, cross-sectional, time series) as primary design; xi) administrative data not used to define outcome; xii) administrative data not used to define exposure. Any disagreements that arise between the reviewers at each stage of the selection process will be resolved through discussion or with a third reviewer, if needed. The results of the search will be reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for Scoping Reviews (PRISMA-ScR) checklist and presented in a PRISMA flow diagram.26
Data will be extracted from papers included in the scoping review by two independent reviewers using a data extraction tool developed by the authors. Draft extraction forms, which have been tested with one eligible paper used to create the search strategy, are provided (see Appendix II). The data extracted from studies will include specific details about the population, concept, context, methods, statistical analyses, and key findings relevant to the review question. Data extraction from the first 10 eligible articles will be reviewed by two reviewers, then agreement on data extracted will be assessed. Any disagreements in data extraction between the reviewers during this pilot phase will be resolved through discussion or with a third reviewer as needed. Any modifications to the data extraction form will be described in the full scoping review. Pending sufficient agreement and any changes of the form after this pilot phase, one reviewer will conduct data extraction for each of the remaining articles with a second reviewer to check and confirm data. Authors of papers will be contacted to request missing or additional data, where required.
Data analysis and presentation
The extracted data will be presented in tabular form in a manner that aligns with the objective of this scoping review. Study design aspects (eg, exposure definition, statistical methods used) will be aggregated and described by the number of papers that used each type of method. An example results summary table is provided in Appendix III. If patterns emerge related to use of certain methods and study characteristics (eg, region, calendar time), those may be presented in a separate table or diagram. The planned presentation of the results will likely evolve as the study progresses, and the final presentation of the results will be justified in the review. The final paper will include a narrative summary of current practices for the application of observational research methods to study fracture with T2DM medications as an exposure case example.
Glyneva Bradley-Ridout, MI, who provided methodological guidance during development of the search strategy and revisions.
This work was supported by the University of Toronto Leslie Dan Faculty of Pharmacy Summer Undergraduate Summer Research Program (USRP) and a Canadian Institutes of Health Research (CIHR) Project Grant (PJT-16913).
KNH is supported by a CIHR Frederick Banting and Charles Best Canada Graduate Scholarships Doctoral Award (GSD-16412), the Drug Safety and Effectiveness Cross-disciplinary Training (DSECT) Program, and the Ontario Drug Policy Research Network (ODPRN).
Appendix I: Search strategy
Search conducted June 9, 2020.
Appendix II: Data abstraction forms
A: Example cohort study data abstraction form
B: Example case-control study data abstraction form
Appendix III: Example results summary table
1. Sherman RE, Anderson SA, Dal Pan GJ, Gray GW, Gross T, Hunter NL, et al. Real-world evidence — what is it and what can it tell us? N Engl J Med 2016;375 (23):2293–2297.
2. U.S. Food and Drug Administration. Real-world evidence [internet]. 2020 [cited 2020 Oct 20]. Available from: https://www.fda.gov/science-research/science-and-research-special-topics/real-world-evidence
3. U.S. Food and Drug Administration. Submitting documents using real-world data and real-world evidence to FDA for drugs and biologics guidance for industry [internet]. 2019 [cited 2020 Nov 2]. Available from: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/submitting-documents-using-real-world-data-and-real-world-evidence-fda-drugs-and-biologics-guidance
4. Health Canada. Elements of real world data/evidence quality throughout the prescription drug product life cycle [internet]. 2020 [cited 2020 Jun 16]. Available from: https://www.canada.ca/en/services/health/publications/drugs-health-products/real-world-data-evidence-drug-lifecycle-report.html
5. Cave A, Kurz X, Arlett P. Real-world data for regulatory decision making: challenges and possible solutions for Europe. Clin Pharmacol Ther 2019;106 (1):36–39.
6. Cadarette SM, Wong L. An Introduction to health care administrative data. Can J Hosp Pharm 2015;68 (3):
7. ClinCalc. ClinCalc Drugstats Database: the top 200 drugs of 2020 [internet]. 2020 [cited 2020 Jun 16]. Available from: https://clincalc.com/DrugStats/
8. Health Canada. For health professionals - association between long-term treatment with Actos (pioglitazone hydrochloride) tablets for type 2 diabetes mellitus
and fractures in women [internet]. 2007 [cited 2021 Oct 14]. Available from: https://www.healthycanadians.gc.ca/recall-alert-rappel-avis/hc-sc/2007/14436a-eng.php
9. Loke YK, Singh S, Furberg CD. Long-term use of thiazolidinediones and fractures in type 2 diabetes: a meta-analysis. Can Med Assoc J 2009;180 (1):32–39.
10. U.S. Food and Drug Administration. FDA Drug Safety Communication: FDA revises label of diabetes drug canagliflozin (Invokana, Invokamet) to include updates on bone fracture risk and new information on decreased bone mineral density [internet]. 2015 [cited 2020 Jun 16]. Available from: https://www.fda.gov/drugs/drug-safety-and-availability/fda-drug-safety-communication-fda-revises-label-diabetes-drug-canagliflozin-invokana-invokamet
11. Fralick M, Kim SC, Schneeweiss S, Kim D, Redelmeier DA, Patorno E. Fracture risk after initiation of use of canagliflozin: a cohort study. Ann Intern Med 2019;170 (3):155.
12. Abrahami D, Douros A, Yin H, Yu OHY, Azoulay L. Sodium–glucose cotransporter 2 inhibitors and the risk of fractures among patients with type 2 diabetes. Diabetes Care 2019;42 (9):e150–e152.
13. Wong L, Hayes K, Cadarette S. Bad to the bone: how pharmacists can help minimize drug-induced fracture risk. Pharm Pract Bus 2018;5 (10):18–23.
14. Warriner AH, Patkar NM, Curtis JR, et al. Which fractures are most attributable to osteoporosis? J Clin Epidemiol 2011;64 (1):46–53.
15. Lix LM, Azimaee M, Osman BA, Caetano P, Morin S, Metge C, et al. Osteoporosis-related fracture case definitions for population-based administrative data. BMC Public Health 2012;12 (1):
16. Jean S, Candas B, Belzile É, Morin S, Bessette L, Dodin S, et al. Algorithms can be used to identify fragility fracture cases in physician-claims databases. Osteoporos Int 2012;23 (2):483–501.
17. Mamza J, Marlin C, Wang C, Chokkalingam K, Idris I. DPP-4 inhibitor therapy and bone fractures in people with Type 2 diabetes – A systematic review and meta-analysis. Diabetes Res Clin Pract 2016;116:288–298.
18. Donnan JR, Grandy CA, Chibrikov E, Marra CA, Aubrey-Bassler K, Johnston K, et al. Comparative safety of the sodium glucose co-transporter 2 (SGLT2) inhibitors: a systematic review and meta-analysis. BMJ Open 2019;9 (1):e022577.
19. Salari-Moghaddam A, Sadeghi O, Keshteli AH, Larijani B, Esmaillzadeh A. Metformin use and risk of fracture: a systematic review and meta-analysis of observational studies. Osteoporos Int 2019;30 (6):1167–1173.
20. Cadarette SM, Ban JK, Consiglio GP, Black CD, Dubins D, Marin A, et al. Diffusion of Innovations model helps interpret the comparative uptake of two methodological innovations: co-authorship network analysis and recommendations for the integration of novel methods in practice. J Clin Epidemiol 2017;84:150–160.
21. Peters M, Godfrey C, McInerney P, Munn Z, Tricco A, Khalil H. Chapter 11: Scoping Reviews (2020 version). In: Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis (formerly JBI Reviewer's Manual) [internet]. Adelaide: JBI; 2020 [cited 2021 Feb 21]. Available from: https://synthesismanual.jbi.global
22. Lipscombe L, Booth G, Butalia S, Dasgupta K, Eurich DT, Goldenberg R, et al. Pharmacologic glycemic management of type 2 diabetes in adults. Can J Diabetes 2018;42: (Suppl 1): S88–S103.
23. Lo C, Toyama T, Wang Y, Lin J, Hirakawa Y, Jun M, et al. Insulin and glucose-lowering agents for treating people with diabetes and chronic kidney disease. Cochrane Kidney and Transplant Group, ed. Cochrane Database Syst Rev 2018;9 (9):CD011798.
24. University of Texas School of Public Health. Search filters for various databases [internet]. n.d. [cited 2020 Oct 20]. Available from: https://libguides.sph.uth.tmc.edu/search_filters
25. Larney S, Kopinski H, Beckwith CG, Zaller ND, Jalais DD, Hagan H, et al. Incidence and prevalence of hepatitis C in prisons and other closed settings: results of a systematic review and meta-analysis. Hepatology 2013;58 (4):1215–1224.
26. Moher D, Liberati A, Tetzlaff J, Altman DG. The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 2009;6 (7):e1000097.