Pragmatic Trials in Maintenance Dialysis: Perspectives from the Kidney Health Initiative : Journal of the American Society of Nephrology

Journal Logo

Up Front Matters: Brief Reviews

Pragmatic Trials in Maintenance Dialysis: Perspectives from the Kidney Health Initiative

Dember, Laura M.*,†; Archdeacon, Patrick‡,§; Krishnan, Mahesh; Lacson, Eduardo Jr.; Ling, Shari M.**; Roy-Chaudhury, Prabir††; Smith, Kimberly A.§; Flessner, Michael F.‡‡

Author Information
Journal of the American Society of Nephrology 27(10):p 2955-2963, October 2016. | DOI: 10.1681/ASN.2016030340
  • Free


The nephrology community has expressed concern about the inadequate evidence base for guiding clinical practice and the slow pace of improving outcomes for patients with kidney disease.1,2 A significant barrier to evidence generation is the inefficiency and expense of clinical trials, the most rigorous method for evaluating treatment approaches. The number of published clinical trials addressing kidney disease is lower than for many other disease areas3,4 and most nephrology trials, like those in other medical specialties, have small sample sizes.5,6 Moreover, interventions found to be beneficial in well conducted trials are often not widely implemented in practice.7–9

In recent years there has been a renewed interest in pragmatic clinical trials, also referred to as “real-world” clinical trials, because of their potential to yield findings that are highly generalizable, identify interventions that are readily translatable into clinical practice, and increase the efficiency of trial conduct.10–12 In contrast to explanatory trials which evaluate efficacy by testing treatments under idealized conditions and in highly selected patients, pragmatic trials evaluate effectiveness of interventions implemented under the usual conditions of clinical care and for the broad group of patients with the condition of interest. Pragmatic trials usually study treatments that are already in widespread use, with the goal of identifying optimal approaches.

Maintenance dialysis is a setting that is well suited for pragmatic clinical trials. In dialysis, encounters with patients are frequent and predictable, data generated through routine care are highly granular and uniform across patients, electronic data systems are widely utilized, and, in the United States, the delivery of care to approximately 90% of the patients by a small number of provider organizations facilitates centralized trial implementation. Recognizing the potential of pragmatic trials in the dialysis setting, the Kidney Health Initiative – a consortium of >75 organizations founded by the American Society of Nephrology and the US Food and Drug Administration (FDA) – assembled a working group with representation from academia, industry, and government to identify key considerations for conducting pragmatic trials in dialysis.13 In this paper, we discuss several of the important scientific, operational, and regulatory issues for such trials. Where applicable, we provide concrete illustrations from experience with the Time to Reduce Mortality in End Stage Renal Disease (TiME) Trial (NCT02019225), an ongoing, large, cluster-randomized pragmatic trial of hemodialysis session duration being conducted through a partnership between academic investigators and two large dialysis organizations as one of the demonstration projects of the National Institutes of Health (NIH) Health Care Systems Research Collaboratory.14

Pragmatic Trial Design

Characteristics of pragmatic trials and their implications for trial design are provided in Table 1.15,16 Pragmatic trials often randomize by “cluster” rather than by patient to facilitate implementation of the intervention(s) by clinical personnel within a health care team and to avoid contamination across treatment groups. Sample size requirements are usually larger for pragmatic than explanatory trials in order to overcome cluster effects (if cluster randomization is used) and because of reductions in treatment effects resulting from trial populations that are not highly selected for patients most likely to adhere to or benefit from the intervention. Approaches to implementing the intervention are typically flexible to ensure that the trial experience generalizes to the clinical setting. End points for pragmatic trials are often limited to those ascertainable from routine, clinically acquired data available in electronic medical or administrative records, unless there are sufficient resources to support large-scale data capture specifically for the trial.

Table 1. - Pragmatic trial characteristics and design implications
Component Characteristics Implications
Eligibility criteria Nonrestrictive in order to increase generalizability Allows for rapid enrollment; longer follow-up is possible if trial duration is fixed
Necessitates large sample size if nonselectivity reduces effect size
Allows for EMR-based identification of participants
Intervention Often already in widespread use Sometimes allows for waiver or modification of informed consent
Implemented by clinicians rather than researchers Reduces required research infrastructure
Can be implemented with flexibility Effect size reduced if incomplete adherence and crossover between treatment groups
Control Alternative to intervention that is also already in widespread use, or “usual care” With usual care control, secular changes during trial conduct should be anticipated
Randomization Often by cluster rather than individual participant If cluster-randomization, sample size determination should incorporate:
Treatment assignment usually not masked   Intracluster correlation coefficient
  Size of clusters; variability of cluster size
  Number of clusters
If cluster-randomization, less potential for contamination across treatment groups but greater possibility of baseline imbalances across treatment groups
If individual participant randomization, smaller sample size is required but sample may be less representative of population with condition
Lack of masking allows for management by clinicians rather than research personnel
Outcomes Often ascertained through electronic health records and/or administrative databases Requires upfront attention to data specifications for outcome ascertainment
Usually clinically important rather than intermediate or surrogate outcomes
Adverse event monitoring Adverse events often ascertained through electronic health records and administrative databases Can necessitate use of targeted rather than comprehensive adverse events
EMR, electronic medical record.

The design of the TiME Trial is summarized in Table 2. In brief, dialysis facilities are randomized to Intervention or Usual Care. Facilities in the Intervention group adopt an approach of prescribing dialysis session durations of at least 4.25 hours for incident patients; facilities in the Usual Care group have no trial-driven approach to session duration. The primary outcome is mortality, and major secondary outcomes are hospitalizations and quality of life. Pragmatic features of the TiME Trial include broad eligibility criteria, the implementation of the intervention by treating nephrologists and dialysis unit personnel rather than by researchers, and reliance on data obtained through routine clinical care. The major trial outcomes (mortality, hospitalization rate, and quality of life) are patient-centered and ascertainable through the electronic data systems of the dialysis providers. To address cluster effects, the sample size determination incorporated an intrafacility correlation coefficient for mortality, the primary outcome of the trial, as well as the number and size distribution of clusters. These factors increased the required number of participating patients by 28%.

Table 2. - TiME Trial design
Design Element Details
Treatment groups Intervention: dialysis facility adopts approach of prescribing hemodialysis session duration of ≥4.25 h
Usual Care: no trial-driven approach to hemodialysis session duration
Randomization Cluster-randomization by dialysis facility
Stratification by provider organization, facility race distribution, and facility catheter use
Target sample size 6432 patients
402 clusters
Eligibility criteria – patients Age ≥18 yr
Initiation of maintenance hemodialysis within the past 120 d
Ability to provide consent for dialysis
Currently treated with in-center hemodialysis
Eligibility criteria – facilities Agreement by facility leadership and nephrologists to adopt trial intervention
Capacity to accommodate treatment session durations of ≥4.25 h for incident patients
Duration of intervention Median 2.5 yr
Source of data Dialysis provider organization electronic data systems
Primary outcome Mortality
Major secondary outcomes Hospitalizations
Quality of life
Consent approach Opt-out
Analysis Primary: intention to treat, patient-level
Secondary: incorporation of intervention adherence
Oversight Single IRB of Record
External Data and Safety Monitoring Board

Partnerships between Researchers and Health Systems

Because pragmatic trials are intentionally integrated into clinical care delivery, health care providers rather than researchers carry out the research protocol. The incorporation of research into care delivery has been described as a “learning health system,” a term reflecting the goals of systematically learning from clinical care encounters and developing models for implementation that are sustainable from one learning activity (e.g., a clinical trial) to another.17–19 In a learning health system, partnerships between researchers and the health care delivery organizations are critical, especially if the goal is to accommodate multiple intervention trials concurrently and over time. Thus, whereas the principal stakeholders for explanatory trials are the investigators, sponsors, and participants, for pragmatic trials health system administrators and clinicians are also key stakeholders.20

Harmonizing interests of investigators and providers is essential for the success of pragmatic trials but can be challenging. Priorities for research questions often differ between investigators and health system leaders.20 Leaders of dialysis organizations, like those of other health care systems, are interested in evaluating interventions that add long-term value or address challenges in care delivery, payer targets, or regulatory requirements.21 In contrast, academic investigators might be more likely to prioritize studies that inform clinical practice guidelines. Additionally, dialysis providers favor studies that provide answers quickly in order to adopt approaches rapidly on the basis of the results and minimize the effect of the trial on competing initiatives. Quality improvement (QI) initiatives are often appealing to health systems as a way to systematically implement what are thought to be best practices. Incorporating features of pragmatic clinical trial design into QI programs allows the effectiveness of such practices to be evaluated with scientific rigor. For example, the use of facility-level randomization for a hemodialysis central venous catheter care QI initiative allowed a dialysis provider organization to not only implement the clinical care approach but to assess its effect on infection outcomes.22

For the TiME Trial, the academic investigators and dialysis provider organizations, Fresenius Medical Care and DaVita, partnered early in the planning stages, before obtaining funding, to develop the research question and implementation approach. Members of the research divisions at both dialysis companies are voting members of the TiME Trial Steering Committee, and the work of implementing the trial is shared by the group. Although engagement by the leadership of the dialysis organizations has been present throughout planning and implementation, one of the more challenging aspects for the TiME Trial has been developing approaches for direct interaction between the researchers and the clinicians at individual dialysis units. Engaging “on-the-ground” clinicians, a frequently reported challenge for large pragmatic trials,23,24should be overcome more readily if clinical trials become routinely incorporated into the delivery of clinical care.25,26

Selection of Trial Sites

Criteria for site selection in pragmatic clinical trials will vary depending on the study design, particularly when cluster randomization is used. Characteristics of clinical facilities can markedly affect the likelihood of successful participant recruitment, protocol implementation, and delivery of information to staff and patients. In the dialysis setting, although it is important to evaluate dialysis units individually, there are efficiencies in collaborating with provider organizations that comprise large networks of facilities that can serve as research sites. Such collaborations streamline the creation of clinical trial agreements and allow for a uniform approach to trial implementation by utilizing existing standardized practices, policies, and operating procedures.12

For individual participating dialysis units, stable structure and function are important for successful implementation of a pragmatic trial that relies on facility personnel for its implementation. Prioritization of study sites can be informed by preliminary assessments of the number of eligible patients, strong leadership from both the medical director and clinical manager, and a track record of compliance with policies and procedures. Review of ongoing clinical trials and QI initiatives within candidate facilities for overlap or conflict with the interventions or effect on the intended outcomes is also important. For the TiME Trial, the initial identification and evaluation of candidate facilities was not performed by the academic investigators but rather by the dialysis organization research teams in collaboration with regional administrators who had the necessary knowledge about the facility dynamics and capacity for trial implementation.

Adherence by Sites and Participants

An important benefit of conducting trials in partnership with health care delivery organizations is the ability to leverage existing systems for trial implementation. Dialysis provider organizations have well developed systems for communicating with large numbers of geographically disparate facilities, launching new initiatives, and providing feedback about performance. These systems can be used to motivate facility adherence to a trial protocol. For the TiME Trial, dialysis units randomized to the Intervention group receive regular facility-specific reports about adherence to the trial intervention. At the individual participant level, approaches used routinely by dialysis personnel to provide feedback about clinical targets (e.g., regular distribution of “patient report cards”) can be leveraged to promote adherence to the trial intervention.

Data Acquisition

Compared with many other health care delivery systems, dialysis has the advantage of highly standardized clinical data collection despite the large numbers of facilities and treating clinicians. Dialysis units maintain information systems with granular patient-specific demographic and clinical data. These systems consolidate the day-to-day flow of laboratory, treatment, dialysis machine, and operational data, most of which are stored as discrete data elements, rather than text fields.27 When using existing data systems for a trial conducted by multiple dialysis provider organizations, it is important to implement data specifications appropriate for the analytic requirements of the trial. Quality measure development and specification information, posted by the Centers for Medicare and Medicaid Services for its purposes, may be applicable for other uses such as data extraction for clinical trials.28

For the TiME Trial, prespecified data elements are transmitted from the data warehouses of DaVita Healthcare Partners and Fresenius Medical Care to the data coordinating center at the University of Pennsylvania. Compared with more conventional trials that rely on study-specific primary data collection, the approach used by the TiME Trial is highly efficient, inexpensive, and less prone to errors associated with manual data entry. Relying exclusively on data obtained by dialysis facilities has important limitations, one of which is the lack of information about events that occur outside of the dialysis facility, but some of these can be reduced through supplementation with other sources such as claims data.29,30

Informed Consent

Obtaining informed consent for large pragmatic trials can pose challenges if: (1) study implementation is highly centralized without research personnel at the study sites, (2) cluster-wide implementation of the intervention is required for either scientific or operational reasons, or (3) the consent requirement will lead to imbalances in participant characteristics across treatment groups (as might occur if the randomized treatment arm for the cluster is known before enrollment of individual participants). For research that is subject to the Department of Health and Human Services (HHS) regulations, waiving or modifying the informed consent process may be permitted if specific criteria are met, one of which is that the research presents no more than minimal risk of harm to participants.31 Although it is intuitively clear that the minimal risk requirement limits the interventions that can be studied when using trial designs that necessitate waiving or modifying consent, it is important to recognize that there is variability across institutional review boards in interpreting minimal risk and active debate among ethicists about how risks should be assessed.32 Additionally, revisions currently under consideration to the federal regulations for human subjects research protection (referred to as the Common Rule), have implications for the concept of minimal risk and how consent is approached in pragmatic trials.33 The TiME Trial met the criteria for waiving consent and is using an opt-out approach for trial participation in combination with an Institutional Review Board (IRB)-approved waiver of Health Insurance Portability and Accountability Act Authorization.34

Specific additional FDA regulations regarding informed consent apply to clinical investigations conducted under FDA oversight. Trials investigating drugs or devices (even if determined to be exempt from Investigational New Drug [IND] or Investigational Device Exemption [IDE] requirements, respectively), must adhere to the FDA regulations directed at human subjects protections, including informed consent requirements as detailed in the Code of Federal Regulations (CFR) 21 CFR 50, and the requirements for IRBs (21 CFR 56).35,36 Unlike the Common Rule, no waivers exist for obtaining informed consent for clinical investigations regulated by FDA except when participants are in life-threatening situations in which consent is not feasible.37 However, a provision does exist that allows an IRB to waive the documentation of informed consent for certain types of minimal risk research.38 Although the TiME Trial is not investigating a drug or device, and thus does not fall under FDA purview, this provision would likely allow for the opt-out consent process used in the TiME Trial and has also facilitated the conduct of pragmatic trials investigating FDA-regulated products.31

Institutional Review Boards

For all multicenter studies, whether pragmatic or explanatory, oversight by numerous local IRBs can pose challenges.39–43 Variability across institutions in approaches and interpretation of regulations are important contributors to the slow pace and expense of large trials, and it has been argued that review by multiple IRBs may paradoxically reduce protections to participants.44 For pragmatic trials, particularly those that use cluster-wide implementation of an intervention, variability across institutions in the interpretation of regulations, such as the criteria for waiving informed consent, can result not only in delays but also in irresolvable impasses. Moreover, for pragmatic trials in which local clinicians implement the intervention but researchers are not directly affiliated with the sites, reporting and responding to local IRBs is not practical. For the TiME Trial, the University of Pennsylvania IRB is serving as the IRB of record, and authorization agreements were created between each dialysis provider organization and the University of Pennsylvania IRB. Although the use of central IRBs or single IRBs of record is becoming increasingly common, willingness of institutions to utilize these models varies, and concerns have been raised about eliminating local context from the review process.43,45 However, both a recent draft policy by the NIH and proposed revisions to the Common Rule by HHS and other federal bodies promote the use of a single IRB for multisite studies, suggesting that in the near future this approach may become the norm rather than the exception.33,46 Of note, the FDA has made clear in guidance documents that it supports the use of a central IRB for drug studies47; however, the Federal Food, Drug, and Cosmetic Act requires local IRB review for device studies.48

Data and Safety Monitoring Committees and Adverse Event Reporting

Clinical trials, particularly those with safety risks, often have Data and Safety Monitoring Boards (DSMBs) to evaluate safety, efficacy, and quality of trial conduct while the study is underway. Although external monitoring is usually desirable for pragmatic trials, design or implementation features might necessitate an approach by the DSMB that differs from that used for most explanatory trials.49 For example, the TiME Trial DSMB does not have individual serious adverse event reports available for review but instead relies on prespecified, clinically acquired laboratory data or trial outcomes to assess safety. Additionally, in the interest of understanding real-world experience, a pragmatic trial DSMB might focus little attention on monitoring adherence to the intervention. On the other hand, for an open-label, cluster-randomized pragmatic trial in which group assignment is known before participant enrollment, a DSMB might be highly interested in comparing participant characteristics across treatment groups to assess selection bias. Although there are not regulatory requirements for DSMBs, efforts currently underway to generate consensus documents about external monitoring will presumably address different types of trials including pragmatic trials.50

FDA IND or IDE Provisions

There are perceptions that trials conducted under IND or IDE provisions cannot use highly pragmatic design or implementation approaches because of data collection and safety reporting requirements51,52; however, this is not necessarily the case. For example, questions have been raised about whether the electronic clinical data systems used by dialysis providers meet the FDA regulations on electronic records described in 21 CFR part 11. FDA guidance that describes how electronic source data should be captured, reviewed, and retained specifically states that the FDA does not intend to assess the “part 11 compliance” of the computer system (e.g., the electronic medical record) used in a clinical investigation.53 Notwithstanding the guidance recommendations, additional clarification from regulatory authorities on proper approaches to handling electronic data from dialysis information systems could encourage sponsors to pursue pragmatic clinical trials of drugs and devices used in dialysis. Another challenge for pragmatic trials conducted under IND or IDE provisions is determining who among the many individuals implementing the trial meets the regulatory definition of “investigator.”42 Because this designation carries specific responsibilities for both sponsors and investigators, the extent to which pragmatic trials can efficiently increase the evidence base will depend on whether clinicians operating in a learning health system are subject to such regulatory requirements.54 From a safety standpoint, FDA requirements for adverse event reporting might be a barrier to pragmatic trials that rely on electronic medical record data rather than investigators’ reports of individual events. However, both the regulations themselves and guidance documents make it clear that the FDA may waive these requirements if certain criteria are met.55,56 Thus, for pragmatic trials comparing well established treatments, alternative approaches to monitoring patient safety proposed by the sponsor might be deemed acceptable by the FDA.

Moving Forward

The use of pragmatic clinical trials to fill the large gaps in our knowledge about caring for patients receiving maintenance dialysis is highly appealing and there are several fundamental questions that, at least at first glance, appear to lend themselves well to this approach. Examples include identifying optimal targets for serum phosphorus concentration, strategies for anemia and iron management, and electrolyte composition of dialysis solutions. However, in order for the full potential of pragmatic trials to be realized there are important challenges that require focused attention (Table 3). Some of these are operational or technical – e.g., the lack of efficient systems for engaging and communicating with “on-the-ground” nephrologists at dialysis units, and limited access to clinical information from care delivered outside of the dialysis unit. Others are cultural or philosophical, such as reluctance by some IRBs to cede oversight to another institution, or differing views about acceptable approaches to adverse event monitoring for trials evaluating treatments already in widespread use. Ethical challenges include reconciling tensions between providing benefits to large populations and maintaining protections for individuals. And, because of the lack of distinction between full and modified informed consent, unless the unmodified informed consent process can be operationalized in large trials incorporated into clinical care delivery, regulatory requirements limit the interventions that can be studied to those that pose minimal risk. Despite these challenges, there is tremendous interest, both nationally and globally, in increasing the momentum for pragmatic trials, and funders of research, including industry sponsors, are increasingly embracing this approach, in part to reduce cost, but also to generate findings that are rapidly translatable to practice.57–64 As more of these trials are conducted in dialysis, in other areas of nephrology, and in medicine more broadly, approaches to overcome operational and technical barriers will continue to be identified, the views of potential research participants on consent approaches and data sharing will be better understood, and perhaps, over a more extended time frame, revisions will be made to current regulations and policies that will expand the scope of questions that can be answered.

Table 3. - Increasing the impact of pragmatic trials
Domain Areas for Focused Efforts
Operational and technical Optimizing systems for interacting with clinicians at health care delivery sites, particularly for nonemployee physicians
Developing approaches to minimize disruptions to clinical work flow by research activities
Developing strategies for handling complex data extraction, creating systems interoperability, addressing missing data, establishing computable phenotypes, and incorporating documentation tools into electronic medical records
Cultural and philosophical Elevating interest by health system leaders, clinicians, and patients in incorporating research into the clinical care setting
Addressing privacy concerns of providers and patients
Harmonizing varied interpretations of research-related risks
Addressing resistance by institutions to the use of a single IRB of record
Refining approaches to data and safety monitoring
Ethical Recognizing tradeoffs between improving outcomes for populations and protecting rights of individuals
Identifying appropriate “gatekeepers” for access to participants
Clarifying the importance of downstream and indirect effects of research on nonparticipants
Regulatory Addressing the lack of distinction by the Common Rule between modifying and waiving informed consent
Addressing adherence to regulations regarding vulnerable populations
Harmonizing relevant Common Rule and FDA regulations
Clarifying acceptable approaches to use of electronic data for trials under IND and IDE provisions
Clarifying options for safety reporting for trials under IND and IDE provisions
Addressing the requirement for research indicator for Medicare claims
Clarifying distinctions between research and quality improvement
Funding Increasing collaborations between academic investigators and health systems to identify and maximize opportunities for incorporating trials into health care delivery
Educating grant reviewers about unique design or implementation considerations for pragmatic trials


E.L.Jr. was an employee of Fresenius Medical Care (Waltham, MA) when this project was initiated. M.K. is employed by and owns stock in DaVita Healthcare Partners.

Published online ahead of print. Publication date available at

This work was supported by the Kidney Health Initiative (KHI), a public-private partnership between the American Society of Nephrology, the US Food and Drug Administration, and >75 member organizations and companies to enhance patient safety and foster innovation in kidney disease. KHI funds were used to defray costs incurred during the conduct of the project, including project management support which was expertly provided by American Society of Nephrology staff members, Melissa West and Ryan Murray. There was no honorarium or other financial support provided to KHI workgroup members. The authors of this paper had final review authority and are fully responsible for its content.

KHI makes every effort to avoid actual, potential, or perceived conflicts of interest that may arise as a result of industry relationships or personal interests among the members of the workgroup. More information on KHI, the workgroup, or the conflict of interest policy can be found at The TiME Trial is funded by the National Institutes of Health Office of the Director and the National Institute of Diabetes and Digestive and Kidney Diseases (UH2-AT007797 and UH3-DK102384 to L.M.D.). The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of any agency of the US Government, DaVita Healthcare Partners, or Fresenius Medical Care.


1. Levin A, Lancashire W, Fassett RG: Targets, trends, excesses, and deficiencies: refocusing clinical investigation to improve patient outcomes. Kidney Int 83: 1001–1009, 201323515054
2. Archdeacon P, Shaffer RN, Winkelmayer WC, Falk RJ, Roy-Chaudhury P: Fostering innovation, advancing patient safety: the kidney health initiative. Clin J Am Soc Nephrol 8: 1609–1617, 201323744001
3. Strippoli GF, Craig JC, Schena FP: The number, quality, and coverage of randomized controlled trials in nephrology. J Am Soc Nephrol 15: 411–419, 200414747388
4. Palmer SC, Sciancalepore M, Strippoli GF: Trial quality in nephrology: how are we measuring up? Am J Kidney Dis 58: 335–337, 201121856491
5. Califf RM, Zarin DA, Kramer JM, Sherman RE, Aberle LH, Tasneem A: Characteristics of clinical trials registered in, 2007-2010. JAMA 307: 1838–1847, 201222550198
6. Inrig JK, Califf RM, Tasneem A, Vegunta RK, Molina C, Stanifer JW, Chiswell K, Patel UD: The landscape of clinical trials in nephrology: a systematic review of Am J Kidney Dis 63: 771–780, 201424315119
7. Eisenberg JM: Physician utilization: the state of research about physicians’ practice patterns. Med Care 40: 1016–1035, 200212409848
8. Grol R, Buchan H: Clinical guidelines: what can we do to increase their use? Med J Aust 185: 301–302, 200616999667
9. McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA: The quality of health care delivered to adults in the United States. N Engl J Med 348: 2635–2645, 200312826639
10. Tunis SR, Stryer DB, Clancy CM: Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA 290: 1624–1632, 200314506122
11. Zwarenstein M, Oxman A; Pragmatic Trials in Health Care Systems (PRACTIHC): Why are so few randomized trials useful, and what can we do about it? J Clin Epidemiol 59: 1125–1126, 200617027421
12. Treweek S, Zwarenstein M: Making trials matter: pragmatic and explanatory trials and the problem of applicability. Trials 10: 37, 200919493350
13. Kidney Health Initative. Available at: Accessed October 15, 2015
14. NIH Health Care Systems Research Collaboratory: UH3 Project: Time to Reduce Mortality in End-Stage Renal Disease (TiME), 2015. Available at: Accessed October 15, 2015
15. Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, Chalkidou K: A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol 62: 464–475, 200919348971
16. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M: The PRECIS-2 tool: designing trials that are fit for purpose. BMJ 350: h2147, 201525956159
17. Institute of Medicine: The Learning Healthcare System: Workshop Summary (IOM Roundtable on Evidence-Based Medicine), Washington, D.C., The National Academies Press, 2007. doi:10.17226/11903
18. Greene SM, Reid RJ, Larson EB: Implementing the learning health system: from concept to action. Ann Intern Med 157: 207–210, 201222868839
19. Institute of Medicine: Best care at lower cost: The path to continuously learning health care in America, Washington, D.C., The National Academies Press, 2013.
20. Johnson KE, Tachibana C, Coronado GD, Dember LM, Glasgow RE, Huang SS, Martin PJ, Richards J, Rosenthal G, Septimus E, Simon GE, Solberg L, Suls J, Thompson E, Larson EB: A guide to research partnerships for pragmatic clinical trials. BMJ 349: g6826, 201425446054
21. Krishnan M, Brunelli SM, Maddux FW, Parker TF 3rd, Johnson D, Nissenson AR, Collins A, Lacson E Jr: Guiding principles and checklist for population-based quality metrics. Clin J Am Soc Nephrol 9: 1124–1131, 201424558050
22. Rosenblum A, Wang W, Ball LK, Latham C, Maddux FW, Lacson E Jr: Hemodialysis catheter care strategies: a cluster-randomized quality improvement initiative. Am J Kidney Dis 63: 259–267, 201424295613
23. DeVoe JE, Likumahuwa S, Eiff MP, Nelson CA, Carroll JE, Hill CN, Gold R, Kullberg PA: Lessons learned and challenges ahead: report from the OCHIN Safety Net West practice-based research network (PBRN). J Am Board Fam Med 25: 560–564, 201222956690
24. Ivers NM, Tu K, Young J, Francis JJ, Barnsley J, Shah BR, Upshur RE, Moineddin R, Grimshaw JM, Zwarenstein M: Feedback GAP: pragmatic, cluster-randomized trial of goal setting and action plans to increase the effectiveness of audit and feedback interventions in primary care. Implement Sci 8: 142, 201324341511
25. Fagnan LJ, Handley MA, Rollins N, Mold J: Voices from left of the dial: reflections of practice-based researchers. J Am Board Fam Med 23: 442–451, 201020616286
26. Gilbert GH, Richman JS, Gordan VV, Rindal DB, Fellows JL, Benjamin PL, Wallace-Dawson M, Williams OD; DPBRN Collaborative Group: Lessons learned during the conduct of clinical studies in the dental PBRN. J Dent Educ 75: 453–465, 201121460266
27. Krishnan M, Wilfehrt HM, Lacson E Jr: In data we trust: the role and utility of dialysis provider databases in the policy process. Clin J Am Soc Nephrol 7: 1891–1896, 201222977218
28. Centers for Medicare & Medicaid Services Measures Management System. Available at: Accessed March 3, 2016
29. Goldstein BA, Chang TI, Mitani AA, Assimes TL, Winkelmayer WC: Near-term prediction of sudden cardiac death in older hemodialysis patients using electronic health records. Clin J Am Soc Nephrol 9: 82–91, 201424178968
30. Winkelmayer WC, Patrick AR, Liu J, Brookhart MA, Setoguchi S: The increasing prevalence of atrial fibrillation among hemodialysis patients. J Am Soc Nephrol 22: 349–357, 201121233416
31. U.S. Department of Health and Human Services: “Code of Federal Regulations - Title 45 Public Welfare CFR 46.” Available at: Accessed March 3, 2016
32. Lantos JD, Wendler D, Septimus E, Wahba S, Madigan R, Bliss G: Considerations in the evaluation and determination of minimal risk in pragmatic clinical trials. Clin Trials 12: 485–493, 201526374686
33. Proposed Rule: Federal Policy for the Protection of Human Subjects, 2015. Available at: Accessed March 3, 2016
34. McKinney RE Jr, Beskow LM, Ford DE, Lantos JD, McCall J, Patrick-Lake B, Pletcher MJ, Rath B, Schmidt H, Weinfurt K: Use of altered informed consent in pragmatic clinical research. Clin Trials 12: 494–502, 201526374677
35. Electronic Code of Federal Regulations: 21 CFR 50. Available at: Accessed March 3, 2016
36. Electronic Code of Federal Regulations: 21 CFR 56. Available at: Accessed March 3, 2016
37. Electronic Code of Federal Regulations: 21 CFR 50. 24 Available at: Accessed March 3, 2016
38. Electronic Code of Federal Regulations: 21 CFR 56.109. Available at: Accessed March 3, 2016
39. Abramovici A, Salazar A, Edvalson T, Gallagher N, Dorman K, Tita A. Review of multicenter studies by multiple institutional review boards: characteristics and outcomes for perinatal studies implemented by a multicenter network. Am J Obstet Gynecol 212: 110 e1–110e6, 2015
40. Higgerson RA, Olsho LE, Christie LM, Rehder K, Doksum T, Gedeit R, Giuliano JS Jr, Brennan B, Wendlandt R, Randolph AG; PALISI PICFlu Study Investigators: Variability in IRBs regarding parental acceptance of passive consent. Pediatrics 134: e496–e503, 201425002659
41. Polito CC, Cribbs SK, Martin GS, O’Keeffe T, Herr D, Rice TW, Sevransky JE: Navigating the institutional review board approval process in a multicenter observational critical care study. Crit Care Med 42: 1105–1109, 201424368345
42. O’Rourke PP, Carrithers J, Patrick-Lake B, Rice TW, Corsmo J, Hart R, Drezner MK, Lantos JD: Harmonization and streamlining of research oversight for pragmatic clinical trials. Clin Trials 12: 449–456, 201526374678
43. Flynn KE, Hahn CL, Kramer JM, Check DK, Dombeck CB, Bang S, Perlmutter J, Khin-Maung-Gyi FA, Weinfurt KP: Using central IRBs for multicenter clinical trials in the United States. PLoS One 8: e54999, 201323383026
44. Menikoff J: The paradoxical problem with multiple-IRB review. N Engl J Med 363: 1591–1593, 201020942660
45. Loh ED, Meyer RE: Medical schools’ attitudes and perceptions regarding the use of central institutional review boards. Acad Med 79: 644–651, 200415234914
46. Request for Comments on the Draft NIH Policy on the Use of a Single Institutional Review Board for Multi-Site Research, 2015, Available at: Accessed October 15, 2015
47. Guidance for Industry: Using a Centralized IRB Review Process in Multicenter Clinical Trials. Available at: Accessed March 3, 2016
48. U.S.C. Federal Food, Drug, and Cosmetic Act, Subchapter V. Available at: Accessed March 3, 2016
49. Ellenberg SS, Culbertson R, Gillen DL, Goodman S, Schrandt S, Zirkle M: Data monitoring committees for pragmatic clinical trials. Clin Trials 12: 530–536, 201526374679
50. Clinical Trials Transformation Initiative: DMCs Project. Available at: Accessed March 3, 2016
51. Electronic Code of Federal Regulations 21 CFR 312.64. Available at: Accessed March 3, 2016
52. Electronic Code of Federal Regulations 21 CFR 312.32. Available at: Accessed March 3, 2016
53. Guidance for Industry: Electronic Source Data in Clinical Investigations. Available at: Accessed March 3, 2016
54. Electronic Code of Federal Regulations: 312 CFR Subpart D. Available at: Accessed March 3, 2016
55. Electronic Code of Federal Regulations 21 CFR 312.10. Available at: Accessed March 3, 2016
56. Guidance for Industry: Determining the Extent of Safety Data Collection Needed in Late Stage Premarket and Postapproval Clinical Investigations: Available at: Accessed March 3, 2016
57. NIH Health Care Systems Research Collaboratory: Available at: Accessed March 3, 2016
58. PCORnet, the National Patient-Centered Clinical Research Network. Available at: Accessed March 3, 2016
    59. Bakerly ND, Woodcock A, New JP, Gibson JM, Wu W, Leather D, Vestbo J: The Salford Lung Study protocol: a pragmatic, randomised phase III real-world effectiveness trial in chronic obstructive pulmonary disease. Respir Res 16: 101, 201526337978
    60. James S, Rao SV, Granger CB: Registry-based randomized clinical trials--a new clinical trial paradigm. Nat Rev Cardiol 12: 312–316, 201525781411
    61. Patient Centered Outcomes Research Institute: Pragmatic Clinical Studies to Evaluate Patient-Centered Outcomes. Available at: Accessed May 22, 2016
      62. NIH HCS Research Collaboratory. Available at: Accessed May 22, 2016
        63. Selby P, Brosky G, Oh P, Raymond V, Arteaga C, Ranger S: A pragmatic, randomized, controlled study evaluating the impact of access to smoking cessation pharmacotherapy coverage on the proportion of successful quitters in a Canadian population of smokers motivated to quit (ACCESSATION). BMC Public Health 14: 433, 201424885542
        64. Koehler M, Donnelly ET, Kalanovic D, Dagher R, Rothenberg ML: Pragmatic randomized clinical trials: a proposal to enhance evaluation of new cancer therapies with early signs of exceptional activity. [published online ahead of print March 30, 2016] Ann Oncol doi: 10.1093/annonc/mdw143

        cluster randomization; health systems; informed consent; electronic medical record; ethical and regulatory issues

        Copyright © 2016 by the American Society of Nephrology