Advances in transplantation medicine have led to substantial improvement in short and intermediate outcomes in kidney transplant recipients worldwide with a 1-year graft survival rate exceeding 95% in some countries.1-5 However, major hurdles to the success of kidney transplantation persist. Long-term graft survival has remained largely unchanged over the past few decades.2,6-11 For recipients with a functioning graft 1 year after transplant, their 10-year graft survival is estimated to be 50% to 70%.6,10,12,13 Furthermore, long-term immunosuppression leads to adverse outcomes including cancer, cardiovascular disease, diabetes, and infection—all major causes of premature death in recipients.8,11,14-18 In this complex setting, decision making should be informed by clinical trials that measure and report outcomes of relevance to patients and clinicians.
Unfortunately, many trials in kidney transplantation are short-term, focused on immunosuppression only, and frequently use surrogate endpoints, such as serum creatinine and glomerular filtration rate, because they require a smaller sample size and less time and resources to determine treatment efficacy.19 However, these outcomes are largely un-validated, ill defined, vary widely across trials, and may not be meaningful or directly relevant to patients.19-23 Such problems have important implications because patients and clinicians make treatment decisions that require individual-level and multiple tradeoffs between graft survival and the risk of side effects, adverse events, and mortality.20,24 Regulatory agencies may also be required to approve products on the basis of unvalidated and/or short-term surrogate endpoints.19 In recent years, there has been a surge in the number of biomarkers used to define acute rejection, delayed graft function, progression of native kidney disease, and late allograft injury,25-30 but the problem of validation has largely remained.6,29
There is substantial heterogeneity in the myriad of outcomes in trials in kidney transplantation, and incomplete reporting of outcomes can render estimates of treatment effect unreliable. Findings from a recent systematic review of trials in kidney transplantation indicated that only 79% provided complete reports of death, less than half reported time to death, and complete reports for graft function were infrequent with 35% and 28% trials reporting estimated glomerular filtration rate and creatinine, respectively. Similarly, studies have shown that kidney function has been assessed using different outcome measures and at varying time points.31,32
As in most research, there is a tendency for trialists in kidney transplantation to measure and report outcomes for reasons of feasibility and efficiency rather than relevance to patients and clinicians. The mismatch between the outcomes of relevance to stakeholders, such as patients and clinicians, compared with what is routinely measured and reported is apparent across medical specialties including kidney transplantation. Although graft loss, mortality, cancer, diabetes, cardiovascular disease, and infection have been shown to be of greatest importance to patients,20,24 long-term studies in kidney transplantation still primarily measure and report surrogate endpoints, such as kidney function (serum creatinine, creatinine clearance) and graft rejection. These surrogate endpoints may not be associated with clinical outcomes.
Immunosuppressive medications are potentially harmful, increase the risk of comorbidities and may cause severe acute side effects that impair quality of life (QoL).24 These adverse outcomes amplify the treatment burden for patients and can potentially lead to nonadherence, which is associated with a 7-fold increase in the risk of graft failure.33,34 Yet, only 2% of trials of immunosuppression in kidney transplantation report QoL outcomes, and 95% of these QoL outcomes favored the intervention, strongly suggestive of a major outcome reporting bias.21 Reliable evidence for treatment decision-making based on QoL outcomes is lacking in kidney transplantation, and the inability to address patient priorities and concerns can contribute to nonadherence.
The problems with outcome selection and reporting are prevalent and increasingly recognized as a major cause of inefficiency and waste in biomedical research across all medical disciplines.35 Many initiatives have been established worldwide to develop core outcome sets—an agreed minimum set of standardized outcomes that should be measured and reported in all trials for a specific clinical area (Figure 1).36-39 Core outcome sets can minimize reporting bias, promote consistency across trials to enable direct comparisons of the effect of different intervention, and ensure outcomes are relevant and important to patients and health professionals who make health care decisions.36,37,39,40 The Outcome Measures in Rheumatology (OMERACT) consensus initiative commenced in 1992 and has been at the forefront in engaging patients, providers, and policy makers to develop core outcome sets for clinical trials in rheumatology that has helped to improve the completeness of outcome reporting in trials.41–43 Then, in 2010, the Core Outcome Measures for Effectiveness Trials was launched to foster the development and implementation of core outcome sets.37,40
In kidney transplantation, there is increasing recognition of the need to develop core outcomes spurring some efforts to review outcomes used in trials in kidney transplantation.22,44-46 The Standardized Outcomes in Nephrology (SONG) initiative was recently formed to establish core outcomes across the full spectrum of chronic kidney disease with an initial focus on hemodialysis.47 The SONG-Tx project aims to establish a core outcome set for trials in kidney transplantation and other types of research including observational studies (eg, registries, quality indicators for clinical care).
MATERIALS AND METHODS
The SONG-Tx project involves 3 phases: systematic review, international online Delphi survey, and a consensus workshop (Figure 2). The methods detailed below are adapted from the processes used in SONG-Hemodialysis, and the OMERACT48 methodological framework that has been recognized by the World Health Organization49 as a valid approach for developing core outcomes.
Phase 1: Systematic Review of Outcome Domains Reported in Randomized Controlled Trials of Interventions for Kidney Transplant Recipients
A systematic review will identify and compare outcome domains and measures reported in randomized controlled trials (RCTs) of interventions for adult kidney transplant recipients.
We will search the Cochrane Database of Systematic reviews to identify all systematic reviews that included at least 1 RCT involving adult kidney transplant recipients. We will retrieve the full text article of all RCTs included in the systematic review, and also search for all additional articles published from the same trial using key word searches in electronic databases (MEDLINE, Embase) and trial registries. No date or language restrictions will be applied.
Types of Studies
For feasibility, the Cochrane Database of Systematic reviews will be used as a sampling frame to identify RCTs in kidney transplantation, as has been done in previous analysis of outcomes reporting in RCTs.22,50 Based on a preliminary search, we expect that more than 300 RCTs will be included.
Types of Interventions
Any intervention (including but not limited to surgical, pharmacological [immunosuppression, antibiotics, antifungal, antivirals, diuretics, statins, vaccines], psychosocial and lifestyle) for kidney transplant recipients will be eligible.
Types of Participants
Adult kidney transplant recipients (aged ≥ 18 years).
Randomized controlled trials that included other solid organ or tissue transplant recipients (eg, lung, liver, heart, pancreas, bone marrow) will be excluded as our primary focus is on kidney transplantation.
Eligibility of Studies
Two reviewers (B.S., N.E.) will independently assess all records obtained from the searches. Full texts of all potentially relevant systematic reviews and RCTs will be assessed independently by the 2 reviewers, and any disagreement on the eligibility of included studies will be resolved through consultation with a third reviewer (A.T.).
One reviewer (B.S.) will extract characteristics of all the trials including: first author, date of publication, country/ies in which the trial was conducted, sample size, participant characteristics (range and mean age, time since transplant, sex), trial duration, name and type of intervention (eg, surgical, pharmacological, psychosocial, lifestyle), and all outcomes as reported in the trial (including definitions, tools for measurement, thresholds, time points or time frames for measurement, change in level or percentage, scores). Two reviewers (N.E., A.T.) will cross check the data extraction.
Data Analysis and Presentation
The data will be entered into Microsoft Excel to assist with data management, tabulation, and analysis. Author B.S. will group all similar outcomes into appropriate outcome domains, which will be reviewed and discussed by the SONG-Tx Steering Group (J.R.C. [Chair], K.B., J.G., M.A.J., L.M., T.P.P., D.R., L.R., A.W., G.W.] and SONG Executive Committee (J.C.C., S.C., J.G., T.H., B.H., B.M., P.T., W.V.B., D.C.W., W.C.W., A.T.). The outcome domains will be broadly classified as surrogate, clinical, or patient-reported outcomes. We will identify the number of trials that reported each outcome domain. For each outcome domain, we will assess the number of different outcomes (including measures) and the number of trials that assessed each specific outcome and perform statistical analyses using the software package R (version 3.2.3).
Phase 2: International Online Delphi Consensus Survey
An international online Delphi survey will be conducted to generate consensus on the outcome domains that are most important to all stakeholder groups. The Delphi has been used to obtain reliable consensus on core outcome sets across a range of health conditions.51-54 This technique involves 2 or 3 rounds of surveys completed sequentially and anonymously by a panel of experts with experience or expertise on the topic38,55 and allows feedback of individual contributions of knowledge, information, and perspectives; assessment of the group view; and opportunities for respondents to revise their opinions.56 The controlled communication in this process minimizes direct confrontation and allows individual respondents to express independent thought and enables equitable contribution from all participants. This study has been approved by the Human Research Ethics Committee of The University of Sydney (2015-228).
Participants and Recruitment
There is no universal agreement on the “minimum” or appropriate sample size for a Delphi panel. The majority Delphi of studies in core outcome development have included less than 200 respondents. Given the large health professional and consumer networks in kidney transplantation and to achieve broad engagement internationally, our minimum target sample size will be 500 respondents with at least 250 patients with chronic kidney disease (either with a functioning graft, on dialysis having lost a kidney transplant, or waiting for a kidney transplant)/family members/living kidney donors) to ensure a sample size balance between patients/family members/donors and health professionals. We will aim to recruit nephrologists and surgeons (minimum n = 150); nursing and allied health professionals (n = 50); and policy makers (including regulators), researchers, and representatives from industry (n = 50); who have experience, expertise, or interest in kidney transplant recipient outcomes.
Purposive sampling to obtain a maximum variation in the demographics, clinical characteristics (patients), and professional experiences and roles (health professionals), and snowballing strategies (where participants can nominate or extend an invitation to other relevant stakeholder members to participate) will be used. Patients/family members will be recruited through participating hospital/university institutions of the SONG Executive, SONG-Tx Steering Group and investigators, patient/consumer organisations, and the SONG Initiative database. Health professionals will be recruited via the collegial networks of the investigators and professional transplantation and nephrology societies.
Generating the List of Outcomes
The Delphi survey will include outcome domains from the systematic review of outcomes reported in RCTs (phase 1). To ensure that patient-important outcomes are considered, we will also identify outcomes from studies that involved patients in prioritizing or reporting outcomes in kidney transplantation. The key studies that will be used to directly inform the Delphi are outlined in the following.
A recent nominal group technique study conducted with 57 kidney transplant recipients who participated in 8 nominal groups in Australia to identify and rank outcomes relevant to immunosuppressant medications.20 All outcomes identified will be considered for inclusion in the Delphi. The top 10 ranked outcomes overall were: kidney rejection leading to graft loss, kidney function, damage to other organs, cancer (all), diabetes, skin cancer, cardiovascular disease, infection, weight gain and excessive appetite, and in addition the top 5 psychosocial or physical symptoms were: impact on family, depression, impact on work, gastrointestinal problem, and concentration and memory.20 Potential outcomes for the Delphi survey will also be extrapolated from a systematic review of qualitative studies on the motivations, challenges, and attitudes to self-management in kidney transplant recipient (eg, graft function, graft rejection, alopecia, pain, anxiety, cardiovascular disease, diabetes, cancer, mouth ulcers, acne, cataracts, diarrhoea, physical appearance).57
A systematic review of patient-reported QoL outcomes in trials of immunosuppressive agents in kidney transplantations identified 2 kidney transplant-specific instruments (Kidney Transplant Questionnaire [physical symptoms, fatigue, uncertainty/fear, appearance, emotional status],58 and a visual analogue scale to assess impact of disease) and 21 symptom-specific instruments (eg, depression, gastrointestinal problems, physical appearance, physical well-being).21 Attributes from these will be used to inform the development of the list of outcomes for the Delphi.
All outcomes will include a plain language definition. The survey will be reviewed by the SONG Executive and SONG-Tx Steering Group, which includes kidney transplant recipients, and piloted with at least 10 kidney transplant recipients.
All participants must register their name and email address via www.songinitiative.org by March 2016 to receive a standard study information sheet. Informed consent will be obtained from all participants. The surveys will be completed online via LimeSurvey. Each participant will be given a unique identifier so their responses from rounds 1 to 3 can be linked anonymously. At least 3 reminders will be sent to the participants during the Delphi Rounds in an attempt to retain at least a 70% response rate across all 3 rounds. Participants who complete all 3 rounds will receive a copy of the preliminary results to provide feedback and comment.
Delphi Survey Round 1
Participants will be asked to rate the importance of outcome domain for research in kidney transplantation (approximately 35) using the GRADE 9-point Likert scale.59 The visual scale used for each outcome will indicate ratings 1 to 3 as “limited importance”; “4 to 6 as “important, but not critical”; and 7 to 9 as “critical importance.” An option “unsure” will also be provided. Responses to the rating questions will be mandatory. Participants can provide comments in a free-text box under each outcome. The sequence of outcomes shown will be randomized to minimize ordering bias. At the end of the round, participants can suggest new outcomes. All new outcomes that are suggested by more than 10% of participants and that do not overlap or duplicate already presented outcomes will be recoded by at least 2 investigators (N.E., B.S.) and reviewed by the SONG-Tx Steering Group, then carried through round 2.
Depending on distribution of scores across all outcomes, an outcome with a median and mean of more than 7 (with ≥ 70% of participants in both patient/family member/donor and health professionals rating the outcome 7-9; based on the OMERACT criteria for consensus in) will be retained in round 2. Any outcomes that were not retained in subsequent rounds will still be considered in the research agenda (Figure 1).
Delphi Survey Round 2:
In round 2, participants will be presented with a column graph of the distribution of scores for each outcome for the following groups: (1) patients/family members/donors, (2) health professionals, and (3) all participants (with scores weighted evenly between groups). An explanation of how to read the graph will be provided to ensure that participants can understand and interpret the graph clearly. They will also see comments from round 1 by patients/family members/donors and health professionals. In the rating scale, their own response from the previous round will be highlighted. Participants will rerate each outcome and any additional outcomes identified in round 1 using the same GRADE 9-point Likert scale. A free-text box will be provided for participants to explain reasons for their rating or to provide responses to the comments.
An outcome with a median and mean of more than 7 (with ≥ 70% of participants in both patient/family member/donor and health professionals rating the outcome 7-9) will be included in round 3.
Delphi Survey Round 3
In this final round, participants will see the distribution of scores for each outcome for all participants and by stakeholder groups and comments from round 2. Participants will see their own score from round 2 highlighted in the rating scale and rerate all outcomes. A free-text box will be provided for additional comments. In the final section of the survey, participants will be asked to complete a forced ranking exercise. They will be presented with a list of all outcomes to arrange into a list in order of importance (top being most important).
For all 3 rounds, we will summarize the distribution of scores and calculate the mean, median, and proportion for ratings and rankings of each outcome. According to the OMERACT prespecified definition of “consensus” for outcomes to be included in the core set, the outcome must have at least 70% participants scoring 7 to 9 and less than 15% of participants scoring as 1 to 3.48 Based on previous initiative, a core set includes 3 to 5 outcomes for feasibility. However, it is possible that more than 5 outcomes will meet the OMERACT threshold for inclusion. Therefore, the identification of preliminary core outcome domains will also be based on means, medians, and proportions in round 3, which will be validated against the OMERACT cutoffs based on proportions. Because it may be necessary to define consensus post hoc to some extent, the preliminary outcomes with the rationale and threshold for inclusion will be detailed in a plain language report. This will be discussed at the consensus workshop (phase 3).
Phase 3: Consensus Workshop
A consensus workshop will be convened with stakeholders to comment and critique the identification of core outcomes (including the potential set of core outcomes) and to discuss strategies for the development of outcome measures and implementation. A member of the SONG-Tx Steering Group will chair the session. We aim to have a minimum of 60 attendees, with at least 20 patients/family members. Health professionals with a range of clinical experience in kidney transplantation (nephrologists, surgeons, nursing, and allied health professionals), expertise in research (epidemiology, clinical trials in kidney transplantation, registries, quality improvement), and leadership or advisory roles in major research and policy organizations (including regulators) and industry will be invited to attend.
Before the workshop, we will send participants a copy of the results from phase 1 and phase 2, so they can reflect on the results to date and be better prepared to contribute their opinions during the workshop. The workshop will include 3 sessions:
- Session 1: Introduction
- We will provide a brief introduction to the SONG-Tx initiative and present the details of the SONG-Tx process and results from phase 1 and phase 2 and the preliminary core outcome set and proposed threshold for inclusion.
- Session 2: Breakout groups
- Participants will be allocated to 1 of 5 breakout groups with up to 12 participants in each group (including a facilitator and cofacilitator). Mixed stakeholder groups with at least 2 patients/family members will be convened to encourage a richer exchange of ideas, explanations of similar or different opinions, and breadth of discussion. A trained facilitator will ask participants to discuss the identification and implementation of core outcomes, and ensure cooperative, respectful and inclusive discussion. All facilitators will attend a briefing session and will be provided with a question guide.
- Session 3: Plenary discussion
- The groups will reconvene to engage in a broader discussion moderated by the workshop chair. A member from each breakout group will present a brief summary of their discussion. The wide group will be invited to give their opinions, reflections on the issues raised by other groups. The moderator will summarize key similarities and differences in the points raised across groups.
Finalization of Core Outcome Domains
The SONG-Tx process (phase 1 to phase 3) and proposed outcomes will be published in a plain language report for circulation to the participants in the Delphi (phase 2) and consensus workshop (phase 3), circulated to stakeholder groups and made available on the website for 3 weeks to obtain public comment. All feedbacks will be reviewed by the SONG-Tx Steering Group and SONG Executive Committee to finalize the SONG-Tx core outcome set.
The SONG-Tx project uses a validated and transparent process that enables broad stakeholder engagement to develop a consensus-based prioritized set of core outcome domains for trials in kidney transplantation that can also be applied to other research contexts including registry studies and quality measurement activities. For each of the core outcome domains, we will identify or develop core outcome measures using the OMERACT filter to ensure that they are valid, discriminative, and feasible.60 We will develop and apply implementation strategies in consultation with key stakeholders so that the core outcomes will be progressively translated into all trials and research.
Consistent and complete reporting of outcomes that are highly important to all stakeholder groups is likely to improve the quality and relevance of the research evidence to better support patients and clinicians in making informed treatment decisions in kidney transplantation. The core outcome set may also serve as a catalyst in driving the research agenda to focus on outcomes that really matter to patients and health care providers.
The SONG-Tx initiative is endorsed by The Transplantation Society. The authors also want to acknowledge the following organizations for their partnerships and support: Cochrane Kidney and Transplant, Kidney Disease Improving Global Outcomes (KDIGO), PKD International, World Transplant Games Federation, Australasian Kidney Trials Network (AKTN), Christchurch Kidney Society, Kidney Health Australia, Transplant Australia, The Kidney Foundation of Canada, British Kidney Patient Association, European Kidney Transplant Association (EKITA) of the European Society of Transplantation, European Renal Best Practice (ERBP), and the UK National Kidney Federation. All SONG Collaborators are listed here: songinitiative.org/who-we-are/partners-and-supporters/.
1. Ekberg H, Tedesco-Silva H, Demirbas A, et al. Reduced exposure to calcineurin inhibitors in renal transplantation. N Engl J Med
2. Lamb KE, Lodhi S, Meier-Kriesche HU. Long-term renal allograft survival in the United States: a critical reappraisal. Am J Transplant
3. Pascual M, Theruvath T, Kawai T, et al. Strategies to improve long-term outcomes after renal transplantation. N Engl J Med
4. ANZDATA Registry. 37th Report, Chapter 8: Transplantation
. Adelaide, Australia: Australia and New Zealand Dialysis and Transplant Registry; 2015.
5. Matas AJ, Smith JM, Skeans MA, et al. OPTN/SRTR 2013 Annual Data Report: Kidney. Am J Transplant
. 2015;15(Suppl 2):1–34.
6. Stegall MD, Gaston RS, Cosio FG, et al. Through a glass darkly: seeking clarity in preventing late kidney transplant failure. J Am Soc Nephrol
7. Chapman JR. The consequences of successful transplantation. Lancet
8. Cole EH, Johnston O, Rose CL, et al. Impact of acute rejection and new-onset diabetes on long-term transplant graft and patient survival. Clin J Am Soc Nephrol
9. Meier-Kriesche HU, Schold JD, Srinivas TR, et al. Lack of improvement in renal allograft survival despite a marked decrease in acute rejection rates over the most recent era. Am J Transplant
10. Watson CJ, Dark JH. Organ transplantation: historical perspective and current practice. Br J Anaesth
. 2012;108(Suppl 1):29–42.
11. Djamali A, Samaniego M, Muth B, et al. Medical care of kidney transplant recipients after the first posttransplant year. Clin J Am Soc Nephrol
12. Matas AJ, Gillingham KJ, Humar A, et al. 2202 kidney transplant recipients with 10 years of graft function: what happens next? Am J Transplant
13. Ojo AO, Morales JM, González-Molina M, et al. Comparison of the long-term outcomes of kidney transplantation: USA versus Spain. Nephrol Dial Transplant
14. Engels EA, Pfeiffer RM, Fraumeni JF Jr, et al. Spectrum of cancer risk among US solid organ transplant recipients. JAMA
15. Jardine AG, Gaston RS, Fellstrom BC, et al. Prevention of cardiovascular disease in adult recipients of kidney transplants. Lancet
16. Vajdic CM, McDonald SP, McCredie MR, et al. Cancer incidence before and after kidney transplantation. JAMA
17. Wong G, Chapman JR, Craig JC. Death from cancer: a sobering truth for patients with kidney transplants. Kidney Int
18. Ojo AO, Hanson JA, Wolfe RA, et al. Long-term survival in renal transplant recipients with graft function. Kidney Int
19. Lachenbruch PA, Rosenberg AS, Bonvini E, et al. Biomarkers and surrogate endpoints in renal transplantation: present status and considerations for clinical trial design. Am J Transplant
20. Howell M, Tong A, Wong G, et al. Important outcomes for kidney transplant recipients: a nominal group and qualitative study. Am J Kidney Dis
21. Howell M, Wong G, Turner RM, et al. The consistency and reporting of quality-of-life outcomes in trials of immunosuppressive agents in kidney transplantation: a systematic review and meta-analysis. Am J Kidney Dis
22. Masson P, Duthie FA, Ruster LP, et al. Consistency and completeness of reported outcomes in randomized trials of primary immunosuppression in kidney transplantation. Am J Transplant
23. Fritsche L, Einecke G, Fleiner F, et al. Reports of large immunosuppression trials in kidney transplantation: room for improvement. Am J Transplant
24. Howell M, Wong G, Rose J, et al. Eliciting patient preferences, priorities and trade-offs for outcomes following kidney transplantation: a pilot best-worst scaling survey. BMJ Open
25. Ho J, Rush DN, Nickerson PW. Urinary biomarkers of renal transplant outcome. Curr Opin Organ Transplant
26. Heng B, Li Y, Shi L, et al. A meta-analysis of the significance of granzyme B and perforin in noninvasive diagnosis of acute rejection after kidney transplantation. Transplantation
27. Mannon RB, Matas AJ, Grande J, et al. Inflammation in areas of tubular atrophy in kidney allograft biopsies: a potent predictor of allograft failure. Am J Transplant
28. Roedder S, Vitalone M, Khatri P, et al. Biomarkers in solid organ transplantation: establishing personalized transplantation medicine. Genome Med
29. Lo DJ, Kaplan B, Kirk AD. Biomarkers for kidney transplant rejection. Nat Rev Nephrol
30. Vavrincova-Yaghi D, Seelen MA, Kema IP, et al. Early posttransplant tryptophan metabolism predicts long-term outcome of human kidney transplantation. Transplantation
31. White CA, Siegal D, Akbari A, et al. Use of kidney function end points in kidney transplant trials: a systematic review. Am J Kidney Dis
32. White C, Akbari A, Hussain N, et al. Estimating glomerular filtration rate in kidney transplantation: a comparison between serum creatinine and cystatin C-based methods. J Am Soc Nephrol
33. Butler JA, Roderick P, Mullee M, et al. Frequency and impact of nonadherence to immunosuppressants after renal transplantation: a systematic review. Transplantation
34. Prendergast MB, Gaston RS. Optimizing medication adherence: an ongoing opportunity to improve outcomes after kidney transplantation. Clin J Am Soc Nephrol
35. Chalmers I, Bracken MB, Djulbegovic B, et al. How to increase value and reduce waste when research priorities are set. Lancet
36. Kirkham JJ, Gorst S, Altman DG, et al. COS-STAR: a reporting guideline for studies developing core outcome sets (protocol). Trials
37. Williamson PR, Altman DG, Blazeby JM, et al. Developing core outcome sets for clinical trials: issues to consider. Trials
38. Sinha IP, Smyth RL, Williamson PR. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med
39. Gargon E, Gurung B, Medley N, et al. Choosing important health outcomes for comparative effectiveness research: a systematic review. PLoS One
40. Wuytack F, Smith V, Clarke M, et al. Towards core outcome set (COS) development: a follow-up descriptive survey of outcomes in Cochrane reviews. Syst Rev
41. Bautista-Molano W, Navarro-Compán V, Landewé RB, et al. How well are the ASAS/OMERACT Core Outcome Sets for Ankylosing Spondylitis implemented in randomized clinical trials? A systematic literature review. Clin Rheumatol
42. Kirkham JJ, Boers M, Tugwell P, et al. Outcome measures in rheumatoid arthritis randomised trials over the last 50 years. Trials
43. Tugwell P, Boers M, Brooks P, et al. OMERACT: an international initiative to improve outcome measurement in rheumatology. Trials
44. Nistor I, Bolignano D, Haller MC, et al. Why creating standardized core outcome sets for chronic kidney disease will improve clinical practice. Nephrol Dial Transplant
. 2015 [published online ahead of print October 23rd, 2015] doi:10.1093/ndt/gfv365.
46. Knight SR, Morris PJ, Schneeberger S, et al. Trial design and end points in clinical transplant research. Transpl Int
. 2016 [published online ahead of print Janurary 8th, 2016]doi: 10.1111/tri.12743.
47. Tong A, Manns B, Hemmelgarn B, et al. Standardised Outcomes in Nephrology—Haemodialysis (SONG-HD): study protocol for establishing a core outcome set in haemodialysis. Trials
48. Boers M, Kirwan JR, Tugwell P, et al. The OMERACT Handbook
49. Stucki G, Boonen A, Tugwell P, et al. The World Health Organisation International Classification of Functioning, Disability and Health: a conceptual model and interface for the OMERACT process. J Rheumatol
50. Ioannidis JP, Horbar JD, Ovelman CM, et al. Completeness of main outcomes across randomized trials in entire discipline: survey of chronic lung disease outcomes in preterm infants. BMJ
51. Blackwood B, Ringrow S, Clarke M, et al. Core Outcomes in Ventilation Trials (COVenT): protocol for a core outcome set using a Delphi survey with a nested randomised trial and observational cohort study. Trials
52. MacLennan S, Bekema HJ, Williamson PR, et al. A core outcome set for localised prostate cancer effectiveness trials: protocol for a systematic review of the literature and stakeholder involvement through interviews and a Delphi survey. Trials
53. Chiarotto A, Deyo RA, Terwee CB, et al. Core outcome domains for clinical trials in non-specific low back pain. Eur Spine J
54. van't Hooft J, Duffy JM, Daly M, et al. A core outcome set for evaluation of interventions to prevent preterm birth. Obstet Gynecol
55. Jones J, Hunter D. Qualitative research: consensus methods for medical and health services research. BMJ
56. Linstone HA, Turoff M. The Delphi Method: techniques and appliations
. London UK: Addison Wesley Publishing Company; 1975.
57. Jamieson NJ, Hanson CS, Josephson MA, et al. Motivations, challenges, and attitudes to self-management in kidney transplant recipients: a systematic review of qualitative studies. Am J Kidney Dis
58. Laupacis A, Pus N, Muirhead N, et al. Disease-specific questionnaire for patients with a renal transplant. Nephron
59. Schunemann H, Brozek J, Oxman AD. GRADE handbook for grading quality of evidence and strength of recommendation
. Vol Version 3.22009.
60. Boers M, Kirwan JR, Wells G, et al. Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0. J Clin Epidemiol