Over the past 50 years, remarkable progress has been made in kidney transplantation, largely owing to treatment advances and the growing number of trials in this area.1 2,3 4,5 6 7
Trials in kidney transplantation are often short-term, and important outcomes, such as mortality, graft loss, and kidney function, are inconsistently and incompletely reported in randomized trials in kidney transplantation; with just one third reporting all 3 outcomes.8 9 10 11
These problems are well recognized across many areas in healthcare, which has driven the development of core outcome sets, defined as an “agreed minimum set of outcomes to be reported in all trials” addressing a specific condition.12 13
MATERIALS AND METHODS
Study Design
The Delphi technique is a validated approach for establishing consensus on core outcomes14-18 19,20 Figure S1, SDC, https://links.lww.com/TP/B436
Participant Selection and Recruitment
Adult patients/caregivers (aged 18 years or older) and health professionals with an interest or experience in kidney transplantation were eligible to participate. Patients/caregivers included kidney transplant recipients, patients with chronic kidney disease stages 1 to 5, or on dialysis, their caregivers, family members, and living kidney donors. Health professionals included physicians (nephrologists, surgeons, and psychiatrists), nurses, allied health professionals (psychologists, social workers, dieticians, and pharmacists), policy makers/regulators, researchers, and industry.
To be broadly inclusive, we used multiple recruitment strategies. Patients/caregivers were recruited by site investigators at participating hospitals, patient organizations worldwide (Table S1, SDC, https://links.lww.com/TP/B436 Table S1, SDC, https://links.lww.com/TP/B436
Data Collection
Selection of Outcome Domains
We identified the outcome domains from a systematic review of outcomes reported in trials in kidney transplant recipients, studies on kidney transplant recipient preferences for outcomes,21 10,22 23
Round 1
Round 1 included 35 outcome domains (Table S2, SDC, https://links.lww.com/TP/B436 24
Round 2
Participants rerated each outcome and provided (optional) comments. Participants reviewed their own score from round 1, and the distribution of the group responses on a column graph displayed by patients/caregivers, health professionals, and the weighted score of both groups combined. Participants were shown the comments from patients/caregivers and health professionals. If outcomes had a mean and median score of more than 7, and more than 50% of participants rated the outcome as 7 to 9, we retained those outcomes for round 3.
Round 3
Participants were provided with their own score, the group responses and comments as per round 2 and asked to rerate the final set of outcomes. To assess the relative importance of outcomes, participants also completed a forced ranking exercise and ordered outcomes in order of importance using a drag-and-drop function.25
Data Analysis
Quantitative Analysis
We assessed the mean, median, and the proportion of participants who rated 7 to 9 for each outcome across all rounds, and separated by patients/caregivers and health professionals. We compared the mean difference in rating scores between both stakeholder groups with a Wilcoxon signed rank test or a t test, significance at P less than 0.05. We performed subgroup analysis by country and age. For the forced ranking scores, we calculated the median and interquartile range (IQR) to determine the rank of each outcome. Statistical analyses were undertaken using R version 3.2.3 (R Foundation for Statistical Computing, Vienna, Austria ).
Definition of Consensus for Core Outcomes
An a priori definition of consensus was not possible because of the unknown distribution of scores for the outcome domains. For feasibility, we aimed to identify the top 3 to 5 outcome domains indicated as critically important by both stakeholder groups. “Consensus” for a critical outcome domain was based on both stakeholder groups having a median score of greater than or equal to 8, a mean score greater than or equal to 7.5, the proportion of participants rating the outcome as ‘critically important’ being greater than or equal to 75%, and a median score of less than 10 in the forced ranking question.
Qualitative Analysis
We imported the comments into HyperRESEARCH (Version 3.7, Randolph, MA) software for qualitative data analysis. Using thematic analysis, KEM coded the text and inductively identified themes focusing on reasons for ratings, changes in ratings across rounds, and differences between stakeholder groups. Two investigators (B.S. and A.T.) who read the qualitative data, reviewed the preliminary analysis to ensure that the themes captured the range and depth of data.
RESULTS
Participant Characteristics
In round 1, 1018 people from 79 countries participated, of whom 461 (45%) were patients/caregivers and 557 (55%) were health professionals. Round 2 was completed by 844 (83% response rate) participants from 70 countries, of whom 387 (46%) were patients/caregivers and 457 (54%) were health professionals. Round 3 was completed by 779 (77% response rate) participants with 360 (46%) patients/caregivers and 419 (54%) health professionals from 68 countries. Overall, the participant characteristics were consistent from rounds 1 to 3 (Tables 1 and 2 ).
TABLE 1: Characteristics of patients/caregivers
TABLE 2: Characteristics of health professionals
Of the 360 patients/caregivers who completed all 3 rounds, 251 (69%) were kidney transplant recipients, 32 (9%) patients on dialysis, 17 (5%) patients not on any kidney replacement therapy, 35 (10%) living kidney donors, and 27 (7%) were caregivers/family members. The patients/caregivers were from 21 countries. Of the 419 health professionals, 281 (59%) were nephrologists, 61 (13%) researchers, and 49 (10%) nurses. Health professionals were from 64 countries.
Ratings and Ranking
Round 1
The mean and median scores for each of the 35 outcome domains and proportion of participants scoring outcomes as 7 to 9 (critical importance) for each outcome are included in Table S3, SDC (https://links.lww.com/TP/B436 Table S3, SDC, https://links.lww.com/TP/B436 Table S4, SDC, https://links.lww.com/TP/B436
Round 2
The mean and median scores for each of the 22 outcome domains and proportion of participants scoring the outcome as 7 to 9 (critical importance) for each outcome are included in Table S5, SDC (https://links.lww.com/TP/B436 Table S5, SDC , https://links.lww.com/TP/B436
Round 3
The mean and median score for each of the 16 outcome domains and the proportion of participants scoring the outcome as 7 to 9 (of critical importance) for each outcome are shown in Table S6 and Figure S2, SDC (https://links.lww.com/TP/B436 Table S7, SDC (https://links.lww.com/TP/B436 Figure S3, SDC, https://links.lww.com/TP/B436
Consensus
Eight of the 16 outcomes in round 3 met all 4 consensus criteria (mean rating, ≥ 7.5; median rating, ≥ 8; proportion of 7 to 9 rating scores, ≥ 75%; median rank score, < 10) in both stakeholder groups. These were: graft function, mortality, acute graft rejection, chronic graft rejection, graft loss, infection, cardiovascular disease and cancer (nonskin). Figure 1 depicts all 16 outcomes from round 3 against the 4 consensus criteria.
FIGURE 1: Consensus criteria and outcome domains. Outcomes included in the potential core outcome set met at least two of the following criteria: mean score of greater than or equal to 7.5; mean score greater than or equal to 8; proportion of participants rating the outcome 7 to 9 (critically important) is greater than or equal to 75, and the median is less than 10 based on the forced ranking question.
Changes in Scores From Rounds 1 to 3
The 8 outcomes that fulfilled the consensus criteria were the same across the 3 rounds for both stakeholder groups. For patients/caregivers, the mean rating scores for 15 of the 16 outcomes in round 3 increased and only the score for blood pressure decreased. For health professionals, the mean scores for 14 of the 16 outcomes in Round 3 increased. The results are presented in Figures 2 and 3 .
FIGURE 2: Mean scores of patients/caregivers in rounds 1 to 3.
FIGURE 3: Mean scores of health professionals in rounds 1 to 3.
Differences Between Stakeholder Groups
The mean differences in ratings between both stakeholder groups across all rounds are shown in Figure 4 . Patients/caregivers rated 11 outcome domains higher than health professionals: skin cancer (mean difference 1.0, P < 0.001), surgical complications (0.7, P < 0.001), cognition (0.6, P < 0.001), blood pressure (0.6, P < 0.001), depression (0.5, P < 0.001), ability to work (0.5, P < 0.001), acute graft rejection (0.3, P < 0.001), non-skin cancer (0.3, P < 0.001), infection (0.3, P < 0.001), diabetes (0.2, P < 0.001), and chronic graft rejection (0.1, P = 0.022). Two outcomes were rated higher by health professionals: mortality (0.4, P < 0.001) and hospitalization (0.2, P = 0.033). The subgroup analyses for countries with more than 10 participants for outcomes rated higher by patients/caregivers (ie, outcomes with a mean difference of 0.5 or more) are shown in Table S8, SDC (https://links.lww.com/TP/B436
FIGURE 4: Differences in mean scores between patients/caregivers and health professionals in rounds 1 to 3. *The top 8 outcome domains for both stakeholder groups that met at least 2 of the following criteria: mean score of greater than or equal to 7.5; mean score greater than or equal to 8; proportion of participants rating the outcome 7 to 9 (critically important) is greater than or equal to 75, and the median is less than 10 based on the forced ranking question.
Themes From Comments
We identified 5 themes that reflected the reasons, changes and differences in the rating of outcomes: capacity to control and inevitability, personal relevance, debilitating repercussions, understanding and gaining awareness of risks, and addressing critical knowledge gaps. The description of the themes in the following section applies to both stakeholder groups unless otherwise specified, and supporting quotations are provided in Table 3 .
TABLE 3: Selected illustrative quotations
Capacity to Control and Inevitability
Although participants recognized some outcomes, such as blood pressure or skin cancer, as important, these were rated of relatively lower importance if perceived as preventable, treatable, or non–life-threatening. Across rounds, some participants decreased their ratings because they realized that certain outcomes could be feasibly improved—“I changed my answer. It's critical, but experienced surgeons make miracles happen.” For patients, some outcomes were considered of lower importance if they were inevitable—“death comes to us all, but maintaining quality of life while living is more important.”
Personal Relevance
Participants considered the importance of outcomes in terms of the patient/caregivers' life stage, health status, and personal experience, and this was maintained across rounds. For example, patients who were retired rated ability to work of lower importance. However, some recognized that fertility and ability to work would be an important outcome for younger people, and thus adjusted their rating to be “sympathetic” to other patients in different stages of life.
Debilitating Repercussions
Outcomes seen as severely debilitating, distressing, or which would lead to serious comorbidities or hospitalization were rated of higher importance. For patients, graft loss meant a return to dialysis, which was described as “devastating” and “worse than the fear of death.” In contrast, some health professionals agreed that losing the graft was a “main hard endpoint,” but “relative to death, [it] cannot receive the highest rating.”
Understanding and Gaining Awareness of Risks
Patients/caregivers increased their ratings across rounds as read comments from health professionals and became “enlightened” about the risks of outcomes. Outcomes were rated higher if they learned that it was associated with a greater risk of graft loss and chronic graft rejection—“I'm naturally going to increase my rating if the pro[fessional]'s think it's critical.” Health professionals expressed appreciation and “respect” of the importance of outcomes to patients, namely psychosocial and disabling outcomes—“underestimated the importance based on patient feedback.”
Addressing Critical Knowledge Gaps
Participants recognized the unresolved challenges of preventing complications including rejection, graft function, cancer, infection, and diabetes, which prompted them to assign higher importance to these outcomes. Health professionals suggested that “long standing issues” that lacked evidence should also be high priorities for research and would benefit patients, for example, ability to work.
DISCUSSION
The outcomes of critical importance for patients/caregivers and health professionals were graft loss, graft function, acute graft rejection, chronic graft rejection, mortality, cardiovascular disease, infection, and cancer (non-skin). Overall, the results were congruent between the 2 stakeholder groups, with the same top 8 outcomes. Graft-related outcomes comprised the top 4 for both patients/caregivers and health professionals, consistent with research priority setting studies.26-28 26,29-33
Although mortality was rated critically important for both stakeholder groups, the mean rating score was higher for health professionals (8.3) and ranked second after graft loss, compared with patients/caregivers (7.6) where mortality ranked fifth (below all the graft related outcomes). This discrepancy could be due to differences in how patients/caregivers and health professionals conceptualize death. Free text comments suggest that patients accept death as unavoidable, beyond their control, and therefore place greater importance on having a good quality of life. For health professionals, patient survival was entrenched as their primary goal of care and deemed to be the principal hard endpoint.34
Patients and caregivers placed greater importance on skin cancer, surgical complications, cognition, depression, blood pressure, and ability to work, compared with health professionals. The discrepancy in the prioritization of skin cancer may reflect geographical variations in the risk of skin cancer. We noted that patients/caregivers in countries with public campaigns for skin cancer prevention (United States, South Africa, Canada and Australia35-38 33,39-41 42 43,44
Unlike for hemodialysis, patient-reported outcomes were not prioritized as highly in kidney transplantation. In the Delphi survey on hemodialysis, ability to travel and dialysis-free time were rated markedly higher by patients/caregivers than providers.17 45,46
The consensus between patients/caregivers and health professionals supports the inclusion of graft outcomes and medical comorbidities in a core outcome set for kidney transplantation. However, the, dominant focus on the clinical challenges of kidney transplantation may have led to patient-reported outcomes being assigned a lower rating score. For example, because quality of life is markedly improved by transplantation, compared to dialysis, it could be assumed as a given provided the graft continued to function. This is pertinent given the increasing emphasis on including patient-reported outcome measures in trials, to assess how patients feel and function.47 48
Compared with other Delphi surveys used to establish core outcomes, our sample size is large with a broad representation of different stakeholders across 79 countries and had a high response rate of 77% by round 3.45,46
Core outcomes should be reported in all trials in kidney transplantation because they are relevant and important to patients and clinicians. Patients and clinicians want to know the effect of an intervention on outcomes they regard as critically important irrespective of whether the intervention makes a difference. Researchers are not required to use a core outcome as a primary endpoint to estimate the sample size required to adequately power a study. Also, researchers may add other outcomes based on other considerations including responsiveness to an intervention and feasibility.
Our international SONG-Tx Delphi survey has achieved clear consensus on high priority outcome domains to report in all trials involving kidney transplant recipients. These pertained to preservation of the graft and common life-threatening medical complications—cancer, infection, and cardiovascular disease. However, there is increasing emphasis on explicitly including patient-reported outcomes, which may be inherent in graft survival and clinical complications. Further stakeholder engagement through consensus workshops is needed to discuss the inclusion of patient-reported outcomes and to ascertain appropriate, feasible, and acceptable approaches for establishing core outcome domains in kidney transplantation. We will present the findings from the Delphi survey at SONG-Tx consensus workshops for review and discussion by patients, caregivers, clinicians, researchers, and policy makers. Once the core outcome domains are established, subsequent work to identify core outcome measures will ensue, which may be informed from complementary initiatives such as the Clinical Data Interchange Standards Consortium. Clinical Data Interchange Standards Consortium recently published the Kidney Transplant Therapeutic Area User Guide to facilitate standardized reporting of data.49
Improving the consistency and relevance of trial endpoints in kidney transplantation can ultimately empower patients and clinicians to make critical treatment decisions based on outcomes that explicitly matter to them.
ACKNOWLEDGMENTS
The authors thank all the participants who participated on the SONG-Tx Delphi Panel. The authors are very grateful to all the collaborating organizations for help in inviting patients, caregivers, and health professionals to participate.
REFERENCES
1. Stegall MD, Gaston RS, Cosio FG, Matas A. Through a glass darkly: seeking clarity in preventing late kidney transplant failure.
J Am Soc Nephrol . 2015;26:20–29.
2. Shapiro R, Basu A, Tan H, et al. Kidney transplantation under minimal immunosuppression after pretransplant lymphoid depletion with thymoglobulin or campath.
J Am Coll Surg . 2005;200:505–515.
3. Lodhi SA, Meier-Kriesche HU. Kidney allograft survival: the long and short of it.
Nephrol Dial Transplant . 2011;26:15–17.
4. Nankivell BJ, Kuypers DR. Diagnosis and prevention of chronic kidney allograft loss.
Lancet . 2011;378:1428–1437.
5. Matas AJ, Gillingham KJ, Humar A, et al. 2202 kidney transplant recipients with 10 years of graft function: what happens next?
Am J Transplant . 2008;8:2410–2419.
6. De Geest S, Moons P. The patient’s appraisal of side-effects: the blind spot in quality-of-life assessments in transplant recipients.
Nephrol Dial Transplant . 2000;15:457–459.
7. Ioannidis JP, Greenland S, Hlatky MA, et al. Increasing value and reducing waste in research design, conduct, and analysis.
Lancet . 2014;383:166–175.
8. Masson P, Duthie FA, Ruster LP, et al. Consistency and completeness of reported outcomes in randomized trials of primary immunosuppression in kidney transplantation.
Am J Transplant . 2013;13:2892–2901.
9. Hussain S, Knight SR. Quality and consistency of outcome reporting in clinical trials of immunosuppression in renal transplantation.
Clin Transplant . 2016;30:1440–1448.
10. Howell M, Wong G, Turner RM, Tan HT, Tong A, Craig JC, et al. The consistency and reporting of quality-of-life outcomes in trials of immunosuppressive agents in kidney transplantation: a systematic review and meta-analysis.
Am J Kidney Dis . 2016;67:762–774.
11. Chalmers I, Bracken MB, Djulbegovic B, et al. How to increase value and reduce waste when research priorities are set.
Lancet . 2014;383:156–165.
12. Williamson PR, Altman DG, Blazeby JM, et al. Developing core outcome sets for clinical trials: issues to consider.
Trials . 2012;13:132.
13. Tong A, Budde K, Gill J, et al. Standardized outcomes in nephrology-transplantation: a global initiative to develop a core outcome set for trials in kidney transplantation.
Transplant Direct . 2016;2:e79.
14. Evangelidis N, Tong A, Manns B, et al. Developing a set of core outcomes for trials in hemodialysis: an international Delphi survey.
Am J Kidney Dis . 2017; S0272-6386(17)30035-5.
15. Van’t Hooft J. A core outcome set for evaluation of interventions to prevent preterm birth: summary for CROWN.
BJOG . 2016;123(Suppl 3):107.
16. McNair AG, Whistance RN, Forsythe RO, et al. Core outcomes for colorectal cancer surgery: a consensus study.
PLoS Med . 2016;13:e1002071.
17. Karas J, Ashkenazi S, Guarino A, et al. A core outcome set for clinical trials in acute diarrhoea.
Arch Dis Child . 2015;100:359–363.
18. Hopkins J, Howes N, Chalmers K, et al. What are important outcomes of bariatric surgery? An in-depth analysis to inform the development of a core outcome set and a comparison between the views of surgeons and other health professionals (the BARIACT study).
Lancet . 2015;385(Suppl 1):S43.
19. Jones J, Hunter D. Consensus methods for medical and health services research.
BMJ . 1995;311:376–380.
20. Murphy MK, Black NA, Lamping DL, et al. Consensus development methods, and their use in clinical guideline development.
Health Technol Assess . 1998;2: i-iv, 1–88.
21. Howell M, Tong A, Wong G, Craig JC, Howard K. Important outcomes for kidney transplant recipients: a nominal group and qualitative study.
Am J Kidney Dis . 2012;60:186–196.
22. Dobbels F, Wong S, Min Y, Sam J, Kalsekar A. Beneficial effect of belatacept on health-related quality of life and perceived side effects: results from the BENEFIT and BENEFIT-EXT trials.
Transplantation . 2014;98:960–968.
23. LimeSurvey Project Team/Carsten Schmitz. LimeSurvey: An Open Source survey tool.
http://www.limesurvey.org . Published 2015.
24. Schunemann H, Brozek J, Oxman A. GRADE Handbook for grading quality of evidence and strength of recommendation.
http://gdt.guidelinedevelopment.org/app/handbook/handbook.html . Published 2009.
25. Krosnick JA. Survey research.
Annu Rev Psychol . 1999;50:537–567.
26. American Society of Nephrology. American Society of Nephrology Renal Research Report.
J Am Soc Nephrol . 2005;16:1886–1903.
27. Tong A, Sainsbury P, Carter SM, et al. Patients’ priorities for health research: focus group study of patients with chronic kidney disease.
Nephrol Dial Transplant . 2008;23:3206–3214.
28. Tong A, Crowe S, Chando S, et al. Research priorities in CKD: report of a national workshop conducted in Australia.
Am J Kidney Dis . 2015;66:212–222.
29. Vajdic CM, McDonald SP, McCredie MR, et al. Cancer incidence before and after kidney transplantation.
JAMA . 2006;296:2823–2831.
30. Jardine AG, Gaston RS, Fellstrom BC, Holdaas H. Prevention of cardiovascular disease in adult recipients of kidney transplants.
Lancet . 2011;378:1419–1427.
31. Chapman JR. The consequences of successful transplantation.
Lancet . 2011;378:1357–1359.
32. Sharif A, Hecking M, de Vries AP, et al. Proceedings from an international consensus meeting on posttransplantation diabetes mellitus: recommendations and future directions.
Am J Transplant . 2014;14:1992–2000.
33. Schipper K, Abma TA. Coping, family and mastery: top priorities for social science research by patients with chronic kidney disease.
Nephrol Dial Transplant . 2011;26:3189–3195.
34. McAlister FA, van Diepen S, Padwal RS, et al. How evidence-based are the recommendations in evidence-based guidelines?
PLoS Med . 2007;4:e250.
35. Doran CM, Ling R, Byrnes J, et al. Benefit cost analysis of three skin cancer public education mass-media campaigns implemented in New South Wales, Australia.
PLoS One . 2016;11:e0147665.
36. Hingle MD, Snyder AL, McKenzie NE, et al. Effects of a short messaging service-based skin cancer prevention campaign in adolescents.
Am J Prev Med . 2014;47:617–623.
37. Schulz EJ. Sun education in South Africa.
Clin Dermatol . 1998;16:531–533.
38. Freiman A, Yu J, Loutfi A, Wang B. Impact of melanoma diagnosis on sun-awareness and protection: efficacy of education campaigns in a high-risk population.
J Cutan Med Surg . 2004;8:303–309.
39. Veater NL, East L. Exploring depression amongst kidney transplant recipients: a literature review.
J Ren Care . 2016;42:172–184.
40. Müller HH, Englbrecht M, Wiesener MS, et al. Depression, anxiety, resilience and coping pre and post kidney transplantation—initial findings from the Psychiatric Impairments in Kidney Transplantation (PI-KT)-Study.
PLoS One . 2015;10:e0140706.
41. Griva K, Thompson D, Jayasena D, Davenport A, Harrison M, Newman SP. Cognitive functioning pre- to post-kidney transplantation—a prospective study.
Nephrol Dial Transplant . 2006;21:3275–3282.
42. Singh P, Ng Y-H, Unruh M. Kidney transplantation among the elderly: challenges and opportunities to improve outcomes.
Adv Chronic Kidney Dis . 2016;23:44–50.
43. Van Sandwijk MS, Ten Berge IJ, Majoie CB, et al. Cognitive changes in chronic kidney disease and after transplantation.
Transplantation . 2016;100:734–742.
44. Kurella Tamura M, Yaffe K. Dementia and cognitive impairment in ESRD: diagnostic and therapeutic strategies.
Kidney Int . 2011;79:14–22.
45. Kostro JZ, Hellmann A, Kobiela J, et al. Quality of life after kidney transplantation: a prospective study.
Transplant Proc . 2016;48:50–54.
46. Czyżewski L, Sańko-Resmer J, Wyzgał J, Kurowski A. Assessment of health-related quality of life of patients after kidney transplantation in comparison with hemodialysis and peritoneal dialysis.
Ann Transplant . 2014;19:576–585.
47. Rahimi K, Malhotra A, Banning AP, et al. Outcome selection and role of patient reported outcomes in contemporary cardiovascular trials: systematic review.
BMJ . 2010;341:c5707.
48. U.S. Department of Health and Human Services FDA Center for Drug Evaluation and Research; U.S. Department of Health and Human Services FDA Center for Biologics Evaluation and Research; U.S. Department of Health and Human Services FDA Center for Devices and Radiological Health. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance.
Health Qual Life Outcomes . 2006;4:79.
49. Clinical Data Interchange Standards Consortium (CDISC) 2016. Kidney Transplant Therapeutic Area User Guide (TAUG-KT) Version 1.0.
https://www.cdisc.org/standards/therapeutic-areas/kidney-transplant/kidney-transplant-therapeutic-area-user-guide-v10 . Published December 31 2016. Accessed 15th February 2017.
