Introduction
Many patients and providers perceive the living kidney donor candidate evaluation process to be difficult to navigate and lengthy, exceeding 10 months at many transplant centers (1–2 3 2 4 5 2 6–7 8 9 10 11 12
The lack of benchmarks for the timeliness of the living donor evaluation process may hinder quality improvement efforts. The development and use of standardized definitions for key terms and quality indicators would facilitate the creation of such benchmarks, provide insight into bottlenecks in the living donor evaluation process, and enable meaningful comparisons within and across transplant centers over time (9 e.g. , scheduling multiple specialist consultations on the same day, evaluating many candidates simultaneously for the same recipient) (1 3
We conducted this study using a Delphi process to develop consensus on the key terms and indicators that can be used to measure how efficiently a transplant center evaluates persons interested in becoming a living kidney donor (13–14 15
Materials and Methods
Consensus-Building Method
We followed published recommendations on how to undertake a Delphi study, which comprised a series of surveys aimed to reach consensus from a group of stakeholders (13 14 16
Survey Development
The surveys included key terms used to define a concept, process indicators that measure program activities, and outcome indicators that measure clinical events. We first identified and developed initial terms and indicators through conversations with various stakeholders and after a scoping review of the literature (Figure 1 ) (1 12
Figure 1.: Delphi process of identifying and defining key terms and final quality indicators. There were initially 12 key terms, 13 process indicators, and ten outcome indicators that were rated by the participants. After rating scores were computed and participant comments reviewed, discussion with the work group produced a total of 12 final key terms, 16 final process indicators, eight final outcome indicators, and two indicators that can be considered both process and outcome indicators.
Survey Participants (Stakeholders)
Eligible participants were adults (age ≥18 years) in Canada who had experience with the living kidney donor candidate evaluation, including kidney transplant recipients or recipient candidates, donors and donor candidates, health care providers (e.g. , nephrologists, surgeons, social workers, living donor coordinators, transplant coordinators), and health care administrators (e.g. , representatives of provincial or national organ procurement organizations).
Participant Sample
We initiated word-of-mouth recruitment by leveraging existing networks known to the research team and relying on snowball sampling, a common technique used in qualitative research (Figure 1 ) (17 18 18 19
Measures of Agreement
We provide an example of how agreement was ascertained, how the definition of one key term was presented to participants, and how the definition was finalized (Boxes 1 and 2). On a nine-point Likert scale from one (strongly disagree) to nine (strongly agree), survey participants rated whether each definition was clear, whether they agreed with each definition, whether each indicator was an important measure to improve, and whether each indicator was suitable to compare across transplant programs. We used the RAND methodology to assess agreement (14 20 14 14
Box 1
Example of agreement calculation. Below are responses from 72 stakeholders to the question “Do you agree with this definition for a living donor candidate: any person who contacts a living donor program with an interest of being evaluated as a living kidney donor or learning more about the living kidney donor process?”
1
1
3
4
4
4
5
6
6
6
7
7
7
7
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
The responses are on a nine-point Likert scale (one, strongly disagree; nine, strongly agree).
Calculations:
30th percentile=8.
70th percentile=9.
Interpercentile range (IPR)=1 (70th–30th percentile).
To adjust for asymmetry:
Central point: (8+9)/2=8.5.
Asymmetry index: absolute value of the middle point (5 in a nine-point Likert scale) minus the central point=|5.0–8.5|=3.5.
IPR adjusted for asymmetry (IPRAS; 2.35 is the IPR required for disagreement when perfect symmetry exists)=2.35+(1.5×asymmetry index)=2.35+1.5×3.5=7.6.
If IPR is greater than IPRAS, then no agreement. In this example, there is agreement.
Note: This method allows for any number of participant responses and better accounts for dispersion and higher weights on the extremes than traditional methods.
Box 2
Example of how the comments were used to modify a key term.
Example of definition of key term:
The first definition provided was “living donor candidate,” defined as:
“Any person who contacts a living donor program with an interest of being evaluated as a living kidney donor or learning more about the living kidney donor process.”
The median score for “I agree with this definition” was eight (important), with a very narrow interpercentile range of eight to nine among respondents. Using the RAND methodology, this signifies agreement (using the calculation in Box 1 ).
Despite agreement, the comments were consistent in recommending that we add the word “potential” to the term, and not count in this definition persons who contacted the program simply to learn more about the living kidney donor process (i.e. , only those expressing an interest of being evaluated as a living kidney donor are counted).
The final definition is now “potential living donor candidate,” defined as:
“Any person who contacts a living donor program with an interest of being evaluated as a living kidney donor.”
Although the definition was changed, the research team decided this minor change did not need to be rated again by participants.
Work Group
A work group (comprising one epidemiologist, three nephrologists, and seven patient partners) met in Montreal, Canada, in May 2019 to review results of the Delphi study, discuss problematic definitions or indicators (e.g. , the extent of revisions or rationale behind the disagreement), and discuss newly developed definitions or indicators. Slight modifications in terminology were not discussed by the work group, nor were they brought back to participants for rerating (Box 2).
Results
Participant Population
We emailed 128 people inviting them to participate in the study, and 77 (60%) did so. Most participants self-identified as White (80%) and female (64%) (Table 1 ). Participants either resided in, received care in, or provided care across Canada, with representation from seven Canadian provinces. Twenty-seven participants (36%) were health care providers, 33 (43%) were living donors , and ten (13%) received a kidney transplant from either a living or deceased donor. Nine transplant recipients previously received dialysis, and 14 persons (12 donors and two recipients) were involved in the Canadian national Kidney Paired Donation Program. Among the 51 nonparticipants, there were 22 men, 17 donors, 16 nephrologists, five recipients, four other health care professionals (surgeon, transplant coordinator, living donor coordinator, psychosocial specialist), three family members of a donor or recipient, and one ineligible donor candidate.
Table 1. -
Participant demographics
Characteristic
n (%), N =77
Age, yr
mean 49 (SD 11), range 24–69
Women
a
47 (64%)
White
a
59 (80%)
Donor or recipient
b
42 (55%)
Living donor, donated
33 (79%)
Living donor candidate, ineligible to donate
1 (2%)
Recipient, from living donor
6 (14%)
Recipient, from deceased donor
4 (10%)
Transplant candidate, has/had a living donor identified
1 (2%)
Health care professional
b
27 (36%)
Transplant nephrologist
10 (37%)
Living donor nephrologist
9 (33%)
General nephrologist
5 (19%)
Donor surgeon
2 (7%)
Transplant surgeon
2 (7%)
Living donor nurse coordinator
6 (22%)
Other
c
5 (19%)
Health care administrator or decision maker
b
7 (9%)
Province of residence or donation/transplant workup
Ontario
34 (44%)
Alberta
10 (15%)
British Columbia
9 (14%)
Manitoba
3 (5%)
Saskatchewan
2 (3%)
Nova Scotia
2 (3%)
Quebec
3 (5%)
Other (United States)
3 (5%)
Missing
11 (17%)
a Three participants did not provide information, and percentages reflect the denominator of 74.
b Some overlap may occur across and within each classification because some categories are not mutually exclusive.
c Includes nephrology trainee, personal support worker, and nurse involved in CKD care.
Overview of Study Results
Initially, 12 key terms were proposed to measure 13 process indicators and ten outcome indicators (Supplemental Appendix 1 3 Supplemental Appendix 4 Figure 1 , Table 2 ). Twenty-two unique data elements were required to enable the measurement of all indicators (Table 3 ). Key terms and quality indicators are italicized throughout.
Table 2. -
List of final key terms, process indicators, and outcome indicators identified and defined through the Delphi process
Glossary
Key terms (see
Supplemental Appendix 5
for detailed definitions)
1.
Potential living donor candidate
2.
First contact date
3.
Decision to start testing date (formerly the testing start date)
4.
Approval date rendered by each responsible physician (e.g. , nephrologist, urologist, etc .)
5.
Withdrawal date
6.
Lost to follow-up date
7.
Evaluation end date (derived)
8.
Hold start date
9.
Hold end date
10.
Potential preemptive living kidney donor transplant
11.
Reasonable amount of time (to complete the evaluation if the goal is a preemptive transplant)
12.
Reasonably long period of time (to be considered lost to follow-up)
Process indicators (see
Supplemental Appendix 6
for detailed descriptions)
1.
Time from decision to start testing until donation
2.
Total evaluation time (time from decision to start testing until evaluation end)
3.
Time from decision to start testing until first diagnostic test is performed
4.
Time from decision to start testing to consultation
5.
Time from decision to start testing to initial compatibility (crossmatch) test
6.
Time from first contact date until recipient referral
7.
Time from when the recipient referral is sent to the transplant center until the recipient referral is received by donor evaluation team
8.
Time from decision to start testing until first blood or urine test
9.
Time from decision to start testing until first 24-h urine test
10.
Time from computed tomography angiogram until evaluation end
11.
Time from decision to start testing until potential initial compatibility test
12.
Time from initial crossmatch until results received by the donor candidate
13.
Time from initial crossmatch until results received by the donor living donor coordinators
14.
Time from initial crossmatch until results received by the donor transplant coordinator
15.
Time from approval until operating room booking
16.
Time from approval until donation
17.
Diagnostic testing done on the same visit
Outcome indicators (see
Supplemental Appendix 7
for detailed descriptions)
1.
Total health care cost of the living donor candidate’s evaluation
2.
The number of visits to the transplant center
3.
Annual number of living donor kidney transplants
4.
Annual number of preemptive living donor kidney transplants
5.
Annual number of living donor kidney transplants done in the first year of dialysis
6.
Annual number of potential preemptive transplants lost
7.
Annual number of deaths of the intended recipient (instead of a living donor transplant)
8.
Annual number of deceased donor transplants received by the intended recipient (instead of living donor transplants)
9.
Annual number of times the intended recipient was rendered ineligible (instead of living donor transplants)
Table 3. -
Unique variables needed to enable all process and outcome indicators
Data Element
Decision to start testing date
Donation date
Approval date rendered by each responsible physician (e.g., nephrologist approval, urologist approval, etc .)
Withdrawal date
Declined date
Loss to follow-up date
Date of diagnostic test (one for each test)
Date of consultation (one for each consultation)
First contact date
Date the recipient referral was received by the transplant program
Date the recipient referral was received by the donor evaluation team
Date the recipients serum was frozen for crossmatch testing
Date the initial crossmatch test results were received by the donor candidate
Date the initial crossmatch test results were received by the living donor coordinator
Date the initial crossmatch test results were received by the transplant coordinator
Date the operating room was booked
Date the intended recipient started dialysis
Date the intended recipient died
Date the intended recipient was rendered ineligible
Date the intended recipient received a deceased donor kidney transplant
Hold start date
Hold end date
Key Terms
Definitions with Agreement (n =10).
These defined several aspects of the donor evaluation process, such as the date a candidate was approved (or declined) by the transplant center, the date a donor candidate withdrew from the evaluation process, and the dates the evaluation was put on hold.
The Revised Definition (n =1).
The purpose of defining a potential preemptive living kidney donor transplant was to estimate the number of missed opportunities for preemptive transplantation. Although the median rating score indicated agreement with the term, this lacked consensus (Supplemental Appendix 4 Supplemental Appendix 5
Definition Removed/Replaced (n =1).
The “evaluation start date” was initially used to define the starting point for most process indicators. Although the original definition was clear to participants (median 8; IPR 7–9), there was disagreement on the definition itself (median 8, IPR 5–9). The comments revealed the first contact date may not always be the evaluation start date: people do not always contact the living donor program to immediately begin the evaluation; others perceived the initial steps of the evaluation process outside the control of the transplant center. Given the lack of consensus, this definition was replaced with the “testing start date,” which was clear to participants (median 9, IPR 8–9) with agreement on the definition (median 9, IPR 8–9). The definition was renamed as the “decision to start testing date.”
Proposed Definitions (n =2).
The first new term discussed with the work group was “the minimum time required to complete a living donor evaluation if the goal is a preemptive transplant,” which is a necessary inclusion criterion to be able to measure missed opportunities for a preemptive transplant. For example, if it is only a few days from when a candidate starts the evaluation to the time the intended recipient starts dialysis, a preemptive transplant from that donor candidate is not possible. The work group considered a value of 90 days, but decided to leave it to the discretion of whoever is using the measure.
The second term was to help define when a candidate is no longer actively being evaluated. For example, although there is no test result that indicates a candidate is ineligible to donate, a candidate's actions may demonstrate that the candidate is no longer interested in pursuing donation (e.g. , do not respond to donor program contact attempts). Thus, we wanted to define the period of time necessary to indicate this has occurred. The work group decided such a definition could be too arbitrary and did not adopt the definition.
Process Indicators
Most process indicators represented wait times between milestones along the evaluation pathway (Figure 2 ) and relied on the key term decision to start testing date as the start point of the potential donor candidate evaluation (Supplemental Appendix 6
Figure 2.: Steps in the living donor workup where definitions of key terms are required to enable the measurement of process indicators. The top line refers to the donor candidate evaluation process. The bottom line refers to key milestones in the recipient’s pathway. Dn etc . En etc . R1 –R3 refer to anchors related to the evaluation of the intended recipient (if there is one).
Retained Process Indicators (n =16).
Participants agreed these were important measures to improve and to compare between programs. For example, the time until donation was defined as the time from the decision to start testing until donation (B–Z in Figure 2 ). The time to any diagnostic test or consult was measured once per test/consult (e.g. , time until first ultrasound, time until first nephrology consultation).
Process indicators that some participants perceived to be beyond the control of the transplant program (e.g. , evaluation times that depended on action from the potential living kidney donor candidate) lacked consensus as to whether they were important measures to improve. These included the time from when a donor candidate started the evaluation until the first blood or spot urine test (median 8, IPR 6–9) and the time until first 24-hour urine test (median 7, IPR 5–8). The work group acknowledged there are still opportunities for transplant centers to improve these measures, such as providing candidates laboratory requisitions earlier and electronically, or educating candidates more clearly about the process and the anticipated effect that delays in testing could have.
Revised Process Indicator (n =1).
Our intent was to improve the actionability of the time until the initial crossmatch test (one of the metrics of B–Dn Figure 2 ). The initial indicator that was intended to measure improvements in this part of the process was rated important (median 7, IPR 5–8) and to compare across transplant centers (median 7, IPR 5–8), but comments suggested clarification was needed as well as cautioned that the requirements for the crossmatch test may differ between programs. After work group discussion, we simplified the original indicator to the time from the decision to start testing until the time the recipient’s serum was available (Supplemental Appendix 6
Removed Process Indicator (n =2).
Unpublished estimates from a medium-sized transplant center suggested that the “time to review the intake medical-social questionnaire within the program” has a short turn-around time (median 8 days, interquartile range 3–18 days) and is not a critical bottleneck. Considering that this process indicator requires two unique data elements, it was removed. The “time until testing start” was removed for a similar reason. If a program still wishes to measure this step, it could use time from first contact date until decision to start testing date without additional data collection.
Added Process Indicators (n =6).
Three new process indicators arose from a work group discussion regarding the crossmatch testing indicator described above: the time from the initial crossmatch testing date until the results are received by (1 ) the potential living donor candidate, (2 ) the living donor coordinators, and (3 ) the transplant recipient coordinator. Two other process indicators were added, including the time from approval until donation (no new data collection needed) and the time from approval until operating room booking (the date the operating room is booked for a future transplant surgery is a new data element). The sixth new indicator is a dual indicator, described below.
Dual Indicators
Two quality indicators focused on reducing the burden of testing on candidates. Each can be considered a process indicator (e.g. , as an efficiency measure) and an outcome indicator (e.g. , to improve the satisfaction of the donor candidate during the evaluation process). One was a new indicator, diagnostic testing done on the same visit , defined by the work group as the completion of a set of tests/consults on the same date (Supplemental Appendix 6 number of visits to the transplant center , was important to reduce and compare between transplant centers.
Outcome Indicators
Indicators measuring actual living donor transplantation (n =3) were important to improve and compare across transplant centers (Supplemental Appendix 7 Supplemental Appendix 4
Outcome indicators measuring the potential for living donor transplantation (n =4) were also rated highly, but with less agreement. One of these indicators, the number of potential preemptive transplants lost , reached agreement that efforts should be made to reduce this event (median 8.5, IPR 7–9), but there was disagreement about whether this should be compared across programs (median 8, IPR 6–9). Comments suggested participant disagreement related to the measurement’s feasibility and its inability to capture the reasons why a recipient started dialysis despite having a living donor. We also observed similar responses for the number of potential recipients who died or became ineligible to receive a transplant despite having a living donor. Comments on the number of times the recipient received a deceased donor transplant (despite having a potential living donor candidate) suggested that disagreement was because of participants prioritizing ethical decision making over efficiency. These indicators were retained.
The total health care cost of the potential living donor candidate’s evaluation was not recommended as a target for improvement (median 7.5, IPR 6–9), but participants agreed it was useful to compare such costs between transplant centers (median 8, IPR 7–9).
Discussion
Multiple quality indicators are needed to describe the efficiency of the living kidney donor evaluation process. Using a modified Delphi method, we established definitions for key terms and developed a list of process and outcome indicators that can be used to measure performance within and across transplant centers over time.
Process indicators (those that measure how health care is delivered) are typically easy to measure, can be produced over a short time horizon (e.g. , on the order of months rather than years), and are modifiable (e.g. , through some change in practice) (21 7 9 21–22 23 7
Unlike actual outcomes, which are readily interpretable and measurable (e.g. , number of living donor transplants), outcome indicators are not absolute. For example, the four outcome indicators aimed to estimate the number of missed opportunities for living donation are not directly measurable because they relate to events that did not occur, and it cannot be known whether these events would have occurred (e.g. , the donor candidate may have been ultimately deemed ineligible to donate). Despite this, these indicators remain meaningful because a higher rate of missed opportunities for transplant may signify a potential area of improvement to a transplant center.
To our knowledge, this study represents the first comprehensive approach to develop a set of quality indicators to measure the efficiency of the donor candidate evaluation. One strength of this study is the way individuals with lived experience were incorporated into the research team from the outset. Patients have rarely been featured in the development of quality indicators in transplantation (24
There are some limitations of this study. First, although our focus was on the efficiency of the evaluation of potential living donor candidates, a similar process is also important for the assessment of transplant recipient candidates. Generally, the living donor evaluation will not begin until the intended recipient has accomplished some pretransplant workup (some programs require the intended recipient to be approved or reasonably likely to be approved for transplant). Second, some final indicators are not applicable to the contexts of nondirected donation or paired donation, which became clear when reviewing participant comments (particularly from one nondirected donor). These should be the focus of future efforts. Third, during our study process, we did not receive suggestions for patient-reported experience measures or measures of ethics. This may include how well candidates felt they were supported by the living kidney donation program, how well candidates felt the program communicated with them, how well informed the candidates were about the process, if the candidates perceived the evaluation to be timely, and if the candidates felt pressure from the transplant center to complete the evaluation more quickly than they would have liked. Such measures are important and should be considered in future research. Fourth, although the decision to start testing may vary between programs, we expect these differences to be small relative to the overall time frame of the evaluation process (days rather than months). Similar dates have been used by some transplant programs (e.g. , “decision to proceed” with a donor) and in other areas for quality improvement (e.g. , the “decision to treat” for estimating wait times) (25–26 27 living donors who have shared the sentiments experienced by nondonor candidates. This is supported by a scoping review of the literature, where wait times and testing burden featured as efficiency-related barriers to donation (1 28 e.g. , operating room availability, access to computed tomography angiography, full-time equivalents of human resources). Structural changes are generally difficult to implement, require large investments in personnel and equipment, and are subject to budget and physical constraints that may be beyond the control of a transplant center. One indicator that can be considered a dual structural/process indicator incorporates replacing certain face-to-face visits with telehealth visits. This is one quality improvement strategy that may improve the efficiency of the evaluation process and should be the focus of future work.
In conclusion, as in other areas of health care, the evaluation process for potential living donor candidates can be measured and ultimately improved (9 29–30 31 32 33 e.g. , efficiency, safety, cost) speaks to the perceived issues that surround the living donor evaluation process. In most developed nations, living donation is considered safe (e.g. , complications are rare), and costs are of comparatively little importance than increasing living donor activity. Thus, most indicators focused on the timeliness of the evaluation process. Future work is needed to identify the reasons for delayed donor candidate evaluations and to identify and prioritize opportunities for improvement.
Disclosures
All authors have nothing to disclose.
Funding
This project, including patient partnerships, was supported by Canadian Institutes of Health Research Kidney Foundation of Canada CIHR CIHR Astellas Pharma
Acknowledgments
Dr. Ann Bugeja reports receiving grants from LeoPharma, outside the submitted work. Dr. Amit Garg is supported by the Dr. Adam Linton Chair in Kidney Health Analytics.
Supplemental Material
This article contains the following supplemental material online at http://cjasn.asnjournals.org/lookup/suppl/doi:10.2215/CJN.03780320/-/DCSupplemental
Supplemental Appendix 1
Supplemental Appendix 2
Supplemental Appendix 3
Supplemental Appendix 4
Supplemental Appendix 5
Supplemental Appendix 6
Supplemental Appendix 7
References
1. Habbous S, Woo J, Lam NN, Lentine KL, Cooper M, Reich M, Garg AX: The efficiency of evaluating candidates for living kidney donation: A scoping review. Transplant Direct 4: e394, 2018 30498771
2. Moore DR, Serur D, Rudow DL, Rodrigue JR, Hays R, Cooper M; American Society of Transplantation: Living donor kidney transplantation: Improving efficiencies in live kidney donor evaluation—Recommendations from a consensus conference. Clin J Am Soc Nephrol 10: 1678–1686, 2015 26268509
3. Habbous S, Arnold J, Begen MA, Boudville N, Cooper M, Dipchand C, Dixon SN, Feldman LS, Goździk D, Karpinski M, Klarenbach S, Knoll GA, Lam NN, Lentine KL, Lok C, McArthur E, McKenzie S, Miller M, Monroy-Cuadros M, Nguan C, Prasad GVR, Przech S, Sarma S, Segev DL, Storsley L, Garg AX; Donor Nephrectomy Outcomes Research (DONOR) Network: Duration of living kidney transplant donor evaluations: Findings from 2 multicenter cohort studies. Am J Kidney Dis 72: 483–498, 2018 29580662
4. Ralph AF, Chadban SJ, Butow P, Craig JC, Kanellis J, Wong G, Logeman C, Tong A: The experiences and impact of being deemed ineligible for living kidney donation: Semi-structured interview study. Nephrology 25: 339–350, 2020
5. Agerskov H, Ludvigsen MS, Bistrup C, Pedersen BD: Living kidney donors’ experiences while undergoing evaluation for donation: A qualitative study. J Clin Nurs 24: 2258–2267, 2015 25753175
6. Habbous S, McArthur E, Dixon SN, McKenzie S, Garcia-Ochoa C, Lam NN, Lentine KL, Dipchand C, Litchfield K, Begen MA, Sarma S, Garg AX: Initiating maintenance dialysis before living kidney donor transplantation when a donor candidate evaluation is well underway. Transplantation 102: e345–e353, 2018 29538259
7. Habbous S, McArthur E, Sarma S, Begen MA, Lam NN, Manns B, Lentine KL, Dipchand C, Litchfield K, McKenzie S, Garg AX; Donor Nephrectomy Outcomes Research (DONOR) Network: Potential implications of a more timely living kidney donor evaluation. Am J Transplant 18: 2719–2729, 2018 29575655
8. Habbous S, Sarma S, Barnieh LJ, McArthur E, Klarenbach S, Manns B, Begen MA, Lentine KL, Garg AX; Donor Nephrectomy Outcomes Research (DONOR) Network: Healthcare costs for the evaluation, surgery, and follow-up care of living kidney donors. Transplantation 102: 1367–1374, 2018 30044364
9. Graham JM, Courtney AE: The adoption of a one-day donor assessment model in a living kidney donor transplant program: A quality improvement project. Am J Kidney Dis 71: 209–215, 2018 29150247
10. Pradel FG, Jain R, Mullins CD, Vassalotti JA, Bartlett ST: A survey of nephrologists’ views on preemptive transplantation. Clin J Am Soc Nephrol 3: 1837–1845, 2008 18832107
11. LaPointe Rudow D, Hays R, Baliga P, Cohen DJ, Cooper M, Danovitch GM, Dew MA, Gordon EJ, Mandelbrot DA, McGuire S, Milton J, Moore DR, Morgievich M, Schold JD, Segev DL, Serur D, Steiner RW, Tan JC, Waterman AD, Zavala EY, Rodrigue JR: Consensus conference on best practices in live kidney donation: Recommendations to optimize education, access, and care. Am J Transplant 15: 914–922, 2015 25648884
12. Getchell LE, McKenzie SQ, Sontrop JM, Hayward JS, McCallum MK, Garg AX: Increasing the rate of living donor kidney transplantation in Ontario: Donor- and recipient-identified barriers and solutions. Can J Kidney Health Dis 4: 2054358117698666, 2017
13. Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C: Using and reporting the Delphi method for selecting healthcare quality indicators: A systematic review. PLoS One 6: e20476, 2011 21694759
14. Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR, Lázaro P: Loo Mvh, McDonnell J, Vader JP, Kahan JP, The RAND/UCLA Appropriateness Method User’s Manual, 2001. Available at:
https://www.rand.org/content/dam/rand/pubs/monograph_reports/2011/MR1269.pdf 15. Kröger E, Tourigny A, Morin D, Côté L, Kergoat MJ, Lebel P, Robichaud L, Imbeault S, Proulx S, Benounissa Z: Selecting process quality indicators for the integrated care of vulnerable older adults affected by cognitive impairment or dementia. BMC Health Serv Res 7: 195, 2007 18047668
16. Gill FJ, Leslie GD, Grech C, Latour JM: Using a web-based survey tool to undertake a Delphi study: Application for nurse education research. Nurse Educ Today 33: 1322–1328, 2013 23510701
17. Thonon F, Boulkedid R, Teixeira M, Gottot S, Saghatchian M, Alberti C: Identifying potential indicators to measure the outcome of translational cancer research: A mixed methods approach. Health Res Policy Syst 13: 72, 2015 26635108
18. Hanson CS, Chadban SJ, Chapman JR, Craig JC, Wong G, Tong A: Nephrologists’ perspectives on recipient eligibility and access to living kidney donor transplantation. Transplantation 100: 943–953, 2016 26425873
19. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG: Research electronic data capture (REDCap)--A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42: 377–381, 2009 18929686
20. Evangelidis N, Tong A, Manns B, Hemmelgarn B, Wheeler DC, Tugwell P, Crowe S, Harris T, Van Biesen W, Winkelmayer WC, Sautenet B, O’Donoghue D, Tam-Tham H, Youssouf S, Mandayam S, Ju A, Hawley C, Pollock C, Harris DC, Johnson DW, Rifkin DE, Tentori F, Agar J, Polkinghorne KR, Gallagher M, Kerr PG, McDonald SP, Howard K, Howell M, Craig JC; Standardized Outcomes in Nephrology–Hemodialysis (SONG-HD) Initiative: Developing a set of core outcomes for trials in hemodialysis: An international Delphi survey. Am J Kidney Dis 70: 464–475, 2017 28238554
21. Donabedian A: Evaluating the quality of medical care. Milbank Q 83: 691–729, 2005 16279964
22. Mant J: Process versus outcome indicators in the assessment of quality of health care. Int J Qual Health Care 13: 475–480, 2001 11769750
23. van der Veer SN, van Biesen W, Couchoud C, Tomson CRV, Jager KJ: Measuring the quality of renal care: Things to keep in mind when selecting and using quality indicators. Nephrol Dial Transplant 29: 1460–1467, 2014 24286978
24. Brett KE, Ritchie LJ, Ertel E, Bennett A, Knoll GA: Quality metrics in solid organ transplantation: A systematic review. Transplantation 102: e308–e330, 2018 29557915
25. Knight SR, Cao KN, South M, Hayward N, Hunter JP, Fox J: Development of a clinical decision support system for living kidney donor assessment based on national guidelines. Transplantation 102: e447–e453, 2018 30028418
26. Anantha RV, Paskar D, Vogt K, Crawford S, Parry N, Leslie K: Allocating operating room resources to an acute care surgery service does not affect wait-times for elective cancer surgeries: A retrospective cohort study. World J Emerg Surg 9: 21, 2014 24669771
27. Ramos S, Gaio R, Ferreira F, Leal JP, Martinse S, Santos JV, Carvalho I, Duarte R: Tuberculosis in children from diagnosis to decision to treat. Rev Port Pneumol (English Ed) 23: 317–322, 2017
28. Birt L, Scott S, Cavers D, Campbell C, Walter F: Member checking: A tool to enhance trustworthiness or merely a nod to validation? Qual Health Res 26: 1802–1811, 2016 27340178
29. Seawright AH, Taylor L: A systematic approach to postoperative management of deceased donor kidney transplant patients with a clinical pathway. Prog Transplant 21: 43–52, 2011 21485942
30. Leaphart CL, Gonwa TA, Mai ML, Prendergast MB, Wadei HM, Tepas JJ 3rd, Taner CB: Formal quality improvement curriculum and DMAIC method results in interdisciplinary collaboration and process improvement in renal transplant patients. J Surg Res 177: 7–13, 2012 22482757
31. Goldstein SL, Mottes T, Simpson K, Barclay C, Muething S, Haslam DB, Kirkendall ES: A sustained quality improvement program reduces nephrotoxic medication-associated acute kidney injury. Kidney Int 90: 212–221, 2016 27217196
32. Collister D, Rigatto C, Hildebrand A, Mulchey K, Plamondon J, Sood MM, Reslerova M, Arsenio J, Coudiere R, Komenda P: Creating a model for improved chronic kidney disease care: Designing parameters in quality, efficiency and accountability. Nephrol Dial Transplant 25: 3623–3630, 2010 20466693
33. Bosek MSD, Sargeant IL: Living kidney donor advocacy program: A quality improvement project. JONAS Healthc Law Ethics Regul 14: 27–28, 2012 22367017
