Introduction
The Executive Order on Advancing American Kidney Health supports initiatives that prevent kidney failure and increase access to preemptive transplantation, particularly for racial and ethnic minorities (1). Patients with CKD with greater knowledge of their underlying condition and its treatment options are more likely to make informed decisions, complete evaluation for transplant, and receive a living donor kidney transplant (2). Yet, many patients with CKD, particularly those at earlier stages and those whose primary language is not English, may not fully appreciate their disease severity and are not informed regarding the benefits of kidney transplantation compared with dialysis as a treatment for kidney failure (3,4).
Although early, high-quality education about CKD and its optimal treatment options can help (5), valid, reliable measures of CKD and transplant knowledge are also needed to identify those patients in most need of structured educational support and to monitor patients’ knowledge change over time. Knowledge of CKD and kidney failure is often low, especially among patients at earlier CKD stages, who self-identify as Black or African American, whose primary language is not English (6,7), or who face low health literacy or low educational attainment (89–10).
We previously developed and validated a novel scale to measure kidney transplant–related knowledge rigorously among 1294 patients living with kidney failure: the Knowledge Assessment of Renal Transplantation 1.0 (KART 1.0). Although currently in use, KART 1.0 does not assess patient understanding of CKD generally or of transplant-related medication side effects, malignancy risk, or postdonation survival prognosis (11). Moreover, the validation study for the KART 1.0 scale did not include patients at earlier stages in their CKD continuum and those whose primary language was not English, leaving a gap in KART 1.0’s validity and scalability.
To provide nephrologists, researchers, and policy makers with comprehensive, valid, and efficient measures that can be implemented into interdisciplinary education programs for CKD care, we sought to update the KART 1.0 by both developing and testing new survey items and expanding the diversity of patients by CKD stage and race and ethnicity (2,12). From there, we developed and validated two novel instruments: the KART 2.0–Transplant Knowledge Scale and the KART 2.0–CKD Knowledge Scale.
Materials and Methods
Study Data and Participants
Kaiser Permanente Southern California (KPSC) manages the care of almost 24,000 patients with CKD stages 3–5 in Southern California, including Los Angeles, San Diego, and surrounding cities, with 19% of these in CKD stages 3b–5. We collected data assessing knowledge about CKD and the risks and benefits of kidney transplantation and living donation from patients who were completing baseline surveys as part of an ongoing educational trial (#NCT03389932). Inclusion criteria required patients to be active members of KPSC at any medical center, between the ages of 18 and 70, and to be receiving nephrology care for CKD stages 3, 4, and 5. We excluded those who were not continuous members in KPSC for at least 1 year before the incidence of CKD stage 3 and those in CKD stages 3 or 4 with GFRs >59 ml/min per 1.73 m2. Because CKD stages 3 and 4 ICD-10 code use is inconsistently recorded, we defined CKD stage 3 as having at least one eGFR between 30 and 59 ml/min per 1.73 m2 occurring in 2005 or later and at least one additional eGFR measurement after first occurrence of CKD stage 3. CKD stage 5 was defined by using ICD-10 codes (N18.3–6).
The University of California at Los Angeles (UCLA; 18–001099) and KPSC Institutional Review Board (11469) reviewed and approved the study. The protocol for this trial has been published previously (13). The baseline survey assessments, taken before any educational intervention, occurred during five recruitment periods starting in July 2018, November 2018, January 2019, April 2019, and August 2019. The surveys were conducted electronically and were self-completed by participants or over the phone via interviewer administration.
KART 2.0 Instrument Development
We took a multiple-stage approach to developing a new version of the KART instrument. Physicians and health psychologists with expertise in kidney disease and experts in measure development convened a working group to propose new items for the KART 2.0 scales. We started with a set of 48 items measuring participants’ knowledge of uremic symptoms, kidney function, kidney transplant eligibility, and risks for morbidity and mortality associated with kidney transplant receipt and donation. The starting set of items were sourced from the original KART measure development (17 items, including one not ultimately used in KART 1.0) (11), a validated measure of CKD knowledge (15 items) (9), and newly written items (16 items). Newly written items were drafted by a health psychologist with expertise in kidney disease and then reviewed by the working group before the draft instrument was taken to cognitive interviews. The rationale for the choice of topics to include in the measure is given in the Supplemental Material.
Cognitive Interview Study.
Before administering the items to patients in a survey, we conducted a qualitative development phase study consisting of cognitive interviews with a subset of 41 patients recruited from UCLA’s dialysis program. During the cognitive interviews, each patient answered the set of 48 starting items. On average, participants were 57 years old. Most of the participants (n=33) completed the cognitive interview in English, while eight completed it in Spanish. A standardized cognitive interview script was developed and followed for each interview. The script contained 48 transplant and CKD items for the patient to complete, along with 12 predefined probes on specific words or phrases patients may have found confusing (e.g., “Is the meaning of term ‘kidney functioning’ clear? What does it mean to you?”). The cognitive interview script is given in the Supplemental Material. Patients were encouraged to give feedback about each item’s clarity, including response options, and to note any reactions to the items. The participating patients were also probed to clarify specific words or phrases perceived as confusing.
The cognitive interview study led to modifications of four items. The item “Patients have better health outcomes if they receive a transplant before starting dialysis” was modified to “Patients have better health if they receive a transplant before starting dialysis” (KTX3). This change was motivated by nine patients reporting confusion about the meaning of the term “health outcomes.” The next item was “A patient on dialysis has the same level of kidney functioning as a patient with a transplanted kidney,” which was modified to “Compared with having two working kidneys, dialysis works just as well as getting a transplant” (KTX4). This item was modified because nine patients did not understand the term “kidney functioning.” For the same reason, the item, “About what percentage of all transplanted kidneys function for at least 1 year?” was modified to “About what percentage of all transplanted kidneys keep working for at least 1 year?” (KTX25). The last item modified was “In general, how long does it take after surgery before donors are able to return to their normal activities?” After modification, the new item was “In general, after surgery, how long does it take for most donors to return to their normal daily activities?” (KTX21). This item was modified because patients found the wording cumbersome. We omitted 11 items due to feedback from the cognitive interviews or because of analyses conducted as part of the development of KART 1.0.
This process resulted in 37 items that were administered in the trial survey. Table 1 shows each of these items, their response options, and the proportion of patients responding correctly. Supplemental Table 1 shows the proportion of patients giving each raw response category. Twenty-four items were true/false format, five were yes/no format, and eight were multiple choice. Each item was administered with a “don’t know” option. For each item, “don’t know” was counted as incorrect.
Table 1. -
Starting set of
CKD and transplant
knowledge items
Item Name # |
Item Wording |
Response Options |
% Correct |
CKD1 |
Does the kidney make urine? |
Yes, No, Don’t Know |
64 |
CKD2 |
Does the kidney make clean blood? |
Yes, No, Don’t Know |
76 |
CKD3 |
Does the kidney help keep bones healthy? |
Yes, No, Don’t Know |
52 |
CKD4 |
Does the kidney help keep blood pressure normal? |
Yes, No, Don’t Know |
67 |
CKD5 |
Shortness of breath is a symptom of chronic kidney disease. |
True, False, Don’t Know |
45 |
CKD6 |
Confusion is a symptom of chronic kidney disease. |
True, False, Don’t Know |
43 |
CKD7 |
Increased fatigue is a symptom of chronic kidney disease. |
True, False, Don’t Know |
77 |
CKD8 |
Chronic kidney disease increases a person’s chance of a heart attack. |
True, False, Don’t Know |
62 |
CKD9 |
Unusual itching is a symptom of chronic kidney disease. |
True, False, Don’t Know |
47 |
CKD10 |
Nausea and/or vomiting is a symptom of chronic kidney disease. |
True, False, Don’t Know |
50 |
CKD11 |
Metal taste/bad taste in the mouth is a symptom of chronic kidney disease. |
True, False, Don’t Know |
46 |
KTX1 |
Patients older than 70 years of age can receive transplants. |
True, False, Don’t Know |
30 |
KTX2 |
The transplant team will let a living donor back out from donating on the day of the surgery. |
True, False, Don’t Know |
50 |
KTX3 |
Patients have better health if they receive a transplant before starting dialysis. |
True, False, Don’t Know |
40 |
KTX4 |
Compared with having two working kidneys, dialysis works just as well as getting a transplant. |
True, False, Don’t Know |
35 |
KTX5 |
Individuals who donate a kidney have a slightly higher chance of high blood pressure in the future compared with people who don’t donate. |
True, False, Don’t Know |
16 |
KTX6 |
The transplant team cannot tell the kidney patient anything about the donor unless the donor agrees. |
True, False, Don’t Know |
54 |
KTX7 |
You can slow down how fast your kidneys fail. |
True, False, Don’t Know |
62 |
KTX8 |
In general, a living kidney donor will need to stay in the hospital more than 1 week after the surgery. |
True, False, Don’t Know |
16 |
KTX9 |
All living donors’ transportation costs to and from the transplant center are covered by Medicare and/or private insurance. |
True, False, Don’t Know |
10 |
KTX10 |
In general, patients can live longer with a kidney transplant than if they stayed on dialysis. |
True, False, Don’t Know |
57 |
KTX11 |
After a patient is listed, they don’t need to return to the transplant center again until a matching kidney is found. |
True, False, Don’t Know |
28 |
KTX12 |
Donors usually feel the most pain the first week after surgery. |
True, False, Don’t Know |
32 |
KTX13 |
In general, most people on dialysis are happier with the quality of their lives than people with transplants. |
True, False, Don’t Know |
49 |
KTX14 |
Transplant recipients are at risk of developing skin cancer. |
True, False, Don’t Know |
12 |
KTX15 |
Kidney transplants that occur before a patient starts dialysis generally last longer than other transplants. |
True, False, Don’t Know |
19 |
KTX16 |
Tremors are a possible side effect of antirejection medications. |
True, False, Don’t Know |
27 |
KTX17 |
If a patient waits long enough on the waitlist, a matching kidney from someone who has died will definitely become available. |
True, False, Don’t Know |
28 |
KTX18 |
Compared with transplants from donors who have died, how long do transplants from living donors last? |
A Shorter Amount of Time, A Longer Amount of Time, The Same Amount of Time, Don’t Know |
18 |
KTX19 |
How long do patients usually wait on the waiting list for a kidney from someone who has died? |
<1 year, 1–2 years, 3–5 years, >5 years, Don't Know |
10 |
KTX20 |
What is the chance that a donor would die while undergoing transplant surgery? |
<1%, 3%, 10%, 25%, Don't Know |
19 |
KTX21 |
In general, after surgery, how long does it take for most donors to return to their normal daily activities? |
1 week, 1 month, 3 months, 6 months, Don’t Know |
15 |
KTX22 |
Do donors have to pay for testing and hospitalization related to kidney donation? |
Yes, No, Don’t Know |
29 |
KTX23 |
After a transplant, how long does the US Government pay for most of the costs of transplant medications? |
1 year, 3 years, 10 years, For the Rest of the Recipient’s Life, Don’t Know |
4 |
KTX24 |
How long have doctors been doing transplants using living donors? |
2 years, 10 years, 25 years, >50 years, Don't Know |
12 |
KTX25 |
About what percentage of all transplanted kidneys keep working for at least 1 year? |
50%, 75%, 90%, Don’t Know |
14 |
KTX26 |
What is the chance that a recipient would die while undergoing transplant surgery? |
<1%, 3%, 10%, 25%, Don’t Know |
10 |
Item Response Theory and Differential Item Functioning Analyses.
We subjected the 37 items to item response theory (IRT) to refine the instrument further. IRT is a modern approach to patient-reported measure development and evaluation that has grown in medical research in the past two decades (1415–16), which offers the ability to understand multiple aspects of survey questions or items that are especially beneficial for developing measures of health-related knowledge. The IRT analyses for this paper used two-parameter logistic models (one each for transplant knowledge and CKD knowledge) to estimate key item properties: (1) item discrimination (represents how well the item differentiates between patients with higher and lower levels of knowledge); and (2) item difficulty (captures the how difficult it is to give a correct answer to the question). We used the results of these models to refine the item set. Detailed IRT methods are given in the Supplemental Material.
After item exclusions made from the IRT analyses, we examined measurement equivalence by testing differential item functioning (DIF) across multiple clinical patient groups, including race and ethnicity (non-Hispanic Black versus other race and ethnicity), CKD stage (stage 3 versus stages 4 and 5), primary language spoken (English versus Spanish), and sex (woman versus man). These tests determine if each item measures the same thing for each group. Notably, these tests do not compare knowledge levels across groups. Detailed DIF methods are given in the Supplemental Material. Any items exhibiting DIF were considered for exclusion. After all exclusions, the remaining item set(s) were put forward as two new KART scales: KART 2.0–CKD Knowledge and KART 2.0–Transplant Knowledge.
Validity
To test construct validity, we examined mean differences in the new KART 2.0 CKD and transplant scale scores between groups of patients who had previously spoken with health professionals (doctors or other medical staff) about transplant, read information about kidney disease, dialysis, kidney transplant, and living kidney donation for <1 hour versus ≥1 hour with independent sample t tests. Cohen d effect sizes for these tests were calculated as the difference in mean score between groups divided by the pooled SD for the scale. Cohen cutoffs for this effect size estimate were used to determine its magnitude (small, 0.2≤d<0.5; medium, 0.5≤d<0.8; large, d≥0.8) (17).
Results
Participants
We assessed 4074 patients for study eligibility. Of these, 977 patients were deemed eligible, randomized to the study, and completed pre-intervention surveys that were used for this analysis (Supplemental Figure 1). On average, these patients were 59 years old (SD=11). The largest proportion were men (n=554; 57%), identified as Hispanic (n=482; 49%), had received a high school diploma or less (n=390; 40%), and had stage 3 CKD (n=399; 41%). Approximately one third (n=280; 29%) spoke Spanish as their primary language. Most patients had previously read about kidney disease for ≥1 hour (n=592; 75%) and about dialysis for ≥1 hour (n=492; 63%). Less than half of the participants had read about kidney transplantation for ≥1 hour (n=360; 47%), and about a third had read about living donor kidney transplant for ≥1 hour (n=246; 33%). A full listing of participant characteristics is given in Table 2.
Table 2. -
Characteristics of patients with
CKD who participated in a survey on
CKD and transplant
knowledge
Patient Characteristic |
N=977 |
Age, yr, mean (SD, range) |
59 (11, 21–72) |
Sex, n (%)
|
Men |
554 (57) |
Women |
423 (43) |
Self-identified race and ethnicity, n (%)
|
Asian |
93 (10) |
African American or Black |
179 (18) |
White |
223 (23) |
Hispanic or Latino |
482 (49) |
CKD stage at study onset, n (%)
|
3 |
399 (41) |
4 |
341 (35) |
5 |
237 (24) |
Primary language, n (%)
|
English |
697 (71) |
Spanish |
280 (29) |
Maximum educational attainment, n (%)
|
High school diploma or less |
390 (40) |
Some college |
240 (25) |
College degree |
208 (21) |
Some professional or graduate education or higher |
139 (14) |
How often do you have someone (e.g., a family member, friend, hospital/clinic worker, or caregiver) help you read hospital materials?, n (%)
|
None of the time/A little of the time/Some of the time |
794 (82) |
Most of the time/All of the time |
179 (18) |
How confident are you filling out forms by yourself?, n (%)
|
Not at all/A little bit/Somewhat |
285 (29) |
Quite a bit/Very much |
684 (71) |
If your family lost your current income, how long could you continue to live in your current situation?, n (%)
|
Do not have an income |
110 (11) |
<1 mo |
104 (11) |
1–6 mo |
269 (28) |
≥7 mo |
335 (35) |
Prefer not to answer |
153 (16) |
In general, would you say your health is…, n (%)
|
Excellent/Very good/Good |
496 (51) |
Fair/Poor |
472 (49) |
Talked to doctors or medical staff about transplants ≥1 h |
386 (50) |
Read about kidney disease ≥1 h |
592 (75) |
Read about dialysis ≥1 h |
492 (63) |
Read about transplant ≥1 h |
360 (47) |
Read about living donor kidney transplant ≥1 h |
246 (33) |
Percentages are out of the total non-missing for each variable.
IRT Analyses
After making item exclusions from the initial round of IRT analyses, we refit separate two-parameter logistic IRT models for the transplant knowledge items (16 remaining items) and CKD knowledge items (10 remaining items). The analyses resulted in exclusion of no additional transplant knowledge items and one additional CKD knowledge item (CKD6 “Shortness of breath is a symptom of chronic kidney disease”). Details of these analyses are given in the Supplemental Material.
DIF
No items flagged for DIF across groups of race and ethnicity, CKD stage, and sex. One item (KTX9 “All living donors' transportation costs to and from the transplant center are covered by Medicare and/or private insurance”) flagged for DIF by a very small amount. However, because no confusion was expressed by Spanish-speaking patients about this item during cognitive interviews, we elected not to exclude it. Therefore, we did not exclude any items due to DIF. Detailed results are given in the Supplemental Material.
Creation of KART 2.0 CKD Knowledge and Transplant Scales
On the basis of the psychometric analyses described above, we retained two sets of items to create the KART 2.0 instrument, to include two different knowledge scales: (1) KART 2.0—Transplant Knowledge and (2) KART 2.0–CKD Knowledge. Table 3 shows the final set of items retained in each of these and how to score the scales. For KART 2.0–Transplant Knowledge scale, 16 total items were retained. Of these, five items are also included in KART 1.0. Two items from KART 1.0 were modified as a result of cognitive interviews. The remaining nine items are new and introduce new content around risks for adverse events after transplant and living donation, preemptive transplantation, and costs and management of living donation. As legislation was passed to provide lifetime Medicare coverage for immunosuppressants in December 2020, the correct answer to KTX23 (“After a transplant, how long does the US Government pay for most of the costs of transplant medications?”) will be changed from “3 years” to “for the rest of the recipient's life.” The KART 2.0–CKD Knowledge scale was created with the nine remaining items, tapping content on basic facts of CKD and CKD symptoms. These items are also featured in Table 2. None of them were included in the KART 1.0 scale.
Table 3. -
Final items included in
Knowledge Assessment of Renal
Transplantation (KART) 2.0 scales
Item Name |
Item Wording |
Response Options |
Included in KART 1.0? |
Transplant Knowledge scale
|
KTX1 |
Patients older than 70 years can receive transplants. |
True
a
/False |
No |
KTX10 |
In general, patients can live longer with a kidney transplant than if they stayed on dialysis. |
True
a
/False |
Modified |
KTX13 |
In general, most people on dialysis are happier with the quality of their lives than people with transplants. |
True/False
a
|
Yes |
KTX11 |
After a patient is listed, they don’t need to return to the transplant center again until a matching kidney is found. |
True/False
a
|
Yes |
KTX14 |
Transplant recipients are at risk of developing skin cancer. |
True
a
/False |
No |
KTX9 |
All living donors’ transportation costs to and from the transplant center are covered by Medicare and/or private insurance. |
True/False
a
|
No |
KTX12 |
Donors usually feel the most pain the first week after surgery. |
True
a
/False |
No |
KTX5 |
Individuals who donate a kidney have a slightly higher chance of high blood pressure in the future compared with people who don’t donate. |
True
a
/False |
No |
KTX6 |
The transplant team cannot tell the kidney patient anything about the donor unless the donor agrees. |
True
a
/False |
No |
KTX8 |
In general, a living kidney donor will need to stay in the hospital more than 1 week after the surgery. |
True/False
a
|
No |
KTX15 |
Kidney transplants that occur before a patient starts dialysis generally last longer than other transplants. |
True
a
/False |
No |
KTX25 |
About what percentage of all transplanted kidneys keep working for at least 1 year? |
50%, 75%, 90%,
a
Don’t Know |
Modified |
KTX18 |
Compared with transplants from donors who have died, how long do transplants from living donors last? |
A Shorter Amount of Time, A Longer Amount of Time,
a
The Same Amount of Time, Don’t Know |
Yes |
KTX22 |
Do donors have to pay for testing and hospitalization related to kidney donation? |
Yes/No
a
|
Yes |
KTX23 |
After a transplant, how long does the US Government pay for most of the costs of transplant medications? |
1 year, 3 years,
b
10 years, For the Rest of the Recipient’s Life,
a
Don’t Know |
Yes |
KTX21 |
In general, after surgery, how long does it take for most donors to return to their normal daily activities? |
1 week, 1 month,
a
3 months, 6 months, Don’t Know |
Modified |
CKD Knowledge scale
|
CKD2 |
Does the kidney make clean blood? |
Yes
a
/No |
No |
CKD3 |
Does the kidney help keep bones healthy? |
Yes
a
/No |
No |
CKD4 |
Does the kidney help keep blood pressure normal? |
Yes
a
/No |
No |
CKD8 |
Chronic kidney disease increases a person’s chance of a heart attack. |
Yes
a
/No |
No |
CKD9 |
Unusual itching is a symptom of chronic kidney disease. |
Yes
a
/No |
No |
CKD7 |
Increased fatigue is a symptom of chronic kidney disease. |
Yes
a
/No |
No |
CKD10 |
Nausea and/or vomiting is a symptom of chronic kidney disease. |
Yes
a
/No |
No |
CKD5 |
Shortness of breath is a symptom of chronic kidney disease. |
Yes
a
/No |
No |
CKD11 |
Metal taste/bad taste in the mouth is a symptom of chronic kidney disease. |
Yes
a
/No |
No |
aCorrect response.
bAt the time of the study, the correct answer to this item was 3 years in the United States. However, as of December 2020, immunosuppression medications are covered for lifetime for kidney transplant recipients. The correct answer to this item has been now changed to “For the patient’s entire life.”
Scores for the KART 2.0–Transplant Knowledge and KART 2.0–CKD Knowledge scales were created by summing the number of correct responses. Although the IRT models used to create these scales can also be used to generate IRT-based scores for each scale, it may be more practical for many users to create scores by summing item responses, especially for clinical use where KART 2.0 administration and scoring may need to occur in real time. Previous research has demonstrated that summed scores and IRT-generated scores for the same scale are often very highly correlated (>0.9) (18), giving confidence that summed scores will sufficiently capture important variation in IRT scores while offering a more practical way to generate the scores for some users.
The distributions and Cronbach’s α reliabilities of the KART 2.0–Transplant Knowledge and KART 2.0–CKD Knowledge scales are given in Table 4. Relative to their possible maximums (16 and nine, respectively), scores were low, indicating low knowledge of transplant and CKD among this sample, particularly transplant knowledge. The Cronbach’s α values for both scales were around the lower bound, indicating good reliability.
Table 4. -
Score distributions for
Knowledge Assessment of Renal
Transplantation 2.0 scales
Statistic |
Transplant Knowledge Scale |
CKD Knowledge Scale |
Number of items |
16 |
9 |
Mean |
4.2 |
5.2 |
SD |
3.3 |
2.6 |
Minimum |
0 |
0 |
25th percentile |
1 |
3 |
Median |
4 |
5 |
75th percentile |
7 |
7 |
Maximum |
15 |
9 |
Cronbach’s α (95% CI) |
0.8 (0.78 to 0.81) |
0.79 (0.77 to 0.81) |
95% CI, 95% confidence interval.
Validity
Construct validity results for both the KART 2.0–Transplant Knowledge and the KART 2.0–CKD Knowledge scales are presented in Table 5. Each of the scales was able to distinguish between groups of patients who engaged in various types of CKD or transplant education for <1 hour or ≥1 hour. For the KART 2.0–Transplant Knowledge scale, the effect sizes (range 0.33–0.45) for these differences were small for education targeted to CKD knowledge but medium (range 0.67–0.76) for education targeted to transplant knowledge. For the KART 2.0–CKD Knowledge scale, effect sizes were medium for all types of education (range 0.54–0.69).
Table 5. -
Construct validity tests for
Knowledge Assessment of Renal
Transplantation 2.0 scales
|
Transplant Knowledge Scale |
CKD Knowledge Scale |
Mean |
P
|
Effect Size |
Mean |
P
|
Effect Size |
Talked to doctors/medical staff about transplant, h
|
<0.001 |
0.33 |
|
<0.001 |
0.54 |
≥1 |
5 |
|
|
6.1 |
|
|
<1 |
3.9 |
|
|
4.7 |
|
|
Reading about kidney disease, h
|
<0.001 |
0.45 |
|
<0.001 |
0.62 |
≥1 |
4.9 |
|
|
5.8 |
|
|
<1 |
3.4 |
|
|
4.2 |
|
|
Reading about dialysis, h
|
<0.001 |
0.36 |
|
<0.001 |
0.69 |
≥1 |
4.9 |
|
|
6.1 |
|
|
<1 |
3.7 |
|
|
4.3 |
|
|
Reading about kidney transplant, h
|
<0.001 |
0.67 |
|
<0.001 |
0.69 |
≥1 |
5.7 |
|
|
6.4 |
|
|
<1 |
3.5 |
|
|
4.6 |
|
|
Reading about living donor kidney transplant, h
|
<0.001 |
0.76 |
|
<0.001 |
0.62 |
≥1 |
6.2 |
|
|
6.5 |
|
|
<1 |
3.7 |
|
|
4.9 |
|
|
Discussion
Receipt of a kidney transplant confers significant benefit to patients in terms of health-related quality of life, cognitive function, and survival (1920–21). Obtaining valid assessments of patients’ knowledge of CKD and options for transplantation earlier in their disease continuum allows for better disease self-management and encourages patients to pursue timely transplantation (222324252627–28). Because many available CKD and transplant knowledge measures do not publish their psychometric properties or validation processes (27,293031323334–35), we developed and validated a new instrument to measure CKD and transplant-related knowledge (KART 2.0) in a racially, ethnically, and linguistically diverse sample of patients with stages 3–5 CKD.
KART 2.0 contains two scales: the Transplant Knowledge scale and the CKD Knowledge scale. These two scales are scored separately and can be used together or independently. KART 2.0 improves upon KART 1.0 by adding new, important content about general CKD facts and symptoms, and additional content on transplantation, especially living donor kidney transplant. This new content expands KART’s applicability to assess CKD and transplant knowledge among patients earlier in their disease continuum and deepens KART’s ability to evaluate educational activities targeted toward living donor kidney transplant.
Critically, the new KART scales demonstrated measurement equivalence across a diverse range of patient groups, indicating that results are not biased toward one particular group over another. In our study sample, >20% of the participants reported low health literacy, and 40% had an educational attainment of high school diploma or less. In addition, many participants identified as racial and ethnic minorities. Notably, 49% identified as Hispanic/Latinx and 29% spoke Spanish as their primary language. Our DIF analyses determined if the KART 2.0 items were equally applicable to different racial and ethnic and language groups (36). Showing no or only trivial DIF is required to make valid comparisons between patient groups because observed differences could be due to DIF (37). Therefore, demonstrating measurement equivalence, or lack of DIF, is particularly important when CKD or transplant knowledge is the end point of an intervention aiming to reduce disparities in CKD outcomes or access to treatment (38). Given persistent, even deepening racial disparities in access to living donor kidney transplant (39) and in early referral for transplant (40), the KART 2.0 scales’ measurement equivalence across Black versus non-Black patients and English and Spanish speakers is particularly important because it indicates their ability to make valid comparisons of knowledge across these groups.
The KART 2.0 CKD Knowledge and Transplant Knowledge scales will be useful for clinical assessment of patients in community CKD clinics and dialysis clinics, or for patients on the transplant waiting list. Although a rigorous approach to item selection using IRT was used for analysis, the KART 2.0’s simple scoring approach makes it easy to implement, even using hand-scoring with paper and pencil administration. With coronavirus disease 2019 hastening the uptake of telemedicine in nephrology (41), the KART 2.0 scales can also be administered remotely as part of virtual education programs. As e-learning and home-based CKD and transplant education opportunities prove to be effective (4243–44), they are likely to become more common.
This study has key limitations and strengths to consider. First, although the sample of patients used to develop and validate the KART 2.0 scales was diverse across a range of factors, it was drawn only from CKD clinics within the Southern California Kaiser Permanente system. Therefore, these results may not generalize easily to patients in other regions of the United States or internationally. Additional work should be undertaken to replicate these results in additional patient samples. Second, although the KART 2.0 CKD Knowledge and Transplant Knowledge scales cover a wide range of content, they do not have knowledge questions about optimal CKD management, dialysis, or alternative transplant options such as paired kidney donation. Future efforts to develop KART further will seek to incorporate these and other emerging topics. Also, policies and clinical realities around kidney transplant and CKD change over time, potentially affecting which item responses are correct. For this reason, the new KART 2.0 scales may need to be updated in the future. Finally, although this study was able to establish several key psychometric properties of the KART 2.0 scales, we were not able to test responsiveness to change. Future studies with longitudinal design should be undertaken for this purpose.
Advancing the patient centeredness of CKD care first requires ensuring that patients have a comprehensive understanding of their disease and treatment options. The KART 2.0–Transplant Knowledge scale and the KART 2.0–CKD Knowledge scale address this need by giving clinicians and researchers valid and reliable tools to identify those patients in most need of education and counseling. Only when patients living with CKD can make informed choices about their care and treatment options regardless of racial and ethnic background, literacy level, or language of origin can we optimize health outcomes for all.
Disclosures
K.L. Cavanaugh reports employment with Tennessee Valley Health Systems-Nashville VA; consultancy agreements with Kidney Health Initiative, REATA Pharmaceuticals, and Responsum Health; ownership interest in HCA Healthcare; serving in an advisory or leadership role for National Kidney Foundation KDQOI education committee, American Journal of Kidney Diseases Editorial Board, CJASN Editorial Board, and Medical Decision Making Editorial Board; and serving as an Associate Editor for Kidney360. B.S. Mittman reports employment with Kaiser Permanente Southern California and serving in an advisory or leadership role with the National Institutes of Health (federal) and Patient-Centered Outcomes Research Institute (nonprofit). D. Nair reports employment with Tennessee Valley Veterans Affairs Healthcare System; serving in an advisory or leadership role for American Journal of Kidney Diseases Editorial Board, American Society of Nephrology Ethics and Professional Standards Committee, CMS CKD/ESRD Expert Workgroup for Measures Development, International Society of Nephrology North American and Caribbean Regional Board, and National Kidney Foundation KDOQI Education Committee; and other interests or relationships with Tennessee Kidney Foundation. J.D. Peipert reports consultancy agreements with AstraZeneca and FACIT.org; research funding from Bristol-Myers Sqibb, Clovis Oncology, Pfizer, and Veloxis Pharmaceuticals; and service on the Journal of Patient Reported Outcomes Editorial Board. I. Purnajo reports employment with Clinical Outcomes Solutions. A.D. Waterman reports consultancy agreements with Redesign Health, serving as National Kidney Registry Education Manager, participating in a speakers’ bureau for Astellas Pharmaceutical Company, and other interests or relationships with Health Literacy Media and Transplant Recipients’ International Organization.
Funding
A.D. Waterman was supported by National Institutes of Health (NIH), National Center for Advancing Translational Sciences, University of California, Los Angeles, CTSI grants UL1 TR000124 and UL1 TR001881, and Health Resources and Services Administration grant R39OT29879. D. Nair and K.L. Cavanaugh were supported by NIH grants K12 HS026395 and P30DK114809.
Acknowledgments
We sincerely thank the patients who contributed the data used in this study. We thank Angelika Alem, Lizeth Ambriz, Crystal Anderson, Jennifer L. Beaumont, Neha Bijjala, Bhanuja Dub, Ron D. Hays, and Shayna L. Henry for their significant contributions to the design and execution of the parent study that made these data possible.
Author Contributions
K.L. Cavanaugh, B.S. Mittman, J. Peipert, and A.D. Waterman conceptualized the study; K.L. Cavanaugh, J. Peipert, and A.D. Waterman were responsible for the methodology; B.S. Mittman, J. Peipert, and A.D. Waterman were responsible for the investigation; D. Nair, J. Peipert and A.D. Waterman wrote the original draft of the manuscript; J. Peipert was responsible for visualization; J. Peipert, I. Purnajo, and A.D. Waterman curated the data and conducted the formal analysis; J. Peipert and A.D. Waterman were responsible for project administration; A.D. Waterman was responsible for funding acquisition and supervision; and all authors reviewed and editing the manuscript.
Supplemental Material
This article contains the following supplemental material online at http://cjasn.asnjournals.org/lookup/suppl/doi:10.2215/CJN.11490821/-/DCSupplemental.
Rationale for choice of topics to include in the KART 2.0 measure.
Supplemental Table 1. Frequency and proportion of raw responses to each item (N=977).
Detailed item response theory (IRT) methods.
Detailed differential item functioning (DIF) methods.
Supplemental Figure 1. Patient recruitment and inclusion flow chart.
Results of tests of IRT assumptions.
Exclusions from the initial round of IRT analyses.
Results of second round of IRT analyses.
Detailed results of DIF analyses.
References.
References
1. Lin E, Ginsburg PB, Chertow GM, Berns JS: The “Advancing American Kidney Health” Executive Order: Challenges and opportunities for the large dialysis organizations. Am J Kidney Dis 76: 731–734, 2020
2. Waterman AD, Peipert JD, Hyland SS, McCabe MS, Schenk EA, Liu J: Modifiable patient characteristics and racial disparities in evaluation completion and living donor transplant. Clin J Am Soc Nephrol 8: 995–1002, 2013
3. Kaltsouda A, Skapinakis P, Damigos D, Ikonomou M, Kalaitzidis R, Mavreas V, Siamopoulos KC: Defensive coping and health-related quality of life in chronic kidney disease: A cross-sectional study. BMC Nephrol 12: 28, 2011
4. Kazley AS, Johnson EE, Simpson KN, Chavin KD, Baliga P: Health care provider perception of chronic kidney disease:
Knowledge and behavior among African American patients. BMC Nephrol 15: 112, 2014
5. Narva AS, Norton JM, Boulware LE: Educating patients about
CKD: The path to self-management and patient-centered care. Clin J Am Soc Nephrol 11: 694–703, 2016
6. Gordon EJ, Mullee JO, Ramirez DI, MacLean J, Olivero M, Feinglass J, Carney P, O’Connor K, Caicedo JC: Hispanic/Latino concerns about living kidney donation: A focus group study. Prog Transplant 24: 152–162, 2014
7. Gordon EJ, Reddy E, Gil S, Feinglass J, Rodde J, Abecassis MM, Caicedo JC: Culturally competent transplant program improves Hispanics’
knowledge and attitudes about live kidney donation and transplant. Prog Transplant 24: 56–68, 2014
8. Wright Nunes JA, Wallston KA, Eden SK, Shintani AK, Ikizler TA, Cavanaugh KL: Associations among perceived and objective disease
knowledge and satisfaction with physician communication in patients with chronic kidney disease. Kidney Int 80: 1344–1351, 2011
9. Wright JA, Wallston KA, Elasy TA, Ikizler TA, Cavanaugh KL: Development and results of a kidney disease
knowledge survey given to patients with
CKD. Am J Kidney Dis 57: 387–395, 2011
10. Lissanu L, Lopez F, King A, Robinson E, Almazan E, Metoyer G, Quinn M, Peek ME, Saunders MR: “I try not to even think about my health going bad”: A qualitative study of chronic kidney disease
knowledge and coping among a group of urban African-American patients with
CKD. J Racial Ethn Health Disparities 6: 625–634, 2019
11. Peipert JD, Hays RD, Kawakita S, Beaumont JL, Waterman AD: Measurement characteristics of the
knowledge assessment of renal
transplantation.
Transplantation 103: 565–572, 2019
12. Gupta N, Salter ML, Garonzik-Wang JM, Reese PP, Wickliffe CE, Dagher NN, Desai NM, Segev DL: Actual and perceived
knowledge of kidney
transplantation and the pursuit of a live donor.
Transplantation 98: 969–973, 2014
13. Waterman AD, Anderson C, Alem A, Peipert JD, Beaumont JL, Henry SL, Dub B, Ambriz L, Bijjala N, Lipsey AF, Mittman B: A randomized controlled trial of
Explore Transplant at Home to improve transplant
knowledge and decision-making for
CKD 3-5 patients at Kaiser Permanente Southern California. BMC Nephrol 20: 78, 2019
14. Cappelleri JC, Jason Lundy J, Hays RD: Overview of classical test theory and item response theory for the quantitative assessment of items in developing patient-reported outcomes measures. Clin Ther 36: 648–662, 2014
15. Hays RD, Morales LS, Reise SP: Item response theory and health outcomes measurement in the 21st century. Med Care 38: II28–II42, 2000
16. Edelen MO, Reeve BB: Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Qual Life Res 16: 5–18, 2007
17. Cohen J: Statistical Power Analysis for the Behavioral Sciences, 2nd Ed., New York, Academic Press, 1988
18. Fan X: Item response theory and classical test theory: An empirical comparison of their item/person statistics. Educ Psychol Meas 58: 357–381, 1998
19. Molnar-Varga M, Molnar MZ, Szeifert L, Kovacs AZ, Kelemen A, Becze A, Laszlo G, Szentkiralyi A, Czira ME, Mucsi I, Novak M: Health-related quality of life and clinical outcomes in kidney transplant recipients. Am J Kidney Dis 58: 444–452, 2011
20. Chu NM, Gross AL, Shaffer AA, Haugen CE, Norman SP, Xue QL, Sharrett AR, Carlson MC, Bandeen-Roche K, Segev DL, McAdams-DeMarco MA: Frailty and changes in cognitive function after kidney
transplantation. J Am Soc Nephrol 30: 336–345, 2019
21. Hart A, Smith JM, Skeans MA, Gustafson SK, Wilk AR, Robinson A, Wainwright JL, Haynes CR, Snyder JJ, Kasiske BL, Israni AK: OPTN/SRTR 2016 annual data report: Kidney. Am J Transplant 18: 18–113, 2018
22. Nair D, Cavanaugh KL: Measuring patient activation as part of kidney disease policy: Are we there yet? J Am Soc Nephrol 31: 1435–1443, 2020
23. King A, Lopez FY, Lissanu L, Robinson E, Almazan E, Metoyer G, Tanumihardjo J, Quinn M, Peek M, Saunders M: Renal replacement
knowledge and preferences for African Americans with chronic kidney disease. J Ren Care 46: 151–160, 2020
24. Havas K, Douglas C, Bonner A: Meeting patients where they are: Improving outcomes in early chronic kidney disease with tailored self-management support (the
CKD-SMS study). BMC Nephrol 19: 279, 2018
25. Alikari V, Tsironi M, Matziou V, Tzavella F, Stathoulis J, Babatsikou F, Fradelos E, Zyga S: The impact of
education on
knowledge, adherence and quality of life among patients on haemodialysis. Qual Life Res 28: 73–83, 2019
26. Ilori TO, Enofe N, Oommen A, Odewole O, Ojo A, Plantinga L, Pastan S, Echouffo-Tcheugui JB, McClellan W: Factors affecting willingness to receive a kidney transplant among minority patients at an urban safety-net hospital: A cross-sectional survey. BMC Nephrol 16: 191, 2015
27. Lopez-Vargas PA, Tong A, Howell M, Craig JC: Educational interventions for patients with
CKD: A systematic review. Am J Kidney Dis 68: 353–370, 2016
28. Zimbudzi E, Lo C, Ranasinha S, Kerr PG, Usherwood T, Cass A, Fulcher GR, Zoungas S: Self-management in patients with diabetes and chronic kidney disease is associated with incremental benefit in HRQOL. J Diabetes Complications 31: 427–432, 2017
29. Rodrigue JR, Cornell DL, Lin JK, Kaplan B, Howard RJ: Increasing live donor kidney
transplantation: A randomized controlled trial of a home-based educational intervention. Am J Transplant 7: 394–401, 2007
30. Rodrigue JR, Cornell DL, Kaplan B, Howard RJ: A randomized trial of a home-based educational approach to increase live donor kidney
transplantation: Effects in Blacks and Whites. Am J Kidney Dis 51: 663–670, 2008
31. Rodrigue JR, Pavlakis M, Egbuna O, Paek M, Waterman AD, Mandelbrot DA: The “House Calls” trial: A randomized controlled trial to reduce racial disparities in live donor kidney
transplantation: Rationale and design. Contemp Clin Trials 33: 811–818, 2012
32. Bornemann K, Croswell E, Abaye M, Bryce CL, Chang CH, Good DS, Freehling Heiles CA, Dew MA, Boulware LE, Tevar AD, Myaskovsky L: Protocol of the KTFT-TALK study to reduce racial disparities in kidney transplant evaluation and living donor kidney
transplantation. Contemp Clin Trials 53: 52–59, 2017
33. Strigo TS, Ephraim PL, Pounds I, Hill-Briggs F, Darrell L, Ellis M, Sudan D, Rabb H, Segev D, Wang NY, Kaiser M, Falkovic M, Lebov JF, Boulware LE: The TALKS study to improve communication, logistical, and financial barriers to live donor kidney
transplantation in African Americans: Protocol of a randomized clinical trial. BMC Nephrol 16: 160, 2015
34. Patzer RE, Basu M, Mohan S, Smith KD, Wolf M, Ladner D, Friedewald JJ, Chiles M, Russell A, McPherson L, Gander J, Pastan S: A randomized controlled trial of a mobile clinical decision aid to improve access to kidney
transplantation: iChoose Kidney. Kidney Int Rep 1: 34–42, 2016
35. Arriola KR, Powell CL, Thompson NJ, Perryman JP, Basu M: Living donor transplant
education for African American patients with end-stage renal disease. Prog Transplant 24: 362–370, 2014
36. Teresi JA, Fleishman JA: Differential item functioning and health assessment. Qual Life Res 16: 33–42, 2007
37. Teresi JA, Ramirez M, Lai JS, Silver S: Occurrences and sources of differential item functioning (DIF) in patient-reported outcome measures: Description of DIF methods, and review of measures of depression, quality of life and general health. Psychol Sci Q 50: 538, 2008
38. Roberts BT, Rodgers GP: NIDDK initiatives addressing health disparities in chronic diseases. J Clin Invest 130: 5036–5038, 2020
39. Purnell TS, Luo X, Cooper LA, Massie AB, Kucirka LM, Henderson ML, Gordon EJ, Crews DC, Boulware LE, Segev DL: Association of race and ethnicity with live donor kidney
transplantation in the United States from 1995 to 2014. JAMA 319: 49–61, 2018
40. Gander JC, Zhang X, Plantinga L, Paul S, Basu M, Pastan SO, Gibney E, Hartmann E, Mulloy L, Zayas C, Patzer RE: Racial disparities in preemptive referral for kidney
transplantation in Georgia. Clin Transplant 32: e13380, 2018
41. Truong T, Dittmar M, Ghaffari A, Lin E: Policy and pandemic: The changing practice of nephrology during the coronavirus disease-2019 outbreak. Adv Chronic Kidney Dis 27: 390–396, 2020
42. Waterman AD, Peipert JD, McSorley AM, Goalby CJ, Beaumont JL, Peace L: Direct delivery of kidney transplant
education to Black and low-income patients receiving dialysis: A randomized controlled trial. Am J Kidney Dis 74: 640–649, 2019
43. Li W-Y, Chiu F-C, Zeng J-K, Li YW, Huang SH, Yeh HC, Cheng BW, Yang FJ: Mobile health app with social media to support self-management for patients with chronic kidney disease: Prospective randomized controlled study. J Med Internet Res 22: e19452, 2020
44. Joyce M, Durant L, Emre S, Haakinson D, Hammers L, Hughes L, Ventura K, Wuerth D, Liapakis A: Expansion of patient
education programming regarding live donor liver
transplantation via virtual group encounters during the COVID-19 pandemic. Transplant Proc 53: 1105–1111, 2021