Share this article on:

Measuring compliance with drug regimens after renal transplantation: comparison of self-report and clinician rating with electronic monitoring

Butler, Janet A.1 5; Peveler, Robert C.1; Roderick, Paul2; Horne, Robert3; Mason, Juan C.4

doi: 10.1097/01.TP.0000110412.20050.36

Nonadherence to immunosuppressants in renal transplant recipients is a major factor affecting graft survival, but it is difficult to detect accurately in clinical practice. Adherence was measured in 153 adult renal transplant recipients using self-report questionnaires and interview, clinician rating, and cyclosporine levels. The sensitivity and specificity of these measures were determined by comparison with electronic monitoring in a randomly selected subsample of 58 subjects. Measures of adherence in current clinical use do not perform well when tested against electronic monitoring. Self-report at a confidential interview was the best measure of adherence for the detection of both missed doses and erratic timing of medication. However, the use of a confidential interview is not directly applicable to a clinical setting. Further research on how best to facilitate disclosure in clinical settings may be the best way to develop adherence measures for use in routine practice.

1 University Mental Health Group, Royal South Hants Hospital, Southampton, United Kingdom.

2 University Healthcare Research Unit, Southampton General Hospital, Southampton, United Kingdom.

3 Centre for Health Care Research, University of Brighton, Falmer, Brighton.

4 Wessex Renal Unit, Queen Alexandra Hospital, Portsmouth, United Kingdom.

Funded by the Medical Research Council as part of a Clinical Research Training Fellowship for Janet Butler, Ph.D.

5 Address correspondence to: Janet A. Butler, Ph.D., University Mental Health Group, Royal South Hants Hospital, Brintons Terrace, Southampton SO14 0YG, United Kingdom.

Received 13 May 2003.

Revised 29 July 2003.

Accepted 12 August 2003.

Noncompliance (also known as nonadherence) with immunosuppressants seems to be a major factor influencing renal transplant survival, but it is difficult to detect in clinical settings. Clinicians use clinical interviews, medical records, and drug levels (1) to indicate poor adherence; however, the accuracy of these measures is not known. Adherence ratings vary between different professionals(2), and biochemical assays are influenced by pharmacokinetic factors and usually only indicate recent drug consumption (3). Self-report can be used, but it is recognized to lack sensitivity (4,5). Although not directly applicable to a clinical setting, disclosure to an independent researcher seems to be more accurate than disclosure to clinical staff (6). Electronic monitoring is now recommended as the best measure of adherence for research studies (7). It is unique in detecting temporal patterns of adherence (3) and therefore could be used to assess other measures (8).

This study uses clinically relevant measures to assess the frequency of nonadherence and to identify the best method for clinical practice by assessing its sensitivity and specificity compared with electronic monitoring.

Back to Top | Article Outline


All patients from one U.K. transplant unit who were aged more than 18 years, who had a functioning renal graft, and who underwent transplantation 6 to 63 months before recruitment were eligible. Subjects residing in the Channel Islands (two subjects) or outside the region served by the unit (22 subjects) and those unable to give informed consent (two subjects) were excluded.

Adherence was measured in all subjects by several self-report questionnaires (Morisky questionnaire (9); Medication Adherence Rating Scale [MARS], R. Horne, Ph.D., personal communication, 1999; single item about dose timing). The Morisky questionnaire is a four-item categoric measure requiring all items (e.g., “do you ever forget to take your medicine?”) to be rated as “yes=1” or “no=0.” However, adherence is likely to be dimensional rather than categoric, and it was hypothesized that subjects may be reluctant to disclose nonadherence. Therefore, the MARS questionnaire was also used. This is composed of items rated from “1=always” to “5=never,” thus allowing for varying degrees of adherence to be reported. The MARS was not used as the sole questionnaire because validation data have not been published. Pilot studies indicate that the questionnaire has good internal reliability and correlates with the Morisky questionnaire (R. Horne, Ph.D., personal communication, 1999). A single item was added to the MARS questionnaire asking about delay in taking medication because patients are often told to take their cyclosporine at 12-hr intervals. Adherence was also assessed by self-report at interview, clinician and interviewer rating, and cyclosporine levels in those taking cyclosporine. Nephrologists and the interviewer, blind to questionnaire responses, rated how often they thought subjects missed or were late in taking their immunosuppressants on 5-point scales designed for this study. The range and lowest of the last six cyclosporine levels before the interview assessment of adherence were used as biochemical measures of adherence.

A stratified random subsample (using time since transplantation) of 60 subjects received electronic monitors containing prednisolone. The number of monitored days was counted between the first and last openings of the bottle. Intervals were excluded if the subject told the researcher that he or she had not used the container for that specified period, such as during hospital admissions. If two openings occurred on 1 day, it was assumed that one opening occurred in error.

Two continuous measures of nonadherence were calculated:

Back to Top | Article Outline

Erratic Dosing

Erratic dosing was defined as follows: daily dosing=standard deviation (SD) of inter-dose intervals less than 48 hr and alternate-day dosing=SD of inter-dose intervals less than 72 hr.

Subjects missing at least 20% of days of medication were defined as nonadherent according to missed medication, and an SD of inter-dose intervals of at least 6 hr was used to indicate erratic dosing.

The consequence of failing to detect nonadherence may be severe (graft loss), so measures to detect nonadherence should be highly sensitive. However, nonadherence is likely to be relatively uncommon, and increased sensitivity of a measure will result in more patients being incorrectly identified as nonadherent. Although unlikely to be harmful, unnecessary intervention to improve adherence will add to the cost. Receiver-operator curves were plotted to identify the cutoff of each measure to optimize both the sensitivity and specificity. The aim was to find a measure that had at least a sensitivity of 80% and a positive predictive value of 60%. These arbitrary figures were chosen to lead to an acceptable number of nonadherent cases being missed whereas targeting an intervention for a feasible number of patients.

Back to Top | Article Outline


Sample Characteristics

There were no significant differences in sociodemographic or transplant-related factors between subjects who consented (153 subjects) and refused (19 subjects). Baseline factors, except gender, did not differ between subjects who were prescribed prednisolone (eligible for electronic monitoring) and those who stopped taking steroids. More men continued to take prednisolone (χ2=7.14, P =0.01, df =1). Sixty monitors were distributed and offered to the next subject in the randomly ordered list of subjects if one refused the monitor (five subjects) or was not prescribed steroids (20 subjects). One monitor was not recovered because the subject died during the monitoring period. Data from another monitor were not used because the subject reported decanting the medication from the container once per week. Therefore, 58 subjects had available data from electronic monitoring (Table 1).

Table 1

Table 1

Back to Top | Article Outline

Distribution of Nonadherence According to Electronic Monitoring

Electronic monitoring showed a highly skewed distribution of the percentage of days when medication was missed but a wide variation in the time of taking medication (Table 2). Although 55% of subjects did not miss any medication during the monitored period, 7 (12%; 95% confidence interval [CI] 4%–20%) and 15 subjects (26%; CI 32%–58%) missed at least 20% or 10% of days, respectively. Twenty-six subjects (45%; CI 32%–58%) had an SD of inter-dose intervals of at least 6 hr. All subjects except one missing at least 20% of days of prednisolone also had an SD of inter-dose intervals of at least 6 hr.

Table 2

Table 2

Back to Top | Article Outline

Distribution of Adherence According to Other Measures

All measures, except the normally distributed lowest cyclosporine level, showed a highly skewed distribution, with most subjects being classified at the adherent end of the measure (Table 2). Ninety-five and 99 subjects (63% and 66%, respectively) reported complete adherence on the MARS (score of 25) and Morisky (score of 0) questionnaires, respectively. Only 14 and 12 subjects (9% and 8%, respectively) scored 23 or less on the MARS questionnaire or 2 or more on the Morisky questionnaire, respectively. “Forgetting” was the most common reason given for nonadherence on the MARS questionnaire, but seven subjects (5%) reported altering the dose, seven subjects (5%) reported taking less medication than instructed, four subjects (3%) reported stopping immunosuppressants intermittently, and four subjects (3%) reported deciding to miss a dose. During the interview, fewer subjects reported perfect adherence, especially for timing of medication (Table 2). Nephrologists tended to estimate lower frequencies of nonadherence than the interviewer (Table 2).

Back to Top | Article Outline

Comparison of Other Measures with Electronic Monitoring

Compared with electronic monitoring, the best measure of adherence was the self-report at the research interview of taking immunosuppressants more than 2 hr late at least occasionally (Table 3). However, even this did not have the desired sensitivity of 80% and positive predictive value of 60%. Self-report questionnaires only misclassified one third of subjects, but they lacked sensitivity. The accuracy of measures was similar when used to identify erratic dosing. No continuous measure of adherence correlated well with the continuous description of adherence from electronic monitors (data available from the authors on request).

Table 3

Table 3

Back to Top | Article Outline


Currently used measures of adherence, cyclosporine levels, and clinician rating did not satisfactorily differentiate adherence between subjects compared with electronic monitoring. The poor performance of cyclosporine levels was surprising and may be attributable to better adherence of subjects just before clinic visits, so-called white-coat adherence (10). Subjects rated by clinicians as “very rarely” missing medication, the cutoff needed to raise the sensitivity of the rating more than 14%, are unlikely to represent those patients whom clinicians consider as at risk from nonadherence.

Self-report measures misclassified the least subjects compared with classification by electronic monitoring. The distribution of self-reported adherence using the MARS questionnaire had less variance than in other patient groups (R. Horne, Ph.D., personal communication, 2002), which may indicate a greater reluctance of transplant recipients to report nonadherence, particularly if responses occur on paper. An interview seemed to improve the sensitivity of self-report.

The sensitivities and specificities calculated in this study need to be treated with some caution because few subjects were identified as nonadherent by electronic monitoring. However, results did not change significantly when subjects missing at least 10% rather than 20% of days of medication (15 instead of 7 subjects) were classified as nonadherent. A further limitation of the study is comparison of electronic monitoring of prednisolone with measurement of adherence to immunosuppressants in general, or cyclosporine specifically, by other measures. However, data from electronic monitors were interpreted to give conservative estimates of nonadherence, and none of the three studies reporting prevalence of nonadherence to different immunosuppressants (11–13) report adherence to prednisolone to be worse than adherence to other immunosuppressants. Although the better performance of self-report at interview compared with questionnaires may be explained by subjects deducing the purpose of the monitor and thus more accurately reporting their adherence, the researcher did not note any evidence of this during the interviews.

Back to Top | Article Outline


Results show two independent patterns of nonadherence: relatively infrequently missed medication and commonly occurring erratic ingestion of tablets. Research is needed to identify to what degree each aspect of nonadherence contributes to clinical risk and whether subjects who both miss medication and take it erratically have a higher risk than subjects who only miss medication. Self-report at interview was the most accurate measure compared with electronic monitoring. Further research on how best to facilitate disclosure in clinical settings may be the best way to develop a measure of adherence to use in routine practice.

Back to Top | Article Outline


1. Hathaway DK, Combs C, De Geest S, et al. Patient compliance in transplantation: a report on the perceptions of transplant clinicians. Transplant Proc 1999; 31: 10S–13S.
2. Green K, Tapson J, Gerstenkorn C, et al. Determining noncompliance by nurse practitioner. Abstract for the First European Conference on Non-compliance in Transplantation 1999; Salzburg February 4–6, 1999.
3. Farmer KC. Methods for measuring and monitoring medication regime adherence in clinical trials and clinical practice. Clin Ther 1999; 21: 1074–1090.
4. George C, Peveler R, Heliger S, et al. Compliance with tricyclic antidepressants: the value of four different methods of assessment. Br J Clin Pharmacol 2000; 50: 166–171.
5. Waterhouse DM, Calzone KA, Mele C, et al. Adherence to oral tamoxifen: a comparison of patient self-report, pill counts, and microelectronic monitoring. J Clin Oncol 1993; 11: 1189–1197.
6. De Geest S, Borgermans L, Gemoets H, et al. Incidence, determinants, and consequences of subclinical noncompliance with immunosuppressive therapy in renal transplant recipients. Transplantation 1995; 59: 340–347.
7. De Geest S, Vanhaecke J. Methodological issues in transplant compliance research. Transplant Proc 1999; 31: 81S–83S.
8. McGavock H. A review of the literature on drug adherence. 1996. Royal Pharmaceutical Society of Great Britain & Merck Sharp & Dohme.
9. Morisky DE, Green LW, Levine DM. Concurrent and predictive validity of a self-reported measure of medication adherence. Med Care 1986; 24: 67–74.
10. Cramer JA, Scheyer RD, Mattson RH. Compliance declines between clinic visits. Arch Intern Med 1990; 150: 1509–1510.
11. Hilbrands LB, Hoitsma AJ, Koene RA. Medication compliance after renal transplantation. Transplantation 1995; 60: 914–920.
12. Siegal BR, Greenstein SM. Postrenal transplant compliance from the perspective of African-Americans, Hispanic-Americans, and Anglo-Americans. Adv Ren Replace Ther 1997; 4: 46–54.
13. Feldman HI, Hackett M, Bilker W, et al. Potential utility of electronic drug compliance monitoring in measures of adverse outcomes associated with immunosuppressive agents. Pharmacoepidemiol Drug Saf 1999; 8: 1–14.
© 2004 Lippincott Williams & Wilkins, Inc.