Medication is a vital component in the treatment of patients with severe mental disorders. Nonadherence to prescribed psychoactive medication greatly increases the risk of illness exacerbation and rehospitalization.1-3 Rates of nonadherence to antipsychotics in schizophrenia patients have been reported in the range from 11% to 89%,4,5 and a recent review found a mean rate of 41% (range, 20%-56%).6 In bipolar disorder, adherence to long-term prophylactic pharmacotherapy ranges from 20% to 66%,7 and the importance of improving adherence has been firmly established.8,9 A recent study of adherence to antipsychotic medication in bipolar disorder found that 48% were partially adherent or nonadherent.10 The variation in reported adherence seems to reflect inconsistency of study design and methods, which makes it difficult to compare results and limits the generalizability of the findings.6,7,9,11,12 However, some of the variation could also reflect true differences in adherence due to characteristics of the samples or the health care systems. Identifying such differences could provide important information about factors related to adherence. Thus, it is of importance to measure adherence in large, well-described samples from different health care systems.
There are several methods for adherence measurements, both direct methods such as observing patients swallowing tablets and measurements of level of medicine or metabolites in the blood and indirect methods such as self-reports and electronic medication monitors.13 Each method has advantages and disadvantages, and no method is considered the criterion standard. Measuring the level of medicine in the blood is an objective direct method that is regarded highly reliable but depends on a strict protocol and is costly.13,14 The method has, however, rarely been used in adherence studies in severe mental illness. In studies of oral antipsychotics, only 7 of 161 studies used blood levels of medicine as a part of adherence evaluation.9 Serum concentrations are used routinely for lithium treatment in bipolar disorders,7,15 but in adherence studies of mood stabilizers, structured interviews16,17 or pharmacy records are mostly used.18 Thus, adherence estimates based on blood levels of a range of medicines in a large, well-characterized sample of patients with severe mental disorders will provide new information.
Because of the important clinical implications of nonadherence, there is a need for a simple adherence tool for both research and clinical practice. Despite obvious problems such as self-presentational and recall biases,19-21 self-reports can simply and effectively measure adherence13 and seems to have high specificity but lower sensitivity.22 Others have addressed the importance of the type of self-report and found that questionnaires are more concordant with objective measures than interviews are.23 The utility of self-report scales in severe mental disorders, such as schizophrenia, has been criticized, but despite more than 160 studies, there have been surprisingly few attempts to validate self-reports with objective measures in these patients.11 Short self-report scales seem to correlate to more thorough self-reports, as shown from the comparison of the Brief Evaluation of Medication Influences and Beliefs scale with the more thorough Drug Attitude Inventory.24 This study used refill records as an objective nonadherence measure. A recent schizophrenia study compared the structured self-report Brief Adherence Rating Scale with electronic medication monitors, with promising results.25 Another structured self-report, the Medication Adherence Report Scale (MARS), was found useful, but compared only with subjective measures.26 In a comparison of simple ratings from schizophrenia patients, research assistants, and prescribers, electronic medication monitors were used.20 Only 1 study in schizophrenia has compared adherence measures to blood level of medication.27 However, this study was limited by a relatively small sample (n = 53) and used individual test-retest variability of medicine concentration as nonadherence criteria, without comparison to standardized levels.27 Scott and Pope28,29 used 3 different self-reports to get a more robust adherence measure in patients taking mood stabilizers. They defined partial adherence as in those who reported missing 30% or more of the medication in the past month. The combined self-report measure had a specificity of 83% and sensitivity of 65% to 84% when compared with plasma levels taken sometime in the past 3 months. Hence, there was not a direct comparison of a standard self-report measure and serum levels. We also find it questionable to use a serum measure that is not taken at the same time as the self-report, but sometime in the past 3 months.28,29 Systematic validation of simple self-reports in a large sample using serum concentrations determined by a group standard has, to the best of our knowledge, not yet been performed.
The Norwegian health care system is catchment area-based and publicly funded and the only provider of psychiatric services. Patients are referred to the specialized psychiatric treatment system from primary care or can be directly admitted during acute phases. The maximum cost for health care including medicines that patients themselves with schizophrenia or bipolar disorder have to pay is approximately US $300 per year. In the present study, we included patients with severe mental disorders from several outpatient clinics in Oslo.
The aims of the present study were first to determine the level of adherence in a representative outpatient sample of severe mental disorders based on blood concentration of pharmacological agents and, second, to evaluate the sensitivity and specificity of a short and simple self-report method for adherence measurement and compare this to the 5-item MARS scale and provider reports.
MATERIALS AND METHODS
This study is a part of the Norwegian Thematic Organized Psychosis Research Study of patients with schizophrenia and bipolar disorders. It is an ongoing study, carried out by the University and the University Hospital in Oslo, Norway.30 Patients were recruited from the psychiatric departments as outpatients. Informed consent was obtained from all participants, and the Regional Committee for Medical Research Ethics and the Norwegian Data Inspectorate approved the study. Exclusion criteria were presence of a diagnosis of developmental disorder (IQ <70) or brain damage. From May 2003 to October 2006, we included 280 patients with bipolar I (n = 66) and II (n = 48) disorders (bipolar group, n = 114), schizophrenia (n = 126), and schizoaffective (n = 30) and schizophreniform (n = 10) disorder (schizophrenia group, n = 166) aged 18 to 65 years. In total, 385 patients were referred during the inclusion period. Eighty-eight patients were not included because they did not meet the diagnostic criteria. Fifteen patients had not started treatment with medication at the time of inclusion and were thus excluded, and 2 patients withdrew their data after inclusion. The patients were recruited by their clinician, and thus, we do not know the exact number of eligible patients who were not referred to the study based on the clinicians' decisions. However, there were 48 clearly eligible patients who were referred but refused to participate.
All patients underwent a structural clinical interview for Diagnostic and Statistical Manual of Mental Disorders, Revised Third Edition by an experienced clinician.31 All interviewers participated in regular diagnostic consensus meetings led by a clinically experienced professor of psychiatry. Interrater reliability of Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition diagnoses was adequate, with a κ of 0.77 (95% confidence interval, 0.60-0.94).
The sample consisted of 141 men and 139 women; the mean (SD) age was 35.1 (11.0) years. When divided into diagnostic groups, the schizophrenia group consisted of 95 men and 71 women; the mean (SD) age was 32.9 (9.6) years. Of these, 78.9% were of European origin, 15.7% were living with a partner, and 19.3% were employed. These patients had been in treatment for their illness for 7.1 (SD, 8.8) years. As to the subgroups within the schizophrenia group, the mean (SD) age of the schizoaffective patients was a bit higher, 36.3 (10.2) years, and 63.3% of them were women and 28.6% were living with a partner. In the bipolar group, there were 46 men and 68 women, and the mean (SD) age was 38.3 (12.1) years; 93.9% were of European origin, 32.5% were living with a partner, and 53.5% were employed. The bipolar disorder patients had been in treatment for 8.8 (SD, 10.0) years. Of the whole group, 23.5% had a lifetime diagnosis of alcohol or substance abuse or dependency, and within that group, 83.3% had used alcohol and 53.3% had used illicit drugs within the last 6 months.
The study is cross-sectional, and all measures were obtained at the time of inclusion in the study. Fasting blood samples were routinely collected between 9 and 11 am from all patients. Patients were instructed not to take their morning dose of prescribed medication before the blood sampling, according to standard protocol. Serum concentrations of medications were analyzed at the Department of Clinical Pharmacology, St Olavs Hospital, Trondheim. To simplify the analysis, a primary therapeutic agent was defined for patients taking more than 1 medicine. When more than 1 psychotropic drug from the same class was used, the primary therapeutic agent was defined as the drug with the highest dose. If the dose range was similar, the medicine that had been used for the longest duration was selected as the primary therapeutic agent. The reference range for each drug has been derived at the laboratory based on their extensive database and long experience with measuring each psychotropic drug.32 Of the 280 patients included, serum concentrations were missing for 25, and they were excluded from analysis. Reasons for the missing data were that some patients refused to give blood samples, and some samples were not analyzed because of technical problems. Thus, a total of 255 participants were included in the analysis, with n = 154 in the schizophrenia and n = 101 in the bipolar group (Table 1).
In the schizophrenia group, 146 patients (94.8%) were prescribed an antipsychotic agent as the primary therapeutic agent. Of those, 55 patients (37.7%) were on monotherapy, and 91 (62.3%) were on combined therapy with other antipsychotics, antiepileptics or lithium, or antidepressants. Forty-nine (33.6%) took 2 medications, 29 (19.9%) took 3, 9 (6.2%) took 4, 2 (1.4%) took 5 medications, and 2 (1.4%) were taking 6 different types of medications. In the bipolar group, 60 patients (59.4%) were prescribed an antiepileptic or lithium as their primary therapeutic agent; 23 (22.8%) were prescribed antipsychotics; and 9 patients (8.9%) had an antidepressant as their primary therapeutic agent. In this group, 34 patients (33.7%) were on monotherapy, 34 (33.7%) took 2 medications, 20 (19.8%) took 3 medications, and 4 patients (4.0%) took 4 medications. In addition, benzodiazepines were prescribed to 24% of the sample.
When considering serum concentrations of psychotropic medicines, the concentration-dose ratio was used because this gives the best picture of drug intake.32 An exception from this was lithium, where the serum concentration was used. By using the concentration-dose ratio, the serum concentration is divided by daily dosage and thus corresponds to the serum concentration per milligram of the medication taken daily by the individual. By using this measure, the values can be compared directly between subjects independently of different drug treatment regimens. To simplify the analysis, the patients were grouped into clusters, as suggested by Velligan et al.11 We defined 3 groups with regard to the concentration-dose ratio provided by the Trondheim laboratory: (1) not detectable, (2) low, and (3) within reference range or higher. To calculate sensitivity and specificity for self-reported adherence, a cutoff was set at 75%, as suggested by Velligan et al.11 Thus, 75% to 100% was considered satisfactory adherence, whereas 74% or less was considered unsatisfactory. The concentration-dose ratio was used as the objective measure, with nonadherence defined as not detectable or low.
Self-report of adherence was obtained from all patients. They were given a questionnaire by the research fellow and marked on a visual analog scale from 0% to 100% how much of their prescribed medication they had taken the past week. The scale had tick marks with corresponding numbers indicating 0%, 25%, 50%, 75%, and 100% level. The patients marked on the scale the percentage of intake for each medication they were taking at the time of assessment. This scale is referred to as self-report throughout the article.
To compare the current simple self-report with other measures, the MARS was administered. The version used here, MARS-5, is a 5-item self-report scale developed in the United Kingdom.33 It was translated into Norwegian, with the back-translation approved by the original author. It has been used in earlier studies as a questionnaire to measure self-reported adherence.33-37 The scale consists of general statements about medicines and is not especially developed for psychiatric patients. The 5 items are as follows: "I forget to take my medicines," "I change the dose of my medicines," "I stop taking my medicines for a while at times," "I decide not to take a dose," and "I take less than I am instructed to." There is no time frame specified in this scale. The item responses are scored on a 5-point Likert scale where 1 = always, 2 = often, 3 = sometimes, 4 = rarely, and 5 = never. Scores range from 5 to 25, with higher scores indicating higher adherence. There were 247 valid scores on the MARS-5, but 8 had missing scores, mainly because they refused to fill in the questionnaire because they had stopped taking their medication. There are other longer versions that include questions about specific types of medication, that is, for lung diseases. In addition, there is a version developed for patients with psychosis, with the same name, but that is somewhat different in that it includes more items and is a yes/no self-report.26,38
The internal validity of the MARS-5 version used here was tested, and Cronbach α was 0.78. If an item was deleted, the value of Cronbach α decreased or increased in the range of 0.70 to 0.81. The internal validity in our sample is similar to other studies using this version.34
To compare self-report with information from the providers, a provider report of adherence was obtained from the patients' case workers, who filled in a scale from 0% to 100% indicating how much of the prescribed medication they estimated their patients had taken the past week. The provider report was present for 182 subjects. The MARS-5 and the provider report were not used as measures of nonadherence in itself, but only as a comparison to the self-report.
Statistical analyses were carried out using SPSS version 14.0 for Windows (SPSS Inc, Chicago, Ill). Mean rates of all adherence measures were calculated. The χ2 was used to test differences between diagnostic groups. Correlations were calculated when comparing the different adherence measures. This method was chosen to add statistical power to the analysis. Because of skewed distribution, correlations were calculated as Spearman ρ. Analyses were conducted first for the whole group and then separately for the 2 diagnostic groups (schizophrenia vs bipolar disorder).
Serum levels and reference ranges for the laboratory are demonstrated in Table 2.
Adherence rate defined by measurements of level of medicine (serum concentration-dose ratio) within reference range or higher was 61.6% in the total sample, 58.4% for schizophrenia and 66.3% for bipolar disorder (Table 3). The difference between diagnostic groups was not significant (P = 0.24).
Adherence rate defined by the self-report score above 75% was 86.3% for the total sample, 87.7% in schizophrenia and 84.2% for bipolar disorder (Table 3). Seventy-seven percent of the patients reported that they had taken all their medications during the past week, and 10.2% reported taking less than 5%. In the group of patients who reported taking 75% or more of their medications in the past week on the self-report, 70.0% had serum concentration-dose ratio within or higher than the reference range, 28.0% were lower than the reference range, and for 2.0%, no medicine was detectable in the blood. When not adjusting for the medication dose (using only the concentration), these figures were somewhat different: 80.5%, 17.5%, and 2.0%, respectively.
The sensitivity for the self-report scale for determining adherence using concentration-dose ratio as the objective measure was 98%, and the specificity was 33%. The positive predictive value was 70%, and the negative predictive value was 91%.
The MARS-5 mean score in the total sample was 22.0 (SD, 2.9; range, 13-25). The single-item scores on the MARS-5 are shown in Figure 1, indicating a fairly similar distribution of the scores except for the "Forgets to take" item. The mean adherence last week according to the provider report was 85.2%.
Table 4 shows the correlations between different measures of adherence. There was a reasonably high correlation between self-report and concentration-dose ratio and provider report last week. The correlation of self-reported adherence with serum concentration was similar to that with concentration-dose ratio (data not shown).
The main finding of this study is an adherence rate of 61.6% in a sample of outpatients with severe mental disorders, with 58.4% in schizophrenia and 66.3% in bipolar disorder, using blood-level measurements. Second, the present results indicate that a simple self-report is a reasonably valid measure of adherence.
The present adherence results are based on medication levels in the blood from a relatively large sample, and the results are in the average to higher range of most studies of adherence in severe mental disorders.7,11 Blood measurement is a direct, objective method, but it is expensive, burdensome to both patient and health care provider, and susceptible to distortion.13 Compared with indirect methods, such as electronic monitoring, blood levels are less likely to give false high-adherence measures. Thus, the present results seem to indicate a reasonably high adherence level in the present outpatient sample. However, the blood levels represent adherence levels only in the last few days before blood draw, and it can be discussed how representative they are for more extended periods.11 In the current study, we used a laboratory with extensive experience in blood-level measurements32 and used established values for reference range based on a database of several hundred cases. We also used the dose-concentration ratio, which controls for the dose of the drug,32 but got comparable results using concentration alone.
In addition to the assessment methods, the adherence level depends on the characteristics of the patient sample. In the present study, we investigated patients who attend outpatient clinics on a regular basis. A similar outpatient study of schizophrenia from Austria reported full adherence in 52% and partial adherence in 39% according to self-report for the prior 3 months.39 Adherence level of 80% based on clinician's ratings was reported in a sample of schizophrenia patients recently referred to an ambulatory psychotic disorder clinic.40 In a Japanese study of outpatients with good adherence before the follow-up period, only 19% were nonadherent.41 In a sample of schizophrenia patients attending an outpatient clinic, 80% reported good compliance in the past 3 months with self-report.42 Another important factor for adherence is the health care service. In Norway, the government owns and runs all psychiatric hospitals and outpatient departments. The service is based on catchment areas and is practically free, and the treatment is integrated with community-based public health care. An important additional factor is the high level of therapeutic medication monitoring in clinical practice in Norway. Serum measurement of medication is a routine clinical practice in most hospitals, also outside university clinics, and for the patients with schizophrenia and bipolar disorder, this intervention can be expected when attending outpatient clinics. Thus, it can be speculated that, in a well-organized health care system, with close collaboration with community services, where the follow-up of the patients is careful and extensive including regular blood-level measurements, a favorable basis for good adherence in this particular patient group is obtained.
There is a large demand for an easy, reliable self-report for adherence.11,25-27 In the current study, we evaluated a very easy self-report by comparing it to objective medicine blood-level measures. Earlier studies have indicated that self-reports seem to overestimate adherence,22 which is supported by the current results (61.6% based on blood level and 86.3% based on self-reports). In our sample, the sensitivity for the self-report was very good (98%), and so was the negative predictive value (89%) for determining adherence. However, the specificity was poor (33%), whereas the positive predictive value was moderate (70%). Our results are different from those of Scott and Pope,28,29 who reported a specificity of 83% and sensitivity of 65% to 84% of their more extensive adherence self-reports. However, that report was based on plasma levels of mood stabilizers obtained during the previous 3 months.28,29 Our results suggest that a simple self-report questionnaire is a reasonably valid method for measuring adherence. When patients admitted not taking their medications, this was usually correct, and when patients claimed taking their medication in the past week, the majority had serum concentrations that confirmed this. Our low specificity is reflected in the fact that 30% of the patients reported regular medication intake the past week, which did not correspond to the result of the serum measurements. However, only 2% of the sample had medication levels below detection limit, and the remaining 28% had serum concentrations that were lower than expected. The self-report used in our study was simple and measures adherence only in the past week. This could make it more reliable than asking about the past month or the past 3 months.27,42 What patients reported to the research personnel in the present study could be different to what is reported to the case worker. A previous study showed that the type of self-report matters when correlating with non-self-report measures.23
In our study, the correlation between self-report and serum concentration was 0.52 (Table 4). This seems to be reasonably high and is highly significant. However, the explained variance is below 30%, and the clinical significance must be further studied in other samples. We also found reasonably good correlation between self-reports and simple provider reports, but less correlation to MARS-5 scores. This correlation is similar to a recent report from an investigation of a somewhat different and more extensive MARS questionnaire for patients with psychotic disorders.26 In the present study, the MARS-5 total scale had a weak but statistically significant correlation with provider-rated medication adherence. The self-report used in our study asks the patient directly about the use of medication in a given period, whereas the MARS-5 questions are more general in their approach to adherence, and there is no given time frame. As such, the MARS-5 seems to be more subjective and should thus be treated with some caution. In a recent study, self-reports and physicians' report of adherence were only weakly correlated with pill count and electronic monitoring and not with serum concentrations.27 This is in contrast to our findings, but the discrepancies might probably be due to the different method for defining nonadherence based on blood-level measurements.27 Most subjects in the current study seem to follow their treatment, and they attend the outpatient clinic on a regular basis. It is cost beneficial to have a valid and easy instrument like the current, simple self-report for screening and routine evaluation. But for patients who seem resistant to treatment or have a history of nonadherence, it could be wise to use direct measures of adherence like blood measurements. The present study investigated both patients with schizophrenia and those with bipolar disorder. Few studies have directly compared these groups. Both diagnostic groups seem to have fairly good adherence, and we did not find any significant differences. Similarities between schizophrenia and bipolar disorders have also been reported for other aspects, ranging from cardiovascular risk factors30 to biological susceptibility.43 Although there exist extensive data on patients receiving lithium treatment where some studies have investigated the course of illness for many years,44-46 few studies have focused on nonadherence.15 There are indications that the prevalence of nonadherence in bipolar disorder increases with longer follow-up interval.47
The present study has several limitations. The study is cross-sectional, and we do not have observations of adherence over time. The study is naturalistic but, in our opinion, fairly representative of patients in an outpatient clinic in the Norwegian health care system. We have not controlled for factors that could affect medicine blood-level measurements, such as interactions with other medication or intraindividual and interindividual differences in the medication metabolism, but this is probably of less importance for the results because of the large sample size. A general concern with adherence studies based on informed consent is the implicit selection of patients, as those patients who deny all treatment usually do not consent to participate in studies. This is an important limitation, which because of ethical reasons is impossible to reduce. More specifically for our study, we cannot rule out that the clinicians who referred patients selected candidates who were more adherent. In addition, participating in a study can be considered an intervention and as such may also improve adherence. The patients were aware of the fact that blood samples would be taken and serum level of medication measured. This might influence the patients' adherence to the better. However, the current blood-level measurement protocol is similar to routine clinical practice in many clinics in Norway.
To conclude, the present results indicate a fairly good adherence in a sample of patients with severe mental disorder treated in outpatient clinics in a public health care system where therapeutic drug monitoring is common. Furthermore, the results indicate good predictive value for a simple self-report measure of adherence.
AUTHOR DISCLOSURE INFORMATION
Dr Jónsdóttir has received travel support from Eli Lilly and Janssen Cilag in Iceland. Dr Birkenaes has received the 2005 annual research grant from Eli Lilly, Norway, and is an advisory board member of Bristol-Myers Squibb, Norway. Dr Ringen has received the 2005 annual research grant from Pfizer, Norway, and is an advisory board member of Eli Lilly, Norway. Dr Opjordsmoen has participated as a clinical investigator in drug trials initiated by AstraZeneca and Janssen-Cilag. He is also an advisory board member for Bristol-Myers Squibb, Norway. Dr Andreassen has received speaker honorarium from the Norwegian departments of Lundbeck, AstraZeneca, Bristol-Myers Squibb, Pfizer, Wyeth, Eli Lilly, and Janssen-Cilag, and research support from Eli Lilly. He has also received travel funding from Eli Lilly, AstraZeneca, and GSK Norway. The rest of the coauthors have no disclosures or competing interests to make.
1. Weiden DI, Glazer W. Assessment and treatment selection for "revolving door" inpatients with schizophrenia. Psychiatr Q
2. Fenton WS, Blyler CR, Heinssen RK. Determinants of medication compliance in schizophrenia: empirical and clinical findings. Schizophr Bull
3. Law MR, Soumerai SB, Ross-Degnan D, et al. A longitudinal study of medication nonadherence and hospitalization risk in schizophrenia. J Clin Psychiatry
4. Kampman O, Lehtinen K. Compliance in psychoses. Acta Psychiatr Scand
5. Gilmer TP, Dolder CR, Lacro JP, et al. Adherence to treatment with antipsychotic medication and health care costs among Medicaid beneficiaries with schizophrenia. Am J Psychiatry
6. Lacro JP, Dunn LB, Dolder CR, et al. Prevalence of and risk factors for medication nonadherence in patients with schizophrenia: a comprehensive review of recent literature. J Clin Psychiatry
7. Lingram R, Scott J. Treatment non-adherence in affective disorders. Acta Psychiatr Scand
8. Vieta E. Improving treatment adherence in bipolar disorder through psychoeducation. Clin Psychiatry
. 2005;66(suppl 1):24-29.
9. Colom F, Vieta E, Tacchi MJ, et al. Identifying and improving non-adherence in bipolar disorder. Bipolar Disord
. 2005;7(suppl 5):24-31.
10. Sajatovic M, Valenstein M, Blow FC, et al. Treatment adherence with antipsychotic medications in bipolar disorder. Bipolar Disord
11. Velligan DI, Lam Y-WF, Glahn DC. Defining and assessing adherence to oral antipsychotics: a review of the literature. Schizophr Bull
12. Awad GA. Antipsychotic medications: compliance and attitudes towards treatment. Curr Opin Psychiatry
13. Osterberg L, Blaschke T. Adherence to medication. N Engl J Med
14. Baumann P, Hiemke C, Ulrich S, et al. Therapeutic monitoring of psychotropic drugs an outline of the AGNP-TDM expert group consensus guideline. Ther Drug Monit
15. Goodwin FK, Jameson KR. Manic Depressive Illness
. New York: Oxford University Press; 2007.
16. Manwani SG, Szilagyi KA, Zablotsky B, et al. Adherence to pharmacotherapy in bipolar disorder patients with and without co-occurring substance use disorders. J Clin Psychiatry
17. Zeber JE, Copeland LA, Good CH, et al. Therapeutic alliance perceptions and medication adherence in patients with bipolar disorder. J Affect Disord
18. Sajatovic M, Valenstein A, Blow F, et al. Treatment adherence with lithium and anticonvulsants medications among patients with bipolar disorder. Psychiatr Serv
19. Rudd P. The measurement of compliance: medication taking. In: Krasnegor NA, Epstein LH, Johnson SB, et al, eds. Developmental Aspects of Health Compliance Behaviour
. Hillsdale, NJ: Lawrence Erlbaum; 1993:185-213.
20. Byerly MJ, Thompson A, Carmody T. Validity of electronically monitored medication adherence and conventional adherence measures in schizophrenia. Psychiatr Serv
21. Kennedy JS, von Moltke LL, Harmatz JS, et al. Validity of self-reports of caffeine use. J Clin Pharmacol
22. Stephenson BJ, Rowe BH, Haynes B, et al. The rational clinical examination. Is this patient taking the treatment as prescribed. JAMA
23. Garber MC, Nau DP, Erickson SR, et al. The concordance of self-report with other measures of medication adherence. A summary of the literature. Med Care
24. Dolder CR, Lacro JP, Warren KA, et al. Brief evaluation of medication influences and beliefs. Development and testing of a brief scale for medication adherence. J Clin Psychopharmacol
25. Byerly MJ, Nakonenzy PA, Rush JA. The Brief Adherence Rating Scale (BARS) validated against electronic monitoring in assessing the antipsychotic medication adherence of outpatients with schizophrenia and schizoaffective disorder. Schizophr Res
26. Fialko L, Garety PA, Kuipers E, et al. A large-scale validation study of the Medication Adherence Rating Scale (MARS). Schizophr Res
27. Velligan DI, Wang M, Diamond P, et al. Relationship among subjective and objective measures of adherence to oral antipsychotic medications. Psychiatr Serv
28. Scott J, Pope M. Nonadherence with mood stabilizers: prevalence and predictors. J Clin Psychiatry
29. Scott J, Pope M. Self-reported adherence to treatment with mood stabilizers, plasma levels and psychiatric hospitalization. Am J Psychiatry
30. Birkenæs AB, Søgaard AJ, Engh JA, et al. Sociodemographic characteristics and cardiovascular risk factors in patients with severe mental disorders compared with the general population. J Clin Psychiatry
31. Spitzer RL, Williams JB, Gibbon M, et al. The structural clinical interview for DSM-III-R
(SCID). I: history, rationale and description. Arch Gen Psychiatry
32. Castberg I, Skogvoll E, Spigset O. Quetiapine and drug interactions: evidence from a routine therapeutic drug monitoring service. J Clin Psychiatry
33. Horne R, Weinman J. Self-regulation and self-management in asthma: exploring the role of illness perceptions and treatment beliefs in explaining non-adherence to preventer medication. Psychol Health
34. Mårdby AC, Åkerlind I, Jörgensen T. Beliefs about medicines and self-reported adherence among pharmacy clients. Patient Educ Couns
35. Johnson G, Kong DCM, Thoman R, et al. Factors associated with medication nonadherence in patients with COPD. Chest
36. Bowskill R, Clatwothy J, Parham R, et al. Patients' perceptions of information received about medication prescribed for bipolar disorder: implications for informed choice. J Affect Disord
37. Byrne M, Walsh J, Murphy AW. Secondary prevention of coronary heart disease: patients beliefs and health-related behaviour. J Psychosom Res
38. Thompson K, Kulkarni J, Sergejew AA. Reliability and validity of a new Medication Adherence Rating Scale (MARS) for the psychoses. Schizophr Res
39. Rettenbacher MA, Hofer A, Eder U, et al. Compliance in schizophrenia: psychopathology, side effects, and patients attitudes toward the illness and medication. J Clin Psychiatry
40. Weiss KA, Smith TE, Hull JW, et al. Predictors of risk of nonadherence in outpatients with schizophrenia and other psychotic disorders. Schizophr Bull
41. Yamada K, Watanabe K, Nemoto N, et al. Prediction of medication non-compliance in outpatients with schizophrenia: a 2-year follow-up study. Psychiatr Res
42. Garavan J, Browne S, Gervin M, et al. Compliance with neuroleptic medication in outpatients with schizophrenia; Relationship to subjective response to neuroleptics; attitudes to medication and insight. Compr Psychiatry
43. Craddock N, Owen MJ. The beginning of the end for Kraepelinian dichotomy. Br J Psychiatry
44. Cavanagh J, Smyth R, Goodwin GM. Relapse into mania or depression following lithium discontinuation: a 7-year follow-up. Acta Psychiatr Scand
45. Schumann C, Lenz G, Berghöfer A, et al. Non-adherence with long-term prophylaxis: a 6-year naturalistic follow-up study of affectively ill patients. Psychiatr Res
46. Johnson RE, McFarland BH. Lithium use and discontinuation in a health maintenance organization. Am J Psychiatry
47. Keck PE, McElroy SL, Strakowski SM, et al. Compliance with maintenance treatment in bipolar disorder. Psychopharmacol Bull