The purpose of this study was to compare compliance estimates based on 3 methods of measuring medication taking: self-report, an electronic medication event measurement system (MEMS), and a biochemical tracer (riboflavin). During the first 4 weeks of a placebo-controlled, double-blind trial, cocaine-dependent participants (N = 55) took their assigned study medications and provided data to assess daily pill-taking behavior. MEMS-based estimates of medication compliance were substantially lower than riboflavin or self-report (28%, 78%, and 87%, respectively). Using MEMS as a reference or “gold standard,” self-report, riboflavin, and their combination demonstrated poor ability to detect non-compliance (AUCs < 0.60). Medication compliance rates vary depending on measurement method, with subjective reports more likely to overestimate actual pill-taking behavior.