The proliferation of prescription opioids has become a critical public health issue in the United States, Canada, and, increasingly, Europe, Australia, and New Zealand.10 In the United States, prescriptions for opioid analgesics have quadrupled over the past 2 decades. Prescription opioid overdoses claimed over 14,000 lives in the United States in 2014.6 Previous studies have found an association between opioid prescription and rates of overdose mortality.3,11,13,16,19,20,26,28,30,32 The increase in opioid-related mortality has prompted scrutiny of prescription drug misuse, diversion, and “doctor shopping.” Prescription drug monitoring programs (PDMPs) have been implemented in 49 States, most of which allow clinicians to track patient prescription histories.21
High-risk opioid use involves patient behaviors, clinician prescribing, and the clinician–patient interaction. Patterns of opioid prescribing to opioid-naive patients are associated with probability of subsequent long-term use.9 Over time, tolerance, and hyperalgesia may lead to patient drug-seeking behavior. In States with PDMPs, clinicians are privy to a patient's prescription history at the time of an encounter. Routine use of the PDMP may allow clinicians to avoid writing for overlapping prescriptions or dangerous interactions.
Prescribers interpret data in the PDMP with little guidance. Some PDMP's use proactive alerts21 to help clinicians identify patients who exceed thresholds for number of prescribers or pharmacies, high dose, or prescription overlap. No definitive set of evidence-based criteria for triggering proactive alerts exists. Accuracy of alerts in predicting overdose, diversion, or abuse is only partly understood.
There may also be some risk of creating “alert fatigue,” as observed with drug interaction alerts in electronic medical record systems.27 The utility of a proactive alert is blunted if the clinician finds it burdensome or irrelevant. A model that generates alerts that are sensitive, but not specific, may be of little value.
Many PDMP systems are focused on identifying drug diversion or abuse for law-enforcement purposes. Models that identify patients at high risk of overdose may differ in both variables used and parameter values. Alerts directed to clinicians should address health outcomes. We developed a model for identifying patients at highest risk for overdose using prescription patterns and patient characteristics in PDMP data. We simplified the model so it could potentially be used as a proactive alert in PDMP systems where complex calculations would limit feasibility and slow system response time, presenting a deterrent to PDMP utilization.
Specific aims of the present research were as follows:
- To evaluate the strength of prescription-related risk factors associated with opioid overdose.
- To develop a predictive model for opioid overdose that could be calculated using PDMP records alone.
- To reduce the model to the fewest and simplest possible data elements that could inform a proactive alert.
- To assess model sensitivity and specificity.
- To assess model generalizability.
The guidelines established in the “Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD)” Statement represent the most rigorous standard for reporting findings from predictive modeling.8 We present our findings in a manner consistent with this standard.
Data were drawn from the Oregon PDMP, Oregon Vital Records, and the Oregon Hospital Discharge Database. Consistent with the State law, an analyst at the Public Health Division prepared a deidentified data set for use by our research team. Potential identifiers were stripped or categorized such that patients, prescribers, and pharmacies were not individually identifiable. The data captured all controlled substance prescription fills, hospitalizations, and deaths between October 1, 2011, and October 31, 2014.
The commercial vendor that hosts Oregon's PDMP matches multiple prescriptions to an individual using a proprietary and largely deterministic algorithm based on name, date of birth, and address. Consequently, the same individual may have numerous records in the system because of use of nicknames, changes of legal name or residence, errors in spelling, or reversed characters or digits. Before deidentification, a Public Health analyst therefore used Link King v7.1 software to match individuals within and between data sets. The software uses name, date of birth, and zip code to classify possible matching records into “linkage certainty levels.”1,5 A random sample of linked record pairs was manually inspected within each certainty level for validation. Where linkage certainty levels fell below 95% positive predictive value, possible record pairs were reviewed individually. This process resulted in an improved unique patient identifier in the PDMP data, which was then used to link patients between PDMP, Vital Records, and Hospital Discharge data sets.
Drugs were classified by pharmaceutical class using Food and Drug Administration National Drug Codes.29 All opioid medications were included in this analysis regardless of therapeutic use, except tramadol, which was not tracked in the PDMP until August of 2014, and combinations of buprenorphine with naloxone or naltrexone. Nonhuman records, such as inventory transfers between pharmacies and drugs prescribed to pets, were also excluded as were prescriptions from out-of-state prescribers.
“Morphine milligram equivalents” (MMEs) were calculated for each prescription using the Centers for Disease Control and Prevention Conversion Reference Tables.22 Where conversion factors were unavailable, a clinical pharmacist used the drug name and other properties to identify equivalent drugs with associated conversion factors. Where data about duration of action (immediate or extended release) were missing, the clinical pharmacist assigned duration of action based on equivalent drugs.23
The study population included Oregon residents older than 12 years who received a prescription for an opioid medication from an Oregon physician during the calendar year 2013. The validation analysis used comparable data from 2012 alone. We restricted analysis to individuals who had only Oregon addresses to capture the most complete patient records.
2.2. Defining adverse outcomes
Our goal was to identify adverse outcomes, defined as opioid overdose mortality or hospitalization. Prescription opioid overdose deaths were identified by ICD-10 codes for underlying cause of death and contributing cause of death indicating poisoning by prescription opioids (T40.2 and T40.3) in the Oregon vital records.
Hospitalizations in the Oregon Hospital Discharge Database were included if discharge diagnoses indicated a definite or probable prescription opioid overdose. Definite overdose-related hospitalizations were identified using ICD-9 codes indicating poisoning by opioid substances (965.0, 965.00, 965.02, 965.09, E850.1, and E850.2). Probable overdoses were identified by codes indicating adverse effects of opioids (E935.1 and E935.2) coupled with codes on the same date suggestive of overdose or poisoning (276.4, 292.11, 292.12, 292.81, 292.89, 486.0, 496.0, 518.81, 518.82, 780.01, 780.02, 780.03, 780.09, 780.97, 786.03, 786.05, 786.09, 799.01, 799.02, 799.10, E950.0, E950.1, E950.2, E950.3, E950.4, E950.5, and E980.0).9 These definitions were based on a synthesis of approaches presented in the literature.11,17,20,26,30
2.3. Predictive measures
Predictive measures (independent variables) in the models were derived from PDMP data, including patient characteristics and risk factors related to prescribing patterns. The PDMP data set contained patient age category and urban-rural designation but no sex or other demographic information. The information collected by the Oregon PDMP is defined in statute and was intentionally limited. At the time of the study, even sex was not recorded in the database. Other variables such as education, race, and income are not recorded.
Prescription-related independent variables were drawn from the emerging body of literature on prescribing practices with high risk of opioid overdose. All variables were calculated over the full 12-month study period. See Table 1 for a list of independent variables and corresponding definitions. The number of prescribers, pharmacies, and prescriptions for opioids per patient was continuous counts. The “4 by 4” metric (≥4 prescribers and ≥4 pharmacies)19 is also discussed here because it is a commonly cited risk metric.
“Overlap” variables indicate that a patient is receiving multiple prescriptions for controlled substances that may be concurrent (opioid–opioid, opioid–benzodiazepine, opioid–benzodiazepine–carisoprodol).4,15,18,24,25,30 Such patients may be at greater risk because of pharmacokinetic or pharmacodynamic interactions, or high cumulative opioid dose.30 An alternative approach using binary indicators for any prescription of the individual drugs during the study year was also considered.
Total MME was calculated for each prescription. Each patient's prescriptions were aggregated, and averaged over each month to generate an average daily dose. The maximum average daily dose for each patient (the highest monthly dose) during the study period was retained. The binary “High Dose” variable was set equal to 1 where the patients' maximum average daily dose was greater than the Centers for Disease Control and Prevention–recommended cutoff of 90 MME.
We first examined the association between patient demographic variables and prescribing patterns with the outcome of overdose events using unadjusted odds ratios (ORs). Metrics were then evaluated for usefulness as predictors. Two logistic regression models are presented here. One evaluated all candidate risk factors, and a second reduced the number of variables to a minimal model. The objective of the logistic regression models is to predict a binary outcome (opioid overdose) as a linear combination of risk factors. The multivariable model separates out the unique variance explained by each variable after controlling for the influence of all the others. The dependent variable in a logistic model is a continuous estimate of the log odds of an overdose occurring, and may be thought of as representing how often we would expect a patient with the given characteristics and history to have experienced an overdose.
We analyzed goodness of fit using an adjusted Hosmer–Lemeshow test. The Hosmer–Lemeshow test divides the population into subgroups based on predicted value of the dependent variable. Within each group, fit is assessed by how well the predicted frequency accords with the observed frequency. The resulting statistic is distributed approximately chi square and permits hypothesis testing for model fit. This test is sensitive to large sample sizes and to the number of subgroups used.
In ordinary least square regression models, R 2 provides a measure of the percentage of variance in the dependent variable explained by the model. In logistic regression, the R 2 cannot be calculated. Several pseudo-R 2 measures have been proposed, including the Cox and Snell and Nagelkerke methods, each with strengths and weaknesses. Cox and Snell performs best when the probability of the dependent variable is near 0.5, and the upper bound approaches 1. In the case of rare events, the upper bound of Cox and Snell R 2 can be quite low, making its interpretation challenging. Nagelkerke R 2 is a rescaled version Cox and Snell calculated by dividing by the upper bound of the test. This normalization of the statistic gives it an interpretation similar to that of the R 2 statistic in ordinary least square regression in the case of rare outcomes.7
Receiver operating characteristic (ROC) curve analysis was also performed: a plot of sensitivity vs 1-specificity of the models, indicating their ability to discriminate between patients who did or did not have an overdose event. The objective of logistic regression is to predict a binary outcome whether the patient will or will not experience an overdose. The dependent variable in a logistic model may be thought of as representing how often we would expect a patient with the given characteristics and history to have experienced an overdose. To generate the binary outcome, we decide on a threshold that delineates positive from negative predictions. As a model becomes more sensitive, by classifying more cases as positive, it also increases the number false positives, reducing specificity.
The receiver operating characteristic curve visualizes this tradeoff between sensitivity and specificity. The perfect predictive model would identify each event and nonevent correctly. For such a model, the ROC plot would be a right angle in the upper left corner. A model that performs no better than chance would have an ROC curve that was a diagonal line from the lower left to the upper right corner of the plot. Most models have an ROC curve that falls somewhere in between. The area under the ROC curve (AUC) quantifies the success of discrimination, ranging from 0.5 (no better than chance) to 1.0 (perfect discrimination).14
In assessing the generalizability of a model, tests of model fit and statistical significance may be inadequate. Overfitting of the model to the data may be an issue.2 To address this concern, a separate data set was compiled from the PDMP for 2012. This data set was used to validate the final model. Observations in the validation data set were classified using the final model, and then performance of the model on the 2 data sets was compared using the AUC.
All analyses were performed using SAS 9.4 for Windows.
3.1. Patients, prescriptions, and events
The analytic data set for 2013 consisted of data on 6,334,197 prescriptions (all controlled drugs) written to 879,402 unique patients. Of the patients represented in the data set, 61% were residents of urban areas. This was consistent between patients who did and did not experience an adverse event. Patient age was grouped into 7 categories 12 to 24 (12%), 25 to 34 (15%), 35 to 44 (15%), 45 to 54 (17%), 55 to 64 (18%), 65 to 74 (13%), and 75+ (9%). Age distribution varied considerably with between patients who did and did not experience an adverse event. Demographic information that may be recorded by the Oregon PDMP is defined in statute and was deliberately limited. During the study period, even patient sex was not recorded. This is discussed further in the Limitations section.
The most commonly prescribed medications by class were opioids (69.71%) followed by benzodiazepines (16.5%), nonbenzodiazepine sedatives (5.99%), and stimulants (3%). The most commonly prescribed opioids were hydrocodone–acetaminophen combinations (33.84%) and oxycodone combinations (20.1%).
Opioid-related mortality from the Oregon Vital Records file included 99 prescription opioid-related deaths (other than methadone) and 53 methadone-related deaths. Allowing for a small number of cases in which both methadone and nonmethadone opioids were implicated, there were 139 prescription opioid-related deaths during the study period. Opioid-related hospitalizations included 893 definite overdose events and 766 probable overdose events. A total of 1409 unique patients experienced an overdose during the study period. Heroin-related events that did not involve a prescription opioid were excluded.
3.2. Aim 1: Unadjusted associations between patient or prescription characteristics and overdose events
Odds ratios (ORs) were calculated for demographic variables and all prescribing variables. Full results of this analysis are reported in Table 1 (supplemental digital content, available online at http://links.lww.com/PAIN/A491). The odds of an opioid-related adverse event increased with each advancing age category, with all age groups showing statistically significant elevation of risk relative to the reference group (12-24 year olds). Rural residency was not significantly associated with likelihood of an overdose event (OR = 1.03, 95% confidence interval [CI]: 0.93-1.15).
Counts of prescribers, pharmacies, and prescriptions all showed an increasing trend in OR of experiencing an opioid-related adverse event. There was no clear empirical threshold. The OR for the “4 by 4” metric was large and statistically significant (OR = 18.59, 95% CI: 15.24-22.68) but slightly less than the sum of the individual ORs for 4 prescribers (OR = 9.82, 95% CI: 8.56-11.27) and 4 pharmacies (OR = 9.96, 95% CI: 8.45-11.72). A model containing both variables and an interaction term indicated that the combination metric was largely a linear combination of the 2 constituent variables with a small negative interaction.
Variables related to overlapping prescriptions for opioids with other opioids, benzodiazepines or nonbenzodiazepine sedatives, or carisoprodol and between long-acting and short-acting opioids, were all significantly associated with the likelihood of an overdose event. We also tested metrics indicating any prescription for a benzodiazepine, carisoprodol, or long-acting opioid prescription at any time during the study year as possible alternatives to more complex overlap variables.
3.3. Aim 2: Logistic regression model 1
A logistic regression model was estimated simultaneously incorporating demographics, prescribers, pharmacies, and prescriptions, overlap variables and high dose. Number of prescribers, pharmacies, and prescriptions were mean centered to address concerns of collinearity. Step-up and step-down approaches were also applied. All 3 approaches suggested the same model (Table 1). Age was associated with increasing risk, but only categories such as age older than 35 years were statistically significant. Magnitudes of effects in the logistic model were smaller than their unadjusted counterparts.
3.4. Aim 3: Logistic regression model 2
A second, more parsimonious model was created based on the results of model 1 (Table 1). Age categories were reduced to only 35 to 44 (adjusted OR [aOR] = 1.47, 95% CI: 1.14-1.88), 45 to 54 (aOR = 1.95, 95% CI: 1.57-2.44), 55 to 64 (aOR = 2.82, 95% CI: 2.29-3.48), 65 to 74 (aOR = 3.68, 95% CI: 2.97-4.57), >75 (aOR = 4.99, 95% CI: 4.02-6.192) with those younger than 35 years used as the reference. The number of prescribers (aOR = 1.15 for each additional prescriber, 95% CI: 1.12-1.18) and the number of pharmacies (aOR = 1.11 for each additional pharmacy, 95% CI: 1.06-1.16) were retained in the model. Variables that required calculating a prescription overlap period were removed and replaced with similar but easier to calculate binary indicators for any prescription for long-acting opioids (aOR = 4.41, 95% CI: 3.93-4.94), benzodiazepines or nonbenzodiazepine sedatives (aOR = 2.50, 95% CI: 2.23-2.79), or carisoprodol (aOR = 1.63, 95% CI: 1.25-2.13) in the study year. The number of prescriptions and high dose were dropped because of small effect sizes after adjustment for other variables.
3.5. Aim 4: Model fit, sensitivity and specificity
For model 2, the Cox and Snell R 2 (0.0026) and Nagelkerke R 2 (0.1071) suggest different degrees of explanatory power. In this situation, with rare outcomes, the Nagelkerke R2 may be the better estimate. A Hosmer–Lemeshow (H–L) test for model fit was calculated for each model. In model 1, the model fit statistic indicated that the model fit the data well, χ2 (8) = 14.58, P = 0.07. For model 2, the model fit statistic indicated that this model does not fit the data as well, χ2 (7) = 38.89, P < 0.0001. Although the simpler model may not fit the data as well as the full model, its explanatory power was not adversely impacted. Explanatory power and goodness of fit of a model are often though not always associated.
The ROC curve in Figure 1A and AUC (0.82) indicate that model 2 performs substantially better than a random model, χ2 (1) = 3182.34, P < 0.0001. Table 2 indicates that there is an almost symmetrical tradeoff between sensitivity and specificity moving from a cutoff value of 0.001 to 0.002. Using a cutoff of 0.002, the sensitivity and specificity of the model were 62% and 82%, respectively. These may be evaluated relative to the total number of patients who did experience an adverse event (n = 1409) and the total number of patients who did not (n = 837,280).
Prescribing data provided information about the risk of adverse outcomes and identified a subgroup among whom adverse events were far more prevalent; however, even among patients identified as high risk by our model, overdose remained a rare event. The positive predictive value of a model is the proportion of positive tests that prove to be correct. The positive predictive value of model 2 using a cutoff of 0.002 was 0.006, meaning that among 1000 patients identified as “high risk”, only 6 were actually hospitalized or died of an opioid-related overdose (Table 2).
3.6. Aim 5: Generalizability
Generalizability of our model was assessed by applying the model estimated using the 2013 training data set to validate data from 2012. The AUC for the validation data set (AUC = 0.82) was virtually identical to that for the training data set, suggesting that the model generalizes well to new data.
Our preliminary analysis (specific aim 1) indicated that all the prescribing risk factors identified from the literature had significant bivariate associations with opioid overdose. When combined into multivariable models, in pursuit of the specific aim 2, many of these relationships were attenuated. Given the low prior probability of major overdose events for any given individual, we found that even among patients identified as being at higher relative risk of opioid overdose, the absolute risk remained small. The predictive value of any proactive alert for overdose risk based solely on prescription data seemed to be low. The results of our research suggest caution in using a PDMP-only model for prescribing decisions support. Nonetheless, the factors that predict overdose-related death or hospitalization should recommend caution in prescribing because they are likely to be important predictors of misuse, abuse, diversion, or less severe overdose as well.
Although there is some overlap between predictors of overdose and other outcomes of interest, there are some important differences. In our data, increasing age and use of long-acting opioids were important predictors of opioid-related overdose. These factors are not emphasized in models built for identifying misuse or diversion for law-enforcement purposes, but have important clinical implications. Older adults may have lower risks of substance abuse disorder than younger adults,12 but age-related changes in drug metabolism and sensitivity to drug effects may render them more likely to experience inadvertent overdoses. More parsimonious use and lower dosing of opioids in this age group may be important. In our models, the single most powerful predictor of overdose, after adjusting for all others, was a prescription for a long-acting opioid. This has obvious implications for clinical practice, particularly in the treatment of chronic pain, where these medications are often used.
In addition to advancing age and long-acting opioids, key independent predictors of opioid overdose in our study included the number of different prescribers, number of different pharmacies, a prescription for a benzodiazepine or other sedative, and any prescription for carisoprodol. Although many models have used threshold metrics for number of prescribers11 and number of pharmacies (most commonly ≥4),11,13,25 we found that continuous predictors resulted in better model fit. Threshold models may be more useful in identifying diversion, as thresholds may clearly demarcate unusual behavior, whereas risk of overdose appears to increase continuously.
In developing the parsimonious model 2 (specific aim 3), we found that replacing prescription overlap variables with indicators of any prescription for long-acting opioids, carisoprodol, benzodiazepine, or other sedatives in the same year did not decrease the predictive power of the model. Several factors may explain this finding. Prescription overlap definitions may be imprecise because of ambiguity or unavailability of the intended duration of a prescription. Overlap variables and binary indicators are likely to be highly collinear among those at high risk. Patients may save pills for later use, resulting in delayed effect of the prescription. Furthermore, coprescriptions may simply be markers of individuals whose medication-taking behavior is more likely to lead to opioid overdose, even if the coprescription is not involved.
Specific aim 4 is a response to a gap in the literature with respect to the sensitivity, specificity, and positive predictive value of models predicting overdose risk. Models may be highly sensitive at the expense of a large number of false positives, which may induce alert fatigue. Even among patients flagged as high risk using this model, the absolute incidence of major overdose outcomes was low. Nevertheless, these patients may be at increased risk of addiction, misuse, minor overdose, or engaged in diversion of medications. Clinicians must integrate PDMP-derived flags with other patient information, including clinical history, physical examination, and laboratory findings. A model constructed in the Veterans Affairs health care system indicated that combining prescription data of the type we examined with patient demographic, clinical, and health care utilization data from medical records resulted in substantially greater predictive accuracy.31
Our fifth specific aim was to assess the generalizability of our model. In many applications, generalizability is not a matter of model fit to training data, but of how well it classifies a test data set that was not used in the process of model building. Such tests of classification provide evidence that an observed relationship is stable over time, or across different populations. The fact that the model generalized well to another year of data suggests that these few variables are good candidate metrics for predicting patient risk.
There are important limitations to our analysis. The PDMP did not collect most patient demographic features and did not have a unique identifier, such as social security number, during the study years. This necessitated probabilistic matching of patients within the PDMP and between the PDMP and hospital and death registries. Information on days' supply for prescriptions was also missing, which had 2 effects. First, overlap variables were imprecise. Second, calculation of average daily dose had to be performed over the full study period, making the model less sensitive to high dose prescriptions of short duration.
Our exclusive use of PDMP data to define predictor variables restricts the types of variables we could include in the model. Consequently, the model does not incorporate variables that have been identified in previous studies such as health status.
This analysis is restricted to overdose events that resulted in death or hospital admission. Overdoses that were fully reversed by first responders or in the community with naloxone are not included. Overdose events that were treated and released from the emergency department or in outpatient clinics also do not appear in our data set, nor do overdose events that were never reported or never interacted with medical professionals.
Restricting PDMP data to Oregon providers and Oregon pharmacies leaves the possibility of out-of-state doctor shopping. In cases in which a patient was engaged in doctor shopping over state lines, the model would underestimate patient risk.
Outcomes were measured in the same year as exposures. Consequently, patients who died early in the year are less likely to display high log odds, and patients who did not experience an adverse event may have higher log odds. A patient who died in January of the study period might not appear to display many high-risk characteristics simply because there is only a single month of data for them, whereas a patient who survives throughout the entire year may switch doctors or pharmacies and come to appear much more high risk. This limitation should result in a bias toward a finding of no effect.
Together, our findings may inform strategies for policy-makers designing practical proactive alerts using PDMP data, but also suggest that incorporation of other data may be necessary to achieve high predictive values. Twenty-six states use PDMP data to identify high-risk patients and provide proactive alerts to clinicians.22 In programming proactive alerts, use of simple indicators decreases the complexity of coding, processing time, and the need for data that may not be present in a PDMP system. Our models suggest that a small and computationally simple set of variables may be useful predictors of patient overdose risk, although the predictive value of a positive test is low for rare events.
Our data suggest that a predictive model based on prescription history alone can discriminate between patients at high and low risk of overdose event. However, like many models of rare events, our model had a weak positive predictive value. Most individuals identified by our model as being at high risk did not have an adverse event. Improvement of the predictive value of such models may require information from sources outside most PDMPs, such as a history of hospitalization for opioid-related complications or other clinical data from electronic health records. Nonetheless, our study helps to identify a parsimonious and computationally simple set of PDMP variables for such alerts, and may complement models built primarily for law-enforcement purposes. Testing the optimal design and impact of proactive alerts remains an important research agenda.
Conflict of interest statement
The authors have no conflicts of interest to declare.
The authors acknowledge Dagan Wright and Lisa Millet from the Injury and Violence Prevention program of the Oregon Health Authority for their assistance. Miguel Marino and Eve Dexter of Oregon Health & Science University provided technical help. The project was funded through a supplement to National Institute on Drug Abuse Grant # 3R01DA031208-03S1.
Supplemental digital content
Supplemental digital content associated with this article can be found online at http://links.lww.com/PAIN/A491.
. Beil H, Preisser J, Rozier R. Accuracy of record linkage software in merging dental administrative data sets. J Public Health Dent 2013;73:89–93.
. Bleeker SE, Moll HA, Steyerberg EW, Donders ART, Derksen-Lubsen G, Grobbee DE, Moons KGM. External validation is necessary in prediction research: a clinical example. J Clin Epidemiol 2003;56:826–32.
. Bohnert AB, Valenstein M, Bair MJ, Ganoczy D, McCarthy JF, Ilgen MA, Blow FC. ASsociation between opioid prescribing patterns and opioid overdose
-related deaths. JAMA 2011;305:1315–21.
. Bramness JG, Furu K, Engeland A, Skurtveit S. Carisoprodol use and abuse in Norway. A pharmacoepidemiological study. Br J Clin Pharmacol 2007;64:210–18.
. Campbell K, Deck D, Krupski A. Record linkage software in the public domain: a comparison of link plus, the link king, and a “basic” deterministic algorithm. Health Inform J 2008;14:5–15.
. Centers for Disease Control and Prevention (CDC). Prescription opioid overdose
data. 2016. Available at: http://http://www.cdc.gov
/drugoverdose/data/overdose.html. Accessed August 10, 2016.
. Cohen J, Cohen P, West SG, Aiken LS. Applied multiple regression/correlation analysis for the behavioral sciences. Mahwah, NJ: Lawrence Erlbaum Associates Inc., 2003.
. Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 2015;162:55.
. Deyo RA, Hallvik SE, Hildebran C, Marino M, Dexter E, Irvine JM, O'Kane N, Van Otterloo J, Wright DA, Leichtling G, Millet LM. Association between initial opioid prescribing patterns and subsequent long-term use among opioid-naive patients: a Statewide Retrospective Cohort Study. J Gen Intern Med 2017;32:21–7.
. Deyo RA, Von Korff M, Duhrkoop D. Opioids for low back pain. BMJ 2015;350:g6380.
. Dunn KM, Saunders KW, Rutter CM, Banta-Green CJ, Merrill JO, Sullivan MD, Weisner CM, Silverberg MJ, Campbell CI, Psaty BM, Von Korff M. Opioid prescriptions for chronic pain and overdose: a cohort study. Ann Intern Med 2010;152:85–92.
. Edlund MJ, Martin BC, Fan M-Y, Devries A, Braden JB, Sullivan MD. Risks for opioid abuse and dependence among recipients of chronic opioid therapy: results from the TROUP study. Drug Alcohol Depend 2010;112:90–8.
. Ekholm O, Kurita GP, Højsted J, Juel K, Sjøgren P. Chronic pain, opioid prescriptions, and mortality in Denmark: a population-based cohort study. PAIN 2014;155:2486–90.
. Fawcett T. An introduction to ROC analysis. Pattern Recognit Lett 2006;27:861–74.
. Forrester M. Ingestions of hydrocodone, carisoprodol, and alprazolam in combination reported to Texas poison centers. J Addict Dis 2011;30:110–15.
. Gomes T, Mamdani MM, Dhalla IA, Paterson J, Juurlink DN. OPioid dose and drug-related mortality in patients with nonmalignant pain. Arch Intern Med 2011;171:686–91.
. Gwira Baumblatt JA, Wiedeman C, Dunn JR, Schaffner W, Paulozzi LJ, Jones TF. High-risk use by patients prescribed opioids for pain and its role in overdose deaths. JAMA Intern Med 2014;174:796–801.
. Jann M, Kennedy WK, Lopez G. Benzodiazepines: a major component in unintentional prescription drug overdoses with opioid analgesics. J Pharm Pract 2014;27:5–16.
. Katz N, Panas L, Kim M, Audet AD, Bilansky A, Eadie J, Kreiner P, Paillard FC, Thomas C, Carrow G. Usefulness of prescription monitoring programs for surveillance-analysis of schedule II opioid prescription
data in Massachusetts, 1996-2006. Pharmacoepidemiol Drug Saf 2010;19:115–23.
. Liang Y, Turner BJ. Assessing risk for drug overdose in a national cohort: role for both daily and total opioid dose? J Pain 2015;16:318–25.
. Manasco AT, Griggs C, Leeds R, Langlois BK, Breaud AH, Mitchell PM, Weiner SG. Characteristics of state prescription drug monitoring programs: a state-by-state survey. Pharmacoepidemiol Drug Saf 2016;25:847–51.
. National Center for Injury Prevention and Control. CDC conversion reference table A compilation of opioid analgesic formulations with morphine milligram equivalent conversion factors. 2014 version. Atlanta, GA: Centers for Disease Control and Prevention, Available at: http://http://www.pdmpassist.org
/pdf/BJA_performance_measure_aid_MME_conversion.pdf. Accessed September 15, 2014.
. O'Kane N, Hallvik SE, Marino M, Van Otterloo J, Hildebran C, Leichtling G, Deyo RA. Preparing a prescription drug monitoring program data set for research purposes. Pharmacoepidemiol Drug Saf 2016. Available at: http://onlinelibrary.wiley.com/doi/10.1002/pds.4039/full. Accessed August 10, 2016.
. Owens C, Pugmire B, Salness T, Culbertson V, Force R, Cady P, Steiner J. Abuse potential of carisoprodol: a retrospective review of Idaho medicaid pharmacy and medical claims data. Clin Ther 2007;29:2222–5.
. Paulozzi LJ. Prescription drug overdoses: a review. J Saf Res 2012;43:283–9.
. Paulozzi LJ, Kilbourne EM, Shah NG, Nolte KB, Desai HA, Landen MG, Harvey W, Loring LD. A history of being prescribed controlled substances and risk of drug overdose death. Pain Med 2012;13:87–95.
. Phansalkar S, van der Sijs H, Tucker AD, Desai AA, Bell DS, Teich JM, Middleton B, Bates DW. Drug—drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J Am Med Inform Assoc 2013;20:489–93.
. Ray WA, Chung CP, Murray KT, Hall K, Stein CM. Prescription of long-acting opioids and mortality in patients with chronic noncancer pain. JAMA 2016;315:2415.
. US Food and Drug Administration. National drug code directory. Available at: http://http://www.fda.gov
/Drugs/InformationOnDrugs/ucm142438.htm. Accessed June 14, 2014.
. Yang Z, Wilsey B, Bohm M, Soulsby M, Roy K, Ritley D, Jones C, Melnikow J. Defining risk for prescription opioid overdose
: pharmacy shopping and overlapping prescriptions among long-term opioid users in medicaid. J Pain 2015;16:445–53.
. Zedler B, Xie L, Wang L, Joyce A, Vick C, Brigham J, Kariburyo F, Baser O, Murrelle L. Development of a risk index for serious prescription opioid-induced respiratory depression or overdose in Veterans' health administration patients. Pain Med 2015;16:1566–79.
. Zedler B, Xie L, Wang L, Joyce A, Vick C, Kariburyo F, Rajan P, Baser O, Murrelle L. Risk factors for serious prescription opioid-related toxicity or overdose among veterans health administration patients. Pain Med 2014;15:1911–29.
Chronic pain management; Opioid prescription; Opioid overdose; Addiction medicine; Predictive modeling
Supplemental Digital Content
© 2018 International Association for the Study of Pain