OBJECTIVE: To report on the development of a questionnaire to screen for hazardous substance use in pregnant women and to compare the performance of the questionnaire with other drug and alcohol measures.
METHODS: Pregnant women were administered a modified TWEAK (Tolerance, Worried, Eye-openers, Amnesia, K[C] Cut Down) questionnaire, the 4Ps Plus questionnaire, items from the Addiction Severity Index, and two questions about domestic violence (N=2,684). The sample was divided into “training” (n=1,610) and “validation” (n=1,074) subsamples. We applied recursive partitioning class analysis to the responses from individuals in the training subsample that resulted in a three-item Substance Use Risk Profile-Pregnancy scale. We examined sensitivity, specificity, and the fit of logistic regression models in the validation subsample to compare the performance of the Substance Use Risk Profile-Pregnancy scale with the modified TWEAK and various scoring algorithms of the 4Ps.
RESULTS: The Substance Use Risk Profile-Pregnancy scale is comprised of three informative questions that can be scored for high- or low-risk populations. The Substance Use Risk Profile-Pregnancy scale algorithm for low-risk populations was mostly highly predictive of substance use in the validation subsample (Akaike's Information Criterion=579.75, Nagelkerke R2=0.27) with high sensitivity (91%) and adequate specificity (67%). The high-risk algorithm had lower sensitivity (57%) but higher specificity (88%).
CONCLUSION: The Substance Use Risk Profile-Pregnancy scale is simple and flexible with good sensitivity and specificity. The Substance Use Risk Profile-Pregnancy scale can potentially detect a range of substances that may be abused. Clinicians need to further assess women with a positive screen to identify those who require treatment for alcohol or illicit substance use in pregnancy.
LEVEL OF EVIDENCE: III
The Substance Use Risk Profile-Pregnancy scale is a new, flexible, and effective questionnaire to screen for use of hazardous substances in pregnancy.
From the Departments of Psychiatry and Obstetrics, Gynecology and Reproductive Sciences and the School of Epidemiology and Public Health, Yale University School of Medicine, New Haven, Connecticut.
Supported by Grant R01 DA 019135 from the National Institute on Drug Abuse to Drs. Yonkers, Kershaw, and Rounsaville.
Corresponding author: Kimberly A. Yonkers, MD, 142 Temple Street, Suite 301, New Haven, CT 06510; e-mail: Kimberly.Yonkers@yale.edu.
Financial Disclosure Dr. Yonkers has received royalties from Up To Date for a chapter on premenstrual syndrome, and she has been a principal investigator for an investigator-initiated grant on postpartum depression from Eli Lilly. The other authors did not report any potential conflicts of interest.
Studies of pregnant women find that approximately 5% use illicit drugs, whereas 15% drink alcohol during pregnancy.1,2 Serious and life-threatening complications such as maternal cardiac dysrhythmias, placental abruption, and uterine rupture can occur with cocaine and other stimulant abuse, whereas withdrawal from alcohol or sedatives can lead to maternal hypertension and seizures.3,4 The potential outcomes for offspring are also a concern and range from intrauterine growth restriction and preterm birth to spontaneous abortion and fetal death.5 Accordingly, the American College of Obstetricians and Gynecologists recommends screening pregnant women for alcohol and illicit substances.6 Unfortunately, there are few screening instruments that have been evaluated for use in pregnant women and most are designed to screen for alcohol misuse. The T-ACE (Tolerance, Annoyance, Cut Down, Eye Opener)7 and TWEAK (Tolerance, Worried, Eye-openers, Amnesia, K[C] Cut Down)8 are two screening questionnaires that have performed acceptably in pregnant women9 but are limited to risky alcohol use. The 4Ps10 and 4Ps plus11,12 screen for a range of substances and can detect pregnant women with lower levels of alcohol or drug use. However, they have low to moderate specificity; thus, many women will screen positive but not have problems with drugs or alcohol. The goals of this study were to 1) use a large screening database to derive a clinically useful screening assessment tool for hazardous substance use; and 2) compare the performance of this screener with the 4Ps and 4Ps Plus and the TWEAK modified for drugs and alcohol.
Between 2005 and 2009, pregnant women who presented for obstetric care at three hospital-based reproductive health clinics were screened for misuse of hazardous substances. The screening questionnaire was designed to determine possible eligibility for a study of on-site treatment services for hazardous substance use in pregnancy, referral to community care, or both of these. Trained research assistants obtained verbal and written informed consent before screening. It required approximately 10 minutes for participants to complete the self-report form. The Yale School of Medicine and Bridgeport Hospital human subjects boards approved the study procedures.
The screening questionnaire was administered as a single self-report survey and included two questions on domestic violence as well as questions from the Addiction Severity Index13 the 4Ps Plus,10–12 and the TWEAK,8 which was modified to ask about drugs as well as alcohol. A summary of questions asked can be found in Box 1. The modified TWEAK was scored as recommended for the TWEAK.8 The 4Ps was scored in two different ways according to algorithms in the 200511 and 200712 articles by Chasnoff et al. As in the 2005 article, women were deemed 1) low risk if they never used alcohol; 2) average risk if they had used alcohol in the past but consumed no alcohol and smoked fewer than three cigarettes in the month before pregnancy; and 3) high risk if they consumed any alcohol or smoked at least three cigarettes in the month before pregnancy. In the 2007 article, women who endorsed any alcohol or cigarette use in the month before pregnancy were classified as positive.
We randomly divided our cohort (N=2,684) into a “training” sample (n=1,610) to derive our measure and a “validation” sample (n=1,074) to test the new screening measure. The gold standard used to designate drug or alcohol misuse was patient self-report of use of a substance in the last 30 days. Although self-report has obvious limitations, we found high agreement between self-report and urine drug tests in previous work (Yonkers K, Howell H, Gotman N, Rounsaville B. Self-report of illicit substance use versus urine toxicology results from at-risk pregnant women. J Subst Abuse. In press).
To construct the new screening measure, we applied recursive partitioning class analysis to the 22 screening items in the training sample (n=1,610). Recursive partitioning class analysis is a nonparametric method based on stepwise splitting of a sample into subgroups that maximally differ on the desired outcome.14 The sample is initially split into two to three subgroups (called nodes) by finding the predictor and the cut point on that predictor (for continuous variables) that maximize differences between groups. This process is repeated for each resulting node until the split produces subgroups that do not differ significantly on the outcome or until group sizes are too small to further split. We used a minimum node size of 50, or 3% of the sample, which is consistent with other studies.15 The analysis resulted in a set of terminal nodes and a “tree” that was used to construct the Substance Use Risk Profile-Pregnancy scale screeners for low- and high-risk populations.
We conducted a series of logistic regression analyses on the validation sample (n=1,064) to compare the ability of the following instruments to predict past-30-day self-reported use of drugs or alcohol: 1) the 4Ps Plus and its published scoring algorithms; 2) the modified TWEAK; and 3) Substance Use Risk Profile-Pregnancy scale results for high- and low-risk populations. Each model used self-report of drug or alcohol use as the dependent variable. We assessed model fit using the Akaike Information Criterion16 and the amount of unique variability accounted for using Nagelkerke R2.17 Lower values of Akaike's Information Criterion indicate better model fit, whereas higher values of Nagelkerke R2 indicate a higher amount of explained variability. We also calculated simple sensitivity and specificity, with associated 95% confidence intervals, for each screening measure. Finally, we assessed ability of the Substance Use Risk Profile-Pregnancy scale to predict use of individual substances. Sensitivity and specificity are presented as well as P values for Fisher's exact test. SPSS AnswerTree 3.1 software was used for recursive partitioning; SAS 9.1.3 was used for other analyses.
The demographic characteristics and substance use history of the 2,684 pregnant women who were screened are shown in Table 1. In the past 30 days, 6% used alcohol, 6% used marijuana, 1% used cocaine, 1% used sedatives, and less than 1% used heroin. Twelve percent of women sampled used at least one of these substances in the past month. There were no statistically significant differences between the training and validation subsamples.
The results of the recursive partitioning class analysis are shown in Figure 1. Within each node, the proportion of past month alcohol or drug use is shown. The analysis resulted in five terminal nodes (nodes 3, 4, 6, 7, and 8) with three questions required to identify node membership. These questions and the pattern of conditional responses constitute the Substance Use Risk Profile-Pregnancy scale.
We categorized nodes as low, medium, or high risk based on the proportion of women within each node who used alcohol or drugs in the past month. Proportions of 0–5% were deemed low risk, 6–20% as medium risk, and 21% or greater as high risk. A simple formula for risk categorization is presented in Box 2. Negative responses for all items yields a low-risk individual, one affirmative response yields a moderate risk individual, and two or three affirmative responses yield a high-risk individual. For low-risk populations, we recommend classifying moderate- and high-risk women as positive. For high-risk populations, we recommend classifying only high-risk women as positive.
Table 2 presents estimates of the ability of the TWEAK, 4Ps, and the Substance Use Risk Profile-Pregnancy scale screener to predict past month alcohol or drug use in the validation subsample (n=1,074). The Substance Use Risk Profile-Pregnancy scale for low-risk populations provided the best model fit (Akaike's Information Criterion=579.75) and accounted for the most variance (R2=0.27). The sensitivity was excellent at 91%, whereas the specificity was adequate at 67%. The Substance Use Risk Profile-Pregnancy scale for high-risk populations had lower sensitivity (57%) but higher specificity (88%).
The sensitivity, specificity, and general association of the Substance Use Risk Profile-Pregnancy scale for drug abuse categories (alcohol, marijuana, cocaine, sedatives, opioids) can be seen in Table 3. The association between Substance Use Risk Profile-Pregnancy scale classification and substance endorsement was statistically significant (P≤.02) in all cases except for endorsement of opioids using the algorithm for low-risk populations, in which it was marginally significant (P=.06). The Substance Use Risk Profile-Pregnancy scale performs nearly as well for detecting other substance use as it does for alcohol or marijuana, although questions about use of other drugs are not among the three questions in the screener.
Our results showed that the Substance Use Risk Profile-Pregnancy scale is a brief, flexible, and effective screening tool for alcohol and illicit substance use in pregnancy. It identified individuals at high risk of cocaine, sedative, and opioid use, although questions about consumption of these substances are not included in the instrument. Of the screeners administered in this study, the Substance Use Risk Profile-Pregnancy scale method was most predictive of alcohol and illicit drug use. For low-risk populations, it had good sensitivity, whereas the Substance Use Risk Profile-Pregnancy scale for high-risk populations had good specificity. This flexibility in the algorithm is an advantage of the Substance Use Risk Profile-Pregnancy scale, but this also illustrates the tension between sensitivity and specificity that is inherent among many screening and case finding scales.18 The algorithm for low-risk populations is useful if sensitivity is of paramount concern, whereas specificity may be more important if the potential consequences of false-positive results are risky or lead to expensive testing. False-positive designation of substance use status is an important consideration because substance use in pregnancy may have legal ramifications.19 For example, 15 states regard substance use during pregnancy to be child abuse and information about substance use can be applied to terminate parental rights. In 14 states, healthcare professionals are mandated to report suspected prenatal drug exposure. To this end, screening results should always be followed with more careful and complete assessment.
Consistent with other work, our findings showed that Chasnoff's 4Ps11 and 4Ps plus12 had moderate to excellent sensitivity (81–95%) for detection of alcohol or illicit substance use in pregnancy but modest specificity (69–45%). The modified TWEAK had lower sensitivity (63%) and higher specificity (86%). However, estimates for sensitivity and specificity may vary according to the populations assessed as shown by Chasnoff et al,10 who found substantial differences cross-nationally. Additionally, the patterns of drug use and abuse change over time and can influence drug use.
There are a number of limitations to our report. One is that our “gold standard” was self-report of substance use rather than a biologic measure such as a urine or hair toxicology test. It is worth noting that our estimate for prior month hazardous substance use is in line with results from the National Survey of Drug Use and Health.20 Moreover, a recent study that evaluated pregnant women's perspectives on screening for alcohol and drugs found that although women have concerns over the consequences of their drug use disclosure, mainly involvement of child protective services, they were not averse to reporting alcohol use in pregnancy.21 This is consistent with other data from our group in which we found that the agreement between urine drug screens and self-report for marijuana and cocaine was 0.74 and 0.79, respectively, in the preceding 28 days (Yonkers K et al, in press). Although urine tests are often considered the gold standard, they have limitations in the detection window and are not useful for alcohol. Similarly, hair testing, although improved lately, is limited by type of hair, use of hair products, processing of the hair, and the difficulty of collecting hair in large samples of women.22,23
Although the Substance Use Risk Profile-Pregnancy scale was derived in one subsample and evaluated in another, the similarity of these groups was a limitation to this study. The instrument may be influenced by factors that are unique to that group. For example, the top two substances of choice in our population were alcohol and marijuana. Accordingly, a screening instrument that asks about marijuana and alcohol use would be expected to perform well. However, we also showed that these questions detected women at high risk for use of a variety of other drugs and that a question about marijuana identifies general risk of alcohol and drug use. Marijuana use continues to be relatively common among young women,1,2 including those who are pregnant,24 and thus our population may not differ substantially from others with regard to this factor. Still, our population may differ from others in a number of other respects. Our cohort was derived from an inner-city hospital clinic and was predominantly African American and Hispanic. Accordingly, the Substance Use Risk Profile-Pregnancy scale will need to be tested on other groups of pregnant women to determine generalizability. Finally, we only considered substance use data from the prior 30 days, and therefore our results do not extrapolate to substance use throughout pregnancy.
Despite the limitations of our study, it has several strengths. To develop the Substance Use Risk Profile-Pregnancy scale, we used several questionnaires that allowed selection of the most diverse, informative questions while eliminating the redundancy within each instrument. The items in the screening were taken from established instruments used in addictions research or were clinically relevant to the issue of substance misuse (eg, domestic violence). The cohort under study was large (N=2,684) and diverse. The Substance Use Risk Profile-Pregnancy scale contains only three items, and scoring is very simple. This is particularly useful because busy clinicians do not have time to administer separate questionnaires that would screen for a variety of substances individually. Given the brevity of the screener, it can be readministered on multiple occasions with minimal burden to the patient or the clinician. Given the high risk of relapse during the postpartum period for women with a history of hazardous substance use,2,25 readministration of the Substance Use Risk Profile-Pregnancy scale in the several months after delivery may be useful. However, future research will need to clarify the incremental benefit of rescreening. As well, it would be of great scientific benefit to determine whether the use of this or other screening instruments leads to increased substance use treatment and more favorable maternal and child health.
1.SAMHSA. Results from the 2001 National Household Survey on Drug Abuse: volume I: summary of national findings. Report No. SMA 02-3758, NHSDA Series H-17. U.S. Department of Health and Human Services Publication. Rockville (MD): Office of Applied Studies; 2002.
2.Ebrahim SH, Gfroerer J. Pregnancy-related substance use in the United States during 1996–1998. Obstet Gynecol 2003;101:374–9.
3.Kuczkowski KM. The effects of drug abuse on pregnancy. Curr Opin Obstet Gynecol 2007;19:578–85.
4.Palmer JM, Indermaur MD, Tebes CC, Spellacy WN. Placenta increta and cocaine abuse in a grand multipara leading to a second trimester rupture of an unscarred uterus: a case report. South Med J 2008;101:834–5.
5.Kaltenbach K, Berghella V, Finnegan L. Opioid dependence during pregnancy. Effects and management. Obstet Gynecol Clin North Am 1998;25:139–51.
6.At-risk drinking and illicit drug use: ethical issues in obstetric and gynecologic practice. ACOG Committee Opinion No. 294. American College of Obstetricians and Gynecologists. Obstet Gynecol 2004;103:1021–31.
7.Sokol RJ, Martier SS, Ager JW. The T-ACE questions: practical prenatal detection of risk-drinking. Am J Obstet Gynecol 1989;160:863–8.
8.Russell M, Martier SS, Sokol RJ, Mudar P, Jacobson S, Jacobson J. Detecting risk drinking during pregnancy: a comparison of four screening questionnaires. Am J Public Health 1996;86:1435–9.
9.Chang G, Wilkins-Haug L, Berman S, Goetz MA, Behr H, Hiley A. Alcohol use and pregnancy: improving identification. Obstet Gynecol 1998;91:892–8.
10.Chasnoff IJ, Neuman K, Thornton C, Callaghan MA. Screening for substance use in pregnancy: a practical approach for the primary care physician. Am J Obstet Gynecol 2001;184:752–8.
11.Chasnoff IJ, McGourty RF, Bailey GW, Hutchins E, Lightfoot SO, Pawson LL, et al. The 4P's Plus screen for substance use in pregnancy: clinical application and outcomes. J Perinatol 2005;25:368–74.
12.Chasnoff IJ, Wells AM, McGourty RF, Bailey LK. Validation of the 4P's Plus screen for substance use in pregnancy. J Perinatol 2007;27:744–8.
13.McLellan AT, Kushner H, Metzger D. The Fifth Edition of the Addiction Severity Index. J Subst Abuse Treat 1992;9:199–213.
14.Zhang HP, Singer B. Recursive partitioning in the health sciences. New York (NY): Springer-Verlag; 1999.
15.Lemon SC, Roy J, Clark MA, Friedmann PD, Rakowski W. Classification and regression tree analysis in public health: methodological review and comparison with logistic regression. Ann Behav Med 2003;26:172–81.
16.Burnham K, Anderson D. Model selection and multimodal inference: a practical information-theoretic approach. 2nd ed. New York (NY): Springer-Verlag; 2002.
17.Nagelkerke NJD. A note on general definition of the coefficient of determination. Biometrika 1991;78:691–2.
18.Grimes DA, Schulz KF. Uses and abuses of screening tests. Lancet 2002;359:881–4.
20.SAMHSA. Results from the 2007 National Survey on Drug Use and Health: national findings. Rockville (MD): Office of Applied Studies; 2007. p. 1–290.
21.Roberts SC, Nuru-Jeter A. Women's perspectives on screening for alcohol and drug use in prenatal care. Womens Health Issues 2010;20:193–200.
22.Kulaga V, Pragst F, Fulga N, Koren G. Hair analysis of fatty acid ethyl esters in the detection of excessive drinking in the context of fetal alcohol spectrum disorders. Ther Drug Monit 2009;31:261–6.
23.Pragst F, Balikova MA. State of the art in hair analysis for detection of drug and alcohol abuse. Clin Chim Acta 2006;370:17–49.
24.Gilchrist LD, Hussey JM, Gillmore MR, Lohr MJ, Morrison DM. Drug use among adolescent mothers: prepregnancy to 18 months postpartum. J Adolesc Health 1996;19:337–44.
25.Ebrahim SH, Luman ET, Floyd RL, Murphy CC, Bennett EM, Boyle CA. Alcohol consumption by pregnant women in the United States during 1988–1995. Obstet Gynecol 1998;92:187–92.
Figure. No caption available.