Individuals who unintentionally overdose (OD) during use of opioids, especially users recently released from prison or jail, are at significant risk for death.1–5 The high rate of relapse to opioid use, combined with the loss of tolerance to opioids during incarceration, contributes substantially to the high morbidity and mortality in this population.1 In light of these risks, there exists a significant opportunity for user-/community-based administration of resuscitative naloxone as an effective antidote by intervening through the peer groups and families of those most at risk.6–15
Investigators conducting a National Institute on Drug Abuse–funded research study created a behavior change theory-based documentary-style educational DVD for the prison population at risk of overdose.16 A companion feasibility study generated a just-in-time protocol to educate, equip, and reassess at-risk, volunteering prison inmates in opioid OD prevention, OD recognition, and intranasal (IN) naloxone administration to prepare them for post-release exposures to OD situations. Using the DVD and standardized didactic materials during a prerelease "teachable moment," investigators accessed members of an especially high-risk cohort to educate and train them in the core knowledge and skills necessary to prevent, recognize, and mitigate OD situations. As part of this experimental program, researchers developed and applied an opioid OD patient simulation methodology to explore its potential role in safely studying subjects' pertinent knowledge and skills. In particular, the specific purpose of the simulation-based substudy was to examine the layperson subjects' postrelease retention of opioid OD training by having them demonstrate their ability to perform essential opioid OD resuscitation actions. This manuscript presents the development, conduct, analyses, and findings of the exploratory simulation-based element within the larger research program.
Setting and Sample
The targeted population was inmates who were within 4 weeks of release from the Rhode Island Department of Corrections in Cranston, RI. The study specifically recruited English-speaking inmates older than 18 years who were (1) involved with and/or likely to be in proximity after release to those involved in opioid abuse and (2) volunteered to participate in the opioid OD prevention and naloxone administration training program with postrelease follow-up. The exclusion criterion was not fulfilling all of the inclusion criteria. Those who expressed an interest in study participation after recruitment announcements and posters were approached privately for written informed consent by study personnel. Participation or nonparticipation in the overall research study did not affect the inmates' release date, privileges, probation, or parole status in any way. The study was approved by the Miriam Hospital Institutional Review Board with a prisoner representative member, the Medical Review Advisory Group of the Rhode Island Department of Corrections, and the Federal Office of Human Rights Protection.
Inmates who agreed to participate first completed baseline interviews detailing their sociodemographics and drug exposure and use. At a separate meeting time, participants were exposed on-site in small groups or one-on-one (as the prison schedule permitted) to an experimental intervention comprising the OD prevention educational DVD (accessible online at prisonerhealth.org) and standardized didactics on OD recognition, layperson opioid OD management (eg, basic airway management, and rescue breathing) and IN naloxone administration. Immediately after this session, subjects were evaluated on OD-related knowledge, including opioid OD recognition and the role of naloxone. Subjects unable to correctly answer all content questionnaire items were invited to attend additional sessions until able to do so. Immediate postintervention simulation practice for demonstration and assessment of subjects' skills in OD recognition and management was not feasible owing to the correctional setting. For similar reasons, educational props such as simulation naloxone rescue kits and cardiopulmonary resuscitation barrier masks were not permitted into the facility during the study intervention. After successfully completing the intervention and initial assessment (total time, approximately 30–40 minutes), each subject was provided with an IN naloxone kit around the time of release and scheduled for a 1-month follow-up assessment. Because the highest risk of postincarceration OD death is concentrated in the first 2 weeks after release, a follow-up period of 1 month was selected to permit monitoring for possible applications of preventive behaviors as well as OD recognition and management skills.
Simulation Scenario and Assessment Tool Development
After a thorough review of the literature, substance abuse, correctional health, emergency medicine, and simulation researchers along with former inmates and active drug users collaboratively developed an opioid OD simulation scenario with accompanying 21-item assessment checklist for a full-body AirwayMan manikin (Laerdal, Wappingers Falls, NY). The scenario called for a patient simulator seated on the floor, with palpable pulses and without respirations; there was no scripting for OD progression or response to naloxone. An IN naloxone kit (identical to those distributed to the subjects at the time of release) and a cardiopulmonary resuscitation barrier mask were made available to the participants during the simulation. Ambient street noise recordings that contained sounds of an approaching police car were played in the background; a cell phone mockup, drug-related props, and distractor decoys (for common OD treatment errors) were provided. The scenario was designed to elicit and record subjects' accessing of emergency services and the performance of basic airway maneuvers (eg, head tilt, mouth opening), rescue breathing with barrier protection, IN naloxone kit assembly and use, avoidance of nonindicated actions, and continued monitoring. (See document, Supplemental Digital Content 1, http://links.lww.com/SIH/A289, which contains the study session equipment list and setup process, orientation script, and simulation protocol.)
The complementary checklist was based on the American Heart Association's Life Support guidelines17,18 and derived with a modified Delphi approach (Table 1).19 Specific checklist elements, for example, “recovery position,” were purposefully included as harm reduction interventions in acknowledgement of the concern that people who use drugs or who were formerly incarcerated may not be willing to stay on-scene after calling 911. Furthermore, formal validation of the scenario and checklist relative to real-world metrics was precluded by challenges associated with accessing confidential public safety service records that contained details of illicit substance abuse.
Two pilot sessions were completed to troubleshoot the simulation study protocol, scenario, and assessment checklist.
Simulation Session Conduct
Each subject was contacted by mail and/or phone to confirm his/her scheduled 1-month follow-up appointment at a community site. On arrival, the subjects were queried as to whether they had experienced, been near, and/or managed ODs (especially with the study naloxone). After written follow-up assessments, a scripted orientation to the simulation setting was conducted with the subject. Ground rules were reviewed, for example termination of simulation in case of subject discomfort. The subject was then instructed to enter the preconfigured simulation space and start the scenario; 1 investigator (M.C.R.) video-recorded and completed real-time checklist coding of subject performance and timeliness. The scenario ended when subjects reported completing their resuscitative interactions with the simulated patient. Each participant received a $50 study incentive at session conclusion after completing a brief structured debriefing with the opportunity to self-correct (unprompted, then with open-ended prompting) any aspect of their resuscitation performance.
Results from the overall experimental research program (including paper-based baseline and follow-up assessment data on subjects' OD recognition and knowledge) are being analyzed and published separately, for example, Green et al. 2014.20 For the simulation study component, investigators used descriptive summary statistics on subjects' sociodemographic information, opioid OD experience at baseline and follow-up, and simulation performance data.
All study session videos were independently recoded offline by a second study reviewer (S.E.B.) for comparison with the initial real-time checklist coding. Training of investigators for video review entailed their rating of 3 study session recordings, followed by comparison and group discussion of their ratings (thereby allowing for cognitive interviewing of the checklist completion process by each rater). This continued as an iterative process until all raters had consistent coding across the 21 checklist items and their time stamps. Subjects' simulation checklist scores were analyzed with Fisher exact and Mann-Whitney U tests relative to their overdose experience (baseline and follow-up) and naloxone administration experience (follow-up).
Owing to the exploratory nature of the study's application of simulation to layperson OD resuscitation research, 2 investigators (T.C.G. and S.E.B.) independently completed post hoc reviews of all videos for a separate, gestalt adjudication of each subject's overall performance as either beneficial or harmful; discrepant reviews were resolved by a third reviewer. Bennett's prevalence-adjusted bias-adjusted kappa coefficient, due to detected prevalence bias,21,22 and interobserver percent agreement were calculated for the overview assessments.
All subject information, responses, and data were handled confidentially and securely stored in a Microsoft Access database with a deidentifying key code system; videos were deleted after data extraction.
One hundred seventy one inmates expressed an interest in study participation, of whom 107 (62.5%) consented to participate and completed the baseline assessment; 104 completed the study intervention training while incarcerated. Thirty-eight (35.5%) and 75 (70.1%) subjects had personally experienced or witnessed an opioid OD, respectively. None had previously been trained to respond to ODs or obtained a naloxone rescue kit in the community.
Ninety-nine subjects received an IN naloxone kit (96%) around the time of their release. Four additional subjects had not received their kit in person or by mail after release and were provided with one kit each at follow-up visit, and one subject could not be accessed for follow-up delivery of study naloxone. Subject demographics were representative of the target population: mean age, 34.8±8.9 years; male, 80%; race: white, 79%; African American, 6.5%; Native American, 6.5%; Hispanic/Latino, 18%.
One-month Postrelease Follow-up Assessment
At 1-month follow-up, 85 participants of the 103 eligible inmates who had been trained and assessed, released and residing in the community were re-assessed. (Five subjects were re-incarcerated before follow-up, 12 did not present for follow-up, and 1 had died from an opioid OD). Complete study and simulation data were available for all 85 follow-up subjects. Seven follow-up subjects (8.2%) had experienced opioid ODs after release, and 3 of the ODs were reversed using the study naloxone. In addition, subjects reported witnessing 8 nonfatal opioid ODs with 4 administrations of study naloxone, 1 performance of rescue breathing, and 1 activation of 911 services. No injuries, hospitalizations, or naloxone-related complications were reported.
Simulation Session Data
Complete data extracted from 85 video-recorded simulation sessions conducted in 2012–2013 were analyzed. Independent reviewers exhibited less than 10% disagreement for checklist coding across sessions, which evolved to full agreement after joint review of the videos. Subjects' median scores on task checklists during simulation scenarios were 12.0 (interquartile range, 11.0–15.0) of 21 total indicated actions. Checklist scores did not exhibit significant differences on bivariate analyses by age, sex, baseline OD and naloxone experience, and follow-up OD and naloxone experience (data not shown).
911 was activated by 72.9% of subjects via instruction to bystander (n = 7) or prop phone (n = 56). At least one basic airway maneuver and delivery of 2 rescue breaths with chest rise were completed by 56 (65.9%) and 78 (91.8%) subjects, respectively. Full naloxone doses were correctly administered into both nares by 44 (51.8%) of subjects at a median of 94.5 (70.8–125.0) seconds from scenario start; IN naloxone kit assembly and medication administration were completed in 58.0 (40.0–66.0) seconds from kit retrieval. Sixty subjects (70.6%) administered at least some naloxone intranasally: 8 participants administered naloxone into 1 nostril only, and 9 participants administered a partial dose. Several subjects attempted to administer naloxone intravenously (n = 4), intramuscularly (n = 1), orally (n = 1), or intranasally without the atomizer (n = 5) to the manikin. Six participants stated they did not know how to put the naloxone kit together during the simulation. Nonindicated actions that were not discussed during the experimental intervention were observed during 42 simulations (49.4%), primarily the performance of chest compressions. See Table 1 for details.
Post hoc review of videos determined that resuscitative actions by 8 (94.1%) 0 of 85 subjects were of potential overall benefit to the simulated opioid OD patient. The prevalence-adjusted bias-adjusted kappa coefficient was 0.81 with interobserver agreement of 90.6% for the 2 initial reviewers. Discrepancies for 8 subjects primarily revolved around whether and when during the scenario chest compressions were administered—these were resolved through review by a third investigator.
With widespread use of opioid pain relievers and a growing heroin crisis nationally, ODs are increasingly a major cause of morbidity and mortality.23 Opioid ODs are on the rise in both incidence and case fatality despite the presence of an effective, accessible, and commonly recognized antidote in the form of naloxone. Institution of programs for specific populations to promote layperson administration of naloxone may help curb OD deaths associated with this epidemic. These efforts may be facilitated by patient simulation, which provides a safe and validated24–27 mechanism for learners to demonstrate and be evaluated on their acquisition and application of knowledge, skills, and poise for high-stakes situations. Accordingly, the study team used simulation to complement existing research tools for investigation into the meaningful performance of layperson opioid OD resuscitative actions in a high-risk cohort.
Even with significant environmental constraints on the program intervention, more than half of the study participants were observed to correctly deliver resuscitative doses of IN naloxone with timeliness comparable to paramedic students.28 Concurrent with its primary evaluative purpose within the research program, study simulations were noted to enable subjects to act out what the experimental intervention had presented and to practice in effect for high-acuity events that are highly probable in their immediate future.29 Study conduct also highlighted the potential access that applied simulation could afford when trying to reach and help at-/high-risk populations. Conversely, subjects' difficulties with IN naloxone delivery device assembly and the frequent observation of impromptu and unrehearsed chest compressions revealed the complexity surrounding intervention efforts for this cohort. The limited and brief nature of opportunities available to train these individuals is expected to forestall traditional, resource-delimited methods such as structured Basic Life Support courses. Additionally, trepidation associated with accessing public safety services, that is, calling 9-1-1, in the context of illicit drug-related activities was reported frequently by subjects and represents a prominent impediment to optimal resuscitative care—expansion of Good Samaritan overdose immunity laws may ameliorate this problem.30 Next steps for community IN naloxone research efforts will need to address these unique operational challenges while ensuring correct device assembly (or alternative delivery mechanisms31,32), appropriate naloxone dose, proper administration technique, and the avoidance of nonindicated actions.
Study conduct demonstrated that simulation can be applied to outreach efforts directed toward inmate target populations housed in intrinsically limiting environments. With corroboration of the high frequency of opioid ODs in and around the study cohort over an extremely short follow-up period, the potential value of medical simulation techniques to assess (and potentially empower) those at risk seems significant. Study findings are being used nationally in ongoing programs to promote community naloxone administration for opioid ODs—continued application of the study simulation protocol is being considered for further investigation and standardized assessment of OD recognition and response in prison and community-based cohorts.
Unforeseen factors related to subject recruitment and attrition may have affected the study sample selection. As simulation could not be made available to subjects inside the correctional setting, subjects were not assessed for their acquisition and retention of opioid OD training immediately after the intervention. Records from public safety and emergency medical services were not available for subjects' self-reported experiences with overdoses and administrations of naloxone. Presimulation assessments at follow-up may have affected subject performance as recorded by scenario checklists, although the significant variability in checklist scores does not indicate a uniform effect. Study design and target population characteristics precluded follow-up beyond the research program's one-time scheduled session.
Investigators applied simulation to the postrelease assessment of opioid OD prevention training retention and skills acquisition in at-risk prison inmates. Findings from this exploratory study suggest the potential utility of simulation as a complementary evaluative methodology for investigation of layperson OD management skills.
The authors acknowledge Anna C. Cousins for her insight and assistance in manuscript preparation, the staff at the Rhode Island Department of Corrections for their help with this study, and all of the participants who volunteered for this research study.
1. Binswanger IA, Stern MF, Deyo RA, et al. Release from prison—a high risk of death for former inmates. N Engl J Med
2. Rosen DL, Schoenbach VJ, Wohl DA. All-cause and cause-specific mortality among men released from state prison, 1980-2005. Am J Public Health
3. Farrell M, Marsden J. Acute risk of drug-related death among newly released prisoners
in England and Wales. Addiction
4. Dietze P, Jolley D, Fry C, et al. Transient changes in behaviour lead to heroin overdose: results from a case-crossover study of non-fatal overdose. Addiction
5. Warner M, Chen LH, Makuc DM. Increase in fatal poisonings involving opioid analgesics in the United States. NCHS Data Brief
6. Kim D, Irwin KS, Khoshnood K. Expanded access to naloxone: options for critical response to the epidemic of opioid overdose mortality. Am J Public Health
7. Doe-Simkins M, Walley AY, Epstein A, et al. Saved by the nose: bystander-administered intranasal naloxone hydrochloride for opioid overdose. Am J Public Health
8. Green TC, Heimer R, Grau LE. Distinguishing signs of opioid overdose and indication for naloxone: an evaluation of six overdose training and naloxone distribution programs in the United States. Addiction
9. Piper TM, Rudenstine S, Stancliff S, et al. Overdose prevention
for injection drug users: lessons learned from naloxone training and distribution programs in New York City. Harm Reduct J
10. Seal KH, Thawley R, Gee L, et al. Naloxone distribution and cardiopulmonary resuscitation
training for injection drug users to prevent heroin overdose death: a pilot intervention study. J Urban Health
11. Sherman SG, Gann DS, Tobin KE, et al. "The life they save may be mine": diffusion of overdose prevention
information from a city sponsored programme. Int J Drug Policy
12. Wagner KD, Valente TW, Casanova M, et al. Evaluation of an overdose prevention
and response training programme for injection drug users in the Skid Row area of Los Angeles, CA. Int J Drug Policy
13. Piper TM, Stancliff S, Rudenstine S, et al. Evaluation of a naloxone distribution and administration program in New York City. Subst Use Misuse
14. Strang J, Powis B, Best D, et al. Preventing opiate overdose fatalities with take-home naloxone: pre-launch study of possible impact and acceptability. Addiction
15. Bird SM, McAuley A, Perry S, et al. Effectiveness of Scotland's national naloxone programme for reducing opioid-related deaths: a before (2006–10) versus after (2011–13) comparison. Addiction
. 2015, epub ahead of print publication. Available at: onlinelibrary.wiley.com/doi/10.1111/add.13265/citedby
. Accessed April 15, 2016.
16. Green TC, Bowman SE, Ray M, et al. Development of an incarceration
-specific overdose prevention
video: staying alive on the outside. Health Educ J
17. Berg RA, Hemphill R, Abella BS, et al. Part 5: Adult Basic Life Support: 2010 American Heart Association guidelines for cardiopulmonary resuscitation
and emergency cardiovascular care. Circulation
18. Vanden Hoek TL, Morrison LJ, Shuster M, et al. Part 12: Cardiac arrest in special situations: 2010 American Heart Association guidelines for cardiopulmonary resuscitation
and emergency cardiovascular care. Circulation
20. Green TC, Ray M, Bowman SE, et al. Two cases of intranasal naloxone self-administration in opioid overdose. Subst Abuse
21. Bennett EM, Albert R, Goldstein AC. Communications through limited response questioning. Public Opinion Q
22. Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol
23. Jones CM, Logan J, Gladden RM, et al. Vital signs: demographic and substance use trends among heroin users—United States, 2002–2013. MMWR Morb Mortal Wkly Rep
24. Stefanidis D, Korndorffer JR Jr, Markley S, et al. Closing the gap in operative performance between novices and experts: does harder mean better for laparoscopic simulator training? J Am Coll Surg
25. Barsuk JH, Cohen ER, Feinglass J, et al. Use of simulation-based education to reduce catheter-related bloodstream infections. Arch Intern Med
26. Barsuk JH, McGaghie WC, Cohen ER, et al. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med
27. Wayne DB, Didwania A, Feinglass J, et al. Simulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: a case-control study. Chest
28. McDermott C, Collins NC. Prehospital medication administration: a randomised study comparing intranasal and intravenous routes. Emerg Med Int
2012; Available at: ncbi.nlm.nih.gov/pmc/articles/PMC3431081
. Accessed April 15, 2016.
29. Frank JW, Andrews CM, Green TC, et al. Emergency department utilization among recently released prisoners
: a retrospective cohort study. BMC Emerg Med
31. Edwards ET, Edwards ES, Davis E, et al. Comparative usability study of a novel auto-injector and an intranasal system for naloxone delivery. Pain Ther
32. Krieter P, Chiang N, Gyaw S, et al. Pharmacokinetic properties and human use characteristics of an FDA approved intranasal naloxone product for the treatment of opioid overdose. J Clin Pharmacol
2016;5. Epub ahead of print. doi: 10.1002/jcph.759.