Share this article on:

Patient Simulation for Assessment of Layperson Management of Opioid Overdose With Intranasal Naloxone in a Recently Released Prisoner Cohort

Kobayashi, Leo MD; Green, Traci C. PhD, MSc; Bowman, Sarah E. MPH; Ray, Madeline C. BA; McKenzie, Michelle S. MPH; Rich, Josiah D. MD

doi: 10.1097/SIH.0000000000000182
Empirical Investigations

Introduction Investigators applied simulation to an experimental program that educated, trained, and assessed at-risk, volunteering prisoners on opioid overdose (OD) prevention, recognition, and layperson management with intranasal (IN) naloxone.

Methods Consenting inmates were assessed for OD-related experience and knowledge then exposed on-site to standardized didactics and educational DVD (without simulation). Subjects were provided with IN naloxone kits at time of release and scheduled for postrelease assessment. At follow-up, the subjects were evaluated for their performance of layperson opioid OD resuscitative skills during video-recorded simulations. Two investigators independently scored each subject's resuscitative actions with a 21-item checklist; post hoc video reviews were separately completed to adjudicate subjects' interactions for overall benefit or harm.

Results One hundred three prisoners completed the baseline assessment and study intervention and then were prescribed IN naloxone kits. One-month follow-up and simulation data were available for 85 subjects (82.5% of trained recruits) who had been released and resided in the community. Subjects' simulation checklist median score was 12.0 (interquartile range, 11.0–15.0) of 21 total indicated actions. Forty-four participants (51.8%) correctly administered naloxone; 16 additional subjects (18.8%) suboptimally administered naloxone. Nonindicated actions, primarily chest compressions, were observed in 49.4% of simulations. Simulated resuscitative actions by 80 subjects (94.1%) were determined post hoc to be beneficial overall for patients overdosing on opioids.

Conclusions As part of an opioid OD prevention research program for at-risk inmates, investigators applied simulation to 1-month follow-up assessments of knowledge retention and skills acquisition in postrelease participants. Simulation supplemented traditional research tools for investigation of layperson OD management.

Supplemental digital content is available in the text.

From the Department of Emergency Medicine (L.K., T.C.G.), Alpert Medical School of Brown University, Providence, RI; Lifespan Medical Simulation Center (L.K.), Providence, RI; Department of Epidemiology (T.C.G., J.D.R.), School of Public Health, Brown University, Providence; Center for Prisoner Health and Human Rights (T.C.G., S.E.B., M.S.M., J.D.R.), The Miriam Hospital, Providence, RI; Department of Emergency Medicine (S.E.B.), Alpert Medical School of Brown University, Providence, RI; Department of Medicine (M.C.R., M.S.M., J.R.D.), Alpert Medical School of Brown University; Providence, RI.

Reprints: Leo Kobayashi, MD, Lifespan Medical Simulation Center, Suite 106, Coro West Building, 1 Hoppin Street, Providence, RI 02903 (e-mail:

The authors declare no conflict of interest.

Aspects of the simulation sessions and the study intervention discussed within the article were presented as a poster abstract at the 2014 International Meeting on Simulation in Healthcare in San Francisco, CA.

This work was supported by NIH grants R21 DA029201, K24 DA022112 and P30 AI042853. The material discussed is based on work supported by the Departments of Emergency Medicine and Medicine at the Alpert Medical School of Brown University, the Lifespan Medical Simulation Center, and the University Emergency Medicine Foundation. The conclusions, opinions, and recommendations expressed in the manuscript are those of the authors and do not necessarily reflect the views of the supporting entities.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (

Individuals who unintentionally overdose (OD) during use of opioids, especially users recently released from prison or jail, are at significant risk for death.1–5 The high rate of relapse to opioid use, combined with the loss of tolerance to opioids during incarceration, contributes substantially to the high morbidity and mortality in this population.1 In light of these risks, there exists a significant opportunity for user-/community-based administration of resuscitative naloxone as an effective antidote by intervening through the peer groups and families of those most at risk.6–15

Investigators conducting a National Institute on Drug Abuse–funded research study created a behavior change theory-based documentary-style educational DVD for the prison population at risk of overdose.16 A companion feasibility study generated a just-in-time protocol to educate, equip, and reassess at-risk, volunteering prison inmates in opioid OD prevention, OD recognition, and intranasal (IN) naloxone administration to prepare them for post-release exposures to OD situations. Using the DVD and standardized didactic materials during a prerelease "teachable moment," investigators accessed members of an especially high-risk cohort to educate and train them in the core knowledge and skills necessary to prevent, recognize, and mitigate OD situations. As part of this experimental program, researchers developed and applied an opioid OD patient simulation methodology to explore its potential role in safely studying subjects' pertinent knowledge and skills. In particular, the specific purpose of the simulation-based substudy was to examine the layperson subjects' postrelease retention of opioid OD training by having them demonstrate their ability to perform essential opioid OD resuscitation actions. This manuscript presents the development, conduct, analyses, and findings of the exploratory simulation-based element within the larger research program.

Back to Top | Article Outline


Setting and Sample

The targeted population was inmates who were within 4 weeks of release from the Rhode Island Department of Corrections in Cranston, RI. The study specifically recruited English-speaking inmates older than 18 years who were (1) involved with and/or likely to be in proximity after release to those involved in opioid abuse and (2) volunteered to participate in the opioid OD prevention and naloxone administration training program with postrelease follow-up. The exclusion criterion was not fulfilling all of the inclusion criteria. Those who expressed an interest in study participation after recruitment announcements and posters were approached privately for written informed consent by study personnel. Participation or nonparticipation in the overall research study did not affect the inmates' release date, privileges, probation, or parole status in any way. The study was approved by the Miriam Hospital Institutional Review Board with a prisoner representative member, the Medical Review Advisory Group of the Rhode Island Department of Corrections, and the Federal Office of Human Rights Protection.

Inmates who agreed to participate first completed baseline interviews detailing their sociodemographics and drug exposure and use. At a separate meeting time, participants were exposed on-site in small groups or one-on-one (as the prison schedule permitted) to an experimental intervention comprising the OD prevention educational DVD (accessible online at and standardized didactics on OD recognition, layperson opioid OD management (eg, basic airway management, and rescue breathing) and IN naloxone administration. Immediately after this session, subjects were evaluated on OD-related knowledge, including opioid OD recognition and the role of naloxone. Subjects unable to correctly answer all content questionnaire items were invited to attend additional sessions until able to do so. Immediate postintervention simulation practice for demonstration and assessment of subjects' skills in OD recognition and management was not feasible owing to the correctional setting. For similar reasons, educational props such as simulation naloxone rescue kits and cardiopulmonary resuscitation barrier masks were not permitted into the facility during the study intervention. After successfully completing the intervention and initial assessment (total time, approximately 30–40 minutes), each subject was provided with an IN naloxone kit around the time of release and scheduled for a 1-month follow-up assessment. Because the highest risk of postincarceration OD death is concentrated in the first 2 weeks after release, a follow-up period of 1 month was selected to permit monitoring for possible applications of preventive behaviors as well as OD recognition and management skills.

Back to Top | Article Outline

Simulation Scenario and Assessment Tool Development

After a thorough review of the literature, substance abuse, correctional health, emergency medicine, and simulation researchers along with former inmates and active drug users collaboratively developed an opioid OD simulation scenario with accompanying 21-item assessment checklist for a full-body AirwayMan manikin (Laerdal, Wappingers Falls, NY). The scenario called for a patient simulator seated on the floor, with palpable pulses and without respirations; there was no scripting for OD progression or response to naloxone. An IN naloxone kit (identical to those distributed to the subjects at the time of release) and a cardiopulmonary resuscitation barrier mask were made available to the participants during the simulation. Ambient street noise recordings that contained sounds of an approaching police car were played in the background; a cell phone mockup, drug-related props, and distractor decoys (for common OD treatment errors) were provided. The scenario was designed to elicit and record subjects' accessing of emergency services and the performance of basic airway maneuvers (eg, head tilt, mouth opening), rescue breathing with barrier protection, IN naloxone kit assembly and use, avoidance of nonindicated actions, and continued monitoring. (See document, Supplemental Digital Content 1,, which contains the study session equipment list and setup process, orientation script, and simulation protocol.)

The complementary checklist was based on the American Heart Association's Life Support guidelines17,18 and derived with a modified Delphi approach (Table 1).19 Specific checklist elements, for example, “recovery position,” were purposefully included as harm reduction interventions in acknowledgement of the concern that people who use drugs or who were formerly incarcerated may not be willing to stay on-scene after calling 911. Furthermore, formal validation of the scenario and checklist relative to real-world metrics was precluded by challenges associated with accessing confidential public safety service records that contained details of illicit substance abuse.



Two pilot sessions were completed to troubleshoot the simulation study protocol, scenario, and assessment checklist.

Back to Top | Article Outline

Simulation Session Conduct

Each subject was contacted by mail and/or phone to confirm his/her scheduled 1-month follow-up appointment at a community site. On arrival, the subjects were queried as to whether they had experienced, been near, and/or managed ODs (especially with the study naloxone). After written follow-up assessments, a scripted orientation to the simulation setting was conducted with the subject. Ground rules were reviewed, for example termination of simulation in case of subject discomfort. The subject was then instructed to enter the preconfigured simulation space and start the scenario; 1 investigator (M.C.R.) video-recorded and completed real-time checklist coding of subject performance and timeliness. The scenario ended when subjects reported completing their resuscitative interactions with the simulated patient. Each participant received a $50 study incentive at session conclusion after completing a brief structured debriefing with the opportunity to self-correct (unprompted, then with open-ended prompting) any aspect of their resuscitation performance.

Back to Top | Article Outline

Data Analysis

Results from the overall experimental research program (including paper-based baseline and follow-up assessment data on subjects' OD recognition and knowledge) are being analyzed and published separately, for example, Green et al. 2014.20 For the simulation study component, investigators used descriptive summary statistics on subjects' sociodemographic information, opioid OD experience at baseline and follow-up, and simulation performance data.

All study session videos were independently recoded offline by a second study reviewer (S.E.B.) for comparison with the initial real-time checklist coding. Training of investigators for video review entailed their rating of 3 study session recordings, followed by comparison and group discussion of their ratings (thereby allowing for cognitive interviewing of the checklist completion process by each rater). This continued as an iterative process until all raters had consistent coding across the 21 checklist items and their time stamps. Subjects' simulation checklist scores were analyzed with Fisher exact and Mann-Whitney U tests relative to their overdose experience (baseline and follow-up) and naloxone administration experience (follow-up).

Owing to the exploratory nature of the study's application of simulation to layperson OD resuscitation research, 2 investigators (T.C.G. and S.E.B.) independently completed post hoc reviews of all videos for a separate, gestalt adjudication of each subject's overall performance as either beneficial or harmful; discrepant reviews were resolved by a third reviewer. Bennett's prevalence-adjusted bias-adjusted kappa coefficient, due to detected prevalence bias,21,22 and interobserver percent agreement were calculated for the overview assessments.

All subject information, responses, and data were handled confidentially and securely stored in a Microsoft Access database with a deidentifying key code system; videos were deleted after data extraction.

Back to Top | Article Outline


One hundred seventy one inmates expressed an interest in study participation, of whom 107 (62.5%) consented to participate and completed the baseline assessment; 104 completed the study intervention training while incarcerated. Thirty-eight (35.5%) and 75 (70.1%) subjects had personally experienced or witnessed an opioid OD, respectively. None had previously been trained to respond to ODs or obtained a naloxone rescue kit in the community.

Ninety-nine subjects received an IN naloxone kit (96%) around the time of their release. Four additional subjects had not received their kit in person or by mail after release and were provided with one kit each at follow-up visit, and one subject could not be accessed for follow-up delivery of study naloxone. Subject demographics were representative of the target population: mean age, 34.8±8.9 years; male, 80%; race: white, 79%; African American, 6.5%; Native American, 6.5%; Hispanic/Latino, 18%.

Back to Top | Article Outline

One-month Postrelease Follow-up Assessment

At 1-month follow-up, 85 participants of the 103 eligible inmates who had been trained and assessed, released and residing in the community were re-assessed. (Five subjects were re-incarcerated before follow-up, 12 did not present for follow-up, and 1 had died from an opioid OD). Complete study and simulation data were available for all 85 follow-up subjects. Seven follow-up subjects (8.2%) had experienced opioid ODs after release, and 3 of the ODs were reversed using the study naloxone. In addition, subjects reported witnessing 8 nonfatal opioid ODs with 4 administrations of study naloxone, 1 performance of rescue breathing, and 1 activation of 911 services. No injuries, hospitalizations, or naloxone-related complications were reported.

Back to Top | Article Outline

Simulation Session Data

Complete data extracted from 85 video-recorded simulation sessions conducted in 2012–2013 were analyzed. Independent reviewers exhibited less than 10% disagreement for checklist coding across sessions, which evolved to full agreement after joint review of the videos. Subjects' median scores on task checklists during simulation scenarios were 12.0 (interquartile range, 11.0–15.0) of 21 total indicated actions. Checklist scores did not exhibit significant differences on bivariate analyses by age, sex, baseline OD and naloxone experience, and follow-up OD and naloxone experience (data not shown).

911 was activated by 72.9% of subjects via instruction to bystander (n = 7) or prop phone (n = 56). At least one basic airway maneuver and delivery of 2 rescue breaths with chest rise were completed by 56 (65.9%) and 78 (91.8%) subjects, respectively. Full naloxone doses were correctly administered into both nares by 44 (51.8%) of subjects at a median of 94.5 (70.8–125.0) seconds from scenario start; IN naloxone kit assembly and medication administration were completed in 58.0 (40.0–66.0) seconds from kit retrieval. Sixty subjects (70.6%) administered at least some naloxone intranasally: 8 participants administered naloxone into 1 nostril only, and 9 participants administered a partial dose. Several subjects attempted to administer naloxone intravenously (n = 4), intramuscularly (n = 1), orally (n = 1), or intranasally without the atomizer (n = 5) to the manikin. Six participants stated they did not know how to put the naloxone kit together during the simulation. Nonindicated actions that were not discussed during the experimental intervention were observed during 42 simulations (49.4%), primarily the performance of chest compressions. See Table 1 for details.

Post hoc review of videos determined that resuscitative actions by 8 (94.1%) 0 of 85 subjects were of potential overall benefit to the simulated opioid OD patient. The prevalence-adjusted bias-adjusted kappa coefficient was 0.81 with interobserver agreement of 90.6% for the 2 initial reviewers. Discrepancies for 8 subjects primarily revolved around whether and when during the scenario chest compressions were administered—these were resolved through review by a third investigator.

Back to Top | Article Outline


With widespread use of opioid pain relievers and a growing heroin crisis nationally, ODs are increasingly a major cause of morbidity and mortality.23 Opioid ODs are on the rise in both incidence and case fatality despite the presence of an effective, accessible, and commonly recognized antidote in the form of naloxone. Institution of programs for specific populations to promote layperson administration of naloxone may help curb OD deaths associated with this epidemic. These efforts may be facilitated by patient simulation, which provides a safe and validated24–27 mechanism for learners to demonstrate and be evaluated on their acquisition and application of knowledge, skills, and poise for high-stakes situations. Accordingly, the study team used simulation to complement existing research tools for investigation into the meaningful performance of layperson opioid OD resuscitative actions in a high-risk cohort.

Even with significant environmental constraints on the program intervention, more than half of the study participants were observed to correctly deliver resuscitative doses of IN naloxone with timeliness comparable to paramedic students.28 Concurrent with its primary evaluative purpose within the research program, study simulations were noted to enable subjects to act out what the experimental intervention had presented and to practice in effect for high-acuity events that are highly probable in their immediate future.29 Study conduct also highlighted the potential access that applied simulation could afford when trying to reach and help at-/high-risk populations. Conversely, subjects' difficulties with IN naloxone delivery device assembly and the frequent observation of impromptu and unrehearsed chest compressions revealed the complexity surrounding intervention efforts for this cohort. The limited and brief nature of opportunities available to train these individuals is expected to forestall traditional, resource-delimited methods such as structured Basic Life Support courses. Additionally, trepidation associated with accessing public safety services, that is, calling 9-1-1, in the context of illicit drug-related activities was reported frequently by subjects and represents a prominent impediment to optimal resuscitative care—expansion of Good Samaritan overdose immunity laws may ameliorate this problem.30 Next steps for community IN naloxone research efforts will need to address these unique operational challenges while ensuring correct device assembly (or alternative delivery mechanisms31,32), appropriate naloxone dose, proper administration technique, and the avoidance of nonindicated actions.

Study conduct demonstrated that simulation can be applied to outreach efforts directed toward inmate target populations housed in intrinsically limiting environments. With corroboration of the high frequency of opioid ODs in and around the study cohort over an extremely short follow-up period, the potential value of medical simulation techniques to assess (and potentially empower) those at risk seems significant. Study findings are being used nationally in ongoing programs to promote community naloxone administration for opioid ODs—continued application of the study simulation protocol is being considered for further investigation and standardized assessment of OD recognition and response in prison and community-based cohorts.

Back to Top | Article Outline


Unforeseen factors related to subject recruitment and attrition may have affected the study sample selection. As simulation could not be made available to subjects inside the correctional setting, subjects were not assessed for their acquisition and retention of opioid OD training immediately after the intervention. Records from public safety and emergency medical services were not available for subjects' self-reported experiences with overdoses and administrations of naloxone. Presimulation assessments at follow-up may have affected subject performance as recorded by scenario checklists, although the significant variability in checklist scores does not indicate a uniform effect. Study design and target population characteristics precluded follow-up beyond the research program's one-time scheduled session.

Back to Top | Article Outline


Investigators applied simulation to the postrelease assessment of opioid OD prevention training retention and skills acquisition in at-risk prison inmates. Findings from this exploratory study suggest the potential utility of simulation as a complementary evaluative methodology for investigation of layperson OD management skills.

Back to Top | Article Outline


The authors acknowledge Anna C. Cousins for her insight and assistance in manuscript preparation, the staff at the Rhode Island Department of Corrections for their help with this study, and all of the participants who volunteered for this research study.

Back to Top | Article Outline


1. Binswanger IA, Stern MF, Deyo RA, et al. Release from prison—a high risk of death for former inmates. N Engl J Med 2007;356:157–165.
2. Rosen DL, Schoenbach VJ, Wohl DA. All-cause and cause-specific mortality among men released from state prison, 1980-2005. Am J Public Health 2008;98:2278–2284.
3. Farrell M, Marsden J. Acute risk of drug-related death among newly released prisoners in England and Wales. Addiction 2008;103:251–255.
4. Dietze P, Jolley D, Fry C, et al. Transient changes in behaviour lead to heroin overdose: results from a case-crossover study of non-fatal overdose. Addiction 2005;100:636–642.
5. Warner M, Chen LH, Makuc DM. Increase in fatal poisonings involving opioid analgesics in the United States. NCHS Data Brief 2009;(22):1–8.
6. Kim D, Irwin KS, Khoshnood K. Expanded access to naloxone: options for critical response to the epidemic of opioid overdose mortality. Am J Public Health 2009;99:402–407.
7. Doe-Simkins M, Walley AY, Epstein A, et al. Saved by the nose: bystander-administered intranasal naloxone hydrochloride for opioid overdose. Am J Public Health 2009;99:788–791.
8. Green TC, Heimer R, Grau LE. Distinguishing signs of opioid overdose and indication for naloxone: an evaluation of six overdose training and naloxone distribution programs in the United States. Addiction 2008;103:979–989.
9. Piper TM, Rudenstine S, Stancliff S, et al. Overdose prevention for injection drug users: lessons learned from naloxone training and distribution programs in New York City. Harm Reduct J 2007;4:3.
10. Seal KH, Thawley R, Gee L, et al. Naloxone distribution and cardiopulmonary resuscitation training for injection drug users to prevent heroin overdose death: a pilot intervention study. J Urban Health 2005;82:303–311.
11. Sherman SG, Gann DS, Tobin KE, et al. "The life they save may be mine": diffusion of overdose prevention information from a city sponsored programme. Int J Drug Policy 2009;20:137–142.
12. Wagner KD, Valente TW, Casanova M, et al. Evaluation of an overdose prevention and response training programme for injection drug users in the Skid Row area of Los Angeles, CA. Int J Drug Policy 2010;21:186–193.
13. Piper TM, Stancliff S, Rudenstine S, et al. Evaluation of a naloxone distribution and administration program in New York City. Subst Use Misuse 2008;43:858–870.
14. Strang J, Powis B, Best D, et al. Preventing opiate overdose fatalities with take-home naloxone: pre-launch study of possible impact and acceptability. Addiction 1999;94:199–204.
15. Bird SM, McAuley A, Perry S, et al. Effectiveness of Scotland's national naloxone programme for reducing opioid-related deaths: a before (2006–10) versus after (2011–13) comparison. Addiction. 2015, epub ahead of print publication. Available at: Accessed April 15, 2016.
16. Green TC, Bowman SE, Ray M, et al. Development of an incarceration-specific overdose prevention video: staying alive on the outside. Health Educ J 2015;74:627–637.
17. Berg RA, Hemphill R, Abella BS, et al. Part 5: Adult Basic Life Support: 2010 American Heart Association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation 2010;122(Suppl 18):S685–S705.
18. Vanden Hoek TL, Morrison LJ, Shuster M, et al. Part 12: Cardiac arrest in special situations: 2010 American Heart Association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation 2010;122(18 Suppl):S829–S861.
19. Linstone HA, Turoff M. The Delphi Method: Techniques and Applications. Boston, MA: Addison Wesley, 1975. Available at: Accessed April 15, 2016.
20. Green TC, Ray M, Bowman SE, et al. Two cases of intranasal naloxone self-administration in opioid overdose. Subst Abuse 2014;35:129–132.
21. Bennett EM, Albert R, Goldstein AC. Communications through limited response questioning. Public Opinion Q 1954;18:303–308.
22. Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol 1993;46:423–429.
23. Jones CM, Logan J, Gladden RM, et al. Vital signs: demographic and substance use trends among heroin users—United States, 2002–2013. MMWR Morb Mortal Wkly Rep 2015;64:719–725.
24. Stefanidis D, Korndorffer JR Jr, Markley S, et al. Closing the gap in operative performance between novices and experts: does harder mean better for laparoscopic simulator training? J Am Coll Surg 2007;205:307–313.
25. Barsuk JH, Cohen ER, Feinglass J, et al. Use of simulation-based education to reduce catheter-related bloodstream infections. Arch Intern Med 2009;169:1420–1423.
26. Barsuk JH, McGaghie WC, Cohen ER, et al. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med 2009;37:2697–2701.
27. Wayne DB, Didwania A, Feinglass J, et al. Simulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: a case-control study. Chest 2008;133:56–61.
28. McDermott C, Collins NC. Prehospital medication administration: a randomised study comparing intranasal and intravenous routes. Emerg Med Int 2012; Available at: Accessed April 15, 2016.
29. Frank JW, Andrews CM, Green TC, et al. Emergency department utilization among recently released prisoners: a retrospective cohort study. BMC Emerg Med 2013;13:16.
30. National Conference of State Legislature. Drug overdose immunity and Good Samaritan laws. April 12, 2016. Available at: Accessed April 15, 2016.
31. Edwards ET, Edwards ES, Davis E, et al. Comparative usability study of a novel auto-injector and an intranasal system for naloxone delivery. Pain Ther 2015;4:89–105.
32. Krieter P, Chiang N, Gyaw S, et al. Pharmacokinetic properties and human use characteristics of an FDA approved intranasal naloxone product for the treatment of opioid overdose. J Clin Pharmacol 2016;5. Epub ahead of print. doi: 10.1002/jcph.759.

Drug overdose; Emergency treatment; Incarceration; Opioid-related disorders; Overdose prevention; Patient simulation; Prisoners; Resuscitation; Self-efficacy; Toxicology

Supplemental Digital Content

Back to Top | Article Outline
© 2017 Society for Simulation in Healthcare