Share this article on:

Controlled Substance Reconciliation Accuracy Improvement Using Near Real-Time Drug Transaction Capture from Automated Dispensing Cabinets

Epstein, Richard H. MD, CPHIMS; Dexter, Franklin MD, PhD; Gratch, David M. DO; Perino, Michael BS; Magrann, Jerry MS

doi: 10.1213/ANE.0000000000001289
Technology, Computing, and Simulation: Research Report

BACKGROUND: Accurate accounting of controlled drug transactions by inpatient hospital pharmacies is a requirement in the United States under the Controlled Substances Act. At many hospitals, manual distribution of controlled substances from pharmacies is being replaced by automated dispensing cabinets (ADCs) at the point of care. Despite the promise of improved accountability, a high prevalence (15%) of controlled substance discrepancies between ADC records and anesthesia information management systems (AIMS) has been published, with a similar incidence (15.8%; 95% confidence interval [CI], 15.3% to 16.2%) noted at our institution. Most reconciliation errors are clerical. In this study, we describe a method to capture drug transactions in near real-time from our ADCs, compare them with documentation in our AIMS, and evaluate subsequent improvement in reconciliation accuracy.

METHODS: ADC-controlled substance transactions are transmitted to a hospital interface server, parsed, reformatted, and sent to a software script written in Perl. The script extracts the data and writes them to a SQL Server database. Concurrently, controlled drug totals for each patient having care are documented in the AIMS and compared with the balance of the ADC transactions (i.e., vending, transferring, wasting, and returning drug). Every minute, a reconciliation report is available to anesthesia providers over the hospital Intranet from AIMS workstations. The report lists all patients, the current provider, the balance of ADC transactions, the totals from the AIMS, the difference, and whether the case is still ongoing or had concluded. Accuracy and latency of the ADC transaction capture process were assessed via simulation and by comparison with pharmacy database records, maintained by the vendor on a central server located remotely from the hospital network. For assessment of reconciliation accuracy over time, data were collected from our AIMS from January 2012 to June 2013 (Baseline), July 2013 to April 2014 (Next Day Reports), and May 2014 to September 2015 (Near Real-Time Reports) and reconciled against pharmacy records from the central pharmacy database maintained by the vendor. Control chart (batch means) methods were used between successive epochs to determine if improvement had taken place.

RESULTS: During simulation, 100% of 10,000 messages, transmitted at a rate of 1295 per minute, were accurately captured and inserted into the database. Latency (transmission time to local database insertion time) was 46.3 ± 0.44 milliseconds (SEM). During acceptance testing, only 1 of 1384 transactions analyzed had a difference between the near real-time process and what was in the central database; this was for a “John Doe” patient whose name had been changed subsequent to data capture. Once a transaction was entered at the ADC workstation, 84.9% (n = 18 bins; 95% CI, 78.4% to 91.3%) of these transactions were available in the database on the AIMS server within 2 minutes. Within 5 minutes, 98.2% (n = 18 bins; 95% CI, 97.2% to 99.3%) were available. Among 145,642 transactions present in the central pharmacy database, only 24 were missing from the local database table (mean = 0.018%; 95% CI, 0.002% to 0.034%). Implementation of near real-time reporting improved the controlled substance reconciliation error rate compared to the previous Next Day Reports epoch, from 8.8% to 5.2% (difference = −3.6%; 95% CI, −4.3% to −2.8%; P < 10−6). Errors were distributed among staff, with 50% of discrepancies accounted for by 12.4% of providers and 80% accounted for by 28.5% of providers executing transactions during the Near Real-Time Reports epoch.

CONCLUSIONS: The near real-time system for the capture of transactional data flowing over the hospital network was highly accurate, reliable, and exhibited acceptable latency. This methodology can be used to implement similar data capture for transactions from their drug ADCs. Reconciliation accuracy improved significantly as a result of implementation. Our approach may be of particular utility at facilities with limited pharmacy resources to audit anesthesia records for controlled substance administration and reconcile them against dispensing records.

Published ahead of print April 22, 2016

From the *Department of Anesthesiology, Perioperative Medicine and Pain Management, University of Miami, Miller School of Medicine, Miami, Florida; Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania; Department of Anesthesia, University of Iowa, Iowa City, Iowa; and §Department of Information Systems and Technology, Thomas Jefferson University Hospital, Philadelphia, Pennsylvania.

Accepted for publication February 16, 2016.

Published ahead of print April 22, 2016

Funding: None.

The authors declare no conflicts of interest.

This material was presented, in part, at the 2013 Postgraduate Assembly of the New York State Society of Anesthesiologists and at the 2014 Annual Meeting of the American Medical Informatics Association.

Reprints will not be available from the authors.

Address correspondence to Richard H. Epstein, MD, Department of Anesthesiology, Perioperative Medicine and Pain Management, University of Miami, Miller School of Medicine, 1400 NW 12th Ave., Suite 3028-8, Miami, FL 33136. Address e-mail to repstein@med.miami.edu.

Accurate accounting of controlled drug transactions by inpatient hospital pharmacies is a requirement in the United States under the Controlled Substances Act (CSA).a The CSA requires “every registrant … to maintain on a current basis a complete and accurate record of each such substance manufactured, imported, received, sold, delivered, exported, or otherwise disposed of…”a At some hospitals, scheduled drugs are signed out to anesthesiologists or nurse anesthetists at central or satellite pharmacy locations for use in their patients. Subsequently, there is accounting for administration to individual patients, with unused drug returned to the pharmacy. However, processes for distribution of controlled substances from pharmacies increasingly are being replaced by peripherally distributed, automated dispensing cabinets (ADCs) as part of hospital efforts to improve drug inventory control and charge capture and to reduce labor costs.1,2

Despite the promise of improved accountability offered by the perioperative use of ADCs, this may not be realized. Vigoda et al.3 reported a 15% prevalence of discrepancies for controlled substances among database records in their anesthesia information management system (AIMS), effectively the prescription and administration record, and their ADC, the dispensing system. A myriad of reasons for the errors were noted (see their Tables 1 and 2), including incorrect amounts documented in the AIMS, incorrect amounts wasted or returned, misspelled names and incorrect medical record numbers (MRNs), duplicate entries in the ADC system, and ADC transactions documented under the wrong patient.3 We noted a comparable incidence of controlled substance discrepancies (15.8%; 95% confidence interval [CI], 15.3% to 16.2%) between AIMS and pharmacy database records at our hospital, with similar explanations for discrepancies adduced by our providers.b Although we were able to reduce (P < 10−6) the discrepancy rate to 10.5% (95% CI, 9.6% to 11.5%) by providing next day reports via e-mail4 to our residents and Certified Registered Nurse Anesthetists for the cases in which they were the sole anesthesia provider,b we were not satisfied with this level of performance. A desire to reduce the error rate further motivated the process improvement project described in this article.

Because most reconciliation errors appear to be clerical,3 we hypothesized that providing near real-time feedback to our providers for their patients would reduce the discrepancy rate. The feedback listed the amounts documented in the AIMS and the balance of transactions in the pharmacy database. We theorized that having this information would allow providers to waste and transfer drugs more accurately, would improve the fastidiousness with which transactions are entered in the AIMS, and would create peer pressure among anesthesia providers, with all 3 contributing to process improvement. We were not concerned that implementation of such feedback would affect the performance of our previously described drug diversion surveillance system, because its algorithms are not based on evaluating reconciliation discrepancies.5,6 Development of the processes described in this article was necessary because our ADC system currently is not able to link the balance of transaction records to administration records in electronic health record systems, only to identify internal count discrepancies in drawer items.c We are unaware of any other ADC system with such cross-platform linking abilities.

We describe the technical details of how we implemented a system to capture ADC activity in near real time and compare (1) the balance of drug transactions with (2) the documentation of administration of those drugs in the AIMS. We also report the resulting changes in the accuracy of controlled drug reconciliation.

Back to Top | Article Outline

METHODS

The Thomas Jefferson University IRB determined that this project did not meet the regulatory definition of “human subjects research.”

Back to Top | Article Outline

Technical Infrastructure

Our infrastructure consists of the following components:

  • Innovian® (Dräger, Telford, PA) as our AIMS
  • Pyxis® Anesthesia System 4000, Pyxis MedStation, and Pyxis MedStation ES systems (CareFusion, a BD Company, San Diego, CA) as our ADC system
  • Cloverleaf interface engine (INFOR, New York, NY)
  • Health Level 7® (HL7, Health Level Seven International, Ann Arbor, MI) message parsing, implemented using the open source Tool Command Languaged (Tcl)
  • TCP/IP port and socket listener, written in Perl v5.14.2e (running on a Windows 7 Virtual Machine under the integrated development environment Padre v0.94f)
  • Comprehensive Perl Archive Network library DBI, used to connect to our AIMS server databaseg
  • SQL Server Native Client 10.0 ODBC driver (Microsoft, Redmond, WA), installed by loading the complimentary version of SQL Server Express 2008 R2 (Microsoft)h
Figure 1

Figure 1

Our controlled substance reconciliation reports comparing ADC and AIMS records were implemented using Transact-SQL (Microsoft) on the primary AIMS server (Microsoft Windows Server Standard SP2). These were exported at 1-minute intervals via a scheduled job to a web server (Microsoft Windows Server 2003 SP2) as HTML documents. Such reports were visible to all anesthesia providers from a link on AIMS workstation computers. Figure 1 shows the schematic representation of the data capture and presentation process.

Back to Top | Article Outline

Pyxis Transaction Capture

Pyxis workstations are located in each operating room (OR) and procedural room where anesthesia care is delivered. In addition, there are workstations located in the OR suite hallways, postanesthesia care units, and holding areas. Providers log on to the workstations using a unique identifier that is stored in both the Pyxis and the AIMS databases. Each user has a primary biologic (fingerprint) and a secondary text password (used only if the fingerprint scanner cannot acquire the provider’s fingerprint). Transactions are executed by selecting a patient, a drug, a quantity, and a type of transaction (dispense, waste, return, or transfer to another patient). This information is stored locally on the Pyxis workstation and retrieved by a local interface server (maintained by the vendor) via a polling process that executes every 1 to 2 minutes, depending on how the local workstation is configured. Transactions are sent to the Pyxis server in a proprietary, pipe delimited, text format; converted to HL7; and then transmitted to the Cloverleaf interface engine.

Table 1

Table 1

As the reference source for determining accuracy, we relied upon Excel workbooks (compressed in the ZIP file format) generated from DataInsight® (CareFusion) and transferred using the secure file transfer protocol. The source database, located on a remote server maintained by the vendor, is populated from the local Pyxis interface server, but data are not available until 1 to 2 hours after transactions have occurred. Thus, this information is not usable for near real-time reconciliation. We were not permitted direct access to the Pyxis database (either locally or remotely) because of performance and security concerns by the vendor. Pyxis data fields used by our near real-time processing are presented in Table 1.

Back to Top | Article Outline

Cloverleaf Interface Engine Processing

Table 2

Table 2

Upon receipt of a drug transaction message from the Pyxis interface server, a Tcl script (Table 2) is executed. To reduce processing requirements, transactions are filtered so that only those involving scheduled drugs originating from workstations accessed by anesthesia providers are processed. The script extracts the desired transactional data from the various designated fields of the HL7 message segments, and then transmits these to the perl listener script as a string, terminated with an end of message indicator (Table 1). Queued transactions are concatenated and sent as a single string. An acknowledgment is sent back to the Cloverleaf interface engine after processing each transaction; however, the acknowledgment was used only during simulation testing to modulate the data flow. The Tcl script was written to transmit all messages immediately upon receipt to the perl script, where they were captured in a large data buffer, provided that the socket connection was active, without waiting for acknowledgment of an individual message receipt.

Back to Top | Article Outline

Perl Processing

Table 3

Table 3

The perl listener script (Table 3) establishes a persistent connection between the Cloverleaf interface engine, using the IP address of the server where Cloverleaf is running and the socket to which Cloverleaf will connect. Once the connection is established, the script loops continuously, waiting for a new message to arrive. After receipt, the message is checked to ensure that the expected authorization code (Table 1) is present at the start of the message, then is split into fields and appended to a local transaction table on the AIMS database server. Received messages are logged to the local display console and to a flat text file, as are any error messages. If a message is received containing multiple transactions (as would occur if the transfer process stops and is restarted), each transaction is processed in a loop, writing to the database after each has been parsed. There is a large (65,536 bytes) data buffer to process multiple messages (sufficient for approximately 24 hours of transactions).

Back to Top | Article Outline

Database Processing and Reconciliation Report Generation

Every 1 minute, a stored SQL procedure is executed that calculates the total amount for each scheduled drug (fentanyl, sufentanil, remifentanil, morphine, hydromorphone, midazolam) that has been documented in the AIMS for all patients entering the OR after 5 PM on the previous date (to include cases running past midnight). (The 5 PM cutoff was selected based on the start times of cases occasionally running into the next day.) For each of these patients, all Pyxis transactions executed by anesthesia providers after the above start time are retrieved from the local Pyxis transaction table (described above). Using the transaction timestamps and the MRN, the unique case identifier (CASE_ID) of the corresponding AIMS case is determined. In situations where the CASE_ID cannot be identified (e.g., a temporary patient had been entered with a missing or incorrect MRN), the patient name is used for the match. The SQL script then generates an HTML document listing each patient, drug, Pyxis balance, amount documented in Innovian, the difference between the Pyxis and Innovian amounts, and the last provider on the case (Appendix A). The HTML document had no ability to log the identity of individuals viewing the reports, so we were unable to assess the relationship between discrepancies and individuals not bothering to check their discrepancies. The last provider logged in to the case is assigned responsibility,i because as part of the handover process, we require that the provider being relieved wastes all controlled substances (except opioids being given by continuous infusion). The relieving provider is required to draw up his or her own controlled substances.

Documentation events were added to the AIMS to allow a provider to document a reason why specified drugs were not in balance (e.g., too much drug was wasted in Pyxis, creating a negative balance that could not be fixed). Such documentation was incorporated in the reconciliation reporting and such cases were noted as having an explained discrepancy. Cases still ongoing (i.e., the patient had not yet left the OR) listed the Pyxis balance, the cumulative AIMS dose, and the difference. Providers had 60 minutes from the anesthesia end time to reconcile their drugs, after which discrepancies were highlighted in red, visible to all providers on the HTML report from a link within the AIMS. At 2:30 and 2:45 PM each day, a text message was sent to the alphanumeric pager7 of all providers with outstanding discrepancies reminding them to fix any problem before they left for the day. The discrepancy report was archived each day at 11:55 PM. For the analysis in this study, the discrepancy report was rerun each day for all patients entering the OR on that day, including drug transactions occurring after midnight for cases spanning >1 day. We made corrections to eliminate apparent discrepancies that resulted from providers not having their Pyxis linking numbers entered into Innovian at the time of the transactions. We also improved the logic matching Pyxis transaction records to the corresponding Innovian records for manually entered patients or cases.

Back to Top | Article Outline

Transaction Processing Testing

Performance of the system was tested via simulation prior to implementation. Time from transfer of transactions from the perl script to the database was measured by sending 100 batches of 100 dummy drug transaction messages, with each sent after acknowledgment of receipt of the prior message was received from the perl script. The mean and SD of the latency were determined among the 100 batches, and accuracy checked by comparing the transmitted string to what was inserted into the database. Latency testing in the production system was performed by comparing a convenience sample of 1384 consecutive transactions retrieved in Excel format from the central Pyxis database (see the section Pyxis Transaction Capture, above) to what was inserted via the near real-time process. Latency was determined by comparing the timestamps of the recorded transaction time at the workstation and the insertion time in the database. We verified that clocks on all devices were synchronized to network time.

Back to Top | Article Outline

Reconciliation System Implementation Details

An initial report was sent to all anesthesia providers on June 27, 2013, listing their scheduled drug reconciliation error rate for cases in which they were the only anesthesia provider and the implementation date of July 1, 2013, for the start of next day reconciliation reports (Appendix B). A sample report is shown in Appendix C. On April 28, 2014, near real-time reporting was instituted, and e-mails were sent to all providers informing them of the new process (Appendix A). Next day reports were discontinued on May 8, 2014. Announcements related to the reconciliation process were made during grand rounds near the start of each implementation phase, and periodically thereafter, reminding everyone to accurately document their drugs and follow department policies regarding scheduled drug handling.

Back to Top | Article Outline

Statistical Methods

Accuracy of data capture by the near real-time process was tested by comparing the percentage of missed transactions in each of 72 one-week bins compared to data stored in the central (reference) Pyxis database. The percentages were compared using the method of batch means.8–13 Data were binned by week to mitigate potential correlation between failures to capture transactions on a given day (e.g., due to server, interface, or software related issues). For comparisons of discrepancy rates, data were binned by month. Larger bins were used because there were many fewer discrepancies than transactions.

Back to Top | Article Outline

RESULTS

Simulation and Acceptance Testing

During simulation, 100% of the 10,000 messages transmitted were accurately inserted into the database. Latency (time from message transmission to insertion in the database) was 46.3 ± 0.4 milliseconds (95% CI, 45.4 to 47.2 milliseconds). The rate of transaction processing among the 100 batches was 1305 ± 10 per minute (95% CI, 1284 to 1325). For comparison, the maximal rate of transactions in any given minute over a 2-month interval prior to implementation was 11.

During acceptance testing of live transactions examined over a 5-day period, only 1 of 1384 transactions showed a difference between what was recorded in the central Pyxis database and the local Pyxis transaction table. This was for a “John Doe” trauma patient whose name had been changed by the time the central database was queried. Mean latency from the time of transaction entry at the ADC to the insertion time in the database was 65.5 ± 37.3 seconds (SD), with 90% of transactions appearing within 108 seconds of their entry. This latency was mostly attributable to the polling frequency from the local Pyxis interface server to the workstations (i.e., every 1 to 2 minutes).

Back to Top | Article Outline

Latency and Accuracy During Actual Use

The average number of scheduled drug transactions in the n = 18 one-month bins was 17,118 (95% CI, 15,988–18,248). Once a transaction was entered at the Pyxis workstation, 84.9% (95% CI, 78.4% to 91.3%) of these transactions were available in the database on the AIMS server within 2 minutes (Fig. 2). Within 5 minutes, 98.2% (95% CI, 97.2% to 99.3%) were available. The median latency was 60.3 ± 3.3 seconds (95% CI, 53.4 to 67.2 seconds). Prolonged latencies were usually clustered, as expected from our previous work involving network communcations.6 These occurrences typically reflected interruptions in the transfer process due to interface server interruptions, physical disconnection of the Pyxis workstation Ethernet cable to the network jack, and other network or software issues.

Figure 2

Figure 2

Of the 145,652 transactions present in the central Pyxis database between April 1, 2014, and September 30, 2015, 24 (0.018%; 95% CI, 0.002% to 0.034%) were missing from the local database created via the near real-time capture process. During 64 of the 72 one-week bins, 100% of transactions were captured. In the remaining 8 one-week bins, the median number of missed transactions was 2, with a range from 1 to 7. Of the 24 missed transactions, 12 were identifiable in the error log file and resulted from an inability to connect to the SQL database because of the server being unavailable for longer than the timeout interval. The reason for the remaining 12 missing transactions could not be determined; however, these were usually correlated events from different workstations, indicating a system problem rather than an individual workstation issue.

Back to Top | Article Outline

Improvement in Reconciliation Accuracy

Implementation of the near real-time reconciliation reporting process resulted in a decrease in the discrepancy rate between Pyxis and AIMS controlled substance transactions compared to the prior epoch when next day reports were being provided. When fentanyl, morphine, hydromorphone, remifentanil, sufentanil, and midazolam were included in the analysis, the discrepancy rate decreased from 8.8% to 5.2% (difference = −3.6%; 95% CI, −4.3% to −2.8%, P < 10−6; Fig. 3). During the Baseline, Next Day Reports, and Near Real-Time Reports epochs, 50% of the discrepancies were accounted for by 13.8% (36/261), 18.8% (33/176), and 12.4% (30/242) of the providers with the greatest numbers of discrepancies executing transactions, respectively (Fig. 4). During these same intervals, 80% of discrepancies were accounted for by 31.8%, 39.8%, and 28.5% of the providers, respectively (Fig. 4).

Figure 3

Figure 3

Figure 4

Figure 4

As a sensitivity analysis, we excluded sufentanil and remifentanil (usually administered by continuous infusion), because pump rate changes are not transmitted automatically to the AIMS. Invariably, the lack of an interface results in small differences between the infusion drug total in the AIMS and what is calculated by the infusion pump. The discrepancy rates decreased from 8.2% to 4.9% (difference = −3.3%; 95% CI, −4.1% to −2.5%; P < 10−6). Our hypothesis that implementing near real-time reconciliation reports would improve the accuracy of controlled drug reconciliation was accepted.

Table 4

Table 4

Figure 5

Figure 5

The absolute values of many discrepancies for fentanyl and midazolam were less than or equal to the typical vial size for each drug (100 μg and 2 mg), with peaks at the vial sizes in use and common incremental doses (50 μg fentanyl and 1 mg midazolam) (Fig. 5). The unit dose discrepancies were often various user errors that resulted in an inability to match the Pyxis transactions with the AIMS record, as well as occasional failures to document administration of midazolam in the holding area, the usual location where this drug is given. Since we do not have AIMS workstations in our holding areas, providers have to remember to back date the entry after the patient arrives in the OR, and sometimes forget to log the administration electronically. Table 4 has examples of scenarios resulting in a discrepancy of a full vial of a drug.

Back to Top | Article Outline

DISCUSSION

The near real-time system we describe for the capture of transactional data flowing over the hospital network was highly accurate and reliable and exhibited acceptable latency. This methodology will allow other organizations to implement similar data capture for transactions from their drug ADCs, as well as other HL7 data flowing through hospital interface servers. Interface servers are ubiquitous in hospital information networks, and all have software to parse and redirect message streams. The software we describe to receive data and write it to a relational database is open source (i.e., no license fees), is well supported by the user community, and is technically straightforward to implement.

Recently, the Drug Enforcement Agency launched an investigation of controlled substance processes and practices at Massachusetts General Hospital related to potential diversion, during which multiple violations related to scheduled drug reconciliation errors in the OR were noted. In addition to paying a $2.3 million dollar fine, the hospital agreed to a corrective action plan that requires, in part, that the “… Department of Pharmacy to conduct daily operating room post-case reconciliation (“PCR”) of controlled substances dispensed, used, or wasted, and, if any discrepancy is not resolved within 72 hours, to report the discrepancy to the Drug Diversion Compliance Officer.”j Thus, we suspect there will be increasing interest by hospital pharmacies to implement more robust auditing of controlled substances in ORs.

Deployment of our near real-time process resulted in a statistically significant improvement in the accuracy with which our providers matched their documentation of controlled substances in the AIMS with the balance of activity in the Pyxis system. Nonetheless, despite considerable effort by the department to encourage providers to improve their reconciliation accuracy, substantive problems remain. We do not have data quantifying the influence on peer pressure from the public display of individual providers’ discrepancies. However, it was not the situation that most discrepancy errors resulted from the actions of a few providers; rather, the discrepancies were widely distributed among the staff, similar to what was found by Vigoda et al.3 Most errors encountered were a result of user inattentiveness, and many could have been fixed easily (e.g., transferring the drug from one patient to another). However, some errors cannot be corrected (e.g., wasting too much drug, since there is no ability to “unwaste”), and others can be difficult to diagnose and fix (e.g., a patient who is prematurely discharged from the Pyxis system and disappears from the patient list). We note that with both process improvement initiatives we attempted (next day and near real-time reports), the effect was apparent within 1 month of implementation, with no substantive improvement thereafter (Fig. 3). This suggests that evaluation of efficacy can be made shortly after deployment rather than waiting many months to collect data.

As hospital pharmacies continue attempts to reduce costs by decreasing human resources, and installing automated drug-dispensing carts, more of a burden is being placed on anesthesia departments to ensure that controlled substances are documented accurately. A consequence of such cost-saving initiatives at our institution is that witnessed wasting of unused controlled substances has been pushed back completely to the providers. Errors made during this process are responsible for a substantive portion of our residual discrepancy issues. Likely such errors would be fewer if unused controlled substances were returned to the pharmacy for auditing and potential forensic evaluation of syringe contents. Recent closings of satellite pharmacies serving our ORs have created further logistical problems, because the main pharmacy is too far away from most of the ORs to be readily accessible, even if the former process of pharmacy returns and auditing were to be resurrected.

The process we describe has several limitations. First, the specific implementation we describe (Table 1) is for a single vendor’s ADC system (i.e., Pyxis). However, all such systems involve entry of transactions at remotely located workstations, with transfer to a central database. Thus, data should be available in near real time either by methods similar to what we describe or through direct access to the pharmacy database (if allowed by the vendor). Second, to implement the near real-time process we describe, information system resources with adequate technical abilities are needed. However, we think it is reasonable to expect that most hospital information systems departments would have such individuals on staff because the required skills are typically needed to support other core processes. Third, since the system operates in near real time, 24/7 monitoring is necessary to identify interruptions to transactional processing. We run such software checking for potential problems every 15 minutes, with text and e-mail messages sent automatically to on-call personnel when it appears that the system has stopped functioning. True problems requiring intervention have occurred infrequently (11 times in the 18 months that the system has been operational), but nonetheless must be anticipated and procedures put in place to restore functionality in a timely fashion. Fourth, when restarting the interface engine is necessary (e.g., to apply a software or operating system update), there is a need to manually restart the perl listener script (as written), since a new connection to the interface engine needs to be established. Our perl script does not include an automatic facility to refresh the communications channels, for example, if no transactions are received within some specified interval based on day of week, time of day, and incorporating holidays. The restart process requires coordination among the people who maintain these systems (e.g., an interface engineer and application analyst). Finally, the ability to capture drug documentation information from the AIMS within minutes of data entry is necessary. Systems that do not routinely make such drug documentation available for querying until the next day (e.g., Epic Anesthesia, Epic, Verona, WI) would need additional software written to extract, translate, and load those data into a suitable format (e.g., a relational database table) for comparison with the ADC records.

In conclusion, we showed that using informatics with reports available within minutes of transactions is a practicable approach. We showed that this process can successfully assist providers in meeting the regulatory requirements of the CSA. Our approach may be of particular utility in situations where pharmacy resources to audit anesthesia records for controlled substance administration and reconcile them against dispensing records are limited.

Back to Top | Article Outline

APPENDIX A

A sample reconciliation report, shown below in the yellow box, is a redacted portion of a near real-time reconciliation report, updated every minute, and visible within from a link within the anesthesia information management system. At the time of this report (10:07 AM), there were discrepancies noted for 5.1% of cases that had concluded. Patient A represents an ongoing case in which there are 100 μg of fentanyl yet to be administered, but all the dispensed midazolam was given. Patient B shows a case that has concluded where the administrated amount matches the balance of Pyxis transactions. Patient C is a case which ended recently, but where the provider has not yet completed documentation. Patient D is a case that concluded more than 60 minutes ago in which there is an outstanding discrepancy of 50 μg between what was dispensed from the Pyxis workstation and what was documented in the AIMS. Patient E is a case where the amount documented in the AIMS is more than what was dispensed from the Pyxis workstation. Patient F shows the balance for a sufentanil infusion in a case that is ongoing. If the case had ended at the time of the report, 171.481 μg would be documented as the amount wasted, based on the timing of infusion rate changes in the AIMS. In the real report, the names of patients and providers are displayed. Reports contain 2 lists, sorted by patient last name, first name, and sorted by provider last name, first name. The latter list allows providers to easily check possible discrepancies on all their patients.

Pyxis-Innovian Reconciliation Report by Patient as of Oct 7 2015 10:07 AM (Press F5 to Refresh) Discrepancy Rate = 5.1%

This report is designed to assist you in reconciling controlled substances between Pyxis and Innovian. In the Pyxis column, the current balance of Pyxis transactions (vend, return, waste, transfer) is displayed for each patient/drug combination, current as of the report time at the top of the report. In the Innovian column is what has been documented in Innovian. The Unreconciled column reflects the difference between the 2 and should be what is left in the syringe containing the drug. If this number is negative, then you have either documented the administration of drugs not taken out on the patient, entered duplicate documentation, or wasted/transferred more drug than was available for such purposes. The Comment column gives the current status of the case. Once the Anesthesia End time is entered, you will have 60 minutes to reconcile your Pyxis and Innovian records. After this, any discrepancies will be flagged in red and marked as a discrepancy.

Lines in green are drugs withdrawn from Pyxis but where the drug is not yet documented as given in Innovian. If you see 2 patients with the same name (or very close), one of which is in green, this likely means that the patient was incorrectly added to Pyxis (e.g., wrong MRN, incorrect spelling). To resolve this, you need to transfer the drug to the correct Pyxis patient (i.e., the one matching the MRN of the Innovian patient). The last person vending will be shown as the provider.

Sample On-Line Reconciliation Report

Sample On-Line Reconciliation Report

It is imperative that you resolve any discrepancies prior to leaving for the day. If you relieve someone who is going home, you must ensure that all controlled substances are in balance, as you will be held responsible for any errors that are not corrected before you accept the case. There should be NO controlled substances given to you in a syringe; all such drugs must be wasted and reconciled accurately by the provider you are relieving. This means that the Unreconciled column should have a value of 0 for all noninfusion drugs. Infusions (e.g., remifentanil, sufentanil) should be carried over to the next provider, and a balance is expected in the Unreconciled column for these substances.

Back to Top | Article Outline

APPENDIX B

Initial Notification E-Mail Describing the Next Day Reconciliation System, Sent June 27, 2013, to Each Anesthesia Provider

To All Anesthesia Providers:

We are having continuing problems with lack of agreement between the doses of controlled substances removed/wasted from the Pyxis machines and the amounts documented in Innovian. Currently, only 84% of our patient records are in agreement, which is creating regulatory problems for the hospital and will potentially result in licensure problems for providers. Federal law mandates that the controlled substances we remove from the Pyxis machines must be accounted for by what is documented as having been given, less doses that are wasted, transferred, or returned. If the current inaccuracies continue, reporting to the DEA may become necessary. This may result in the loss of medical licensure, termination from TJUH, and even criminal prosecution. This is a serious matter that you must address. We must account for all doses of controlled substances that we remove from the Pyxis machines and administer using Innovian.

Below is your personal record over the past 18 months for cases in which you were the sole provider and where the patient only went to the OR one time on the date of surgery. Only fentanyl, morphine, hydromorphone, and midazolam are included, and infusions (remifentanil and sufentanil) were excluded. Thus, errors in these cases are solely attributable to you.

Error rate = (# of case-drug combinations that were out of balance)/(total # of case-drug combinations).

Provider Name: Anne Esthesia

# of Qualifying Drug Doses: 353

Your Error Rate: 23%

Average Department Error Rate: 16%

In reviewing samples of the discrepant records, we identified a number of patterns that were responsible for the errors, some of which are described below:

1. Not documenting drug given after the record was printed

2. Apparent computational errors in calculating the wasted amounts

3. Drawing up drug on patient A and giving it to patient B

4. Wasting drug on patient A that should have been wasted on patient B

5. Careless manual entry of misspelled patient names or incorrect/incomplete MRNs

6. Failure to waste nonadministered doses

7. Apparent splitting of drugs between patients

8. Use of generic patients (e.g., *Ultrasound Patient) instead of the correct patient when dispensing

The bottom line is that you are personally responsible for accounting for every microgram of a controlled drug that you dispense from the Pyxis. The drug needs to be administered, wasted, transferred to another patient, or returned such that the balance of the transactions sums to 0.

The following practices should be followed:

  1. If you are relieved from a case for the day, you need to waste whatever controlled drugs you have drawn up in syringes. These may NOT be given to the next provider, even if nothing has been administered from the syringe.
  2. At the time of the relief, in addition to the handover of clinical care, you must go through each scheduled drug and make sure that everything is in balance.
  3. Controlled substances should be wasted at the end of every case, preferably with your attending, and not accumulated until the end of the day.
  4. If you give additional drug after the record is printed, you must reopen the record, record the missing amounts, reprint the record, and replace the original copy in the PACU.
  5. Infusions should be left running and not wasted. The final provider on the case is responsible for accounting for the infusion amounts. We are allowing a 10% quantitative error in the infusion drug amounts. You can use the Pyxis Reconciliation Report in the Innovian browser to help do the calculations.
  6. For brief reliefs (e.g., lunch or bathroom breaks), you do not need to waste drugs in a syringe. But on your return, you should ensure that any drug given in your absence was properly documented in Innovian.
  7. Do not use the generic patients for removal of scheduled drugs. In OB, the *OB Patient can be used, but only if you do not know the patient, and the drug must be promptly transferred once the patient is identified.
  8. If you have to manually enter a patient into Pyxis, be sure to accurately spell his or her name and provide the complete 8 digit MRN, including leading 0’s, if present.

We are now getting an hourly feed of all Pyxis transactions, which are being automatically processed and compared with the Innovian totals. Each morning at 6 AM, starting on July 1, 2013, any discrepancies will be identified and the involved providers contacted with an e-mail message. Do not under any circumstance try to alter the Innovian or Pyxis record after the fact to “fix” the problem. If a large error was involved, you need to contact Dr. Gratch for further instructions.

These are really serious matters that must be addressed. We are not expecting perfection, but the target for the department is to reduce the percentage of cases where noninfusion-controlled substance accounting does not balance to ≤2.5%. Individuals whose error rates are >5% will be subject to counseling, initiation of a performance review, and further consequences, as indicated.

We are expecting everyone to cooperate in this endeavor.

David Gratch, DO

Rich Epstein, MD

Zvi Grunwald, MD

Back to Top | Article Outline

APPENDIX C

Sample next day discrepancy report

To: Anne Esthesia

Report Date: June 15, 2013

On June 14, 2013, there was an apparent discrepancy between Pyxis and Innovian records for Midazolam on the following patient that you took care of: PATIENT, Imyour

The net amount of drug dispensed − wasted − credited from Pyxis = 1000 μg

The total amount documented in Innovian = 2000 μg

Discrepancy (Innovian − Pyxis) = 2000 μg

This indicates that you documented drug administration in Innovian that was in excess of the net amount dispensed from Pyxis. The drug came from somewhere, but where?

Try to recall if you entered the patient manually into Pyxis, might have accidentally administered drug dispensed on a different patient, were given the drug by another provider, or got the drug from a non-OR Pyxis machine. You must enter the patient’s full MRN and spell their name accurately if you do a manual entry. If no Pyxis transactions are listed for the patient, there is a good chance that this error was made.

It is mandatory that you reply by e-mail copying both Drs. Gratch and Epstein within 24 hours with an explanation of what you think caused this discrepancy. See the details below for the Pyxis and Innovian records retrieved from the respective databases. Those not replying in writing will have an appropriate notice placed in their personnel file. Failure to respond may result in further action.

Some of these apparent discrepancies may be resolved following further investigation and will be noted as such. Indicating that you looked but could not figure out what happened is an acceptable response. But you must reply in writing.

DO NOT ALTER THE INNOVIAN OR PYXIS RECORDS TO TRY TO MAKE THE TRANSACTIONS BALANCE UNDER ANY CIRCUMSTANCE. ADJUSTMENTS WILL BE DEALT WITH BY US ADMINISTRATIVELY, IF NECESSARY.

Richard H. Epstein, MD

David M. Gratch, DO

Zvi Grunwald, MD

{Transaction details on the patient in question, as well as all transactions by the recipient of the e-mail were provided, but have been redacted}

Back to Top | Article Outline

DISCLOSURES

Name: Richard H. Epstein, MD, CPHIMS.

Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript. He is the archival author, responsible for maintaining the study records.

Attestation: Richard H. Epstein has approved the final manuscript

Name: Franklin Dexter, MD, PhD.

Contribution: This author helped design the study and write the manuscript.

Attestation: Franklin Dexter has approved the final manuscript.

Name: David M. Gratch, DO.

Contribution: This author helped write the manuscript.

Attestation: David M. Gratch has approved the final manuscript.

Name: Michael Perino, BS.

Contribution: This author helped write the manuscript.

Attestation: Michael Perino has approved the final manuscript.

Name: Jerry Magrann, MS.

Contribution: This author helped write the manuscript.

Attestation: Jerry Magrann has approved the final manuscript.

Back to Top | Article Outline

RECUSE NOTE

Dr. Franklin Dexter is the Statistical Editor for Anesthesia & Analgesia. This manuscript was handled by Dr. Maxime Cannesson, Section Editor for Technology, Computing, and Simulation, and Dr. Dexter was not involved in the editorial process or decision in any way.

Back to Top | Article Outline

FOOTNOTES

a Code of Federal Regulations. General requirements for continuing records. 21 CFR 1304.21. http://www.gpo.gov/fdsys/pkg/CFR-1999-title21-vol9/pdf/CFR-1999-title21-vol9.pdf. Accessed September 9, 2015.
Cited Here...

b Epstein RH, Gratch DM. Automated reconciliation of controlled substances dispensed from an automated anesthesia drug cart and an anesthesia information management system. 67th Annual Meeting of the New York State Society of Anesthesiologists. December 2013.
Cited Here...

c Masengale C. Reconciling diversion activity utilizing the Pyxis MedStation® system. https://www.ncsbn.org/2014DCM_CMasengale.pdf. Accessed October 18, 2015.
Cited Here...

d Tcl Core Team. http://core.tcl.tk/. Accessed September 9, 2015.
Cited Here...

e The Perl Programming Language. www.perl.org. Accessed September 9, 2015.
Cited Here...

f Padre Developers. http://padre.perlide.org/developers.html. Accessed September 9, 2015.
Cited Here...

g Bunce T. DBI. http://search.cpan.org/~timb/DBI-1.634/DBI.pm. Accessed September 9, 2015.
Cited Here...

h Microsoft. http://www.microsoft.com/en-us/download/details.aspx?id=30438. Accessed September 9, 2015.
Cited Here...

i Staff were notified by e-mail and during grand rounds just prior to the rollout of the near real-time reconciliation reports in May 2014 that the relieving provider would be held responsible for any errors not corrected prior to taking over the case. This created an incentive for the staff to ensure that the documentation was accurate before taking over the case. In addition, this attribution of responsibility is printed at the top of the reconciliation report (Appendix A). Forcing providers to waste their unused scheduled drugs in syringes prior to relief simplified the reconciliation process (i.e., the balance between the Pyxis and Innovian records should be 0) but is not strictly necessary. If syringes are allowed to be transferred between providers for an ongoing case, the balance would need to be checked against the (nonzero) syringe amount.
Cited Here...

j Department of Justice. Settlement Agreement. http://www.justice.gov/usao-ma/file/778651/download. Accessed January 11, 2016.
Cited Here...

Back to Top | Article Outline

REFERENCES

1. Ray MD, Aldrich LT, Lew PJ. Experience with an automated point-of-use unit-dose drug distribution system. Hosp Pharm 1995;30:18, 20–3, 2730.
2. Cooper RL, Merry A. Stonemetz J, Ruskin K, Medication management. In: Anesthesia Informatics. New York: Springer, 2009:209–26
3. Vigoda MM, Gencorelli FJ, Lubarsky DA. Discrepancies in medication entries between anesthetic and pharmacy records using electronic databases. Anesth Analg 2007;105:1061–5.
4. Epstein RH, Dexter F, Patel N. Influencing anesthesia provider behavior using anesthesia information management system data for near real-time alerts and post hoc reports. Anesth Analg 2015;121:678–92.
5. Epstein RH, Gratch DM, Grunwald Z. Development of a scheduled drug diversion surveillance system based on an analysis of atypical drug transactions. Anesth Analg 2007;105:1053–60.
6. Epstein RH, Gratch DM, McNulty S, Grunwald Z. Validation of a system to detect scheduled drug diversion by anesthesia care providers. Anesth Analg 2011;113:160–4.
7. Epstein RH, Dexter F, Rothman B. Communication latencies of wireless devices suitable for time-critical messaging to anesthesia providers. Anesth Analg 2013;116:911–8.
8. Ledolter J, Dexter F, Epstein RH. Analysis of variance of communication latencies in anesthesia: comparing means of multiple log-normal distributions. Anesth Analg 2011;113:888–96.
9. Dexter F, Epstein RH, Marcon E, Ledolter J. Estimating the incidence of prolonged turnover times and delays by time of day. Anesthesiology 2005;102:1242–8.
10. Dexter F, Epstein RH. Increased mean time from end of surgery to operating room exit in a historical cohort of cases with prolonged time to extubation. Anesth Analg 2013;117:1453–9.
11. Dexter F, Marcon E, Epstein RH, Ledolter J. Validation of statistical methods to compare cancellation rates on the day of surgery. Anesth Analg 2005;101:465–73.
12. Dexter F. High-quality operating room management research. J Clin Anesth 2014;26:341–2.
13. Austin TM, Lam HV, Shin NS, Daily BJ, Dunn PF, Sandberg WS. Elective change of surgeon during the OR day has an operationally negligible impact on turnover time. J Clin Anesth 2014;26:343–9.
© 2016 International Anesthesia Research Society