Enhanced Free-Text Search for Aggregated Medication Error Report Analysis and Risk Detection

Objectives Detecting medication errors (MEs) and learning from them are the key elements of medication safety management in health care. While the aggregation of the data and learning across the ME reports could help detect and manage organizational risks, the inconsistent and partly missing structural data complicate the analysis. The objective of this study was to examine whether an analysis of free-text data of aggregated ME reports could contribute to the detection of organizational risks. Methods A retrospective, cross-sectional analysis of ME reports from a patient safety incident reporting system in a tertiary hospital 2017–2021. Clustering of characteristics and variables of ME reports with an enhanced free-text search of the 10 most frequent active substances (TOP10) related to ME reports using Microsoft Excel. Validity analysis of the four most frequent active substances of the search results (TOP4). Evaluation of the possible impact of the enhanced free-text search method on ME report analysis and risk detection. Results The enhanced free-text search increased significantly the number of relevant ME reports of TOP10 active substances from 698 reports to 1578 reports. The validity of the enhanced free-text search results in TOP4 active substances was more than 74%. The enhanced free-text search revealed also new ME findings. Conclusions Enhanced free-text search can contribute to the aggregate analysis of clustered ME reports and to the improvement of ME risk detection. The enhanced free-text search method enables more comprehensive analysis of the free-text data with commonly available software and provides new insights into medication safety improvement.

variable in HaiPro data. 8When filling a ME report into HaiPro, the system suggests a suitable option to choose from the medicine register based on the first letters to be entered into the "medication name" data field.However, it is possible to leave the suggested medicine unselected, in which case the system saves the written text as the name of the medicine.Misspelled or otherwise self-entered medication names that differ from the information in the register are not classified under the medicine in question and its active substance, respectively.Thus, the data classification of the medication names and active substances may well be incomplete and fragmented.
The free-text data entry is used in the HaiPro system in some data fields to record descriptive information, for example, the description of the error.Thus, the medication names and active substances can be mentioned in the free-text description of the error without filling anything into the medicine name data field at all.In this respect, while relevant information may be available, it will not automatically be classified into the system's statistics.

Data Analysis
Data analysis was performed by the researcher (V.V.).The ME report data were aggregated by active substances categorized originally by the individual reporters and the report managers.The 10 most frequent active substances (TOP10) relating to MEs were defined and analyzed.The categorization according to active substances was used in this analysis as it is the official classification applied in the HaiPro system and it is commonly used in the ME report free texts along with the trade names.An enhanced free-text search was conducted to find missing ME reports relating to these TOP10 active substances also from the unclassified data entries.The enhanced free-text search was performed using the Excel COUNTIF-function to search for the mentions of the TOP10 active substances and medicine trade names in the Finnish market from the free-text data of "medication name" and "incident description" data fields (Fig. 1).
The enhanced free-text search used customized search terms to cover the postposition inflection of the TOP10 active substances and trade names.The customized terms were formulated by abbreviating the active substances and trade names (Table 1).For example, search terms "furesi*" and "furosemi*" were used to collect reports information relating to furosemide from the selected free-text fields because that is the way that the drug's trade name Furesis is inflected in the Finnish language from the middle of the word.As an example, the sentence "adverse reaction of Furesis" would translate into Finnish as "Furesiksen haittavaikutus."

Outcome Validation and Classification
The search results were filtered to cover only error reports with a search result higher than zero; in other words, we used reports with at least one mention of the targeted search terms in the "medication name" or "incident description" data field (Fig. 1).
After the enhanced free-text search, a more detailed analysis was performed of the sample data set to ensure the validity of the enhanced free-text search results.The 4 most common active substances with the most ME reports (TOP4) were selected as the sample data set, as they covered the 3 most common active substances within both the original classification and after the enhanced free-text search even though the order changed after the search.In the validity analysis, it was determined whether the ME reports found by the enhanced free-text search were actually related to the targeted active substances by reading and analyzing the free-text case descriptions.The validity analysis was performed by the researcher (V.V.) and all ambiguous cases were discussed and resolved by the research team (V.V., S.S., K.H., M.T.).Actual citations of the incident descriptions translated from Finnish to English were included into the study results to provide a wider perspective of the validity analysis.
The TOP4 active ME reports were examined whether they provided new ME insights into risk management in addition of the original ME reports.The new ME findings that were only present in the search results were also classified with the National Coordinating Council for Medication Error Reporting and Prevention taxonomy (NCCMERP) by the researcher (V.V.). 14All ambiguous cases were resolved by the whole research team (V.V., S.S., K.H., M.T.).

Ethical Considerations
The study design obtained a research permission from the KUH in the autumn of 2021.Ethics approval was not required according to the Finnish National Ethics Committee, as the study was based solely on anonymous registry data. 15

Detection of Events
During 2017-2021 altogether, 5463 MEs were reported in the KUH.Of these reports, 46.4% (n = 2536) included the related  medication name selected from the register and thus the active substance had been classified originally (Fig. 2).Some of the unclassified data entries were misspelled or contained wrong information; for example, the active substance had been entered in the medication name data field.
The most common active substance in all MEs was enoxaparin sodium (n = 133, 2.4% of the reports) according to the original classification (Fig. 3).
By exploiting the enhanced free-text search, the frequency of the 10 most common active substance increased significantly from 698 to 1578 reports (Fig. 4).The frequency of the most commonly reported active substance, enoxaparin sodium increased by 114% (from 133 to 284 reports) increasing the proportion of the whole ME data reports relating to enoxaparin sodium from 2.4 to 5.2%.The most dramatic increase was seen with paracetamol as its frequency increased by 220% (from 70 to 224 reports).
In numerical terms, the largest group of ME reports in both the original classification and the enhanced free-text search was attributable to enoxaparin sodium.However, the order of the next largest groups changed after use of the enhanced free-text search.The ME report number of paracetamol increased to 224 and oxycodone to 223, dropping cefuroxime from being the second largest group to the fourth with 184 ME reports.

Validation
When analyzing the contents of these TOP4 active substances ME reports, it was found that 74.1% to 90.8% of the reports found with enhanced free-text search were valid and the search result covered all the original reports (Table 2).In the remaining reports, the active substance or trade name had been mentioned in the free text for some other purpose, for example, as a corrective measure used to improve the error situation, or as a part of the patient's long medication list, but the error itself was not actually related to it.When removing the original reports from the search results and examining only the ME reports identified with the enhanced free-text search, then the validity of the search results varied from 62.3% to 79.3%.
An example of an incident description as translated from Finnish into English from an ME report, which was identified with the enhanced free-text search and related to enoxaparin sodium: "...it was noticed that Klexane [trade name of enoxaparin sodium] 60 Â 2 had been stopped from the patient 1 day earlier.However, it remained on the list (so the card had not been changed).The patient received the medicine two more times after the medicine should have been terminated." An example of an incident description translated from Finnish into English from an ME report identified with the enhanced free-text search but not related enoxaparin sodium: "In the recent weeks, the ward had stopped the labeling of, e.g., Klexane [trade name of enoxaparin sodium] and other s.c.medications.The patient is taking Caprilon [trade name of tranexamic acid] in IV form…."

Types of Errors
Analysis of the TOP4 active substances revealed that the enhanced free-text search resulted in completely new MEs, which were not present in the original ME reports (Table 3).
The enhanced free-text search did not reveal any completely new ME types with oxycodone, which would not have been present in the original reports.In this respect, oxycodone differed from the other 3 active substances.However, the search did reveal a substantial increase in the number of oxycodone ME reports relating to duplicate therapy with the number of reports increasing from 1 (1.0%) to 10 (4.9%).In addition, the ME reports relating to inadequate pain management increased from 10 (10.2%) to 22 (14.3%).The analysis of enhanced free-text results of paracetamol revealed a formerly unidentified look-alike, sound-alike (LASA) risk.Several reports described a mix-up between paracetamol (IV) with sodium bicarbonate (IV) resulting in the wrong medicine being administered to patients.None of these reports were originally classified as an error relating to paracetamol, and therefore they would not have been noticed as a LASA risk.

Statement of Key Findings
In this retrospective study of data from a Finnish tertiary hospital, we have examined whether an enhanced free-text search could contribute to aggregate analysis of clustered ME reports and ME risk detection.This study found the enhanced free-text search analysis represented a useful approach in the analysis of ME reports and thus may represent an improvement in ME risk detection.
In our study, we focused on analyzing the ME reports of the TOP4 active substances most commonly producing ME reports in our study data.The amount of ME reports relating to the TOP4 active substances increased from 403 original reports to 915 reports representing an increase of 2.3-fold.We found that 74.1% to 90.8% of the reports found with enhanced free-text search identified with the TOP4 active substances search were valid and truly related to the active substance being targeted in the search.When taking into account only the valid ME reports of the TOP4, the report amount almost doubled from the 403 to 772, that is, an increase of 1.9-fold.
The validity varied between the analyzed active substances.The enhanced free-text search results of paracetamol had lowest validity (74.1%).This may be due to the frequent use of paracetamol and therefore it was mentioned as a part of the patient medication listing in the error reports more commonly than the other TOP4 active substances, even although the reports were linked to other medications.

Strengths and Limitations
To our knowledge, the enhanced free-text search method has not previously been used to analyze MEs.The enhanced free-text search strategy produced more ME reports to the data analysis.There was good validity with ME reports relating to the TOP4 active substances.The analysis of enhanced free-text results revealed formerly unidentified MEs, for example, a systematically occurring LASA risk, which would not have been identified without the newly introduced search method, that is, the problem would not have been noticed and therefore the necessary changes to avoid this error would never have been implemented.Of course, the use of enhanced free-text search needs to be validated also with other active substances to evaluate whether the validity is as good as with the 4 substances used here as a sample.One potential target for the use of the enhanced free-text search would be to analyze different aspects of the ME data, which are normally difficult to focus on due to the nature of the data and the default classification, for example, patient falls.However, further studies are needed to ensure the effectiveness and validity of the enhanced free-text search in other topics and in other data sets.
The study data contains some limitations, which have to be considered from the perspective of this research approach.For example, the default setting with the HaiPro error reporting system makes it possible to choose only one medication in each error report.This feature is most likely also one important reason for the incomplete data regarding the medication names or active substances.When the error involves multiple medications, the reporters may (1) leave the field empty, (2) choose one medication, or (3) enter the complete list medications as a free text.In contrary, for example, commonly used NCCMERP taxonomy of MEs guides to classify each medication involved in ME with precise coding principles. 14The reporting quality in the HaiPro system could be improved by allowing multiple medications to be added to events where appropriate.However, the use of this kind of enhanced free-text search enables the analysis of the unclassified reports and reports relating to multiple medications, for example, taking into account both medications in LASA risk pairs.Nevertheless, we would argue that enhanced free-text search results should not be used alone or solely for statistical purposes without a content analysis or a validation of the results, as the validity of the results was not 100%.However, it is important to bear in mind that also the original classifications are imperfect and quite often misclassified or unclassified. 8,9,16][19] One limitation relating to the use of enhanced free-text search is that the search targets must be defined and suitable and relevant search terms must be generated before performing the actual search itself.Thus, the use of enhanced free-text search requires some kind of preanalysis of the data and an insight into what is being sought.Thus, we had to exploit a relatively narrow search strategy and conducted the enhanced free-text search such that it only encompassed the TOP10 active substances.With more sophisticated tools, it would be also possible to undertake a more comprehensive search approach as revealed by Laatikainen et al, 9 as they categorized ME reports into ATC groups according to the involved medicines with search terms created out of every trade name and active substance name on the Finnish market.

Interpretation Within the Context of the Wider Literature
The 10 most commonly reported active substances in the study data did correspond to previous studies conducted from Finnish ME report data in HaiPro system. 9,20,213][24][25] However, surprisingly, no antidiabetic drug was found here from the TOP10 ME report list.This might be due to our approach as we examined the data at the level of the active substance.There are rather many different active substances used in the diabetes care and therefore no individual antidiabetic substance is found in TOP10 listing.
As a result of the enhanced free-text search, the frequency of MEs among TOP10 common active substances increased significantly from the original report amounts.The use of enhanced free-text search revealed that there was both underreporting and misclassifications in the ME reports.These findings are consistent with previous studies conducted by aggregating the ME reports from HaiPro data indicating that the data quality can be variable. 8,9,16The data categories can be interpreted in various ways depending on the classifier, which may lead to inconsistent data and thus to biased perspective.Therefore, it is crucial to conduct further analyses of the ME report data to avoid biases in the interpretation of the data, for example, when planning safety improvements.

Implications for Future Research and Practice
In this study, our enhanced free-text search demonstrated that it is possible to conduct a light version free-text analysis of ME reports data even with Excel, which is often available for many  researchers by default.A manual review of the study data as a whole and finding the missing ME reports of the TOP10 active substances or even one of them without the enhanced free-text search would not have been possible, as the data set was so extensive.The use of enhanced free-text search method could be beneficial, for example, in the process of identification and classification of organizational high-alert medications, as the search method can enable a larger number of ME reports to be taken into account in the analysis. 20,211][12] The exploitation of AI could also improve the reliability of the classifications of ME reports as they would be uniform and not dependent on the person making the classification of the report.However, these more advanced tools are often unavailable in day-to-day medication safety improvement work, whereas the enhanced free-text search is possible to conduct with Excel®.

CONCLUSIONS
The enhanced free-text search can contribute to the aggregate analysis of clustered ME reports and to the improvement of ME risk detection.The enhanced free-text search provided a significant increase in relevant ME reports for the analysis, with a validity of more than 74%.The analysis of the enhanced free-text search results revealed new ME findings, which otherwise would have remained unobserved.The enhanced free-text search method makes it possible to conduct a more comprehensive analysis of the free-text data while using commonly available and easily accessible software.

FIGURE 1 .
FIGURE 1.The enhanced free-text search protocol.

FIGURE 3 .
FIGURE 3. TOP10 active substances relating to ME reports in the KUH 2017-2021.

FIGURE 4 .
FIGURE 4. TOP10 active substances relating to original ME reports and ME reports found with enhanced free-text search in the KUH 2017-2021.

TABLE 1 .
Search Terms in the Enhanced Free-Text Search of TOP10 Active Substances in the KUH ME Report Data 2017-2021 (Continued next page)

TABLE 2 .
Validity of the Enhanced Free-Text Search Results of ME Reports: Whether the ME Report in Search Results Related to the Targeted Active Substances in the KUH 2017-2021 Medication error reports according to the original active substance classification.†New ME reports found with enhanced free-text search (original reports excluded).‡All reports found with enhanced free-text search (original reports included). *

TABLE 3 .
New ME Findings Resulting From TOP4 Active Substances With Enhanced Free-Text Search in the KUH 2017-2021 J Patient Saf • Volume 20, Number 4, June 2024