Using the Generic Analysis Method to Analyze Sentinel Event Reports Across Hospitals: A Retrospective Cross-Sectional Study

Abstract

Objectives 

Improving patient safety by investigating sentinel events (SEs) is hampered by the focus on isolated events within hospitals and a narrow scope of traditional root cause analysis methods. We aimed to examine if performing cross-hospital aggregate analysis of SEs applying a novel generic analysis method (GAM) bearing a human factor perspective can enhance learning from SEs.

Methods 

A retrospective cross-sectional review of SE reports from 28 Dutch general hospitals using the GAM to reanalyze events was performed. A qualitative approach was used to identify contributing factors and system issues. Findings were discussed with a patient safety expert panel. Descriptive statistics and measures of associations between domains were calculated.

Results 

Sixty-nine SE reports were reviewed. Applying the GAM provided a more holistic SE analysis than a traditional method. Of the 405 identified contributing factors in all SEs, the majority was related to the persons involved (patients and professionals, n = 146 [36.2%]) and the organization (n = 121 [30%]). The most frequently recurring pattern was the combination of factors related to the persons involved, the technology used, the tasks of professionals, and organizational factors influencing the event. Cross-hospital aggregate GAM analysis of SEs helped to identify system issues and propose more system-oriented overarching recommendations.

Conclusions 

This study found that applying the GAM to analyze SEs across hospitals can help to improve learning from SEs and may result in proposing stronger recommendations. The method can support hospitals, working together in a network of hospitals, to jointly learn from SEs.

In-hospital adverse events threaten patient safety^1,2 and result in substantial societal costs.³ Learning from these events is vital to continuously improve patient safety. Hospitals therefore thoroughly monitor and investigate events. In particular, sentinel events (SEs),^4,5 which can be defined as unintended and unexpected events, are related to the quality of care and having caused death or serious patient harm.⁶ In SE investigations, hospitals aim to reconstruct what happened by combining multiple sources of data. After gathering a precise description of the event, they perform a root cause analysis and formulate possible recommendations to prevent event recurrence. These SE investigations and analyses are documented in a detailed report that bear great learning opportunities. Although the SE investigations and analyses have great potential to improve patient safety, important barriers exist to exploit its full potential.

One important constraint is that hospitals focus on singular events within their own organization,^4,7 instead of pursuing to learn from multiple SEs across hospitals. This leads to recurring events with similar causes and contributing factors that remain hidden.⁴ Hospital-level analysis is ineffective in identifying patterns and trends in causes and contributing factors on a system level, and hinders hospitals to jointly learn from SEs. Aggregate cross-hospital analyses of SEs might help to overcome these concerns, addressing more deeply engrained systemic and organizational issues.^4,5,8 Although in some countries (e.g., the United States^9,10) hospitals made attempts to jointly learn from aggregate analysis, other countries are still lagging behind. In the Netherlands, clustering SEs for cross-hospital analysis is hindered by among other issues the application of differing methods for analyzing SEs.⁸ Dutch hospitals often use Prevention and Recovery Information System for Monitoring and Analysis (PRISMA-medical),¹¹ Systematic Incident Reconstruction and Evaluation (SIRE),¹² or Tripod-bèta¹³ to perform root cause analyses,¹⁴ all with their own approach and focus, making it difficult to perform an aggregate analysis of SEs across hospitals.

A second constraint is that traditional SE-analysis methods typically focus on finding “the one linear” root cause, whereas SEs are more likely to result from interactions of multiple contributing factors.^4,7,15 This narrow scope of traditional analysis underestimates the complexity of health care. According to the emergent human factor scholarship, understanding of interactions between people (e.g., health care professionals and patients) and other sociotechnical elements (e.g., technology, organization, or the environment) within the complex work systems of health care might enlighten root cause analysis.^16,17 Traditional methods for SE analysis, however, are not based on such human factors thinking.

The disaggregated hospital-level approach and the lack of a human factors perspective in traditional SE-analysis methods may support the formulation of weak and ad hoc recommendations after SE investigations, instead of—more effective—system-level recommendations.¹⁸ This frustrates hospitals in their ambition to learn from SEs and prevent recurrence. Therefore, a novel generic analysis method (GAM) was developed.¹⁹ The GAM is an operationalization of the human factors based Systems Engineering Initiative for Patient Safety models.^20,21 It consists of a framework and affiliated questionnaire to analyze SEs, in which the foci and some important elements of 3 root cause analysis methods used by Dutch hospitals are integrated. The GAM could ease aggregate SE analysis across hospitals, irrespective of the initial root cause analysis method used, and enhance SE analysis.¹⁹ This could result in stronger recommendations.

In this article, we describe the results of an aggregate cross-hospital analysis of SEs applying the GAM. Using the rich data of SE reports, we aim to (1) exemplify what the GAM approach yields as opposed to an initial root cause analysis, (2) explore which contributing factors and eventual patterns emerge after aggregate cross-hospital GAM analysis, and, most importantly,(3) propose how these findings can help strengthen recommendations.

METHODS

A retrospective cross-sectional review of SE reports of Dutch general hospitals was performed. In the Netherlands, hospitals are obliged to report and investigate possible SEs and share a detailed report on the findings with the Dutch Health and Youth Care Inspectorate. The inspectorate assesses the reports on the quality of the investigation and appropriateness of recommendations.⁶ For investigating SEs, each hospital has an independent multidisciplinary committee involving quality and safety officers, and clinicians. These committees use multiple sources to elicit the course of events. The most important source is an in-depth interview with involved (health care) personnel, and patients and/or family, which forms the foundation for a detailed description of the event. Information gathered through interviews is then combined with a review of documentation such as medical records, test results (laboratory results, medical images, etc.), and guidelines and work instructions. This completes an information-rich, comprehensive reconstruction of the event, which forms the first part of the SE report. The committee of each hospital is trained in performing a method to identify root causes (PRISMA-medical,¹¹ SIRE,¹² or Tripod-bèta¹³) and formulate possible recommendations. Description of identified causes and formulated recommendations forms the second part of the report.

Data Collection

The SE reports used for this study were shared by 28 Dutch general hospitals that cooperate in a network aiming to improve patient safety. All associated hospitals were asked to share reports of closed investigations of SEs between 2017 and 2019.

Four safety critical themes were formulated to structure the selection of reports and collect a sufficient amount of, somewhat, similar events. This could increase the chances in finding eventual patterns and trends in causes and contributing factors, and at the same time help to include events relating to various care settings. The 4 safety critical themes were as follows: (1) diagnostic process in the emergency department, (2) medical technology, (3) anticoagulation, and (4) critically ill patients (of whom 1 or more vital functions, such as breathing, circulation, and consciousness, were disturbed and in danger of failure). These themes were based on their incidence in former Dutch adverse event studies^22–24 and inclusion in patient safety programs.²⁵ The processes underlying these themes cover a large part of care provided by hospitals, making a diagnosis, applying medical devices, prescribing and administering medication, and providing care to vulnerable patient groups.

Hospitals were asked to provide 1 or 2 anonymized SE reports per theme for analysis. After the hospital association checked the reports for anonymity, they were securely sent to the researcher (M.C.B.) for analysis.

Instrument: GAM

The reports were analyzed using the GAM, which is explained in detail in a former study.¹⁹ GAM consists of 3 main pillars (Fig. 1).

FIGURE 1: Overview of the GAM, operationalization of the Systems Engineering Initiative for Patient Safety models,^20,21 and elements of PRISMA,¹¹ SIRE,¹² and Tripod¹³ are adopted.

First, the method gathers some general information of the event (e.g., day, time, location, summary of the event). Second, sociotechnical domains in the work system are evaluated. Interactions and interdependencies between persons involved and factors related to other domains of the sociotechnical system (tasks, technologies, organization, and the physical and external environment) are mapped. This focus on interactions and interdependencies between centrally positioned persons and other domains of the work system reflects the human factor foundation of this method. Third, the method assesses the consequences of SEs for patients, their families, health care professionals, and the organization, as well as possible recommendations formulated by the hospital committee. The GAM was operationalized in an online survey environment that followed the GAM framework and included all the questions of the affiliated questionnaire (Supplemental Appendix 1, https://links.lww.com/JPS/A535).

Data Analysis

Analysis of SE reports was a 4-step process. First, the researcher (M.C.B.), as a GAM developer and expert, studied the event descriptions and reanalyzed these following the GAM. Second, a representative of each of the participating hospitals also reanalyzed SE reports using the GAM, to assess whether the general findings of the researcher aligned with what the hospital representative found. Agreement was qualitatively assessed by comparing a selection of answers of the most important open-ended questions. Third, findings were presented to and discussed with a panel of experts from the patient safety network, consisting of quality and safety officers and clinicians from 13 of the 28 participating hospitals, to check and verify our findings. Eventual differences in the interpretation between both analysts (hospital representative and researcher) were reflected upon. In case of uncertainties or irregularities in the data, initial SE reports were reassessed. The panel also made suggestions for recommendations based on the aggregated cross-hospital GAM analysis. Minutes were taken during meetings, and transcribed notes were sent to the participants afterward for verification. As a last step and double check, the data and general findings of the first theme (diagnostics at emergency departments) were verified by a second researcher (S.M.V.S.). There was a large overall agreement in findings between both analysts and the second researcher.

Collected data were then qualitatively analyzed to find relevant characteristics from the various sociotechnical domains in the set of open-ended questions. A sociotechnical characteristic was identified as a contributing factor whenever it (1) was explicitly described in the comprehensive event reconstruction in the SE report and (2) might have contributed in any matter to the event (as interpreted by the researcher). The factor did not necessarily have to be labeled as a root cause or basic risk factor. This aligns with the human factors philosophy, which proposes that we should evaluate interactions and interdependencies between all factors of the sociotechnical system and not merely focus on the final, hierarchal and linearly identified cause(s). We specifically tried to identify system issues, by evaluating all contributing factors. Factors that interacted with many other factors and those that recurred across events and hospitals and were related to the system were signaled as possible system issues.

After identifying the (system) factors that contributed to the events, we calculated descriptive statistics. Associations between the sociotechnical domains were measured using nonparametric tests to identify to what extent the various domains simultaneously occurred within one SE and which domains were associated. Descriptive statistics and correlations measures were calculated using Stata 15 software (StataCorp LLC, College Station, Texas).

RESULTS

Sixty-nine SE reports were studied. The majority was filed under the theme diagnostic process in the emergency department (n = 23). Twenty of the reports related to medical technology, 14 to anticoagulation, and 12 to critically ill patients. Of the patients involved, more than half was female (n = 35), and most patients were in the adult, working age class of 18 to 65 years (n = 36). Other patients were between 66 and 79 years (n = 18) and 80 years and older (n = 12), and 3 cases involved minors. Event characteristics varied across themes (Supplemental Appendix 2, https://links.lww.com/JPS/A536).

A Human Factors Perspective

Figure 2 presents an example of how an event was analyzed by the hospital using a traditional root cause analysis method (PRISMA-medical), and shows the results of analysis of the event description in the SE report using the GAM. PRISMA-analysis identified 3 independent root causes, whereas using the GAM to study the same event resulted in a total of 12 identified contributing factors and the evaluation of interactions between factors. Based on mapping the factors in this example and the identified interactions, possible system issues arose.

FIGURE 2: Example of an SE, analyzed using traditional root cause analysis; PRISMA-medical (left) and the GAM (right).

Aggregate Analysis: Contributing Factors and System Issues

Cross-hospital aggregate GAM analysis of SEs resulted in 405 identified contributing factors (mean per report, 6) from various sociotechnical domains. Table 1 shows that most factors were identified for the domains organization (n = 121; e.g., resources, culture, scheduling, or training policies) and person(s): the professionals (n = 103; e.g., knowledge, experience, tiredness, or cooperation) and patients (n = 45; e.g., frailty, or cooperativeness). Factors related to the technology used (n = 62; e.g., limited user-friendliness, technical failures, or misapplication) and tasks performed (n = 48; e.g., complexity of the task or coinciding tasks) also frequently contributed. The domains physical environment (n = 11; e.g., arrangement of the room or noise) and the external environment (n = 15; e.g., legislation or budget cuts) provided for less factors. The distribution of factors per domain differed only slightly between the 4 safety critical themes, with medical technology as most divergent (Fig. 3).

TABLE 1 - Sociotechnical Factors Identified in the SE Reports (n = 69) and Distribution of Contributing Factors (n = 405)

Sociotechnical Domain	Events* (n = 69), n (%)	Factors† (n = 405), n (%)	Examples of Contributing Factors
Person(s)	38 (55.1)	45 (11.1)	Patient Does not speak the language, which hampers the physical exam and medical history taking Stops taking medicine earlier than planned
	63 (91.3)	103 (25.4)	(Health care) Personnel Minimal experience in specific treatment leading to misapplication of a surgical instrument Suboptimal cooperation and communication between staff of different departments, which results in the omission of performing tests of vital parameters
Tools and technology	42 (60.9)	62 (15.3)	Not user-friendly electronic medication prescription system hinders correct medication prescription The tip of an electrosurgical instrument breaks off during surgery
Tasks	38 (55.1)	48 (11.9)	Complexities in the diagnostic reasoning process (i.e., tunnel vision) lead to missing a high-risk diagnosis (Related to high workload) vital parameters are not measured and registered, contributing to missing the deterioration of a critically ill patient
Organization	59 (85.5)	121 (29.9)	Anticoagulation work instructions are not up-to-date inducing a wrong medication policy Insufficient patient safety culture leads to taking high risks
Physical environment	10 (13)	11 (2.7)	Patient room is too small so caregivers get in each other’s way during resuscitation Positioning of monitors and alarm screens hampers accessible patient monitoring information (e.g., cardiac activity, blood pressure, and oxygen saturation) for caregivers
External environment	13 (19.1)	15 (3.7)	Closure of wards in a nearby hospital causes extra bed pressure, contributing to the decision to not admit a patient Shortage of emergency doctors hinders sufficient staffing during weekends

*Number of SEs in which 1 or more factors of this domain contributed and corresponding percentage.

^†Number of identified factors in all SEs and corresponding percentage.

FIGURE 3: Distribution of the sociotechnical factors for the SEs for each theme and all SEs together.

Identified system issues were related, for example, to suboptimal cooperation (e.g., poor communication between hospital departments and information transfer between chain partners), limited user- and patient-friendliness of medical technology (e.g., usability problems related to electronic health records and prescription systems, or suboptimal technology design for clinical practice), and poor work structures (e.g., high workload, scheduling, staffing, high staff turnover or multitasking).

Patterns of Interactions

Figure 4 shows commonly identified interactions between domains, based on domains that mutually contributed within one event. Characteristics of the person(s) involved contributed to nearly all SEs and therefore interact with most other domains such as organization (n = 57), technology (n = 39), and tasks (n = 37). The domains technology and organization (n = 36), tasks and organization (n = 35), and technology and tasks (n = 24) also often contributed simultaneously. At the same time, it shows the rather small relation between the physical and external environment and other domains. None of the relations between domains were statistically significant associated (Supplemental Appendix 3, https://links.lww.com/JPS/A537).

FIGURE 4: Interactions between sociotechnical domains over all reports. Thick lines indicate the most frequently observed combination of—more than 2—contributing domains.

The most frequently observed combination of—more than 2—sociotechnical domains contributing to an event was the combination of the persons involved, the technology used, the tasks of (health care) professionals, and organizational factors influencing the event (n = 14 [20.3%]).

Improving Patient Safety

In the studied reports, hospital committees formulated 211 possible recommendations (mean per report, 3). A majority of the formulated recommendations were related to writing, adapting, or raising awareness to protocols and work instructions (n = 102). Recommendations regarding technology and information and communication technology systems (n = 24), improving staff training (n = 23), and communication (n = 17) were also regularly suggested. Measures aimed at improving collaboration (n = 7) were proposed less frequently, and changes in the organization of care were rarely suggested. Other types of recommendations (n = 38) focused on, for example, case discussion in department meetings or with chain partners. Most hospital committees formulated possible recommendations narrowly directed toward the identified root causes, as were identified with traditional root cause analysis methods.

Findings from the cross-hospital aggregated GAM analysis were used by the expert panel of the patient safety network to formulate possible recommendations on identified recurring patterns of factors and unruly organization or system-level issues. Examples were to consider appointing a case manager for anticoagulation who can be easily approached by all units for questions regarding the anticoagulation policy, or evaluate and adapt the predefined lists of medication options in administrative systems, making it more intuitive and less prone to mistakes.

DISCUSSION

Sixty-nine SE reports from 28 Dutch general hospitals were retrospectively analyzed using the GAM. We exemplified what the method yields as opposed to traditional root cause analysis methods, provided insights in the contributing (system) factors of SEs and patterns herein across SEs, and illustrated how these findings can be used to improve possible recommendations.

Studying an event applying the GAM provided a richer and more holistic analysis than a traditional root cause analysis method. This finding aligns with previous studies that showed how traditional methods are characterized by their focus on one or a few linear, independent and detached causal factor(s).^4,26 Alternative approaches using human factors principles to analyze events typically identify a variety of interdependent factors, such as methods guided by the Systems Engineering Initiative for Patient Safety model^27,28 or other models (e.g., Human Factors Analysis Classification System²⁹ and Systems Theoretic Accident Modelling and Processes³⁰).

Aggregated cross-hospital SE analysis using the GAM indicates that a majority of contributed factors were related to the persons and organization domains. Human and organizational factors are known to be the most frequently identified causes in SE or adverse event studies.^8,31,32 Not many contributing factors were identified that related to the physical and external environment. A possible explanation could lie in the fact that traditional root cause analysis methods do not or only limitedly address these domains and thus may be underreported in the SE reports. At the same time, the more system-related issues (e.g., the external environment)^4,33 are often less explicit and thus more difficult to identify in SE investigations. Nevertheless, we found some important system issues by looking at interacting factors that were recurring across events and hospitals.

In line with the identification of contributing factors, factors related to the persons and organizations domain most frequently coincided within one event. Although no domains were statistically significantly associated, systematically mapping factors of the various domains provided a valuable insight in the complexity and combinations of factors underlying SEs. The most frequently observed pattern (i.e., more than 2 coinciding factors) was the combination of factors related to the persons, organization, technology, and tasks. This confirms the complex multidimensional nature of SEs, as also found in other SE studies.^8,34

Recommendations in the studied SE reports were often narrowly directed toward the identified root causes, and the majority was related to improving work instructions and guidelines. These are considered common pitfalls. As discussed by Trbovich and Shojania,³³ root cause analysis teams tend to align their recommendations closely to identified causal factor(s) and resort to actions that are based on the specific individual case rather than on system issues. Other studies evaluating recommendations as reported by root cause analysis teams also found that recommendations are mainly aimed at, for example, policy changes and administrative actions.^18,35 These types of recommendations are considered to be weak. Bos et al³⁶ confirmed this finding regarding generally weak recommendations, as they concluded that the majority of recommendations in 115 Dutch SE reports did not meet quality criteria and thus may have limited effect in improving patient safety. On the other hand, aggregate cross-hospital GAM analysis helped to formulate more system-oriented recommendations aimed at solving recurring issues across events and organizations. The examples provided, about adaptations in technology and institutional-level changes are considered more effective and sustainable.¹⁸ Such system-level recommendations, however, may require substantial (financial) resources. Literature suggests SE investigators typically avoid these types of interventions.^33,36 Because the aggregated cross-hospital GAM analysis recommendations are based on multiple SEs from various hospitals, they may provide more relevance and urgency than case-specific recommendations. In turn, this could help to mobilize the appropriate resources to implement and follow-up on recommendations.

There are several study limitations that should be considered when interpreting our findings. Our practice sample was purposefully selected and restricted to Dutch general hospitals. Therefore, caution is required when drawing generalizable conclusions. We think that our findings regarding the core of the identified patterns in contributing factors and trends in possible recommendations may be theoretically generalizable to other event types and settings, as they merely differed among the 4 investigated themes. However, results could differ when SEs would be included randomly. Likewise, if we would have included SEs from other types of hospitals, the substantive results might have been different because of, for example, a more complex patient population in academic hospitals. As a result of the study design, the numbers of SEs per theme do not reflect their true incidence. For example, numbers per theme may rather be more affected by the timing of data collection (e.g., data on the latter 2 themes were collected during the COVID pandemic) than by their occurrence in practice. In addition, although 69 SE reports are not a particularly small sample, examining a greater number of reports would enable us to perform more extensive statistical analysis.

Applying a new analysis method to existing SE reports in itself also holds some limitations. Researchers were not able to ask specific additional questions to those involved in the SE but were bound to the comprehensive event descriptions in the report. This may have resulted in an underestimation of factors related to the domains that are only minimally covered in traditional SE investigations and analyses, such as the physical and external environment. Nonetheless, the event descriptions in most SE reports were written in such detail that we were able to determine additional contributing factors, also for the domains that are less embedded in the traditional approaches.

Our study has some important implications. Hospitals can use the substantive insights into recurring and system-level issues as identified in the current study. More importantly, we recommend hospitals to use existing networks and its infrastructures to periodically perform a cross-hospital analysis of SEs, to identify patterns and trends in contributing (system) factors and put urgency on suggested recommendations. The GAM can assist to perform such retrospective aggregate cross-hospital analysis of SE reports, irrespective of the variety in underlying root cause analysis methods. Variation in root cause analysis methods has proven to be a major limitation in learning from SEs; it is therefore important to strive for more standardization.^8,37,38 Ideally, the issue of unstandardized SE analysis is solved upfront, for example, by hospitals including the GAM as an alternative method for directly investigating and analyzing SEs. This will ease the analysis of SEs across hospitals and embeds a human factors perspective in future SE investigations. Further validation and evaluation of the usability of the method is recommended though.

We also propose that hospitals should train SE investigators in human factors principles and GAM analysis. Training of SE investigators could help them to set the focus during SE investigations on interlinked human factors domains and ask the right questions during interviews with those involved. Embedding human factors expertise in the SE-investigation team could enlighten the SE analysis and may improve the quality of possible recommendations.¹⁶ For specific cases, hospitals could also consider to consult other types of independent experts, for example, clinical technicians, health-IT specialists, or organizational or behavioral psychologists.

CONCLUSIONS

In conclusion, this study underlines the value of the GAM to learn from SEs. It provides a richer analysis than traditional root cause analysis methods, as it combines the foci of 3 existing methods. The human factors approach of the method stimulates to evaluate the complex nature of SEs. Applying the method for aggregate analysis of SEs across hospitals helps to identify trends and patterns, and may assist in formulating system-oriented recommendations. Wider application of the GAM has the potential to further improve patient safety.