Secondary Logo

Journal Logo

Original Articles

High Rate of Implementation of Proposed Actions for Improvement With the Healthcare Failure Mode Effect Analysis Method

Evaluation of 117 Analyses

Öhrn, Annica PhD, RN; Ericsson, Carin BSc; Andersson, Christer PhD, MD; Elfström, Johan MD

Author Information
doi: 10.1097/PTS.0000000000000159
  • Open

Abstract

The Healthcare Failure Mode Effect Analysis (HFMEA) is a prospective risk assessment method first introduced by DeRosier et al1 in 2002. The HFMEA emanates from the Failure Mode Effect Analysis method used in, for example, the chemical industry for decades. The objective of HFMEA is to correct latent system errors in health care before they lead to adverse events. The HFMEA approach includes the following steps: constructing a multidisciplinary team, diagramming process, identifying failure mode and causes, prioritization of potential risks by determining the severity and the probability of a failure mode, a decision tree analysis, and, finally, defining actions and outcome measurements.1

The use of HFMEA increased considerably in the previous decade, but its impact is hardly described in the literature. Although many studies describe how risk analyses are used,1–8 only a few focus on the effects of an organization. The method has also been criticized for being time consuming.9

In Sweden, a modified HFMEA model was implemented and described in a national handbook, published in 2005,10 and at the same period was introduced into the health care organizations. The Swedish modified model includes all steps in the HFMEA mentioned above except the decision tree.

The main objective of this study was to investigate what kind of impact the HFMEA had on the organization in 1 county council in Sweden and the method of working for multidisciplinary teams performing HFMEA. There were 3 main outcome measures: the quality of the documentation from the HFMEA, fulfillment of the primary goal of the HFMEA, and whether proposed actions were implemented.

METHODS

The present study was performed at the County Council of Östergötland, Sweden. Swedish health care is publicly funded, and hospitals and primary health care are provided by county councils. Sweden has 9 million inhabitants, and Östergötland has nearly 420,000 inhabitants. There are 3 hospitals, 1 of which is a university hospital, and 41 primary health care centers. Almost 12,000 people are employed by the county council. The health care organization in the county council is divided into 15 divisions.

The HFMEA was implemented in 2006 as part of a comprehensive patient safety program initiated in 2000 in the county council. An HFMEA is ordered by the head of the county council, a division or a department. A multidisciplinary team is formed with an assigned team leader. After performing a conventional HFMEA, the team completed a documentation containing a 6- to 10-page summary together with a table outlining the failure modes, underlying causes, and recommended actions for improvement. For both the documentation and the table, a template was used to obtain a certain degree of uniformity.

Of all HFMEAs (n = 156) registered between years 2006 and 2010, a total of 39 HFMEAs had to be excluded because of insufficient documentation. Thus, 117 HFMEAs were finally included in the study.

The study consisted of 3 steps:

  • First, the research group studied the documentation of each performed HFMEA (n = 117).
  • Second, the team leaders (n = 47) of the HFMEAs were interviewed (n = 17) or answered a questionnaire (n = 30).
  • Third, the managers (n = 51) who had ordered the HFMEA either were interviewed (n = 20) or answered a questionnaire (n = 31).

The purpose of the first and second steps was to evaluate how the method of HFMEA was used and the quality of the work in the teams. The purposes of the third step were to elucidate the motives for the HFMEAs and to evaluate the outcomes, fulfillment of the primary goal of the HFMEA, and whether proposed actions were implemented.

For the first step, a protocol was designed by the research group. The structured protocol contained 26 variables for the initiation of the HFMEA, the work within the team, the characteristics of the failure modes, their causes, and the proposed actions for improvement. The authors then split into 2 groups to study the documentation for the HFMEA. The causes of the identified failure modes were classified into 5 areas (communication and information, education and competence, work environment, equipment and technique, and, finally, routines and procedures). The proposed actions for improvement were classified whether they could be judged as concrete (could be measured and followed up). The documentation of the HFMEA was also classified according to quality aspects, primarily whether the proposed actions were concrete; completeness of the documentation; and precision of the language. After a pilot test, when the first 8 cases were studied by all 4 authors to ascertain that the variables were judged consistently, the protocol went through minor changes.

In the second and third steps, the respondents (team leaders and managers) were invited to either attend an interview or complete a questionnaire. The team leaders and managers involved in 2 or more HFMEAs were interviewed; the remaining received a questionnaire by e-mail with 2 reminders. The team leaders answered questions about (1) the HFMEA planning process as well as (2) the process performance of the HFMEA and particularly the efficiency of the work in the teams. To evaluate what kind of impact the HFMEA had on the organization, the managers answered (1) whether they considered that their initial goal with the HFMEA had been fulfilled and (2) what proportion of the proposed actions had been implemented. The interviews were semistructured, and 7 and 8 variables were registered from the team leaders and the managers, respectively. The responses for the variables were coded and recorded for analysis in an Excel file.

Two authors were involved in each interview, 1 leading the interview and the other documenting the answers. The interviews took place during the first half of 2012. This means that the time delay since an HFMEA was carried out ranged from 1.5 to 6 years.

The data obtained were registered in an Excel file, and the information was processed with the help of the “Data Analysis package” of the same application. The outcome variables for proposed actions underwent multiple regression analysis by means of the SPSS package to identify possible underlying causes. Variables from all 3 parts are shown in Table 1.

TABLE 1
TABLE 1:
Variables and Values Used in Analysis of Documentation and Interviews and Questionnaire

RESULTS

The review of the 117 HFMEAs showed that the motive was a proposed organizational change in 81 cases (70%). This included change in organizational structure, routines, or equipment. The remainder of the HFMEAs was performed as a consequence of the registration of a great number of adverse events in a unit or because of a general feeling of insecurity.

When the managers of the units were asked for their opinion on the most important motive for the HFMEA, they admitted that 1 motive was to convince the staff that the change could be implemented without danger to patients. In 8 cases (10%), this was the main motive. Of the 117 HFMEAs, 69 (59%) dealt with various aspects of medical treatment. Laboratory processes or information/documentation was studied in 16 cases (14 %) of the HFMEAs. The problem studied was confined to 1 department in 80 cases (68%). More than 1 department was involved in 24 HFMEAs (21%), and in 13 (11%), the problem was related to the whole county council. An example of the latter was the implementation of computerized medical records.

Method of Working in the Teams

The HFMEAs consumed 70 hours (median of man-hours) during a median of 5 weeks. The mean cost of an HFMEA was estimated to be €1909. The analysis teams were led by 53 different team leaders. Their education and experience varied to a large extent. Sixty-six HFMEAs (56%) were performed by team leaders with a solid education (more extensive university courses corresponding to 7.5 credits); and the remainder, by team leaders who had participated in a 2-day course. Forty-seven (40%) HFMEAs were performed by 6 team leaders known to the authors as very experienced. The number of participants in the teams varied between 2 and 20; the median was 8 participants. Registered nurses dominated the teams, constituting 40% of all members. Physicians participated in the teams in 79% of the HFMEAs.

The teams identified many failure modes in each HFMEA, from a few up to more than 100 (median of 29 failure modes); 33% were classified as serious. The causes of the failure modes were almost evenly distributed among the areas defined. Only “routines and procedures” were more common.

The documentations of HFMEAs were generally of high quality; only 19 were judged defective by the research team. It seems that a solid education and extensive experience improved the documentation (χ2 test, P < 0.01; Table 2).

TABLE 2
TABLE 2:
Quality of Documentation in Relation to the Education Level and Experience of the Team Leaders

Teams with more than 10 participants were considered to be less efficient than smaller teams as judged by the team leaders (χ2, p < 0.05; Table 3). The work in the teams was disturbed for various reasons in 30% of cases. The most common causes for these disturbances were opposition to the proposed change and incomplete attendance of members with key roles.

TABLE 3
TABLE 3:
Influence of Team Size on Efficiency in Team Work

Implementation of Proposed Actions for Improvement

The number of proposed actions in an HFMEA varied between 0 and more than 100 (median of 10). The proposed actions were considered concrete in 69% of HFMEAs. In 78% of HFMEAs, a large part or all proposed actions were considered to have been implemented (Table 4).

TABLE 4
TABLE 4:
Proportion of Proposed Actions That Were Implemented

Factors enhancing implementation were looked for, focusing on the quality of the reports, the experience of the team leaders, and the presence of physicians and how concrete the proposed measures were. However, none attained significance in the regression analysis (P = 0.14 at best).

DISCUSSION

The main findings in the present study were that many proposals for improvements were implemented to a substantial degree. The quality of the reports was high, and it correlated with the education and experience of the leader of the teams. It indicates that the HFMEA is a well-accepted tool of research in patient safety work for an organization and that the analyzing teams have the required skills. On the other hand, we also found that some units did not follow up the proposals for improvement. In addition, it was not possible to identify factors promoting the implementation of measures for risk improvement.

The study design was retrospective, and 2 important outcome measurements emanated from the interviews with the managers. Thus, we did not follow up the proposed actions for improvement ourselves. This drawback was discussed when the study was planned, and the option to follow up all proposed actions by ourselves was not considered feasible. It could also be questioned whether the managers were able to recall the necessary details some years afterward. However, most of the analyses dealt with a change in the organization. Such are not so common, which also arouse a lot of emotion in the staff, and therefore, it is reasonable to think that the analyses were well remembered by the managers. In summary, we consider our conclusions reasonably reliable.

More than two-thirds of the analyses involved a proposed change in the organization, and consequently, the treatment processes already in use were seldom questioned. Considering the number of adverse events normally registered in hospitals, this was unexpected. We can only speculate on possible reasons. Perhaps root cause analysis (RCA) was considered sufficient or the organization may have used the incident reporting system to make aggregated analyses of common causes. Moreover, it may be difficult to enroll ordinary physicians and nurses into the HFMEA teams because of their primary duties, considering that HFMEAs are more time consuming than are RCAs. Thus, it may be understandable that organizational change was given priority. In addition, the estimated cost for HFMEAs was twice as high as the cost of RCAs in our organization. Therefore, it may be argued that the balance between organizational change and processes in actual use was economically efficient.

The policy in our organization was to educate a number of persons in this modified HFMEA method. However, this study shows that only a few of those gained thorough experience, and therefore, we now consider it sufficient to have only 10 to 15 well-trained individuals in an organization of our size. This presupposes that they are given enough time and others are given resources to fulfill their task.

A well-written report is often mandatory for the project. However, in this study, it was not possible to demonstrate a correlation between the quality of the reports and the outcome measurements used. In our organization, the report is accompanied by an oral presentation. Perhaps the managers considered the oral report to be sufficient for making necessary decisions and then handed over the report to a subordinate, who was assigned to implement the risk improvement measures. Writing reports takes several hours, which could perhaps be spent more efficiently. Therefore, we believe that the written report should be simplified.

The ultimate question that has to be answered is whether the risk improvement measures that were implemented really influenced patient safety. To our knowledge, there are no such studies in the medical literature, and such studies would greatly support the decision on when to use the HFMEA. Although the HFMEA method is considered a useful tool by many, including the authors of the present study, its effectiveness has to be challenged.11,12 There is a need for further studies on its cost-effectiveness.

CONCLUSIONS

Most of the proposed actions were implemented. The use of the HFMEA can be improved using fewer team leaders but with more experience. The work involved in writing a report can be reduced without loss of impact on the organization.

REFERENCES

1. DeRosier J, Stalhandske E, Bagian JP, et al. Using health care Failure Mode and Effect Analysis: the VA National Center for Patient Safety's prospective risk analysis system. Jt Comm J Qual Improv. 2002;28:248–267, 209.
2. Ashley L, Armitage G, Neary M, et al. A practical guide to failure mode and effects analysis in health care: making the most of the team and its meetings. Jt Comm J Qual Patient Saf. 2010;36:3518.
3. Ashley L, Armitage G. Failure mode and effects: an empirical comparison of failure mode scoring procedures. J Patient Saf. 2010;6:210–215.
4. Abujudeh HH, Kaewlai R. Radiology failure mode and effect analysis: what is it? Radiology. 2009;252:544–550.
5. Thornton E, Brook OR, Mendiratta-Lala M, et al. Application of failure mode and effect analysis in a radiology. Radiographics. 2011;31:281–293.
6. Nickerson T, Jenkins M, Greenall J. Using ISMP Canada's framework for failure mode and effects analysis: a tale of two FMEAs. Healthc Q. 2008;11:40–46.
7. van Tilburg CM, Leistikow IP, Rademaker CM, et al. Health care failure mode and effect analysis: a useful proactive risk analysis in a pediatric oncology ward. Qual Saf Health Care. 2006;15:58–63.
8. Habraken MMP, Van der Schaaf TW, Leistikow IP, et al. Prospective risk analysis of health care processes: a systematic evaluation of the use of HFMEA in Dutch health care. Ergonomics. 2009;52:809–819.
9. Vlayen A. Evaluation of time- and cost-saving modifications of HFMEA: an experimental approach in radiotherapy. J Patient Saf. 2011;7:165–168.
10. Socialstyrelsen, Swedish Association for Local Authorities and Regions. Riskanalys & Händelseanalys—Handbok för patientsäkerhetsarbete [Risk Analysis and Event Analysis—Handbook for Patient Safety Work]. 2nd ed. Stockholm, Sweden: Socialstyrelsen; 2009.
11. Franklin BD, Shebl NA, Barber N. Failure mode and effect analysis: too little for too much? Qual Saf Health Care. 2012;21:607–611.
12. Shebl NA, Franklin BD, Barber N. Failure mode and effect analysis outputs: are they valid? BMC Health Serv Res. 2012;12:150–156.
Keywords:

HFMEA; evaluation; implementation; patient safety

Copyright © 2018 Wolters Kluwer Health, Inc. All rights reserved