Share this article on:

An Exploration of Contributing Factors to Patient Safety

Zadvinskis, Inga, M., PhD, RN; Salsberry, Pamela, J., PhD, RN, FAAN; Chipps, Esther, M., PhD, RN, NEA-BC; Patterson, Emily, S., PhD; Szalacha, Laura, A., PhD; Crea, Kathryn, A., PharmD, BCPS, CPPS

doi: 10.1097/NCQ.0000000000000284
Articles

One in 3 patients is estimated to experience health care–related harm during hospitalization. This descriptive, cross-sectional study used the Safety Attitudes Questionnaire to measure interprofessional staff perceptions of safety and teamwork climate and a retrospective, modified Global Trigger Tool chart review methodology to measure unit-level patient outcomes. Safety climate and teamwork did not have a statistically significant relationship with the frequency of adverse events identified by the Global Trigger Tool. Researchers may consider the Global Trigger Tool for detecting unit-level adverse events.

Nursing Excellence, OhioHealth, Columbus (Dr Zadvinskis); Health Behavior and Health Promotion, College of Public Health, The Ohio State University, Columbus (Dr Salsberry); The Ohio State Wexner Medical Center and The Ohio State University College of Nursing, Columbus (Dr Chipps); School of Health and Rehabilitation Sciences, College of Medicine, The Ohio State University, Columbus (Dr Patterson); Research Methods and Statistics, College of Nursing, The University of Arizona, Tucson (Dr Szalacha); and Department of Quality, Accreditation and Patient Safety, OhioHealth Riverside Methodist Hospital, Columbus (Dr Crea).

Correspondence: Inga M. Zadvinskis, PhD, RN, OhioHealth Riverside Methodist Hospital, 3535 Olentangy River Rd, Ste 205 North Medical Bldg, Columbus, OH 43214 (Inga.Zadvinskis@ohiohealth.com).

The authors declare no conflicts of interest.

Accepted for publication: June 21, 2017

Published ahead of print: August 24, 2017

EVIDENCE suggests that 1 in 3 patients is harmed by an adverse event (AE) during a hospital admission,1 costing $17 to $29 billion per year in health care expenses, lost worker productivity, and disability.2 Although patient harm occurs frequently with significant costs, morbidity, and mortality, translating evidence-based safety interventions into practice remains slow.3 Lack of progress in preventing harm may be partially explained by lack of an organizational culture of safety.4

Back to Top | Article Outline

MEASURING SAFETY CLIMATE, TEAMWORK CLIMATE, AND AEs

Adverse events with patient harm continue to occur during hospitalization. Organizations promote safety culture and teamwork as 2 defenses to reduce risk of AEs. Safety culture includes acknowledgement of high-risk activities, a blame-free environment, collaboration across disciplines to seek solutions, and organizational commitment of resources to address safety concerns.5 Disagreement exists among researchers regarding whether safety culture differs from safety climate,6 but safety culture represents a more stable characteristic of an organization, such as its “personality,” while climate represents a more visible mood state at a point in time.7

There are several reliable and valid safety climate surveys available, such as the Agency for Health care Research and Quality (AHRQ) Hospital Survey on Patient Safety Culture, the Patient Safety Climate in Health care Organizations, and the Safety Attitudes Questionnaire (SAQ).8 Although the US Centers for Medicare & Medicaid Services (CMS) does not require safety climate measurement, the CMS aims to make health care safer by supporting safety climate to prevent health care–related harm.9 In 2016, 680 US hospitals voluntarily submitted AHRQ Hospital Survey on Patient Safety Culture data to a comparative database.10

The relationship between safety climate and patient outcomes is dependent on the outcome measured. In a 2015 systematic review, 16 of 17 studies reported significant relationships between safety climate and outcomes such as decreased mortality, medication errors, the AHRQ Patient Safety Indicators (PSI) composite score, and nurse-sensitive PSIs.11 On the contrary, a meta-analysis found no significant relationships between safety climate and pressure ulcers, falls, medication errors, nurse-sensitive outcomes, and postoperative outcomes.12

Teamwork is another organizational defense used to reduce risk of AEs. Teamwork is “the perceived quality of collaboration between personnel.”13 (p3) Teamwork climate provides a way to measure surface features of attitudes about working together from diverse professionals with unique work experience at a certain time point, similar to safety climate.6 , 14 The relationship between teamwork and AEs is supported in literature. Hospitals with better teamwork tend to have lower rates of PSIs (standardized regression coefficient: 0-0.31, moderate size effect),15 AEs, patient falls,16 , 17 catheter-associated urinary tract infections,17 and odds of developing a pressure ulcer.18 Conversely, one study did not demonstrate a significant relationship between teamwork and number of reported safety events.19

Understanding and interpreting the relationships between safety climate, teamwork, and patient outcomes is complex due to measurement issues and variation in perceptions across staff groups.20 Researchers can measure safety climate using a variety of surveys. Similarly, patient outcomes such as AEs can be measured using a variety of approaches, such as voluntary reporting, administrative data analysis, and chart review. Comparing results across studies that use different measures of safety climate and patient outcomes can be challenging. Voluntary self-report can also influence AE frequency. Practitioners tend to underreport errors because of fear, the attitude of administration, system barriers, and staff perceptions of error.21 Since assessment of safety climate is not mandatory, hospitals that choose to measure safety climate may differ from a hospital population that does not assess safety climate.15

In 2009, the Institute for Healthcare Improvement (IHI) published the Global Trigger Tool (GTT) for Measuring Adverse Events,22 and it has become the most widely used global patient safety measure.23 The GTT measures AEs with harm resulting from health care that requires additional monitoring, treatment, hospitalization, or death.22 Reviewers scan the health record for triggers that may or may not indicate that an AE has occurred. For example, if a reviewer finds the trigger naloxone (Narcan), it likely represents an AE (unless drug abuse or self-inflicted overdose is involved).22 The GTT represents significant progress in AE detection because it finds 10 times more confirmed, serious events than voluntary reporting and AHRQ PSIs.1 Although experts designed the GTT to measure AE rates for an entire hospital, some researchers have applied it to the nursing unit or department level. Research in Norway indicates a statistically significant correlation with unit-level teamwork and GTT-identified AE occurrence, but the relationship between safety climate and AEs was not statistically significant, possibly due to small sample size (n = 4 units).24

Back to Top | Article Outline

Specific aims

The purpose of this study was to explore relationships between organizational (safety climate) and human factors (teamwork) with AEs that resulted in patient harm detected using an IHI (GTT)-modified trigger tool methodology. The aims of this study were to (1) describe self-reported safety climate and teamwork climate among an interprofessional group of providers working on 32 hospital inpatient units, measured using the SAQ; (2) explore the nature of AEs identified using an IHI (GTT)-modified trigger tool methodology; and (3) examine to what extent unit-level safety climate and teamwork climate predict AEs as detected via an IHI (GTT)-modified trigger tool methodology chart review format.

Back to Top | Article Outline

METHODS

This study used a descriptive, nonexperimental, cross-sectional design, with a retrospective chart review methodology. The study population was inpatient, nursing units/departments that served adult patients, with unit size (number of beds) ranging from 5 to 86. The convenience sample comprised 32 nursing units/departments from one 750+-bed Midwestern US regional, acute care, Magnet teaching hospital that employed more than 1000 nurses.

Units were required to have a response rate of 40% or greater on the SAQ to be included. Safety Attitudes Questionnaire respondents included nurses, nursing assistants, therapists (respiratory, physical, occupational, etc.), radiology technologists, pharmacists, technicians (pharmacy and operating room), physicians, and “other” clinical personnel.

Patient outcome data were collected from individual patient records to determine the occurrence of AEs. The GTT inclusion criteria were adapted with specifications unique to the proposal. The inclusion criteria were (1) patient age greater than 18 years, (2) length of stay at least 24 hours and admitted to the hospital on January 1-31, 2013, (3) no psychiatric or addictive disease admission or admission to a rehabilitation program, and (4) closed and completed medical record, with completed discharge and coding summaries. Safety Attitudes Questionnaire administration occurred between February 11, 2013, and March 15, 2013, and by selecting records from the preceding month, the team avoided the potential effect of SAQ survey administration on AE prevalence. A random selection process (SAS code) selected 10 medical record numbers for review for each unit/department. The patient's unit was selected on the basis of the first place (unit) the patient went after leaving the emergency department or immediately after surgery. For patients who moved back and forth between multiple units, the reviewer determined the date of transfer on the basis of provider orders and focused on the selected unit's care.

Back to Top | Article Outline

Variables

This study was designed to determine unit-level differences in contributing factors to patient outcomes. The independent variables were safety climate and teamwork climate, and the dependent variable was frequency of AEs, measured at the nursing unit/department level. Potential confounding variables were unit characteristics, size (number of patient beds/bays), and unit type (critical care, intermediate care, medical surgical, obstetrics/gynecology, and procedural departments).

Back to Top | Article Outline

Measures

Safety Attitudes Questionnaire

The SAQ is a 33-item survey with established validity13 and reliability (Cronbach α = 0.68-0.81,25 Raykov ρ coefficient = 0.90)13 and association with outcomes. Two domains of the SAQ are safety climate and teamwork climate. Both were measured with 1 to 5 Likert scale, ranging from 1 strongly disagree to 5 strongly agree. Safety climate comprised 7 items. Respondents indicate their level of agreement with statements, such as, “I would feel safe being treated here as a patient.” Teamwork climate was assessed with 6 items, such as, “Nurse input is well received in this setting.” Safety climate and teamwork climate were measured at the unit/department level via positive percentage agreement, which reflects the percentage of respondents who on average responded positively to the items in that domain. Percent agreement scores range from 0% to 100%. Previous researchers have proposed a minimum threshold scale of 60% positive percent agreement to indicate good teamwork and safety climate over time.26

Back to Top | Article Outline

GTT for measuring adverse events

The GTT is becoming more common for measuring AE rates, and it frequently detects higher rates of AE than other methods.27 Global Trigger Tool chart reviewers do not attempt to determine AE preventability, as per IHI GTT recommendations. Rather, users of the GTT record all events that are unintentional effects of medical care, regardless of perceived preventability. Harms that appear to be nonpreventable today may become preventable in the future28 due to advancements in technology and procedures. Conclusions regarding AE rates may not be drawn until 12 data points are available.22

Researchers have evaluated the ability of the GTT to detect the incidence of AEs compared with AHRQ PSIs and voluntary sentinel event reporting. The GTT had a sensitivity to detect patients with an AE of 94.9% and a specificity to detect patients with no AE of 100%,1 whereas the PSI method had a sensitivity of 8.5% and specificity of 98.5%, and a local hospital's voluntary reporting system had a sensitivity of 0% and a specificity of 100%. Because researchers compared the GTT with existing measures and demonstrated its ability to detect AE as well (if not better) than existing measures, the GTT has demonstrated adequate criterion-related validity. Global Trigger Tool reliability may be affected by variation in harm definition,29 reviewer training,30 and number of reviewers.31 Studies have established substantial agreement between GTT reviewers (κ = 0.61-0.80)32 , 33 and 78% agreement between analysts for harm identification.34

Global Trigger Tool modifications were necessary to assess unit-level AEs. Because patients may stay in several nursing units during a hospital admission, determining the unit that corresponded with the AE was challenging. When patients were moved back and forth between multiple units, the reviewer determined the date of transfer on the basis of provider orders to assist with AE onset. As an example, for a patient transferred from the intensive care unit to an intermediate unit, if the record for review was selected to represent the intermediate unit, the reviewer focused on the sections for triggers on the basis of care provided in that unit, not the intensive care unit.

Back to Top | Article Outline

Procedures

The SAQ was administered to 2076 staff members via e-mail, with 2 e-mail reminders to respond. The SAQ provided safety and teamwork climate data, with climate scores aggregated to the unit level. The primary reviewer reviewed all 317 randomly selected electronic health records, and the secondary reviewer reviewed 32 records to detect AEs using GTT methodology. The university and hospital data collection's site institutional review boards reviewed and approved the study.

Back to Top | Article Outline

Methods for assessing accuracy of the data

To determine interrater reliability, the team compared agreement between the primary reviewer and the secondary reviewer regarding presence of AEs. The interrater reliability sample comprised a random selection of 32 charts (10% of the intended overall sample size of 320). The observed percentage agreement regarding presence of an AE was 93.9%. The Cohen κ score was 0.835, which may be interpreted as almost perfect agreement.35 The primary record reviewer reexamined all AEs at the end of data collection to verify data accuracy, because the reviewer's skills may have improved over the 4-month data collection period. Both reviewers discussed 23 complicated cases of AEs to verify agreement prior to data analysis.

Back to Top | Article Outline

Analysis

Data analysis required descriptive statistics and bivariate tests: independent t tests, analysis of variance, and Pearson correlations. Adverse events per 1000 patient-days were calculated by summing the number of events for the unit, divided by the total length of stay × 1000. Adverse events per 100 admissions were calculated by summing the number of events for the unit, divided by the number of records reviewed × 100. A series of hierarchical regression models was fit with AEs per 1000 patient-days as the outcome and dichotomized teamwork climate, safety climate, and a term interacting teamwork climate and safety climate, controlling for unit type. Because 8 units did not experience any AEs, an alternative hypothesis was developed to examine the likelihood that a unit would experience an AE. A logistic regression analysis was conducted to predict the likelihood that a unit would experience an AE, with 3 predictor variables: unit type, dichotomized teamwork climate, and safety climate. IBM SPSS Statistics version 22 (IBM Corp, Armonk, New York) served as the software program for data analyses. A PhD-prepared statistician reviewed all analyses.

Back to Top | Article Outline

RESULTS

The mean SAQ response rate was 62.75% (range: 40.22%-100%, SD = 14.37). This sample's response rate was consistent with benchmarking data collected from more than 1500 employees from 11 inpatient units in US hospitals.13 The emergency department response rate was 37.84%, which was below the 40% rate for inclusion, so it was excluded.

Back to Top | Article Outline

Unit/department characteristics

The mean unit size (number of beds per unit) was 28 beds (SD = 17.57) (Table). The sample of 32 units consisted of 3 intermediate care, 3 obstetrics/gynecology, 5 critical care, and 10 medical surgical units, and 11 procedural areas. Procedural areas included areas such as surgery, multiple cardiac departments, radiology, endoscopy, and inpatient dialysis. The reliability of safety and teamwork climates via Cronbach α was 0.82 and 0.80, respectively.

Table

Table

Back to Top | Article Outline

Safety and teamwork climates

The average teamwork and safety climate scores were 70.07% (SD = 0.19) and 75.61% (SD = 0.16). Safety climate was normally distributed whereas teamwork climate was negatively skewed. Unit types with the strongest safety climate were medical surgical units (M = 0.80, SD = 0.11), followed by intermediate care (M = 0.78, SD = 0.06) and critical care (M = 0.77, SD = 0.15). Unit types with the strongest teamwork climate were critical care (M = 0.77, SD = 0.17), intermediate care (M = 0.77, SD = 0.10), and medical surgical (M = 0.74, SD = 0.13).

Safety climate and teamwork climate were highly correlated with each other (r = 0.89, n = 32, P < .001). Neither low nor high teamwork units significantly differed in unit size (F = 0.36, df = 1,30, P = .554). Safety climate did not differ significantly by unit size (F = 70.84, df = 30,1, P = .094). Safety climate and dichotomized teamwork climate means did not statistically differ between medical surgical units and other types of units (t = 1.09, df = 30, P = .286; t = 4.28, df = 30, P = .672).

Back to Top | Article Outline

Adverse events

The mean patient age was 61.4 years (range: 19-96 years, SD = 19.04) and 182 (57.4%) of the patients were women, with a mean length of hospital stay of 4.1 days (range: 1-43 days; SD = 4.62). Sixty-nine AEs occurred per 1000 patient-days and 21 AEs occurred per 100 admissions. About 20% of admissions experienced an AE. Fifty-three AEs were associated with least severe harm category E (temporary harm that required intervention), and 16 events were associated with harm category F, which represented temporary harm to the patient who required initial or prolonged hospitalization.

The most frequently identified triggers were antiemetic administration, diphenhydramine (Benadryl) administration, and transfusion or use of blood products. To determine the positive predictive value of each trigger, the number of AEs identified with the trigger was divided by the number of triggers found in the patient record.36 For example, the reviewer found the diphenhydramine trigger 23 times, but only 3 doses were consistent with an AE. The diphenhydramine positive predictive value (PPV) was calculated as 3/23 = 0.13 (not very predictive). Six triggers had positive predictive values of greater than 75%: health care–associated infections (100%), injury, repair, or removal of organ (100%), oversedation/hypotension (100%), any procedure complication (90%), antiemetic use (84%), and any operative complication (75%).

One-quarter (25%) of the sample nursing units did not experience any AEs. Unit types with the greatest frequency of AEs per 1000 patient-days were medical surgical (M = 125.67, SD = 81.86), procedural (M = 58.93, SD = 79.63), and critical care (M = 38.57, SD = 9.97).

Back to Top | Article Outline

Safety and teamwork climates and adverse events

A series of hierarchical multiple regression models were fit to examine how predictive unit type, teamwork climate, safety climate, and a potential interaction between teamwork and safety climate were on the frequency of AEs (per 1000 patient-days). Thirty percent of the variance in AEs was predicted by unit type (R 2 = 0.30, F 4,27 = 2.84; P = .04) (Table). Neither teamwork climate nor safety climate was a significant predictor of AEs. Based on the logistic regression, none of the predictors (unit type, safety climate, teamwork climate) were significant in predicting the likelihood of an AE.

Back to Top | Article Outline

DISCUSSION

The purpose of this study was to explore relationships between safety climate and teamwork climate with AEs that resulted in patient harm, detected using a modified GTT methodology. Neither teamwork climate nor safety climate was a statistically significant predictor of AEs. The lack of relationship between safety climate/culture and patient safety outcomes (such as AEs pressure ulcers and falls) is consistent with a meta-analysis.12 Similarly, a systematic review found non-significant relationships between safety culture and patient outcomes in 4 of 17 studies.11 The most likely reason for lack of association is due to similar SAQ scores between units, limiting variation and ability to detect statistically significant differences. Respondents may have reported positive safety climate and teamwork for social desirability reasons.

A strength of this project was the application of the GTT for identifying unit-level AEs, in contrast to typical use for identifying hospital AE rates. Global Trigger Tool interrater reliability was strong. However, the small sample size limits generalizability and cross-sectional data limit ability to detect cause/effect and trends over time. The GTT was not designed for unit-level analyses, and modifications affect instrument validity and reliability.

Although medical surgical units experienced the strongest safety climate scores in this study, they also experienced the greatest frequency of AE. Medical surgical nurses may perceive that they work on a unit with positive safety climate, but their patients experience a greater number of AEs than patients admitted to other unit types. Hospital staff perceptions of safety climate may not accurately reflect patient outcomes such as AEs. A cross-sectional study found that a unit with the highest (best) self-assessed safety climate experienced more AEs than a unit with lower self-assessed safety climate.37 These unexpected findings raise questions regarding the validity and reliability of safety climate instruments.37

Patients admitted to medical surgical units are at higher risk for AEs, which provides guidance for focusing future interventions. Research has demonstrated that a lower number of patients per nurse was associated with significantly lower mortality.38 Improved nurse staffing and decreased workload may decrease frequency of AE in medical surgical units. Caring for patients with complex health problems increases nursing workload, which may also influence AEs. Previous research indicates that a high nursing workload increases risk of AEs.39

Back to Top | Article Outline

CONCLUSIONS

Adverse events cause significant patient morbidity, mortality, and increased health care costs. Because of alarming AE rates, hospitals promote safety culture and teamwork as defenses to reduce risk of patient harm, but this study demonstrated no statistically significant association between unit-level safety and teamwork climates with AEs. Unit type may be an important consideration for understanding safety incidents, as it predicted 30% of the variance in AEs. Medical surgical units experience more AEs than other unit types, which may be due to numerous contributing factors, such as nurse staffing. Researchers may consider using the GTT to detect unit-level AEs not detected via other methods. Future research and advancements in health information technology will contribute to GTT automation and might detect AEs in real time to improve patient safety.

Back to Top | Article Outline

REFERENCES

1. Classen DC, Resar R, Griffin F, et al “Global Trigger Tool” shows that adverse events in hospitals may be ten times greater than previously measured. Health Aff. 2011;30(4):581–588.
2. National Quality Forum. National voluntary consensus standards for patient safety: a consensus report. http://http://www.qualityforum.org/Publications/2012/03/National_Voluntary_Consensus_Standards_for_Patient_Safety__A_Consensus_Report.aspx. Published 2012. Accessed May 29, 2017.
3. Rutberg H, Borgstedt Risberg M, Sjödahl R, Nordqvist P, Valter L, Nilsson L. Characterisations of adverse events detected in a university hospital: a 4-year study using the Global Trigger Tool method. BMJ Open. 2014;4:e004879.
4. Chassin MR. Improving the quality of health care: what's taking so long? Health Aff. 2013;32(10):1761–1765.
5. Agency for Healthcare Research and Quality. Safety culture. Patient Safety Primer. https://psnet.ahrq.gov/primers/primer/5/safety-culture. Published 2016. Accessed May 27, 2017.
6. Halligan M, Zecevic A. Safety culture in healthcare: a review of concepts, dimensions, measures and progress. BMJ Qual Saf. 2011;20(4):338–343.
7. O'Connor P, Buttrey SE, O'Dea A, Kennedy Q. Identifying and addressing the limitations of safety climate surveys. J Safety Res. 2011;42:259–265.
8. Weaver SJ, Sydney D, Lubomski LH, Wilson R. Promoting a culture of safety. Making Health Care Safer II: An Updated Critical Analysis of the Evidence for Patient Safety Practices. Rockville, MD: Agency for Healthcare Research and Quality; https://archive.ahrq.gov/research/findings/evidence-based-reports/ptsafetyII-full.pdf. Published 2013. Accessed July 29, 2017.
9. U.S. Centers for Medicare & Medicaid Services. CMS quality strategy 2016. https://www.cms.gov/medicare/quality-initiatives-patient-assessment-instruments/qualityinitiativesgeninfo/cms-quality-strategy.html. Published 2016. Accessed February 17, 2017.
10. Agency for Healthcare Research and Quality. Hospital user comparative database reports. https://www.ahrq.gov/professionals/quality-patient-safety/patientsafetyculture/hospital/hosp-reports.html. Published 2016. Accessed February 17, 2017.
11. DiCuccio MH. The relationship between patient safety culture and patient safety outcomes: a systematic review. J Patient Saf. 2015;11(3):135–142.
12. Groves PS. The relationship between safety culture and patient outcomes: results from pilot meta-analyses. West J Nurs Res. 2014;36(1):66–83.
13. Sexton JB, Helmreich RL, Neilands TB, et al The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res. 2006;6:44.
14. Valentine MA, Nembhard IM, Edmondson AC. Measuring teamwork in health care settings: a review of survey instruments. Med Care. 2015;53(4):e16–e30.
15. Mardon R, Khanna K, Sorra J, Dyer N, Famolaro T. Exploring relationships between hospital patient safety culture and adverse events. J Patient Saf. 2010;6(4):226–232.
16. Brown DS, Wolosin R. Safety culture relationships with hospital nursing sensitive metrics. J Healthc Qual. 2013;35(4):61–74.
17. Rahn DJ. Transformational teamwork: exploring the impact of nursing teamwork on nurse-sensitive quality indicators. J Nurs Care Qual. 2016;31(3):262–268.
18. Taylor JA, Dominici F, Agnew J, Gerwin D, Morlock L, Miller MR. Do nurse and patient injuries share common antecedents? An analysis of associations with safety climate and working conditions. BMJ Qual Saf. 2012;21:101–111.
19. Malloy PA. Examining the Relationship Between Work Climate and Patient Safety Among Nurses in Acute Care Settings [dissertation]. ProQuest Dissertations Publishing, Ann Arbor, Michigan: Nursing, The University of Rhode Island; 2012.
20. Pannick S, Beveridge I, Wachter RM, Sevdalis N. Improving the quality and safety of care on the medical ward: a review and synthesis of the evidence bas. Eur J Intern Med. 2014;25(10):874–887.
21. Soydemir D, Seren Intepeler S, Mert H. Barriers to medical error reporting for physicians and nurses [published online ahead of print September 30, 2016]. West J Nurs Res. doi:10.1177/0193945916671934.
22. Griffin FA, Resar RK. IHI Global Trigger Tool for Measuring Adverse Events. IHI Innovation Series White Paper. 2nd ed. Cambridge, MA: Institute for Healthcare Improvement; 2009: http://http://www.ihi.org/resources/Pages/IHIWhitePapers/IHIGlobalTriggerToolWhitePaper.aspx. Accessed April 21, 2014.
23. Pronovost PJ, Wachter RM. Progress in patient safety: a glass fuller than it seems. Am J Med Qual. 2014;29(2):165–169.
24. Deilkas ET, Hofoss D. Psychometric properties of the Norwegian version of the Safety Attitudes Questionnaire (SAQ), generic version (short form 2006). BMC Health Serv Res. 2008;8(191):1–10.
25. Colla JB, Bracken AC, Kinney LM, Weeks WB. Measuring patient safety climate: a review of surveys. Qual Saf Health Care. 2005;14:364–366.
26. Paine LA, Rosenstein BJ, Sexton JB, Kent P, Holzmueller CG, Pronovost PJ. Republished paper: assessing and improving safety culture throughout an academic medical centre: a prospective cohort study. Postgrad Med. 2011;87:428–435.
27. Hibbert PD, Molloy CJ, Hooper TD, et al The application of the Global Trigger Tool: a systematic review. Int J Qual Health Care. 2016;28(6):1–10.
28. Parry G, Cline A, Goldmann D. Deciphering harm measurement. JAMA. 2012;307(20):2155–2156.
29. Deilkas ET. Imprecision regarding the Global Trigger Tool. BMJ Qual Saf. 2013;22:271.
30. Classen DC, Lloyd RC, Provost L, Griffin F, Resar RK. Development and evaluation of the Institute for Healthcare Improvement Global Trigger Tool. J Patient Saf. 2008;4(3):169–177.
31. Hanskamp-Sebregts M, Zegers M, Vincent C, VanGurp PJ, DeVet HCW, Wollersheim H. Measurement of patient safety: a systematic review of the reliability and validity of adverse event detection with record review. BMJ Open. 2016;6:e011078.
32. Hwang J-I, Chin HJ, Chang Y-S. Characteristics associated with the occurrence of adverse events: a retrospective medical record review using the Global Trigger Tool in a fully digitalized tertiary teaching hospital in Korea. J Eval Clin Pract. 2014;20(1):27–35.
33. Kirkendall ES, Kloppenborg E, Papp J, et al Measuring adverse events and levels of harm in pediatric inpatients with the global trigger tool. Pediatrics. 2012;130(5):e1206–e1214.
34. Garrett PR Jr, Sammer C, Nelson A, et al Developing and implementing a standardized process for global trigger tool application across a large health system. Jt Comm J Qual Patient Saf. 2013;39(7):292–297.
35. Viera AJ, Garrett JM. Understanding interobserver agreement: the Kappa statistic. Fam Med. 2005;37(5):360–364.
36. Carnevali L, Krug B, Amant F, et al Performance of the adverse drug event trigger tool and the Global Trigger Tool for identifying adverse drug events: experience in a Belgian hospital. Ann Pharmacother. 2013;47(11):1414–1419.
37. Farup PG. Are measurements of patient safety culture and adverse events valid and reliable? Results from a cross sectional study. BMC Health Serv Res. 2015;15:186.
38. Aiken LH, Sloane DM, Cimiotti JP, et al Implications of the California nurse staffing mandate for other states. Health Serv Res. 2010;45(4):904–921.
39. Kang J-H, Kim C-W, Lee S-Y. Nurse-perceived patient adverse events depend on nursing workload. Osong Public Health Res Perspect. 2016;7(1):56–62.
Keywords:

Global Trigger Tool; organizational culture; outcome assessment (health care); patient safety; safety climate

Copyright © 2018 Wolters Kluwer Health, Inc. All rights reserved