The need for the improvement of quality and safety in patient care has been essential since the debut of the publication Building a Safer Health System in 2000.1 More than 2 decades later, medical errors remain prevalent in the United States, becoming the third leading cause of death.2 Therefore, many regulatory and accrediting organizations have recognized medical errors to be an ongoing crisis that is preventable with quality improvement (QI) initiatives.3–6 Given nurses comprise a majority of the health care workforce with 3.8 million nurses,7 nurses are poised to engage in QI initiatives to improve care.5,8,9 Unfortunately, evidence shows that only 53% of frontline nurses are engaged in the QI process,10 with recent results from Tschannen et al11 reporting it to be less than 50%. One identified barrier related to nurse engagement includes nurses not possessing adequate QI knowledge12; thus, further exploration of nurses' knowledge of QI is warranted.
BACKGROUND
Findings from a recent integrative review identified several barriers to nurses participating in QI initiatives, including leadership, education and training, resource constraints, data, culture, and time.13 Knowledge of QI was a major identified barrier; however, there are few tools available that accurately measure QI knowledge. The majority of tools evaluate a nurse's perception and attitude toward QI but fail to objectively measure knowledge. For example, Kovner et al10 and Djukic et al12 focused on assessing a new nurse's education of QI using a scale that measured the nurse's perceived “preparation” of their QI knowledge. The only tool that directly evaluates knowledge, the Nursing Quality Improvement in Practice tool, includes only 7 knowledge items focused primarily on QI terminology.14 An objective tool is needed to better understand nursing knowledge of QI. With better understanding of nurses' knowledge of QI, strategies can be put into place to improve their engagement with QI.
QI knowledge is a key identified barrier to nurses engaging in QI initiatives; unfortunately, there are insufficient tools available to measure knowledge. The revised Quality Improvement Knowledge Application Tool (QIKAT-R) has been utilized for physicians.15 The QIKAT-R uses physician-based scenarios to assess knowledge and provides a standardized way to evaluate the knowledge through a rubric/rating scale aligned with 3 primary areas: (1) aim, (2) measure, and (3) change.15 Although studies to date have shown strong interrater reliability with the use of the QIKAT-R,16–18 nursing has not used this tool, as the current scenarios in the QIKAT-R are not within the scope of nursing practice. Therefore, the purpose of this study was to evaluate QI knowledge utilizing the QIKAT-R, with newly created nursing scenarios, among frontline nurses and nurse leaders.
METHODS
Study sample and implementation
Using a cross-sectional design, this study evaluated nurses' QI knowledge utilizing the QIKAT-R, with newly created nursing scenarios. All nurses employed within one Midwest health system who provided direct care were eligible for participation, including those working on inpatient units, ambulatory care clinics, emergency, procedural, and home health areas. This included staff nurses, advanced practice nurses (eg, clinical nurse specialist, nurse practitioner, and certified registered nurse anesthetist), and nurse leaders (eg, supervisors, managers, directors). Nurses employed in departments where direct care was not provided (eg, quality department, informatics department, human resources, coding) were excluded from the study. The institutional review board reviewed the study and deemed it exempt prior to study initiation.
Recruitment for this study included 2 approaches. The first approach recruited nurses from a larger study that was aimed at understanding current levels of QI competence (eg, knowledge, skills, and attitudes) and barriers/facilitators to engagement in QI among frontline nurses. Nurses participating in the focus groups for the larger study were included in this study. The second approach solicited frontline nurses employed on one unit via an email invitation introducing the study. Two email invitations were sent in hopes to increase participation.
Instrument
The QIKAT-R was used to measure QI knowledge among the frontline nurses. The QIKAT-R was originally developed to objectively evaluate QI knowledge among resident physicians.15 The tool consists of 3 questions the learner answers after reading a short-case scenario: (1) What would be the aim? (2) What would you measure to assess the situation? and (3) Identify one change that might be worth testing. A rubric is provided with the QIKAT-R, providing a standardized approach to evaluating the learner responses.
Since the tool had not been used for nurses prior to this study, several steps were taken to ensure that the QIKAT-R was consistent with nursing care and standards. Specifically, the SQUIRE website, which included case scenarios, case prompts, and a scoring rubric for the QIKAT-R, was reviewed to determine the relevance of the material to nursing. The case scenarios included, which were initially developed and tested with resident physicians, did not provide the needed details to align with the nursing scope of practice and workload. For this reason, 2 scenarios were developed in alignment with the QIKAT-R cases but with a greater focus on the nursing role in patient care. As noted in Table 1, the scenarios included an issue with communication among the team of a critical laboratory value in an inpatient setting and a breakdown in processes related to specimens in an outpatient oncology clinic. The cases provided enough information for the learner to identify gaps in practice and a subsequent need for change. The full scenarios are located in Supplemental Digital Content Table 1 (available at: https://links.lww.com/JNCQ/B47).
Table 1. -
Abbreviated QI Nursing Scenarios and Scoring Rubric
Inpatient Medical-Surgical Unit:
The nurse is working on a busy medical-surgical unit. One of the patients on this nurse's assignment is a 71 years old man admitted for exacerbation of heart failure. The patient has been receiving multiple medication changes to address his heart failure and elevated blood pressure. These medication changes affect his electrolyte levels, thus has orders for frequent blood tests. The nurse receives a very hurried report from the off-going nurse. Furthermore, when they are leaving from their shift, they forget to notify the oncoming nurse of the patient's recent potassium level. The next day the patient codes with the cause being hyperkalemia. |
Outpatient Oncology Clinic:
The APRN is working in a busy oncology clinic and agrees to see a “squeeze-in” patient who has acute myelogenous leukemia and started experiencing a temperature. After seeing the patient, the APRN orders STAT laboratory tests and a CXR and notifies the patient to wait in the office for the results so that outpatient treatment can be started immediately, per guidelines for neutropenic fever. The MA draws the blood and places it in the “transfer to lab” box. An hour later, the APRN notices the blood is still sitting in the box. When discussing the issue with the MA, the MA expresses, “Just a busy day ... that's what seems to happen.” This event has happened several times in the past. |
| Revised QIKAT Scoring Rubric (QIKAT-R) |
| Each item receives 1 point if the response adequately addresses the item and zero points if it does not. The total possible score is 9 points for each scenario.
|
3 points for the AIM. The AIM ...
-
A1: is focused on the system level of the problem presented.
-
A2: includes direction of change (increase or decrease).
-
A3: includes at least one specific characteristic such as magnitude (%change) or time frame.
|
3 points for the MEASURE. The MEASURE...
-
M1: is relevant to the aim.
-
M2: is readily available so data can be analyzed over time.
-
M3: captures a key process or outcome.
|
3 points for the CHANGE. The CHANGE...
-
C1: is linked directly with the aim.
-
C2: proposes to use existing resources.
-
C3: provides sufficient details to initiate a test of change.
|
Abbreviations: APRN, advanced practice registered nurse; CXR, chest radiograph; MA, medical assistant; QI, quality improvement; QIKAT-R, revised Quality Improvement Knowledge Application Tool.
To ensure the appropriateness of the scenarios and QIKAT-R tool for evaluating QI knowledge, a pilot test was done among senior level nursing students at one university to evaluate clarity in the cases and instructions. A total of 13 students in a Leadership and Management course were asked to review one of the 2 scenarios and subsequently answer the case prompts. Following this, students were asked to provide feedback on the case as it related to clarity, realism, and overall presentation. Feedback received was very positive as students reported the cases to be clear and realistic. Students also noted satisfaction with using case studies, such as these, for evaluating knowledge related to QI.
Study procedure
Data for this study were captured in 2 phases. The first phase included the frontline nurses participating in the focus groups for the larger study. At the beginning of the session, each nurse was given a case scenario based on their current place of employment; thus, nurses working inpatient areas were given the inpatient medical-surgical scenario and those working in the ambulatory care areas were given the outpatient oncology clinic scenario (Table 1). As in the initial testing of the tool, the nurses were asked to review the case and answer the 3 case prompts (eg, aim, measure, and change). Additional questions on the tool included years as a nurse, highest degree, and current work area (see Supplemental Digital Content Table 2, available at: https://links.lww.com/JNCQ/B48). Nurses were asked to turn in the completed QIKAT-R to the facilitator at the conclusion of the session.
The second phase of data collection included frontline nurses providing direct patient care on one unit within the health system. An email invitation with instructions for completion of the QIKAT-R was sent to all eligible nurses within the unit. The email, which was sent at 2 different time points, included an electronic link to the QIKAT-R.
Data analysis
All data were collected and prepared for study reviewers to code using the QIKAT-R coding rubric (Table 1). Prior to scoring the study data, interrater reliability was established among the research team (n = 4) using the nursing student responses collected during the pilot QIKAT-R testing in an effort to attain interrater reliability (IRR). Independent scores from each researcher were entered into an Excel file to determine interrater reliability. Scores were calculated for each aspect of the rubric such that total aim (0-3 points), total measure (0-3 points), total change (0-3 points), and total knowledge score (0-9 points) were computed (Table 1). Initial IRR was 72% (±1 point) among the 4 research team members. The team members met and discussed variations in scoring, which subsequently resulted in a few additions to the scoring rubric to aid in clarity. For example, A3 describes magnitude of change: clarifying words of always or never were added as being necessary to describe change.
Once reliability was obtained by the full research team, the decision was made to have 2 research team members review and score the study data using a similar process as stated earlier, with the opportunity to engage another research team member in the event there was disagreement. Initial IRR for the review between the 2 reviewers was 77%, with approximately 50% in total agreement. The 2 reviewers met to discuss differences in scores (eg, those that were more than 1 point different in scoring), coming to a consensus on each score component. After the review was scored, descriptive statistics using frequencies for categorical variables and mean, standard deviation, minimum, and maximum for continuous variables. Skewness and kurtosis were examined to assess normality. Correlational analyses were computed to determine if education or experience were confounding factors; no confounding variables were found. Mean scores for aim, measure, change, and total were compared using independent-samples t test to determine if there were differences between nurse leaders and frontline nurses. The significance level was set at P < .05 for all statistical analyses.
RESULTS
The researchers enrolled a total of 64 nurses from a large Midwestern teaching hospital, who were either frontline nurses (n = 44) or nurse leaders (n = 11). Nine participants did not identify their classification and some nurses did not answer all of the questions. Thus, the sample (n = 55) was used with the descriptive characteristics and when comparing the variable scores of the frontline nurses to the nurse leaders, whereas the total sample (n = 64) was used for the variable scores. In the study (n = 55), there was an average of almost 20 years' experience as a registered nurse. More than half had an undergraduate degree. Almost three-fourths of the nurses who responded to the survey worked in the inpatient area (see Supplemental Digital Content Table 2, available at: https://links.lww.com/JNCQ/B48). Correlational analyses showed a significant relationship between total knowledge score and educational level (rpb = 0.259, P = .039) but not between total knowledge score and years of experience (r = 0.063, P = .668).
Overall, QI knowledge scores were low (Table 2). The sample (n = 64) mean total knowledge score was 5.88 (SD = 1.990), with frontline nurses' total score being higher than nurse leaders; however, this did not reach the level of significance (P = .306). When reviewing the knowledge of nurses (n = 64) within the various QI categories (aim, measure, change), aligned with the QIKAT-R, overall mean scores for aim (M = 1.59, SD = 0.610) was lower than both measure (M = 2.23, SD = 1.137) and change (M = 2.08, SD = 1.044). When considering variations among roles, frontline nurses had higher average scores for aim and measure, but nurse leaders had higher average change scores. Only the average aim score demonstrated a significant difference between frontline nurses and nurse leaders (P = .008).
Table 2. -
QI Knowledge Scores Between Nurse Leaders and Frontline Nurses
a
| Variable |
Total Sample
(N = 64), M (SD) |
Nurse Leaders
(N = 11), M (SD) |
Frontline Nurses
(N = 44), M (SD) |
t-Statistic
(P Value) |
| Aim score |
1.59 (0.610) |
1.18 (0.405) |
1.73 (0.624) |
2.749 (0.008) |
| Measure score |
2.23 (1.137) |
2.18 (1.250) |
2.27 (1.107) |
0.237 (0.813) |
| Change score |
2.08 (1.044) |
2.45 (0.522) |
2.05 (1.056) |
−1.242 (0.220) |
| Total score |
5.88 (1.990) |
5.82 (1.328) |
6.01 (1.978) |
0.306 (0.761) |
Abbreviation: QI, quality improvement.
aEqual variances assumed. Nine participants did not indicate whether they were leader or frontline.
DISCUSSION
The purpose of this study was to evaluate QI knowledge utilizing the QIKAT-R, with newly created nursing scenarios, among frontline nurses and nurse leaders. Findings from this study revealed that overall QI knowledge among frontline nurses and nurse leaders was lower than expected. Specifically, QI knowledge among nurse leaders was lower than staff nurses, although not significant. Factors that could contribute to low overall knowledge in QI include education and experience level.
In this study, an experience level of almost 20 years could indicate that nursing education in QI was unavailable. Nurses educated prior to 2010 may not have received formalized training in QI. The American Association of Colleges of Nurses led a national effort to enhance education related to QI in nursing school curriculum in 2010.19 Since the sample for this study included highly experienced nurses (a mean of almost 20 years), their QI knowledge more likely developed through work experience, not formal academic study. Furthermore, nursing programs are still challenged in providing curriculum and instruction where graduates are prepared to substantially provide positive change to patient care, thereby improving outcomes.20 Therefore, shared experiences through multigenerational initiatives would support partnerships between newly graduated nurses, frontline nurses, and nurse leaders promoting shared learning experiences and increasing QI knowledge. Through the creation of this shared relationship, colleagues may address concerns and improve QI knowledge on a unit level, which has been shown to be successful in other disciplines. Specifically, Goodman et al21 demonstrated success with their multigenerational experiential learning collaborative, which included a learning collaboration between students, educators, and clinical QI mentors.
Although not significant, frontline nurses scored higher in the QIKAT-R than nurse leaders. This finding, along with work from other authors, supports the need for all nurses, at all levels, to gain QI knowledge. Using the QIKAT-R with revised nursing scenarios can assist in increasing QI knowledge within multigenerational nursing teams. Previous researchers identified time and workload constraints as barriers.13 Further study, using an integrative approach to include time and workload into a QI project, should be reviewed using the QIKAT-R instrument in a pre- and posttest study.
Limitations
One limitation of this study included a small sample size related to staffing issues from COVID-19 surges. Second, the study was conducted at one institution, limiting the generalizability of the results. Therefore, further research assessing QI knowledge should be explored in larger samples of frontline nurses, across multiple settings. Another potential limitation is the demographics of this particular sample. Nurses in this study had a high number of years of experience (almost 20 years average). This may have skewed the results; thus, further study is warranted.
Implications for nursing practice
Health systems aimed at improving quality need to focus on how to remove barriers to nurse engagement in QI, including the assurance that nurses have the knowledge needed to be a full member of the QI team. This will require some targeted strategies toward nurse engagement and education for both nurse leaders and staff nurses. Researchers may want to utilize the QIKAT-R, with the newly created nursing scenarios, to assess QI knowledge when developing and implementing the targeted strategies. Nurse leaders should have the opportunity to create QI teams that include multigenerational members with varying QI experience. Nurse leaders can support frontline nurses in QI initiatives by providing mentorship, fellowship opportunities, integrating QI discussion in a job description, finding ways to reduce the barriers to QI knowledge by addressing time and workload, and taking advantage of interprofessional education (IPE) opportunities. These IPE opportunities can have a dual purpose of providing knowledge in QI and engagement with other professionals who are struggling with QI.
CONCLUSIONS
The importance of evaluating nurses QI knowledge using the QIKAT-R requires further study. Nurses QI knowledge in practice is imperative for patient safety. Nursing has an opportunity to increase knowledge and engagement through targeted strategies aimed at frontline nursing staff and nurse leaders. Tools such as the QIKAT-R with newly created nursing scenarios can be used to more accurately measure QI knowledge.
REFERENCES
1. Kohn LT, Corrigan JM, Donaldson MS, eds. To Err Is Human: Building a Safer Health System. National Academy Press; 2000. Accessed January 3, 2018.
http://www.nap.edu/catalog/9728.html
2. Makary MA, Daniel M. Medical error—the third leading cause of death in the US. BMJ. 2016;353:21–39. doi:10.1136/bmj.i2139
3. The Joint Commission. National patient
safety goals effective January 1, 2016: hospital accreditation program. Published 2016. Accessed January 3, 2018.
https://www.jointcommission.org/standards/national-patient-safety-goals/
4. Centers for Medicare & Medicaid Services. Quality initiatives—general information. Published 2015. Accessed January 3, 2018.
https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityInitiativesGenInfo
5. Agency for Healthcare Research and Quality, AHRQ. Homepage. Accessed January 3, 2018.
http://www.ahrq.gov/
6. American Association of Colleges of Nursing. Hallmarks of the professional nursing practice environment. Published 2006. Accessed January 3, 2016.
https://www.aacnnursing.org/News-Information/Position-Statements-White-Papers/Hallmarks-Practice
7. American Association of Colleges of Nursing. News & information: nursing fact sheet. Updated 2019. Accessed January 3, 2020.
https://www.aacnnursing.org/news-Information/fact-sheets/nursing-fact-sheet
8. Institute for Healthcare Improvement. Transforming care at the bedside (TCAB). Published 2012. Accessed January 3, 2020.
http://www.ihi.org/Engage/Initiatives/Completed/TCAB/Pages/default.aspx
9. Djukic M, Kovner CT, Brewer CS, Fatehi FK, Bernstein I. Early-career registered nurses' participation in hospital quality improvement activities. J Nurs Care Qual. 2013;28(3):198–207. doi:10.1097/NCQ.0b013e31827c6c58
10. Kovner CT, Brewer CS, Yingrengreung S, Fairchild S. New nurses' views of quality improvement education. Jt Comm J Qual Patient Saf. 2010;36(1):29–35. doi:10.1016/S1553-7250(10)36006-5
11. Tschannen D, Alexander CC, Tovar EG, et al. Quality improvement engagement and competence: a comparison between frontline nurses and nurse leaders. Nurs Outlook. 2021;69(5):836–847. doi:10.1016/j.outlook.2021.02.008
12. Djukic M, Kovner CT, Brewer CS, Fatehi FK, Seltzer JR. A multi-state assessment of employer-sponsored quality improvement education for early-career registered nurses. J Contin Educ Nurs. 2013;44(1):12–19. doi:10.3928/00220124-20121115-68
13. Alexander CC, Tschannen D, Hays D, et al. An integrative review of the barriers and facilitators to nurse engagement in quality improvement in the clinical practice setting. J Nurs Care Qual. 2021;37(1):94–100. doi:10.1097/NCQ.0000000000000562
14. Tschannen D, Alexander C, Tovar EG, Ghosh B, Zellefrow C, Milner KA. Development of the nursing quality improvement in practice tool: advancing frontline nursing practice. J Nurs Care Qual. 2020;35(4):372–379. doi:10.1097/NCQ.0000000000000457
15. Singh MK, Ogrinc G, Cox KR, et al. The Quality Improvement Knowledge Application Tool Revised (QIKAT-R). Acad Med. 2014;89(10):1386–1391. doi:10.1097/ACM.0000000000000456
16. Kiger ME, Bertagnoli TA. A project-based, resident-led quality improvement curriculum within a pediatric continuity clinic. MedEdPORTAL. 2018;14:10738. doi:10.15766/mep_2374-8265.10738
17. Cioletti A, Sweidan S. A joint quality improvement and high-value care curriculum in a limited-resource setting. MedEdPORTAL. 2017;13:10545. doi:10.15766/mep_2374-8265.10545
18. Ogrinc G, Cohen ES, van Aalst R, et al. Clinical and educational outcomes of an integrated inpatient quality improvement curriculum for internal medicine residents. J Grad Med Educ. 2016;8(4):563–568. doi:10.4300/JGME-D-15-00412.1
19. American Association of Colleges of Nursing. Quality &
Safety Education for Nurses (QSEN). Accessed January 3, 2021.
https://www.aacnnursing.org/Quality-Safety-Education
20. Balakas K, Smith JR. Evidence-based practice and quality improvement in nursing education. J Perinat Neonatal Nurs. 2016;30(3):191–194. doi:10.1097/JPN.0000000000000197
21. Goodman CW, Justo J, Merrow C, Prest P, Ramsey E, Ray D. An experiential learning collaborative on quality improvement for interprofessional learners. J Interprof Care. 2022;36(2):327–330. doi:10.1080/13561820.2021.1901673
