Incorporating Patient Safety and Quality Into the Medical School Curriculum: An Assessment of Student Gains

Abstract

Background 

Global efforts are being made to improve health care standards and the quality of care provided. It has been shown through research that the introduction of patient safety (PS) and quality improvement (QI) concepts in the medical curriculum prepares medical students to face future challenges in their professional careers.

Purpose 

This study aimed to evaluate how a brief course on QI and PS affects the knowledge, efficacy, and system thinking of medical students.

Methods 

A 5-day QI and PS intervention course was implemented at the Aga Khan University medical college for 98 third-year medical students in March 2021. This weeklong course of lectures, interactive sessions, and hands-on skill workshops was conducted before the students began their clinical rotations. Students’ knowledge, self-efficacy, and system thinking were assessed with pretest and posttest. Students were also asked to write personal reflections and fill out a satisfaction survey at the end of the intervention.

Results 

Comparisons of pretest and posttest scores showed that the course significantly improved students’ knowledge by a mean of 2.92 points (95% confidence interval, 2.30–3.53; P < 0.001) and system thinking by 0.16 points (95% confidence interval, 0.03–0.29; P = 0.018) of the maximum scores of 20 and 5 points, respectively. The students’ self-assessment of PS knowledge also reflected statistically significant increases in all 9 domains (P < 0.001). Students reported positive experiences with this course in their personal reflections.

Conclusions 

The medical students exhibited increases in knowledge, self-efficacy, and system thinking after this weeklong intervention. The design of the course can be modified as needed and implemented at other institutions in low- and middle-income countries. A targeted long-term assessment of knowledge and attitudes is needed to fully evaluate the impact of this course.

Medical errors cause needless yet significant patient harm, are one of the leading causes of patient death, and can substantially increase health care costs.¹ According to the current literature, between 2.9% and 16.6% of patients suffer at least 1 inpatient adverse event, half of which are considered to be preventable.^2–5 According to a 2019 survey, an estimated 161,250 preventable deaths occur each year in U.S. hospitals.⁶ The adverse event rate is even higher in low- and middle-income countries (LMICs). For LMICs, the rate was reported to be between 2.5% and 18.4%,⁷ and 83% were considered to be preventable.⁷ Although the findings show a decline in numbers, limited progress has been made in reducing preventable harm despite increased efforts over the last decade.⁸ To improve patient safety (PS), a comprehensive approach targeting multidimensional areas is needed. A report from the Institute of Medicine highlighted the need for global emphasis on integrating PS and quality improvement (QI) principles into the practice of clinical medicine.¹

Adequate training in PS science is needed to help reduce preventable harm. Wong et al.⁹ shed light on the importance of inculcating PS from the earliest stage of medical education. Formal introduction of QI and PS to curricula would allow undergraduate and postgraduate trainees to develop PS competencies and promote a change in attitude and skills.¹⁰ In a systematic review, Kirkman et al.¹¹ showed that the number of PS educational interventions has been increasing. However, most of the studies came from high-income countries. In response to the global problem, the Institute for Healthcare Improvement launched an open school community that provides an online PS course curriculum. More than 700,000 residents and medical students across more than 85 countries have completed the course.¹² The World Health Organization also introduced a curriculum on PS for medical students¹³ that has been evaluated by nine countries so far.

In LMICs, health care delivery lacks financial resources, medical expertise, and advanced information technology, leading to subpar quality of care.¹⁴ Although some studies have identified knowledge gaps and documented awareness about the importance of PS and QI principles among health care providers in LMICs,^15–17 there is a dearth of literature about their existing knowledge of the subject. A multicenter study from India stressed the need for the development of a comprehensive training material and operational modules on PS to cater to the needs of health care professionals from various disciplines.¹⁸

There is no curriculum dedicated to QI and PS in Pakistan at present. Some concepts pertaining to these topics are taught at the discretion of the teaching institution, but they are spaced out and vary significantly within the country.

Although a few studies have assessed the level of awareness of medical school faculty and students about the importance of PS in Pakistan,^16,17 no formal PS education has been introduced. Here, we report on the implementation and evaluation of a 5-day PS and QI course that was taught to third-year medical students before the start of their clinical rotations. The course consisted of lectures, case studies, group discussions, and interactive activities. The aim of this course was to increase the students’ knowledge and competency in identifying key issues pertaining to QI and PS. Thus, this course can be used to design a curriculum on PS and QI that caters specifically to providers in Pakistan and South Asia at large.

METHODS

This study implemented and evaluated a course on PS tailored for LMICs. The impact of the course was assessed with a pre-post design. The bachelor of medicine, bachelor of surgery is a 5-year program for medical education in Pakistan that consists of 2 preclinical years followed by 3 years of clinical clerkships. All participants in our study were enrolled in year 3 of the bachelor of medicine, bachelor of surgery program at the Aga Khan University Medical College, a private medical college affiliated with a tertiary care teaching hospital. Two months before they began their first clinical rotation, the students attended a 5-day course that consisted of lectures, case-based interactive discussions and activities, and hands-on skill development workshops on QI and PS as part of the bench-to-bedside module. The bench-to-bedside module consists of multidisciplinary clinical and nonclinical teaching sessions to prepare third-year medical students for their first clinical rotations. Our course was modeled after a 3-day PS course for medical students taught at the Johns Hopkins University School of Medicine.¹⁹ Qualified QI and PS experts were consulted to review the course and its evaluations. They included the director and senior manager from the Center for Patient Safety at the Aga Khan University Hospital in Pakistan as well as the director of the Armstrong Institute for Engagement and Patient-Centered Innovations at the Johns Hopkins University. We spread the course for 5 days instead of 3 by dividing the students into 2 smaller batches for in-person sessions to ensure compliance with COVID-19–related standard operating procedures.

Goals and Objectives

The 3 main goals of the course were to teach students the following:

• 1) How medical errors occur, how health care professionals can learn from them, and ways to prevent the recurrence of medical errors at multiple levels
• 2) The knowledge and skills necessary to practice medicine safely both individually and within health care teams at large
• 3) How to improve system-based thinking as a way to improve PS and quality of care

Implementation

Strategies

The bench to bedside module offers opportunity for various subdivisions within the institute to teach innovative curriculum deviating from traditional clinical science to medical students during the 1 week of its 2-month duration. Previous attempts to implement a course on quality and PS had failed because of the inability to teach the entire batch of medical students simultaneously. Hence, this module offered a way past this obstacle for our team. Subsequently, our course was chosen by the selection committee for implementation. We identified contextualized key subject matters in PS and QI after consultation with local experts. This was followed by recruitment of local and international faculty speakers from various disciplines, including generalist and specialist physicians, health services researchers, nurses, health care safety experts, infection control practitioners, and other allied health care professionals. The topics of the lectures included (1) Science of Patient Safety, (2) Effective Communication, (3) Conflict Management, (4) Error Disclosure, (5) Learning from Defects, (6) Human and System Factors, and (7) Medical Record Documentation. On each of its 5 half days, the course started with a full class online lecture covering 1 of the key subject matters followed by two 60- and 90-minute breakout sessions by dividing the students in groups of 10. The speakers delivered the lecture using contextualized examples for the ease of their application in local settings. The breakout sessions were designed to equip students with knowledge and skills required to practice safely as individual health care providers and as integral members of the team. They included group discussion around case scenarios related to the topics covered in the lecture on the same day followed by presentations to the entire batch of students. In this manner, key take-home messages from each scenario were communicated to each student. The activities in the second half included role plays by the students on clinical scenarios pertaining to “learning from defects,” “communication skills,” and infection control measures such as appropriate methods for donning and doffing of personal protective equipment.

Evaluation

We evaluated this course using a precourse and postcourse knowledge test and survey. The research team designed the survey by using items and domains from the Health Professional Education in Patient Safety Survey,²⁰ the Systems Thinking Scale (STS),²¹ and other items in the Johns Hopkins PS course evaluation.¹⁹ The survey consisted of the following sections.

• 1) Demographics
• 2) Knowledge assessment: This section comprised 20 multiple choice questions. The assessment was developed with resources on PS from the Institute for Healthcare Improvement and the Agency for Healthcare Research and Quality.^22,23
• 3) Self-efficacy: Efficacy was assessed through 18 statements scored on a 5-point Likert scale that were grouped into 9 domains: (1) teamwork, (2) communication, (3) risk management, (4) human and environmental factors, (5) recognition and prevention of harm, (6) culture of safety, (7) infection control, (8) error disclosure, and (9) medical documentation. The tool was modified and adapted from the Health Professional Education in Patient Safety Survey²⁰ and the PS course at Johns Hopkins School of Medicine.¹⁹ An additional contextualized item designed by the research team was also included. The post-pre survey uses a unique approach in that the survey is administered only after the intervention (i.e., the PS course) is complete. Upon completion of the postcourse assessment survey, the students are asked to mark their preassessment ratings retrospectively. This method allows participants to use their current level of knowledge to create consistent measurement/ratings for the precourse and postcourse assessment.^24,25 With the post-pre approach, the students can ascertain the exposure required to rate the pretest survey. In our view, this method prevents overestimation of ratings on the pretest survey.^24,25
• 4) STS: Systems thinking is defined as the approach that analyses a problem within its system. It considers the subcomponents that interact with the problem or are affected by it, and as a whole form a process that achieves the goal of the system. We use a validated scale composed of 20 items²¹ scored on a 0 to 4 Likert-type scale, with composite scores ranging from 0 to 80 for this purpose.
• 5) Personal reflections: Students were asked to reflect on their experience of attending the course in 90 words. They were given a few points to brainstorm on, including the lessons learned from the course, personal experiences of adverse events, and novel ideas to prevent medical errors in the future. Subsequently, we analyzed the major themes qualitatively.
• 6) Student course evaluation: The students were asked to provide feedback on the course after the intercession using a standardized questionnaire provided by the department of undergraduate medical education.

The tool is available as a supplemental material, https://links.lww.com/JPS/A475.

Data Analysis

A descriptive analysis was conducted using means, medians, and 95% confidence intervals (CIs). Composite scores for precourse and postcourse assessments were calculated for the knowledge assessment, STS, and knowledge efficacy. The data from participants who did not attempt or complete the precourse assessments for each section before attempting the postcourse assessment were excluded as missing data during analysis. Differences in the scores for knowledge assessment and STS were compared with a paired-sample t test, and those for knowledge efficacy were compared with a Wilcoxon signed rank test. Students’ ratings on the 5-point Likert scale for knowledge efficacy were dichotomized into a binary variable. Adopting the approach laid down by Aboumatar et al.,¹⁹ we grouped “agree” and “strongly agree” into one category and all other responses into a second category for comparison. The precourse and postcourse composite scores for this binary variable were compared by using the McNemar test. The data were analyzed in Microsoft Excel (Redmond, Washington) and SPSS version 25 (IBM Corp., Armonk, NY).

Qualitative Analysis

Two independent reviewers assigned themes to all reflections using Microsoft Excel (Redmond, Washington). Assignments were compared, and any discrepancies were resolved via discussion in the presence of a senior reviewer. Mentions for each theme were then counted and tabulated.

Ethical Approval

Exemption was obtained from Aga Khan University’s Ethics and Review Committee for this study (2021-5976-16957).

RESULTS

A total of 98 students attended the module on PS held on March 22–27, 2021. Mean age of the students was 21.13 ± 0.8 years. None of the students had received prior training on PS or QI. The demographic information of the participants is summarized in Table 1.

TABLE 1 - Demographics of the Students Who Took the Course

Characteristic	Summary Statistics
Age, mean ± SD, y	21.13 ± 0.8
Sex, n (%)
Male	49 (52.7)
Female	44 (47.3)
English spoken, n (%)	91 (97.8)

Knowledge Assessment and System Thinking Scale

Table 2 represents mean preintervention and postintervention knowledge test and STS scores with 95% CIs on the difference. Of the 98 students, 83 (84.7%) took the knowledge assessment section and showed a mean increase of 2.92 points (22.12%) of a maximum of 20 points from pretest to posttest (95% CI, 2.30–3.53; P < 0.001). The pretest and posttest mean scores were 13.16 ± 2.76 and 16.07 ± 2.00, respectively. Similarly, 89 (90.8%) of the students completed the preintervention and postintervention STS. The mean scores increased significantly by 0.16 (4.17%) points (95% CI, 0.03–0.29; P = 0.018) from 3.85 ± 0.55 on the preintervention assessment to 4.00 ± 0.63 on the postintervention assessment of a maximum of 5 points.

TABLE 2 - Knowledge Assessment (n = 83) and STS Scores (n = 89)

Assessment	Pretest Score*	Posttest Score*	Mean Difference†	95% CI	P ‡
Knowledge (maximum, 20)	13.16 ± 2.76	16.07 ± 2.0	2.92 (22.12)	2.30–3.53	<0.001
STS (maximum, 5)	3.85 ± 0.55	4.00 ± 0.63	0.16 (4.16)	0.03–0.29	0.018

*Values for pretest and posttest scores are presented as mean ± SD.

^†Mean difference is presented as absolute and (%) difference.

^‡P values were calculated by paired-sample t test.

Self-Assessment of Knowledge

Table 3 shows the post- and post-pre self-efficacy ratings by the students. Of the 98 students, 88 (89.8%) completed both post- and post-pre assessment. The results showed a statistically significant (P < 0.001) increase in knowledge self-assessment ratings for all 9 domains. These participants scored particularly high in 3 domains: infection control (85.2%), teamwork (84.4%), and human and environmental factors (82.2%) compared with the other domains on postcourse assessment.

TABLE 3 - Student Self-Assessment of PS Knowledge (n = 88)

Domain	Pretest		Posttest		Comparison of Means
	Mean	Percent Agree/Strongly Agree	Mean	Percent Agree/Strongly Agree	z	P*
Teamwork	3.19	43.2	3.89	84.4	−5.551	<0.001
Effective communication	3.29	43.2	3.99	76.7	−4.625	<0.001
Risk management	3.10	35.2	3.86	68.9	−4.713	<0.001
Human and environmental factors	3.34	45.5	4.14	82.2	−5.43	<0.001
Recognition and reduction of harm	3.15	42	3.91	80	−6.223	<0.001
Culture of safety	3.33	43.2	4.07	78.9	−4.445	<0.001
Infection control	3.16	44.9	4.20	85.2	−5.634	<0.001
Error disclosure	2.73	29.2	3.65	55.7	−5.25	<0.001
Medical documentation	2.80	29.2	3.76	61.4	−5.10	<0.001

*Wilcoxon signed rank test.

Qualitative Analysis

Ninety-four students wrote personal reflections on the module. From these, we identified 5 overarching themes with 13 subthemes as listed hereinafter and depicted in Figure 1:

FIGURE 1: Common themes from personal reflections of the students.

• 1) Importance of PS: awareness regarding PS in developed countries, awareness regarding PS in developing countries, and general awareness regarding PS
• 2) Defect investigation: critical thinking and consequences of medical errors
• 3) System thinking: human factors and system factors
• 4) Effective communication: conflict management, teamwork, and standardized communication process
• 5) Skill development: hands-on experience, gearing up for clinical rotations, and using standardized tools

An overwhelming majority of the respondents mentioned the consequences of medical errors (n = 64 [68.1%]) and importance of PS (n = 71 [75.5%]). Similarly, many students talked about skill development (n = 75 [79.8%]), felt that the course prepared them for their upcoming clinical rotations, and thought that it helped them learn the use of standardized tools to avoid patient harm. Another common theme was effective communication (n = 71 [75.5%]) encompassing conflict management and the use of structured communication approaches such as SBAR (situation, background, assessment and recommendation) and ALEEN (anticipate, listen, empathize, explain and negotiate). Excerpts from some of the reflections are displayed in Figure 2.

FIGURE 2: Excerpts from personal reflections of the students. VLE, virtual learning experience, a portal for virtual learning content of the Aga Khan University.

Course Evaluation

The results of the course evaluation are summarized in Table 4. Participants showed greatest satisfaction with content delivery followed by gain in knowledge. The overall responses by the students showed a positive trend with high levels of satisfaction.

TABLE 4 - Course Evaluation by the Students (n = 95)

Question	Mean Score ± SD*
1. The session helped in gaining new knowledge.	4.13 ± 0.90
2. The session helped me in gaining new skill.	4.04 ± 0.93
3. The session helped in learning how to interact with the patients.	3.96 ± 0.89
4. There were ample opportunities for interaction with faculty during the session.	3.98 ± 1.02
5. I felt engaged during the session.	3.88 ± 1.02
6. I feel the online modality can be used for the teaching of this skill.	3.73 ± 1.08
7. The session was well planned for delivery of relevant skill.	3.91 ± 0.92
8. The facilitators were well prepared to deliver the content.	4.27 ± 0.75
9. I enjoyed peer to peer learning during the session.	3.96 ± 0.95
10. The learning resource uploaded on VLE helped in optimizing learning during the session	3.98 ± 0.88

*The means were calculated from the ratings on a Likert scale with a range of 1 to 5, where 1 represents strongly disagree; 2, disagree; 3, neutral; 4, agree; and 5, strongly agree.

VLE, virtual learning experience.

DISCUSSION

The introduction of QI and PS courses in medical school curricula is currently being evaluated across the globe.¹³ In this study, we show that a 5-day dedicated course on QI and PS in Pakistan led to significant increases in students’ PS knowledge, self-efficacy, and system thinking.

As in reports of other interventions, our students showed a significant increase in knowledge.^26–33 Most previous interventions focused on medical errors, safety culture, error disclosure, human and system factors, and effective communication.^9,30,34 Ours also included topics on learning from defects, conflict management, and medical record documentation. The gain in knowledge and positive response from the students can be attributed to the content of the modules and teaching methods. This course helped the students view PS as an integral part of their education and not just a soft science as perceived earlier.¹⁹ Every activity was preceded by a lecture on the respective topic to reinforce learning. Didactic approaches as such are proven to be effective at enhancing knowledge and attitude.^35,36 Keeping the sessions interactive optimized student engagement. Group discussions also promoted a better understanding of the subject. The students seemed to appreciate and enjoy these teaching strategies, as indicated in their personal reflections.

Our study revealed a significant increase in system thinking of the participants after the intervention. This finding is consistent with previously published reports, which have documented such positive changes in composite scores after interventions in the high-income countries.¹⁹ Although the Institute of Medicine’s report “To Err Is Human” attributes the largest proportion of medical errors to “system factors,” it is not uncommon to see health care professionals being exclusively blamed.^37,38 Current medical literature emphasizes the need for medical providers to focus on the system factors to detect pitfalls and develop system-based solutions for prevention.³⁹ Moreover, LMICs suffer from a combination of lack of accountability and rampant blame culture.^14,40 Hence, this finding reflects a potentially significant milestone in mitigating this problem in underresourced health care systems. The module succeeded in equipping the medical students with analytical skills to enhance system thinking. Furthermore, scheduling this workshop right before the commencement of clinical rotations should help to further reinforce system thinking during clinical rotations.

Despite the challenges of incorporating PS into the medical curriculum, interventional studies carried out in the past have shown that knowledge and self-efficacy increase after implementation of short courses on PS.^26–33 Our study had similar findings. Unlike many other studies that used a pre-post survey design, we used the post-pre survey.^4,41 Students reported on this survey a particularly high percentage of “agree/strongly disagree” responses in areas of infection control, teamwork, and human and environmental factors. It is important to note that we implemented this course during the COVID-19 pandemic. The current pandemic situation has once again accentuated the need to reinforce various PS components, including infection control and prevention and health care workforce safety.⁴² The pandemic has altered the perspective of all personnel in health care, including medical students, as evident by the highest percentage of positive responses being for infection control.

The personal reflections provided an insight into major takeaways from the intervention and included effective communication, the importance of PS, and consequences of medical errors. The standardized communication processes taught to students, such as the ALEEN and SBAR approach, may help increase the autonomy of students in improving the safety of their own patients during clinical clerkships. Such tools will help them engage in self-improvement efforts at an individual level and become a responsible member of the team. Several respondents remarked on the importance of their role as a part of the larger team delivering care. Individual efforts are very important in the context of LMICs, where the system factors cannot be controlled owing to a lack of resources and accountability.¹⁴ In fact, some studies have documented that the increased emphasis on system factors has minimized the importance of individual accountability.^38,39 Hence, these reflections illustrate that the objectives of the course were successfully achieved.

To our knowledge, our study is the first to introduce a PS and QI curriculum for medical students in Pakistan. Several barriers prevent most developing countries from progressing beyond the planning and consideration stage of the PS curriculum. These include lack of cooperation to address challenges of implementation, lack of expertise to enable implementation, and lack of commitment by the stakeholders, senior leadership, and government for implementation.⁴³ Our team faced several challenges while trying to implement this module. The first challenge was to introduce QI and PS as a science and change perceptions regarding its importance. One of our lessons learnt was regarding persistence in bringing this to fruition. Significant directed efforts over multiple years were needed the Center for Patient Safety team to get this workshop on the agenda for institutional leadership, and even then, it required a planned redesign of the overall curriculum to allow for it to occur with dedicated time over a whole week rather than dispersed over the entire 5 years. Second, the limited number of QI and PS experts in Pakistan translated into some initial difficulties in finding faculty. A valuable lesson learned was the feasibility of using clinical experts in ways they were not used to, such as a cardiologist for documentation errors given their role and personal experiences on relevant hospital committees. Moreover, with the increasing use of remote conferencing technologies, securing lectures from international subject matter experts also became feasible. However, the lack of local experts probably led to fewer facilitators for the interactive activities. We anticipate that such activities would have resulted in greater student engagement if conducted in smaller groups. Despite these challenges, we succeeded in implementing the course in our setting and hope that this pilot study will serve as a prototype that can be replicated across South Asia for teaching various health care professionals. Furthermore, effective implementation requires a shift from stand-alone modules and courses^9,44 to incorporating PS education in clinical settings.⁴⁵ Our study incorporated PS and QI teaching modules in a 2-month-long bench-to-bedside clinical teaching module for clinical year medical students. The bench-to-bedside module consists of multidisciplinary clinical and nonclinical teaching sessions to prepare third-year medical students for their first clinical rotations. We expect that the real-life clinical experiences and simulation scenarios will reinforce the concepts that we taught.

This study had some limitations worth noting. First, some of the interactive sessions had to be simulated virtually to ensure compliance with the standard operating procedures for COVID-19 control and created technical issues. Second, the pre-post study design is subject to social desirability bias. Third, as suggested in the students’ reflections, a long-term assessment of the concepts and skills taught must be conducted at the end of either year 3 or all clinical years to gain full insight into course effectiveness at improving PS. Perhaps, a refresher course toward the end of the clinical years will further solidify learned concepts. Lastly, this study was conducted at 1 medical school at an academic medical center. Other public sector institutions may lack the resources and expertise that were available to our institution.

CONCLUSIONS

We developed and conducted a 5-day course on PS and QI that was taken by third-year medical students before they began their clinical rotations in Pakistan. Students reported an increase in their knowledge, self-efficacy, and system thinking scores after the intervention. This design can be replicated and modified to incorporate PS curriculum across LMICs for medical students and other health care professionals. Additional research is needed to assess the long-term impact of PS courses on student knowledge, attitudes, and skills.