Srinivasan, Malathi MD; Wilkes, Michael MD, PhD; Stevenson, Frazier MD; Nguyen, Thuan MS, MD; Slavin, Stuart MD
Contemporary medical practice employs a variety of small-group, case-based discussion strategies.1–3 Small-group teaching methods emphasize teamwork and problem solving, yet they are extremely faculty- and resource intensive.4 Controversy remains about which method of small-group learning is most effective, time efficient, and palatable to the learner and teacher. In this study, we assessed two methods of small-group teaching: the classical problem-based learning (PBL) method5 and an innovative case-based learning (CBL) strategy.6
The two teaching methods
In PBL small groups, the group focuses on the process of discovery by learners—to stimulate problem solving, independent learning, and teamwork.7 Usually, facilitators play a minimal role and do not guide the discussion, even when learners explore tangents. In this format, learners are presented a problem, often using a clinical case as a starting point for discussion. Learners then have time to struggle and define the problem, explore related issues (during and/or after sessions), and grapple with problem resolution. Compared with traditional lecture-based educational approaches, PBL instructional methods have demonstrated improved learner/faculty satisfaction but without changes in test scores from responses to multiple-choice questions.8 Many hypothesize that when confronted with a new problem, learners participating in PBL instruction may emerge as better problem solvers than would learners from purely lecture-based instruction. Some have called PBL an open inquiry approach.
In CBL small groups, the group focuses on creative problem solving, with some advance preparation.9 Discovery is encouraged in a format in which both students and facilitators share responsibility for coming to closure on cardinal learning points. As in the PBL format, learners are presented with a clinical problem and have time to struggle, define, and resolve the problem. However, when learners begin to explore tangents, the facilitators will use guiding questions to bring them back to the main learning objective. Additionally, students prepare in advance for the session, and they may ask questions of the local experts during the session. They typically have little postsession work, although this varies depending on the group’s interest in pursuing additional issues. Some have called CBL a guided inquiry approach. Key similarities and differences between these two methods are illustrated in Figure 1.
PBL proponents argue that PBL methods encourage lifelong learning, simulate clinical practice, encourage curiosity, and create a broader understanding of the complexity of medicine.5,10 PBL detractors argue that the PBL process is time inefficient, frustrating for time-pressured medical learners, and often leads to erroneous conclusions.9 Additionally, there is little guarantee that the students will learn how to apply the material necessary for clinical practice in the absence of appropriate clinical direction.1 PBL detractors also state that the presence and expertise of the faculty is wasted if it not harnessed in more than a passive manner.
CBL proponents argue that CBL still provides for open-ended exploration of issues and encourages debate, discussion, and exploration of ambiguity while providing more structure for the learner in an efficient, goal-directed manner. Further, CBL proponents argue that CBL helps focus the learners on the key points of a clinical case, encourages a structured approach to clinical problem-solving, and allows each learner to be a “content expert” for part of the session. Importantly, they argue that facilitators can correct incorrect assumptions of the learner—which usually does not happen in PBL. Faculty can moderate the influence of louder, more contributory students. CBL detractors argue that providing answers (or direction towards answers) to key clinical or ethical questions effectively stifles curiosity. Detractors also argue that without intensive faculty development, the CBL format may encourage faculty to lecture instead of facilitate. They feel that CBL encourages a spoon-feeding mentality of learners, in which they always expect their peers or teachers to have the correct answers.
To date, no study has directly compared these extremely different types of small-group teaching methods. Previous barriers to this comparison included the need to train facilitators in both methods, the preference of faculty for one particular method of small-group teaching, and the need to recreate a new curriculum for students with the alternate method.
Curricular shift at two academic health centers
The University of California, Davis, School of Medicine (UCD) and the University of California, Los Angeles, David Geffen School of Medicine at UCLA (UCLA), each undertook a shift in teaching methods from a longitudinal PBL course to a longitudinal CBL course, via an existing small-group course called “Doctoring.” This shift occurred between 1998 and 2001 at UCLA and between 2002 and 2004 at UCD. Although the PBL course at each school had received good reviews from students and faculty based on standard course evaluations, the shift was undertaken to assess the impact of a guided inquiry approach over an open-ended approach to small-group teaching.
At the time of the change from PBL to CBL, the surveyed students had at least one year of experience with the PBL format, with similar content and learning issues. At UCLA, these medical students were finishing the second year (with PBL in their first year) and their third year (with PBL in their first two years). At UCD, these medical students were finishing their third year (with PBL in their first two years).
At both schools in the PBL format, students were presented with a clinical case using a sequential management problem (Table 1). Over the course of multiple sessions, students identified their own learning issues for the case, explored those issues, and brought back new material to the small group to inform the rest of the group. Many of the cases were triggered by standardized patient interviews. Video-trigger tapes or standardized patient interviews were used to illustrate key points, and students were encouraged to seek outside sources of information to share with the group on topics. The topics related to clinical management, etiology, epidemiology, and pathophysiology. Typically, obtaining and preparing additional material took less than an hour. If students explored tangential material or had incorrect assumptions about the material/approach, these were left uncorrected, because mistakes were understood to be part of the learning process. Trigger material was provided to the students to encourage discussion of ethical, economic, and cultural themes relevant to the session. Faculty were encouraged to provide guidance only when asked, and to throw back the questions to the students whenever possible.
At both schools in the CBL format, students were presented with a clinical case using progressive disclosure in a fixed order throughout the session. Students had preassigned general readings, which were often complemented by specific readings assigned to individual students. Typically, reading and preparing for the sessions ahead of time took less than one hour. Sessions typically began with a check-in phase, in which students shared recent professional experiences. Case discussion began with either a discussion of the main clinical problem or a presentation of a patient’s case, via a triage note. During each session, one or two students would interview a standardized patient. Faculty and interviewing students could call time-outs at key times to discuss the ongoing interview content to help the student problem solve. During time-outs, students in the group would provide feedback and assistance to the interviewing students. At the conclusion of the session, there was no requirement for students to read about the completed case, because the next case would be on a different topic. In several sessions, the same standardized patient would return for follow-up visits, allowing students to follow the development of illness over time, but the patient would present with different issues than those shown in the initial presentation. Ethical, economic, and cultural themes were interwoven into the discussion. Each week, facilitators were provided with a facilitator guide that contained learning objectives, sample questions, sample answers to the questions, and a time-management guide for the format of the class. Faculty were encouraged to provide guidance when the group seemed off track or unsure about the content. However, they were also asked to throw back questions to the students and facilitate discussion. The CBL curriculum was developed at both institutions by a common faculty member (MSW).
First-, second-, and third-year medical students, respectively, participated in Doctoring 1, 2, or 3 courses. Both PBL and CLB courses had about the same number of student contact hours per school. Depending on the course, students would meet from 15 to 25 sessions with their small groups. Sessions ranged from two to three hours. Each course ranged from 60 to 120 hours, depending on the year. Clinical cases were used to anchor the discussions and would often run over several sessions in both formats. There were roughly the same number of sessions per year, per school. At both schools, before and after the curricular shift, each small group consisted of two faculty and seven to nine students. These small groups worked together for the full year. Some of the curricular changes were highlighted in the first few Doctoring sessions. The paired faculty had different but complementary backgrounds. The pairs would include one clinical faculty member (pediatrician, internist, emergency room, etc.) and a mental health or social work faculty member (psychologist, psychiatrist, social worker, etc.)6,9. Each student in the two medical schools was assigned to a small group, with attention to balancing sex and ethnicity. All participants evaluated each other at the midpoint and end of the academic year.
Faculty at both schools were recruited from among those with the highest teaching evaluations and with a genuine interest in medical education. Faculty were assigned to small groups and worked with the same group all year. Faculty participated in specific training regarding the teaching methods. Training included a half-day to one-day seminar about teaching methods. We developed the seminars and focused on teaching techniques, use of standardized patients in teaching encounters, student evaluation methods, and discussion of teaching experiences. All faculty also participated in a required weekly hour-long faculty development sessions associated with each class. Faculty in both the PBL and CBL courses had little difficulty in adapting their style to either type of small-group format, based on their communications with us.
After the curricular shift, we assessed the effect of each teaching method on the target population of our educational intervention. Below, we report how we assessed the target population, the learners’ and faculty members’ preferences for the two teaching methods, the factors that contributed to those preferences, and how those involved in our study felt about the effectiveness of the two methods in reaching their educational goals.
We developed a 24-item survey instrument about the strengths and weaknesses of each curricular model and preferred choice of learning. The instrument was piloted with ten fourth-year students. The survey was administered approximately 10 months after each group had the opportunity to participate in the new curriculum: in 2001 to 215 students at UCLA and in 2004 to 89 students at UCD.
Questions focused on identifying the strengths of each teaching method based on the assumptions of each proponent, achievement of specific learning objectives, general curricular questions, and overall preference for each teaching method. For instance, theoretical advantages of CBL included fewer unfocused tangents or more opportunities for clinical skills application. Theoretical advantages of PBL included increased time interacting with and getting to know faculty and greater self-directed learning. The general curricular questions were assessed on a five-point Likert scale to allow learners a neutral choice if they had no strong preference about the question.
At UCD, third-year Doctoring students and their faculty were surveyed, and at UCLA, both second- and third-year Doctoring students were surveyed. Only faculty who had taught in both formats were included in the survey. The questionnaire took approximately 10 minutes to complete. Anonymity was achieved by deidentifying the data during analysis. We compared all students with all faculty, UCLA second-year with third-year students, and UCD third-year students with UCLA third-year students. We did not compare faculty responses between schools, because the sample sizes were small. Mean preference scores were compared using two-sample t tests; proportions preferring a specific format or rating a feature as a strength were compared by chi-square tests. We also explored the degree to which specific opinions on strengths or weaknesses led to a preference for PBL versus CBL, using logistic regression. All tests of hypotheses were two tailed at level 0.05, and all analyses used SAS version 6.02. The study was approved as exempt by each school’s institutional review board, because the responses were only examined in aggregate, all data were analyzed in a deidentified manner, and no adverse outcome accrued to either faculty or student for participation/nonparticipation. Because the study examined the effect of a curricular shift, and not learner-specific outcomes that could affect students’ careers, individual consent was not necessary for the survey.
At UCLA, 209 (97%) of the surveyed students and 23 (92%) of the surveyed faculty responded; at UCD, 77 (86%) students and 8 (100%) faculty responded. For both schools, this meant that 286 (94%) of the students and 31 (94%) of the faculty responded.
As seen in Table 2, students and faculty overwhelmingly preferred the CBL method; at UCLA, 189 (90%) of students and 18 (78%) of faculty; at UCD, 66 (85%) of students and 8 (100%) of faculty. Between 60% and 80% of students at both institutions identified as comparative strengths of the CBL format that it made better use of time, had fewer unfocused tangents, and decreased outside work and busy work (required work without perceived benefit). About half of the learners felt they had more opportunity for clinical problem solving. In addition, more than half of the faculty felt that CBL offered the opportunity to apply the skills learned in the CBL sessions in different practice settings. More than half of the faculty at both schools felt that PBL offered more opportunities for application of problem-solving skills within the session. Fewer than 5% of student and faculty felt that there were no advantages to the CBL format.
In comparison, the majority of faculty felt that PBL was advantageous in only two areas. Specifically, faculty felt that PBL provided more emphasis on independent learning and that it encouraged self-directed learning. However, student responses were significantly lower for these items, and responses differed significantly with the faculty. Additionally, fewer than half of the students agreed with the advantages of the PBL methods that have been proposed by PBL proponents. For instance, only 9% felt that less course structure provided opportunities for more participation in class or that PBL really promoted student participation. Only a quarter of all students agreed that PBL promoted the ability to explore a single case in any depth, emphasized independent learning, or encouraged stronger resource-utilization skills, and 23% of students felt that the PBL format had no advantages at all.
When asked directly about particular PBL advantages, students and faculty thought that PBL was somewhat valuable or not valuable and that it neither promoted nor inhibited discussion (see Table 3). Overall, they reported that PBL was somewhat valuable or not valuable in promoting listening skills. They reported that both PBL methods somewhat promoted self-directed learning. There were statistically significant differences between groups on a few of these items, but these differences were generally small in magnitude.
Comparison of learners and faculty perceptions, between institutions.
We were interested in seeing whether learners at different schools perceived the benefits of the curriculum differently—as reflecting a school’s individual culture. Because Doctoring 2 and Doctoring 3 emphasize different content and have students who have transitioned between the preclinical and clinical years, we only compared differences between schools for learners at the same stage of training (Doctoring 3 students), using two-tailed t tests. There were few significant differences between groups, as noted in Table 3. Compared with UCD students, UCLA students were more likely to agree that CBL decreased outside work and busy work and that it offered more opportunities for clinical problem solving. Compared with UCLA students, UCD students felt that there were somewhat more benefits to PBL.
Effect of learner level.
We were also interested in the effect of the learner’s stage of training. UCLA Doctoring 2 and 3 students differed on only a few points. For instance, compared with Doctoring 3 students, more Doctoring 2 students felt that the structured CBL environment enhanced knowledge.
Comparison with faculty ratings.
Faculty ratings of CBL were quite positive. A few differences were noted in comparison with students. For instance, students were more likely to report that CBL decreased outside work, busy work, or the need for presentation. Faculty ratings of PBL were less positive than their ratings for CBL. Faculty saw more benefits to PBL than did the students. For instance, faculty were more likely to report that PBL stimulated independent learning and encouraging self-directed learning. Only 3% of faculty (in comparison with 23% of students) found no value to the PBL method.
Associations with preferred instructional methods.
We also explored which perceived strengths of the two approaches were associated with a preference for one curricular method over the other, using logistic regression analysis. Separate models examined the role played by perceived advantages of CBL and of PBL. Students who agreed that having fewer unfocused tangents was an advantage of CBL were substantially more likely to prefer that method: odds ratio (OR) 4.10, P = .01. A preference for less busy-work and fewer student presentations also predicted a preference for CBL (OR 3.97, P = .01). Students who saw greater opportunities for clinical skill application in CBL preferred that method (OR 25.6, P < .002), whereas those who enjoyed more opportunity to meet with faculty tended to prefer PBL (OR 0.15, P < .001). These analyses suggest that all of these features of the two methods contributed somewhat independently to preferences, but they should be interpreted with caution because the number of students preferring PBL was small—only 27 in each model.
Educators are struggling to find practical instructional methods to promote critical inquiry and sustainable self-directed learning.1,5,6,11 PBL has been heavily promoted in the literature and in educational conferences as the preferred method of promoting critical inquiry. This study assessed the outcomes of a major, similar curricular shift at two academic medical centers to examine methods to promote inquiry and development of related skills. In this shift, PBL was replaced with CBL during a period of two to three years. Overwhelmingly, students and faculty at both institutions preferred the CBL instructional method. In this small-group method, both students and faculty contribute to discussion, learning issues are preidentified, and preparatory readings are assigned, while student discussion and guided inquiry around clinical problems are still promoted. Guided inquiry, over open inquiry, was preferred by these early learners and experienced faculty.
Although it is conceivable that individuals at both institutions had identical cultures and responded similarly because of similarities in their West Coast environment, these institutions are, in fact, quite different. The institutions differ in their urban/suburban settings, their in-state/out-of-state student mix, residency choices by students, faculty expectations, and institutional size. Outside of their common Doctoring curricula, other curricular initiatives are quite different. At each institution, both the PBL and the CBL courses were carefully conceived, well executed, well supported, and involved numerous faculty. The initial PBL courses were only somewhat similar in content, but they were nearly identical in method. They both adhered to the strict definitions of PBL described by Barrows.5 Although the CBL-initiating faculty member (MSW) was the same at both institutions, the faculty implementing the curriculum and the students at each site were different. The CBL curricula at each site, while quite similar, were not identical except in method of instruction. Thus, the robustness of our findings is increased by the variation in the schools and courses themselves. Additionally, both students and faculty rated important questionnaire items for both PBL and CBL very similarly—with few statistical differences—further strengthening our findings.
Why did these students prefer the CBL over PBL? Did they eschew the fully open inquiry method of PBL? Our findings indicate that the determinants of student preference for the CBL format were related to perceptions of efficient use of time, not an opposition to open inquiry. Medical learners are operating in a time-pressured setting, in which learners are multitasking clinical, academic, and personal responsibilities—often in excess of 60 to 80 hours a week. The lack of closure of the PBL method, with additional work between cases, and exploration of blind alleys, seemed like busy work to the learners—as opposed to free exploration and inquiry. Student presentations were often inefficient, and some presented inaccurate or incomplete information on their topics. The open inquiry method is also not mirrored in the clinical arena, where the learners develop a sound clinical approach with guidance from faculty and more senior learners. It may be that more advanced learners (who have the context for their work already developed) or a subset of early learners might benefit more from a process of open inquiry in a time-limited context. At our institutions, learners participated in no other courses taught in the PBL format. One might hypothesize that if an entire school were created using the PBL model, with sufficient time given for exploration and inquiry, the outcomes of this study might have been quite different.
Even after 15 to 20 hours of faculty development, the faculty at these two institutions preferred the CBL method. In our study, faculty impressions of PBL, although more positive than those of the students, were not very positive when compared with their impressions of CBL. Our faculty did see the value of the PBL course in stimulating self-directed learning. But the CBL method allows faculty to contribute to discussions to guide the learners without dominating their discussions. CBL uses faculty as a consultants and allows the two faculty (clinical and mental health) to interact with students in their areas of expertise. It also allows faculty to balance discussion among individuals, as opposed to having the group moderate itself and unduly favor the more aggressive students.
Additionally, in the PBL format, faculty do not provide learners with feedback about their performance. From a theoretical perspective, many early learners are unskilled, unaware of their lack of skill, and have difficulty identifying methods to improve their skills. Feedback is thought to be essential to development of reasoning skills among learners. Although fully open inquiry may promote scientific curiosity among learners who are aware of their own skills and deficits, it may not benefit the larger group. However, trained faculty at the two institutions were unable to achieve the levels of learner satisfaction and acceptance that have been seen at other institutions using the PBL instructional technique.
No curriculum will satisfy every learner or faculty group, but incorporating learner preferences within a curricular framework holds greater promise in creating a sustainable, accepted curriculum. With both the PBL and CBL curricula, we have noted that between 10 and 15% of learners are persistently unhappy with being pulled from other duties to participate in small-group learning sessions. However, the quick acceptance of a major curricular shift (where there is normally resistance) points to a value that both learners and faculty placed on the revised CBL instructional method.
Should learner preferences be considered when creating curricula? Some might argue that learners are too naive to know what is most beneficial for their future careers, and that learners do not know best. We disagree with this assessment of our adult learners. Medical faculty and learners often trade short-term challenges for long-term skills improvement. Yet, if both learners and experienced faculty find little benefit to an instructional method—when given a choice and experience in competing methods—then educational planners must give serious consideration to their viewpoints. Similar to 360-degree evaluations in other contexts, feedback about instruction may not be pleasant, but it needs to be incorporated into planning for future curricula. As adults in their mid-to-late 20s (and sometimes 30s), medical students have a rich sense of utility, and they are quick to passively oppose activities that they find less valuable. Although they may not know what they don’t know, they usually recognize activities that are beneficial to them, in which they see real gains in skills, techniques or thought processes.
This study has several limitations. First, only 60% of faculty who taught in the PBL course taught in the CBL course. Thus, we might have a biased sample of educators who chose to stay during the curricular shift. However, we had an over 90% response rate from all students who had participated in the curricular shift at both schools, and participation was mandatory in both curricula. Second, the study took place over several years, and the characteristics of the learners at each institution may have changed. Third, although we have studied the perceived utility of CBL and PBL among faculty and learners, we do not know whether one of these methods is more effective in producing physicians who are more prepared for the complexity of patient care encountered in practice. This longitudinal final outcome cannot be studied based on our curricular shift, because a number of influences (rotation experience, clinical feedback, non-Doctoring skills development, residency program, and related experiences) shape the clinical performance of these medical students. Fourth, it might be that faculty who may have disliked some aspects of PBL (such as not correcting student errors) may have been unconsciously biased toward disliking PBL teaching as a whole. Although it is unlikely that this occurred with all 84% of our faculty who preferred the CBL method, a more global like/dislike may have biased a subset of the faculty. Fifth, we did not compare our CBL method of small-group learning with other PBL variants, such as student-directed PBL, virtual PBL, etc. Sixth, and most importantly, our survey was conducted retrospectively, after students and faculty had experienced a new course format. Their recall of their previous course may have faded, or they may have been biased by their participation in the second course structure. This potential recall bias might affect the generalizability of our results
Even with these limitations, our study’s findings at two distinct universities are striking, and the multiinstitutional aspect decreases the likelihood that these findings are spurious. When given a choice, learners and faculty preferred the guided-inquiry method of CBL to the open-inquiry method of PBL. These findings raise important questions for the use of small-group learning techniques in educational settings. Small-group learning methods in medical education are expensive and faculty- and staff intensive, and they require tradeoffs with other more directly clinical activities. Our findings call for a careful analysis of the effect of curricular interventions on learners and faculty to guide the best use of limited resources in medical education.