Secondary Logo

Journal Logo

PAPERS: KNOWING WHAT WE KNOW

Cognitive Benefits of Problem-based Learning

Do They Persist through Clinical Training?

LIEBERMAN, STEVEN A.; STROUP-BENHAM, CHRISTINE A.; LITWINS, STEPHANIE D.

Editor(s): Albanese, Mark PhD

Author Information
Erratum

The Lieberman, Stroup-Benham, and Litwins report of the October 2001 Research in Medical Education supplement, contains an inaccurate entry in Table 2. Under the column heading, “Scale Range,” “Conceptualization (C)” should read, “9-63” rather than “12-84.” The correction shows the scale range values of Mitchell's Cognitive Behavior Survey.

Academic Medicine. 77(5):445, May 2002.

The use of problem-based learning (PBL) is increasing in preclinical medical education. In comparison with students in traditional lecture-based curricula, students completing two-year PBL curricula report favorable changes in cognitive behaviors, specifically decreased reliance on rote memorization and greater reflection on the material they learn and how they learn.1,2 Students also report increased relevance of and greater satisfaction with their learning experience.2–6 Improved teamwork, an expected consequence of the small-group PBL environment, has been indirectly studied by assessing students' relations with their peers, with mixed results in several studies.1–5,7

Of course, preclinical education represents only the first two years of medical training, which typically spans seven to ten years. It is not clear whether benefits derived from preclinical curriculum changes persist through the undergraduate clinical years, not to mention postgraduate training. We undertook the present study to examine whether differences in cognitive behaviors and intellectual satisfaction resulting from traditional and PBL preclinical curricula persisted through a common clinical curriculum.

Method

Beginning in 1995 at the University of Texas Medical Branch in Galveston, a PBL curriculum for 24 students was conducted in parallel to the traditional lecture-based curriculum (TC) with approximately 180 students. Students in the TC were assessed almost exclusively by multiple-choice exams, while the PBL track used facilitator assessments of small groups, standardized patient exams, and written exams with short essays. PBL students were selected from a pool of volunteers by stratified sampling so that the resultant cohort of PBL students was similar to that of the TC students with respect to gender, age, and ethnicity. Following two years of independent preclinical education, PBL and TC students received a uniform clinical education consisting of traditional clerkships in surgery (eight weeks), internal medicine (eight weeks), obstetrics—gynecology (six weeks), psychiatry (six weeks), family medicine (four weeks), and pediatrics (four weeks), in addition to a 12-week primary care community-based clerkship in pediatrics, internal medicine, and family medicine. Assessment in these clerkships was multimodal, including written multiple-choice exams, standardized patient exams, and clinical performance evaluations.

All students matriculating in 1995 and 1996 completed several surveys during orientation in a testing environment. These surveys included the Medical School Learning Environment Survey (MSLES, Form A, querying expectations of the learning environment) and Mitchell's Cognitive Behavior Survey. Students were resurveyed at the end of year two and again during year four using the MSLES Form B (perceptions of the learning environment during the preceding two years) and the Cognitive Behavior Survey.

Five scales of the Cognitive Behavior Survey relate to students' study habits and their opinions of the academic environment. The memorization scale measures the emphasis students place on rote learning. The conceptualization scale assesses the degree to which students construct conceptual models or visual representations, develop overviews, and use higher-order cognitive techniques (e.g., analogies or metaphors). The reflection scale measures students' emphasis on reviewing and integrating previously learned material and discussing or reflecting on their own study habits. The peer-to-peer interactions scale assesses the quantity and quality of students' interrelations and group study. The positive learning experience scale indicates the degree to which students find medical school stimulating, motivating, and rewarding.

The MSLES consists of six subscales and a total scale score. Of interest for this investigation is the meaningful learning experience scale, which addresses a broader scope of intellectual satisfaction than does Mitchell's positive learning experience scale, including the emphasis on concepts rather than facts, application of knowledge, relevance of information to medical practice, and integration of basic and clinical science. Reliability and construct validity of the Cognitive Behavior Survey and the MSLES have been previously reported.2,8,9

Our goal was to examine whether the divergent effects of the dissimilar preclinical curricula persisted through similar clinical education. The six scales described above served as endpoints. Data from PBL and TC students at three time points (baseline, year two, and year four) were compared by two-factor analysis of variance (curriculum × time). In addition, multiple regression analyses were performed for each dependent variable (using a standard statistical software package). In addition to curriculum, the independent variables entered into the model include age, gender, race, first-time Medical College Admission Test (MCAT) scores, U.S. Medical Licensing Examination (USMLE) Step 1 and Step 2 scores, and the score of the corresponding dependent variable at baseline (for year-two analyses) or at year two (for year-four analyses). In addition, a class-by-curriculum interaction term was included as an independent variable to determine whether it was significant. Equations with significant interaction terms were rerun with three class-by-curriculum dichotomous dummy variables as independent variables rather than the individual class and curriculum variables. These multivariate analyses provided evidence of whether curriculum was a significant predictor of cognitive behaviors and learning experiences at the end of year two and during year four, after accounting for the other independent variables.

Baseline demographic data for students in each curriculum were compared by Student's t-tests (for continuous variables) or chisquare analyses (for dichotomous variables). Similar analyses were performed to determine differences between the classes (1995 and 1996 matriculants) or between those with and those without complete data sets.

Results

Of the 204 students matriculating in 1995, complete data sets (baseline, year two, and year four) were available for 131 of 180 (72.8%) TC students and 21 of 24 (87.5%) PBL students. Of the 200 who matriculated in 1996, complete data were available for 128 of 177 (72.3%) TC students and 20 of 23 (87.0%) PBL students. There was no significant difference between students included in the analyses and those omitted due to incomplete data with regard to baseline scores for any of the seven endpoints (data not shown). Students omitted were slightly older (25.0 ± 4.0 versus 24.1 ± 3.8, p < .05) and had lower MCAT (26.2 ± 5.9 versus 27.6 ± 4.9, p < .05) and Step 1 scores (205.0 ± 26.4 versus 212.0 ± 19.6, p < .05).

Table 1 provides comparisons of demographic and standardized testing data for students in the individual curricula and matriculating classes. In general, differences were few and small, justifying the use of combined data. Interaction terms (class × curriculum) were used in all regression analyses, with dummy variables incorporated when the interaction was significant.

TABLE 1
TABLE 1:
Demographic Data and Standardized Test Scores for Students Matriculating in 1995 and 1996 into the Traditional Curriculum and the Problem-based Learning Track*

Table 2 shows the least-squares means for the individual curricula over time and comparisons of the two curricula at each time point. Two scales, memorization and meaningful learning experience, were different for students in the two curricula at baseline. At the end of year two, students in the TC had a significant increase over baseline in memorization, with concomitant decreases in the conceptualization, reflection, and peer-interaction scales. Both intellectual satisfaction scales also decreased significantly. Changes in memorization and reflection reverted to baseline during years three and four, while preclinical decreases in conceptualization and peer interaction persisted. Both measures of intellectual satisfaction improved during clinical training for students who had completed the TC.

TABLE 2
TABLE 2:
Cognitive Behaviors and Intellectual Satisfaction during Medical School among Students in Two Different Two-year Preclinical Curricula Followed by a Uniform Clinical Curriculum*

Among students in the PBL curriculum, memorization decreased and reflection and meaningful learning experience increased during years one and two. Other measures showed no significant change. The decrease in memorization persisted through clinical clerkships, while the preclinical increase in reflection reverted to baseline and the increase in meaningful learning experience dropped to below baseline.

Multiple regression analysis indicated significant curricular differences in memorization (PBL lower), reflection (PBL higher), and meaningful learning experience (PBL higher) at the end of year two, after accounting for the other specified independent variables. By year four, curricular differences were significant for meaningful learning experience only.

Discussion

This study confirms the previously reported beneficial effects of a PBL preclinical curriculum on students' cognitive behaviors and intellectual satisfaction,2 but it also shows that these gains are not sustained through a traditional clinical curriculum. Specifically, the increases in reflective behavior and intellectual satisfaction (as measured by the meaningful learning experience scale of the MSLES) seen with the PBL curriculum disappeared during clinical clerkships. In marked contrast, the students in the traditional preclinical curriculum showed undesirable changes in memorization and reflection and decreased intellectual satisfaction during years one and two, yet these changes were reversed during years three and four. Significant differences between the curricula in these three scales were found at the end of year two, but only the meaningful learning experience scale remained significantly different at year four.

The divergent effects of the same clinical curriculum on students from different preclinical curricula could derive from any of several factors: differences at the time of matriculation between students entering the two tracks, divergent expectations for clinical training following different preclinical experiences, and the nature of the clinical education experience relative to the preclinical experience. Students enrolled in the PBL track were chosen from volunteers (approximately two volunteers for each available slot) by random selection stratified in order to produce similar demographic profiles for the two groups. Although demographic comparability was achieved, there were differences between the two groups at baseline, most notably in their prior reliance on memorization and their expectations of the intellectual environment of medical school. It seems reasonable to suggest that relative de-emphasis on memorization would contribute to self-selection for the PBL track, that the instruction and evaluation methods of the PBL track would tend to reinforce this prior behavior, and that the methods of the TC would reinforce prior reliance on memorization.

The reversal of the increase in memorization among TC students during subsequent clinical education may reflect the diminished utility of rote learning, both in patient care and in clerkship-evaluation methods. Using the Entwhistle Learning Style Inventory, Martin et al. reported that medical students' volume of clinical experience correlated more highly with a deep learning approach than with a superficial, memorization-oriented approach.10 Although causation cannot be inferred from their cross-sectional data, taken together with the current study, it appears that clinical experience promotes deeper learning than traditional lecture-based preclinical education.

Baseline differences in the meaningful learning experience scale might be argued to predispose students to a more (PBL) or less (TC) satisfying preclinical experience, in essence a self-fulfilling prophecy. However, the reversal of preclinical changes during years three and four argues strongly against such predetermination. Because this scale reflects emphasis on concepts, application of knowledge, relevance of information, and integration of basic and clinical science, it is not surprising to find an increase in this measure among PBL students and a decrease among TC students—both of these have been previously reported.2 While it is reassuring that TC students' intellectual satisfaction improved during years three and four, the decrease in meaningful learning experience among PBL students to below matriculation levels is unexpected. Although the reasons for this dramatic change are unclear, it seems unlikely that the students perceived less application of knowledge or less relevance of information during clinical clerkships. They may have perceived less integration of basic and clinical science or greater emphasis on facts over concepts in years three and four than in their PBL experience. Whatever the cause, these results serve to alert medical educators that although preclinical curricular reform can have desirable effects on students' cognitive behaviors and intellectual satisfaction, clinical education must adapt to capitalize on these changes, lest they be lost.

References

1. Moore GT, Block SD, Style CB, Mitchell R. The influence of the New Pathway curriculum on Harvard medical students. Acad Med. 1994;69:983–9.
2. Lieberman SA, Stroup-Benham CA, Peel JL, Camp MG. Medical student perception of the academic environment: a prospective comparison of traditional and problem-based curricula. Acad Med. 1997;72(10 suppl):S13–S15.
3. Moore-West M, Harrington DL, Mennin SP, Kaufman A, Skipper BJ. Distress and attitudes toward the learning environment: effects of a curriculum innovation. Teach Learn Med. 1989;1:151–7.
4. Kaufman DM, Mann KV. Comparing students' attitudes in problem-based and conventional curricula. Acad Med. 1996;71:1096–9.
5. Blumberg P, Eckenfels E. A comparison of student satisfaction with their learning environment in a traditional and problem-based curriculum. Proc Annu Conf Res Med Educ. 1988;27:60–5.
6. Bligh J, Lloyd-Jones G, Smith G. Early effects of a new problem-based clinically oriented curriculum on students' perceptions of teaching. Med Educ. 2000;34:487–9.
7. Clarke RM, Feletti GI, Engel CE. Student perceptions of the learning environment in a new medical school. Med Educ. 1984;18:321–5.
8. Mitchell R. The development of the Cognitive Behavior Survey to assess medical student learning. Teach Learn Med. 1994;6:161–7.
9. Marshall RE. Measuring the medical school learning environment. J Med Educ. 1978;53:98–104.
10. Martin IG, Stark P, Jolly B. Benefiting from clinical experience: the influence of learning style and clinical experience on performance in an undergraduate objective structured clinical examination. Med Educ. 2000;34:530–4.

Section Description

Research in Medical Education: Proceedings of the Fortieth Annual Conference. November 4–7, 2001.

© 2001 by the Association of American Medical Colleges