In medical education, educators encourage learners to identify their learning goals. This educational activity is supported by the precept of Adult Learning Theory that teaching is more effective when directed toward educational needs identified by the learners themselves.1 By understanding learners' goals, educators are able to design instructional strategies that meet learners' needs. Goals affect performance by directing one's action, energizing individuals, influencing persistence, and indirectly leading to the arousal, discovery, and/or use of task-relevant knowledge and strategies.2
Goal setting involves self-assessment of previous performances. This is considered a key component of professional development. Unfortunately, many studies have questioned the ability to self-assess even in practicing physicians.3 As such, a goal setting exercise may be a challenging task for medical students who are transitioning to adult learners. Most of such self-assessment studies have compared “self-ratings” of particular clinical activities and external indicators, concluding that the personal judgment of one's ability is poor.3,4 Eva and Regehr cautioned that “self-rating” is only part of the complicated self-assessment that is occurring in the daily practice of medicine.5 When self-assessment is evaluated as awareness of whether the current situation is going well, rather than to perform an overall judgment of ability, studies have demonstrated that students can correctly identify general areas of strengths and weaknesses.6
Goal setting is a formative process that calls for learners' awareness of their strengths and weaknesses. Despite frequent use in medical education, studies about medical students' goal setting are very limited. In this study, investigators sought to identify the themes of third-year medical students' self-identified learning goals and analyze how their prior learning experiences and demographic characteristics influence the selection of their learning goals. The hypothesis is that understanding trends and characteristics of learners' goals help educators to assess learners' curricular needs and make appropriate adjustments.
In academic year 2006–2007, all third-year students at two medical schools wrote their learning goals for one of their clerkships as part of the educational activities. These sites were selected based on the similarity in the content of the goal setting exercise. At both schools, students were provided a form immediately before the start of their clerkship that asked, “What are your learning goals for this clerkship?” All students were encouraged to list three goals in free-text format and instructed to submit them to the clerkship coordinator by the first day of their clerkship. The form had personal identifiers so that the clerkship director and the student's instructors could share their student's learning goals to guide their teaching.
At one school, this form was used for their inpatient internal medicine clerkship, and at the other, it was used for their combined internal medicine and pediatrics ambulatory clerkship. Both clerkships were eight-week required courses for third-year students and were offered throughout the academic year.
At the end of the academic year, after all students completed their clerkship, these forms were collected. An honest broker deidentified the forms from both sites and assigned a unique identifier. Additional demographic information including gender, school, and the period of the clerkship (whether the student completed the clerkship during the first six months or the second six months of the academic year) was collected and assigned a matching identifier to permit data recombination. The investigators were blinded to the demographic information until after the coding of the goals was completed.
The themes and frequency of goals were analyzed using a “quasi-statistical” qualitative methodology.7 First, two investigators (H.M.T. and D.M.E.) independently categorized 10% of all goals into themes. After the independent coding process, two coders met and synthesized an emerged list of themes to create an initial codebook. Using this codebook, two investigators coded another 10% of all goals to determine a kappa score that assessed the agreement between the coders. By using the same process, each goal was also categorized into either “Specific” (focused and explicit) or “General” (broad, nonspecific, and lacking detail) goal. After confirming substantial agreement between the two investigators (k > 0.70), one investigator (H.M.T.) coded all the goals. Assigned codes were reviewed by the second investigator (D.M.E.). The codes were processed multiple times until complete agreement was achieved between them. Through this process, a final codebook was developed, and this was used to count the frequency of goals in each category.
The association between gender, school, and period of clerkship and the distributions of goals was analyzed using chi-square testing or Fisher exact testing where appropriate and Student's t test. Atlas.ti version 5.2 (Atlas.ti Scientific Software Development GmbH, Berlin, Germany) was used in qualitative analysis, and Stata 10.0 (StataCorp LP, College Station, Texas) was used in statistical analysis. The institutional review boards at both schools approved this as an exempt study.
A total of 290 students submitted the form (overall response rate, 82%). Of them, 190 were from the inpatient clerkship (response rate, 92%), and 100 were from ambulatory clerkship (response rate, 67%). About half (152; 52%) were men, and about half (143; 49%) completed the clerkship during the first six months of the academic year.
Investigators obtained 879 goals with a mean of three goals per student (range, 0–7 goals; four students completed forms without goals). Interrater agreement of category and specificity assigned by investigators was substantial (k = 0.73). Through the qualitative analysis, five domains of goals emerged. Acquisition of clinical skills was the most frequently identified, listed by 258 (89%) students, followed by medical knowledge (44%), other (20%), career choice (17%), and attitudinal goals (15%). Examples of goals in each domain are listed in Table 1.
Investigators looked for associations between demographic variables and goals (Table 1). No gender differences in distributions of goals were seen (P > .3 for all). There were few school/clerkship differences. More inpatient clerkship students identified “career choice” goals (22% versus 7%, P < .01), and more outpatient clerkship students identified “other” goals (14% versus 30%, P < .01). When the distributions of goals identified during the first six months of the academic year were compared with those identified during the second six months, “career choice” goals were more frequently identified during the first six months (22% versus 12%, P = .01). For all other domains, there was no difference between the two periods.
The clinical skills domain was a composite of 12 skills. These 12 subcategories and examples of goals in each subcategory are shown in Table 2. When the contents of clinical skills goals were compared between the first six months of the academic year and the second six months, goals related to improving oral presentation skills and critical-thinking ability were identified more frequently during the first six months (15% versus 7%, P = .05 and 16% versus 8%, P = .04). Goals related to patient-management skills were more frequently identified during the second six months of the academic year (18% versus 31%, P = .01) (Table 2).
Students' goals were more often “General” than “Specific,” with half of students (146; 50%) listing only “General” goals. A third of the students listed one “Specific” goal, and about 20% listed two “Specific” goals. Only three students (1%) listed three or more “Specific” goals. Examples of “Specific” and “General” goals are shown in Tables 1 and 2. There were no differences in the proportion of “Specific” goals between genders, schools, or periods of clerkship (P > .05 for all).
In this study, the investigators identified the themes of third-year medical students' learning goals. Regardless of student gender or the structure or timing of the clerkship, their goals were targeted toward the acquisition of various clinical skills. However, students' focus within the clinical skills shifted over time. Some of these shifts seem to trend according to the Reporter-Interpreter-Manager-Educator (RIME) taxonomy, a validated measure of learners' progress in clinical training.8,9 On the basis of this taxonomy, oral presentation skills and critical-thinking ability that were more frequently identified during the first six months of an academic year are categorized as “Reporter” and “Interpreter” stage skills, whereas patient-management skills identified more frequently during the second six months are categorized as “Manager” stage skills. This shift would indicate that students' goals shifted from basic to more advanced skills as an academic year progressed. This shows not only the expected maturation of students' clinical skills but also students' awareness of their areas of weakness or learning needs based on their clinical experiences and progress.
Students listed more “General” than “Specific” goals, and the specificity did not increase over the academic year. A study of internal medicine residents reported a similar result with more than 70% of residents' learning objectives being “General.”10 Goal setting theory and subsequent studies on goal setting mainly conducted in nonmedical fields have implicated that specific goals had a much greater chance of being accomplished than general goals. Moreover, having specific and appropriately challenging goals was shown to have positive impacts on performance.2 Students having more specific learning goals may also benefit medical educators when providing feedback to learners. Feedback should be based on well-defined and mutually agreed-on goals.11 Thus, helping learners identify specific goals could help medical educators formulate more effective feedback. Because learners do not seem to acquire the skills of generating specific goals naturally, medical educators, when asking learners their learning goals, may need to help formulate them.
There are several limitations to this study. First, this study collected goals from two schools with different clerkship structures. This makes the interpretation of the differences identified between the two clerkships difficult. However, the clerkship differences seem to highlight the overall similarities in the students' goals. Second, the goals were obtained from only one academic year, which may limit the reproducibility. However, over 870 learning goals were analyzed, and the investigators believe that thematic saturation was achieved. Finally, individual students listed their clerkship goals only once. Thus, the individual students' changes in their learning goals over time were not assessed. Although individual students' goal changes over time are of interest, it is difficult to assess because each student goes through the clerkship only once. However, students' goal changes within the eight-week clerkship period may elicit interesting insights into students' learning processes, such as the association between feedback and subsequent goal setting.
This study provides medical educators with information on third-year students' learning goals for internal medicine clerkships. Because medical education has been shifting more and more toward a truly leaner-centered curriculum, this will help medical educators understand learners better, target clinical teaching, and align curricula in more effective ways that match students' self-identified educational needs. This study identified that students' selection of goals is influenced by their previous clinical experiences and that goals advance over time as described in the RIME taxonomy. The results also highlight a potential issue associated with students' goal setting. Although future study is necessary to assess the effect of goal setting on students' performance, satisfaction, or behavioral changes, educators, when encouraging students to identify their learning goals, may teach them how to articulate specific, targeted goals with the hope of having more positive impacts on student's learning outcomes.
The investigators would like to thank Dr. Susan Zickmund for qualitative analysis instructions and Dr. James Bost for statistical support.
This study was funded by the Thomas H. Nimick, Jr. Competitive Research Fund, a part of Shadyside Foundation Grant.
1 Laidley TL, Braddock CL. Role of adult learning theory in evaluating and designing strategies for teaching residents in ambulatory settings. Adv Health Sci Educ. 2000;5:43–54.
2 Locke EA, Latham GP. Building a practically useful theory of goal setting and task motivation. A 35-year odyssey. Am Psychol. 2002;57:705–717.
3 Davis DA, Mazmanian PE, Fordis M, Van Harrison, Thorpe K, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. JAMA. 2006;296:1094–1102.
4 Duffy FD, Holmboe ES. Self-assessment in lifelong learning and improving performance in practice: physician know thyself. JAMA. 2006;296:1137–1139.
5 Eva KW, Regehr G. Self-assessment in the health professions: A reformulation and research agenda. Acad Med. 2005;80(10 suppl):S46–S56.
6 Eva KW, Regehr G. Knowing when to look it up: A new conception of self-assessment ability. Acad Med. 2007;82(10 suppl):S81–S84.
7 Miller W, Crabtree B. Primary care research: A multi typology and qualitative road map. In: Crabtree B, Miller W, eds. Doing Qualitative Research. London, UK: Sage; 1992:23–28.
8 Pangaro LN. A new vocabulary and other innovations for improving descriptive in-training evaluations. Acad Med. 1999;74:1203–1207.
9 Battistone MJ, Sande CM, Sande MA, Pangaro LN, Hemmer PA, Shomaker TS. The feasibility and acceptability of implementing formal evaluation sessions and using descriptive vocabulary to assess student performance on a clinical clerkship. Teach Learn Med. 2002;14:5–10.
10 Caverzagie KJ, Shea JA, Kogan JR. Resident identification of learning objectives after performing self-assessment based upon the ACGME core competencies. J Gen Intern Med. 2008;23:1024–1027.
11 Ende J. Feedback in clinical medical education. JAMA. 1983;250:777–81.