Secondary Logo

Journal Logo

Research Reports

In Pursuit of Meaningful Use of Learning Goals in Residency: A Qualitative Study of Pediatric Residents

Lockspeiser, Tai M. MD, MHPE; Li, Su-Ting T. MD, MPH; Burke, Ann E. MD; Rosenberg, Adam A. MD; Dunbar, Alston E. III MD, MBA; Gifford, Kimberly A. MD; Gorman, Gregory H. MD, MHS; Mahan, John D. MD; McKenna, Michael P. MD; Reed, Suzanne MD; Schwartz, Alan PhD; Harris, Ilene PhD; Hanson, Janice L. PhD, EdS

Author Information
doi: 10.1097/ACM.0000000000001015

Abstract

Physicians must continue learning after completing their training to maintain their proficiency in an ever-evolving field.1 Self-regulated learning (SRL) theory provides a conceptual framework for understanding physician lifelong learning.2 SRL theory proposes that learners can manage their own learning by using a cyclical approach with three main phases: self-reflection (reflecting on one’s experiences and learning needs), forethought (creating goals and planning for learning), and performance (learning by working on goals).2–4 Goal setting serves as a focal point for SRL because effective goal setting requires reflection and prepares a learner for performance.

Learning goals have been used to develop SRL skills in learners across the continuum of medical education, from undergraduate medical education5 and graduate medical education6–10 to many specialty board maintenance of certification programs.1 Developing these skills prior to the end of residency is crucial, as the completion of graduate medical education marks a critical transition where physicians must assume full responsibility for regulating their own learning. The Accreditation Council for Graduate Medical Education (ACGME) acknowledges the importance of SRL skills through the requirements of the practice-based learning and improvement (PBLI) competency, which specify that residents must develop the skills needed to identify their own deficiencies, set learning goals, and select appropriate learning activities.7 In addition, 5 of the 24 ACGME core residency review committees (neurology, nuclear medicine, pediatrics, preventive medicine, and radiology) explicitly require the documentation of individu alized learning goals as part of PBLI.8

Yet, despite the recognized importance of learning goals, research in medical education and in other fields shows that few learners consistently integrate these goals into their daily practice.2,3,10,11 For residents in particular, lack of time and lack of skills in using goals are significant barriers.6,11–13 Residents struggle with identifying specific goals and formulating effective plans to achieve them.6 Goal characteristics, learner characteristics, and types of goal set have been identified in the literature as factors that influence success in achieving goals.14–16 Other factors in the learning environment associated with success in achieving goals are not known.15 To facilitate the use of learning goals in a way that is meaningful to residents, medical educators must better understand what these other factors are and how they interact.

The purpose of this study was to explore the role of the learning environment as residents progress through the phases of SRL, with a focus on residents’ use of learning goals. The specific aims of this study were as follows: (1) to use qualitative methods to elicit pediatric residents’ perspectives on the use of learning goals in their residency program, and then (2) to develop explanations that suggest how different aspects of the learning environment facilitate or hinder the meaningful use of learning goals.

Method

We selected pediatric residency programs as the setting for this study because pediatrics is one of the five ACGME specialties that mandates the creation and documentation of individualized learning goals for all residents at least annually.8 Pediatric residency programs were invited to participate in the study at the 2012 Association of Pediatric Program Directors meeting. The study was described at a plenary session, and a follow-up e-mail with a brief survey was sent to all program directors. Directors of 30 programs expressed interest in participating. We used the results of this brief e-mail survey to select a purposeful sample of 12 U.S. pediatric residency programs with maximum diversity of size, geographic location, type of institution (private or public, academic medical center or community hospital), and current use of learning goals.

Brief interviews (20–40 minutes) were conducted with the program director or associate program director responsible for resident learning goals at each of the 12 participating programs. The in-person interviews were conducted by a trained qualitative interviewer. Program directors were asked to describe how their program used learning goals as well as to share their thoughts on what worked well in their program and the challenges they faced. Transcripts of these interviews and copies of the information that programs provided to residents about goals (supplied to us during the interviews) were examined to ensure that our sample included programs with diverse approaches to using learning goals as well as to gain an understanding of how each program used learning goals in order to inform our analysis of the resident focus group results (described below).

Third-year (senior) residents at each of the 12 sites were sent an e-mail invitation to participate in one-hour semistructured focus groups led by researchers trained in qualitative methods. Senior residents were chosen as the informants for this study because, having used learning goals for at least two years, they had the most comprehensive resident perspective on the use of learning goals in their program. The focus group format was chosen to allow for discussion among participants and to encourage participants to build off the comments of others.

The focus group guide was developed to elicit residents’ perspectives about the creation and use of learning goals in their residency program, the strategies they used to achieve their own goals, the programmatic factors that facilitated or hindered their success in using goals meaningfully, and the changes they would recommend to enhance the use of goals. (The focus group guide is available as Supplemental Digital Appendix 1 at http://links.lww.com/ACADMED/A317.) SRL theory informed our initial development of the focus group questions, which we then piloted in interviews with residents at the University of Colorado and iteratively revised. We intentionally did not define “meaningful” use of goals because criteria for the successful use of goals have not been established in the literature. Rather, we sought residents’ perspectives on what constitutes meaningful use.

All interviews and focus groups were conducted during the 2012–2013 academic year. Participants gave verbal consent to participate prior to the start of the session and received no incentives. All interviews and focus group sessions were audio-recorded and transcribed, and identifying information was removed from the transcripts. This study was approved by the Colorado Multiple Institutional Review Board and the IRB at each participating site.

Data were analyzed as they were collected to ensure that data collection continued until theoretical saturation was achieved (i.e., no new concepts were emerging and the concepts were well developed). The program director interview and resident focus group transcripts were compiled into one dataset and analyzed iteratively using the constant comparative method associated with grounded theory.17–19 Data were analyzed to identify initial codes (open coding) and relationships among the codes that suggested major themes and subthemes. After all the data were coded and themes identified, further analysis compared which themes frequently occurred together to strengthen the understanding of the relationships between the themes that influence the meaningful use of goals. These explanations led to a grounded theory that was developed through iterative discussions, throughout data analysis. Grounded theory was chosen as the framework for this analysis because we were particularly interested in the relationships between the themes and the resulting explanations that could lead to a better understanding of the complex process of using learning goals in residency.

At least two investigators independently coded each transcript, compared their coding, and reconciled disagreements through discussion until they reached consensus. Investigators never coded transcripts from their own institution. All investigators were involved in the development of the grounded theory (i.e., explanations). Our research team included program directors, pediatric clinician–educators, and PhD educators who brought practical experience, knowledge of relevant education theory, and qualitative research expertise to the analysis and interpretation of the data.

The final list of themes and subthemes was reviewed by a subset of the participating program directors to provide member checking to enhance the trustworthiness of the analysis.

Results

Ninety-five senior residents and 12 faculty members participated in the study. The 12 participating pediatric programs were diverse with regard to geographic location, size and type of program, and use of learning goals (Table 1). Nineteen focus groups were conducted (one to four focus groups per program), and 3 to 10 residents participated in each focus group. The sample of residents included residents with MD and DO degrees as well as residents participating in combined residencies such as medicine–pediatrics and pediatrics–neurology.

Table 1
Table 1:
Characteristics of the 12 U.S. Pediatric Residency Programs Participating in a Qualitative Study on Meaningful Use of Learning Goals in Residency, 2012–2013

Themes

Across the transcripts, 21 subthemes were identified and were grouped into 5 themes, as follows:

  • program support: orientation to goal setting, documentation system, frequent use of goals, protected time for creating and achieving goals, matching goals to learning experiences, peer support, and faculty development;
  • faculty roles: faculty support for reflection, for creation of goals, and for achieving goals;
  • goal characteristics and purposes: important, specific, measurable, realistic, includes plan, and consideration of multiple goal purposes;
  • resident attributes: buy-in, reflection, and initiative; and
  • accountability and goal follow-through: internal accountability and external accountability to help residents achieve goals.

The themes are described below from the perspective of residents, using selected quotations to illustrate their views and experience. Additional representative quotations from residents and program directors are presented in Appendix 1.

Program support.

Program support for the use of learning goals is central to residents’ success in achieving their goals and to their finding the process meaningful. This support must start early, with an orientation that provides residents with an understanding of how to select, formulate, and use goals. Residents also benefit from discussing the creation and use of goals with peers. One resident commented:

When you hear what other people are talking about, it actually makes you stop and reflect on yourself. So I feel like … that makes your individual goals a little more meaningful because you have the chance to … reflect on other people’s experience and normalize yourself.

Programs must provide accessible and easy-to-use tools (either computer- or paper-based) for documenting goals. Programs should be mindful of the influence forms have on how goals are written and used. Two residents explained:

I think making it shorter and easier to access … so you could refer to it more frequently [would help].

Somehow transferring it to a page makes it more … I don’t know, makes you commit more to them.

Residents commented that the frequency of goal creation has an impact on their learning, and some suggested the potential value of more frequent creation of goals that are specifically related to their rotations. Given the competing priorities in training, residents need protected time, separate from their patient care duties, to create, review, and revise goals. Patient care and other responsibilities can be aligned with residents’ learning goals by offering residents opportunities to select clinical experiences that relate to their goals. One resident said:

I think our schedule is very pliable. We have a lot of ability to pick our electives and kind of mold our year around what you want … to get to your ultimate goal.

Faculty should encourage residents to consider their upcoming rotations and to formulate goals that allow them to tailor their required clinical experiences to those goals. As one resident explained:

But to the extent that the resident is willing to be proactive … with help from their mentor, identifying things that are important to learn.… Residents specifically will go to a clinic and say.… What I want to see in dermatology today is kids with eczema and how you manage the kids with eczema.

Faculty development on how to develop and use learning goals is crucial to creating a shared understanding between faculty and residents. Residents described variability in support from faculty and emphasized the importance of faculty development. One resident stated:

Well, realistically for [goals] to be helpful, the faculty member you are meeting with would have to have training in how the [form to document goals] is set up and what the goals of it are.

Faculty roles.

Faculty support is necessary for residents to succeed in formulating and achieving goals. Longitudinal partnerships between residents and faculty help residents reflect on specific deficiencies in their skills, formulate meaningful goals, and follow through with achieving their goals. Rapport with faculty members is important because it helps residents candidly discuss their learning needs and create feasible strategies to achieve their goals.

Residents recognized that learning goals foster dialogue with faculty and help facilitate individualized learning. They appreciated working with faculty who ask about their goals during rotations. One resident commented:

I think setting rotation-specific goals with the attending at the beginning of the month and then at the end of the month reviewing whether or not you met those goals is more … of a real-time thing. And I think that would be more useful.

Residents indicated that they find it helpful when faculty ask about their progress in achieving their goals or provide feedback specifically related to their goals. Such interactions stimulate residents to focus on their goals and encourage forward momentum in achieving those goals. As one resident stated, “The most important thing is for the attending to take part in achieving these objectives.”

Goal characteristics and purposes.

Residents identified several characteristics of “meaningful” goals that aligned with aspects of the ISMART mnemonic for characteristics of useful goals (e.g., important, specific, measurable, accountable, realistic, timeline) that has been documented in the literature.14,20 Residents stated that to be meaningful, goals must be important, which they defined as relevant to future career plans, a self-identified gap in knowledge or skills, or a current learning opportunity. One resident said that goals should be “something personal you are invested in that you actually want to achieve.” Residents emphasized that goals should be specific and include a measurable and realistic outcome. Writing a goal in this way may involve breaking larger, long-term goals into smaller parts: One resident explained that goals should be written as “meaningful bite-sized chunks” that can be pursued in short-term increments. Residents recognized that a useful goal includes a description of how to know that the goal has been achieved. One resident commented: “So if you are going to be learning something, you need to be able to measure it. Am I actually learning this? Being able to apply it?” Useful goals were described as having a realistic scope that is achievable in a reasonable time frame. In addition, a well-thought-out plan for achieving the goal is crucial for success and allows residents to “delineate the steps that need to be taken in order to achieve a goal.”

Resident attributes.

Residents’ skills and attitudes influence their ability to formulate and achieve their goals. Residents felt they would be more likely to succeed in achieving their goals if they believed goal setting to be important, used self-reflection to develop goals that were meaningful to them, and took initiative in achieving their goals. Residents who did not view goal setting as important described putting little effort into the process and did not find the outcomes valuable. One resident commented:

If you do it once a year, you rush through it in 15 minutes late at night. And whatever prints out the next morning you don’t revisit it. It’s even more worthless than the time you put into it.

Residents who reflected on their goals and experiences felt they were able to develop meaningful goals. A resident explained:

If you take time to think critically about the things that you need to create the goals for yourself, and then create a system to help you achieve them and actually devote yourself to that, then you are going to be more likely to meet those goals and have an end result that you are pleased with.

Residents also recognized the importance of “taking initiative to seek opportunities to learn and practice.”

Accountability and goal follow-through.

Residents emphasized the importance of accountability—both internal and external—in actually following through on the learning goals they create. They described many different ways to ensure internal accountability, such as making time to think about and write down goals and motivating themselves to work on them. As one resident commented, “Even if we talked about it every month, it doesn’t mean I am going to go do it, unless I have some inner motivation to do those things.”

Residents also described supplementing their internal accountability with external accountability. External accountability can be facilitated by program requirements for setting goals, goal monitoring by faculty, and opportunities to discuss goals with others. One resident explained:

If I write something down and then I tell other people about it—this is what I want to do, and these people are going to help me stay accountable to it, and also the writing it down helps me personally stay accountable to it.

Many residents commented that more frequent meetings with faculty to discuss their goals would make goals more meaningful. One resident stated:

While my goals might be individualized, it has helped having faculty provide insight and keep you on task.… That there are other people to help you stick to your goals and work towards obtaining those goals.

Some residents described a different approach to ensuring external accountability—namely, communicating with other team members about their goals. One resident stated, “I would make myself more accountable by telling my interns and second years—keep me honest and expect me to teach you.” The residents were clear that support from others—the program director, faculty, peers, and/or medical students—is crucial to following through on their goals.

Program director perspectives

Although the same themes emerged in the program director interviews as in the resident focus groups, program directors’ intentions regarding learning goals did not always match residents’ descriptions of their experiences. For example, for several participating programs, the program director described well-thought-out plans for the use of goals, but the residents’ description of their experiences did not match those plans. Similarly, many program directors acknowledged that they had not yet found a way to make learning goals meaningful for their residents. One program director commented, “I would love to have the residents really start connecting their learning goals to their rotations and not think of it as an outside administrative thing to do.”

Relationships between themes

Review of the overlap and relationships between the themes revealed that program support provides a strong foundation for the meaningful use of learning goals (Figure 1). First, the residency program supports the faculty by giving them time to teach and faculty development regarding learning goals. Then, the faculty and the program support the residents in creating goals, with attention to goal characteristics and purposes, through providing orientation, time to work on and systems for documenting goals, and opportunities for faculty and peer support. Program and faculty support also help residents buy in to using goals and can foster resident motivation and self-reflection. When residents see goals as part of their lifelong learning, they become more invested in learning how to create and use them meaningfully. Finally, once the goals are created, accountability—both internal and external—encourages residents to follow through and achieve their goals or to learn from their goals and revise them. The program, the faculty, the goal characteristics, and ultimately the resident provide accountability that leads to follow-through. This pyramid starts with a foundation of program support that facilitates the layers above it, ultimately leading to goal follow-through. Providing the support needed at each level of the pyramid allows residents to use goals meaningfully as part of their SRL.

Figure 1
Figure 1:
Relationships between the five themes related to meaningful use of goals during residency, identified using qualitative data from senior resident focus groups and faculty interviews at 12 U.S. pediatric residency programs in a 2012–2013 study. All five layers in this pyramid support the meaningful use of learning goals. Each layer is supported by and builds on all of the layers below. For example, faculty roles are supported by program support, and accountability and goal follow-through are supported by the four layers below.

Discussion

Through focus groups of senior residents and interviews with program directors at 12 U.S. pediatrics residency programs, we identified five aspects of the learning environment that influence the meaningful use of learning goals in residency: program support, faculty roles, goal characteristics and purposes, resident attributes, and accountability and goal follow-through. Program support serves as the foundation of a pyramid of support and affects all of the other layers that lead to residents’ meaningful use of learning goals.

The themes regarding learning goals that emerged from this study demonstrated all three phases of SRL: resident attributes include self-reflection, goal characteristics represent forethought, and accountability encourages performance.2 Faculty can help residents during these three phases by promoting reflection, guiding goal creation, and providing external accountability. Ultimately, program support facilitates each step of the process by providing residents and faculty with training on goal setting, supplying a documentation system to help with goal creation, encouraging resident buy-in by emphasizing and encouraging faculty to role model the importance of goal setting, protecting time for reflection and work on goals, and enabling residents to actuate their learning plan by aligning goals with clinical experiences.

Our findings suggest that residents struggle to use their goals on their own—without external support—and place a high value on support from the faculty and the program. Nothnagle and colleagues13 have reported a paradox: Learners need external support to be effective self-directed learners. This is just what we found. The residents in our study reported that their internal motivation was not sufficient to enable them to create and use their goals for effective SRL. They needed ongoing support throughout the process, from reflecting on their past performance (the self-reflection phase of SRL), to creating goals (the forethought phase of SRL), to working on their goals in the clinical setting (the performance phase of SRL). Thus, although SRL is an internal process, it does not happen autonomously but, rather, requires external input and support.

As has been seen in other aspects of medical education, our results suggest that attention must be directed to both the explicit curriculum (what learners are directly taught; e.g., how to write learning goals) and the implicit curriculum (what learners learn from organizational and cultural influences; e.g., whether learning goals are worth the time needed to revisit them frequently).21–23 Much of what has been discussed previously in the literature regarding learning goals focuses on aspects of the explicit curriculum—that is, characteristics of effective goals, how to document goals, or how frequently to update goals.11,15,16 However, our results suggest that the implicit curriculum also has a substantial impact on learning from goals. This impact is evident in the contrast between the program directors’ descriptions of their program’s well-thought-out and detailed approaches to using goals and the residents’ perceptions that using learning goals is not always meaningful. A strong foundation of program support would include attention to aspects of the implicit curriculum such as competing priorities that may interfere with goal creation, faculty role modeling of goal use, and an emphasis on the importance of reflection.

There were several limitations to this study. Although a diverse sample of pediatric residency programs was purposefully selected for data collection and theoretical saturation was achieved, all of the programs were pediatric residencies, which require the use of learning goals. This may limit transferability to other medical education programs with learners at different stages of training, in different specialties, or without learning goal requirements. Further, data were not collected about actual learning outcomes related to using goals, and analyses for variations among programs were not performed. Finally, residents’ different definitions and understandings of the concept of meaningful use of goals may not have always been clearly elucidated.

In conclusion, this study provides a starting point for understanding the aspects of the learning environment that contribute to the meaningful use of learning goals in residency. Future research should test the grounded theory from this study by comparing the approaches that residency programs in different specialties use to support the optimal use of learning goals and tease apart the interactions between the five themes we identified. Rigorous methods for assessing the meaningful use of learning goals will need to be defined to identify the impact of each of these five themes on successful learning that results from goals and, ultimately, to determine the impact of using learning goals on achieving the SRL skills needed for physician lifelong learning.

Acknowledgments: The authors thank the following program directors and residency training programs for participating in this study: Rich Robus (Blank Children’s Hospital); Su-Ting Li (University of California, Davis); Adam Rosenberg (University of Colorado); Elizabeth Wedemeyer (Columbia University); Kimberly Gifford (Children’s Hospital at Dartmouth–Hitchcock); Jerry Rushton and Mike McKenna (Indiana University); Jenna Ross and Kimberly Northrip (University of Kentucky); Sharon Riesen (Loma Linda University); Gregory Gorman (National Capital Consortium); John Mahan and Suzanne Reed (Nationwide Children’s Hospital/Ohio State University); Alston Dunbar (Our Lady of the Lake Children’s Hospital); and Ann Burke (Wright State University Boonshoft School of Medicine).

References

1. Miller SH. American Board of Medical Specialties and repositioning for excellence in lifelong learning: Maintenance of certification. J Contin Educ Health Prof. 2005;25:151156.
2. Zimmerman BJ. Self-regulated learning and academic achievement: An overview. Educ Psychol. 1990;25:317.
3. Zimmerman BJ. Becoming a self-regulated learner: An overview. Theory Pract. 2002;41:6472.
4. Artino AR Jr, Jones KD. AM last page: Self-regulated learning—a dynamic, cyclical perspective. Acad Med. 2013;88:1048.
5. Challis M. AMEE Medical Education Guide No. 19: Personal learning plans. Med Teach. 2000;22:225236.
6. Li ST, Favreau MA, West DC. Pediatric resident and faculty attitudes toward self-assessment and self-directed learning: A cross-sectional study. BMC Med Educ. 2009;9:16.
7. Accreditation Council for Graduate Medical Education. Common program requirements. https://www.acgme.org/acgmeweb/Portals/0/PFAssets/ProgramRequirements/CPRs_07012015.pdf. Accessed October 5, 2015.
8. Accreditation Council for Graduate Medical Education. Program and institutional accreditation: Accreditation Review Committee Web page links. http://www.acgme.org/acgmeweb/tabid/83/ProgramandInstitutionalAccreditation.aspx. Accessed September 20, 2014.
9. Nothnagle M, Goldman R, Quirk M, Reis S. Promoting self-directed learning skills in residency: A case study in program development. Acad Med. 2010;85:18741879.
10. Sagasser MH, Kramer AW, van Weel C, van der Vleuten CP. GP supervisors’ experience in supporting self-regulated learning: A balancing act. Adv Health Sci Educ Theory Pract. 2015;20:727744.
11. Li ST, Burke AE. Individualized learning plans: Basics and beyond. Acad Pediatr. 2010;10:289292.
12. Stuart E, Sectish TC, Huffman LC. Are residents ready for self-directed learning? A pilot program of individualized learning plans in continuity clinic. Ambul Pediatr. 2005;5:298301.
13. Nothnagle M, Anandarajah G, Goldman RE, Reis S. Struggling to be self-directed: Residents’ paradoxical beliefs about learning. Acad Med. 2011;86:15391544.
14. Li ST, Paterniti DA, Co JP, West DC. Successful self-directed lifelong learning in medicine: A conceptual model derived from qualitative analysis of a national survey of pediatric residents. Acad Med. 2010;85:12291236.
15. Li ST, Paterniti DA, Tancredi DJ, Co JP, West DC. Is residents’ progress on individualized learning plans related to the type of learning goal set? Acad Med. 2011;86:12931299.
16. Li ST, Tancredi DJ, Co JP, West DC. Factors associated with successful self-directed learning using individualized learning plans during pediatric residency. Acad Pediatr. 2010;10:124130.
17. Hanson JL, Balmer DF, Giardino AP. Qualitative research methods for medical educators. Acad Pediatr. 2011;11:375386.
18. Glaser BG, Strauss AL. The Discovery of Grounded Theory: Strategies for Qualitative Research. 2009.New Jersey, NJ: Transaction Books.
19. Corbin J, Strauss A. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. 2014.Thousand Oaks, Calif: Sage Publications.
20. Doran GT. There’s a SMART way to write management’s goals and objectives. Manage Rev. 1981;70:3536.
21. Hafferty FW. Beyond curriculum reform: Confronting medicine’s hidden curriculum. Acad Med. 1998;73:403407.
22. Hafler JP, Ownby AR, Thompson BM, et al. Decoding the learning environment of medical education: A hidden curriculum perspective for faculty development. Acad Med. 2011;86:440444.
23. Eisner EW. The Educational Imagination: On the Design and Evaluation of School Programs. 2002.3rd ed. Upper Saddle River, NJ: Prentice Hall.

Appendix 1

Themes and Subthemes Related to Meaningful Use of Learning Goals in Residency, With Representative Quotations From Senior Resident Focus Groups and Program Director Interviews, 2012–2013a

Supplemental Digital Content

Copyright © 2016 by the Association of American Medical Colleges