A needs assessment or situational analysis is typically recommended as the foundation of any curriculum development process.1 Steinert et al,2 in their Best Evidence Medical Education review of faculty development programs in teaching, advise that the goals and objectives of these programs must address the local context and continually adapt in response to institutional priorities and needs in order to be effective for educational change. These notions are aligned with adult learning principles that promote the need for curricula to be relevant and timely to the individual practitioner as well as to the practitioner’s context.3
The typical time lag between needs assessment and implementation of related faculty development curricula assumes a certain stability of individual and contextual needs. Typically, a cross-sectional needs assessment is followed by program development and implementation of new curricula. However, institutional priorities can shift during the development and implementation period as health professional schools respond to external pressures, distributed campuses, changes in health care delivery, and interest in curricular improvement and innovation.4 Once the program is being implemented, any new needs that have emerged during this time lag are often not addressed, regardless of how relevant or important they may be. From a participant perspective, Skeff et al5 have also noted that individuals often do not fully appreciate their faculty development needs until they have been exposed to the subject area. Similarly, until participants learn a language of practice, their thoughts about perceived needs in education can be constrained.6 Precurriculum needs assessments can therefore be limited in their ability to tap into what may ultimately feel relevant once the course is underway. These issues greatly challenge faculty developers to be able to provide relevant and responsive curricula for the individuals and their contexts in real time.
Strategy in Curriculum Development
Similar to strategic management of organizations, curriculum development involves a process of defining directions and decision making about how these directions can be realized. Mintzberg7 describes strategy as a pattern of actions that are consistent over time that may or may not be preceded by decisions. The term strategy is used to describe either an intention that is formulated intentionally (deliberate strategy) or an outcome that forms and becomes apparent organically (emergent strategy).8 Purely deliberate strategy precludes learning once the strategy is formulated, whereas emergent strategy fosters learning. Thought and action are often not separated in time, and strategies can emerge from feedback as action informs thought through learning. In practice, all strategy making involves some degree of deliberation and emergence so that flexibility and learning are coupled with some control.
Traditional planning in health professional faculty development tends toward deliberate strategy, focusing on the program meeting the needs of individuals within a context.9 Strategy is outlined and implemented to achieve certain outcomes with little adjustment once initiated. Deliberate efforts to identify emergent needs of individuals and their context in order to adapt curricular position or perspective are rarely described.10,11 Strategic visioning is a deliberate process that addresses the philosophy and goals of a program. Once a program is underway, emergent processes can be encouraged if feedback is provided to inform and instruct. These could include strategic venturing in the development of new content, methods, or innovations and strategic learning from the emergence of new paradigms, philosophies, or goals.
An umbrella strategy is a mixed approach that encourages emergence within certain boundaries.12 Such a blend is useful in situations that require great expertise as well as creativity so that implementers are also designers who can use their knowledge of the situation at hand and their technical expertise to adapt the program in progress. General guidelines or boundaries for direction, which are broad in outline and not specific in detail, are defined. These boundaries are usually defined within a strategic vision that is flexible in scope so that venturing and learning can occur. This intentionally creates conditions under which input, learning, and experimentation can facilitate and inform adaptation of the execution (content and methods) as well as the vision (goals and philosophy) of the strategy. Applied to curriculum design, this approach suggests that curricula need to have clear goals, intentions, and limits defined at the outset, but that conditions must be created to allow grassroots input, learning, and experimentation. Such a process should identify emergent needs and adapt in real time to respond to these issues. This approach enables innovation and organizational learning.13 Its use has not been described in the faculty development field.
In this article, I describe how an umbrella strategy informed the relevant, timely, and continuous evolution of the University of Toronto 2004–2006 Education Scholars Program (ESP). The ESP was created in response to a perceived need to address the neglect of education and the advancement of educators in our research-intensive university. Its development was itself part of a strategic plan for a central and coordinated program of faculty development. The ESP is a two-year, half-day-per-week faculty development program for 15 health professions educators of mixed professional backgrounds from the Faculty of Medicine (FoM) that enrolls a new class biannually. The FoM has more than 5,000 full- and part-time faculty members spanning 13 fully affiliated and 13 partially affiliated hospitals in the University of Toronto FoM. Candidates were nominated by their department chairs and selected by a selection committee consisting of the leadership of the ESP and the Centre for Faculty Development. Departments supported participation by providing protected time and tuition support for participants. The umbrella strategy illuminated the development of a strategic process of continuous and iterative feedback that informed curricular evolution in real time. Specific methods were implemented to enable enrolled participants, as they became increasingly aware of and able to articulate their needs, to inform curriculum development by participating in an ongoing feedback process. Using such a strategy in the design of faculty development and continuing medical education programs not only enhances individual and contextual relevance but also participants’ motivation, learning, and self-direction.14,15
Fifteen multiprofessional participants enrolled in the first class of the program in 2004, representing eight clinical departments (six MD, two other health professions). These eight men and seven women had been on faculty for an average of 5.6 years (range: 1–19). Their ranks included seven lecturers and eight assistant professors. Scholars attended 87.54% of all classes (range: 80.43% to 93.17%) during the two years and, according to evaluation data, were extremely satisfied with the program. During the two years of the program, the semiannual program evaluations yielded a 4.3 (1 = strongly disagree; 5 = strongly agree) rating of whether the individual sessions should be repeated for the next cohort (range of average score for each class: 3.6–4.9). Detailed program evaluation findings will be reported in another paper.
Informing the Curriculum
In this section, I describe how the umbrella strategy specifically informed the selection of various processes that were used initially or iteratively throughout the ESP to identify emergent needs and perspectives. A broad-based needs assessment at baseline informed the development of the goals for the program, and these goals became the broad vision and boundaries. Subsequent to this, a variety of methods were implemented to solicit input iteratively from participants in order to identify emergent needs, learn, and experiment with evolving the curriculum in a timely way.
Creating the vision: The umbrella
In the six months before curriculum implementation, the needs assessment incorporated an environmental scan internally and externally, including review of multiple sources of information as well as interviews with key potential stakeholders and opinion leaders both within and outside the University of Toronto.16–18 The intention of the needs assessment was to identify perceived needs of local leadership and potential participants, understand barriers and enablers of the context, extract best practices, clarify information, and identify key forces and trends. For internal and external data sources, see Table 1.
Accepted applicants were interviewed by the course leadership to discuss their goals and expected outcomes for the program, and their application materials were reviewed to extract redundant themes.19 Interviews of sponsoring chairpersons and clinical chiefs focused on the faculty development needs of their departments as well as their goals and expectations for the sponsored scholar. External experts were contacted to discuss their opinions on best practices and to answer questions about unique features and strengths of their programs, as well as challenges and barriers to implementation and sustainability. All interviews were manually transcribed by the course director, and predominant themes were summarized at the end of the interview for review by interviewees to ensure accurate representation of their views.
Similar to other scholar programs, the need to promote the professional development of educators in the health professions in the areas of teaching, education scholarship, and leadership was a priority need of all formal and informal sources.20–22 A recurrent theme in local leadership interviews was the expectation for participants to be able to provide faculty development in education to their peers through such activities as program development, peer review, and consultations. Additionally, the development of a community of practice of educators emerged as essential to sustaining these geographically distributed participants beyond the duration of the program as well as to promote inter- and intraprofessional collaboration. With such an enormous and distributed faculty, this demand for participants to disseminate what they are learning and to collaborate was not surprising.
The ESP vision/program goals were threefold: to develop (1) scholarly health professional educators with enhanced abilities in teaching, education scholarship and education leadership,23 (2) faculty developers, and (3) a community of practice.24
A core curriculum was defined that provided exposure to key constructs in teaching, education scholarship, and leadership that would span the program’s two years (Table 2). Contributing course activities and requirements were designed to facilitate the integration and application of these constructs into the actual practice contexts of participants. Teaching practica, lectures, workshops, and scholarly project assignments provided scholars with the opportunity to integrate and apply what they were learning in the classes. Apprentice, lecture, project, and reflection discussions as well as written apprentice reports supported the development of faculty development skills, including small-group facilitation and oral and written feedback. In reflection discussions, participants learned to reflect on and deconstruct issues and assumptions in their practices as educators. Dossier and career action plans assisted participants in defining career and leadership goals and plans to achieve them. “Hot topics” classes focused on critical appraisal of selected “hot” literature with an emphasis on discussion of application to practice. At the outset of implementation, the first-term curriculum was defined and a working draft of the second term was in development. The plan was to leave up to half of the topics in the second year for emergent needs that evolved from the first year of the program. By identifying core curriculum goals while reserving curricular space for identifying and addressing emergent needs, we successfully applied the umbrella approach to curriculum planning.
Methods to enable emergence
To avoid the traditional time lag between identifying emergent issues and incorporating them into curricula, we developed a number of strategies to solicit input from participants during the two years of the program. These methods were selected on the basis of their practicality, availability, and minimal added burden to the participants and administrators of the course. Students were advised at the outset that in keeping with a learner-centered collaborative program, the course was prepared to evolve the content and process in response to emergent needs. They were, therefore, encouraged to provide feedback on any issues or needs as they arose. It was apparent that they were pleased with their potential to influence the course design.
Creating a dialogue space.
Flexibility is not possible without formal structured curriculum time allocated to dialogue about emerging needs. Therefore, a number of spaces were built into curriculum time for discussion of course issues and participant needs. A half hour of class time weekly was devoted to “housekeeping” discussions. Both the course director and participants were invited to raise any questions, issues, or business related to the program content, process, and administration and potential responses to the issues. Participants were also invited to meet with or e-mail course leadership outside class time. Participants kept journals on their educator practices throughout the course and brought them to the 12, one-hour classroom reflection discussions. These discussions provided further opportunity to identify shared needs. Participants also raised questions and issues for further learning in the apprentice discussions that followed every class. These discussions involved deconstructing the curriculum design and teaching methods of the preceding class with the intention of critically reflecting on the process as well as crafting a formative feedback report for the faculty member. The course director kept a journal on the course to keep track of emergent issues and themes.
Class and program evaluations.
In addition, weekly class evaluations solicited information about what could be improved and allowed participants to submit general comments. These evaluations were synthesized and reviewed weekly so that any questions about the results could be discussed in housekeeping time with the class. Semiannual program evaluations inquired about what could be improved and what was missing from the program. Any unclear findings were discussed with the class at the beginning of the following term.
At the end of each term in the first year, students prepared reflection papers that addressed how they were applying the course in their practice. These reflection papers provided course leadership with a window into the challenges and contexts in which participants were translating their knowledge to practice.
Opinion leader interviews.
The course director met annually with key formal and informal local opinion leaders in health professional education to scan for local preoccupations, driving forces, and trends that they felt should inform the course curriculum.
Formal needs assessments during the course.
Participants were explicitly informed that curriculum time in the second year had been protected in order to respond to their emerging needs. Attendees participated in three modified Delphi surveys during the two years of the program.25 The “Wildest Dreams Needs Assessment,” toward the end of the first year of the program, invited participants to identify and rate topics that they would love to have in the second year of the curriculum. At the end of year one, a focus group was conducted and thematically analyzed by an external consultant to provide directors with evaluative feedback and curriculum suggestions. The second Delphi, at the end of the first term of the second year, inquired what they would like to learn in a future career planning session. Toward the end of the final term, the “What Next” Delphi inquired how they would like to stay in touch after the program, and what would facilitate their ongoing development. The results of the final Delphi were then circulated to the class to facilitate a dialogue about their future plans beyond the course (Table 2).
These various methods provided course leadership with rich information about how the curriculum needed to evolve in real time during the two years. This section outlines the issues that were identified through this iterative process and how the curriculum continuously adapted in response.
Strategic venturing in course content and process
In this section, the various issues that were identified and responded to are described with attention to how they were identified and the specific modifications that were ventured to the program content and process. Housekeeping and reflection discussions as well as the weekly class evaluations resulted in the identification of the following concerns on the part of participants.
Participants repeatedly emphasized their desire for classroom activities that ensured application of concepts directly to their contexts through exercises, simulations, or discussions of application to practice. The 34 faculty who taught in the program were advised of this desire and assisted in adapting their curriculum designs when indicated.
Scholarly project challenges.
Scholars reported a variety of challenges in the development and implementation of their projects. More support was needed in ethics submissions and literature searches. Heads of local ethics boards were invited to teach, an ethics submission guide was created for the students, and individual ethics consultations were provided on request by the course administrative team. A session taught by a medical education librarian was arranged in a computer laboratory, where participants were provided with floppy disks linking them to appropriate search tools. The librarian also offered participants follow-up telephone support. A number of participants were provided with readings on focus-group methodology and linked with local experts for advice on this subject area. Project management tools were created to assist them with organization and timelines for their projects.
The electronic reflection discussions.
In the beginning of the second year, reflection discussions were moved to the electronic discussion board. Participants expressed significant resistance to this learning methodology. Within a few months, participants adapted to the technology and presented a cogent argument to restore these discussions to class time. They decided to forego class time for project work in order to incorporate this adaptation.
Bring back the apprentice discussions.
The apprentice discussions were only offered in the first year of the curriculum in the interest of managing the workload of the course. In the second year of the program, scholars discussed missing these weekly discussions. They elected to extend class time from 4:30 to 5:00 pm in the second year to enable these discussions to be restored. Because time was protected until 5:00 pm, this did not require any renegotiation of commitments.
Journaling and the reflection papers.
Semiannual program evaluations and first-year focus groups revealed ambivalence about independent journaling and reflection papers. Subsequently, the class decided that they would like more reflection discussions in class as well as protected time to journal in class.
Local educational opinion leaders’ views centered around the following themes: interprofessional education, simulations, use of technology for teaching, and increasing enrollment of students. The results of all three Delphi surveys showing the top 15 topics are indicated in List 1. Beyond the predetermined second-year core curriculum and the emergent issues previously described, the remaining curriculum time was informed by the first-year Delphi survey, opinion leader interviews, trends in the literature, as well as unique strengths of local and visiting faculty. These topics were primarily distributed among the scholars’ choice workshops and hot topics classes (List 2). Writing for publication was felt to be more appropriate for an alumni program.
Strategic learning about course goals and philosophy
Housekeeping discussions, reflection papers, term evaluations, and focus groups indicated that participants experienced the course as a safe, supportive, and mentoring environment. They noted that they had not experienced the level of trust and comfort in being able to talk about issues of mutual interest anywhere else in their careers. The strong mentoring and advising feature of the program had not been intended. The goal of the development of a community of practice within the class was affirmed by the program evaluations and focus groups. An unexpected outcome was that participants also felt connected to and a part of a greater community of practice of educators beyond the course. The mentoring and advising as well as connection to the greater community of educators have now become explicit aspects of the course goals and philosophy.
The use of an umbrella strategy illuminated an iterative methodology for continuous adaptation of a faculty development curriculum in real time to emergent needs. Andragogy, constructivism, and self-directed learning principles suggest that such a student-centered and collaborative process is likely to be motivating and engaging to adult learners.26 It was apparent throughout the course that participants welcomed the opportunity to contribute to curriculum evolution. When given the opportunity to communicate their needs in an iterative way, course participation increased their awareness and ability to articulate ideas and concerns about relevant curricular topics and methods. Ensuring that curriculum time was available to respond to these emergent needs was critical to real-time adaptation. This builds on the work of Mintzberg,27 who describes creating a white space in the curriculum that is unstructured and can be filled in by the learners. Consistent with Patton’s28 recommendations, evaluators worked closely with stakeholders’ key questions and issues and involved them in the interpretation of the findings. Westley et al29 have proposed that this type of partnering for developmental evaluation is critical to nurturing the exploration and adaptation essential to innovation. This work distinguishes itself in that the participants are empowered to adjust the curriculum in collaboration with the course leadership in real time rather than feeding findings back into curriculum development after the cohort has completed the program.
An umbrella strategy recognizes the need to develop methods that define the boundaries and core curriculum and that enable curricular emergence based on participant and stakeholder experience. This approach could be of particular value with a mixed professional group of students whose shared needs cannot be comprehensively anticipated. An iterative approach to identify emergent opportunities and adjustments must be combined with a flexible course leadership and protected class time for venturing.
This methodology can be applied to a faculty development or continuing education program of larger size or shorter duration, although the choice of specific needs assessment methods must be context specific. In general, class, Delphi, and module evaluations were quick, simple, and easy methods to anonymously identify emergent needs. Creativity is required in considering how to create a safe dialogue space in class or electronically, and this will vary according to size, frequency, and preferred classroom methods.
The notion of curriculum design in faculty development as a planned strategy limits the ability of curriculum planners to adapt their curricula once implemented. Applying an umbrella strategy helps faculty and professional developers to construct intentionally an iterative needs assessment program with curricular space and flexibility to respond to the individual and contextual needs of the learners. This strategy marries the need to define a core curriculum with a process of collaboratively engaging learners in iterative identification of emergent needs and creatively evolving program content and process. Formal and informal scanning within the program as well as externally can provide course directors with important direction regarding relevant and timely curriculum adjustments as well as motivating and engaging participants. Although this does require some additional work on the part of course directors, it is well worth it to reap the benefit of highly engaged and interested learners.
The author would like to thank Drs. Ivan Silver, Brian Hodges, and Scott Reeves for their thoughtful review and feedback on the manuscript.
2 Steinert Y, Mann K, Centeno A, et al. A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Med Teach. 2006;28:497–526.
3 Knowles MS, ed. Andragogy in Action. San Francisco, Calif: Jossey-Bass; 1984.
4 McLeod PJ, Steinert Y, Meagher T, McLeod T. The ABCs of pedagogy for clinical teachers. Med Educ. 2003;37:638–644.
5 Skeff KM, Stratos GA, Bergen MR. Evaluation of a medical faculty development program: A comparison of traditional pre/post and retrospective pre/post self-assessment ratings. Eval Health Prof. 1992;15:350–366.
6 Lingard L, Haber RJ. Teaching and learning communication in medicine: A rhetorical approach. Acad Med. 1999;74:507–510.
7 Mintzberg H. The strategy concept I: 5 P’s for strategy. Calif Manage Rev. 1987;30:11–32.
8 Mintzberg H, Waters JA. Of strategies deliberate and emergent. Strateg Manag J. 1986;6:257–272.
9 Farquharson A. Assessing what needs to be learned in teaching in practice. In: How Professionals Can Work Effectively With Clients Patients and Colleagues. San Francisco, Calif: Jossey-Bass; 1995.
10 Wilkerson LA, Uijtdhaage S, Relan A. Increasing the pool of educational leaders for UCLA. Acad Med. 2006;81:954–958.
11 Cole KA, Barker LR, Kolodner K, Williamson P, Wright SM, Kern DE. Faculty development in teaching skills: An intensive longitudinal model. Acad Med. 2004;79:469–480.
12 Mintzberg H. Crafting strategy. Harv Bus Rev. July–August 1987:66–75.
13 Mintzberg H, Ahlstrand B, Lampel J. The learning school. In: Strategy Safari. New York, NY: Free Press; 1998.
14 Candy PC. Self-Direction for Lifelong Learning: A Comprehensive Guide to Theory and Practice. San Francisco, Calif: Jossey-Bass; 1991.
15 Kaufman DM, Mann KV, Jennett P. Teaching and Learning in Medical Education: How Theory Can Inform Practice [monograph]. London, UK: Association for the Study of Medical Education; 2000.
16 Hatch TF, Pearson TG. Using environmental scans in educational needs assessment. J Contin Educ Health Prof. 1998;18:179–184.
17 Aherne M, Lamble W, Davis P. Continuing medical education, needs assessment, and program development: Theoretical constructs. J Contin Educ Health Prof. 2001;21:6–14.
18 Scholes K. Stakeholder mapping: A practical tool for public sector managers. In: Johnson G, Scholes K, eds. Exploring Public Sector Strategy. Essex, England: Pearson Education Ltd.; 2001.
19 Crandall SJS. Using interviews as a needs assessment tool. J Contin Educ Health Prof. 1998;18:155–162.
20 Steinert Y, McLeod PJ. From novice to informed educator: The Teaching Scholars Programs for Educators in the Health Sciences. Acad Med. 2006;81:969–974.
21 Robins L, Ambrozy D, Pinsky L. Promoting academic excellence through leadership development at the University of Washington: The Teaching Scholars Program. Acad Med. 2006;81:979–983. \yts18
22 Searle NS, Thompson BM, Perkowski LC. Making it work: The evolution of a medical educational fellowship program. Acad Med. 2006;81:984–989.
23 Glassick CE. Reconsidering scholarship. J Public Health Manag Pract. 2000;6:4–9.
24 Wenger EC, Snyder WM. Communities of practice: The organizational frontier. Harv Bus Rev. January–February 2000:139–145.
25 De Villiers MR, De Villiers PJT, Kent AP. The Delphi technique in health sciences education research. Med Teach. 2005;27:639–643.
26 Kaufman DM. ABC of learning and teaching in medicine: Applying educational theory in practice. BMJ. 2003;326:213–216.
27 Mintzberg H. Managers Not MBAs: A Hard Look at the Soft Practice of Managing and Management Development. San Francisco, Calif: Berrett-Koehler; 2004.
28 Patton MQ. Utilization-Focused Evaluation: The New Century Text. Thousand Oaks, Calif: Sage; 1997.
© 2009 Association of American Medical Colleges
29 Westley F, Zimmerman B, Patton MQ. Getting to Maybe: How the World Is Changed. Toronto, Canada: Random House Canada; 2006.