Toomey, Patricia MD, MSc; Lovato, Chris Y. PhD; Hanlon, Neil PhD; Poole, Gary PhD; Bates, Joanna MDCM
Health care professional shortages are common in rural, remote, and northern regions of Canada.1 These shortages also occur in other countries that have a generally adequate physician supply.2,3 Numerous medical schools have developed undergraduate regional training sites to attract medical students who are well suited to practice in underserved areas.4–6 Although evaluation of these programs has predominantly focused on academic outcomes7 and physician recruitment,5,8 interest in assessing the broader impact on host communities has increased. Kaufman and colleagues9 provide evidence of strengthened community development, increased health care access, and improved indices of community health following a medical education initiative in New Mexico. Several Canadian medical schools have explored socioeconomic and other impacts on host communities.10,11 A recent publication by Snadden and colleagues4 describes the importance of considering community impact as part of the evaluation framework of distributed medical education programs.
In 2004, the University of British Columbia (UBC) implemented a four-year medical education program at the University of Northern British Columbia (UNBC), in the community of Prince George (PG),12 which is located in the vast expanse of northern British Columbia, Canada. This program, the Northern Medical Program (NMP), is one of three distributed campuses—all part of UBC’s undergraduate medical education program—that were created to address physician shortages. Further information regarding the structure and operation of the program is detailed elsewhere.4,12 After the implementation of the NMP, we conducted, as part of our evaluation strategy, a qualitative pilot study with eight community leaders representing the education, health, economic, media, and political sectors of PG.13 Our goal then was to evaluate community leaders’ perceptions of the NMP’s early impact. Findings suggested that those 2005 participants perceived a wide range of effects outside the health sector as well as important community-level benefits, including enhanced community pride and development.13 Related studies indicate that the program has increased social capital (defined as community resources used collectively to achieve a common goal) in the medical and wider PG area.14–16
It is unclear whether the findings from the 2005 pilot study were either limited by the small sample size (N = 8) or influenced by the program’s novelty (that study was conducted during the NMP’s first year of operation). Thus, the purpose of the current study was to follow up with a larger sample size and recruit participants from a variety of community sectors whose involvement with the implementation and operation of the NMP varied. A second objective was to identify outcomes relevant to the community, as well as indicators of these outcomes that could be quantified in future studies (see Method for more information on outcomes and indicators).
PG, the NMP’s host community, is located 800 kilometers (about 500 miles) north of Vancouver, Canada. Its economy is largely composed of natural-resource-based industries, such as mining and forestry. According to 2011 census data, 84,232 people live in PG.17 The most recently available survey data from the community (2006) indicate that approximately 10% of respondents self-identify as Aboriginal (i.e., North American Indian, Inuit, Métis, Registered and non-Registered Indian, and members of an Indian Band or First Nation).18 Only 9% of residents 20 to 64 years of age completed university (the provincial average is 62%), and 11% did not complete high school.19 PG has been identified as being underserved in terms of health care professionals.20
This qualitative study comprised 23 semistructured interviews conducted in September and November 2007 (List 1 shows the guide used for all interviews). One of us (P.T.) conducted 20 of the interviews face-to-face and 3 over the phone (for logistical reasons). The study population comprised community leaders, including 6 participants from the first study (2005).
We contacted all 8 of the participants from the pilot study and interviewed those who agreed to be reinterviewed. We used purposeful sampling to identify leaders in Aboriginal health services, social services, allied health, and community development. Using snowball sampling, we then asked participants to provide the names of leaders whom they felt could further contribute to our knowledge base. Even though we reached information saturation21 before interviewing everyone recruited, we decided to conduct all interviews in an effort to ensure an inclusive community perspective.
We audiotaped the interviews and recorded field notes to capture nuances. We transcribed interviews verbatim and anonymized them before entering all data into NUD*IST (6.0; QSR International [Americas] Inc, Burlington, Massachusetts). Analysis, which was an iterative process, commenced after the first interview. Three of us (P.T., C.L., and J.B.) read the transcripts to develop an overall sense of the data, discussing passages and repeatedly coding and recoding text as necessary. First, we used the analytic induction technique to describe content thematically,21 coding data according to an a priori framework based on the five community sectors identified in 2005 (i.e., education, health, the economy, media, and politics). Next, we used an open-coding methodology to uncover new, emerging themes and to compare these with themes described in 2005. We resolved disagreements through discussion and consensus.
To enhance the trustworthiness of findings, we used source and analyst triangulation (i.e., a diversity of participants and analysts of different backgrounds and skill sets).21 In addition, we employed member checking, which included both the production of a document summarizing findings that we sent to interviewees, and the active solicitation of feedback from community leaders who attended the British Columbia Rural and Remote Health Conference in PG.22
In this study, we define “community impact” broadly as a change on any of the sectors affected by the NMP. We refer to “outcomes” as the anticipated effects of the program (e.g., increased research capacity). Finally, “indicators” are specific, observable measures that show progress toward achieving the anticipated outcomes (e.g., the number of faculty publications).
One of us (P.T.) identified outcomes directly from the transcripts and categorized them by sector. Quantifiable indicators, which emanate from the transcripts, reflect outcomes and areas of impact that are important to the community. These were either explicitly named by participants or extrapolated by the research team. We matched the indicators to participant-identified outcomes and, more broadly, to the five broad areas of impact noted above. We achieved consensus on a final list of indicators that appeared to be practical, feasible, and relevant to our research agenda whilst maintaining the integrity of what participants reported.
The research ethics board at UBC approved this study. We told participants that we were conducting a study to evaluate the impact of the NMP on the community, and we obtained informed consent from each of them. We offered no incentives, and participation was voluntary. Audiotapes were stored in a locked file cabinet, and computer-based data analyses were password protected.
Of the 8 participants we invited to participate in 2005, 6 (75%) agreed to be reinterviewed; 2 declined, citing lack of time. Of 20 potential new participants, 17 (85%) agreed to be interviewed. Three individuals from social service organizations declined, claiming insufficient knowledge of the topic. We reached information saturation after 16 interviews, but we still decided to conduct the remaining 7 interviews. On average, the interviews lasted about 40 minutes.
The 6 participants from the 2005 study held positions in business (i.e., local business owners), health administration (i.e., officials from the health authority), medicine (i.e., physicians working in PG), education (i.e., professors and administrators of UNBC and/or the NMP), politics (i.e., politicians at the municipal level), and media (i.e., PG journalists and media personnel). All of these participants were involved with the NMP to some extent (i.e., they were UNBC and NMP administrators; or they were politicians, media personnel, and/or business owners in PG who had lobbied for the NMP, reported on its implementation, and/or assisted with its early operation).
The 17 newly recruited leader participants represented Aboriginal health services (i.e., administrators of local organizations providing resources to Aboriginal communities), social services (i.e., administrators of local organizations providing resources for PG citizens, in general), allied health (i.e., nurse managers and regional hospital administrators), and community development (i.e., officials of local economic development initiatives). Of the 17 new participants, only 2 were formally linked to the NMP in some way (i.e., fundraising, related community initiatives).
The health sector (including Aboriginal health services) was represented by 8 leader participants; the social services and business sectors (including community development) were represented by 5 participants each; politics by 3 participants; and education and media by 2 each. As is common in small, tight-knit communities, many participants held multiple positions, thus representing multiple sectors. For example, a participant could hypothetically be a local politician as well as a business owner who had served on a fundraising committee that works to assist incoming medical students.
Perceptions of impact
The participants described the current and anticipated impact of the NMP as related to education, health services, the economy, business, media, and politics. Considerable overlap between the outcomes reported for the economy and business sectors led to the merger of these two broad areas of impact in our analysis. All 23 participants commented on the impact of the NMP on the education, health services, and economy of PG. Comments related to the media and politics were limited to the leaders representing those sectors (2 and 3 participants, respectively). Perceptions regarding the program’s overall impact on the community were overwhelmingly positive, though participants did report some negative perspectives for each sector save the economy.
Eleven participants believed the NMP represents increased awareness of, access to, and opportunity for local Aboriginal and non-Aboriginal secondary students to pursue medical and allied health education. To illustrate, one observed:
[I]n talking with young people, having the program here … has increased [their] interest in seeking a medical profession of any type…. Because the university has one of the highest percentages of Aboriginal students, there’s [sic] groups of them that are now looking at the medical profession as well. (Participant 09)
Participants discussed the NMP’s impact not only on potential applicants but also on the MD admission process itself. They described how the NMP, as a program on a distributed campus, helped to broaden, but not lower, admission criteria. One community leader commented:
[I]f you look at the statistics of people who go to medical school, it clearly favours, overwhelmingly, students who come from an urban background. The social indicators were all related to urban-type activities. We had to change that … the social indicators, not the grade average. (Participant 04)
Participants felt that the advantages of the NMP extended beyond admissions. Six of them reported that medical students benefit from the program through exposure both to the challenges and rewards of rural practice and to the effects of social conditions on health (e.g., Aboriginal health issues, geographic isolation).
Furthermore, four participants anticipate increases in faculty and student research capacity and productivity over time. To illustrate, one leader noted:
[W]e definitely have significant expectations of what the Northern Medical Program is bringing us in terms of faculty and research interests and students who want to be engaged, particularly in health services and health policy research. Clinical and biomedical research too, but the health services and policy research in a rural, northern environment. We’re looking for that and we’ve already seen some success in that regard. (Participant 6)
Although the majority of the education-related comments were positive, participants did note one negative effect in the education sector. Because of the wide interest in incoming students, 14 participants believed that early abundant publicity and excitement surrounding the birth of the NMP created tension between existing university faculties and the program. One of them said:
Ask yourself what it feels like to be an undergrad social work student or nurse 200 yards from this. One of the few downsides of the Northern Medical Program … is that [UNBC] is still a small, closely knit school … [the NMP] really had a significant divisive impact at some levels and there was a fair bit of ill will generated. (Participant 10)
All participants reported that physicians were more easily recruited to the region at least in part because of the presence of the program. One community leader observed:
Northern Health [the regional health authority] is starting to have greater success recruiting. I believe part of that is being able to say that we have a medical teaching program. People are aware of it. It puts you on the map to have a teaching hospital. So there’s another impact. (Participant 01)
The participants believed the increase in the region’s pool of generalists and specialists would allow patients to remain in PG to receive and recover from health care services. One interviewed leader noted this difference:
We now have six orthopedic specialists in Prince George who are flat out working around the clock. At one point we had one. Now a person who has to have something done is able to stay in the community and recoup with their family. (Participant 20)
The leaders participating in the interviews anticipate that these newly recruited physicians will remain in PG as a result of increased interprofessional stimulation and a positive work environment. Two commented:
I think physicians derive significant professional strength by having other individuals to share their practice, to help deal with the pressures of being on call, and to maintain the professional stimulation, in terms of bringing back or being exposed to new ideas in the field … it has an intrinsic value, having more colleagues in practice with them. (Participant 11)
I actually think training environment also serves to retain staff. It is interesting. People in health care go through training programs where they train in hospitals … and by and large I think they want to give back. So that retention, that ability to teach, to pass on keeps you fresh, keeps you current, and I believe it helps retain staff. (Participant 01)
Participants also anticipate retention of NMP graduates, particularly family physicians, on graduation. One noted,
[T]he ability to train physicians and particularly to retain some of them later to residency programs that might come to the North was seen to be a huge strategic gain in making sure that our communities had access to physicians in the next decade. (Participant 10)
Continual improvement of access to family physicians and specialists will, our participants presumed, decrease reliance on emergency services for primary health care. One leader commented:
There’s nothing worse, as we know, than somebody having some kind of major illness and never having established that relationship with a health care provider. So the program is about trying to make sure that doesn’t happen anymore, trying to make sure that people aren’t using the emergency walk-in clinics as their primary source of health care. Because there’s no continuity, there’s no plan, there’s no relationships. (Participant 02)
One health services leader further anticipates medical teaching to encourage evidence-based thinking in physicians, to the benefit of patients.
So [the NMP] challenges or supports the continued professionalization of doctors and even nurses … it brings that level of inquiry to think about, “Well, what is the evidence out there for doing things? We’ve always done things this way, because we’ve always done that. But is there evidence to support that?” There’s [sic] challenges to habits people have gotten into, whether it’s medicine or nursing…. So I think having students increases the learning for everybody. I think that whole level of inquiry, that whole kicking things up a notch or two, is just great for patients. (Participant 22)
The participants felt, however, that the price of these anticipated improvements to health care would be added strain on health system resources. For example, one commented:
I just know that the [Northern Health Authority] is a small organization and that means, in our organization, that people wear multiple hats, and that goes for physicians as well as our internal staff. Anything like [the NMP] does stretch our capacity and infrastructure. (Participant 06)
Participants believed that the presence of medical students in the regional hospital would not only reportedly reduce physical work space for hospital staff but also might decrease physician incomes, assuming that teaching students reduces time available to see patients. One community leader reported:
[Teaching] is a time-consuming thing, so therefore [a physician] is not giving his time to his patients and probably losing some income. And so if medical coverage in this area is so important, I think the government has to come to us and say “Ok, if you are going to supervise x number of residents, then there will be x amount of money for you to make up for what you’ve lost in your practice.” (Participant 03)
Economy, business, and community development
According to 17 leaders, the NMP is indirectly attracting new businesses and workers to the region. Two of them described anecdotal evidence whereby migration from elsewhere in Canada was partially due to a sense of security in the quality of and access to health services associated with the presence of the NMP. One leader credited program success with more than just stimulating business opportunities:
The program means a lot just in terms of the credibility of the North and these communities. The fact that we can have a program of this sophistication operating here successfully does legitimize other potential opportunities. It shows that if you can run a Northern Medical Program and train doctors here, why can’t you establish other successful businesses?… It shows that northerners can, in fact, offer world-class services. (Participant 08)
Eight leaders further described the program’s impact on community development. For example, one participant commented:
By bringing professionals to town, by expanding that base of professionals, it supports things like arts and sports as well. So, it’s an economic base. It’s a population that has a disposable income that can go out and go to plays and go to hockey games … buys new cars, all of that. So it’s different than some of the other types of economic development. It’s not cyclical like the resource industry, so the program is a stabilizing influence on Prince George. (Participant 01)
Five leaders specifically mentioned community efforts to retain the influx of professionals. One remarked:
One of the things that Prince George was battling for years was the rough logging community type of personality … as soon as the university and the program opened, that started to change. We saw that the people coming in, in order to retain them, required a more sophisticated entertainment: the symphony, the theatre. So we have a very nice mix of all those things right now and we’re working on putting together a performing arts facility here. (Participant 20)
As mentioned above, all of the comments regarding economic and community development were positive.
Twelve leaders described how early political backing united politicians at both the municipal and the provincial level, which boosted momentum to ensure continued NMP support. To illustrate, one participant discussed how community leaders lobbied to expand the region’s postgraduate residency programs to accommodate graduating NMP students:
And the provincial support, administrative health support, colleagues tell me that there has been an increase in residency positions. So that’s an indication that [political parties] have fulfilled their part of the deal and certainly … put this all together. So at the political level and at the level of the community partners, I think there is very solid support. (Participant 04)
Furthermore, ground-level support from the region’s citizens led to the creation of the NMP Trust, a fund created to assist students in financing their education.
Although most of the comments about the political sector were positive, four leaders reported that some politicians used the NMP as a bargaining tool to strengthen their own political positions. One participant noted that
people on one side of the house or the other are always using health care, and in this case, the [NMP], as a political football. (Participant 09)
Various parties, political and otherwise, also took advantage of the opportunity [the NMP] to hype this into a world-changing event. (Participant 10)
The NMP continues to receive abundant media coverage at the local, host university, and to a lesser extent, provincial and national levels. One participant observed:
[E]very year, when the new class starts in January, we have a meet-and-greet with the local media. We get 100% attendance from the local media. It’s incredible. And the students, I think, get a rise out of it. (Participant 15)
As PG citizens, two leaders from the media sector reported a strong sense of responsibility to convey the significance of the program to the public. One expressed the sentiment this way:
We knew that [the NMP] was history in the making at the time, and you don’t always know that. But we knew that then. This was history in the making. So [we] were very cognisant in making sure we were documenting everything and making videos. (Participant 15)
Many participants (n = 14) noted that the prominent media attention during the NMP’s implementation may have overshadowed UNBC faculties; two participants believed this attention overshadowed the other UBC MD distributed campuses as well.
Outcomes and indicators
Table 1 lists outcomes and quantifiable indicators that we judged to be relevant to our research agenda and reflective of participant perceptions. We believe the indicators reflect data that evaluators can feasibly collect and quantify. We identified 16 indicators for three sectors (i.e., education, health services, and the economy). Of these 16 indicators, 10 pertain to health services and relate specifically to physician recruitment and retention and to population health indicators; 3 indicators are relevant to education and focus specifically on enrollment and scholarship, and the remaining 3 indicators pertain to economic impact—specifically, capital investment and new businesses. We did not include indicators from the political and media sectors because we deemed the outcomes that participants described to be impractical and unfeasible to quantify in future studies.
As explained above, some of the indicators reflect outcomes participants explicitly mentioned, whereas we derived others, by extension, from participant responses. For instance, an indicator directly corresponding to the anticipated, explicitly mentioned outcome of “increased physician recruitment” is “number of physicians recruited in a given time period”; and we identified widely used population health indicators, such as mortality, as proxies for the participant-expressed outcome of “improved population health.”
Discussion and Conclusions
The results of our interviews with 23 local leaders suggest that after 3 years, the broadly perceived impact of the NMP, a regional undergraduate medical education program, on its host community extends beyond the traditionally evaluated outcomes of physician recruitment and retention and student performance to the health, education, and economic sectors.
Comparison of current findings with those from the pilot study in 200513 suggests that initial areas of perceived program impact on education have endured. In both studies, participants report that since the NMP’s implementation, local students, including those of Aboriginal or First Nations’ communities, have conveyed greater ambition to pursue a medical career. UNBC administrators theorize that the NMP enhances the credibility of its host university as a postsecondary institution and amplifies student interest in the bachelor of health sciences undergraduate (i.e., baccalaureate) degree program as a gateway into medical school.
Participants also reported that through NMP-mediated changes in UBC’s admissions process,23 more rural and northern Canadian students are not just expressing interest in medicine, but also being accepted into the health sciences program, as an entrance into medicine. Research has shown that rural background and experience in rural settings are associated with physician retention in underserved areas.24,25 Individuals with rural backgrounds and those of Aboriginal descent are severely underrepresented in all health disciplines.26 Increased interest in pursuing medicine and increased admissions of rural, northern, and Aboriginal students could, thus, translate into more rural-raised and Aboriginal physicians in underserved areas.
Another finding that aligns with our 2005 study is the anticipated strain on the health care system, especially on the regional hospital, as students moved into clinical clerkship in their third year.13 Both new and previously interviewed participants reported that the demands of training have strained the physical space allotted to regional hospital physicians and staff, decreased revenue, and increased workload. These findings align not only with initial analyses from administrators that show that a greater number of inpatient diagnostic tests have been ordered for learning purposes27 (the cost of which the hospital absorbs) but also with research that indicates that PG physicians who take on teaching responsibilities have an increased workload,27 as do physicians in other community hospitals that have been converted to teaching hospitals.28,29
The primary health-services-related outcomes our participants noted were increases in the region’s physician workforce, greater access to health care, and improved health status of the population over time. These results corroborate previous findings linking NMP-associated physician recruitment with additional opportunities for physician communication and interaction.27 The findings also support previous research showing that higher degrees of physician-to-physician contact are associated with increased physician retention—as are economic development and political support.30
The presence of the NMP, according to our participants, has also led to, among other changes, the development of novel cultural amenities in the community. This development within the community aligns with previous findings.16 Although this other research reports development,16 the extent to which the presence of NMP has precipitated economic growth and cultural expansion, described by our participants, was unanticipated. Future researchers should carefully track all economic development.
Improving the health status of northern British Columbians in the long term has been a widely anticipated impact of the NMP. Reported effects of the program on health services include not simply increased physician recruitment but, importantly, more long-term patient–physician relationships (which, in turn, lead to superior compliance with treatments) and greater continuity of care, which can all contribute to improving health.31 The program may also improve health by influencing an array of developments in education, politics, media, and the economy, all of which are documented determinants of health.32,33 Therefore, the presumption that the program will succeed in improving the health status of the community is plausible.
A second aim of this study was to identify and construct a list of outcomes and quantifiable indicators that can be feasibly tracked over time. The largest concentration of quantifiable outcomes and indicators pertains to improvements in health services. This concentration is not surprising given the origins of, and the needs that engendered the development of, the NMP. Collecting and communicating the data to measure NMP’s outcomes as prescribed by our indicators over time is important.
Although we sought to identify quantitative indicators, we recognize that some indicators are not readily quantifiable (e.g., tension with[in] the host university, political impact). A qualitative approach allowed us to identify meaningful positive and negative outcomes that may not have otherwise come to the forefront. This duality is consistent with previous work suggesting that the societal impact of academic clinical partnerships varies from highly quantitative to qualitative.34
Our next step is to conduct a feasibility study to assess the availability and usability of institutional, local, and provincial datasets that can be matched to the indicators we identified. We plan to conduct a longitudinal study that uses both qualitative and quantitative methods to evaluate long-term outcomes and program impact.
One limitation of this study relates to the fact that the NMP was implemented in 2004; the impact of the program on physician workforce will not be fully known for a decade or more. Secondly, perceptions of community leaders are not necessarily representative of PG citizens. Nevertheless, we think that soliciting the perspective of leaders from a variety of sectors has provided us with a broad, overarching impression of the NMP’s impact that could not have been elucidated by sampling the general public. Despite these limitations, results of this and other research11–13 provide credible evidence that the NMP has contributed to community changes that span beyond increasing physician workforce.
In summary, community leaders perceive broad, multisector impacts of a regional undergraduate medical education program on its medically underserved host community located in northern British Columbia, Canada. From our dataset we have identified areas of impact and a set of outcomes and indicators that are important to community leaders and feasible to measure quantitatively. Results may be applicable to other medical education programs with regional sites operating in similar rural, remote, and/or medically underserved regions. In the long term, it is reasonable to expect that the NMP will lead to improvements in health, including better access to health care and improvements in other indicators of population health. Future studies will reveal whether expectations align with actual change.
Acknowledgments: The authors wish to thank the participants interviewed in this study for their contributions. We also wish to thank Dr. Dan Pratt for consultation regarding interview procedures, Dr. Kevin Eva for helpful comments on the manuscript, and Ms. Joan Gray and Ms. Rita Quill for editorial assistance.
Funding/Support: Research funding was provided by the Canadian Institutes of Health Research Canada Graduate Scholarship–Master’s Award (Project Grant #17W33879). This study was also supported by the Evaluation Studies Unit, Faculty of Medicine, University of British Columbia.
Other disclosures: None.
Ethical approval: This study was approved by the University of British Columbia Behavioural Research Ethics Board.
Previous presentations: The authors have previously presented findings from this study as follows:
* Evaluation of the impact of the Northern Medical Program on the community: Perceptions of community stakeholders. Canadian Conference on Medical Education (Poster P-07). May 2–6, 2009, Edmonton, Alberta, Canada.
* Evaluating the impact of the Northern Medical Program on the community: Perceptions of community leaders. Knowledge in Motion Inaugural Conference. (Poster) Oct 16–18, 2008, St. Johns, Newfoundland and Labrador, Canada.
* Evaluating the impact of the Northern Medical Program on the community: Perceptions of community leaders. BC Rural and Remote Health Research Network Inaugural Conference: Building Research Together. Abstract 14. May 13–14, 2008, Prince George, British Columbia, Canada.
* The Northern Medical Program: Bridging medical education and the community. Knowledge in Motion Inaugural Conference. Session: Role of teaching in outreach—Moving beyond the degree/diploma and making the most out of teaching. Oct 16–18, 2008, St. Johns, Newfoundland and Labrador, Canada.