Laos, or Lao People’s Democratic Republic (Lao PDR), is one of the least developed countries in east Asia. Its population is estimated at six million people, and approximately 80% of the population lives in rural areas. The health status of the Lao people is considered by the World Bank as being among the worst in the region, with social indicators comparable with those of countries in sub-Saharan Africa.1 Life expectancy at birth for the total population is 55 years (compared with 78 years in the United States). The infant mortality rate is 87.1/1,000 live births (compared with 6.4/1,000 live births in the United States). More than 75% of the Lao people live on less than two U.S. dollars per day. An estimated 40% of the population lives below the national poverty line.2 Malaria, pneumonia, and diarrhea—preventable and treatable conditions—remain leading causes of morbidity and mortality.3 Although socioeconomic and health indicators are improving, Lao PDR is ranked 135 out of 177 countries by the United Nations Health Development Index and remains among the world’s least developed nations.4
Developing a New Family Medicine Curriculum
In 2004, the Faculty of Medical Sciences (FMS) at the National University of Laos (NUOL) and the Lao Ministries of Health and Education joined with the University of Calgary Faculty of Medicine to develop a new primary care program. In particular, it was the Lao Ministry of Health that first identified that a resource of well-trained primary care Lao physicians was needed to improve the health status of the Lao people. With this in mind, representatives from the FMS-NUOL, the ministries, and the University of Calgary developed a two-year primary care training program called the Family Medicine Specialist Program (FMSP). According to the proposal, the first year would be a hospital-based curriculum with rotations in internal medicine, surgery, pediatrics, and obstetrics–gynecology in the capital city of Vientiane. The second year of the program would be a community medicine experience in which trainees would work in the provincial hospitals, district health centers, and rural villages. The expressed goal of the FMSP was for Lao trainees to acquire practical, hands-on experience in the care of patients, so that they would gain the competence and confidence to practice in isolated rural settings independently with minimal assistance. Graduates of the FMSP would be labeled “specialists” to confer status of the program in the Lao medical community and to cultivate the recognition of primary care as a medical discipline in its own right. The intended overall outcome for the FMSP was “to train the family doctor to be a care provider, health promoter, decision maker, community health leader, communicator and manager.”5
The international community has advocated a mandate to deliver primary health care services throughout the world.6 This need is especially grave in developing nations. In Lao PDR, medical curriculum development faces many challenges that are specific to its low-income status. Obstacles include a lack of health care professionals with adequate training and remuneration, the poverty and the predominantly rural location of the general population, a lack of basic resources in health services, a lack of educational resources for professional development, a lack of a professional infrastructure for medical personnel, a lack of informational infrastructure in medical institutions, the poor health-seeking behaviors of the general public, difficult physical access to rural villages and health care centers, and many others. Lao PDR also suffers from barriers to the global development of family practice, such as failure to appreciate primary care as its own specialty, failure to integrate clinical care with community health services, preference for selective and subspecialized care over comprehensive care, disproportionate funding toward tertiary care services, development of urban centers over rural services, lack of training opportunities and leadership in primary care, and a medical education system biased toward subspecialty postgraduate training.7
The only medical school in the country is located at the NUOL. At graduation, a small number of students pursue further training in a foreign country or they enter one of a few small postgraduate specialty-training programs in Lao PDR.8 Other students resort to nonclinical employment within the urban center of Vientiane. Very few medical graduates go on to work independently in district health centers or provincial hospitals, where care is most needed. In 2002, the Lao government assessed its status of medical education in the country and reported severe needs in the area of health human resources development. Medical graduates were found to be unwilling and unable to practice in the remote and rural areas of the country because of a lack of adequate training and, hence, a lack of ability and confidence to provide frontline primary care.9 Before the launch of the FMSP in January 2005, there was no educational training program dedicated to providing primary health care in the country.
A Need for Needs Assessment
The sizable task of defining a new FMSP curriculum that reflected the health care needs of the country’s tropical Asian environment and that was feasible to achieve Lao PDR’s social, cultural, and economic context required innovative methodology. The first step of planning a new medical curriculum is to identify the health care needs that the program intends to meet.10 The first year of the FMSP was developed to utilize the existing clinical teachers in the urban hospitals where medical training was already taking place and where teachers were familiar with the presence of trainees. Furthermore, a lack of trained clinical teachers in the rural areas prevented use of that underserved community at the outset of the program. Because the initial implementation of the program was dependent on the existing allocation of human resources, it became apparent that preparing resources for the first- and second-year curricula would be different tasks. Developing the first-year curriculum required identifying objectives in four existing specialty areas (internal medicine, pediatrics, obstetrics–gynecology, and surgery) for the purpose of increasing the clinical competence of the trainees in curative medicine.
A modified Delphi
We implemented a modified Delphi technique modeled after a study described by Lawrence et al11 to serve as the needs assessment for the first-year curriculum. The technique consisted of the following features:
1. An “expert panel” of Lao medical education leaders who would establish a preliminary set of knowledge and skill objectives to be met by the hospital-based curriculum.
2. Consultation with Lao clinicians and teachers to assess the health care priorities relevant to the cultural and resource needs of Lao PDR and to define the competencies to be acquired by an FMSP trainee.
3. Development of prioritized lists for the teaching and learning of knowledge and skills in the FMSP. Such lists would serve to guide administrators, teachers, and trainees toward successful achievement of the program objectives.
The Delphi technique has been used extensively in the health care field in many modified and adapted forms.12–14 Generally, four key characteristics are described: anonymity, iteration with controlled feedback, statistical group response, and expert input.12
The expert panel
The Lao Ministry of Health and FMS leadership at NUOL established an expert panel called “the Key Leaders” to tackle multiple curricular innovations in medical education in Lao PDR, including defining the curricular objectives of the FMSP. This group consisted of senior-level officials from the Ministry of Health, the Ministry of Education, and the National Institute of Public Health; the dean and associate deans of the NUOL FMS; the directors of three central hospitals in Vientiane; and heads of departments in surgery, internal medicine, pediatrics, and obstetrics– gynecology. This group generated a list of recommended clinical objectives based on the panel members’ long-standing experience in Lao clinical medicine and as administrative stakeholders of this new national program. The group divided this list into domains of knowledge and skills for the four specialty areas that were the focus of the first-year curriculum: surgery, internal medicine, pediatrics, and obstetrics–gynecology. University of Calgary consultants in family medicine, surgery, and public health collaborated with the group to ensure that crucial aspects of basic medical care were considered according to international standards. This step was the first iteration of the Delphi process.
Consultation with clinicians and teachers
We identified three groups of clinicians to form the second iteration of the Delphi process:
1. Clinical specialists in surgery, internal medicine, pediatrics, and obstetrics–gynecology.
2. Emergency physicians.
3. FMS teachers with previous rural or provincial clinical experience.
For this step, we had to keep in mind that no existing primary care practitioners were officially recognized in Lao PDR. Only specialists serve the urban Vientiane hospitals. Emergency physicians are few and serve the large outpatient departments at the hospitals. Physicians in rural and provincial areas were difficult to access and identify for this needs assessment. Medical practitioners exist in outlying facilities, but the government does not acknowledge them as providing any formal primary care service. Therefore, we identified existing medical teachers with previous clinical experience at the district and provincial levels to represent a rural health care perspective in the assessment. These three clinical groups were considered important stakeholders of the program; each group offered a different perspective of the health care needs to be addressed by the new curriculum.
We constructed a questionnaire based on the list of objectives created by the Key Leaders. The clinicians and teachers we consulted were asked to rank each knowledge and skill objective on the basis of priority for the health care needs of the Lao population and the level of competence to be achieved by an FMSP graduate. We asked these groups to rate knowledge objectives on a four-point rating scale ranging from “none” (not required of an FMSP graduate) to “in-depth” (comprehensive knowledge required of an FMSP graduate). The groups rated skills objectives on a similar scale appropriate for investigative/procedural skills, ranging from “irrelevant” (skill not required of an FMSP graduate) to “proficiency” (mastery of this skill required of an FMSP graduate).
We administered the questionnaires with an introduction sheet describing the FMSP and instructions for the questionnaire within small focus groups or in individual interviews. We asked specialists to participate in the questionnaire relevant to their specialty objectives, and we asked emergency physicians and the teachers with previous rural/provincial clinical experience to participate in the questionnaires for all four specialty areas. The material was presented in two languages, English and Lao. We also elicited input from the groups through written responses within the questionnaires and through the interviews.
Developing priority lists
As the final component of our Delphi technique, we created prioritized objective lists for teaching and learning by combining the responses of the above three clinician groups, collectively called “the expert clinician group.” We scored the responses for each questionnaire according to a weighted averaging system.11 We assigned numeric equivalents to the scaled responses to the knowledge questions: 0 for “none,” 1 for “familiarity,” 2 for “moderate,” and 3 for “in depth.” We calculated responses to skills questions similarly, using 0 for “irrelevant,” 1 for “observed,” 2 for “performed,” and 3 for “proficiency.” For each item, we determined the number of responses in each of the four levels, multiplied that number by its assigned numeric equivalent, and then summed the products and divided them by the total number of responses to determine a mean score for the item (range of 0.0–3.0). Using this method, we calculated mean scores for all the objectives, which allowed them to be ranked by the relative overall importance attributed by the respondents.
We conducted meetings and interviews with the clinical specialists, emergency physicians, and teachers with rural/ provincial experience from May 18 to July 1, 2004. We accepted responses to questionnaires until July 1, 2004. After we had gathered all the information, we produced a “priority list” for the teaching and learning of knowledge and skills in each specialty area of the first-year curriculum. We also calculated the mean scores of the objectives for each respondent group. We compared the mean scores across groups, using the Kruskal-Wallis test for nonparametric values. Objectives that we found to have differences that were statistically significant (P < .05) across groups were then compared between groups using Mann–Whitney tests for nonparametric values comparing two means. All statistical analysis was performed using software package, SPSS version 10.
Finally, we reviewed the priority lists with the clinical specialists and FMSP leadership to confirm consensus for these objectives as the basis of the curriculum. We did ask clinicians for comments and suggestions, both on the questionnaires and in interviews, but, in our opinion, describing these suggested additions is beyond the scope of this paper.
Results of a Consultative Approach
Of the 111 individuals we considered eligible to participate in our interviews and questionnaires, 95 were clinical specialists, 7 were emergency physicians, and 9 were teachers with rural/provincial experience. A total of 143 questionnaires were returned by 96 respondents, and we divided these into groups by the four specialty disciplines (see Table 1). The individual response rates for each group of participants per questionnaire administered were 84.2% (80/95) for the clinical specialists (subgroups: surgery, 84% [21/25]; internal medicine, 89% [33/37]; pediatrics, 87.5% [14/16]; obstetrics/gynecology, 70.6% [12/17]); 96.4% (27/28) for the emergency physicians; and 100% (36/36) response from the teachers with rural/provincial experience. The total response rate was 90% (143/159). The response rate was notably high, as the majority of questionnaires were distributed during a focused individual or group interview. Only 2.6% of questionnaires were distributed without an interview. We defined nonresponders as participants who did not return a questionnaire despite being present for an interview, who returned a ruined or defaced questionnaire, or who could not be contacted during the study period. Fifteen clinical specialists qualified as nonresponders. One emergency physician completed all but the pediatrics objectives.
Tables 2 and 3 are examples of the prioritized lists of objectives we obtained. We recognized that some knowledge and skill objectives, despite their perceived higher level of importance by the authors, were ranked low in priority by this method. This possibly reflected either unwillingness or an inability of clinicians and teachers to teach the objective to trainees.
Comparisons between groups’ responses revealed some notable differences across the specialty areas, confirming the benefit of consulting the different clinician groups. For example, within the pediatrics knowledge objectives, the teachers with rural/provincial experience rated some objectives higher than the pediatricians did: “Knowledge to perform neonatal resuscitation,” “Assess and treat severe dehydration,” “Assess and manage major pediatric trauma and burns,” “Assess and manage a child presenting with failure to thrive,” “Assess and manage jaundice in neonates and children,” “Diagnose and manage meningitis,” “Diagnose and manage anemia,” and “Diagnose and manage HIV infections in children.” For the surgery skills objectives, the emergency physicians ranked the following objectives lower than the surgeons did: “Do anoscopy/rectoscopy,” “Apply cervical spine precautions before and during airway management of a trauma patient,” “Reduce joint dislocations,” and “Administer regional blocks (i.e., digital).” We attributed such differences to the different priorities allotted to the respective objectives by the clinical groups. In this instance, we assume that pediatric health care needs in the rural communities likely differ from those of the urban hospitals, and that emergency physicians place less emphasis on some procedural skills than do the surgeons/specialists.
Assessing Our Needs Assessment
By using multiple stakeholder groups in a consultative process, we achieved a robust assessment of the needs to be met by a new primary care training program in Lao PDR. Each group we consulted contributed a unique perspective (ranging from administrative to rural) to the future FMSP curriculum. Many insights emerged through this interactive process, identifying potential challenges that might be encountered in the teaching and learning of the new FMSP. The educational program that evolved was a “made in Laos” solution to a lack of available primary care; the program was created by Laotians for Laotians. The role of the University of Calgary consultants was mainly to help with process, not to formulate content.
One consistent dilemma the respondents faced was the polarity between wishing to teach what ought to be learned versus what can be practiced within the restricted resources of rural Laos. Some clinicians chose to rank objectives on the basis of the availability of the resources in the rural health centers, whereas other clinicians prioritized the objectives on the basis of what should be taught according to the health care needs of the rural population, despite the lack of resources. One prime example was the lower ranking of cardiopulmonary resuscitation (CPR) in both the surgery and pediatrics objectives. Some teachers were unwilling to teach CPR because of a lack of appropriate intensive care services to support a resuscitated patient or because of a lack of their own experience in successful recovery of patients with CPR, even in the urban hospitals. Furthermore, some clinicians expressed concern about teaching invasive procedures such as the insertion of a chest tube for the diagnosis of pneumothorax. Senior surgeons identified that the religious beliefs of the Lao people can prevent the use of lifesaving external devices; in the Lao Buddhist religion, the body is viewed as a “holy temple,” and patients and family members would often consider invasive maneuvers a violation of their beliefs.
A criticism of the curriculum that was acknowledged by the Key Leaders group in the first stage of our assessment was the lack of attitudinal objectives present in the curriculum. Attitudes are more difficult to define within a foreign or culturally relevant context, and although they are considered important, this area was deferred as more appropriate to develop within the community-based second-year curriculum. Furthermore, we did not consider other international and regional curricula immediately applicable to the FMSP, because importing a foreign curriculum would be unlikely to serve the health care needs in Lao PDR within its unique political climate and resource limitations.
Language equivalency and translation of the study materials from English into Lao represented a weakness in this assessment. Lao physicians and teachers can be fluent in multiple languages, but not necessarily in English. The only official language of the country is Lao. Translation of the questionnaire and FMSP objectives was potentially hindered by the limited medical vocabulary available in the Lao language, resulting in objectives that were “simplified” in their content.
A further weakness of this assessment was the inability to include existing rural practitioners. Involving practicing physicians from rural centers would have further contributed to the consultative process. Moreover, many Delphi studies incorporate the use of existing or recently graduated trainees as a group to be consulted in the needs assessment. However, because the FMSP is a new program, such a trainee group was not available for consideration. We did consult medical students (pretrainee level), but their contributions were not considered at an “expert” level for the purpose of needs assessment.
The strength of this study is that the process served a dual purpose. First, it identified the health care priorities to be addressed by the FMSP. Second, it functioned as a faculty-development tool and introduced not only the Lao medical education leadership to novel methods of curriculum development and needs assessment, but it also involved the clinicians who would be the new teachers of the program. This study actively engaged the available stakeholders in the present health care system to participate in the development of a new curriculum. By doing so, the participants gained a sense of ownership of the project. The teachers realized they would be accountable for the content and skills that they would be teaching to the new trainees. The high level of participation in the study led to increased cooperation and enthusiasm of the participants toward constructive development of the new program.
In addition to providing faculty development, the process used a key characteristic of the Delphi method—anonymity. Within the Lao medical education system, hierarchy between teachers and clinicians can play a dominant role, which may hinder the accuracy of a needs-assessment process. By involving all available clinicians (junior and senior physicians) in the Delphi process, each participant was able to contribute equally in a nonthreatening environment and, thus, have a voice in the new curriculum.
The needs assessment we performed was rooted in developing relationships with the relevant stakeholders to gather information that would otherwise be immeasurable and unattainable by standard quantitative means. Statistical data in Laos is fragmentary and unreliable at best, even at the national level. The qualitative data obtained through interacting with the Lao clinicians and educators uncovered unique issues, such as creating public awareness for the program, so that rural Laotians will attend local medical centers, which are currently underused. Similarly, participants raised the issue of the need to increase provision of basic equipment and resources to improve rural health care facilities, so that future FMSP graduates will be able to provide adequate and high-quality care to the population. Dialogue among the participants revealed multiple insights into future challenges facing the implementation of the FMSP.
There have been many advantages to producing the prioritized lists for teaching and learning at the outset of the FMSP. Study guides and logbooks based on the priority objective lists were developed to support both trainers and trainees. Using these lists, teachers have been directed to teach more on topics of greater clinical relevance and less on topics that are rare within the Lao context. The lists continue to serve as a guide for trainees of the common medical conditions that are considered prevalent in Laos and help focus skills training on procedures considered relevant in the rural environment, as determined by this needs assessment. The learning needs of trainees have been highlighted for the faculty so that efficient use of existing resources and learning opportunities can be maximized. Our results also have elucidated areas for targeting future faculty development. For example, the low ranking of resuscitative measures by specialists may warrant further training at the faculty level. We identified a need for more faculty training in student-centered learning strategies, because the priority lists emphasized the learning needs of the trainees. This was considered a major shift in teaching methods for many of the teachers. Finally, the priority lists of knowledge and skills have been incorporated as a component of formative and summative assessment for the program. FMSP leaders plan to use the prioritized lists as a core component of program evaluation when the first graduates complete their training in December 2006, which had not yet occurred when this article went to press.
The Next Steps: Meeting the Needs
The FMSP in Lao PDR is a new endeavor in Lao medical education designed to improve rural primary health care services in this low-income country. The challenges facing Lao PDR in the realms of medical education and the attainment of adequate health care for all Lao citizens are numerous. The foundation for developing a new primary care curriculum rested in identifying the health care needs and priorities to be met by the ultimate product of the training program, and our innovative needs assessment was used to develop the hospital-based curriculum for the first year of the FMSP. Our experience has shown that a culturally relevant needs assessment for defining a medical curriculum can be performed under circumstances of extremely limited reliable data and minimal resources. Future program evaluation will determine whether FMSP participants indeed acquire the competencies and confidence they need to provide primary care in rural areas as defined by this tailor-made needs assessment. We hope that other low-income countries that face challenges similar to those in Lao PDR will find our tailored approach to curriculum development adaptable to their environment with comparable gains.
The authors would like to acknowledge the significant contribution of Dr. Clarence Guenter, professor emeritus of the University of Calgary, for his mentorship and leadership in this project, and without whom this work would not be possible. Also, the authors would like to thank Mr. Greg Kanashiro and Dr. Ketsomsouk Bouphavanh for their managerial and clerical support. This project has received financial support from the Government of Canada provided through the Canadian International Development Agency (CIDA) and managed by the Association of Universities and Colleges of Canada; the Government of the Grand Duchy of Luxembourg; the World Health Organization; the University of Calgary, Canada; and many private donors.
3 Ministry of Health, State Planning Committee. Report of the National Health Survey: Health Status of the People in Lao PDR. Vientiane, Lao PDR: Ministry of Health; 2001.
5 Nammavongmixay K. Draft Curriculum of Family Medicine Program, A Joint Collaboration between Faculty of Medical Sciences, National University of Laos, Ministry of Education and Ministry of Health. Lao PDR; 2004.
6 Garcia-Barbero M. Medical education in the light of the World Health Organization Health for All strategy and the European Union. Med Educ. 1995;29:3–12.
7 Haq C, Ventres W, Hunt V, et al. Where there is no family doctor: the development of family practice around the world. Acad Med. 1995;70:370–380.
8 Gordon G, Vongvichit E, Hansana V, Torjesen K. A model for improving physician performance in developing countries: a three-year postgraduate training program in Laos. Acad Med. 2006;81:399–404.
9 Guenter C. Association of Universities and Colleges of Canada–Canadian International Development Agency (AUCC-CIDA), University Partnerships in Cooperation and Development Program (UPCD) Revised Project Description, Tier 2, Lao Human Resources for Community Health, Canada. Re: The Lao Health Master Planning Study. Canada; 2004.
10 Harden RM. Ten questions to ask when planning a course or curriculum. Med Educ. 1986;20:356–365.
11 Lawrence PF, Alexander RH, Bell RM, et al. Determining the content of a surgical curriculum. Surgery. 1983;94:309–317.
12 Goodman CM. The Delphi technique: a critique. J Adv Nurs. 1987;12:720–734.
13 Stewart J, O’Halloran C, Harrigan P, Spencer JA, Barton JR, Singleton SJ. Identifying appropriate tasks for the preregistration year: modified Delphi technique. BMJ. 1999;319:224–229.
© 2007 Association of American Medical Colleges
14 Mullen PM. Delphi: myths and reality. J Health Organ Manag. 2003;17:37–52.