Secondary Logo

Share this article on:

Strengthening Public Health Management Capacity in Vietnam: Preparing Local Public Health Workers for New Roles in a Decentralized Health System

Do, Mai, Hoa, MD, PhD; Bui, Thi Thu, Ha, PhD; Phan, Tuong, PhD; Nguyen, Hoang, Long, MD, MPH, PhD; Duong, Thuy, Anh, PhD; Le, Bao, Chau, MD, MPH; Pham, Thi Thuy, Linh, MPH; Nguyen, Minh, Hoang, MPH; Nguyen, Phuong, Thuy, MPH; Malison, Michael, MD, MPA

Journal of Public Health Management and Practice: March/April 2018 - Volume 24 - Issue - p S74–S81
doi: 10.1097/PHH.0000000000000755
Research Full Reports

Health sector decentralization has created an urgent need to strengthen public health management capacity in many countries throughout the developing world. This article describes the establishment of a national management training network in Vietnam that used Project-Based Learning to strengthen management competencies of HIV program workers and linked training to measurable improvement in HIV/AIDS public health program outcomes. Skills were taught using a combination of classroom learning and mentored fieldwork. From 2005 to 2015, 827 HIV/AIDS program managers were trained with this method throughout Vietnam by trainers in 3 regional training centers. A total of 218 applied learning projects were carried out by trainees during this period; 132 resulted in measurable improvements in HIV/AIDS program outputs, and 86 produced well-organized plans for implementing, monitoring, and evaluating HIV/AIDS intervention strategies. Vietnam's management training network represents an important advancement in public health workforce development that helps prepare workers for new roles and responsibilities in a decentralized health system.

Institute of Health Management Training, Hanoi University of Public Health, Hanoi, Vietnam (Drs Do, Bui, Phan and Le, Ms Pham, Mr. Nguyen, and Ms Nguyen); Vietnam Ministry of Health, Vietnam Administration for HIV/AIDS Control, Hanoi, Vietnam (Drs Nguyen and Duong); and International Health, Emory Rollins School of Public Health, Atlanta, Georgia (Dr Malison).

Correspondence: Michael Malison, MD, MPA, International Health, Emory Rollins School of Public Health, 1560 Clifton Rd, Atlanta, GA 300333 (michael.malison@gmail.com).

The authors declare no conflicts of interest.

The authors wish to thank Janna Brooks, Sara Andrist, Asia Nguyen and the many others from the US Centers for Disease Control and Prevention (CDC) who provided technical support for this project. Funding for this project was provided by a grant from the President's Emergency Program for AIDS Relief (PEPFAR). Dr. Malison, the founder and former Director of CDC's Sustainable Management Development Program, was supported to assist in the preparation of this manuscript by a grant from the Fulbright Foundation US Scholar's Program.

Vietnam's

(renovation) market-oriented reforms implemented in 1986 had a profound impact on economic and social development.1 All state-run enterprises were affected; collective agricultural production effectively ceased, as did the majority of price controls and subsidies.2 Under this new system, hospitals and service providers were allowed to implement user fees and generate profits. As a result, health care improved significantly—for those who could afford it.

Although millions were lifted out of poverty, these reforms had several unintended consequences, perhaps the most important of which was increasing health disparities among the poor.3 , 4 One of Vietnam's policy responses to address this challenge has been to decentralize health services, transferring more authority for decision making and resource allocation to the provincial and district levels.5 Decentralization is intended not only to improve the efficiency of health resource allocation but also to empower and motivate local health workers. In practice, however, decentralization seldom lives up to these lofty promises.6 One of the often-cited reasons for this is that the shortage of management skills among workers at the periphery of the health system poses an important barrier that prevents decentralization from achieving its goals.7 This problem is compounded by the fact that in many developing countries, training infrastructure is weak and training curricula often lack a focus on cognitive (problem-solving and critical thinking) and behavioral skills (teamwork and communication), both of which are critical in decentralized health systems where responsibility for higher-level functions such as assessment and policy development has shifted to the periphery of the health system.

The Hanoi School of Public Health (HSPH) has worked with international partners over the past 2 decades to adapt and modernize its management training curricula and implement training strategies that address this important workforce development need. In this article, we describe the establishment of a national management training network for HIV/AIDS personnel that demonstrates the value and feasibility of Project-Based Learning 8 to provide evidence-based management skills and link the training to direct improvements in HIV/AIDS program efficiency and effectiveness in a decentralized health system environment.

Back to Top | Article Outline

Methods

In 1992, the US Centers for Disease Control and Prevention (CDC) established the Sustainable Management Development Program (SMDP) in response to a growing demand for public health management training, particularly among developing countries undergoing health sector decentralization. SMDP's mission was to strengthen management training capacity and increase the use of evidence-based tools to improve the effectiveness and efficiency of public health programs.9 Its strategy included an annual 6-week training-for-trainer (ToT) management course taught in collaboration with Emory University, followed by technical assistance to support alumni in planning, implementing, and evaluating in-country management capacity development programs.

From 1992 to 2013, 425 management trainers from 70 countries attended SMDP's ToT course and established management capacity development programs in a variety of settings including nongovernmental organizations,10 academia,11 and ministries of health.12 A key feature of SMDP's in-country approach to management capacity development was the use of Project-Based Learning to reinforce skill transfer and link management training to tangible improvements in public health program outcomes.

Ten faculty members from the HSPH attended SMDP's ToT course from 1997 to 2013. During this period, content and SMDP teaching methods were introduced into pre-service curricula of HSPH undergraduate and graduate degree programs and were also used to develop in-service training initiatives for health workers in Vietnam's Reproductive Health13 and TB Control programs.14

In 2004, Vietnam became one of 15 focus countries of the US President's Emergency Program for AIDS Relief (PEPFAR). From 2005 to 2015, PEPFAR provided the HSPH with funding to develop a national management and leadership training network, called Vietnam Leadership and Management Capacity Strengthening (VLMCS), that would result “in specific and measurable improvements in the effectiveness of HIV/AIDS programs and the quality of HIV/AIDS services.”15

To accomplish this objective, the HSPH developed and coordinated a network of regional training centers across Vietnam that included the Pasteur Institute in Ho Chi Minh City (HCMC), the Da Nang Provincial AIDS Center (PAC), the Institute of Public Health in HCMC, and the Da Nang Center for Preventive Health (Figure 1). Trainers from these institutions attended ToT courses taught by HSPH faculty who themselves had attended the CDC/SMDP ToT course at CDC in Atlanta. Under HSPH direction, these institutions then carried out management and leadership training courses for provincial and district HIV/AIDS personnel within their respective regions. The target audience included staff from the PACs who managed and directed Vietnam's HIV/AIDS intervention programs at the subnational level. The goal of the VLMCS program was to improve efficiency, service quality, and health outcomes within these programs through hands-on, project-based learning in a decentralized work environment. Figure 2 illustrates the relationship of VLMCS program inputs to outputs, outcomes, and goals in a logic model framework. The management training curriculum included 2 core modules (each with a mentored, field-based project) supplemented by several short classroom-based modules (Table).

FIGURE 1

FIGURE 1

FIGURE 2

FIGURE 2

TABLE

TABLE

The core Quality Improvement (QI) module provided participants with tools to analyze and improve the efficiency of public health programs. Skills included critical thinking, evidence-based problem-solving, teamwork, and communication. The tools included:

  • Brainstorming, multivoting, and theme selection matrices to develop consensus and prioritize problems for improvement;
  • Tables and figures to synthesize performance data and assess work processes;
  • Flowcharts and Ishikawa (fishbone) diagrams to break down complex problems into root causes; and
  • Plan, Do, Check, and Act cycle

The core Project Management module provided learners with tools for planning, implementing, and monitoring new HIV/AIDS interventions. Tools in this module included:

  • Log-frame for project planning;
  • Job analysis to define duties, tasks, responsibilities, and performance indicators;
  • Time management, communication, and risk management plans; and
  • Developing indicators for project monitoring and evaluation.

Provincial- and district-level HIV/AIDS program managers typically attended the training courses in teams of 2 to 3 persons from different work sites (∼25 participants per training). Both the QI and Project Management modules were conducted in 3 stages: (1) an initial 3- to 5-day classroom session to explain the tools and provide mentored practice sessions; (2) 5 to 6 months of fieldwork during which participants applied the tools to identify a problem and improve a work process, or develop a plan to roll out a new HIV/AIDS intervention with a budget, timeline, and framework for monitoring and evaluation. Participants typically received 2 supervisory visits during the fieldwork period; and (3) a final 3-day classroom session during which participants presented their field projects to an audience of peers and stakeholders. During these presentations, trainees defended their operational decisions and demonstrated the extent to which their projects met HIV/AIDS program goals. Awards or prizes were given for “best project,” and winners presented their projects at annual national dissemination workshops and national HIV/AIDS conferences.

Back to Top | Article Outline

Results

From 2005 to 2015, 111 participants attended in-country ToT courses led by HSPH faculty who had attended the CDC/SMDP ToT courses. Among these, 28 (18 from Hanoi, 5 from Da Nang, and 5 from HCMC) became the core faculty for the regional HIV/AIDS management training network. This core group trained 827 HIV/AIDS program managers throughout Vietnam. During the training, provincial- and district-level participants carried out 218 applied learning projects; 132 focused on improving the efficiency of a process that impacted HIV/AIDS service delivery; the remaining 86 involved plans to roll out HIV/AIDS intervention projects or services at the subnational levels. The success rate (defined as met or exceeded targets/goals) among the QI and managing new HIV/AIDS intervention projects was 94.7% and 95.3%, respectively.

Both types of projects provided mentored, hands-on learning and enabled participants to apply tools to collect and utilize data that improved HIV/AIDS program outcomes. Through project-based learning, participants developed cognitive (problem-solving and critical thinking) and behavioral skills (teamwork and communication) that enabled them to work more effectively in a decentralized work environment.

The following summaries of 3 QI projects, one from each region, illustrate how this strategy was effective at both skill transfer and linking training to improvements in HIV/AIDS program outcomes.

Back to Top | Article Outline

Increasing the Proportion of Clients Returning for HIV/AIDS Test Results in Nam Dinh Province (Northern Region)

Nam Dinh is a coastal province located in the Red River delta of northeastern Vietnam with a population of approximately 2 million. From 2003 to 2006, the annual number of new HIV/AIDS cases in Nam Dinh increased from 295 to 506. In May 2006, a team from the Nam Dinh PAC attended a QI training offered by faculty from the Northern Regional Management Training Center in Hanoi. Following the initial classroom session, participants organized a QI team that included other workers involved in various aspects of HIV/AIDS prevention and treatment. The team developed a list of core processes involved in their work, and using a multivoting selection matrix, identified voluntary counseling and testing (VCT) as a priority area for improvement.

A flowchart of the VCT process was developed to identify areas for improving efficiency. Prospective data were collected from June to November (2006) to determine the number of client visits for VCT, the proportion accepting HIV/AIDS testing, and the proportion returning for test results. While the acceptance rate for HIV/AIDS testing was 87% (program target = 90%), the proportion of clients returning for test results was only 60%—well below the target. The team used an Ishikawa (fishbone) diagram to identify root causes that contributed to low return rates and then used a countermeasure matrix and barrier/aid analysis to identify and implement the following actions:

  • Increased supervisory oversight of counseling procedures;
  • Eliminated duplicate client reporting by outreach workers;
  • Improved training and supervision for community outreach workers; and
  • Developed new information, education, and communication tools that explain and motivate clients to return for results.

Following implementation of these countermeasures, the return rate for test results increased from 65% in November 2006 to 92% in January 2007. The countermeasures were integrated into standard operating procedures (SOPs) and return rates for test results were monitored thereafter. As of the end of April 2008, they remained above 90%.

Back to Top | Article Outline

Reducing Nonadherence Rate to Antiretroviral Therapy Among Persons With HIV/AIDS in Khanh Hoa Province (Central Region)

Khanh Hoa Province (population ∼1.2 million) is located along Vietnam's south central coast. Its capital city (Nha Trang) is a popular tourist destination. From 1993 to 2006, Khanh Hoa reported 1871 cumulative cases of HIV/AIDS; 830 had clinical AIDS and 623 died. The majority of HIV transmission in Khanh Hoa was associated with intravenous drug use, although the number of cases associated with commercial sex work was increasing.

Antiretroviral therapy (ART) was initiated in Khanh Hoa in August 2005. In September 2005, a team from Khanh Hoa Province attended a 3-stage QI training offered by the Central Regional Training Center in Da Nang. Following the initial 5-day training, team members returned to Khanh Hoa to begin their field project. Using brainstorming, multivoting, and a theme selection matrix, the team chose “nonadherence to ART” as the focus of their field project. They defined clients as “nonadherent” if any of the following occurred at least 3 times per month: forgot to take their medication, took the wrong medication, failed to take the proper dosage, or failed to take medication at the correct time of day.

From September to December 2005, 28 clients on ART were followed to determine adherence with ART. During this period, 5 (18%) were nonadherent. The team set a goal to reduce nonadherence from 18% to less than 5% by August 2006.

An Ishikawa (fishbone) diagram was used to identify the following root causes that contributed to high rates of nonadherence: shortage of peer counselors, inadequate counselor training and supervision, and lack of family and community support for clients. Action plans were developed and implemented to address each of these root causes. Although the number of clients on ART had increased from 28 to 44 by August 2006, the number not adhering dropped from 5 of 28 (18%) to 2 of 44 (4.5%). Following the success of these interventions, they were all integrated into SOPs.

Back to Top | Article Outline

Improving HIV Rapid Testing at Number 1 Pediatric Hospital, HCMC (Southern Region)

In 2004, the Department of Microbiology at Number 1 Pediatric Hospital in HCMC performed more than 3000 HIV rapid tests (HRTs) to identify and reduce perinatal mother-to-child transmission of HIV. A 2-person team from the department attended a QI workshop in 2005 and using tools from the workshop, chose to work on improving HRTs, noting that 90% of all HRTs at the hospital did not meet the following 4 criteria for “proper testing”: (1) the sample should be delivered directly from where it was collected to the laboratory and registered by a staff member of the Department of Microbiology; (2) the sample should include at least 1 mL of nonhemolyzed blood; (3) the sample should be properly labeled with the patient's information; and (4) the test should be completed within 1 hour after delivery to the laboratory. Further analysis revealed that 80% of specimens were not tested within 1 hour of being received in the laboratory, 30% were hemolyzed or contained less than 1 mL of blood, and 10% of specimens were lost.

To address these problems, the 2 staff members who participated in the QI workshop organized a QI team that consisted of key staff from the Microbiology Department as well from the clinical departments that collect and send specimens to the laboratory. The team set a goal of reducing the number of HRTs that did not comply with all 4 criteria by half (from 90% to 45%) within 4 months from September to December 2005.

The team developed a flowchart of the HRT delivery process and collected data by observing the delivery process, extracting data from administration records, and by interviewing staff from the laboratory and other departments involved with HRT specimen collection, transport, and testing. The analysis revealed that about 60% of the HRT samples that did meet the delivery criteria were from just 3 departments—the Emergency Department, Pulmonary Medicine, and Infectious Diseases. An Ishikawa (fishbone) diagram identified several root causes that contributed to improper HRTs:

  • Nurses from the clinical wards were unaware that specimens for HRT had special delivery requirements;
  • Some staff members in the Department of Microbiology forgot to sign in HRT specimens;
  • Nurses from the clinical wards were unaware there was a bell at the delivery point to notify the laboratory staff when an HRT specimen arrived; and
  • Department heads were unaware there was a problem with HRTs (no monitoring or feedback), so they did not supervise their staff to follow proper procedures.

The following countermeasures were implemented to address the root causes:

  • Clear instructions and a description of the correct procedure for the collection and delivery of HRT specimens were sent to all clinical departments.
  • Oversight of department heads was increased, and they were expected to provide an update on their department's progress to improve HRT specimen handling to the hospital director at their weekly meeting.
  • A notice was placed on the wall of the laboratory specimen delivery window to remind nurses to ring the bell when dropping off HRT specimens.
  • A monitoring system was put into place to track improvement in the 3 departments that accounted for 60% of the deficiencies.

By December 2005, the proportion of improperly delivered HRT samples was reduced from 90% to 48%. Among the 3 departments that accounted for 60% of the improperly delivered specimens, the rate was reduced from 94% to 37%. While the overall rate (48%) did not meet the team's goal of reducing improper specimen delivery to 45%, further investigation revealed that during the follow-up period, there was a surge in the number of HRT samples submitted by the Kidney and Dialysis Unit and none (0%) of their specimen deliveries met the criteria. The team worked with the hospital's chief nurses of all departments to integrate countermeasures into SOPs and also focused on helping the Kidney and Dialysis Unit implement the new SOPs.

Back to Top | Article Outline

Discussion

The global shortage of management capacity has been referred to as a “binding constraint” that impedes progress in scaling up services to achieve Millennium Development Goals.16 While there is consensus on the importance of the management competency gap and the need to address it, there is little agreement on the best approaches to accomplish this goal other than general agreement that competency-based, on-the-job training is superior to classroom training alone. The World Health Organization has developed a useful framework for mapping the challenges and responses to the management competency gap but points out that “documentation of successful approaches is scarce and their dissemination even more so.”16

In Vietnam, the HSPH has focused its in-service management training efforts on providing frontline public health program managers with evidence-based problem-solving skills that can have an immediate and measurable impact on public health program outcomes. The training content was taught to teams rather than individuals, and learners were challenged to apply new skills to solve real workplace problems. This Problem-Based Learning approach provided managers with skills needed to succeed in a decentralized health system and encouraged innovation and teamwork, helping create what the World Health Organization refers to as an “enabling working environment” essential for producing lasting behavioral change.17 This strategy also embraces the concept of “transformative learning” aimed at achieving core competencies for effective teamwork and represents a creative adaptation of a new educational model to address local priorities.18

A similar team-based and action-learning approach has been used in the US CDC's Management Academy for Public Health training program since 1998.19 , 20 This program also focuses on strengthening public health practice and blending new skills in a field-based setting to address local public health priorities.21 This approach has been shown to significantly improve public health practice, resulting in better programming and health outcomes.20–22 By promoting a similar approach, SMDP has helped countries undergoing decentralization adapt and improve health planning and management among both governmental11 and nongovernmental organizations.10

In addition to producing public health program managers who are competent problem solvers, the SMDP approach generates a valuable database of completed QI projects, each of which identifies a priority problem, its underlying causes, and defines countermeasures that once implemented improve program outputs. These projects represent more than just a vehicle for team building and competency development—they provide stakeholders with a rare opportunity to see a direct return on training investment in terms of program output improvement and also represent a database of “best practices” that can be used to refine and improve program norms.

Finally, this initiative resulted in the development of a national management training network that began with only 3 HSPH faculty members trained by CDC/SMDP in the late 1990s. Similar to the integration of the Management Academy for Public Health program into the South Carolina workforce development strategy,23 this network in Vietnam represents the culmination of 20 years of effort established with resources from a series of donors, each interested in different categorical program outcomes. Although the most recent support came from PEPFAR, the network has the flexibility to work across categorical programs in the future and is not dependent on PEPFAR for recurrent costs.

In 2014, Vietnam signed an agreement with the World Bank for a US $106 million project, titled “Health Professionals Education and Training for Health System Reforms.”24 This 6-year project includes a US $12 million component to “Strengthen Management Competencies in the Health Sector” of which US $6 million is earmarked specifically to “Strengthen Health Management Training.” The HSPH and its southern regional management training network partner, the HCMC Institute of Public Health, are the designated co-leads for this subcomponent and are expected to train about 4000 managers at various levels of the health system by 2020.

While the goals of this project are broad—improving the quality of health service delivery and reducing health disparities in a subset of low-income provinces—the project provides another opportunity for the HSPH and its network partners to continue scaling up national management capacity development efforts, this time without the constraints imposed by categorical program funding. “Strengthening health sector management capacity” was a high priority in Vietnam's 2011-2015 Five Year Health Sector Development Plan25 and will likely remain so in the foreseeable future. It is our hope that the work described in this article guides future health sector management development policy in Vietnam and ensures that it remains evidence- and competency-based.

Back to Top | Article Outline

Implications for Policy & Practice

  • Vietnam's collaboration with CDC/SMDP demonstrates the success of an international cascade ToT model. Standardized training materials and frequent supervisory visits were important components of ensuring training quality as was documenting the impact of the training on health program outputs.
  • Project-based learning, although more labor- and cost-intensive than classroom training alone, is a practical method for developing and reinforcing management competencies among public health workers. These projects are designed to address local public health needs and serve as a database of best practices and that can be shared allowing others to benefit from local operational research.
  • Health sector decentralization typically shifts responsibility for higher-level public health functions toward the periphery of the health system. To realize the potential benefits of decentralization, local health workers must develop new management skills to meet these new responsibilities. Vietnam's VLMCS demonstrates how a practical competency-based training model can be scaled up and integrated into a national training network. This network, built primarily with HIV/AIDS program funding, can now serve the broader national workforce capacity development needs of other health programs.
Back to Top | Article Outline

References

1. Vuong QH. Vietnam's political economy in transition. https://www.stratfor.com/the-hub/vietnams-political-economy-transition-1986-2016. Accessed January 8, 2017.
2. Witter S. “” and health: the effect of economic reforms on the health system in Vietnam. Int J Health Plan Manag. 1996;11:159–172.
3. Målqvist M, Lincetto O, Nguyen H, Burgess C, Dinh T. Maternal health care utilization in Viet Nam: increasing ethnic inequity. Bull WHO. 2013;91:254–261.
4. Målqvist M, Dinh TPH, Thomsen S. Causes and determinants of inequity in maternal and child health in Vietnam [published online ahead of print August 11, 2012]. BMC Public Health. 2012;12:641.
5. The World Bank. Skilling Up Vietnam: Preparing the Workforce for a Modern Market Economy. Vietnam Development Report. Washington, DC: The World Bank; 2014; http://documents.worldbank.org/curated/en/610301468176937722/pdf/%20829400AR0P13040Box0379879B00PUBLIC0.pdf. Accessed January 18, 2017.
6. Lieberman S, Capuno J, Hoang V. Decentralizing health: lessons from Indonesia, the Philippines, and Vietnam. In: East Asia Decentralizes: Making Local Government Work. Washington, DC: The World Bank; 2005:155–178.
7. Omar M. Health sector decentralization in developing countries: unique or universal! World Hosp Health Serv. 2002;38(2):24–30.
8. Condliffe B. Project-Based Learning: A Literature Review. San Rafael, CA: Luca Education Research; 2016. https://s3-us-west-1.amazonaws.com/ler/MDRC+PBL+Literature+Review.pdf. Accessed January 7, 2017.
9. Setliffe R, Porter J, Malison M, Frederick S, Balderson T. Strengthening the public health workforce: three CDC programs that prepare managers and leaders for the challenges of the 21st century. J Pub Health Manag Pract. 2003;9(2):91–102.
10. McEwan E, Conway M, Bull D, Malison M. Developing public health management training capacity in Nicaragua. Am J Public Health. 2001;91(10):1586–1588.
11. Simmons N, Brborovic O, Fimka T, et al Public health capacity building in Southeastern Europe: a partnership between the Open Society Institute and the US Centers for Disease Control and Prevention. J Public Health Manag Pract. 2005;11(4):351–356.
12. Sucaldito N, Tayag E, Roces M, Malison M, Robie B, Howze E. The Philippines Field Management Training Program: strengthening management capacity in a decentralized public health system. Int J Public Health. 2014;59(6):897–903.
13. UNFPA. Strengthening the Vietnam Ministry of Health's Capacity to Implement the National Reproductive Health Programme. New York City, NY: UNFPA; 1997. Project VIE/97/P10.
14. Umble K, Brooks J, Lowman A, et al Management training in Vietnam's National Tuberculosis Program: an impact evaluation. Int J Tuberc Lung Dis. 2009;13(2):238–246.
15. US Centers for Disease Control and Prevention. Management capacity building activities in Vietnam. https://www.cdc.gov/globalhealth/SMDP/pdf/Vietnam_factsheet.pdf. Published 2010. Accessed January 7, 2017.
16. Egger D, Travis P, Dovlo D, Hawken L. Making Health Systems Work: Working Paper No. 1. Strengthening Management in Low Income Countries. Geneva, Switzerland: WHO Press; 2005. http://http://www.who.int/management/working_paper_1_en_opt.pdf?ua=1. Accessed January 8, 2017.
17. World Health Organization. Building Leadership and Management Capacity in Public Health. Geneva, Switzerland: World Health Organization; 2007. http://http://www.who.int/management/FrameworkBrochure.pdf. Accessed January 6, 2017.
18. Frenk J, Chen L, Bhutta Z, et al Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010;376:1923–1958.
19. Porter J, Johnson J, Upshaw V, Orton S, Deal KM, Umble K. The Management Academy for Public Health: a new paradigm for public health management development. J Public Health Manag Pract. 2002;8(2):66–78.
20. Umble KE, Orton S, Rosen B, Ottoson J. Evaluating the Impact of the Management Academy for Public Health: developing entrepreneurial managers and organizations. J Public Health Manag Pract. 2006;12(5):436–445.
21. Orton S, Umble KE, Rosen B, McIver J, Menkens AJ. The Management Academy for Public Health: program design and critical success factors. J Public Health Manag Pract. 2006;12(5):409–418.
22. Orton S, Umble K, Zelt S, Porter J, Johnson J. Management Academy for Public Health: creating entrepreneurial managers. Am J Public Health. 2007;97(4):601–605.
23. Cumbey DA, Ellison LA. The Management Academy for Public Health: the South California experience. J Public Health Manag Pract. 2006;12(5):468–474.
24. The World Bank. Health Professionals Education and Training for Health System Reforms Project. Washington, DC: The World Bank; 2014. Report No. 82318-VN. http://documents.worldbank.org/curated/en/427731468316154123/%20pdf/%20823180PAD0IDA0010Box385188B00OUO090.pdf. Accessed January 19, 2017.
25. Vietnam Ministry of Health. Five-year health sector development plan 2011-2015. http://http://www.wpro.who.int/health_services/VTN_2011-2015.pdf. Accessed January 18, 2017.
Keywords:

health sector decentralization; project-based learning; public health management training

Copyright © 2018 Wolters Kluwer Health, Inc. All rights reserved.