Evaluation of an Ecohealth Approach to Public Health Intervention in Ha Nam, Vietnam

Abstract

Context: 

Ecohealth is a transdisciplinary research approach that considers socio-economic, cultural, and environmental factors. Ecohealth program assessment is sometimes unable to capture the process of change, especially when the evidence is not well documented. As such, there is a need to better understand how ecohealth approaches are understood, integrated, and adapted in practice to support the sustainability of the approach.

Objective: 

To evaluate the use of an ecohealth approach to a community-based intervention to improve environmental sanitation and draw lessons learned for similar public health initiatives.

Design: 

An iterative evaluation approach involving 27 in-depth interviews and 9 focus group discussions was used to gather feedback on the intervention activities and outcomes from all participants.

Setting and Participants: 

The study site was Hoang Tay commune, Kim Bang district, Ha Nam province. The participants included 12 farmers, 4 local policy makers, and 7 researchers from Hanoi University of Public Health.

Results: 

The farmers provided specific shortcomings of the biogas procedure steps, while the local authorities identified new and more effective ways to promote sanitation guidelines. Outcomes, as behavior changes, in 3 participant groups were captured.

Conclusion: 

Participation in ecohealth interventions should be collegial to give opportunities for all related stakeholders to build capacity, support, and achieve the transdisciplinary principle. This also helps ensure that the community-based solutions are incorporated in public health interventions. Participatory monitoring and evaluation should support the understanding of the implementation process to capture intervention outcomes.

Ecosystem approach to health (Ecohealth) recognizes that humans, animals, and their environment operate functionally as complex socioecological systems. Promoted mainly by the International Development Research Centre, Canada, the approach appraises “the effective multi-sectoral collaboration and the engagement of multiple types of stakeholder” through its 6 integrated principles, namely, system thinking, transdisciplinary research, participation, gender and social equity, knowledge to action, and sustainability.¹ These 6 principles can be achieved through a long process of collaboration, negotiation, interaction, and learning among related stakeholders.¹ As ecohealth approach is broad and holistic, the understanding and application of its principles are varied among scholars and practitioners around the world.² Therefore, “an explanation of the process implemented is required to give the readers the ability to understand and evaluate a study that is classified as ecohealth.”² Research is recommended to “concentrate on the reporting and evaluation of process to more rigorously guide ecohealth to develop from concept to practice.”²

Ecohealth scholars encourage a culture of evaluation where the members of the research teams can “keep abreast of findings, developments, interventions and stakeholder positions.”¹ This culture supports researchers to “examine projects in-progress to determine their consistency with ecohealth concepts.”³ For that reason, evaluation approaches and tools in ecohealth research and intervention should facilitate the understanding of process of changes to support related stakeholders in their decision making.¹ Evaluation tools and approaches that focus on outcomes rather than outputs (such as Outcome Mapping and Developmental Evaluation) are preferred in ecohealth research and interventions.¹

While there have been efforts to share challenges and opportunities in doing ecohealth research and interventions around the world,^4,5 publications on evaluation ecohealth are rare and do not necessarily focus on describing the process of change in stakeholders.² Besides, evaluation is often carried out from the donor site, which is sometimes unable to record the process of change when the evidence is not well documented.⁶ As such, there is a need to better understand how an ecohealth approach and principles are understood, integrated, and adapted in practice using a suggested evaluation approach to support ecohealth sustainability.⁷ This article, in an attempt to fill this gap, presents our experience in using an iterative participatory evaluation framework to determine the outcomes of an intervention, along with its stakeholders' feedback in using an ecohealth approach to improve animal waste management (biogas operation) in Ha Nam province of Vietnam. This will contribute to a learning process when applying ecohealth intervention research and serve as lessons learned for similar public health initiatives in developing countries.

Methods

Study site

Hoang Tay commune, a periurban agricultural area, located in Kim Bang district, Ha Nam province, in Northern Vietnam, was chosen as the study site for the intervention phase (Figure 1). The economic basis of the commune relies on both livestock and crops. The total swine population commune was 6500 in 2015 and almost all of them were raised at small household scale. Household biogas was used to treat manure and human excreta. While the treatment technology looked promising and clean, the treatment quality and health risk posed by biogas effluent are questionable.⁸

FIGURE 1: Map of the Intervention Site (Hoang Tay Commune, Kim Bang District, Ha Nam Province)

The intervention

The program “Field Building Leadership Initiative (FBLI): Advancing Ecohealth in South East Asia and China,” funded by the International Development Research Centre, Canada, and implemented in 4 countries, Vietnam, Indonesia, Thailand, and China, sought to understand and address agricultural practices and associated health risks. In Vietnam, the FBLI aimed to better understand the impact of livestock intensification on human, animal, and environment health, along with implementing ecohealth intervention to better manage the livestock waste and reduce the associated health risks. It commenced in July 2012 with the 2 phases: the baseline investigation from July 2012 to September 2014, focused on identifying local issues using a participatory approach and household surveys and the intervention phase from October 2014 to September 2016, aimed to develop and implement an intervention to solve the local public health use using an ecohealth approach. Health risks from human and animal waste treatment was identified as a local priority from previous research conducted by the same team from 2007 to 2011, which was funded by the Swiss National Centres of Competence in Research North-South (herewith addressed as NCCR research).^2,9 Besides, the results from the baseline investigation showed that using biogas wastewater in daily agriculture activities such as vegetable, crop, and fruit irritation was popular among the local community while pathogens such as Escherichia coli, Giardia lamblia, and Cryptosporidium parvum remained at high concentrations in the biogas wastewater.^8,10 Local farmers' health was at risk, with a diarrhea risk ranging from 45.7% to 89.4%.¹⁰ These results suggested an intervention that could “enhance the awareness of people when handling biogas wastewater, and hence promoting practices of using personal protective measures.”⁸

The intervention, targeted at developing and trialing a sustainable solution to improve the local sanitation issue, consisted of 3 stages: discussion stage (from October 2014 to December 2014), pilot and assessment stage (January to August 2015), and finally scaling up stage (September 2015 to September 2016). In the discussion stage, researchers worked with local farmers and authority, who voluntarily took part in the intervention, to find solutions for the sanitation issue that had been found in the investigation phase. At the end of the stage, a 6-step biogas safety procedure and a plan on how to promote Chapter 6 on sanitation in the village regulation to enhance local people awareness on sanitation came out as final products (Figure 2). The former targeted at providing local farmers with information on proper operation of household biogas systems through establishing safe routines such as wearing personal protection equipment and using appropriate water level when cleaning the pigpens or checking biogas meters regularly during the operation. The latter aimed at revitalizing Huong Uoc—a traditional village document that has no legal power but was considered as behavior/manner norms for the local villagers. In the pilot stage, the plan was to promote Chapter 6 on sanitation in Huong Uoc through big graphical posters hanging in public places such as open wet markets, communal cultural houses, and loud speakers. The 6-step biogas procedure was trialed and then assessed with a group of 12 voluntary farmers. In the final stage, it was expected that the farmers, upon seeing the benefits from the 6-step safety procedure, would use peer-to-peer communication to widely promote this model at a higher level. The local policy makers hoped to witness the impacts of the promotion of Chapter 6 in supporting the improvement of local sanitation. Ethical approval for this research was obtained from Hanoi University of Public Health on 19 February 2013 (Decision No 041/2013-HD3).

FIGURE 2: (A) A Poster Depicting That 6-Step Biogas Safety Procedure Was Displayed at a Farmer's House in Hoang Tay Commune, Kim Bang District, Ha Nam Province. (B) A Graphical Poster Promoting Chapter 6 on Sanitation in the Traditional Village Regulation (Huong Uoc)

Evaluation approach and design

The evaluation followed a developmental evaluation approach that “supports development of innovations and adaptation of interventions in dynamic environments.”¹¹ Rather than based on a fixed logical framework, this approach works toward developing “measures and tracking mechanisms quickly as outcome emerge and updated through under conditions of complexity.”¹¹ When applying a developmental approach, an iterative evaluation framework is often employed on the basis of one of the key features of developmental evaluation approach that provides continuous data for program improvement. In this framework, evaluation is often conducted several times during program/project cycle to keep the implementers informed of any developments amid the changing and complex environments.¹¹ In theory, applying an iterative framework with the same participants at each evaluation assists implementers with capturing the process of changes in participants' action, behavior, and attitude in an ecohealth intervention.¹

The evaluation design adapted outcome harvesting tool, which enables researchers to retroactively collect qualitative evidences of changes generated during the intervention.¹² Outcome is defined as changes in behavior, attitude, action, activities, and relationship of any related stakeholders or organizations involved in the intervention.^12,13 These changes are logically linked to a program's activities and defined by programs' stakeholders. In these approaches, “the most successful programs are those that devolve power and responsibility to endogenous actors.”¹³

Our evaluation framework was designed to gather qualitative information with 3 iterations: the first at the end of the discussion stage, the second at the end of the pilot and assessment stage, and the last at the end of the scaling up stage. Each iteration consisted of 3 sequential steps (ie, document review, in-depth interview, and focus group discussion). Documents such as technical reports, meeting notes, and field notes from the previous stage were reviewed to collect information on participants' feedback, possible outcomes, and challenges. The information was then complimented during in-depth interviews with each participant. In the focus group discussions, participants from each group reach consensus on feedback, outcomes, and challenges generated from the intervention activities during the previous stage. A semistructured interview guide was developed that used general and open-ended questions such as the following: (i) What changes in behavior, attitude, action, and relationship of any participant who took part in the activities did you observe during the previous stage? (ii) Who initiated the(se) change(s)? When and where did the(se) change(s) take place? (iii) How were these changes developed? (iv) What things did you not expect to happen, but happened during the previous stage? The interview guide was used for both in-depth interview and focus group discussion. Probes were used where needed. The findings from in-depth interviews and focus group discussion at each stage were compiled and finalized by the first, second, and third authors and then presented in an internal evaluation report to the research team. Participants' responses were coded and thematically analyzed independently by 2 researchers (first and second authors).¹⁴ Qualitative data were analyzed using Nvivo software.

Participants

Three groups of participants directly took part in the intervention, namely, FBLI Vietnam research team members (7), local farmers in Hoang Tay commune (12), and local authorities in Hoang Tay commune (4). The local authorities included the vice chairman of the Commune People's Committee (CPC), the cultural officer at the CPC, the head of the health station, and an animal health worker. A total of 27 in-depth interviews and 9 focus group discussions were conducted with these participants. In-depth interviews lasted approximately 30 minutes while focus group discussions lasted up to an hour. All interviews and focus group discussions were conducted in Vietnamese and were recorded with permission from the participants.

Results

Farmer feedback on 6-step biogas safety procedure

After the pilot and assessment stage, the farmers indicated shortcomings of the 6-step biogas safety procedure (Table 1). The practice of the 6-step biogas procedure was hindered by the local summer weather conditions and the current infrastructural situation of household biogas systems. The farmers also indicated that they did not feel comfortable checking on each other since they were neighbors and relatives. “It is not our culture to check on our neighbors and relatives” (interview with a farmer in Hoang Tay commune who trialed the 6-step biogas safety procedure, August 2015).

TABLE 1 - Farmer Feedback on Biogas Safety Procedure

Step	Key Recommendation	Feedback
1	Wear PPE while cleaning pigpen	Not practical during hot seasons (April-September)
		Not convinced that PPE protects them from bacterial exposure
2	Check water level in the biogas tank before cleaning pigpen	Not practical because tank cover is heavy and requires 2 people to lift it up
3	Check biogas tank (inside house) water level before cleaning pigpen	Easy to do and fits with daily schedule of work
4	Manually clean out pigpen (feces) with shovel before washing pigsClean once a day (waste input for biogas digester)Do not use high-pressure hose in this step	Not practical for smallholder farms with more 50 pigsNot practical for smallholder farms that raise sows (too much feces)More convenient to use high-pressure hose
5	Limit water use for washing pigs	Need to use amount of water that will clean pigpen floor and cool down pigs
6	Check the waste water outlet after cleaning out pigpen and washing pigs (step 4)	Easy to do and fits with daily schedule of work

Abbreviation: PPE, personal protection equipment.

The issuance of local sanitation guideline

In September 2015, the second evaluation iteration revealed that the vice chairman of the CPC at commune level made 2 significant decisions regarding the sanitation issue of commune. The first was to issue an official sanitation guideline and the second was to prioritize sanitation in its rural development plan. According to the vice chairman, these decisions were prompted upon seeing the positive impacts of the graphical posters promoting Chapter 6 in the village regulation on local people. The research team considered these decisions as unexpected outcomes of the intervention.

Outcomes of the intervention

All 3 participant groups, namely, farmer, local authority, and researcher demonstrated a progressive change in attitudes over the timeline of the intervention (Table 2). The researchers identified changes in attitude toward how to apply transdisciplinary research in practice. Farmers illustrated a tendency to become parts of a transdisciplinary research team by giving their feedback on the shortcomings of the intervention at the end of the second intervention stage. Local authority showed a significant change in their attitude toward using intervention results (promotion of Chapter 6) by voluntarily issuing a legal guideline for Chapter 6 and advancing the sanitation targets in the rural development plan.

TABLE 2 - Key Outcomes per Participant Group (Researcher, Farmer, and Local Authority) per Evaluation Iteration

Iteration	Themes in Outcomes	Sample Quotes
1	Researchers gained an applied understanding of transdisciplinary research.	“Our team included researchers from diverse backgrounds and we also considered local people in as our team members.”
2	Researchers acknowledged the key role of local authority and time investment required for meaningful interparticipant discussion.	“We did not invent the six-step safe use of biogas model and the idea of using Huong Uoc [village regulation] as the tool to promote the locals' awareness of sanitation. They are the results of the long-time discussion with the locals.”
3	Researchers learned how to do ecohealth research through the feedback of local community.	“The core group farmer feedbacks are very important to us, since we are not the one who implement the six-step safe use of biogas model.”
1	Local farmers expressed their interest in sanitation issues they considered important.	“We want to know more about the issues like pesticides or waste management in the commune because those are very burning sanitation issue in the area.”
2	Local farmers took action to solve sanitation issues that affected them.	“We agree to take part in the intervention because we hope that it could solve the sanitation issues in our local area.”
3	Local farmers identified both practical and cultural procedure shortcomings.	“It is very hot in the summer, thus wearing gloves and less water for bathing the pigs as suggested in the [six-step] model are impossible.”
1	Local authority indicated an interest in adapting Huong Uoc to promote the importance of sanitation issues.	“Huong Uoc is the traditional document which does not have any legal power but very useful if it can be promoted and used, we believe that revitalizing the Huong Uoc raise the local's awareness on sanitation.”
2	Local authority now an active participant in promoting the graphical poster of the sanitation regulations.	“We hang the posters which animated Chapter 6 in the culture houses, health station, the Committee's office and especially the open market where people gather every day for purchasing food.”
3	Local authority issued a “red-stamp” document specifically for Chapter 6: Sanitation guideline. Local authority advanced the sanitation targets in the rural development plan.	“The posters attracted the attention of the local people and visitors. Sanitation issue became hot topics. The People Committee has issued a guideline to reinforce the implementation of Chapter 6 on sanitation.” “We decided to push up the rural development plan a head of two years because we think that we will meet the sanitation requirements thanks to the local people's attention on this issue.”

Discussion

Within the scope of this article, our discussion focuses on how an ecohealth approach has been used in the intervention in Hanam province and its implications for future ecohealth interventions. Our evaluation results suggested that 3 principles of transdisciplinarity, participation, and knowledge to action have been explicitly reflected in this intervention. We also compared our results with the findings from the previous NCCR research to shed light on the changes in behaviors, attitude, and actions of related stakeholders.

Transdisciplinarity and participation

Scholars and practitioners around the world offered varied and diverse definitions of transdisciplinarity.¹⁵ The concept is central around 3 elements, namely, the integration of disciplines, the participation of different stakeholders, and the addressing of real-world problems.¹⁵ These elements are the starting basis to assess transdisciplinarity in ecohealth research and interventions.¹⁵ The integration of disciplines encourages the utilization of “different research methodologies, tools across disciplines, non-academic perspectives and knowledge” in ecohealth research.¹⁶ Johnston et al¹⁷ in a research on ecohealth and Aboriginal people in Australia showed an example of how integration of the Aboriginal views about the interconnectedness of people and their land into health intervention would be “an important and culturally relevant strategy.” A literature review conducted by Burgess et al¹⁸ considered that effective health interventions required “trans-disciplinary, holistic approaches that explicitly incorporate Indigenous health beliefs and engage with social and cultural drivers of health.” In our case, the 6-step biogas safety procedure and the plan for promotion of Chapter 6 in Huong Uoc, which came out at the end of the discussion stage, were the products of collaboration between researchers and local community. The researchers brought their scientific technical information potential risks of contracting helminth through improper treatment of human and livestock waste management^8,19 and village sanitation issues as the science base for the solution. Meanwhile, the farmers provided commune-level context and the practical constraints of the existing biogas system. Non-academic knowledge was also utilized when the farmers were active participants in providing their critique of the 6-step procedure (Table 1). In ecohealth research, the 2 principles of transdisciplinarity and participation go hand in hand.¹⁶ Four modes of participation included the following:

(1) contractual, where members of the local communities were contracted in the projects of the researchers to take part in the experiments with no decision power; (2) consultative, where local people were asked for their opinions as input for research and actions which were under the control of the researchers; (3) collaborative, where community members worked together with researchers to determine priorities but responsibility remained with researchers for directing the process; and (4) collegiate, where the local people and researchers shared their knowledge to create new understandings and work together to form action plans under the control of the local people.^20,21

In our intervention case, modes of participation could be categorized as “collegiate.” The intervention design allowed all participants in participating in defining the problems and designing the solutions (discussion stage), trialing the solution to test whether it is fit for the purpose of the target community and its socioeconomic and cultural practices (pilot and assessment stage), and finally adjusting and scaling up the solutions (scaling up stage). It is worth considering the previous NCCR research, which used disciplinary methods (epidemiological surveys) and limited the participation of many stakeholders where the researchers simply “obtained data for their research” through community members.² In the FBLI intervention, researchers transitioned from their traditional role of data receivers/knowledge providers to knowledge receivers who received feedback on the shortcomings of the 6-step biogas and the possibility of using a traditional document as legal power in addressing the sanitation issues (Table 2). The community members (farmers and local authority) were not just data and information providers but took part in the intervention in the role of knowledge contributors and users (Table 2). This transition in behavior and attitude reflected the sharing knowledge among stakeholders in creating a new way of solving local sanitation issues. The research problem was “real” since it was identified and tested by the locals. Transdisciplinary and participatory approach in this case enabled mutual learning and built capacity for each stakeholder and offered solutions to overcome the impediments identified in the previous research.²

Knowledge to action

Knowledge to action principle, focusing on the goal of the ecohealth process, is the reflection of how research or intervention results are used to inform the action of knowledge user who is likely to be able to use research results to make informed decisions about health policies.¹⁶ Research results can only be translated into policy through negotiation, interaction, and collaborative exchange among related stakeholders.¹⁶ In the FBLI intervention, the effectiveness of the promotion of Chapter 6: Sanitation through graphical posters prompted the vice chairman of the CPC to take further actions to sustain and enhance the positive impacts of this communication tool on addressing the local sanitation issue. Science research and public policy “do not always speak the same language” but can find a common cause through a long-term effective engaging process of collaboration.^16,22,23 These actions were the results of the year-long collaboration between the researchers and the local authority in Hoang Tay commune, which stemmed from the previous NCCR research.²⁴ The iterative and recursive intervention design, in this case, allowed the local authorities' reflection on the effectiveness of the intervention, which resulted in the knowledge turning into actions. Intervention results, which were assessed after piloting, were used as material/evidences for local sanitation policy. These actions also met the expectation of the previous NCCR research in “building up an evidence that provides a rationale for changing the way research and decision making is done.”²⁴ While ecohealth calls for policy changing at large scale and higher level, it should start with the local level where best practice can be learned, adapted, and maintained by the locals themselves.

To sustain ecohealth and similar integrated approaches in the absence of a relevant level of funding, the core principles of these approaches need to be “well-understood, integrated within and adapted” in relevant intervention.⁷

Implications for Policy & Practice

Our research described how a ecohealth approach was used and how its principles were applied in practice. The results gave important implications for further ecohealth research and interventions as well as other public health community-based interventions.

• First of all, participation in ecohealth interventions should be collegial and give opportunities for all related stakeholders to build their capacity. This could support the achieving of the transdisciplinarity principle. This also helps ensure the sustainability of community-based solutions when local community is motivated to sustain the improvements it has recommended for any single intervention.
• Ecohealth outcomes, in some cases, need to be captured in process. As such, a participatory monitoring and evaluation component should be an integrated part of ecohealth programs.
• Participants should be able to understand the developmental purpose of evaluation that supports the learning and decision making during program interventions.

Limitation of study

In applying a developmental evaluation approach, the feedback information from each evaluation iteration should be rapid. In our case, the data collection was, at times, hindered by fluctuation of the personnel. During the 2-year intervention, the position of a PhD student in the research team was occupied by different candidates. Some farmers in the core group left in the middle of the intervention leading to recruiting of new ones. These factors led to the fact that interviews were not always conducted with the same informants during all 3 evaluation iterations as expected. Yet, key informants, such as the head of the research team, the research assistant who conducted main field work, the entire group of local authority, and key farmers, were all included in the interviews and focus group discussions. Besides, the lack of quantitative data also resulted in shortage of indicators to compliment the outcomes. However, we deem it important to have a separated quantitative research that focuses on technical aspects of 6-step biogas' safety procedures and/or knowledge, attitude, and practices (KAP survey) of the involved participants to fully assess the effectiveness of this ecohealth intervention model.