In recent years, there has been a growing recognition of the need for operational research (OR) to provide evidence that would inform programmes, particularly in resource-poor settings [1–3].
Although there are many definitions of OR, there is general agreement that OR is directly concerned with applied issues, and that what distinguishes it from other types of research is the involvement of ‘stakeholders’ alongside scientists in the process of defining policy-relevant projects and feeding results back to programmes . Yet although the role of stakeholders in OR is considered to be key to the research process, there is no unified definition of stakeholders and no clarity about what their participation entails. International guidance documents show considerable variations in how stakeholders are identified, with some  considering ‘programme administrators’ as possible partners, others defining stakeholders as ‘policy-makers and managers’ , ‘endemic country health professionals’  or ‘policy-makers and decision-makers’ . In recent documents on OR on HIV, the range of possible stakeholders appears to be broader and to include key programme managers, donors, programme staff, programme beneficiaries, affected people and communities, and it is recommended that they be included from the start: ‘The process of priority setting should be informed by inputs from multiple players, including national programs, national research institutes, international donors, technical agencies, academic institutions, and NGOs, including PLWHA associations’ . Such statements indicate a shift, in that individuals and groups that were previously considered simply as ‘key audiences’  are now to be fully involved in the research.
Documents from the 3 × 5 (later Universal Access) initiative highlight the contribution of community-based, non-governmental organizations (NGO) and faith-based organizations to treatment and care. This contribution has been crucial in those settings in which HIV prevalence is high and trained health professionals are few , so the emphasis on country ownership and community participation reflects the reality of how care is provided. The shift in definitions of stakeholders is also likely to be an outcome of advocacy on the part of people living with HIV/AIDS (PLWA), who called for including affected groups in research and programmes. Global trends probably influenced this broader approach to stakeholder involvement, including the more prominent role given to community and patient representatives in the management of health services in general . Recommendations that OR build on ‘collaboration with local counterparts and decision-makers’ and be ‘grounded in local culture’  underscore a more participatory approach to research and a better understanding of local contexts, and thus have technical and managerial implications for the conduct of research.
The emphasis on stakeholders' participation in recent guidelines raises a number of questions. What impact does it have on the research process? Is it feasible and desirable to define categories of stakeholders that should be involved in OR? What form does stakeholders' participation take at various stages of a research project? Are there negative aspects to such participation and can there be reasons to limit it? Although there are no standard answers to these questions, it is useful to examine the feasibility of participation of stakeholders in OR and to discuss the extent to which this represents a ‘best practice’. We examine these issues on the basis of an OR project conducted in Burkina Faso, where broad stakeholder involvement was a central element. The analysis is based on our own experience with this project, on reviews that were conducted by outside experts, and on more general discussions, in Burkina and in international organizations, of the process of stakeholder participation.
Recent experience with operational research on HIV care in Burkina Faso
OR on services for PLWA was conducted in Burkina Faso, as part of a larger project supported by the World Health Organization's (WHO) HIV Department and TDR Program (project ID A50479), and implemented in five African countries between 2006 and 2008. The research theme was defined through a consultative process with representatives from each country and local and international experts. The team from Burkina Faso identified as a research priority the understanding and management of treatment and care practices in public, private, faith- and community-based facilities in the country. The objective was to describe and analyse the provision of care from the point of view of clients and providers; to assess the quality of care in various health facilities; to identify obstacles and difficulties, as well as ‘best practices’ and local solutions to address them; and to define ways for the different sectors to collaborate and harmonize care practices.
An advisory committee and a research team were set up. The principal investigator was the Comité Ministériel de Lutte contre le Sida (CMLS) from the health ministry, which is in charge of the national HIV treatment programme. The research team was led by a group based at the national health research institute, Institut de Recherche en Sciences de la Santé. The research process included three phases: situational analysis, survey, and participative/dissemination phase.
The situational analysis showed that 10 644 persons were on antiretroviral therapy (ART) in April 2006, and an estimated 23 500 needed ART, at a time when HIV prevalence was estimated at 2%  (when the process was launched in 2005, estimated prevalence was 4.5%). The survey showed that HIV care and treatment were provided in very different ways in the different facilities, with some specialized in medical treatment and others providing comprehensive care including social support. It was not possible to collect data at private facilities because of high refusal rates, although such facilities are thought to account for a considerable proportion of HIV care. Because patients may access care from several sites and multiple registrations are frequent, patient records do not provide good estimates of the numbers treated at the different facilities.
All the health facilities that provided ART to at least 20 patients were included in the study. There were 26 such facilities, located in two main cities (Ouagadougou and Bobo-Dioulasso) and in rural districts. Quantitative and qualitative methods were used to measure the provision of care, through questionnaires and interviews at health facilities. A total of 740 patients and 114 caregivers (out of the 780 and 156 targeted) was interviewed about their practices and their experience. A variety of topics were included: patients' socioeconomic characteristics, access to care, expenses related to care, adherence to treatment and stigma, caregivers' experiences, perceptions about HIV, HIV patients and HIV care, practices related to the prevention of sexual transmission among those on ART, referrals and follow-up. Quantitative data were analysed using Epi Data and SPSS 10.0, while content analysis was conducted on qualitative data. Comparisons were made among public and community/NGO/faith-based facilities and between urban and rural facilities.
After preliminary analyses were completed, the results were shared with stakeholders at workshops lasting 1.5 days; 46 stakeholders participated in three workshops that brought together caregivers, patients' representatives, programme directors and members of NGO in Ouagadougou and Bobo-Dioulasso. During plenary and group sessions participants discussed the meaning of the results, their implications for programmes, as well as potential interventions that could be implemented.
Scaling-up HIV care in Burkina Faso: context and constraints
To understand how the project was defined and implemented in Burkina Faso, it is useful to summarize information about the context of HIV treatment scale-up in the country. The first case of HIV was diagnosed in 1987. While the public health system was slow to react, local AIDS organizations mobilized to provide information, prevention, and later counselling and testing. In 2003, 479 community-based and NGOs were providing HIV care and support, a considerable number for a primarily rural country of 12 million inhabitants. Antiretroviral care was first widely provided by Médecins Sans Frontières and the Red Cross, in collaboration with the health ministry. The unavailability of antiretroviral drugs in public facilities led some organizations to open care facilities and to employ health workers along with social workers, and the number of these facilities increased after 2000. At the time of the study, more than 12 community-based facilities provided HIV care including ART, primarily in urban areas. The provision of medical treatment outside medical facilities was not easily accepted by health professionals who were concerned about the lack of resources in public hospitals. The non-governmental sector received funds from private donors and international agencies, keen to support ‘civil society’. Different donors promoted different models of care, so that by 2004 there were, for example, four different electronic toolkits for patients records. All caregivers suffered from the heterogeneity in the provision of care and tended to distrust patients who might be registered in several facilities. Doctors from the public sector came to question the legitimacy of medical care provided by NGOs (associations), partly as a result of a lack of knowledge about it. Inequalities in salaries and/or working conditions compounded the distrust between the community and the public health sectors. While this complex situation prevailed in urban areas, the situation of rural areas was not documented.
The advisory committee reviewed the results of the study and made recommendations for follow-up regarding the supply of antiretroviral drugs, access to ART and care, care practices and patients follow-up, and coordination among care facilities. These are summarized as follows:
Administrative and management changes were recommended to shorten the excessive delays in ART supply.
The analysis showed that the monthly cost of ART (UD$10) and biological follow-up (US$6) in public facilities were responsible for treatment interruptions, and that the problem was especially acute in rural areas, where NGO and community-based organizations are few and patients cannot get support to meet these costs.
Patients' narratives highlighted ethical problems, especially concerning confidentiality in the private sector, but very few caregivers or patients in the private sector were willing to participate in the study. A further study was recommended.
A high level of satisfaction with the quality of care was reported, particularly in NGO/community/faith-based facilities. Comprehensive care, as provided in these facilities, was acknowledged as a ‘best practice’, and some caregivers from public facilities decided to set up or reinforce their collaboration with associations through formal agreements.
The study revealed the complex itineraries of patients, as they try to obtain the necessary care from various facilities. As caregivers everywhere have little time and few resources to ensure patients' continuity of care, it was recommended that patients be allowed to choose a main health facility for follow-up, that more caregivers be trained in ART management, and that referral mechanisms be set up to facilitate comprehensive care and reduce the duplication of services and records.
Stakeholders' participation at different stages of the project
At the initial phase of the project when priorities were set and research questions defined, a consultation brought together a representative of Burkina Faso CMLS, the directors of HIV teams from two public health research centres and the head of the national PLWA network (two scientists and two stakeholders) (Table 1). Despite their divergent perspectives, they shared the same concerns with the fragmentation of HIV care, itself rooted in its social history. The scope of the selected OR project, which sought to assess care in public and NGO facilities, reflected the composition of the team and the inclusion of a representative of the associative sector, who ensured that the project was relevant for PLWA and accepted by NGO. During the research design and planning phase, exchanges were facilitated by the relatively small size of the group; interactions among scientists, administrative and PLWA representatives were relatively easy and egalitarian, and previous collaborations among members who had worked together on AIDS interventions fostered the emergence of trust and a common understanding about the research goals.
The advisory committee was composed of approximately 30 stakeholders including ministry of health programme directors and managers, representatives of public and private practitioners, PLWA associations, university staff, members of faith-based organizations and NGO and international agencies. During the first meeting, the project and its articulation with other initiatives were vigorously discussed; this was important later for a wide recognition and ownership of the project.
As it was agreed to represent multiple stakeholders in the implementation phase, the research team included 14 individuals from research centres, the health ministry, associations and AIDS international programmes. The key research questions, methodology and tools were agreed upon during a 4-day meeting. This phase allowed discussions about practices or perceptions that ought to be considered, and allowed the sharing of experiences among all participants. Field studies were conducted by a team of 10 qualified interviewers including members of PLWA groups. Stakeholders' participation in the research team was made possible by the capacity of some individuals to play multiple roles, and members of PLWA support associations were included on the basis of their professional skills in sociology and field research. A first set of results was presented to the advisory committee, and they drew attention to a number of specific issues and requested complementary analyses.
The third phase of an OR project is usually devoted to dissemination. In this project outcomes were defined as those key recommendations that were made based on discussions of quantitative and qualitative results. This was done through workshops that brought together approximately 20 participants, including 15 local stakeholders, and encouraged the sharing of experiences and lessons learned, as well as discussion of particular cases. In the early days of ART provision, it was common to engage in such exchanges, and this approach has since been adopted by those actors who are involved in HIV care, for example in Bobo-Dioulasso. In addition to sharing information and strengthening capacities, these discussions among stakeholders contributed to fostering collaborative efforts among those working with patients in different parts of the health system. They also provided highly relevant local data and rich practical suggestions, although they may not be readily generalizable or amenable to scientific analysis.
Discussion: achievements and limits of stakeholder participation
Research in developing countries is often defined at the international level with insufficient consideration of local priorities, and the non-involvement of local stakeholders in the research process is a limit to the relevance and applicability of results. In this research project on the scale-up of HIV care, a broad participation of stakeholders was implemented from the early stages. There were several reasons for this. Because the selected theme included care practices in the private, public, community-based/faith-based/non-governmental sectors, and at various levels of the health system, the participation of caregivers and managers from these sectors was seen as necessary to define research questions that were relevant for each sector. Stakeholder involvement was also necessary to facilitate the acceptance of the study in the field, and avoid the potential reluctance that may have emerged if the study was perceived as an external evaluation. Moreover, when tackling harmonization of care at the level of the health system, the presence of various representatives in the advisory committee helped clarify specific aspects and discuss issues related to collaboration among facilities.
During the data collection phase, the participation of PLWA and members of PLWA associations in research teams helped build confidence among individuals who were interviewed and facilitated their participation in the study. They accepted to speak on sensitive topics such as sexuality or relationships with healthcare workers, which might not have been the case with a research team including only scientists, and it is very likely that the quality of data improved as a result.
The presence of numerous stakeholders in the research team and the advisory committee did, however, create some difficulties: with such a broad research theme, each member sought to include additional questions for the study, and it was not always easy for the principal investigator to negotiate competing demands; this also resulted in research instruments that were excessively long and difficult to manage. With a large group of stakeholders and a relatively large research team, the involvement of investigators in the implementation of research was uneven, perhaps as a result of a lack of felt ownership for the project among some, or to the difficulty of harmonizing personal and collective agendas. More general obstacles regarding stakeholder participation were the limited time that some members – particularly administrative members – were able to allocate to the project; and also the urge to identify immediate solutions, particularly when caregivers were not familiar with the research process. Although differences among stakeholders in social status and in familiarity with scientific language and methods were handled relatively well within the advisory committee and research team, such differences cannot be completely ignored and they may explain the low participation of some stakeholders in the final phase of results writing. Also, in spite of the inclusion of a representative of private practitioners in the advisory committee, the study failed to enlist sufficient participation from caregivers and patients in this sector. It is unclear whether this was caused by the lack of representativeness of the committee member, a general reluctance towards enquiries in the private sector, greater time pressures or opportunity costs of those in private facilities, or a combination of these.
Overall, the advantages of a broadly defined research topic and the early involvement of multiple stakeholders exceeded the disadvantages. The research topic might not have been chosen by an academic researcher nor by a programme director alone, as the former might have preferred a more focused and more scientifically designed study and the latter might have felt little legitimacy for tackling issues beyond the limits of his programme. The theme and scope of the study, however, fit particularly well with the goals considered by the head of CMLS as priority throughout the health system and including all sectors and types of care.
Stakeholders' reactions to the results of the study varied, depending on whether they called for practical responses or for broader reformulations of care. Some results showed gaps that required technical answers; on such topics as delays in ART provision, the committee meetings and workshops were an opportunity to gather key persons and discuss possible innovative responses. Other topics necessitated that a common perception of the problem be built before proposals for change could be made. This was the case for the question of cost obstacles to AIDS care. Although Burkina Faso's ART programme is among the last in West Africa to request payments from patients, there was little advocacy on this topic, mainly because facilities in community-based organizations and NGO had found ways to exempt the majority of patients from fees by classifying them as ‘indigent’. The results of the study, showing the need for free access to ART and care especially in rural areas, led stakeholders to a shared view of the problem and helped mobilize them around this issue. Whereas the specific study results were important in that respect, it was the research process itself that contributed to this common perspective, by gradually building a consensus around the issue and the need to address it. As the decision about pricing must be taken by health authorities or at a higher level, the strategy chosen by stakeholders was to organize an international conference on the management and effects of free access to ART care. The research process opened a social space for a ‘peaceful protest’; this was important, as the involvement of PLWA support associations in providing care now makes it more difficult for them to voice patients' demands and critiques. Advisory committee and workshop meetings also facilitated exchanges among individual members about specific topics, and probably contributed to the development of collaborative interactions and more open discussions of issues regarding health care.
The experience analysed here shows that the effective participation of stakeholders, including community and patient representatives, at all stages of an OR process is more than a rhetorical point, and can make a difference to the relevance, quality and implementation of OR. These findings are consistent with the results of a review of the WHO/GFTAM Program including four other research projects (C. Obermeyer, F. Samuels, W. Koster, J. Kinsman, O. Ky-Zerbo, J. Simbaya, 2007, unpublished report to the Global Fund to Fight AIDS Tuberculosis and Malaria), which highlighted the role of the appropriate involvement of stakeholders and the way in which an advisory committee can function to ensure such multiple participation. Based on the analysis of that and other projects, it is possible to recommend that stakeholders' participation be a necessary component of OR, because if properly defined and applied, it can ensure greater relevance of the selected topic, better implementation of the research, and more direct application of results to programmes.
This analysis also shows that stakeholder participation does not happen just because individuals are invited to participate in a project, as some may, for a variety of reasons, be reluctant to join. A recent review of OR projects provides some advice about specific practices that can enhance stakeholders' participation , but it is clear that a broad commitment is necessary from several key players in order for the process to work. There may, in addition, be an optimal level of stakeholder participation, both in terms of size (not too big) and composition (not only prominent members) of the group, beyond which disadvantages become apparent. These may include an overly broad scope of the project, delays in implementation, difficulties in coordination and unequal involvement of members of research and advisory teams at different phases of the work.
Whereas this analysis sought to exercise a degree of reflexivity in examining the process and identifying lessons learned based on data collected in the course of a specific project, in general, the evidence about stakeholder participation is patchy, and more systematic information is needed. Perceptions and incentives of various participants need to be considered, and some observers have recommended including stakeholder involvement as a component of the evaluation of reproductive health programmes . More in-depth studies using tools from the social sciences , and including structural/contextual factors can examine the process and its consequences for OR in different contexts, and throw light on the long-standing question of translating recommendations into improved practices.
The authors wish to acknowledge the contribution of the following members of the Burkina research team: D. Bakouan, A. Berthe, B. Bila, F. Bocoum Yaya, B. Doulougou, S. Kouanda, N. Meda, P. Mselatti, A. Ouiminga, M.J. Sanou, M. Sawadogo, B. Sondo, A. Somda, M. Yameogo, and to thank the many women and men who participated in the study and shared their sometimes painful experiences of HIV.
Sponsorship: The project (ID A50479) on which this analysis is based was supported by the HIV Department, WHO, under the 3 × 5 initiative. This paper, however, represents the views of its authors and not of the World Health Organization.
Conflicts of interest: None.
1. World Health Organization. Operational research to support HIV treatment and prevention in resource-limited settings: summary of activities, July 2004 to January 2006
. Geneva: World Health Organization, Department of HIV/AIDS and Special Programme for Research and Training in Tropical Diseases; 2006. Available at: http://www.who.int/hiv/pub/operational/or_summary_2006.pdf
. Accessed: February 2009.
2. World Health Organization and the Global Fund to Fight AIDS, Tuberculosis and Malaria. Guide to operational research in programs supported by the Global Fund
. Geneva: World Health Organization, Department of HIV/AIDS and the Global Fund to Fight AIDS, Tuberculosis and Malaria; 2008. Available at: http://www.who.int/hiv/pub/epidemiology/SIR_operational_research_brochure.pdf
. Accessed: February 2009.
3. Obermeyer C, Makhlouf, Bott S, Carrieri P, Parsons M, Pulerwitz J, et al
. HIV testing, treatment and prevention: generic tools for operational research
. Geneva: World Health Organization; 2009.
4. World Health Organization. Expanding capacity for operations research in reproductive health: summary report of a consultative meeting
. Geneva, WHO; 2003. http://www.popcouncil.org/pdfs/frontiers/reports/WHO_expand_capacity.pdf
. Accessed: October 22, 2009.
5. Fisher AA, Laing JE, Stoeckel JE, Townsend JW. Handbook for family planning operations research design
. New York, Population Council; 1991. Available at: http://www.popcouncil.org/pdfs/frontiers/Capacity_Bldg/HbkFPOR.pdf
. Accessed: October 22, 2009.
6. World Health Organization/Tropical Diseases Research. Guidelines for preparing a grant application to the TDR Steering Committee for Implementation Research
. Geneva: WHO/TDR; 2005.
7. Sanders D, Haines A. Implementation research is needed to achieve international health goals. PLoS Med 2006; 3:719–722, e186.
8. World Health Organization. Evidence to improve HIV treatment and prevention programs: operational research to support HIV treatment and prevention in resource-limited settings: summary of activities, July 2004 to January 2006
. Geneva: WHO; 2006.
9. Marin MC, Bertrand JT. Evaluation manual: evaluating operations research utilization: guidelines for assessing process and impact
. Washington, DC: Population Council, FRONTIERS Programme; 2003. Available at: http://www.popcouncil.org/pdfs/frontiers/Evaluation_Manual.pdf
. Accessed: October 22, 2009.
10. Van Damme W, Kober K, Kegels G. Scaling up antiretroviral treatments in Southern African countries with human resource shortage: how will health systems adapt? Soc Sci Med 2008; 66:2108–2121.
11. Cornwall A, Shankland A. Engaging citizens: lessons from building Brazil's national health system. Soc Sci Med 2008; 66:2173–2184.
12. Comité National de Lutte contre le Sida et les IST-Burkina Faso, ONUSIDA. Rapport UNGASS du Burkina Faso. Suivi de la déclaration d'engagement sur le VIH/SIDA, 2008
. Available at: http://data.unaids.org/pub/Report/2008/burkina_faso_2008_country_progress_report_fr.pdf
. Accessed: October 22, 2009.
13. Hennink M, McEachran J. How to use the conceptual work to increase research participation.
In: WHO/HRP. Turning research into practice: suggesting actions from case-studies of sexual and reproductive health research
. Geneva: WHO/HRP; 2006. pp. 20-27. Available at: http://www.who.int/reproductivehealth/publications/general/9241594837/en/index.html
. Accessed: November 26, 2009.
14. Brugha R, Varvasovszky Z. Stakeholder analysis: a review. Health Pol Plan 2000; 15:239–246.