Skip Navigation LinksHome > July 2008 - Volume 22 - Issue > Why and how to monitor the cost and evaluate the cost-effect...
AIDS:
doi: 10.1097/01.aids.0000327626.77597.fa
Section I: Informing Scale-up and Resource Allocation

Why and how to monitor the cost and evaluate the cost-effectiveness of HIV services in countries

Beck, Eduard Ja; Santas, Xenophon Mb; DeLay, Paul Ra

Free Access
Article Outline
Collapse Box

Author Information

From the aEvidence, Monitoring and Policy Department, UNAIDS, Geneva, Switzerland

bGlobal AIDS Division, Centers for Disease Control and Prevention, Atlanta, Georgia, USA.

Correspondence to Eduard J. Beck, Evidence, Monitoring and Policy Department, UNAIDS, 20 Avenue Appia, Geneva 27, 1211 Switzerland. Tel: +41 22 791 5521; e-mail: becke@unaids.org

Collapse Box

Abstract

The number of people in the world living with HIV is increasing as HIV-related mortality has declined but the annual number of people newly infected with HIV has not. The international response to contain the HIV pandemic, meanwhile, has grown. Since 2006, an international commitment to scale up prevention, treatment, care and support services in middle and lower-income countries by 2010 has been part of the Universal Access programme, which itself plays an important part in achieving the Millennium Development Goals by 2015. Apart from providing technical support, donor countries and agencies have substantially increased their funding to enable countries to scale up HIV services. Many countries have been developing their HIV monitoring and evaluation systems to generate the strategic information required to track their response and ensure the best use of the new funds. Financial information is an important aspect of the strategic information required for scaling up existing services as well as assessing the effect of new ones. It involves two components: tracking the money available and spent on HIV at all levels, through budget tracking, national health accounts and national AIDS spending assessments, and estimating the cost and efficiency of HIV services. The cost of service provision should be monitored over time, whereas evaluations of the cost-effectiveness of services are required periodically; both should be part of any country's HIV monitoring and evaluation system. This paper provides country examples of the complementary relationship between monitoring the cost of HIV services and evaluating their cost-effectiveness. It also summarizes global initiatives that enable countries to develop their own HIV monitoring and evaluation systems and to generate relevant, robust and up-to-date strategic information.

Back to Top | Article Outline

Introduction

Middle and lower-income countries that are scaling up HIV services need access to robust and contemporary strategic information to develop and implement these services, and monitor and evaluate them once they are established. Such strategic information includes financial information on resources received and spent, but also needs to include information on the cost and cost-effectiveness of these programmes and services. In addition to the scaling up of services, new prevention and therapeutic intervention will continue to come on stream, all of which will affect the way services are provided to the increasing number of people living with HIV around the world.

The estimated number of people living with HIV in 2007 was 33.2 million (range 30.6–36.1 million), of whom 93% were adults over 15 years of age and 46% were women [1]. Worldwide, an estimated 2.5 million (range 1.8–4.1 million) new HIV infections occurred in 2007, 84% of which were in adults, an annual rate that has not changed substantially since 2001. In 2007 an estimated 2.1 million deaths (range 1.9–2.4 million) were attributable to HIV infection or AIDS, which has decreased from 3.9 million (range 3.3–5.8 million) in 2001 [1].

Although HIV infection is a biomedical problem, successful containment of the pandemic requires integrated national and international multisectoral responses, as recognized in the 2001 UN Declaration on HIV/AIDS [2]. As part of the 2001 Declaration of commitment, UN member states agreed to collect and submit regularly a defined set of indicators as part of their United Nations General Assembly Special Session (UNGASS) reporting requirements, including information on health and social services as well as economic and financial indicators [3]. Subsequently, the country-led universal access programme was unanimously endorsed by United Nations member states in the 2006 Political Declaration on HIV/AIDS [4]. Its focus is for middle and lower-income countries to scale up HIV prevention, treatment, care and support services by 2010, while recognizing that individual countries need to set their own ambitious yet realistic targets [4,5]. It has also become an important stepping stone towards achieving the Millennium Development Goals (MDG) in 2015 [6].

Within this context, many countries are now in the process of scaling up HIV prevention, treatment, care and support services within the framework of the ‘Three Ones’ [7], which provides a focus around which HIV programmes can be developed, maintained and improved. It recommends that each country set up: one agreed national HIV action framework; one national agency or authority with appropriate technical capacity responsible for coordinating the efforts of all relevant stakeholders; one national multisectoral monitoring and evaluation framework to track the response.

As part of the international response to the HIV pandemic, financial resources made available to middle and lower-income countries have increased substantially since 2001 [8] (Fig. 1). These increased resources have come from countries through the World Bank's Multicountry AIDS Programme, the WHO/UNAIDS ‘3 by 5’ initiative, the US President's Emergency Program for AIDS Relief (PEPFAR), the establishment of the Global Fund to Fight AIDS, Tuberculosis and Malaria, the Clinton Foundation HIV/AIDS Initiative, and other multilateral and bilateral programmes [9].

Fig. 1
Fig. 1
Image Tools

In 2007, an estimated US$10 billion was made available to HIV-related programmes in middle and lower-income countries, an increase over US$8.9 billion in 2006 [8]. Even more funding will be needed to achieve universal access. If the scale-up of HIV services continues at the recent pace, however, the funding levels– projected to reach US$15.4 billion in 2010 and US$22.5 billion in 2015– will not be sufficient to achieve universal access by either 2010 or 2015. On the basis of these increases, only 4.7 million people would be able to receive antiretroviral drugs in 2010, or two-thirds of those people living with HIV estimated to need antiretroviral therapy (ART) in 2006. By 2015, funding would be available for ART for an estimated 8 million people [8].

Meanwhile, lower-income countries account for approximately half of the estimated global resource needs for 2008, whereas lower middle-income countries will account for approximately one third [10]. The need to monitor and evaluate HIV financial resources critically over time as part of the overall response to the HIV pandemic is even greater in these countries that already have more limited resources and a higher burden of HIV infection. This paper therefore focuses on: (i) the need to recognize cost and cost-effectiveness information as important strategic information required by countries scaling up HIV services; (ii) the complementary relationship between monitoring the cost of HIV service provision and evaluating the cost-effectiveness of programmes or services over time; (iii) a brief overview of how such strategic information can be collected, stored, transferred and used within a country's HIV monitoring and evaluation system.

Back to Top | Article Outline

Cost and cost-effectiveness as strategic information

Strategic information

Strategic information can be defined as information that is important or essential in relation to a plan of action [11] for programmes or services. For any programme to succeed in the long run, it must be sustainable and strengthen local services [12]. The success of any programme or service needs to be evaluated in terms of its effectiveness, efficiency, equity of coverage and acceptability to both users and providers [12,13]. In this context ‘effectiveness’ refers to the outcome or impact of programmes or services; ‘efficiency’ focuses on the resources required to achieve a certain outcome or impact; ‘equity’ considers who benefits from programmes or services; and ‘acceptability’ refers to both the acceptability of programmes or services to users and providers and to the improvements in quality of life they achieve [13]. Although applicable to all countries, the need to monitor and evaluate these criteria critically is more urgent in middle and low-income countries where infection rates are higher and resources more limited.

For many health professionals, however, ‘strategic information’ has to date referred to information on the use of services, including inputs, outputs and coverage by services; outcome, reflecting changes in the health status of individuals [13,14]; and impact of services, reflecting changes in health status at the population level [12]. For a variety of reasons, financial and economic information has until recently not usually been considered to be part of ‘strategic information’. The higher burden of HIV infection rates in middle and low-income countries, their more limited resources, the chronicity of HIV infection, the cost of scaling up interventions, programmes and services, as well as the increasing international financial response have all helped change our understanding of what comprises ‘strategic information’. Without adequate financial resources, countries will be unable to implement necessary programmes or scale up services at community, health facility, subnational and national levels, especially if these need to be sustained over the long term. Any such failure will not only make it difficult to meet their universal access targets, it will also affect their ability to meet their MDG targets.

Back to Top | Article Outline
Financial strategic information

Financial information for service provision falls broadly into two categories: information on how much is spent and the actual cost of service provision. The amount of money spent on services can come from the government sector or civil society, with the latter including both non-governmental organizations and the private sector. In well-established nation states, central, regional or state government organization provides financial resources to communities or facilities where services are provided. One needs to differentiate between resource or budget allocation, which refers to ‘planning for using available resources… to achieve goals for the future. It is the process of allocating resources among the various projects or business units’ [15], aiming for the maximum possible output from a given combination of resources. Expenditure, on the other hand, refers to the funds actually being spent at the various levels, which for government organizations varies from expenditure at central level down to facility or community levels.

Countries now have various methods to track how HIV-related resources are allocated and spent. These methods include the use of budget tracking [16], general national health accounts (NHA) [17] and HIV-specific health accounts such as the national AIDS spending assessments (NASA) [18]. Whereas the NHA provide countries with information on general expenditure across the health sector, the NASA provide more detailed HIV-related information required by those working within the HIV sector. The NASA, however, cover both the health sector and non-health sectors. A functional integration between NHA and NASA is now being developed collaboratively by UNAIDS, WHO, USAID and Abt Associates. When completed, it should make more detailed information available to countries, integrating information on general health sector expenditure with specific HIV spending. Although reasonable information can be obtained at global or national levels, however, information at subnational, health facility or community levels has been more difficult to obtain to date.

Information on the allocation or expenditure of financial resources may not coincide with the financial resources actually required to provide a service. Determining the financial resources needed to provide services requires information directly obtained from those facilities that provide them. For service providers, two additional types of financial information are therefore important: one relates to the cost of service provision and the other to the efficiency of services.

Costs, estimated through costing or ‘cost-of-illness’ studies, refer to the monetary resources required to implement an intervention or provide a particular service [13]. It is estimated by multiplying indices on the use of services by their respective unit costs and answers questions like ‘what does it cost to run an HIV service? Is the service affordable?’

Costs include direct costs, which are the costs involved with the direct provision of services in facilities or communities and indirect costs. Indirect costs include lost productivity, valuation of time and expenses incurred by users of services and family members as a consequence of a particular illness and its management, informal care provided by family and friends, and intangible costs of stress and anxiety caused by that disease. When considering only the public sector, direct costs provide a public sector perspective, whereas when indirect costs are also included this is said to provide a societal perspective [13].

For costing studies, the costs of services provided require the valuation of personnel, therapeutic or preventive interventions, tests and care facilities such as the use of hospital beds, clinics and programme costs. Often routine cost data may not be available and prices or charges have to be used as the unit of valuation. It has long been recognized, however, that prices or charges of healthcare resources may not be synonymous with costs; in fact charges may often seriously underestimate and prices may overestimate actual costs [19,20].

Costing studies can be complex, difficult and time-consuming but need not be performed annually [21]. Two main obstacles include scarcity of resources to perform the costing exercise and the lack or inaccessibility of data. Costs can be calculated using either a ‘top-down’ or ‘bottom-up’ approach [13]. In the top-down method, information on expenditures is allocated to specific services provided, which is more easily administered and takes less time to perform. Services provided in this context can refer to the number of people counselled and tested for prevention services or the number of people living with HIV treated in a health facility. Relationships between the use of services, costs and client characteristics are only weakly defined [22]. The bottom-up or ingredients approach, which collects all relevant resource inputs to provide the service, allows greater flexibility in relating the use of services to costs and patient characteristics, although the method can be time-consuming and expensive. Furthermore, some items, such as hospital overheads, can often not be exclusively attached to individual clients but need either to be averaged for all patients or identified as a separate cost. The bottom-up approach is, however, generally regarded as superior for cost estimation [13,22].

Efficiency concerns the resources required to provide a particular service and achieve a particular outcome or impact. It always involves combining information on costs with information on outcomes as well as a comparison, either comparing the activity with doing nothing or comparing a new intervention, programme or service with an established one. Efficiency studies answer questions such as ‘what does it cost to achieve a certain outcome or impact? Does a new intervention add value compared to doing nothing or an existing intervention?’ In order to determine the efficiency of interventions, programmes or services, costs are linked to outcome measures, as done through cost-effectiveness, cost-minimization, cost-utility or cost-benefit studies. In cost-effectiveness studies, costs are linked with biological or clinical outcomes such as life-years gained for treatment or infections averted for prevention services. For cost-minimization studies, the effectiveness of the interventions being compared is similar, but the resources required to achieve a particular outcome may differ. In cost-utility studies, biological outcomes are converted into disability-adjusted life-years or quality-adjusted life-years, which weight life expectancy by a value of the quality of disability or life, respectively. Cost-benefit studies translate biological outcomes into monetary ones, allowing for the overall analysis to be expressed in monetary terms [9,13]. The cut-off point for what constitutes a ‘cost-effective’ intervention requires the implicit acceptance of a threshold of what a society is ‘willing to pay’ for another year of life or other outcome. These thresholds vary among countries [23].

Back to Top | Article Outline
Financial gaps

By combining financial information on what is spent on HIV services and comparing it with the financial estimates of the cost of providing these services, policy makers can determine if a financial gap exists between what is spent and what is needed and make the necessary adjustments by now being able to answer questions such as ‘are additional resources required to provide services?’ Similarly, with information obtained from cost-effectiveness studies, informed judgements can be made on whether introducing an intervention, programme or service will require additional financial resources or whether the new programme is more efficient than existing programmes and saves resources.

Back to Top | Article Outline
Monitoring and evaluating the HIV response in countries

Information on both cost and cost-effectiveness within countries will be required over time. Apart from information for scaling up existing programmes, new prevention and therapeutic interventions continue to be developed, some of which will be included in routine services. The most recent prevention intervention now being introduced into routine services is male circumcision, whereas work continues to be done in the area of microbicides, the development of an HIV vaccine, pre-exposure prophylaxis and others [24–26]. Increasing numbers of antiretroviral drugs are being developed and coming on the market [9,27,28], whereas more middle and lower-income countries are now confronting the situation that people are failing first-line ART and need to go on second-line therapy [9].

To monitor information on the cost over time, robust and routinely available cost information is necessary. Cost-effectiveness studies, in contrast, not only require detailed information on the cost of a specific intervention, programme or service, but also need to be combined with information on its effectiveness. Such information is less likely to be routinely available and therefore more likely to require a special study or evaluation. Such evaluations are made easier, however, if the data they require are stored in an accessible manner and are of good quality.

Back to Top | Article Outline
Monitoring costs and evaluating cost-effectiveness: an example from Singapore

The aim of any national monitoring and evaluation framework is to generate the strategic information to identify which programmes are required, guide their implementation and monitor and evaluate their outcome and impact. Within this context, monitoring is the regular production and review of defined indicators; evaluation is the more in-depth analysis of trends, based on scientific research methods, assessing why certain targets were or were not achieved. Both monitoring and evaluation are complementary parts of operational research [12], which can be defined as ‘a method of mathematically based analysis for providing a quantitative basis for management decisions’ [27].

Indicators are often based on aggregated data, which can be derived from individual-level information, and provide useful summary information on whether a particular programme or service is achieving its objectives. If monitored over time, indicators can describe time trends. By themselves, however, indicators rarely explain why certain results were obtained, or why certain trends were observed. Such explanations usually require more in-depth analyses based on the inputs and processes involved, and outcome or impact achieved, requiring access to the relevant primary data, some of which will be individual level.

Two studies performed in Singapore demonstrate the complementary relationship between cost and cost-effectiveness studies and the information coming out of them. In the second half of 1997, discussions took place in Singapore on whether the government should continue to subsidize ART. This debate was precipitated by new fiscal constraints at the Singaporean Centre for Disease Control (CDC), Tan Tock Seng Hospital, the main public centre for treating people living with HIV. Physicians at CDC had been able to prescribe ART free at the point of delivery since June 1996, when CDC started to manage substantial numbers of people living with HIV. In September 1997, however, fiscal constraints imposed by the Government prevented physicians from continuing to prescribe ART free at the point of delivery. Since then, only those who were insured or were able to pay themselves could continue to receive ART at CDC [28,29].

At the end of 1997, a study reviewed monthly trends of the number of mean inpatient days per patient, the proportion of people on ART and hospital charges per HIV-infected person managed at CDC [28]. As the proportion of persons on ART increased, the mean number of inpatient days per month decreased significantly, as did average monthly hospital charges (Fig. 2). This change suggested a relationship between the increasing proportion of people on ART, the reduction in inpatient days and the levelling off and subsequent reduction in monthly hospital charges.

Fig. 2
Fig. 2
Image Tools

When these indicators were analysed within a multivariate regression model setting monthly hospital charges as the dependent variable, the adjusted R2 was 0.63 and P < 0.001. So 63% of the changes in monthly hospital charges could be explained by changes in mean monthly inpatient days and the proportion of people on ART. Within this model, a significant direct relationship was observed between monthly hospital charges and mean monthly inpatient days (t = 5.14, P = 0.0002) and to a lesser extent between monthly hospital charges and the proportion of people on ART (t = 2.13, P = 0.05). When mean monthly inpatient days was set as the dependent variable, the adjusted R2 was 0.65 with P < 0.001 and a significant inverse relationship was observed between the mean monthly inpatient days and the monthly proportion of people on ART (t = −2.37, P = 0.03).

In summary, the higher proportion of people on ART resulted in a reduction of inpatient days, which itself reduced the monthly hospital charges, despite increased spending on ART. The exact relationship between the use of different types of antiretroviral regimens, dual or triple therapy, the cost of service provision, the outcome for people on dual or triple antiretroviral regimens and therefore the cost-effectiveness of ART could only be inferred from this analysis. For example, during the early period June 1996 to September 1997, patients were started initially on dual therapy, whereas later in 1997 more people were started on triple therapy. In order to obtain a more detailed understanding of the relationships between these indicators and the cost-effectiveness of different ART regimens in this setting, a more comprehensive evaluation was performed in 2003.

The follow-up study covered the period January 1996 to December 2001 and was based on prospectively collected individual-level information stored in an HIV disease registry [29]. That study demonstrated the cost-effectiveness of dual and triple ART compared with no ART, with triple being more cost-effective under some conditions compared with dual therapy (Table 1). The cost-effectiveness study thus confirmed the inferences based on the 1997 study and provided more detailed information on the outcome of different antiretroviral regimens.

Table 1
Table 1
Image Tools

The 1997 study used aggregate indicators to review trends over time and correlations could be calculated between these indicators, whereas the cost-effectiveness study required disaggregated individual-level information, enabling more in-depth evaluation to be performed. The Singaporean example highlights the complementary nature between monitoring, using specific indicators, and the evaluation and explanation of trends, using more detailed sources of information. The monthly charges, generated routinely within the hospital, indicated a certain trend but in themselves had little explanatory power. This explanatory power was improved when correlated with some other hospital indicators, i.e. the monthly mean number of inpatient days and monthly proportion on ART. Only when a full cost-effectiveness analysis was performed, however, was the cost-effectiveness of ART of different drug combinations demonstrated in this setting. In order to be able to perform the cost-effectiveness analysis, however, individual-level data had to be available beyond the aggregated information routinely produced by the hospital.

These analyses were performed in an industrialized country with a fairly sophisticated information infrastructure, which could both produce aggregated indicators and collect relevant disaggregated information prospectively. However, even within this setting data collection and analysis faced challenges, raising the question as to whether middle and lower-income countries can develop the conditions required to carry out such analyses routinely. Because countries indicate that they would like access to strategic information relevant to their own context, which steps should these countries take in order to achieve such goals?

Back to Top | Article Outline

Requirements for generating robust financial and other strategic information in countries

When the published HIV cost literature was reviewed in 2000, there was relatively little published information on the cost of HIV services in middle or lower-income countries [30], whereas an updated literature review, covering the years 2000–2005, identified an additional 12 published costing studies from these countries [21]. A review of published studies on the cost-effectiveness of HIV-related interventions primarily covering the years 1994–2004 revealed that most of them were set in the United States or Europe, and only a few specifically examined sub-Saharan Africa or Asia; none were identified from Latin America [31]. Of those few studies performed in resource-limited settings, most relied on outcomes or costs generated from multiple sources, either from within or outside the country studied, or they had to use modelling studies because of insufficient effectiveness and cost data [31]. As reported elsewhere, many of the published studies for health facilities and households used data collection and analysis methods, which were either unclear or non-standardized [32,33].

To address some of these problems, a number of national and international initiatives have been undertaken to overcome these logistic issues and enable countries to collect and analyse relevant strategic information over time, including standardized information on the cost of service provision, as well as enabling them to perform cost-effectiveness or other evaluation studies.

In some of these initiatives, members of international agencies have worked with national experts to determine which data need to be collected and for what purpose; how such information can be collected, stored and transferred; and how to use and analyse the strategic information generated. Data on service provision are usually collected for two distinct but related purposes. First, service providers maintain records of services provided to individuals, important for both preventive and therapeutic health services as well as social services. For people living with HIV, the existence of good longitudinal service records is extremely important to optimize services. The second important role for such information is as a source of data to generate strategic information to monitor and evaluate the success of programmes. As described in the Singaporean example, such information can exist either in disaggregate or aggregate formats. The existence of routinely available, robust and up-to-date strategic information is, however, contingent on the availability of high quality primary data, collected at the site of service provision.

Back to Top | Article Outline
Data to be collected

To guide middle and lower-income countries on which data to collect, several ‘minimum’ or ‘essential’ datasets have been proposed and agreed on between representatives and experts from relevant national and international organizations. WHO organized the development of a clinical minimum HIV dataset in March 2004 for scaling up HIV treatment and care [34,35], which formed the basis for the agreement on indicators for monitoring the scale-up of HIV services at subnational, national and international levels achieved at a UNAIDS-sponsored meeting in October 2004 [36]. In 2006 a minimum socioeconomic dataset was developed, which identified those data that could be collected by countries at patient, household, facility and programme levels [21]. Similar datasets and guidelines are in the process of being defined, including those on HIV care and treatment in children and HIV/tuberculosis co-infection. The Interagency Task Team on Prevention of Mother-to-Child Transmission has been working on defining data to collect for service provision and monitoring and evaluation, and a similar working group is being set up to address testing and counselling. To date no consensus has been reached on secondary or behavioural surveillance. Although these datasets and guidelines provide guidance to countries, their implementation is ultimately guided by specific country requirements and conditions.

Back to Top | Article Outline
Data collection, storage and transfer

The primary reason for collecting service information is to improve the quality of services for individuals. Such data are collected and need to be stored in facilities or communities where services are provided. Countries and international organizations have developed paper-based collection systems based on agreed datasets. Eventually, data collected on paper will need to be transferred into an electronic format to ease the transfer of information and analyses. In the health area, electronic medical records are now being developed and implemented for use in resource-limited settings [37,38]. For transferring data between electronic systems, an HIV-specific Health Level 7 (HL7) electronic message now enables the transfer of individual-level information between sites [39]. Similarly, the transfer of HIV indicators is now possible using an extensible markup language (XML) message [40].

Many countries are also developing national chronic disease registries for HIV, based on individual-level clinical information. An extension of these disease registries has been the creation of data warehouses to store individual-level information in central electronic data systems. Unlike chronic disease registries, HIV data warehouses aim to store data from multiple sources, including health and social services as well as other societal sectors. A number of middle and lower-income countries, which have been developing HIV disease registries, including Brazil, South Africa, Botswana and Rwanda, are now converting these registries into national data warehouses.

Under the guidance of the UNAIDS Monitoring and Evaluation Reference Group, an indicator repository has been developed to assist countries establish their own indicator registries for storing national and international indicators [41]. Countries can populate them with country-specific numerator and denominator data and generate their national indicators. Indicators can then be sent from this registry to healthcare professionals at local, subnational, or national levels and international agencies. Alternatively, they can be sent to the various indicator databases such as the Country Response Indicator System (CRIS–UNAIDS), DEVInfo (UNICEF), Health Mapper (WHO) and KIDS (WFP). To facilitate this process, international agencies and country experts have, under guidance of the Monitoring and Evaluation Reference Group, reviewed all HIV-related indicators produced by international organizations since 2000, harmonized them, agreed on standard definitions and reduced the number of indicators that international organizations require from middle and lower-income countries [3].

Back to Top | Article Outline
Use of information

A multisectoral response to a country's HIV epidemic requires strategic information from the country's health, social services, educational, labour, economic and other sectors [3]. Any national HIV monitoring and evaluation system will therefore need to capture information from a variety of sources (Fig. 3), including health data, vital statistics, social services or welfare statistics, labour statistics, census data and data from demographic health surveys or other studies.

Equation (Uncited)
Equation (Uncited)
Image Tools
Fig. 3
Fig. 3
Image Tools

Such a system will require a stable, trained staff for data management and analysis. Whereas all facilities will need their own trained staff to fulfil basic functions, others can link up with academic or other institutions, which can provide analytical support on the data stored in facilities, chronic disease registries or data warehouses. A number of high-income countries, including the United Kingdom, the Netherlands and France [42–44], have implemented multicentre prospective monitoring systems that collect HIV health and socioeconomic data and regularly produce reports and analyses based on them.

Another challenge is to ensure that the data collected and generated at the various levels is not only timely but also robust and of optimum quality. One way is to conduct data quality checks at regular intervals. Another method, which can provide an incentive for professionals to ensure the quality of the information collected, is to provide regular feedback of analyses useful to those sites, which produced the data in the first place, so that professionals at those facilities can use the data to improve their own services. One such example is the NPMS-HHC in the United Kingdom. This multicentre prospective monitoring system currently receives information covering approximately 75% of people living with HIV who attend National Health Service HIV clinics. Routine analyses are provided on the use, cost, outcome and impact of HIV services at clinic, regional and national levels. These analyses enable clinics to compare themselves with regional or national averages and also supply information on the cost of their services, which is used in their negotiations with local funders [42,45]. In addition, based on data collected through The National Prospective Monitoring System on Use, Cost and Outcome of HIV Service Provision in UK Hospitals– HIV Health-economics Collaboration (NPMS-HHC), evaluations can be performed. A recent evaluation estimated the cost-effectiveness of different drug regimens for first, second and third-line HAART [46,47].

Back to Top | Article Outline
Confidentiality and security issues

Scaling up HIV services in countries will lead to the collection of more and more personal and sensitive information. Whereas various parts of the data collection, storage, analysis and use process may differ in specific technological requirements, all need to aim to preserve patient and facility confidentiality. Health facilities or community projects using paper-based systems need to address and agree upon security and confidentiality for paper records. Health facilities using electronic medical records will need to ensure the security of data stored therein, as well as for electronic data interchange, especially for individual-level information using HL7 but also for transfers of indicators with XML.

Individual-level data to be stored in data warehouses or HIV disease registries must be pseudo-anonymized to maintain confidentiality, removing personally identifying variables, which could pose a risk to confidentiality. Pseudo-anonymizing individual records involves generating a unique number for the data on each individual, allowing linkages to be made to the clinic of origin if needed. Access to electronic databases must be strictly regulated, requiring the creation of specific steering groups to oversee operations and to agree on the rules of access. Such rules must be enforced by robust procedural and electronic security measures. The analyses performed on aggregate data or the indicators generated can then be made accessible to the public. To assist countries in this work, interim ‘Guidelines on protecting the confidentiality and security for HIV information’ were recently developed, which describe methods by which most securely to collect, store and transfer information [48].

In conclusion, this paper has reinforced the importance of financial information as part of strategic information required to monitor and evaluate a country's response to its HIV epidemic; it also underscores the need for standardized data to permit analyses within and between countries, as well as regional or global analyses. Financial information in countries relates to monies spent on services, which is not necessarily equivalent to the amount of money required to provide such services. Whereas methods are being developed and implemented to track expenditures within countries, more emphasis should be placed on obtaining good information on the cost and efficiency of service provision. Information on costs is not only complementary to cost-effectiveness information; without robust and up-to-date costs, the results of any cost-effectiveness analysis are suspect. Many countries are actively scaling up HIV services within the context of universal access and the wider goals of the MDGs. As new prevention and therapeutic interventions continue to become available, it will therefore be necessary to continue to monitor the changing cost of service provision as well as to evaluate the cost-effectiveness of those services. Furthermore, comparing what is spent on certain services with what is required to provide these services will enable policy makers to make appropriate policy adjustments.

The examples cited in this paper used routinely collected data to monitor costs over time. The example from Singapore showed that aggregate information on the cost of service provision, especially when correlated with other indicators, provided insights into how adding ART into routine service provision changed the costs and nature of service provision. To understand such changes fully, however, a cost-effectiveness evaluation produced a more detailed analysis of the effect of different antiretroviral drug regimens on routine service provision. The multicentre prospective monitoring mechanisms in the United Kingdom, France and the Netherlands, in use since the 1990s, now not only produce trend analyses but also store the data for cost-effectiveness analyses.

Even in many industrialized countries, simply obtaining robust and timely strategic information has its challenges, let alone obtaining such information routinely. When routine information is unavailable or not of robust and contemporary quality, policy makers and others may have to rely on information obtained from other sources, especially randomized controlled trials (RCT) or modelling studies. Ideally for any new intervention, an experimental study is performed before implementation into a programme or service (Fig. 4). RCT measure the efficacy or outcome of an intervention under experimental conditions for those populations, which are part of the trial (internal validity). Combining this information with cost data, estimates the cost-efficacy of an intervention. Translating the RCT findings to real-life situations (external validity) can, however, be more problematical, either because the characteristics of the populations using the intervention in real life are different, or because of the effects of other factors that were not part of the original experimental framework but are nonetheless present in real life [12]. Therefore when robust and contemporary data are unavailable through monitoring or evaluation, experimental studies can provide information for policy makers, from which cost-effectiveness can be inferred. In addition, modelling studies are also widely used to estimate the cost-effectiveness of an intervention, programme or service [31]. Both methods have been extensively used, especially for prevention interventions, and they comprise an important part within the research cycle, although they address different questions to those addressed by operational research [12] (Fig. 4). The limitations of extrapolating information from experimental or modelling studies can, however, be profound, especially in the absence of robust primary data, as has recently been reviewed and discussed [31].

Middle and lower-income countries are in the process of developing HIV disease registries or data warehouses as part of developing their HIV monitoring and evaluation systems. Progress has been made in providing resource-limited countries with guidance and technical assistance to develop the necessary information and analytical systems. Much remains to be done, however, as implementation in many countries remains difficult. For some countries, even the most basic requirements are still challenging, such as data collection, management, storage, and transfer of information for the provision of services for individuals; further still is the ability to use such data for monitoring and evaluation at any level, from facilities to central government. The training and retention of staff with the necessary data management and analytical skills also provides challenges, whereas the confidentiality and security of such sensitive information needs to be assured at all levels of the system.

Fig. 4
Fig. 4
Image Tools

Although many of these activities are primarily related to scaling up HIV services in these countries, ideally they should also leverage improvements in the countries' general health information systems. Disease-specific information systems in countries, as described in this paper, can feed directly into and strengthen projects that aim to develop health sector-wide information systems, such as the Health Metric Network [49]. Conversely, the multisectoral approach, necessitated by the nature of the HIV pandemic, may hopefully encourage the development of similar multisectoral approaches for other diseases currently ravaging middle and lower-income countries. Although the question then arises concerning the relationship between vertical programmes and horizontal services in these countries and how the former can strengthen the latter [12], it is a challenge that we should resolve satisfactorily for the sake of the millions of individuals around the world who need access to basic services, including health and social services.

Back to Top | Article Outline

Acknowledgements

E.J.B. would like to thank Drs Wong SinYew and Nick Paton for inviting him to work and learn from the Singaporean experience. The authors would like to thank Ms Sundhiya Mandalia for assistance in performing some of the analyses presented. They are grateful to all those country and international experts and people living with HIV, who have spent much time and energy in developing guidelines and conceptual frameworks, some of which have been described in this paper. Finally, the authors wish to thank Ms Lucy Honing and the reviewers of the paper for their extremely useful suggestions, which have strengthened the paper.

Conflicts of interest: None.

Back to Top | Article Outline

References

1. UNAIDS/WHO. AIDS epidemic update 2007. Available at http://www.unaids.org/en/HIV_data/2007EpiUpdate/default.asp. Accessed: 26 November 2007.

2. UNAIDS. Keeping the promise: summary of the declaration of commitment on HIV/AIDS. United Nations General Assembly Special Session on HIV/AIDS 25–27 June 2001, New York. Available at: http://data.unaids.org/pub/Report/2002/JC668-KeepingPromise_en.pdf. Accessed: May 2007.

3. United Nations General Assembly Special Session on HIV/AIDS. Monitoring the Declaration of Commitment on HIV/AIDS: guidelines on construction of core indicators– 2008 reporting. March 2007. Available at: http://www.unaids.org/en/AboutUNAIDS/Goals/UNGASS/2008_UNGASS_Reporting.asp. Accessed: March 2008.

4. UNAIDS. Towards universal access: assessment by the Joint United Nations programme on HIV/AIDS on scaling up HIV prevention, treatment, care and support. UN General Assembly document A/60/737. New York: UNAIDS; 2006.

5. UNAIDS. UNGASS reporting 2008. Technical guidance for reporting on the Declaration of Commitment 2001, Geneva: UNAIDS; 2007. Available at: http://data.unaids.org/pub/Manual/2007/20070411_ungass_core_indicators_manual_en.pdf. Accessed: July 2007.

6. United Nations. The UN millennium development goals: keep the promise. Available at: http://www.un.org/millenniumgoals/goals.html. Accessed: 31 January 2008.

7. UNAIDS. The Three Ones. Geneva: UNAIDS; 2007. Available at: http://www.unaids.org/en/Coordination/Initiatives/three_ones.asp. Accessed: May 2007.

8. UNAIDS. Financial resources required to achieve universal access to HIV prevention, treatment, care and support. Geneva: UNAIDS; 2007. Available at: http://data.unaids.org/pub/Report/2007/20070925_advocacy_grne2_en.pdf. Accessed: February 2008.

9. Beck EJ, Walensky RP. The outcome and impact of ten years of HAART. In: Zuniga JM, Whiteside A, Ghaziani A, Bartlett JG, editors. A decade of HAART. Oxford, UK: Oxford University Press; 2008. pp. 45–62.

10. Report of the Secretary General. Declaration of commitment on HIV/AIDS and political declaration on HIV/AIDS: focus on progress over the past 12 months. New York: UN General Assembly; 2007. Available at: http://data.unaids.org/pub/Report/2007/20070418_sg_%20progress_report__en.pdf. Accessed: May 2007.

11. The American Heritage Dictionary of the English Language, 4th ed. Houghton Mifflin Company, Boston, MA, USA, 2007. Available at: http://www.answers.com/topic/strategic. Accessed: July 2007.

12. Beck EJ, Mays N. Some lessons learned. In: Beck EJ, Mays N, Whiteside A, Zuniga J, editors. The HIV pandemic: local and global implications. Oxford: Oxford University Press; 2006. pp. 757–776.

13. Beck EJ, Miners AH. Effectiveness and efficiency in the delivery of HIV services: economic and related considerations. In: Gazzard B, Johnson M, Miles A, editors. The effective management of HIV/AIDS. UK key advances in clinical practice. London: Aesculapius Medical Press; 2001. pp. 113–138.

14. Donabedian A. Explorations in quality assessment and monitoring, Vol. III. The methods and findings of quality assessment and monitoring. Ann Arbor Michigan: Health Administration Press; 1969.

15. Wikipedia. Resource allocation. Available at: http://en.wikipedia.org/wiki/Resource_allocation. Accessed: 25 January 2008.

16. Roberts RA, Hickey A, Rosner Z. The role of community involvement in HIV programmes in South Africa. In: Beck EJ, Mays N, Whiteside A, Zuniga J, editors. The HIV pandemic: local and global implications. Oxford: Oxford University Press; 2006. pp. 720–733.

17. WHO. National health accounts (NHA). Geneva: World Health Organization. Available at: http://www.who.int/nha/en/. Accessed: May 2007.

18. UNAIDS. National AIDS spending assessment (NASA): a notebook on methods, definitions and procedures for the measurement of HIV and AIDS financing flows and expenditures at country level. Version 21. Geneva: UNAIDS; 2006.

19. Finkler SA. The distinction between costs and charges. Ann Intern Med 1982; 96:102–109.

20. Beck EJ, Beecham J, Mandalia S, Griffith R, Walters MDS, Boulton M, Miller DL. What is the cost of getting the price wrong? J Public Health Med 1999; 21:311–317.

21. Beck EJ, Blandford J, Boulle A, Carr L, Cleary S, Eholié S, et al. Essential information for countries to monitor and evaluate the economic aspects of HIV service provision: proceedings from a workshop. Geneva: WHO/TDR Generic Tools Workshop, 16–18 January 2006. Available at: http://www.who.int/hiv/events/GenericToolsWorkshop.pdf. Accessed: May 2007.

22. Tolley K, Gyldmark M. Towards a standardized framework for costing HIV and AIDS treatment and care in Europe. In: Fitzsimons D, Hardy V, Tolley K, editors. The economic and social impact of AIDS in Europe. London: Cassell; 1995. pp. 25–39.

23. Garber A. Advances in cost-effectiveness analysis of health interventions. In: Culyer A, Newhouse J, editors. Handbook of health economics, Vol. 1A, Chapter 4. Amsterdam: Elsevier; 2000. pp. 181–221.

24. Hankins C, Strathdee S, Karim SA. HIV transmission and its prevention in Africa. In: Volberding PA, Sande MA, Lange J, Green WC, Gallant JE, editors. Global HIV/AIDS medicine. Philadelphia, USA: Elsevier; 2008. pp. 565–575.

25. Brass AL, Dykxhoorn DM, Benita Y, Yan N, Engelman A, Xavier RJ, et al. Identification of host proteins required for hiv infection through a functional genomic screen. Science 2008. Available at: http://www.sciencemag.org/cgi/rapidpdf/1152725v1.pdf. Accessed: 31 January 2008.

26. Flexner C. HIV drug development: the next 25 years. Nat Rev 2007; 6:959–966.


28. Beck EJ. The prospects of setting up a Singaporean prospective monitoring system on the use, cost and outcome of HIV service provision: Communicable Disease Centre, Tan Tock Seng Hospital, Singapore, 12–23 October 1997. Health Manpower Development Plan Visit Report. London: Imperial College; 1997.

29. Paton NI, Chere A, Chapman CA, Sangeetha S, Mandalia S, Bellamy R, Beck EJ. Cost-effectiveness of antiretroviral therapy for HIV infection in Singapore, 1996–2001. Int J STD AIDS 2006; 17:699–705.

30. Beck EJ, Miners AH, Tolley K. The cost of HIV treatment and care. A global review. Pharmacoeconomics 2001; 19:13–39.

31. Harling G, Wood R, Beck EJ. Efficiency of interventions in HIV infection, 1994–2004. Dis Manag Health Out 2005; 13:371–394.

32. Rosen L, Long L. How much does it cost to provide antiretroviral therapy for HIV/AIDS in Africa? Health and development discussion paper no. 9. Center for International Health and Development, Boston University, School of Public Health; October 2009.

33. Le Booysen F, Arntz T. The methodology of HIV/AIDS impact studies: a review of current practices. Soc Sci Med 2003; 56:2391–2405.

34. WHO. HIV patient ART monitoring meeting. Geneva: World Health Organization; 2004. Available at: http://www.who.int/3by5/publications/documents/artmonitoring/en/. Accessed: May 2007.

35. WHO. Patient monitoring guidelines for HIV care and antiretroviral therapy (ART). Geneva: World Health Organization; 2006. Available at: http://www.who.int/3by5/capacity/ptmonguidelinesfinalv1.PDF. Accessed: May 2007.

36. WHO. HIV patient ART monitoring meeting, International Conference Centre, Geneva, 29–31 March 2004 and beyond. Geneva: World Health Organization; 2004. Available at: http://www.who.int/3by5/publications/documents/en/hivpatientartmonitoringmeeting2004.pdf. Accessed: May 2007.

37. Beck EJ, Bailey C, Boucher P, Travers P. WHO– Electronic Medical Record Meeting, Stanley Hotel, Nairobi Kenya, 30 and 31 August 2004. Geneva: World Health Organization; 2004. Available at: http://www.who.int/kms/initiatives/EMR_Meeting_Report_2004.pdf. Accessed: May 2007.

38. Tierney WM, Beck EJ, Gardner RM, Musick B, Shields M, Shiyonga NM, Spohr MH. Viewpoint: apragmatic approach to constructing a minimum dataset for care of patients with HIV in developing countries. J Am Med Inform Assoc 2006; 13:253–260.

39. WHO Anti Retroviral Therapy Program. Anti retroviral therapy messaging: transfer of patient health record implementation guide using HL7 version 2.5. Geneva: World Health Organization, Knowledge Management Unit; 2006.

40. UNAIDS. Data exchange with Country Response Information System and UN agency software: a step by step guide. Geneva: UNAIDS; 2006. Available at: http://data.unaids.org/pub/BaseDocument/2007/cris_de_web_final_en.pdf. Accessed: May 2007.

41. UNAIDS Indicator Registry. Geneva: UNAIDS; 2006. Available at: http://www.unaids.org/en/KnowledgeCentre/HIVData/CRIS/CRIS_Data.asp. Accessed: May 2008.

42. Beck EJ, Mandalia S. The cost of HIV treatment and care in England since HAART: Part 2. Br J Sex Med 2003; 27:21–23.

43. De Wolf F, Lange JMA, Bossuyt PMM, Dijkgraaf MGW, Burger DM, Nievwkerk PT, et al, for the Athena Project. Monitoring of human immunodeficiency virus type 1 (HIV-1) infection in the Netherlands. Amsterdam: Stichting HIV Monitoring, Natec, AMC; 2001.

44. Direction des Hospitaux, Mission SIDA. DMI2: an information system built around the patient. INSERM– SC4. Paris: Ministère de la Santé et de l'Action Humanitaire; 1993.

45. Beck EJ, Mandalia S. The cost of HIV treatment and care in England since HAART: Part 1. Br J Sex Med 2003; 27:19–23.

46. Beck EJ, Mandalia S, Brettle R, Fisher M, Gompels M, Kinghorn G, et al., for the NPMS-HHC Steering Group. Cost-effectiveness of NNRTI compared with PI-containing regimens for first-, second- and third-line HAART regimens in UK NPMS-HHC Clinics 1996–2002. In: XVIth International AIDS Conference. Toronto, Canada, 13–18 August 2006 [Abstract TUPE0073].

47. Beck EJ, Mandalia S, Youle M, Brettle R, Fisher M, Gompels M, et al, for the NPMS-HHC Steering Group. Treatment outcome and cost-effectiveness of different HAART regimens in the UK 1996–2002. Int J STD AIDS 2008; 19:297–304.

48. UNAIDS. Guidelines on protecting the confidentiality and security of HIV information: proceedings from a workshop. Geneva: UNAIDS; May 2007. Available at: http://data.unaids.org/pub/Manual/2007/confidentiality_security_interim_guidelines_15may2007_en.pdf. Accessed: May 2007.

49. WHO. Health metrics network biennial report 2005/2006. Geneva: World Health Organization; 2007. Available at: http://www.who.int/healthmetrics/governance/HMN_biennial_report.pdfs. Accessed: July 2007.

Keywords:

cost and cost-effectiveness; HIV information systems; Millennium Development Goals; monitoring and evaluation; strategic information; universal access

© 2008 Lippincott Williams & Wilkins, Inc.

Login

Search for Similar Articles
You may search for similar articles that contain these same keywords or you may modify the keyword list to augment your search.