Secondary Logo

Journal Logo

EPIDEMIOLOGY & SOCIAL

Epidemiological analysis of the quality of HIV sero-surveillance in the world: how well do we track the epidemic?

Walker, Neffa; Garcia-Calleja, Jesus M.a; Heaton, Laurab; Asamoah-Odei, Emila,c; Poumerol, Gillesc; Lazzari, Stefanoc; Ghys, Peter D.a; Schwartländer, Bernharda; Stanecki, Karen A.b

Author Information

Abstract

Introduction

Epidemiological surveillance has been defined as ‘the ongoing and systematic collection, analysis, and interpretation of health data in the process of describing and monitoring a health event’[1]. The HIV/AIDS epidemic represents a health event of unprecedented magnitude.

One purpose of HIV/AIDS surveillance is to determine the extent of the epidemic. A more important purpose is tracking the changes or trends in the epidemic over time. Given the long incubation period between infection and the development of AIDS, a surveillance system that relies solely on AIDS case reporting is not effective. In resource poor countries, health infrastructure is not developed to a level that can provide reporting at a level of completeness that makes AIDS or HIV cases a reliable measure of the epidemic. In addition the definition of AIDS becomes increasingly meaningless with the provision of therapy, making AIDS cases reporting even less useful.

In 1988, the World Health Organization (WHO) proposed the introduction of HIV sentinel surveillance systems [2] to monitor the extent and trends of the HIV epidemic. Most recently UNAIDS and WHO have proposed the use of second generation surveillance tools [3] to improve the understanding of the epidemic. This approach includes collection of behavioural data and provides information on a set of HIV/AIDS indicators that will increase the ability to track the course of the epidemic and assess the impact of interventions.

A recent trend has been to try to rationalize the process of setting the public health agenda by quantifying the cost associated with a disease or condition [4]. The primary method is based on the concept of burden of disease. In this approach, morbidity and mortality are attributed to a disease, disorder or disability. Based on the comparative burden of a disease and the cost and effectiveness of available prevention and treatment interventions, rational goals can be set for public health. Although there have been criticisms of the burden of disease approach [5,6] as well as inherent limitations of composite scores [7], the goal of this approach is certainly worthy. Its value, however, is highly dependent on the availability of data on which to base the calculation of burden. The estimates of burden due to HIV/AIDS are based on the data provided by the sentinel surveillance systems.

This article provides an overview of the quality of HIV sentinel surveillance systems in countries. The data produced by these systems are the basis for estimates of HIV/AIDS prevalence and mortality produced by WHO and UNAIDS [8]. These in turn are used in the calculation of burden of disease attributable to HIV/AIDS. The paper will not review the methods used to produce estimates of HIV/AIDS prevalence, as they have been presented elsewhere [9] nor will it review other aspects of surveillance related to HIV/AIDS.

Methods

Data sources

Data on sero-surveillance of HIV/AIDS in countries came primarily from the HIV/AIDS Surveillance Database developed by the United States Census Bureau [10]. This database is a compilation of HIV/AIDS prevalence data from published journal articles, abstracts of conference presentations, and country reports of HIV surveillance. Data for countries in Europe came from a database maintained by the European Centre for the Epidemiological Monitoring of AIDS. Finally, all country reports on HIV, AIDS and sero-surveillance available to WHO or UNAIDS were reviewed.

Defining appropriate sentinel surveillance

The level of the HIV epidemic was determined before evaluating the quality of the surveillance system, as both the groups tested and the extent of the system should vary by epidemic level. The countries were divided into three states of the epidemic: generalized, concentrated and low. These definitions were based on surveillance data that were available and if not, on the 1999 estimates of adult HIV prevalence [8]. These states are defined below [3].

Low

HIV prevalence has not consistently exceeded 5% in any sub-population whose behaviour places them at highest risk. At this level of the epidemic, HIV surveillance should be carried out in the groups at highest risk in the country.

Concentrated

HIV prevalence consistently over 5% in at least one sub-population at highest risk, and prevalence below 1% in the general adult population (age 15–49 years) in urban areas. At this level of the epidemic, surveillance should continue in the groups at highest risk and surveillance should be started in the general population in urban areas.

Generalized

HIV prevalence reaches 1% in the general adult population (age 15–49 years). When this level of epidemic has been reached in urban areas, coverage of the general population in rural areas should be added to the continuing surveillance in the general population in urban settings and in at least one group at highest risk.

All countries have groups that are at higher risk of infection than the general population. For the purposes of this analysis, groups at high risk were defined as attenders of sexually transmitted infection clinics; injecting drug users (IDUs); sex workers; and men who have sex with men.

Data that are representative of the general adult population in the most sexually active years (age 15–49 years) are also hard to gather. According to WHO guidelines [2,11]; WHO/GPA. Field guidelines for HIV sentinel surveillance, World Health Organization. Geneva, August 1989.(unpublished)] on HIV sentinel surveillance, women attending antenatal care may be used as a proxy for the general adult population. Antenatal clinics often provide the most accessible cross-section of healthy, sexually active women in the general population. Blood is often drawn for syphilis screening, so the additional procedure of unlinked anonymous testing for HIV on blood left over from these samples provides a low-cost and ethically acceptable method for sero-surveillance. Data from pregnant women do not provide representative samples of men and women, but population-based studies which compare prevalence in samples from antenatal clinic attendees and men and women in the general population can be used to calibrate the relation between this sample and the general population [8,9]

Defining the quality of sentinel surveillance systems

Coding schemes to represent four dimensions related to the quality of the system were developed. These dimensions were:

  •  frequency and timeliness of data collection;
  •  appropriateness of populations under surveillance;
  •  consistency of the sites/locations and groups measured over time; and
  •  coverage/representativeness of the groups for the adult populations.

Countries with generalized epidemics

In countries with generalized epidemic, HIV/AIDS has spread into the general population. In this type of the epidemic, coverage of the general adult population becomes more important (in all areas of the country).

Initial coding of data

A spreadsheet was created that contained six variables. The first three related to the frequency of data collection: (1) number of surveillance rounds in the 1990s; (2) number of rounds in the last 5 years; and (3) number of rounds in the last 2 years (i.e. during 1998 and 1999) among attendees of antenatal clinics. The fourth variable was the number of sites testing pregnant women. The fifth variable was the number of years in which at least one site tested a group at highest risk. The final variable was the ratio of the number of sites among pregnant women per million adult population.

The next step in coding was to locate the sentinel sites used in the last 5 years on a map of the country. In addition to the sites, the maps also had overlays of population density and major roads. These maps were used to judge the coverage provided by the surveillance system sites.

Scoring sentinel surveillance system quality

Scores were computed for the four dimensions of quality: frequency and timeliness, appropriateness, consistency, and coverage. The scores for each of these dimensions were then combined to create an overall score of quality of the surveillance system.

Frequency and timeliness

The scoring for frequency and timeliness was a simple arithmetic operation. The number of times surveillance data had been collected since 1994 was counted (range 0 to 5). Countries where data had been collected in the last 2 years were given a one, others a zero. The sum of the two variables was used as the overall measure of frequency and timeliness.

Appropriateness

An appropriate system was defined as one in which data had been collected in the last 5 years in urban and rural sites for pregnant women and from at least one high-risk group during that period. Countries with an appropriate system were scored as one, all other were scored as zero.

Consistency

A judgement about the consistency of data was made by reviewing the sites for pregnant women. Trends in the epidemic cannot be measured if data are not collected repeatedly from some sites. The scoring judgement was made on a three-point scale, with zero representing no pattern of consistency in urban and rural sites for pregnant women. A score of two was given if there was a clear pattern of repeating sites over each data collection cycle. This does not mean that all sites must be repeated, as new sites are added over time, or in some cases groups of rural sites are alternated between data collection cycles. A score of two based on a judgement that there was sufficient consistency among sites to measure trends in the epidemic among pregnant women in both urban and rural areas. A score of one was awarded when there was some repetition, providing trend information for a few sites. Judgements of consistency were made independently by two judges.

Coverage

The judgement of coverage provided by the sentinel surveillance system was based on the likelihood that the sites in the surveillance system provided results for a representative sample of pregnant women. Two pieces of data were used to make this judgement. First, the surveillance site maps were reviewed to see if the sites provided sufficient coverage of the population. Ideally this review would be based on actual health care coverage provided by the sites, however, using the site maps and the population density overlay, judgements could be made about approximate coverage provided by the system. In addition, the ratio of the number of antenatal sites per million population was also reviewed to help make the judgement of coverage. Scores were given on a three-point scale with zero representing very poor coverage, one representing fair coverage, and two representing coverage that included a good sample of sites in both urban and rural areas.

Overall quality of the sentinel surveillance systems

The overall quality of the surveillance system was the combination of the scores for each of the four dimensions (range 0–11). The countries were then ranked based on the quality score. This distribution was reviewed to help determine categories of quality. Three categories were used. To be characterized as a fully implemented, high-quality system, the system had to be judged to provide timely and consistent data that came from the appropriate populations and provided fairly representative data. The countries in this category had total quality scores of eight or above.

Country surveillance systems were characterized as of medium quality if they met most or some of the features of a high-quality system, but not all. For many countries, the weakness in the system was the timeliness of data collection. For others it was poor coverage or consistency. For countries in this category some data were available to track the epidemic, but improvements are needed. Countries were placed in this category when the total scores ranged from five to seven. The poor or non-functioning surveillance system category was applied to all other countries.

Countries with concentrated and low-level epidemics

Scoring surveillance system quality for countries with low-level or concentrated epidemics was carried out differently than for countries with generalized epidemics. In countries with low-level or concentrated epidemics, the risk of spread of HIV/AIDS is concentrated in groups that have risk behaviours. Therefore the information needed for surveillance must come from groups at the highest risk and coverage of the general population is less important.

Initial coding of data

A spreadsheet was created that contained data for all groups tested in the surveillance system. The groups included the four primary groups at highest risk: men who have sex with men, sex workers, IDU, and attendees of sexually transmitted infections (STI) clinics. Data collected from pregnant women were also recorded. In addition to these five categories a sixth category labelled ‘other’ which included data from other groups tested in the surveillance system (e.g., military, police) was coded for countries. For each category, the number of years that data had been collected since 1994 was recorded.

Scoring surveillance system quality

In categorizing surveillance systems for countries with low-level and concentrated epidemics, the four dimensions of quality were scored in slightly different ways. Frequency and timeliness were again captured as the number of times data had been collected since 1994. Appropriateness and coverage were combined and defined by which populations had been under surveillance. In low-level and concentrated epidemics it is not necessary to have full geographic coverage as high-risk groups are often concentrated in urban areas. Therefore both appropriateness and coverage were defined by the groups from which data had been collected. For countries with low-level epidemics, data should be collected from all groups at highest risk for infection. In all countries this includes sex workers, men who have sex with men, and attendees of STI clinics, as it was assumed that these groups are present in all countries. In addition, where significant injecting drug use is present in a country, there should be surveillance among this population. For countries with concentrated epidemics, there should also be surveillance among pregnant women in urban areas as this group will be the most likely avenue of spread from high-risk populations into the general population.

Consistency was judged by reviewing the sites over time. For consistency, one point was given for consistent use of data sources, 2 for some consistency, and 3 for no or very little consistency. For each population a summary measure was created by dividing the number of times data had been collected by the measure of consistency.

Overall quality of the sentinel surveillance systems

An average was used which was based on the summary score for the groups that should be under surveillance. Specifically, pregnant women in urban areas were excluded in the scoring for countries with low-level epidemics and for countries where there is little evidence of injecting drug use, the summary score for surveillance excluded data from this population.

Results

The results for 167 countries are presented in Table 1 by geographic region and level of the epidemic. As can be seen in the table, 55 countries have generalized epidemics. In addition, India has four states that have generalized epidemics. Forty-six countries have concentrated epidemics, and 53 have low-level epidemics.

Table 1
Table 1:
Adult prevalence of HIV/AIDS, state of epidemic and quality of surveillance system rating.

Figure 1 presents a summary map showing the quality of HIV surveillance systems in all countries. Among the 167 countries whose surveillance systems were rated, 47 were judged to have fully implemented sentinel surveillance systems; 51 were judged to have systems that had some or most aspects of a good HIV sentinel surveillance system in place, and 69 were rated as having poorly functioning or non-existent sentinel surveillance systems.

Fig. 1.
Fig. 1.:
Quality of HIV sero-surveillance.

Sub-Saharan Africa

This region is the most affected by the HIV/AIDS pandemic. Among 42 countries scored in this region, all but three countries have generalized epidemics. Surveillance systems in this region have considerable variability in quality. Twelve countries have systems that we categorized as being fully implemented. On the other hand, 16 countries do not have the basic elements required for a HIV/AIDS surveillance system. The remaining 14 countries have some or most of the components of a fully implemented surveillance system in place, but have yet to build a system that is capable of providing the data required to track the epidemic accurately. Fortunately, most of the countries that are most affected by the epidemic also have systems that are fully implemented or have systems with some or most aspects implemented.

Asia (excluding India and China)

The three countries in this region that have generalized epidemics all have well-developed sentinel surveillance systems. These countries, Thailand, Cambodia, and Myanmar, all have systems that have produced systematic data over the past few years. The systems in Cambodia and Myanmar do not provide the same level of coverage among rural areas as does the system in Thailand, but the systems still fall in the highest category.

Among the remaining 17 countries, six have surveillance systems that are well developed. Four countries have some aspects of a well-developed system in place. Bangladesh is one of the countries that falls in this intermediate category. Recently, Bangladesh has upgraded its sentinel surveillance system and its last round of data collection was quite good. However, as the scoring was based on data produced during the last 5 years, its system was scored as having some or most of the aspects of a fully implemented system. Finally, seven countries in the region have poorly functioning sentinel surveillance systems.

India

India, with its large population, is perhaps best characterized as having many different epidemics. Although most states have low-level or concentrated epidemics, at least four states have generalized epidemics. In 1998, India implemented a new and expanded surveillance system increasing the number of sentinel sites nation-wide from fewer than 100 to 166. The new system relies heavily on data from antenatal clinics and clinics that treat sexually transmitted infections. The system also has sites that cover injecting drug users in some states. Some states have well-functioning systems. However, in the states with a generalized epidemic the coverage among the general population in rural areas is poor. In other states where the epidemic is low-level or concentrated, the data from STI clinics provide the majority of data for the groups at highest risk. Overall, India has the basic components of a high-quality surveillance system, but lacks sufficient to provide a complete picture of the course of the epidemic in all states.

China

China, like India, is best thought of as having multiple epidemics. In some provinces and autonomous regions the spread of HIV has been quite rapid in the injecting drug user population. In some regions the epidemic has spread among sex workers. In other regions the epidemic has not yet spread to any large degree. The national surveillance system in China has grown substantially with almost 100 sites, covering all provinces except one. The programme covers six population groups (injecting drug users, STI patients, truck drivers, sex workers, pregnant women, and repeat blood donors).

In general the sentinel surveillance system in China provides good coverage of most groups at highest risk with the exception of men who have sex with men. In addition, although the growth of the number of sites in China has been substantial in the last few years, it has a huge and diverse population that may require more sites to provide sufficient coverage to track the epidemic in each province.

Newly independent states

The 15 countries in this region all have low-level or concentrated epidemics. However, in both Russia and Ukraine the recent growth of the epidemic has been quite rapid. Until recently, the countries in this region relied on mandatory or routine testing among certain groups to track the course of the HIV/AIDS epidemic [12]. However, this policy has been dropped in favour of voluntary testing and the development of routine sero-surveillance systems. Of the countries in this region, eight have some or most of the aspects of a fully implemented sentinel surveillance system, and seven countries have poorly functioning or non-existent sero-surveillance systems.

Eastern/Central Europe

The 13 countries in this region all have low-level or concentrated epidemics. The surveillance systems in these countries are a mixture of reporting and sentinel surveillance. Among these countries, only three have fully implemented sentinel surveillance systems (Poland, Slovakia, and the Czech Republic), and three countries have some or most of the aspects of a well-developed sentinel surveillance system. The remaining seven countries of the region have poorly functioning surveillance systems and must rely primarily on case reporting and blood screening to track the epidemics.

Latin America and the Caribbean

There are 12 countries in this region that have generalized epidemics. Among these countries, only three have sentinel surveillance systems that provide sufficient data to track the epidemic. Two other countries have systems that have some or most of the aspects of a fully developed system. The other countries with generalized epidemics have systems that do not provide the basic information needed.

There are 15 countries in this region with low-level or concentrated epidemics. Of these, three have fully implemented sentinel surveillance systems and three have systems that provide some data for tracking the epidemic. The remaining countries have low-quality sero-surveillance systems.

Highly industrialized countries

The 24 highly industrialized countries all have low-level or concentrated epidemics. These countries have better developed health care systems and have therefore relied heavily on AIDS and different types of HIV case reporting as a primary component of their surveillance systems. Yet even with high rates of case reporting, surveillance data on population prevalence are needed to help track the epidemic and to guide the interpretation of trends in case reporting.

The highly industrialized countries are roughly equally split between those that have fully developed surveillance data to augment their case reporting, and those who do not. Fourteen countries have implemented sentinel surveillance systems or surveys that can be used with case reporting to provide high-quality data about the status and trend of the HIV/AIDS epidemics. The remaining 10 countries rely almost exclusively on case reporting, and a few studies about prevalence. For a few countries it may well be that the quality of case reporting is sufficiently high to eliminate the need for sentinel surveillance. In addition, most of these countries test pregnant women for the purposes of prophylaxis. The use of the resulting data from this testing and from other studies to determine rates of coverage for testing and treatment remains unknown.

North Africa and the Middle East

The countries in this region all have low-level or concentrated epidemics, with the exception of Sudan, where the epidemic is generalized. Although four of the countries have the basic components of a surveillance system in place, no country has a system that produces the quality of data required to track the epidemic. Fortunately, most of these countries have sufficient information in the form of blood screening or occasional studies to suggest that HIV has not spread significantly in the general population. However, few countries have developed a surveillance system that would detect the beginning of the epidemic in groups at highest risk.

Discussion

This analysis of the quality of sentinel surveillance systems for HIV/AIDS suggests that the quality of systems is quite variable. Overall, 98 countries out of the 167 scored have sero-surveillance systems that were either fully implemented or have most of the attributes of a quality sero-surveillance system. Among these countries, 47 were found to have fully implemented sero-surveillance systems. However, 69 countries had poor or non-functioning surveillance systems.

In addressing the question of the quality of data available for producing estimates of prevalence of HIV/AIDS, we must look more closely at which countries had the poor systems. As shown in Table 1, many of the countries with poor or non-functioning systems appear to be the countries with few people living with HIV/AIDS. Using the adult prevalence estimates in the table, one sees that 90% of the people living with HIV/AIDS live in countries with somewhat or fully developed sero-surveillance systems. Although data quality does not guarantee quality estimates of prevalence or burden of disease, it is a prerequisite for good estimates. This analysis supports the argument that in general current global estimates of HIV/AIDS are based on data of fairly high quality. However, because many countries have little data on which to base our estimates it may be that some of these countries have many more people living with HIV/AIDS than are currently estimated.

The analysis of the sentinel surveillance systems in the countries revealed general patterns of weakness in the systems. One weakness was that few countries outside the highly industrialized have sentinel surveillance among men who have sex with men. For many countries with low-level and concentrated epidemics the failure to carry out surveillance among this population is the greatest shortcoming of the system. A second weakness of many systems is the poor coverage of rural populations. For some countries with generalized epidemics, little is known about HIV prevalence away from the cities. Finally, for many countries consistency of data over time is the major weakness.

One limitation of this analysis is that all data available from a country may not have been included. Although several methods were used to ensure the completeness of our data set, inevitably some data will have been missed. In addition, this analysis does not address the manner in which countries use data relating to HIV prevalence that might be available from other sources which are outside their surveillance system. One example is that almost all countries screen blood for HIV. These data can be very useful in tracking the epidemic in countries with low-level epidemics if the data are used in conjunction with knowledge about the characteristics of people who are donating blood.

Another limitation of this analysis is its inability to capture distinctions between countries that fall in the broad categories of quality. In a category there are countries that are rapidly improving their systems by expanding coverage of populations and working to ensure quality of testing. For example, Congo and Liberia have revamped their surveillance programmes and have begun to collect data on sero-prevalence. However, there are also countries, such as Madagascar and Lesotho, that had good systems in the recent past, but due to financial or other restrictions, have not recently collected data. Another issue about the scoring of quality occurs for countries, which, due to limited resources, have chosen to collect data only every 2 years. For example, both Namibia and Zambia have chosen to collect data every 2 years. Given limited resources and high levels of HIV prevalence, this approach may be appropriate.

This analysis has necessarily limited itself in scope. It has not addressed issues related to quality of testing procedures and, in addition it has not reviewed the surveillance systems for most countries with small populations. These countries, although perhaps having small numbers of people living with HIV/AIDS and therefore adding little to the global burden of HIV/AIDS, could have large epidemics that would have profound impact on their population. Finally, this review has not dealt with the ethical issues related to sentinel surveillance systems. WHO and later UNAIDS have recommended unlinked anonymous methodology as the main strategy for sentinel surveillance. This approach to surveillance ensures that respect for privacy, human rights and confidentiality are part of central conditions of any HIV sentinel surveillance system [2,3].

This paper looked at the extent to which countries have established sound sero-surveillance systems for tracking HIV prevalence in key populations. As the epidemic has evolved and experience in attempting to track and explain changes in HIV prevalence has grown, it has become clear that data generated by sero-surveillance are not enough to explain fully the dynamics of the epidemic. Sero-prevalence data need to be analysed by age group and calibrated with occasional information on groups not included in the regular sentinel surveillance system. In addition, behavioural data and data of sexually transmitted infections are needed in order to investigate whether observed changes in prevalence might be attributed to a successful national response, or are simply an effect of the natural course of the epidemic. In concentrated epidemics, these same data can also act as an early warning system, guiding surveillance managers in their choice of populations or sites for future sero-surveillance activities.

This analysis has shown encouraging evidence that many of the countries worst affected by HIV have sound and functioning surveillance systems that can be gradually expanded to meet the needs of second generation surveillance systems. However it also identifies weakness in many countries. Here, more must be done to establish the basics of a functional sero-surveillance system. Finally, the analysis suggests that most countries current sero-surveillance systems do produce sufficient data for estimating current HIV/AIDS prevalence and HIV-related burden of disease.

Acknowledgements

First we wish to express our appreciation to the national HIV/AIDS programmes in countries who have made their data available. We would also like to acknowledge the work of the regional and national programme offices of WHO and country programme advisers of UNAIDS for their work in compiling data on HIV/AIDS surveillance and for the regional offices of WHO for reviewing the surveillance system scoring for countries in their regions. Finally, our appreciation to the people at the European Centre for the Epidemiological Monitoring of AIDS and the United States Census Bureau whose work in creating and maintaining databases on HIV/AIDS surveillance has made this and other work possible.

References

1. World Health Organization. Protocol for the Evaluation of Epidemiological Surveillance Systems. Geneva, Switzerland: WHO; 1997 (WHO/EMC/DIS/97.2).
2. World Health Organization. Sentinel Surveillance for HIV Infection. Geneva, Switzerland: World Health Organization; 1988 (WHO/GPA/DIR/88.8).
3. WHO/UNAIDS Guidelines for Second Generation HIV Surveillance. Geneva, Switzerland: World Health Organization and the Joint United Nations Programme on HIV/AIDS; 2000.
4. World Bank. World Development Report: Investing in Health. Washington DC: World Bank; 1993.
5. Arnaud S, Hansen K. Disability-adjusted life years: a critical review. J Health Econ 1997, 16: 658–702.
6. Williams A. Calculating the global burden of diseases: time for a strategic reappraisal? Health Econ 1998, 8: 1–8.
7. Walker N, Catrambone R. Aggregation and the use of regression in evaluating models of human performance. Hum Factors 1993, 35: 397–411.
8. UNAIDS. Report on the global HIV/AIDS epidemic, June 2000. Geneva, Switzerland: United Nations Joint Programme on HIV/AIDS; 2000.
9. Schwartländer B, Stanecki KA, Brown T. et al. Country-specific estimates and models of HIV and AIDS: methods and limitations. AIDS 1999, 13: 2445–2458.
10. United States Bureau of the Census, United States Bureau of the Census, Population Division, United States Bureau of the Census, Population Division, International Programs Center. HIV/AIDS surveillance data base, June 2000. Washington, DC: United States Bureau of the Census; 2000.
11. Chin J. Public health surveillance of AIDS and HIV infections. Bull WHO 1990, 68: 529–536.
12. Russian Federal AIDS Centre. HIV-infection in Eastern Europe & Central Asia, 1993–1998. Moscow: Russian Federal AIDS Centre; 1999.
Keywords:

HIV; surveillance; data quality; AIDS; estimates of HIV/AIDS

© 2001 Lippincott Williams & Wilkins, Inc.