The UNAIDS/WHO Workbook method  is based on the identification of groups at different risk of HIV infection in each region of interest. The method assumes that pregnant women attending antenatal clinics are representative of the low-risk female population, whereas no low-risk men are envisaged. For each group, estimates of lower and upper bounds on size and HIV prevalence are required as input. After multiplying each combination of lower and upper bounds of group size and prevalence, the average of the four resulting figures is taken as the estimate of the number of subgroup-specific infections. The method, which is implemented in Excel, allows estimation of both point prevalence and prevalence trends through mathematical interpolation; however, here only point estimates were obtained. The national estimate is obtained by summing regional estimates . For each risk group, most recently available data on population sizes and HIV prevalence (Tables 2 and 3) were acquired.
Multiparameter evidence synthesis method
The MPES approach was introduced in the United Kingdom in response to the need to adopt an estimation method that would use all available data, notably on diagnosed infections, and provide a realistic representation of the uncertainty inherent in the estimates . In MPES, the population is subdivided into a number of mutually exclusive groups (g) (Tables 2 and 3 and Supplement 1, http://links.lww.com/QAD/A96), residing in different geographical regions (r). Heterosexual STI clinic attendees were further classified by ethnicity as SSA migrants, CRB migrants and nonmigrants. The method combines all available information to estimate three basic parameters for each subgroup and region combination: ρ g,r, the proportion of the population in subgroup g in region r; π g,r, the corresponding HIV prevalence; and δ g,r, the corresponding proportion of infections that are diagnosed.
From these estimates, using census information on the size of the total population, an estimated total number of infections, subdivided into diagnosed and undiagnosed infections, can be derived. Although in some cases, direct information is available on ρ g,r, π g,r and δ g,r, evidence is mostly available indirectly, that is, on functions of these basic parameters. MPES allows effective use of all information, both direct and indirect, in a unified Bayesian model. The available data are, within this model, combined with initial beliefs or expert opinion on the basic parameters, expressed in terms of probability distributions (priors), to produce a ‘posterior’ distribution for the basic parameters and any functions of these. This posterior distribution conveys our final in-and out-of-sample knowledge on the quantities of interest. The uncertainty in the estimation of ρ g,r, π g,r and δ g,r is then propagated through the posterior distributions of the number of people living with diagnosed or undiagnosed HIV infection. This is usually summarized via medians and 95% credible intervals (CrIs), respectively providing a point estimate and a measure of its accuracy (A 95% credibility interval indicates the range within which the parameter of interest falls with probability 95%. It differs from a 95% confidence interval in that the latter is expected to include the parameter in 95% of hypothetical replications of the experiment, which led to the actual sample. Credibility and confidence intervals are expression of different statistical paradigms, that is, the Bayesian and frequentist approaches.). The use of multiple data sources can lead to conflicting evidence. Discrepant items of data were reconciled in MPES through feedback from epidemiologists and data suppliers. Additionally, a number of assumptions were introduced as detailed in Supplement 3, http://links.lww.com/QAD/A96.
Estimation and Projection Package and Spectrum methods
EPP was developed by UNAIDS/WHO, initially for generalized epidemics, to provide short-term projections of HIV prevalence and incidence . EPP fits a transmission model to surveillance time-series of prevalence data, to produce a national estimate of the HIV prevalence. This transmission model is described by a set of three differential equations governing the dynamics of the sizes of three mutually exclusive population compartments: a group not at risk of HIV infection; a group at risk of infection; and a group infected with HIV . These differential equations are parameterized by the rate r of growth of the epidemic; the fraction f 0 of the population at risk of infection at the start of the epidemic; the start year t 0 of the epidemic; and a behavioral response parameter Φ, describing how the epidemic levels off after reaching its peak . In concentrated epidemics, a fifth parameter (d) is additionally included in the model to account for the turnover of people from the higher risk population subgroups (for example, sex workers ‘returning’ to the low-risk group after a median 3 years of stay in the ‘FSWs’ risk group). Estimates of the above system parameters are obtained via a Bayesian approach, and the uncertainty surrounding them is then propagated through the model via Monte Carlo simulation, thereby yielding a sample of HIV prevalence curves. The median and 2.5–97.5% percentiles of the resulting set of curves at any given year are conventionally reported as final summaries. The latest (2009) EPP release allows incorporation of data on (combination) ART. Retrospective prevalence data from the beginning of the epidemic (1985 in the Netherlands) are required. Ideally, for each subgroup of interest, the model requires at least three corresponding prevalence data points over time (Supplement 2, http://links.lww.com/QAD/A96).
The output produced by EPP can then be imported into the complementary Spectrum package to produce a richer array of demographic and epidemiological HIV/AIDS descriptors (such as number of new HIV infections or number needing ART)  (Brown T, Stover J; 2009, personal communication). The EPP and Spectrum packages are designed to complement each other (Brown T, Stover J; 2009, personal communication). For the purpose of producing a national HIV estimate, two Spectrum modules are used. The first is demography, a program computing population projections based on the current population size and fertility, mortality and migration rates. The second module used is the AIDS Impact Model, a program predicting the consequences of the AIDS epidemic, including the number of PLWHA, given an assumed adult HIV prevalence . The demographic projection is modified by the AIDS Impact Model through information on AIDS-related deaths and the impact of HIV infections on fertility.
Using the Workbook method, it was estimated that 23 969 PLWHA aged 15–70 years were living in the Netherlands as of 1 January 2008 (Table 4). Compared to the observed number of 12 649 HIV cases that were registered in care and alive on 1 January 2008 , this implies that 53% were in care. Compared to the 2005 estimate of 18 500, the estimated number of PLWHA has increased by almost 30%. The method itself does not estimate proportions of infections undiagnosed nor provides a formal measure of the uncertainty surrounding the estimates.
Multiparameter Evidence Synthesis approach
MPES estimated that 21 444 (95% CrI 17 204–28 694) people aged between 15 and 70 years were living with HIV/AIDS in the Netherlands as of January 2008 (Table 4). The adult HIV prevalence was estimated at 0.2% (95% CrI 0.15–0.24%). Figure 1 illustrates subgroup-specific and region-specific estimates of the proportions diagnosed with HIV. Overall, it was estimated that 40% (95% CrI 25–55%) of infected individuals lived with an undiagnosed infection. Notably, large differences in the proportion undiagnosed were found between the regions, especially among migrants not attending a STI clinic.
Estimation and Projection Package/Spectrum method
EPP/Spectrum produced an estimated HIV prevalence in the Netherlands for 2008 of 0.2%, with the number of PLWHA aged 15 years or more estimated at 19 115 (95% CrI 15 902–22 577) in 2008 (Table 4). For 2013, it is projected that 21 287 (95% CrI 15 884–26 093) people aged 15 years or more will live with HIV in the Netherlands (Fig. 2). Spectrum provided a 95% credibility interval around the aggregate (but not subgroup-specific) PLWHA.
On the basis of results from MPES, we estimate that there were 21 444 (95% CrI 17 204–28 694) PLWHA in the Netherlands as of January 2008, aged between 15 and 70 years. Overall, an estimated 40% of the HIV-infected population remained undiagnosed, with a particularly high proportion outside Amsterdam and Rotterdam. The HIV prevalence is estimated at 0.2% based on EPP/Spectrum implying a projected number of 19 115 PLWHA in the Netherlands. Workbook estimated a national figure of 23 969 PLWHA.
An important implication of this work is the confirmation that even in the Netherlands – a western European country with reasonably good access to testing and counseling – a substantial proportion of HIV infections were estimated as undiagnosed. Although high compared to the UK and the US where 28 and 21%, respectively, were unaware of their infection [31,32], this estimate is plausible as the Netherlands introduced an active testing policy rather late. Especially among migrant populations, it is known that access to testing is limited . This is reflected in the proportion of HIV-infected patients who present late (CD4 cell count <200) in care. Among heterosexuals (usually originating from HIV endemic countries), this was almost 40% in 2009 . It stresses the need for further efforts in developing strategies to achieve greater uptake of testing, counseling and referral into care.
Accuracy and biases
The contrasted methods produced three different outcomes, though within a relatively similar range. Moreover, the results from both UNAIDS tools fit well within the credibility interval of the MPES method (17 204–28 694). The MPES method produced a cross-sectional estimate of 21 444 PLWHA that is consistent with all information, as it relies on all data available, including HIV diagnosis data for registered HIV cases in care. EPP/Spectrum produced an estimate of 19 115 PLWHA relying on a projection of HIV incidence curves from retrospective prevalence data spanning several years. Demographic assumptions in Spectrum contributed to reducing the EPP estimate. In comparison with MPES and EPP/Spectrum, the somewhat higher Workbook outcome of 23 696, despite the assumption of no low-risk men being infected with HIV, relied on fewer data. Referring to the Workbook outcome, the number of PLWHA has increased by 30% compared to the previous estimate in 2005, which is much steeper (unlikely realistic) increase than the 18% increase in registered cases in care . Furthermore, Workbook only provides a single point estimate, with a minimum-maximum range, but with no formal uncertainty measure attached. Both MPES and EPP/Spectrum are likely to provide more accurate prevalence estimates than Workbook as they employ a richer collection of data and rely on a more refined set of assumptions. However, only MPES provides estimates that are based on, and hence consistent with, all available information, fully reflecting the uncertainty in the data and estimation process. By allowing detection and modeling of biased or conflicting evidence, MPES encourages a critical re-appraisal of the whole body of evidence. For these reasons, although a formal comparison between methods relying on different data, assumptions and structure remains difficult, we suggest that the most reliable estimate of PLWHA is provided by MPES. In addition, MPES provides subgroup-specific and region-specific estimates, allowing to identify risk groups and regions with the highest proportions undiagnosed and, therefore, most in need of intervention measures. This information is especially useful in guiding public health strategies.
Data on population size and HIV prevalence for each subpopulation and region are needed to both Workbook and MPES for the target year (Table 1). On the other hand, EPP/Spectrum requires retrospective prevalence data from the beginning of the epidemic and data on ART use. MPES uses data on proportions diagnosed (derived from anonymous unlinked HIV prevalence surveys) and numbers of HIV cases registered in care. The latter is an advantage of MPES, because data on diagnosed HIV cases are now routinely collected in most high-income and many low-income and middle-income countries , whereas the other methods are not designed to use this information.
Applicability and expertise needed
Of the three tools, Workbook is the easiest to use: it is very user-friendly and requires no statistical expertise to be understood and used. Once data on input parameters are collected, Workbook takes only a short time to perform. Although less straightforward than Workbook, EPP/Spectrum is also user-friendly and the accompanying user guide illustrates the estimation process comprehensively. No statistical or mathematical knowledge is required to implement EPP/Spectrum. In contrast to Workbook and EPP/Spectrum, MPES is less user-friendly and more difficult to apply. It is implemented in the freely available WinBUGS software  (code available from the authors on request); however, the code is bespoke and needs to be adapted to the case-study at hand, depending on the available data. Hence, both statistical and epidemiological knowledge are needed to implement a MPES approach. Biases and contradictions in the data must be discussed and well understood prior to being modeled. The method is, therefore, more challenging than the others. To increase the applicability of MPES, a more user-friendly and specifically designed programming environment would be useful.
An important limitation of Workbook is that it only allows low-risk women in the exposure group. Furthermore, Workbook was initially developed to estimate HIV prevalence among the adult population aged 15–49 years. In many high-income countries, persons aged 50 and over account for a substantial proportion of PLWHA [5,25]. We recommend that when using the Workbook method, this should also include the population older than 50 years. For EPP/Spectrum, data on future ART uptake are required; this can be done only by extrapolating our current knowledge and expectations for the coming years. Furthermore, as Spectrum assumes that women live longer than men after contracting HIV, this will result in a somewhat heavier mortality in the case of mainly male-driven epidemics (T. Brown, J. Stover, 2009, personal communication). Thus, in epidemics that are mainly MSM-driven (such as in many western European countries and, to some extent, in the United States) or IDU-driven (as in eastern Europe), these assumptions may have an excessive impact on the projected number of PLWHA. Different methods are being used to estimate the size of risk groups, but there is currently no consensus on the best and most appropriate methods to use . In addition, the data used in the models were not all from the same year: for instance, FSW estimates relied upon rather outdated statistics on population size.
The choice of a method necessarily depends on the objectives, data availability and time constraints. For a realistic reflection of the uncertainty inherent in both the estimation process and the data, MPES could be most appropriate. Moreover, only MPES yields subgroup-specific and region-specific estimates of the proportions undiagnosed. On the other hand, if the aim is not just to obtain annual estimates, but also short-term projections, then EPP/Spectrum could be more suitable. Overall, we recommend MPES as the most statistically sound method, but EPP/Spectrum is the most practical package in terms of data and expertise needed to provide estimates. Future work should focus on developing methods that include data on HIV diagnoses and on methods for risk group size estimation.
M.v.V. was involved in the design of the study, interpreted data and parameters and carried out Workbook and EPP method. M.v.V. was also involved in interpreting MPES and wrote the article. A.P. designed various aspects of the MPES model for Dutch data and was involved in critically reviewing and revising the article. S.C. programmed and implemented the MPES model, was involved in the statistical analysis and critically reviewed and edited the article, especially the Materials and Methods. M.X. was involved in the design of the study and critically reviewed and revised the article, especially the Discussion section of the text. A.R.S. was involved in the design of the study and reviewed the article. M.D. was involved in the design of the study and reviewed and edited the article. A.v.S. supplied data of HIV diagnosed cases and reviewed and edited the article. M.v.d.S. was involved in interpretation of the data and methods and reviewed and revised the article. D.D.A. was involved in the design of the study, interpreted data and results from the MPES model and reviewed and revised the article.
The authors would like to thank the WHO Regional Office for Europe for partially funding this project through the partnership program between WHO and the Netherlands Ministry of Health, Welfare and Sport. They would like to thank, in particular, Stine Nielsen (currently at the Robert Koch Institute, Berlin) for her assistance and suggestions. We are very grateful to Eline Op de Coul from the National Institute for Public Health and the Environment (RIVM) for her contribution to the estimation process. Roel Coutinho (RIVM) is acknowledged for his comments and criticism on the article. Mary Mahy (UNAIDS), John Stover (Futures Institute), and Tim Brown (East-West Centre) are acknowledged for their explanation of the UNAIDS/WHO packages EPP/Spectrum. The authors are grateful to Gijs Baaten, Marcel Buster, Anneke van den Hoek, Rik Koekenbier, Anneke Krol, Martijn van Rooijen, Ineke Stolte, and Daan Uitenbroek from the Amsterdam Health Service for providing data from the Amsterdam Health Monitor, the Amsterdam pregnancy screening, drug monitor, anonymous unlinked HIV survey among STI clinic attendees, laboratory results of mantotman.nl, and data from the Amsterdam Cohort Studies among MSM and drug users. We thank Hannelore Götz and Bianca Stam (Municipal Health Service Rotterdam-Rijnmond) for providing data from Rotterdam pregnant screening and the Rotterdam Health Monitor and Nicole Dukers (South Limburg Public Health Service) for providing data from the Amsterdam Health Monitor. Femke Koedijk and Liesbeth Mollema from the RIVM are thanked for providing data from the STI centers and PIENTER study. Floor Bakker of Rutgers Nisso is acknowledged for providing data on the sexual health survey. Tobias Dorfler (Schorer) and Harm Hospers (University Maastricht) are acknowledged for providing data from the Schorer monitor.
1. Walker N, Bryce J, Black RE. Interpreting health statistics for policymaking: the story behind the headlines. Lancet 2007; 369:956–963.
2. Kitahata MM, Gange SJ, Abraham AG, Merriman B, Saag MS, Justice AC, et al
. Effect of early versus deferred antiretroviral therapy for HIV on survival. N Engl J Med 2009; 360:1815–1826.
3. Sterne JA, May M, Costagliola D, de Wolf F, Phillips AN, Harris R, et al
. Timing of initiation of antiretroviral therapy in AIDS-free HIV-1-infected patients: a collaborative analysis of 18 HIV cohort studies. Lancet 2009; 373:1352–1363.
4. Marks G, Crepaz N, Janssen RS. Estimating sexual transmission of HIV from persons aware and unaware that they are infected with the virus in the USA. AIDS 2006; 20:1447–1450.
5. European Centre for Disease Prevention and Control. HIV prevention in Europe: action, needs and challenges; 2–3 October 2006. Meeting report. Stockholm: ECDC; 2006.
6. Hamers FF, Phillips AN. Diagnosed and undiagnosed HIV-infected populations in Europe. HIV Med 2008; 9(Suppl 2):6–12.
7. Brookmeyer R, Damiano A. Statistical methods for short-term projections of AIDS incidence. Stat Med 1989; 8:23–34.
8. Gail MH, Brookmeyer R. Methods for projecting course of acquired immunodeficiency syndrome epidemic. J Natl Cancer Inst 1988; 80:900–911.
9. Giesecke J, Johnson A, Hawkins A, Noone A, Nicoll A, Wadsworth J, et al
. An estimate of the prevalence of human immunodeficiency virus infection in England and Wales by using a direct method. J R Statist Soc 1994; 157:89–103.
10. Lyerla R, Gouws E, Garcia-Calleja JM, Zaniewski E. The 2005 Workbook: an improved tool for estimating HIV prevalence in countries with low level and concentrated epidemics. Sex Transm Infect 2006; 82(Suppl 3):iii41–iii44.
11. Boulos D, Yan P, Schanzer D, Remis RS, Archibald CP. Estimates of HIV prevalence and incidence in Canada, 2005. Can Commun Dis Rep 2006; 32:165–174.
12. Goubar A, Ades A, De Angelis D, McGarrigle C, Mercer C, Tookey P, et al
. Estimates of HIV prevalence and proportion diagnosed based on Bayesian multiparameter synthesis of surveillance data. JRSS(A) 2008; 171:541–580.
13. Phillips AN, Sabin C, Pillay D, Lundgren JD. HIV in the UK 1980–2006: reconstruction using a model of HIV infection and the effect of antiretroviral therapy. HIV Med 2007; 8:536–546.
14. Chin J. The AIDS pandemic: the collision of epidemiology and political correctness. Oxford: Radcliffe Publishing; 2007.
15. Ghys PD, Walker N, McFarland W, Miller R, Garnett GP. Improved data, methods and tools for the 2007 HIV and AIDS estimates and projections. Sex Transm Infect 2008; 84(Suppl 1):i1–i4.
16. Brown T, Salomon JA, Alkema L, Raftery AE, Gouws E. Progress and challenges in modelling country-level HIV/AIDS epidemics: the UNAIDS Estimation and Projection Package 2007. Sex Transm Infect 2008; 84(Suppl 1):i5–i10.
17. Stover J, Johnson P, Zaba B, Zwahlen M, Dabis F, Ekpini RE. The Spectrum projection package: improvements in estimating mortality, ART needs, PMTCT impact and uncertainty bounds. Sex Transm Infect 2008; 84(Suppl 1):i24–i30.
18. Op de Coul ELM, Van Sighem AI, Laar MJWvd. Estimation of the number of adults living with HIV/AIDS in the Netherlands in 2005. Infectieziekten Bulletin 2006; 17:398–403.
19. Walker N, Stover J, Stanecki K, Zaniewski AE, Grassly NC, Garcia-Calleja JM, et al
. The workbook approach to making estimates and projecting future scenarios of HIV/AIDS in countries with low level and concentrated epidemics. Sex Transm Infect 2004; 80(Suppl 1):i10–13.
20. Goubar A, Ades A, De Angelis D, McGarrigle C, Mercer C, Tookey P, et al. Bayesian multiparameter synthesis of HIV surveillance data in England and Wales, 2001
. Health Protection Agency Centre for Infections Technical Report.
London: Health Protection Agency (HPA); 2006.
21. Ghys PD, Brown T, Grassly NC, Garnett G, Stanecki KA, Stover J, et al
. The UNAIDS Estimation and Projection Package: a software package to estimate and project national HIV epidemics. Sex Transm Infect 2004; 80(Suppl 1):i5–9.
22. Brown T, Grassly NC, Garnett G, Stanecki K. Improving projections at the country level: the UNAIDS Estimation and Projection Package. Sex Transm Infect 2006; 82(Suppl 3):iii34–iii40.
23. Presanis AM, De Angelis D, Spiegelhalter D, Seaman S, Goubar A, Ades A. Conflicting evidence in a Bayesian synthesis of surveillance data to estimate HIV prevalence. JRSS (A) 2008; 171:915–937.
24. Koedijk F, Vriend HJ, van Veen MG, Op de Coul ELM, van den Broek IVF, van Sighem AI, et al
. Sexually transmitted infections, including HIV, in the Netherlands in 2008
. Bilthoven: Centre for Infectious Disease Control – National Institute for Public Health and the Environment (RIVM); 2009.
25. Gras L, Sighem Av, Smit C, Zaheri S, Schuitemaker H, Wolf Fd. Monitoring of human immunodificiency virus (HIV) infection in the Netherlands
. Report 2008
. Amsterdam: HIV Monitoring Foundation (SHM); 2008.
26. van den Broek IVF, Koedijk FDH, van Veen MG, Op de Coul ELM, van Sighem AI, van der Sande MAB. Sexually transmitted infections in the Netherlands in 2007
. Bilthoven: Centre for Infectious Disease Control – National Institute for Public Health and the Environment (RIVM); 2008.
27. van Veen MG, Gotz HM, van Leeuwen PA, Prins M, van de Laar MJ. HIV and sexual risk behavior among commercial sex workers in the Netherlands. Arch Sex Behav 2010; 39:714–723.
28. Alkema L, Raftery AE, Brown T. Bayesian melding for estimating uncertainty in national HIV prevalence estimates. Sex Transm Infect 2008; 84(Suppl 1):i11–i16.
29. UNAIDS/WHO. Quick Start Guides for Estimation and Projection Package & Spectrum 2009
30. Stover J. AIM: a computer program for making HIV/AIDS projections and examining the demographic and social impacts of AIDS
. USAID Health Policy Initiative
31. HPA. Testing times
. HIV and other sexually transmitted infections in the United Kingdom; 2007
. London: Health Protection Agency; 2007.
32. Centers for Disease Control and Prevention (CDC). HIV prevalence estimates: United States, 2006
. MMWR Weekly
33. Fakoya I, Reynolds R, Caswell G, Shiripinda I. Barriers to HIV testing for migrant black Africans in western Europe. HIV Med 2008; 9(Suppl 2):23–25.
34. Gras L, Sighem Av, Smit C, Zaheri S, Schuitemaker H, Wolf Fd. Monitoring of human immunodificiency virus (HIV) infection in the Netherlands
. Report 2009
. Amsterdam: HIV Monitoring Foundation (SHM); 2009.
35. European Centre for Disease Prevention and Control/WHO regional Office for Europe. HIV/AIDS surveillance in Europe in 2008
. Stockholm: European Centre for Disease Prevention and Control; 2009.
36. Lunn D, Thomas A, Best N, Spiegelhalter D. WinBUGS: a Bayesian modelling framework – concepts, structure, and extensibility. Stat Comp 2000; 10:325–337.
Estimation and Projection Package; estimation tools; HIV; Multiparameter Evidence Synthesis; prevalence; Spectrum; Workbook
Supplemental Digital Content
© 2011 Lippincott Williams & Wilkins, Inc.