Mathematical models are widely used in the evaluation of sexually transmitted infection (STI) control strategies. However, different STI models often yield substantially different conclusions,1–5 and it is necessary to understand what factors account for these discrepancies. Most previous model comparisons have attempted to explain model differences in terms of differences in input parameters and differences in model structure (e.g., the degree of heterogeneity in risk behavior,1,3 the representation of STI natural history and immunity,5,6 and age stratification3). However, few have focused specifically on the role of different modeling frameworks.
When contrasting disease modeling frameworks, the distinction that is most frequently made is the distinction between deterministic (or compartmental) and microsimulation (or agent-based) models.7,8 Deterministic models divide the population into compartments of individuals and calculate expected numbers of events in each compartment on the assumption that individuals in each compartment are homogeneous. Microsimulation models, however, simulate each individual in the population as a separate unit, and randomly assign characteristics and events to individuals by sampling from specified probability distributions. In the context of STI modeling, a further distinction is made between frequency-dependent models, pair models, and network models.9 Frequency-dependent models make the simplifying assumption that the rate of STI acquisition is proportional to STI prevalence in the pool of potential partners. In most frequency-dependent models, this means modeling the risk of transmission in a relationship as a once-off probability that applies at the start of the relationship, so that the duration of the relationship is irrelevant. In network models and pair models, individuals are classified according to the infection status of their current partners, so that the rate of STI acquisition is dependent only on the infection status of the current partners. This distinction is illustrated in Figure 1: in pair models, there is a delay between when an individual acquires an STI and when they can transmit that STI to another partner, whereas in frequency-dependent models, it is implicitly assumed that an individual can transmit their STI to another partner immediately after they become infected. Pair models are traditionally deterministic and assume no concurrency,10 whereas network models are microsimulation models that typically allow for concurrent partnerships. A more detailed summary of the different classes of STI models is included in Supplemental Digital Content 1, http://links.lww.com/OLQ/A126.
Microsimulation and deterministic models of STIs can produce similar results, if it is assumed that there is no partner concurrency11–15 or if the dynamics of partnership formation and dissolution are ignored.16 However, microsimulation network models and deterministic frequency-dependent models can potentially produce different conclusions when partnership formation and dissolution is modeled and allowance is made for partner concurrency.17,18 The frequency-dependent assumption can lead to overestimation of STI prevalence,9,16 and frequency-dependent models also predict more rapid growth in STI prevalence, when new STIs emerge.19 However, most comparisons of frequency-dependent and network/pair models have compared STI prevalence in hypothetical populations, without any model calibration to STI prevalence data or sexual behavior data, and without evaluating intervention impacts. The few comparisons that have involved calibrated models have compared estimates of intervention impact but assumed no concurrency.4,5 It is thus not clear how conclusions about STI epidemiology are likely to be biased by the frequency-dependent assumption in a “real-world” setting, particularly where there are high levels of concurrency. We therefore aim to assess a deterministic, frequency-dependent model approximation to a microsimulation network model of STI transmission in South Africa, both before and after model calibration to STI prevalence data.
MATERIALS AND METHODS
The network and frequency-dependent models both classify the South African population by age and sex, and project the growth in population over time, starting in 1985. The adult population is also divided into “high-risk” and “low-risk” groups, the former being defined as individuals who have a propensity for concurrent sexual partnerships, and the latter being defined as individuals who never have more than 1 partner at a time. Three types of sexual relationship are modeled: long-term marital/spousal relationships, short-term relationships, and interactions between female sex workers and their clients. In both models, individuals are classified according to their number of current partners, which depends on assumed rates of partnership formation and dissolution. All long-term relationships are assumed to begin as short-term relationships, so that individuals can only enter the married state if they currently have at least 1 short-term partner. Rates at which short-term partnerships are formed differ by age, sex, risk group, and marital status. Men in the high-risk group are assumed to have contact with sex workers at rates that depend on their age and current number of partners. The frequency of sex in other relationships depends on the age of the partners and whether the partnership is short term or long term. A more detailed description of the sexual behavior component of the model, as well as the data sources on which it is based, is provided elsewhere20 and in Supplemental Digital Content 2, http://links.lww.com/OLQ/A127.
The models simulate the transmission of 6 STIs: HIV, genital herpes, syphilis, gonorrhea, chlamydia, and trichomoniasis. In each case, parameters determine the probability of transmission per act of sex with an infected partner. The modeling of the natural history of each STI is explained in Supplemental Digital Content 2, http://links.lww.com/OLQ/A127. Assumptions about the average duration in each stage of disease determine the rate at which individuals progress from one disease state to the next in the absence of treatment. The models allow for changes over time in the quality of STI treatment and the level of condom usage, in response to syndromic management and condom distribution programs, respectively.21 To test the hypothesis that it is the sexually transmitted nature of STIs that accounts for model differences, the models also simulate the incidence of 2 non-STIs in women (bacterial vaginosis and vaginal candidiasis).
The network and frequency-dependent models are structurally identical in all but 3 respects. First, for computational convenience, the size of the starting population (in 1985) is set to 20,000 individuals in the network model, whereas the frequency-dependent model works with the actual South African population size. Second, a date of birth is assigned to each individual in the network model, whereas in the frequency-dependent model, individuals are classified only by 5-year age group. Third and most important, the network model randomly links an individual to a specific partner whenever a new partnership is formed (a more detailed explanation of the partner matching algorithm is provided in Supplemental Digital Content 2, http://links.lww.com/OLQ/A127). In the frequency-dependent model, transmission is modeled taking into account the STI prevalence levels in the pool of potential partners, stratified according to the individual's age, sex, risk group, marital status, and partner risk group.22
Two analyses were performed to compare the frequency-dependent model against the “gold standard” network model: the first compares model estimates of STI prevalence levels when input parameters are identical in the 2 models, and the second compares model estimates of natural history parameters and predictions about intervention impacts when both models are calibrated to the same STI prevalence data. In the first analysis, probability distributions were specified to represent uncertainty around key natural history and transmission parameters (Table 1; further details are provided in Supplemental Digital Content 2, http://links.lww.com/OLQ/A127). For each STI, 100 parameter combinations were randomly sampled from these distributions. For each parameter combination, both models predicted the STI prevalence that would be expected in the absence of any changes to baseline behavior and STI treatment. The ratio of the prevalence predicted by the network model to that predicted by the frequency-dependent model was calculated. We hypothesized that the extent to which this ratio differed from 1 would be dependent on the fraction of STI transmission occurring in the early stage of infection, because the frequency-dependent model ignores the delay between when individuals acquire infection and transmit infection to others (Fig. 1), and this delay is relatively more significant when the duration of infectiousness is short. The “early transmission fraction” was therefore defined to be the fraction of transmission that would be expected to occur during the first 6 months after STI acquisition (the assumed average duration of short-term relationships) if all contacts were with susceptible partners; mathematical details are provided in the Supplemental Digital Content 2, http://links.lww.com/OLQ/A127.
In the second analysis, a likelihood function was defined for each STI, to represent the degree of model consistency with South African STI prevalence data. For each STI, a sample of 20,000 parameter combinations were drawn from the same probability distributions as before. Both models were run for each parameter combination, and the likelihood values were calculated. The 100 parameter combinations that yielded the highest likelihood values were compared, and for each of the 100 parameter combinations, the model was used to predict the effect of 6 different hypothetical changes: a 50% reduction in the male rate of sex worker contact, a 50% reduction in the fraction of sex acts that are unprotected in nonspousal relationships, the same reduction in spousal relationships, a 50% reduction in rates of sexual debut, a 100% reduction in the rate at which secondary partners are acquired (no concurrency), and a doubling of the rate of STI health seeking. The 2 models were compared in terms of the predicted cumulative reduction in STI incidence for 10 years (relative to the incidence that would be expected in the absence of any change in behavior or STI treatment).
When input parameters were the same in the 2 models, estimates of STI prevalence were higher in the frequency-dependent model than in the network model (Table 2). Relative differences in prevalence were greatest in the case of syphilis and gonorrhea (network model estimates were 34% and 35%, respectively, of those in the frequency-dependent models), but were modest in the case of long-term STIs (HIV and genital herpes). In the case of bacterial vaginosis and vaginal candidiasis (infections that are assumed not to be sexually transmitted), the 2 models produced very similar estimates of prevalence.
To test the hypothesis that the difference between the 2 models is due to the delay between STI acquisition and transmission in the network model, we calculated the early transmission fraction for each STI (Table 2). The early transmission fraction was highest for gonorrhea and syphilis (86% and 94%, respectively) and lowest for genital herpes and HIV (3% and 17%, respectively). There was a strong negative correlation between the early transmission fraction and the ratio of the network model STI prevalence to the frequency-dependent model STI prevalence, across the 6 STIs (r = −0.93). (In other words, the extent to which the frequency-dependent model overestimated prevalence increased as the early transmission fraction increased.) Similar negative correlations were observed when the curable STIs were considered individually, with parameter combinations being randomly sampled from the prior distributions in Table 1 (Fig. 2): relative differences in STI prevalence between the 2 models were smallest for the parameter combinations that yielded the lowest early transmission fractions (r = −0.50 for syphilis, −0.50 for gonorrhea, −0.51 for trichomoniasis, and −0.54 for chlamydia).
When the 2 models were fitted to the same STI prevalence data, the models yielded similar fits to the data (Supplemental Digital Content 2, http://links.lww.com/OLQ/A127). However, the best-fitting model parameters differed substantially between the 2 models in some cases (Table 1). Transmission probabilities per sex act tended to be lower in the frequency-dependent model than in the network model, especially in the case of female-to-male transmission. Average durations of asymptomatic infection also tended to be shorter in the case of the frequency-dependent model, although not in the case of syphilis (for which latent infection is assumed to be noninfectious). The average duration of immunity was also longer in the frequency-dependent model than in the network model. HIV infectiousness during primary infection was estimated to be substantially lower in the frequency-dependent model than in the network model.
For the best-fitting parameter combinations, the models predicted the effects of various hypothetical changes to sexual behavior and STI treatment (Fig. 3). The 2 models predicted similar effects of reductions in the rate of sexual debut and increases in rates of health seeking. However, the network model consistently estimated a greater impact than the frequency-dependent model when considering reductions in levels of commercial sex and concurrent partnerships. In contrast, the frequency-dependent model estimated a greater impact than the network model when considering increases in condom use in spousal and short-term relationships (not including commercial sex). The extent of the difference between the impacts estimated by the 2 models tended to be greatest for the STIs with high early transmission fractions (gonorrhea and syphilis) and relatively small for genital herpes, which has the lowest early transmission fraction.
This study shows that frequency-dependent models tend to estimate higher levels of STI prevalence than do network models. This difference cannot be attributed to the deterministic/stochastic natures of the respective models, because we found that model results were virtually identical when considering infections that are not sexually transmitted (bacterial vaginosis and vaginal candidiasis). Rather, the differences between the models are due to the way in which sexual transmission is modeled. The simplifying assumption made in frequency-dependent models—that individuals are at risk for transmitting STIs to other partners immediately after becoming infected—is problematic because in reality much of their transmission potential is “wasted” while they remain in contact with the partner from whom they acquired the infection. Even in situations where there are high levels of concurrency, some bias will exist because not all individuals will have multiple partners. These results show that the bias is greatest when a high proportion of the transmission potential is concentrated during the average duration of a short-term relationship. For chronic STIs such as HIV and herpes, where it is common for transmission to occur many years after infection has been acquired, the bias due to the frequency-dependent assumption is limited, and frequency-dependent models may be accurate enough for most purposes.
These findings are consistent with the results of Lloyd-Smith et al.,9 who show that frequency-dependent models tend to estimate higher levels of STI prevalence than do pair models, particularly for short-term STIs. However, our analysis extends this earlier modeling work by considering a more realistic model of sexual behavior, allowing for concurrency, commercial sex, heterogeneity in sexual behavior, and stratification of the simulated population by age and sex. Lloyd-Smith et al. show that the extent of the agreement between frequency-dependent and pair models is strongly dependent on the rate of partner change: in scenarios where there is high partner turnover (such as might be expected in sex work/casual sex environments), differences between frequency-dependent and pair models are likely to be small because the implicit assumption of the frequency-dependent model (that people are at risk of transmitting the STI to susceptible partners immediately after acquiring the infection) is approximately true. This suggests that frequency-dependent models may be appropriate for modeling short-term STIs in settings where STI transmission is largely contained in high-risk groups with rapid partner turnover, but are less appropriate in settings where STIs commonly occur in relationships of longer duration. In settings such as South Africa, where STIs are common even in low-risk groups such as pregnant women,23–25 modeling of short-term STIs using a frequency-dependent model may be problematic. It would be valuable to conduct similar model comparisons in other settings with different sexual behavior patterns, to establish whether the bias is consistent.
These results have important implications for studies that aim to model the likely effect of STI interventions that are targeted to particular behavioral groups. Frequency-dependent models are likely to underestimate the importance of interventions that are targeted at high-risk groups, while overestimating the impact of interventions targeted at low-risk groups. This is because frequency-dependent models overestimate the ease with which STIs can be sustained in low-density networks (where few individuals have concurrent partners and partner change is slow); in high-density networks, where partner turnover is rapid, the difference between frequency-dependent and network modeling approaches is relatively small.9
Although frequency-dependent models are likely to be less biased in the case of HIV than in the case of short-term STIs, problems may arise when considering certain drivers of the HIV epidemic. Mathematical models are often used to estimate the fraction of HIV transmission attributable to commercial sex,20,26,27 but frequency-dependent models may underestimate the contribution of commercial sex (Fig. 3A). Models are also often used to estimate the fraction of HIV transmission occurring in the acute stage of HIV infection.28–30 However, frequency-dependent approaches are likely to exaggerate the actual fraction of transmission occurring during the acute phase of infection, because in reality many acutely infected individuals are in contact only with the partner from whom they acquired HIV. This analysis does not directly estimate the fraction of HIV transmission from acute HIV infection, as the early transmission fraction that we have defined is specific to the hypothetical scenario in which all sexual contacts are with susceptible partners.
These results also have important implications for attempts to estimate STI natural history and transmission parameters using Bayesian and other likelihood-based methodologies.6,31,32 Estimates of the extent of immunity derived from frequency-dependent models6,32 may be exaggerated, whereas estimates of transmission probabilities are likely to be underestimated.
Although network models have the advantage of being more realistic then frequency-dependent models, they have the disadvantage of being more computationally intensive, due to the time that it takes to randomly allocate an individual partner whenever a new partnership is formed. There is also the danger, when applying network models to small populations, that the number of infections may randomly run to zero (extinction), which may imply a bias relative to a larger (more realistic) population, in which the probability of the number of infections randomly dropping to zero is very small.33,34
Although we have emphasized the distinction between network models and frequency-dependent models, it is worth noting that hybrid approaches are possible. A number of attempts have been made to extend deterministic pair models to allow for concurrency using moment closure approximations, which rely on the frequency-dependent assumption in some form.17,18,35 Bauch and Rand17,18 show that such moment closure approximations produce results reasonably consistent with “pure” network models when there is rapid partner turnover and when STI transmission and resolution is “slow,” but not in other scenarios.
The distinction between network models and frequency-dependent models is often conflated with the distinction between microsimulation and deterministic models, because almost all network models are microsimulation models, and most frequency-dependent models are deterministic. However, there are examples of deterministic models that are pair models10,15,36 and microsimulation models that are frequency dependent,37,38 and it is important not to confuse the 2 classifications. A limitation of this model comparison study is that it, to some extent, suffers from the same conflation. Although we believe the differences illustrated in this article to be due to the frequency-dependent assumption and have advanced a number of arguments to support this, we cannot rule out the possibility that the differences may be due to differences between the stochastic and deterministic approaches. The only other model comparison study that has shown substantial differences between deterministic and stochastic model estimates of STI prevalence is a study that similarly conflates the stochastic/deterministic distinction and the network/frequency-dependent distinction, as previously described.17,18
Sexually transmitted infections continue to contribute substantially to the global burden of disease,39 and mathematical models have an important role to play, not only in evaluating the potential future impact of new control strategies but also in understanding why previous control strategies have failed and why previous predictions have erred.40 This study advances our understanding of what constitutes a realistic model by quantifying the extent to which the frequency-dependent assumption may bias STI model results.
1. Althaus CL, Turner KM, Schmid BV, et al. Transmission of Chlamydia trachomatis
through sexual partnerships: A comparison between three individual-based models and empirical data. J R Soc Interface 2012; 9: 136–146.
2. Eaton JW, Johnson LF, Salomon JA, et al. HIV treatment as prevention: Systematic comparison of mathematical models of the potential impact of antiretroviral therapy on HIV incidence in South Africa. PLoS Med 2012; 9: e1001245.
3. Hontelez JA, Lurie MN, Bärnighausen T, et al. Elimination of HIV in South Africa through expanded access to antiretroviral therapy: A model comparison study. PLoS Med 2013; 10: e1001534.
4. Heijne JC, Althaus CL, Herzog SA, et al. The role of reinfection and partner notification in the efficacy of chlamydia screening programs. J Infect Dis 2011; 203: 372–377.
5. Van de Velde N, Brisson M, Boily MC. Understanding differences in predictions of HPV vaccine effectiveness: A comparative model-based analysis. Vaccine 2010; 28: 5473–5484.
6. Johnson LF, Dorrington RE, Bradshaw D. The role of immunity in the epidemiology of gonorrhoea, chlamydial infection and trichomoniasis: Insights from a mathematical model. Epidemiol Infect 2011; 139: 1875–1883.
7. Garnett G. An introduction to mathematical models in sexually transmitted disease epidemiology. Sex Transm Infect 2002; 78: 7–12.
8. Mishra S, Fisman DN, Boily MC. The ABC of terms used in mathematical models of infectious diseases. J Epidemiol Community Health 2011; 65: 87–94.
9. Lloyd-Smith JO, Getz WM, Westerhoff HV. Frequency-dependent incidence in models of sexually transmitted diseases: Portrayal of pair-based transmission and effects of illness on contact behaviour. Proc Biol Sci 2004; 271: 625–634.
10. Dietz K, Hadeler K. Epidemiological models for sexually transmitted diseases. J Math Biol 1988; 26: 1–25.
11. Chick S, Adams A, Koopman J. Analysis and simulation of a stochastic, discrete-individual model of STD transmission with partnership concurrency. Math Biosci 2000; 166: 45–68.
12. Koopman JS, Chick SE, Riolo CS, et al. Modeling contact networks and infection transmission in geographic and social space using GERMS. Sex Transm Dis 2000; 27: 617–626.
13. Kretzschmar M, Morris M. Measures of concurrency in networks and the spread of infectious disease. Math Biosci 1996; 133: 165–195.
14. Mode CJ. A stochastic model for the development of an AIDS epidemic in a heterosexual population. Math Biosci 1991; 107: 491–520.
15. Kretzschmar M. Deterministic and stochastic pair formation models for the spread of sexually transmitted diseases. J Biol Syst 1995; 3: 789–801.
16. Eames KT, Keeling MJ. Modeling dynamic and network heterogeneities in the spread of sexually transmitted diseases. Proc Natl Acad Sci U S A 2002; 99: 13330–13335.
17. Bauch C, Rand DA. A moment closure model for sexually transmitted disease transmission through a concurrent partnership network. Proc Biol Sci 2000; 267: 2019–2027.
18. Bauch CT. A versatile ODE approximation to a network model for the spread of sexually transmitted diseases. J Math Biol 2002; 45: 375–395.
19. Kretzschmar M, Dietz K. The effect of pair formation and variable infectivity on the spread of an infection without recovery. Math Biosci 1998; 148: 83–113.
20. Johnson LF, Dorrington RE, Bradshaw D, et al. Sexual behaviour patterns in South Africa and their association with the spread of HIV: Insights from a mathematical model. Demogr Res 2009; 21: 289–340.
21. Johnson LF, Dorrington RE, Bradshaw D, et al. The effect of syndromic management interventions on the prevalence of sexually transmitted infections in South Africa. Sex Reprod Healthc 2011; 2: 13–20.
22. Johnson LF, Dorrington RE, Bradshaw D, et al. The role of sexually transmitted infections in the evolution of the South African HIV epidemic. Trop Med Int Health 2012; 17: 161–168.
23. Moodley D, Moodley P, Sebitloane M, et al. High prevalence and incidence of asymptomatic sexually transmitted infections during pregnancy and postdelivery in KwaZulu Natal, South Africa. Sex Transm Dis 2015; 42: 43–47.
24. Sturm PD, Connolly C, Khan N, et al. Vaginal tampons as specimen collection device for the molecular diagnosis of non-ulcerative sexually transmitted infections in antenatal clinic attendees. Int J STD AIDS 2004; 15: 94–98.
25. Govender L, Hoosen AA, Moodley J, et al. Bacterial vaginosis and associated infections in pregnancy. Int J Gynecol Obstet 1996; 55: 23–28.
26. Steen R, Hontelez JA, Veraart A, et al. Looking upstream to prevent HIV transmission: Can interventions with sex workers alter the course of HIV epidemics in Africa as they did in Asia? AIDS 2014; 28: 891–899.
27. Case KK, Ghys PD, Gouws E, et al. Understanding the modes of transmission model of new HIV infection and its use in prevention planning. Bull World Health Organ 2012; 90: 831–838.
28. Eaton JW, Hallett TB. Why the proportion of transmission during early-stage HIV infection does not predict the long-term impact of treatment on HIV incidence. Proc Natl Acad Sci U S A 2014; 111: 16202–16207.
29. Powers KA, Ghani AC, Miller WC, et al. The role of acute and early HIV infection in the spread of HIV and implications for transmission prevention strategies in Lilongwe, Malawi: A modelling study. Lancet 2011; 378: 256–268.
30. Rapatski BL, Suppe F, Yorke JA. HIV epidemics driven by late disease stage transmission. J Acquir Immune Defic Syndr 2005; 38: 241–253.
31. Kim JJ, Kuntz KM, Stout NK, et al. Multiparameter calibration of a natural history model of cervical cancer. Am J Epidemiol 2007; 166: 137–150.
32. Bogaards JA, Xiridou M, Coupé VM, et al. Model-based estimation of viral transmissibility and infection-induced resistance from the age-dependent prevalence of infection for 14 high-risk types of human papillomavirus. Am J Epidemiol 2010; 171: 817–825.
33. Sleeman CK, Mode CJ. A computer exploration of some properties of non-linear stochastic partnership models for sexually transmitted diseases with stages. Math Biosci 1999; 156: 123–145.
34. Gallop RJ, Mode CJ, Sleeman CK. Determination of threshold conditions for a non-linear stochastic partnership model for heterosexually transmitted diseases with stages. Math Biosci 2002; 177–8: 287–315.
35. Ferguson NM, Garnett GP. More realistic models of sexually transmitted disease transmission dynamics: Sexual partnership networks, pair models, and moment closure. Sex Transm Dis 2000; 27: 600–609.
36. Chen MI, Ghani AC, Edmunds J. Mind the gap: The role of time between sex with two consecutive partners on the transmission dynamics of gonorrhea. Sex Transm Dis 2008; 35: 435–444.
37. Phillips AN, Pillay D, Garnett G, et al. Effect on transmission of HIV-1 resistance of timing of implementation of viral load monitoring to determine switches from first to second-line antiretroviral regimens in resource-limited settings. AIDS 2011; 25: 843–850.
38. Bendavid E, Brandeau ML, Wood R, et al. Comparative effectiveness of HIV testing and treatment in highly endemic regions. Arch Intern Med 2010; 170: 1347–1354.
40. Eaton JW, Bacaër N, Bershteyn A, et al. Assessment of epidemic projections using recent HIV survey data in South Africa: A validation analysis of ten mathematical models of HIV epidemiology in the antiretroviral therapy era. Lancet Glob Health 2015; 3: e598–e608.