To the Editor:
We read with interest the letter by Lin1 on our recent systematic review and meta-analysis reporting the incidence and risk factors for prediabetes and diabetes mellitus in HIV-infected adults on antiretroviral therapy (ART).2 Lin questioned the appropriateness of conducting a meta-analysis of studies with substantial heterogeneity.
Heterogeneity, which may be defined as a disparity in true effect sizes underlying different studies, is inevitable in a meta-analysis; it is highly unlikely that several studies conducted in different populations with different methods and by different investigators all end up providing an identical estimate of a parameter of interest. Therefore, formal meta-analysis of studies can be misleading owing to the inherent variability across the included primary studies. However, the fact that, in the words of Egger et al,3 “insufficient attention is often given to heterogeneity does not mean that researchers ould return to writing highly subjective narrative reviews.” Furthermore, from the point of view that heterogeneity is inescapable, heterogeneity is often considered acceptable, provided stringent predefined inclusion criteria are met.4
One of the principles of systematic reviews and meta-analyses is that a protocol should be written in advance, and the decision to pool estimates from several studies should be taken a priori based on subject-matter knowledge rather than on the basis of the actual point estimates. It would be biased to combine only studies that have similar results and therefore lead to less heterogeneity. The criteria to conduct meta-analysis should be planned in advance in a protocol,3 considering the use of random-effects model and meta-regression analysis if the investigators think that there may be a difference between studies.5 Additionally, the use of random-effect meta-analysis models specifically takes into account the between-study heterogeneity and the within-study sampling error. Accordingly, we published a protocol for our review beforehand. Moreover, when a high heterogeneity is found, investigating potential sources of heterogeneity becomes paramount. This is exactly what was done in our meta-analysis.2
First, we applied strict selection criteria as described in the methods section, to ensure that all included studies were similar enough to be pooled together. Then, as long as we reported the pooled incidence of diabetes mellitus and prediabetes in HIV-infected individuals on ART, we investigated, through subgroup and meta-regression analyses, several possible sources of heterogeneity among the populations’ characteristics, the exposure, and the outcomes. These data are presented in details in the appendix of our review. Furthermore, we clearly acknowledged other possible sources of heterogeneity such as ART-related information that we could not assess because they were not reported in primary studies.
In conclusion, the high heterogeneity found in our meta-analysis is acceptable and does not invalidate its findings. As recommended, we explored potential sources of heterogeneity. As in all areas of applied statistics, careful and sound judgment is required to interpret results of a meta-analysis.
Jean Joel Bigna
School of Public Health
Faculty of Medicine
University of Paris Sud XI
Le Kremlin Bicêtre, France
Jobert Richie Nansseu
Department of Public Health
Faculty of Medicine and Biomedical Sciences
University of Yaoundé 1
Arnaud D. Kaze
Department of Medicine
University of Maryland Medical Center Midtown Campus
Jean Jacques Noubiap
Department of Medicine
University of Cape Town and Groote Schuur Hospital
Cape Town, South Africa, firstname.lastname@example.org
1. Lin L. Re: incidence and risk factors for prediabetes and diabetes mellitus among HIV-infected adults on antiretroviral therapy: a systematic review and meta-analysis (Letter). Epidemiology 2018;29:e58.
2. Nansseu JR, Bigna JJ, Kaze AD, Noubiap JJ. Incidence and risk factors for prediabetes and diabetes mellitus among HIV-infected adults on antiretroviral therapy: a systematic review and meta-analysis. Epidemiology. 2018;29:431–441.
3. Egger M, Schneider M, Davey Smith G. Spurious precision? Meta-analysis of observational studies. BMJ. 1998;316:140–144.
4. Higgins JP. Commentary: heterogeneity in meta-analysis should be expected and appropriately quantified. Int J Epidemiol. 2008;37:1158–1160.
5. Ioannidis JP, Patsopoulos NA, Rothstein HR. Reasons or excuses for avoiding meta-analysis in forest plots. BMJ. 2008;336:1413–1415.