To the Editor:
We read the recent article by Nansseu et al1 with great interest, and we congratulate the authors for conducting the novel systematic review and meta-analysis to summarize the global evidence on the rates and drivers of progression from diabetes in antiretroviral treatment-exposed HIV-infected people. However, we have some concerns about the appropriateness of combining the results from the 44 extremely heterogeneous studies.
It is widely accepted that heterogeneity is a critical issue in meta-analysis, and too heterogeneous studies may not be suitably combined. Also, meta-analyses of observational studies often produce too precise but equally spurious results, so the statistical combination of data may not be a prominent component of reviews of such studies.2 In the authors’ meta-analyses regarding incidence rates and cumulative incidences of diabetes mellitus and prediabetes, the I2 statistics were at least 96% globally, indicating that heterogeneity contributed to nearly all variations in the combined results.
One important aim of meta-analysis is to deliver more precise evidence than individual studies. However, as a consequence of the overwhelming heterogeneity, the combined results did not provide better estimates than many individual studies. For example, the 95% confidence interval of the global overall incident rate of diabetes mellitus based on a total of 29 studies was wider than those in 14 individual studies (48.3%). Therefore, the combined results may lack interpretability and generalizability for general populations.
Meta-analysis is usually only considered as a method to combine information and produce a single overall effect estimate. However, assessing and modeling the consistency between existing studies and improving the understanding of moderator variables and generalizability are also important ingredients of meta-analysis.3 Instead of relying on combining evidence, the authors may focus on presenting and exploring the differences between studies in the presence of overwhelming heterogeneity. They may specify the diverse trends in modeling, adjustments, and reporting the available risk factors among the collected studies.4 Although the adjusted effect sizes were provided in the supplementary file, their adjustments were likely based on very different sets of predictors in different statistical models. Such information, including the missingness of certain moderator variables and risk factors, should be clearly reported to help audience understand the potential sources of heterogeneity.
In summary, for meta-analyses with overwhelming heterogeneity, researchers may wish to emphasize the investigation of how heterogeneity arose rather than simply combining the results.
Department of Statistics
Florida State University
Tallahassee, FL, [email protected]
1. Nansseu JR, Bigna JJ, Kaze AD, Noubiap JJ. Incidence and risk factors for prediabetes and diabetes mellitus among HIV-infected adults on antiretroviral therapy: a systematic review and meta-analysis. Epidemiology. 2018;29:431441.
2. Egger M, Schneider M, Davey Smith G. Spurious precision? Meta-analysis of observational studies. BMJ. 1998;316:140144.
3. Ioannidis JP, Patsopoulos NA, Rothstein HR. Reasons or excuses for avoiding meta-analysis in forest plots. BMJ. 2008;336:14131415.
4. Serghiou S, Patel CJ, Tan YY, Koay P, Ioannidis JP. Field-wide meta-analyses of observational associations can map selective availability of risk factors and the impact of model specifications. J Clin Epidemiol. 2016;71:5867.