The optimum item was “Comprehensive literature search”(100%). Items of “Provide a priori design” (97.5%), “duplicate study selection and data extraction” (97.5%), “quality of included studies assessment” (97.5%), “formulated conclusions concerning the quality of the included studies” (97.5%), “Provide characteristics of the included studies” (82.5%) and “methods used to combine the findings” (70%) were acceptable. However, severe flaws existed in 4 items: “Publication bias assessment” (37.5%), “status of publication used as an inclusion criterion” (22.5%), “interest conflict” (12.5%), and the worst compliant item “list of studies (included and excluded) provided” (0%).
For 12 items, over 80% articles are in compliance with the criteria, but for the item of “structured summary,” “study selection,” “summary of evidence,” and “conclusions,” all articles have met the criteria. However, there were still 11 items whose compliance rates were below 50% (20/40), which were “Objectives,” “Protocol and registration,” “Search,” “Data items,” “Assessment of inconsistency,” “Risk of bias across studies,” “Results of individual studies,” “Exploration for inconsistency,” “Risk of bias across studies,” “Results of additional analyses” and “Funding.” Then, the quality of the remaining 9 items was moderate between 50% and 80% accordance with PRISMA-NMA checklist.
Throughout the reporting of Methods and Results sections, issues of inadequate or selective reporting also existed. Two NMAs[28,57] (5%) reported to assess risk of bias within individual studies in Methods part (item 12) but not really did in Results (item 19). The same flaw showed in reporting of inconsistency assessment (items S2 and S5) in 4 NMAs[37,46,51,55] (10%) and risk of bias across studies (items 15 and 22) in 1 NMA (2.5%).
TCM is well known as a complementary and alternative therapy for its use of Chinese herbal combinations to treat the functional disorders. However, few studies directly revealed the relationship between multitargets and multi-ingredients in Chinese herbal formula by utilizing the network pharmacology methodologies due to the complexity of Chinese medicine in chemical composition and molecular mechanisms. The network pharmacology may contribute to generate the hypothesis, and further experimental validation was still needed.
NMAs could provide useful evidence on relative effectiveness of different treatments for decision-making when head-to-head comparison trials are insufficient. To the best of our knowledge, this is the first study to comprehensively assess the methodological and reporting quality of NMAs in TCM, although several reviews that focused on the methodological or reporting problems of NMAs in other fields have been conducted.[17–19,68,69]
This study has several limitations. First, there has been no standard tool to assess the methodological robustness of NMAs recently, although AMSTAR was widely used in the quality assessment of systematic review and meta-analysis. Second, though the use of the summary AMSTAR and PRISMA score was validated in the previous studies,[22,24] these checklist was not originally designed as a scored instrument. Third, even though thorough search strategy was employed, we cannot guarantee that all relevant articles were identified.
Authorship: FY and JZ conceived the study, developed the criteria, and wrote the paper. JT and LG searched the literature. HW and XJ exacted the data. JZ and YC assessed the methodological quality. HW and XJ assessed the reporting quality. XL and MSL revised the manuscript. All authors read and approved the final manuscript.
. Li S, Zhang B. Traditional Chinese medicine
network pharmacology: theory, methodology and application. Chin J Nat Med 2013;11:110–20.
. Zhang JH, Zhu Y, Fan XH, et al. Efficacy-oriented compatibility for component-based Chinese medicine
. Acta Pharmacol Sin 2015;36:654–8.
. Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med 1997;126:376–80.
. Estellat C, Ravaud P. Lack of head-to-head trials and fair control arms: randomized controlled trials of biologic treatment for rheumatoid arthritis. Arch Intern Med 2012;172:237–44.
. Volpp KG, Das A. Comparative effectiveness-thinking beyond medication A versus medication B. N Engl J Med 2009;361:331–3.
. Hochman M, McCormick D. Characteristics of published comparative effectiveness studies of medications. JAMA 2010;303:951–8.
. Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med 2004;20:3105–24.
. Ioannidis JP. Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. CMAJ 2009;8:488–93.
. Lee AW. Review of mixed treatment comparisons in published systematic reviews shows marked increase since 2009. J Clin Epidemiol 2014;2:138–43.
. Bafeta A, Trinquart L, Seror R, et al. Reporting of results from network meta-analyses
: methodological systematic review. BMJ 2014;348:g1741.
. Cipriani A, Higgins JP, Geddes JR, et al. Conceptual and technical challenges in network meta-analysis. Ann Intern Med 2013;159:130–7.
. Salanti G, Higgins JPT, Ades AE, et al. Evaluation of networks of randomized trials. Stat Methods Med Res 2008;17:279–301.
. Lee AW. Review of mixed treatment comparisons in published systematic reviews shows marked increase since 2009. J Clin Epidemio 2014;67:138–43.
. Li L, Tian JH, Moher D, et al. Network meta-analyses
could be improved by searching more sources and by involving a librarian. J Clin Epidemiol 2014;67:1001–7.
. Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Synth Methods 2012;3:80–97.
. Bafeta A, Trinquart L, Seror R, et al. Analysis of the systematic reviews process in reports of network meta-analyses
: methodological systematic review. BMJ 2013;347:f3675.
. Petropoulou M, Nikolakopoulou A, Veroniki AA, et al. Bibliographic study showed improving statistical methodology of network meta-analyses
published between 1999 and 2015. J Clin Epidemiol 2016;82:20–8.
. Zarin W, Veroniki AA, Nincic V, et al. Characteristics and knowledge synthesis approach for 456 network meta-analyses
: a scoping review. BMC Med 2017;15:3.
. Nikolakopoulou A, Chaimani A, Veroniki AA, et al. Characteristics of networks of interventions: a description of a database of 186 published networks. PLoS One 2014;22:e86754.
. Yang ZX, Li X, Wang L, et al. Background research for the decision support tool of evidence-based evaluation for essential medicines: 1. Overview of quality
assessment tools for systematic review of randomized controlled trials. Chin J Evid-based Med 2015;15:723–9.
. Shea BJ, Rrimshaw JM, Wells GA, et al. Development of AMSTAR
: a measurement tool to assess the methodological quality
of systematic reviews. BMC Med Res Methodol 2007;7:10.
. Shea BJ, Hamel C, Wells GA, et al. AMSTAR
is a reliable and valid measurement tool to assess the methodological quality
of systematic reviews. J Clin Epidemiol 2009;62:1013–20.
. Shea BJ, Bouter LM, Peterson J, et al. External validation of a measurement tool to assess systematic reviews (AMSTAR
). PLoS One 2007;2:e1350.
. Tian JH, Zhang JH, Ge L, et al. The methodological and reporting quality
of systematic reviews from China and the USA are similar. J Clin Epidemiol 2017;85:50–8.
. Hutton B, Salanti G, Caldwell DM, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses
of health care interventions: checklist and explanations. Ann Intern Med 2015;11:777–84.
. Li JL, Ge L, Ma JC, et al. Quality
of reporting of systematic reviews published in “evidence-based” Chinese journals. Syst Rev 2014;3:58.
. Yang XJ, Wei XC, Jiang L. Network meta-analysis of 5 kinds of TCM injections in the treatment of malignant pleural effusion. China Pharmacy 2017;28:4686–90.
. Ding LL. Chinese patent medicine injection as adjunctive therapy for angina pectoris: a network meta-analysis and overview. Kunming Medical University. Master's Thesis 2017.
. Liu S. Assessing the clinical effectiveness of SMICs for ACI and UAP by network meta-analysis. Beijing University of Chinese Medicine
. Master's Thesis 2017.
. Xiang Y. Comparative effectiveness research of Chinese herb injections for activating blood circulation for stroke. Guangzhou University of Chines Medicine. Master's Thesis 2017.
. Feng JS, Li JY, Yang YE, et al. Network meta-analysis on the comparison of the efficacy of five traditional Chinese medicine
injections combined with sulfasalazine in the treatment of mild to moderate ulcerative colitis in active phase. J Guangdong Med Univ 2017;35:496–9.
. Han Q. Network Meta-analysis of 5 Chinese Herb Injections Combined with ARB or ACEI for Early Diabetic Nephropathy. Dalian Medical University. Master's Thesis 2017.
. Liang FT, Fang JY. Network meta-analysis of the effect of traditional Chinese medicine
for the treatment of pregnancy-induced hypertension syndrome. Shenzhen J Integra Tradit Chin Wes Med 2017;27:190–3.
. Wang HB, Gu YJ, Liu XY, et al. Cost-effectiveness analysis of shugan Jieyu capsule in treating depressive disorder based on network meta-analysis. China J Pharmaceutical Econ 2017;12:19–24.
. Wu ZL, Tan WJ, Pan SS, et al. Network meta-analysis of Chinese medicine
injections combined with L-OHP-based chemotherapy regimen in treating colorectal neoplasms. Chin J Experi Tradit Med Formu 2017;23:203–11.
. Zhang YF, Ke XT, Li BC, et al. Meta-analysis on indirect comparison of Erigeron breviscapus injection and Breviscapus injection in treatment of acute ischemic stroke. China J Mater Medica 2017;42:1194–201.
. Han SY, Hong ZY, Xie YH, et al. Therapeutic effect of Chinese herbal medicines for post stroke recovery: a traditional and network meta-analysis. Medicine (Baltimore) 2017;96:e8830.
. Zhang D, Zheng JW, Ni MW, et al. Comparative efficacy and safety of Chinese herbal injections combined with the FOLFOX regimen for treating gastric cancer in China: a network meta-analysis. Oncotarget 2017;8:68873–89.
. Zhang D, Wu JR, Liu S, et al. Network meta-analysis of Chinese herbal injections combined with the chemotherapy for the treatment of pancreatic cancer. Medicine (Baltimore) 2017;96:e7005.
. Wei XC, Zhu LQ, Wang H, et al. Efficacy of traditional Chinese medicines in preventing oxaliplatin-induced peripheral neurotoxicity in cancer patients: a network meta-analysis. Chin Herbal Med 2017;9:161–8.
. Jin YH, Zhao C, Gan H, et al. Effect of nursing interventions on chemotherapeutic phlebitis: a network meta-analysis. J Nursing Sci 2016;31:85–90.
. Li G, Zhao C, Si JH, et al. Network meta-analysis on clinical effect of 19 Chinese herb injections combined with TACE in treatment of advanced hepatocellular carcinoma. J Lanzhou Univ (Med Sci) 2016;42: 57–63+72.
. Lou LL, Xie W, Zhang P, et al. Effectiveness of traditional Chinese medicine
plus chemotherapy in treatment of esophageal cancer: a network Meta-analysis. J Lanzhou Univ (Med Sci) 2016;42:55–60.
. Shi FY, Lou LL, Xie W, et al. Network meta-analysis of Chinese medicine
injections combined with transcatheter arterial chemoembolization for liver cancer. Chin J Experi Tradit Med Formu 2016;22:180–7.
. Su YY, Liu YY, Zhang YY, et al. Network meta-analysis of 9 kinds of Chinese herb injections combined with transcatheter arterial chemoembolization in treatment of hepatocellular carcinoma. J Jilin Univ (Med Edit) 2016;42:1126–31.
. Tian JH, Ge L, Lou LL, et al. Effectiveness and safety of traditional Chinese medicine
injection combined with CF chemotherapy regimen in the treatment of breast cancer: network meta-analysis and trial sequential analysis. Chin Gen Pract 2016;19:3326–35.
. Wang NN, Zhao RH, Liu Y, et al. Network meta-analysis of Chinese herbal treatment and GnRH-a to prevent thepost-operative recurrence of pelvic endometriosis. Int J Trad Chin Med 2016;38:1120–7.
. Wei XC, Wang H, Zhu LQ, et al. Network meta-analysis of efficacy of 2 kinds of Chinese medicine
formulas in preventing oxali-platin-induced peripheral neurotoxicity in cancer patients. Liaoning J Chin Med 2016;43:1948–50.
. Wei XC, Zhu LQ, Wang CG, et al. Network meta-analysis of efficacy of 3 kinds of Chinese medicine
injections in preventing oxali-platin-induced peripheral neurotoxicity in cancer patients. Chin Hosp Pharm J 2016;36:1567–71.
. Ge L, Wang YF, Tian JH, et al. Network meta-analysis of Chinese herb injections combined with FOLFOX chemotherapy in the treatment of advanced colorectal cancer. J Clin Pharm Ther 2016;41:383–91.
. Chung VC, Wu X, Ma PH, et al. Chinese herbal medicine and salmeterol and fluticasone propionate for chronic obstructive pulmonary disease. Medicine (Baltimore) 2016;95:e3702.
. Wang HL, Jiang Q, Feng XH, et al. Tripterygium wilfordii Hook F versus conventional synthetic disease-modifying anti-rheumatic drugs as monotherapy for rheumatoid arthritis: a systematic review and network meta-analysis. BMC Complement Altern Med 2016;16:215.
. Yang QT, Xiong WJ, Kuang MH, et al. Effectiveness and safety of Chinese medical injections in combination with radiotherapy for esophageal carcinoma: a network meta analysis. Prac Prev Med 2016;23:663–7.
. Xiong WJ. Network Meta-analysis of Chinese patent medicine from National Medicare list for the treatment of Chronic Hepatitis B. Beijing University of Chinese Medicne. Master's Thesis 2016.
. Li JK. To evaluated the curative effect and security of ventricular premature beat treated by wenxin granule in different therapeutic ways of differentiation syndromes and differentiation of diseases: a network meta-analysis. Henan University of Chinese Medicne. Master's Thesis 2016.
. Dong AA, Yu HZ, Qiu DD, et al. Network meta-analysis of Shensongyangxin capsules combined with amiodarone for paroxysmal atrial fibrillation. Chin J Mod Appl Pharm 2016;33:1573–80.
. Xu YC. Network meta-analysis of Chinese herb injections for non-small cell lung cancer. World Latest Med Information 2015;15:152.
. Ge L, Mao L, Tian JH, et al. Network meta-analysis on selecting Chinese medical injections in radiotherapy for esophageal cancer. China J Mater Medica 2015;40:3674–81.
. Liu C, Fan Y, Li MM, et al. A network meta-analysis of Tanreqing injection for radiation pneumonitis. Chin J Drug Evalu 2015;32:228–32.
. Tian JH, Zhao Y, Li JL, et al. Network meta-analysis of 10 Chinese herb injections combined with vinorelbine and cisplatin for non-small cell lung cancer. Chin J Drug Evalu 2015;32:45–9.
. Wu ZS, Lin SL, Luo QH, et al. Pharmacoeconomics analysis of Chinese herb injection combined with NP chemotherapy in treatment of non-small cell lung cancer. Chin J Experi Tradit Med Formu 2015;21:199–202.
. Tian JH, Zhao Y, Li JL, et al. Network meta-analysis of 12 Chinese herb injections combined with gemcitabine and cisplatin for non-small cell lung cancer. Chin J Drug Evalu 2014;32:350–5.
. Zhao Y, Liu DL, Li JL, et al. Chinese herb injections combined with paclitaxel plus cisplatin/carboplatin for non-small cell lung cancer: a network meta-analysis. Chin J Drug Evalu 2014;31:295–9.
. Wang JC. Network meta-analysis of Chinese herb injection combined with chemotherapy for gastric cancer. Lanzhou University. Doctor's Thesis 2014.
. Wang JC, Tian JH, Ge L, et al. Which is the best Chinese herb injection based on the FOLFOX regimen for gastric cancer? A network meta- analysis of randomized controlled trials. Asian Pac J Cancer Prev 2014;15:4795.
. Tian JH. Network meta-analyses
of chinese herb injection combined with chemotheropy for non-small cell lung cancer. Lanzhou University. Doctor's Thesis 2012.
. Liu YF, Ai N, Keys A, et al. Network pharmacology for traditional Chinese medicine
research: methodologies and applications. Chin Herbal Med 2015;7:18–26.
. Chambers JD, Naci H, Wouters OJ, et al. An assessment of the methodological quality
of published network meta-analyses
: a systematic review. PLoS One 2015;10:e0121715.
. Ge L, Tian JH, Li XX, et al. Epidemiology characteristics, methodological assessment and reporting of statistical analysis of network meta-analyses
in the field of cancer. Sci Rep 2016;6:37208.
. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol 2009;62:e1–34.