Overview of statistical methods usage in Indian anaesthesia publications : Indian Journal of Anaesthesia

Secondary Logo

Journal Logo

Original Article

Overview of statistical methods usage in Indian anaesthesia publications

Tyagi, Asha; Garg, Devansh; Mohan, Aparna; Salhotra, Rashmi; Vashisth, Ishita2; Agrawal, Ananya1; Deshpande, Sanika1; Deep, Sonali; Das, Sacchidananda; Malhotra, Rajeev K3; Pradhan, Rajeev4; Panda, Aparajita5

Author Information
doi: 10.4103/ija.ija_667_22
  • Open

Abstract

INTRODUCTION

Medical research is the core of clinical practice and its advancements. The importance and relevance of medical research has led to the genesis and acceptance of ‘evidence-based’ clinical practice. Statistical methods form the mainstay of organising collected data and using it to draw inferences during research.[12] Even if not actively conducting medical research, an understanding of statistical methods is required for critical evaluation, understanding and clinical implementation of published research. The importance of statistical methods and the errors associated with them has been emphasised.[3]

Statistics can be simply understood as a field of mathematics that pertains to analysis of data.[4] It is required to not only interpret results, but also explain variations and predict future data.[4] While the importance of statistical methods is often appreciated, it is also acknowledged that understanding them is difficult, and their incorrect application is common.[25] All of this signifies the need for greater focus and understanding on the usage of statistics in research.

The variety of statistical methods now includes much more complex and advanced procedures. The importance of choosing the correct statistical methods during dissertation writing has also been emphasised.[6] International anaesthesia journals such as the ‘Anesthesia & Analgesia’ (A&A) have recognised the increasing importance of statistical tools and regularly publish educational articles on related subjects, including the advanced and complex methods. However, despite accepting the importance of knowledge regarding statistical methods, we were unable to locate any attempts to examine the pattern of their usage in Indian journals.

The present literature survey hence aimed to examine the use of various statistical methods in a two-year sample (2019-2020) of representative Indian anaesthesia journals and also compare it with an international top-ranked journal in the subject. The primary objective was to note the type of various statistical methods used amongst the original research articles. The secondary objectives were to note the type of study design, software used for statistical analysis (when mentioned) and overall percentage of original research articles.

METHODS

The study was undertaken from 4 August 2021 to 4 June 2022, for data available in the public domain. Hence, no ethical concerns were identified by the Institutional Ethical Committee (Ref no. IECHR-2021-50-13).

‘Indian Journal of Anaesthesia’ (IJA) and ‘Journal of Anaesthesiology Clinical Pharmacology’ (JOACP) were chosen as representative of Indian anaesthesia publications. The international journal included for comparison was A&A, the official publication of International Anesthesia Research Society from the USA.

Statistical method usage was audited only for ‘original research’ articles published in these journals. Original research articles were identified from the table of contents of each issue (clinical or basic). Each included article was assessed to identify the statistical method from the sections of statistical analysis, tables, figures as well as the results section. All the tests used (even if not mentioned in the section of statistical methods) were noted. For identifying the studies, abstracting information and entering it into a Google sheet, four teams with two independent researchers were designated. To ensure unbiased assessment, each team was allotted three issues of IJA (2019) and A&A (2019) since both are monthly publications and one issue of JOACP (2019), since it is published quarterly. Each included article was scrutinised independently by both the allocated team members. Any discrepancy was resolved by the referee team of other three clinical researchers and a biostatistician. This referee team also conducted several educational online sessions for all team members to discuss relevant aspects of the statistical methods involved in the study. The same process was repeated for the 2020 publications as well.

The statistical method used was identified and categorised as being descriptive, elementary, multivariable, complex multivariate analysis, others (diagnostic/classification) or those associated with machine learning and data mining.[17] These categories were adapted from previously published classifications of statistical methods.[17] The individual methods included in each of the categories were as follows—Descriptive: mean [standard deviation (SD)], median [interquartile range (IQR)], number, percentage, ratio or proportions; Elementary: Chi-square, t-test, analysis of variance (ANOVA), Fisher’s exact, Kaplan–Meier, Wilcoxon rank, and correlation; Multivariable: Cox proportional hazard, linear regression, logistic regression; Complex multivariate: weighted logistic regression, unconditional logistic regression, conditional logistic regression, Poisson regression, pooled logistic regression, nonlinear regression, negative binomial regression or generalised estimating equation; Classification/diagnostic: relative risk, risk ratio, precision, recall, sensitivity and specificity. Newer type of analysis linked to machine learning and data mining includes the Bayesian networks, decision trees, artificial neural networks, support vector machines and clustering.[7]

Additionally, the type of study design in terms of randomised controlled trials/cohorts and software used for statistical analysis were noted as well. The number of original research articles was also noted as a percentage of all published articles (excluding letter to editor) in each journal over the two-year period.

All variables were coded as binary outcomes into an excel sheet. Results are presented as frequencies/percentages. Comparison of percentages between journals was done using the Chi-square/Fisher’s exact test. Statistical Package for the Social Sciences (SPSS) version 28 (International Business Machines Corp; USA) statistical software was used for statistical analysis. No sample size was calculated since it was a time-limited data extraction (publications of 2019-2020); however, the expected sample size was large, considering the frequency and regularity of publication of the journals.

RESULTS

A total of 748 original research articles were evaluated, including 179, 125 and 444 in IJA, JOACP and A&A, respectively. The percentage of original research articles was 179/450 (39.8%) in IJA, 125/233 (53.6%) in JOACP and 444/865 (51.3%) in A&A during the period surveyed.

A comparison of usage for individual statistical methods was done between both Indian journals (JOACP versus IJA) and then by clubbing both Indian journals together versus A&A [Table 1]. There was no significant difference between the usage of any individual statistical method when comparing JOACP with IJA [Table 1]. There were, however, significant differences between Indian versus the international journal. These comparisons were further analysed.

T1
Table 1:
Distribution of various statistical methods

From amongst the various descriptive methods (i.e., mean (SD), median [IQR], and percentages/ratio/numbers), the use of mean (SD) as well as percentages or related terms was significantly greater (P = 0.000 each) in the Indian journals, while it was significantly lesser for median [IQR] (P = 0.000) [Table 1].

Amongst the elementary methods (including Chi-square/Fisher’s exact test, t-test, ANOVA, Kaplan–Meier, Wilcoxon rank, Mann–Whitney and Pearson/Spearman correlation), Indian journals showed a significantly lesser usage of Wilcoxon and Pearson/Spearman correlation tests (P = 0.000 each) and significantly greater usage of Chi-square, t-test and Mann–Whitney test as compared to A&A (P = 0.000, 0.000 and 0.001, respectively) [Table 1]. The use of ANOVA and Kaplan–Meier analysis remained statistically similar for Indian journals versus A&A (P = 0.082) [Table 1].

Amongst the multivariable tests, there was significantly lesser usage in Indian journals versus A&A for logistic regression (7.6% versus 31.3%; P = 0.000), linear regression (3.9% versus 18.7%; P = 0.000) and Cox proportional hazard test (0.7% versus 5%; P = 0.001).

The frequency of using complex multivariate methods was almost nil in Indian journals (0% for each of the individual tests in JOACP; while in IJA also it was 0% for all, except conditional regression [1.1%]). In contrast, in A&A, there was a presence albeit low, for all the individual methods except unconditional regression which was not used [Table 1]. Poisson regression and generalised estimating equation achieved significantly greater usage in A&A versus the Indian journals together (P = 0.028 and 0.006 each) [Table 1].

The use of classification/diagnostic methods (viz. ROC analysis, sensitivity/specificity, recall, precision and relative risk/risk ratio) was also significantly lesser in Indian journals versus A&A (ROC analysis, P = 0.022; sensitivity/specificity, P = 0.000; precision, P = 0.006; and relative risk/risk ratio, P = 0.010) [Table 1].

There were three studies (all from A&A) that used methods for evaluating machine learning and none in JOACP or IJA. The frequencies for usage of each of the statistical methods were also derived [Table 1].

When considering all journals together, the usage was commonest for descriptive (96.5%), followed by elementary (78.4%), multivariable (31.3%), diagnostic/classifying (16.6%), complex multivariate (4%) and machine learning methods in decreasing order of frequencies [Figure 1]. Even when each of the three journals was considered independently, the same trend was apparent [Figure 1].

F1
Figure 1:
Distribution for categories of statistical methods. Values are depicted as percentages. IJA: Indian Journal of Anaesthesia; JOACP: Journal of Anaesthesiology Clinical Pharmacology

The software used for analysis was mentioned in 489/748 (65.3%) of the original research articles, including 54.7%, 80.8% and 81.0% each for A&A, JOACP and IJA, respectively. The commonest software used in A&A was R-software (90/243 = 37%), while it was SPSS for both Indian journals, i.e., 88.1% and 90.3% for JOACP and IJA, respectively.

Randomised controlled trials were commoner in the Indian journals (IJA = 60.9%, JOACP = 61.6%) than in A&A (19.8%). Cohort studies were published in greater numbers in A&A than in JOACP and IJA (40.1% versus 16.8% and 20.7%, respectively).

DISCUSSION

The study observations depict that the commonest categories of statistical analysis used in all journals included descriptive or elementary methods. The more advanced multivariable, complex multivariate and diagnostic/classifying tests were, however, more commonly used in the international journal (A&A) as compared to Indian journals [Figure 1]. The array of statistical methods has grown over time to extend from simple descriptive narratives such as mean and medians to the most advanced and complex procedures.[8-10]

The current observation of the use of descriptive/elementary methods amongst journals has been previously observed too, although not for anaesthesia literature.[7] This finding reiterates that the most popular statistical methods are often the simplest, and perhaps have been used for decades for this reason.[11] Educational narrative reviews on basic statistical tools can also be found with ease.[91213] Descriptive methods include measures of frequency, central tendency as well as variability. These measures help in consolidating a large amount of data for easier representation and comparisons.[4] It is undoubtedly acceptable that clinicians are most familiar with these simple tests and their utility, and hence the increased usage. A lack of awareness and knowledge regarding biostatistical methods and advancements thereof as well as the non-availability of qualified biostatisticians contributes to the use of more basic methodology in research. It is often noted that understanding and correctly applying biostatistical methods is difficult for clinicians.[251415]

Both the Indian journals, i.e., IJA and JOACP, had statistically similar usage of all the evaluated methods. However, when comparing them to A&A, there emerged a distinct and less encouraging difference in the pattern of usage.

Amongst the findings of the present study, the seemingly innocuous practice of using mean (SD) more frequently than median [IQR] itself is noteworthy. A median [IQR] representation is preferred with samples that do not have a normal distribution of data. To assess normal distribution, specific techniques such as a Kolmogorov–Smirnov test or assessing histograms, etc., are advocated. With no specific mention of such methods to analyse the type of distribution of data in the section of statistical analysis, the under-usage of median [IQR] may be explained. If we average the usage of Wilcoxon and Mann–Whitney tests (both for nonparametric data), it depicts a similar usage across all journals (A&A = 16.6%, JOACP = 15.2%, IJA = 16.7%). This implies that there may be an erroneous under-utilisation of median [IQR] in the Indian journals.

The Indian journals showed a significantly lesser use of correlation using Pearson/Spearman tests (A&A = 13.5%, JOACP = 5.6%, IJA = 4.5%), even though they are considered as basic elementary methods.[1116] Correlation testing is meant to be applied when looking for a possible association/relationship between variables. The under-utilisation of correlation testing in Indian journals may be a consequence of the greater unsubstantiated emphasis on randomised trials. The typical study design for carrying out tests of association/correlation is non-experimental, i.e., observational (cohort or cross sectional). The percentage of cohort studies was however much lesser in the Indian journals. Randomised controlled trials are often conducted with the ambitious aim to discover causation and hence dictate therapy. However, well-conducted cohort/cross-sectional trials may end up generating excellent research questions as well, even though not testing a hypothesis.[1116] The complete potential of correlation testing in exploring research questions was thus under-appreciated in the Indian journals.

The use of various ‘regression models’ aims to evaluate the association between variables.[17] We included regression models into multivariable and complex multivariate categories of statistical methods in our study.[1116] Nevertheless, the usage of regression was exceedingly low in Indian journals.

It is thus apparent that Indian researchers are not using the more complex and advanced methods. This may be the result of mere lack of knowledge amongst clinical researchers, even though the tests could be better suited to answer the research question. On the other hand, advanced statistical methods improve the quality and accuracy of results as well, and thus, the associated chances of publication. Involving a statistician at the stage of conceptualisation itself with a focus on appropriate study design and statistical tests required would be highly desirable. In a recent literature survey, it was noted that of 22,298 articles in top six international anaesthesia journals over a 10-year period, only 1.08% were contributed from India, and amongst these, 20% were original research articles with the majority being correspondences.[18] With the augmentation of statistical methods to more complex and advanced models, the quality of publications from the country will improve as well.

We chose IJA and JOACP to represent Indian anaesthesia journals given the fact that both are indexed with PubMed and remain credible databases. Thus, the results obtained could be expected to accurately reflect the use of biostatistical methods by Indian researchers in the field of anaesthesia.

The international journal we chose for comparison was ‘A&A’ since it remains one of the top-ranking anaesthesia journals (within first five ranked as per most recent rankings),[19] with a high impact factor. The journal features contributions by Indian anaesthesia researchers as well.[18] Additionally, the journal features regular articles focusing on statistical methods and concepts and has dedicated statistical reviewers. Hence, it was felt that the journal could be considered for a fruitful comparison/bench-marking.

There are limitations to the current study. It is depictive of a randomly chosen sample of journals. It is not mandatory that the findings are applicable across all Indian or international journals and specialities. Also, since a significant number of research articles must have been rejected and not accepted for publication, their statistical test usage cannot be commented upon. Lastly, the list of tests we have included in each category is not all-inclusive and thus remains modifiable.

CONCLUSION

The observations of the study show a limited use of advanced complex statistical tests in Indian anaesthesia journals. The tests usually employed are restricted to descriptive or elementary tests, and a strong bias towards using randomised controlled designs. The findings suggest an urgent and focussed need on training in research methodology, including statistical methods, during postgraduation and continued medical training.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

REFERENCES

1. Windish DM, Huot SJ, Green ML Medicine residents'understanding of the biostatistics and results in the medical literature JAMA 2007 298 1010 22
2. Gore A, Kadam Y, Chavan P, Dhumale G Application of biostatistics in research by teaching faculty and final-year postgraduate students in colleges of modern medicine:A cross-sectional study Int J Appl Basic Med Res 2012 2 11 6
3. Dhulkhed VK, Tantry TP, Kurdi MS Minimising statistical errors in the research domain:Time to work harder and dig deeper! Indian J Anaesth 2021 65 567 71
4. Raines DA Are you statistically literate? A basic overview of statistics Neonatal Netw 2013 32 289 94
5. Berwick DM, Fineberg HV, Weinstein MC When doctors meet numbers Am J Med 1981 71 991 8
6. Harsoor SS, Panditrao MM, Rao S, Bajwa SJS, Sahay N, Tantry TP Dissertation writing in post graduate medical education Indian J Anaesth 2022 66 34 46
7. Scotch M, Duggal M, Brandt C, Lin Z, Shiffman R Use of statistical analysis in the biomedical informatics literature J Am Med Inform Assoc 2010 17 3 5
8. Ma Y, Mazumdar M, Memtsoudis SG Beyond repeated-measures analysis of variance:Advanced statistical methods for the analysis of longitudinal data in anesthesia research Reg Anesth Pain Med 2012 37 99 105
9. Dexter F, Shafer SL Narrative review of statistical reporting checklists, mandatory statistical editing, and rectifying common problems in the reporting of scientific articles Anesth Analg 2017 124 943 7
10. Bajwa SJ Basics, common errors and essentials of statistical tools and techniques in anesthesiology research J Anaesthesiol Clin Pharmacol 2015 31 547 53
11. Schober P, Schwarte LA Correlation coefficients:Appropriate use and interpretation Anesth Analg 2018 126 1763 8
12. Ali Z, Bhaskar SB Basic statistical tools in research and data analysis Indian J Anaesth 2016 60 662 9 Erratum in:Indian J Anaesth 2016 60:790
13. Pandit JJ The analysis of variance in anaesthetic research:Statistics, biography and history Anaesthesia 2010 65 1212 20
14. Ganasegeran K, Ch'ng ASH, Jamil MFA, Looi I Clinicians'perceived understanding of biostatistical results in the medical literature:A cross-sectional study Medicina (Kaunas) 2019 55 227 doi:10.3390/medicina55060227
15. Laopaiboon M, Lumbiganon P, Walter SD Doctors'statistical literacy:A survey at Srinagarind Hospital, Khon Kaen University J Med Assoc Thai 1997 80 130 7
16. Harvey LA Relationships, associations, risk factors and correlations:Nebulous phrases without obvious clinical implications Spinal Cord 2020 58 1 2
17. Aoyama K, Pinto R, Ray J G, Hill A, Scales D C, Fowler RA Determining associations and estimating effects with regression models in clinical anesthesia Anesthesiology 2020 133 500 9
18. Kar P, Kar AK, Gopinath R Publication performance of Indian authors in high impact anesthesiology journals:Are we doing enough? J Anaesthesiol Clin Pharmacol 2021 37 505 8
19. Staffa SJ, Zurakowski D Recent trends in utilization of statistical methods in anesthesia research:2012-2017 Trends Anes Surg 2018 1 doi:10.15761/TAS.1000102
Keywords:

Anaesthesia; anaesthesiology; biomedical research; research design; ROC curve

Copyright: © 2022 Indian Journal of Anaesthesia