Radiomics-Based Machine Learning to Predict Recurrence in Glioma Patients Using Magnetic Resonance Imaging : Journal of Computer Assisted Tomography

Secondary Logo

Journal Logo

Neuroimaging: Brain

Radiomics-Based Machine Learning to Predict Recurrence in Glioma Patients Using Magnetic Resonance Imaging

Hu, Guanjie MSc∗,†,‡; Hu, Xinhua PhD∗,†; Yang, Kun PhD∗,†; Yu, Yun PhD§; Jiang, Zijuan PhD; Liu, Yong PhD∗,†; Liu, Dongming MSc; Hu, Xiao MSc; Xiao, Hong PhD; Zou, Yuanjie PhD∗,†; You, Yongping PhD; Liu, Hongyi PhD∗,†; Chen, Jiu PhD†,¶

Author Information
Journal of Computer Assisted Tomography 47(1):p 129-135, 1/2 2023. | DOI: 10.1097/RCT.0000000000001386

Abstract

According to the classification criteria of the World Health Organization, glioma is a common malignant tumor in the brain.1 Although the prognosis of glioma is poor, some recur several months after surgery, whereas few remain asymptomatic for years.2 The poor prognosis and recurrence of glioma are related to many factors: the invasive growth of glioma,3 genetic mutations (isocitrate dehydrogenase [IDH] and 1p/19q),4,5 radiation resistance,6 the lack of effectively targeted chemotherapeutic drugs,7 and immunotherapy resistance.8 In clinical practice, recurrence commonly occurs in glioma patients. Unfortunately, at present, it is not possible to predict which patients will develop recurrence sufficiently in advance, which affects the ability to provide effective treatment for recurrence.

Machine learning, which is a category of artificial intelligence9 that simulates the human learning mode, summarizes rules from historical experience, and analyzes the prediction of new objects, may offer a tool for the construction of a recurrence prediction model in patients with glioma. In particular, machine learning is performed based on statistics and in combination with computer science. Several machine learning methods are available, including linear regression analysis, the support vector machine (SVM), decision trees, the random forest, and neural networks.10 Furthermore, the application of various methods may aid in identifying suitable models for a particular data set. Machine learning involves numerous applications and is being used increasingly in the field of medicine. Such methods can help physicians with auxiliary diagnoses and provide improved efficiency through the use of computer-aided testing and diagnostic images.11 Furthermore, machine learning is currently used extensively in computed tomography angiography for segmenting lung embolisms,12,13 virtual colonoscopies or computed tomography diagnoses of colon polyps,14,15 breast cancer detection and diagnosis,16 brain segmentation under magnetic resonance imaging (MRI),17,18 the detection of the brain cognitive state, and the diagnosis of neurological diseases based on MRI features.19,20 Therefore, machine learning offers potential significance in precision medicine and may provide a valuable tool for predicting recurrence in patients with a history of glioma.

At present, MRI, which is a noninvasive examination tool, plays an important role in the detection of glioma disease.21 The radiomic features that are extracted from MRI have been applied extensively in the construction of predictive models, including those for diagnosis, prognosis, and therapeutic response.22 Moreover, many studies have revealed a strong relationship between MRI features and biological information in glioma patients using machine learning. Several researchers have successfully predicted H3K27M mutations with radiomic features in glioma patients using machine learning.23–25 Radiomic information has also been used effectively in the prediction of other gene mutations in glioma patients, including mutations of IDH and 1p/19q.26–32 Despite the heterogeneity of the patients who were enrolled in these studies, machine learning achieved effective prediction of gene mutations in glioma. In addition to genetic mutations, radiomic information has been demonstrated as useful in predicting tumor grades and overall survival in glioma patients.32–37 Furthermore, researchers successfully predicted recurrence in high-grade glioma in a pilot study based on deep learning using multimodal MRI features.38 Interestingly, in another study, the subsequent location of the recurrence was prospectively predicted using machine learning, and promising results were exhibited in glioblastoma.39 Although the aforementioned studies focused on glioma recurrence, they all had a small sample size and included only high-grade gliomas. However, prospective studies of patients with multiple gioma types, in which those with and without recurrence are distinguished, are lacking. The prediction of recurrence in advance can aid physicians in performing targeted and individualized treatment much earlier and may improve the prognosis of patients.

Recurrence is one of the primary reasons for the poor prognosis of glioma. Therefore, in this study, we aimed to predict glioma recurrence through the identification of predictive radiomic features using machine learning. We hypothesized that the recurrence of glioma could be predicted successfully using machine learning models.

MATERIALS AND METHODS

Patient Enrollment

This study, in which 105 glioma patients were initially enrolled, was approved by the Institutional Ethical Committee for Clinical Research of the Affiliated Brain Hospital of Nanjing Medical University (Nanjing, China). Owing to data loss and poor data quality in certain instances, a total of 77 patients were ultimately enrolled. These patients were recruited from the Department of Neurosurgery at the Affiliated Brain Hospital of Nanjing Medical University from 2012 to 2015 and were confirmed as glioma patients upon subsequent pathological diagnosis after surgery. None of the patients had any other head diseases or MRI contraindications. Patients who exhibited alcohol and substance abuse were excluded. Furthermore, 20 patients who met the aforementioned criteria were recruited as an independent validation set from the Department of Neurosurgery at the First Affiliated Hospital of Nanjing Medical University between 2014 and 2019. Owing to privacy and ethical restrictions, the data are not publicly available.

MRI Examinations

Based on follow-up data of 12 months,40 neuroradiologists reviewed the MRI images of patients upon reexamination after discharge and those upon first admission to determine whether the patients had relapsed according to the Response Assessment in Neuro-Oncology criteria.41 Finally, the different states of the 2 groups of patients were regarded as predictive categorical labels. Images of the MRI were acquired using 3.0T Verio Siemens scanners. The scanned images included nonenhanced T1-weighted images, nonenhanced T2-weighted images, and enhanced T2-weighted images. Because the full information of tumors including edema is included in T2-weighted images,42 the machine learning analysis was performed on the nonenhanced T2-weighted images of the first admission before surgery. The acquisition parameters of the images were as follows: repetition time, 3000 milliseconds; echo time, 104 milliseconds; flip angle, 150 degrees; and slice thickness, 6 mm.

Tumor Segmentation and Feature Extraction

After the bones of the skull were removed using FMRIB Software Library,43 the tumor and its maximum edema area were manually segmented on the nonenhanced T2-weighted images of the first admission before surgery by 2 experts with 5 and 20 years of diagnostic experience using the ITK-SNAP software (http://www.itksnap.org).44,45 The final region of interest (ROI) was obtained by the superposition of the segmentations that were generated by the 2 experts. However, if the overlap was less than 80% of the full ROI, the final ROI was defined by a third investigator.25 After the aforementioned steps, the ROI of the manually segmented tumor was as close as possible to that of the real tumor.

Once the ROIs were defined, we attempted to extract the radiomic features of each patient using the open-source code PyRadiomics with the default parameters in Python 3.7.46,47 The extracted radiomic features included 14 shape, 18 first-order, 24 gray-level co-occurrence matrix (GLCM), 16 gray-level run length matrix, 16 gray-level size zone matrix (GLSZM), 14 gray-level dependent matrix, and 5 neighborhood gray-tone difference matrix (NGTDM) features.

Classification Procedure

In this study, feature selection, hyperparameter optimization, and model selection were performed. The models, which included the SVM, random forest, logistic regression, decision tree, gradient boosting, and extra trees, were constructed using the Tree-based Pipeline Optimization Tool (TPOT; http://epistasislab.github.io/tpot/).48,49 The Tree-based Pipeline Optimization Tool is an automated machine learning tool in Python that involves several steps, including feature selection, model pipelines selection, and hyperparameter optimization. The resource-intensive manual model selection and parameter optimization in machine learning can be eliminated or notably reduced using TPOT. In a previous study, the H3K27M mutation was identified using TPOT, which yielded fairly good results.25 Before the runtime analysis, the subjects were randomly divided into a training cohort (58 patients) and testing cohort (19 patients) according to previous studies.25,29,34,50 When TPOT was run, each iteration maintained the best pipeline until the end. We adopted the default settings of TPOT, but the number of generations was set to 1000. After 1000 generations, TPOT generated the best pipeline and the corresponding code for the current number of generations.

We performed the aforementioned process 10 times to produce a reliable model; therefore, 10 independent models were generated. We used 10-fold cross-validation to determine the average accuracy, average area under the curve (AUC) of the generated receiver operating characteristic (ROC) curves, average specificity, average sensitivity, and average F1 score of the models. Furthermore, we conducted 5-fold cross-validation to avoid bias in the results. The aforementioned indicators were compared to identify the best model. Subsequently, we used an independent validation set of 20 patients to verify the effectiveness of the optimal model. Figure 1 presents the overall workflow. Moreover, we ranked the features in these models according to their contribution to the models using the feature importance code in the Python environment.

F1
FIGURE 1:
Flowchart of machine learning. (1) T2-weighted images of 77 glioma patients; (2) manual segmentation of glioma using ITK-SNAP software; (3) radiomic features extraction based on open-source code PyRadiomics in Python 3.7 including shape, texture, and first-order; (4) statistical analysis, establishment of machine learning model in recurrence predicting using TPOT. Figure 1 can be viewed online in color at www.jcat.org.

We applied a manual machine learning method in addition to using TPOT. In the MATLAB environment, the SVM was selected to construct the machine learning model for the data set. In this process, we adopted data normalization, feature selection (Pearson correlation), and leave-one-out cross-validation.

Statistical Analysis

Statistical analyses were performed using SPSS version 22. The sexes and tumor grades among the groups were compared using the χ2 test. Differences in the age and overall survival were determined by the 2-sample t test. The statistical significance was defined as P < 0.05.

RESULTS

Patient Characteristics

Images on the MRI from 77 glioma patients were obtained, including 57 newly diagnosed glioma patients (age, 54.40 ± 15.34 years) and 20 glioma patients with recurrence (age, 44.65 ± 15.47 years). A significant age difference between the 2 groups was observed (P = 0.02), but no significant differences in the overall survival were observed (P = 0.66). Furthermore, no significant differences in the sex (P = 0.86) or glioma grade (P = 0.84) were observed between the 2 groups by the χ2 test. Table 1 summarizes the patient characteristics. In the validation set of 20 patients, the patient ratio was similar to that of the model construction: 15 patients (age, 56.87 ± 13.23) were newly diagnosed, and 5 (age, 54.80 ± 10.62 years) were recurrent. No significant differences in the age and sex between the groups were observed.

TABLE 1 - Demographic Characteristics of Glioma Patients
Variable Incipient (n = 57) Recrudesce (n = 20) χ2/t/F (P Value)
Age, y 54.40 ± 15.34 44.65 ± 15.47 0.02
Overall survival (0.5 mo) 20.88 ± 11.84 19.75 ± 8.95 0.66
Sex 0.86
 Male 30 11
 Female 27 9
Grade
 Low (2) 16 5 0.84
 High (3) 16 7
 High (4) 25 8
P values were determined using one-way analysis of variance for age and overall survival. The χ2 test was used for sex ratio and grade.

Feature Importance Rank

A total of 107 features were extracted from the nonenhanced T2-weighted images using PyRadiomics. The most important features of the models are listed in Table S1, https://links.lww.com/RCT/A163. The top 10 crucial features for the best model (model 6) included 5 GLCM features, 2 shape features, 1 NGTDM feature, 1 first-order feature, and 1 GLSZM feature (Table 2 and Fig. 2).

TABLE 2 - The First 10 Features of Model 6
Model 6
Features Values
Original GLDM large dependence low-gray level emphasis 35.45113369
Original GLCM Id −33.32783439
Original GLCM cluster tendency 12.75824795
Original GLCM Imc2 −12.62423302
Original NGTDM complexity 8.73055665
Original first-order 10 percentile −8.34846336
Original shape maximum 3D diameter −8.17939938
Original GLCM maximum probability 5.7446031
Original GLSZM small-area low gray-level emphasis −5.7062086
Original shape mesh volume 5.1023505
GLDM indicates gray level dependent matrix.

F2
FIGURE 2:
The first 10 features of model 6. Figure 2 can be viewed online in color at www.jcat.org.

Model Comparison and Final Model

The accuracy of the 10 models that were generated by TPOT ranged from 0.72 to 0.81, the AUC value ranged from 0.67 to 0.85, the specificity ranged from 0.56 to 0.69, the sensitivity ranged from 0.58 to 0.68, and the F1 score ranged from 0.56 to 0.67. After the model comparison, model 6 was selected as the optimal model, with an accuracy of 0.81, an AUC value of 0.85, a specificity of 0.69, a sensitivity of 0.68, and an F1 score of 0.67. The specific details of the 10 models are presented in Table 3. The results of the 5-fold cross-validation were similar to the aforementioned findings and are listed in Table S2, https://links.lww.com/RCT/A163. In the 5-fold cross-validation, the best model (model 6) achieved an accuracy of 0.77, an AUC value of 0.73, a specificity of 0.67, a sensitivity of 0.65, and an F1 score of 0.64. We also plotted the ROC curve for model 6 (Fig. 3: 10-fold cross-validation; Figure 4: 5-fold cross-validation), which exhibited the best performance. The parameters of model 6 were as follows: logistic regression (c = 20.0, dual = false, penalty = “l1”). Moreover, model 6 included the steps of data normalization (RobustScaler) and model stacking. The stacking was based on the results of the base learner as the input of the meta-learner to obtain the final result. The meta-learner of model 6 was logistic regression, and the base learners of model 6 were the K-neighbor classifier (n_neighbors = 44, p = 2, weights = “uniform”), K-neighbor classifier (n_neighbors = 45, p = 2, weights = “uniform”), and XGB classifier (learning_rate = 1.0, max_depth = 2, min_child_weight = 2, n_estimators = 100, nthread = 1, subsample = 0.2). The parameters of the remaining nine models are described in Table S3, https://links.lww.com/RCT/A163. When the manual machine learning method was run in the MATLAB environment, we obtained an accuracy of 0.73 and an AUC value of 0.66 using the SVM (Fig. S1, https://links.lww.com/RCT/A163). Furthermore, we achieved an accuracy of 0.75 and an AUC value of 0.87 for model 6 in the validation set (Fig. S2, https://links.lww.com/RCT/A163).

TABLE 3 - Specific Information for Each Model Using 10-Fold Cross-Validation
Variable Accuracy Specificity Sensitivity F1 Score AUC
Model 1 0.755357 0.59881 0.6 0.577463 0.71
Model 2 0.726786 0.593095 0.623333 0.588007 0.78
Model 3 0.757143 0.682857 0.641667 0.637929 0.81
Model 4 0.767857 0.678929 0.64 0.629471 0.74
Model 5 0.717857 0.607381 0.615 0.589782 0.71
Model 6 0.808929 0.688690 0.675 0.665634 0.85
Model 7 0.728571 0.688452 0.675 0.655286 0.7
Model 8 0.755357 0.623929 0.633333 0.610258 0.76
Model 9 0.764286 0.564881 0.595 0.565994 0.74
Model 10 0.742857 0.574048 0.583333 0.564351 0.67

F3
FIGURE 3:
Area under the curve of model 6 based on 10-fold cross-validation. The x coordinate is the false-positive rate, and the y coordinate is the true-positive rate. The dotted lines represent diagonals, and the real black line represents ROC. The AUC of model 6 is 0.85. Figure 3 can be viewed online in color at www.jcat.org.
F4
FIGURE 4:
Area under the curve of model 6 based on 5-fold cross-validation. The x coordinate is the false-positive rate, and the y coordinate is the true-positive rate. The dotted lines represent diagonals, and the real black line represents ROC. The AUC of model 6 is 0.73. Figure 4 can be viewed online in color at www.jcat.org.

DISCUSSION

Machine learning is playing an increasingly significant role in precision medicine. Based on previous research,51–53 we conducted 10-fold and 5-fold cross-validation in this study. The algorithm that was generated by machine learning could noninvasively predict the recurrence of glioma with an accuracy of 0.81 and an AUC value of 0.85 based on the 10-fold cross-validation, and an accuracy of 0.77 and AUC value of 0.72 based on the 5-fold cross-validation. In the independent validation data set, we obtained an accuracy of 0.75 and an AUC value of 0.87 with model 6. The superior results of the 10-fold cross-validation are consistent with those that were obtained previously. An increase in the K value has been reported to be associated with improved results.51 Note that the data set in this study was not large, and a small K value affects the modeling. Therefore, the selection of a larger K value for cross-validation could provide better results.

The edema features around the tumor were extracted in several previous studies, which may lead to feature loss of the glioma information. Therefore, we included the tumor and its surrounding edema in the delineation of the ROIs of each patient to minimize the loss of tumor information. PyRadiomics,46 which is an open-source tool that is implemented in Python and used extensively in cancer research, was applied for the feature extraction of each glioma patient. PyRadiomics is a standardized algorithmic definition with good reproducibility and comparability of results.46 Therefore, feature extraction using PyRadiomics is a fairly promising approach.

In general, traditional machine learning requires the manual selection of models and manual adjustment of parameters, which is a tedious process.10 However, automated machine learning may omit such laborious processes. Although a longer time and additional computation are required, the generated machine learning algorithm can achieve satisfactory results.48,49 Existing automated machine learning methods include Auto-Sklearn, AutoKeras, TPOT, H2O AutoMl, and Python AutoMl. The Tree-based Pipeline Optimization Tool automatically optimizes the model parameters and selects appropriate features to improve the classification performance.49 Moreover, in this study, the final results of the automated machine learning TPOT were superior to those obtained by the SVM model with manual parameter adjustment, which indicates the promising performance of TPOT. A total of 10 models, including the random forest, decision tree, logistic regression, gradient boosting, and extra trees, were obtained after 10 repeated runs with 1000 generations per run. The best logistic regression model achieved an accuracy of 0.81 and an AUC value of 0.85, which was an encouraging result. The prediction model constructed by TPOT may provide novel perspectives for the prediction of glioma recurrence.

In this study, model 6, which used logistic regression with model stacking, was the best model. Logistic regression is a classification algorithm that is commonly used for dichotomies. It converts the input value into a predicted value in linear regression that is subsequently mapped into a sigmoid function. The value is considered as a variable on the x axis, whereas the y axis denotes the probability. A value that is closer to 1 indicates greater similarity to the predicted value. However, better fitting does not mean a better effect because overfitting may occur. Furthermore, logistic regression can be used for multiple classifications.54 Model stacking is essentially a hierarchical structure model. The principle of each algorithm varies slightly, and different data may be suitable for different algorithms; thus, an appropriate classification method for the data can only be determined through continuous fitting. Therefore, logistic regression with model stacking was the most suitable prediction model for our data.

The top 10 features of model 6 included 5 GLCM features, 2 shape features, 1 NGTDM feature, 1 first-order feature, and 1 GLSZM feature. The shape played a nonnegligible role in the prediction of glioma recurrence, which was consistent with previous studies. To the best of our knowledge, gliomas often cause morphological changes in the tumor areas and surrounding tissues, and a previous study also confirmed that shape features make a significant contribution to improved model performance.55 Researchers previously found that the shape plays an important role in predicting H3K27M and IDH mutations.25,26 In another study, the maximum tumor length was the most crucial feature in machine learning predictive models for spinal cord diffuse midline gliomas.23 Therefore, we speculated that a strong relationship may exist between the shape and glioma recurrence; however, further investigation and verification are required. The textural features of glioma, including the GLCM, NGTDM, and GLSZM, are represented by the grayscale distributions of pixels. In previous studies, convincing performance was achieved in predicting IDH mutations in glioma patients based on the textural tumor features.56,57 Thus, consistent with previous studies, the textural glioma features of the surrounding spatial neighborhood were crucial to the recurrence prediction. Moreover, the first-order features, in which the statistics describe the distributions of the voxel intensities within an extracted ROI through commonly used and basic metrics, contained important information regarding the prediction. First-order features have been applied extensively and have exhibited good results in both glioma and lung cancer patients in machine learning settings.58,59 As in previous studies, the textural and first-order features achieved promising prediction performance.56,59,60 Thus, we speculate that alterations in the textural and first-order features may serve as biomarkers for glioma recurrence.

CONCLUSIONS

The machine learning algorithm that was generated in this study could achieve high-accuracy prediction of glioma recurrence using features that were extracted using PyRadiomics and an automated machine learning algorithm (TPOT). The TPOT algorithm eliminates or reduces several resource-intensive steps of machine learning, and the resulting generated algorithm exhibited promising results. Thus, we believe that TPOT will be used increasingly because it continues to evolve. Our preliminary results provide a reference for predicting the clinical recurrence of glioma. Therefore, radiomics may play a valuable role in the prediction of glioma recurrence.

This study exhibits several limitations. First, the sample size was limited and imbalanced. We will continue to expand the sample size to construct a more general and balanced machine learning model. Second, the inclusion of excessive features may lead to overfitting, but the feature selection was performed using TPOT and the demographic information of patients was not included. In the future, the demographic information of patients could be included for further analysis provided that a sufficient number of samples is collected. Furthermore, we only performed analysis in the T2-weighted images of glioma patients, which may be a reason for the classification effect not being impressive. The information of the tumor itself and its surrounding areas involving edema was extracted, thereby maximizing the retention of tumor radiomics information. We believe that the performance of machine learning models may be improved by applying multiple imaging models, which will be our focus in future studies.

REFERENCES

1. Louis DN, Perry A, Reifenberger G, et al. The 2016 World Health Organization classification of tumors of the central nervous system: a summary. Acta Neuropathol. 2016;131:803–820.
2. van den Bent MJ, Wefel JS, Schiff D, et al. Response assessment in neuro-oncology (a report of the RANO group): assessment of outcome in trials of diffuse low-grade gliomas. Lancet Oncol. 2011;12:583–593.
3. Angelopoulou E, Piperi C. Emerging role of plexins signaling in glioma progression and therapy. Cancer Lett. 2018;414:81–87.
4. Waitkus MS, Diplas BH, Yan H. Isocitrate dehydrogenase mutations in gliomas. Neuro Oncol. 2016;18:16–26.
5. Eckel-Passow JE, Lachance DH, Molinaro AM, et al. Glioma groups based on 1p/19q, IDH, and TERT promoter mutations in tumors. N Engl J Med. 2015;372:2499–2508.
6. Bao S, Wu Q, McLendon RE, et al. Glioma stem cells promote radioresistance by preferential activation of the DNA damage response. Nature. 2006;444:756–760.
7. Stepanenko AA, Chekhonin VP. Recent advances in oncolytic virotherapy and immunotherapy for glioblastoma: a glimmer of hope in the search for an effective therapy?Cancers (Basel). 2018;10:492.
8. Jackson CM, Choi J, Lim M. Mechanisms of immunotherapy resistance: lessons from glioblastoma. Nat Immunol. 2019;20:1100–1109.
9. Erickson BJ, Korfiatis P, Akkus Z, et al. Machine learning for medical imaging. Radiographics. 2017;37:505–515.
10. Giger ML. Machine learning in medical imaging. J Am Coll Radiol. 2018;15(3 Pt B):512–520.
11. Schoepf UJ, Costello P. CT angiography for diagnosis of pulmonary embolism: state of the art. Radiology. 2004;230:329–337.
12. Schoepf UJ, Schneider AC, Das M, et al. Pulmonary embolism: computer-aided detection at multidetector row spiral computed tomography. J Thorac Imaging. 2007;22:319–323.
13. Dundar MM, Fung G, Krishnapuram B, et al. Multiple-instance learning algorithms for computer-aided detection. IEEE Trans Biomed Eng. 2008;55:1015–1021.
14. Summers RM. Improving the accuracy of CTC interpretation: computer-aided detection. Gastrointest Endosc Clin N Am. 2010;20:245–257.
15. Yoshida H, Nappi J. CAD in CT colonography without and with oral contrast agents: progress and challenges. Comput Med Imaging Graph. 2007;31:267–284.
16. Chan HP, Lo SC, Sahiner B, et al. Computer-aided detection of mammographic microcalcifications: pattern recognition with an artificial neural network. Med Phys. 1995;22:1555–1567.
17. Bauer S, Wiest R, Nolte LP, et al. A survey of MRI-based medical image analysis for brain tumor studies. Phys Med Biol. 2013;58:R97–R129.
18. Mohseni Salehi SS, Erdogmus D, Gholipour A. Auto-context convolutional neural network (Auto-Net) for brain extraction in magnetic resonance imaging. IEEE Trans Med Imaging. 2017;36:2319–2330.
19. Mitchell TM, Shinkareva SV, Carlson A, et al. Predicting human brain activity associated with the meanings of nouns. Science. 2008;320:1191–1195.
20. Davatzikos C, Fan Y, Wu X, et al. Detection of prodromal Alzheimer's disease via pattern classification of magnetic resonance imaging. Neurobiol Aging. 2008;29:514–523.
21. Antonios JP, Soto H, Everson RG, et al. Detection of immune responses after immunotherapy in glioblastoma using PET and MRI. Proc Natl Acad Sci U S A. 2017;114:10220–10225.
22. Chaddad A, Kucharczyk MJ, Daniel P, et al. Radiomics in glioblastoma: current status and challenges facing clinical implementation. Front Oncol. 2019;9:374.
23. Jung JS, Choi YS, Ahn SS, et al. Differentiation between spinal cord diffuse midline glioma with histone H3 K27M mutation and wild type: comparative magnetic resonance imaging. Neuroradiology. 2019;61:313–322.
24. Pan CC, Liu J, Tang J, et al. A machine learning-based prediction model of H3K27M mutations in brainstem gliomas using conventional MRI and clinical features. Radiother Oncol. 2019;130:172–179.
25. Su X, Chen N, Sun H, et al. Automated machine learning based on radiomics features predicts H3 K27M mutation in midline gliomas of the brain. Neuro Oncol. 2020;22:393–401.
26. Zhang B, Chang K, Ramkissoon S, et al. Multimodal MRI features predict isocitrate dehydrogenase genotype in high-grade gliomas. Neuro Oncol. 2017;19:109–117.
27. Zhou H, Vallieres M, Bai HX, et al. MRI features predict survival and molecular markers in diffuse lower-grade gliomas. Neuro Oncol. 2017;19:862–870.
28. Zhou H, Chang K, Bai HX, et al. Machine learning reveals multimodal MRI patterns predictive of isocitrate dehydrogenase and 1p/19q status in diffuse low- and high-grade gliomas. J Neurooncol. 2019;142:299–307.
29. van der Voort SR, Incekara F, Wijnenga MMJ, et al. Predicting the 1p/19q codeletion status of presumed low-grade glioma with an externally validated machine learning algorithm. Clin Cancer Res. 2019;25:7455–7462.
30. Kocak B, Durmaz ES, Ates E, et al. Radiogenomics of lower-grade gliomas: machine learning-based MRI texture analysis for predicting 1p/19q codeletion status. Eur Radiol. 2019.
31. Lu CF, Hsu FT, Hsieh KL, et al. Machine learning-based radiomics for molecular subtyping of gliomas. Clin Cancer Res. 2018;24:4429–4436.
32. Chaddad A, Desrosiers C, Abdulkarim B, et al. Predicting the gene status and survival outcome of lower grade glioma patients with multimodal MRI features. IEEE Access. 2019;7:75976–75984.
33. Macyszyn L, Akbari H, Pisapia JM, et al. Imaging patterns predict patient survival and molecular subtype in glioblastoma via machine learning techniques. Neuro Oncol. 2016;18:417–425.
34. Takahashi S, Takahashi W, Tanaka S, et al. Radiomics analysis for glioma malignancy evaluation using diffusion kurtosis and tensor imaging. Int J Radiat Oncol Biol Phys. 2019;105:784–791.
35. Liu L, Zhang H, Wu J, et al. Overall survival time prediction for high-grade glioma patients based on large-scale brain functional networks. Brain Imaging Behav. 2019;13:1333–1351.
36. Chaddad A, Zhang M, Desrosiers C, et al. Deep Radiomic Features From MRI Scans Predict Survival Outcome of Recurrent Glioblastoma. Cham, Switzerland: Springer International Publishing; 2020.
37. Chaddad A, Daniel P, Sabri S, et al. Integration of radiomic and multi-omic analyses predicts survival of newly diagnosed IDH1 wild-type glioblastoma. Cancers (Basel). 2019;11:1148.
38. Bacchi S, Zerner T, Dongas J, et al. Deep learning in the detection of high-grade glioma recurrence using multiple MRI sequences: a pilot study. J Clin Neurosci. 2019;70:11–13.
39. Akbari H, Macyszyn L, Da X, et al. Imaging surrogates of infiltration obtained via multiparametric imaging pattern analysis predict subsequent location of recurrence of glioblastoma. Neurosurgery. 2016;78:572–580.
40. Guo L, Li X, Chen Y, et al. The efficacy of hypofractionated radiotherapy (HFRT) with concurrent and adjuvant temozolomide in newly diagnosed glioblastoma: a meta-analysis. Cancer Radiother. 2021;25:182–190.
41. Nayak L, DeAngelis LM, Brandes AA, et al. The neurologic assessment in neuro-oncology (NANO) scale: a tool to assess neurologic function for integration into the response assessment in neuro-oncology (RANO) criteria. Neuro Oncol. 2017;19:625–635.
42. Kirste S, Treier M, Wehrle SJ, et al. Boswellia serrata acts on cerebral edema in patients irradiated for brain tumors: a prospective, randomized, placebo-controlled, double-blind pilot trial. Cancer. 2011;117:3788–3795.
43. Jenkinson M, Beckmann CF, Behrens TE, et al. FSL. Neuroimage. 2012;62:782–790.
44. Yushkevich PA, Piven J, Hazlett HC, et al. User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage. 2006;31:1116–1128.
45. Yushkevich PA, Pashchinskiy A, Oguz I, et al. User-guided segmentation of multi-modality medical imaging datasets with ITK-SNAP. Neuroinformatics. 2019;17:83–102.
46. van Griethuysen JJM, Fedorov A, Parmar C, et al. Computational radiomics system to decode the radiographic phenotype. Cancer Res. 2017;77:e104–e107.
47. Liu Q, Jiang P, Jiang Y, et al. Prediction of aneurysm stability using a machine learning model based on PyRadiomics-derived morphological features. Stroke. 2019;50:2314–2321.
48. Orlenko A, Kofink D, Lyytikäinen LP, et al. Model selection for metabolomics: predicting diagnosis of coronary artery disease using automated machine learning. Bioinformatics. 2020;36:1772–1778.
49. Le TT, Fu W, Moore JH. Scaling tree-based automated machine learning to biomedical big data with a feature set selector. Bioinformatics. 2020;36:250–256.
50. Wu S, Meng J, Yu Q, et al. Radiomics-based machine learning methods for isocitrate dehydrogenase genotype prediction of diffuse gliomas. J Cancer Res Clin Oncol. 2019;145:543–550.
51. Zhang YL, Yang YH. Cross-validation for selecting a model selection procedure. J Econom. 2015;187:95–112.
52. Krstajic D, Buturovic LJ, Leahy DE, et al. Cross-validation pitfalls when selecting and assessing regression and classification models. J Cheminform. 2014;6:10.
53. Jung Y. Multiple predicting K-fold cross-validation for model selection. J Nonparamet Stat. 2018;30:197–215.
54. Meurer WJ, Tolles J. Logistic regression diagnostics: understanding how well a model predicts outcomes. JAMA. 2017;317:1068–1069.
55. Menze BH, Jakab A, Bauer S, et al. The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans Med Imaging. 2015;34:1993–2024.
56. Moiseev A, Snopova L, Kuznetsov S, et al. Pixel classification method in optical coherence tomography for tumor segmentation and its complementary usage with OCT microangiography. J Biophotonics. 2018;11:e201700072.
57. Jakola AS, Zhang YH, Skjulsvik AJ, et al. Quantitative texture analysis in the prediction of IDH status in low-grade gliomas. Clin Neurol Neurosurg. 2018;164:114–120.
58. Zhang N, Xia M, Qiu T, et al. Reorganization of cerebro-cerebellar circuit in patients with left hemispheric gliomas involving language network: a combined structural and resting-state functional MRI study. Hum Brain Mapp. 2018;39:4802–4819.
59. Moran A, Daly ME, Yip SSF, et al. Radiomics-based assessment of radiation-induced lung injury after stereotactic body radiotherapy. Clin Lung Cancer. 2017;18:e425–e431.
60. Li Y, Qian Z, Xu K, et al. MRI features predict p53 status in lower-grade gliomas via a machine-learning approach. Neuroimage Clin. 2018;17:306–311.
Keywords:

glioma; machine learning; recurrence; tree-based pipeline optimization tool

Supplemental Digital Content

Copyright © 2022 Wolters Kluwer Health, Inc. All rights reserved.