Journal Logo

Research Article: Diagnostic Accuracy Study

Discriminating malignant and benign clinical T1 renal masses on computed tomography

A pragmatic radiomics and machine learning approach

Uhlig, Johannes MD, MPHa,b; Biggemann, Lorenz MDa; Nietert, Manuel M. PhDc; Beißbarth, Tim PhDc; Lotz, Joachim MDa,d; Kim, Hyun S. MDb,e; Trojan, Lutz MDf; Uhlig, Annemarie MD, MPHf,∗

Editor(s): Lucarelli., Giuseppe

Author Information
doi: 10.1097/MD.0000000000019725

Abstract

1 Introduction

Renal cell carcinoma (RCC) is the most common renal malignancy worldwide, accounting for approximately 175,000 annual cancer-related deaths.[1] In recent years, RCC incidence has been increasing, which is partially attributable to technical advancements and wider availability of cross-sectional imaging.[2] In particular, the incidence of small renal masses has increased, now comprising up to 40% of all renal masses in the United States.[3,4] Although the majority of renal masses are malignant, studies report that 16% to 19% of renal masses are benign.[5,6]

Renal masses are often incidentally detected on cross-sectional imaging that has been performed for other indications.[7] In these cases, imaging studies might not be optimized for characterization of renal masses and might, thus, lack crucial information. For example, routine computed tomography (CT) studies might lack contrast enhanced images in corticomedullary renal phase.

This monophasic imaging studies might complicate renal mass assessment, since classical CT features are not available, such as enhancement patterns over time.[8] In patients with incidentally detected renal masses on monophasic studies, repeated imaging with dedicated multiphasic CT might be performed. Still, this exposes patients to radiation and high doses of iodinated contrast media, and even multiphasic CT studies do not allow unequivocal classification of renal masses in all cases.[8] Nuclear medicine studies, magnetic resonance imaging, as well as ultrasound and renal biopsy might be indicated to further assess incidental renal masses. Still, these procedures might not be timely available for all patients and are themselves associated with costs and procedural risk. Therefore, the questions remain whether advanced image analyses could aid in assessment of incidental renal masses on monophasic CT studies.

Radiomic feature analyses and machine learning algorithms have recently been shown to perform well on a range of different radiological imaging types, including magnetic resonance imaging and CT.[9,10] Still, to date there is no literature to evaluate the diagnostic performance of these techniques for the assessment of clinical T1 renal masses in a real-world setting with various CT scanners, acquisition protocols, and potential artifacts.

The aim of this study was to train a machine learning algorithm for discrimination of malignant and benign clinical T1 renal masses that were detected on venous CT and compare its diagnostic performance to experienced radiologists.

2 Material and methods

This retrospective, STARD-compliant study received previous approval by the ethics committee of the University Medical Center Goettingen (No 2/4/17) and is compliant with the Declaration of Helsinki.

2.1 Patient inclusion

Adult patients presenting for surgical resection of renal masses consecutively between 2012 and 2017 at the University Medical Center Goettingen were considered for inclusion if preoperative, contrast-enhanced CT studies in venous phase were available. Only renal masses radiologically staged T1 with maximal diameter of 70 mm in any direction were included in this study. Exclusion criteria were diffuse infiltrative renal disease (ie, lymphoma) and primarily cystic lesions. A study flow-chart is provided in Figure 1.

Figure 1
Figure 1:
Study flow chart and prospective clinical application.

2.2 Radiological Imaging

In a pragmatic, real-life approach, we evaluated contrast-enhanced CT studies in venous phase only. Our study was not restricted to CT studies performed at our tertiary center but included patients that were referred with imaging from external centers as well. No restrictions were made regarding CT scanner, slice thickness, or imaging artifacts.

2.3 Radiomic features

The open source software 3D Slicer was used for renal mass segmentation and radiomic feature analyses.[11] Radiomic features in 3D slicer are based on standardized and reproducible algorithms in accordance with feature definitions by the Imaging Biomarker Standardization Initiative.[12,13] A bin width of 25 was chosen.

Renal masses were segmented (delineation of region of interest) on axial venous CT images by 2 radiologists in consensus. The total number of assessed CT slices varied with renal mass size and CT slice thickness.

From the segmented renal masses, a total of 120 distinct radiomic features were analyzed, with extended details provided online.[12] Radiomic features are subdivided into 8 classes:

  1. First-order statistic (describing renal mass voxel intensity)
  2. 3D shape features (describing 3-dimensional size and shape of renal mass)
  3. 2D shape features (describing 2-dimensional size and shape of renal mass)
  4. Gray-level co-occurrence matrix features (GLCM; describing second-order joint probability function of renal mass)
  5. Gray-level size zone matrix features (GLSZM; quantifying gray level zones in renal mass)
  6. Gray-level run length matrix features (GLRLM; quantifying gray level runs in renal mass)
  7. Neighboring gray tone difference matrix features (NGTDM; quantifying the difference between a gray value and average gray value of its neighbors in renal mass)
  8. Gray-level dependence matrix features (GLDM; quantifying gray level dependencies of renal mass).

2.4 Renal mass assessment

As criterion standard, all renal mass specimens underwent histopathological assessment at the Department of Pathology, University Medical Center Goettingen. Renal masses were categorized as malignant (including clear cell, chromophobe, and papillary RCC) and benign (including oncocytoma and angiomyolipoma [AML]). Histopathological assessment was performed on partial or radical nephrectomy specimens using hematoxylin-eosin staining, and immunostaining for cytokeratin 7, CD10, CD117, as well as Vimentin, following international recommendations.[14,15] Diagnosis of AMLs was further based on Melan-A, human melanoma black 45 and actin staining.[16,17] Representative histopathological slides of malignant and benign renal masses are presented in the appendix, https://links.lww.com/MD/E31.

All CT studies were independently assessed by 2 radiologists (with 3 and 5 years of dedicated experience in abdominal imaging) that were blinded to each other and final histopathological diagnosis. Renal masses were radiologically assessed using a Likert scale defining the probability of malignancy (POM) ranging from 1 (definitely benign renal mass) to 10 (definitely malignant) with increments of 1.

2.5 Machine learning

In a first step, preprocessing of the radiomic features was conducted with centering (subtracting the mean from individual values) and scaling (dividing values by standard deviation) of each feature.

Second, a feature selection was conducted using recursive feature elimination (RFE). For RFE, a full logistic regression model was fit with all potential predictors, ranking the importance of each predictor. At each RFE iteration, only the most important predictors were retained; the model was refitted followed by an assessment of its diagnostic accuracy. In this study, RFE was conducted using 10-fold cross-validation to avoid overfitting. For this cross-validation, the full dataset is divided into 10 subsamples of which 9 subsamples are used as training data and the remaining subset for testing. The process is repeated 10 times using each subset for validation once.

Third, machine learning algorithms were modeled to predict the probability of malignancy of a specific renal mass (outcome) given its radiomic features (predictors). Several machine learning algorithms were considered a priori according to Wolperts’ no free lunch theorem.[18] The following machine learning algorithms were trained: extreme gradient boosting (XG boost), random forest (RF), neural network, support vector machines (SVM), and k-nearest neighbors. Details for each machine learning algorithm have been published previously and are provided in a short summary in the appendix, https://links.lww.com/MD/E31. All machine learning algorithms were trained and tested using a leave-one-out cross validation: training was conducted on n-1 observations and the model performance tested on the left-out observation. This procedure was repeated n-times to obtain the final model.

2.6 Diagnostic performance assessment

Diagnostic accuracy was assessed using the receiver-operating characteristics curve (ROC) and the area under the ROC curve (AUC). AUCs of radiologists and machine learning algorithms were compared using 2000 bootstrap samples. The Youden Index was used to determine the optimal cutoff values from radiologists and machine learning algorithms for calculation of sensitivity and specificity. The McNemar test was used for comparison of sensitivity and specificity between radiologists and machine learning algorithms. Interobserver agreement was evaluated comparing renal mass assessment between both radiologists; intraobserver agreement was evaluated comparing renal mass assessment of one radiologist at 2 separate timepoints. Inter- and intraobserver agreement were quantified using the inter-/intraobserver correlation coefficient (ICC) ranging from 0–1.[19] ICC <0.4 was rated as poor, 0.4 to 0.59 as fair, 0.6 to 0.74 as good, and 0.75 to 1 as excellent. Readings of both radiologists were combined to simulate an “average radiologist” for comparison of diagnostic accuracy to the machine learning algorithms.

Machine learning algorithm implementation and statistical analyses were performed using R and RStudio [20,21] with the R package “caret.”[22] An alpha level of 0.05 was chosen to indicate statistical significance. All provided P values are 2-sided.

3 Results

3.1 Study cohort

A total of 94 patients met were included in our study (female, n = 28, 29.8%; male, n = 66, 70.2%) with median age of 64.4 years (interquartile range [IQR]: 54.9–73 years). The histopathological assessment revealed 76 malignant lesions (clear cell RCC, n = 67; papillary RCC, n = 7; chromophobe RCC, n = 2) and 18 benign lesions (oncocytoma, n = 9; AML, n = 9). A study flowchart is provided in the appendix, https://links.lww.com/MD/E31.

3.2 Radiological imaging and assessment

The median renal mass diameter was 46.5 mm (IQR: 35–56.8 mm). Radiological imaging was acquired from 18 different CT scanners (see appendix for further details, https://links.lww.com/MD/E31) with median slice thickness of 2.5 mm (IQR: 1–5 mm). Imaging artifacts were present in 15 CT studies (15.9%).

Interobserver agreement in renal mass assessment between both radiologists was fair with an ICC = 0.513. As shown in Figure 2, interobserver agreement was good for those renal masses with very low or very high average probability of malignancy. Intraobserver agreement for repeated renal mass assessment of one radiologist was fair with ICC = 0.435.

Figure 2
Figure 2:
Dotplot and Whisker-boxplot depicting the discrepancies between both radiologists’ probability of malignancy (POM) assessment of renal lesions. Notably, the radiologists’ discrepancies are smaller in cases with very low (1–2) or very high (9–10) POM.

3.3 Machine learning algorithms versus radiological assessment

Table 1 summarizes the diagnostic accuracy of different machine learning algorithms to predict renal mass malignancy. Among the machine learning algorithms, RF achieved the numerically highest AUC = 0.83.

Table 1
Table 1:
Diagnostic accuracy of machine learning algorithms and radiologist readers measured by AUC. Sensitivity and specificity obtained using the Youden Index.

As demonstrated in Figure 3, the AUC of RF (0.83) was higher when compared to the radiologists (AUC = 0.68, P = .047). According to the Youden Index, the optimal cut-off to distinguish benign and malignant lesions for the RF algorithm was 67% and for the radiologists a POM 5/10. After dichotomization, RF sensitivity (0.88) was significantly higher than the radiologists (sensitivity = 0.80, P = .045). RF specificity (0.67) was numerically higher, but did not reach statistical significance (radiologist specificity = 0.50, P = 0.083). Notably, cases of classical AMLs with macroscopic fat (n = 5) were assigned a low POM score by both radiologists. Case studies for malignancy prediction of renal masses are provided in Figures 4 and 5.

Figure 3
Figure 3:
Receiver-operating characteristic curves for random forest machine learning algorithm and radiologists. POM = probability of malignancy.
Figure 4
Figure 4:
Case study of a 38-year-old male patient presenting with heterogeneous renal mass of the right upper ventral pole (arrow; A axial plane; B 3D reformation with renal mass highlighted in green). Relevant beam-hardening artifacts are evident from dorsal instrumentation. Probability of malignancy was 99% by random forest machine learning, and rated 10/10 by radiologist 1 and 7/10 by radiologist 2. Histopathology diagnosed a clear cell renal cell carcinoma.
Figure 5
Figure 5:
Case study of a 73 year-old female presenting with heterogeneous right-sided renal mass of the lateral circumference (arrow; A axial plane; B 3D reformation with renal mass highlighted in green). Probability of malignancy was 54% by random forest machine learning (and therefore rated as “benign” using the Youden Index cutoff at 67%), and rated 6/10 radiologist 1 and 7/10 by radiologist 2 (therefore “malignant” using the Youden Index cutoff at 5/10). Diagnosis upon histopathological assessment was oncocytoma.

4 Discussion

The discrimination of malignant and benign renal masses is an ongoing radiological challenge, especially in the case of CT studies not specifically tailored to renal imaging and small renal lesions.

In our study, radiomic features and machine learning algorithms demonstrated a high diagnostic accuracy for prediction of renal mass malignancy on preoperative, venous CT. The final RF algorithm robustly performed on a heterogeneous population with CT studies acquired on a range on different scanners. Even in cases with large slice thickness, beam-hardening or motion artifacts, accurate malignancy predictions were achieved. This robustness corroborates the utility of machine learning algorithms in a real-life clinical scenario.

Among evaluated machine learning algorithms, RF demonstrated superior performance. Due to their algorithmic characteristics, RF excel in cases of high-dimensional and highly correlated data, such as the radiomic features analyzed in this study.[23]

Compared to 2 experienced radiologists, the RF algorithm demonstrated superior diagnostic accuracy for renal mass assessment. Although sensitivity and specificity were numerically higher for the RF algorithm after dichotomization using the Youden index, no statistically significant differences were evident compared to the radiologists. This could be attributed to a lack of statistical power given the limited sample size.

The observed suboptimal interobserver agreement between both radiologists underlines the need for additional diagnostic tools to reliably assess renal masses. Notably, the radiologists performed well in cases with macroscopic fat, which were accurately rated as AMLs. The overall lower diagnostic accuracy of the radiologists compared to the RF algorithm might therefore be driven by cases with fat-poor AMLs and oncocytomas, which were falsely described as malignant masses by the radiologists.

The study cohorts’ characteristics of our study are in line with population-based analyses, showing a male predominance and peak incidence between age 60 and 70.[24,25] Further, the proportion of benign renal masses with 19% in our study is comparable to the literature ranging from 20% to 30%.[5,6]

In renal imaging, machine learning studies are scarce. Recently, Feng et al used SVMs to discriminate fat-poor AMLs from RCC in 58 patients.[26] Kocak et al[27] evaluated 68 RCC cases and were able to discriminate histological subtypes with moderate accuracy. Yu et el[9] evaluated radiomics and machine learning for discrimination of histological RCC subtypes in 119 patients. Initial studies also highlighted that RCC mutational status correlated with CT imaging features and were predictable using machine learning algorithms.[28,29] Although these studies showed promising results, most of them were designed to evaluate CT studies which were acquired in a standardized and partially multiphasic manner, which might limit their applicability to a heterogeneous clinical setting. Our study, in contrast, demonstrated the feasibility and accuracy of radiomic feature analyses and machine learning algorithms even in a pragmatic scenario with diverse CT scanners and relevant artifacts.

Our RF algorithm using only venous CT imaging demonstrated similar diagnostic accuracy to previous multiphasic studies: using contrast enhancement analyses to differentiate renal mass subgroups in 200 patients with multiphasic CT, Coy et al[30] reported an AUC = 0.85, which is comparable to findings in our study (AUC = 0.83).

It still remains unclear, how exactly radiomic features discriminate benign and malignant renal masses. One hypothesis might be that radiomic features quantify neovascularity and malignant cell transformation. Another potential mechanism is the correlation of radiomic features with well described alterations in renal cancer cell metabolism, such as the metabolic flux through glycolysis, mitochondrial bioenergetics, and oxidative phosphorylation.[31–33]

Further, it has to be highlighted that radiomic feature analyses and machine learning are not the only method to discriminate between benign and malignant renal masses. In recent years, there have been several reports on the emerging role of biomarkers in renal cancer diagnosis: for example, Papale et al described RKIP/p-RKIP as an urinary biomarker for ccRCC, whereas Lucarelli et al reported on serum circulating CA 15–3, CA 125 and beta-2 microglobulin as prognostic markers in RCC.[34,35] Further studies have been published on the prognostic impact of the autocrine motility factor, soluble serum αKlotho, and kynurine pathway in ccRCC.[36–38] It remains to be evaluated whether the combination of radiomic feature analyses and biomarkers can further assist in accurate renal mass diagnosis.

Our study is not devoid of limitations: first, only patients of white race and from a restricted European region were evaluated, which may limit the generalizability of our findings. Still, there is no literature available on race-specific differences in radiologic appearance of renal mass. Second, our sample size was not large enough to allow for an independent validation dataset and results might thus be overfitted to our specific population. Specifically, the class imbalance with a small number of benign cases limits the generalizability of our findings. Third, we excluded primarily cystic and diffuse infiltrative renal masses, as well as those >70 mm in diameter. Thereby, the proposed machine learning algorithm cannot be applied to any case presenting in clinical routine. Nevertheless, diffuse infiltrative renal disease is clinically rare with primary renal lymphoma reportedly accounting for <1% of renal lesions.[39] For renal masses of >70-mm diameter, patient stratification is less important than for smaller renal masses, as even benign masses like oncocytomas might be resected due to risk of hemorrhage.[40] Using a 2-reader consensus approach in our study, the variability of renal mass segmentation and its effect on malignancy prediction were not assessible. Finally, no validation dataset was available for measuring the algorithm's performance in unknown data. To bypass this shortcoming, we used cross-validation which, though being widely accepted as an internal validation method, lags behind a completely independent, external validation dataset, which will be the subject of future studies.

Despite this study's’ apparent limitation, its innovative approach and future implementations should be highlighted: our pragmatic approach carries the potential for developing a clinically applicable software that not only supports radiologists in daily routine, but also multidisciplinary tumor board decisions. Further research should aim to validate and improve our algorithm on independent datasets. Moreover, automated renal mass segmentation has a high potential to streamline implementation of our radiomic and machine learning approach, which further lowers the threshold for clinical application.

5 Conclusions

Our study suggests that radiomic features and machine learning yield good diagnostic accuracy for discrimination of malignant and benign renal masses on CT studies, which was significantly higher than that of radiologists. Although limited by a small sample size and low number of benign renal masses, the presented RF algorithm robustly performs in a real-life scenario with nonstandardized CT studies from various referring centers. Further studies should aim to validate our findings in independent datasets. A future clinical pipeline should incorporate not only radiomic analyses and machine learning algorithms, but also automated detection and segmentation of renal lesions to streamline renal mass diagnostics.

Author contributions

Conceptualization: Johannes Uhlig, Manuel M Nietert, Tim Beißbarth, Joachim Lotz, Hyun S. Kim, Lutz Trojan, Annemarie Uhlig.

Data curation: Johannes Uhlig, Lorenz Biggemann, Manuel M Nietert, Annemarie Uhlig.

Formal analysis: Johannes Uhlig, Manuel M Nietert, Tim Beißbarth, Annemarie Uhlig.

Investigation: Johannes Uhlig, Lorenz Biggemann, Joachim Lotz, Lutz Trojan, Annemarie Uhlig.

Methodology: Johannes Uhlig, Lorenz Biggemann, Tim Beißbarth, Joachim Lotz, Hyun S. Kim, Annemarie Uhlig.

Project administration: Johannes Uhlig, Joachim Lotz, Hyun S. Kim, Lutz Trojan, Annemarie Uhlig.

Resources: Johannes Uhlig, Tim Beißbarth, Joachim Lotz, Hyun S. Kim, Lutz Trojan, Annemarie Uhlig.

Software: Joachim Lotz, Hyun S. Kim, Lutz Trojan.

Supervision: Johannes Uhlig, Manuel M Nietert, Tim Beißbarth, Joachim Lotz, Hyun S. Kim, Lutz Trojan, Annemarie Uhlig.

Validation: Johannes Uhlig, Lorenz Biggemann, Annemarie Uhlig.

Visualization: Annemarie Uhlig.

Writing – original draft: Annemarie Uhlig.

Writing – review & editing: Johannes Uhlig, Lorenz Biggemann, Manuel M Nietert, Tim Beißbarth, Joachim Lotz, Hyun S. Kim, Lutz Trojan, Annemarie Uhlig.

Johannes Uhlig orcid: 0000-0003-3557-3194.

References

[1]. Bray F, Ferlay J, Soerjomataram I, et al. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 2018;68:394–424.
[2]. Hollingsworth JM, Miller DC, Daignault S, et al. Rising incidence of small renal masses: a need to reassess treatment effect. J Natl Cancer Inst 2006;98:1331–4.
[3]. Nguyen MM, Gill IS, Ellison LM. The evolving presentation of renal carcinoma in the United States: trends from the Surveillance, Epidemiology, and End Results program. J Urol 2006;176(6 pt 1):2397–400. discussion 2400.
[4]. Kane CJ, Mallin K, Ritchey J, et al. Renal cell cancer stage migration: analysis of the National Cancer Data Base. Cancer 2008;113:78–83.
[5]. Frank I, Blute ML, Cheville JC, et al. Solid renal tumors: an analysis of pathological features related to tumor size. J Urol 2003;170(6 pt 1):2217–20.
[6]. Kutikov A, Fossett LK, Ramchandani P, et al. Incidence of benign pathologic findings at partial nephrectomy for solitary renal mass presumed to be renal cell carcinoma on preoperative imaging. Urology 2006;68:737–40.
[7]. O’Connor SD, Pickhardt PJ, Kim DH, et al. Incidental finding of renal masses at unenhanced CT: prevalence and analysis of features for guiding management. Am J Roentgenol 2011;197:139–45.
[8]. Pierorazio PM, Hyams ES, Tsai S, et al. Multiphasic enhancement patterns of small renal masses (</=4 cm) on preoperative computed tomography: utility for distinguishing subtypes of renal cell carcinoma, angiomyolipoma, and oncocytoma. Urology 2013;81:1265–71.
[9]. Yu H, Scalera J, Khalid M, et al. Texture analysis as a radiomic marker for differentiating renal tumors. Abdom Radiol (New York) 2017;42:2470–8.
[10]. Juntu J, Sijbers J, De Backer S, et al. Machine learning study of several classifiers trained with texture analysis features to differentiate benign from malignant soft-tissue tumors in T1-MRI images. J Magn Reson Imaging 2010;31:680–9.
[11]. Fedorov A, Beichel R, Kalpathy-Cramer J, et al. 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magn Reson Imaging 2012;30:1323–41.
[12]. van Griethuysen JJM, Fedorov A, Parmar C, et al. Computational radiomics system to decode the radiographic phenotype. Cancer Res 2017;77:e104–7.
[13]. Zwanenburg A, Leger S, Vallières M, et al. Image biomarker standardisation initiative—feature definitions 2016; 17–132, version 11, arXiv:1612.07003.
[14]. Trpkov K, Grignon DJ, Bonsib SM, et al. Handling and staging of renal cell carcinoma: the International Society of Urological Pathology Consensus (ISUP) conference recommendations. Am J Surg Pathol 2013;37:1505–17.
[15]. Reuter VE, Argani P, Zhou M, et al. Best practices recommendations in the application of immunohistochemistry in the kidney tumors: report from the International Society of Urologic Pathology consensus conference. Am J Surg Pathol 2014;38:e35–49.
[16]. Makhlouf HR, Ishak KG, Shekar R, et al. Melanoma markers in angiomyolipoma of the liver and kidney: a comparative study. Arch Pathol Lab Med 2002;126:49–55.
[17]. Stone CH, Lee MW, Amin MB, et al. Renal angiomyolipoma: further immunophenotypic characterization of an expanding morphologic spectrum. Arch Pathol Lab Med 2001;125:751–8.
[18]. Wolpert DH. The lack of a priori distinctions between learning algorithms. Neural Comput 1996;8:1341–90.
[19]. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychological bulletin 1979;86:420–8.
[20]. R: A language and environment for statistical computing. [computer program]. Vienna, Austria: R Foundation for Statistical Computing. Available at: https://www.r-project.org; 2008.
[21]. RStudio: Integrated Development for R [computer program]. Boston, MA, USA, available at: https://www.rstudio.com2015.
[22]. Kunh M. building predictive models in R using the caret package. Journal of Statistical Software 2008 2008;28:26.
[23]. Boulesteix A-L, Janitza S, Kruppa J, et al. Overview of random forest methodology and practical guidance with emphasis on computational biology and bioinformatics. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 2012;2:493–507.
[24]. Capitanio U, Bensalah K, Bex A, et al. Epidemiology of renal cell carcinoma. Eur Urol 2019;75:74–84.
[25]. Hidayat K, Du X, Zou SY, et al. Blood pressure and kidney cancer risk: meta-analysis of prospective studies. J Hypertens 2017;35:1333–44.
[26]. Feng Z, Rong P, Cao P, et al. Machine learning-based quantitative texture analysis of CT images of small renal masses: Differentiation of angiomyolipoma without visible fat from renal cell carcinoma. Eur Radiol 2018;28:1625–33.
[27]. Kocak B, Yardimci AH, Bektas CT, et al. Textural differences between renal cell carcinoma subtypes: Machine learning-based quantitative computed tomography texture analysis with independent external validation. Eur J Radiol 2018;107:149–57.
[28]. Kocak B, Durmaz ES, Ates E, et al. Radiogenomics in clear cell renal cell carcinoma: machine learning-based high-dimensional quantitative CT texture analysis in predicting PBRM1 mutation status. AJR Am J Roentgenol 2019;212:W55–w63.
[29]. Karlo CA, Di Paolo PL, Chaim J, et al. Radiogenomics of clear cell renal cell carcinoma: associations between CT imaging features and mutations. Radiology 2014;270:464–71.
[30]. Coy H, Young JR, Douek ML, et al. Quantitative computer-aided diagnostic algorithm for automated detection of peak lesion attenuation in differentiating clear cell from papillary and chromophobe renal cell carcinoma, oncocytoma, and fat-poor angiomyolipoma on multiphasic multidetector computed tomography. Abdom Radiol (New York) 2017;42:1919–28.
[31]. Lucarelli G, Rutigliano M, Sallustio F, et al. Integrated multi-omics characterization reveals a distinctive metabolic signature and the role of NDUFA4L2 in promoting angiogenesis, chemoresistance, and mitochondrial dysfunction in clear cell renal cell carcinoma. Aging (Albany NY) 2018;10:3957–85.
[32]. Lucarelli G, Loizzo D, Franzin R, et al. Metabolomic insights into pathophysiological mechanisms and biomarker discovery in clear cell renal cell carcinoma. Expert Rev Mol Diagn 2019;19:397–407.
[33]. Bianchi C, Meregalli C, Bombelli S, et al. The glucose and lipid metabolism reprogramming is grade-dependent in clear cell renal cell carcinoma primary cultures and is targetable to modulate cell viability and proliferation. Oncotarget 2017;8:113502–15.
[34]. Papale M, Vocino G, Lucarelli G, et al. Urinary RKIP/p-RKIP is a potential diagnostic and prognostic marker of clear cell renal cell carcinoma. Oncotarget 2017;8:40412–24.
[35]. Lucarelli G, Ditonno P, Bettocchi C, et al. Diagnostic and prognostic role of preoperative circulating CA 15-3, CA 125, and beta-2 microglobulin in renal cell carcinoma. Dis Markers 2014;2014:689795.
[36]. Lucarelli G, Rutigliano M, Sanguedolce F, et al. Increased expression of the autocrine motility factor is associated with poor prognosis in patients with clear cell-renal cell carcinoma. Medicine (Baltimore) 2015;94:e2117.
[37]. Gigante M, Lucarelli G, Divella C, et al. Soluble serum alphaKlotho is a potential predictive marker of disease progression in clear cell renal cell carcinoma. Medicine (Baltimore) 2015;94:e1917.
[38]. Lucarelli G, Rutigliano M, Ferro M, et al. Activation of the kynurenine pathway predicts poor outcome in patients with clear cell renal cell carcinoma. Urol Oncol 2017;35:461e415-461.e427.
[39]. Stallone G, Infante B, Manno C, et al. Primary renal lymphoma does exist: case report and review of the literature. J Nephrol 2000;13:367–72.
[40]. Kuusk T, Biancari F, Lane B, et al. Treatment of renal angiomyolipoma: pooled analysis of individual patient data. BMC Urol 2015;15:123–123.
Keywords:

carcinoma; computer-assisted; image interpretation; machine learning; multidetector computed tomography; renal cell

Supplemental Digital Content

Copyright © 2020 the Author(s). Published by Wolters Kluwer Health, Inc.