Secondary Logo

Journal Logo

Thoracic Imaging

Computed Tomography–Based Radiomic Features for Diagnosis of Indeterminate Small Pulmonary Nodules

Liu, Qin MD; Huang, Yan MD; Chen, Huai MD; Liu, Yanwen MD; Liang, Ruihong MD; Zeng, Qingsi MD, PhD

Author Information
Journal of Computer Assisted Tomography: January/February 2020 - Volume 44 - Issue 1 - p 90-94
doi: 10.1097/RCT.0000000000000976


According to recent cancer statistics, the most common cause of cancer death worldwide is still cancer of the lung and bronchus.1–3 Individuals diagnosed as having lung cancer generally have a poor prognosis, as most patients are diagnosed at stage III or IV because of an absence of discriminating symptoms at early stages of the disease.4 Early precision diagnosis and treatment of lung cancer are challenging clinical problems.

The detection of small pulmonary nodules (SPNs; ≤10 mm) is increasing largely due to the common use of computed tomography (CT). The rate of noncalcified SPN is 23%, as reported by Patz EF et al5 in the Early Lung Cancer Action Project. Hazelrigg et al6 found an even higher rate (39.5%) in patients undergoing pulmonary reduction surgery. The international guidelines for the management of SPN recommend that every indeterminate nodule should be followed with serial CT through at least 2 years for solid nodules and 3 years for ground-glass nodules.7,8 Therefore, early accurate differential diagnosis of SPN can reduce health care costs and the excessive CT scans with little benefit. An advanced noninvasive diagnostic tool is warranted to provide guidance for the management of indeterminate SPN.

Radiomics” is an advanced noninvasive diagnostic method that involves the high-throughput extraction of quantitative and objective imaging features from medical images.9,10 These image features parameters are mined with sophisticated bioinformatics tools to develop models that may potentially improve diagnostic, prognostic, and predictive accuracy.11 For example, radiomics had been used to investigate tumor phenotypes, histological subtypes of lung adenocarcinoma, and the pathological response of non–small cell lung cancer.12–14 However, to date, there is no study that has evaluated the classification performance of indeterminate SPN in a radiomics approach. Thus, the aim of this study is to investigate the diagnostic ability of radiomic signature based on preoperative contrast-enhanced CT for SPN with size 10 mm.


Patients and Pulmonary Nodules

This retrospective study was approved by our institutional review board, and the requirement for informed consent was waived. Between January 2011 and March 2017, a total of 210 SPNs (maximum diameter ≤10 mm) were collected from 197 patients who had underwent surgical resections. The inclusion criteria of this study were as follows (at the level of nodules): (1) greatest nodule diameter of 10 mm (n = 1000), (2) pathologically confirmed benign or malignant SPN (n = 453), (3) availability of clinical characteristics (n = 423), (4) with complete baseline CT data sets and medical history at the authors' institution (n = 356), and (5) a CT scanner with same imaging parameters and reconstruction slice thickness (n = 210). The exclusion criteria were as follows: (1) with pretreatment before the CT examination and (2) have imaging artifacts in the CT images. The 210 SPNs were randomly divided into the training cohort (n = 140) and validation cohort (n = 70) at a ratio of 2:1.

Contrast-Enhanced CT Images Acquisition

All patients underwent contrast-enhanced CT examination with a 64-detector-row CT scanner (Siemens Definition AS + 128, Forchheim, Germany). Contrast-enhanced CT image acquisitions were as follows: intravenous administration of iodinated contrast material (1–1.1 mL/kg: Ultravist 370; Bayer Pharma AG, Berlin, Germany) at a rate of 3.5 mL/s with a pump injector (Mallinckrodt OptiVantage DH; Tyco Healthcare, Cincinnati, Ohio), and the images were obtained after a 30-second delay. Image parameters were as follows: rotation time, 0.5 second; detector collimation, 64 × 0.625 mm; field of view, 500 mm; 120 kV; 120 mA; and matrix, 512 × 512. All image data were reconstructed with a slice thickness of 2 mm.

Radiomic Feature Extraction

All contrast-enhanced CT images were retrieved from our institution's picture archiving and communication system (Neusoft). An in-house feature extraction algorithm was applied in A.K. (Artificial Intelligence Kit) software developed by GE Healthcare. A total of 385 radiomic features including (a) histogram features, (b) form factor features, and (c) textural features (such as gray-level co-occurrence matrix, gray-level run length matrix, and gray-level size zone matrix) were extracted from contrast-enhanced CT images (Fig. 1). A series of primary feature extraction algorithms and descriptions were presented in the Supplementary Information ( For each SPN, a volume of interest (VOI) was generated by delineating regions of interest of the nodule margins on each slice of the CT images with ITK-SNAP software ( All manual delineations were constructed by a radiologist with 5 years' experience in lung cancer, and then validated by a senior radiologist with 20 years' experience. Figure 2 demonstrates the region-of-interest delineations for malignant and benign nodules, respectively.

Radiomic features used in this study including histogram, form factor, gray-level co-occurrence matrix (GLCM), and gray level run-length matrix (GLRLM) features. Figure 1 can be viewed online in color at
Two examples of patients with benign tumor (hamartoma, A–D) and malignant tumor (minimally invasive adenocarcinoma, E–H), respectively. Volumes of interest were generated from 4 regions of interest of thin-slice CT images. Figure 2 can be viewed online in color at

Feature Selection and Prediction Model Building

Feature selection and reduction of parameter redundancy were performed using the Kruskal-Wallis test, Spearman rank correlation coefficient, and principal component analysis (PCA). Kruskal-Wallis test was firstly used to select features that were statistically associated with the outcome of interest. Then, we used the Spearman rank correlation coefficient test to exclude the features with a correlation greater than 0.9. Finally, PCA was used to further reduce the feature vector dimensions and to increase the discriminative capability, which is described in Supplementary Information ( Those principal components that sufficiently accounted for 99.5% of the significant feature subset variability were selected for further modeling. After feature selection, we applied random forest (RF) machine learning algorithm to build prediction model. Random forest makes a decision by generating multiple random subsets and calculating the weighted average number, which is suitable for radiomic features classification. To obtain robust performance of model, we applied the bootstrap method with 1000 randomly selected samples. Then, we used training cohort to train the RF model, in which 501 decision trees with the same weights were set up. We also calculated and compared the relative importance (mean decrease in Gini index) of the features within the RF model.

Classification Performance of Radiomic Features

The diagnostic performance of the radiomic model was evaluated using the area under the curve (AUC), sensitivity, specificity, and accuracy in the training cohort and validation cohort. The AUC value can range from 0.5 to 1, which indicates no discriminative ability and perfect ability to distinguish between benign and malignant SPN, respectively.

Statistical Analysis

Statistical analysis was performed using R statistical and computing software version 3.2.1 (; R Foundation for Statistical Computing, Vienna, Austria). Differences in age and sex of patients between the benign and malignant groups were compared by using the 2-sample t test and χ2 test, respectively. Statistical significance of each influential feature for discrimination between benign and malignant SPN was analyzed using the Kruskal-Wallis test. The correlations between the features were compared using Spearman correlation analysis by conducting a correlation matrix. Principal component analysis was conducted by using “psych” package. The RF modeling was performed using the RF package. A 2-tailed P < 0.05 was considered statistically significant.


Clinical Characteristics

A total of 197 patients with 210 SPN underwent total resections. The clinical characteristics of patients are shown in Table 1. In the benign SPN group, 51.9% (41/79) were men and 48.1% (38/79) were women, with a mean age of 50.24 years (range, 29–72 years). In the malignant SPN group, 64.1% (84/131) were men and 35.9% (47/131) were women, with a mean age of 51.52 years (range, 17–75 years). There were no significant difference in age and sex between the benign and malignant groups (P = 0.422 and P = 0.080, respectively). Among the nodules, 79 (37.6%) were classified as benign lesions, including tuberculomas (n = 15), fibrous nodules (n = 13), lymph nodes (n = 11), hamartomas (n = 13), pulmonary crytococcosis (n = 10), inflammatory nodules (n = 8), inflammatory granuloma (n = 4), aspergillosis (n = 3), pulmonary sclerosing hemangiomas (n = 1), and smooth muscle (n = 1), and 131 (62.5%) were malignant lesions, consisting of invasive adenocarcinomas (n = 44), minimally invasive adenocarcinoma (n = 58), adenocarcinoma in situ (n = 20), and atypical adenomatous hyperplasia (n = 9).

Clinical Characteristics of Patients

Feature Selection and Prediction Model Building

A total of 385 radiomic features were extracted from the VOI of contrast-enhanced CT images. First, we selected 302 features that were statistically associated with the outcome by using the Kruskal-Wallis test. Subsequently, we used Spearman rank correlation analysis to exclude the features with a correlation greater than 0.9. As a result, 65 radiomic features were remained, which are most strongly associated with the SPN classification. Finally, 15 principal component features were proven to have robust classification power after PCA. Hence, RF model was built based on the 15 principal component features. Figure 3 shows the mean decrease in Gini index or relative importance of the 15 features, of which, feature 2 was the most important feature, and features 3, 6, 9, 10, 11, and 15 were considered equivocal in differentiating benign and malignant nodules.

The receiver operating characteristic curve of the radiomic signature for the classification of benign and malignant SPN in both training cohort (A) and validation cohort (B). The gray areas indicate the 95% CIs. Figure 3 can be viewed online in color at

Classification Performance of Radiomic Model

The AUC, sensitivity, specificity, and accuracy of RF model in the training cohort were 0.957 (95% confidence interval [CI], 0.923–0.992), 93.4% (95% CI, 87.2%–94.9%), 91.9% (86.3%–97.1%), and 92.9% (95% CI, 88.1%–94.3%), respectively (Fig. 4). In the validation cohort, RF model obtained AUCs of 0.877 (95% CI, 0.795–0.959), 81.8% (95% CI, 72.0%–90.9%), and 77.4% (95% CI, 63.9%–89.3%) and an accuracy of 80.0% (95% CI, 72.0%–86.7%; Fig. 4).

Bar plot represents the mean decrease in Gini index (relative importance) of the 15 principal component features. Figure 4 can be viewed online in color at


We developed and validated a CT-based radiomic model for individualized preoperative prediction of SPN, which showed favorable discrimination and calibration. These results demonstrate that radiomic features extracted from CT images can be applied as a noninvasive prediction tool to identify patients with a high risk of malignant SPN.

For most cancer types, a delay in diagnosis at a more advanced stage is a negative factor in cancer prognosis, whereas diagnosis at an earlier stage usually has better survival.15 For example, the overall 5-year relative survival rate for patients with female breast cancer has significantly improved from 74.8% in 1975 to 1977 to 90.3% in 2003 to 2009. This increase is largely due to earlier diagnosis as a result of the widespread use of mammography for breast cancer screening.16 Currently, SPN diagnosis is still a common clinical problem. Early accurate differential diagnosis of SPN (≤10 mm) is crucially important for detection and therapy of early lung cancer, which, if possible, provides the potential to improve lung cancer survival. The risk of malignancy ranged from 6% to 28% in pulmonary nodules that measured between 5 and 10 mm, whereas the prevalence of malignancy in pulmonary nodules that measured <5 mm was very low (range, 0%–1%).17 In this study, the malignant lesions accounted for 62.5%, which might be due to the choice of SPN clinically and radiologically suspected to be lung cancer.

The diagnosis of pulmonary nodules largely relies on the qualitative findings of CT images, such as size, spiculation, location, and pleural indentation.18 Computed tomographic screening for pulmonary nodules has the potential to reduce mortality; however, it is also likely to detect indolent tumors with resulting overdiagnosis.5 To provide an early intervention opportunity for cure, some radiologists recommended continuous follow-up for all indeterminate SPNs that are detected as an incidental finding on a CT.7 The overdiagnosis of indeterminate SPN may cause potential morbidity and mortality, poor utilization of limited resources, increased health care costs, unnecessary patient anxiety, loss of credibility for radiologists, and increased radiation burden for the affected population.7 In addition, because of the peripheral location of many pulmonary nodules, CT-guided percutaneous biopsy tends to be a more common approach to determine tumor histology. In clinical practice, the sensitivity of percutaneous biopsy was reduced in progressively smaller nodules,19,20 and its accuracy was also affected by nodule morphology and longer needle path.18 Moreover, percutaneous biopsy is clinically limited by its invasiveness and its high risk of complications.21 Thus, noninvasive tools are warranted to classify indeterminate SPN.

Recently, the great development of medical image analysis has driven increasing amounts of studies to investigate the radiomics of lung cancer. It has been proven that radiomic signatures are beneficial to decode lung cancer bioinformatics based on medical imaging, such as tumor development,22 gene expression patterns,23 therapy response,24,25 and patient survival.26,27 Moreover, whether radiomics could improve the differential diagnosis of pulmonary nodules has been investigated against conventional visual inspection on CT,28,29 but most nodules were smaller than 30 mm in diameter. In this study, we used radiomic approach to diagnose SPN with a size less than 10 mm and obtained promising results. The results demonstrated that the radiomic signature provides an advanced, noninvasive, and cost-effective diagnostic method for indeterminate SPN.

A large amount of radiomic features has the potential ability to quantifying intratumoral heterogeneity. However, the redundancy among the high-dimensional features always exists, which results in poor classification performance. In this study, a total of 385 features were extracted from VOIs on contrast-enhanced CT images. After feature selection by multiple methods, only 15 radiomic features who had robust predictive power were eligible for modeling. In addition, prediction models with high accuracy, reliability, and efficiency are thought to be crucial toward stable and clinically relevant radiomic biomarkers to provide a noninvasive way of quantifying and monitoring tumor heterogeneity in clinical practice. Some previous studies reported that RF algorithm had the highest prognostic performance when compared with other machine-learning classifiers.30,31 Therefore, we chose RF method to build prediction model, and it indeed achieved high performance in the training and validation cohorts.

This study has some limitations. First, most malignant nodules were adenocarcinomas because adenocarcinomas are the predominant histological subtype of lung cancer. Second, this radiomic study was performed in a single center, and images were obtained using the same scanner and the same imaging protocol; external validation in other cohorts is warranted to confirm the performance of the radiomic model.

In conclusion, this study demonstrates the feasibility of applying radiomic features extracted from CT images for preoperative prediction of SPN status. This study provides clinicians a valuable model as a noninvasive tool for individualized preoperative prediction of SPN to inform the instruction of long-term follow-up and selective surgical management. Further external validation is warranted to determine the performance of the model before implementing it in clinical practice.


1. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2017. CA Cancer J Clin. 2017;67:7–30.
2. Torre LA, Bray F, Siegel RL, et al. Global cancer statistics, 2012. CA Cancer J Clin. 2015;65:87–108.
3. Chen W, Zheng R, Baade PD, et al. Cancer statistics in China, 2015. CA Cancer J Clin. 2016;66:115–132.
4. Ruparel M, Quaife SL, Navani N, et al. Pulmonary nodules and CT screening: the past, present and future. Thorax. 2016;71:367–375.
5. Patz EF, Pinsky P, Gatsonis C, et al; NLST Overdiagnosis Manuscript Writing Team. Overdiagnosis in low-dose computed tomography screening for lung cancer. JAMA Intern Med. 2014;174:269–274.
6. Hazelrigg SR, Boley TM, Weber D, et al. Incidence of lung nodules found in patients undergoing lung volume reduction. Ann Thorac Surg. 1997;64:303–306.
7. MacMahon H, Austin JH, Gamsu G, et al. Guidelines for management of small pulmonary nodules detected on CT scans: a statement from the Fleischner Society. Radiology. 2005;237:395–400.
8. Sayyouh M, Vummidi DR, Kazerooni EA. Evaluation and management of pulmonary nodules: state-of-the-art and future perspectives. Expert Opin Med Diagn. 2013;7:629–644.
9. Lambin P, Rios-Velazquez E, Leijenaar R, et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer. 2012;48:441–446.
10. Kumar V, Gu Y, Basu S, et al. Radiomics: the process and the challenges. Magn Reson Imaging. 2012;30:1234–1248.
11. Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data. Radiology. 2016;278:563–577.
12. Choi ER, Lee HY, Jeong JY, et al. Quantitative image variables reflect the intratumoral pathologic heterogeneity of lung adenocarcinoma. Oncotarget. 2016;7:67302–67313.
13. Yoon HJ, Sohn I, Cho JH, et al. Decoding tumor phenotypes for ALK, ROS1, and RET fusions in lung adenocarcinoma using a radiomics approach. Medicine. 2015;94:e1753.
14. Coroller TP, Agrawal V, Narayan V, et al. Radiomic phenotype features predict pathological response in non–small cell lung cancer. Radiother Oncol. 2016;119:480–486.
15. Ott JJ, Ullrich A, Miller AB. The importance of early symptom recognition in the context of early detection and cancer survival. Eur J Cancer. 2009;45:2743–2748.
16. DeSantis CE, Lin CC, Mariotto AB, et al. Cancer treatment and survivorship statistics, 2014. CA Cancer J Clin. 2014;64:252–271.
17. Wahidi MM, Govert JA, Goudar RK, et al. Evidence for the treatment of patients with pulmonary nodules: when is it lung cancer? ACCP evidence-based clinical practice guidelines (2nd edition). Chest. 2007;132:94S–107S.
18. Callister ME, Baldwin DR. How should pulmonary nodules be optimally investigated and managed? Lung Cancer. 2016;91:48–55.
19. Kothary N, Lock L, Sze DY, et al. Computed tomography–guided percutaneous needle biopsy of pulmonary nodules: impact of nodule size on diagnostic accuracy. Clin Lung Cancer. 2009;10:360–363.
20. Wallace MJ, Krishnamurthy S, Broemeling LD, et al. CT-guided percutaneous fine-needle aspiration biopsy of small (< or =1cm) pulmonary lesions. Radiology. 2002;225:823–828.
21. Wiener RS, Schwartz LM, Woloshin S, et al. Population-based risk of complications following transthoracic needle lung biopsy of a pulmonary nodule. Ann Inern Med. 2011;155:137–144.
22. Hawkins S, Wang H, Liu Y, et al. Predicting malignant nodules from screening CT scans. J Thorac Oncol. 2016;11:2120–2128.
23. Aerts HJ, Velazquez ER, Leijenaar RT, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun. 2014;5:4006.
24. Aerts HJ, Grossmann P, Tan Y, et al. Defining a Radiomic response phenotype: a pilot study using targeted therapy in NSCLC. Sci Rep. 2016;6:33860.
25. Huynh E, Coroller TP, Narayan V, et al. CT-based radiomic analysis of stereotactic body radiation therapy patients with lung cancer. Radiother Oncol. 2016;120:258–266.
26. Grove O, Berglund AE, Schabath MB, et al. Quantitative computed tomographic descriptors associate tumor shape complexity and intratumor heterogeneity with prognosis in lung adenocarcinoma. PLoS One. 2015;10:e0118261.
27. Huang Y, Liu Z, He L, et al. Radiomics signature: a potential biomarker for the prediction of disease-free survival in early-stage (I or II) non–small cell lung cancer. Radiology. 2016;281:947–957.
28. He L, Huang Y, Ma Z, et al. Effects of contrast-enhancement, reconstruction slice thickness and convolution kernel on the diagnostic performance of radiomics signature in solitary pulmonary nodule. Sci Rep. 2016;6:34921.
29. Bae JM, Jeong JY, Lee HY, et al. Pathologic stratification of operable lung adenocarcinoma using radiomics features extracted from dual energy CT images. Oncotarget. 2017;8:523–535.
30. Parmar C, Grossmann P, Bussink J, et al. Machine learning methods for quantitative radiomic biomarkers. Sci Rep. 2015;5:13087.
31. Zhang B, He X, Ouyang FS, et al. Radiomic machine-learning classifiers for prognostic biomarkers of advanced nasopharyngeal carcinoma. Cancer Lett. 2017;403:21–27.

pulmonary nodules; computed tomography; radiomics; diagnosis

Supplemental Digital Content

Copyright © 2020 Wolters Kluwer Health, Inc. All rights reserved.