Colorectal cancer (CRC) is one of the most common malignant tumors, with the third highest incidence among cancers, while its mortality is the fourth highest.1 2 3 4 5
Magnetic resonance imaging scans have high resolution and are regarded as the best examination method for evaluating whether invasion is present in CRM.6 7 8 9 10 11–15 16
In our study, we explored the feasibility of using Faster R-CNN to perform automatic computer recognition of positive CRM regions in high-resolution MRIs of rectal cancer that are used to assist imaging diagnosis.
METHODS
General Information and Establishment of the High-Resolution MRI Database
We retrospectively studied 318 patients with rectal cancer from among more than 2000 treated patients in the Affiliated Hospital of Qingdao University from July 2016 to August 2018, who were determined to be CRM positive and who underwent pretreatment high-resolution MRI. After excluding cases with substantially degraded images, such as cases after neoadjuvant therapy, cases of recurrent cancer after surgery, cases with poor-quality images, and those with artifacts and extensive necrosis, 240 patients were included in this study. Among these, 162 patients were confirmed to be CRM positive by postoperative pathology, and the remaining 78 patients were treated with neoadjuvant therapy. In the 78 patients who subsequently received neoadjuvant therapy, 18 were confirmed to have developed liver and lung metastases, and 46 had T4 lesions. Among the 162 patients with postoperative pathological results, T3 and T4 lesions occurred in 147 and 15 patients. According to sex and tumor position, the patients were assigned to either the training group (192 cases) or the validation group (48 cases) at a ratio of 8:2. The detailed patient and tumor characteristics are listed in Table 1 .
TABLE 1.: Patient and tumor characteristics on MRI images in the training and validation groups
This study was approved by the ethics committee of the Medical Department of Qingdao University. All the methods were conducted following the relevant guidelines and regulations and registered with the Chinese Clinical Trial Registry (registration number: ChiCTR-1800017410).
Parameters of High-Resolution MRI and the Scanning Process
GE Signa 3.0T superconducting MR scanner and 16-channel phased-array surface coils were used for scanning. The primary scanning parameters and sequences are shown in Table 2 . Only those T2-weighted imaging (T2WI) and diffusion-weighted imaging scans with the highest diagnostic value were extracted.
TABLE 2.: Sequence parameters of the GE Signa 3.0T HDX MR scanner
Marking of the Positive CRM Images by Radiologists
Two senior imaging experts read and analyzed the images of each patient to independently diagnose each image and identify the CRM areas most likely to be invaded by cancer in the related images. When the experts’ diagnoses were not consistent, consensus was reached through discussion by referring to relevant examination reports such as those of pathology and colonoscopy.
The operational principles for the specific identification process are as follows (see Fig. 1). The CRM-positive region was marked by combining images of 3 planes. The shortest distance from the low or middle tumors to the levator ani was determined by the T2WI coronal view. The distance from the tumor to the anal verge or the levator ani plane was determined by the T2WI sagittal view, which was used to differentiate among the low, middle, and high positions within the rectum. The axial T2WI was vertical to the intestinal wall and accurately displayed the transverse plane of the intestine and local tumor infiltration and metastasis, which was the primary plane for evaluating CRM. Locating a tumor by using 3 planes can avoid misjudgments caused by a distorted orientation of the intestine. Diffusion-weighted imaging can be used to maximally distinguish between inflammatory reactions, fibrosis around the intestine, and tumor invasion, and can further verify tumor invasion and metastasis. The yellow square was used to identify the CRM-positive area, and the intersection point of the cancer invading the mesorectal fascia of the rectum was used as the marker. The identified area covered approximately half of the tumor tissue and half of the adjacent tissue; the dividing line was the mesorectal fascia by default.
FIGURE 1.: Image identification process description. Each column shows a high-resolution MRI from the same patient, determined by the sagittal and axial positions in the first column and by the coronal and axial positions in the second column. The white arrow indicates where the circumferential resection margin was violated, and the yellow box defines the segmentation area.
Using tumors located between the plane of the levator ani and anal canal and the reflection of the peritoneum as marking targets, a total of 1220 axial T2WI high-resolution MRI images with the best logo were selected. Among the identified images, 1151 were caused by primary tumor invasion; there were 57 images of lymph node metastasis and 12 images of extramural vascular invasion. In 162 patients with pathological results, 144 were caused by primary tumor invasion, 7 involved lymph node metastases, and 11 were caused by a mixture of these 2 factors. The selected images were categorized into a training group (1020 images) and a validation group (200 images) according to the previous patient grouping.
During model training and validation, the imaging department and the Faster R-CNN platform not only needed to recognize labeled images, but were also required to learn continuous normal images from the same patients, in such a way that its ability to distinguish and memorize could be trained. The continuous images we selected for CRM+/CRM– are from the same group of patients, and the ratio of approximately 1:2 constitutes a positive sample and a negative sample data set, including 3430 training images (including 1020 positive and 2410 negative images) and 600 validation images (including 200 positive and 400 negative images).
Faster R-CNN Method and Procedure
In this study, establishing an artificial intelligence-aided imaging diagnosis system of positive CRM in rectal cancer involves 2 procedures: training and validation.
Because of the relatively small scale of the selected training data set, we fine-tuned the images to achieve an additional data augmentation by flipping along the axial and coronal plane and rotation in different angles with 30°, 60°, and 90°. In application data augmentation technology, the network has to capture the higher semantic features, avoiding being stuck in simple low-level density features. Finally, 11,658 images (including 3626 positive and 8032 negative images) were placed in the Faster R-CNN for training. On the basis of the previous training experience (pretrained networks) and the migration learning model, the Faster R-CNN system applied in this article has completed the study of normal anatomical MRI images of pelvic cavity. After the validation set and training set data are finally added, the entire database will contain 12,258 rectal T2WI high-resolution MRI images.
Faster R-CNN Principles and Training Processes
The automatic detection of the invasion of CRM in rectal cancer images using a Faster R-CNN was studied (see Supplemental files for detailed methods, https://links.lww.com/DCR/B89 Fig. 2 for the detailed network architecture). In this experiment, RPN and Fast R-CNN were alternately trained in 2 stages, and the parameters were fine-tuned during the iterations.
FIGURE 2.: The network architecture of the region proposal network (RPN) and Faster R-CNN. Faster R-CNN improves Fast R-CNN by the addition of a RPN. The CNN network includes 2 phases: feature representation and score map reconstruction. In the feature representation phase, the network consists of 4 Pool-Conv-Relu layer blocks and 9 Convo-Relu blocks. Pool = pooling; conv = convolution; convo = convolution; relu= rectified linear unit; cls = classification; bbox pred= bounding box prediction; ROI = region of interest.
The images of positive CRM for training in the database were input into the Faster R-CNN. The 4-step iterative training of the Faster R-CNN was performed 80,000 times; the training parameters are listed in Table 3 . We then exported the Faster R-CNN detector, the RPN network, and the loss-of-function values of the entire network obtained during the training process. The results show that the Faster R-CNN network obtained good attenuation after 240,000 training iterations (see Supplemental Figure 1, https://links.lww.com/DCR/B89
TABLE 3.: Training parameters of the Region-based Convolutional Neural Network
Faster R-CNN Database Validation Experiment
A selection of 600 axial T2WI images from 48 patients in the validation group, including 200 images that were considered positive CRM by radiologists, were used as the experimental data. The times (in seconds) used for both identification processes was recorded. The images were input into the trained Faster R-CNN model. First, a convolutional feature image was generated by using the feature extraction network. The RPN was then used to screen the feature images to generate possible positive CRM regions. Finally, the convolutional feature images and their derived regions were further regressed and classified by using local feature vectors, and the position and probability of positive CRM were generated.
Statistical Analysis
The data were analyzed using SPSS 20.0 software (SPSS, Chicago, IL). All categorical data are presented as numbers of cases and percentages; continuous data are shown as either the mean ± SD (range) or as the median and interquartile range, depending on the data distribution. We used χ2 and Fisher exact tests for the categorical variables, and the Mann-Whitney U and Student t tests for the continuous variables. p values ≤0.05 were considered statistically significant. At the same time, the sensitivity, false positivity, and precision of the diagnoses were analyzed, and receiver operating characteristic (ROC) curves were generated. The areas under the curve of the ROC (AUC) were calculated using the trapezoidal rule.
RESULTS
Evaluation of the Training Effects of the Artificial Intelligence Platform
To assess the learning effects of the Faster R-CNN deep neural network, we input the test data into the trained Faster R-CNN. To reveal details related to how the machine accomplished its learning during the training process, we plotted the precision recall curve by recording the precision and recall rates. The AUC was 0.367 or average precision = 0.367(Fig. 3 ), which indicated that the training of the Faster R-CNN has a good effect. Therefore, we concluded that the Faster R-CNN had been effectively trained for images of positive CRM in rectal cancer.
FIGURE 3.: Mean average precision of recognition during Faster R-CNN training. MAP = average precision of the main set.
The recognition of the test data by the Faster R-CNN model was compared with the marked images in the database, and the mean average precision score of recognition in the Faster R-CNN training process was calculated. The average precision of the main set (MAP) is calculated as the average of the mean average precision of each topic. In this study, the MAP reflects the recognition precision of the Faster R-CNN; the closer the MAP is to 100%, the more accurate the model is.
Clinical Validation of the Diagnosis of Positive CRM in Rectal Cancer by the Artificial Intelligence System
In an image verified by the artificial intelligence (AI) platform, there are multiple identified regions with scores between 0 and 1. The larger the value, the larger the positive may be. According to the regulations, 0.7 is used as the demarcation point between the positive and negative areas. Compared with the positive CRM images identified by the radiology experts, the accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of the CRM status determined by the trained Faster R-CNN AI approach were 0.932, 0.838, 0.956, 0.825, and 0.959. We categorized all the labeled areas in the test set into true-positive or false-positive categories and obtained the true-positive rate and false-positive rate under different probability thresholds. The rates were used to plot the ROC curves, as shown in Figure 4 . The AUC was calculated to reflect the precision of the data recognition in the test set. Using the ROC curve shown in Figure 3 as a major aspect of analysis, the AUC for the Faster R-CNN AI-aided diagnosis was calculated using the trapezoidal rule; the value of AUC was 0.953. In this test, the time to automatically recognize an image was 0.2 seconds.
FIGURE 4.: The receiving operator characteristics curve of Faster R-CNN in diagnosing positive circumferential resection margin. AUC = areas under the curve.
Test Results
The results of the position and probability of the positive CRM are shown in Figure 5 . The Faster R-CNN test set contained 184 images of positive CRM with recognition probabilities above 0.7, and there were 175 images that had an overlap rate above 0.7 with the radiologist-marked areas. These identified images were all caused by primary tumor invasion and did not accurately relate to other types.
FIGURE 5.: Annotation samples of the positive circumferential resection margin. The left side shows the results marked by the computer, whereas the right side shows the results marked by the radiology experts.
DISCUSSION
Necessity of MRI for the Assessment of Rectal CRM
Positive CRM is an independent risk factor for local recurrence and has a low survival rate. The second edition of the NCCN guidelines published in 2017 recommended that accurate clinical staging is needed to develop a therapeutic plan for patients before treating rectal cancer. Based on the European Society for Medical Oncology Clinical Practice Guidelines for rectal cancer, CRM is defined as involved if it is ≤1 mm from the mesorectal fascia, resulting in increased risks of local recurrence, distant metastases, and poorer survival. In the era of TME, most guidelines recommend the use of preoperative or postoperative long-course chemoradiotherapy for patients who have rectal cancer with stage II or III disease.17 18 19 18
Feasibility of Faster R-CNN
Faster R-CNN combined with a RPN greatly reduced repetitive calculations and achieved fast real-time target recognition.20 , 21 22 Fig. 2 ), the low-level features were combined into high-level features through the convolutions. Using repeated algorithm iterations, we significantly enhanced the correlative characteristics of the targeted area and gradually suppressed the irrelevant features. Artificial intelligence approaches may be able to identify more relevant features using the VGG16 model, not only the “sandwich” type, to extract the most useful features and achieve better segmentation results. This technique makes it possible to obtain high-precision images using only a small number of the images in an image data set, while still achieving accurate automatic segmentation of the positive CRM. The advantages of Faster R-CNN are as follows: 1) The accuracy and efficiency of Faster R-CNN have been demonstrated for many solid tumors. 2) The Faster R-CNN model can automatically identify images with real-time input-output, which is more feasible. 3) This platform is reliable in terms of both its consistency and replicability, and it can gradually learn to improve through feedback. With the advancements in MRI technology, including multidimensional and multiparametric imaging, a computer-aided system can assist radiologists in making a correct and timely diagnosis by considering all the various images and factors in a much shorter time frame.
Based on previous research experiences and the advantages of using a Faster R-CNN for recognition, after training, the AI-aided diagnosis platform accurately identified CRM-positive images caused by the primary tumors. The visual areas identified by machine learning were evaluated based on expert discriminations of the images in the test set and their identifications of CRM-positive areas. Statistical evaluation of validation data shows that AI-aided diagnosis achieves higher accuracy, sensitivity, specificity, and improved ROC curve. The AUC of the ROC for AI platform-aided imaging diagnosis was 0.953, which suggests that the diagnostic ability of the trained Faster R-CNN AI approach is equivalent to the diagnostic abilities of radiology experts. Based on the current recognition ability of our Faster R-CNN model, it takes approximately 0.2 seconds to automatically recognize an image. Thus, the AI platform has the advantage of high efficiency and feasibility in identifying positive CRM caused by pretreatment tumor invasion.
This study shows that it is possible to make a significant breakthrough in conducting comprehensive study of locally invasive rectal cancer. However, this is a single-center retrospective study. Due to insufficient data, the classification accuracy could not be validly assessed, such as its ability to identify lymph nodes or extramural vascular invasion. Consequently, in future work, it is necessary to increase the number of cases and strictly select cases with high-quality TME surgery and pathological specimens to improve the auxiliary diagnostic platform. In future studies, we plan to improve our CNN architecture by, for example, conducting training directly with 3-dimensional images and integrating dynamic scanning information.
CONCLUSIONS
In high-resolution MRI images of colorectal cancer acquired before treatment, the application of a Faster R-CNN model to segment the positive CRM has high accuracy and feasibility.
ACKNOWLEDGMENTS
We thank all the participants and the doctors who contributed to this project.
REFERENCES
1. Siegel RL, Miller KD, Fedewa SA, et al. Colorectal cancer statistics, 2017. CA Cancer J Clin. 2017;67:177–193.
2. Chen W, Zheng R, Baade PD, et al. Cancer statistics in China, 2015. CA Cancer J Clin. 2016;66:115–132.
3. Amin MB, Edge S, Greene F. AJCC Cancer Staging Manual. 2017.8th ed. New York, NY: Springer International Publishing.
4. Martling AL, Holm T, Rutqvist LE, Moran BJ, Heald RJ, Cedemark B. Effect of a surgical training programme on outcome of rectal cancer in the County of Stockholm. Stockholm Colorectal Cancer Study Group, Basingstoke Bowel Cancer Research Project. Lancet. 2000;356:93–96.
5. Monson JR, Weiser MR, Buie WD, et al.; Standards Practice Task Force of the American Society of Colon and Rectal Surgeons. Practice parameters for the management of rectal cancer (revised). Dis Colon Rectum. 2013;56:535–550.
6. Torkzad MR, Påhlman L, Glimelius B.
Magnetic resonance imaging (MRI) in rectal cancer: a comprehensive review. Insights Imaging. 2010;1:245–267.
7. Al-Sukhni E, Milot L, Fruitman M, et al. Diagnostic accuracy of MRI for assessment of T category, lymph node metastases, and circumferential resection margin involvement in patients with rectal cancer: a systematic review and meta-analysis. Ann Surg Oncol. 2012;19:2212–2223.
8. Benson AB, Venook AP, Al-Hawary MM, et al. Rectal Cancer, Version 2.2018, NCCN Clinical Practice Guidelines in Oncology. J Natl Compr Canc Netw. 2018;16:874–901.
9. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436–444.
10. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542:115–118.
11. Setio AA, Ciompi F, Litjens G, et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging. 2016;35:1160–1169.
12. Yang X, Liu C, Wang Z, et al. Co-trained convolutional neural networks for automated detection of prostate cancer in multi-parametric MRI. Med Image Anal. 2017;42:212–227.
13. Nishio M, Sugiyama O, Yakami M, et al. Computer-aided diagnosis of lung nodule classification between benign nodule, primary lung cancer, and metastatic lung cancer at different image size using deep convolutional neural network with transfer learning. PLoS One. 2018;13:e0200721.
14. Trebeschi S, van Griethuysen JJM, Lambregts DMJ, et al. Deep learning for fully-automated localization and segmentation of rectal cancer on multiparametric MR. Sci Rep. 2017;7:5301.
15. Takiyama H, Ozawa T, Ishihara S, et al. Automatic anatomical classification of esophagogastroduodenoscopy images using deep convolutional neural networks. Sci Rep. 2018;8:7497.
16. Ren S, He K, Girshick R, Zhang X, Sun J. Object detection networks on convolutional feature maps. IEEE Trans Pattern Anal Mach Intell. 2017;39:1476–1481.
17. Wong RK, Berry S, Spithoff K, et al.; Gastrointestinal Cancer Disease Site Group. Preoperative or postoperative therapy for stage II or III rectal cancer: an updated practice guideline. Clin Oncol (R Coll Radiol). 2010;22:265–271.
18. Nougaret S, Reinhold C, Mikhael HW, Rouanet P, Bibeau F, Brown G. The use of MR imaging in treatment planning for patients with rectal carcinoma: have you checked the “DISTANCE”? Radiology. 2013;268:330–344.
19. Beets-Tan RG, Beets GL, Vliegen RF, et al. Accuracy of
magnetic resonance imaging in prediction of tumour-free resection margin in rectal cancer surgery. Lancet. 2001;357:497–504.
20. Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell. 2017;39:1137–1149.
21. Chollet F. Deep learning with separable convolutions. arXiv preprint arXiv. 2016;1610.02357.
22. Xie X, Xiao H, Quan L, Shi G. Visualization and pruning of SSD with the base network VGG16. Proc. 2017 Intl. Conf. on Deep Learning Technologies. 2017 Chengdu, China: ACM; 2017:90–94.
