After image-guided biopsy, most lesions identified on mammograms turn out to be benign or malignant. The rest (as high as 14% of lesions) are diagnosed as high-risk breast lesions, meaning they may upgrade to cancer later on. The majority of high-risk lesions, however, do not upgrade and remain benign. However, distinguishing between high-risk lesions that upgrade to cancer and those that do not is difficult, so high-risk lesions are often surgically removed to ensure none become cancerous. The result is women with high-risk lesions are usually overtreated with surgery.
“We knew that across the country there is such wide variation in how these women currently are managed,” said senior author Constance Lehman, MD, PhD, Professor at Harvard Medical School and Chief of the Breast Imaging Division at Massachusetts General Hospital, Boston. She explained that some medical centers recommend excision of all high-risk lesions and other centers are very selective when recommending surgery. It was this wide variation, she explained, that made this condition “ripe for integration of machine learning.”
Predicting Breast Lesions
Lehman and her colleagues from Massachusetts Institute of Technology, Massachusetts General Hospital, and Harvard Medical School conducted a proof-of-concept, single-center study that showed a machine learning model could predict whether a patient's high-risk breast lesion would upgrade to cancer at the time of surgery to remove the lesion. Being able to predict whether the lesion will upgrade would allow these women to forgo unnecessary surgery along with the stress and costs associated with surgery. The study was published in October in Radiology (2017; doi:10.1148/radiol.2017170549).
“It's mainly safe to say that this is the first study using machine learning and this specific kind of data to do this work,” noted Shandong Wu, PhD, Assistant Professor of Radiology, Biomedical Informatics, and Bioengineering at the University of Pittsburgh. “I believe there are relevant works in the literature targeted on similar questions, but this work is very encouraging.”
The researchers used a traditional machine learning model called a random forest classifier to retrospectively predict which high-risk lesions had a low likelihood of upgrading to cancer at the time of surgery. The model was trained with data from 671 high-risk lesions (654 total patients) in which the model knew the surgical outcomes. The data provided to the model included demographic factors, such as age and race, and information from mammographic reports, biopsy reports, and pathology reports.
Then, the model was tested on data from 335 high-risk lesions (332 total patients) that the model had never seen before. Without knowing the surgical outcomes, the model predicted which lesions would be upgraded at the time of surgery and which ones would not.
Researchers compared the model predictions with the actual surgical outcomes and found that of the 335 tested high-risk lesions, the model correctly identified 37 of 38 lesions as cancerous (97.4%) and 91 of 297 lesions as benign (30.6%). In theory, the model could have informed treatment decisions and prevented about one-third of these women from having an unnecessary surgery.
“Machines, when trained appropriately, can do better than humans in predicting future events in our patients,” said Lehman. “With our machine learning model, we could predict better which women would have cancer diagnosed at a site of a high-risk lesion and which ones wouldn't. This allows us to reap more benefits from our screening techniques and reduce the false positives and the unnecessary surgeries that are associated with any screening program.
“We think this has widespread applications in cancer care,” she added.
Upon closer investigation, the researchers found that the one lesion that upgraded to cancer but was misclassified by the model was from a patient with Cowden syndrome. This is a rare hereditary condition characterized by noncancerous growths on the skin and mucous membranes called hamartomas and is associated with an increased risk of developing breast cancer. The patient's history of Cowden syndrome was not provided to the model. Lehman said the researchers plan to include Cowden syndrome and other syndromes in the data provided to the model in the future.
Although the model accurately identified about one-third of women with lesions that did not upgrade to cancer, it missed two-thirds. To improve the accuracy of the machine learning model, Lehman said the number of cases seen by the model could be increased and feedback could be provided to the model.
“The more you can continually train, the smarter and better the machine becomes,” she noted.
“They showed that this technique works for this scenario,” Wu said. However, he explained that a newer technique called deep learning works differently from the traditional machine learning technique used in this study and has shown better performance in many other applications. He wonders whether a newer technique, such as deep learning, would have performed better than the machine learning method used.
Regarding the data provided to the model, Wu stated he would like to see some of the imaging data incorporated into the machine learning frame work. “Inclusion of the imaging itself would be even more relevant and could potentially improve the model.”
He added that the model needs to be tested on data from other medical centers to further evaluate and validate the model.
“The very careful and important next step is to begin clinical implementation, and that has to be done in a very careful way,” Lehman said. The model will be incorporated in the clinic at Massachusetts General Hospital next year for further testing and evaluation.
As for when this model could be available to physicians outside of Massachusetts General Hospital, Lehman said the model must first demonstrate consistency and the performance must be evaluated at Massachusetts General Hospital. Also, they must determine how best to implement this technology in the clinic.
Lehman emphasized this model is intended to assist, not replace the physician's decision, in the long-term.
“It would be hard to imagine, in this particular area of breast cancer, we would depend solely on the machine,” she said. “We still have a lot of work to do to have the machine assist and improve the radiologist's or the physician's decision.”
Lehman noted that at this time a cost analysis to estimate the potential cost savings of using this machine learning model has not been done, but she said the group would like to.
“There are many areas in healthcare that are not that straightforward,” Lehman said. “That's where we think the machine learning tools can be really helpful.”
Christina Bennett is a contributing writer.