A model for explainable malignancy assessment of pulmonary nodules on CT images
- 作者: Dumaev R.I.1, Molodyakov S.A.1, Utkin L.V.1
-
隶属关系:
- Peter the Great St. Petersburg Polytechnic University
- 期: 编号 4 (2024)
- 页面: 123-134
- 栏目: Analysis of Signals, Audio and Video Information
- URL: https://journals.rcsi.science/2071-8594/article/view/278310
- DOI: https://doi.org/10.14357/20718594240410
- ID: 278310
如何引用文章
全文:
详细
To increase the transparency of modern computer-aided diagnosis (CAD) systems for assessing the malignancy of lung nodules, an interpretable model based on applying the generalized additive models and the concept-based learning is proposed. The model detects a set of clinically significant attributes in addition to the final malignancy regression score and learns the association between the lung nodule attributes and a final diagnosis decision as well as their contributions into the decision. The proposed concept-based learning framework provides human-readable explanations in terms of different concepts (numerical and categorical), their values, and their contribution to the final prediction. Numerical experiments with the LIDC-IDRI dataset demonstrate that the diagnosis results obtained using the proposed model, which explicitly explores internal relationships, are in line with similar patterns observed in clinical practice. Additionally, the proposed model shows the competitive classification and the nodule attribute scoring performance, highlighting its potential for effective decision-making in the lung nodule diagnosis.
作者简介
Rinat Dumaev
Peter the Great St. Petersburg Polytechnic University
编辑信件的主要联系方式.
Email: dumaevrinat@gmail.com
Graduate student
俄罗斯联邦, St. PetersburgSergey Molodyakov
Peter the Great St. Petersburg Polytechnic University
Email: samolodyakov@mail.ru
Doctor of technical sciences, docent, Professor
俄罗斯联邦, St. PetersburgLev Utkin
Peter the Great St. Petersburg Polytechnic University
Email: lev.utkin@gmail.com
Doctor of technical sciences, professor, Head of the Research Laboratory of Neural Network Technologies and Artificial Intelligence
俄罗斯联邦, St. Petersburg参考
- Majkowska A., Mittal S., Steiner D. F., Reicher J. J., McKinney S. M., Duggan G. E., Eswaran K., Cameron Chen P.-H., Liu Y., Kalidindi S. R., et al. Chest radiograph interpretation with deep learning models: assessment with radiologist-adjudicated reference standards and population-adjusted evaluation // Radiology. 2020. V. 294. No 2. P. 421–431.Xu Y., Kong M., Xie W., Duan R., Fang Z., Lin Y., Zhu Q., Tang S., Wu F., Yao Y.-F. Deep sequential feature learning in clinical image classification of infectious keratitis // Engineering. 2021. V. 7. No 7. P. 1002–1010.
- Bonavita I., Rafael-Palou X., Ceresa M., Piella G., Ribas V., Ballester M. A. G. Integration of convolutional neural networks for pulmonary nodule malignancy assessment in a lung cancer classification pipeline // Computer methods and programs in biomedicine. 2020. V. 185. P. 105172.
- Wang J., Zhu H., Wang S.-H., Zhang Y.-D. A review of deep learning on medical image analysis // Mobile Networks and Applications. 2021. V. 26. P. 351–380.
- Rudin C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead // Nature Machine Intelligence. 2019. V. 1. No 5. P. 206–215. Access mode: http://dx.doi.org/10.1038/s42256-019-0048-x.
- Adebayo J., Gilmer J., Muelly M., Goodfellow I., Hardt M., Kim B. Sanity checks for saliency maps // Advances in neural information processing systems. 2018. V. 31.
- Hendricks L. A., Hu R., Darrell T., Akata Z. Grounding visual explanations // Proceedings of the European conference on computer vision (ECCV). 2018. P. 264–279.
- Zhang Z., Xie Y., Xing F., McGough M., Yang L. Mdnet: A semantically and visually interpretable medical image diagnosis network // Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. P. 6428–6436.
- Kim B., Wattenberg M., Gilmer J., Cai C., Wexler J., Viegas F., et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav) // International conference on machine learning. PMLR. 2018. P. 2668–2677.
- Chen Z., Bei Y., Rudin C. Concept whitening for interpretable image recognition // Nature Machine Intelligence. 2020. V. 2. No 12. P. 772–782.
- Ribeiro M. T., Singh S., Guestrin C. ”Why should i trust you?”: Explaining the predictions of any classifier // Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016. P. 1135–1144.
- Fong R., Patrick M., Vedaldi A. Understanding deep networks via extremal perturbations and smooth masks // Proceedings of the IEEE/CVF international conference on computer vision. 2019. P. 2950–2958.
- Wang H., Wang Z., Du M., Yang F., Zhang Z., Ding S., Mardziel P., Hu X. Score-CAM: Score-weighted visual explanations for convolutional neural networks // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. 2020. P. 24–25.
- Chen C., Li O., Tao D., Barnett A., Rudin C., Su J. K. This looks like that: deep learning for interpretable image recognition // Advances in neural information processing systems. 2019. V. 32.
- Fang Z., Kuang K., Lin Y., Wu F., Yao Y.-F. Concept-based explanation for fine-grained images and its application in infectious keratitis classification // Proceedings of the 28th ACM international conference on Multimedia. 2020. P. 700–708.
- Graziani M., Andrearczyk V., Mu ̈ller H. Regression concept vectors for bidirectional explanations in histopathology // Understanding and Interpreting Machine Learning in Medical Image Computing Applications: First International Workshops, MLCN 2018, DLF 2018, iMIMIC 2018, Held in Conjunction with MICCAI 2018, Granada, Spain. 2018. Proceedings 1 / Springer. 2018. P. 124–132.
- Lucieri A., Bajwa M. N., Braun S. A., Malik M. I., Dengel A., Ahmed S. On interpretability of deep learning based skin lesion classifiers using concept activation vectors // 2020 international joint conference on neural networks (IJCNN) / IEEE. 2020. P. 1–10.
- Truong M. T., Ko J. P., Rossi S. E., Rossi I., Viswanathan C., Bruzzi J. F., Marom E. M., Erasmus J. J. Update in the evaluation of the solitary pulmonary nodule // Radiographics. 2014. V. 34. No 6. P. 1658–1679.
- Agarwal R., Melnick L., Frosst N., Zhang X., Lengerich B., Caruana R., Hinton G. E. Neural additive models: Interpretable machine learning with neural nets // Advances in neural information processing systems. 2021. V. 34. P. 4699–4711.
- Yang Z., Zhang A., Sudjianto A. GAMI-Net: An explainable neural network based on generalized additive models with structured interactions // Pattern Recognition. 2021. V. 120. P. 108192.
- Kumar N., Berg A. C., Belhumeur P. N., Nayar S. K. Attribute and simile classifiers for face verification // 2009 IEEE 12th international conference on computer vision / IEEE. 2009. P. 365–372.
- Lampert C. H., Nickisch H., Harmeling S. Learning to detect unseen object classes by between-class attribute transfer // 2009 IEEE conference on computer vision and pattern recognition / IEEE. 2009. P. 951–958.
- Kazhdan D., Dimanov B., Jamnik M., Li`o P., Weller A. Now you see me (CME): concept-based model extraction // arXiv preprint arXiv:2010.13233. 2020.
- Koh P. W., Nguyen T., Tang Y. S., Mussmann S., Pierson E., Kim B., Liang P. Concept bottleneck models // International conference on machine learning / PMLR. 2020. P. 5338–5348.
- Wickramanayake S., Hsu W., Lee M. L. Comprehensible convolutional neural networks via guided concept learning // 2021 International Joint Conference on Neural Networks (IJCNN) / IEEE. 2021. P. 1–8.
- Chen S., Qin J., Ji X., Lei B., Wang T., Ni D., Cheng J.-Z. Automatic scoring of multiple semantic attributes with multi-task feature leverage: a study on pulmonary nodules in CT images // IEEE transactions on medical imaging. 2016. V. 36. No 3. P. 802–814.
- Liu L., Dou Q., Chen H., Qin J., Heng P.-A. Multi-task deep model with margin ranking loss for lung nodule analysis // IEEE transactions on medical imaging. 2019. V. 39. No 3. P. 718–728.
- Dai Y., Yan S., Zheng B., Song C. Incorporating automatically learned pulmonary nodule attributes into a convolutional neural network to improve accuracy of benign-malignant nodule classification // Physics in Medicine & Biology. 2018. V. 63. No 24. P. 245004.
- Ost D. E., Gould M. K. Decision making in patients with pulmonary nodules // American journal of respiratory and critical care medicine. 2012. V. 185. No 4. P. 363–372.
- MacMahon H., Naidich D. P., Goo J. M., Lee K. S., Leung
- N., Mayo J. R., Mehta A. C., Ohno Y., Powell C. A., Prokop M., et al. Guidelines for management of incidental pulmonary nodules detected on CT images: from the Fleischner Society 2017 // Radiology. 2017. V. 284. No 1. P. 228–243.
- Cruickshank A., Stieler G., Ameer F. Evaluation of the soltary pulmonary nodule // Internal Medicine Journal. 2019. V. 49. No 3. P. 306–315.
- Shen S., Han S. X., Aberle D. R., Bui A. A., Hsu W. An interpretable deep hierarchical semantic convolutional neural network for lung nodule malignancy classification // Expert systems with applications. 2019. V. 128. P. 84–95.
- Wu B., Zhou Z., Wang J., Wang Y. Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction // 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) / IEEE. 2018. P. 1109–1113.
- Mehta K., Jain A., Mangalagiri J., Menon S., Nguyen P., Chapman D. R. Lung nodule classification using biomarkers, volumetric radiomics, 3D CNNs // Journal of Digital Imaging. 2021. P. 1–20.
- Armato III S. G., McLennan G., Bidaut L., McNitt-Gray M. F., Meyer C. R., Reeves A. P., Zhao B., Aberle D. R., Henschke C. I., Hoffman E. A., et al. The lung image data-base consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans // Medical physics. 2011. V. 38. No 2. P. 915–931.
- Hancock M. C., Magnan J. F. Lung nodule malignancy classification using only radiologist-quantified image features as inputs to statistical learning algorithms: probing the Lung Image Database Consortium dataset with two statistical learning methods // Journal of Medical Imaging. 2016. V. 3. No 4. P. 044504–044504.
- He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition // Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. P. 770–778.
- Huang G., Liu Z., Van Der Maaten L., Weinberger K. Q. Densely connected convolutional networks // Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. P. 4700–4708.
- Hu J., Shen L., Sun G. Squeeze-and-excitation networks // Proceedings of the IEEE conference on computer vision and pattern recognition. 2018. P. 7132–7141.
- Snoeckx A., Reyntiens P., Desbuquoit D., Spinhoven M. J., Van Schil P. E., van Meerbeeck J. P., Parizel P. M. Evaluation of the solitary pulmonary nodule: size matters, but do not ignore the power of morphology // Insights into imaging. 2018. V. 9. P. 73–86.
- Yip R., Yankelevitz D. F., Hu M., Li K., Xu D. M., Jirapatnakul A., Henschke C. I. Lung cancer deaths in the National Lung Screening Trial attributed to nonsolid nodules // Radiology. 2016. V. 281. No 2. P. 589–596.
- Seemann M., Staebler A., Beinert T., Dienemann H., Obst B., Matzko M., Pistitsch C., Reiser M. Usefulness of morphological characteristics for the differentiation of benign from malignant solitary pulmonary lesions using HRCT // European radiology. 1999. V. 9. No 3. P. 409–417.
- Gurney J. Determining the likelihood of malignancy in solitary pulmonary nodules with Bayesian analysis. Part I. Theory // Radiology. 1993. V. 186. No 2. P. 405–413.
- Meldo A., Utkin L., Kovalev M., Kasimov E. The natural language explanation algorithms for the lung cancer computer-aided diagnosis system // Artificial intelligence in medicine. 2020. V. 108. P. 101952.
- Dumaev R. I., Molodyakov S. A. Classification and Prediction of Lung Diseases According to Chest Radiography // 2023 IV International Conference on Neural Networks and Neurotechnologies (NeuroNT) / IEEE. 2023. P. 48–51.
补充文件
