ARTIFICIAL INTELLIGENCE AND DECISION MAKINGARTIFICIAL INTELLIGENCE AND DECISION MAKINGИскусственный интеллект и принятие решений2071-859427831010.14357/20718594240410Analysis of Signals, Audio and Video InformationАнализ сигналов, аудио и видео информацииResearch ArticleA model for explainable malignancy assessment of pulmonary nodules on CT imagesМодель для объяснимой оценки злокачественности легочных узелков на КТ-изображенияхDumaevRinat I.ДумаевРинат Ильгизович
Russian Federation

Graduate student

Аспирант

dumaevrinat@gmail.comMolodyakovSergey A.МолодяковСергей Александрович
Russian Federation

Doctor of technical sciences, docent, Professor

Доктор технических наук, доцент, профессор

samolodyakov@mail.ruUtkinLev V.УткинЛев Владимирович
Russian Federation

Doctor of technical sciences, professor, Head of the Research Laboratory of Neural Network Technologies and Artificial Intelligence

Доктор технических наук, профессор, заведующий научно-исследовательской лабораторией нейросетевых технологий и искусственного интеллекта высшей школы технологии искусственного интеллекта

lev.utkin@gmail.comPeter the Great St. Petersburg Polytechnic UniversityСанкт-Петербургский политехнический университет Петра Великого1012202441231342801202528012025Copyright ©; ,Copyright ©; ,https://journals.rcsi.science/2071-8594/article/view/278310

To increase the transparency of modern computer-aided diagnosis (CAD) systems for assessing the malignancy of lung nodules, an interpretable model based on applying the generalized additive models and the concept-based learning is proposed. The model detects a set of clinically significant attributes in addition to the final malignancy regression score and learns the association between the lung nodule attributes and a final diagnosis decision as well as their contributions into the decision. The proposed concept-based learning framework provides human-readable explanations in terms of different concepts (numerical and categorical), their values, and their contribution to the final prediction. Numerical experiments with the LIDC-IDRI dataset demonstrate that the diagnosis results obtained using the proposed model, which explicitly explores internal relationships, are in line with similar patterns observed in clinical practice. Additionally, the proposed model shows the competitive classification and the nodule attribute scoring performance, highlighting its potential for effective decision-making in the lung nodule diagnosis.

Для решения проблемы непрозрачности современных систем оценки злокачественности образований легких предложена основанная на понятиях объяснимая модель с использованием обобщенных аддитивных моделей. Модель обнаруживает набор клинически значимых признаков в дополнение к окончательному показателю злокачественности новообразований и изучает связь и вклад между атрибутами узелков в легких и окончательным решением. Она предоставляет понятные человеку объяснения с точки зрения различных признаков, таких как плотность, внутренняя текстура, их значений и вклада в окончательный прогноз. Численные эксперименты показали, что результаты диагностики, полученные с использованием модели, соответствуют аналогичным закономерностям, наблюдаемым в клинической практике между атрибутами узелков в легких и показателем злокачественности новообразований. Приведены примеры прогнозов, сгенерированных с помощью разработанной модели.

explainable artificial intelligencemedical image processingpulmonary nodulesgeneralized additive modelneural networkconcept-based learningобъяснимый искусственный интеллектобработка медицинских изображенийлегочные узелкиобобщенная аддитивная модельнейронная сетьобучение на основе понятийMajkowska A., Mittal S., Steiner D. F., Reicher J. J., McKinney S. M., Duggan G. E., Eswaran K., Cameron Chen P.-H., Liu Y., Kalidindi S. R., et al. Chest radiograph interpretation with deep learning models: assessment with radiologist-adjudicated reference standards and population-adjusted evaluation // Radiology. 2020. V. 294. No 2. P. 421–431.Xu Y., Kong M., Xie W., Duan R., Fang Z., Lin Y., Zhu Q., Tang S., Wu F., Yao Y.-F. Deep sequential feature learning in clinical image classification of infectious keratitis // Engineering. 2021. V. 7. No 7. P. 1002–1010.Majkowska A., Mittal S., Steiner D. F., Reicher J. J., McKinney S. M., Duggan G. E., Eswaran K., Cameron Chen P.-H., Liu Y., Kalidindi S. R., et al. Chest radiograph interpretation with deep learning models: assessment with radiologist-adjudicated reference standards and population-adjusted evaluation // Radiology. 2020. V. 294. No 2. P. 421–431.Bonavita I., Rafael-Palou X., Ceresa M., Piella G., Ribas V., Ballester M. A. G. Integration of convolutional neural networks for pulmonary nodule malignancy assessment in a lung cancer classification pipeline // Computer methods and programs in biomedicine. 2020. V. 185. P. 105172.Xu Y., Kong M., Xie W., Duan R., Fang Z., Lin Y., Zhu Q., Tang S., Wu F., Yao Y.-F. Deep sequential feature learning in clinical image classification of infectious keratitis // Engineering. 2021. V. 7. No 7. P. 1002–1010.Wang J., Zhu H., Wang S.-H., Zhang Y.-D. A review of deep learning on medical image analysis // Mobile Networks and Applications. 2021. V. 26. P. 351–380.Bonavita I., Rafael-Palou X., Ceresa M., Piella G., Ribas V., Ballester M. A. G. Integration of convolutional neural networks for pulmonary nodule malignancy assessment in a lung cancer classification pipeline // Computer methods and programs in biomedicine. 2020. V. 185. P. 105172.Rudin C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead // Nature Machine Intelligence. 2019. V. 1. No 5. P. 206–215. Access mode: http://dx.doi.org/10.1038/s42256-019-0048-x.Wang J., Zhu H., Wang S.-H., Zhang Y.-D. A review of deep learning on medical image analysis // Mobile Networks and Applications. 2021. V. 26. P. 351–380.Adebayo J., Gilmer J., Muelly M., Goodfellow I., Hardt M., Kim B. Sanity checks for saliency maps // Advances in neural information processing systems. 2018. V. 31.Rudin C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead // Nature Machine Intelligence. 2019. V. 1. No 5. 206–215. Access mode: http://dx.doi.org/10.1038/s42256-019-0048-x.Hendricks L. A., Hu R., Darrell T., Akata Z. Grounding visual explanations // Proceedings of the European conference on computer vision (ECCV). 2018. P. 264–279.Adebayo J., Gilmer J., Muelly M., Goodfellow I., Hardt M., Kim B. Sanity checks for saliency maps // Advances in neural information processing systems. 2018. V. 31.Zhang Z., Xie Y., Xing F., McGough M., Yang L. Mdnet: A semantically and visually interpretable medical image diagnosis network // Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. P. 6428–6436.Hendricks L. A., Hu R., Darrell T., Akata Z. Grounding visual explanations // Proceedings of the European conference on computer vision (ECCV). 2018. P. 264–279.Kim B., Wattenberg M., Gilmer J., Cai C., Wexler J., Viegas F., et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav) // International conference on machine learning. PMLR. 2018. P. 2668–2677.Zhang Z., Xie Y., Xing F., McGough M., Yang L. Mdnet: A semantically and visually interpretable medical image diagnosis network // Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. P. 6428–6436.Chen Z., Bei Y., Rudin C. Concept whitening for interpretable image recognition // Nature Machine Intelligence. 2020. V. 2. No 12. P. 772–782.Kim B., Wattenberg M., Gilmer J., Cai C., Wexler J., Viegas F., et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav) // International conference on machine learning. PMLR. 2018. 2668–2677.Ribeiro M. T., Singh S., Guestrin C. ”Why should i trust you?”: Explaining the predictions of any classifier // Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016. P. 1135–1144.Chen Z., Bei Y., Rudin C. Concept whitening for interpretable image recognition // Nature Machine Intelligence. 2020. V. 2. No 12. P. 772–782.Fong R., Patrick M., Vedaldi A. Understanding deep networks via extremal perturbations and smooth masks // Proceedings of the IEEE/CVF international conference on computer vision. 2019. P. 2950–2958.Ribeiro M. T., Singh S., Guestrin C. ”Why should i trust you?”: Explaining the predictions of any classifier // Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016. 1135–1144.Wang H., Wang Z., Du M., Yang F., Zhang Z., Ding S., Mardziel P., Hu X. Score-CAM: Score-weighted visual explanations for convolutional neural networks // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. 2020. P. 24–25.Fong R., Patrick M., Vedaldi A. Understanding deep networks via extremal perturbations and smooth masks // Proceedings of the IEEE/CVF international conference on computer vision. 2019. P. 2950–2958.Chen C., Li O., Tao D., Barnett A., Rudin C., Su J. K. This looks like that: deep learning for interpretable image recognition // Advances in neural information processing systems. 2019. V. 32.Wang H., Wang Z., Du M., Yang F., Zhang Z., Ding S., Mardziel P., Hu X. Score-CAM: Score-weighted visual explanations for convolutional neural networks // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. 2020. P. 24–25.Fang Z., Kuang K., Lin Y., Wu F., Yao Y.-F. Concept-based explanation for fine-grained images and its application in infectious keratitis classification // Proceedings of the 28th ACM international conference on Multimedia. 2020. P. 700–708.Chen C., Li O., Tao D., Barnett A., Rudin C., Su J. K. This looks like that: deep learning for interpretable image recognition // Advances in neural information processing systems. 2019. V. 32.Graziani M., Andrearczyk V., Mu ̈ller H. Regression concept vectors for bidirectional explanations in histopathology // Understanding and Interpreting Machine Learning in Medical Image Computing Applications: First International Workshops, MLCN 2018, DLF 2018, iMIMIC 2018, Held in Conjunction with MICCAI 2018, Granada, Spain. 2018. Proceedings 1 / Springer. 2018. P. 124–132.Fang Z., Kuang K., Lin Y., Wu F., Yao Y.-F. Concept-based explanation for fine-grained images and its application in infectious keratitis classification // Proceedings of the 28th ACM international conference on Multimedia. 2020. 700–708.Lucieri A., Bajwa M. N., Braun S. A., Malik M. I., Dengel A., Ahmed S. On interpretability of deep learning based skin lesion classifiers using concept activation vectors // 2020 international joint conference on neural networks (IJCNN) / IEEE. 2020. P. 1–10.Graziani M., Andrearczyk V., Mu ̈ller H. Regression concept vectors for bidirectional explanations in histopathology // Understanding and Interpreting Machine Learning in Medical Image Computing Applications: First International Workshops, MLCN 2018, DLF 2018, iMIMIC 2018, Held in Conjunction with MICCAI 2018, Granada, Spain. 2018. Proceedings 1 / Springer. 2018. P. 124–132.Truong M. T., Ko J. P., Rossi S. E., Rossi I., Viswanathan C., Bruzzi J. F., Marom E. M., Erasmus J. J. Update in the evaluation of the solitary pulmonary nodule // Radiographics. 2014. V. 34. No 6. P. 1658–1679.Lucieri A., Bajwa M. N., Braun S. A., Malik M. I., Dengel A., Ahmed S. On interpretability of deep learning based skin lesion classifiers using concept activation vectors // 2020 international joint conference on neural networks (IJCNN) / IEEE. 2020. P. 1–10.Agarwal R., Melnick L., Frosst N., Zhang X., Lengerich B., Caruana R., Hinton G. E. Neural additive models: Interpretable machine learning with neural nets // Advances in neural information processing systems. 2021. V. 34. P. 4699–4711.Truong M. T., Ko J. P., Rossi S. E., Rossi I., Viswanathan C., Bruzzi J. F., Marom E. M., Erasmus J. J. Update in the evaluation of the solitary pulmonary nodule // Radiographics. 2014. V. 34. No 6. P. 1658–1679.Yang Z., Zhang A., Sudjianto A. GAMI-Net: An explainable neural network based on generalized additive models with structured interactions // Pattern Recognition. 2021. V. 120. P. 108192.Agarwal R., Melnick L., Frosst N., Zhang X., Lengerich B., Caruana R., Hinton G. E. Neural additive models: Interpretable machine learning with neural nets // Advances in neural information processing systems. 2021. V. 34. P. 4699–4711.Kumar N., Berg A. C., Belhumeur P. N., Nayar S. K. Attribute and simile classifiers for face verification // 2009 IEEE 12th international conference on computer vision / IEEE. 2009. P. 365–372.Yang Z., Zhang A., Sudjianto A. GAMI-Net: An explainable neural network based on generalized additive models with structured interactions // Pattern Recognition. 2021. V. 120. P. 108192.Lampert C. H., Nickisch H., Harmeling S. Learning to detect unseen object classes by between-class attribute transfer // 2009 IEEE conference on computer vision and pattern recognition / IEEE. 2009. P. 951–958.Kumar N., Berg A. C., Belhumeur P. N., Nayar S. K. Attribute and simile classifiers for face verification // 2009 IEEE 12th international conference on computer vision / IEEE. 2009. P. 365–372.Kazhdan D., Dimanov B., Jamnik M., Li`o P., Weller A. Now you see me (CME): concept-based model extraction // arXiv preprint arXiv:2010.13233. 2020.Lampert C. H., Nickisch H., Harmeling S. Learning to detect unseen object classes by between-class attribute transfer // 2009 IEEE conference on computer vision and pattern recognition / IEEE. 2009. P. 951–958.Koh P. W., Nguyen T., Tang Y. S., Mussmann S., Pierson E., Kim B., Liang P. Concept bottleneck models // International conference on machine learning / PMLR. 2020. P. 5338–5348.Kazhdan D., Dimanov B., Jamnik M., Li`o P., Weller A. Now you see me (CME): concept-based model extraction // arXiv preprint arXiv:2010.13233. 2020.Wickramanayake S., Hsu W., Lee M. L. Comprehensible convolutional neural networks via guided concept learning // 2021 International Joint Conference on Neural Networks (IJCNN) / IEEE. 2021. P. 1–8.Koh P. W., Nguyen T., Tang Y. S., Mussmann S., Pierson E., Kim B., Liang P. Concept bottleneck models // International conference on machine learning / PMLR. 2020. 5338–5348.Chen S., Qin J., Ji X., Lei B., Wang T., Ni D., Cheng J.-Z. Automatic scoring of multiple semantic attributes with multi-task feature leverage: a study on pulmonary nodules in CT images // IEEE transactions on medical imaging. 2016. V. 36. No 3. P. 802–814.Wickramanayake S., Hsu W., Lee M. L. Comprehensible convolutional neural networks via guided concept learning // 2021 International Joint Conference on Neural Networks (IJCNN) / IEEE. 2021. P. 1–8.Liu L., Dou Q., Chen H., Qin J., Heng P.-A. Multi-task deep model with margin ranking loss for lung nodule analysis // IEEE transactions on medical imaging. 2019. V. 39. No 3. P. 718–728.Chen S., Qin J., Ji X., Lei B., Wang T., Ni D., Cheng J.-Z. Automatic scoring of multiple semantic attributes with multi-task feature leverage: a study on pulmonary nodules in CT images // IEEE transactions on medical imaging. 2016. V. 36. No 3. P. 802–814.Dai Y., Yan S., Zheng B., Song C. Incorporating automatically learned pulmonary nodule attributes into a convolutional neural network to improve accuracy of benign-malignant nodule classification // Physics in Medicine & Biology. 2018. V. 63. No 24. P. 245004.Liu L., Dou Q., Chen H., Qin J., Heng P.-A. Multi-task deep model with margin ranking loss for lung nodule analysis // IEEE transactions on medical imaging. 2019. V. 39. No 3. 718–728.Ost D. E., Gould M. K. Decision making in patients with pulmonary nodules // American journal of respiratory and critical care medicine. 2012. V. 185. No 4. P. 363–372.Dai Y., Yan S., Zheng B., Song C. Incorporating automatically learned pulmonary nodule attributes into a convolutional neural network to improve accuracy of benign-malignant nodule classification // Physics in Medicine & Biology. 2018. V. 63. No 24. P. 245004.MacMahon H., Naidich D. P., Goo J. M., Lee K. S., LeungOst D. E., Gould M. K. Decision making in patients with pulmonary nodules // American journal of respiratory and critical care medicine. 2012. V. 185. No 4. P. 363–372.N., Mayo J. R., Mehta A. C., Ohno Y., Powell C. A., Prokop M., et al. Guidelines for management of incidental pulmonary nodules detected on CT images: from the Fleischner Society 2017 // Radiology. 2017. V. 284. No 1. P. 228–243.MacMahon H., Naidich D. P., Goo J. M., Lee K. S., Leung N., Mayo J. R., Mehta A. C., Ohno Y., Powell C. A., Prokop M., et al. Guidelines for management of incidental pulmonary nodules detected on CT images: from the Fleischner Society 2017 // Radiology. 2017. V. 284. No 1. 228–243.Cruickshank A., Stieler G., Ameer F. Evaluation of the soltary pulmonary nodule // Internal Medicine Journal. 2019. V. 49. No 3. P. 306–315.Cruickshank A., Stieler G., Ameer F. Evaluation of the solitary pulmonary nodule // Internal Medicine Journal. 2019. 49. No 3. P. 306–315.Shen S., Han S. X., Aberle D. R., Bui A. A., Hsu W. An interpretable deep hierarchical semantic convolutional neural network for lung nodule malignancy classification // Expert systems with applications. 2019. V. 128. P. 84–95.Wu B., Zhou Z., Wang J., Wang Y. Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction // 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) / IEEE. 2018. P. 1109–1113.Mehta K., Jain A., Mangalagiri J., Menon S., Nguyen P., Chapman D. R. Lung nodule classification using biomarkers, volumetric radiomics, 3D CNNs // Journal of Digital Imaging. 2021. P. 1–20.Armato III S. G., McLennan G., Bidaut L., McNitt-Gray M. F., Meyer C. R., Reeves A. P., Zhao B., Aberle D. R., Henschke C. I., Hoffman E. A., et al. The lung image data-base consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans // Medical physics. 2011. V. 38. No 2. P. 915–931.Armato III S. G., McLennan G., Bidaut L., McNitt-Gray M. F., Meyer C. R., Reeves A. P., Zhao B., Aberle D. R., Henschke C. I., Hoffman E. A., et al. The lung image data-base consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans // Medical physics. 2011. V. 38. No 2. 915–931.Hancock M. C., Magnan J. F. Lung nodule malignancy classification using only radiologist-quantified image features as inputs to statistical learning algorithms: probing the Lung Image Database Consortium dataset with two statistical learning methods // Journal of Medical Imaging. 2016. V. 3. No 4. P. 044504–044504.Hancock M. C., Magnan J. F. Lung nodule malignancy classification using only radiologist-quantified image features as inputs to statistical learning algorithms: probing the Lung Image Database Consortium dataset with two statistical learning methods // Journal of Medical Imaging. 2016. V. No 4. P. 044504–044504.He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition // Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. P. 770–778.Huang G., Liu Z., Van Der Maaten L., Weinberger K. Q. Densely connected convolutional networks // Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. P. 4700–4708.Hu J., Shen L., Sun G. Squeeze-and-excitation networks // Proceedings of the IEEE conference on computer vision and pattern recognition. 2018. P. 7132–7141.Snoeckx A., Reyntiens P., Desbuquoit D., Spinhoven M. J., Van Schil P. E., van Meerbeeck J. P., Parizel P. M. Evaluation of the solitary pulmonary nodule: size matters, but do not ignore the power of morphology // Insights into imaging. 2018. V. 9. P. 73–86.Yip R., Yankelevitz D. F., Hu M., Li K., Xu D. M., Jirapatnakul A., Henschke C. I. Lung cancer deaths in the National Lung Screening Trial attributed to nonsolid nodules // Radiology. 2016. V. 281. No 2. P. 589–596.Seemann M., Staebler A., Beinert T., Dienemann H., Obst B., Matzko M., Pistitsch C., Reiser M. Usefulness of morphological characteristics for the differentiation of benign from malignant solitary pulmonary lesions using HRCT // European radiology. 1999. V. 9. No 3. P. 409–417.Gurney J. Determining the likelihood of malignancy in solitary pulmonary nodules with Bayesian analysis. Part I. Theory // Radiology. 1993. V. 186. No 2. P. 405–413.Meldo A., Utkin L., Kovalev M., Kasimov E. The natural language explanation algorithms for the lung cancer computer-aided diagnosis system // Artificial intelligence in medicine. 2020. V. 108. P. 101952.Dumaev R. I., Molodyakov S. A. Classification and Prediction of Lung Diseases According to Chest Radiography // 2023 IV International Conference on Neural Networks and Neurotechnologies (NeuroNT) / IEEE. 2023. P. 48–51.