Diagnosis of intracranial hemorrhages based on brain computed tomography with artificial intelligence

Cover Image

Cite item

Full Text

Abstract

BACKGROUND: Intracranial hemorrhages are associated with high mortality and risk of disability, requiring prompt and accurate diagnosis, particularly within the first 24 hours. The use of artificial intelligence technologies in analyzing brain computed tomography scans can shorten diagnostic time and improve diagnostic quality. The relevance of this study is emphasized by the limited number of certified artificial intelligence services for detecting intracranial hemorrhages in Russia and lacking data on their long-term effectiveness, highlighting the need for multicenter monitoring to assess the stability and accuracy of such systems in clinical practice.

AIM: The study aimed to assess the diagnostic accuracy and stability of an artificial intelligence service in detecting intracranial hemorrhages on non-contrast brain computed tomography scans in a multicenter clinical monitoring setting for 18 months.

METHODS: Anonymized brain computed tomography scans were used. The artificial intelligence service underwent a three-phase evaluation to evaluate its diagnostic accuracy and clinical performance using limited datasets. Two radiologists specializing in neuroimaging examined 80 brain computed tomography scans each month for 18 months, which had been preprocessed by the artificial intelligence service and randomly selected from the clinical workflow. The results were analyzed using ROC analysis with sensitivity, specificity, accuracy, and area under the curve.

RESULTS: During clinical monitoring, 1200 brain computed tomography scans were analyzed, with signs of intracranial hemorrhage detected in 48.3% of the scans. Based on the binary classification of intracranial hemorrhage presence or absence performed by the artificial intelligence service, the following diagnostic metrics were obtained: sensitivity, 97.4% (95.8–98.5); specificity, 75.4% (71.8–78.7); accuracy, 86.0% (83.9–87.9); and area under the curve, 94% (92.6–95.3). Eventually, a significant moderate positive correlation was observed in most diagnostic metrics and the time variable, except for sensitivity, which was affected by an update to the service version. However, full concordance between artificial intelligence-based markings and radiologist conclusions was noted in 28.5% of cases of identified intracranial hemorrhage, whereas discrepancies were found in 71.5%. The refined diagnostic metrics for cases with complete agreement with the radiologists’ report were as follows: sensitivity, 26.6%; specificity, 73.8%; accuracy, 50.1%; and area under the curve, 49.6%.

CONCLUSION: The current configuration of the artificial intelligence service allows ruling out intracranial hemorrhage with very high probability, which may be useful in the initial triaging of patients in emergency settings. However, low values of refined metrics indicate considerable discrepancies between radiologist reports and service-generated results regarding the interpretation of pathological findings.

About the authors

Anna N. Khoruzhaya

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Author for correspondence.
Email: KhoruzhayaAN@zdrav.mos.ru
ORCID iD: 0000-0003-4857-5404
SPIN-code: 7948-6427

MD

Russian Federation, Moscow

Kirill M. Arzamasov

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Email: ArzamasovK@zdrav.mos.ru
ORCID iD: 0000-0001-7786-0349
SPIN-code: 3160-8062

MD, Dr. Sci. (Medicine)

Russian Federation, Moscow

Maria R. Kodenko

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Email: KodenkoM@zdrav.mos.ru
ORCID iD: 0000-0002-0166-3768
SPIN-code: 5789-0319

Cand. Sci. (Engineering)

Russian Federation, Moscow

Elena I. Kremneva

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies; Russian Center of Neurology and Neurosciences

Email: KremnevaE@zdrav.mos.ru
ORCID iD: 0000-0001-9396-6063
SPIN-code: 8799-8092

MD, Dr. Sci. (Medicine)

Russian Federation, Moscow; Moscow

Dmitry V. Burenchev

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Email: BurenchevD@zdrav.mos.ru
ORCID iD: 0000-0003-2894-6255
SPIN-code: 2411-3959

MD, Dr. Sci. (Medicine)

Russian Federation, Moscow

References

  1. Li X, Zhang L, Wolfe CDA, Wang Y. Incidence and long-term survival of spontaneous intracerebral hemorrhage over time: a systematic review and meta-analysis. Frontiers in Neurology. 2022;13:819737. doi: 10.3389/fneur.2022.819737 EDN: MLOQRJ
  2. Hemorrhagic stroke: clinical guidelines. Moscow: Ministry of Health of the Russian Federation; 2022. (In Russ.) [cited 2024 Dec 12]. Available from: https://ruans.org/Text/Guidelines/hemorrhagic-stroke-2022.pdf
  3. Hostettler IC, Seiffge DJ, Werring DJ. Intracerebral hemorrhage: an update on diagnosis and treatment. Expert Review of Neurotherapeutics. 2019;19(7):679–694. doi: 10.1080/14737175.2019.1623671 EDN: JWSYUZ
  4. Woo D, Comeau ME, Venema SU, et al. Risk factors associated with mortality and neurologic disability after intracerebral hemorrhage in a racially and ethnically diverse cohort. JAMA Network Open. 2022;5(3):e221103. doi: 10.1001/jamanetworkopen.2022.1103 EDN: BVHNLU
  5. Yaghi S, Dibu J, Achi E, et al. Hematoma expansion in spontaneous intracerebral hemorrhage: predictors and outcome. International Journal of Neuroscience. 2014;124(12):890–893. doi: 10.3109/00207454.2014.887716
  6. Gong B, Khalvati F, Ertl-Wagner BB, Patlas MN. Artificial intelligence in emergency neuroradiology: current applications and perspectives. Diagnostic and Interventional Imaging. 2025;106(4):135–142. doi: 10.1016/j.diii.2024.11.002 EDN: DHXSGS
  7. Arbabshirani MR, Fornwalt BK, Mongelluzzo GJ, et al. Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration. npj Digital Medicine. 2018;1(1):9. doi: 10.1038/s41746-017-0015-z EDN: BORIWC
  8. Seyam M, Weikert T, Sauter A, et al. Utilization of artificial intelligence–based intracranial hemorrhage detection on emergent noncontrast CT images in clinical workflow. Radiology: Artificial Intelligence. 2022;4(2):e210168. doi: 10.1148/ryai.210168 EDN: HEPSBX
  9. Davis MA, Rao B, Cedeno PA, et al. machine learning and improved quality metrics in acute intracranial hemorrhage by noncontrast computed tomography. Current Problems in Diagnostic Radiology. 2022;51(4):556–561. doi: 10.1067/j.cpradiol.2020.10.007 EDN: NHQFYC
  10. O’Neill TJ, Xi Y, Stehel E, et al. Active reprioritization of the reading worklist using artificial intelligence has a beneficial effect on the turnaround time for interpretation of head CT with intracranial hemorrhage. Radiology: Artificial Intelligence. 2021;3(2):e200024. doi: 10.1148/ryai.2020200024 EDN: LCDGTM
  11. Smorchkova AK, Khoruzhaya AN, Kremneva EI, Petryaikin AV. Machine learning technologies in CT-based diagnostics and classification of intracranial hemorrhages. Burdenko's Journal of Neurosurgery. 2023;87(2):85. doi: 10.17116/neiro20238702185EDN: JVZDST
  12. Yu KH, Kohane IS. Framing the challenges of artificial intelligence in medicine. BMJ Quality & Safety. 2018;28(3):238–241. doi: 10.1136/bmjqs-2018-008551
  13. Allen B, Dreyer K, Stibolt R, et al. Evaluation and real-world performance monitoring of artificial intelligence models in clinical practice: try it, buy it, check it. Journal of the American College of Radiology. 2021;18(11):1489–1496. doi: 10.1016/j.jacr.2021.08.022 EDN: NMKGVD
  14. Recht MP, Dewey M, Dreyer K, et al. Integrating artificial intelligence into the clinical practice of radiology: challenges and recommendations. European Radiology. 2020;30(6):3576–3584. doi: 10.1007/s00330-020-06672-5 EDN: WWDEXB
  15. Vasiliev YuA, Vlazimirskyy AV, Omelyanskaya OV, et al. Methodology for testing and monitoring artificial intelligence-based software for medical diagnostics. Digital Diagnostics. 2023;4(3):252–267. doi: 10.17816/DD321971 EDN: UEDORU
  16. Morozov SP, Vladzimirsky AV, Klyashtornyy VG, et al. Clinical acceptance of software based on artificial intelligence technologies (radiology). Moscow: Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies; 2019. EDN: GWJIMI
  17. Morozov SP, Vladzimirsky AV, Andreychenko AE, et al. Regulations for the preparation of data sets with a description of approaches to the formation of a representative data sample. Moscow: Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies; 2022. (In Russ.) EDN: XENAJE
  18. Chetverikov SF, Arzamasov KM, Andreichenko AE, et al. Approaches to sampling for quality control of artificial intelligence in biomedical research. Sovremennye tehnologii v medicine. 2023;15(2):19. doi: 10.17691/stm2023.15.2.02 EDN: FUKXYC
  19. Kodenko MR, Bobrovskaya TM, Reshetnikov RV, et al. Empirical approach to sample size estimation for testing of AI algorithms. Doklady Mathematics. 2024;110(S1):S62–S74. doi: 10.1134/S1064562424602063 EDN: VJHJRD
  20. Salehinejad H, Kitamura J, Ditkofsky N, et al. A real-world demonstration of machine learning generalizability in the detection of intracranial hemorrhage on head computerized tomography. Scientific Reports. 2021;11(1):17051. doi: 10.1038/s41598-021-95533-2 EDN: SXLMCH
  21. Zia A, Fletcher C, Bigwood S, et al. Retrospective analysis and prospective validation of an AI-based software for intracranial haemorrhage detection at a high-volume trauma centre. Scientific Reports. 2022;12(1):19885. doi: 10.1038/s41598-022-24504-y EDN: IWNBET
  22. Ginat DT. Analysis of head CT scans flagged by deep learning software for acute intracranial hemorrhage. Neuroradiology. 2019;62(3):335–340. doi: 10.1007/s00234-019-02330-w EDN: WTOITQ
  23. Voter AF, Meram E, Garrett JW, Yu JPJ. Diagnostic accuracy and failure mode analysis of a deep learning algorithm for the detection of intracranial hemorrhage. Journal of the American College of Radiology. 2021;18(8):1143–1152. doi: 10.1016/j.jacr.2021.03.005 EDN: GPJYDS
  24. McLouth J, Elstrott S, Chaibi Y, et al. Validation of a deep learning tool in the detection of intracranial hemorrhage and large vessel occlusion. Frontiers in Neurology. 2021;12:656112. doi: 10.3389/fneur.2021.656112 EDN: FFIXVV
  25. Kundisch A, Hönning A, Mutze S, et al. Deep learning algorithm in detecting intracranial hemorrhages on emergency computed tomographies. PLOS ONE. 2021;16(11):e0260560. doi: 10.1371/journal.pone.0260560 EDN: QPACKZ
  26. Del Gaizo AJ, Osborne TF, Shahoumian T, Sherrier R. Deep learning to detect intracranial hemorrhage in a national teleradiology program and the impact on interpretation time. Radiology: Artificial Intelligence. 2024;6(5):e240067. doi: 10.1148/ryai.240067 EDN: EHHAOO
  27. Pettet G, West J, Robert D, et al. A retrospective audit of an artificial intelligence software for the detection of intracranial haemorrhage used by a teleradiology company in the United Kingdom. BJR|Open. 2023;6(1):tzae033. doi: 10.1093/bjro/tzae033 EDN: DWNYCF
  28. Mäenpää SM, Korja M. Diagnostic test accuracy of externally validated convolutional neural network (CNN) artificial intelligence (AI) models for emergency head CT scans – A systematic review. International Journal of Medical Informatics. 2024;189:105523. doi: 10.1016/j.ijmedinf.2024.105523 EDN: HLVVYQ
  29. Eldaya RW, Kansagra AP, Zei M, et al. Performance of automated RAPID intracranial hemorrhage detection in real-world practice: a single-institution experience. Journal of Computer Assisted Tomography. 2022;46(5):770–774. doi: 10.1097/rct.0000000000001335 EDN: GRDZTF
  30. Schmitt N, Mokli Y, Weyland CS, et al. Automated detection and segmentation of intracranial hemorrhage suspect hyperdensities in non-contrast-enhanced CT scans of acute stroke patients. European Radiology. 2021;32(4):2246–2254. doi: 10.1007/s00330-021-08352-4 EDN: OLFWXI
  31. Warman R, Warman A, Warman P, et al. Deep learning system boosts radiologist detection of intracranial hemorrhage. Cureus. 2022;undefined:. doi: 10.7759/cureus.30264 EDN: IRZKDY
  32. Buchlak QD, Tang CHM, Seah JCY, et al. Effects of a comprehensive brain computed tomography deep learning model on radiologist detection accuracy. European Radiology. 2023;34(2):810–822. doi: 10.1007/s00330-023-10074-8 EDN: ZHIFOG
  33. Ngiam KY, Khor IW. Big data and machine learning algorithms for health-care delivery. The Lancet Oncology. 2019;20(5):e262–e273. doi: 10.1016/S1470-2045(19)30149-4
  34. Kiefer J, Kopp M, Ruettinger T, et al. Diagnostic accuracy and performance analysis of a scanner-integrated artificial intelligence model for the detection of intracranial hemorrhages in a traumatology emergency department. Bioengineering. 2023;10(12):1362. doi: 10.3390/bioengineering10121362 EDN: EPLIBY

Supplementary files

Supplementary Files
Action
1. JATS XML
Download
2. Fig. 1. Characteristic curves of calibration tests of the artificial intelligence service designed for automatic analysis of medical computed tomographic images of the brain for the presence of intracranial hemorrhages: a — first; b — second; c — third.

Download (195KB)
Indexing metadata
3. Fig. 2. Scheme of analysis of computed tomography studies of the brain during expert assessment under clinical monitoring conditions: ICH+ — presence of intracranial hemorrhages; ICH− — absence of intracranial hemorrhages; IP — true positive result; FP — false positive result; NF — false negative result; TN — true negative result.

Download (233KB)
Indexing metadata
4. Fig. 3. Dynamics of diagnostic metrics of the artificial intelligence service operation relative to the results of two calibration tests: abscissa axis — metric values; ordinate axis — months. The dotted line indicates the results of the metrics obtained during the calibration tests.

Download (289KB)
Indexing metadata
5. Fig. 4. Examples of partially correct responses of the artificial intelligence service: a — correct definition of the hemorrhage type and incorrect segmentation; b — correct segmentation of hemorrhage areas, erroneous definition of their type; c — partial selection of some hemorrhages and omitting others, incorrect both segmentation and type definition.

Download (246KB)
Indexing metadata
6. Fig. 5. Examples of false positive (a) and false negative (b) responses from the artificial intelligence service.

Download (213KB)
Indexing metadata

Copyright (c) 2025 Eco-Vector

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Согласие на обработку персональных данных с помощью сервиса «Яндекс.Метрика»

1. Я (далее – «Пользователь» или «Субъект персональных данных»), осуществляя использование сайта https://journals.rcsi.science/ (далее – «Сайт»), подтверждая свою полную дееспособность даю согласие на обработку персональных данных с использованием средств автоматизации Оператору - федеральному государственному бюджетному учреждению «Российский центр научной информации» (РЦНИ), далее – «Оператор», расположенному по адресу: 119991, г. Москва, Ленинский просп., д.32А, со следующими условиями.

2. Категории обрабатываемых данных: файлы «cookies» (куки-файлы). Файлы «cookie» – это небольшой текстовый файл, который веб-сервер может хранить в браузере Пользователя. Данные файлы веб-сервер загружает на устройство Пользователя при посещении им Сайта. При каждом следующем посещении Пользователем Сайта «cookie» файлы отправляются на Сайт Оператора. Данные файлы позволяют Сайту распознавать устройство Пользователя. Содержимое такого файла может как относиться, так и не относиться к персональным данным, в зависимости от того, содержит ли такой файл персональные данные или содержит обезличенные технические данные.

3. Цель обработки персональных данных: анализ пользовательской активности с помощью сервиса «Яндекс.Метрика».

4. Категории субъектов персональных данных: все Пользователи Сайта, которые дали согласие на обработку файлов «cookie».

5. Способы обработки: сбор, запись, систематизация, накопление, хранение, уточнение (обновление, изменение), извлечение, использование, передача (доступ, предоставление), блокирование, удаление, уничтожение персональных данных.

6. Срок обработки и хранения: до получения от Субъекта персональных данных требования о прекращении обработки/отзыва согласия.

7. Способ отзыва: заявление об отзыве в письменном виде путём его направления на адрес электронной почты Оператора: info@rcsi.science или путем письменного обращения по юридическому адресу: 119991, г. Москва, Ленинский просп., д.32А

8. Субъект персональных данных вправе запретить своему оборудованию прием этих данных или ограничить прием этих данных. При отказе от получения таких данных или при ограничении приема данных некоторые функции Сайта могут работать некорректно. Субъект персональных данных обязуется сам настроить свое оборудование таким способом, чтобы оно обеспечивало адекватный его желаниям режим работы и уровень защиты данных файлов «cookie», Оператор не предоставляет технологических и правовых консультаций на темы подобного характера.

9. Порядок уничтожения персональных данных при достижении цели их обработки или при наступлении иных законных оснований определяется Оператором в соответствии с законодательством Российской Федерации.

10. Я согласен/согласна квалифицировать в качестве своей простой электронной подписи под настоящим Согласием и под Политикой обработки персональных данных выполнение мною следующего действия на сайте: https://journals.rcsi.science/ нажатие мною на интерфейсе с текстом: «Сайт использует сервис «Яндекс.Метрика» (который использует файлы «cookie») на элемент с текстом «Принять и продолжить».