Experience with artificial intelligence algorithms for the diagnosis of vertebral compression fractures based on computed tomography: from testing to practical evaluation

Cover Image

Cite item

Full Text

Abstract

BACKGROUND: Osteoporosis is often diagnosed at the stage with complications, i.e., low-energy fractures. Vertebral compression fractures, which are complications of osteoporosis and predictors of subsequent fractures, are often asymptomatic. Compression fractures can be found by computed tomography performed for other indications with vertebral morphometry. Approaches to using artificial intelligence algorithms designed for diagnosing vertebral compression fractures were analyzed.

AIM: Testing artificial intelligence algorithms to conduct morphometric analysis of vertebrae on chest computed tomography scans and assess the possibility of their implementation in medical organizations of the Moscow Healthcare Department.

MATERIALS AND METHODS: To set a clinical task for artificial intelligence algorithms, basic diagnostic requirements in the area of “vertebral compression fractures (osteoporosis)” were formulated. The testing of the artificial intelligence algorithms included the following stages: self-testing, functional and calibration testing, practical evaluation, and operation testing. The first three stages of testing were performed using previously generated datasets. At practical evaluation and operation testing, artificial intelligence algorithms analyzed the data from computed tomography performed in medical organizations. The expert group of radiologists assessed the diagnostic accuracy and functional capacity of the AI algorithms at all stages. The resulting quantitative metrics of the accuracy of artificial intelligence algorithms were compared with the required target values.

RESULTS: From June 2021 to June 2022, two artificial intelligence algorithms (Nos. 1 and 2) with different methods of detecting compression fractures were tested. Both artificial intelligence algorithms successfully passed the self-testing (6 tests), functional (5 tests), and calibration (100 tests) stages. The area under the ROC curve for artificial intelligence algorithm No. 1 was 0.99 (95% CI, 0.98–1), and for artificial intelligence algorithm No. 2, it was 0.91 (95% CI, 0.85–0.96). Artificial intelligence algorithm No. 1 passed the practical evaluation stage without any significant remarks, whereas algorithm No. 2 was sent for fine-tuning. After the operation testing stage, the following accuracy metrics were obtained: the areas under the ROC curve for artificial intelligence algorithm Nos. 1 and 2 were 0.93 (95% CI, 0.89–0.96) and 0.92 (95% CI, 0.90–0.94), respectively. At all stages, both artificial intelligence algorithms demonstrated sufficient metrics for clinical validation.

CONCLUSION: Artificial intelligence algorithms for the automatic diagnosis of vertebral compression fractures have been tested, demonstrating the high quality of their operation. artificial intelligence algorithms can be applied as a supplementary tool in the medical decision support system.

About the authors

Zlata R. Artyukova

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Author for correspondence.
Email: zl.artyukova@gmail.com
ORCID iD: 0000-0003-2960-9787
SPIN-code: 7550-2441
Russian Federation, Moscow

Alexey V. Petraikin

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Email: alexeypetraikin@gmail.com
ORCID iD: 0000-0003-1694-4682
SPIN-code: 6193-1656

MD, Dr. Sci. (Medicine), Assistant Professor

Russian Federation, Moscow

Nikita D. Kudryavtsev

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Email: KudryavtsevND@zdrav.mos.ru
ORCID iD: 0000-0003-4203-0630
SPIN-code: 1125-8637
Russian Federation, Moscow

Fedor A. Petryaykin

Lomonosov Moscow State University

Email: feda.petraykin@gmail.com
ORCID iD: 0000-0001-6923-3839
SPIN-code: 7803-1005
Russian Federation, Moscow

Dmitry S. Semenov

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Email: semenovds4@zdrav.mos.ru
ORCID iD: 0000-0002-4293-2514
SPIN-code: 2278-7290

Cand. Sci. (Engineering)

Russian Federation, Moscow

Daria E. Sharova

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Email: SharovaDE@zdrav.mos.ru
ORCID iD: 0000-0001-5792-3912
SPIN-code: 1811-7595
Russian Federation, Moscow

Zhanna E. Belaya

Endocrinology Research Centre

Email: jannabelaya@gmail.com
ORCID iD: 0000-0002-6674-6441
SPIN-code: 4746-7173

MD, Dr. Sci. (Medicine)

Russian Federation, Moscow

Anton V. Vladzimirskyy

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies; The First Sechenov Moscow State Medical University

Email: VladzimirskijAV@zdrav.mos.ru
ORCID iD: 0000-0002-2990-7736
SPIN-code: 3602-7120

MD, Dr. Sci. (Medicine)

Russian Federation, Moscow; Moscow

Yuriy A. Vasilev

Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Email: VasilevYA1@zdrav.mos.ru
ORCID iD: 0000-0002-0208-5218
SPIN-code: 4458-5608

MD, Cand. Sci. (Medicine)

Russian Federation, Moscow

References

  1. Belaya ZhE, Belova KYu, Biryukova EV, et al. Federal clinical guidelines for diagnosis, treatment and prevention of osteoporosis. Osteoporosis and Bone Diseases. 2021;24(2):4–47. doi: 10.14341/osteo12930
  2. Petraikin A, Artyukova Z, Nisovtsova LA, et al. Analysis of the effectiveness of implementing screening of osteoporosis. Manager Zdravoochranenia. 2021;2:31–39. doi: 10.21045/1811-0185-2021-2-31-39
  3. Alacreu E, Moratal D, Arana E. Opportunistic screening for osteoporosis by routine CT in Southern Europe. Osteoporosis International. 2017;28(3):983–990. doi: 10.1007/s00198-016-3804-3
  4. Ziemlewicz TJ, Binkley N, Pickhardt PJ. Opportunistic Osteoporosis Screening: Addition of Quantitative CT Bone Mineral Density Evaluation to CT Colonography. Journal of the American College of Radiology. 2015;12(10):1036–1041. doi: 10.1016/j.jacr.2015.04.018
  5. Rebello D, Anjelly D, Grand DJ, et al. Opportunistic screening for bone disease using abdominal CT scans obtained for other reasons in newly diagnosed IBD patients. Osteoporosis international. 2018;29(6):1359–1366. doi: 10.1007/s00198-018-4444-6
  6. Artyukova ZR, Kudryavtsev ND, Petraikin AV, et al. Using an artificial intelligence algorithm to assess the bone mineral density of the vertebral bodies based on computed tomography data. Medical Visualization. 2023;27(2):125–137. doi: 10.24835/1607-0763-1257
  7. Jang S, Graffy PM, Ziemlewicz TJ, et al. Opportunistic osteoporosis screening at routine abdominal and Thoracic CT: Normative L1 trabecular attenuation values in more than 20 000 adults. Radiology. 2019;291(2):360–367. doi: 10.1148/radiol.2019181648
  8. Smets J, Shevroja E, Hügle T, et al. Machine Learning Solutions for Osteoporosis-A Review. J Bone Miner Res. 2021;36(5):833–851. doi: 10.1002/jbmr.4292
  9. Petraikin AV, Skripnikova IA. Quantitative Computed Tomography, modern data. Review. Medical Visualization. 2021;25(4):134–146. doi: 10.24835/1607-0763-1049
  10. Lenchik L, Rogers LF, Delmas PD, et al. Diagnosis of Osteoporotic Vertebral Fractures: Importance of Recognition and Description by Radiologists. American Journal of Roentgenology. 2004;183(4):949–958. doi: 10.2214/ajr.183.4.1830949
  11. Pinto A, Berritto D, Russo A, et al. Traumatic fractures in adults: Missed diagnosis on plain radiographs in the Emergency Department. Acta Biomedica. 2018;89:111–123. doi: 10.23750/abm.v89i1-S.7015
  12. Carberry GA, Pooler BD, Binkley N, et al. Unreported vertebral body compression fractures at abdominal multidetector CT. Radiology. 2013;268(1):120–126. doi: 10.1148/radiol.13121632
  13. Vladzimirskii AV, Vasil’ev YuA, Arzamasov KM, et al. Computer Vision in Radiologic Diagnostics: The First Stage of the Moscow Experiment: Monograph. 2nd edition, revised and supplemented. Moscow: Izdatel’skie resheniya; 2023. (In Russ.) EDN: FOYLXK
  14. Genant HK, Wu CY, Cornelis van K, et al. Vertebral fracture assessment using a semiquantitative technique. Journal of Bone and Mineral Research. 1993;8(9):1137–1148. doi: 10.1002/jbmr.5650080915
  15. Mosmed.ai [Internet]. State Budgetary Institution of Healthcare of the City of Moscow “Scientific and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Department of Healthcare of the City of Moscow” [cited 2024 Mar 14]. (In Russ.)Available from: https://mosmed.ai/
  16. Clinical guidelines. Osteoporosis. [Internet]. Ministry of Health of the Russian Federation. [cited 2023 Oct 24]. Available from: https://cr.minzdrav.gov.ru/schema/87_4
  17. The Adult Official Positions of the ISCD [Internet]. The International Society For Clinical Densitometry [cited 2023 Oct 24]. Available from: https://iscd.org/official-positions-2023/
  18. ACR–SPR–SSR practice parameter for the performance of quantitative computed tomography (QCT) bone mineral density [Internet]. American College of Radiology [cited 2023 Oct 24]. Available from: https://www.acr.org/-/media/ACR/Files/Practice-Parameters/qct.pdf
  19. Certificate of the Russian Federation on state registration of the database № 2023621171/ 11.04.2023. Vasil’ev YuA, Turavilova EV, Vladzimirskii AV, et al. MosMedData: CT scan with signs of osteoporosis of the spine. Available from: https://www.elibrary.ru/download/elibrary_52123357_73775308.PDF [cited 2023 Oct 23]. (In Russ.) EDN: SHLWTC
  20. Pisov M, Kondratenko V, Zakharov A, et al. Keypoints Localization for Joint Vertebra Detection and Fracture Severity Quantification. In: Martel AL, et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science. Vol. 12266. Springer; 2020. P:723–732. doi: 10.1007/978-3-030-59725-2_70
  21. Bar A, Wolf BL, Orna A, et al. Compression fractures detection on CT. Medical Imaging 2017: Computer-Aided Diagnosis. 2017;10134:1013440. doi: 10.48550/arXiv.1706.01671
  22. Lesnyak O, Baranova I, Belova K, et al. Osteoporosis in Russian Federation: epidemiology, socio-medical and economical aspects (review). Traumatology and Orthopedics of Russia. 2018;24(1):155–168. doi: 10.21823/2311-2905-2018-24-1-155-168
  23. Seo JW, Lim SH, Jeong JG, et al. A deep learning algorithm for automated measurement of vertebral body compression from X-ray images. Sci Rep. 2021;11(1):13732. doi: 10.1038/s41598-021-93017-x
  24. Murata K, Endo K, Aihara T, et al. Artificial intelligence for the detection of vertebral fractures on plain spinal radiography. Sci Rep. 2020;10(1):20031. doi: 10.1038/s41598-020-76866-w
  25. Dong Q, Luo G, Lane NE, et al. Deep Learning Classification of Spinal Osteoporotic Compression Fractures on Radiographs using an Adaptation of the Genant Semiquantitative Criteria. Acad Radiol. 2022;29(12):1819–1832. doi: 10.1016/j.acra.2022.02.020
  26. Tomita N, Cheung YY, Hassanpour S. Deep neural networks for automatic detection of osteoporotic vertebral fractures on CT scans. Computers in Biology and Medicine. 2018;98:8–15. doi: 1016/j.compbiomed.2018.05.011
  27. Valentinitsch A, Trebeschi S, Kaesmacher J, et al. Opportunistic osteoporosis screening in multi-detector CT images via local classification of textures. Osteoporosis International. 2019;30(6):1275–1285. doi: 10.1007/s00198-019-04910-1
  28. Yasaka K, Akai H, Kunimatsu A, et al. Prediction of bone mineral density from computed tomography: application of deep learning with a convolutional neural network. Eur Radiol. 2020;30(6):3549–3557. doi: 10.1007/s00330-020-06677-0
  29. Nam KH, Seo I, Kim DH, et al. Machine Learning Model to Predict Osteoporotic Spine with Hounsfield Units on Lumbar Computed Tomography. J Korean Neurosurg Soc. 2019;62(4):442–449. doi: 10.3340/jkns.2018.0178
  30. Zhang J, Liu F, Xu J, et al. Qingqing. Automated detection and classification of acute vertebral body fractures using a convolutional neural network on computed tomography. Frontiers in Endocrinology. 2023;14(1132725):1–10. doi: 10.3389/fendo.2023.1132725
  31. Pickhardt PJ, Dustin PB, Travisи L, et al. Opportunistic Screening for Osteoporosis Using Abdominal Computed Tomography Scans Obtained for Other Indications. Annals of internal medicine. 2013;158(8):588. doi: 10.7326/0003-4819-158-8-201304160-00003
  32. Del Lama RS, Candido RM, Chiari-Correia NS, et al. Computer-Aided Diagnosis of Vertebral Compression Fractures Using Convolutional Neural Networks and Radiomics. J Digit Imaging. 2022;35(3):446–458. doi: 10.1007/s10278-022-00586-y
  33. Morozov SP, Gavrilov AV, Arkhipov IV, et al. Effect of artificial intelligence technologies on the CT scan interpreting time in COVID-19 patients in inpatient setting. Russian Journal of Preventive Medicine. 2022;25(1):14–20. doi: 10.17116/profmed20222501114
  34. Vladzymyrskyy AV, Kudryavtsev ND, Kozhikhina DD, et al. Effectiveness of using artificial intelligence technologies for dual descriptions of the results of preventive lung examinations. Russian Journal of Preventive Medicine. 2022;25(7):7–15. doi: 10.17116/profmed2022250717
  35. Shelepa AA, Petraikin AV, Artyukova ZR, et al. Artificial intelligence for bone mineral density assessment: general population data. Digital Diagnostics. 2022;3(S1):23–24. doi: 10.17816/DD10571

Supplementary files

Supplementary Files
Action
1. JATS XML
Download
2. Fig. 1. Sequence of passing the artificial intelligence service to the stage of trial operation. ERIS EMIAS — Unified radiological information service "Unified medical information and analytical system"; AI-service — artificial intelligence service; CT OGK — computed tomography of the chest organs.

Download (151KB)
Indexing metadata
3. Fig. 2. An example of the Genant-IRA service: an additional curvilinear reconstructed series of a computed tomography study with marking of the target pathology - a compression fracture of the ThXII vertebral body.

Download (138KB)
Indexing metadata
4. Fig. 3. Examples of the HealthVCF service: an additional series of computed tomography studies with marking of the target pathology - compression fracture.

Download (307KB)
Indexing metadata
5. Fig. 4. Results of calculating accuracy metrics with 95% confidence interval: a — Genant-IRA service; b — HealthVCF service.

Download (252KB)
Indexing metadata
6. Fig. 5. ROC curve for determining compression fractures: a — Genant-IRA service (360 studies); b — HealthVCF service (520 studies).

Download (233KB)
Indexing metadata
7. Fig. 6. Examples of errors in the Genant-IRA service: a — false positive result: the service marked the calcified intervertebral disc ThXI–ThXII as the vertebral body ThXII with compression deformation >40% (Genant 3); b — false positive result: the service marked the pronounced osteophyte LI as the vertebral body ThXII with compression deformation >40% (Genant 3), while the vertebral body ThXII is not marked; c — false positive result: a patient with severe scoliosis experienced a critical failure of the algorithm (the so-called algorithm breakdown), a failure in constructing the curvilinear reconstruction, and, as a result, incorrect marking of the vertebrae and assessment of the degree of their compression deformation; d — false positive result: erroneous marking and compression deformation of >40% (Genant 3) of the "bodies" of the ThVII and ThVIII vertebrae was detected due to pronounced ring artefacts due to a malfunction of the computer tomography scanner detector. Data on the scanner defect were forwarded to the technical service of the Moscow City Health Department.

Download (265KB)
Indexing metadata
8. Fig. 7. Examples of errors in the HealthVCF service: a — false positive result and incorrect localization assessment: no compression deformation >25%; b — false negative result (most likely due to severe kyphosis in the patient): the service did not note compression deformation of the vertebral bodies >25% (ThIV, ThV, ThVI, ThVII, LII), notes were made by an expert (red frames); c — false positive result: no compression deformation >25% of the ThIX vertebral body with Schmorl's node; d — false positive result: the service erroneously identified compression deformation >25% of the ThVIII vertebral body due to severe "ring artefacts" caused by a malfunction of the CT scanner detector. Data on the scanner defect were forwarded to the technical service of the Moscow City Health Department.

Download (237KB)
Indexing metadata

Copyright (c) 2024 Eco-Vector

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Согласие на обработку персональных данных с помощью сервиса «Яндекс.Метрика»

1. Я (далее – «Пользователь» или «Субъект персональных данных»), осуществляя использование сайта https://journals.rcsi.science/ (далее – «Сайт»), подтверждая свою полную дееспособность даю согласие на обработку персональных данных с использованием средств автоматизации Оператору - федеральному государственному бюджетному учреждению «Российский центр научной информации» (РЦНИ), далее – «Оператор», расположенному по адресу: 119991, г. Москва, Ленинский просп., д.32А, со следующими условиями.

2. Категории обрабатываемых данных: файлы «cookies» (куки-файлы). Файлы «cookie» – это небольшой текстовый файл, который веб-сервер может хранить в браузере Пользователя. Данные файлы веб-сервер загружает на устройство Пользователя при посещении им Сайта. При каждом следующем посещении Пользователем Сайта «cookie» файлы отправляются на Сайт Оператора. Данные файлы позволяют Сайту распознавать устройство Пользователя. Содержимое такого файла может как относиться, так и не относиться к персональным данным, в зависимости от того, содержит ли такой файл персональные данные или содержит обезличенные технические данные.

3. Цель обработки персональных данных: анализ пользовательской активности с помощью сервиса «Яндекс.Метрика».

4. Категории субъектов персональных данных: все Пользователи Сайта, которые дали согласие на обработку файлов «cookie».

5. Способы обработки: сбор, запись, систематизация, накопление, хранение, уточнение (обновление, изменение), извлечение, использование, передача (доступ, предоставление), блокирование, удаление, уничтожение персональных данных.

6. Срок обработки и хранения: до получения от Субъекта персональных данных требования о прекращении обработки/отзыва согласия.

7. Способ отзыва: заявление об отзыве в письменном виде путём его направления на адрес электронной почты Оператора: info@rcsi.science или путем письменного обращения по юридическому адресу: 119991, г. Москва, Ленинский просп., д.32А

8. Субъект персональных данных вправе запретить своему оборудованию прием этих данных или ограничить прием этих данных. При отказе от получения таких данных или при ограничении приема данных некоторые функции Сайта могут работать некорректно. Субъект персональных данных обязуется сам настроить свое оборудование таким способом, чтобы оно обеспечивало адекватный его желаниям режим работы и уровень защиты данных файлов «cookie», Оператор не предоставляет технологических и правовых консультаций на темы подобного характера.

9. Порядок уничтожения персональных данных при достижении цели их обработки или при наступлении иных законных оснований определяется Оператором в соответствии с законодательством Российской Федерации.

10. Я согласен/согласна квалифицировать в качестве своей простой электронной подписи под настоящим Согласием и под Политикой обработки персональных данных выполнение мною следующего действия на сайте: https://journals.rcsi.science/ нажатие мною на интерфейсе с текстом: «Сайт использует сервис «Яндекс.Метрика» (который использует файлы «cookie») на элемент с текстом «Принять и продолжить».