Analysis of the diagnostic and economic impact of the combined artificial intelligence algorithm for analysis of 10 pathological findings on chest computed tomography

Cover Image

Cite item

Full Text

Abstract

BACKGROUND: Artificial intelligence technology can help solve the significant problem of missed findings in radiology studies. An important issue is assessing the economic benefits of implementing artificial intelligence.

AIM: To evaluate the frequency of missed pathologies detection and the economic potential of artificial intelligence technology for chest computed tomography compared and validated by experienced radiologists.

MATERIALS AND METHODS: This was an observational, single-center retrospective study. The study included chest computed tomography without IV contrast from June 1 to July 31, 2022, in Clinical Hospital in Yauza, Moscow. The computed tomography was processed using a complex artificial intelligence algorithm for 10 pathologies: pulmonary infiltrates, typical for viral pneumonia (COVID-19 in pandemic conditions); lung nodules; pleural effusion; pulmonary emphysema; thoracic aortic dilatation; pulmonary trunk dilatation; coronary artery calcification; adrenal hyperplasia; and osteoporosis (vertebral body height and density changes). Two experts analyzed computed tomography and compared results with artificial intelligence. Further routing was determined according to clinical guidelines for all findings initially detected and missed by radiologists. The hospital price list determined the potential revenue loss for each patient.

RESULTS: From the final 160 computed tomographies, the artificial intelligence identified 90 studies (56%) with pathologies, of which 81 (51%) were missing at least one pathology in the report. The “second-stage” lost potential revenue for all pathologies from 81 patients was RUB 2,847,760 ($37,251 or CNY 256,218). Lost potential revenue only for those pathologies missed by radiologists but detected by artificial intelligence was RUB 2,065,360 ($27,017 or CNY 185,824).

CONCLUSION: Using artificial intelligence as an “assistant” to the radiologist for chest computed tomography can dramatically minimize the number of missed abnormalities. Compared with the normal model without artificial intelligence, using artificial intelligence can provide 3.6 times more benefits. Using advanced artificial intelligence for chest computed tomography can save money.

About the authors

Valeria Yu. Chernina

IRA Labs

Email: v.chernina@ira-labs.com
ORCID iD: 0000-0002-0302-293X
SPIN-code: 8896-8051
Scopus Author ID: 57210638679
ResearcherId: AAF-1215-2020
Russian Federation, Moscow

Mikhail G. Belyaev

IRA Labs

Email: belyaevmichel@gmail.com
ORCID iD: 0000-0001-9906-6453
SPIN-code: 2406-1772

Cand. Sci. (Phys.-Math.), Professor

Russian Federation, Moscow

Anton Yu. Silin

Clinical Hospital on Yauza

Email: silin@yamed.ru
ORCID iD: 0000-0003-4952-2347
SPIN-code: 4411-8745
Russian Federation, Moscow

Ivan O. Avetisov

Clinical Hospital on Yauza

Email: avetisov@yamed.ru
ORCID iD: 0009-0007-3550-7556
Russian Federation, Moscow

Ilya A. Pyatnitskiy

IRA Labs; The University of Texas at Austin

Email: i.pyatnitskiy@ira-labs.com
ORCID iD: 0000-0002-2827-1473
SPIN-code: 6150-4961
Russian Federation, Moscow; Austin, Texas, USA

Ekaterina A. Petrash

IRA Labs; N.N. Blokhin National Medical Research Center of Oncology

Email: e.a.petrash@gmail.com
ORCID iD: 0000-0001-6572-5369
SPIN-code: 6910-8890

MD, Cand. Sci. (Med.)

Russian Federation, Moscow; Moscow

Maria V. Basova

IRA Labs

Email: m.basova@ira-labs.com
ORCID iD: 0009-0000-3325-8452
Russian Federation, Moscow

Valentin E. Sinitsyn

Lomonosov Moscow State University; Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Email: vsini@mail.ru
ORCID iD: 0000-0002-5649-2193
SPIN-code: 8449-6590

MD, Dr. Sci. (Med.), Professor

Russian Federation, Moscow; Moscow

Vitaly V. Omelyanovskiy

The Center for Healthcare Quality Assessment and Control; Russian Medical Academy of Continuous Professional Education; Scientific and research financial institute

Email: vvo@rosmedex.ru
ORCID iD: 0000-0003-1581-0703
SPIN-code: 1776-4270

MD, Dr. Sci. (Med.), Professor

Russian Federation, Moscow; Moscow; Moscow

Victor A. Gombolevskiy

IRA Labs; Artificial Intelligence Research Institute

Author for correspondence.
Email: gombolevskii@gmail.com
ORCID iD: 0000-0003-1816-1315
SPIN-code: 6810-3279

MD, Cand. Sci. (Med.)

Russian Federation, Moscow; Moscow

References

  1. Aberle DR, Adams AM, Berg CD, et al.; National Lung Screening Trial Research Team. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med. 2011;365(5):395–409. doi: 10.1056/NEJMoa1102873
  2. Pastorino U, Silva M, Sestini S, et al. Prolonged lung cancer screening reduced 10-year mortality in the MILD trial: new confirmation of lung cancer screening efficacy. Ann Oncol. 2019;30(7):1162–1169. doi: 10.1093/annonc/mdz117
  3. Grover H, King W, Bhattarai N, et al. Systematic review of the cost-effectiveness of screening for lung cancer with low dose computed tomography. Lung Cancer. 2022;170:20–33. doi: 10.1016/j.lungcan.2022.05.005
  4. Nikolaev AE, Gombolevskiy VA, Gonchar AP, et al. Incidental findings during lung cancer screening by low-dose computed tomography. Tuberculosis Lung Dis. 2018;96(11):60–67. (In Russ). doi: 10.21292/2075-1230-2018-96-11-60-67
  5. Kilsdonk ID, de Roos MP, Bresser P, et al. Frequency and spectrum of incidental findings when using chest CT as a primary triage tool for COVID-19. Eur J Radiol Open. 2021;(8):100366. doi: 10.1016/j.ejro.2021.100366
  6. Beltsevich DG, Melnichenko GA, Kuznetsov NS, et al. Russian Association of Endocrinologists clinical practice guideline for adrenal incidentalomas differential diagnosis. Endocrine Sur. 2016;10(4):31–42. (In Russ). doi: 10.14341/serg2016431-42
  7. Emphysema of the lungs. Clinical recommendations (approved by the Ministry of Health of the Russian Federation, 2021). (In Russ). Available from: https://www.garant.ru/products/ipo/prime/doc/402775957/. Accessed: 25.08.2021.
  8. Thoracic and thoracoabdominal aortic aneurysms. Clinical recommendations (approved by the Ministry of Health of the Russian Federation, 2023). (In Russ). Available from: https://www.angiolsurgery.org/library/recommendations/2022/aneurysms_thoracic_thoracoabdominal_aorta/recommendation.pdf. Accessed: 25.08.2021.
  9. Vasiliev YuA, Vladzimirsky AV, Arzamasov KM, et al. Computer vision in radiation diagnostics: the first stage of the Moscow experiment. Monograph. 2nd ed., revised and updated. Ed. by Yu.A. Vasiliev, A.V. Vladzimirsky. Moscow: Izdatel’skie resheniya; 2023. 376 p. (In Russ).
  10. Zakharov A, Pisov M, Bukharaev A, et al. Interpretable vertebral fracture quantification via anchor-free landmarks localization. Med Image Anal. 2023;(83):102646. doi: 10.1016/j.media.2022.102646
  11. Goncharov M, Pisov M, Shevtsov A, et al. CT-Based COVID-19 triage: Deep multitask learning improves joint identification and severity quantification. Med Image Anal. 2021;(71):102054. doi: 10.1016/j.media.2021.102054
  12. Shirokikh B, Shevtsov A, Dalechina A, et al. Accelerating 3D medical image segmentation by adaptive small-scale target localization. J Imaging. 2021;7(2):35. doi: 10.3390/jimaging7020035
  13. Goncharov M, Pisov M, Belyaev M, et al. Quantification of epicardial adipose tissue in low-dose computed tomography images. Lecture Notes Electrical Engineering. 2022;784(1):98–107. doi: 10.1007/978-981-16-3880-0_11
  14. Husereau D, Drummond M, Augustovski F, et al.; CHEERS 2022 ISPOR Good Research Practices Task Force. Consolidated health economic evaluation reporting standards 2022 (cheers 2022) statement: Updated reporting guidance for health economic evaluations. Value Health. 2022;25(1):3–9. doi: 10.1016/j.jval.2021.11.1351
  15. Morozov SP, Vladzimirsky AV, Kljashtorny VG, et al. Clinical trials of software based on intelligent technologies (radiology diagnostics). Preprint № CDT-2019-1. Best Practice in Radiation Oncology and Instrumental Diagnostics. Vol. 23. Moscow: Center for Diagnostics and Telemedicine; 2019. 33 р. (In Russ).
  16. Gombolevskiy VA, Blokhin IA, Laipan AS, et al. Methodological guidelines for lung cancer screening. Best Practice in Radiation and Instrumental Diagnostics. Vol. 56. Moscow: Center for Diagnostics and Telemedicine; 2020. 57 р. (In Russ).
  17. Morozov SP, Protsenko DN, Smetanina SV, et al. Radiation diagnosis of coronavirus disease (COVID-19): Organization, methodology, interpretation of results: Guidelines. Best Practices in Radiation and Instrumental Diagnostics Series. Vol. 65. Moscow: Center for Diagnostics and Telemedicine; 2020. 78 р. (In Russ).
  18. Temporary methodological recommendations. Prevention, diagnosis and treatment of new coronavirus infection (COVID-19). Ministry of Health of the Russian Federation. Version 17 (12/14/2022). (In Russ). Available from: https://static-0.minzdrav.gov.ru/system/attachments/attaches/000/061/254/original/%D0%92%D0%9C%D0%A0_COVID-19_V17.pdf?1671088207. Accessed: 17.03.2023.
  19. Lynch DA, Austin JH, Hogg JC, et al. CT-definable subtypes of chronic obstructive pulmonary disease: A statement of the Fleischner Society. Radiology. 2015;277(1):192–205. doi: 10.1148/radiol.2015141579
  20. Hersh CP, Washko GR, Estépar RS, et al. Paired inspiratory-expiratory chest CT scans to assess for small airways disease in COPD. Respir Res. 2013;146(1):42. doi: 10.1186/1465-9921-14-42
  21. Walker C, Chung JH. Muller’s imaging of the chest e-book: Expert radiology series. Elsevier Health Sciences; 2018.1104 р.
  22. Erbel R, Aboyans V, Boileau C et al. ESC Committee for Practice Guidelines. 2014 ESC Guidelines on the diagnosis and treatment of aortic diseases: Document covering acute and chronic aortic diseases of the thoracic and abdominal aorta of the adult. The task force for the diagnosis and treatment of aortic diseases of the European society of cardiology (ESC). Eur Heart J. 2014;35(41):2873–2926. doi: 10.1093/eurheartj/ehu281
  23. Wanhainen A, Verzini F, Van Herzeele I, et al. Clinical practice guidelines on the management of abdominal aorto-iliac artery aneurysms. Eur J Vascul Endovascul Sur. 2019;57(1):8–93. doi: 10.1016/j.ejvs.2018.09.020
  24. Jurgens PT, Carr JJ, Terry JG, et al. Association of abdominal aorta calcium and coronary artery calcium with incident cardiovascular and coronary heart disease events in black and white middle-aged people: The coronary artery risk development in young adults study. J Am Heart Assoc. 2021;10(24):23–37. doi: 10.1161/JAHA.121.023037
  25. Galiè N, Humbert M, Vachiery JL, et al. 2015 ESC/ERS Guidelines for the diagnosis and treatment of pulmonary hypertension: The joint task force for the diagnosis and treatment of pulmonary hypertension of the European society of cardiology (ESC) and the European respiratory society (ERS): Endorsed by: Association for European paediatric and congenital cardiology (AEPC), international society for heart and lung transplantation (ISHLT). Eur Respir J. 2015;46(4):903–975. doi: 10.1183/13993003.01032-2015
  26. Agatston AS, Janowitz WR, Hildner FJ et al. Quantification of coronary artery calcium using ultrafast computed tomography. J Am Coll Cardiol. 1990;15(4):827–832. doi: 10.1016/0735-1097(90)90282-t
  27. Zhuravlev KN. Coronary CT angiography. Best practices in radiation and instrumental diagnostics series. Vol. 65. Moscow: Center for Diagnostics and Telemedicine; 2020. 35 р. (In Russ).
  28. Mayo-Smith WW, Song JH, Boland GL, et al. Management of incidental adrenal masses: A white paper of the ACR incidental findings committee. J Am Coll Radiol. 2017;14(8):1038–1044. doi: 10.1016/j.jacr.2017.05.001
  29. Belaya ZE, Belova KYu, Biryukova EV, et al. Federal clinical guidelines for diagnosis, treatment and prevention of osteoporosis. Osteoporosis Bone Diseases. 2021;24(2):4–47. (In Russ). doi: 10.14341/osteo12930
  30. Genant HK, Jergas M. Assessment of prevalent and incident vertebral fractures in osteoporosis research. Osteoporosis Int. 2003;14(3):43–55. doi: 10.1007/S00198-002-1348-1
  31. Pathological fractures complicating osteoporosis. Clinical recommendations. Moscow; 2018. 90 p. (In Russ). Available from: https://neirodoc.ru/wp-content/uploads/2019/10/patologicheskie-perelomy-oslozhnyayuschie-osteoporoz-2018.pdf. Accessed: 17.03.2023.
  32. Kolbin AS, Gomon YuM, Balykina YuE, et al. Socioeconomic and global burden of COVID-19. Good Clin Pract. 2021;(1):24–34. (In Russ). doi: 10.37489/2588-0519-2021-1-24-34
  33. Pickhardt PJ, Correale L, Hassan C. AI-based opportunistic CT screening of incidental cardiovascular disease, osteoporosis, and sarcopenia: Cost-effectiveness analysis. Abdom Radiol (NY). 2023;(3):1181–1198. doi: 10.1007/s00261-023-03800-9
  34. Behr CM, Koffijberg H, Degeling K, et al. Can we increase efficiency of CT lung cancer screening by combining with CVD and COPD screening? Results of an early economic evaluation. Eur Radiol. 2022;32(5):3067–3075. doi: 10.1007/s00330-021-08422-7
  35. Chintanapakdee W, Mendoza DP, Zhang EW, et al. Detection of extrapulmonary malignancy during lung cancer screening: 5-year analysis at a tertiary hospital. J Am Coll Radiol. 2020;17(12):1609–1620. doi: 10.1016/j.jacr.2020.09.032
  36. Girling A, Lilford R, Cole A, et al. Headroom approach to device development: Current and future directions. Int J Technol Assess Health Care. 2015;31(5):331–338. doi: 10.1017/S0266462315000501
  37. Ijzerman MJ, Steuten LM. Early assessment of medical technologies to inform product development and market access: A review of methods and applications. Appl Health Econ Health Policy. 2011;9(5):331–347. doi: 10.2165/11593380-000000000-00000
  38. Izerman MJ, Koffijberg H, Fenwick E, et al. Emerging use of early health technology assessment in medical product development: A scoping review of the literature. Pharmacoeconomics. 2017;35(7):727–740. doi: 10.1007/s40273-017-0509-1
  39. Sahiner B, Pezeshk A, Hadjiiski LM, et al. Deep learning in medical imaging and radiation therapy. Med Phys. 2019;46(1):1–36. doi: 10.1002/mp.13264
  40. Allen BJ, Seltzer SE, Langlotz CP, et al. A road map for translational research on artificial intelligence in medical imaging: From the 2018 national institutes of health/RSNA/ACR/The academy workshop. J Am Coll Radiol. 2019;16(9):1179–1189. doi: 10.1016/j.jacr.2019.04.014
  41. Xia C, Rook M, Pelgrim GJ, et al. Early imaging biomarkers of lung cancer, COPD and coronary artery disease in the general population: rationale and design of the ImaLife (Imaging in Lifelines) Study. Eur J Epidemiol. 2020;35(1):75–86. doi: 10.1007/s10654-019-00519-0
  42. Ziegelmayer S, Graf M, Makowski M, et al. Cost-Effectiveness of artificial intelligence support in computed tomography-based lung cancer screening. Cancers (Basel). 2022;14(7):1729. doi: 10.3390/cancers14071729
  43. McDonald RJ, Schwartz KM, Eckel LJ, et al. The effects of changes in utilization and technological advancements of cross-sectional imaging on radiologist workload. Acad Radiol. 2015;22(9):1191–1198. doi: 10.1016/j.acra.2015.05.007
  44. Sokolovskaya E, Shinde T, Ruchman RB, et al. The effect of faster reporting speed for imaging studies on the number of misses and interpretation errors: A pilot study. J Am Coll Radiol. 2015;12(7):683–688. doi: 10.1016/j.jacr.2015.03.040
  45. Geijer H, Geijer M. Added value of double reading in diagnostic radiology, a systematic review. Insights Imaging. 2018;9(3):287–301. doi: 10.1007/s13244-018-0599-0
  46. Lauritzen PM, Hurlen P, Sandbæk G, et al. Double reading rates and quality assurance practices in Norwegian hospital radiology departments: Two parallel national surveys. Acta Radiol. 2015;56(1):78–86. doi: 10.1177/0284185113519988
  47. Morozov SP, Chernina VYu, Andreychenko AE, et al. How does artificial intelligence effect on the assessment of lung damage in COVID-19 on chest CT scan? Digital Diagnostics. 2020;2(1):27−38. (In Russ). doi: 10.17816/DD60040

Supplementary files

Supplementary Files
Action
1. JATS XML
Download
2. Fig. 1. Study design.

Download (395KB)
Indexing metadata
3. Fig. 2. Study result by the number of findings detected with and without AI.

Download (161KB)
Indexing metadata
4. Fig. 3. Number of findings (ranked by the number of significant missed pathological findings).

Download (210KB)
Indexing metadata
5. Fig. 4. Analysis of the cost of medical services not provided because of missed pathological findings, and all CT scans performed.

Download (319KB)
Indexing metadata
6. Fig. 5. Range of costs of medical services not provided due to the use of the combined AI service for chest CT scans in a clinic. AI, artificial intelligence; CT, computed tomography; CNMS, cost of non-provided medical services.

Download (286KB)
Indexing metadata
7. Fig. 6. An example of AI use. Patient B, 76 years old. A radiologist correctly identified bilateral hydrothorax and emphysematous changes but did not describe the lung nodule in the right lung. An AI algorithm revealed all three pathological findings: hydrothorax is highlighted with a yellow line, emphysematous changes are highlighted in orange, and the lung nodule is indicated by a red square.

Download (196KB)
Indexing metadata
8. Fig. 7. An example of AI use. Patient B., 79 years old. Chest CT scans: a) axial section: a radiologist and an algorithm correctly identified a lung nodule in the left lung (indicated by a red square) and coronary calcification (outlined by an orange line). In addition, the algorithm indicated an increase in the volume of epicardial fat (filled in yellow; this pathological finding was not considered in the study); b) sagittal section: a radiologist and an algorithm correctly identified compression fractures of Th6 and Th9 vertebral bodies, Genant 3 (three columns are marked with red lines); however, the radiologist did not indicate deformities of Th5 and Th12 vertebral bodies, Genant 2 (three columns are marked with yellow lines) in the protocol.

Download (263KB)
Indexing metadata
9. Fig. 8. Potential cost of medical services not provided because a combined AI service was not used for chest CT scans in a clinic, considering the cost of using AI.

Download (242KB)
Indexing metadata

Copyright (c) 2023 Eco-Vector

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

This website uses cookies

You consent to our cookies if you continue to use our website.

About Cookies