Analysis of the diagnostic and economic impact of the combined artificial intelligence algorithm for analysis of 10 pathological findings on chest computed tomography
- Authors: Chernina V.Y.1, Belyaev M.G.1, Silin A.Y.2, Avetisov I.O.2, Pyatnitskiy I.A.1,3, Petrash E.A.1,4, Basova M.V.1, Sinitsyn V.E.5,6, Omelyanovskiy V.V.7,8,9, Gombolevskiy V.A.1,10
-
Affiliations:
- IRA Labs
- Clinical Hospital on Yauza
- The University of Texas at Austin
- N.N. Blokhin National Medical Research Center of Oncology
- Lomonosov Moscow State University
- Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies
- The Center for Healthcare Quality Assessment and Control
- Russian Medical Academy of Continuous Professional Education
- Scientific and research financial institute
- Artificial Intelligence Research Institute
- Issue: Vol 4, No 2 (2023)
- Pages: 105-132
- Section: Original Study Articles
- URL: https://journals.rcsi.science/DD/article/view/146880
- DOI: https://doi.org/10.17816/DD321963
- ID: 146880
Cite item
Abstract
BACKGROUND: Artificial intelligence technology can help solve the significant problem of missed findings in radiology studies. An important issue is assessing the economic benefits of implementing artificial intelligence.
AIM: To evaluate the frequency of missed pathologies detection and the economic potential of artificial intelligence technology for chest computed tomography compared and validated by experienced radiologists.
MATERIALS AND METHODS: This was an observational, single-center retrospective study. The study included chest computed tomography without IV contrast from June 1 to July 31, 2022, in Clinical Hospital in Yauza, Moscow. The computed tomography was processed using a complex artificial intelligence algorithm for 10 pathologies: pulmonary infiltrates, typical for viral pneumonia (COVID-19 in pandemic conditions); lung nodules; pleural effusion; pulmonary emphysema; thoracic aortic dilatation; pulmonary trunk dilatation; coronary artery calcification; adrenal hyperplasia; and osteoporosis (vertebral body height and density changes). Two experts analyzed computed tomography and compared results with artificial intelligence. Further routing was determined according to clinical guidelines for all findings initially detected and missed by radiologists. The hospital price list determined the potential revenue loss for each patient.
RESULTS: From the final 160 computed tomographies, the artificial intelligence identified 90 studies (56%) with pathologies, of which 81 (51%) were missing at least one pathology in the report. The “second-stage” lost potential revenue for all pathologies from 81 patients was RUB 2,847,760 ($37,251 or CNY 256,218). Lost potential revenue only for those pathologies missed by radiologists but detected by artificial intelligence was RUB 2,065,360 ($27,017 or CNY 185,824).
CONCLUSION: Using artificial intelligence as an “assistant” to the radiologist for chest computed tomography can dramatically minimize the number of missed abnormalities. Compared with the normal model without artificial intelligence, using artificial intelligence can provide 3.6 times more benefits. Using advanced artificial intelligence for chest computed tomography can save money.
Full Text
##article.viewOnOriginalSite##About the authors
Valeria Yu. Chernina
IRA Labs
Email: v.chernina@ira-labs.com
ORCID iD: 0000-0002-0302-293X
SPIN-code: 8896-8051
Scopus Author ID: 57210638679
ResearcherId: AAF-1215-2020
Russian Federation, Moscow
Mikhail G. Belyaev
IRA Labs
Email: belyaevmichel@gmail.com
ORCID iD: 0000-0001-9906-6453
SPIN-code: 2406-1772
Cand. Sci. (Phys.-Math.), Professor
Russian Federation, MoscowAnton Yu. Silin
Clinical Hospital on Yauza
Email: silin@yamed.ru
ORCID iD: 0000-0003-4952-2347
SPIN-code: 4411-8745
Russian Federation, Moscow
Ivan O. Avetisov
Clinical Hospital on Yauza
Email: avetisov@yamed.ru
ORCID iD: 0009-0007-3550-7556
Russian Federation, Moscow
Ilya A. Pyatnitskiy
IRA Labs; The University of Texas at Austin
Email: i.pyatnitskiy@ira-labs.com
ORCID iD: 0000-0002-2827-1473
SPIN-code: 6150-4961
Russian Federation, Moscow; Austin, Texas, USA
Ekaterina A. Petrash
IRA Labs; N.N. Blokhin National Medical Research Center of Oncology
Email: e.a.petrash@gmail.com
ORCID iD: 0000-0001-6572-5369
SPIN-code: 6910-8890
MD, Cand. Sci. (Med.)
Russian Federation, Moscow; MoscowMaria V. Basova
IRA Labs
Email: m.basova@ira-labs.com
ORCID iD: 0009-0000-3325-8452
Russian Federation, Moscow
Valentin E. Sinitsyn
Lomonosov Moscow State University; Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies
Email: vsini@mail.ru
ORCID iD: 0000-0002-5649-2193
SPIN-code: 8449-6590
MD, Dr. Sci. (Med.), Professor
Russian Federation, Moscow; MoscowVitaly V. Omelyanovskiy
The Center for Healthcare Quality Assessment and Control; Russian Medical Academy of Continuous Professional Education; Scientific and research financial institute
Email: vvo@rosmedex.ru
ORCID iD: 0000-0003-1581-0703
SPIN-code: 1776-4270
MD, Dr. Sci. (Med.), Professor
Russian Federation, Moscow; Moscow; MoscowVictor A. Gombolevskiy
IRA Labs; Artificial Intelligence Research Institute
Author for correspondence.
Email: gombolevskii@gmail.com
ORCID iD: 0000-0003-1816-1315
SPIN-code: 6810-3279
MD, Cand. Sci. (Med.)
Russian Federation, Moscow; MoscowReferences
- Aberle DR, Adams AM, Berg CD, et al.; National Lung Screening Trial Research Team. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med. 2011;365(5):395–409. doi: 10.1056/NEJMoa1102873
- Pastorino U, Silva M, Sestini S, et al. Prolonged lung cancer screening reduced 10-year mortality in the MILD trial: new confirmation of lung cancer screening efficacy. Ann Oncol. 2019;30(7):1162–1169. doi: 10.1093/annonc/mdz117
- Grover H, King W, Bhattarai N, et al. Systematic review of the cost-effectiveness of screening for lung cancer with low dose computed tomography. Lung Cancer. 2022;170:20–33. doi: 10.1016/j.lungcan.2022.05.005
- Nikolaev AE, Gombolevskiy VA, Gonchar AP, et al. Incidental findings during lung cancer screening by low-dose computed tomography. Tuberculosis Lung Dis. 2018;96(11):60–67. (In Russ). doi: 10.21292/2075-1230-2018-96-11-60-67
- Kilsdonk ID, de Roos MP, Bresser P, et al. Frequency and spectrum of incidental findings when using chest CT as a primary triage tool for COVID-19. Eur J Radiol Open. 2021;(8):100366. doi: 10.1016/j.ejro.2021.100366
- Beltsevich DG, Melnichenko GA, Kuznetsov NS, et al. Russian Association of Endocrinologists clinical practice guideline for adrenal incidentalomas differential diagnosis. Endocrine Sur. 2016;10(4):31–42. (In Russ). doi: 10.14341/serg2016431-42
- Emphysema of the lungs. Clinical recommendations (approved by the Ministry of Health of the Russian Federation, 2021). (In Russ). Available from: https://www.garant.ru/products/ipo/prime/doc/402775957/. Accessed: 25.08.2021.
- Thoracic and thoracoabdominal aortic aneurysms. Clinical recommendations (approved by the Ministry of Health of the Russian Federation, 2023). (In Russ). Available from: https://www.angiolsurgery.org/library/recommendations/2022/aneurysms_thoracic_thoracoabdominal_aorta/recommendation.pdf. Accessed: 25.08.2021.
- Vasiliev YuA, Vladzimirsky AV, Arzamasov KM, et al. Computer vision in radiation diagnostics: the first stage of the Moscow experiment. Monograph. 2nd ed., revised and updated. Ed. by Yu.A. Vasiliev, A.V. Vladzimirsky. Moscow: Izdatel’skie resheniya; 2023. 376 p. (In Russ).
- Zakharov A, Pisov M, Bukharaev A, et al. Interpretable vertebral fracture quantification via anchor-free landmarks localization. Med Image Anal. 2023;(83):102646. doi: 10.1016/j.media.2022.102646
- Goncharov M, Pisov M, Shevtsov A, et al. CT-Based COVID-19 triage: Deep multitask learning improves joint identification and severity quantification. Med Image Anal. 2021;(71):102054. doi: 10.1016/j.media.2021.102054
- Shirokikh B, Shevtsov A, Dalechina A, et al. Accelerating 3D medical image segmentation by adaptive small-scale target localization. J Imaging. 2021;7(2):35. doi: 10.3390/jimaging7020035
- Goncharov M, Pisov M, Belyaev M, et al. Quantification of epicardial adipose tissue in low-dose computed tomography images. Lecture Notes Electrical Engineering. 2022;784(1):98–107. doi: 10.1007/978-981-16-3880-0_11
- Husereau D, Drummond M, Augustovski F, et al.; CHEERS 2022 ISPOR Good Research Practices Task Force. Consolidated health economic evaluation reporting standards 2022 (cheers 2022) statement: Updated reporting guidance for health economic evaluations. Value Health. 2022;25(1):3–9. doi: 10.1016/j.jval.2021.11.1351
- Morozov SP, Vladzimirsky AV, Kljashtorny VG, et al. Clinical trials of software based on intelligent technologies (radiology diagnostics). Preprint № CDT-2019-1. Best Practice in Radiation Oncology and Instrumental Diagnostics. Vol. 23. Moscow: Center for Diagnostics and Telemedicine; 2019. 33 р. (In Russ).
- Gombolevskiy VA, Blokhin IA, Laipan AS, et al. Methodological guidelines for lung cancer screening. Best Practice in Radiation and Instrumental Diagnostics. Vol. 56. Moscow: Center for Diagnostics and Telemedicine; 2020. 57 р. (In Russ).
- Morozov SP, Protsenko DN, Smetanina SV, et al. Radiation diagnosis of coronavirus disease (COVID-19): Organization, methodology, interpretation of results: Guidelines. Best Practices in Radiation and Instrumental Diagnostics Series. Vol. 65. Moscow: Center for Diagnostics and Telemedicine; 2020. 78 р. (In Russ).
- Temporary methodological recommendations. Prevention, diagnosis and treatment of new coronavirus infection (COVID-19). Ministry of Health of the Russian Federation. Version 17 (12/14/2022). (In Russ). Available from: https://static-0.minzdrav.gov.ru/system/attachments/attaches/000/061/254/original/%D0%92%D0%9C%D0%A0_COVID-19_V17.pdf?1671088207. Accessed: 17.03.2023.
- Lynch DA, Austin JH, Hogg JC, et al. CT-definable subtypes of chronic obstructive pulmonary disease: A statement of the Fleischner Society. Radiology. 2015;277(1):192–205. doi: 10.1148/radiol.2015141579
- Hersh CP, Washko GR, Estépar RS, et al. Paired inspiratory-expiratory chest CT scans to assess for small airways disease in COPD. Respir Res. 2013;146(1):42. doi: 10.1186/1465-9921-14-42
- Walker C, Chung JH. Muller’s imaging of the chest e-book: Expert radiology series. Elsevier Health Sciences; 2018.1104 р.
- Erbel R, Aboyans V, Boileau C et al. ESC Committee for Practice Guidelines. 2014 ESC Guidelines on the diagnosis and treatment of aortic diseases: Document covering acute and chronic aortic diseases of the thoracic and abdominal aorta of the adult. The task force for the diagnosis and treatment of aortic diseases of the European society of cardiology (ESC). Eur Heart J. 2014;35(41):2873–2926. doi: 10.1093/eurheartj/ehu281
- Wanhainen A, Verzini F, Van Herzeele I, et al. Clinical practice guidelines on the management of abdominal aorto-iliac artery aneurysms. Eur J Vascul Endovascul Sur. 2019;57(1):8–93. doi: 10.1016/j.ejvs.2018.09.020
- Jurgens PT, Carr JJ, Terry JG, et al. Association of abdominal aorta calcium and coronary artery calcium with incident cardiovascular and coronary heart disease events in black and white middle-aged people: The coronary artery risk development in young adults study. J Am Heart Assoc. 2021;10(24):23–37. doi: 10.1161/JAHA.121.023037
- Galiè N, Humbert M, Vachiery JL, et al. 2015 ESC/ERS Guidelines for the diagnosis and treatment of pulmonary hypertension: The joint task force for the diagnosis and treatment of pulmonary hypertension of the European society of cardiology (ESC) and the European respiratory society (ERS): Endorsed by: Association for European paediatric and congenital cardiology (AEPC), international society for heart and lung transplantation (ISHLT). Eur Respir J. 2015;46(4):903–975. doi: 10.1183/13993003.01032-2015
- Agatston AS, Janowitz WR, Hildner FJ et al. Quantification of coronary artery calcium using ultrafast computed tomography. J Am Coll Cardiol. 1990;15(4):827–832. doi: 10.1016/0735-1097(90)90282-t
- Zhuravlev KN. Coronary CT angiography. Best practices in radiation and instrumental diagnostics series. Vol. 65. Moscow: Center for Diagnostics and Telemedicine; 2020. 35 р. (In Russ).
- Mayo-Smith WW, Song JH, Boland GL, et al. Management of incidental adrenal masses: A white paper of the ACR incidental findings committee. J Am Coll Radiol. 2017;14(8):1038–1044. doi: 10.1016/j.jacr.2017.05.001
- Belaya ZE, Belova KYu, Biryukova EV, et al. Federal clinical guidelines for diagnosis, treatment and prevention of osteoporosis. Osteoporosis Bone Diseases. 2021;24(2):4–47. (In Russ). doi: 10.14341/osteo12930
- Genant HK, Jergas M. Assessment of prevalent and incident vertebral fractures in osteoporosis research. Osteoporosis Int. 2003;14(3):43–55. doi: 10.1007/S00198-002-1348-1
- Pathological fractures complicating osteoporosis. Clinical recommendations. Moscow; 2018. 90 p. (In Russ). Available from: https://neirodoc.ru/wp-content/uploads/2019/10/patologicheskie-perelomy-oslozhnyayuschie-osteoporoz-2018.pdf. Accessed: 17.03.2023.
- Kolbin AS, Gomon YuM, Balykina YuE, et al. Socioeconomic and global burden of COVID-19. Good Clin Pract. 2021;(1):24–34. (In Russ). doi: 10.37489/2588-0519-2021-1-24-34
- Pickhardt PJ, Correale L, Hassan C. AI-based opportunistic CT screening of incidental cardiovascular disease, osteoporosis, and sarcopenia: Cost-effectiveness analysis. Abdom Radiol (NY). 2023;(3):1181–1198. doi: 10.1007/s00261-023-03800-9
- Behr CM, Koffijberg H, Degeling K, et al. Can we increase efficiency of CT lung cancer screening by combining with CVD and COPD screening? Results of an early economic evaluation. Eur Radiol. 2022;32(5):3067–3075. doi: 10.1007/s00330-021-08422-7
- Chintanapakdee W, Mendoza DP, Zhang EW, et al. Detection of extrapulmonary malignancy during lung cancer screening: 5-year analysis at a tertiary hospital. J Am Coll Radiol. 2020;17(12):1609–1620. doi: 10.1016/j.jacr.2020.09.032
- Girling A, Lilford R, Cole A, et al. Headroom approach to device development: Current and future directions. Int J Technol Assess Health Care. 2015;31(5):331–338. doi: 10.1017/S0266462315000501
- Ijzerman MJ, Steuten LM. Early assessment of medical technologies to inform product development and market access: A review of methods and applications. Appl Health Econ Health Policy. 2011;9(5):331–347. doi: 10.2165/11593380-000000000-00000
- Izerman MJ, Koffijberg H, Fenwick E, et al. Emerging use of early health technology assessment in medical product development: A scoping review of the literature. Pharmacoeconomics. 2017;35(7):727–740. doi: 10.1007/s40273-017-0509-1
- Sahiner B, Pezeshk A, Hadjiiski LM, et al. Deep learning in medical imaging and radiation therapy. Med Phys. 2019;46(1):1–36. doi: 10.1002/mp.13264
- Allen BJ, Seltzer SE, Langlotz CP, et al. A road map for translational research on artificial intelligence in medical imaging: From the 2018 national institutes of health/RSNA/ACR/The academy workshop. J Am Coll Radiol. 2019;16(9):1179–1189. doi: 10.1016/j.jacr.2019.04.014
- Xia C, Rook M, Pelgrim GJ, et al. Early imaging biomarkers of lung cancer, COPD and coronary artery disease in the general population: rationale and design of the ImaLife (Imaging in Lifelines) Study. Eur J Epidemiol. 2020;35(1):75–86. doi: 10.1007/s10654-019-00519-0
- Ziegelmayer S, Graf M, Makowski M, et al. Cost-Effectiveness of artificial intelligence support in computed tomography-based lung cancer screening. Cancers (Basel). 2022;14(7):1729. doi: 10.3390/cancers14071729
- McDonald RJ, Schwartz KM, Eckel LJ, et al. The effects of changes in utilization and technological advancements of cross-sectional imaging on radiologist workload. Acad Radiol. 2015;22(9):1191–1198. doi: 10.1016/j.acra.2015.05.007
- Sokolovskaya E, Shinde T, Ruchman RB, et al. The effect of faster reporting speed for imaging studies on the number of misses and interpretation errors: A pilot study. J Am Coll Radiol. 2015;12(7):683–688. doi: 10.1016/j.jacr.2015.03.040
- Geijer H, Geijer M. Added value of double reading in diagnostic radiology, a systematic review. Insights Imaging. 2018;9(3):287–301. doi: 10.1007/s13244-018-0599-0
- Lauritzen PM, Hurlen P, Sandbæk G, et al. Double reading rates and quality assurance practices in Norwegian hospital radiology departments: Two parallel national surveys. Acta Radiol. 2015;56(1):78–86. doi: 10.1177/0284185113519988
- Morozov SP, Chernina VYu, Andreychenko AE, et al. How does artificial intelligence effect on the assessment of lung damage in COVID-19 on chest CT scan? Digital Diagnostics. 2020;2(1):27−38. (In Russ). doi: 10.17816/DD60040