Technological defects in software based on artificial intelligence

Мұқаба

Дәйексөз келтіру

Толық мәтін

Аннотация

BACKGROUND: Technological defects in the use of artificial intelligence software are critical when deciding on the practical applicability and clinical value of artificial intelligence software.

AIM: To conduct an analysis and systematization of technological defects occurring when artificial intelligence software analyzes medical images.

MATERIALS AND METHODS: As part of the experiment on the use of innovative computer vision technologies for the analysis of medical images and further application in the Moscow healthcare system, technological parameters of all artificial intelligence software are monitored at the testing and operation stages of the trial. This article presents graphical information on the average number of technological defects in mass mammography screening in 2021. This period was chosen as the most indicative and characterized by the active development of artificial intelligence software and increased technical stability of its performance. To assess the applicability of the analysis for technological defects, a similar analysis was conducted for the direction of detection of intracranial hemorrhage on computed tomography scans of the brain for 2022–2023.

RESULTS: During the study, artificial intelligence software used for mammography (two algorithms) and brain computed tomography (one algorithm) were analyzed. Fourteen mammography samples were collected for technological monitoring during the identified period, each from 20 studies, and 12 brain computed tomography samples were obtained, each from 80 studies. Graphs were constructed for each type of defect, and trend lines were plotted for each modality. The coefficients of the trend line equations indicate a downward tendency in the number of technological defects.

CONCLUSION: This analysis allows tracing a downward trend in the number of technological defects, which may indicate a refinement of artificial intelligence software and an increase in its quality because of periodic monitoring. It also shows the versatility of use for both preventive and emergency methods.

Толық мәтін

##article.viewOnOriginalSite##

Авторлар туралы

Viktoria Zinchenko

Scientific and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Хат алмасуға жауапты Автор.
Email: ZinchenkoVV1@zdrav.mos.ru
ORCID iD: 0000-0002-2307-725X
SPIN-код: 4188-0635
Ресей, Moscow

Kirill Arzamasov

Scientific and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Email: ArzamasovKM@zdrav.mos.ru
ORCID iD: 0000-0001-7786-0349
SPIN-код: 3160-8062

MD, Cand. Sci. (Med.)

Ресей, Moscow

Elena Kremneva

Scientific and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Email: KremnevaEI@zdrav.mos.ru
ORCID iD: 0000-0001-9396-6063
SPIN-код: 8799-8092

MD, Cand. Sci. (Med.)

Ресей, Moscow

Anton Vladzymyrskyy

Scientific and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Email: VladzimirskijAV@zdrav.mos.ru
ORCID iD: 0000-0002-2990-7736
SPIN-код: 3602-7120

MD, Dr. Sci. (Med.)

Ресей, Moscow

Yuriy Vasilev

Scientific and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Email: VasilevYA1@zdrav.mos.ru
ORCID iD: 0000-0002-0208-5218
SPIN-код: 4458-5608

MD, Cand. Sci. (Med.)

Ресей, Moscow

Әдебиет тізімі

  1. Vladzimirskii AV, Vasil’ev YuA, Arzamasov KM, et al. Computer Vision in Radiologic Diagnostics: the First Stage of Moscow experiment. Vasil’ev YuA, Vladzimirskii AV, editors. Publishing solutions; 2022. (In Russ).
  2. Ranschaert ER, Morozov S, Algra PR, editors. Artificial Intelligence in Medical Imaging. Berlin: Springer; 2019. doi: 10.1007/978-3-319-94878-2
  3. Gusev AV, Dobridnyuk SL. Artificial intelligence in medicine and healthcare. Information Society Journal. 2017;(4-5):78–93. (In Russ).
  4. Shutov DV, Sharova DE, Abuladze LR, Drozdov DV. Artificial intelligence in clinical physiology: How to improve learning agility. Digital Diagnostics. 2023;4(1):81–88. doi: 10.17816/DD123559
  5. Meldo AA, Utkin LV, Trofimova TN. Artificial intelligence in medicine: current state and main directions of development of the intellectual diagnostics. Diagnostic radiology and radiotherapy. 2020;11(1):9–17. doi: 10.22328/2079-5343-2020-11-1-9-17
  6. Recht MP, Dewey M, Dreyer K, et al. Integrating artificial intelligence into the clinical practice of radiology: challenges and recommendations. European radiology. 2020;30(6):3576–3584. doi: 10.1007/s00330-020-06672-5
  7. Larson DB, Harvey H, Rubin DL, et al. Regulatory Frameworks for Development and Evaluation of Artificial Intelligence-Based Diagnostic Imaging Algorithms: Summary and Recommendations. Journal of the American College of Radiology. 2021;18(3 Pt A):413–424. doi: 10.1016/j.jacr.2020.09.060
  8. Zinchenko V, Chetverikov S, Ahmad E, et al. Changes in software as a medical device based on artificial intelligence technologies. International Journal of Computer Assisted Radiology and Surgery. 2022;17:1969–1977. doi: 10.1007/s11548-022-02669-1
  9. Nomura Y, Miki S, Hayashi N, et al. Novel platform for development, training, and validation of computer-assisted detection/diagnosis software. International Journal of Computer Assisted Radiology and Surgery. 2020;15(4):661–672. doi: 10.1007/s11548-020-02132-z
  10. Methodological recommendations on the procedure for expert examination of quality, efficiency and safety of medical devices (in terms of software) for state registration under the national system FGBU «VNIIIMT» Roszdravnadzor. Moscow; 2021. (In Russ).
  11. Pemberton HG, Zaki LAM, Goodkin O, et al. Technical and clinical validation of commercial automated volumetric MRI tools for dementia diagnosis — a systematic review. Neuroradiology. 2021;63:1773–1789. doi: 10.1007/s00234-021-02746-3
  12. Order of the Moscow City Health Department No. 51 dated 26.01.2021 «On approval of the procedure and conditions for conducting an experiment on the use of innovative technologies in the field of computer vision to analyze medical images and further application in the health care system of the city of Moscow in 2021». (In Russ).
  13. Chetverikov SF, Arzamasov KM, Andreichenko AE, et al. Approaches to Sampling for Quality Control of Artificial Intelligence in Biomedical Research. Modern Technologies in Medicine. 2023;15(2):19. doi: 10.17691/stm2023.15.2.02
  14. Zinchenko VV, Arzamasov KM, Chetverikov SF, et al. Methodology for Conducting Post-Marketing Surveillance of Software as a Medical Device Based on Artificial Intelligence Technologies. Modern Technologies in Medicine. 2022;14(5):15–25. doi: 10.17691/stm2022.14.5.02
  15. Altman DG. Statistics and ethics in medical research: III How large a sample? British medical journal. 1980;281(6251):1336. doi: 10.1136/bmj.281.6251.1336
  16. Tyrov IA, Vasilyev YuA, Arzamasov KM, et al. Assessment of the maturity of artificial intelligence technologies for healthcare: methodology and its application based on the use of innovative computer vision technologies for medical image analysis and subsequent applicability in the healthcare system of Moscow. Medical Doctor and IT. 2022;(4):76–92. doi: 10.25881/18110193_2022_4_76
  17. Vladzimirsky AV, Gusev AV, Sharova DE, et al. Health Information System Maturity Assessment Methodology. Medical Doctor and IT. 2022;(3):68–84. doi: 10.25881/18110193_2022_3_68
  18. Order of the Moscow City Health Department No. 160 dated 03.11.2022 «On Approval of the Procedure and Conditions for Conducting an Experiment on the Use of Innovative Technologies in Computer Vision for Analyzing Medical Images and Further Application in the Moscow City Health Care System in 2022». (In Russ).

Қосымша файлдар

Қосымша файлдар
Әрекет
1. JATS XML
Жүктеу
2. Fig. 1. Changes in detection of the average number of each technological defect for software based on artificial intelligence for mammography. Defects are divided into groups in accordance with order no. 51 of the Moscow Department of Health dated January 26, 2021.

Жүктеу (136KB)
Метадеректер
3. Fig. 2. Changes in the detection of the average number of each technological defect for software based on artificial intelligence for the brain computed tomography modality (presence or absence of intracranial hemorrhage). Defects are divided into groups in accordance with order no. 160 of the Moscow Department of Health dated November 3, 2022.

Жүктеу (157KB)
Метадеректер
4. Fig. 3. Defect: not all necessary images have been evaluated. Modality: mammography.

Жүктеу (117KB)
Метадеректер
5. Fig. 4. Defect: off-target markings; Modality: mammography.

Жүктеу (78KB)
Метадеректер
6. Fig. 5. Defect: an incorrect series was evaluated (contrast-enhanced computed tomography instead of native on). Modality: computed tomography.

Жүктеу (124KB)
Метадеректер
7. Fig. 6. Defect: off-target markings, contrast-enhanced computed tomography instead of native computed tomography. Modality: computed tomography.

Жүктеу (114KB)
Метадеректер
8. Fig. 7. Number of defects in each group over time; modality: mammography.

Жүктеу (214KB)
Метадеректер
9. Fig. 8. Number of defects in each group over time; modality: computed tomography.

Жүктеу (215KB)
Метадеректер

© Eco-Vector, 2023

Creative Commons License
Бұл мақала лицензия бойынша қол жетімді Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Осы сайт cookie-файлдарды пайдаланады

Біздің сайтты пайдалануды жалғастыра отырып, сіз сайттың дұрыс жұмыс істеуін қамтамасыз ететін cookie файлдарын өңдеуге келісім бересіз.< / br>< / br>cookie файлдары туралы< / a>