Research and development of algorithms for the formation of an effective ensemble of convolutional neural networks for image classification

Capa

Citar

Texto integral

Resumo

The object of the research is artificial neural networks (ANN) with convolutional architecture for image classification. The subject of the research is the study and development of algorithms for constructing ensembles of convolutional neural networks (SNS) in conditions of limited training sample. The aim of the study is to develop an algorithm for the formation of an effective model based on an ensemble of convolutional SNS using methods of averaging the results of each model, capable of avoiding overfitting in the process of improving the accuracy of the forecast and trained on a small amount of data, less than 10 thousand examples. As a basic network, an effective SNA architecture was developed as part of the ensemble, which showed good results as a single model. The article also examines methods for combining the results of ensemble models and provides recommendations for the formation of the SNA architecture.  The research methods used are the theory of neural networks, the theory of machine learning, artificial intelligence, methods of algorithmization and programming of machine learning models, a comparative analysis of models based on different algorithms using classical ensembling with simple averaging and combining the results of basic algorithms in conditions of limited sampling, taking into account weighted average. The field of application of the obtained algorithm and model is medical diagnostics in medical institutions, sanatoriums during primary diagnostic admission, using the example of a research task, the model is trained to classify dermatological diseases according to input photographs. The novelty of the study lies in the development of an effective algorithm and image classification model based on an ensemble of convolutional NS that exceed the prediction accuracy of basic classifiers, the process of retraining an ensemble of classifiers with deep architecture on a small sample volume is investigated, from which conclusions are drawn on the design of an optimal network architecture and the choice of methods for combining the results of several basic classifiers. As a result of the research, an algorithm has been developed for the formation of an ensemble of SNS based on an effective basic architecture and weighted average averaging of the results of each model for the classification task of image recognition in conditions of limited sampling.

Bibliografia

  1. Thoma M. Analysis and optimization of convolutional neural network architectures, 2017.
  2. Cruz Y. J. et al. Ensemble of convolutional neural networks based on an evolutionary algorithm applied to an industrial welding process //Computers in Industry. – 2021. – Т. 133. – С. 103-530.
  3. Yang S. et al. An ensemble classification algorithm for convolutional neural network based on AdaBoost //2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS). – IEEE, 2017. – С. 401-406.
  4. Basili V. R., Briand L. C., Melo W. L. A validation of object-oriented design metrics as quality indicators //IEEE Transactions on software engineering. – 1996. – Т. 22. – №. 10. – С. 751-761.
  5. Нейронные сети. Переобучение-что это и как этого избежать, критерии останова обучения. [Электронный ресурс]. URL: https://proproprogs.ru/neural_network/ pereobuchenie-chto-eto-i-kak-etogo-izbezhat-kriterii-ostanova-obucheniya (дата обращения 09.02.2024).
  6. Воронецкий Ю. О., Жданов Н. А. Методы борьбы с переобучением искусственных нейронных сетей // Научный аспект. 2019. №2. [Электронный ресурс] URL: https://na-journal.ru/2-2019-tehnicheskie-nauki/1703-metody-borby-s-pereobucheniem-iskusstvennyh-neironnyh-setei (дата обращения: 10.02.2024).
  7. Li C. et al. Improving forecasting accuracy of daily enterprise electricity consumption using a random forest based on ensemble empirical mode decomposition // Energy. – 2018. – Т. 165. – С. 1220-1227.
  8. Omisore O. M. et al. Weighting-based deep ensemble learning for recognition of interventionalists’ hand motions during robot-assisted intravascular catheterization // IEEE Transactions on Human-Machine Systems. – 2022. – Т. 53. – №. 1. – С. 215-227.
  9. Ансамблирование моделей нейронных сетей с использованием библиотеки Keras. [Электронный ресурс]. URL:https://se.moevm.info/lib/ exe/fetch.php/courses:artificial_ neural_ networks:pr_8.pdf (дата обращения 11.02.2024).
  10. Метод оптимизации Нелдера — Мида. Пример реализации на Python. [Электронный ресурс]. URL:https://habr.com/ru/articles/332092/ (дата обращения 09.02.2024).
  11. Клюева И. А. Методы и алгоритмы ансамблирования и поиска значений параметров классификаторов. [Электронный ресурс]. URL:https://dissov.pnzgu.ru/files/dissov .pnzgu.ru/2021/tech/klyueva/ dissertaciya_ klyuevoy _i_a_.pdf (дата обращения 08.02.2024).
  12. Микрюков, А. А. Классификация событий в системах обеспечения информационной безопасности на основе нейросетевых технологий / А. А. Микрюков, А. В. Бабаш, В. А. Сизов // Открытое образование. – 2019. – Т. 23. № 1. – C. 57-63.
  13. Gizluk D. Adaptive optimization methods. // Neural networks are simple (part 7). 2020. №7. [Электронный ресурс]. URL:https://www.mql5.com/ru/articles/8598#para21 (дата обращения: 10.02.2024).
  14. Mason L. et al. Boosting algorithms as gradient descent //Advances in neural information processing systems. – 1999. – Т. 12.
  15. Zaheer R., Shaziya H. A study of the optimization algorithms in deep learning //2019 third international conference on inventive systems and control (ICISC). – IEEE, 2019. – С. 536-539.
  16. Староверов Б. А., Хамитов Р. Н. Реализация глубокого обучения для прогнозирования при помощи ансамбля нейронных сетей //Известия Тульского государственного университета. Технические науки. – 2023. – №. 4. – С. 185-189.
  17. Onan A., Korukoğlu S., Bulut H. A multiobjective weighted voting ensemble classifier based on differential evolution algorithm for text sentiment classification // Expert Systems with Applications. – 2016. – Т. 62. – С. 1-16.
  18. Kim H. et al. A weight-adjusted voting algorithm for ensembles of classifiers //Journal of the Korean Statistical Society. – 2011. – Т. 40. – №. 4. – С. 437-449.
  19. Yao X., Islam M. M. Evolving artificial neural network ensembles //IEEE Computational Intelligence Magazine. – 2008. – Т. 3. – №. 1. – С. 31-42.
  20. Anand V. et al. Weighted Average Ensemble Deep Learning Model for Stratification of Brain Tumor in MRI Images //Diagnostics. – 2023. – Т. 13. – №. 7. – С. 1320.
  21. The International Skin Imaging Collaboration. [Электронный ресурс].-URL: https://www.isic-archive.com (дата обращения 12.02.2024).
  22. Alexandropoulos S. A. N., Kotsiantis S. B., Vrahatis M. N. Data preprocessing in predictive data mining // The Knowledge Engineering Review. – 2019. – Т. 34. – С. e1.
  23. García S., Luengo J., Herrera F. Data preprocessing in data mining. – Cham, Switzerland: Springer International Publishing, 2015. – Т. 72. – С. 59-139.
  24. Liang G., Zheng L. A transfer learning method with deep residual network for pediatric pneumonia diagnosis // Computer methods and programs in biomedicine. – 2020. – Т. 187. – С. 104-964.
  25. InceptionV3. [Электронный ресурс].-URL: https://keras.io/api/ applications/inceptionv3/ (дата обращения 13.02.2024).
  26. InceptionResnNetV2. [Электронный ресурс]. URL: https://keras.io/api/ applications/inceptionresnetv2/ (дата обращения 13.02.2024).
  27. VGG16. [Электронный ресурс]. URL: https://keras.io/api/ applications/vgg/#vgg16-function (дата обращения 13.02.2024).
  28. Щетинин Е. Ю. О некоторых методах сегментации изображений с применением свёрточных нейронных сетей // Информационно-телекоммуникационные технологии и математическое моделирование высокотехнологичных систем. – 2021. – С. 507-510.
  29. Rosebrock A. Change input shape dimensions for fine-tuning with Keras. // AI & Computer Vision Programming. 2019. [Электронный ресурс]. URL:https://pyimagesearch.com/2019/06/24/ change-input-shape-dimensions-for-fine-tuning-with-keras/ (дата обращения 14.02.2024).
  30. Костин К. А. и др. Адаптивный классификатор патологий для компьютерной диагностики заболеваний с использованием сверточных нейронных сетей по медицинским изображениям и видеоданным: магистерская диссертация по направлению подготовки: 01.04. 02-Прикладная математика и информатика. – 2017.
  31. A. Krizhevsky, I. Sutskever, G.E. Hinton. ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of Advances in Neural Information Processing Systems 25 (NIPS 2012), 2012, Pp. 1097-1105.
  32. Wang J., Lin J., Wang Z. Efficient hardware architectures for deep convolutional neural network // IEEE Transactions on Circuits and Systems I: Regular Papers. – 2017. – Т. 65. – №. 6. – pp. 1941-1953.
  33. Phung V. H., Rhee E. J. A high-accuracy model average ensemble of convolutional neural networks for classification of cloud image patches on small datasets //Applied Sciences. – 2019. – Т. 9. – №. 21. – С. 4500.
  34. The differential evolution method. [Электронный ресурс]. URL: https://docs.scipy.org/ doc/scipy/reference/generated/scipy.optimize.differential_evolution.html (дата обращения: 13.02.2024).
  35. Как разработать средневзвешенный ансамбль для глубоких обучающих нейронных сетей. // 2018. [Электронный ресурс]. URL: https://machinelearningmastery.ru/ weighted-average-ensemble-for-deep-learning-neural-networks/# (дата обращения: 13.02.2024)

Arquivos suplementares

Arquivos suplementares
Ação
1. JATS XML
Baixar

Согласие на обработку персональных данных

 

Используя сайт https://journals.rcsi.science, я (далее – «Пользователь» или «Субъект персональных данных») даю согласие на обработку персональных данных на этом сайте (текст Согласия) и на обработку персональных данных с помощью сервиса «Яндекс.Метрика» (текст Согласия).