Method of automatic search for the structure and parameters of neural networks for solving information processing problems

封面

如何引用文章

全文:

详细

Neural networks are actively used in solving various applied problems of data analysis, processing and generation. When using them, one of the difficult stages is the selection of the structure and parameters of neural networks (the number and types of layers of neurons, activation functions, optimizers, and so on) that provide the greatest accuracy and, therefore, the success of solving the problem. Currently, this issue is being solved by analytical selection of the neural network architecture by a researcher or software developer. Existing automatic tools (AutoKeras, AutoGAN, AutoSklearn, DEvol and others) are not universal and functional enough. Therefore, within the framework of this work, a method of automatic search for the structure and parameters of neural networks of various types (multilayer dense, convolutional, generative-adversarial, autoencoders, and others) is considered for solving a wide class of problems. The formalization of the method and its main stages are presented. The approbation of the method is considered, which proves its effectiveness in relation to the analytical solution in the selection of the architecture of the neural network. A comparison of the method with existing analogues is carried out, its advantage is revealed in terms of the accuracy of the formed neural networks and the time to find a solution. The research results can be used to solve a large class of data processing problems for which it is required to automate the selection of the structure and parameters of a neural network.

作者简介

Artem Obukhov

Tambov State Technical University

106 Sovetskaya St., Tambov 392000, Russia

参考

  1. Бескровный А. С., Бессонов Л. В., Иванов Д. В., Кириллова И. В., Коссович Л. Ю. Использование сверточной нейронной сети для автоматизации построения двумерных твердотельных моделей позвонков // Известия Саратовского университета. Новая серия. Серия: Математика. Механика. Информатика. 2020. Т. 20, вып. 4. С. 502–516. https://doi.org/10.18500/1816-9791-2020-20-4-502-516
  2. Li H., Yuan D., Ma X., Cui D., Cao L. Genetic algorithm for the optimization of features and neural networks in ECG signals classification // Scientific Reports. 2017. Vol. 7, iss. 1. P. 1–12. https://doi.org/10.1038/srep41011
  3. He X., Zhao K., Chu X. AutoML: A Survey of the State-of-the-Art // Knowledge-Based Systems. 2021. Vol. 212. P. 106622. https://doi.org/10.1016/j.knosys.2020.106622
  4. Jin H., Song Q., Hu X. Auto-keras: An efficient neural architecture search system // Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2019. P. 1946–1956. https://doi.org/10.1145/3292500.3330648
  5. Real E., Aggarwal A., Huang Y., Le Q. V. Regularized evolution for image classifier architecture search // Proceedings of the AAAI Conference on Artificial Intelligence. 2019. Vol. 33. P. 4780–4789. https://doi.org/10.1609/aaai.v33i01.33014780
  6. Budjac R., Nikmon M., Schreiber P., Zahradnikova B., Janacova D. Automated machine learning overview // Research Papers Faculty of Materials Science and Technology Slovak University of Technology. 2019. Vol. 27, iss. 45. P. 107–112. https://doi.org/10.2478/rput-2019-0033
  7. Feurer M., Klein A., Eggensperger K., Springenberg J., Blum M., Hutter F. Efficient and robust automated machine learning // Automated Machine Learning. 2019. P. 113–134. https://doi.org/10.1007/978-3-030-05318-5_6
  8. Le Q., Zoph B. Using machine learning to explore neural network architecture // Google Research Blog. 2017. URL: https://research.googleblog.com/2017/05/using-machine-learning-to-explore.html (дата обращения: 05.04.2022).
  9. Gong X., Chang S., Jiang Y., Wang Z. Autogan: Neural architecture search for generative adversarial networks // Proceedings of the IEEE International Conference on Computer Vision. 2019. P. 3224–3234. https://doi.org/10.1109/ICCV.2019.00332
  10. Liu C., Zoph B., Neumann M., Shlens J., Hua W., Li L. J., Murphy K. Progressive Neural Architecture Search // Computer Vision – ECCV 2018. ECCV 2018 / eds.: V. Ferrari, M. Hebert, C. Sminchisescu, Y. Weiss. Cham : Springer, 2018. P. 19–34. (Lecture Notes in Computer Science, vol. 11205). https://doi.org/10.1007/978-3-030-01246-5_2
  11. Cai H., Gan C., Han S. Once for all: Train one network and specialize it for efficient deployment // arXiv preprint arXiv:1908.09791. 2019.
  12. Truong A., Walters A., Goodsitt J., Hines K., Bruss C. B., Farivar R. Towards automated machine learning: Evaluation and comparison of AutoML approaches and tools // 2019 IEEE 31st International Conference on Tools with Artificial Intelligence. 2019. P. 1471–1479. https://doi.org/10.1109/ICTAI.2019.00209
  13. Moen E., Bannon D., Kudo T., Graf W., Covert M., Van Valen D. Deep learning for cellular image analysis // Nature Methods. 2019. Vol. 16, iss. 12. P. 1233–1246. https://doi.org/10.1038/s41592-019-0403-1
  14. Obukhov A., Krasnyanskiy M. Quality assessment method for GAN based on modified metrics inception score and Frechet inception distance // Proceedings of the Computational Methods in Systems and Software. 2020. P. 102–114. https://doi.org/10.1007/978-3-030-63322-6_8
  15. MNIST handwritten digit database, Yann LeCun, Corinna Cortes and Chris Burges. URL: http://yann.lecun.com/exdb/mnist/ (дата обращения: 05.04.2022).
  16. UCI Machine Learning Repository. URL: https://archive.ics.uci.edu/ml/datasets.php (дата обращения: 05.04.2022).
  17. Obukhov A., Siukhin A., Dedov D. The model of the automatic control system for a treadmill based on neural networks // 2020 International Multi-Conference on Industrial Engineering and Modern Technologies (FarEastCon). 2020. P. 1–5. https://dx.doi.org/10.1109/FarEastCon50210.2020.9271589
  18. Обухов А. Д., Краснянский М. Н. Нейросетевой метод обработки и передачи данных в адаптивных информационных системах // Вестник Удмуртского университета. Математика. Механика. Компьютерные науки. 2021. Т. 31, вып. 1. С. 149–164. https://doi.org/10.35634/vm210111


Creative Commons License
此作品已接受知识共享署名 4.0国际许可协议的许可
##common.cookie##