A method for detecting objects in images based on neural networks on graphs and a small number of training examples

Cover Page

Cite item

Full Text

Abstract

In the presented work, the object of research is computer vision systems. The subject of the study is a method for detecting objects in images based on neural networks on graphs and a small number of training examples. Such aspects of the topic as the use of a structural representation of the scene to improve the accuracy of object detection are discussed in detail. It is proposed to share information about the structure of the scene based on neural networks on graphs and training from "multiple shots" to increase the accuracy of object detection. Relationships between classes are established using external semantic links. To do this, a knowledge graph is pre-created. The method contains two stages. At the first stage, object detection is performed based on training with "multiple shots". At the second stage, the detection accuracy is improved using a neural network on graphs. The basis of the developed method is the use of convolution based on spectral graph theory. Each vertex represents a category in the knowledge graph, and the edge weight of the graph is calculated based on conditional probability. Based on the convolution, information from neighboring vertices and edges is combined to update the vertex values. The scientific novelty of the developed method lies in the joint use of convolutional networks on graphs and training from "multiple shots" to increase the accuracy of object detection. A special contribution of the author to the research of the topic is the use of a convolutional network based on a knowledge graph to improve the results of the object detection method using a small number of training examples. The method was studied on test sets of images from the field of computer vision. Using the PASCAL VOC and MS COCO datasets, it is demonstrated that the proposed method increases the accuracy of object detection by analyzing structural relationships. The average accuracy of object detection using the developed method increases by 1-5% compared to the "multiple shots" training method without using a structural representation.

References

  1. Zou Z., Chen K., Shi Z., Guo Y., Ye J. Object Detection in 20 Years: A Survey // Proceedings of the IEEE. 2023. Vol. 111 (3). Pp. 257-276.
  2. Redmon J., Divvala S., Girshick R., Farhadi A. You only look once: Unified, real-time object detection // IEEE Conference on Computer Vision and Pattern Recognition. 2016. Pp. 779-788.
  3. Liu W., Anguelov D., Erhan D., Szegedy C., Reed S., Fu C.Y., Berg A. C. Ssd: Single shot multibox detector // European Conference on Computer Vision. 2016. Pp. 21-37.
  4. Lin T.Y., Goyal P., Girshick R., He K., Dollar P. Focal loss for dense object detection // IEEE Transactions on Pattern Analysis and Machine Intelligence. 2018. Vol. 42(2). Pp. 318-327.
  5. Girshick P. Fast R-CNN // 2015 IEEE International Conference on Computer Vision (ICCV). 2015. Pp. 1440-1448.
  6. Ren S., He K., Girshick R., Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks // Advances in Neural Information Processing System. 2015. Pp. 91-99.
  7. He K., Gkioxari G., Dollar P., Girshick R. Mask R-CNN // Proceedings of the IEEE International Conference on Computer Vision. 2017. Pp. 2961-2969.
  8. Köhler M., Eisenbach M., Gross H. M. Few-Shot Object Detection: A Survey // IEEE Transactions on Neural Networks and Learning Systems. 2024. Vol. 35 (9). Pp. 11958-11978.
  9. Huang G., Laradji I., Vazquez D., Lacoste-Julien S., Rodriguez P. A Survey of Self-Supervised and Few-Shot Object Detection // IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023. Vol. 45(4). Pp. 4071-4089.
  10. Wu J., Liu S., Huang D., Wang Y. Multi-scale positive sample refnement for few-shot object detection // European Conference on Computer Vision. 2020. Pp. 456-472.
  11. Wang X., Huang T. E., Gonzalez J., Darrell T., Yu F. Frustratingly simple few-shot object detection // Proceedings of the 37th International Conference on Machine Learning (ICML). 2020. Pp. 9919-9928.
  12. Kang B., Liu Z., Wang X., Yu F., Feng J., Darrell T. Few-shot object detection via feature reweighting // 2019 IEEE/CVF International Conference on Computer Vision. 2019.
  13. Захаров А.А., Тужилкин А.Ю. Сегментация спутниковых изображений на основе суперпикселей и разрезов на графах // Программные системы и вычислительные методы. 2018. № 1. С. 7-17. doi: 10.7256/2454-0714.2018.1.25629 URL: https://e-notabene.ru/itmag/article_25629.html
  14. Захаров. А.А., Титов Д.В., Жизняков А.Л., Титов В.С. Метод визуального внимания на основе ранжирования вершин графа по разнородным признакам изображений // Компьютерная оптика. 2020. Т. 44, № 3. С. 427-435.
  15. Barinov A.E., Zakharov A.A. Clustering using a random walk on graph for head pose estimation // International Conference on Mechanical Engineering, Automation and Control Systems, MEACS. 2015.
  16. Cao P., Zhu Z., Wang Z., Zhu Y., Niu Q. Applications of graph convolutional networks in computer vision // Neural Computing and Applications. 2022. № 34. Pp. 13387-13405.
  17. Kipf T.N. Deep Learning with Graph-Structured Representations, Universiteit van Amsterdam, 2020.
  18. Li W., Liu X., Yuan Y. SIGMA++: Improved Semantic-Complete Graph Matching for Domain Adaptive Object Detection // IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023. Vol. 45 (7). Pp. 9022-9040.
  19. Chen C., Li J., Zhou H.Y., Han X., Huang Y., Ding X., Yu Y. Relation matters: Foreground-aware graph-based relational reasoning for domain adaptive object detection // IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023. Vol. 45 (3). Pp. 3677-3694.
  20. Chen T., Lin L., Chen R., Hui X., Wu X. Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognition // IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022. Vol. 44 (3). Pp.1371-1384.
  21. Liu Z., Jiang Z., Feng W., Feng H. OD-GCN: Object Detection Boosted by Knowledge GCN // 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). 2020.

Supplementary files

Supplementary Files
Action
1. JATS XML
Download

Согласие на обработку персональных данных

 

Используя сайт https://journals.rcsi.science, я (далее – «Пользователь» или «Субъект персональных данных») даю согласие на обработку персональных данных на этом сайте (текст Согласия) и на обработку персональных данных с помощью сервиса «Яндекс.Метрика» (текст Согласия).