Email this article Gesture control of small unmanned aerial vehicle flight
- Authors: Abramov N.S.1, Sattarova V.V.2, Fralenko V.P.1, Khachumov M.V.1,3,4,5
-
Affiliations:
- Ailamazyan Program Systems Institute of RAS
- RUDN University
- Federal Research Center "Computer Science and Control" of RAS
- MIREA - Russian Technological University
- Russian state university for the humanities
- Issue: Vol 15, No 2 (2024)
- Pages: 21-36
- Section: Articles
- URL: https://journals.rcsi.science/2079-3316/article/view/299198
- DOI: https://doi.org/10.25209/2079-3316-2024-15-2-21-36
- ID: 299198
Cite item
Full Text
Abstract
The problem of constructing gesture commands for controlling a small unmanned aerial vehicle, such as a quadcopter, is considered. Commands coming from a video camera are identified by a classifier based on a convolutional neural network, and the multimodal control interface equipped with an intelligent solver converts them into control commands for the quadcopter. Neural networks from the Ultralytics neural network library allow selecting targets in a frame in real-time. The commands are sent to a specialized program on a smartphone, developed on the basis of DJI SDK flight simulators, which then sends commands via the remote control channel.The quality of recognition of developed gesture commands for DJI Phantom 3 standard edition quadcopters is investigated, and a brief guide in the form of operator work scenarios with unmanned vehicles is provided. The prospects of gesture control of several vehicles in extreme conditions have been revealed, considering the complex safety challenges of joint flight and interaction of aircraft in confined space.
About the authors
Nikolai Sergeevich Abramov
Ailamazyan Program Systems Institute of RAS
Email: n-say@nsa.pereslavl.ru
Vita Viktorovna Sattarova
RUDN University
Email: 1032201655@pfur.ru
Vitaly Petrovich Fralenko
Ailamazyan Program Systems Institute of RAS
Email: alarmod@pereslavl.ru
Mikhail Vyacheslavovich Khachumov
Ailamazyan Program Systems Institute of RAS; Federal Research Center "Computer Science and Control" of RAS; MIREA - Russian Technological University; Russian state university for the humanities
Email: khmike@inbox.ru
References
- Абрамов Н. С., Талалаев А. А., Фраленко В. П., Хачумов М. В.. «Система мультимодального управления и визуализации полета беспилотного летательного аппарата», Авиакосмическое приборостроение, 2023, №9, с. 3–11.
- Алфимцев А. Н.. Разработка и исследование методов захвата, отслеживания и распознавания динамических жестов, М., 2008, 167 с.
- Нагапетян В. Э.. Методы распознавания жестов руки на основе анализа дальностных изображений, М., 2008, 117 с.
- Oikonomidis I., Kyriazis N., Argyros A. A.. “Efficient model-based 3D tracking of hand articulations using Kinect”, Proceedings of the 22nd British Machine Vision Conference 2011, BMVC'11 (Dundee, UK, August 29–September 2, 2011), 2011, ISBN 1-901725-43-X, 11 pp.
- Shaowei C., Tanaka J.. “Interacting with a self-portrait camera using motion-based hand gestures”, Proceedings of the 11th Asia-Pacific Conference on Computer-Human Interaction 2013, APCHI'13 (Bangalore, India, September 24–27, 2013), ACM, New York, 2013, ISBN 978-1-4503-2253-9, pp. 93–101.
- Maraqa M. R., Al-Zboun F., Dhyabat M., Zitar R. A.. “Recognition of Arabic Sign Language (ArSL) using recurrent neural networks”, Journal of Intelligent Learning Systems and Applications, 4:1 (2012), pp. 41–52.
- Pugeault N., Bowden R.. “Spelling it out: real-time ASL fingerspelling recognition”, Proceedings of the IEEE International Conference on Computer Vision Workshops 2011, ICCV'11 (Barcelona, Spain, 06–13 November 2011), 2011, pp. 1114–1119.
- Zhao R., Wang K., Divekar R., Rouhani R., Su H., Ji Q.. “An immersive system with multi-modal human-computer interaction”, Proceedings of the 13th IEEE International Conference on Automatic Face & Gesture Recognition 2018 (Xi'an, China, 15–19 May 2018), 2018, pp. 517–524.
- Sanna A., Lamberti F., Paravati G., Manuri F.. “A kinect-based natural interface for quadrotor control”, Entertainment Computing, 4:3 (2013), pp. 179–186.
- Нагапетян В. Э., Хачумов В. М.. «Распознавание жестов руки в задаче бесконтактного управления беспилотным летательным аппаратом», Автометрия, 51:2 (2015), с. 103–109.
- Абрамов Н. С., Емельянова Ю. Г., Талалаев А. А., Фраленко В. П., Хачумов М. В.. «Архитектура мультимодального интерфейса для управления беспилотным летательным аппаратом», Известия высших учебных заведений. Авиационная техника, 2022, №3, с. 55–63.
- Hu M. K.. “Visual pattern recognition by moment invariants”, IRE Transactions on Information Theory, 8:2 (1962), pp. 179–187.
- Hochreiter S., Schmidhuber J.. “Long short-term memory”, Neural Computation, 9:8 (1997), pp. 1735–1780.
- Жуковская В. А., Пятаева А. В.. «Рекуррентная нейронная сеть для распознавания жестов русского языка с учетом языкового диалекта Сибирского региона», ГрафиКон 2022: материалы 32-й Международной конференции по компьютерной графике и машинному зрению (Рязань, 19–22 сентября 2022 г.), 2022, с. 538–547.
- Булыгин Д. А., Мамонова Т. Е.. «Распознавание жестов рук в режиме реального времени», Научный вестник НГТУ, 2020, №1(78), с. 25–40.
- Zhao Y., Lv W., Xu S., Wei J., Wang G., Dang Q., Liu Y., Chen J.,. DETRs beat YOLOs on real-time object detection, 2023, 14 pp.
Supplementary files
