Building an Online Learning Model Through a Dance Recognition Video Based on Deep Learning
- Authors: Hung N.V1, Loi T.Q1, Binh N.H1, Nga N.T1, Huong T.T2, Luu D.L3
-
Affiliations:
- East Asia University of Technology
- Hanoi University of Science and Technology
- University of Science and Technology
- Issue: Vol 23, No 1 (2024)
- Pages: 101-128
- Section: Artificial intelligence, knowledge and data engineering
- URL: https://journals.rcsi.science/2713-3192/article/view/267189
- DOI: https://doi.org/10.15622/ia.23.1.4
- ID: 267189
Cite item
Full Text
Abstract
Jumping motion recognition via video is a significant contribution because it considerably impacts intelligent applications and will be widely adopted in life. This method can be used to train future dancers using innovative technology. Challenging poses will be repeated and improved over time, reducing the strain on the instructor when performing multiple times. Dancers can also be recreated by removing features from their images. To recognize the dancers’ moves, check and correct their poses, and another important aspect is that our model can extract cognitive features for efficient evaluation and classification, and deep learning is currently one of the best ways to do this for short-form video features capabilities. In addition, evaluating the quality of the performance video, the accuracy of each dance step is a complex problem when the eyes of the judges cannot focus 100% on the dance on the stage. Moreover, dance on videos is of great interest to scientists today, as technology is increasingly developing and becoming useful to replace human beings. Based on actual conditions and needs in Vietnam. In this paper, we propose a method to replace manual evaluation, and our approach is used to evaluate dance through short videos. In addition, we conduct dance analysis through short-form videos, thereby applying techniques such as deep learning to assess and collect data from which to draw accurate conclusions. Experiments show that our assessment is relatively accurate when the accuracy and F1-score values are calculated. More than 92.38% accuracy and 91.18% F1-score, respectively. This demonstrates that our method performs well and accurately in dance evaluation analysis.
Keywords
About the authors
N. V Hung
East Asia University of Technology
Author for correspondence.
Email: hungnv@eaut.edu.vn
Ky Phu - Ky Anh -
T. Q Loi
East Asia University of Technology
Email: thangquangloi21@gmail.com
Xuan Long, Yen Binh -
N. H Binh
East Asia University of Technology
Email: binhnh@eaut.edu.vn
Chien Thang - Thi Tran Xuan Mai - Chuong My -
N. T Nga
East Asia University of Technology
Email: ngantt@eaut.edu.vn
Ta Thanh Oai - Thanh Tri -
T. T Huong
Hanoi University of Science and Technology
Email: huong.truongthu@hust.edu.vn
Dai Co Viet St. 1
D. L Luu
University of Science and Technology
Email: ldlich@dut.udn.vn
Nguyen Luong Bang St. 54
References
- Zhai X. Dance movement recognition based on feature expression and attribute mining. Complexity. 2021. vol. 2021. pp. 1–12.
- Krishna V.B. Ballroom dance movement recognition using a smart watch. arXiv preprint. 2020. arXiv:2008.10122.
- Sun Y., Chen J. Human movement recognition in dancesport video images based on chaotic system equations. Advances in Mathematical Physics. 2021. vol. 2021. pp. 1–12.
- Zhang J., Sun J., Wang J., Yue X.-G. Visual object tracking based on residual network and cascaded correlation filters. Journal of ambient intelligence and humanized computing. 2021. vol. 12. pp. 8427–8440.
- Zhou Q., Wang J., Wu P., Qi Y. Application development of dance pose recognition based on embedded artificial intelligence equipment. Journal of Physics: Conference Series. 2021. vol. 1757(1). no. 012011.
- Nguyen H., Dao T.N., Pham N.S., Dang T.L., Nguyen T.D., Truong T.H. An accurate viewport estimation method for 360 video streaming using deep learning. EAI Endorsed Transactions on Industrial Networks and Intelligent Systems. 2022. vol. 9. no. 4. doi: 10.4108/eetinis.v9i4.2218.
- Wang S., Li J., Cao T., Wang H., Tu P., Li Y. Dance emotion recognition based on laban motion analysis using convolutional neural network and long short-term memory. IEEE Access. 2020. vol. 8. pp. 124928–124938.
- Ding Y., Zhang Z., Zhao X., Cai Y., Li S., Deng B., Cai W. Self-supervised locality preserving low-pass graph convolutional embedding for large-scale hyperspectral image clustering. IEEE Transactions on Geoscience and Remote Sensing. 2022. vol. 60. pp. 1–16.
- Hung N., Loi T., Huong N., Hang T.T., Huong T. Aafndl – an accurate fake information recognition model using deep learning for the vietnamese language. Informatics and Automation. 2023. vol. 22. no. 4. pp. 795–825.
- Hao S., Zhou Y., Guo Y. A brief survey on semantic segmentation with deep learning. Neurocomputing. 2020. vol. 406. pp. 302–321.
- Cai W., Song Y., Duan H., Xia Z., Wei Z. Multi-feature fusion-guided multiscale bidirectional attention networks for logistics pallet segmentation. Computer Modeling in Engineering and Sciences. 2022. vol. 131. no. 3. pp. 1539–1555.
- Zhao M., Chang C.H., Xie W., Xie Z., Hu J. Cloud shape classification system based on multi-channel cnn and improved fdm. IEEE Access. 2020. vol. 8. pp. 44111–44124.
- Bakalos N., Rallis I., Doulamis N., Doulamis A., Protopapadakis E., Voulodimos A. Choreographic pose identification using convolutional neural networks. 11th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games). IEEE, 2019. pp. 1–7.
- Rani C.J., Devarakonda N. An effectual classical dance pose estimation and classification system employing convolution neural network–long shortterm memory (CNN-LSTM) network for video sequences. Microprocessors and Microsystems. 2022. vol. 95. no. 104651.
- Yang Y., Yu D., Yang C. Video transaction algorithm considering fisco alliance chain and improved trusted computing. PeerJ Computer Science. 2021. vol. 7. no. e594.
- Hu K., Jin J., Zheng F., Weng L., Ding Y. Overview of behavior recognition based on deep learning. Artificial Intelligence Review. 2023. vol. 56. no. 3. pp. 1833–1865.
- Matsuyama H., Aoki S., Yonezawa T., Hiroi K., Kaji K., Kawaguchi N. Deep learning for ballroom dance recognition: A temporal and trajectory-aware classification model with three-dimensional pose estimation and wearable sensing. IEEE Sensors Journal. 2021. vol. 21. no. 22. pp. 25437–25448.
- Ng L.H.X., Tan J.Y.H., Tan D.J.H., Lee R.K.-W. Will you dance to the challenge? predicting user participation of TikTok challenges. Proceedings of the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining. 2021. pp. 356–360.
- He H., Luo Q. Online teaching mode of college sports dance course under the background of internet plus. International Conference on Information Technology and Contemporary Sports (TCS). 2021. pp. 160–164.
- Zhu X. Research on the application of digital media technology in sports dance teaching. International Conference on Education, Information Management and Service Science (EIMSS). 2021. pp. 22–26.
- Hu Z. Research on the application of virtual reality technology in the teaching of sports dance in colleges and universities. 2nd International Conference on Artificial Intelligence and Education (ICAIE). 2021. pp. 414–418.
- Kritsis K., Gkiokas A., Pikrakis A., Katsouros V. Danceconv: Dance motion generation with convolutional networks. IEEE Access. 2022. vol. 10. pp. 44982–45000.
- Chen Y., Li X. Research on the application of flipped classroom model in college sports dance teaching. International Conference on Information Technology and Contemporary Sports (TCS). 2021. pp. 508–511.
- Li Y., Xu K. Online sports dance body contour extraction and training algorithm based on dsp chip intelligent high-definition camera image processing. 7th International Conference on Communication and Electronics Systems (ICCES). 2022. pp. 908–911.
- Kaur M., Mohta A. A review of deep learning with recurrent neural network. International Conference on Smart Systems and Inventive Technology (ICSSIT). 2019. pp. 460–465.
- Rallis I., Voulodimos A., Bakalos N., Protopapadakis E., Doulamis N., Doulamis A. Machine learning for intangible cultural heritage: a review of techniques on dance analysis. Visual Computing for Cultural Heritage. 2020. pp. 103–119.
- Biswal A. Recurrent neural network (RNN) tutorial: Types, examples, LSTM and more. Simplilearn. Com. Retrieved. 2022. Available at: https://www.simplilearn.com/tutorials/deep-learning-tutorial/rnn (accessed: 05.10.2023).
- Zargar S. Introduction to sequence learning models: RNN, LSTM, GRU. Department of Mechanical and Aerospace Engineering, North Carolina State University. 2021. doi: 10.13140/RG.2.2.36370.99522.
- Zhang W., Li H., Tang L., Gu X., Wang L., Wang L. Displacement prediction of jiuxianping landslide using gated recurrent unit (gru) networks. Acta Geotechnica. 2022. vol. 17. no. 4. pp. 1367–1382.
- Li W., Wei Y., An D., Jiao Y., Wei Q. LSTM-TCN: Dissolved oxygen prediction in aquaculture, based on combined model of long short-term memory network and temporal convolutional network. Environmental Science and Pollution Research. 2022. vol. 29. no. 26. pp. 39545–39556.
Supplementary files
