AI-feedback in education: user experience analysis

Cover Page

Cite item

Full Text

Abstract

The subject of the study is the subjective experience and perception of students' feedback system based on generative artificial intelligence (AI) in real conditions of the educational process. The aim of the work was to identify the key advantages and disadvantages of the AI-assessor technology and to determine the factors affecting the trust and satisfaction of its users. The research methodology is based on a qualitative approach using combined data analysis. The empirical base of the study: 69 spontaneous user feedbacks obtained during the pilot implementation of the “AI-evaluator” system in the programs of additional professional education. The methods of quantitative (tone analysis, frequency analysis) and qualitative (thematic analysis, argumentation analysis) analysis were used. The scientific novelty consists in the qualitative analysis of the subjective experience gained by students in the course of direct interaction with educational content within the framework of domestic educational programs. For the first time the influence of the unique aspect of AI “characters” design on students' perception of the technology has been considered in detail. The study revealed polarized user attitudes toward the AI assistant. On the one hand, responsiveness, 24/7 availability and innovative elements, particularly the use of different AI ‘characters’, were highly valued, increasing engagement and creating a safe educational environment. On the other hand, significant problems are recorded: inconsistency and errors in assessment, formalism, lack of understanding of context and technical glitches that undermine trust in the system and cause frustration. The data analysis showed that the polarization of opinions is due to the conflict between the high innovation potential of the technology and its current functional limitations. The need is emphasized not only for technical improvement of algorithms (including dealing with fairness and bias), but also for the development of hybrid interaction models that combine automation with human control and expertise to build epistemic trust in users. The results can be applied by developers of educational AI tools as well as educators and methodologists to improve practices of implementation and use of such system in work.

References

  1. Tzirides A. O., Zapata G. C., Bolger P., Cope B., Kalantzis M., Searsmith D. Exploring instructors' views on fine-tuned generative AI feedback in higher education // International Journal on E-Learning. 2024. No. 23(3). Pp. 319-334.
  2. Anthony J., Fisch-Ferguson J. Generative AI: Student perception on ChatGPT prompt feedback in post-secondary online education // Proceedings of the World Conference on Social Sciences and Humanities. 2024. No. 1(1). Pp. 1-15.
  3. Sain Z. H., Vasudevan A., Thelma C. C., Asfahani A. Harnessing ChatGPT for effective assessment and feedback in education // Journal of Computer Science and Informatics Engineering. 2024. No. 3(2). Pp. 74-82. doi: 10.55537/cosie.v3i2.856 EDN: IGJOQS.
  4. Li H., Xing W., Li C., Zhu W., Lyu B., Zhang F., Liu Z. Who should be my tutor? Analyzing the interactive effects of automated text personality styles between middle school students and a mathematics chatbot // In Proceedings of the 15th International Learning Analytics and Knowledge Conference (LAK '25). Association for Computing Machinery, 2025. P. 910-917.
  5. Lyu B., Li C., Li H., Oh H., Song Y., Zhu W., Xing W. Exploring the role of teachable AI agents' personality traits in shaping student interaction and learning in mathematics education // In Proceedings of the 15th International Learning Analytics and Knowledge Conference (LAK '25). Association for Computing Machinery, 2025. P. 887-894.
  6. Lee S. S., Moore R. L. Harnessing generative AI (GenAI) for automated feedback in higher education: A systematic review // Online Learning. 2024. No. 28(3). Pp. 82-104.
  7. Qadeer A. The mediating impact of student engagement on the association between generative AI-based feedback and academic performance in higher education // Journal of Research, Innovation, and Strategies for Education. 2025. No. 2(1). Pp. 29-44.
  8. Kwako A., Wan Y., Zhao J., Chang K.-W., Cai L., Hansen M. Does BERT exacerbate gender or L1 biases in automated English speaking assessment? // In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023). Association for Computational Linguistics, 2025. P. 668-681.
  9. Lai P., Lau I., Pang R. Exploring the efficacy of rubric-based AI feedback in enhancing student writing outcomes // In 2024 6th International Workshop on Artificial Intelligence and Education (WAIE). IEEE, 2024. P. 220-224.
  10. Сысоев П. В., Филатов Е. М., Хмаренко Н. И., Мурунов С. С. Преподаватель vs искусственный интеллект: Сравнение качества предоставляемой преподавателем и генеративным искусственным интеллектом обратной связи при оценке письменных творческих работ студентов // Перспективы науки и образования. 2024. № 71(5). С. 694-712. doi: 10.32744/pse.2024.5.41 EDN: XZGVGM.
  11. Боголепова С. В. Возможности инструментов искусственного интеллекта для проверки письменных работ и формулировки обратной связи // Дискурс профессиональной коммуникации. 2025. № 7(1). С. 70-88. doi: 10.24833/2687-0126-2025-7-1-70-88 EDN: IMTFUB.
  12. Cheong Y. A Study of the effectiveness of student perception-based AI feedback in college writing classes // Korean Journal of General Education. 2024. No. 18(5). Pp. 159-173.
  13. Ahmed I., Liu W., Roscoe R. D., Reilley E., McNamara D. S. Multifaceted assessment of responsible use and bias in language models for education // Computers. 2025. No. 14(3). P. 100. doi: 10.3390/computers14030100 EDN: KSZZRK.
  14. Zhang M., Johnson M., Ruan C. Investigating sampling impacts on an LLM-based AI scoring approach: Prediction accuracy and fairness // Journal of Measurement and Evaluation in Education and Psychology. 2024. No. 15. Pp. 348-360.
  15. Khlaif Z. N., Ayyoub A., Hamamra B., Bensalem E., Mitwally M. A. A., Ayyoub A., Hattab M. K., Shadid F. University teachers' views on the adoption and integration of generative AI tools for student assessment in higher education // Education Science. 2024. No. 14(10). P. 1090.
  16. Сысоев П. В. Этика и ИИ-плагиат в академической среде: Понимание студентами вопросов соблюдения авторской этики и проблемы плагиата в процессе взаимодействия с генеративным искусственным интеллектом // Высшее образование в России. 2024. № 33(2). С. 31-53. doi: 10.31992/0869-3617-2024-33-2-31-53 EDN: VTAIUO.
  17. Keshtkar F., Rastogi N., Chalarca S., Bukhari S. A. C. AI tutor: Student's perceptions and expectations of AI-driven tutoring systems: A survey-based investigation // The International FLAIRS Conference Proceedings. 2024. No. 37(1).
  18. Токмакова Ю. В., Саенко Е. С. Использование корректирующей обратной связи от генеративного искусственного интеллекта в обучении профессиональному иностранному языку студентов аграрного вуза // Вестник Тамбовского университета. Серия: Гуманитарные науки. 2025. № 30(1). С. 50-66. doi: 10.20310/1810-0201-2025-30-1-50-66 EDN: GSFFPP.

Supplementary files

Supplementary Files
Action
1. JATS XML
Download

Согласие на обработку персональных данных

 

Используя сайт https://journals.rcsi.science, я (далее – «Пользователь» или «Субъект персональных данных») даю согласие на обработку персональных данных на этом сайте (текст Согласия) и на обработку персональных данных с помощью сервиса «Яндекс.Метрика» (текст Согласия).