🔧На сайте запланированы технические работы
25.12.2025 в промежутке с 18:00 до 21:00 по Московскому времени (GMT+3) на сайте будут проводиться плановые технические работы. Возможны перебои с доступом к сайту. Приносим извинения за временные неудобства. Благодарим за понимание!
🔧Site maintenance is scheduled.
Scheduled maintenance will be performed on the site from 6:00 PM to 9:00 PM Moscow time (GMT+3) on December 25, 2025. Site access may be interrupted. We apologize for the inconvenience. Thank you for your understanding!

 

Cognitive processing of educational polycode text: An experimental eye-tracking study


Cite item

Full Text

Abstract

This study investigates the cognitive processing of educational PCTs by Russian-speaking readers, addressing the research gap concerning how this process is executed, regulated, and influenced by various factors. The primary aim is to explore the principles of text-image integration during the reading of an educational PCT by native Russian-speaking readers. Employing the concept that eye movements correspond to the logic of cognitive processing, the study uses eye-tracking methods to analyse this process. It relies on eye-movement data from 67 Russian students in the socio-humanities who read an educational polycode biology text in Russian. Findings reveal that the verbal component plays a dominant role in the cognitive processing of this text type, suggesting that this dominance is not dependent on the national educational culture. At the same time, the reading pattern is influenced by the learners’ experience with educational texts of a specific content type - in this case, biology. The study also confirms and identifies the mechanisms of the regulatory function of the verbal component during the reading of an educational PCT. The implications of this study are both theoretical and practical: it contributes to the understanding of how the cognitive processing of educational texts is carried out and what factors influence it and underscores the importance of the mindful use of images in the design of educational materials. Moreover, its findings can be applied in developing principles for teaching visual literacy.

About the authors

Inna V. Tubalova

Tomsk State University

Email: tina09@inbox.ru
ORCID iD: 0000-0001-6519-0739

Alena V. Garina

Tomsk State University

Email: alyonushka050700@gmail.com
ORCID iD: 0009-0006-9821-1675

References

  1. Anisimov, V. N., & Luzhin, A. O. (2020). Method of determining efficiency of visual presentation of text materials. Patent No. 2722440 C1. Russian Federation. EDN: JAVCJT
  2. Anisimova, E. E. (2003). Text linguistics and intercultural communication (on the material of creolized texts). Academia.
  3. Babakova, T. A. (2022). Scientific-methodical project on higher education pedagogy in the context of academic writing. Lifelong Education: The 21st Century, 3(39), 30-43. https://doi.org/10.15393/j5.art.2022.7846
  4. Bakhtin, M. (1986). The problem of the text. In C. Emerson & M. Holquist (Eds.), Speech genres and other late essays (pp. 101-131). University of Texas Press.
  5. Barthes, R. (2004). Introduction to the structural analysis of narratives. In M. Bal (Ed.), Narrative theory: Critical concepts in literary and cultural studies (pp. 65-116). Routledge.
  6. Bernatskaya, A. A. (2000). On the problem of text ‘creolization’: History and current state. Speech Communication: Specialized Messenger, 3, 104-110.
  7. Blinova, E., & Shcherbakova, O. (2020). Patterns of intellectual behaviour manifested when working with verbal and iconic texts. In A. V. Shaboltas (Ed.) Psychology of the XXI century: Current challenges and achievements (pp. 78-88). Skifiya-Print.
  8. Chernyavskaya, V. E. (2009). Text linguistics: Polysemy, intertextuality, interdiscursivity. LIBROKOM.
  9. de Beaugrande, R., & Dressler, W. U. (1981). Introduction to text linguistics. Longman.
  10. Derrida, J. (2016). Of grammatology. Jhu Press.
  11. Dragomilov, A. G., & Mash, R. D. (2008). Biology. 8th grade: Textbook for students of general educational institutions. Ventana-Graf.
  12. Eiger, G. V., & Yukht, V. L. (1974). Towards the construction of a typology of texts. In Proceedings of Research Conference on Text Linguistics (pp. 103-109). Moscow State Linguistic University Press.
  13. Fairclough, N. (2023). Critical discourse analysis. In M. Handford & J. P. Gee (Eds.), The Routledge handbook of discourse analysis (pp. 11-22). Routledge. https://doi.org/10.4324/9781003035244
  14. Forceville, C. (2006). Non-verbal and multi-modal metaphor in a cognitivist framework: Agendas for research. In G. Kristiansen, M. Achard, R. Dirven, & F. J. R. de Mendoza Ibáñez (Eds.), Cognitive linguistics: Current applications and future perspectives (pp. 379-402). Mouton de Gruyter.
  15. Foucault, M. (1971). Orders of discourse. Social Science Information, 10(2), 7-30. https://doi.org/10.1177/053901847101000201
  16. Halliday, M. A. K., & Hasan, R. (1976). Cohesion in English. Longman.
  17. Hannus, M., & Hyönä, J. (1999). Utilization of illustrations during learning of science textbook passages among low- and high-ability children. Contemporary Educational Psychology, 24(2), 95-123. https://doi.org/10.1006/ceps.1998.0987
  18. Hegarty, M. (1992). Mental animation: Inferring motion from static diagrams. Spatial Cognition & Computation, 2(3-4), 211-235.
  19. Hegarty, M., & Just, M. A. (1993). Constructing mental models of machines from text and diagrams. Journal of Memory and Language, 32(6), 717-742. https://doi.org/10.1006/jmla.1993.1036
  20. Hoover, W. A., & Gough, P. B. (1990). The simple view of reading. Reading and Writing, 2(2), 127-160. https://doi.org/10.1007/BF00401799
  21. Jewitt, C. (Ed.). (2009). The Routledge handbook of multimodal analysis. Routledge.
  22. Jian, Y.-C. (2013). How do students read diagrams with text in high school biology textbooks: An eye-tracking study. International Journal of Science Education, 35(3), 467-496.
  23. Jian, Y.-C. (2015). Visual literacy does not simply mean reading diagrams: The role of text-diagram integration and cognitive processing in comprehending illustrated science text. Journal of Science Education and Technology, 24, 43-55.
  24. Jian, Y.-C. (2016). Fourth graders’ cognitive processes and learning strategies for reading illustrated biology texts: Eye movement measurements. Reading Research Quarterly, 51(1), 93-109. https://doi.org/10.1002/rrq.125
  25. Jian, Y.-C., & Wu, C.-J. (2012). The effect of arrows in an illustration when reading scientific text: Evidence from eye movements. Chinese Journal of Psychology, 54, 385-402.
  26. Jian, Y.-C., & Wu, C.-J. (2015). Using eye tracking to investigate semantic and spatial representations of scientific diagrams during text- diagram integration. Journal of Science Education and Technology, 24(1), 43-55. https://doi.org/10.1007/s10956-014-9519-3
  27. Kalugin, Y. E., & Prokhorov, A. V. (2023). Understanding of educational texts. Universum: Psychology and Education, 4(106), 46-49.
  28. Kibrik, A. A. (2010). Multimodal analysis of discourse: A new area of research and a training course. Bulletin of Moscow State University. Series 9: Philology, 5, 114-125.
  29. Kolesov, D. V., Mash, R. D., & Belyaev, I. N. (2016). Biology. Human: Textbook for the 8th grade of general educational institutions. Drofa.
  30. Kosenko, V. S. (2019). The impact of a creolized text non-verbal and verbal parts on its perception (an experimental study). Polylinguality and Transcultural Practices, 16(3), 310-321. https://doi.org/10.22363/2618-897X-2019-16-3-310-321
  31. Kress, G. (2009). Multimodality: A social semiotic approach to contemporary communication. Routledge. https://doi.org/10.4324/9780203970034
  32. Kress, G., & van Leeuwen, T. (2020). Reading images: The grammar of visual design. Routledge. https://doi.org/10.4324/9781003099857
  33. Kristeva, J. (2009). Narration et transformation. De Gruyter Mouton.
  34. Kunitsyna, E. Y. (2010). Features of the perception of creolized texts (an experimental study on the material of agitational-political texts). Political Linguistics, 2, 115-119.
  35. Kuryanovich, A. V. (2022). Educational texts as didactic and methodological material in the professional activity of a Russian language teacher. In Proceedings of the 20th International Research Conference ‘Personal Development in the Educational Space’ (pp. 55-60). Shukshin Altai State Humanitarian Pedagogical University.
  36. Lacan, J. (1981). The language of the self: The function of language in psychoanalysis. JHU Press.
  37. Lee, W.-K., & Wu, C.-J. (2017). Eye movements in integrating geometric text and figure: Scanpaths and given-new effects. International Journal of Science and Mathematics Education, 16(4), 699-714. https://doi.org/10.1007/s10763-016-9790-2
  38. Lemke, J. L. (1998). Multiplying meaning: Visual and verbal semiotics in scientific text. In J. R. Martin & R. Veel (Eds.), Reading science: Critical and functional perspectives on literacies of science (pp. 87-113). Routledge.
  39. Levin, J. R. (1981). On functions of pictures in prose. In F. J. Pirozzolo & M. C. Wittrock (Eds.), Neuropsychological and cognitive processes in reading (pp. 203-228). Academic Press. https://doi.org/10.1016/B978-0-12-185030-2.50013-5
  40. Levin, J. R., Anglin, G. J., & Carney, R. N. (1987). On empirically validating functions of pictures in prose. In D. M. Willows & H. A. Houghton (Eds.), The psychology of illustration: Basic research (pp. 51-85). Springer-Verlag. http://dx.doi.org/10.1007/978-1-4612-4674-9_2
  41. Lin, J. J. H., & Lin, S. S. (2014). Cognitive load for configuration comprehension in computer-supported geometry problem solving: An eye movement perspective. International Journal of Science and Mathematics Education, 12(3), 605-627. https://doi.org/10.1007/s10763-013-9479-8
  42. Luria, A. R. (2009). The main problems of neurolinguistics. LIBROKOM.
  43. Machin, D. (2007). Introduction to multimodal analysis. Hodder Arnold.
  44. Mason, L., Tornatora, M. C., & Pluchino, P. (2013). Do fourth graders integrate text and picture in processing and learning from an illustrated science text? Evidence from eye-movement patterns. Computers & Education, 60(1), 95-109. https://doi.org/10.1016/j.compedu.2012.07.011
  45. Mayer, R. E. (2005). The Cambridge handbook of multimedia learning. Cambridge University Press.
  46. Mayer, R. E. (2009). Multimedia learning (2nd ed.). Cambridge University Press.
  47. Mayer, R. E., & Gallini, J. K. (1990). When is an illustration worth ten thousand words? Journal of Educational Psychology, 82(4), 715-726. https://doi.org/10.1037/0022-0663.82.4.715
  48. Nekrasova, E. D. (2014). On multimodal perception of the text (a psycholinguistic experiment). Tomsk State University Journal, 378, 45-48. EDN: RZCHTJ
  49. O’Halloran, K. L. (Ed.). (2004). Multimodal discourse analysis: Systemic-functional perspectives. Continuum.
  50. Paivio, A. (1986). Mental representations: A dual coding approach. Oxford University Press.
  51. Pêcheux, M., Hak, T., Helsloot, N., & Macey, D. (1995). Automatic discourse analysis. Rodopi.
  52. Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3), 372-422. https://doi.org/10.1037/0033-2909.124.3.372
  53. Rayner, K., Rotello, C. M., Stewart, A. J., Keir, J., & Duffy, S. A. (2001). Integrating text and pictorial information: Eye movements when looking at print advertisements. Journal of Experimental Psychology: Applied, 7(3), 219-226. https://doi.org/10.1037/1076-898X.7.3.219
  54. Reichle, E. D., Rayner, K., & Pollatsek, A. (2009). The E-Z reader model of eye-movement control in reading. Behavioral and Brain Sciences, 26(4), 445-476. https://doi.org/10.1017/s0140525x03000104
  55. Rokhlov, V. S., & Trofimov, S. B. (2007). Biology. 8th grade: Man and his health. Textbook for general educational institutions. Mnemozina.
  56. Sabinina, A. A. (2009). Training text: Structure and pragmatics. Herzen University Journal of Humanities & Sciences, 97, 222-225. EDN: KJBLDZ
  57. Sapin, M. R., & Bilich, G. L. (2010). Normal human biology: Book 1. Medical Information Agency.
  58. Schnotz, W. (2005). An integrated model of text and picture comprehension. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 49-69). Cambridge University Press.
  59. Schnotz, W., & Horz, H. (2009). On the role of embodiment in the comprehension of static and animated displays. In R. Lowe & W. Schnotz (Eds.), Learning with animation: Research and implications for design (pp. 89-111). Cambridge University Press.
  60. Scollon, R., & Scollon, S. W. (2003). Discourses in place: Language in the material world. Routledge. https://doi.org/10.4324/9780203422724
  61. Sériot, P. (1997). Ethnos et demos: la construction discursive de l'identité collective. Langage & Société, 79(1), 39-51.
  62. Shelestyuk, E. V. (2010). Laws and stages of text comprehension. Issues of Cognitive Linguistics, 2(23), 85-90. EDN: MVDFNZ
  63. Slough, S. W., McTigue, E. M., Kim, S., & Jennings, S. (2010). Science textbooks’ use of graphical representation: A descriptive analysis of four sixth-grade science texts. Reading Psychology, 31(3), 301-325.
  64. Sonin, A. G. (2005). Modelling the mechanisms of understanding polycode texts [Doctoral dissertation, Institute of Linguistics, Russian Academy of Sciences]. Russian State Library. https://search.rsl.ru/en/record/01003258360
  65. Sorokin, Y. A., & Tarasov, E. F. (1990). Creolized texts and their communicative function. In R. G. Kotov (Ed.), Optimisation of speech impact (pp. 180-186). Nauka.
  66. Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4), 295-312. https://doi.org/10.1016/0959-4752(94)90003-5
  67. Tyutkova, I. A. (2016). Pedagogical thesaurus: A study guide. Institute for Lifelong Learning.
  68. Valgina, N. S. (2003). Text theory. Logos.
  69. van Dijk, T. A. (1997). Discourse as structure and process. Sage.
  70. van Dijk, T. A. (2015). Critical discourse analysis. In D. Tannen, H. E. Hamilton, & D. Schiffrin (Eds.), The handbook of discourse analysis (pp. 466-485). Wiley. https://doi.org/10.1002/9781118584194.ch22
  71. van Leeuwen, T. (2005). Introducing social semiotics: An introductory tectbook. Routledge.
  72. Vashunina, I. V., Tarasov, E. F., Nistratov, A. A., & Matveev, M. O. (Eds.). (2020). Creolized text: Semantic perception. Institute of Linguistics, Russian Academy of Sciences.
  73. Wodak, R. (2004). Critical discourse analysis. Qualitative Research Practice, 185, 185-204.
  74. Zlokazov, K. V., & Lipnitsky, A. V. (2018). Violence in a creolized text: A study of the patterns of understanding. Political Linguistics, 71, 135-143.

Supplementary files

Supplementary Files
Action
1. JATS XML
Download

Copyright (c) 2025 Training, Language and Culture