Aspects of Studying Gestures in Languages of Different Modalities: Signed and Spoken Languages

Cover Page

Cite item

Full Text

Abstract

This article analyzes gestures in languages of visual and spoken modalities, as well as their role in these communicative systems. The authors view gestures as an important element of spoken interaction, emphasizing their key role not only in nonverbal communication but also in the structure of sign languages. The purpose of the article is to review current trends in multimodal research, with particular attention to the comparison of gestures in the languages of the deaf and co-speech gestures in spoken languages. The article also presents several gesture types and examines their role and interaction in communication. Special focus is placed on the differences between gestures as extralinguistic means (in both spoken and sign languages) and gestures as part of linguistic systems. The material draws on interdisciplinary data and results of comparative studies. Based on the literature review, the authors conclude that multimodal studies are critical, as they clarify issues related to the role of gestures in communication across modalities.

About the authors

Anna Vasilievna Leonteva

Moscow State Linguistic University

Author for correspondence.
Email: lentevanja27@gmail.com

PhD in Philology, Head of the Laboratory for Sign Language Research, Senior Researcher at the Center for Socio-Cognitive Discourse Studies

Russian Federation

Varvara Denisovna Kharitonova

Moscow State Linguistic University

Email: varvaradk@yandex.ru

Junior Researcher at the Laboratory for Sign Language Research at the Center for Socio-Cognitive Discourse Studies

Russian Federation

References

  1. Iriskhanova, O. K. (2021). Polymodal discourse as an object of research. In Iriskhanova, O. K. (Ed.), Polimodal’nye izmereniya diskursa (pp. 15–33). Moscow: Izdatel’skiy Dom YaSK. (In Russ.)
  2. Goldin-Meadow, S., Brentari, D. (2017). The coming of age of sign language and gesture studies. Behavioral and Brain Sciences, 40, e59. doi: 10.1017/S0140525X15001247.
  3. Duncan, S. (2005). Gesture in Signing: Analyses of “Gesture” in the Signed Narrative of Learners of Taiwan Sign Language. Gesture, 5(1), 43–74.
  4. Sandler, W. (2009). Symbiotic symbolization by hand and mouth in sign language. Semiotica, 174(1/4), 241–275.
  5. McNeill, D. (1992). Hand and mind: what gestures reveal about thoughts. Chicago, IL: University of Chicago Press.
  6. Singleton, J. L., Morford, J. P., Goldin-Meadow, S. (1993). Once is not enough: standards of well-formedness in manual communication created over three different timespans. Language, 69, 683–715.
  7. Schembri, A., Jones, C., Burnham, D. (2005). Comparing action gestures and classifier verbs of motion: evidence from Australian Sign Language, Taiwan Sign Language, and nonsigners’ gestures without speech. Journal of Deaf Studies and Deaf Education, 10, 272–290.
  8. Emmorey, K., McCullough, S., Brentari, D. (2003). Categorial perception in American Sign Language. Language and Cognitive Processes, 18(1), 21–45.
  9. Emmorey, K. et al. (2011). Sign language and pantomime production differentially engage frontal and parietal cortices. Lang Cogn Process, 26(7), 878–901. doi: 10.1080/01690965.2010.492643.
  10. Emmorey, K. et al. (2013). The biology of linguistic expression impacts neural correlates for spatial language. Journal of Cognitive Neuroscience, 25, 517–533.
  11. Stokoe, W. C. (1960). Sign Language Structure: An Outline of the Visual Communication Systems of the American Deaf. Buffalo, NY: University of Buffalo Press.
  12. Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge: Cambridge University Press.
  13. Fenlon, J. et al. (2019). Comparing sign language and gesture: insights from pointing. Glossa, 4(1), 2. doi: 10.5334/gjgl.499.
  14. Pfau, R., Steinbach, M. (2006). Modality-independent and modality-specific aspects of grammaticalization in sign languages. Linguistics in Potsdam, 24, 5–98. Potsdam: University-Verlag.
  15. Meier, R. P., Lillo-Martin, D. (2010). Does spatial make it special? On the grammar of pointing signs in American Sign Language. In Gerdts, D. B., Moore, J. C., Polinsky, M. (Eds.), Hypothesis A/hypothesis B: linguistic explorations in honor of David M. Perlmutter (pp. 345–360). Cambridge, MA: MIT Press.
  16. McBurney, S. L. (2002). Pointing in Japanese Sign Language and spoken Japanese. Sign Language Studies, 3(4), 398–412.
  17. Quinto-Pozos, D., Parrill, F. (2015). Signers and co-speech gesturers adopt similar strategies for portraying viewpoint in narratives. Topics in Cognitive Science, 7, 12–35. doi: 10.1111/tops.12120.
  18. Filimonova, E. V. (2023). Ret¬siprok v russkom zhestovom yazyke i russkoy zhestikulyatsii: sopostavitel’nyy analiz. In Slovo i zhest (pp. 58–62): proceedings of a scientific conference, dedicated to the memory of E. A. Grishina (“Grishinskiye chteniya”. (In Russ.)
  19. Frederiksen, A. T. (2021). Emerging ASL distinctions in sign-speech bilinguals’ signs and co-speech gestures in placement descriptions. Frontiers in Psychology, 12, 686485. doi: 10.3389/fpsyg.2021.686485.
  20. Casey, S., Emmorey, K., Larrabee, H. (2012). The effects of learning American Sign Language on co-speech gesture. Bilingualism, 15, 677–686. doi: 10.1017/S1366728911000575.
  21. Baus, C., Carreiras, M., Emmorey, K. (2012). When does iconicity in sign language matter for word learning? Frontiers in Psychology, 3, 232. doi: 10.3389/fpsyg.2012.00232.
  22. Campbell, R., Martin, P., White, T. (1992). The processing of iconic meaning in sign language and its application to word learning. Applied Psycholinguistics, 13, 473–489.
  23. Lieberth, A. K., Gamble, M. E. (1991). Iconicity and meaningfulness in American Sign Language. American Journal of Psychology, 104, 381–398.
  24. Van Hoey, L., Cormier, K., Lepic, R. (2023). Iconicity in sign language learning: Associations between form and meaning. Language Learning, 73, 319–346. doi: 10.1111/lang.12636.
  25. Ortega, G., Morgan, G. (2015). Iconicity and gesture jointly facilitate learning of second language signs at first exposure in hearing nonsigners. Language Learning, 65, 647–671. doi: 10.1007/s1366728911000575.
  26. Janke, V., Marshall, C. R. (2017). Using the hands to represent objects in space: Gesture as a substrate for signed language acquisition. Frontiers in Psychology, 8, 2007. doi: 10.3389/fpsyg.2017.02007.

Supplementary files

Supplementary Files
Action
1. JATS XML
Download


Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Согласие на обработку персональных данных

 

Используя сайт https://journals.rcsi.science, я (далее – «Пользователь» или «Субъект персональных данных») даю согласие на обработку персональных данных на этом сайте (текст Согласия) и на обработку персональных данных с помощью сервиса «Яндекс.Метрика» (текст Согласия).