The Impact of Video and Subtitle Speed on Subtitle Reading

An Eye-tracking Replication Study

Authors

DOI:

https://doi.org/10.47476/jat.v7i1.2024.283

Keywords:

subtitling, subtitle speed, reading speed, replication, concurrent video, foreign language, L2 processing, subtitling reading

Abstract

We present results of a direct replication of Liao et al.’s (2021) study on how subtitle speed and the presence of concurrent video impact subtitle reading among British and Polish viewers. Our goal was to assess the generalisability of the original study’s findings on a cohort of Australian English. The study explored both subtitle-level and word-level effects, considering the presence or absence of concurrent video and three subtitle speeds: 12 characters per second, 20 cps, and 28 cps. Overall, most of the original results were replicated, confirming that the presence of video and the speed of the subtitles have a measurable impact on processing across different viewer groups. Additionally, differences in how native and non-native speakers process subtitles emerged, in particular related to wrap-up, word frequency and word length effects. The paper describes the replication in detail, presents the findings, and discusses some of their implications.

Lay summary

In our study we were interested in the effects that the presence of video and various subtitle speeds have on how viewers watch subtitled videos and how they understand them. We also wanted to know if the previous results obtained in a study by Liao et al. (2021) in Australia hold true for other viewers living in different locations. With this goal in mind, we repeated Liao et al.’s (2021) study on British and Polish viewers. The study explored both subtitle-level and word-level effects, considering the presence or absence of video and three subtitle speeds: 12 characters per second, 20 cps, and 28 cps. Overall, most of the original results were confirmed, showing that the presence of video and the speed of the subtitles have an impact on processing across different viewer groups. Additionally, differences in how native and non-native speakers process subtitles emerged, in particular related to well-known linguistic effects from reading studies, such as wrap-up, word frequency and word length effects. The paper describes the replication in detail, presents the findings, and discusses some of their implications.

Downloads

Download data is not yet available.

Author Biographies

Agnieszka Szarkowska, Faculty of Applied Linguistics, University of Warsaw

Agnieszka Szarkowska is University Professor in the Institute of Applied Linguistics at the University of Warsaw, Head of the research group Audiovisual Translation Lab (AVT Lab), and Honorary Research Associate at University College London. Agnieszka is a researcher, academic teacher, ex-translator, and translator trainer. Her research projects include eye-tracking studies on subtitling, audio description, multilingualism in subtitling for the deaf and the hard of hearing, and respeaking. Drawing on her passion for teaching, she has co-founded AVT Masterclass, an online platform for professional audiovisual translation education. Agnieszka is a member of the European Association for Studies in Screen Translation (ESIST) and a recipient of the Jan Ivarsson Award 2022.

Valentina Ragni, Univesity of Warsaw

Valentina Ragni is Assistant Professor in the WATCH-ME project at the Institute of Applied Linguistics of the University of Warsaw. In her research projects, she has investigated the effects of watching reversely subtitled videos on L2 mnemonic retention in advanced L2 Italian learners through an eye-tracking as well as assessing the impact of productivity-enhancing technologies – such as machine translation and behaviour-tracking tools – on professional translators. She is particularly interested in the cognitive and psycholinguistic aspects of translation, both as a learning tool and as a professional practice.

David Orrego-Carmona, School of Modern Languages and Cultures, University of Warwick, and Department of Linguistics and Language Practice, University of the Free State

David Orrego-Carmona is an Associate Professor at the University of Warwick and a Research Associate at the University of the Free State (South Africa). David’s research deals primarily with translation, technologies and users. Using qualitative and quantitative research methods, his work explores the societal affordances and implications of translation and technologies. He is treasurer of ESIST, the European Association for Studies in Screen Translation, associate editor of the journal Translation Spaces and deputy editor of JoSTrans, the Journal of Specialised Translation.

Sharon Black, School of Politics, Philosophy, Language and Communication Studies, University of East Anglia

Sharon Black is Lecturer in Interpreting and Translation at the University of East Anglia (UK). Her principal research interests are in audiovisual translation and media accessibility. Sharon is working on research projects investigating how adults and children watch subtitled video using eye tracking, and has collaborated on European projects on digital media and arts accessibility tools, services and training. Sharon is President of the European Association for Studies in Screen Translation (ESIST).

Sonia Szkriba, Faculty of Applied Linguistics, University of Warsaw

Sonia Szkriba is a doctoral candidate at the Doctoral School of Humanities, University of Warsaw. In her PhD, she studies the processing of interlingual and intralingual subtitles. She is also a subtitling practitioner specialising in theatre surtitling.

Jan-Louis Kruger, Department of Linguistics, Macquarie University, and UPSET focus area, North-West University

Jan-Louis Kruger is professor of Linguistics at Macquarie University in Sydney, Australia, and extraordinary professor in the UPSET focus area at North-West University in South Africa. His research focuses on the processing of language in multimodal contexts, specifically in audiovisual translation, reading, and interpreting.

Krzysztof Krejtz, Faculty of Psychology, SWPS University

Krzysztof Krejtz is Associate Professor at the Institute of Psychology and Head of EyeTracking Research Centre at SWPS University in Warsaw. His research interests focus around eye tracking, human-computer interaction, visual attention and multimedia learning. He is a member of Association for Computer Machinery (ACM) and Steering Committee of Eye Tracking Research and Applications (ACM ETRA).

Breno Silva, Faculty of Modern Languages, University of Warsaw

Breno Silva is an Assistant Professor at the Institute of English Studies, University of Warsaw. His research interests include cross-linguistic influences, incidental and intentional vocabulary acquisition, multilingualism, and statistical modelling. He has also been an English as a Foreign Language teacher for over 20 years.

Downloads

Additional Files

Published

2024-07-05

How to Cite

Szarkowska, A., Ragni, V., Orrego-Carmona, D., Black, S., Szkriba, S., Kruger, J.-L., … Silva, B. (2024). The Impact of Video and Subtitle Speed on Subtitle Reading: An Eye-tracking Replication Study. Journal of Audiovisual Translation, 7(1), 1–23. https://doi.org/10.47476/jat.v7i1.2024.283

Issue

Vol. 7 No. 1 (2024)

Section

Research articles