CC BY
23International Annual Edition of Applied Psychology : Theory, Research, and Practice Volume 3, Issue 1, 2016.
OCULOMOTOR ACTIVITY OF THE RUSSIAN LEARNERS OF ENGLISH DURING THE PERCEPTION OF VERBAL STIMULI IN RUSSIAN AND
ENGLISH LANGUAGES
Natalia I. Igolkina
Saratov State University, Saratov, Russia nataigolkina@mail.ru
Tatyana V.Belykh
Saratov State University, Saratov, Russia tvbelih@mail.ru
Abstract
Utilising modern technologies in second language acquisition (SLA) studies widens the horizon for the researchers in this field. EEG, PET, fMRI and eye-tracking give an opportunity to investigate the processes involved in SLA which are concealed from the researchers. In the last decade eye-trackers have become more available, which has given a rise to the number of studies conducted with the use of this equipment. We used eye movement measures of first-language and second-language text reading to investigate peculiarity of oculomotor activity which the Russian learners of English demonstrate during the perception of verbal stimuli in Russian and English languages. The data show that the mechanisms of oculomotor activity during the perception of texts in English and Russian are different. It is found that fixation count and fixation frequency have maximum values for the text in Russian and these values are closer to the ones registered during the image perception. Saccade count and saccade frequency demonstrate maximum values during the perception of the text in English. Moreover, the study shows that some learners' peculiarities of neuro- and psychodynamics correlate with the certain characteristics of the oculomotor activity registered during the perception of the stimuli in different languages.
Key words: oculomotor activity, second language acquisition, verbal stimuli in Russian and English, eye-tracking, neuro- and psychodynamics characteristics.
Modern technologies in second language acquisition research
Availability of highly sophisticated tools such as neuroimaging devices and eye-trackers for the scientists involved in human studies led to a marked increase in neurolinguistic enquiry. These studies allow the researchers to examine the 'black box' constructs involved in second language acquisition which are hidden from the researchers observing language learner behaviour. Such investigations can contribute to our understanding of the processes the learners undergo while mastering a foreign/second language.
Electroencephalographies (EEG), measurements of event-related potentials (ERPs), positron emission tomography (PET) are the techniques which provide a broader and deeper view of the acquisition processes [1-3]. One of the most advanced technologies which are applied nowadays by some research groups is neuroimaging. Functional magnetic resonance imaging (fMRI) identifies which parts of the brain are activated when learners are asked to perform a language task and plots the activity of the different parts of the brain involved in language use and learning. It allows the researchers to observe the changes in neural activity when participants are asked to perform a task involving receptive or productive language use.
Neural functioning during tasks performance in the target language has become one of the promising research areas within the theory of second language acquisition which will be able to assist in elucidating the methodology employed [4]. Neuroimaging studies of bilinguals have emphasised the potential role of such factors as age of acquisition, level of proficiency, degree of exposure, dominance on functional brain mapping of multiple languages. Abutalebi J., S. Cappa and D. Peranai conducted a review of neuroimaging studies and concluded that 'attained proficiency, and maybe language exposure, are more important than the age of acquisition as a determinant of the cerebral representation of languages in bilinguals/polyglots' [5, p. 179].
Eye-tracking as an investigation method
The eye-movement method, also known as eye-tracking, provides researchers with a highly accurate picture of where people look and for how long when they read a sentence, speak a word, or view a visual scene. It also involves software algorithms for pupil detection, image processing, data filtering and recording eye movements by means of fixation point, fixation duration and saccade and many other parameters. In usability studies and market research eye tracking provides unique methods to evaluate how users and consumers experience and perceive different media and communication messages. Eye tracking is used to answer an endless array of research questions in fields ranging from psychology, infant and reading research to neuroscience and vision research.
Eye-trackers are successfully used as a means to investigate how learners' cognitive abilities and their learning styles effect the whole learning process and the outcome irrespectively the subject to be learnt [6-8]. Wide introduction of computers and multimedia technologies into educational systems at different levels forced researchers to conduct series of studies in this field aiming not only to shed light on the peculiarities of the mechanisms involved in information processing mediated by the computer [9, 10].
Eye tracking is used by linguistic researchers to investigate human language development, language skills, and reading behaviour. In language processing, eye movements are closely linked to the current focus of attention. Linguistic abilities are assessed by tracking and recording eye movements in response to predetermined verbal and visual stimuli [11].
Eye-tracking in second language acquisition research
Eye tracking has been used successfully as a technique for measuring cognitive load in reading, psycholinguistics, writing, language acquisition and other research
areas for some time now. In the last decade a number of studies have been dealing with issues in SLA theory and peculiarities of cognitive processes of bilinguals. Combination of traditional SLA research methods with those that capture the moment-by-moment interpretation of the target language, such as eye-tracking, allows the researchers to investigate new fields and get evidences to support provoking hypothesis and assumptions. The major benefit of the eye-tracking technology is that it gives the opportunity to tap into real-time or online comprehension processes during the uninterrupted processing of the input. The obtained data can be compared to those elicited by other, more metalinguistic tasks to offer a broader picture of language acquisition and processing. L. Roberts and A. Siyanova-Chanturia conducted a survey of topics within SLA studies utilizing eye-tracking technology and showed 'how eye-tracking data can be used to (a) investigate language-related topics and (b) inform key debates in the fields of L2 acquisition and L2 processing'[12, p.2013].
The results which the researchers can gain using eye-tracking method shed lights not only on theoretical and fundamental issues but contributes a lot to understanding of the learning processes and consequently to the development of teaching methods. For instance, the study carried out by H. Kang demonstrated that metacognitive reading skills are critical to fluent reading in online environment [13]. H.Kang got the results which indicate that "... L1 and L2 readers are heterogeneous when language proficiency is considered but they are homogeneous in many other aspects when the focus is on their online reading patterns and comprehension. While L1 readers read much faster than L2 readers, their attention distribution and performance on reading comprehension test are similar to L2 readers". This study concludes that the essential online reading competency factors are similar in L1 and L2's online reading when the "rapid", "purposeful" and "comprehending" attributes of fluent reading are under investigation. When the core vocabulary of a language is mastered, it is metacognitive reading skills that play a major role in fluent reading in the context of new literacy.
Eye-trackers are widely used by the researchers posing the questions on differences in mental behaviour of bilinguals and investigating such research topic as context specific nature of bilingual cognitive advantages. For instance, eye movement recordings as well as lexical decision showed that high-frequency words are recognised more quickly than low-frequency words. This finding is known as word frequency effect. This effect plays a crucial role in determining the structure of the mental lexicon and ways lexical forms are accessed [14,15]. Another eye tracking research showed that 'the relative amount of bilinguals' L1 and L2 experience is as key determinant of lexical activation and, consequently, frequency effect during bilingual reading. Thus, L2 knowledge and usage affect how bilinguals process L2 words and... L1 words' [16].
Oculomotor activity of Russian learners during the perception of texts in
different languages
It is known that SLA makes adult learners mobilise their cognitive and personal resources. While studying foreign languages learners acquire new cognitive skills or use the old ones in another context. Different factors influence the process of second language acquisition and one of these is the combination of native and target languages. Moreover, individual differences influence performance on tasks that call for dynamic oculomotor control [17]. That is why it is supposed that studying oculomotor activities during the perception of texts in English and Russian can give information on peculiarities of cognitive activity during the acquisition of the English language by Russian learners. These peculiarities can interrelate with certain individual learner differences. The assumption is that oculomotor activity of non-proficient language learners during the perception of texts in English and Russian is different and interrelates with neuro- and psychodynamics peculiarities of the language learners.
Participants and experimental set-up
Twenty three Russian learners of English with different levels of achievement in English (except proficient level) participated in the experiment. The participants were undergraduates and post graduate students specialising in natural sciences. Participants had normal or corrected-to-normal vision and no self-reported history of speech, learning, or hearing disorders. Stimuli consisted of one text in Russian and one text in English. The Russian text consisted of 115 words (708 symbols with spaces) and the English text consisted of 136 words (794 symbols with spaces). The contents of the texts were close to each other. The experiment was conducted on the RED 500 System produced by SensoMotorik Instruments GmbH (Germany). The minimal fixation duration was set equal to 50 ms. The texts were presented in black on a grey background. The participants could read each text for one minute and on completing it they were supposed to recall in details its content. The participants were also presented a nonvisual stimulus (a photo) and were supposed to describe the photo after the presentation. All the participants were later tested with Rusalov's and Strelau's questionnaires to define their psychodynamics and temperamental characteristics. Statistical processing of experimental data in order to receive Pearson product-moment correlation coefficient (r) and p-values was performed with the help of Statistical Package for the Social Sciences (SPSS), version 22.
Experimental results
Statistically significant differences were registered during the perception of the English and Russian texts (Table 1). Values of fixation count and fixation frequency during the perception of the Russian text is significantly higher comparatively to the ones of the English text (/?<0,001). These values for the English text are closer to the values for the nonverbal stimulus (for fixation count r=0,334, p=0,12 and for fixation
frequency r=0,473, p=0,022). Statistical significance of fixation duration maximum during the perception of the English text is considerably higher comparatively to this value registered during the perception of the image (r=0,679, /?<0,001). Statistical significance of the saccade count and saccade frequency during the perception of the English text is higher comparatively to the ones of the Russian text (p=0,017) and perception of the image (/?<0,001).
Table 1.
Mean values of oculomotor activity during the perception of tree stimuli (1 stimulus - English text, 2 stimulus - Russian text, 3 stimulus - photo).
Stimuli Blink Count (Fixation Count) (Fixation Frequency [count/s]) Fixation Duration Maximum [ms] Scanpath Length [px] Saccade Count Saccade Frequency [count/s] Saccade Amplitude Average [°]
1 12,5 142,1 2,4 820 15402 399 6,7 3,4
2 12,6 204 3,4 1015,8 24536,8 298,3 5 3,3
3 15,1 148 2,5 1233,4 13867,5 267 4,6 3,5
It was found that peculiarities of neuro- and psychodynamics of the Russian learners of English correlate with some characteristics of oculomotor activity registered during the perception of the stimuli in different languages (Table 2 and Table 3). Moreover for different texts the correlations differ depending on the text being perceived. There are statistically reliable evidences that during the perception of the English text values of fixation count correlate with excitement level (r=0,819, p<0,01), extinction level r=0,559, p<0,05) and strength equability (r=0,706 p<0,01). Fixation frequency correlate with excitement level (r=0,824, p<0,01), extinction level
(r=0,564, p<0,05) and strength equability (r=0,716, p<0,01). Scanpath length correlate with excitement level (r=0,697, p<0,01) and strength equability (r=0,743, p<0,01). Social activity has an only correlation with saccade count (r=0,654, p<0,05). All the correlations mentioned above are positive ones. Negative correlations are observed between excitement level and such values of oculomotor activity as saccade count and (r=-0,557, p<0,05) and saccade frequency (r=-0,558, p<0,05) (see Table 2).
Table 2.
Correlation of neuro- and psychodynamics with OA values during the perception of the English text (*- correlation at p<0,05, ** - correlation at p<0,01).
"\OA Values Personal ^^ Characteristics ^^ Fixation Count Fixation Frequency Scanpath Length Saccade Count Saccade Frequency
Social Activity 0,654
Excitement Level ** 0,819 ** 0,824 ** 0,697 * -0,557 * -0,558
Extinction Level * 0,559 * 0,564
Strength Equability ** 0,706 ** 0,716 ** 0,743
Analysis of OA values during the perception of the Russian text revealed a number of statistically significant correlations with learners' personal characteristics, both positive and negative. There is a positive correlation between social activity and minimal fixation duration (r=0,586, p<0,05) as well as saccade count (r=0,654, p<0,05). Velocity as a personal characteristics also correlates with minimal fixation duration (r=0,546, p<0,05). Saccade amplitude average has an only correlation with emotiveness (r=0,529, p<0,05). Negative correlations were marked between blink count and flexibility (r=-0,506, p<0,05), extinction level (r=-0,504, p<0,05) and
lability (r=-0,504, p<0,05). Extinction level negatively correlates with minimal fixation duration (r= -0,499, p<0,05). Saccade velocity average has correlation with flexibility (r=-0,613, p<0,01).
Table 3.
Correlation of nuero- and psychodynamics with OA values during the perception of the Russian text. (*- correlation at p<0,05, ** - correlation at p<0,01).
\ OA Values PersonaK Charac- \ teristics \ Blink Count Fixation Count Fixation Frequency Fixation Duration (min) Saccade Count Saccade Amplitude Average Saccade Velocity Average
Social Activity 0,586* 0,654*
Flexibility -0,506* -0,613**
Velocity 0,546*
Emotiveness 0,529*
Extinction Level -0,504* -0,499*
Lability -0,514*
Strength Equability 0,706** 0,716**
Discussion of the experiment results
Comparison of nuero- and psychodynamics values which have correlations with OA values during the perception of Russian and English texts shows that the only common correlation for both verbal stimuli is the one between saccade count and social activity (r=0,654, p<0,05 in both cases). It is also worth emphasising that the largest number of OA values which have correlations with a certain personal characteristic has been registered during the perception of the English text. These characteristic are excitement level and strength equability (five items and three items
correspondingly) (see Table 2). These evidences allows us to suppose that, firstly, the mechanisms of processing texts in the target language (in our case English) differ from the mechanisms of processing texts in the native language. Secondly, peculiarities of OA can be predetermined by the learners' personal characteristics. The fact that some OA values during the perception of the English text are closer to the ones registered for image perception makes us presume that non proficient leaners while reading the text in the target language use the mechanisms of image perception rather than verbal information perception. Possibly, it happens when learners come across unknown lexical and grammar items. It may mean that mastering effective reading strategies will contribute to successful second language acquisition.
References
[1] Reiterer, S., Pereda, E., & Bhattacharya, J. (2009). Measuring Second Language Proficiency with EEG Synchronization: How Functional Cortical Networks and Hemispheric Involvement Differ as a Function of Proficiency Level in Second Language Speakers. Second Language Research, 25, 77-106.
[2] Steinhauer, K. (2014). Event-related Potentials (ERPs) in Second Language Research: A Brief Introduction to the Technique, a Selected Review, and an Invitation to Reconsider Critical Periods in L2. Applied Linguistics, 35, 393-417.
[3] Indefrey, P., & Davidson, D. J. (2009). Second Language Acquisition. In L. R. Squire (Ed.), Encyclopedia of Neuroscience. London: Academic Press, 517-523.
[4] Ellis, R. (2008). The Study of Second Language Acquisition. 2nd Edition. Oxford: Oxford University Press. 1142 p.
[5] Abutalebi, J., Cappa, S. & Peranai, D. (2001). The Bilingual Brain as Revealed by Functional Neuroimaging. Bilingualism: Language and cognition, 4, 179-190.
[6] Toker, D., Conati, C., Steichen, B., & Carenini, G. (2013) Individual User Characteristics and Information Visualization: Connecting the Dots through Eye Tracking. In Proceedings of CHI 2013 ACM SIGCHI Conference on Human Factors in Computing Systems, 295-304.
[7] Nisiforou, E.A. & Laghos, A. (2013). Do the Eyes Have it? Using Eye Tracking to Assess Students Cognitive Dimensions. Educational Media International, 1-19.
[8] Nisiforou, E.A. (2013). Using Eye Tracking and Electroencephalography to Assess and Evaluate Students' Cognitive Dimensions. Proceedings of the Doctoral Consortium at the European Conference on Technology Enhanced Learning 2013 co-located with the EC-TEL 2013 conference (EC-TEL 2013), 79-86.
[9] Tsianos, N., Germanakos, P., Lekkas, Z., Mourlas, C., & Samaras, G. (2009) Eye-tracking Users' Behavior in Relation to Cognitive Style within an E-Learning Environment. In Proceedings of the 9th IEEE International Conference on Advanced Learning Technologies (ICALT), Riga, Latvia: IEEE Computer Society Press, 329-333.
[10] Van Gog, T. & Scheiter, K. (2010). Eye Tracking as a Tool to Study and Enhance Multimedia Learning. Learning and Instruction, 20. 95-99.
[11] Lupu, R. G. & Ungureanu, F. (2013). A Survey of Eye Tracking Methods and Applications. Buletinul Institutului Politehnic din Ia§i. Gheorghe Asachi Technical University of Iasi, 57-65.
[12] Roberts, L. & Siyanova-Chanturia, A. (2013). Using Eye-Tracking to Investigate Topics in l2 Acquisition and l2 Processing. Studies in Second Language Acquisition, 35, 213-235.
[13] Kang, H., 2014. Understanding Online Reading through the Eyes of First and Second Language Readers: An Exploratory Study. Computers & Education 73, 1-8.
[14] Rayner, K., & Duffy S. A. (1986). Lexical Complexity and Fixation Times in Reading: Effects of Word Frequency, Verb Complexity, and Lexical Ambiguity. Memory & Cognition,12. 590-606.
[15] Rayner, K. (1998). Eye Movements in Reading and Information Processing; 20 Years of Research. PsychologicalBulletetin, 124, 372-422.
[16] Whitford, V. & Titone D. (2012). Second-Language Experience Modulates First- and Second-Language Word Frequency Effects: Evidence From Eye Movement Measures of Natural Paragraph Reading. Psychonomic Bulletin & Review, 19, 73-80.
[17] Singh, N. & Mishra, R.K. (2015). The Modulatory Role of Second Language Proficiency on Performance Monitoring: Evidence from a Saccadic Countermanding Task in High and Low Proficient Bilinguals. Frontier Psychology. 5. 14-81.
