CC BY
75UDC 81-139
VERBAL ASSOCIATIONS TO VISUAL STIMULI: QUANTITATIVE ANALYSIS
A.A. Miklashevski
Philological faculty of the National Research Tomsk State University (Tomsk, Russian Federation).
E-mail: [email protected]
Abstract. The paper is devoted to a quantitative analysis of the results of an associative experiment, which was held previously. Some of the Paivio's conclusions about dual coding of information (verbal and visual) are applied into the research of subjective visual images, information conversion from a visual to a verbal mode. Within the research visual and verbal information processing is related to the response on concrete and abstract nouns. The results of a quantitative analysis and intermediate conclusions are presented. Dynamics of verbal and visual coding during an associative experiment is being studied.
Keywords: verbal processing; visual processing; dual coding theory; associative experiment.
Introduction
The problem of the relation between verbal and visual components in human's cognition attracts attention of researchers in linguistics, psycholin-guistics, cognitive linguistics, cognitive psychology, art studies and other areas. However, researchers haven't reached an agreement on the issue of this relation: Are these components equal? Or, maybe, one of them is leading? In addition, no generally accepted model of the relation between verbal and visual components has been created either.
The absence of a universal model leads to a large diversity of methods and approaches which are used to investigate this problem. Relatively speaking, we can distinguish two approaches among the works reviewed: analytical and holistic. Some researchers focus on modularity of visual and verbal processing, their greater or less autonomy. At the same time the others, on the contrary, seek to demonstrate their indivisible unity. The choice of the method is partly justified by this difference.
Holistic approach
For example, linguistics and psycholinguistics use questionnaire method and quantitative analysis, which are traditional for these disciplines. The linguo-psychological dictionary "The senses, emotions and adjectives of the Russian language" [1] was compiled using these methods. Images in this dictionary (in broad terms, i.e. smell images or tactile images and etc.) are
researched through the prism of a language; dependence of an image semantic component of a word on general psychophysiological genesis of the subject is demonstrated.
A similar view is held by T.A. Sternin and M.Ya. Rosenfeld, the authors of the monograph "The Word and the Image" [2]. Using similar methods the authors consider image component as an integral part of a semantic structure of a word and classify reactions in their associative experiment on this theoretical basis.
Associative experiment was also used in Bebchuck's study [3], and images obtained during the experiment can be interpreted culturologically and cognitively as components of concepts [4].
Analytical approach
There is another tradition in foreign cognitive studies. In Paivio's works [5, 6] DCT, Dual Coding Theory was formulated, stating that a word and an image act as units of two equal systems of information coding. This approach focuses on preservation and retrieving of information, which was coded in these two different systems. Naturally, this model is in high demand in the area of second language acquisition and education in general [7]. It is worth noting that this viewpoint is not unique: there is an opinion about much greater role of a language and a subordinate role of an image during information coding (so-called conceptual-propositional hypothesis by J.R. Anderson and G.N. Bower [8]).
Previous research
In our previously published research [9] we have tried to develop a method to analyze the transition from a visual code to a verbal one, i.e. description strategy. We used the data of the chain associative experiment as a material with pictures as stimuli and verbal reactions. There were 67 participants, 40 - 12-14-year-old and 27 - 17-20-year-old. There were 21 men among them. 13 pictures were proposed to the participants for evaluation; they had to give 3 verbal associations to each picture. 67 questionnaires with 871 associative arrays were received. 3 questionnaires were excluded: in two questionnaires more than 25% of reactions were missing and the third contained a lot of abstract words, poorly (from our viewpoint) associated with the content of pictures. In addition some associative arrays were excluded due to illegibility of participant's handwriting. In general 74 associative arrays were excluded, i.e. 8,5% of the total number (871).
In the same study we proposed a method for analysis of the association content: the following associative strategies were distinguished:
1. Strategies based on language mechanisms (syntagmatics (phrases, quotes etc.) and paradigmatics (hyponyms and hyperonyms, synonyms and antonyms within one array)) - 46%; 8% of which was assigned to this group only.
2. Strategies based on image properties (nomination of shape, color, size of the object, visual perception indications) - 26% of all associative arrays; 7% of which was assigned to this group only.
3. Strategies based on properties of a typical situation (association, referring to familiar frames; propositions, describing typical situations; associative arrays, indicating subject's physical involvement into interrelation with the object from the picture) - 74%; 33% of which was assigned to this group only.
It can be concluded, that all groups listed here overlap, and only a small number of associative arrays belongs to one group only. It seems obvious that such an approach isn't free from researcher's subjectivity and we encountered some contradictions during the analysis: there are different types in different combinations within one array, and analysis of these combinations doesn't allow making unambiguous conclusions. To overcome the difficulty we performed a quantitative analysis of the results of the same experiment, using more strict regularities.
Quantitative analysis of reactions
The data of the associative experiment, which was performed in 20132014 among pupils of a school and students of a university, were used at this stage of the research.
Purpose and objectives of research. The purpose of the research is to elaborate an objective method for the analysis of description strategies, ie conversion of a visual code into a verbal one. To achieve the goal we have solved the problem of a qualitative analysis of the results of an associative experiment, three groups of associations were described. In this research we make a quantitative analysis, consider time dynamics of verbal and visual processing.
Hypothesis. Initial hypothesis claimed that each associative array was coded in one of two systems (verbal or visual), but qualitative analysis rejected this hypothesis. As a result another hypothesis was proposed: during an associative process (under given conditions) verbal and visual processing work in parallel; associative array is a result of their joint work.
Materials and subjects. The results of the experiment described above (see Previous research) were taken as a material for the given study.
Research methods. Quantitative analysis method was used in this research.
In Paivio's works the following relation between verbal / visual processing and abstract / concrete words was demonstrated.
T a b l e 1
Coding systems for different types of stimuli [10, 11]
Type of the stimulus Coding system* visual verbal
Picture + + + + +
Concrete word + + + +
Abstract word - - + + +
* The number of pluses shows relative participation.
Using this table 1 we have made a conclusion that the presence of abstract words can be indirect evidence of verbal processing, and the presence of concrete words indicates more visual processing.
The proportion of nouns and non-nouns
The next step was to determine part of speech characteristics of the responses, discriminating abstract and concrete nouns. Their proportions are presented in table 2.
T a b l e 2
Association 1 Association 2 Association 3
Concrete nouns 399 (50%) 313 (39%) 293 (37%)
Abstract nouns 263 (33%) 327 (41%) 368 (46%)
Nouns in total 83% 80% 83%
Adjectives 29 (4%) 45 (6%) 50 (6%)
Adverbs 14 (2%) 36 (5%) 19 (2%)
Verbs 1 (>1%) 10 (1%) 7 (1%)
Interjections 0 0 1 (>1%)
Numerals 0 0 1 (>1%)
Pronouns 0 1 (>1%) 0
Phrases 91 (11%) 65 (8%) 58 (7%)
Non-nouns in total 17% 20% 17%
At this stage it can be seen that, first, nouns dominate in all reactions, and the number of nouns in total doesn't change from one reaction to the other one. Second, it can be noted, that from the first reaction to the third reaction the number of concrete nouns decreases (from 50% to 37%), and the number of abstract nouns grows (from 33% to 46%). This trend can be explained by characteristics of a thinking process, moving from the perception of a received picture-stimulus to language abstraction. But, of course, the trend needs a more detailed review.
The proportion of concrete and abstract nouns
At the next stage all non-nouns were excluded from the analysis. Thus, only such arrays as "noun - noun - noun", 482 associative arrays or 60% of all reactions were left.
To find out what factors influence the change of proportion of concrete and abstract nouns from reaction to reaction the presence / the absence of distinction between types of neighboring reactions in each associative array was investigated. Besides, a quantitative analysis of these distinctions was performed. These distinctions are the following (the question mark shows a reaction not-considered in an array; as in general there are two types of these distinctions in each array out of three reactions, table 3 presents a percentage of the total number of all distinctions 964 (482 * 2); the number of cases is depicted in brackets):
T a b l e 3
Association 1 Association 2 Association 3 The number Presence / absence of the change Type of the change
concrete > abstract ? 12% (115) The change happens (38%) concrete > abstract (23%)
? concrete > abstract 11% (104)
abstract > concrete ? 7% (72) abstract > concrete (15%)
? abstract > concrete 8% (77)
concrete > concrete ? 17% (162) The change doesn't happen (62%) concrete, doesn't change (30%)
? concrete > concrete 13% (130)
abstract > abstract ? 14% (133) abstract, doesn't change (32%)
? abstract > abstract 18% (171)
As the data in table 3 show the change of proportion of concrete / abstract nouns occurs due to more transfers (by 8%) from concrete to abstract nouns, but this transition is evenly distributed in time, there is no correlation between the reaction ordinal number and the speed of the increased number of abstract nouns.
Conclusion
Based on the above it can be concluded, that:
1. Nouns constitute the largest share of all reactions under given conditions, that's why they can be used as an indicator for the analysis.
2. The number of abstract nouns increases evenly under given conditions, and the number of concrete nouns decreases evenly, which can be indirect evidence for language-based "recoding" of images-stimuli.
3. Such "recoding" happens gradually, and is distributed evenly in time; in other words, there is no rigid boundary between verbal and visual coding systems under given conditions, their interrelation takes place at all stages of association process.
It seems likely that the data on the dynamics of concrete / abstract nouns will correlate with such factors as sex, age, education of respondents and etc. These correlations can be a subject for further study.
References
1. KOLBENEVA, M.G., and ALEXANDROV, Yu.I., 2010. The senses, emotions and adjec-
tives of the Russian language: linguo-psychological dictionary. Moscow: Languages of Slavonic cultures.
2. STERNIN, I.A., and ROSENFELD, M.Ya., 2008. The Word and the Image. Voronezh:
Istoki.
3. BEBCHUCK, E.M., 1991. The image component in a lexical structure of a Russian noun.
Author's abstract on competition of a scientific degree of candidate of philological Sciences. Voronezh.
4. POPOVA, Z.D., and STERNIN, I.A., 2001. Essays on cognitive linguistics. Voronezh,
рр. 58-64.
5. PAIVIO, A., 1971. Imagery and verbal processes. New York: Holt, Rinehartand Winston.
6. PAIVIO, A., 1986. Mental representations: a dual coding approach. Oxford. England:
Oxford University Press.
7. CLARK, J.M., and PAIVIO, A., 1991. Dual Coding Theory and Education. Educational
Psychology Review, 3 (3).
8. ANDERSON, J.R., and BOWER, G.H., 1973. Human associative memory. Washington,
DC: Winston & Sons.
9. MIKLASHEVSKI, A.A., 2014. Verbal and visual processing of information: experimental
research. Tomsk State University Journal, 383, рр. 56-61.
10. PAIVIO, A., and BEGG, I., 1971. Imagery and comprehension latencies as a function of sentence concreteness and structure. Perception and Psychophysics, 10. рр. 408-412.
11. SOLSO, R.L., 2011. Cognitive Psychology. Saint-Petersburg: Piter, 333 р.
ВЕРБАЛЬНЫЕ АССОЦИАЦИИ К ВИЗУАЛЬНЫМ СТИМУЛАМ: КОЛИЧЕСТВЕННЫЙ АНАЛИЗ
Миклашевский А.А. Национальный исследовательский Томский государственный университет (Томск, Россия). E-mail: [email protected]
Аннотация. Статья посвящена описанию результатов количественного анализа результатов ассоциативного эксперимента, проведенного ранее. Исследование субъективных визуальных образов, перевода информации из образного кода в вербальный опирается на теорию двойного кодирования A. Paivio. В настоящей работе образная и вербальная обработка связываются с реагированием абстрактными vs конкретными существительными. Представлены промежуточные выводы, рассматривается динамика реагирования абстрактными vs конкретными существительными в ходе цепочечного ассоциативного эксперимента.
Ключевые слова: вербальное кодирование; образное кодирование; теория двойного кодирования; ассоциативный эксперимент.
