CC BY
219
ТЕОРИЯ И МЕТОДИКА ПРЕПОДАВАНИЯ
y^K 372.881.111.1
DOI: 10.17223/19996195/42/8
COGNITIVE SKILLS IN EDUCATION: TYPOLOGY AND DEVELOPMENT
R. Millrood, I. Maksimova
Abstract. This publication addresses the issue of cognitive skills as a challenge for the teachers and an asset for the learners. Despite the unanimity in accepting cognitive skills development as a rightful educational agenda, teaching practitioners give preference to the development of lower-order cognitive skills in learners. The reason lies not only the backwash effect of closed-ended testing tasks, but also teachers' beliefs. The hypothesis of the research consists in the following proposition: cognitive skills growth in the classroom depends on explicit training of lower- and higher-order cognitive skills. This research draws on teachers' opinion poll, follow-up interviews and a case study of teaching students of engineering specialties a set of higher-order cognitive skills in their lessons of English. Cognitive skills enable the learner to work out the four types of knowledge such as, factual (facts and events), conceptual (theories and models), procedural (methodology and processes), and metacog-nitive (awareness of ways and practices of critical thinking). These types of knowledge, based on higher-order thinking, enable the learners to make well-informed decisions as a result of productive thinking. Creative procedures of knowledge generation and application enhance learners' cognitive abilities further on. The article considers the barrier raised by the teaching community that gives preference to challenging learners with acquiring the ready-made knowledge rather than with the knowledge-producing tasks. Ready-made knowledge acquisition seems to be more appealing to both teachers and students, because of the fast-gained results, while cognitive skills development bears fruit much later. The truth is that there is no fast track towards higherorder cognitive skills development. Therefore, declarative knowledge prevails. Keywords: cognitive skills; lower-order cognitive skills; higher-order cognitive skills; logical reasoning; critical thinking.
Introduction
Definition
Skill is a critical asset of human capital, increasing productivity of individuals and communities. Hence, there is growing interest in exploring the area of cognitive skills and philosophy of cognition in education. Cognitive skills are knowledge- and competence-related mechanisms. This relationship consists in that learner competence is a function of skills.
There is a whole scope of terms related to skills in cognition: cognitive universals, cognitive strategies, metacognitive skills, study skills, study competence etc. [1-3].
Cognitive universals refer to the mentally invariant models, in which individuals' minds map the reality. Cognitive strategies are learning techniques that help acquire contents through rote learning or deep processing, word-for-word reproduction or material transformation, looking for the meaning or guessing from context etc. Metacognitive skills involve ideas about learning activities and goal awareness, choosing the required steps and working out strategies, monitoring the process and evaluating the results. Study skills include the mastery of tasks performance such as note-taking, essay writing, web-quest, reading for gist or details and others. Taken together, study skills underlie the degree of excellence in learning known as "study competence".
The overview of skills pertinent to learning leads us to the following definition of the learner cognitive skill: Learner cognitive skill is a functional mechanism that enables an individual to process information and produce new knowledge, creating, integrating and utilizing the means of resolving situational challenges and problem.
Problematizing learner cognitive skills
Cognitive skills in education have become a trendy research strand since the second half of the 20th century. In search of pedagogical means to increase the efficiency of instruction, attention of scholars has focused on controlling memory processes, developing intelligence functions, such as, decision making and problem solving, embedding cognitive operations in teaching procedures, dealing with challenged learners in the inclusive environment [4].
Despite the unabated interest in cognitive functions, there is a growing bend towards other personality resources [5-9].
One of the research strands in the area of learner efficacy is shifting emphasis away from cognitive to non-cognitive skills, such as motivation, effort, self-regulated learning, self-efficacy and self-concept (belief in one's potential to achieve and learning abilities), prosocial behavior and learner resilience to stress [10]. The research has shown the importance of non-cognitive attributes to learners' success opening new horizon of positive pedagogy.
Going deeper into the problem of cognitive skills development, it is necessary to point out that the improvement of learners' standardized achievements in tests is often the result of building up subject competences rather than cognitive skills in learners [11]. This means that the focus of attention is in many cases put primarily on the subject knowledge overlooking
the development of fluid intelligence. Teaching contents rather than cognitive skills, seems like taking a shortcut, which is always a faster track. One of the reasons is that it is unknown how the enhancement of fluid intelligence translates into gains in the learners' examination performance.
Instructional practices have always been under the influence of testing procedures [12]. The technicalities of testing have become part and parcel of classroom practices in pursuit of students' higher scoring. The backwash effect of testing can be traced back to the teaching context [13].
The question is whether testing techniques frustrate or facilitate teaching cognitive skills in the classroom. Namely, it is important to find out whether closed-ended formats, such as cloze procedure, matching, true-false or multiple choice, promote any higher-order thinking operations. If "yes", testing procedures can be welcome in the lessons as cognition boosters.
A comparative study of national reading tests in Denmark, Norway and Sweden shows that testing formats can differ in cognitive engagement of test takers. They may involve students in developing multiplicity of stances towards the text by producing a variety of opinions and hypothesizing about the text, or linking ideas across different texts for reading and listening as requested by the testing tasks.
It is worth noting that the ability to solve "cloze" (gap-filling) tasks is not constrained by lack of global comprehension of the whole text, meaning that successful performance in cloze tests is possible by attending to details and and ignoring whole-meaning processing. Emphasis on making local inferences lies lower in the order of thinking than interpreting implied meaning, reflecting and commenting on the content processing the text globally [14].
Reading proves to be essential in boosting learners' cognitive skills. Research experiments with eye tracking during reading prove that successful readers show markedly distinct cognitive operations [15].
Successful readers direct their gaze along the meaningful thread in the text, process the meaning of larger text chunks beyond the clause-sentence level, process the text at a number of levels from details to the global message. The chief finding is that whole text processing points to higher-order thinking during reading in contrast to local inferences from words, clauses or sentences. This means that global meaning processing is a separate cognitive skill standing higher in the hierarchy of cognitive operations.
The controversy lies, therefore, in the three opposite approaches: teaching cognitive skills vs. non-cognitive skills, developing cognitive skills vs. subject knowledge, training learners' cognitive skills vs. testing skills.
Research hypotheses
The hypothesis of the research consisted of the proposition: cognitive skills growth in the classroom depends on explicit training of lower- and
higher-order cognitive skills with the specificity of their development at every level.
Methodology of research
Research methodology included literature analysis aimed at establishing key ideas about cognitive skills development, teachers' opinion poll with a purpose to have a cut of massive pedagogical practices, follow-up interviews with the teachers, action research into teaching cognitive skills to students of engineering specialties in the open-ended writing tasks during their lessons of English.
Literature analysis
According to the published research, developing cognitive skills in learners is an issue of state concern because there is a close relationship between educational achievement and Gross Domestic Product, which means that school policy is a social tool that can spur economic growth. Paradoxically, the shares of basic literates and high performers do not affect in any significant degree the country's economy. In fact, the group of high performers can be larger in poorer countries due to increased motivation to part with poverty. Important here is the fact that schools contribute to economic success by producing a positive effect on the learners' cognitive skills [16].
To continue the link of school-developed cognitive skills to the economic success of graduates, there is research in employers' skill preferences regarding job applicants. According to the data, cognitive skills matter most in a number of European countries including Czech and Irish labor markets [17]. Incidentally, these countries have recently shown remarkable economic growth.
In view of the importance attached to cognitive skills in school learning and future employment, there is continuous interest in Bloom's taxonomy of deeper learning, which involves higher order thinking and the capacity to transfer knowledge to a great variety of tasks and contexts [18].
As Bloom's taxonomy suggests, knowledge development proceeds with the growing complexity of cognitive operations, starting with retention of discrete pieces and over to comprehension through transformation, application by transfer to new circumstances, analysis and distinguishing between facts and opinions, synthesis in a new intellectual product, and evaluation as critical reflection.
Cognitive skills enable the learner to work out four types of knowledge such as, factual (facts and events), conceptual (theories and models), procedural (methodology and processes), and metacognitive (awareness of ways and practices). These types of knowledge, based on higher-order
thinking, enable higher-order cognitive behavior in learners. Creative procedures of knowledge generation in learners enhance cognitive abilities fUrther on.
Bolstering learners' cognitive skills is a specific educational target achieved with the help of dedicated pedagogy. There are teaching techniques that appear to be more conducive to cognitive skills development compared to others. Among successful techniques, researchers name such activities as information gathering, segmentation and restructuring of data, analysis and interpretation, problematizing and questioning other views, recognizing and treating constraints, articulating ideas, evaluating procedures and results [19].
Information analysis implies identifying components and their attributes, recognizing patterns and causal relationships, differentiating between central and peripheral ideas, identifying and treating factual and interpretative errors, i.e. commenting on "what is wrong".
Ideas generation characterizes productive minds capable of inferring, predicting and elaborating theories.
Thought evaluation includes establishing criteria, data and verifiable indicators for informed assessment of the products and processes.
It is necessary to emphasize that students' environment, which is learner-designed and self-directed, flexibly structured and periodically reorganized, less predictable and sometimes "adventurous" makes a markedly greater impact on cognitive skills development than traditional setting of predictable procedures and results [Ibid.].
Emphasizing the development of higher-order cognitive skills may produce a false impression that lower-order skills are of lesser significance in the learners' cognitive development. In reality, lower- and higher-order cognitive operations build upon each in accumulating thinking experience. This explains the necessity to train memory, supported with comprehension skills. Only on this condition will learners' minds be ready for exercising creativity skills, producing and implementing novel ideas from scratch [20].
Special attention in the development of higher order cognitive skills rightfully goes to the reading curriculum because learner's interaction with the written text is important in terms of thought stimulation. The following while-reading activities are among the most efficient: comparing and contrasting ideas, distinguishing the perspectives of the author and the heroes, explaining cause-and-effect relationship, interpreting circumstances, inferring insights and resolving puzzles [21].
The real value of learners' cognitive development is not so much the ability to solve problems, but the skill of learning to learn [22]. The significance ascribed to the skill of learning is not accidental for a number of good reasons. It is a measurable indicator of educational effectiveness, key competence for lifelong learning, and valid predictor of successful studies and careers. This prompts the necessity to develop complex problem solving tests that require demonstration of ability to learn on the spot.
Considering the research carried out in the area of higher order learn-ing-to-learn skills, a new dimension of teaching taxonomy emerges - learning taxonomy [23]. The reason for such shift of attention is that course-books and teaching practices putting special emphasis on comprehension and knowledge-in-memory questions still disregard transfer-application-analysis-synthesis-evaluation chain. Therefore, there is a need to teach learners to engage with knowledge, thus challenging stereotypes of "proper teaching".
Practically speaking, the taxonomy of learning to learn skills includes a variety of thinking types, because eventually learners, in the course of education, acquire not knowledge, whether declarative or procedural, but thinking skills. Thus, the outcome of education may be either reduced to the skill of memorizing the knowledge "on offer", or extended to the whole arsenal of learners' intellect. This means that the following types of thinking can make cognitive curriculum of teaching: executive, logical, judicial, creative, and critical [24].
The whole range of discussions on the issue of cognitive skills comes down to the following cognitive skills taxonomy:
Lower-order cognitive skills:
- owning knowledge;
- utilizing knowledge;
- demonstrating knowledge.
Higher-order cognitive skills:
- generating knowledge;
- generating problems;
- generating solutions.
Quasi-cognitive skills:
- metacognitive reflections;
- procedure organization;
- language use.
Cognitive skills can make an important part of language teaching curriculum. However, there are both proponents of and opponents to the cognitive-skills approach who either support or take issue with the higher-order cognitive skills in the language classroom [25, 26]. The opponents argue that not every language student possesses the higher-level skills of cognition demanded by some language learning activities. Instead, they suggest training students to perform in English what they already can do in their mother tongue from the point of cognitive operations. The crucial point of the debate here is whether language teachers should or should not address cognitive processes in learners.
Research and discussion
Teachers' opinion poll
In search of the teachers' views on the cognitive agenda in the English language classroom, we conducted an internet-based opinion poll among
139 teachers from various regions of Russia. The question was about the most frequent teaching activities in the lessons of English. The results are in diagram 1.
Diagram 1. Results of teacher's opinion poll on the most frequent language classroom activities
Key to the diagram:
1 - retrieving previously learned information;
2 - testing language and communicative skills;
3 - argumentative reasoning in discussion and writing;
4 - retelling previously learned texts;
5 - transferring knowledge to a different situation;
6 - consolidating and memorizing lesson material;
7 - comparing / contrasting objects, images and opinions;
8 - questioning contents;
9 - finding controversial information in books and on the internet;
10 - generating learner's own material.
The diagram shows that in Russia's classroom context, "retrieving previously learned information" is taking the lead, with "testing language and communicative skills" coming second. Unlike the tendencies of the pre-communicative era, "retelling previously learned texts" has moved a step down the list, letting ahead "learner reasoning in discussion and writing". Previously popular "consolidating and memorizing lesson material" has descended further down in favor of "transferring knowledge to a different situation". However, such activities as "questioning contents", "finding contro-
versial information in books and on the internet", "generating learners' own material" still occupy subsidiary positions in the chart.
Follow-up interviews
During the interviews, the teachers reflected on their opinions. The purpose of the interviews was to probe into the most favored activities in the English language lessons.
Most frequent activities according to the teachers' interviews are in chart 1.
Chart 1. Most frequent teaching activities according to the teachers' interviews
As the chart shows, the key activity most frequently mentioned by the teachers during interviews was text-based teaching. This implied text comprehension committing the content to memory, whether voluntarily or involuntarily.
An important feature of the "proper lesson" was teaching grammar and vocabulary with a variety of sentence- and text-based techniques.
Grammar-and-vocabulary activities, according to the teachers' model, had a "creative" follow-up. The arsenal of the tasks referred to as "creative" included storytelling (based on the studied texts), dialogues and role-plays (dramatized learner-made scripts), text- or problem-based discussions, summarizing a text or, making WebQuest presentations. Language testing was a destination point of the teaching / learning trajectory.
Most teachers mentioned the text as the premise for a variety of teaching and learning techniques. Quite a few teachers would mention "creative
tasks" in their lessons, though failing to unwrap the term. Many of them preferred to sideline material memorization although there were those who emphasized their trust in memory-driven exercises. Testing techniques featured prominently and the teachers almost unanimously acknowledge the wash-back effect of using language tests as training tools. Closed-ended tests dominated over open-ended tasks whether written or oral. Some teachers estimated teaching-testing ratio in their lessons as 50-50. They gave preference to closed-ended testing blanks as time saving and instrumental techniques for checking up vast areas of vocabulary and grammar, as well as text comprehension during reading or listening.
Discussion of the teachers' opinion poll and interview results
The priority given by Russian teachers of English to working with texts and retrieving previously learned information comes as no surprise because "learning", as it came out in the interviews, was to a large degree "the filling of memory storage with knowledge details and the taught skills".
By comparison, the ultimate goal of language instruction in the Chinese culture is either to read the needed materials and to study or work abroad. It comes as no surprise that the text functions as the springboard for training pronunciation, expanding vocabulary, practicing grammar points, speaking on the text-related subjects, writing compositions and doing translations [27].
It is true that one of the frequently used text-based cognitive operations is the retrieval of information from memory. Though it is not a higherorder cognitive skill, research proves that retrieval of rhetorical units is necessary for ample text comprehension. This means that retrieving
information from memory is the necessary pre-skill for higher-order text processing [28]. This explains why the teachers often prioritize memory-based teaching tasks.
It is also worth noting that in the Test of English Language Learning (TELL) one of the testing formats is "Listen and Retell" type. The students hear an extended narrative and retell it in as much detail as they can. The scoring rubrics do not require students to use the same words presented in the text. The focus is on the accurate reproduction of the ideas in the narrative [29].
With all the importance attached to text-based teaching and task-based amendments, the awareness of the need to boost cognitive load for better intelligence training, especially in content and language integrated learning (CLIL) is becoming apparent. The newly springing tendency is utilizing higher-order cognitive skills as a hallmark of high-quality teaching.
The most recent tendency is to integrate language learning and content studies. Until now, the tradition has been to overlook lower- / higher-order
cognitive skills in language instruction even though emphasis has long been on language and content / context teaching. The challenge to enrich the language curriculum with explicitly taught cognitive skills is now on the agenda [30]. Content and language brought together in the learners minds call for processing.
Deeper content understanding is not the only gain brought by cognitive skills training in the English language lessons. As L. Vygotsky powerfully proved, thought and speech though stemming from different roots merge into a unified function that enables social interaction and the skill of reasoning [31].
Reasoning consists in the ability to move logically, from premises to addon ideas, building up discourse structure with arguments buttressing assumptions. As research shows, the task of teaching reasoning is not training learners to argue, but creating conditions for seeking and using argumentation [32].
The attention of Russia's teachers of English to the text-based instruction prompted the next step of our research. It was the case study of teaching argumentative reasoning based on texts.
Action research rationale
The problem with teaching cognitive skills was that such skills were more difficult to develop and assess than retention of information from the text in memory. We decided to combine text-based techniques with cognitive activities in the open-ended writing tasks [33].
In our action research we proceeded from the premise that the key cognitive skill to be developed in students was argumentative reasoning as logical arrangement of evidence generated and evaluated for the purpose of problem solving in the course of cognition [34]. To teach argumentative reasoning we created conditions for the learners to utilize contextual knowledge of the text to produce their own "small theory".
According to the framework of California Critical Thinking Disposition Inventory, the cognitive construct characterizes learners with critical thinking skills as truth seeking, open-minded, analytical, systematic, confident, judgmental and inquisitive individuals capable of dealing with a problem. The procedures for developing critical thinking skills in learners as described in literature [35] include: questioning one's own beliefs; generating evidence both supporting and refuting; reasoning logically and honestly as conscientious believers; seeking alternatives in the most ample and unbiased way; weighing up arguments irrespective of your prior views; avoiding cognitive bias by giving full credit to arguments "against"; reflecting back on one's own critical reasoning with a possible change of mind [36]. These skills were explicated and taught to the students.
Additionally, we taught our learners to observe and describe, interpret and explain facts, use evidence, make connections and create a complexity of views, looking for and dealing with controversies, thinking critically about thinking [37].
In our lessons, the post-graduate students at the technical university were given a series of texts containing discursive reasoning and the task was to write their own deliberations on the subject.
The task ran as follows:
Read the text and write your own reflection on it. Remember to give a careful thought to the ideas in the text by comparing, contrasting and commenting on them, stating your initial thesis, developing it and writing a restatement in the end in a well-informed logical conclusion. Make your ideas transparent, the text organized, and the language accurate.
For assessing the learners' cognitive skills, we used a set of criteria -points of evaluating learners' performance. The criteria were paired with verifiable (observable) indicators that could be detected in the learner's discourse. The evaluation framework is shown in chart 2.
C h a r t 2
Critical thinking evaluation framework
Critical Thinking Evaluation Framework
Criteria Indicators
Content
Thesis relevant mature defendable
Evidence credible sufficient ample
Reasoning cohesive coherent informed
Ideas insightful original transparent
Conclusion restating premised convincing
Mechanics
Language accurate expressive adequate
Organization goal-driven structured highlighted
Critical thinking evaluation framework contains the "communicative contents" and " language mechanics" areas of assessment to estimate the quality of learners' thought process. The "communicative content" area breaks down into the "thesis", "evidence", "reasoning", "ideas" and "conclusion" items. They are assessed according to a series of appropriate indicators. The "language mechanics" area features the two items of assessment: "voclabulaty / grammar" and "logical organization". Each is assessed with the help of observable indicators.
Action research lasted for ten two-hour lessons (one lesson per week). Nine post-graduate students were taking part.
According to the assessment scheme, each verifiable indicator, if confirmed, brought the student one point.
The results of action research case study
The results of the action research are represented in chart 3.
C h a r t 3
The results of action research (based on the Critical Thinking Evaluation Framework)
Criteria Indicators Scoring
Content
Thesis relevant mature defendable 2.1
Evidence credible sufficient ample 2.3
Reasoning cohesive coherent informed 2.2
Ideas insightful original transparent 1.1
Conclusion embracing premised convincing 1.3
Mechanics
Language (vocabulary / grammar) accurate expressive adequate 2.4
Logical organization goal-driven structured sign-posted 2.7
Indicators in italics made the "short-term" agenda of instruction. These sub-skills developed in learners after a series of training sessions within a relatively short period of teaching (ten weeks). In contrast, indicators in bold type constituted a long-term agenda because apparently such sub-skills of critical thinking take considerably longer time to develop.
Our results were consistent with another independent research, in which training time was given to the development of cognitive skills. Short-term training raised the performance of crystallized intelligence, i.e. imitable and declarative knowledge. Fluid intelligence in the form of inimitable and procedural skills did not now show a reliable rise in scoring during extra training sessions. However, fluid intelligence indicators increased individually with the years [38-43].
The sub-skills marked in our Critical Thinking Evaluation Framework in italics referred to crystallized intelligence because the students were able to "follow and demonstrate" them. The sub-skills in bold type referred to as fluid intelligence and apparently proved more difficult to demonstrate as "learned".
By giving our post-graduate students short-term training, we succeeded in teaching them to:
- use the given data;
- generate their own evidence by accessing resources;
- follow the suggested format;
- organize the text structurally;
- achieve discourse cohesion with connectives.
Short-term training given to the participants failed to:
- broaden background and general knowledge;
- achieve maturity in evidence and argumentation;
- deal with the cognitive bias;
- overcome stereotypical thinking;
- learn the art of hypothesizing and theory building.
Short-term training proved to be relatively successful in teaching learners the knowledge that can be located in the text, such as, creating in
learners' memory a certain complexity of facts and assumptions. These attainments referred to the lower-order cognitive skills.
Higher-order cognitive skills consisting in critical approach to the established propositions, questioning stereotypes, creating controversies, seeking solutions and building theories apparently require a longer program of instruction.
Conclusion
The research into the development of cognitive skills in students has produced a number of findings that shed light on the issue suggesting integration of subject knowledge build-up, cognitive skills development and successful performance in a unified pedagogical process.
The learning process can boost the growth of lower- and higher-order cognitive skills as a merger of learners' ability to analyze localized details and to synthesize new knowledge by processing facts and opinions critically, reasoning logically and drawing conclusions in a well-informed way.
Processing localized ("pinpointed") details in the text is the way to develop skills of comprehension and analysis in learners, important but not sufficient for the ample actualization of intellectual functions. In contrast, whole-text message elicitation is the activity that trains the skills of synthesis and productive thinking, necessary for generating new knowledge.
The reality of learners' cognitive development is that the lower-order cognitive skills of processing localized knowledge develop sooner than the higher-order cognitive skills as the ability to generate knowledge through critical thinking. The latter develops through lasting and concerted efforts taking a long path of life-long learning.
The teaching community seems to give preference to teaching knowledge comprehension and learning rather than knowledge production, because the former brings fast and tangible results, while the latter may bear fruit much later. There appears to be no such thing as a fast track towards higher-order cognitive development of learners.
References
1. Eluemuno A., Uchenna A-O. (2013) The effect of metacognitive skills on performance in
English language among senior secondary school students. Journal of Emerging Trends in
Educational Research and Policy Studies 4. 3. pp. 33-39.
2. Khezrlou S. (2012) Cognitive strategy training; improving reading comprehension in the
language classroom. The Journal of Teaching Language Skills 4. 3. pp. 77-98.
3. Waters M., Waters A. (1992) Study skills and study competence: getting the priorities
right. English Language Teaching Journal 46. 3. pp. 264-273.
4. Sweller J. (2016) Cognitive load theory, evolutionary educational psychology, and instructional
design. Evolutionary Perspectives on Child Development and Education, edited by David C.
Geary and Daniel B.Berch. Switzerland: Springer International Publishing. pp. 291-306.
5. Atkinson R., Shiffrin R. (1968) Human Memory: A Proposed System and Its Control
Processes. Psychology of Learning and Motivation. Vol. 2, edited by Kenneth Spence and Janet Taylor Spence. New York: Academic. pp. 114-18.
6. Berardi-Coletta B., Dominowski R., Buyer L., Rellinger E. (1995) Metacognition and prob-
lem solving: A process-oriented approach. Journal of Experimental Psychology: Learning, Memory, and Cognition 21. pp. 205-223.
7. Borkowski J., Muthukrishna N. (1992) Moving metacognition into the classroom: "Work-
ing models" and effective strategy teaching. Promoting academic competence and literacy in school, edited by Michael Pressley, Karen R. Harris, and John T. Guthrie. San Diego, CA: Academic. pp. 477-501.
8. Duckworth A., Quinn P., Tsukayama E. (2012) What No Child Left Behind leaves behind:
The roles of IQ and self-control in predicting standardized achievement test scores and report card grades. Journal of Educational Psychology 104. 2. pp. 439-451.
9. Van Lehn K. (1996) Cognitive skill acquisition. Annual Review of Psychology 47. 1. p. 629.
10. Rosen J., Glennie E., Dalton B., Lennon J., Bozick R. (2010) Noncognitive Skills in the Classroom: New Perspectives on Educational Research. North Carolina: RTI International. doi: 10.3768/rtipress.2010.bk.0004.1009 (Accessed: March 1, 2017).
11. Finn A., Kraft M., West M., Leonard J., Bish C., Martin R., Sheridan M., Gabrieli C., Gabrieli J. (2014) Cognitive Skills, Student Achievement Tests, and Schools. Psychological Science 25. 3. pp. 736-744. doi: 10.1177/0956797613516008 (Accessed: February 27, 2017).
12. Prodromou L. (1995) The backwash effect: from testing to teaching. ELT Journal 49. 1. pp. 13-25.
13. Sabio J., Balagtas M., David A. (2015) Backwash effect of testing on learning mathematics. The Normal Lights 9. 2. pp. 156-179.
14. Tengberg M. (2017) National reading tests in Denmark, Norway and Sweden: a comparison of construct definitions, cognitive targets and response formats. Language Testing 34. 1. pp. 83-100.
15. Bax S. (2013) The cognitive processing of candidates during reading tests; evidence from eye-tracking. Language Testing 30. 4. pp. 441-465.
16. Hanushek E., Woessmann L. (2012) Do better schools lead to more growth? Cognitive skills, economic outcomes, and causation. Journal of Economic Growth 17. 1. pp. 267-321.
17. Kurekova L., Beblavy M., Haita C., Thum A-E. (2016) Employers' skill preferences across Europe: between cognitive and non-cognitive skills. Journal of Education and Work 29. 6. pp. 662-687.
18. Adams N. (2015) Bloom's Taxonomy of Cognitive Learning Objectives. Journal of Medical Library Association 103. 3. pp. 152-153.
19. Liu M. (2003) Enhancing learners' cognitive skills through multimedia design. Interactive Learning Environments 11. 1. pp. 23-39.
20. Goksy I., Gulcu A. (2016) The evaluation of the cognitive learning process of the renewed Bloom Taxonomy using a web-based expert system. The Turkish Online Journal of Educational Technology 15. 4. pp. 135-149. http://www.tojet.net/articles/v15i4/15413.pdf (Accessed: February 25, 2017).
21. Assaly I., Smadi O. (2015) Using Bloom's Taxonomy to Evaluate the Cognitive Levels of Master Class Textbook's Questions. English Language Teaching 8. 5. pp. 100-110 doi: 10.5539/elt. v8n5p 100 (Accessed: March 2, 2017).
22. Vainikainen M-P., Wurstenberg S., Kupiainen S., Hotulainen R., Hautamaki J. (2015) Development of learning to learn skills in primary schools. International Journal of Lifelong Education 34. 4. pp. 376-392.
23. Yildiring S., Baur S. (2016) Development of learning taxonomy for an undergraduate course in architectural engineering program. Australaian Journal of Engineering Education 13. 1. pp. 1-10.
24. Sun R., Hui E. (2012) Cognitive competence as a positive youth development construct: a conceptual review. The Scientific World Journal. http://dx.doi.org/10.1100/2012/210953
25. Newton J. (2016) Comprehending misunderstanding. ELT Journal 71. 2. pp.1-8. doi: https://doi.org/10.1093/elt/ccw096 (Accessed: March 3, 2017).
26. Swan M., Walter C. (2017) A response to Jonathan Newton. ELT. doi: 10.1093/elt/ccw095 (Accessed: February 23, 2017).
27. Cortazzi M., Lixian J. (1996) English teaching and learning in China. Language teaching 29. 2. pp. 61-80.
28. Machida S. (2011) Reading foreign language text - what separates skilled and unskilled FL readers? Electronic Journal of Foreign Language Teaching 8. 2. pp. 141-153, http://e-flt.nus.edu.sg/v8n22011/machida.pdf (Accessed: February 28, 2017).
29. Bonk W. (2016) Scoring TELL (Test of English Language Learning). Pearson Education. http://cdn2.hubspot.net/hubfs/559254/TELL/TELL_White_Paper_4_-_Scoring_TELL.pdf?t=1489783009334 (Accessed: February 26, 2017).
30. Spratt M. (2016) Focus on content-based language teaching. ELT Journal 70. 4. pp. 476478. doi: https://doi.org/10.1093/elt/ccw054
31. Vygotsky L. (1986) Thought and Language. Cambridge, Massachusetts: The MIT Press.
32. Rex L., Thomas E., Engel S. (2010) Applying Toulmin: teaching logical reasoning and argumentative writing. English Journal 99. 6. pp. 56-62.
33. Saavedra A., Opfer D. (2012) Learning 21st-Century Skills Requires 21st-Century Teaching. Phi Dalta Kappan 94. 2. pp. 8-13. http://dx.doi.org/10.1177%2 F003172171209400203 (Accessed: March 5, 2017).
34. Cavagnetto A., Kenneth K. (2016) Promoting students' attention to argumentative reasoning patterns. Science Education 100. 4. pp. 625-644.
35. Leicester M. (2010) Teaching Critical Thinking Skills. London, New York: Continuum International Publishing Group.
36. Zin Z., Wong B. (2014) Relationship between critical thinking disposition and critical reading skills of Malaysian EFL learners. The Asian EFL Journal Quarterly 16. 3. pp. 4168.
37. Maley A., Peachy N. (2015) Creativity in the English language classroom. London: British Council.
38. Carlson M, Dahl G., Rooth D-O. (2015) The effect of schooling on cognitive skills. The Review of Economics and Statistics 97. 3. pp. 533-547.
39. Auletta G., Colage I., D'Ambrosio P., Tocal L. (2011) Integrated Cognitive Strategies in a Changing World. Rome: G&B Press.
40. Kalypso I., Kendeou P., Beker K. (2016) Argumentative Reasoning. Handbook of Epis-temic Cognition, edited by Jeffrey A. Greene, William A. Sandoval and Ivar Braten. New York: Routledge. pp. 39-53.
41. Mageean B. (2000) Cognitive universals and embodied knowing. International Journal of Social Education 14. 2. pp. 233-45.
42. Tahtinen-Pacheco S., Merchant J. (2014) Promising instructional strategies for English learners in the science classroom. MinneTESOL Journal. http://minnetesoljourn al.org/spring-2014/promising-instructional-strategies-for-english-learners-in-the-science-class room (Accessed: February 23, 2017).
43. Winterton J., Delamare F., Stringfellow E. (2006) Typology of knowledge, skills and competence: clarification of the concept and prototype. Luxembourg: European Centre for the Development of Vocational Training.
Information about the authors:
Millrood R.P., Dr.Sc. (Education), Professor, Department of International Professional and
Scientific Communication, Tambov State Technical University (Tambov, Russia). E-mail:
rad_millrood@mail.ru
Maksimova I.R., Ph.D., Associate Professor, Department of Foreign Languages, Russian
Academy of FSIN (Ryazan, Russia). E-mail: kaf-inyaz-academy@mail.ru
Received 3 May 2018
