CC BY-ND
54
Journal of Language & Education
Volume 1, Issue 2, 2015
Teachers’ written feedback: Does the delivery method matter?
Natalia Koliadina
National Research University Higher School of Economics
Responding to student writing, which is a widely researched area, is still one of the most challenging parts of the EAP (English for Academic Purposes) teacher’s job. Little attention has been given to analyzing the role of systematic feedback on students’ improvement of writing at the university. The paper reports on the results of a small-scale action research conducted among first-year undergraduate students, which explored the effect of record sheets, used as a tool to track student progress in writing argumentative essays. Apart from student portfolios and record sheets, the 8-week study used other methods of data collection that included recorded semi-structured interviews and a survey. Findings show that providing consistently structured (praise and criticism) selective (global and local) feedback to students has a positive effect both on the teacher and on student perception of feedback and, generally, their achievements in developing writing skills. The study may motivate EAP practitioners to change their current classroom practices and seek more effective ways of responding to student writing.
Keywords: feedback to student writing, record sheets, consistent feedback, tracking student progress at university, EAP
Introduction
Feedback on students’ academic development plays an important role in helping them adjust to university life. For instance, feedback on students’ writing allows them to see how others perceive their work and to learn about the nature of the university education, about the “role of writing in learning, about their identity as students, [and] about their competence as writers” (Hyland, 2006, p. 102).
While responding to student writing is a widely researched area, and both teachers and students feel that teacher written feedback should be an integral part of the writing process (Ferris, 2001; Hyland, & Hyland, 2001; Montgomery, 2007), it is still one of the most time-consuming and challenging parts of the English for Academic Purposes (EAP) teacher’s job.
Natalia Koliadina is an EAP instructor at the International College of Economics and Finance (ICEF), National Research University Higher School of Economics, Moscow, Russia. Her interests are in the field of academic skills development, assessment and feedback, materials design. She is also involved in teaching and learning projects at ICEF.
E-mail: nkoliadina@hse.ru
Previous research suggests that there are “different and legitimate ways to respond to student writing that may vary across students, teachers and specific contexts” (Ferris, 2007, p. 175); however, little attention has been given to analyzing the role of systematic feedback on students’ improvement of writing. In this article, systematic feedback, is defined as a regular commentary on students’ written assignments, provided in a consistent way, to “express a teacher’s stance and beliefs about writing and negotiating a relationship with learners” (Hyland, 2001, p. 186).
From both theoretical and practical points of view, it is important to understand whether tracking students’ progress in writing helps to “create a supportive teaching environment” (Hyland, p. 186) and serves the purposes of motivating and helping students to develop their writing skills. This study was based on several assumptions: feedback should follow relevant assessment criteria (Ferris, 2007); comments should highlight students’ strengths and items/areas that need improvement (e.g. Hyland, 2001; Treglia, 2008); feedback should be both content-focused and form-focused (Bitchener, 2008; Ferris, 2001, 2007; Hyland, 2001) and be tailored to the needs and progress of individual students; it should address the students’ reactions to these comments (Treglia, 2008) and “find
27
NATALIA KOLIADINA
the correct balance between intervention (helpful) and appropriation (harmful)” (Ferris, p. 167). With this in mind, the primary aim of this research was to explore whether record sheets, context-specific forms used to provide regular systematic feedback, help develop students’ essay writing skills.
Literature review
Many studies on feedback in second language writing suggest that it is a “crucial means of encouraging students’ learning and writing” (Leki, 2006, as cited in Poverchjuc 2011, p. 144; see also Black & William, 1998; Hyland & Hyland, 2006), while at the same time, responding to student writing is one of the most challenging tasks that writing instructors have to face.
The extent to which students benefit from feedback has been debated since Truscott (1996) questioned the effectiveness of grammar correction, by which he meant “correction of grammatical errors for the purpose of improving a student’s ability to write accurately” (p. 329) and suggested that the practice should be abandoned in favour of other aspects of writing, such as “organization and logical development of arguments” (p. 356). While that study was limited to discussing corrective feedback on grammar, a more balanced approach that incorporates content-focused and form-focused or global and local feedback (Bitchener, 2007; Connors & Lunsford, 1993; Ferris, 2006, 2007; Montgomery, 2007), has been widely adopted since then. However, empirical studies have shown that teachers primarily “address local issues of grammar and mechanics, which might suggest to students that they should prioritize local errors” (e.g. Montgomery, 2007). In her seminal paper, “Preparing teachers to respond to student writing”, Ferris (2007) argues that the balance between content-focused and language-focused feedback should be found and that it should be selective, in other words, it should focus on a limited number of items, as the goal of motivating students to improve their writing and become autonomous learners is more important than fixing particular problems.
Students’ reaction to teacher comments is another controversial issue. The students may not understand and, hence, fail to make good use of feedback provided to them (Hyland & Hyland, 2006 , as reviewed in Pov-erjuc, 2011, p. 144; see also Price, O’Donnovan & Rust, 2007). Research also shows that their understanding of the teacher comment might be affected by different factors, such as the wording (directive vs. mitigated), functions (praise, criticism, and suggestion) and delivery method (cyber-tutoring, face-to-face conferencing, written commentary: marginal or end/cover memo) of the feedback (Ferris, 2007; Hyland & Hyland, 2001; Tre-glia, 2008). For example, too much criticism may affect
students’ self-confidence and motivation (e.g. Connors & Lunsford, 1993). At the same time, students find commentary providing “some acknowledgment of their writing, [offering] specific suggestions, and [giving] them choices” being most helpful (see, e.g., Treglia, p. 105). Moreover, teachers do not have to mitigate their critical remarks, but should be aware that “comments that combine praise with constructive criticism are an effective tool to provide students with the confidence and motivation they need to actively engage in the [learning] process” (Treglia, p. 130). Hyland and Hyland (2001) argue that teachers make decisions when responding to student writing, which are based on a “desire to negotiate interactions that recognize both the
learner’s struggle to make meaning.. and the fragile
intimacy of teacher-student relationship” (p. 192).
While this literature review provides theoretical support for teachers providing feedback on student writing, it also points to the possible reasons why feedback may fail to be a successful tool in improving student writing. However, apart from several studies that mention interactive cover sheets/cover memos or end notes as a method of feedback delivery, little research has been done on providing consistently structured feedback to students and its effect on student perception of their achievements in developing their writing skills.
Methods
Research context
The International College of Economics and Finance (ICEF) is a double degree BSc programme, where English is used as a medium of instruction. EAP classes are aimed at supporting the ICEF students as they adjust to their studies, so they can make considerable progress in their academic development as future economists. The EAP writing course is taught to first year students, aged 16-17, whose level is B2 or higher on the Common European Framework of Reference (CEFR) scale. Student performance is assessed formatively through weekly home assignments (essays and other writing tasks) and summatively through a midterm and a final exam, International English Language Testing System (IELTS).
Although there is no official policy, the EAP teachers are expected to provide feedback on both local (grammar, vocabulary and mechanics) and global issues (content and organisation). Students usually complete only one draft, on which they are given feedback; essay revisions are rare, so the feedback they get is more like feedforward to their future essays.
The study was designed to answer the following research questions: Will the use of record sheets
28
TEACHERS’ WRITTEN FEEDBACK: DOES THE DELIVERY METHOD MATTER?
as a tool of tracking feedback help develop students’ writing skills. If so, how?
Participants
Although most students submitted their essays during the first semester, only a few students did that regularly and observed the deadlines; so, for the purposes of the project, it was decided to keep it small and, out of thirty students, a group of 10 students was chosen, those who were willing to participate. The study’s intervention took place over one semester. The students wrote an in-class 40-minute diagnostic IELTS-type essay at the end of the first semester. The essays were assessed by independent English teachers, i.e. not the students’ teachers; the students were informed of the final results.
Data collection tools
The data for the study comprised ten student portfolios of at least six essays, with feedback, that the students submitted during 8 weeks in the second semester and the essay they wrote at the end of the first semester. All the essays that students wrote during the second semester were formatively assessed. The deadlines for essay submission (one essay per week) were set, which the students were expected to meet. The students had four hours of EAP classes per week, which included a two-hour writing component.
Besides the teacher feedback samples taken from the essays in student portfolios, data were collected from the feedback record sheets (Appendix 1), which were completed for each student on a weekly basis. The record sheets were adapted from a tool developed at the University of East Anglia, and their design incorporated best practices for responding to student feedback. The record sheets stated features of academic writing as well as summary of level of student writing skills development as praise or criticism e.g. Done well, Needing work (week 1), Continuing to do well from week 1, Now doing well, Now needing work (week 2). Unlike cover sheets, which are in use at some universities, record sheets helped to track the students’ progress over two months and were supposed to give the teacher a balanced view of the progress the students made, and at the same time reinforced the classes as made it possible for the teacher to refer to the students’ essays and provide generic feedback. The record sheet information replicated the one students received as an endnote, a short comment written at the end of the essay, which summarized in the form of praise and criticism (Done well, Improvement needed) and were based on the marginal and in-text comments made in the essay. The initial plan was to use record sheets in oral feedback sessions with the students, but the interviews showed that some stu-
dents preferred written feedback to oral feedback, so the endnote was chosen for the students to see the summary of the main things they did well and things they needed to improve, and left the record sheets to the teacher, making quick references to them while talking to the students during office hours.
By the beginning of the second semester, the students had been introduced to the basic concepts and features of writing, such as academic style, coherence and cohesion, paragraphing, references, hedging, as well as the criteria on which their essays had been assessed, namely, Task response, Coherence and cohesion, Lexical resource, Grammatical range and accuracy, which sometimes included referencing to sources. Making students aware of these criteria was important in order to provide feedback that would “ help them progress toward meeting the standards” (Ferris, p. 170). In the second semester, several features of writing were reviewed, the choice of which was based on the formative essay assessment. Student essays informed the teacher about the specific things that needed to be reviewed in the areas of content (addressing the topic, relevance of information), organization (thesis statements, cohesive device), accuracy (passive forms, conditionals, relative clauses, subject-verb agreement) and mechanics (punctuation in relative clauses, referencing). In the feedback, a selective, rather than comprehensive approach was taken by focusing on two to four most significant feedback points in the essay, such as the quality of thesis statement and how it was developed in the main body, or, the use of topical vocabulary, rather than addressing every single problem.
Data analysis tools
In the study’s second week, a semi-structured interview was conducted with each participant receiving feedback from the students on their attitude towards classroom practices. The students were asked to evaluate their progress in writing in the last two months, answer six questions and explain their choices. These questions (Appendix 2) asked students which of the listed activities (self-evaluation; peer evaluation; analysis of model texts; written feedback from peers; oral feedback from peers; written feedback from the teacher; oral feedback from the teacher) they found most and least useful. They were also asked whether they understood the assessment criteria and could identify the areas they needed to improve most in their writing.
In week four, the students were invited to participate in a survey (Appendix 3), which was aimed at collecting data on their perceptions of teacher feedback, which had been provided in written (marginal or in text) and oral formats. The survey included closed items; the students could add comments on the questions and
29
NATALIA KOLIADINA
their answers. The survey questions addressed topics such as: how they felt when they finished writing an essay; whether they read all of the written comments or just some of them; how they felt after reading the teacher’s after reading the teacher’s comments; which feedback method they preferred and why. The answers and comments were categorized according to recurring themes and then analysed.
Results
Submission rate
On a general note, essay submission increased -8 students out of 10 met the deadlines and completed at least one essay per week, which could be the result of setting the deadlines for essay submission. If the students failed to submit their essay on time, they could not get feedback from the teacher the following week or participate in class discussions and other activities, e.g. self-evaluation and peer evaluation. Having a system in place and doing things consistently seemed to motivate both the students and the teacher.
Student perception of own progress and needs: Interview
The results of the interview showed that the students were enthusiastic about receiving timely feedback and asked many questions not so much about the grades, but about particular items in the feedback.
Table 1 shows that eight students out of ten evaluated their progress in writing (01) as Good or Very Good; only two of them thought that they made Fair progress - those students had actually submitted the smallest number of essays during the period under consideration. All the students found written feedback from the teacher (WF(T)) to be most useful; Five chose self-evaluation (SE), analysis of model texts
(AMT) and oral feedback from the teacher (OF(T)) to be equally important (02). Among the four criteria areas they needed to improve Lexical resource (LR) and Grammatical range (GR) accounted for eight and five respectively. Four students mentioned Coherence and Cohesion (CC) as their improvement area; only two mentioned Task response (TR) as an area they thought they needed to improve, which did not always correspond to what the results of the diagnostic test showed (03, 04).
Surprisingly, some students preferred written feedback to oral feedback. Below is an example of part of the interview, in which the student explains his preference :
T: Which of the listed activities do you find most useful?
S: The most useful for me is self-evaluation and, of course, written feedbackfrom the teacher...
T: Why?
S:Mmm.... becauseyou can understand what you did wrongand then deal with it.
T: Can you explain why you find self-evaluation useful?
S: Well, I think it’s the most important because if you can’t understand it, you can’t attain yourgoal.
T: And oralfeedback?
S: I have a good short-term memory. I need to see it (feedback) written to remember it later.....
It is not clear whether the student would choose written feedback only because he thinks he has a bad long-term memory. According to some studies, learners may have “cultural or social inhibition about engaging informally with authority figures, such as teachers, let alone questioning them” (Hyland & Hyland, 2006, p. 83), which might be the case with Russian students.
Student perception of feedback: Survey
The results of the Survey showed that the majority of students (78%) felt optimistic after they finished
Table 1
Results ofinterviews on student perceptions and preferences
SI S2 S3 S4 S5 S6 S7 S8 S9 S10
01 Fair Good Very Good Very Good Good Fair Very Very
good good good good
02 WF(T) WF(T) WF(T) WF(T) WF(T) WF(T) WF(T) WF(T) WF(T) WF(T)
SE OF(T) SE PE AMT SE AMT OFS SE MT
MT PE OF(P) OF(T) OF(T) OFT
03 Y Y Y Y Y Y Y Y Y N
N Y Y N Y Y Y Y Y Y
04 LR GR GR LR LR ТА LR CC ТА ТА
GR CC CC
30
TEACHERS’ WRITTEN FEEDBACK: DOES THE DELIVERY METHOD MATTER?
writing an essay; 22% felt they could do better. All students reported that they read all the comments in the feedback. 75% students felt that they were encouraged after reading the comments.
When asked about the methods of feedback they prefer (Fig.3), the majority of the students mentioned feedback on grammar (89%) and a short commentary (90%) at the end of the essay, endnote. 67 per cent of the students expected the teacher to give feedback on content / ideas, which is nearly twice as many as those who chose feedback on organization and mechanics. Surprisingly, vocabulary was chosen by only 44 per cent of the students. Contrary to popular belief, students preferred that the teacher wrote a moderate number of comments to writing a lot of or very few comments.
The study also provided evidence that feedback to student writing should not be limited to justifying the grade only and that the teacher’s comments evolved from standard phrases taken from the essay marking criteria and descriptors to more personalised comments that would be ‘a response to a person rather than to a script’ in other words, ‘construct a context that relates feedback to specific learners’ (Hyland & Hyland, 2006, p. 206, 213). This became clear from the survey, the interview and the conversations during office hours. Below are some examples from the survey open comments:
“I do not think that the number of comments is important. I would prefer every helpful piece of information that you give to me in comments. If there is less information, I can ask you about it.”
“I think you should emphasise our weaknesses, but also tell us about our strengths in order to show some good tendencies”.
Discussion
The results of this study may suggest that responding to student writing in a systematic way may have a positive influence on the students. Firstly, if properly organised, it helps to shape students’ expectations and track their achievements. Many students seem to think that addressing local errors (grammar and mechanics) is the most important part of writing classes. Nevertheless, it does not follow that teachers should not change their own and their students’ practices and expectations, as, for example, the emphasis on local issues may suggest to students that they should prioritize local errors. Secondly, keeping a record of student progress, in the long run, is likely to result in limiting the feedback to the most important issues, which could benefit the students,
I pessimistic (33%) I optimistic (67%)
Figure 2. Student perception of teacher feedback.
%
Endnote I Marginal comments В Positive comments В Critical comments В Feedback on grammar В Feedback on vocabulary В Feedback on content/ideas В Feedback on organization В Feedback on mechanics
Figure 3. Student preferences of feedback methods.
because they are more likely to have a positive perception of the teacher feedback and to respond to the feedback they get, which facilitates the process of their writing skills development. Thirdly, there is widespread opinion among EFL educators in Russia that students can benefit from extensive feedback. The study showed that the students seem to prefer moderate number of comments, which would combine praise and criticism. More research is needed
31
NATALIA KOLIADINA
in order to understand how both types of comments affect individual students as they have the “potential to construct the kinds of relationships which could either facilitate or undermine a student’s writing development” (Hyland & Hyland, 2001, p. 207).
Finally, while it is useful to have a bank of phrases and use them appropriately in teacher feedback comments, teacher comments should go beyond simple decisions to address form or content or to praise mechanics or criticize organization. EAP instructors need to provide helpful advice and, at the same time, negotiate interpersonal relationship that will facilitate its development (p. 208) as “thoughtful feedback tailored to the needs of an individual student and his/ her evolving text and writing is a gift, and perhaps the most important thing a writing instructor can do for his/her students” (Ferris, p. 169).
6 Conclusion and final thoughts
“Response to student writing is extremely challenging to do well, and it takes considerable reflection and experience” (Ferris, p. 179). Further research is necessary into the role of feedback on teaching and learning and methods of its delivery to students. It is important to encourage students to be responsible and to develop a sense of ownership over their writing. Even though they may not be highly proficient in English, they are mature enough to engage in a dialogue about their writing with their instructors. The teacher should be aware that combining criticism with praise can be an effective tool to provide students with the confidence and motivation they need to actively engage in the writing process (Treglia, 2008). One of the ways to achieve this is through a systematic and constructive dialogue with their students and using different tools to monitor students’ progress, which has a role to play in the process.
References
Bitchener, J. (2008). Evidence in support of written corrective feedback. Journal of Second Language Writing, 17,102-118.
Ferris, D. (2007). Preparing teachers to respond to student writing. Journal of Second Language Writing, 16, 165-193. doi: 10.1016/j.jslw.2007.07.003
Hyland, F., & Hyland, K. (2001). Sugaring the pill. Praise and criticism in written feedback. Journal of Second Language Writing, 10, 185-212. doi: 10.1016/ S1060-3743(01)00038-8
Hyland, K., Hyland, F. (2006). ‘Interpersonal aspects of response: constructing and interpreting teacher written feedback’. In K.Hyland and F.Hyland (eds), Feedback in Second Language Writing: Contexts and Issues (pp. 206224). Cambridge: Cambridge University Press.
Hyland, K., & Hyland, F. (2006). ‘Feedback on second language students’ writing’. Lang. Teach. 39, 77-95. doi:10.1017/S0261444806003399 Cambridge University Press
Montgomery, J. L., & Baker, W. (2007). Teacher-written feedback: Student perception, teacher self-assessment, and actual teacher performance. Journal of Second Language Writing, 16, 82-99. doi: 10.1016/j. jslw.2007.04.002
Poverjuc, O. (2011). The experiences of feedback practices on Academic Writing undergone by students with English as an additional language in a Master’s programme: a multiple case study. Journal of Academic Writing, 1(1),143-152.
Reid, J. (1994). Responding to ESL students’ texts: The myths of appropriation. TESOL Quarterly, 28, 273-292. doi: 10.2307/3587434
Treglia, M. (2008). Feedback on feedback: Exploring student responses to teachers’ written commentary. Journal ofBasic Writing, 27(1), 105-137.
Truscott, J. (1996). The case against grammar correction in L2 writing classes. Language Learning, 46(2), 327369. doi: 10.1111/j.l467-1770.1996.tb01238.x
32
TEACHERS’ WRITTEN FEEDBACK: DOES THE DELIVERY METHOD MATTER?
Appendix 1
Record sheet (example)*
Student (name)__________________
Features Notes Action
of Plan
Academic Writing
WEEK 1
Done well
Needing
work
Needing
work
WEEK 2
Continuing to do well from week 1
Needing work from week 1
Now doing well
Now needing work
WEEK 3
Now doing well
Now needing work
Needing work from week 2
Appendix 2
Questionnaire April 9, 2014
1. How would you evaluate your progress in writing during the past months?
Excellent Verygood Good Fair Poor
2. Which of these activities do you find most useful? Explain.
о Self-evaluation о Peer evaluation о Analysis of model texts о Written feedback from your peers о Oral feedback from your peers о Written feedback from your teacher о Oral feedback from the teacher
3. Do you understand writing assessment criteria/ rubrics/descriptors?
4. Which of the four criteria areas for your essays do you think you need to improve most?
5. What goals do you currently have to improve your essays? Explain.
6. How are you going to achieve these goals?
7. Comments
33
NATALIA KOLIADINA
Appendix 3
Quick survey (May)
1. How do you feel when you finish writing an essay?
□ I am optimistic about having done a good job.
□ I usually feel I could have done better.
2. When I return your essay, do you read all of the written comments or just some of them?
□ All of them
□ Most of them
□ Some of them
□ None of them
3. How do you usually feel after reading my comments?
□ Encouraged
□ Same as before
□ Discouraged
4. Which feedback method do you prefer?
Please choose ALL the options that reflect your answer.
□ Endnote (my short commentary at the end of your essay)
□ Marginal comments (e.g. sp - spelling, s-v - s-v agreement., ss - sent structure, ww - wrong word)
□ Positive comments
□ Critical comments
□ Feedbackongrammar
□ Feedbackonvocabulary
□ Feedback on content/ideas
□ Feedback on organisation
□ Feedback on mechanics (e.g. punctuation)
5. Do you prefer that I write.
Please explain the reason for your preference.
□ alotofcomments?
□ amoderatenumberofcomments?
□ veryfewcomments?
6. Thankyou!
34
