Available online at www.jlls.org JOURNAL OF LANGUAGE AND LINGUISTIC STUDIES ISSN: 1305-578X Journal of Language and Linguistic Studies, 13(2), 561-577; 2017 Digital self-review and anonymous peer feedback in Turkish high school EFL writing* Ayten Kayacan a , Salim Razı b ** a Ministry of National Education, Tekirdağ, Turkey b Çanakkale Onsekiz Mart University, Çanakkale, Turkey APA Citation: Kayacan, A., & Razı, S. (2017). Digital self-review and anonymous peer feedback in Turkish high school EFL writing. Journal of Language and Linguistic Studies, 13(2), 561-577. Submission Date:01/08/2107 Acceptance Date:22/09/2017 Abstract Although writing is considered one of the most essential foreign language skills, to nurture and develop writing skills among students is challenging. To overcome this, teachers have recently considered benefiting from digital technology. Self-monitoring and self-evaluation, as sub-skills of metacognition, in addition to scaffolding, might be beneficial in accordance with the Zone of Proximal Development; hence both peer feedback and self- monitoring could be integrated with digital platforms in language classrooms. This quasi-experimental research study aimed to investigate the impact of exchanging self and anonymous peer feedback on writing assignments in a digital environment among Turkish EFL high school learners. Data were collected both quantitatively and qualitatively by means of open-ended questions and writing task scores using an analytic scoring rubric. Forty- six students in two intact classes exchanged both self and anonymous peer feedback through four writing assignments submitted via Edmodo as the digital environment. Participants were categorized as good, moderate and weak in each group and each student exchanged self and peer feedback on four written assignments in reverse order. Findings revealed that both self and peer feedback contribute positively to the revision of papers by student authors as they scored significantly better in the revised versions. Their writing scores indicated improvement in five major components: organization, content, grammar, vocabulary, and format. The participants also reported improvements related to content, grammar and format and indicated a positive attitude towards digital self and peer feedback. Since both digital self and peer feedback were found to be beneficial, EFL teachers should be encouraged to implement them in their writing classes. Keywords: digital environment; Edmodo; EFL writing; peer feedback; self-review 1. Introduction Writing is seen as among the most crucial skills by foreign language learners due to its productive role in communication. A student-centered approach to teaching writing enhances student authors` reflection on their own writing process and consequently they produce their own strategies for prewriting, drafting and rewriting. In this way, the main focus is on the process itself leading to the final version of the writing task (Shih, 1986). Therefore, giving feedback is seen as a crucial tool for development of second language (L2) and foreign language (FL) writing skills for learners to express * This study reports the findings of a master‟s thesis written by the first author under the supervision of the second author. The preliminary findings were presented at the GlobELT 2017 Conference. ** Corresponding author. Tel.: +90 286 2171303 ext: 3661 E-mail address: [email protected] 562 Ayten Kayacan, Salim Razı. / Journal of Language and Linguistic Studies, 13(2) (2017) 561-577 meaning effectively with the help of multiple drafts (Hyland & Hyland, 2006). „L2‟ and „FL‟ are used interchangeably in this study. In sum, similar to the suggestions of Kroll (2001), learners are expected to benefit from feedback effectively in order to improve their writing skills. As awareness regarding the importance of learner autonomy rises, research on self-review has received more attention. Self-assessment depends on the notion of learner autonomy; therefore, if teachers encourage students to reflect on their own learning, they can equip them with an effective tool for future learning. Once learners are involved in their own assessment, their awareness of learning improves as a useful outcome (Harmer, 2001). Over the last decade, new technology has played a significant role in L2 classes and brought a great number of benefits to writing classrooms. Therefore, computer-based instruction offers an alternative to traditional materials such as pen and paper (Hyland, 2003a). Technological developments have brought innovative ways of exchanging feedback in language classes. These technological developments can be observed in use in the Turkish educational system. For example, the FATİH Project (Action Plan to increase Opportunities and Develop Technology) aims to make technology accessible to all learners and teachers. Turkish schools are equipped with interactive boards bringing the internet into the classroom and tablet PCs are distributed to students and teachers. The main aim of the project is to integrate technology into the teaching and learning process by using information technology tools to develop effective educational materials at home and in school (“Ministry of National Education”, 2012). The foreign language curriculum of the Ministry of National Education has expectations of enabling opportunities for the more effective use of digital materials and platforms in the learning and/or teaching process. In sum, self-monitoring and self-evaluation as sub-skills of metacognition, in addition to scaffolding, might be beneficial in accordance with the Zone of Proximal Development (ZPD). Hence, both peer feedback and self-monitoring could be integrated with digital platforms in language classrooms to improve writing performance and facilitate language learning. Taking into account these aspects, the present study aims to shed light on the effect of exchanging self and anonymous peer feedback on writing assignments in a digital environment among Turkish EFL high school learners. 1.1. L2 writing A number of perspectives on teaching writing has emerged in recent years. Even though teaching writing involves various approaches and techniques, they tend to include the same core features (Badger & White, 2000). Relevant research has provided ideas on how to teach and learn L2 writing by considering the issue from three aspects, namely, focusing on process, product and social cultural theory. Each aspect has a different effect on L2 writing. The process-based approach, according to Kroll (2001), is an umbrella expression that covers various types of writing courses. The model includes a cyclical approach rather than an individual approach. In other words, this kind of approach focuses on developing student skills so that they are able to organize and define a problem and then propose and assess solutions (Hyland, 2003b). According to Hyland`s model of the writing instruction process; planning, drafting, revising and editing generates in a recursive order an ability that enables learners to revise and evaluate interactively and simultaneously before producing the final written material. It focuses on the author as an independent composer of the written task; and further, it helps teachers guide students to perform better in the writing process. In this model, the teacher's role is to make learners aware of the production procedure with the help of pre-writing and brainstorming activities, such as composing drafts, exchanging peer feedback, and encouraging/generating ideas and discussions (Raimes, 1992). In this approach, according to Hyland, teachers should develop the students` metacognitive awareness .Ayten Kayacan, Salim Razı / Journal of Language and Linguistic Studies, 13(2) (2017) 561-577 563 of their processes in order to enable them to use writing strategies effectively. That is to say, not only students but also teachers play crucial roles in planning, drafting, revising and editing the written material. 1.2. Self-review and peer feedback in writing The initial idea of focusing on the product has been transformed into the notion of exchanging feedback in writing classes (Zamel, 1982). In this respect, recent studies have concentrated on understanding L2 writing via the process approach, whereby students revise writing tasks by generating strategies to develop ideas, work on multiple drafts, provide responses and review their texts (Chenowith, 1987; Raimes, 1985, 1987). Hence, teachers have been directed to use alternative options regarding teacher feedback instructions in order to concentrate broadly on the aspects of meaning and the writing process itself (Cumming, 1985; Zamel, 1985). In Vygotsky`s (1978) concept of the zone of proximal development, it is essential that peer feedback facilitates internalization and improvement. Vygotsky clarified this as a crucial aspect of learning by generating the ZPD. In other words, learning stimulates a number of internal progressive procedures that learners can control by cooperating with others and with the assistance of their peers. These procedures become a feature of the learner's success after they are adopted. Accordingly, learners can be taught strategies for generating ideas in the collaboration process through assisting and scaffolding with an adult or a more efficient peer. As learners gradually progress in developing their learning and thinking skills, they need less guidance and eventually are able to think and learn independently without any assistance. As briefly mentioned above, peer response corresponds to a shift from a product-based to process- based learning style and is consistent with multiple drafts and a number of revisions, which is a component of the process approach (Hyland & Hyland, 2006). Partridge (1981) contends that teacher feedback can enhance greater understanding for prompt correction by learners; however, peer feedback could also make an important contribution to the judgment and sensitivity of the reading audience in the long term. Furthermore, Leki (1993) suggests that providing peer feedback enables students to raise their awareness of audience considerations. Consequently, peer response is an essential tool for developing writing skills by scaffolding in process-based writing classes. Although peer editing has been extensively studied, self-monitoring has been neglected. Yet, as awareness regarding the importance of learner autonomy rises, it has received more and more attention. Studies dealing with the notion of self-monitoring base their theoretical discussion on Flavell‟s (1976, 1979) notion of metacognition, which refers to awareness of one's own learning, dealing with the writer's experiences or regulation of learning. Metacognitive regulation refers to how learners monitor and control their own cognitive processes (Nelson & Narens, 1990). Apart from the teachers‟ evaluation, learners can also monitor and judge their own learning. Learners have a tendency to consider how well they have been doing. Teachers can easily encourage learners to improve this awareness so that they can enhance their own learning process. Self-assessment depends on the notion of learner autonomy; therefore, if teachers foster students‟ reflection on their own learning, they can equip them with an effective tool for future learning. Once learners are involved in their own assessment, there is a useful outcome in that their own awareness of learning improves (Harmer, 2001). All in all, learner autonomy is characterized in the frame of teaching pedagogically as the control of learning being in the hands of the learners; hence they are encouraged to engage in learning individually (Benson & Voller, 1997). Within this framework, Ferris (2002, p. 87) suggests that L2 authors to “be aware of … [their] own individual error patterns”. Similarly, according to Sun and Feng 564 Ayten Kayacan, Salim Razı. / Journal of Language and Linguistic Studies, 13(2) (2017) 561-577 (2009), successful writers should be aware of how to assess their own language and how to develop their own writing by checking and searching for their problems. Thus, the students will become better at writing. 1.3. Digital feedback Over the last decade, new technology has played a significant role in L2 classes, which has resulted in dramatic changes in the methodology of language learning and teaching (Kern & Warschauer, 2000), as in the case of computer-based instruction. Such changes have brought a large number of benefits into writing classes. As an alternative to the conventional approach, a great number of innovations in L2 writing have taken place enabling the use of pedagogical approaches in the digital environment, such as conferencing and peer editing (O`Brien, 2004). Accordingly, new technology enables learners to submit their texts in digital environments for peer feedback (Taylor & Ward, 1998). Digital environments facilitate interaction among students for peer response more locally (Crawford, Honan, Knobel & Lankshear, 1998), as well as searching for and publishing texts online and improving their communication skills outside school (Dudeny, 2000). 2. Method The present study was designed as a quasi-experimental research study in which participants from two intact classes were labeled Group A and Group B. In this mixed-method study, quantitative data were collected from the writing task scores of students by means of an analytic scoring rubric (adapted from Bursa Technical University, School of Foreign Languages) and qualitative data were collected from open-ended questions in a survey. Furthermore, a counterbalanced design was followed in which all treatments and assignments were assigned to all participants in a different order. Both groups performed the same tasks, but in reverse order, i.e. while Group A exchanged peer feedback on Assignments 1 and 2, Group B self-reviewed the same assignments. In reverse order, for Assignments 3 and 4, Group A self-reviewed whereas Group B exchanged peer feedback. 2.1. Research questions The present study aimed to investigate the effects of self-review and peer feedback on students` writing performance in a digital environment by revealing whether there was an improvement between the students` first and final draft writing task scores, either in the self-review or peer feedback groups. It also aimed to shed light on the perceptions of participants on exchanging self and anonymous peer feedback regarding writing assignments in a digital environment. For this purpose, the following research questions were formulated: 1. What is the effect of exchanging self and anonymous peer feedback on Turkish EFL high school students` writing task scores in a digital environment? a) Is there a significant difference between the first and final draft writing task scores of the self- review group? b) Is there a significant difference between the first and final draft writing task scores of the anonymous peer feedback group? 2. What is the effect of changing type of feedback (self-review or peer) in groups on the writing tasks in a digital environment? .Ayten Kayacan, Salim Razı / Journal of Language and Linguistic Studies, 13(2) (2017) 561-577 565 a) How does replacing feedback type affect the contribution of feedback in Group A? b) How does replacing feedback type affect the contribution of feedback in Group B? 3. What are the perceptions of students on self-reviewing and exchanging peer feedback regarding their writing assignments? 2.2. Participants The study was conducted at a state vocational high school in Tekirdağ, Turkey in 2016. The participants were 46 Turkish EFL high school students from two different intact classes; 26 in Group A and 20 in Group B. All participants were students in the Information Technology Department who had been using Edmodo actively for two years at the time of data collection. The English proficiency level of the students was A2. The average age of the students was 17 (M = 17.3, SD = 0.59). All participants received a total of four hours of English instruction per week. The participants were asked to write opinion essays on four topics that they decided previously. They submitted an essay on their future plans including their future profession in Assignment 1, advantages and disadvantages of social networks for teenagers in Assignment 2, internet addiction in Assignment 3 and finally they discussed the reasons of going to university in Assignment 4. Each assignment consisted of between 150 – 200 words organized in at least three paragraphs. 2.3. Instruments The data in the present study were collected by means of the following instruments: (a) writing assignments, (b) analytic scoring rubric, and (c) open ended questions. The participants were required to write four opinion essays as writing tasks. Each task consisted of an approximately 150-200-word essay organized in at least three paragraphs. In their assignments, students were encouraged to support their opinions by providing relevant examples. A writing rubric was used by two raters to the score students‟ essays. The same rubric was also used by the students to self-review and exchange peer feedback. The rubric consisted of five sections for organization, content, grammar, vocabulary and format. For each section, the raters labeled it as „excellent‟, „satisfactory‟, „fair‟ or „developing‟ in consideration of the essay. The original rubric was in English; however, in order to avoid any possible miscomprehension due to the student weakness in the target language, the rubric was translated into Turkish by the researchers. Back translation was administered to establish the reliability of the translated instrument. The two raters used the English version of the rubric, whereas student raters used its Turkish version throughout the study. The students were asked open-ended questions in a three-part questionnaire designed to learn their perceptions regarding the implementation. The first part of the questionnaire consisted of demographic information. The eight questions in the second part aimed to reveal participants` perceptions on self- reviewing and exchanging peer anonymous feedback on writing assignments in a digital environment. The final part consisted of a table to reveal the writing performance development of students according to each section of the writing rubric. To investigate participants‟ attitudes to the development of their writing skills, they were asked to refer to each component of the writing rubric and report how they performed in the tasks using the labels „poor‟, „fair‟, „average‟, „good‟ or „excellent‟. 2.4. Data collection procedures The present study benefited from the „Anonymous Multi-mediated Writing Model‟ (Razı, 2016, 2017) to establish the procedure of data collection. Before implementing the study, several training 566 Ayten Kayacan, Salim Razı. / Journal of Language and Linguistic Studies, 13(2) (2017) 561-577 sessions were held. The participants were presented with sample opinion essays considered to be „good‟, „moderate‟ or „weak‟. They were also familiarized with the analytic writing rubric. The sample essays were scored by students using the rubric to enable the students to better understand the descriptions in the rubric. Hence, all students provided feedback on these three sample assignments in the classroom under the guidance of the EFL teacher. Before commencing the four assignments in order to collect data, the students were asked to write an essay for piloting purposes. This pre-test assignment, on one hand, helped the students to practice how they were expected to exchange feedback throughout the study. On the other hand, the scores from the pilot task were also used to establish interrater reliability and set up the groups. All the writing tasks in the present study were scored by two raters, including one of the researchers and the EFL teacher, and the researchers aimed to avoid any possible bias by running Kendall`s Tau test considering the correlation value between the two raters` scores. The results indicated significant positive correlation between the scores of the two raters (p < .001, r = .93). To crosscheck interrater reliability, the correlation between the two raters‟ scores for the four main assignments was checked. The results again indicated significant positive correlation between the two raters‟ scores (p < .001, r = .92.). With regard to these high positive correlation values, it can be concluded that interrater reliability was established. In addition, an independent samples t-test analysis on the pilot study results demonstrated that the two groups (experimental and control groups), in other words, the two intact classes, were similar to each other in terms of the participants‟ proficiency in writing (MGroup A = 59.28, SD = 19.82, MGroup B = 60, SD = 20.30; p > .05). The pilot results were also used to subcategorize the participants in each group as „weak‟, „moderate‟, or „good‟ authors, which would later be used for matching the peers. Although the participants were aware of this subcategorization, they were not informed into which category they were placed. In order to control the feedback exchange process, students exchanged feedback at the departmental computer laboratory throughout the study in both groups within the mainstream teaching. For each feedback session, they were given 40 minutes to complete their tasks. To enable the exchange of peer feedback anonymously on Edmodo, the participants (in Group A for Assignments 1 and 2, in Group B for Assignments 3 and 4) were given pseudonyms. Later, the assignments were submitted online on Edmodo and the participants exchanged peer feedback online using the writing rubric. While exchanging peer feedback, they were expected to highlight mistakes so that their peers could recognize them. The assignments were then submitted on Edmodo and the researchers sent them back to the student-authors. After receiving peer feedback, the student-authors revised their papers accordingly and submitted the final version of their writing task again on Edmodo. Following this process, the researcher sent the assignments to the same peers once more, this time to exchange anonymous peer feedback for the revised versions. The peers repeated the same procedure as they had done earlier. The whole process of exchanging took two weeks for Assignment 1. In the following two weeks, the students were asked to repeat a similar procedure for Assignment 2. The participants in Group B for Assignments 1 and 2 and in Group A for Assignments 3 and 4 also self-reviewed on Edmodo. After submitting their first drafts, they self-reviewed by highlighting their own mistakes and scoring their own assignments. They later revised their papers and resubmitted the final versions. They were also asked to self-review their final versions. Self-reviewing took two weeks for each assignment and the whole experiment for the four assignments took eight weeks, either in Group A or Group B. .Ayten Kayacan, Salim Razı / Journal of Language and Linguistic Studies, 13(2) (2017) 561-577 567 2.5. Data analysis In order to answer RQ1 concerning the total scores of the four written assignments of the self- review and peer feedback groups, descriptive statistics were run. Additionally, a paired-samples t-test was administered to find any differences between the first and final writing task scores of both the self-review and peer feedback groups. To answer RQ2, descriptive statistics were used related to the scores of the first and final versions of the four writings tasks for both groups. Finally, to answer RQ3, dealing with the perceptions of participants on self-reviewing and exchanging peer feedback in a digital environment, descriptive statistics and content analysis were used. 3. Results 3.1. RQ1 In response to RQ1 (effect of self-reviewing and exchanging anonymous peer feedback) concerning the total scores of the four written assignments by the self-review and peer feedback groups, descriptive statistics were utilized. Table 1. Descriptive statistics of self-review and peer feedback groups‟ final score (N = 46) Feedback type Assignment 1 Assignment 2 Assignment 3 Assignment 4 M SD M SD M SD M SD Peer 64.27 9.95 63.12 8.34 69.45 7.43 67.65 9.52 Self 63.85 11.33 62.00 12.71 64.23 8.48 64.50 9.18 Table 1 indicates the peer feedback group‟s superiority over the self-review group on the final versions of the writing task. Although the two groups‟ scores vary slightly from each other in the first two tasks, the difference increases with the third task. Since the data were collected through a counterbalanced design in which the two groups performed the same tasks in reverse order, running inferential statistics at this stage does not reveal which type of feedback contributes more. This will be the concern of the last sub research question. Yet, with regards to the descriptive statistics, it can be concluded that the participants in the peer feedback group scored higher than those of the self-review group. 3.1.1. Self-review gain scores In order to illustrate how self-review contributed to the participants‟ scores in the final version of the four writing tasks, paired-sample t-tests were conducted. This enabled us to compare the scores of the self-review group in the two versions of the four writing tasks. The results demonstrated that there was a significant difference between the scores of the first draft (M = 54.53, SD = 8.64) and the final draft (M = 63.74, SD = 10.21); [t(91) = ‒12.53, p < .001, d = ‒0.97]. A large impact was observed. This makes it clear that the participants in the self-review group performed significantly better in their final versions. To analyze the scores of the first and final versions of the four assignments written by the self- review group, descriptive statistics were also applied. Table 2 shows that the participants` final version scores were almost 8-10 points higher than their first version scores, considering the mean value retrieved from the four written assignments. 568 Ayten Kayacan, Salim Razı. / Journal of Language and Linguistic Studies, 13(2) (2017) 561-577 Table 2. Descriptive statistics for first and final writing task scores of self-review group First Version Final Version M SD M SD Assignment 1 55.35 8.40 64.27 9.95 Assignment 2 52.96 7.90 63.12 8.34 Assignment 3 59.05 10.04 69.45 7.43 Assignment 4 60.60 8.57 67.65 9.58 3.1.2. Peer feedback gain scores To reveal how peer feedback contributed to the participants‟ scores in the final version of the four writing tasks, paired-sample t-tests were again conducted. We were thus able to compare the scores of the peer feedback group in the two versions of the four writing tasks. The results demonstrated a significant difference between the scores of the first drafts (M = 56.62, SD = 9.06) and the final drafts (M = 65.8, SD = 9.14); [t(91) = ‒12.38, p < .001, d = ‒1.01]. The size of the impact makes it clear that the peer feedback group performed better in their final writing assignments. In order to better illustrate the scores of the first and final versions of the four assignments written by the peer feedback group, descriptive statistics were also conducted. Table 3 shows that the participants` final version scores were higher than their first version scores in consideration of the mean value retrieved from the four assignments. It could be noted that the final version scores of the participants were almost 7-10 points higher than their first version scores. Table 3. Descriptive statistics for first and final writing task scores of peer feedback group First Draft Final Draft M SD M SD Assignment 1 53.10 9.51 63.85 11.33 Assignment 2 52.25 8.74 62.00 12.71 Assignment 3 55.46 7.97 64.23 8.48 Assignment 4 5 6 . 4 6 8.43 64.50 9.18 In addition, an independent-sample t-test was conducted to compare the final task scores of the self-review and peer feedback groups. Although the results revealed the superiority of the peer feedback group, they did not indicate any significant difference between the peer feedback group (M = 65.8, SD = 9.16) and self-review group (M = 63.74, SD = 10.21); t(91) = ‒1.45, p > .05. It could be concluded that both self-review and peer feedback contributed to better performance in the final versions. However, neither of these two types of feedback exhibited statistical superiority over the other. 3.2. RQ2 To answer RQ2 (effect of changing type of feedback in the two intact classes), the final version scores of each intact class in the four writing tasks were taken into consideration. As illustrated in Figure 1, exchanging peer feedback enabled learners to perform better in the final version of their writing assignments for each task, regardless of which class they were in. .Ayten Kayacan, Salim Razı / Journal of Language and Linguistic Studies, 13(2) (2017) 561-577 569 Figure 1. Mean difference of groups in terms of final version scores of four assignments. 3.2.1. Replaced feedback in Group A To learn how replacing the type of feedback affected the contribution of feedback in Group A, initially, their mean scores from the first and final versions of peer feedback assignments (Assignments 1 & 2) were taken into consideration regarding the five components in the assessment rubric. Figure 2. Group A peer feedback comparison of first and final scores regarding rubric components. As presented in Figure 2, the scores of the first and final versions of Assignments 1 and 2 demonstrated that the participants in Group A benefited from anonymous peer feedback in all five components of the rubric (organization, content, grammar, vocabulary, format). Participants showed the highest improvement in the „format‟ component of both assignments, whereas the contribution appeared to be lower in the others, especially „grammar‟ and „vocabulary‟. Figure 3. Group A self-review comparison of first and final scores regarding rubric components. 570 Ayten Kayacan, Salim Razı. / Journal of Language and Linguistic Studies, 13(2) (2017) 561-577 Figure 3 illustrates the contribution of self-review to the performance of Group A participants related to the scores of the first and final versions in their Assignments 3 and 4. Similar to the findings presented related to the contribution of peer review, the participants also benefited from self-review, considering all aspects of the rubric. The component of „format‟ is once more highlighted as receiving the greatest improvement, whereas the contribution was limited in the other four components of „content‟, „organization‟, „grammar‟, and „vocabulary‟. Consequently, both self-reviewing and exchanging peer feedback contributed to the writing performance of Group A. It could be inferred that both types of feedback, self-review and peer feedback, made a contribution to the participants‟ writing skills by encouraging them to revise and reorganize their texts. 3.2.2. Replaced feedback in Group B To answer the question of how replacing the type of feedback affected the contribution of feedback in Group B, their mean scores from the first and final versions of the self-review assignments (Assignments 1 & 2) and peer feedback assignments (Assignments 3 & 4) were taken into consideration regarding the five components in the assessment rubric. Figure 4. Group B self-review comparison of first and final scores regarding rubric components. Figure 4 reveals that the participants increased their writing scores in all five components of the rubric in the two assignments which they reviewed themselves. Not unlike the findings in Group A, the highest improvement again appeared in the „format‟ component in both assignments, whereas a lower contribution was observed in the other four components („content‟, „organization‟, „grammar‟, „vocabulary‟). Figure 5. Group B peer feedback comparison of first and final scores regarding rubric components.