English Language Teaching; Vol. 10, No. 5; 2017 ISSN 1916-4742 E-ISSN 1916-4750 Published by Canadian Center of Science and Education Developing Model Assesement for Learning (AFL) to Improve Quality and Evaluation in Pragmatic Course in IAIN Surakarta Woro Retnaningsih1, Djatmiko1 & Sumarlam1 1 LinguisticsPragmatics Department, Universitas Negeri Sebelas Maret Surakarta, Indonesia Correspondence: Woro Retnaningsih, LinguisticsPragmatics Department, Universitas Negeri Sebelas Maret Surakarta, Indonesia. E-mail: [email protected] Received: February 9, 2017 Accepted: April 18, 2017 Online Published: April 19, 2017 doi: 10.5539/elt.v10n5p97 URL: http://doi.org/10.5539/elt.v10n5p97 Abstract The research objective is to develop a model of Assessment for Learning (AFL) in Pragmatic course in IAIN Surakarta. The research problems are as follows: How did the lecturer develop a model of AFL? What was the form of assessment information used as the model of AFL? How was the results of the implementation of the model of assessment. The method used in this study is Research, Development and Diffusion. There were three steps activities in this model. The first step, the researcher done the activities included doing the basic scientific inquiry, investigation issues of education, data collection and designing the operational research planning. The second step, the researcher was composing AFL modeling, data validation from the experts and practitioners, compossing readability test; included trial operation models to find solutions to the problems, planning an educational programs, testing, and evaluating the programs. The third step was diffusion, the reseacher informing the target system, demonstrations programs, training to use the target system and program solutions, servicing and maintaining. The population of this study were 150 students from fives classes. From the data analysis shown than the application of AFL model for Pragmatic course could be improved in understanding the materials and English performing. The average score of Pragmatic course was 3.18 from 5 parallel classes, while the average scores of Vocabulary course is 2.40 from 5 parallel classes. The data analyzis shown that AFL method was more suitable to teach English Pragmatic course than English Vocabulary course. Keywords: assessment for learning, quality and evaluation, pracmatics in English teaching learning 1. Introduction 1.1 Research Problems Evaluation was one of a series of activities to enhance the quality, performance or productivity of an organization in carrying out a program. The purpose of evaluation was to see and know the processes that occur in the learning process. As Mardapi (2008) said that through evaluation will obtain nformation about what has been accomplished and what was not accomplisded. There were three terms that are often used in the evaluation: measurement, assessment and evaluation. According to Griffin and Nix (1991) measurements, assessment and evaluation are hierarchical. The criteria of measuremen is to compare the observations, assessment describes and interprets the results of measurement, and to evaluate the determination of the value or behavioral implications. Quality of learning can be seen from the results of the assessment. Atkin (2001) distinguished a judgment based on the goal of formative assessment and summative assessment. The formative assessment has not been done properly because there are three factors. The first is the method of assessment. Teacher-made tests like Midle semester test or Final semester test. The tests have not been designed as a part of the learning process yet. The second factor was the ability of teachers to use assessment to improve the quality of learning is still low, so the feedback is rarely given to student. The third factor was the students lazy to learn and practice the material after teaching learning procces. 1.2 The Importance of Developing Model Assessment for Learning (AFL) Assessment for Learning (AFL) is the assessment that occurs in the classroom and involves students in every aspect of the assessment to build their confidence and maximize achievement. This assessment was suitable for all situations and provides benefits, for faculty and students in identifying next learning steps necessary to make progress (CEA, 2003; Assessment Reform Group, 1999). It was also commented by Stiggins (2002:9) that, in the 97 elt.ccsenet.org English Language Teaching Vol. 10, No. 5; 2017 AFL draft has to provide insight into the learning benefits to students and teachers. The benefits that could be derived for the students, namely (a) they will become confident learners because they were able to see themselves and (b) They were able to understand what the meaning of responsibility for their own learning to monitor greater success in their future (c) they were motivated to learn, and (d) the learning progress could be delivered in accurate information. The global communication and interaction put English as one of the media of comunication and interaction were absolutely necessary. Without adequate in English proficiency, graduates studets from IAIN Surakarta will face many problems in establishing global interaction. Crystal (2000) stated that English is a global language. His statement has a meaning that English is a global language, used by different nations to communicate with other nations around the world. English is a tool to communicate orally and in writing. Understanding English was intended to communicate, to deliver, to accept, to understand, and to express information, thoughts, feelings and developing science, technology, and culture in the English. The statement was supported by the Ministry of Education (2003), that the ability to communicate in a full understanding is the ability of discourse. 1.3 Assessment for Learning in IAIN Surakarta Similarly, in the context of education, the English language serves as a tool to communicate in order to access information, and in the context of everyday life, as a means to foster the relationships, share information and enjoy the aesthetics of language in English culture. Hopefully State Islamic Institute of Surakarta (IAIN) Surakarta given the importance of mastering English to students. This was due to English proficiency; the student will open insight and knowledge internationally. Therefore, teachers need to devise to innovative the methods of teaching in order to explore, understand, explain, and describe English as well as explore innovative methods. One of the innovative methods that can be done to address the problem of English teaching learning was by using the model of Assessment for Learning (AFL). AFL was a model of assessment that could clearly reveal the student mastery of the English concept step by step. Based on the explanation above, the researcher saw that the basic concepts and principles applied to the AFL match in English teaching learning process. The English concept of assessment and orientation on the process rather than the results were expected that students have motivation, growing awareness, building their confidence and responsibility to improve and enhance the quality of English language learning in the classroom. 1.4 Hypotheses Based on the background of the above issues, the following research problems are: How did the lecturer develop the model of AFL in English teaching learning in State Islamic Institute of Surakarta (IAIN) Surakarta? How did the forms of assessment used by teacher in implementing the AFL model in English teaching learning in IAIN Surakarta? Was the model of AFL can serve to enhance understanding, behavior, and the ability of students in English lessons at IAIN Surakarta? How did the report of the assessment by using the model AFL in English teaching learning in IAIN Surakarta? 2. Method The method used in this study is research, development and diffusion used by Hopkins & Clark (Havelock, 1976). There were three steps activities in this model. The first step, the researcher done the activities included doing the basic scientific inquiry, investigation issues of education, data collection and designing the operational research planning. The second step, the researcher was composing AFL modeling, data validation from the experts and practitioners, compossing readability test; included trial operation models to find solutions to the problems, planning an educational programs, testing, and evaluating the programs. The third step was diffusion, the reseacher informing the target system, demonstrations programs, training to use the target system and program solutions, servicing and maintainingthe application. Methods used by Hopkinsand Clark can be seen at the following figure: 98 elt.ccsenet.org English Language Teaching Vol. 10, No. 5; 2017 Figure 1. Development and diffusion research method by Hopkins & Clark Based on the figure above, the developed method used by Hopkins & Clark has some advantages: (a) especially developed in education, (b) the set data in the planning and problem solution were obtained through the two stages of research, so that the information is very accurate. At this stage of development, the activities carried out: designed AFL modeling, seeked validation from the experts and practitioners, composing readability test, training the teachers, construting test models, and giving evaluation. The obervation of student behavior designed to examine the students’ behavior during the teaching learning process by using AFL model. Students’ observation sheet designed to observe students’ behavior at the beginning of the class, during the teaching learning process, and in the assessment. Students’ observation sheet would be hold by the teacher. 2.1 Participant (Subject) Characteristics The subject of this study was the student of English class in the academic year of 2015/2016. Students need language proficiency in accordance with inter-cultural context as well as in their own culture. The reason of choosing Vocabulary classes that, students aware more things to be studied continuously since learning concepts derived from foreign languages that sometimes are not equivalent in the language and culture of indigenous students. In the development of data collection instruments consisted of two phases of each meeting. They are behavioral observation checklist and student assessment sheet. The approach used to find the parameters of the ability is Graded Response Model (GRM). 2.2 Procedures Firts of all the lecturer given the student’ opportunity to assess and adjust themselves in teaching strategies to get complite comphehenshion. AFL given emphasis on the process undertaken by the faculty to seek information related to the problems faced by students. It requires assessment instruments that can clearly reveal the problems faced by the students to obtain such information, as shown at the figure below: Figure 2. Assesement instruments In this regard lecturer suggested by AFL cycle of self-esteem. Students are able to learn, face new challenges in a 99 elt.ccsenet.org English Language Teaching Vol. 10, No. 5; 2017 state of preparedness, a statement of the maximum to take risks and learn. The AFL’s goal written by CEA (2003: 2) are: (a) give insight into student learning to faculty and students, (b) increase success for all, (c) assist in the establishment of learning goals, (d) allowing continuous reflection on what students know today and what they need to know next, (e) measure what is valued, and (f) improve the standards obtained by the students on adages of capability. 2.2.1Sample Size, Power, and Precision The respondents were 10 classes with five classes for Pragmatic and five classes for Vocabulary courses. The reasons for choosing the Pragmatics course because pragmatic studys have considered a number of rather different delimitations of the field (of pragmatics. The most promising are the definitions that equate pragmatics with 'meaning minus semantics,' or with a theory of language understanding that takes context into account, in order to complement the contribution that semantics makes to meaning. They are not, however, without their difficulties, as we have noted. To some extent, other conceptions of pragmatics may ultimately be consistent with these. The definition of pragmatics as concerned with encoded aspects of context may be less restrictive than it seems at first sight; for if in general (a) principles of language usage have as corollaries principles of interpretation, and (b) principles of language usage are likely in the long run to impinge on grammar (and some empirical support can be found for both propositions), then theories about pragmatic aspects of meaning will be closely related to theories about the grammaticalization of aspects of context. So the multiplicity of alternative definitions may well seem greater than it really is (Levinson, 1983). The Vocabulary class was chossed because studying vocabulary is also concerning with maning. Meaning was studied at semantics. Both Pragmatics and Semantics deal with meaning, so there was an intuitive sense in which the two fields are closely related. There is also an intuitive sense in which the two are distinct: Most people feel they have an understanding of the 'literal' meaning of a word or sentence as opposed to what it might be used to convey in a certain context. Upon trying to disentangle these two types of meaning from each other, however, things get considerably more difficult" (Birner, 2012). The inter-rater agreement (inter-rater reliability) of the self-assessment sheet instruments, they are sheet behavioral observations, questionnaires effectiveness model, the application of models in the classroom, and the results of the model validation assessment sheet. 2.2.2 Measures and Covariates Observations activities conducted in three phases, namely: introduction, core activities, and clossing. This sheet comes with the criteria for assessing the activity of lecturers. In this study, the researchers adopted the criteria of Mansur (2009: 108) as follows: Introduction: opening, delivering basic competence, delivering evaluation, indicators of achievement, reminds students on previous learning, and motivating students. Core Activities: presenting the subject matter along with an example, charge sheet student behavioral observations, giving students opportunity to ask questions, sharing the stage assignment - 1, observe and record student behavior during the task phase - 1, distribute self-assessment sheet, checking the work of the stage - 1, restoring the work phase - 1, give students a chance to express the problems encountered during the study, shows the work of one of the students who fit the criteria, give feedback in the form of comments, suggestions for improvement are generally based on the work of the task phase - 1, and student self-assessment. Closing: providing task stage - 2 to be completed at home, and closing the lesson. The researchers adopted the scooring criteria for this purpose from Anwar (2005) as the following table: Tabel 1. Scooring criteria Numbers Interval Mean Score Specification 4 3.25 < score ≤ 4.0 VeryGood 3 2.5 <score ≤ 5.25 Good 2 1.75 <score ≤ 2.50 Fair 1 0.00 ≤ score ≤ 1.75 Poor Self-assssmentwas done by usingtraffic light, which is atechnique that provides sign colors (green ="get", yellow="know in part", red ="do not know"). 2.2.3 Research Design The model of AFL consists of several components, namely: (a) Observation consists of observations of students’ 100 elt.ccsenet.org English Language Teaching Vol. 10, No. 5; 2017 behavior: at the beginning and during the teaching learning proses. Observations made by a lecturer in English that is used exclusively during the learning process. (b) Task Two Phase, In phase-1 (stage-1) the teacher gave the package to students about 10-20 minutes to complete. The results of the work carried out as a stage-1 the basis in providing the questions on the stage-2 to be completed at home. (c) Self- assessment provided the students assessed themselves during the learning process by using the strategy of traffic light (TL). (d) Feedback. Besides developing the instruments mentioned above, also the assessment criteria were developed (PK), and scoring rubric (RP). The components described above can be seen at the following figure: Figure 3. The model of AFL Based on the explaining above, it appears that the effectiveness of a judgment must meet the three main criteria, namely: valid, systematic, and practical. Nitko (2007: 47) gave the opinion that the effectiveness of educational assessment must meet four main criteria, namely cost, efficiency, practicality, and instructional features. In connection with the opinion of Nitko &Brookhart, an indicator that an effective evaluation model if: (a) the model accommodate the characteristics of different students, (b) evaluation models easily implemented by teachers, (c) evaluation models provide rapid assessment results to guide instruction: assessment results and uncover meaningful individual differences, assessment results to identify student misconceptions and assessment model more efficient. 3. Results The data presented in this paper was the data obtained after the research stage of development. The indicators of Student Behavior Assessment in Vocabulary (V) and Pragmatic (P) are as follows: 1). There are no students listened intently to the teacher's explanation, among others: Looking forward to the board or to the teacher (V: 30%/P: 80%), Drowsy (V: 33%/P: 0%), Shaking (V: 30%/P: 1%), Frowning (V: 45%/P: 1.2%), Smile (V: 55%/P: 99%) 2). Taking note ofthe important things that teacher’s described, Noting what is written or drawn on the board (V: 30%/P: 80%), Recording oral explanation of teachers that are not written on the board, (V: 30%/M: 80%) and A note of the answers given to the student and teacher (V: 30%/M: 80%) 3). Asking the teacher: This question is particularly informative (V: 30%/P: 90%), analytical questions (V: 30%/P: 80%), a significant question (V: 60%/P: 80%), 4). Answering the teacher's question: complete response (V: 30%/P: 85%), response according to what is asked (V: 30%/P: 80%) 5). Expressing opinions: connection to the material (V: 30%/P: 80%), Proposals (V: 23%/P: 87%), 6). Concentration: Sitting politely (V: 78%/P: 98%), not interfere friend (V: 30%/P: 86%), Working quietly (V: 30%/P: 84%), Discipline (V: 38%/P: 89%). The effectiveness model AFS are for Vocabulary and Pragmatics are: a) Instrument model can be used to assess the attitudes, experiences, skills and attitudes in the English language learning (Validity) (V1/P4). 101 elt.ccsenet.org English Language Teaching Vol. 10, No. 5; 2017 b) Instrument model ifused to assess the attitudes, experience, language skills and behavior in learning will repeatedly give the same results (Reliability) (V1/P4). c) This model canbe used to obtain informationas it is about the attitudes, experiences, skills and attitudes in learning English (Objective) (V4/P4). d) This model isused in a systematic and continuous learning in every classroom (Systematic) (V4/P4). e) This model is practically used to collect information relating to the implementation of learning and teaching classroom (Practical) (V2/P4). 4. Discussion AFL is an assessment model used to assess students' progress so that teachers can modify learning to the real needs of students. The development of this research includes the development of a model and instrument development model. Development of the model is to obtain the effectiveness, practicality, and applicability model. The developing instrument is to prove the validity and reliability. AFL-model development is done by using adjustment R, D, & D model of Hopkins and Clark, which in practice is limited to the development phase. The validation activities carried out are the legibility testing and limited testing. The implementation ofAFL’s Model in Vocabulary and Pragmatics Class, in early learning activities stated clearly (V4/P4). During the learning activities stated clearly (V4/P4). Activity feedback clearly stated (V4/P4), and the activities at the end of the lesson clearly stated (V4/P4). The practicability of model AFL in Vocabulary and Pragmatics classes are both of them complete and useable those are in Vocabulary achievement 36.42% and 71% in Pragmatics. The Effectiveness of AFL Model with criteria 1, 2, 3, 4 for Vocabulary are 2 (less effective) and for Pragmatics 4 (very good/effective). The implementation and evaluation for Vocabulary are 2 is pretty good and for Pragmatics with score 4 is very good. Student’s average-performing scores are 2.4 for Vocabulary and 3.18 for Pragmatics. Acknowledgments The conclusion of this research is: Model AFL on learning English in Higher Education developed by the method of research and development. The stages include pre-survey research, problem analysis, analysis of curriculum, research studies, expert consultations, preparation of prototype models. The stages of development include, limited testing, and testing was expanded to fit the model theoretically and empirically. The information displayed when using model-AFL in learning is accurate and in accordance with the real needs of students in terms of student understanding of the learning materials, student behavior during learning, and English language skills of students. It can be used of the information model-AFL assessment through feedback and reflection. The application of model-AFL Pragmatic learning could improve: (a) understanding, (b) behavior, and (c) the ability to score an average of 3:18 of 5 parallel classes. While learning Vocabulary 3 is an average score of 2.4 out of 5 parallel classes, it cannot improve: (a) understanding, (b) behavior, and (c) the ability of student vocabulary. Student learning abilities displayed by individual profiles and classroom profiles. Both of these profiles show an understanding of the development trend of the individual (or class) of the learning materials, student behavior during learning, and the students' English language abilities for each meeting. References Anwar. (2005). Evaluasi Kinerja Belajar. Jakarta Refika: Aditama. Atkin, J. M., Black, P., & Cofey, J. (2001). Classroom Assessment and the National Science Education Standards. New York: National Academy Press. Birner, B. J. (2012). Introduction to Pragmatics. Wiley:Blackwell CEA. (2003). Quality Statement on Assessment Practice (Second). Retrieved 5 December, 2010, from http://www.aaia.org.uk Crystal, D. (2000). English as a Global Language. Cambridge: Cambridge University Press. Field, A. (2000). Discovering Statistics Using SPSS for Windows Advanced Techniques for The Beginner. London: SAGE Publications. Griffin, P., & Nix, P. (1991). Educational Assessment And Reporting. Sidney: Harcout Brace Javanovich Publisher. 102 elt.ccsenet.org English Language Teaching Vol. 10, No. 5; 2017 Group, Assesment Reform. (1999). Assessment for Learning: Beyond the Black Box. University of Cambridge School of Education. Retrieved 1th December, 2010, from http://www.Assesment-reform -group.org.uk Havelock, R. G. (1973). The Change Agent’s Guide to Innovation in Education. Englewood, NJ: Education Technology Publication. Krathwohl, D. R. (1998). Methods of educational &social science: An Integrated Approach. New York: Longman. Levinson, S. C. (1983). Pragmatics. Cambridge: University Press. Mansur. (2009). PembelajaranBahasaInggris.Yogyakart: UNY Press. Mardapi, D. (2008). Teknik Penyusunan Instrumen Tes dan Nontes. Yogyakarta: Mitra Cendikia Press. Nasional, Departemen Pendidikan. (2003). Kurikulum 2004, Stándar Kompetensi, Mata Pelajara: Bahasa Inggris Sekolah Menengah Atas. Jakarta: Pusat Kurikulum, Balitbang Depdiknas. Nitko, A. J., & Brookhart, S. M. (2007). Assessment and Grading in Classrooms. New York: Pearson. Stiggins, R. J. (1994). Student-centered Classroom Assessment. India: Phi Delta Kappan. Stiggins, R. J. (2002). Assessment Eurasia: The Absence of Assessment for Learning. India: Phi Delta Kappan. Copyrights Copyright for this article is retained by the author(s), with first publication rights granted to the journal. This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/). 103