RESEARCH PAPERS QUALITY ASSURANCE IN DISTANCE AND OPEN LEARNING By MOHAMMED HASAN MAHAFZAH Faculty of Information Technology, Computer Science Department, Philadelphia University, Amman, Jordan. ABSTRACT E-learning has become an increasingly important teaching and learning mode in educational institutions and corporate training. The evaluation of E-learning, however, is essential for the quality assurance of E-learning courses. This paper constructs a three-phase evaluation model for E-learning courses, which includes development, process, and product evaluation, called the DPP evaluation model. Development evaluation includes, course material design, E- learning platform, course Web site design, learning resource, interactivity, assessment, and tutor support. Process evaluation includes technical support, Web site utilization, learning interaction, learning evaluation, learning support, and flexibility. Product evaluation includes student satisfaction, teaching effectiveness, learning effectiveness, and sustainability. This research uses the DPP model to evaluate a purely E-learning course in Distance and open Learning, developed by Philadelphia University. According to summative evaluation through a student E-learning experience survey, the majority of students were satisfied on all E-learning dimensions of an E-learning course. The majority of students thought that the learning effectiveness of this course was equivalent, even better, than face-to-face learning because of cross-border collaborative learning, sufficient learning support, and learning flexibility. This study shows that a high quality of teaching and learning might be assured by using the systematic DPP evaluation procedure. It is hoped that the DPP evaluation model can provide a benchmark for establishing a wider E-learning quality assurance mechanism in educational institutions. Keywords: E-Learning, Quality Assurance, Evaluation, Moodle, Learning Management System (LMS), E-learning Platform, Pedagogical Model. INTRODUCTION when they considered pursuing a continuing education E-learning has become widely used in conventional program, while 83.9% of them hoped to try the blended education, continuing education, and corporate training learning approach. because of its flexibility, richness, and cost-effectiveness. With the rapid development of e-learning, there is also an United Nations Educational, Scientific and Cultural increasing interest in e-learning research. Among all the Organization (UNESCO) statistics show that over 455 million research topics, quality assurance of e-learning has people around the world received education and training attracted the greatest concern. Jung, Wong, Cheng, through the Internet in 2008, UNESCO Institute for Statistics Baigaltugs and Belawati (2011) found that various national, (2011). Over 70% of universities in the USA were providing E- regional, and international initiatives have been learning courses, and more than 6.1 million university undertaken with regard to quality assurance in E-learning. students were taking at least one e-learning course, which Endean, Bai, and Dui (2007) stated that those concerned accounted for over 31% of the total number of university about online learning have been developing and students in the USA, Allen and Seaman (2011). publishing ideas for over a decade about how to manage With the rapid development of information technology, the quality of the learning experience of those trying to student attitudes towards E-learning are becoming more study through the Internet. positive. According to the Survey on the Demand for However, Jung and Latchem (2007) found that most Continuing Education in Hong Kong 2007-2008, 40.4% of institutions apply the same quality criteria for E-learning as the respondents showed positive attitudes to e-learning for the other modes of delivery. Endean, Bai, and Dui (2010) i-manager’s Journal of Educational Technology, Vol. 9 l No. 3 l October - December 2012 27 RESEARCH PAPERS pointed that new entrants to the e-learning field were development, which is compiled and prepared by the unlikely to have existing internal procedures to cover quality Faculty Council. The Course material design deals with the assurance of this new mode of operation. formation of the course team and its members' roles, course background, course introduction, course Course quality is assured by a series of evaluations, and e- objectives, learner analysis, requirements for learning learning should be no exception. In this paper, the author facilities and skills, course modules/units, learning materials, proposes an e-learning course evaluation model, called assessment and examination, communication and DPP model, for quality assurance and analyzes its concrete collaboration in learning, learner support services, application using an E-learning course developed by teaching model, course materials writing schedule, and Philadelphia University-Jordan. quality assurance. Construction of an E-learning Course Evaluation Model When the course material design "Pedagogical Model" Referring to the real experience in producing e-learning evaluation has been completed, it is followed by learning courses at Philadelphia University, the author proposes a resources, interactivity, assessments, analyzing the system for evaluating E-learning course that consists of construction of the E-learning platform, course website, three evaluation activities: development evaluation, and the training of tutorial staff, which is implemented by process evaluation, and product evaluation; in short, the the deanship of distance learning at the Philadelphia DPP model. Based upon the proposed DPP model and in University. Table 2 lists the e-learning course development line with the components and E-learning characteristics, and evaluation steps. the e-learning evaluation model consists of 17 items as Since this course was developed by the Deanship of shown in Table 1. Distance Learning at Philadelphia University-Jordan, the Development evaluation of E-learning courses involves evaluation of the material design, learning resources, analyzing every component of course development, interactivity, and assessments procedures, followed by the including the course material design, E-learning platform, course development procedures of the Deanship of course website design, learning resource, student-student Distance Learning at Philadelphia University-Jordan. interactions, Assessment, and Tutors. Evaluating of e- Outside experts were invited to do an external evaluation. learning teaching process should include the following six The requirements for external experts were professors in dimensions: technical support, website utilization, learning education with at least 8 years experience in distance interaction, learning evaluation, learning support, and education research and teaching. flexibility. Product evaluation measures the learners' The principle of six types of interactions was also satisfaction, teaching effectiveness, learning effectiveness, emphasized, including the interactions between students and sustainability, which depend on the results of the and interface, between students and teachers, among abovementioned analyses. students, between students and learning content, between Development Evaluation Course content Responsible Organization Evaluation The first step in e-learning development evaluation is to Course material design Deanship of Distance Course development analyze the course material design according to the "Pedagogical Model". Learning team “Faculty Pedagogical Model used in E-learning course Learning resources Philadelphia University Council And Quality Jordan Assurance team”; Interactivity Development Evaluation Process Evaluation Product Evaluation Diagnostic and Formative Distance Learning Assessments Council. Course material design Technical support Student satisfaction E-learning platform Website utilization Teaching effectiveness Using E-Learning platform Avicenna center for E- Avicenna Center for Course website design Learning interaction Learning effectiveness “Moodle” Learning E-learning; Learning resources Learning evaluation Sustainability Course Website Philadelphia University Multimedia designer Interactivity Learning support Jordan and producer; Tutors Training Assessment Flexibility Module external Tutors assessor. Table 1. The DPP Evaluation Model for E-Learning Courses Table 2. E-Learning Course Development and Evaluation 28 i-manager’s Journal of Educational Technology, Vol. 9 l No. 3 l October - December 2012 RESEARCH PAPERS students and learning objectives, between students and students' questions are not answered promptly or only multimedia learning resources, and between students and ambiguous answers are provided, or if there are not many time management, Zhang (2009). posts in the discussion forum area, the distance learning council would take immediate action to contact the tutors The Moodle e-learning platform was used on Learning and solve the problem and inspire student's learning Management System (LMS). The LMS functions could be enthusiasm. classified into five categories: course content functions, communication and collaboration functions, feedback Process evaluation is an accurate process, which involves and evaluation functions, assignment and assessment continuous evaluation throughout the course. Tutors need functions, and administration and management functions. to plan carefully to maintain students' learning enthusiasm and help them achieve the final learning objectives. Process Evaluation Product Evaluation Process evaluation refers to evaluating the process of course delivery, including the technical support, website Product evaluation of an e-learning course is mainly utilization, learning interaction, learner support, and conducted through quantitative research, supplemented flexibility. Process evaluation mainly uses three with students' feedback and suggestions. For the first intake approaches: survey of students' learning experience and of the proposed E-learning course, the online questionnaire feedback; survey of tutors' opinions; and the deanship of method was used and all 80 students were surveyed. Sixty distance learning council's monitoring of the e-learning valid data sets were received; the response rate was 75%. tutorials. Tables 3 to 5 shows the students' evaluation of course The approach to understanding students' learning effectiveness, teaching effectiveness, and learning experience and feedback is as follows: establishing a effectiveness. Table 3 shows the results of overall feedback special feedback area on the course website, establishing on course effectiveness; such feedback is required for all E- email communication between tutors and students, and learning courses at Philadelphia University. The results of internal reviewer's interviews with tutors and students. For students' evaluation of various E-learning components of example, the students were asked to familiarize themselves the course are shown in Tables 4 & 5. with various functions of the e-learning platform in the first In order to understand these results relative to those of the week, referring to the course website guidance. The face-to-face teaching mode, we adopted the evaluation students needed to report to their tutor their degree of statistics labels used for face-to-face teaching in familiarity and time spent for this purpose. It was found that Philadelphia University and calculated the average all the students learnt to use this course platform in three to percentage of each item in the questionnaire survey on a five hours. Likert-type scale, Norman (2010). The scale of the grades is In the middle of this course, the reviewers and the deanship explained as follows: 0% – 49.9%, Fail; 50.0% – 59.9%, of distance learning council conducted formative Pass; 60% – 67.9%, Satisfied; 68% – 75.9%, Good; 76% – evaluation. At the fourth week of this course, individual 83.9%, Very Good; 84% – 100%, Excellent . interviews were conducted and a virtual classroom was Overall Strongly Agree Un Disagree Strongly Average organized for evaluators to gather students' learning Feedback Agree decided Unsatisfied feedback, including overall evaluation, learning The course has 21% 70% 9% 0% 0% 85% been effective in experiences, difficulties, and suggestions so that timely helping me learn. adjustments could be made. The teacher has 48% 46% 6% 0% 0% 8 been effective in Monitoring e-learning tutorials is one of the most important helping me learn. Attending the 80% 18% 2% 0% 0% 93% tasks for the distance learning council, who needs to log course has been worthwhile. into the course website at least once every two weeks to observe students' learning progress and difficulties. If Table 3. Overall Feedback on Course Effectiveness i-manager’s Journal of Educational Technology, Vol. 9 l No. 3 l October - December 2012 29 RESEARCH PAPERS Table 3 shows that students' evaluation of learning Conclusion effectiveness, teaching effectiveness, and course worth The DPP model for evaluating E-learning courses was reached the level of "Excellent" using the same evaluation designed and proposed based upon the real experience statistics method for face-to-face teaching at Philadelphia in producing E-learning courses at Philadelphia University University. evaluation model (i.e., development evaluation, process It can be seen in Tables 4 and 5 that student evaluation of evaluation, and product evaluation.) In line with the degree of satisfaction, teaching products and learning characteristics and process of E-learning teaching and products in e-learning components reached the level of learning, 17 evaluation items were identified within the DPP “Excellent” based on the course evaluation criteria used at model. Using the DPP model, the author took an E-learning Philadelphia University. Table 4 shows that satisfaction with course, which was produced by Philadelphia University as a tutors was highest with an average score of 90.4, but case study to describe and analyze the series of evaluation interactivity was lowest with an average score of 77.8. In this activities. The research results show that this DPP evaluation course, the tutors were required to answer student model could effectively ensure the quality of the E-learning questions within 48 hours and it was very much course in terms of both teaching and learning appreciated by the students. However, there were no effectiveness. However, the use of the DPP model in this requirements for students to respond to other students' study measures only one purely E-learning course and enquiries in discussion forums. Therefore, interaction further studies are needed. The author hope that this model between students was not so active compared with the could be one reference in establishing E-learning quality interaction between tutors and students. assurance models for other educational institutions. Strongly Satisfied Un Un Strongly Average Acknowledgement Satisfied decided satisfiedUnsatisfied The researcher would like to gratefully acknowledge the Web site design 31% 49% 12% 8% 8% 78.6% Video material 59% 41% 0% 0% 0% 89.3% financial support of the Deanship of Scientific Research in Tutors 70% 30% 0% 0% 0% 90.4% Philadelphia University. Interactivity 29% 59% 12% 0% 0% 77.80% E-learning study 38% 57% 5% 0% 0% 80.5% References sessions Flexibility of learning 47% 43% 10% 0% 0% 85.7% [1]. Allen, I.E., & J. Seaman (2011), "Going the distance: Assessment 44% 45% 11% 0% 0% 79.6% Online education in the United States", retrieved from: E-learning 40% 47% 13% 0% 0% 81% environment http://www.onlinelearningsurvey.com/reports/goingthedist Course quality 46% 50% 4% 0% 0% 83% ance.pdf. Table 4. Students' Degree of Satisfaction with the E-learning Course (%) [2]. Endean, M., Bai, B., and Dui, R. (2010). "Quality Strongly Agree Un Disagree Strongly Average standards in online distance education", International Agree decided Disagree Journal of Continuing Education and Lifelong Learning, Teaching pace was 51% 40% 9% 0% 0% 87.7% 3(1). PP 53-73. appropriate video program [3]. Jung, I., & Latchem, C. (2007). "Assuring quality in Asian The lecture was 77% 23% 0% 0% 0% 94.9% knowledgeable open and distance learning", Open Learning, 22 (3). PP about the subject 235-250. The tutor was 660% 24% 10% 0% 0% 88.7% knowledgeable [4]. Jung, I., Wong, T.M., Chen, L. (2011). Baigaltugs, S., and about subject The tutor explains 69% 30% 1% 0% 0% 85.2% Belawati, T.," Quality assurance in Asian distance clearly education: Diverse approaches and common culture". The tutor has 38% 55% 7% 0% 0% 81.6% been effective International Review of Research in Open and Distance in helping me to learn Learning,12(6). PP 63-83. Table 5. Students' Evaluation of Teaching Effectiveness (%) [5]. Norman, Geoff. (2010). “Likert scales, levels of 30 i-manager’s Journal of Educational Technology, Vol. 9 l No. 3 l October - December 2012 RESEARCH PAPERS measurement and the laws of statistics”, Advances in the world”, retrieved from www.uis.unesco.org/Education/ Health Science Education, 15(5). PP 625-632. Documents /ged-2011-en.pdf. [6]. UNESCO Institute for Statistics (2011). “Global [7]. Zhang, W. (2009). "The theory and application of Education Digest, comparing Education statistics across interaction-centered online interaction", Modern Distance Education Research, 2009(5). PP 10–13. ABOUT THE AUTHOR Dr. Mohammed Mahafzah is Associate Professor of Computer Science. Since 2009, he has been the Dean of Distance Learning at Philadelphia University, Jordan. He graduated from the Yarmouk University, Jordan with a BSC in a Computer Science and gained an MSC and a PhD from the George Washington University, USA, both in Computer Science. As an academic, he continued an interest in Computer Algorithms, Data Compression, Parallel Computer Architectures, and Fault-tolerant Computing. He was awarded a Chair in Computer Science by Mu'tah University, Jordan in 1999. In 2003, he was the Dean of Computing for Graduate Studies by Amman Arab University, Jordan, for one year. In 2007, he was the Dean of Information Technology Faculty at Philadelphia University for two years. Since 2005, he has been a Member of Supreme Accreditation council at the Ministry of Higher Education and Scientific Research, Jordan. Since 2010, he has been a Member of trustee's council at Al-Balqa Applied University, Jordan. He has chaired, programme chaired and been a member of the programme committee of many major conferences in different areas in Computer Science and Distance and open learning. He is the author of more than 20 scientific papers in refereed Conferences and Journals. i-manager’s Journal of Educational Technology, Vol. 9 l No. 3 l October - December 2012 31