International Education Studies; Vol. 9, No. 2; 2016 ISSN 1913-9020 E-ISSN 1913-9039 Published by Canadian Center of Science and Education An Analysis of Grades, Class Level and Faculty Evaluation Scores in the United Arab Emirates Lee Waller1 1 School of Arts & Sciences: American University of Ras Al Khaimah, United Arab Emirates Correspondence: Lee Waller, American University of Ras Al Khaimah, PO Box 10021, Ras Al Khaimah, UAE. Tel: 971-56-156-5977. E-mail: [email protected] Received: August 6, 2015 Accepted: September 11, 2015 Online Published: January 25, 2016 doi:10.5539/ies.v9n2p66 URL: http://dx.doi.org/10.5539/ies.v9n2p66 Abstract This study examined the results of a student evaluation of faculty against the grades awarded and the level of the course for a higher education institution in the United Arab Emirates. The purpose of the study was to determine if the grades awarded in the course and/or level of the course impacted the evaluation scores awarded to the faculty member. The study utilized a 25-question student perception survey coupling course results with the overall course grade point average (GPA) and the course level. All courses were undergraduate. Descriptives for the responses were obtained prior to conducting a factor analysis for the purposes of dimension reduction. The analysis included 184 course pairings. The data set was examined to verify satisfaction of assumptions appropriate for factor analysis. Reliability analysis yielded a Chronbach’s alpha of 0.974. The factor analysis identified three underlying factors accounting for 80.07% of the variance. These three factors were identified as (1) overall perception of instruction, (2) the relationship of the grade and course level and (3) course management. Results of the study did not indicate that the grades given in a class nor the level of the course significantly affected the evaluations provided by the students. Grades and the level of the course were found to align. Student achievement in the course was also found to relate to the student’s perception of fair treatment by the faculty member. Keywords: student course evaluation, instructor evaluation, grade impact, course-level impact, United Arab Emirates 1. Introduction Instructional evaluation by students continues to be a hotbed of concern for many faculty members and their respective institutions. Sojka, Gupta, and Detter-Schmetz (2002) reference the long-standing, love-hate relationship between higher education and student course evaluations of instruction beginning in the early 1930’s and continuing into the present day. These researchers point out that many issues are perceived as affecting student ratings of the instructional effectiveness of faculty. These issues range from the physical and demographic characteristics of faculty members and students to grade leniency and other issues traditionally associated with the rigor of courses. A few years earlier, Greenwald and Gillmore (1997) found that higher grades led to higher ratings of faculty. These researchers also determined that faculty teaching courses deemed as more demanding were likely to receive lower student evaluations than faculty teaching courses deemed as less demanding. The more demanding courses could be said to occur in the junior and senior levels of baccalaureate programs where higher levels of specialization are expected. 2. Review of the Literature Sojka, Gupta, and Detter-Schmetz (2002) found that students and faculty generally concur on some issues relating to student evaluations of instruction while disagreeing on other issues. Both parties were found to agree that the evaluation processes needed to be more specific as opposed to the more nebulous style of questioning. The acceptance of faculty peer evaluations as a means for instructional improvement was also a common ground. The importance of student evaluations as a means of assessing faculty performance proved a major area of disagreement. Students viewed the evaluations with a much greater relevance than did faculty. Students also expected feedback relating to the student evaluations while faculty did not deem student feedback to be of importance. Faculty viewed the process as one of many components relating to any holistic evaluation of instructional expertise. In spite of these differences, the researchers found that neither students nor faculty 66 www.ccsenet.org/ies International Education Studies Vol. 9, No. 2; 2016 desired the elimination of student evaluations of instruction. Hatfield and Coyle (2012) examined the likelihood that students would complete a course evaluation of a faculty’s instruction. Their research indicated that course grades did not influence the likelihood of completing a faculty evaluation. Students with high and low grades were equally likely to complete the course evaluation. Findings did indicate that a student’s gender, age and ethnicity were correlated to the likelihood for completing a student evaluation of faculty. Female students born prior to 1987 were more likely to participate in the evaluation process. Younger males were less likely. Asian students were the only ethnicity found to be more likely to complete student evaluations of faculty. Their findings also indicate that the type and level of the course taken influenced the likelihood of participation in the evaluation process. While the focus of these findings does not address the impact of grades or the level of the course on the actual evaluation scores, the findings did indicate that grades do not influence the likelihood of completing the student evaluation of instruction while the level of the course does influence the completion of the evaluations. These researchers appear to concur with Greenwald and Gillmore (1997) relative to impact of student course evaluations though from opposite ends of the spectrum. Greenwald and Gillmore assert that the difficulty of the course negatively impacts student evaluation scores while Hatfield and Coyle found that the level of the course played a role in the likelihood that a student would complete the evaluation. Though the debate over the structure and usability of student course evaluations of instruction continues, some researchers assert the usefulness of these evaluations as a means of transforming teaching and learning. Bubb et al. (2013) strongly assert the usability of formative student input with regard to instructional improvement. While these researchers point out the many and various difficulties associated with low response rates as institutions transition to the electronic collection of course evaluation results, they demonstrate the usefulness of a formative mid-course evaluation by students. This research builds on the prior work of Stark-Wroblewski, Ahlering, and Brill (2007). Both groups of researchers concur regarding the potential usefulness of student evaluations of faculty instruction. Review of the available literature supports the purpose and intent of this study. Student evaluation of instruction is often perceived as being impacted by course grades and the level of the course in question. Though the usefulness of the student evaluations to impact teaching and learning is not examined in this study, the impact of course grades and/or course level on student evaluation scores clearly holds the potential to empower practice and guide future research. 3. Purpose of the Study The study examined the relationship of undergraduate course grades (GPA) and course level with 25 student evaluation scores of instruction for 184 individual courses in a fall semester of study. The student course evaluation posted a 68.5% response rate. The purpose of the study was to determine if a relationship existed between or among the course grade point average (GPA), course level (freshman, sophomore, junior, senior) and the 25 student evaluation responses. 4. Research Methodology 4.1 Research Questions and Hypotheses Two research questions guided the study. These research questions follow. 1) What are the course grade GPA’s, course levels, and student course evaluation responses in the fall 2014 semester for an institution of higher education in the United Arab Emirates? 2) Do relationships exist between or among course GPA, course level and student course evaluation responses in the fall 2014 semester for an institution of higher education in the United Arab Emirates? The following null and alternate hypotheses were utilized to guide this study and support the analysis required by the study’s research question 2. Ho No relationships exist between or among course GPA, course level and student course evaluation responses in the fall 2014 semester for an institution of higher education in the United Arab Emirates. Ha Relationships exist between or among course GPA, course level and student course evaluation responses in the fall 2014 semester for an institution of higher education in the United Arab Emirates. 4.2 Research Approach The study utilized a quantitative research methodology. The data set was collected and developed by pairing the overall course GPA and course level with the cumulative evaluation scores for 184 courses. The course GPA was established by averaging all course grades. Course grades were assigned the traditional range of 4 for an A, 3 for 67 www.ccsenet.org/ies International Education Studies Vol. 9, No. 2; 2016 a B, 2 for a C, 1 for a D and 0 for an F. The course level was assigned a score of 4 for a senior level course, 3 for a junior level course, 2 for a sophomore course, and 1 for a freshman course. The course GPA and course level were then paired with the student evaluation scores for the course developed using a 5-point Likert scale. A Chronbach alpha was utilized to establish the reliability of the data set. Descriptives were then identified to answer Research Question 1. The Bartlett’s Test of Sphericity was utilized to establish the existence of a relationship between or among the numerous variables in answer to Research Question 2. The four assumptions required for factor analysis were then examined. Afterwards dimension reduction using factor analysis was conducted to determine and examine underlying relationships within the data set. Factors with Eigenvalues greater than 1.0 were deemed significant. Lumadue and Waller (2013) established four assumptions required to conduct factor analysis. These assumptions follow. 1) The data set must not include outliers as these can substantially impact the analysis. 2) The sample size must be adequate. The number of cases must be greater than the number of factors. 3) The model must be linear in nature. 4) Interval data are assumed. Examination of the data set indicated the presence of no outliers. The sample size included 27 variables that far exceeded the number of identified factors thus satisfying the expectations regarding sample size. Additionally, a Kaiser-Meyer-Olkin measure of sampling adequacy indicated a score of 0.950 further establishing the adequacy of the sample size for purposes of factor analysis. The linear nature of the data set was established in response to Research Question 2 via a Bartlett’s Test of Sphericity. The data set met the expectation requiring that the data be at least interval in nature. 4.3 Limitations, Delimitations and Assumptions This study was limited by the following factors. 1) Data were available for the fall 2014 semester only. 2) Data were available only for the institution under examination. The researcher delimited the study as follows. 1) The study was restricted to data retrieved utilizing the survey instrument, grade analysis and determination of grade level. 2) The collection of information was delimited to the fall 2014 semester for the institution under examination. 3) The research examined only the course GPA, course level and course summary scores to the faculty evaluation instrument. The following assumptions were made for this study. 1) The data were accurate and correctly recorded. 2) The data were usable and appropriate for this study. 3) Examination of the data held the potential to impact future practice and research. 5. Findings and Analysis Findings of the study are delineated into responses regarding two research questions. The responses follow. 5.1 Research Question 1 Research question 1 required the analysis of descriptive data for the variables associated with the study. The findings are provided in Table 1. 68 www.ccsenet.org/ies International Education Studies Vol. 9, No. 2; 2016 Table 1. Descriptive analysis of the study variables (continued on subsequent page) Questions N Mean Std. Dev. 1. Course GPA 184 2.41 0.693 2. Course Level 184 2.23 1.308 3. Was prepared for class. 184 4.59 0.466 4. Explained the material clearly. 184 4.67 0.549 5. Corrects assignments in a timely manner. 184 4.42 0.539 6. Comments and suggestions were helpful. 184 4.44 0.526 7. Encouraged student/teacher interaction. 184 4.48 0.548 8. Encouraged questions and class preparation. 184 4.50 0.525 9. Encouraged all students to participate. 184 4.45 0.632 10. Treated students impartially & with respect. 184 4.54 0.576 11. Was available during office hours. 184 4.51 0.464 12. Covered content as outlined. 184 4.50 0.526 13. Encouraged active involvement. 184 4.47 0.563 14. Used different learning activities. 184 4.27 0.672 15. Provides examples to explain concepts. 184 4.50 0.556 16. Sticks to the subject matter. 184 4.45 0.545 17. The course was well organized. 184 4.45 0.562 18. Course description was accurate. 184 4.47 0.539 19. Course requirements were clear. 184 4.46 0.505 20. Learning outcomes were achieved. 184 4.42 0.578 21. Grading criteria was clear. 184 4.39 0.593 22. Assignments helped master the course. 184 4.46 0.588 23. Integrated web materials into the course. 184 4.31 0.635 24. Encouraged access to external information. 184 4.44 0.574 25. Utilized materials outside the textbook. 184 4.27 0.648 26. Overall rating of the course. 184 4.33 0.620 27. Overall rating of the teaching. 184 4.39 0.696 The average course GPA was 2.41. This falls squarely in the mid-C range with 2.0 constituting the threshold for a grade of C and 3.0 constituting the threshold for a grade of B. The average course level was 2.23. This value places the average level of the courses at just above second year. This means that freshman and sophomore courses were a more plentiful than junior and senior courses. The responses to questions relating to the student evaluation of instruction ranged from a high of 4.67 relating to the clear explanation of the materials to a low of 4.27 relating to the utilization of different learning materials and materials from outside the textbook. The scores may be said to indicate a classroom with extensive focus on clear explanations with limited utilization of materials from sources other than the textbook. Overall, the course grading might also said to be in the rigorous range with an average grade of C. 5.2 Research Question 2 Research question 2 examined the possibility of relationships existing between or among the various variables. The Bartlett’s Test of Sphericity was utilized to determine if these relationships existed and was used to establish the linear nature of the data set. This test returned an approximate Chi-Squared value of 7,494 with a significance of less than 0.001. The null hypothesis was subsequently rejected in favor of the alternate hypothesis. Relationships were found to exist between or among course GPA, course level and student course 69 www.ccsennet.org/ies Internationnal Education Stuudies Vol. 9, No. 2;2016 evaluationn responses in the fall semeester for an innstitution of hiigher educatioon in the Unitted Arab Emirrates. Factor anaalysis was thenn conducted too determine thee nature of thee identified relaationships. Finndings are inclluded in Table 2 as follows. Table 2. Tootal variance eexplained Initiall Eigenvalues Extraction Suums of Squaredd Loadings Commponent Tootal % of Variance Cummulative % TTotal % of VVariance Cuumulative % 1 18..776 69.5542 669.542 18.776 699.542 69.542 2 1.6612 5.9969 775.511 11.612 55.969 75.511 3 1.2241 4.5596 880.107 11.241 44.596 80.107 Extractionn Method: Prinncipal Componnent Analysis. Three facttors were deemmed to be siggnificant as inddicated in Tabble 2. These thhree factors exxplained a tottal of 80.107% oof the variancee in the data sett. These factorrs are displayedd in the Scree Plot provided in Figure 1. Figurre 1. Scree plott The factorr loadings for tthe three identiified factors arre provided in Table 3. Table 3. Component (facctor loadings) mmatrix Questioons 1 22 3 1. Course GPA .294 .7442 .417 2. Course Level .276 .8005 .297 3. Was pprepared for class. .849 -.2201 .236 4. Explaained the mateerial clearly. .867 -.142 .133 5. Correects assignmennts in a timely manner. .677 -.104 .291 6. Commments and sugggestions were helpful. .744 -.0062 .078 7. Encouraged studentt/teacher interaaction. .908 .0884 -.223 8. Encouraged questioons and class ppreparation. .873 .1225 -.2877 9. Encouraged all studdents to particiipate. .873 .1223 -.335 10. Treaated students immpartially & wwith respect. .789 .2330 -.4699 11. Wass available durring office houurs. .780 .1661 -.035 70 www.ccsenet.org/ies International Education Studies Vol. 9, No. 2; 2016 12. Covered content as outlined. .861 -.152 .256 13. Encouraged active involvement. .909 .063 -.121 14. Used different learning activities. .872 .015 .079 15. Provides examples to explain concepts. .908 -.089 .097 16. Sticks to the subject matter. .884 -.106 .020 17. The course was well organized. .905 -.170 .201 18. Course description was accurate. .924 -.124 .169 19. Course requirements were clear. .919 -.081 .027 20. Learning outcomes were achieved. .920 -.132 .112 21. Grading criteria was clear. .891 -.010 -.048 22. Assignments helped master the course. .851 -.106 .158 23. Integrated web materials into the course. .825 .135 -.067 24. Encouraged access to external information. .828 .207 -.300 25. Utilized materials outside the textbook. .846 .101 -.207 26. Overall rating of the course. .899 -.123 .057 27. Overall rating of the teaching. .921 -.098 -.052 Extraction Model: Principle. The three factors were identified and named via examination of the factor loadings provided in Table 3. These factors are (1) overall perception of instruction, (2) the relationship of the grade and course level and (3) course management. Factor 1 included a response to every question on the student evaluation of instruction. Factor 2 included the two issues of course GPA and course level. Factor 3 included the course GPA and the student’s perception of being respectfully treated. 6. Conclusions and Implications Examination of the factor loadings for factor 1 indicates the existence of a global perception of the faculty member’s instructional expertise. The existence of a global perception implies that student’s formulated an overall impression of the faculty member’s instructional expertise and allowed this perception to guide responses for the student evaluation of instruction. The faculty member was evaluated within the bounds of this holistic vision rather than on the basis of the individual questions relating to the varied instructional issues. Accordingly, the student evaluation of faculty should be viewed as one overall assessment of instructional ability. The non-inclusion of the course GPA and level in this holistic perception demonstrates that grade and course level did not play a significant role in shaping the responses to the evaluation instrument. Nor did these factors shape the student’s holistic perception of the faculty member’s instructional expertise. More succinctly, neither grades nor the course level shaped the student’s evaluation of instruction. The perception espoused by Greenwald and Gillmore (1997) that high grades led to high evaluations is not born out by factor 1, nor is the impression that differing levels of courses are more likely to receive differing evaluations. Simply put, students in this case evaluated the faculty member’s instructional ability based on a holistic perception of the faculty member’s instructional expertise. Factor 2 included the course GPA and course level only. This indicates that GPA and grade level are related. Course GPA and course level have a high positive correlation, Pearson r, of 0.644. This factor implies that students improve their academic performance as they continue through the educational process to attain a degree. However, the findings indicate that neither the course GPA nor progress through the educational system affected the student’s evaluation of the faculty member’s instructional expertise. The third factor provided an interesting relationship between the course GPA and the student’s perception that he or she has been treated with respect. While the loadings for factor 3 indicate a reciprocal effect for these two variables in constituting the identified factor, further examination found that the two variables have a moderate positive correlation, Pearson r, of 0.236. This factor indicated that students are more likely to earn higher grades in an environment in which they perceive themselves as recipients of respectful treatment. 71 www.ccsenet.org/ies International Education Studies Vol. 9, No. 2; 2016 This study found that a faculty member’s course evaluations are based on the student’s holistic perception of the faculty member’s instructional expertise. This holistic perception guides responses on the course evaluation instrument. Individual question responses rise or fall based on this holistic perception. Course grades and course level are not part of the holistic perception of teaching ability and do not significantly impact responses on the course evaluation instrument. Course grades and course level are positively associated with each other. As student progress in their fields of study, they are more likely to earn higher grades. Faculty members should also recognize that student performance increases when students perceive that they are treated respectfully. Findings of the study align with observations of Bartels, Bommer, and Rubin (2000). These researchers indicated the weaknesses of traditional student evaluation of instruction methodologies. This study indicated that the student evaluation of instruction was guided by a major perception of the instructional ability of the faculty under evaluation. The implications are far-reaching. Alternate methodologies for evaluating instruction and aligning student perception are required. Further research should be conducted to compare the effectiveness these of alternate methodologies. Care should be taken in generalizing this study to a broader context as the study was conducted for one semester for one institution in the United Arab Emirates. However, the finding that course GPA and course level did not impact the student evaluation of faculty is of interest and directly contradicts early studies in the United States. Perhaps a culture difference has been identified. Obviously further research is warranted. References Bartels, L., Bommer, W., & Rubin, R. (2000). Student: Performance: Assessment centers versus traditional classroom evaluation techniques. Journal for Business Education, 75(4), 198-201, http://dx.doi.org/10.1080/08832320009599014 Bubb, D., Schraw, G., James, D., Brents, B., Kaalberg, K., Marchand, G., . . . Cammett, A. (2013). Making the case for formative assessment: How it improves student engagement and faculty summative course evaluations. Assessment Update, 25(3), 8-12. Greenwald, A. G., & Gillmore, G. M. (1997). Grading leniency is a removable contaminant of student ratings. American Psychologist, 52, 1209-1217. http://dx.doi.org/10.1037/0003-066X.52.11.1209 Hatfield, C., & Coyle, E. (2012). Factors that influence student completion of course and faculty evaluations. American Journal of Pharmaceutical Education, 77(2). Sojka, J., Gupta, A., & Deeter-Schmelz, D. (2002). Student and faculty perceptions of student evaluations of teaching: A study of similarities and differences. College Teaching, 50(2), 1-6. http://dx.doi.org/10.1080/87567550209595873 Stark-Wrobelewski, K., Ahlering, R. F., & Brill, F. M. (2007). Toward a more comprehensive approach to evaluating teaching effectiveness: Supplementing student evaluations of teaching with pre-post learning measures. Assessment & Evaluation in Higher Education, 32(4), 403-415, http://dx.doi.org/10.1080/02602930600898536 Waller, L., & Lumadue, R. (2013). Factor Analysis (1st ed.). Cupertino, CA: Apple iTunes Connect. Retrieved from https://itunes.apple.com/us/book/factor-analysis/id656956844?ls=1 Copyrights Copyright for this article is retained by the author(s), with first publication rights granted to the journal. This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/). 72