International Education Studies; Vol. 6, No. 6; 2013 ISSN 1913-9020 E-ISSN 1913-9039 Published by Canadian Center of Science and Education Curriculum Development Based on the Big Picture Assessment of the Mechanical Engineering Program Mohd Anas Mohd Sabri1,2, Nor Kamaliana Khamis1,2, Mohd Faizal Mat Tahir1,2, Zaliha Wahid1,2, Ahmad Kamal Ariffin Mohd Ihsan1,2, Abu Bakar Sulong1,2 & Shahrum Abdullah1,2 1 Department of Mechanical and Materials Engineering, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia, Selangor, Malaysia 2 Centre for Automotive Research, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia, Selangor, Malaysia Correspondence: Mohd Anas Mohd Sabri, Department of Mechanical and Materials Engineering, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia, 43600 UKM Bangi, Selangor, Malaysia. Tel: 60-389-216-517. E-mail: [email protected] Received: February 7, 2013 Accepted: April 1, 2013 Online Published: May 30, 2013 doi:10.5539/ies.v6n6p213 URL: http://dx.doi.org/10.5539/ies.v6n6p213 Abstract One of the major concerns of the Engineering Accreditation Council (EAC) is the need for an effective monitoring and evaluation of program outcome domains that can be associated with courses taught under the Mechanical Engineering program. However, an effective monitoring method that can determine the results of each program outcome using Bloom’s Taxonomy has not yet been established for each course. The purpose of this research is to conduct a Big Picture Assessment to achieve Outcome-Based Learning. Big Picture Assessment is a comprehensive monitoring tool of courses with studied program outcome domains. The tool applies the three main domains of Bloom’s Taxonomy, namely, psychomotor, cognitive, and affective, in its monitoring process. Furthermore, the identification of program outcomes for each course is evaluated to meet standards set by the EAC. The results of this study will facilitate continuous improvement on existing courses. Keywords: big picture, program outcomes, course, cognitive, affective, psychomotor 1. Introduction The Faculty of Engineering and the Built Environment (FKAB) of the Universiti Kebangsaan Malaysia (UKM) prepare themselves to meet the criteria set by the Enginerring Accreditation Council (EAC), which acts as a recognition body of engineering programs in Malaysia. This undertaking is aimed at fulfilling the aspirations of UKM in becoming a leading research university with an accredited study program. In 2004, all engineering programs in public higher education institutions in Malaysia have begun to implement the Outcome-Based Education curriculum (OBE) (Shahrir et al., 2008). In line with the implementation of OBE into the engineering program, emphasis on the curriculum and the methods of delivery, assessment, and measurement is very important and should always be continued to ensure that the Program Education Objectives and Program Outcomes (PO) can be applied in teaching and learning. One of the requirements seriously considered by the EAC is an effective assessment of PO domains that can be associated with courses taught in engineering programs. (EAC, 2012) The purpose of this paper is to conduct the Big Picture Assessment on the mechanical engineering (KM) program in UKM and include the findings in the EAC self-assesment report. Big Picture Assessment is a comprehensive monitoring tool that can be used to assess all KM courses of study with applied domain POs. The results for each course program are assessed to determine if these programs meet the standards set by the EAC to facilitate improvements when needed. 2. Methodology Big Picture Rating was developed by Excel software. Mapping between POs and KM courses taken in each semester was created. The maximum number of POs used was only 9; before 2010, the number was 12. Table 1 shows the details and definition of each PO. Meanwhile, the courses were matched based on the existing list in the FKAB Undergraduate Handbook 2011–2012. Given that the focus was only on the compulsory courses of the 213 www.ccsenet.org/ies International Education Studies Vol. 6, No. 6; 2013 KM programme, courses such as electives, engineering fundamentals, and general education were not considered. Table 1. Details of POs (source: FKAB Undergraduate Handbook 2011–2012) PO1 Ability to apply knowledge in mathematics, science, and engineering PO2 Ability to identify, formulates, solve, and improve engineering problems using techniques, skills, and modern engineering tools necessary for engineering practice PO3 Ability to design a component, system, or process to meet desired needs PO4 Understanding of professional and ethical responsibility based on knowledge in environmental and contemporary issues PO5 Ability to understand and apply in-depth knowledge of one or more areas of specialization within mechanical engineering PO6 Ability to design and conduct experiments, as well as to analyze and interpret data PO7 Ability to communicate and function effectively in a team PO8 Recognition of the need for and an ability to engage in lifelong learning PO9 Knowledgeable in project management, administration, business acumen, and entrepreneurship In the matrix assessment, two color indicators, red and yellow, were used. Red represents a high correlation, while yellow means moderate correlation. If no color is indicated, the PO and course have a low or almost no correlation. All assessment criteria were listed in each color matrix, which describe how the POs were evaluated for each course. Table 2 lists the assessment criteria for evaluating the POs–KM courses matrix. Table 2. Assessment methods used in the matrix of POs–KM courses e Exam r Report a Task d Design p Project l Laboratory work c Case study s Software PB Problem based learning m Drawing g Co-curiculum activity n Observation o Oral presentation f Evaluation (peers) h Survey q Evaluation (lecturer) Each filled matrix was evaluated according to Bloom’s Taxonomy for each KM course. Introduced in 1956, Bloom’s Taxonomy categorizes skills and objectives to be achieved by the students into three major domains, namely, cognitive, affective, and psychomotor (Aqilah et al., 2008). Each course offered does have certain course outcomes based on the results of the cognitive domain in Bloom’s taxonomy. Initiatives have been taken to update all the results of the course based on the cognitive domain of Bloom’s taxonomy for engineering programs (Hairi et al., 2008). The cognitive domain includes knowledge, comprehension, and critical thinking about a certain topic. Traditional education tends to emphasize the skills in this domain. The cognitive domain consists of a number of the learning processes that should be implemented, such as the process of remembering, understanding, applying, analyzing, evaluating, and creating. The ordering of the processes is illustrated in Figure 1. 214 www.ccsenet.org/ies International Education Studies Vol. 6, No. 6; 2013   Figure 1. Cognitive domain Skills in the affective domain describe the way people react emotionally and their ability to feel sadness or happiness towards other living things. The affective domain has five levels, from the lowest to the highest process: receiving, responding, evaluating, organizing, and characterizing (Figure 2).   A5–Characterizing by Value A4‐ Organizingand Conceptualizing Figure 2. Affective domain The third domain is the psychomotor domain, which describes the ability to manipulate something physically. The objectives of the psychomotor domain usually focus on change and the development of attitudes and skills. The psychomotor domain includes the processes of perception, set, guided response, mechanism, complex overt reaction, adaptation, and origination, as shown in Figure 3. 215 www.ccsenet.org/ies International Education Studies Vol. 6, No. 6; 2013 Figure 3. Psychomotor domain 3. Results and Discussion The matrix between POs and KM courses that reflect the Big Picture Assessment of this program are displayed in one sheet only. However, for this paper, the matrix is divided into two phases, from the 1st to the 4th semester, and from the 5th to the 8th semester. Figure 4 shows the matrix from the 1st to the 4th semester. The majority of the KM courses emphasized on PO1, PO2, PO5, PO6, and PO7. Only a few of courses incorporated PO3. Based on the details of these POs, much emphasis has been made on basic skills in science and engineering, environmental issues, and the application of knowledge. These matters are given more focus in most courses, enabling students to form a strong foundation during their 1st and 2nd years before continuing into their 3rd and 4th years of study. In terms of Bloom’s Taxonomy, all the courses successfully incorporated all three domains. Most courses incorporated C3 to C5 of the cognitive domain. For the psychomotor domain, the majority of courses incorporated P3 and P5. Meanwhile, for the affective domains, many courses included A3 and A4. Thus, several of these courses dealt with these domains on an intermediate level and rarely incorporated the highest levels during the 1st and 2nd years of study. For the assessment method adopted, most courses used e (examination), a (assignment), p (project), r (report), and l (lab). Only certain courses included o (oral presentation) as one of their assessment methods. Thus, the majority of courses used more traditional evaluation methods that are commonly used in evaluating basic POs and that are less difficult in assessing their students. 216 www.ccsenet.org/ies International Education Studies Vol. 6, No. 6; 2013 Sem 1 2 3 4 3 4 4 4 4 4 4 3 3 4 4 4 4 1 1 2 4 2 1 1 1 1 2 4 2 2 5 9 3 3 0 3 1 0 5 7 3 1 3 1 1 1 1 1 2 2 2 2 2 2 2 2 M M M M M M M M M M M M M K K K K K K K K K K K K K K K K K K K K K K K K K K de K K K K K K K K K K K K K OPurotcgoramme Course & Co Materials Science Engineering Design Graphic Introduction to Electrical Engineering sEngineering MechanicI Computer Programming Measurement & Instrumentation Thermodynamics I Introduction to Mechanical and Electrical Engineering Materials Manufacturing Processes Mechanics of Materials Fluid Mechanics I sEngineering MechanicII PO1 C3,P3,A3 C3,P5,A3 C4,P3,A3 C4,P3,A3 C5,P5,A4C5,P6,A4 C4,P3,A3 C3,P3,A3 C5,P3,A4 C3,P3,A3 C5,P3,A4 C4,P3,A3 C5,P6,A4 (e,a,k) (e,a,p,s) (e,a,p,r,l) (e,a,l) (e,a,p,l) (e,a,p,r,l) (e,a,p,r,l,o(e,a,p,PB) (e,a,p,r,l) (e,p,r,l,o) (e,a,l) (e,p,o,r,l) (e,a,r,l) ) PO2 C3,P5,A3 C4,P3,A3 C4, P3, A3 C5,P5,A4C5,P6,A4 C4,P3,A3 C3,P3,A3 C5,P3,A4 C3,P3,A3 C5,P3,A4 C4,P3,A3 C5,P6,A4 (e,a,p,s) (l) (e,a,l) (e,a,p,l) (p,r,l) (e,a,p,r,l,o(e,a,p,PB) (e,a,p,r,l,o (e,p,r,l,o) (e,a,p,o,r,l)(e,p,o,r,l) (e,a,r,l) ) ) PO3 C3,P5,A3 C4,P3,A3 C5,P5,A4C5,P6,A4 C3,P3,A3 C5,P3,A4 C3,P3,A3 C4,P3,A3 (e,a,p,s) (e,a,l) (e,a,p,l) (e,a,p,r,l) (e,a,p,PB) (e,a,p,r,l) (e,p,r,l,o) (e,p,o,r,l) PO4 C4,P3A3 C5,P3,A4 C4,P3,A3 (e,a) (e,a,p,r,l,o (e,p,o) ) PO5 C3,P3,A3 C3,P5,A3 C4,P3,A3 C5,P5,A4C5,P6,A4 C4,P3,A3 C5,P3,A4 C5,P3,A4 C4,P3,A3 C5,P6,A4 (e,a) (e,a,p,s) (e,a,p,r,l) (e,a,p,l) (e,a,p,r,l) (e,a,p,r,l,o (p) (p,o,r,l) (e,p,o,r,l) (e,a,r,l) ) PO6 C3,P3,A3 C4,P3,A3 C5,P6,A4 C4,P3,A3 C3,P3,A3 C5,P3,A4 C3,P3,A3 C5,P3,A4 C4,P3,A3 C5,P6,A4 (a,k) (e,a,p,r,l) (e,a,p,r,l) (e,a,p,r,l,o(e,a,p,PB) (p) (e,p,r,l,o) (r,l) (e,p,o,r,l) (r,l) ) PO7 C3,P3,A3 C3,P5,A3 C4,P3,A3 C4,P3,A3 C5,P5,A4C5,P6,A4 C4,P3,A3 C3,P3,A3 C5,P3,A4 C3,P3,A3 C5,P3,A4 C5,P6,A4 (a,k) (p) (l) (e,a,l) (p,l) (p,r,l) (e,r,l) (e,a,p,PB) (e,l) (e,p,r,l,o) (p,o,r,l) (e,a,PB,r,l ) PO8 C3,P3,A3 C4,P3,A3 C5,P3,A4 C3,P3,A3 C5,P3,A4 C4,P3,A3 (e,a) (e,a,p,r,l) (e,l,p) (e,p,r,l,o) (p,o,r,l) (e,p,o,r,l) PO9 C4,P3,A3 (e,a,l) Figure 4. Big picture for KM from the 1st to the 4th semester Figure 5 shows the courses taken from the 5th to the 8th semester (3rd and 4th years). Many courses incorporated PO1 to PO8. Thus, 3rd and 4th year students were tested on all POs compared with 1st and 2nd year students. In terms of the cognitive domain, many courses included C5 and C6. For the psychomotor domain P3, P4, P5, and P7 were used, and, in the affective domain, A3 to A5. Thus, when the students enter their 3rd and 4th years, all of the highest levels of all three domains are covered. Furthermore, the results are consistent with previous studies in indicating that more than 90% of curricula in institutes of higher learning emphasize cognitive skills over psychomotor ones (Maizura et al. 2008). Furthermore, during this period, students have to undergo a more challenging evaluation in the form of PBL and case studies. Even an oral presentation is more frequent than in the 1st and 2nd years. Thus, students in their final years are exposed to actual problems faced in the engineering industry and become more capable of providing solutions in the various mechanical engineering fields. 217 www.ccsenet.org/ies International Education Studies Vol. 6, No. 6; 2013 5 6 7 8 M3014 M3114 M3314 F3XX4 M3934 M3324 M3344 M3124 M3144 M3944 P4013 M4314 M4034 M4955 M4025 M4044 K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K K OPurotcgoramme Numerical Methods Thermodynamics II Dynamics and System Simulation Engineering Ethic and Technology Development Design of Machine Components Machine & Power Electronics Control Engineering Fluid Mechanics II Heat Transfer System Design Project I Mechanical Vibration Engineering Economy and Entrepreneurship Design Project Project II Project Management PO1 C4,P4,A4 C6,P4,A6 C6,P4,A4 C6,P4,A3 C5,P5,A4 C4,P3,A3 C6,P4,A4 C5,P4,A3 C5,P6,A4 C5,P6,A4 C3 C4,P6,A3 C6,P3,A3 C6,P7,A5 C6,P7 C6,P7,A5 (e,a,p,r,l) (e,a,p,o,r,P(e,a,p,PB,o(e,p,PB,r,l)(e,a,p,r,o,l)(e,p,r) (e,p,r) (e,a,p,PBr(e,a,c,PB)(e,a,p,PB, (c,e,I,s) ( e,a) ( e,a,c,PB) ( e,p,r,o) (c,e,I,s) (e,a,p,e,o, BL) ,r,l) ,l) r,o) PB) PO2 C4,P4,A4 C6,P4,A6 C6,P4,A4 C6,P4,A3 C5,P5,A4 C4,P3,A3 C6,P4,A4 C5,P4,A4 C5,P6,A4 C5,P6,A4 C3 C4,P6,A3 C6,P3 P7,A5 C6 C6,P7,A5 (e,a,p,r,l) (e,a,p,o,r,Pe,a,p,PB,r,l(e,p,PB,r,l)(e,a,p,r,o,l)(e,p,r) (e,a,p,PB,(e,a,c,PB)(e,a,p,PB, (c,e,I,s) ( e,a) (e,a,PB,f ) (p,r ) (c,e,I,s) (e,a,p,e,o, BL) ) r,l) r,o) PB) PO3 C6,P4,A6 C6,P4,A3 C5,P5,A4 C6,P4,A4 C5,P4,A4 C5,P6,A4 C5,P6,A4 C3 (c,e,r) C4,P3,A5 C6,P7,A5 ( C6 C6,P7,A5 (e,a,p,o,r,P (e,p,PB,r,l)(e,a,p,r,o,l) (e,p,PB,o, (e,a,p,PB,(e,a,c,PB)(e,a,p,PB, (e,a,c,PB ) e,p,r,o) (c,e,I,s) (e,a,p,e,o, BL) r) r,l) r,o) PB) PO4 C6,P4,A6 C5,P5,A4 C5,P4,A4 C5,P6,A4 C5,P6,A4 C3,P2 C2,P3,A3 P7,A5 C3,A4 C6,P7,A5 (e,a,p,o,r,P (e,p,r,o) (e,p,r) (e,a,c,PB)(e,p,PB,r, (c,e,I,s) (e,a,c,PB ) (p,r,o) (c,e,I,s) (e,a,p,e,o, BL) o) PB) PO5 C4,P4,A4 C6,P4,A6 C6,P4,A4 C6,P4,A3 C5,P5,A4 C4,P3,A3 C6,P4,A4 C5,P4,A4 C5,P6,A4 C5,P6,A4 C3 C4,P6,A3 C4,P2 C6,P7,A5 C4,P5 C6,P3,A3 (e,a,p,r,l) (e,a,p,o,r,P(e,a,p,PB,o(e,p,PB,r,l)(e,a,p,r,o,l)(e,p,r) (e,p,PB,o, (e,a,p,PB,(e,a,c,PB)(e,a,p,PB, (c,e,I,s) ( e,a) (e,a,c,PB ) ( e,p,r,o) (c,e,I,s) ( e,a,p,r) BL) ,r,l) r,l) r,l) r,o) PO6 C6,P4,A6 C6,P4,A4 C6,P4,A3 C5,P5,A4 C4,P3,A3 C6,P4,A4 C5,P4,A4 C5,P6,A4 C5,P6,A4 C3 C^,P3,A5 C6,P7 (e,a,p,o,r,P (e,p,PB,r,l)(e,p,r,o) (e,p,r) (PB,o,r) (e,a,p,PB,(e,a,c,PB)(e,p,PB,r, (c,e,I,s) (e,a,c,PB (c,e,I,s) BL) r,l) o) ,f) PO7 C4,P4,A4 C6,P4,A6 C6,P4,A4 C6,P4,A3 C5,P5,A4 C4,P3,A3 C6,P4,A4 C5,P4,A4 C5,P6,A4 C3,P3,A3 C6,P3,A5 C6,P7,A5 C3,A6 C6,P7,A5 (e,a,p,r,l) (e,a,p,o,r,P(PB,o,r,l) (e,p,PB,r,l)(e,a,p,r,o,l)(e,p,r) (e,p,PB,o, (e,p,r,o, (e,a,p,PB, (a,c,e,o,r, (e,a,c,PB,f)( e,p,r,o) (c,e,g,I,s(e,a,p,e,o, BL) r) PB) r,o) s) ) PB) PO8 C6,P4,A6 C6,P4,A3 C5,P5,A4 C4,P3,A3 C6,P4,A4 C5,P4,A3 C5,P6,A4 C5,P6,A4 C3,P2 C2, P3, A3 P7 (p,r ) C3,A4 C6,P7,A5 (e,a,p,o,r,P (e,p,PB,r,l)(e,a,p,r,o,l)(e,p,r) (PB,o,r) (e,p,r,o, (e,a,c,PB)(e,a,p,PB, (c,e,I,s) (e,a,c,PB) (c,e,I,s) (e,a,p,e,o, BL) PB) r,o) PB) PO9 P7,A5 C6,P7,A5 (p,r,o ) (e,a,p,e,o, PB) Figure 5. Big picture for KM from the 5th to the 8th semester 4. Conclusion The evaluation of the prevailing curriculum used in KM courses by Big Picture Assessment was successfully conducted. The majority of the courses under the mechanical engineering program of UKM incorporated PO1, PO2, PO5, PO6, and PO7 during the 1st and 2nd years, whereas the remaining levels were covered in the last 2 years of the program. The students were also tested on the highest levels of the three domains of Bloom’s Taxonomy during their 3rd and 4th years. However, based on the results, most of the courses tended to use the cognitive domain as the primary evaluation method over the affective and psychomotor domains. Lecturers were also more likely to insert the element of PBL and case studies in the final year of the course. Comparisons with the manufacturing program are proposed for future studies to achieve a more thorough analysis. Acknowledgements The authors would like to express their gratitude for the support given by the head and members of the Department of Mechanical and Materials Engineering and to the National University of Malaysia for the financial support through research grant number PTS-2012-097. References Aqilah B., Huddin, W. M., Diyana, W., Zaki, Aini Hussain, Hafizah Husain, & Nor Fadzilah Abdullah. (2008). Pelaksanaan Pengunaan Taksonomi Bloom Bagi Mengukur Kesesuaian Tahap Dan Menjamin Outcome Kursus. Seminar Pendidikan Kejuruteraan dan Alam Bina 2008. Engineering Accreditation Council (EAC). (2012). Self Assesment Report, Department of Mechanic & Materials Engineering, 2012. Fakulti Kejuruteraan, & dan Alam Bina. (2011). Universiti Kebangsaan Malaysia, Panduan Prasiswazah Fakulti Kejuruteraan dan Alam Bina,: Sesi Akademik 2011-2012, Fakulti Kejuruteraan dan Alam Bina, UKM. Mohd Hairi Mohd Zaman, Badariah Bais, Norbahiah Misran, Salina Abdul Samad, & Aini Hussain. (2008). Penilaian hasil kursus melalui kaedah penilaian langsung dan tak langsung. Seminar Pendidikan Kejuruteraan dan Alam Bina 2008. Seri Mastura Mustaza, Muhd Fauzi Aminuddin Shazi, Aini Hussain, Iskandar Yahya, & Mohd Marzuki Mustafa. 218 www.ccsenet.org/ies International Education Studies Vol. 6, No. 6; 2013 (2008). Penandaarasan program akademik untuk tujuan penambahbaikan, Jaminan Kualiti dan Akreditasi. Seminar Pengajaran dan Pembelajaran Berkesan 2008. Shahrir Abdullah, Riza Atiq Abdullah Ok Rahmat, Azami Zaharim, Norhamidi Muhamad, Baba Md. Deros, Noorhisham Tan Kofli, & Che Husna Azhari. (2008). Implementing continual review of programme educational objectives and outcomes for OBE Curriculum based on stakeholders’ input. Proceedings of the 7th WSEAS International Conference on Education & Educational Technology (EDU’08), Venice, Italy, 21-23 November 2008, 218-223. 219