International Journal of Education and Development using Information and Communication Technology (IJEDICT), 2013, Vol. 9, Issue 1, pp. 100-111   Using Self-Efficacy to measure primary school teachers’ perception of ICT: results from two studies F. Fanni, I. Rega and L. Cantoni NewMinE Lab, Università della Svizzera italiana, Switzerland ABSTRACT The aim of this article is twofold. First, the final results of two research projects, which investigated the impact of Information and Communication Technology (ICT) on primary schools teachers in disadvantaged areas in Brazil (BET k-12) and South Africa (MELISSA), are presented and discussed. Second, the Self-Efficacy construct is proposed as a tool to measure how teachers’ perception of being able to use technology (CSE - Computer Self-Efficacy) affects teachers’ perception of being an effective teacher (TSE - Teacher Self-Efficacy). This article intends to provide data gathered from two case studies in which the Self-Efficacy construct has been applied to measure the impact of ICT in teaching experiences. One out of four surveys confirmed the hypotheses of the abovementioned projects, namely, an increased CSE caused by the improvement of technological skills and a correlation between CSE and TSE. Hence, the authors considered the possibility of creating a new tool to better understand the impact of ICT on teacher training through Self-Efficacy. Keywords: Teacher Self-Efficacy; Computer Self-Efficacy; Teacher Training; Information and Communication Technology; Impact; Case study. INTRODUCTION This article addresses the issue of measuring the impact that teacher training on Information and Communication Technology (ICT) has on primary school teachers. It uses the Teacher Self- Efficacy (TSE) and Computer self-Efficacy (CSE) constructs (Bandura 1977) as a theoretical framework through which to enlighten the role of Information and Communication Technology (ICT) in a teacher’s perception of being a good teacher. Final results of a research project, named MELISSA, are presented, discussing them in comparison with results of a similar previous project, called BET K-12. Both projects presented in this article aimed at (1) introducing ICTs in teachers’ practice and (2) evaluating their impact. In particular, BET K-12 - Brazilian eLearning Teacher Training in K-12 – project, developed from 2005 to 2008, aimed at training primary teachers in community schools in a disadvantaged area of Salvador (State of Bahia, Brazil) in the use of ICTs and in the introduction of ICTs in their teaching activities, and assessing them in terms of CSE and TSE. In order to confirm/disconfirm BET K-12 project’s results, a related project, named MELISSA, has been designed and run from 2009 to 2011. Main goal of MELISSA – Measuring E-Learning impact in primary Schools in South African disadvantaged areas – was to evaluate the impact of ICTs teacher training curriculum on teachers working in disadvantaged primary schools in the Western Cape Province, South Africa. On one hand, in order to achieve goal (1), a teacher training programme, delivered twice during both projects, was proposed to the teachers, introducing them to ICTs practices and exploring the integration of ICTs in their teaching activities. Primary school teachers’ perception of ICT 101   On the other hand, to accomplish goal (2), a mixed investigative method was applied in both projects, merging quantitative and qualitative methodologies, in a quasi-experimental setting in BET K-12, and in an experimental setting in MELISSA. In particular, impact was investigated in terms of changes in teachers’ perceptions of being effective educators as they become more confident in ICTs, hence studying the relationship between CSE and TSE. THEORETICAL FRAMEWORK A high level of knowledge and skills in ICTs use does not necessary mean an actual use of ICTs. In fact, what we know, the skills we possess, or what we have previously accomplished are not always good predictors of subsequent attainments because the beliefs we hold about our capabilities powerfully influence the ways we behave”. (Madewell & Shaughnessy 2003, p. 381) In Social Cognitive Theory, human functioning is viewed as a dynamic interplay of personal, behavioural, and environmental influences. How people interpret the results of their own behaviour informs and alters their environments and the personal factors they possess, which, in turn, inform and alter subsequent behaviour. This is the foundation of Bandura’s (1986) conception of reciprocal determinism, the view that personal factors - in the form of cognition, affect, and biological events -, behaviour, and environmental influences create interactions that result in a triadic reciprocality (Usher et al. 2011). Social Cognitive Theory provides an agentic view of human behaviour in which individuals, through their own self-referent thoughts and feelings, can in part determine the course of actions they take. Of these self-referent thoughts, none is more important than the beliefs individuals hold about their own capabilities, or Self-Efficacy beliefs (Bandura 1995). Albert Bandura (1986) defines the term ‘Self-Efficacy’ as: People’s judgment of their capabilities to organize and execute courses of action required to attain designated types of performances. (p. 391) Bandura identifies four main sources of influence on Self-Efficacy: mastery experiences, vicarious experiences, social persuasion, and emotional states. • Mastery experiences are the most effective means of creating a sense of Self-Efficacy. These in fact represent the memories of past successful experiences that individuals may revert to while facing current or future situations. Positive mastery experiences reinforce Self-Efficacy, while negative mastery experiences weaken it. • Vicarious experiences emanate from the observation of peers or “models”: a process of comparing oneself to other individuals. Seeing these models succeed may increase the observer’s Self-Efficacy, while seeing them fail may weaken Self-Efficacy. This process is intensified if the observer regards him- or herself as similar to the model. • Social persuasion represents positive (verbal) reinforcement. It is possible here that one’s Self-Efficacy may increase if encouraged or motivated by others. Despite social persuasions being less powerful than mastery experiences, they may yet exert a strong influence on self- belief. 102 IJEDICT   • Emotional states (psychological factors) represent the final source of Self-Efficacy according to Bandura. Individuals often consider that their skills are (strictly) related to the way they feel in a particular moment, where a state of stress or tension may be an indication of failure. Individuals with a high sense of Self-Efficacy may employ these kinds of emotional states to improve their performance. Those individuals with a low(er) sense of Self-Efficacy consider these states as a negative influence on the activities they are engaged in. (Bandura 1977) Given the importance of beliefs in understanding the actual integration of ICTs in teaching activities (Ertmer 2005), customized quantitative measurement instruments have been developed. On one hand, several researchers designed measurement instruments for studying Teacher Self-Efficacy (Ashton, et al. 1982; Gibson & Dembo 1984; Bandura 1995; Tschannen- Moran & Woolfolk Hoy 2001; Henson, et al. 2001); on the other hand, Self-Efficacy about the use of ICT has been extensively investigated too (Ertmer, et al. 1994; Compeau & Higgins 1995; Marakas et al. 1998; Cassidy & Eachus 2002; Khorrami-Arani 2001). Furthermore, many scholars investigated Self-Efficacy beliefs of teachers using ICT in a variety of contexts, e.g. pre-service teacher training and science high school teachers (Albion 1999; Wang et al. 2004; Milbrath & Kinzie 2000; Abbitt & Klett 2008). For these research projects, the construct has been applied to two specific contexts: the use of ICT (Computer Self-Efficacy – CSE) and teaching activity (Teacher Self-Efficacy – TSE). CSE represents “an individual perception of his or her ability to use computers in the accomplishment of a task” (Compeau & Higgins 1995, p. 192), while TSE can be defined as a teacher’s: “Judgment of his or her capabilities to bring about desired outcomes of student engagement and learning, even among those students who may be difficult or unmotivated” (Bandura 1977, cited in Tschannen-Moran & Woolfolk Hoy 2001, p. 783). METHODOLOGY In order to measure the impact of ICT on teacher practices, a questionnaire was designed to evaluate Computer and Teacher Self-Efficacy and their changes (if any) along both projects (pre- post test). The part on Computer Self-Efficacy is based on the questionnaire validated by Compeau and Higgins (1995). This contains 10 items that refer to the use of software in a given educational context; for each item a Likert scale (1 to 10) is provided, where 1 is “not at all confident” and 10 is “totally confident”. The 10 items are repeated for all the technologies presented in the curriculum. For Teacher Self-Efficacy, the Teacher’s Sense of Efficacy Scale validated by Tschannen-Moran and Woolfolk Hoy (2001) has been adopted. In this scale, 12 sections – divided into 3 categories: “student engagement”, “instructional strategies” and “classroom management” – refer to different aspects of the teaching activity; for each question a Likert scale (1 to 9) is provided, where 1 is “nothing” and 9 is “a great deal”. Teachers were required to answer these questions by indicating how much they would feel able to accomplish given teaching activities. Teachers for both projects have been selected according to the following criteria: • can access computers and network facilities, in order to be trained and to access the online part of the curriculum; • show a great motivation in the learning experience; • lack prior computer skill; Primary school teachers’ perception of ICT 103   • respectively for the Brazilian project be part of the BET K-12 network, and for the South African project be part of the MELISSA network, and agreed with project research and educational statement. Data have been gathered through paper questionnaires in the first project and through an online questionnaire designed with Survey Monkey in the second project. Statistical analysis has been performed using SPSS software. The research hypotheses that have been tested through the abovementioned methodology were: • H1: the increase of technological skills promoted by the attendance to the proposed curriculum causes an increase in CSE; • H2: the increase of CSE influences an increase in TSE. FIRST STUDY: BET K-12 In Brazil, the Federal Government’s law n. 4019/2004 requested teachers to obtain a university degree in order to keep on teaching. Even if the first foreseen deadline – year 2012 – has been postponed, this law has promoted an important and positive mobilization among teachers, which resulted in a significant growth in the demand of updating-courses, especially for in-service teachers who did not own the required university degree. Even if the goal of the government’s decision is the improvement of teachers’ preparation and, as a consequence, of the quality of Brazilian school system, it could cause negative side effects, such as the closure of disadvantaged schools for a lack of graduate teachers. For this reason the training of teachers, particularly of those who live in disadvantaged areas, is still a crucial issue for the Brazilian school system (Cantoni et al. 2009). In this context BET K-12 has been run, delivering a ICTs teacher training curriculum twice (November 2006 – December 2007, and March – October 2008) to a total of 79 primary school teachers working in disadvantaged areas. TSE and CSE have been measured in the first round in a semi-experimental context with a group of 44 teachers attending project’s curriculum, and results supported both hypotheses (H1) and (H2). The graph below shows that CSE mean values significantly increased along the project; TSE grew along the first three data collections, but decreased in the last one. 104 IJEDICT   Figure 1: Teacher and Computer Self-Efficacy during the project time span – First Round (data are normalized to a 10 grade scale) A significant and positive correlation was detected in all four data collections: in November 2006 α value was 0.19** (R²=0.16), in March 2007 α=0.23* (R²=0.11), in July 2007 α=0.28** (R²=0.26), and α=0.31** (R²=0.11) in the last data collection. Researchers decided then to adopt an experimental setting with an experimental group composed by 35 teachers attending the course and a control group composed by 30 teachers, but in this case the first hypothesis (H1) was verified, while the second one (H2) was not, as the figure below shows. Primary school teachers’ perception of ICT 105   Figure 2: Teacher and Computer Self-Efficacy during the project time span – Experimental Group Second Round (data are normalized to a 10 grade scale) Even though CSE decreased in the last measurement, (H1) was confirmed; this result may be explained by the fact that once a teacher learns how to use a new tool, after nine months the novelty is no longer perceived. It is also important to consider that in the last data collection only 7 questionnaires were done (July 2009), which could be a bias in the comparison of the results and reduce its significance. H2 was refused, because no statistical regression between the two variables has been detected in the four measurements. Furthermore, control group results, as shown in figure 3, highlights that (H1) was confirmed: the attendance of an ICT course is a factor that increases the CSE of teachers. On the other hand, the growth of TSE values was not caused only by the increase of CSE, but also by other external factors, since it increased during the course, even if those teachers did not attend the course. No significant correlation has been detected. Figure 3: Teacher and Computer Self-Efficacy during the project time span – Control Group Second Round (data are normalized to a 10 grade scale) Therefore, researchers decided to further investigate the matter in a similar context to properly understand if: • Self-efficacy is a construct that can be adopted to explain changes in teachers’ attitudes towards technologies • ICT-based courses can affect teachers’ perception of being good teachers (Fanni et al. 2010). 106 IJEDICT   SECOND STUDY: MELISSA In recent years, the South African Department of Education (DoE) has outlined Information and Communication Technologies as an integral part of modern education, especially in terms of computer-assisted teaching (Fanni et al. 2010). This has spawned a renewed interest in distance education and technological learning in the national degree programme, stipulated as part of the National Policy Framework for Teacher Education and Development. Furthermore, it has become pertinent for the DoE to introduce technological infrastructures within under-resourced schools. One of these DoE’s initiatives is taking action in the Western Cape Province, under the name of “Khanya project”. Khanya project was initiated by the Western Cape Department of Education in 2001 as a programme to equip schools in the Province with ICT infrastructure. Goal of Khanya was to empower educators in every school of the Province to use appropriate and available technology to deliver curriculum to all learners in the province by 2012 (Khanya 2012). A total of 110 teachers were involved in the MELISSA project, attending a ICTs training course designed upon the curriculum developed within the previous research project. Teachers were selected from six disadvantaged primary schools that received fully kitted computer laboratories, as they took part to the Khanya initiative. The course was delivered twice during the three years of the project; the first round from July 2009 to May 2010, and the second from July 2010 to June 2011. MELISSA research was developed with an experimental setting, so among the 110 primary school teachers involved in the project, 42 ones were randomly assigned to the experimental group, and received training from July 2009 to May 2010 (referred to as “Group A”); whilst 68 teachers were assigned to a control group, who received training along the second round (referred to as “Group B”). Questionnaires were conducted with Group A on four occasions: beginning of their course (July 2009), middle of their course (January 2010), end of their course (May 2010), and a follow-up (July 2011); and four times with group B: July 2009 and January 2010 (they participated as the control group), January 2011 (middle of their course), and July 2011 (end of their course). Due to organizational problems, data concerning the beginning of the second course delivery were not available. Moreover, data collection during 2010 has been severely constrained by an extensive public sector strike, which closed schools for several weeks. The graphs below show the results of all the measurement conducted during the project, figure 4 for Group A and figure 5 for Group B. Mean values of Computer Self-Efficacy (CSE) and Teacher Self-Efficacy (TSE) of the teachers involved in the project are shown in the graphs. Primary school teachers’ perception of ICT 107   Figure 4: Teacher and Computer Self-Efficacy during the project time span – Group A (data are normalized to a 10 grade scale) The first questionnaire (July 2009), reveals a CSE rate of 5.7 out of 10, and a TSE level of 8 out of 10. In this first survey, the two variables are not significantly correlated (α=0.13**, R²=0.04). During the training, in January 2010, the trend of the two variables remains statically the same (CSE=6.1, TSE=7.4, both with positive F-Test), with no significant correlation (α=0.30*, R²=0.20). At the end of the training, in May 2010, the CSE rate increases to 6.1 out of 10 (negative F-Test); TSE rate, instead, remains statistically unchanged (7.8 out of 10, positive F-Test). Also in May 2010, the variables appear to have no significant correlation (α=0.16*, R²=0.97) (Rega & Fanni 2012). A follow up survey (July 2011) let the authors confirm changes in CSE and TSE values along the course: comparing mean values in July 2009 and July 2011, CSE increased from 5.7 to 7.2 out of 10, while TSE decreased from 8 to 7.5 out of 10 (positive F-Test). No significant correlation has been detected between CSE and TSE mean values along the course time, not even in July 2011 (α=0.37*, R²=0.003). 108 IJEDICT   Figure 5: Teacher and Computer Self-Efficacy during the project time span - Group B (data are normalized to a 10 grade scale) Similarly, analyzing Group B data through the same procedure, CSE mean values slightly increased along the project time span; from the beginning to the end of the project CSE mean value statistically grows from 5.7 to 6.3 out of 10 (negative F-Test). TSE mean values, on the other hand, decreased from 8.0 (July 2009) to 7.6 (July 2011) out of 10 (negative F-Test). A statistical correlation has been observed only in July 2011 (α=0.58**). On the whole, results from the statistical analysis show that CSE increased as the training progressed both in Group A and Group B, validating the first research hypothesis (H1); while TSE decreased. No correlation between the two variables can be detected, except for Group B in July 2011. Considering that a positive correlation between the two variables occurred with no specific pattern just once along the four surveys completed by both Group A and Group B, the second research hypothesis (H2) was not confirmed by statistical analysis. DISCUSSION AND CONCLUSIONS This article presented results from two field studies where Teachers Sense of Self-Efficacy Scale (Tschannen-Moran & Woolfolk Hoy 2001) has been applied, together with Computer Self-Efficacy Scale (Compeau & Higgins 1995) as tools to measure teacher’s perception of ICT use. Specifically, studying the relationship between TSE and CSE the authors tested the hypothesis that (H1) an ICT teacher training course can improve CSE, and that (H2) the increase of CSE can positively influence TSE. While the first research confirmed both hypotheses in the first round, its second round, instead, confirmed only the first one (H1), but refused the second one (H2). Authors tested the same hypotheses in a similar context through the second project, validating the first hypothesis (H1), but refusing the second one (H2): CSE slightly increased during the project period; conversely, teachers’ perception of being good educators decreased along the project time span. Furthermore, CSE was not significantly correlated to TSE, suggesting that MELISSA teachers, Primary school teachers’ perception of ICT 109   even though they felt themselves more capable of using ICTs, did not perceive themselves as being better teachers. On the whole, despite CSE level in teachers did improve, TSE has not been positively affected as expected. Nevertheless, authors consider the use of ICT as one of the main skills, which a teacher has to master in the so-called Knowledge Society (Rivoltella 2008); as a matter of fact, teachers have to be able not only to teach through ICT, but also to teach how to properly use them (Bates & Sangrà 2011). Nowadays, in fact, ICT are permeating our life, affecting also the teaching and learning experience (Rapetti & Cantoni 2012, OECD 2012); in this context, CSE of teachers should be somehow interpreted within TSE as an integral part of it. In conclusion, both project’s results suggest to explore new research paths regarding the methodology applied. On one hand, they confirm the relevance of Self-Efficacy construct as theoretical framework to describe teachers’ perception of ICTs use; on the other hand, they reveal a need for a more suitable tool to better measure the role of ICTs in teacher experiences. Specifically, authors are exploring the option to integrate Computer and Teacher Self-Efficacy creating a new tool measuring Teacher Self-Efficacy in the Knowledge Society. ACKNOWLEDGEMENT BET K-12 project was run by Università della Svizzera italiana (Switzerland), in partnership with a Brazilian NGO (CEAP – Centro de Estudo e Assesoria Pedagogica), and the Universidade Federal de Bahia, and was funded by the Swiss National Science Foundation and the Swiss Agency for Development and Cooperation. MELISSA is a joint research project in partnership with the Università della Svizzera italiana, the University of Cape Town, and the Cape Peninsula University of Technology in South Africa, and funded by the Swiss Secretariat for Education and Research. REFERENCES Abbitt, J. T. & Klett, M. D. 2008. “Identifying influences on attitudes and self-­‐efficacy beliefs towards technology integration among pre-­‐service educators”. Electronic Journal for the Integration of Technology in Education, vol. 6. Albion, P. R. 1999. “Self-efficacy beliefs as an indicator of teachers' preparedness for teaching with technology“, Computers in the Social Studies, vol. 7, no. 4. Ashton, P. T., Olejnik, S., Crocker, L. & McAuliffe, M. 1982. “Measurement problems in the study of teachers’ sense of efficacy”, Proceedings of the Annual Meeting of the American Educational Research Association, New York. Bandura, A. 1977. “Self-Efficacy: towards a unifying theory of behavioral change”, Psychological Review, vol. 84, no. 2, pp. 191-215. Bandura, A. 1986. “Social foundations of thought and action: a social cognitive theory”, Englewood Cliffs, NJ: Prentice-Hall. Bandura, A. 1995. “Self-Efficacy in changing societies”, Cambridge University Press, New York.