Synthesizing Technology Adoption and Learners’ Approaches Towards Active Learning in Higher Education Kevin Chan1, George Cheung1, Kelvin Wan1, Ian Brown 2 and Green Luk 2 1Department of Applied Social Sciences, Hong Kong Polytechnic University, Hunghom, Kowloon, Hong Kong 2Educational Development Centre, Hong Kong Polytechnic University, Hunghom, Kowloon, Hong Kong [email protected] [email protected] [email protected] [email protected] [email protected] Abstract: In understanding how active and blended learning approaches with learning technologies engagement in undergraduate education, current research models tend to undermine the effect of learners’ variations, particularly regarding their styles and approaches to learning, on intention and use of learning technologies. This study contributes to further examine a working model for learning outcomes in higher education with the Unified Theory of Acceptance and Use of Technology (UTAUT) on SRS adoption attitude, and the Study Process Questionnaire (SPQ) on students’ approach to learning. Adopting a cross-section observational design, the current study featured an online survey incorporating items UTAUT and SPQ. The survey was administered to 1627 undergraduate students at a large comprehensive university in Hong Kong. Relationships between SRS adoption attitude, learning approaches, and learning outcomes in higher-order thinking & learning and collaborative learning were analyzed with a structural equation model (SEM). A total of 3 latent factors, including four factors from UTAUT in Performance Expectancy, Effort Expectancy, and Deep Learning Approach from the SPQ, were identified in the structural model on students’ intention to adopt SRS in classes. Current results suggested that a model of active learning outcomes comprising both UTAUT constructs and deep learning approach. Model presented in the present study supported the UTAUT in predicting both behavioral intention and in adopting SRS in large classes of undergraduate education. Specifically, positive attitudes towards SRS use measured with the UTAUT, via a learning approach towards deep learning, accounted for variation on high-impact learning including higher-order thinking and collaborative learning. Results demonstrated that the process of technology adoption should be conceptualized in conjunction with learners’ diversity for explaining variation in adoption of technologies in the higher education context. Keywords: Technology adoption; Learning Approaches; Students Response System (SRS); Higher Education 1 Background Prevailing ideologies in higher education are advocating student-oriented, constructivist, and collaborative learning among students and their faculties (Al-Huneidi and Schreurs, 2013, Slavich and Zimbardo, 2012), in which learners must assume an active role in constructing knowledge in a collaborative and interactive learning environment. Active and blended learning approaches with learning technologies have been identified as instrumental in engaging large classes at the undergraduate level in higher education (Kerr, 2011, Exeter et al., 2010). Current research has proposed various models for outlining the process in which learners adopted such learning technologies (Venkatesh and Bala, 2008, King and He, 2006, Davis et al., 1989) and enhance their learning experience and performance. While giving feedback individually to students in large classes remains a daunting task with limited time in face-to-face lecture, the use of SRS allows mass, yet personalized feedback to students in the face-to-face learning process with aid from adopting the blended learning approach. The use of students’ response system (SRS) with mobile devices has been demonstrated as an effective tool for engaging students in active learning under the large-class context in higher education (Velasco and Çavdar, 2013, Chan et al., 2013, Cobb et al., 2010, Shapiro, 2009, Caldwell, 2007, Schell et al., 2013). Use of SRS for blended learning is influenced by the learner’s propensity to adopt and use the technology in their learning practices. One of the more recent and prevailing models for understanding use of learning technology is the Unified Theory of Acceptance and Use of Technology (UTAUT), a comprehensive (Venkatesh ISSN 1479-4403 431 ©ACPIL Reference this paper as Chan K, Cheung G, Wan K, Brown I and Luk G “Synthesizing Technology Adoption and Learners’ Approaches Towards Active Learning in Higher Education” The Electronic Journal of e-Learning Volume 13 Issue 6 2015, (pp431-440) available online at www.ejel.org The Electronic Journal of e-Learning Volume 13 Issue 6 2015 et al., 2003) and cross-cultural (Lidia et al., 2007) model for understanding factors to behavioral intentions and actual use of technology. Core components in the UTAUT include expectancies towards performance from adopting the technology, effort required in adopting the technology, social influence to adopt the technology, facilitating conditions for adopting the technology. These core components are hypothesized to influence satisfaction and behavioral intention to continue use of the targeted technology. However, these current models, when applied in an educational context, tend to undermine the effect of learners’ variations, particularly regarding their styles and approaches to learning, on intention and use of learning technologies. To accommodate the moderating effect of learners’ approaches, a multi-dimensional model is proposed for delineating the relationships between attitude towards learning technologies, learners’ approaches to learning, and behavioral intention towards continuous use of learning technology as one of the strategies for practicing active learning. Recent research suggested that behavioral changes with SRS in classes influenced students’ approaches or styles to learning (Tlhoaele et al., 2015, Wang et al., 2012) towards deep approach to understanding of subject matters. Whether such deep learning stimulated by SRS in higher education would translate into learning outcomes calls for a theoretical integration and an empirical investigation. In this context, this study contributes to further examine a working model for students’ learning engagement with behavioral intention towards SRS usage with the Unified Theory of Acceptance and Use of Technology (UTAUT) (Dwivedi et al., 2011, Lidia et al., 2007, Venkatesh et al., 2003), and the Revised Two-Factor Study Process Questionnaire (SPQ) (Justicia et al., 2008, Biggs et al., 2001) on students’ approach to learning. The present study is set to address the research question of whether students with high readiness towards use of Clickers towards active learning (UTAUT) would be more likely to adopt deep learning approach (R-SPQ-2F) and subsequent learners’ engagement in higher order thinking and collaborative learning (NSSE). 2 Methods This study features a cross-section observational study at a large comprehensive university in Hong Kong. An online survey incorporating items UTAUT and SPQ was administered to 1623 undergraduate students. Participants in this study were enrolled in undergraduate courses using students response system (SRS, a.k.a. Clickers) for active learning activities in classes. Details on Clickers administration in the participating courses were reported in a previous paper by the research team (Chan et al., 2013). Implementation of SRS in the current study was evaluated with an online quantitative instrument on various aspects of SRS implementation with 5-point Likert-type and non-Likert type questions. • Instruments • Unified Theory of Use and Acceptance of Technology (UTAUT) The Unified Theory of Acceptance and Use of Technology (UTAUT) augments the original technology acceptance model (Davis et al., 1989) with addition of social influence and facilitating conditions in the technology deployment environment towards measurement of technology adoption. By integrating other technology adoption models, the UTAUT proposed a unified model with four determinants influencing both behavioral intention (BI) and actual use of a technology; these determinants are: • Performance expectancy (PE): perceived gains on task performance by adopting the targeted technology • Effort expectancy (EE): ease of use in adopting the targeted technology • Social influence (SI): degree to which significant others influence the user's intention or actual use of the targeted technology • Facilitating conditions (FC): perceived level of organizational and technical infrastructure support to facilitate use of the targeted technology Attitude towards usage of SRS as a learning tool in classroom was assessed with the Unified Theory of Use and Acceptance of Technology (UTAUT) (Venkatesh et al., 2003), a 5-domain 20-item instruments on a 5-point www.ejel.org 432 ©ACPIL Kevin Chan, George Cheung, Kelvin Wan et al likert scale measuring performance expectancy (PE), effort expectancy (EE), social influence (SI), facilitating conditions (FC), and satisfaction and behavioral intention to continue use of the technology adopted (S/BI). • Revised Two-Factor Study Process Questionnaire (R-SPQ-2F) Learners’ learning approaches were assessed with the 20-item Revised Two-Factor Study Process Questionnaire (R-SPQ-2F) (Biggs et al., 2001), with learners classified as adopting surface or deep learning approaches for outlining their motives and strategies towards their learning. • Higher-Order Learning and Collaborative Learning – National Survey of Students’ Engagement (NSSE- HO / NSSE-CL) Students’ generic learning outcomes were captured with two subscales, higher-order learning (NSSE-HO) and collaborative learning (NSSE_CL) from the National Survey of Student Engagement 2013 (Indiana University Center for Postsecondary Research, 2013). Higher order learning refers to challenging learning tasks including applying information learned in solving practical problems, synthesizing information from multiple sources, and forming new ideas from information sourced. Collaborative learning refers to the degree that students master skills and concepts through discussion and team learning with other students toward formative and summative assessments. 3 Data Analysis Quantitative data derived from this study was analyzed with a structural equation model (SEM). Data were analyzed using confirmatory factor analysis (CFA) and structural equation modeling (SEM) (Bollen, 1989). Descriptive and correlations were generated with the IBM SPSS Statistics 22 software. All SEM models were estimated with the IBM SPSS Amos 22 software. Standard maximum likelihood estimations were applied in the SEM with no observed missing data. The research model on students' learning outcomes was based on two latent variables, namely satisfaction and behavioral intention to use of SRS in class in the Unified Theory of Use and Acceptance of Technology (UTAUT), and students' deep approach to learning with the Revised Study Process Questionnaire - 2 Factor Structure (R-SPQ-2F). The research model is illustrated in Figure 1 and the research hypotheses are listed in Table 1. Figure 1: Research model Table 1: Research hypotheses www.ejel.org 433 ISSN 1479-439X The Electronic Journal of e-Learning Volume 13 Issue 6 2015 H1 Performance expectancy (PE) will have a positive and significant influence on satisfaction and behavioral intention to use SRS (S/BI) H2 Effort expectancy (EE) will have a positive and significant influence on satisfaction and behavioral intention to use SRS (S/BI) H3 Social influence (SI) will have a positive and significant influence on satisfaction and behavioral intention to use SRS (S/BI) H4 Facilitating conditions (FC) will have a positive and significant influence on satisfaction and behavioral intention to use SRS (S/BI) H5 Satisfaction and behavioral intention to use SRS (S/BI) will have a positive and significant influence on deep learning approach (Deep) H6 Deep learning approach (Deep) will have a positive and significant influence on NSSE Higher-Order Thinking / Learning (NSSE-HO) H7 Deep learning approach (Deep) will have a positive and significant influence on NSSE Collaborative Learning (NSSE-CL) The proposed model fit was evaluated using conventional fit indices (Hu and Bentler, 1999) including Chi- 2 Square Statistics ((cid:662) ), global fit index (GFI), Tucker-Lewis index (TLI), comparative fit index (CFI), incremental fit index (IFI), root mean square error of approximation (RMSEA), and the standardized root mean square residuals (SRMRs). 4 Results 4.1 Participants and demographics A total of 1623 undergraduate students from 39 courses at a 4-year university participated in this study. The mean age of participating students is 18.93 year old, with 30.44% of the study sample being freshmen (year-1 students) at age 17 or below, 67.1% being sophomores between age 18-23 or below, and 2.46% being mature students age 24 or above. Figure 2 illustrated the distribution of participants’ age. Figure 2: Age of participating students www.ejel.org 434 ©ACPIL Kevin Chan, George Cheung, Kelvin Wan et al Students in this sample represented a balanced mix of academic disciplines, with business and health / social sciences students accounting for 60% of the study population while science / technology / engineering (STE) students (19%), tourism & hospitality students (16%), and humanities / design students (5%) rounded up the remaining 40% of the study sample. Figure 3 depicted distribution of students’ disciplines. Figure 3: Disciplines of participating students Structural model and path coefficients of learning technology use, deep learning, and learning outcomes Figure 4: Structural Equation Model of UTAUT constructs, behavioral intention and satisfaction towards learning technology and R-SPQ-2F deep learning approach on NSSE higher-order learning and collaborative learning www.ejel.org 435 ISSN 1479-439X The Electronic Journal of e-Learning Volume 13 Issue 6 2015 The proposed structural model is illustrated in Figure 4, where Table 2 presents correlation among model variables and Table 3 highlights the critical ratios of exogenous (predictor) variables included in the research model. Table 4 presents the path coefficients and corresponding statistical significance for each of the hypotheses. U(cid:393)(cid:381)(cid:374) (cid:296)(cid:349)(cid:410)(cid:410)(cid:349)(cid:374)(cid:336) (cid:410)(cid:346)(cid:286) (cid:400)(cid:410)(cid:437)(cid:282)(cid:455) (cid:282)(cid:258)(cid:410)(cid:258)(cid:853) (cid:396)(cid:286)(cid:400)(cid:437)(cid:367)(cid:410)(cid:400) (cid:381)(cid:296) (cid:410)(cid:346)(cid:286) (cid:393)(cid:396)(cid:381)(cid:393)(cid:381)(cid:400)(cid:286)(cid:282) (cid:396)(cid:286)(cid:400)(cid:286)(cid:258)(cid:396)(cid:272)(cid:346) (cid:373)(cid:381)(cid:282)(cid:286)(cid:367) (cid:286)(cid:454)(cid:346)(cid:349)(cid:271)(cid:349)(cid:410)(cid:286)(cid:282) (cid:258) (cid:336)(cid:381)(cid:381)(cid:282) (cid:296)(cid:349)(cid:410)(cid:855) (cid:894)(cid:662)(cid:1006) (cid:1089) (cid:1007)(cid:1005)(cid:1006)(cid:1010)(cid:856)(cid:1005)(cid:1004)(cid:1008)(cid:853) (cid:282)(cid:296) (cid:1089) (cid:1010)(cid:1012)(cid:1006)(cid:853) (cid:662)(cid:1006)(cid:876)(cid:282)(cid:296) (cid:1089) (cid:1008)(cid:856)(cid:1009)(cid:1012)(cid:1008)(cid:853) GFI (cid:1089)(cid:1004)(cid:856)(cid:1012)(cid:1013)(cid:1005)(cid:853) TLI (cid:1089) (cid:1004)(cid:856)(cid:1013)(cid:1007)(cid:1008)(cid:853) CFI (cid:1089) (cid:1004)(cid:856)(cid:1013)(cid:1007)(cid:1013)(cid:853) IFI (cid:1089) (cid:1004)(cid:856)(cid:1013)(cid:1007)(cid:1013)(cid:853) RMSEA (cid:1089) (cid:1004)(cid:856)(cid:1004)(cid:1008)(cid:1011)(cid:853) SRMR(cid:1089) (cid:1004)(cid:856)(cid:1004)(cid:1007)(cid:1012)(cid:895)(cid:856) O(cid:448)erall, five out of seven hypotheses were supported by the data. Two out of four hypotheses (H1 and H2) representing the relationship among the main UTAUT constructs (PE, EE) to S/BI were supported in this study. As shown in Table 4, performance expectancy (PE) positively predicted satisfaction and behavioral intention (0.415, p < 0.001); therefore, H1 was supported. Similarly, effort expectancy (EE) significantly predicted satisfaction and behavioral intention (0.258, p < 0.001); therefore, H2 was supported. The hypotheses that were not supported was H3: SI to S/BI and H4: FC to S/BI. Social Influence (SI) (2.644, (cid:393)(cid:1089)(cid:856)(cid:1004)(cid:1004)(cid:1012)) and Facilitating Condition (FC) (1.630, (cid:393)(cid:1089)(cid:856)(cid:1005)(cid:1004)(cid:1007)) did not significantly predict satisfaction and behavioral intention to continual use of SRS; therefore, H3 and H4 were not supported. Satisfaction and behavioral intention to continue use of SRS (S/BI) significantly predict deep learning approach (Deep) (0.287, p < 0.001); therefore, H5 was supported. Regarding learning outcomes, deep learning approach (Deep) positively predicted usage higher-order thinking and learning (NSSE-HO) (0.523, p < 0.001) and collaborative learning (CL) (0.153, p < 0.001); thus H6 and H7 were supported. Table 2: Correlation coefficient of model variables EE SI FC Satisfaction Deep HO CL Age .690** .729** .676** .783** .468** .226** .184** -.134** PE .637** .744** .738** .387** .254** .182** -.103** EE .711** .713** .463** .239** .223** -.083** SI .726** .421** .307** .240** -.100** FC .420** .231** .182** -.117** Satisfaction .386** .283** -.053* Deep .428** -.017 HO .037 CL PE: Perceived ease of use; EE: Effort Expectance; SI: Social influence; FC: Facilitating Conditions; Deep: Deep approach; HO: Higher Order Thinking; CL: Collaborative Learning; **p < 0.01; *p <0.05 www.ejel.org 436 ©ACPIL Kevin Chan, George Cheung, Kelvin Wan et al Table 3: Standard error and critical ratios for each of the parameters Estimate S.E. C.R. P Label 0.452 0.023 20.034 PE < .001 par_52 0.531 0.024 21.744 EE < .001 par_53 0.405 0.02 19.937 SI < .001 par_54 0.316 0.019 16.922 FC < .001 par_55 6.714 0.236 28.478 Age < .001 par_56 Table 4: Structural equation model path coefficient results Standardized path coefficient Path (Hypothesis) (Beta) t-value Hypothesis testing result 0.415 H1 S/BI <-- PE 8.839 *** Supported 0.258 H2 S/BI <-- EE 6.555*** Supported 0.249 H3 S/BI <-- SI 2.644n.s. Not supported 0.16 H4 S/BI <-- FC 1.630 n.s. Not supported 0.287 H5 S/BI <-- Deep 16.018*** Supported 0.523 H6 NSSE-HO <-- Deep 14.466*** Supported 0.153 H7 NSSE-CL <-- Deep 5.014*** Supported M(cid:381)(cid:282)(cid:286)(cid:367) (cid:296)(cid:349)(cid:410) (cid:349)(cid:374)(cid:282)(cid:349)(cid:272)(cid:286)(cid:400)(cid:855) (cid:662)(cid:1006) (cid:1089) (cid:1007)(cid:1005)(cid:1006)(cid:1012)(cid:856)(cid:1009)(cid:1012)(cid:1011)(cid:853) (cid:282)(cid:296) (cid:1089) (cid:1010)(cid:1012)(cid:1007)(cid:853) (cid:662)(cid:1006)(cid:876)(cid:282)(cid:296) (cid:1089) (cid:1008)(cid:856)(cid:1009)(cid:1012)(cid:1005)(cid:853) GFI (cid:1089)(cid:1004)(cid:856) (cid:1012)(cid:1013)(cid:1005)(cid:853) TLI (cid:1089) (cid:1004)(cid:856) (cid:1013)(cid:1007)(cid:1008)(cid:853) CFI (cid:1089) (cid:1004)(cid:856) (cid:1013)(cid:1007)(cid:1013)(cid:853) IFI (cid:1089) (cid:1004)(cid:856) (cid:1013)(cid:1007)(cid:1013)(cid:853) RMSEA (cid:1089) (cid:1004)(cid:856) (cid:1004)(cid:1008)(cid:1011)(cid:853) SRMR(cid:1089) (cid:1004)(cid:856) (cid:1004)(cid:1007)(cid:1012)(cid:854) (cid:910)(cid:910)(cid:910)p < 0.001; n.s. Not significant. 5 Discussion and implications A model of 3 latent factors, including two factors from UTAUT in Performance Expectancy and Effort Expectancy along with Deep Learning Approach from the SPQ, were identified in the structural model on students’ learning outcomes upon adopting SRS in classes. Hypotheses H1 & H2 were statistically significant, indicating evidence for substantial influence on behavioral intention to adopt education technology based on perceived performance and effort expectancies. Particularly, the observed effect size of performance expectancy with standardized coefficient of 0.415 in the research model was substantially larger than average coefficient estimated in a recent meta-analysis on UTAUT of 0.343 (Dwivedi et al., 2011). Performance expectancy may play a significant role in adoption of education technology towards learning goals in the Asian culture. It has been suggested that Asian, particularly Chinese students, tend to value the avoidance of poor performance and extrinsic motivation (D’Lima et al., 2014) high in formulating their learning goal orientation. If adoption of certain educational technology could significantly improve on their performance and avoid failure, it is likely that students would value performance expectancy in making decision to adopt technology for learning over other factors. Hypotheses H3 & H4 did not reach statistical significance in the current research model. While Social Influence (SI) and Facilitating Conditions (FC) did not predict satisfaction and behavioral intention, the overall model nonetheless suggested linkage between readiness towards SRS and tendency to engage in deep learning, which in turn positively predict higher-order thinking and learning and collaborative learning with peers. www.ejel.org 437 ISSN 1479-439X The Electronic Journal of e-Learning Volume 13 Issue 6 2015 Though regarded as one of the more significant constructs in UTAUT, the contribution of SI is subjected to further investigations since it is sometimes omitted in systematic review and meta-analysis because of its context and cultural specificity (Lidia et al., 2007). Indeed, non-significant path between SI and other model variables parallels findings about the inconsistent relationship of this construct with the overall model in the technology adoption literature (Lee et al., 2003) in general. Exclusion of SI in the UTAUT-based research model has also been resonated in a similar non-significant finding from a study population of Asian origin (Koh et al., 2010). Further investigation of cross-cultural variation of this dimension is required to delineate the role of social influence in education technology adoption in the Asian context, particularly when level of individualism and avoidance of uncertainty that vary significantly across cultures (Nistor et al., 2014). In the higher education context, findings on the role of facilitating conditions (FC) in UTAUT have been mixed – from key correlates to effort expectancy (Terzis and Economides, 2011)] to non-significant factor excluded from the research model (Al-Hujran et al., 2014). A possible explanation to the non-significant path observed in the current study is that the participating classes using SRS were part of an ongoing e-learning project that provides centralized support and training on developing and delivering SRS questions in classes. Further analyses are required to examine whether uniformity in technical and user support explains the lack of variation in terms of facilitating conditions towards technology adoption. Hypotheses 5-7 were supported in the research model presented. Findings from the current study confirms the effect of SRS on encouraging complex cognitive effort (Brady et al., 2013a) and collaborative learning (Jones et al., 2012, Blasco-Arcas et al., 2013), while bridging these learning outcomes via activation of deep learning approach being facilitated by satisfaction and intention to use SRS. Soliciting students’ responses in learning environment has been suggested to facilitate retrieval-based learning (Karpicke and Grimaldi, 2012, Campbell and Mayer, 2009) Through retrieval of recently acquired information, students consolidate their acquired information in the higher education setting with active learning effort and timely feedback (Lantz and Stawiski, 2014). The student-initiated effort in SRS also resonates with the generation effect (Hirshman and Bjork, 1988, Lantz, 2010), a theory in cognitive and learning psychology hypothesizing better memorization and retention in learning when individuals are required to generate learning artifacts rather than simply receiving and encoding information. These cognitive processes in a learning context possibly account for the demonstrated association between deep learning approach and readiness to SRS demonstrated in this study. The presented research model highlights the long-term impact of using SRS in classes beyond immediate gains such as students’ attention and class engagement (Velasco and Çavdar, 2013, Vaterlaus et al., 2012, Kay and LeSage, 2009, Morling et al., 2008). Appropriate and effective use of SRS should be regarded as an integral mean to achieve deep learning strategies towards enriched, applied, and collaborative learning, which are considered high-impact practices in higher education in the 21st century (Kilgo et al., 2014). During SRS session, the level of students monitoring their cognitive effort could vary substantially, leading to corresponding variation in learning outcome through SRS. Recent studies suggested that SRS questions and subsequent students’ responses effect moderately on heightening students’ metacognitive effort by verifying their knowledge mastery with the responses presented and discussed in classes (Brady et al., 2013b, Brady et al., 2013a);Schell, 2013 #1066]. The current model could possibly address students’ varying level of metacognitive activities during SRS sessions with deep learning approach, an orientation calling for metacognition and mastery of knowledge acquisition, partially contributing to students’ motivation and effort to think through the questions presented towards a reasoned response. 5.1 Limitations of the current study Learning approach and attitude towards acceptance and use of technology are limited and subjected to recall and social desirability biases by self-reported items from survey. In delineating the moderating effect of learning approach between intention and behavior, further inquiries through qualitative data in a mixed method design would provide more accurate explanations on the findings currently presented in this paper. Implication for learning with technology in higher education: Findings from the present study suggested that using SRS, when students are properly motivated with its utility on learning performance, could influence deep learning approach that in turn predicts higher-order www.ejel.org 438 ©ACPIL Kevin Chan, George Cheung, Kelvin Wan et al learning activities such as higher-order thinking and collaborative learning. Teachers using SRS to facilitate deep learning could focus on SRS questions that are deemed fitting to such learning approach, such as critical thinking or application questions (Bruff, 2009) that optimize stimulations in their students towards higher- order learning. References Al-Hujran, O., Al-Lozi, E. & Al-Debei, M. M. (2014) "Get Ready to Mobile Learning”: Examining Factors Affecting College Students' Behavioral Intentions to Use M-Learning in Saudi Arabia", Jordan Journal of Business Administration, 10, pp 111-128. Al-Huneidi, A. & Schreurs, J. (2013) Constructivism Based Blended Learning in Higher Education, In: LYTRAS, M., RUAN, D., TENNYSON, R., ORDONEZ DE PABLOS, P., GARCÍA PEÑALVO, F. & RUSU, L. (eds.) Information Systems, E-learning, and Knowledge Management Research. Springer Berlin Heidelberg. Biggs, J., Kember, D. & Leung, D. Y. P. (2001) "The revised two-factor Study Process Questionnaire: R-SPQ-2F", British Journal of Educational Psychology, 71, pp 133-149. Blasco-Arcas, L., Buil, I., Hernández-Ortega, B. & Sese, F. J. (2013) "Using clickers in class. The role of interactivity, active collaborative learning and engagement in learning performance", Computers & Education, 62, pp 102-110. Bollen, K. A. (1989) Structural equations with latent variables, New York, NY, Wiley. Brady, M., Seli, H. & Rosenthal, J. (2013a) "“Clickers” and metacognition: A quasi-experimental comparative study about metacognitive self-regulation and use of electronic feedback devices", Computers & Education, 65, pp 56-63. Brady, M., Seli, H. & Rosenthal, J. (2013b) "Metacognition and the influence of polling systems: How do clickers compare with low technology systems", Educational Technology Research and Development, 61, pp 885-902. Bruff, D. (2009) Teaching with classroom response systems: Creating active learning environments, San Francisco, CA, Jossey-Bass. Caldwell, J. E. (2007) "Clickers in the Large Classroom: Current Research and Best-Practice Tips", CBE-Life Sciences Education, 6, pp 9-20. Campbell, J. & Mayer, R. E. (2009) "Questioning as an instructional method: Does it affect learning from lectures?", Applied Cognitive Psychology, 23, pp 747-759. Chan, K., Brown, I., Chung, I. C. B., Lu, H.-J. & Luk, G. W.-T. (Year) Published. Using Students Response System via Mobile Devices in Large Introductory Psychology Classes. In: IVALA, E., ed. International Conference on eLearning, June 27- 28, 2013 2013 Cape Peninsula University of Technology, Cape Town, South Africa. Academic Conferences and Publishing International Limited, 539. Cobb, S., Heaney, R., Corcrcoran, O. & Henderson-Begg, S. (2010) "Using Mobile Phones to Increase Classroom Interaction", Journal of Educational Multimedia and Hypermedia, pp. D’lima, G. M., Winsler, A. & Kitsantas, A. (2014) "Ethnic and Gender Differences in First-Year College Students’ Goal Orientation, Self-Efficacy, and Extrinsic and Intrinsic Motivation", The Journal of Educational Research, 107, pp 341- 356. Davis, F. D., Bagozzi, R. P. & Warshaw, P. R. (1989) "User Acceptance of Computer Technology: A Comparison of Two Theoretical Models", Management Science, 35, pp 982-1003. Dwivedi, Y. K., Rana, N. P., Chen, H. & Williams, M. D. (2011) A Meta-analysis of the Unified Theory of Acceptance and Use of Technology (UTAUT), Governance and Sustainability in Information Systems. Managing the Transfer and Diffusion of IT. Springer. Exeter, D. J., Ameratunga, S., Ratima, M., Morton, S., Dickson, M., Hsu, D. & Jackson, R. (2010) "Student engagement in very large classes: the teachers' perspective", Studies in Higher Education, 35, pp 761-775. Hirshman, E. & Bjork, R. A. (1988) "The generation effect: Support for a two-factor theory", Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, pp 484-494. Hu, L. T. & Bentler, P. M. (1999) "Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives", Structural Equation Modeling: A Multidisciplinary Journal, 6, pp 1-55. Indiana University Center for Postsecondary Research. 2013. National Survey of Student Engagement (NSSE) 2013 Codebook. Available: http://nsse.iub.edu/2013_Institutional_Report/data_codebooks/NSSE 2013 Codebook.pdf. Jones, M. E., Antonenko, P. D. & Greenwood, C. M. (2012) "The impact of collaborative and individualized student response system strategies on learner motivation, metacognition, and knowledge transfer", Journal of Computer Assisted Learning, 28, pp 477-487. Justicia, F., Pichardo, M. C., Cano, F., Berbén, A. B. G. & De La Fuente, J. (2008) "The Revised Two-Factor Study Process Questionnaire (R-SPQ-2F): Exploratory and confirmatory factor analyses at item level", European Journal of Psychology of Education, 23, pp 355-372. Karpicke, J. & Grimaldi, P. (2012) "Retrieval-Based Learning: A Perspective for Enhancing Meaningful Learning", Educational Psychology Review, Online First, pp 1-18. Kay, R. H. & Lesage, A. (2009) "Examining the benefits and challenges of using audience response systems: A review of the literature", Computers & Education, 53, pp 819-827. Kerr, A. (2011) Teaching and Learning in Large Classes at Ontario Universities: An Exploratory Study., Toronto, Higher Education Quality Council of Ontario. www.ejel.org 439 ISSN 1479-439X The Electronic Journal of e-Learning Volume 13 Issue 6 2015 Kilgo, C., Ezell Sheets, J. & Pascarella, E. (2014) "The link between high-impact practices and student learning: some longitudinal evidence", Higher Education, pp 1-17. King, W. R. & He, J. (2006) "A meta-analysis of the technology acceptance model", Information & Management, 43, pp 740- 755. Koh, C. E., Prybutok, V. R., Ryan, S. D. & Wu, Y. (2010) "A Model for Mandatory Use of Software Technologies: An Integrative Approach by Applying Multiple Levels of Abstraction of Informing Science", Informing Science, 13, pp. Lantz, M. E. (2010) "The use of 'Clickers' in the classroom: Teaching innovation or merely an amusing novelty?", Computers in Human Behavior, 26, pp 556-561. Lantz, M. E. & Stawiski, A. (2014) "Effectiveness of clickers: Effect of feedback and the timing of questions on learning", Computers in Human Behavior, 31, pp 280-286. Lee, Y., Kozar, K. A. & Larsen, K. R. T. (2003) "The technology acceptance model: Past, present, and future", Communications of the Association for Information Systems, 12, pp 752-780. Lidia, O., Paul, C. & Harold, T. (2007). Validating the unified theory of acceptance and use of technology (UTAUT) tool cross- culturally. Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI...but not as we know it - Volume 2. University of Lancaster, United Kingdom: British Computer Society. Morling, B., Mcauliffe, M., Cohen, L. & Dilorenzo, T. M. (2008) "Efficacy of Personal Response Systems ('Clickers') in Large, Introductory Psychology Classes", Teaching of Psychology, 35, pp 45-50. Nistor, N., Lerche, T., Weinberger, A., Ceobanu, C. & Heymann, O. (2014) "Towards the integration of culture into the Unified Theory of Acceptance and Use of Technology", British Journal of Educational Technology, 45, pp 36-55. Schell, J., Lukoff, B. & Mazur, E. (2013) Catalyzing Learner Engagement Using Cutting-Edge Classroom Response Systems in Higher Education, In: CHARLES, W. (ed.) Increasing Student Engagement and Retention Using Classroom Technologies Classroom Response Systems and Mediated Discourse Technologies. Bingley: Emerald. Shapiro, A. M. (2009) "An Empirical Study of Personal Response Technology for Improving Attendance and Learning in a Large Class", Journal of scholarship of teaching and learning, 9, pp 13-26. Slavich, G. & Zimbardo, P. (2012) "Transformational Teaching: Theoretical Underpinnings, Basic Principles, and Core Methods", Educational Psychology Review, 24, pp 569-608. Terzis, V. & Economides, A. A. (2011) "The acceptance and use of computer based assessment", Computers & Education, 56, pp 1032-1044. Tlhoaele, M., Hofman, A., Winnips, K. & Beetsma, Y. (2015) "Exploring the Relationship Between Factors That Contribute to Interactive Engagement and Academic Performance", Journal of Education and Training, 2, pp 61-80. Vaterlaus, J. M., Beckert, T. E., Fauth, E. & Teemant, B. (2012) "An Examination of the Influence of Clicker Technology on College Student Involvement and Recall", International Journal of Teaching and Learning in Higher Education, 24, pp 293–300. Velasco, M. & Çavdar, G. (2013) "Teaching Large Classes with Clickers: Results from a Teaching Experiment in Comparative Politics", PS: Political Science & Politics, 46, pp 823-829. Venkatesh, V. & Bala, H. (2008) "Technology Acceptance Model 3 and a Research Agenda on Interventions", Decision Sciences, 39, pp 273-315. Venkatesh, V., Morris, M. G., Gordon, B. D. & Davis, F. D. (2003) "User Acceptance of Information Technology: Toward a Unified View", MIS Quarterly, 27, pp 425-478. Wang, X., Su, Y., Cheung, S., Wong, E. & Kwong, T. (2012) "An exploration of Biggs’ constructive alignment in course design and its impact on students’ learning approaches", Assessment & Evaluation in Higher Education, 38, pp 477-491. www.ejel.org 440 ©ACPIL