International Journal of Education and Development using Information and Communication Technology (IJEDICT), 2015, Vol. 11, Issue 1, pp. 101-121 MLCMS actual use, perceived use, and experiences of use Edgar Napoleon Asiimwe and Åke Grönlund Örebro University, Sweden ABSTRACT Mobile learning involves use of mobile devices to participate in learning activities. Most e- learning activities are available to participants through learning systems such as learning content management systems (LCMS). Due to certain challenges, LCMS are not equally accessible on all mobile devices. This study investigates actual use, perceived usefulness and user experiences of LCMS use on mobile phones at Makerere University in Uganda. The study identifies challenges pertaining to use and discusses how to improve LCMS use on mobile phones. Such solutions are a cornerstone in enabling and improving mobile learning. Data was collected by means of focus group discussions, an online survey designed based on the Technology Acceptance Model (TAM), and LCMS log files of user activities. Data was collected from two courses where Moodle was used as a learning platform. The results indicate positive attitudes towards use of LCMS on phones but also huge challenges which are content related and technical in nature. Keywords: Mobile learning; LCMS; MUELE; TAM; Mobile phones INTRODUCTION Information and communications technology (ICT) mediated learning has increasingly become important in higher education (Simkova et al. 2012; Fu 2013). Electronic learning tools, especially online tools, allow teachers and learners to share educational resources, work on assessments, communicate and collaborate smoothly (Lonn et al. 2011; Lonn et al. 2009; Liaw et al. 2008). Increasingly, mobile technologies are being used for ubiquitous access in learning. There are various meanings of mobile learning. Tagoe and Abakah (2014) demonstrate how mobile learning has been defined over time and further shows that, some of the definitions are technology oriented, e-learning oriented, location oriented, or learner- centered, and are contextualized based on social and cultural perspectives. Wang et al. (2009) takes a technology stand to define mobile learning as: “the delivery of learning to students anytime and anywhere through the use of wireless Internet and mobile devices” (p. 92). Considering the perspectives in this paper, the above definition was expanded to encompass the divergent views on mobile learning that are cited in Tagoe and Abakah (2014), we use “mobile learning” to mean the process of exchanging and acquiring knowledge, and delivering learning instructions and content to students through the use of wireless Internet, mobile devices, web and mobile applications. The backbone for mobile learning includes a mobile communication infrastructure and mobile devices such as cell phones which can support technologies that assist individuals and groups to learn anywhere anytime (Sharples et al. 2002). Johnson et al. (2011) name mobile devices as a priority technology for next generation learning and note that they: “enable ubiquitous access to information, social networks, tools for learning and productivity...are capable computing devices in their own right — and they are increasingly a user’s first choice for Internet access” (p. 5). 102 IJEDICT   Mobile learning uses supporting applications such as Mobile Learning Content Management Systems (MLCMS). Such applications provide simplicity in content management, and ensure proper display and functionality for various mobile devices to enable efficiency in data transmission (Simkova et al. 2012). Gleason (2002) and Mohmoud (2008) suggest that learning system components should include MLCMS that support downloading and managing repositories for mobile content. Besides other m-learning application solutions available, MLCMS solve device constraints such as size that limit content access. Attractive factors of mobile devices include mobility and portability that provide the capability to carry or move the devices easily. Sariola et al. (2001) describe mobile learning from technological and educational theorist perspectives and note that mobility is the most interesting aspect since matters of who is moving (tutor or learner), where they are moving (environment) and why they are moving are important in understanding the context of learning. The mobility factor, for example, comes along with convenience, faster communication, flexibility and full time connectivity (Ducut & Fontelo 2008). Alvarez et al. (2011) note that such opportunities have made mobile learning attractive to educational institutions. Therefore, in terms of flexibility, collaboration and communication, mobile technologies can play a critical ‘freedom of choice’ role regarding how and where to learn, which is core in distance education (Parsons 2009). However, there is a need to re-conceptualize learning for the mobile age through understanding the essential role of mobility and communication in the learning process (Sharples et al. 2005). Understanding the importance of context in establishing meaning and supporting virtual communities that transcend barriers of age and culture is equally important. In pursuit of the “anywhere and anytime” ideal, different researchers have investigated issues related to mobile learning. Most areas addressed in the literature are problems of use, access, design and infrastructure (Westera 2011; Chu et al. 2005), communication and collaboration (Alvarez et al. 2011), content delivery (Macdonald & Chiu 2011), and many more. However none of these researchers discuss the technical aspects of MLCMS. Research either discusses mobile phone use in learning or learning management systems separately, but not together (Asiimwe & Grönlund 2014). We define MLCMS as LCMS that can store and deliver learning content and services to mobile computing devices. The aim is to identify ways of adapting LCMS services for mobile phone users. This aim is pursued by: • Studying actual use, perceived ease of use and usefulness of MLCMS (mobile LCMS) • Investigating challenges involved in use of MLCMS and suggest remedies The main research questions of this study are: • What are user perceptions and actual experiences of MLCMS use on mobile phones? • What affects MLCMS use? There are various theories that discuss use of technology. This study uses TAM (Technology Acceptance Model) as a reference model (Davis et al. 1989; Venkatesh et al. 2000). TAM helps to explain perceived usefulness and usage intentions of an information system. There are several models that have been created for analyzing the relationship between technology and users; TAM is one of the oldest and most used. While models differ in details and scope, they all in some way or another draw on the idea that ease of use and usefulness, as perceived by the user, are the basic factors that lead to use of information systems. We therefore used the TAM general framework to formulate our research instruments. MLCMS actual use, perceived use, and experiences of use 103   Factors affecting actual use of learning technology Information systems research that discusses technology adoption and acceptance e.g., Davis et al (1989), retain that perceived ease of use and perceived usefulness determine use. Task performance is stimulated when a system is easy to use; at the same time for the user to be at all interested in using it, s/he must see some point in doing so – the system must be perceived as being potentially useful. Perceived ease of use is further linked to intentions to use (Venkatesh 1999). The linkage is both direct and indirect via its impact on perceived usefulness (Venkatesh & Davis 2000, p.192). The TAM framework focuses on particular aspects, which in this paper are referred to as “TAM keywords” i.e., “behavior intentions,” “attitudes,” “usefulness,” “ease of use” etc., all seen in the context of a “system,” an information system. From a perspective of learning and pedagogy, this focus of user-to-system may be criticized as many learning studies suggest a rather different direction i.e., focusing on the learner (Ramsden 2003; Light 2001). Although some of the contemporary Information system studies have re-constructed TAM, it should be remembered that TAM was constructed in the 1980s when computer use was very different from now. Then, most use was professional and task-oriented; today computer use is more open; a palette of tools is available, the user often has a choice, and many design features supporting ease-of-use are incorporated in industry standards as well as in the thinking and experience of users. Even so, any new technology requires revisiting the interaction between users and technology as the preconditions change. Mobile technologies are very much an example of such change. While many general functions of mobile technology are already well established, many specialized ones are not. One of the yet unexplored functions is the integration of mobile technologies in teaching and learning environments and processes. For this reason we revisit the TAM factors in the context of m-learning. User experience of information and communication technology (ICT) is an enabling factor for continuous use of ICT (Liu, et al. 2010). Past online learning experience, for example, shapes perceived interaction and perceived usefulness of online learning programs which subsequently motivates intentions for using online learning resources, thus, “the greater the online learning experiences of users, the stronger their intention to use an online learning community” (p.603). Experience is also mentioned as an empowerment tool in terms of enjoyment and concentration during learning discourse. Learning requires a focused and attentive mind driven by interest – what Csikszentmihalyi (1997) describes as a “flow state”; a feeling of complete involvement in an activity. This learning state of mind can be affected by user skills and ambitions as well as by perceptions of ease of use and usefulness of the system. Faith in ICT efficacy is a significant factor shaping intentions to integrate technology in learning and teaching (So et al. 2012; Fanni et al. 2013). ICTs emerge as effective, efficient and productive tools for supporting the performance of a variety of tasks, and this perception can be improved by training (Fanni et al. 2013). Ming-Chi Lee (2010) empirically validates the hypothesis that confirmation of expectations of users is positively related to perceived usefulness of e-learning tools. ICT efficacy raises expectations and when expectations are met it leads to positive learner experiences and satisfaction. Empirical studies by Sun, et al. (2008) and Lee & Lehto (2013) show a positive relationship between perceived usefulness and user satisfaction on electronic learning. Conversely, unsatisfactory perceptions hamper students’ motivation. User satisfaction has, “...a direct impact on the formation of behavioral intention. In educational settings, it is considered a prerequisite for the users’ intent to use a learning system” (Lee & Lehto 2013, p.195). 104 IJEDICT   Thus behavioral intentions or attitude shapes perceived usefulness and ease of use (Venkatesh et al. 2003) leading to increased ICT efficacy. There are also other factors that affect ease of use and usefulness, including good interface design, good content design, and technical support (Cheung & Vogel 2013). Content and interface design affect learners’ perceptions, particularly mobile learners as mobile systems introduce more restrictions to the design. User Interface Design (UID) is an important factor in computer applications development (Liu et al. 2010). Good UID enforces compatibility across different devices. Compatibility has an “influence on ease of use associated with a new technology” (Cheung & Vogel 2013, p.165). All in all, both system and content design affect users’ perception towards technology acceptance and use. MLCMS technology: impact and challenges Several advantages and challenges of mobile phone use with LCMS are discussed in the literature. An empirical study on course content distribution using mobile technology by Mohmoud (2008) used a case to show how access to online learning resources via mobile phones is a preferred learning solution, but notes that the solution requires fast Internet connections and must be affordable. Mohmoud also notes that mobile technology is “the most complex solution” (p. 281) since a website has to be designed for different screen layouts and file formats. Parsons (2009) categorizes challenges of using mobile devices into three fields: • specification and usability i.e. qualities of the device such as screen size, battery life, storage space, flash application capabilities etc.; • lifecycle of the devices; and • diversity and lack of standards. The lifecycle of the device refers to the continuous development of new devices that leads to demand of responsive applications, which is challenging in that content creation is also affected and new requirements must be met. This rapid process of making changes, however, affects the ability to create and adhere to standards and may or may not prompt learning content creators and e-learning website designers to follow standards and instructional design guidelines. Casany et al. (2012b) mention challenges such as lack of teacher confidence and training on technology use and technical difficulties with mobile devices which affect the attitudes towards use. These limitations can be overcome by user training and providing supporting information in the form of a manual. Mobile learning also faces challenges with integrating mobile applications with mainstream e- learning applications. Casany et al. (2012b) suggest that these challenges can be overcome by integration of learning content management systems. This integration can facilitate interoperability improvements across various devices. However, Casany et al. (2012a) note that integrating external m-learning applications into the learning content management systems is a disadvantage due to difficulties in maintaining and extending the integrated external systems. Despite the challenges, literature suggests the existing challenges are contemporary and can be overcome given constant advancements in technology. Thus, MLCMS remain necessary tools for e-learners due to their positive contribution towards learning performance and collaboration. MLCMS actual use, perceived use, and experiences of use 105   THE MUELE CASE MUELE is an online learning management system used as the default e-learning platform at Makerere University (http://muele.mak.ac.ug/). MUELE is customized based on Moodle (Modular Object-Oriented Dynamic Learning Environment). Moodle is an open source learning management system (LMS) developed and supported by the Moodle Project (http://moodle.org). MUELE provides tools to manage and support learning in a virtual environment. Functions of the system include: learners’ activity reporting, creation of online quizzes, content/learning material management, chat rooms, discussion forum, wikis, communication (e-mailing), course creation and management and user management (teachers, students and administrators). MUELE was set up at Makerere University in 2009 because it is open source and hence served to avoid license costs that were incurred on the LMS that was previously in use (Blackboard; blackboard.com). The main purpose of having an LMS is to facilitate e-learning. Most users are students and teachers at all university campuses. The system is hosted and managed locally by DICTS (Directorate of Information and Communications Technology Support; http://dicts.mak.ac.ug/). DICTS is responsible for ICT implementation and support services at Makerere campus. Use and implementation of MUELE is an ongoing activity with no specified timeframe, and the implementation is monitored and evaluated by DICTS through performance and system usage reports. The system is updated regularly in accordance with Moodle updates. The university has 145 undergraduate programmes and 139 postgraduate ones. The estimated number of MUELE registered users is 53,000 but the actual (active) number of users was 30,000 as of April 2014. We conducted an information search on the university intranet and webpages and found that there was no information for students and teachers on how to use MUELE on mobile devices specifically mobile phones. The information was created for desktop users. User support is given when requested. Training on how to use the system is provided for teachers only. Within the system settings, different display templates have been installed to support information access across various devices, but not all devices are supported. Besides perceptions of use, this study took the MUELE case to investigate challenges faced by users so that we could find solutions for mobile users. METHOD This study used focus group discussions (FGD) and an online questionnaire as the primary methods to collect data. We further examined activity logs of participants which were extracted from the learning platform. Informants in both surveys were students and teachers. The informants were divided in three focus groups and handed the same questions (in appendix B). The groups discussed the questions and wrote down their shared views as guided by the facilitator (one of the researchers). After 90 minutes, the three groups convened for 60 minutes to share and discuss their answers to the questions. Answers from each group were recorded by the appointed group leader and answers from all groups were recorded by one of us (the researchers). After the focus group discussions, a link to the online survey (appendix A) was sent to all participants. System logs covering six months of user activity were reviewed. A descriptive analysis of the data collected was made, and then data from the three sources – focus groups, survey and log files were contrasted and compared. 106 IJEDICT   Demographics Survey data was collected from twenty-eight students and two teachers. Three respondents were females, 27 males. Respondents were students and teachers of two particular Information Technology (IT) courses offered during the 2013 fall semester at Makerere University main campus. These IT courses are offered to second year students in the Bachelor of Information Technology programme. The courses include BIS2104 (Introduction to Database Systems with 550 students) and BIT2108 (Advanced Information Technology with 1320 students). The courses run for a full academic semester which is six months. Respondents were between age 20 and 34 and had experience of using Makerere University Electronic Learning Environment (MUELE) on mobile phones. Table one shows the number of respondents in the online questionnaire. The survey link was sent to all 30 respondents. Reminders to fill in the questionnaire were sent to all 30 respondents and eventually 23 (77%) responded. Table 1: Gender and age groups of online respondents (n=23) Age Group 18-24 25-29 30-34 35++ Gender Female 2 0 0 0 Male 18 0 3 0 Selection of respondents Students and teachers in BIS2104 and BIT2108 courses were invited to participate in the survey. Teachers were included in the study because they had previously taken the same courses as students and had used the same learning platform during their studies; they thus had their individual experiences with the system as previous students despite their current teacher roles. One requirement for participation was having a mobile phone (of any kind) that could access the Internet. Those who did not have mobile phones that could access the Internet were excluded. Many students were interested in taking part in the survey but were excluded by this criterion, which led to a sample of twenty-eight respondents. Participants who met the criterion were registered and briefed on the aim of the research and on how to access and use MUELE on their mobile phones. Among the selected participants, some had smart phones while others had semi-smart phones (mobile phones with basic functions and Internet capabilities). Data collection Data were collected from FGDs and an online survey. The FGDs included thirty respondents who were divided in three groups, each with 11, 10 and 9 respondents respectively. The FGDs lasted for 90 minutes in each group. The web link to the online questionnaire (Appendix A) was sent to everyone who participated in FGDs. Twenty-three out of thirty participants responded as shown in Table 1. We further examined respondents’ activity logs (Figure 1) mined from MUELE. The purpose of examining activity logs was to identify what kind of information and tools the respondents accessed. For example, did they access and use the discussion forums, chat rooms, web mail, assignments, course content, etc.? MLCMS actual use, perceived use, and experiences of use 107   Figure 1: Sample respondents’ activity logs. Internet Protocol (IP) addresses and usernames are hidden for ethical purposes Data analysis This paper uses descriptive analysis. Respondents’ opinions and some of the TAM factors are used to analyze correspondences in opinions regarding perceived ease of use and usefulness. Descriptive analysis interprets information patterns that might emerge from data and summarizes the findings in a meaningful way. The descriptive analysis was used mainly because most of the data was qualitative. The comparisons of opinions from the online questions were compared to the views from FGDs so as to serve as a measure of triangulation (using different methods to obtain data on the same phenomenon). The frame of reference for the study was the Technology Acceptance Model (Davis et al. 1989; Venkatesh & Davis 2000), which has been used widely by information systems researchers to explain factors that lead to acceptance of information systems (Lin & Fang 2011). The model includes six essential factors; (1) external variables such as demographic ones; (2) perceived usefulness (personal belief that a system will enhance a task performance); (3) perceived ease of use (personal belief that a system will be simple to operate); (4) attitudes towards use (personal desires to use the system) which are solely determined by perceived usefulness and perceived ease of use, and significantly affects behavioral intention (Thomas 2013); (5) behavioral intention to use the system resulting from attitude towards use and perceived usefulness; and (6) actual use of a system resulting from behavioral intention (van Biljon & Renaud 2009). Figure 2: Technology Acceptance Model (Davis et al. 1989, p.185) 108 IJEDICT   In this study TAM was used to frame some of the questions in the online survey. The framework helped to relate answers from the online survey to opinions discussed in FGDs as shown in results and analysis section. Subsequently we were able to analyze both responses on perceived ease of use and perceived usefulness and assess the assumption that the two factors are the primary factors that lead to actual system use. Davis et al. (1989) acknowledges various studies that discuss other factors linked to attitude and use of information systems and considers usefulness and ease of use of technology as “statistically distinct dimensions” (p. 185). In this study therefore, we identified factors linked to use as those mentioned by van Biljon and Renaud (2009). RESULTS AND ANALYSIS In this section we present the findings and discuss the various factors that affect perceptions on ease of use and usefulness and that influence the use of LCMS on mobile phones. Usefulness and ease of use In the online questionnaire, students were asked how they perceived the usefulness and the ease of use of the MLCMS functions on mobile phones. The questions were grouped under “perceived usefulness” and “ease of use” and the responses from each category were compared with the opinions expressed in the FGDs and the other answers from the online questionnaire. It is the combination of these given responses that are considered the determinants of LCMS use on mobile phones. The responses from the online questionnaire on “perceived usefulness” and “ease of use” are presented in Figure 3 and the following reflections were made on these responses in relation to focus group discussions: Use, interaction and access difficulties affect attitudes toward and behavioral intentions to use the system. However, such difficulties do not avert continued actual system use. More than half of the respondents (53%) noted that it was frustrating for them to use and operate MUELE on mobile phones and that they could not do every task on mobile phones. However, this did not deter them from using the system because there was demand and benefits (external factors) such as “cheap costs and portability of mobile phones, instant access to Internet resources,” that were mentioned by most of the respondents. Moreover, most respondents perceived the use of MUELE on phones improved their productivity (85%), gave them greater control over their learning activity (90%) and increased access to course material (100%). The system can still attract users even if the intentions to use are not fulfilled. Although most students could not perform all tasks in MUELE on mobile phones (94%), FGDs show that they still preferred mobile phones for particular reasons. For example, one student said that “I use MUELE on my phone if I want to quickly see what updates are available from the teacher such as course materials and assignments” while most of the students recited “access to Internet” as a necessity. Task knowledge and experience have an effect on use. Knowing how to perform a task requires knowledge i.e., ‘how to,’ thus an effort is needed to attain such experience (on ‘how to’). Knowing how to use is crucial for users. More than half of all respondents (53%) noted that it requires a lot of effort to know how to perform tasks while 29% stated that it requires an effort to become skillful at using the system. Results on experience from the online survey further show that most of those who frequently used MUELE and had used it for more than four months indicated having had less difficulty in using the system and regarded the system to be most useful. System efficacy shapes attitude towards use. All respondents (100%) perceived MUELE use on phone to be useful. Many reasons were given as to why MUELE on mobile phones was MLCMS actual use, perceived use, and experiences of use 109   perceived to be useful and continued to be used despite the challenges learners faced. Even the person who had not used MUELE on phone before considered it useful; “I think it is more flexible to use a phone compared to a PC.” This particular response shows that efficacy can shape attitude. Most perceived benefits that respondents strongly agreed to were increased access to learning materials (70 %), ability to accomplish learning tasks quickly (50 %), ability to communicate and improved productivity (45 %). Figure 3: Students’ perceived ease of use (n=17) and usefulness (n=20) of MUELE on mobile phone Overall, 45 % strongly agreed and 55 % agreed on MUELE’s perceived usefulness. As for overall ease of use, 35 % strongly agreed, 52 % agreed and 11% disagreed. Given the different responses received, the highly perceived benefit of MUELE on mobile phones was access to learning materials. 110 IJEDICT   Design and Technical challenges In FGDs respondents mentioned challenges faced when using MUELE on their mobile phones. The challenges included: (1) ineffectiveness i.e., the system is perceived to be too slow to load pages on mobile phones; (2) poor design leading to poor system pages optimization on phone screens. Students noted that, “pages become so compact on the screen and the words get mixed,” (3) need for a lot of virtual and physical memory for the phone; (4) upload restrictions and compatibility problems i.e., “difficulty to attach files, images …and receiving files that are not in formats supported by the phones,” (5) high costs. For example it was mentioned that, “it is costly to access the system using mobile Internet,” (6) communication problem, for example course updates were not sent to students automatically, “the system lacks automatic notification functions.” It emerged that technical challenges affect perceptions of use, but do not affect use. For example, despite the technical problems students mentioned, they were confident they would continue using the system for the purposes it served. Students’ desires were more focused on the user benefits rather than the technical difficulties. Other Use Dimensions Frequency of use Figure 4: Frequency of MUELE use on mobile phone (n=23) Frequency of use converts to experience due to navigation knowledge regularly acquired. Most of the students had used MUELE on their phones less than six months while three had used it six months or more. Figure 4 shows how often students used MUELE on their mobile phones. It also shows the “Other” category where one respondent noted not to have used MUELE on the phone on a weekly basis. Six students used it daily. In the FDGs as well as in the online survey, respondents listed the tasks they performed in MUELE using their phones. These included (1) reading course material and downloading course content, (2) checking for communications from the students and lectures and any updates from the lectures regarding their respective courses, (3) participating in discussion forums and chat rooms, (4) accessing assignments and, (5) web-mail services. These tasks were at least performed once a week, except for forums and chats which were only used infrequently (once a month) and only by some of the students. The responses were coherent with the activity logs that were extracted from the learning platform. The logs showed that students mostly accessed course material. There were also other activities that were not mentioned, but appeared in the activity logs. For example, searching and viewing users’ profiles (students viewing other students’ profile information) and forum searching.