European Journal of Open, Distance and e-Learning Vol. 18 / No. 2 – 2015 DOI: 10.1515/eurodl-2015-0013 WHO ARE THE OPEN LEARNERS? A COMPARATIVE STUDY PROFILING NON-FORMAL USERS OF OPEN EDUCATIONAL RESOURCES Robert Farrow, Beatriz de los Arcos, Rebecca Pitt, Martin Weller, Institute of Educational Technology, The Open University, United Kingdom Abstract Open educational resources (OER) have been identified as having the potential to extend opportunities for learning to non-formal learners. However, little research has been conducted into the impact of OER on non-formal learners. This paper presents the results of a systematic survey of more than 3,000 users of open educational resources (OER). Data was collected between 2013 and 2014 on the demographics, attitudes and behaviours of users of three repositories. Questions included a particular focus on the behaviours of non-formal learners and the relationship between formal and non-formal study. Frequency analysis shows that there are marked differences in patterns of use, user profiles, attitudes towards OER, types of materials used and popularity of different subjects. The experience of using OER is fairly consistent across platforms in terms of satisfaction and impact on future behaviour. On the whole, non-formal learners surveyed were highly positive about their use of OER and believe they will continue to use them. With regards to this making formal study more likely some degree of polarization was observed: some believed formal study was now more likely, while others felt it made this less likely. On the whole, while non-formal learners are enthusiastic about using free and online resources, the language and concept of OER does not seem to be well understood in the groups surveyed. A range of findings relating to OER selection and use as well as differences between repositories are explored in the discussion. Abstract in Spanish A pesar del potencial de los recursos educativos abiertos (REA) a la hora de ampliar las oportunidades de aprendizaje, existe poca investigación sobre su impacto en el aprendizaje no formal. Este estudio presenta los resultados de una encuesta realizada entre 2013 y 2014 para recoger datos demográficos, de actitud y comportamiento de más de 3.000 usuarios de tres diferentes repositorios de REA, con especial atención a la relación entre el estudio formal y no formal. El análisis de frecuencias muestra marcadas diferencias en las pautas de uso, perfil del usuario, actitudes hacia los REA, tipos de recursos utilizados y popularidad de diferentes materias. Las distintas plataformas registran experiencias de uso de REA similares en lo que se refiere a satisfacción e impacto sobre comportamiento futuro. Aquellas personas que aprenden en un contexto no formal se muestran muy positivas acerca del uso de REA y manifiestan su intención de continuar usando estos recursos. La transición hacia el aprendizaje formal tras el uso no formal de REA se presenta, sin embargo, polarizada entre aquellos que perciben el estudio formal más probable y los que lo perciben menos probable. En general, mientras que las personas que aprenden en un contexto no formal se muestran entusiastas del uso de recursos gratuitos en la red, se observa la falta de entendimiento del concepto de REA entre los diferentes grupos encuestados. La última sección explora una serie de conclusiones entre las que se encuentran, entre otras, diferencias en la selección y uso de recursos en los repositorios estudiados. Keywords: OER, data, non-formal learning, Saylor, OpenLearn, iTunesU, MOOC Who are the Open Learners? A Comparative Study Profiling non-Formal Users of Open Educational Resources Robert Farrow et al. Introduction According to the definition provided by Hewlett (n.d.), open educational resources are “teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use and re-purposing by others”. These can include whole courses of open content, textbooks, multimedia, software and any other materials which may be used to teach or support learning such as lesson plans and curricula. There remains some debate about what should qualify as an ‘open’ resource, with some definitions emphasizing open access to resources and others focusing on the affordances for revising and repurposing afforded by open licenses (see Creative Commons, 2013). However, any disagreements tend to be limited to the specific kinds of licenses for educational that are termed ‘open’ and whether they should permit specific forms of re-use (such as only allowing non-commercial re-use). Evaluating the effects that OER have on learning is problematic for several reasons. Once materials are published as OER they can be adapted and repurposed in accordance with their licensing restrictions. This is often done to improve or update a resource, or to make it more closely aligned to a specific educational need. But this possibility for adaptation can also make it hard to judge the efficacy of a particular resource, and makes longitudinal studies problematic. Furthermore, even when data can be collected comparing similar cohorts who have consistently used OER and non-OER it can be difficult to reliably attribute any change in performance to the openness of the resources used. Most research exploring the impact of open educational resources (OER) has focused on learners who are registered for a course of study at an educational institution where OER – often in the form of open textbooks – are used. A recent review of efficacy and perception studies (Hilton, 2014) found only twelve instances of peer-reviewed empirical research where the focus was on OER used as primary learning materials. These studies have indicated a high level of student satisfaction with the quality of OER available to them. Students have also consistently reported – particularly in the case of open textbooks – that OER have helped them financially (Pitt, 2015). OER use has also been correlated with higher test scores and lower rates of attrition (Hilton & Laman, 2012) though it is more common to find that use of OER delivers equivalent student satisfaction and performance at massively reduced cost (Feldstein et al., 2012; Wiley et al., 2012). The use of open educational resources (OER) by learners who are not registered for a formal programme of study has been subject to increased attention in recent years, with a recognition that there is a need for greater understanding of how to leverage non-formal learning in support of formal learning and support the transition from the former to the latter (Latchem, 2014; McGreal et al., 2014). Sangrà and Wheeler (2013) have suggested that non-formal learning should increasingly be viewed as a viable alternative to formal education, but recognize that more research is needed to establish effective forms of support. Miyazoe and Anderson (2013) argue that: “the availability of ever-growing amounts of OER and the consequent non-formal learning opportunities fuel this ‘opening’ of the traditional education systems. These free and open opportunities for both interpersonal and student-content interaction create an interaction surplus that can be used to augment and enhance formal educational curricula and systems”. The emergence of non-formal learning networks, facilitated by social networking and MOOC platforms, are increasingly recognised as a possible catalyst for non-formal learning. Siemens (2005) and Downes (2007) have outlined the ways in which such networks can be understood to support non-formal learning through the prism of Connectivism, which diminishes the distinction between formal and non-formal learning. European Journal of Open, Distance and e-Learning – Vol. 18 / No. 2 50 ISSN 1027-5207 © 2015 EDEN Who are the Open Learners? A Comparative Study Profiling non-Formal Users of Open Educational Resources Robert Farrow et al. “Non-formal learning is a significant aspect of our learning experience. Formal education no longer comprises the majority of our learning. Learning now occurs in a variety of ways – through communities of practice, personal networks, and through completion of work-related tasks… Learning is a continual process, lasting for a lifetime. Learning and work related activities are no longer separate. In many situations, they are the same.” (Siemens, 2005) The relationship between non-formal learning and formal study is increasingly an area for interest for higher education providers. Most research into non-formal learning concentrates on either the relevance of open materials for educational institutions, or on recognition of non-formal learning for professional development. As a result, little is known about the ‘non-formal’ OER learners; their ways of using these resources; or their reasons for studying in this way. Miyazoe and Anderson (2013) call for more attention to be paid to the “changing role of formal education in an era of learning opportunity where online educational resources and opportunities are readily accessible and in many cases completely free of cost to the learner”. Advocacy for non-formal learning tends to be grounded in the role that open resources can play in supporting formal learners who are registered for a course of study at an institution. OER can be used in this way to supplement curriculum, brush up on an area of study, for research purposes, or to otherwise complement traditional learning activities (Schmid et al., 2015). Much attention has been paid to possibilities for recognizing and accrediting the learning that takes place through self directed study which makes use of OER (Yang, 2015). OER also comprise a strategy for exposing non-formal learners with access to university level content; affording them opportunities to become more confident with self directed study. Institutionally produced OER can thus be seen to have both pedagogical and business dimensions. Many universities contribute to MOOC on the strategic assumption that in the long run they will be a source of future registrants. Part of the reason that non-formal learning has risen to greater prominence is a growing consensus that a flexible approach to learning and assessment can contribute to economic development and a richer understanding of how people learn in situ. But finding adequate ways to assess the effectiveness of non-formal learning remains a challenge. Mozilla Foundation (n.d.) launched the ‘Open Badges’ initiative to provide a framework for providing recognition for non- formal learning achievements through non-proprietary digital badges, which once awarded can be presented as part of a portfolio of learning. Badges tend to focus on skill acquisition rather than retention of ‘academic’ knowledge (Goligoski, 2012). As Glover and Latif (2013, p.1398) note, badges have the potential to support both student retention (through increased motivation) and future employability (through a digital record of achievement, or a portfolio of badges which can be shared with others as evidence of learning). A more traditional approach to the recognition of prior or non-formal learning is to follow private study with an examination equivalent to those taken by formal students. Conrad et al. (2013, p.46) have argued that digital and openly licensed learning materials are well suited to form the basis of a course of private study that can run parallel to institutional presentation and yet “[t]he greatest barriers to participation in open assessment and accreditation practices are identified as the lack of availability of committed staff members to support such activities, and the potential costs of redeveloping courses as OER”. Institutions who wish to promote recognition of prior learning therefore need to adopt a coherent, holistic strategy. “Within institutions, key factors for the success of open assessment and accreditation implementation appear to be a reliance on a strong base of support within the institution— both in terms of leadership and resources—and an existing culture of openness that includes policies and practices around the creation and use of OER […] Policies that enable either European Journal of Open, Distance and e-Learning – Vol. 18 / No. 2 51 ISSN 1027-5207 © 2015 EDEN Who are the Open Learners? A Comparative Study Profiling non-Formal Users of Open Educational Resources Robert Farrow et al. open access or recognition of prior learning via credit transfer or RPL are also important.” (McGreal et al., 2014, p.130) The advantage of harnessing this kind of non-formal learning could be significant. Cofer (2000) estimates that every hour spent in formal study inspires up to four hours of non-formal learning, but insists that non-formal learning should not be seen as subordinate to formal study but valuable in its own right. Similarly, Latchem (2014) argues that as much as 70-90% of lifelong learning results from non-formal learning and skill acquisition; and yet the majority of research focuses on institutional learning. Evidently, there remains much to be known about non-formal learning and how best to integrate it with existing systems of assessment and accreditation. This study aims to improve our understanding of this topography by providing information on those who use online repositories of OER for private study. It should be noted that our aim is not to identify or profile distinct learning styles (Kolb, 1984) for non-formal learners. Not only has this approach been criticized as lacking in empirical support (Coffield et al., 2004) but such a categorization would not be possible without prior work on how resources are used by non-formal learners; their learning objectives; and study techniques used. Methodology This research took place within the context of OER Research Hub project which ran from 2013- 2015 (OERRH, 2013). OERRH provided a focus for research on OER, and was designed to describe how openness is making a difference to learning and teaching practices. The project operated an open collaboration model and worked with a range of projects, initiatives and organisations across four education sectors (K12, college, higher education and non-formal learning). In addition to the survey based research presented here, the project collected data through interviews, focus groups, institutional visits and through research fellowship. The open collaboration model involved negotiating the parameters and design of the research according to the needs of the collaboration partner. In order to ensure that the areas of focus remained consistent throughout the project the research was guided by eleven key hypotheses about open education (de los Arcos et al., 2014). The main hypotheses investigated across the project focus on the impact of OER on learners and the ways in which open licensing affects sharing and use of resources: • Use of OER leads to improvement in student performance and satisfaction; • The open aspect of OER creates different usage and adoption patterns than other online resources. In addition, several hypotheses pertained directly to non-formal learning: • Non-formal learners use a variety of indicators when selecting OER; • Non-formal learners adopt a variety of techniques to compensate for the lack of formal support, which can be supported in open courses; • Open education acts as a bridge to formal education, and is complementary, not competitive, with it; • Non-formal means of assessment are motivators to learning with OER. The survey was designed by the research team to collect evidence relevant to the project hypotheses concerned with non-formal learning from repository users. European Journal of Open, Distance and e-Learning – Vol. 18 / No. 2 52 ISSN 1027-5207 © 2015 EDEN Who are the Open Learners? A Comparative Study Profiling non-Formal Users of Open Educational Resources Robert Farrow et al. Three OER repositories were selected for the study: iTunes U (Apple, 2014), OpenLearn (2015) and Saylor Academy (2015). iTunes U makes use of Apple’s iTunes content delivery systems – primarily used for commercial purposes – to make educational content available for free. This often comes in the form of podcasts and videos, and includes OER from museums and other cultural institutions as well as from higher education. The Open University (UK) runs has the most popular educational channel on iTunes U, though most major OER providers are represented in some way. Institutions can set up their own channel although can’t necessarily upload content to Apple servers. There is a course builder facility which is designed to help users create a programme of study within the platform. iTunes U is restricted to users of proprietary software/hardware made by Apple. Simply put, learners who wish to access these materials must at some point buy into the Apple ecosystem – even if only minimally. The advantage of this for the student is the assurance that all the technology should work adequately together, though there is a case for arguing that this is a significant impediment to openness and accessibility. OpenLearn (2014) is The Open University (UK) collection of free online learning materials, many of which are taken directly from courses presented to fee-paying students at the University. These include games and interactive media, materials from both undergraduate and postgraduate programmes as well as complete certified short courses. More than 33 million unique visitors have visited the OpenLearn website (which receives about 3 million yearly visitors) (Perryman, Law & Law, 2015). OpenLearn does more to accommodate non-formal learners who may lack confidence, providing short introductions to subjects as well as “interactive games, videos, blogs, [and] podcasts” (OpenLearn, 2015). A newsletter can also be subscribed to in order to get information about new courses as they come online. Most of the content on OpenLearn is designed to be explored through a web browser, and includes some multimedia content. It is structured so as to encourage the site user to browse their way straight into the content that interests them rather than ‘choosing a course’ from the onset. This ‘magazine’ style may be intended to ameliorate the potentially intimidating nature of the content for learners who are inexperienced or lacking confidence. On the homepage for OpenLearn there are often links made between current events (e.g. local and European elections) and relevant course content. This seems to be designed to make best use of pre-existing user interests, current TV shows and topicality to encourage the personal inquiry of the learner as the start of a learning pathway. The third repository in this study is Saylor Academy. Founded in 2008, its mission is to use technology to drive the cost of a college education to zero. Working towards this through curating and compiling open content, Saylor acts as an aggregator of existing content, compiling OER into complete courses that emulate or complement existing college courses. Where Saylor is unable to source adequate open content they commission academic consultants to write additional content to fill gaps in the curriculum. In addition to curating college- length courses Saylor also offer a range of featured pathways through several OR collections. These can offer coverage similar to an introductory level liberal arts course, or training in core professional competencies. Upon completing a course of study with Saylor students can be examined. For successfully completing a Saylor commissioned programme of study students can earn a ‘Saylor Certificate of Completion’ which, while conveying no institutional credit, provides evidence of learning and may be recognised differently in the future. Hilton, Murphy and Ritter (2014) have provided an account of the pedagogical and organizational theories informing the provision of OER as well as describing how these have been applied in the case of Saylor Academy resources. They argue for improved institutional recognition of non-formal learning, noting anecdotal evidence that students have gained college credit after studying non-formally using Saylor materials and European Journal of Open, Distance and e-Learning – Vol. 18 / No. 2 53 ISSN 1027-5207 © 2015 EDEN Who are the Open Learners? A Comparative Study Profiling non-Formal Users of Open Educational Resources Robert Farrow et al. subsequently passing the same invigilated exam as formal learners following a non-open curriculum. One thing to note about the different repositories of OER is that they tend to be characterized by being free at the point of delivery to learners but the different platforms exhibit different potentials for re-use, re-appropriation and modularization of learning materials. Though Weller (2014) has argued that we need to remain open about openness and the possibility that it can mean different things in different contexts, what qualifies as open in different contexts remains contested. For some, an open licence which legitimizes re-use and re-appropriation is a minimal requirement. In addition, we might expect that materials are natively formatted and licensed in such a way as to encourage re-use and re-purposing (iTunes U, for example, would not meet this requirement). Collecting data from non-formal learners presents particular challenges around sampling and validity because of difficulties with verification. A multi-dimensional analysis (including factor analysis and smallest space analysis) of research literature published in the fields of non-formal and informal learning concluded that “the familiar qualitative and quantitative tools already in the hands of researchers, such as interviewing, observing, surveying, etc. are valuable and appropriate for studying non-formal education” and that no new tools or approaches were required to produce meaningful results” (Cohen, 2007). Data Collection OER repositories who acted as OER Research Hub collaborators – iTunes U, OpenLearn, and Saylor – circulated an invitation to complete the survey to non-formal learners who used these sites between 2013 and 2014. 2299 usable responses were received from users of Saylor Academy, 725 from OpenLearn users and 110 from users of iTunes U. The total sample size was 3127 (though not all respondents answered all questions and some questions were omitted from the iTunes U survey). Precise sample sizes for each question are provided in the reporting below. Data was collected through SurveyMonkey, refined and compiled before being analysed in SPSS and Excel. Responses The survey was answered by OER users from a wide range of countries, as shown in Figure 1. Most responses came from the USA (n = 862) or the UK (n = 473) though India (n = 117) Canada (n = 87) and Brazil (n = 84) also contributed significant amounts of data. Most countries were represented and people from every continent contributed information. Excluding Africa, only Bolivia, French Guyana, Greenland, Kyrgyzstan, Suriname, Turkmenistan, Uzbekistan, and Venezuela were not represented. Most countries recorded between 5 and 40 responses. European Journal of Open, Distance and e-Learning – Vol. 18 / No. 2 54 ISSN 1027-5207 © 2015 EDEN Who are the Open Learners? A Comparative Study Profiling non-Formal Users of Open Educational Resources Robert Farrow et al. Figure 1. Geographical spread of survey responses Results A set of demographic questions across the surveys asked about age, gender, prior qualifications and employment status to facilitate profiling OER users. As Figure 2 shows, the repositories exhibit clear differences in the age profile of their users. Figure 2. Age profiles of repository users (N = 3127) Users of iTunes U showed a much lower average age profile with 71.8% of their users aged below 35. By contrast, OpenLearn users tended to be older, with 69% aged 35 or over and relatively few younger users. The pattern of user age profiles was closer to a standard deviation European Journal of Open, Distance and e-Learning – Vol. 18 / No. 2 55 ISSN 1027-5207 © 2015 EDEN Who are the Open Learners? A Comparative Study Profiling non-Formal Users of Open Educational Resources Robert Farrow et al. for Saylor users (perhaps reflective of a larger sample size). Similar patterns of difference can be discerned when considering the gender of respondents. Figure 3. Gender profiles of repository users (N=3090) Figure 3 indicates that Saylor users were more evenly split between male and female (and had a greater proportion of users who identified as transgender) while iTunes U users were more slightly more likely to be male. OpenLearn showed the largest difference in the gender of their users, with female users outnumbering male users by approximately half. Non-formal learners were also asked about their highest academic qualification and their employment status to ascertain differences in patterns of users across repositories. As is perhaps to be expected given the age profiles associated with the different repositories (Figure 3) iTunes U users, generally younger, were most likely to report holding a school leaving qualification as their highest. But as Figure 3 shows, the general trend was that users of these platforms tended to report already holding a degree. 59% of Saylor users held at least an undergraduate degree. The proportion of university graduates for OpenLearn and iTunes U were 46% and 38.2% respectively. European Journal of Open, Distance and e-Learning – Vol. 18 / No. 2 56 ISSN 1027-5207 © 2015 EDEN Who are the Open Learners? A Comparative Study Profiling non-Formal Users of Open Educational Resources Robert Farrow et al. Postgraduate/Graduate School University degree Undergraduate/Bachelors University degree College diploma or certificate OpenLearn (n=719) iTunesU (n=104) Vocational qualification (i.e. practical, trade-based) Saylor (n=2215) School leaving qualification (16-18 years) No formal qualification 0% 5% 10% 15% 20% 25% 30% 35% 40% Figure 4. Educational profiles of repository users (N = 3038) The age of non-formal users of OER also seems important when profiling employment status. Figure 5 illustrates the main differences in pattern. Saylor users were much more likely to be in full time employment (52%, n = 2300) while the majority of iTunes U users (43%, n = 110) were in full time education. iTunes U also had the highest proportion of users who volunteer part time (15%, n = 110). Approximately 40% of OpenLearn users were in full-time employment, and OpenLearn had the highest proportion of retired users (14%, n = 732). Figure 5. Employment profile of repository users (N = 3142) OER repository users were also asked about the ways that they have typically connected to the internet in the previous three months. Figure 6 summarizes this information. European Journal of Open, Distance and e-Learning – Vol. 18 / No. 2 57 ISSN 1027-5207 © 2015 EDEN Who are the Open Learners? A Comparative Study Profiling non-Formal Users of Open Educational Resources Robert Farrow et al. Figure 6. Internet access profiles of repository users (N = 3116) When taken in conjunction with the geographical diversity of the sample these patterns suggest that those accessing these online resources in developing countries are more likely to do so through a smartphone, tablet, or home broadband. A small proportion (less than 10%) of respondents accessed the internet through a dialup connection. iTunes U users were more likely to access materials through a tablet or while at an educational institution or community facility. The final part of background information collected concerned disability. For the combined sample of all three repositories, 10% of users disclosed a disability (n = 3160). OpenLearn had the highest proportion of users who identified as disabled (15.7%. n = 737) while Saylor reported the lowest proportion of disabled users (7.9%, n = 2298). The relatively high rate of declaration by OpenLearn students may reflect similarities to the approach taken to improving accessibility at The Open University through an inclusive framework which covers human and technical elements from course production through to evaluation (McAndrew, Farrow & Cooper, 2012). Figure 7 provides more information on the types of disability reported, and shows that long term or chronic illnesses and mobility impairments were more commonly reported by OpenLearn users while mental health problems more likely to be reported by iTunes U users. iTunes U users were also much more likely to declare a speech disability. It should be noted that some respondents declared more than one disability. European Journal of Open, Distance and e-Learning – Vol. 18 / No. 2 58 ISSN 1027-5207 © 2015 EDEN