Running head: MOTIVATIONS FOR ONLINE READING QUESTIONNAIRE 1 Title: Characteristics and Validity of an Instrument for Assessing Motivations for Online Reading to Learn This is a pre-copy-edited, author-produced pdf of an article accepted for publication in Reading Research Quarterly in 2020 following peer review. Elena Forzani, Boston University, Boston, MA, USA Donald J. Leu, Eva Yujia Li, and Christopher Rhoads, University of Connecticut, Storrs, CT, USA John T. Guthrie, University of Maryland College Park, MD, USA Betsy McCoach, University of Connecticut, Storrs, CT, USA Author Note Elena Forzani, Department of Language & Literacy, Boston University; Donald J. Leu, Departments of Curriculum and Instruction and Educational Psychology, University of Connecticut; Eva Yujia Li and Christopher Rhoads, Department of Educational Psychology, University of Connecticut; John T. Guthrie, Department of Human Development and Quantitative Methodology, University of Maryland College Park; Betsy McCoach, Department of Educational Psychology, University of Connecticut, Storrs, CT, USA. Eva Yujia Li is now at Institutional Research, Yale University, New Haven, CT. Portions of this material are based on work funded by the Institute for Educational Sciences under Awards R305G050154 and R305A170370 and the U.S. Department of Education under Award R305A090608. Opinions expressed herein are solely those of the author and do not represent the position of the Institute od the U.S. Department of Education. We wish to express our appreciation to members of the New Literacies Research Lab at the University of Connecticut for work with developing assessments and collecting data: Ian O’ Byrne, Heidi Everett-Cacopardo, Greg McVerry, Lisa Zawilinski, Cheryl Maykel, Clint Kennedy, Julie Corrigan, and Nicole Timbrell. We also wish to thank Julie Coiro, Jonna Kulikowich, and Nell Sedransk for help with developing the ORCA assessments and the scoring system; Mike Hillinger and Mark Lorah from Lexicon Systems for their help with conceptualizing the ORCA assessments and with the important programming development required; and Anthony Gambino for his work on The FACTORS Project. The Scientific Advisory Boards for both The ORCA Project (Irwin Kirsch, Glenn Kleiman, P. David Pearson, Rand Spiro, and Elizabeth Stage) and the FACTORS Project (John Guthrie, Irwin Kirsch, Yaacov Petscher, and Suzanne Wilson) also made important contributions to many issues, and we thank them for their insights. Finally, we thank Donna Bone for coordinating these complex projects. MOTIVATIONS FOR ONLINE READING QUESTIONNAIRE 2 Correspondence concerning this article should be addressed to Elena Forzani, Department of Language & Literacy, Boston University, Boston, MA 02215. Email: [email protected]. Abstract Motivation for reading is important to comprehension and has been studied extensively in offline reading contexts. However, we know little about the role of motivation in online reading, a new and increasingly important context for reading. This is largely because we lack valid and reliable instruments to estimate a student’s motivation for online reading. This study reports on the development of a Motivations for Online Reading Questionnaire (MORQ) among 1,798 seventh grade students in two states. Results from confirmatory factor analysis revealed a three factor solution for the MORQ: curiosity/value, self-efficacy, and self-improvement beliefs. Additionally, measurement invariance across female and male students was established. Predictive validity of the MORQ was supported by the positive and significant contribution of MORQ to the Online Research and Comprehension Assessment, an established measure of online reading comprehension. Results help establish the MORQ as a well validated instrument for measuring online reading motivation. Results are discussed in relation to theory, research, and practice. Keywords: motivation/engagement, online reading, comprehension, digital/media literacy, assessment MOTIVATIONS FOR ONLINE READING QUESTIONNAIRE 3 Characteristics and Validity of an Instrument for Assessing Motivations for Online Reading to Learn Many factors affect a reader’s ability to comprehend and learn during reading, such as prior knowledge (Bråten, Anmarkrud, Brandmo, & Strømsø, 2014; Kintsch, 1998), cognitive skills and strategies (Duke & Pearson, 2002; Pressley & Afflerbach, 1995), texts, activities, and contexts (RAND, 2002), and motivation (De Naeghel, Van Keer, Vansteenkiste, & Rosseel, 2012; Schiefele & Schaffner, 2016; Wigfield & Guthrie, 1997). While these factors are relatively well understood in offline contexts, they are less well understood online, given the recency of this new reading context (Afflerbach & Cho, 2010). Initial studies that have investigated reading online have made important contributions to our understanding of reading but have focused largely on the cognitive and metacognitive factors, prior knowledge, and texts and contexts that affect readers’ ability to comprehend, ignoring potential influences of motivation (e.g. Afflerbach & Cho, 2010; Leu et al., 2015; Cho, 2014; Coiro, 2011; Coiro & Dobler, 2007). The prominence of the Internet as a reading context in today’s world, however, combined with the profound interest and engagement of our youth with online media, suggests that reading researchers need to better understand students’ motivations and abilities when reading online (Leu, Kinzer, Coiro, Castek, & Henry, 2019; Buckingham & Willet, 2013; Hutchison & Reinking, 2011; Hutchison, Woodward, & Colwell, 2016). As we seek to understand the relationship between motivation and comprehension when reading on the Internet, developing a valid and reliable instrument to measure motivations for reading online would be useful. MOTIVATIONS FOR ONLINE READING QUESTIONNAIRE 4 Why should we develop an instrument to estimate online reading motivations when we already have instruments that estimate offline reading motivation? Research suggests that motivation may function somewhat differently online for several reasons. First, while offline and online reading share many similar characteristics, empirical results suggest they are not the same (Afflerbach & Cho, 2009; Coiro & Dobler, 2007; Coiro, 2011; Goodwin, Cho, Reynolds, Brady, & Salas, 2019). Online contexts may require greater attention to locating, evaluating, and self- monitoring (Cho, 2014; Coiro & Dobler, 2007) but may also position readers to be more critical and agentic, since readers in this context often create, rather than merely consume, knowledge and ideas, and they do so within an interactive, social context (Hutchison et al., 2016). Second, because online reading is often driven by the need to gather information to address a personal or professional issue (Stadtler, Scharrer, Brummernhenrich, & Bromme, 2013; Wineburg & Reisman, 2015), readers’ motivations in this context may be more affected by a need to act on information rather than by curiosity alone. Third, students’ varying Internet access, at home and at school, appears to play a role in their ability to read in this newer context (Leu et al., 2015; Mullis, Martin, Foy, & Hooper, 2017), likely influencing motivations. Such factors suggest that readers’ motivations may be somewhat different online versus offline, necessitating the development of a tool to measure motivations for online reading, specifically. Thus, to meet important needs for both research and practice, this study developed a multidimensional Motivations for Online Reading Questionnaire (MORQ) with a sample of seventh graders. In so doing, this study sought to analyze the factor structure of the MORQ and to test measurement invariance across gender. It also sought to evaluate predictive validity by investigating the relationship between the MORQ and an established measure of online reading MOTIVATIONS FOR ONLINE READING QUESTIONNAIRE 5 known as the Online Research and Comprehension Assessment, or ORCA (Leu et al., 2014, 2015). Theoretical and Empirical Perspectives Online Research and Comprehension We situate our definition of online reading within a broader concept of digital reading involving readers engaged in “multifaceted meaning-making experiences” with multiple texts and for particular purposes within diverse digital contexts (Coiro, 2020, p. 4). The digital context in this study is the Internet. Reading on the Internet, or online reading, can take many forms, and we frame this study around the new literacies of online research and comprehension (Leu et al., 2015; Leu et al., 2019). This theory views online reading as a process involving both traditional and additional skills, strategies, and dispositions and suggests that online, readers often engage in inquiry to learn more about a particular issue or set of issues (Leu et al., 2019). This process includes several components that appear to function reciprocally: locating, evaluating, synthesizing, and communicating (see Forzani, 2016 and Leu et al., 2015 for elaboration). As students read to learn online, they encounter many forms of text, including informational graphics, newspapers, magazines, websites that present informational and narrative texts (e.g., biographical and travel reports, stories), and many more. This study includes all of these potential forms. We direct our focus to reading to learn since much of online information is used this way (Kiili et al., 2018b; Leu et al., 2019; OECD, 2011b). Given the importance of reading to learn online, several related assessments have been recently developed to measure this ability: The PISA Digital Reading Assessment (OECD, 2011a); ePIRLS (Mullis & Martin, 2020); and the ORCA (Leu et al., 2014, 2015). The ORCA is a performance-based measure of readers’ ability to locate information; evaluate author, MOTIVATIONS FOR ONLINE READING QUESTIONNAIRE 6 publisher, and content; synthesize within and across texts, and communicate findings via email or wiki. The ORCA was used in this study to measure online reading during an inquiry task in order to validate the MORQ with online reading performance using an established measure. While offline and online reading overlap in many ways, research suggests they are not the same. This appears to be the case in less complex contexts, such as when reading on paper is compared to reading the same text on a screen without hyperlinks or other dynamic features (Goodwin et al., 2019). For example, Goodwin et al. (2019) found that highlighting was correlated negatively with comprehension in a paper context but positively in a screen context, and suggested that this may point to the increased importance of strategic behaviors in digital contexts. It also appears to be the case within the hyperlinked and dynamic context that defines the Internet (Leu et al., 2015; Coiro, 2011; Afflerbach & Cho, 2009). These demonstrations that offline and online reading differ suggest that motivations for reading in offline and online contexts also may differ. There may be several reasons for this. First, the distinctive features of an online context appear to influence reading processes, and this may have consequences for motivational differences, too. Compared to offline reading, readers online encounter many more texts, and of a greater range of type and quality (Coiro, 2020; Leu et al., 2019). The constant and iterative selection and integration of information across these multiple texts with diverse perspectives requires readers to evaluate relevancy and credibility both more frequently and more vigilantly (Cho, 2014; Kiili, Leu, Marttunen, Hautala, & Leppanen, 2018a; Metzger & Flanagin, 2013). Additionally, the dynamic and stimulating nature of online elements such as video, hyperlinks, and advertisements make self-monitoring, to avoid distraction, more challenging (Cho, 2014; Coiro, 2020; Goldman, Braasch, Wiley, Graesser, & Brodowinska, 2012) and motivation more important. Online contexts also afford MOTIVATIONS FOR ONLINE READING QUESTIONNAIRE 7 readers greater opportunity to contribute to collective understandings and to voice opinions through shared tools such as blogs, wikis, and social media (Hutchison et al., 2016). Readers who enjoy this social context may be especially motivated to read online, even when the primary motivation is not social. These, and other factors, also suggest greater criticality and autonomy, which may be highly engaging and supportive for some readers but more challenging for others. Second, readers’ purposes for reading online, compared to offline, may be somewhat more specific. In offline contexts, readers read many types of texts for a variety of purposes, such as for enjoyment, to learn new information out of curiosity, or to gather information to serve a goal. In online contexts, readers may read most often to obtain specific information with which to act (Kiili et al., 2018a); Stadtler, et al., 2013). This may place more emphasis on being motivated by valuing information rather than by satisfying one’s curiosity alone. Third, several studies have found that greater access to digital devices in the home correlates with higher average online reading comprehension (Leu et al., 2015; Mullis et al., 2017). Lack of access and practice, among those without these advantages at home or at school, may lower readers’ self-efficacy and beliefs about whether they can get better with practice, since such students may have less experience and thus fewer opportunities to develop such beliefs. In turn, this could directly influence their reading. Finally, preliminary research on gender differences suggests readers may be motivated somewhat differently in online compared to offline contexts. Research in offline contexts suggests that girls are more motivated to read than boys (Logan & Johnston, 2010; McKenna, Conradi, Lawrence, Jang, & Meyer, 2012). Yet, research in digital contexts indicates that boys have more positive relationships with technology and online reading (Liu & Huang, 2008). This work suggests that online reading motivation may be somewhat different than offline motivation. MOTIVATIONS FOR ONLINE READING QUESTIONNAIRE 8 Approaches to and Dimensions of Reading Motivation in Offline and Online Contexts Despite these differences, there is also much overlap between offline and online reading (Afflerbach & Cho, 2009, 2010; Coiro, 2011; Leu et al., 2015), implying overlap in motivations. For example, in both offline (Fuchs, Fuchs, Hosp, & Jenkins, 2001) and online (Kanniainen, Kiili, Tolvanen, Aro, & Leppänen, 2019) contexts, readers employ foundational skills in decoding, word identification, and fluency in order to comprehend. This frees up resources, allowing readers to focus on comprehending (Fuchs et al., 2001). In both contexts, skilled readers engage in many of the same comprehension processes, such as using prior knowledge to understand new information, reasoning inferentially, evaluating, and monitoring (Duke & Pearson, 2002; Pressley & Afflerbach, 1995; Afflerbach & Cho, 2009; Coiro, 2011). These cognitive systems are recruited, energized, and sustained by motivational processes during reading (Barber, Levush, & Klauda, 2019). Given these findings, theory and research in offline contexts may provide useful direction for mapping the complexities of motivations online. In offline contexts, researchers often conceptualize reading motivation as a multidimensional construct representing a person’s interests, values, beliefs, goals, and dispositions for reading (Malloy, Marinak, Gambrell, & Mazzoni, 2013; Schiefele, Schaffner, Möller, & Wigfield, 2012; Wigfield & Guthrie, 1997). Such motivations can be viewed as involving multiple factors and can vary by recreational versus academic context (De Naeghel et al., 2012; Schiefele & Schaffner, 2016). Indeed, instruments exist that measure offline reading motivations using such dimensions, broadly conceived, as interests, values, beliefs, goals, and dispositions (see, for example, McKenna, Kear, & Ellsworth, 1995; Gambrell, Palmer, Codling, & Mazzoni, 1996; Schiefele & Schaffner, 2016; Wigfield, 1997). We sought to benefit from this perspective and develop an instrument with multiple dimensions. MOTIVATIONS FOR ONLINE READING QUESTIONNAIRE 9 In order to develop a measure that could explain performance efficiently (i.e., using the fewest dimensions possible), we drew from multiple theoretical approaches in conceiving of motivation and in selecting dimensions that would positively predict comprehension. Consequently, we drew from three important theories of motivation: self-determination theory (Ryan & Deci, 2000), social-cognitive theory (Bandura, 1977; Dweck & Legett, 1988), and modern expectancy-value theory (Eccles & Wigfield, 2002). These theories undergird the Motivations for Reading Questionnaire, or MRQ (Wigfield & Guthrie, 1997), and related work for online contexts, such as the Survey of Online Reading Attitudes and Behaviors, or SORAB (Putman, 2014). It is important to note that different approaches to motivation have been used to generate measures of motivation representing different dimensions. Some approaches suggest that factors like beliefs (e.g., self-efficacy) are antecedents or consequences of motivation, rather than motivations in themselves (see, for example, Schiefele & Schaffner, 2016; Schiefele, Schaffner, Möller, & Wigfield, 2012). Other approaches, represented by the MRQ (Wigfield & Guthrie, 1997) and the SORAB (Putman, 2014), view such factors as motivations which themselves provide the direct energy for a behavior. Drawing on several theories, we framed this study using this latter perspective, defining motivation as providing the direct energy for a behavior, and viewing beliefs as motivations. Bandura defined motivation as involving the “activation and persistence of behavior” (Bandura, 1977, p. 193). According to expectancy-value theory, beliefs like expectancies (similar to self- efficacy) provide such activation and persistence by directly energizing and sustaining behavior: “expectancies…directly influence performance, persistence, and task choice” (Eccles & Wigfield, 2002, p. 118). Moreover, it seems plausible that beliefs can be motivations in MOTIVATIONS FOR ONLINE READING QUESTIONNAIRE 10 themselves, in the sense that they directly energize behavior. According to self-determination theory, people have an innate desire to “exercise [their] capacities” (Ryan & Deci, 2000, p. 70) and might engage in an activity for the feeling of competence it offers. Imagine a child learning to read and feeling the competence that comes as they become skilled with a new and stimulating activity. It is conceivable as well that readers choose to read online because they feel competent in this particular context. This approach is also in line with Author’s (Guthrie & Klauda, 2016) heuristic model of motivation and engagement, in which factors such as efficacy, intrinsic motivation, and value provide the direct energy required for engaging in reading, leading to enhanced comprehension. Thus, we decided to develop our initial questionnaire by starting with one of the more expansive and comprehensive approaches, represented by the MRQ (Wigfield & Guthrie, 1997). The MRQ suggests eleven dimensions of motivation that include: curiosity, involvement, importance (i.e., value), self-efficacy, challenge, recognition, grades, social, competition, compliance, and reading avoidance. While some studies (e.g., Guthrie, Klauda, & Ho, 2013) suggest that five of these dimensions (recognition, grades, competition, compliance, and reading avoidance) predict achievement in offline contexts, this correlation seems tenuous for an online context and risked invalid interpretations. For example, students might not be motivated to read for recognition in classrooms in which online reading is not yet highly valued, as is offline reading. The same line of thinking can be applied to competition, compliance, and avoidance. Thus, students’ responses on items from these factors may indicate something other than motivation for online reading. Therefore, these were not included. The sixth (social) is likely to be relevant online, but we chose not to include it here because interpretations from items used in the MRQ seemed suspect when applied to an online context. Some items, for example, focus on