DOCUMENT RESUME IR 017 971 ED 397 783 Cennamo, Katherine; Braunlich, Elizabeth AUTHOR The Effects of Relevance on Mental Effort. TITLE PUB DATE 96 13p.; In: Proceedings of Selected Research and NOTE Development Presentations at the 1996 National Convention of the Association for Educational Communications and Technology (18th, Indianapolis, IN, 1996); see IR 017 960. Research/Technical (143) PUB TYPE Reports Speeches/Conference Papers (150) MF01/PC01 Plus Postage. EDRS PRICE College Students; Desktop Publishing; Higher DESCRIPTORS Education; Instructional Materials; Learning Motivation; *Media Selection; Printed Materials; Questionnaires; Reaction Time; *Relevance (Education); *Student Motivation; Time on Task; Videotape Recordings ABSTRACT According to Keller's Model of Motivation, one technique for increasing learners' motivation, and thus their mental effort, involves increasing the personal relevance of the material. When instruction is perceived as relevant, learners perceive that important personal needs are being met by the learning situation. The purpose of this study was to manipulate the degree of relevance present in both a videotaped and print version of a lesson to determine whether increasing the relevance of a lesson increases the amount of material effort invested. Students enrolled in a sophomore level education course at a large Midwestern university were invited to participate in a two-hour lesson on desktop publishing. Instructional materials consisted of four lessons: a "standard" (1) (2) a motivationally enhanced videotape; video on dusktop publishing; a text version of the standard lesson; and (4) a text version of (3) the motivationally enhanced lesson. Assessment measures consisted of a self-report questionnaire, a computer program that measured response rates, and a posttest. Findings were consistent with the results of previous studies conducted with elementary and middle school students. Participants reported in the questionnaire that they invested more mental effort in learning from print versions of the lessons than in video-based versions. The students who received the video-based lesson had significantly longer reaction times to the secondary task measure. (Contains 26 references.) (AEF) *********************************************************************** * Reproductions supplied by EDRS are the best that can be made from the original document. *********************************************************************** U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) O This document has been reproduced as received from the person or organization originating it. O Minor changes have been made to improve reproduction quality. rel Points of view or opinions stated in this 00 document do not necessarily represent official OERI position or policy Title: CI% (21 f1-1 The Effects of Relevance on Mental Effort Authors: Katherine Cennamo & Elizabeth Braunlich Purdue University "PERMISSION TO REPRODUCE THIS C.) MATERIAL HAS BEEN GRANTED BY BEST COPY AVAILABLE M. Simonson 125 2 c\'\ TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC)" The Effects of Relevance on Mental Effort passive medium, and thus invest little mental It is a commonly accepted that learners view video as an easy, assumption is primarily based on a series of studies that effort in processing the information (the "couch potato"). This 1984) conducted in the early eighties. In these studies. Salomon (Salomon, 1983; Salomon, 1984; Salomon & Leigh, learn from television than books, reported spending less mental Salomon found that learners perceived it to be "easier" to recalled less from a video-based version of a story than from a effort in learning from videotape than from books, and of video or television as "easy" resulted in the print -based version. Salomon assumed that learners perceptions learning achievement, when compared to reading books. investment of less mental effort, and consequently less provide instruction in a printed form; however, We could assume from these conclusions that it is preferable to particularly in the training environment. In fact, a recent video remains a popular iaeans of conveying instruction, training was exceeded magazine reported that the use of videotape for conducting corporate survey conducted by 'raining only by stand-up instruction (Froiland, 1993). invested in instructional videotapes has been conducted The majority of research investigating the mental effort deer Voort. 1993; Sherman, Salomon, 1983; Salomon, with elementary and middle school children (Beentjes & van researchers have investigated the amount of effort that college 1984; Salomon & Leigh, 1984). Although several 1978; Muth, & Glynn, 1986; Britton, Piha, Davis, & Wehausen, students invest in processing print lessons (Britton, Strickwerda, & Sumner, 1993; Reeves & Thorson, 1986; and and others) and television commercials (Lang, Geiger, investigated the mental effort older learners expend in others), there has been very little research that has systematically learning older learners may have more well developed strategies for learning from video-based instructional materials. As students, it is important to investigate the extent to which Salomon's from videotape that elementary and middle school conclusions apply to the adult population. effort" as the "number of non-automatic elaborations applied Salomon (1983a) defines the construct of "mental processing is automatic processing that is fast and effortless, non-automatic to a unit of material" (p. 42). In contrast to the individual. As new information is received, the new deliberate, conscious, and very much under the control of is (E. Gagne, 1985) stored in the learner's schemata. The learner information cues the retrieval of related prior knowledge and information retrieved from prior knowledge; this process then able to make connections between the new information from increased contact with the learner's mental schemata that results is referred to as elaboration (E. Gagne, 1985). The mental effort, is presumed to facilitate the retention and the conscious, non-automatic generation of elaborations, or retrieval of the new material. learners invest in video-based instruction has been hindered The search for techniques to increase the effort that of mental effort. Early work (Salomon, 1983; by the limitations of the instruments used to assess the construct the amount of effort learners invested in print and video-based Salomon, 1984; Salomon & Leigh, 1984) that investigated learners' effort expenditures. However recently, researchers instruction used self-report questionnaires to document the identified a need for more precise methods of assessing (Beentjes, 1989; Cennamo, Savenye, & Smith, 1991) have studies in which mental effort is assessed by Beentjes (1989) accurately states that "... validation mental effort. multiple methods are called for." ( p. 56). in a variety studies using print (Britton, Muth, & Secondary task techniques have been used to assess effort and others) and television commercials (Lang, Geiger, Glynn, 1986; Britton, Piha, Davis & Wehausen, 1978; and others), and seem to offer a promising method of assessing Strickwerda, & Sumner, 1993; Reeves & Thorson, 1986; When using the secondary task technique, a participant completes a the mental effort invested in video-based lessons. the same time is instructed to respond to a secondary task such as a primary task (reading, watching a video, etc.) and at that as learners spend more cognitive resources on the beep or flash of light as quickly as possible. Researchers assume allocate to the secondary task. They assume that slower reaction primary task, they have less cognitive resources left to cognitive effort is being allocated to the primary task. times to the secondary task indicate that additional task technique are assumed to both measure mental effort, Although Salomon's questionnaire and the secondary question. Beentjas and van der Voort (1993) assessed the effort that the results of a recent study call that assumption into stories and the "structurally equivalent" video-based versions. fourth and sixth grade students invested in two print-based responded to a secondary task, an audible beep, by pressing a key While students were reading or viewing the stories, they beep. After they completed the stories, they responded to a self-report as quickly as possible after hearing the instrument. Then they completed an achieiement test. The questionnaire of mental effort that was based on Salornon's the questionnaire and the secondary task measures of researchers found very low correlations between students scores on and achievement scores on a test of recall inferences effort. In addition, the correlations between scores on both measures the search for a valid means of assessing mental effort, students reported were also very low. To further complicate 3 126 spending significantly more effort in reading the text versions than in viewing the videotapes. However, when reaction times to the secondary task were used as a measure of mental effort. the opposite effects were found; reaction times were significantly longer, indicating more mental effort, for the videotaped versions than the text versions. Beentjas and van der Voort state that further research is needed on the validity of both methods of assessing mental effort. They suggest that one promising area of investigation may be to determine to what extent both measures are sensitive to manipulations that are intended to increase mental effort. But what manipulation? No prior studies have manipulated variables designed to increase mental effort and assessed increases in effort using the secondary task technique and a questionnaire. To select a manipulation that has been shown to increase mental effort as measured by a self-report questionnaire (fun/learn for example) may be biased toward demonstrating effects on the questionnaire. To select a variable that has been shown to increase mental effort as measured by the secondary task technique (related or unrelated cuts, for example) may be biased toward demonstrating effects on the secondary task measure . To further complicate the selection of variables, the vast majority of research that has used reaction times to a secondary task as a dependent variable has manipulated features of either text sar video, not both. To select a variable that is effecti., e in print may bias the results toward print; conversely, to select a variable that is effective in videotape may bias the results toward that medium. Mental effort refers to the choice on the part of the learner to allocate cognitive resources to elaborating on an idea, thus, it has motivational as well as cognitive components (Salomon, 1983). According to Keller (1983), motivation "refers to the choices people make as to what experiences or goals they will approach or avoid, and the degree of effort they will exert in that respect." (p. 389, italics in original). He further distinguishes between effort and performance: "Pefformance means actual accomplishment, whereas effort refers to whether the individual is engaged in actions aimed at accomplishing the task. Thus effort is a direct indicator of motivation" (p. 391). Based on the literature in motivational theory, Keller's ARCS Model of Motivation (1983) breaks the concept of motivation down into four parts (attention, relevance, confidence and satisfaction). Keller provides specific strategies that instructional designers can use to increase learner's motivation, or the effort expended in accomplishing an instructional task. According to Keller's (1983) ARCS model, one promising technique for increasing learner's motivation, and thus their mental effort, involves increasing the personal relevance of the material. When instruction is perceived as relevant, learners perceive that important personal needs are being met by the learning situation" (p.406, italics in original) Nwagbara (1993) presented learners with two versions of a videotape to examine the effects of relevance on learners motivation toward learning the steps involved in creating a document using a desktop publishing system. Half the learners received the "standard" videotape and half received a "motivationally enhanced" version of the program that included scenes to increase the relevance of the lesson for the learners. He found that learners reported a greater willingness to expend effort in watching the motivationally enhanced videotape than in watching the same program without the motivational enhancements. Although the actual effort expended in the lesson was not assessed, these results suggest that increasing the perceived relevance of a lesson may encourage learners to invest more mental effort in learning the material. The purpose of this study was to manipulate the degree of "relevance" present in both a videotaped and print versions of a lesson to determine whether increasing the relevance of a lesson increases the amount of mental effort invested in the lesson. In addition, mental effort was assessed using both the secondary task technique and a self-report questionnaire to determine the more valid way of assessing mental effort. And finally, the study was conducted to determine the extent to which Salmon's conclusions apply to learners of college age. The following research questions were addressed: a)Do learners invest more effort in print-based versions of the lessons than in video-based lessons? b) Do learners invest more mental effort in motivationally enhanced lessons than in "standard" lessons which lack these characteristics? c) IN learners' self-reports of effort correlate with their "on-line" measurements of effort collected using a sec.Indarv task technique? d) Do le4rt,ers mental effort scores (from secondary task or self-report measures) correlate with their post-test scores on recall measures Methods For this study, a 2 (text vs. video) X 2 (enhanced vs. standard versions) X 2 (secondary task vs. non-secondary task) design was used. Eight sessions were scheduled over a two week period with treatments randomly assigned to each of the eight sessions. 1274 BEST COPY AVAILABLE Participants Students enrolled in a sophomore level education course at a large Midwestern University were invited to volunteered participate in a two hour lesson on desktop publishing. Of 225 students invited to participate, 130 students questionnaire, for the study. Thirteen cases were eliminated due to incomplete data (failed to complete the mental effort eliminating failure to respond to the secondary task, or to problems with the data collection equipment). After female (N=82). incomplete data sets, 117 participants remained in the study. Participants were both male (N= 35) and Instructional Materials: Instructional materials consisted of four lessons: a) a "standard" video on desktop publishing, b) a motivationally enhanced videotape , c) a text version of the "standard" lesson, and d) a text version of the motivationally enhanced lesson. The two videotapes were used in Nwagbara's (1993) previous study. The "standard" video tape is a commercially available product. The motivationally enhanced videotape was created by editing segments designed to increase the personal relevance of the material into the commercial videotape. Keller (1987) suggests that to increase the relevance of a lesson, instructional designers should provide goal of instruction to orientation, motive matching, and familiarity. Goal orientation can be created by relating the benefits learners. getting a job, getting a promotion, or improved job performance, or other goals that may be of value to the their own goals. In The goals of the lesson should be clearly presented or learners should have the opportunity to set For learners learners must understand how the concepts and skills are related to their present or future goals. addit future oriented learners, who are present oriented, the instructional content can be related to their current interests. For in the future. Motive matching can be the instruction must provide a rationale as to how the information will be helpful role models. Learners enhanced by providing personal achievement opportunities, cooperative activities, and positive should promote perceptions should be encouraged to assume responsibility for his or her own behavior and the materials be increased by using of self-improvement on the part of the learners (McClelland, 1965; Keller, 1983). Familiarity can testimonials about specific people or things concrete examples from settings familiar to the learner; stories, pictures, or familiar to the learner; and through the use of analogies (Keller, 1984). Following Keller's (1983; 1987) recommendations, motivational elements were added to a) illustrate concrete through testimonials and provide practical examples, b) provide goals for the learners, c) provide positive role models future skills, e) from product users, d) illustrate how the lesson might help the learner improve their present and and e) illustrate how the skills are useful. For example, a encourage learners to assume responsibility for their learning, section was added that stated: " Some of you may have used the computer at home or in school to do some assignments, write term the things you already know about computers, like papers, and other interesting things. You'll see that some of be how to turn the power on, how to use the mouse, or your familiarity with the buttons on the key board will handy and helpful to you in the process." (Familiarity) "Gaining the knowledge and skills in this presentation will open a variety of windows of opportunities for salaries, and of course, prospects for promotions whether you in the world of work in terms of good jobs, good of your won desktop publishing business." (goal orientation) you chose to work in the industry or as the owner between the narrators and In addition, Nwagbara a) removed non-instructional graphic images, b) condensed time result in two versions of the lesson that c) eliminated or reduced the time between illustrations of the computer screens to were both approximately 35 minutes in length. prepared For this study, text versions were prepared from Nwagbara's tapes. The text of the print versions was viewed the videotapes and read the text by transcribing the narration from the videotapes. Two undergraduate students their recommendations, 35 versions to identify visual images that they believed to be important to the text. Based on illustrate key points. For example, images were digitized from the videotape and added to the text to provide interest and Visual images of the the text booklet begins with a series of images created using desktop po.'Aishing packages. the computer screen. "speaker" accompany the testimonials. And finally, the text versions i clude numerous images of such as The verbatim transcription was modified slightly for the print version. When the narrator makes statements "now you see" , the text version makes reference to the appropriate figure (in Figure I, you see.."). 5 128 Text versions were analyzed to ensure that they were equivalent to the vid?otaped versions. One by one, four graduate students in instructional design independently reviewed the text and video materials. Each student reviewed the regular text and video versions together, then the motivationally enhanced materials were reviewed together. As differences were identified between text and video versions, revisions were made to the text version. After two rounds of revisions, two reviewers agreed that the text and video versions were equivalent. Assessment measures Self-report questionnaire: The self-report measure of effort was created by modifying questions used by other researchers (Salomon, 1983b; Salomon & Leigh; 1984; Sherman, 1993). When Salomon's questions were used in a previous study with college students, the instrument only had Chronbach's alpha of .55 (Cennamo, Savenye, & Smith, 1991). Beentjas and van der Voort (1993) reported alphas of .60 to .64 when using similar questions With children. Thus, the instrument was modified in an attempt to increase the reliability of the instrument. Four of the questions were modifications of questions used by Salomon (1983b, 1984) and five questions were modifications of questions used by Sherman (1993) who reported a Chronbach's alpha of .69 for his mental effort questionnaire. Participants were asked to respond to questions such as "While reading the booklet, I concentrated hard on what was shown in the pictures." "The lesson made me think very hard." "I tried hard to understand the information presented in this lesson." Participants In this responded on a 5-point Liken scale which rated responses from "Strongly Agree" (5) to "Strongly Disagree" (1). study, the questionnaire had a Chronbach's alpha of .78 , indicating an acceptable degree of internal consistency (Borg and Gall, 1983). Additional questions on the self-report questionnaire gathered information regarding participants' gender, previous experience with Macintosh computers, experience using desktop publishing software, and other demographic information. Secondary task measure. The secondary task measure consisted of a computer program that presented an audible beep at random intervals then measured and the time between the beep and the moment when the participant clicked the mouse button in "tics" (a tic is equivalent to 1/60 of a second). During the practice phase, participants received seven practice beeps randomly generated in 10 to 30 second intervals. By averaging the seven practice response times, a baseline response rate was established. During the actual use with instructional materials, participants responded to a beep randomly generated over intervals ranging from 30 to 90 seconds. Secondary task reaction times were calculated for each student by averaging the used by another researcher response time for each participant engaged with a lesson. Consistent with a procedure (Grimes, 1990), responses that were more than three standard deviations from the individual's mean were ignored for analysis. For example, if a participant failed to respond to a beep or responded after a much longer interval than usual, that particular score was eliminated from the final average score. Post-test. The post-test consisted of a cued recall test, where students wrote down all they could remember about topics presented in the lessons. Participants were first asked to recall the ideas that impressed them most. The next five questions corresponded to the five major topics of the videotape. Participants responded to questions such as: In Super Paint, what tools do you remember and what were their uses?" and "What steps would you follow to create a news letter with a page-layout program?" Finally students were provided with the opportunity to list other information they recalled from the lesson ("What else do you remember from the lesson?" ) A scoring key was developed that categorized responses to the post-test. The primary researcher and two research assistants (a graduate student in instructional design and an undergraduate education student) each reviewed 15 recall post- tests and clustered responses that seemed to be of a similar type. Responses seemed to directly =all information presented in the lessons, make inferences based on the information presented in the lessons, summarize information that was presented in the lessons, or ;valuate the lessons. Differences in coding responses among the three individuals were discussed and coding categories were finalized. The final coding categories consisted of recall responses (this software leaves room to correct mistakes, I can design my own letterhead), inferences (it seemed easy, it seems convenient to use), and summary statements (there were different photos of parts of the computer, I remember the letters from people in the industry). Evaluative comments (the information was presented in good form; I was falling asleep) were not coded. The two research assistants then coded 15 other post-test using the coding scheme. After finding an acceptable inter-rater reliability of .97 between the two scorers, the tests were scored by one of the research assistants using the scoring key. Participants received a recall score, inference score, summary score, and total post-test score. 6 129 BE3T COPY AVAILABLE Para lum All sessions were held in a computer lab which contained approximately 40 computers. Groups using the secondary task measure had a demonstration of the data collection program And practice with the program. During the practice session, a baseline measure of reaction time to the secondary task was collected. All participants received either a motivationally enhanced videotape, "standard " videotape, motivationally enhanced print booklet, or "standard" print booklet. All learners were asked to read or view the lesson to learn as much as Participants in four of the eight sessions were asked to respond to a secondary possible about the topic of the lesson. task while attending to their lesson. They were informed that their primary task was to learn as much as possible from the lesson and also to press a key on a computer keyboard as soon as they heard a beep. Participants participating in the secondary task sessions wore earplugs to avoid sound carrying throughout the room. Participants receiving one of the videotaped versions viewed the lesson as a group from a large video monitor placed in the front of the room. Participants receiving the text versions read the lesson at their own pace from individual booklets. Following completion of one the four versions of the lesson, all participants completed the mental effort questionnaire. Finally, all participants completed the post-test, then created a logo and letterhead using the software demonstrated in the lessons. The participants were provided an opportunity to apply the skills demonstrated in the lesson to create a logo and letterhead; however, these products were not analyzed. Resulta Multivariate Analysis of Variance (MANOVA) was used to determine if there were significant differences among the eight groups for post-test and self-..-eport mental effort scores. The analysis indicated several significant effects. Interaction between media and version for post-test scores. The interaction between media (text or video) and lesson version (standard or motivationally enhanced ) was significant, F (2, 107) = 4.37, p=.01. Univariate analysis indicated that the interaction effects were caused by significant differences among groups on the post-test scores, F (1, 108). 8.38, p=.005. An examination of the means (see Table 1) revealed that post-test scores were higher for the standard version of video (M= 30.58) than the motivationally enhanced version (M= 21.23) , and higher for the motivationally enhanced version of text (M=29.64) than for the standard version (M= 24.49). (See Figure 1.) Mean Post-test Scores Figure 1: 35 30 25 20 15 Text 10 Video 5 Enhanced Standard Mental effort assessment condition and recall scorea. There were also significant differences in the post-test scores between groups who responded to the secondary task measure of effort and those who only responded to the mental Univariate analysis revealed that the significant difference was due to effort questionnaire , F (2, 107)= 4.10, p=.02. significant differences in achievement test scores, F (1,108) = 6.71, p=.01. Participants who responded to the secondary 7 130 of cued recall task while reading or viewing the lesson received significantly lower scores (M= 23.33) on the post-test (See Table 1.) than those participants who only responded to the mental effort questionnaire after the lesson (M= 28.70). means Table 1: Post-test enhanced standard enhanced standard video TOTAL text text video 23.33 15.75 23.11 27.67 23.94 Secondary 10.66 11.60 task 9.53 8.77 10.80 46 8 9 12 17 28.70 24.36 34.53 30.76 25.00 Questionnaire 13.29 13.36 12.98 12.17 13.24 31 14 17 21 18 21.23 30.58 29.64 24.49 Totals 13.16 12.77 11.95 11.23 22 26 33 35 Effect of media on mental effort scores. MANOVA indicated a significant effect for media, F(2, 107)= 6.83, p=.002. Univariate analysis revealed that the significant effect was due to differences among groups on their responses to the mental effort questionnaire, F(1, 108). 11.78, p=.001. Students who received the text version of the lesson reported spending significantly more effort (M=29.53) in the lesson than those who received the videotaped version (M=26.23). (See Table 2.) Means on Mental Effort Questionnaire Table 2: Text Video 26.23 29.53 Means 4.52 5.59 SD 48 68 N Note: Possible scores ranged from 9 to 45 Secondary task data. Initially, Analysis of Variance was used to analyze differences among group means for baseline reaction times collected during the practice session. There were no significant differences in the baseline reaction times among groups for media, F( 1, 42) = 1.43. p=.24, lesson version F(1, 42)= .08. p=.78, or the interaction between media and lesson version, F(1, 42)= .31, p=.58. Since there were no significant differences among groups for the baseline reaction times (see Table 3 for mean scores on practice task), Analysis of Variance was used to determine differences among group means for reaction times to the secondary task. The interaction between media and lesson version was significant, F(1,42)= 4.17, p=.047. There also were significant differences among groups for media, F (1,42)= 7.28, p=.01, and lesson version, F(1,42)= 11.39, p =.002. An examination of the group means revealed that reaction times for those who received the video versions were significantly longer (M= 78.13) than those who received the text versions (M=55.38). In addition, the reaction time means for those who received the motivationally enhanced versions of the lesson (M=78.33) were significantly longer than those who received the standard versions of the lesson (M=52.59). (See Table 4 for means for secondary task.) Although the analysis revealed a significant intervtion between media and lesson version (see Figure 2), an examination of the means indicates that the effects of media and lesson version was much stronger than the interaction. The significant interaction effect appears to be caused by the larger difference in reaction time scores between the standard video group (M=56.27 ) and the enhanced video group (M=102.71) than between the standard text group (M= 50.( 5) and the enhanced text group (M= 62.08). 8 131 reaction scores Table 3: Mean baseline Text Video Standard 43.62 47.76 18.05 14.24 SD 9 n 17 Enhanced 41.84 53.24 25.45 27.50 SD 8 n 12 Note: reaction time measured in "tics", equivalent to 1/60 of a second times to secondary task Mean reaction Table 4: Total Text Video 52.59 56.27 50.65 Slandard 23.83 20.31 18.68 scl 26 9 17 Enhantod 78.33 62.08 102.71 40.02 34.42 36.79 9:1 20 8 12 Totals 78.13 55.38 38.04 26.41 29 17 Note: reaction time measured in "tics"; equivalent to 1/60 of a second Figure 2: Mean Reaction Times to Secondary Task 120 100 80 60 40 Text 0-- VKleo 20 0 Standard Enhanced Correlations. Correlations among scores on the three dependent measures (post-test, mental effort questionnaire, and secondary task reaction measure) were tested using a Pearson's r test of correlation. The correlation between reaction time means on the secondary task measure and the scores on the self-report of mental effort were very low, r=-.05, p=.61 (N=46). Likewise, the correlations were low between post-test scores and responses to the self-report measure of effort r=-.069, p= .46 (n=117). The correlation between post-test scores and the secondary task response times was somewhat higher, r=-.28, p= .056 (n= 46) but in a negative direction. Correlations between the number of inferences generated on the post-test and scores on the mental effort questionnaire and secondary task measure were also low, with values of r=- .06, p=.79 (questionnaire) and r= .22, p= .52 (secondary task) respectively. Post hoc analysis Several analysis were conducted to determine whether participants' prior experience with desktop publishing or Macintosh computers may have influenced participants scores on the post-test and mental effort questionnaire. When prior experience with desktop publishing (experienced/ no experience) and Macintosh computers (experienced/ no experience) were used as independent variables in a MANOVA with mental effort and post-test scores as the dependent variables, there were no significant interactions, F(2, 103)= 1.20, p=.31, differences due to prior experience with Macintosh computers, F(2, 103)= 2.17, p=.12, or desktop publishing software (F(2,103) = .32, p=.'73). Discussion The purpose of this study was to investigate a) whether college-age learners invest more effort in print-based lessons desigried to teach the skills involved in desktop publishing than in video-based versions of the lessons, b)whether learners inves' more mental effort in motivationally enhanced lessons than in "standard" lessons which lack motivational enhancements, c) the extent to which learners' self-reports correlate with their "on line" measurements of effort collected using a secondary task technique and d) the correlation between mental effort scores (from secondary task or self-report measures) and post-test scores on a recall measure. As in Beentjas and van der Voort's (1993) study, the two measures of mental effort ar eared to assess different things. Correlations between the mental effort questionnaire and the secondary task measure of effort were very low. 133 BEST COPY AVAILABLE