DOCUMENT RESUME ED 355 905 IR 015 964 AUTHOR Perkins, Bob TITLE Differences between Computer Administered and Paper Administered Computer Anxiety and Performance Measures. PUB DATE Feb 93 NOTE 14p. PUB TYPE Reports Research/Technical (143) EDRS PRICE MF01/PC01 Plus Postage. DESCRIPTORS *Anxiety; Comparative Analysis; *Computer Assisted Instruction; *Computer Assisted Testing; Computer Science Education; Graduate Students; Higher Education; Hypermedia; Inservice Teacher Education; *Performance Factors; Predictor Variables; Preservice Teacher Education; Pretests Posttests; Sex Differences; Student Attitudes; *Test Format; Undergraduate Students IDENTIFIERS *Computer Anxiety; *Paper and Pencil Tests ABSTRACT This research investigated whether computer anxiety is different if the measure is administered by computer rather than by paper and pencil. The study compared two groups of students (N=83) who were gathered from three undergraduate sections and one graduate section of a required computer class for in-service and pre-service teachers using anxiety level and performance as measures. Both groups took a written pretest; but one group was administered an anxiety scale (pre and post) and posttest on the computer using a HyperCard stack, while the other group used a paper and pencil version of these measures. Other variables, including owning a computer, graduate versus undergraduate status, previous use of computers, and gender, were also investigated. Statistical analysis of the data revealed there was no significant difference on any variable between the two test administration formats; computer ownership had an effect on both performance and anxiety; anxiety level was higher for graduate students; and females, while starting and ending with lower performance and anxiety levels, made higher gains in performance and decreased their anxiety levels. (Contains 12 references.) (ALF) *********************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. *********************************************************************** U.S. DEPARTMENT Of EDUCATION sr Office 01 Educational Research and improvement EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) O This document Las been reproduced as received from the person or organization originating it 0 Minor changes have been made to improve reproduction Quality Points of view or opinions stated in this docu- ment do not necessarily represent official 0E141 position or policy Differences Between Computer Administered and Paper Administered Computer Anxiety and Performance Measures Bob. Perkins School of Education University of Charleston, SC Charleston, SC 29424 Phone: (803) 792-5699 PERMISSION TO REPRODUCE THIS INTERNET: [email protected] MATERIAL HAS BEEN GRANTED BY Robert Perkins 2 TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) MIME REST MY Anxiety and Performance 1 Abstract This study. compares two groups of students using anxiety level and performance as measures. Both groups took a written pretest but one group was administered an anxiety scale (pre and post) and post test on the computer using a HyperCard stack while the other group used paper versions of these measures. Other variables including owning a computer, graduate versus undergraduate, previous use of computers, and gender were also investigated. Introduction With the proliferation of computers in education, and especially with computer aided instruction, computer maiiaged instruction and integrated learning systems as instruments of teaching, many factors related to them must be measured. Many schools are investing large amounts of money into Integrated Learning Systems (ILS) as a means of educating students. These systems teach, evaluate, and direct a student's education. Because hypermedia programs such as HyperCard can be used as authoring systems, classroom teachers are also using these programs more frequently in their classrooms to design lessons. Computerized testing can be an integral part of the Computer-Aided Instruction (CAI) stacks that are created by these teachers. Computers are able to grade, score, and report student grades very quickly, thus freeing the teacher to do more teaching. If there is a relationship between computer anxiety and performance, and more specifically if the testing is done by computer, the use and design of Integrated Learning Systems as well as classroom hypermedia stacks could he effected. If no relationship is found between anxiety and performance, and if performance is not effected by the evaluating being done by computer, then computer testing should be promoted. This research is investigating whether computer anxiety is different if the measure is administered by computer than if it is administered by paper and pencil. Also being assessed in this study is a computer administered performance measure versus paper and pencil administration. These comparisons were done to measure the relationship between anxiety and performance. The amount of computer anxiety and performance level exhibited may differ if the measure is administered by computer. Anxiety, if it has an effect on performance, may also be higher if the r- of Anxiety and Performance 2 performance measure, in the form of a post test, were also administered by computer. Review of the Literature The computer band wagon is rolling on faster and faster. With the efforts to quickly reform and improve education and student achievement, school systems are trying to find tile optimal way to incorporate the new technologies into the curriculum (Ryan, 199i). Are these new technologies the needed panacea for the ailing educational system? Research on teachers' attitudes related to computers are favorable for incorporation of computers into the curriculum (Dupagne & Krendl, 1992). Computers are seen to have much potential for classroom instruction (Dupagne & Krendl). These technologies have included stand-alone microcomputers and related peripherals and complex and expensive Integrated Learning Systems (ILS). Many things can effect the success of these investments. Research has shown that factors are: amount of teacher training and amount of administrative commitment (Ryan, 1991). Many Integrated Learning systems incorporate computerized testing into their systems. Immediate feedback, test correction, student diagnosis, standardized presentation, ease of data storage, and records management are all advantages of these systems (Gwinn & Beal, 1987/1988; Davis & Cowles, 1989; Van Horn, 1991). Many schools do not have the finances to be able to purchase these large computer systems. Many educators use microcomputers offline to create tests (Gwinn & Beal, 1987/1988) with word processing or desktop publishing programs or test generating programs. Microcomputers can also be used to administer tests, grade them, and keep the records current. Many teachers are also incorporating hypermedia-created lessons in their classroom by using them as authoring systems (Johnson, 1991). These hypermedia programs can also be used to create tests that can also be scored the computer and generate a student progress report. The variables that effect large Integrated Learning Systems may also effect the students using hypermedia generated tests. Further research on computer testing is necessary because of problems related to previous -,tudies. In a meta-analysis of 40 studies, Ryan (1991) states that it has been difficult to determine variable relationships ID student achievement because of under- reporting of study and sample characteristics. Underreported variables in these studies included: Anxiety and Performance 3 characteristics of color, sound, music, speech generation, and input device (Ryan). There has been much research involving attitudes toward computers. Both test anxiety and computer anxiety are important concepts when considering computerized testing and they may be related. Most research in the area of attitude relies on self-administered surveys or interviews (Dupagne & Krendl, 1992). Attitudes of prospective teachers, including anxiety, have been found to be related to the computer literacy level of those prospective teachers (Hignite & Echternacht, 1992). Computer anxiety has been a measure in many studies (Reed & Palumbo, 1987/1988; Liu, Reed & Phillips, 1990; Davis & Cowles, 1989; Banda los & Benson, 1990; Gwinn & Beal, 1987/88; Dimock & Cormier, 1991). Gwinn & Beal (1987/1988) in their anatomy and physiology classes, used an individualized, self paced instruction and testing system. Most of the students had little or no computer experience prior to these classes. They reported that students in their classes (nursing, allied health, medical assisting, dietetics, and physical education students) using computer administered tests found that general test anxiety using the Mandler and Sarason Test Anxiety Questionnaire increased slightly (( =.001 for females and .05 for males) from the beginning of the semester to the end. Test anxiety was also found to be negatively correlated with achievement. Gwinn & Beal (1987/1988) also state that there are two types of test anxiety, facilitating and debilitating test anxiety. They found that paper administered tests initially had higher facilitating and computer administered tests had higher debilitating anxiety. A survey done fourteen weeks later reported that this had inverted, computer administered tests had higher facilitating and paper tests had higher debilitating anxiety. Computer anxiety can influence test scores (Dimock & Cormier, 1991). This effect could be compounded if the test is administered by computer. Differences have been reported between traditional testing and computerized testing, but Dimock & Cormier state that this could be because of format differences between the tests. Typically, conventional tests present a group of questions at one time while computerized tests only include a single question on the screen at one time. Their research, however, indicated that this was true for an initial test, but a second test with different presentation formats showed no significant differences between the groups. Anxiety and Performance 4 Computer anxiety may have an effect on student performance when students are required to take tests administered by computer. Studies have snown that computer anxiety is real and measurable, and that it does decrease when the student is involved with learning computer applications or computer literacy (Reed & Palumbo, 1987/1988; Liu, Reed & Phillips, 1990). However, not all students using CAI also use application programs or have taken classes in computer literacy. Research has also been conducted to determine the effect a computer training module on anxiety (Reed & Palumbo, 1987/1988; Liu, Reed, & Phillips, 1990) using a modified version of Spielberger's Self-Evaluation Questionnaire as as a pre and post measure. These studies did not compare administration of the anxiety measure when that measure was administered by computer. If computer anxiety is real and measurable, then it may be more intense if the measurement is conducted by computer. Computer anxiety has been compared with math anxiety, especially when gender of students enters into the picture (Banda los & Benson, 1990). It was speculated that these two concepts may be related because of personality variables. Since women have higher levels of math anxiety, their computer anxiety level may also be higher (Banda los & Benson). It was found that after administering the Computer Anxiety Scale, however, that when they applied a Goodness of Fit Index, that there was no difference between males and females or undergraduate and graduate students enrolled in educational measurements and statistics courses. Spielberger's State-Trait Personality Inventory was developed to assess three emotions: anxiety, curiosity, and anger (Ben-Zur & Zeidner, 1988). Anxiety is further broken down into two components-state anxiety which is characterized as feelings of tension and is accompanied by increased physiological arousal and trait anxiety which is representative of the frequency and proneness to state anxiety (Ben-Zur & Zeidner). This study sought to compare Israeli students to American students (using Spielberger's data). They found that both cultures reported higher trait anxiety for females (and almost three times as high for Israeli females compared to American females) than males, but that state anxiety showed no difference for gender or culture Davis & Cowles (1989) reported that students that took tests on computer gave responses that were related to low trait anxiety as measured by Endler's Multidimensional Anxiety Scales and Anxiety and Performance 5 gave socially desirable answers on the Marlow-Crowne Social Desirability Scale. Students who were administered paper versions of the test did not have this interaction. They speculated that individuals that give socially desirable answers can "fake good" on the scales to invite approval. Sample The students in this study (N=83) were from three undergraduate sections and one graduate section of a required class for in-service and pre-service teachers called Computers for Teachers. This is an introductory course that teaches students how to word process, create and use databases, use hypermedia to create instruction, and evaluate software. Students are also exposed to technologies such as CD-ROM, telecommunications, laserdiscs, and video and audio digitizing. Demographic information was collected from the students including gender, major, their computer experience before this class (word processing, database, and computer programming) and whether they own their own computer. DIRECTIONS- Think about yourself and hov you feel about computers when you respond to each statement listed on the folloving screens_ Ansver by pressing the button under the number that accurately describes hov you feel. If you have not used computers, imagine hov you might feel the first time you use one. 1=3trongly Disagree 2=Disagree 3=Agree 4=Strongly Agree 2 3 4 1 I feel calm when I think about computers. 0 0 0 0 press if you need to return to previous previous question Figure 1 All students were administered a pretest consisting of questions from the previous MST COPY iVAILABLE Anxiety and Performance 6 semester's exams by paper. One undergraduate and the graduate course also were administered a paper and pencil version of the revised Speilberger's Self-Evaluation Questionnaire The two . remaining undergraduate sections were administered the computer version of the revised Self- Evaluation Questionnaire on computer using a stack created by the author using HyperCard (see Figure 1 for a sample card with the question). The stack asked for the student's name and social security number, and then provided the same written instructions (which stayed on the screen while each question was on the screen) as the paper version of the survey. Students answered the questions by moving the mouse over the selected answer, and clicking the mouse. After the students were finished, the stack closed itself after saving a text file with all of the students' answers and their score. Throughout the stack, the students were allowed to return to previous answers if they wanted to change their answer. Results The pre-test was used to determine if the groups were equivalent before the instruction took place in the four classes. An ANOVA revealed no significant difference between the computer and and the paper and pencil group F(1,81), p=.71. Based on these results, the computer group (n=44) and non-computer group (n=39) were equivalent at the beginning of the study. A two factor (group X Anxiety as a repeated measure) ANOVA was performed using pre and post Self-Evaluation Questionnaire scores. This anxiety measure can produce scores between 20 and 80 with 80 being the high anxiety level. Since no interaction between group and measure was found F(1, 81), 12.-. .95 it was determined that there was no difference between the groups in their level of anxiety either at the pre or post instructional administration of the anxiety measure. There was, however, a significant difference in anxiety apparent, F(1,81), D=.0001, as it was reduced from a mean of 48.38 to 37.95. The the same questions that were administered as a pretest on paper to all students was administered as a post test, but the same group that took the computer anxiety measure took the post test administered by computer. Again, no significant difference between groups was produced, F(1,81), p=.99. There was a significant gain frOm pre to post test F(1,81), a=.0001. Students scores went from a mean of 46.7 to 80.04 on a test that had 100 for a maximum score. Since there were no differences between the computer and non computer groups, the data U Anxiety and Performance 7 was collapsed to compare it to the demographic data. These analyses compared the dependent measures of anxiety and performance against the independent measures of gender, owning a computer, previous computer experience, and undergraduate/graduate scat's. Other previous studies had mentioned these as potential factors that could effect anxiety and performance. In comparing those students who owned computers to those who did not on the performance measure, there was a significant difference between the groups, F(1,81),12=.02 and also a gain for the repeated measure, F(1,81), g= 0001. The students who owned computers (n=18) had a higher mean at pre and post test (84.22 and 54.44 respectively) than those that do not own computers (n=65) at pre and post test (78.89 and 44.55 respectively). There was no interaction because both groups gained equally on the test. The same was found to be true for these two groups on the anxiety measure. Those students that owned computers were significantly lower on the anxiety measure, F(1,81),12=.005 than those who did not (mean = 41.11 and 50.39 respectively). They also ended the study at a lower anxiety level (mean= 33.11 and 39.11 respectively). No significant differences were found between undergraduate students (n=64) and graduate students (n=19) on the performance measure at pm test (mean = 47.70 and 46.38 respectively) and post test (mean = 81.26 and 79.69 respectively). Both groups started equal and ended equal to each other. Both of these groups also showed no significant differences at the beginning and end of the study on the anxiety measure, but both significantly reduced their anxiety level. The anxiety level of graduate students was slightly higher than undergraduates at the start of the study (mean= 49.53 and 44.32 respectively) and at the end of the study (mean= 38.81 and 35.05, respectively), but not significantly F(1,81), p=.08. 5 Anxiety and Performance 8 Prior Computer Experience 90 80 70 60 II experience c4 50 II no experience 40 30 pre-test post-test Figure 2 Computer experience for the purpose of this paper was defined as any prior experience using word processing, data base programs, desktop publishing, or computer programming for at least a year. A two factor (prior experience vs. performance) ANOVA using the test as a repeated measure produced an interaction, F(1, 81),12=.02 (see Figure 2). Those students that did not have prior experience (n=52) had a lower mean on the pre-test than those who had eviw..rience (n=31) (38.26 and 51. 73), a slightly lower post test mean (76.90 and 81.92 respectively), and gained more (38.65 and 30.19 respectively). Prior Computer Experience 80 70 60 1. 50 II experience 40 rip II no experience 30 20 pre-anxiety post-anxiety Figure 3 A two factor (prior experience vs. anxiety) ANOVA using the anxiety measure as a repeated measure also produced an interaction, F(1, 80,12..02 (Figure 3). Students without prior ti