Volume 8, Number 1 http://isedj.org/8/1/ Jan 11, 2010 In this issue: Critical Thinking Measurement in ICT Li-Jen Shannon Solomon Schneider Sam Houston State University Sam Houston State University Huntsville, TX 77341 Huntsville, TX 77341 Judith F. Bennett Sam Houston State University Huntsville, TX 77341 Abstract: This study examined the status of critical thinking (CT) and reasoning skills in infor- mation communication and technology (ICT) for 190 college students in a higher education system. It analyzed how the students performed in CT, reasoning, and internet copyright and ethical issues. A CT assessment was designed to analyze the CT and reasoning skills. The findings showed that the students were not capable of (a) interpreting the chart and question accordingly; (b) processing problem-solving and proposing the proper solutions for the scenario; (c) composing a proper legal action toward the copyright issues; and (d) recognizing internet ethics to treat data legally. Due to limited resources of CT measurements in ICT fields, this paper might be used as the significant evidence promoting students’ CT and reasoning skills in educational systems. Keywords: critical thinking, copyright, Internet ethics, information ethics, ICT, CAT Recommended Citation: Shannon, Schneider, and Bennett (2010). Critical Thinking Measurement in ICT. Information Systems Education Journal, 8 (1). http://isedj.org/8/1/. ISSN: 1545-679X. (A preliminary version appears in The Proceedings of ISECON 2009: §1753. ISSN: 1542-7382.) This issue is on the Internet at http://isedj.org/8/1/ ISEDJ 8 (1) Information Systems Education Journal 2 The Information Systems Education Journal (ISEDJ) is a peer-reviewed academic journal published by the Education Special Interest Group (EDSIG) of the Association of Information Technology Professionals (AITP, Chicago, Illinois). • ISSN: 1545-679X. • First issue: 8 Sep 2003. • Title: Information Systems Education Journal. Variants: IS Education Journal; ISEDJ. • Phys- ical format: online. • Publishing frequency: irregular; as each article is approved, it is published immediately and constitutes a complete separate issue of the current volume. • Single issue price: free. • Subscription address: [email protected]. • Subscription price: free. • Electronic access: http://isedj.org/ • Contact person: Don Colton ([email protected]) 2010 AITP Education Special Interest Group Board of Directors Don Colton Thomas N. Janicki Alan R. Peslak Brigham Young Univ Hawaii Univ NC Wilmington Penn State EDSIG President 2007-2008 EDSIG President 2009-2010 Vice President 2010 Scott Hunsinger Michael A. Smith Brenda McAleer George S. Nezlek Appalachian State High Point Univ U Maine Augusta Grand Valley State Membership 2010 Secretary 2010 Treasurer 2010 Director 2009-2010 Patricia Sendall Li-Jen Shannon Michael Battig Mary Lind Merrimack College Sam Houston State St Michael’s College North Carolina A&T Director 2009-2010 Director 2009-2010 Director 2010-2011 Director 2010-2011 Albert L. Harris S. E. Kruck Wendy Ceccucci Kevin Jetton Appalachian St James Madison U Quinnipiac University Texas State JISE Editor ret. JISE Editor Conferences Chair 2010 FITE Liaison 2010 Information Systems Education Journal Editors Don Colton Thomas Janicki Alan Peslak Brigham Young U Hawaii Univ NC Wilmington Penn State University Editor Associate Editor Associate Editor ThispaperwasselectedforinclusioninthejournalastheISECON2009BestPaper. Theacceptance rateistypically1%forthiscategoryofpaperbasedonblindreviewsfromsixormorepeersincluding three or more former best papers authors who did not submit a paper in 2009. EDSIG activities include the publication of ISEDJ and JISAR, the organization and execution of the annual ISECON and CONISAR conferences held each fall, the publication of the Journal of Information Systems Education (JISE), and the designation and honoring of an IS Educator of the Year. •TheFoundationforInformationTechnologyEducationhasbeenthekeysponsorofISECON over the years. • The Association for Information Technology Professionals (AITP) provides the corporate umbrella under which EDSIG operates. (cid:13)c Copyright 2010 EDSIG.Inthespiritofacademicfreedom,permissionisgrantedtomakeand distribute unlimited copies of this issue in its PDF or printed form, so long as the entire document is presented, and it is not modified in any substantial way. (cid:13)c 2010 EDSIG http://isedj.org/8/1/ Jan 11, 2010 ISEDJ 8 (1) Shannon, Schneider, and Bennett 3 Critical Thinking Measurement in ICT Li-Jen Shannon [email protected] Solomon Schneider [email protected] Judith Bennett [email protected] Department of Computer Science Sam Houston State University Huntsville, TX 77341 Abstract This study examined the status of critical thinking (CT) and reasoning skills in information communication and technology (ICT) for 190 college students in a higher education system. It analyzed how the students performed in CT, reasoning, and internet copyright and ethical is- sues. A CT assessment was designed to analyze the CT and reasoning skills. The findings showed that the students were not capable of (a) interpreting the chart and question accor- dingly; (b) processing problem-solving and proposing the proper solutions for the scenario; (c) composing a proper legal action toward the copyright issues; and (d) recognizing internet eth- ics to treat data legally. Due to limited resources of CT measurements in ICT fields, this paper might be used as the significant evidence promoting students’ CT and reasoning skills in edu- cational systems. Keywords: Critical Thinking, Copyright, Internet Ethics, Information Ethics, ICT, CAT 1. INTRODUCTION that individuals have the intellectual abilities of reasoning and critical thinking to con- Many studies show college students’ infor- struct a framework for learning how to learn. mation communication technology (ICT) le- vels are not as competent as the public When measuring the levels of ICT, the appli- perceives (“How the new generation,” 2007; cation skill measurement seems to be the Kelly & Haber, 2006; Shannon, 2008; Sulli- most direct and simple category to be as- van, 2008). Many educators assume tech- sessed. However, when measuring the stu- nology skills are purely technical; therefore, dents’ critical thinking and reasoning skills in since these students seem adept with tech- ICT, we have not found valid instruments or nology they do not need any formal instruc- studies which applied to measuring the stu- tion (Allen, 2007). dents’ higher level of cognitive learning in ICT. The Information Literacy Competency Stan- dards for Higher Education states that “In- National Educational Technology Standards formation literacy is a key component of, for Students (NETS) entitled six categories and contributor to, lifelong learning (Asso- to assess the students’ ICT proficiency for PK ciation of College and Research Libraries – 12 systems. The standards are listed as [ACRL], 2009)”. ACRL (2009) emphasizes follows: that colleges and universities should provide 1. Creativity and Innovation the foundation for continued growth throughout the students’ careers to ensure 2. Communication and Collaboration (cid:13)c 2010 EDSIG http://isedj.org/8/1/ Jan 11, 2010 ISEDJ 8 (1) Shannon, Schneider, and Bennett 4 3. Research and Information Fluency tive learning outcome (Bean, 2001; Ogle, 1993; Vanderpool & Robinson, 2003). 4. Critical Thinking, Problem Solving, and Decision Making While facing a limited resource of critical thinking assessment in ICT fields, the Educa- 5. Digital Citizenship tional Testing Service (ETS) claimed that the 6. Technology Operations and Concepts iSkill assessment “is the only ICT literacy (ISTE, 2009) test that assesses critical thinking in the dig- ital environment (iSkill, 2009)”. Irvin (2007) The profiles of NETS for students were de- stated that the iSkill assessment focuses on tailed for each grade level group in each the cognitive problem-solving and critical- standard mentioned above. However, the thinking skills associated with using technol- implementation and assessment process ogy to handle information without a mul- seems to lack a connection for each stan- tiple-answer format. We are unable to re- dard. view the reliability and validity of research With more than 30 years of teaching in ICT supporting iSkill assessments in CT and rea- fields from the authors, we found that a ma- soning skills. Several institutions such as the jority of college students are not proficient in California State University, the University of all of the above standards, even though a Wisconsin, and others that applied the iSkill majority of them took one or more computer assessment; but the supporting documents courses before they enrolled into college are still not in place. (Shannon, Bennett, & Schneider, 2009). When compared to other instruments that From the introductory computer courses, we measure critical thinking and intellectual found that the students’ ICT skill levels can performance evaluated by a broad spectrum be improved in many ways. However, we of faculty across the U.S. in Science Tech- encountered an enormous hurdle of motivat- nology Engineering & Mathematics (STEM) ing the students to apply their critical think- and non-STEM disciplines, the CAT instru- ing and reasoning skills in this digital life ment proved to have a high face validity environment. (Stein, Haynes, Redding, Ennis, & Cecil, Sullivan stated that the millennial and post- 2007). Many studies emphasized how writ- millennial generations of young adults “don’t ing is linked to learning and critical thinking understand ethical uses of technology or the (Bean, 2001; Vanderpool & Robinson, concept of intellectual property rights. Their 2003). The pencil and paper form with a critical thinking skills are notoriously weak short answer essay method of CAT provided and their reflective capabilities sorely lacking the tool for our students using writing to (2008).” perform their thinking process. With a burning desire to study how we can Critical Thinking help our students practice their CT skills, we In a college learning environment, the stu- implemented the theory from the critical dents are expected to think at higher levels thinking assessment test (CAT) funded by and demonstrate their knowledge beyond the National Science Foundation to conduct that given in the classrooms. Jalongo, this study. Twiest, and Gerlach (1999) stated critical ICT Literacy Skill Assessment thinking evolves with the following stages: The National Assessment data have hig- • Apply: use knowledge and under- hlighted the problem that has emerged from standing to complete a practical an overemphasis on skill instruction and task. multiple-answer testing. Ogle (1992) stated • Analyze: break things down into that students can select the correct answer, their component parts. but lack the ability to explain why they chose their answers or to substantiate their • Synthesize: combine and integrate thinking about the choices they make. By various sources of information. using case study scenarios, the students • Evaluate: assess the value, merit, or have the opportunity to map their thinking worth of something. process and provide a higher level of cogni- (cid:13)c 2010 EDSIG http://isedj.org/8/1/ Jan 11, 2010 ISEDJ 8 (1) Shannon, Schneider, and Bennett 5 Paul (1995) defined critical thinking as a When facing the copyright and ethical issues self-directed, self-disciplined, self- of the internet, it is vital for students to ex- monitored, and self-corrective thinking skill ercise their CT and reasoning skills to enable which guides the thinker who possesses a them to make correct decisions concerning set of effective dispositions. When the stu- the legal intricacies of copyright laws as well dents internalize their CT competency, they as ethical considerations. will develop their ability to: The U.S. Copyright Office defined copyright • raise vital questions and problems “is a form of protection provided by the laws of the United States (title 17, U. S. Code) to • gather and assess relevant informa- the authors of “original works of authorship,” tion including literary, dramatic, musical, artistic, • come to well-reasoned conclusions and certain other intellectual works” (US and solutions Copyright Office, 2009). Since the material is original and not borrowed or quoted from • think open-mindedly within alterna- another writer, then these writings are con- tive systems of thought, and sidered the property of the writer. Since the • communicate effectively with others writings are from the intellect of the individ- in figuring out solutions to complex ual(s) involved, the term “intellectual prop- problems (Paul, 1995). erty” was coined (WIPO, 2008). “Intellec- tual property refers to anything created by By definition, critical thinking applies skills the mind, such as literary works (books, that contribute to information literacy. Criti- poems, essays), artwork (drawings, paint- cal thinking and information literacy both ings), inventions, ideas, logos or symbols, require making a distinction between as- names, designs, and images or photographs sumption and fact, suspending personal opi- (Taylor, Arth, Solomon, & Williamson, 2007, nion and bias in favor of objectivity, and p.179)”. considering issues from multiple perspec- tives and in adequate depth (Taylor, Arth, To infringe the copyright or right of the au- Solomon, & Williamson, 2007). It includes thor, infringement of copyright is defined as possible processes of reflecting upon a tang- “Anyone who violates any of the exclusive ible or intangible item in order to form a sol- rights of the copyright owner … or who im- id judgment that reconciles scientific evi- ports copies or phonographic records into dence with common sense. the United States … (US Copyright Office, 2009)”. Due to the convenience of using the Without critical thinking skills, an individual internet to download resources, many in- is at a disadvantage and may make a wrong stances of unethical behaviors were related decision because of their inability to discern to the ease of copying internet resources, accurate, precise, relevant and logical infor- especially in the academic settings (Karim, mation. Zamzuri, & Hidayah Ahmad Nor, 2009). The personality variables and unethical Internet Copyright and Internet ethical behaviors were identified as conceptualized issues through Internet-triggered academic disho- It is imperative that the college students are nesty: (1) agreeableness, (2) conscientious- able to use the critical information resources ness, and (3) emotional stability (Karim et in the higher educational system. With the al., 2009). new generation of computer literate students To identify the ethical theories, Quinn and the vast amount of networked informa- (2006) listed various theories, such as: Sub- tion available it is necessary to develop the jective relativism, Cultural relativism, Divine ability to use information resources properly command theory, Categorical imperative, (Kwon, 2008). Act Utilitarianism, Rule utilitarianism, Social Kwon (2008) stated that critical thinking contract theory…etc. Each theory presented dispositions work in carrying out information the pro and con of supporting the ethical search tasks which enables the student to issues. While the arguments sustain socially, retrieve their existing knowledge and per- politically, or psychologically, it is vital for form cognitive tasks more effectively. the students processing their critical thinking (cid:13)c 2010 EDSIG http://isedj.org/8/1/ Jan 11, 2010 ISEDJ 8 (1) Shannon, Schneider, and Bennett 6 and reasoning skills to maintain their integri- Evaluating Ideas and Other Points of View ty in this networked society. • Identify and evaluate evidence for a theory. 2. METHODOLOGY • Identify new information that might In spring 2009, we implemented the theo- support or contradict a hypothesis. retical framework from the Critical thinking Assessment Test (CAT) to conduct our • Explain how new information can study. change a problem. Reliability Learning and Problem Solving The CAT instrument has been applied by a • Separate relevant from irrelevant in- broad range of institutions across the coun- formation. try since 2007. The National Science Foun- • Integrate information to solve prob- dation (NSF) has provided support for many lems. CAT activities. “The CAT Instrument is a unique tool designed to assess and promote • Learn and apply new information. the improvement of critical thinking and • Use mathematical skills to solve real-world problem solving skills (Critical real-world problems. Thinking Assessment Test, 2009).” Communication The NSF is supporting efforts to disseminate the CAT instrument to a diverse group of • Communicate ideas effectively (CAT, institutions (through train-the-trainer work- 2009). shops) to prepare representatives from 20 The CAT instrument utilized the graph and institutions to lead scoring workshops for the case scenario to assess the students’ level in CAT instrument at their own institution from evaluating the given information. Merging 2007 to 2010 (Critical thinking Assessment with the students’ reasoning and problem Test, 2009). Two of our authors participated solving skills, the CAT instrument is well de- in this train-the-trainer workshop, and re- fined to analyze students’ critical thinking ceived the CAT instrument together with skill levels. support for conducting two scoring work- shops on their own campus for the founda- Following the guidelines of the CAT instru- tion of science project under the Quality En- ment, we designed a small scale test to ana- hancement Program (QEP) to meet the crite- lyze the above four domains from CAT. In ria of the Southern Association of Colleges addition to use graph and software product and Schools (SACS). The other author also scenario, we included the second scenario of participated in the campus scoring activity copyright statement to assess the students for the QEP project in spring 2009. with two cases. To summarize the findings, we reported the students’ ability to (a) in- Instrument terpret the graph and questions and re- The CAT instrument provided by the National sponse precisely, (b) use a case scenario to Science Foundation’s CCLI (Course, Curricu- process problem solution/s, and (c) use an lum, and Laboratory Improvement) Program online copyright statement to determine assesses the following critical thinking skills: whether the student will be tempted to in- fringe the copyright legal issues. Evaluating Information The scenario of this CT survey is based on • Separate factual information from in- the statement listed on Appendix. ferences. Research Hypotheses • Interpret numerical relationships in graphs. H1: The students are capable of interpreting the chart and question accordingly. • Understand the limitations of corre- lated data. H2: The students are capable of processing problem-solving and propose the proper so- • Identify inappropriate conclusions. lutions for the scenario. (cid:13)c 2010 EDSIG http://isedj.org/8/1/ Jan 11, 2010 ISEDJ 8 (1) Shannon, Schneider, and Bennett 7 H3: The students are capable of composing This question had a maximum value of three a proper legal action toward the copyright points. One point was issued for choosing issues. “Product B”, and an additional point was is- sued for choosing “Product D”. One more H4: The students are capable of recognizing point was issued, if the participants stated internet ethics and to retrieve data legally. the reason that “both products provide spy- Grading Procedures ware and adware protection”. A zero score was issued for any other answer. To analyze the critical thinking skills, a mixed method was designed for this study. After this problem-solving question, we add- The grading process was to have multiple ed one more new condition and asked the graders to review the written answers and participants to choose the final product. quantify the answers to the score of 0 to 5 Question 4: Due to the budget limita- respectively. A third scorer was required to tion, Mary will not be able to afford review the items when the assigned scores more than $65.00 to protect her sys- were not identical from the other two grad- tems. According to Mary’s circums- ers. The final score was determined by the tances, please recommend one final mean of the three scores. The Statistical product and explain why the product Package for the Social Sciences (SPSS, Ver- will fit Mary’s needs. sion 15.0) was then used to analyze the numerical data. A descriptive method was This question had a maximum value of five also implemented to determine the degree points. One point was issued for choosing of responses from the data. “Product D”. Four additional points were possible; one point for stating the reason of Question 1: Briefly summarize the pat- “under $65.00”, one point for stating the tern of the products in this graph. reason of “Trojan Horse Protection”, one This question had a maximum value of two point for stating the reason of “Adware Pro- points. One point was issued for providing tection”, and one point for stating the reason the statement “increase rate”. An additional of “Spyware Protection”. one point was issued for providing the The next two questions were based on a statement “from Product A to Product E”. A copyright statement from the internet (see zero score was issued for providing any sug- Appendix). gestion or other statement not related to the above statements. Question 5: Will you download the files from this website to post them on your Question 2: Briefly explain what product website? Mary should choose from and provide the reason/s. There was only one point issued for this question: one point for “No” and zero points Question 2 was designed to see whether the for “Yes”. students can provide reasonable solutions based on the knowledge they had on com- Question 6: Please provide the reasons, puter security issues. This question had a why you chose the specific answer for maximum value of two points. One point the question above. was awarded for stating “no product should This last question had a maximum value of be recommended”. An additional point was two points. One point was issued for stating issued for stating the reason that “the chart the reason “the foundation does not provide is insufficient for recommending any prod- any warranty regarding the copyright sta- uct”. A zero score was issued for providing tus”. An additional one point was issued for any other statement and recommendation. stating the reason “the user should be re- To continue question 2, we provided a table sponsible for obtaining the copyright status”. of specification for each product for question A zero score was issued for any other state- 3 that follows: ment. Question 3: Based on Mary’s needs, Participants please recommend 2 products that Mary The survey was conducted during the spring should consider choosing from and ex- 2009 semester having an enrollment of 298 plain the reasons. (cid:13)c 2010 EDSIG http://isedj.org/8/1/ Jan 11, 2010 ISEDJ 8 (1) Shannon, Schneider, and Bennett 8 students from 1111 sseeccttiioonnss ooff introductory sshhoowweedd tthhaatt tthheerree wweerree oonnllyy ttwwoo oouutt ooff 119900 computer courses. TThheerree wweerree 190 students ssttuuddeennttss wwhhoo ssttaatteedd tthhaatt tthhee ddaattaa wwee pprro- (63.8%) that vvoolluunnttaarriillyy ccoommpplleetteedd tthhiiss ssuur- vviiddeedd wweerree nnoott ssuuffffiicciieenntt ttoo pprrooppoossee aannyy vey at the bbeeggiinnnniinngg ooff tthhee sseemmeesstteerr. The product from the chart.. TThhee rreesstt ooff ssttuuddeennttss participants’ majors wweerree ffrroomm the College of (99%) ssuuggggeesstteedd pprroodduucctt EE as having the AArrttss aanndd SScciieenncceess,, CCoolllleeggee ooff CCrriimmiinnaall JJuus- hhiigghheesstt ““vviirruuss pprrootteeccttiioonn rraattee””.. The students ttiiccee,, aanndd CCoolllleeggee ooff EEdduuccaattiioonn. were not aware that ttoo ddeeppeenndd ssoolleellyy oonn tthhee vviirruuss pprrootteeccttiioonn rraattee wwaass nnoott ssuuffffiicciieenntt ttoo 33.. FFIINNDDIINNGGSS make a suggestion.. WWee wwoouulldd aappppllaauudd tthhee oonnllyy ttwwoo ssttuuddeennttss wwhhoo ppooiinntteedd oouutt tthhee ddiif- A descriptive analysis aanndd ccoorrrreellaattiioonn tteesstt ffeerreenncceess ooff mmaallwwaarree aammoonngg ccoommppuutteerr vviirruuss,, was applied iinn tthhiiss ssttuuddyy.. The findings will TTrroojjaann hhoorrssee,, aanndd ssppyywwaarree.. be shown in the sub-hheeaaddeerrss of Graph and Pattern, Problem-SSoollvviinngg,, CCooppyyrriigghhtt aanndd SSuuggggeessttiioonn ffoorr PPrroodduucctt SSeelleeccttiioonn Ethical IIssssuueess,, OOvveerraallll CCTT SSccoorreess,, aanndd CCoorrrre- lations. 1.0% Graph and Pattern Score 1-Correct suggestions QQuueessttiioonn 11:: BBrriieeffllyy ssuummmmaarriizzee tthhee ppaatttteerrnn ooff 99.0% the products in this graph. Score 0 -Wrong 38.9% of the ppaarrttiicciippaannttss ddiidd nnoott pprroovviiddee suggestions any correlations shown iinn tthhee cchhaarrtt.. 1188..99%% of the ppaarrttiicciippaanntt pprroovviiddeedd a partial state- ment from the chart. TThheessee rreessuullttss explained tthhaatt aapppprrooxxiimmaatteellyy 5588%% ooff tthhee ppaarrttiicciippaannttss Figure 2. Suggestion PPrroodduucctt SSeelleeccttiioonn DDiissttrriibbuuttiioonn failed to provide a pprrooppeerr ssttaatteemmeenntt ttoo eex- plain the ppaatttteerrnn ooff pprroodduuccttss sshhoowweedd iinn tthhee QQuueessttiioonn 33:: BBaasseedd oonn MMaarryy’’ss nneeeeddss,, pplleeaassee chart. A majority ooff tthhee rreessppoonnsseess pprreeddiicctted rreeccoommmmeenndd 22 pprroodduuccttss tthhaatt MMaarryy sshhoouulldd or suggested tthhee pprroodduucctt//ss ffoorr tthhiiss sscceennaarriioo ccoonnssiiddeerr cchhoooossiinngg ffrroomm aanndd eexxppllaaiinn tthhee rreea- wwhhiicchh tthhee ssttuuddeennttss wweerree nnoott aasskkeedd ttoo pprro- sons. vviiddee tthheeiirr ooppiinniioonnss ffoorr tthhiiss qquueessttiioonn (see Figure 1). With aa pprrooppeerr ssppeecciiffiiccaattiioonn lliisstteedd iinn tthhee ttaabbllee for references, 1122..11%% ooff ppaarrttiicciippaannttss that correctly chose products BB aanndd DD bbaasseedd oonn IInntteerrpprreettiinngg NNuummeerriiccaall tthhee sscceennaarriioo.. 2266..33%% ooff tthhee ppaarrttiicciippaannttss RReellaattiioonnsshhiippss iinn GGrraapphhss cchhoossee oonnee ooff tthhee ccoorrrreecctt pprroodduuccttss.. OOvveerraallll,, 87.9% of students ffaaiilleedd ttoo ssuuggggeesstt tthhee SSccoorree 00 -Answers were not propeerr ssoolluuttiioonnss bbaasseedd oonn tthhee nneeeeddss pprro- rreellaatteedd ttoo tthhee qquueessttiioonn.. ppoosseedd iinn tthhiiss ccaassee ssttuuddyy sscceennaarriioo (see Figure 42.1% 38.9% 3). SSccoorree 11 -One condition was rreeppoorrtteedd.. IInntteeggrraattiinngg IInnffoorrmmaattiioonn ffoorr 18.9% SSuuggggeesstteedd PPrroodduuccttss SSccoorree 22 -Two conditions wweerree rreeppoorrtteedd.. Score 0 -Incorrect Figure 1. IInntteerrpprreettiinngg NNuummeerriiccaall RReela- 12.1% tionships in Graphs Distribution Score 1 -One 24.2% correct condition 26.3% Problem-Solving 37.4% Score 2 -Two QQuueessttiioonn 22:: BBrriieeffllyy eexxppllaaiinn wwhhaatt pprroodduucctt correct conditions MMaarryy sshhoouulldd cchhoooossee ffrroomm aanndd pprroovviiddee tthhee Score 3 -Three reason/s. correct conditions This is the question wwhheerree we asked the par- ttiicciippaannttss ttoo pprroovviiddee tthheeiirr ssuuggggeessttiioonn ooff wwhhaatt Figure 3. IInntteeggrraattiinngg IInnffoorrmmaattiioonn ffoorr pprroodduucctt tthheeyy wwoouulldd rreeccoommmmeenndd.. The results SSuuggggeesstteedd PPrroodduuccttss DDiissttrriibbuuttiioonn (cid:13)c 2010 EDSIG http://isedj.org/8/1/ Jan 11, 2010 ISEDJ 8 (1) Shannon, Schneider, and Bennett 9 QQuueessttiioonn 44:: AAccccoorrddiinngg ttoo MMaarryy’’ss cciirrccuumms- ttaanncceess,, pplleeaassee rreeccoommmmeenndd oonnee ffiinnaall pprroodduucctt EEtthhiiccaall RReeaassoonniinnggss and explain why tthhee pprroodduucctt wwiillll ffiitt MMaarryy’’ss needs. 4.2% Score 0 - WWiitthh tthhee ssppeecciiffiicc iinnssttrruuccttiioonnss added in the Incorrect question, 4455..99%% ooff ppaarrttiicciippaannttss ffaaiilleedd ttoo cchhoooossee tthhee ccoorrrreecctt pprroodduucctt aanndd pprroovviiddee rreea- 36.8% Score 1 - One sonable statements. OOnnllyy 21% of partici- 58.9% Condition pants were able to cite rreeaassoonnss ffoorr cchhoooossiinngg the correct product ((sseeee FFiigguurree 4). Score 2 -Two RReeaassoonniinngg ffoorr SSuuggggeesstteedd FFiinnaall conditions PPrroodduucctt Figure 6.. EEtthhiiccaall RReeaassoonniinngg DDiissttrriibbuuttiioon Score 0 -Incorrect Overall Scores 4.7% 16.8% Score 1 -1 Reason AA ttoottaall ooff ppoossssiibbllee 1155 ppooiinnttss wwaass aavvaaiillaabbllee from this CT study.. TThhee mmeeaann ooff tthhee rrees- 45.8% Score 2 -2 Reasons ponses was 5.01 (N=190, sseeee FFiigguurree 7). The 6.3% ddeessccrriippttiivvee ssttaattiissttiiccss aanndd hhiissttooggrraamm eexxhhiibbiitteedd 14.7% 11.6% Score 3 -3 Reasons the overvieeww ooff CCTT lleevveellss iinn tthhiiss ssttuuddyy.. Score 4 - 4 30 Reasons Figure 4. RReeaassoonniinngg ffoorr SSuuggggeesstteedd Final PPrroodduuccttss DDiissttrriibbuuttiioonn 20 CCooppyyrriigghhtt aanndd EEtthhiiccaall IIssssuueess y c n QQuueessttiioonn 55:: WWiillll yyoouu ddoowwnnllooaadd tthhee ffiilleess ffrroomm ue q tthhiiss wweebbssiittee ttoo ppoosstt tthheemm oonn yyoouurr wweebbssiittee?? e Fr 63.2 percent ooff ppaarrttiicciippaannttss aannsswweerreedd ““NNoo”” 10 meaning nnoott ttoo ddoowwnnllooaadd tthhee ffiilleess ffrroomm tthhiiss specific website ((sseeee FFiigguurree 5). CCooppyyrriigghhtt OOppttiioonnss 0 -5 0 5 10 15 Totals 36.8% Score 0 -Yes Figure 7. Total CCrriittiiccaall TThhiinnkkiinngg SSccoorree 63.2% DDiissttrriibbuuttiioonn Score 1 -No AA ppoossiittiivvee vvaalluuee ooff sskkeewwnneessss (.449) indicates a pile-uupp ooff ssccoorreess oonn tthhee lleefftt ooff tthhee ddiissttrriibbu- tion ((FFiieelldd,, 22000000)).. TThhee ddiissttrriibbuuttiioonn on the histogram sshhoowweedd tthhaatt 6611..11%% ooff tthhee ppaarrttiicci- Figure 5.. CCooppyyrriigghhtt OOppttiioonnss DDiissttrriibbuuttiioonn pants had their CT scores bbeellooww tthhee mmeeaann of QQuueessttiioonn 66:: PPlleeaassee pprroovviiddee tthhee rreeaassoonnss wwhhyy 5.01. you chose the specific aannsswweerr ffoorr tthhee qquuees- Correlations tion above. TThhee rreessuullttss sshhoowweedd tthhaatt 5588..99%% ooff partici- WWee tteesstteedd tthhee ccoorrrreellaattiioonn wwiitthh aallll ooff tthhee vva- pants discovered neither ccooppyyrriigghhtt nnoorr eetthhi- rriiaabblleess aanndd ffoouunndd tthhaatt tthhee ppaarrttiicciippaannttss’’ aabbiilli- cal issues for this case. HHoowweevveerr,, 3366..88%% ooff ty to define a ggrraapphh ppaatttteerrnn iiss ssiiggnniiffiiccaannttllyy ssttuuddeennttss ddiidd ssttaattee tthhaatt ccooppyyrriigghhtt iinnffrriinngge- ccoorrrreellaatteedd wwiitthh tthhee aabbiilliittyy ooff pprroobblleemm-solving mmeenntt wwaass tthhee rreeaassoonn tthheeyy ddiidd nnoott wwaanntt ttoo sskkiillllss ttoo rreeccoommmmeenndd tthhee pprrooppeerr pprroodduuccttss and ddoowwnnllooaadd aanndd uussee tthhee ffiilleess (see Figure 6). copyright options ((sseeee TTaabbllee 11)).. IInn ootthheerr words, the lower level tthhee ppaarrttiicciippaannttss ccaann (cid:13)c 2010 EDSIG http://isedj.org/8/1/ Jan 11, 2010 ISEDJ 8 (1) Shannon, Schneider, and Bennett 10 read and respond accordingly, the lower products, the participants who did well on their problem-solving skills are. this item also presented skills on interpreting the graph pattern, reasoning their decisions, and recognizing the copyright and internet Correlation Significant ethical issues. Graph Interpre- .030 .685 tation Correlation Significant Suggested Prod- ucts -.260 .001* Graph Pattern -.260 .001 Fwiinthal reasoPnrosd uct -.514 .001* Gtartaiopnh Interpre- -.020 .785 Copyright -.180 .013 Final Product -.206 .004 with reasons Ethical Reason- ing -.035 .636 Copyright -.308 .001* Note: df= 187, * P<.01 Ethical Reason- -.272 .001* ing Table 1. Correlation of Graph Pattern Note: df= 187, * P<.01 versus Other Items Table 3 Correlation of Suggest Product When testing the variable of “Graph Inter- versus Other Items pretation” with the rest of the items, we found that it is significantly correlated with The item of “Final Product with Reasons” the item of “Final product with reasons”. was to retest the participants’ reading and The score of “Graph Interpretation” was not responding skills from the previous item normally distributed due to the lack of know- “Suggested Products”. The findings con- ledge in computer security issues. 98.9% of firmed the same significant results as Table students received a zero score for this item; 3 (see Table 4). therefore, the correlation analysis for “Graph Interpretation” could not represent the find- ings properly (see Table 2). For the rest of Correlation Significant finding discussion, we would remove the Graph Pattern -.514 .001* item of “Graph Interpretation” to report valid findings. Graph Interpre- -.200 .006 tation Suggested Prod- Correlation Significant -.206 .004 ucts Graph Pattern .030 .685 Copyright -.287 .001* Suggested -.020 .785 Ethical Reason- Products -.462 .001* ing Final Product -.200 .006 Note: df= 187, * P<.01 with reasons Table 4. Correlation of Final Product Copyright .035 .629 with Reasons versus Other Items Ethical Reason- .025 .732 The copyright options showed that it is sig- ing nificantly correlated with the participants’ Note: df= 187, * P<.01 ability to interpret graph pattern, suggest proper products, provide reasons, and rec- Table 2. Correlation of Graph Interpre- ognize the internet ethical issues (see Table tation versus Other Items 5). In other words, the participants’ ability to We found that the item of “Suggest Prod- choose a proper copyright option is related ucts” is significantly correlated with the rest to the ability of their problem-solving and of variables (see Table 3). After reading the reasoning skills. provided information to suggest proper (cid:13)c 2010 EDSIG http://isedj.org/8/1/ Jan 11, 2010