Innovative Assessment for the 21st Century · Valerie J. Shute Betsy Jane Becker Editors Innovative Assessment for the 21st Century Supporting Educational Needs 123 Editors ValerieJ.Shute BetsyJaneBecker CollegeofEducation CollegeofEducation FloridaStateUniversity FloridaStateUniversity 1114W.CallStreet 1114W.CallStreet 32306Tallahassee,Florida 32306Tallahassee,Florida USA USA [email protected] [email protected] ISBN978-1-4419-6529-5 e-ISBN978-1-4419-6530-1 DOI10.1007/978-1-4419-6530-1 SpringerNewYorkDordrechtHeidelbergLondon LibraryofCongressControlNumber:2010931400 ©SpringerScience+BusinessMedia,LLC2010 Allrightsreserved.Thisworkmaynotbetranslatedorcopiedinwholeorinpartwithoutthewritten permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY10013,USA),exceptforbriefexcerptsinconnectionwithreviewsorscholarlyanalysis.Usein connectionwithanyformofinformationstorageandretrieval,electronicadaptation,computersoftware, orbysimilarordissimilarmethodologynowknownorhereafterdevelopedisforbidden. Theuseinthispublicationoftradenames,trademarks,servicemarks,andsimilarterms,eveniftheyare notidentifiedassuch,isnottobetakenasanexpressionofopinionastowhetherornottheyaresubject toproprietaryrights. Printedonacid-freepaper SpringerispartofSpringerScience+BusinessMedia(www.springer.com) Acknowledgements This book was made possible through the generous support of the College of Education at Florida State University. As she describes in the Foreword to this book, Dean Marcy Driscoll initiated a series of annual symposia that bring lead- ing scholars to Florida State. Our symposium focused on new ways of thinking aboutassessmentandservedasthespringboardforthisbook.We’dliketoacknowl- edgetheoutstandingassistanceofSoojeongIngrisonewhohelpedwithallaspects of our symposium planning. In addition, we want to thank Kuzey Bilir, Oktay Donmez, and Yanyun Yang for helping us with many of the symposium logistics, andNashMcCutcheonandEmilyKeelerfortheircreativeeffortsinpromotingthe symposium. The quality of the book was enhanced due to the work of our cadre of dili- gentreviewers:BarbaraDavis,MalcolmBauer,JillBurstein,GinetteDelandshere, Vanessa Dennen, Oktay Donmez, Rebekah Dorn, Eric Hansen, Dirk Ifenthaler, Tristan Johnson, Yoon-Jeon Kim, Qian Liu, Iskandaria Masduki, Christie Suggs, and Diego Zapata-Rivera. In addition, we are grateful for the exceptional efforts of Yoon-Jeon Kim in helping us format the book chapters. Finally, we want to acknowledgeSpringerPublishingfortheirinterestinthisbook. v Foreword Thepaperscomprisingthisvolumewerepresentedatthe2008Dean’sSymposium sponsoredbytheCollegeofEducationatFloridaStateUniversity. The impetus for a symposium series came from a blue ribbon committee that I chargedwithfindingwaystofacilitateacultureofinquiryinthecollege,topromote the exchange of ideas among faculty but also with the constituents of our work. Theuniversity’slocationinthestatecapitalanditsproximitytothelegislatureand stateagenciesofferusauniqueopportunitytopartnerwithconstituentgroupsand influence education policy in the state. To take advantage of this opportunity, the blueribboncommitteesuggestedanannualsymposiumthatwouldprovideaforum fordiscussionabouttopicsofkeeninteresttopolicymakersandresearchersalike, bothstatewideandnationally. Our first effort was to sponsor an invited lecture in Spring 2006 that brought teachereducatorMarilynCochran-Smithtocampus.Thatwasawonderfuloppor- tunitynotonlytohearabouttheworkshe’sbeendoingwiththeTeachersforaNew Era project, but also to engage her in conversation with our faculty and doctoral studentsaboutteachereducationresearch,practice,andpolicy. In Fall 2007, we held our first true symposium, interspersing panel and roundtable discussionswithpresentations frominvitedspeakers.Itwasonteacher quality, a topic that is central to education reform efforts in the state of Florida. Wewereexcitedtodraweducatorsfromaroundthestateandfrommanysectors— legislativepolicymakers,superintendentsandschoolleaders,educationresearchers and administrators. To our great delight, three state legislators also attended and participatedinlivelydiscussion. Our second symposium, held in Fall 2008, resulted in this book. Co-Editors ValerieShuteandBetsyBeckertooktheleadinorganizingasetofpresentationsand opportunitiesfordiscussionaroundthetopicofAssessment.InstateslikeFlorida, theconceptofassessmenttendstoevokestrongandoftennegativefeelingsbecause it is most commonly associated with high stakes testing and accountability. Who, afterall,doesn’thaveanopinionabouttheFCAT?YetFloridaisalsoaleaderamong statesinlinkingassessmentdatatoinstructionalintervention.Teachersandprinci- palsareexpectedtomonitorindividualstudentprogressandensurethatappropriate instructional strategies are employed so that each student can learn. How assess- mentcangobeyondprovidingnumberstoprovidinginsightwasthethemeofthis vii viii Foreword symposium.Itisacriticalissuethatbringstogetherpolicymakers,agencyofficials, schoolleaders,andeducationresearchers. The presentations did not disappoint, and I believe the readers of this book will find much to consider. The authors challenge us to think differently, more expansively about assessment and its role in teaching and learning, as well as accountability. They help us to envision what a culture of assessment might look like, incorporating perspectives from teachers and students to state testing person- nel. It is our sincere hope that these papers will change the conversations around assessmentandleadtoadvancesthatwillfacilitatelearningandteachinginthe21st century. Tallahassee,Florida MarcyP.Driscoll June,2009 Contents 1 Prelude:Assessmentforthe21stCentury . . . . . . . . . . . . . . 1 ValerieJ.ShuteandBetsyJaneBecker 2 HumanActionandSocialGroupsastheNaturalHomeof Assessment:Thoughtson21stCenturyLearningandAssessment 13 JamesPaulGee 3 GrowingLearningandAssessmentinthe21stCentury . . . . . . 41 KingD.BeachIII 4 AimingatLearning:AssessmentastheCriticalLink . . . . . . . 49 MariPearlman 5 Sharpening the Aim: Making Strides to Create an AssessmentCultureinSchools . . . . . . . . . . . . . . . . . . . . 69 LynnWicker 6 UsingEvidenceCenteredDesigntoThinkAboutAssessments . . 75 RussellG.Almond 7 ThinkingAboutAssessmentsinaTransitionalTime . . . . . . . . 101 KrisEllingtonandVincentVerges 8 ParticipatoryAssessmentof21stCenturyProficiencies . . . . . . 107 DanielT.Hickey,MichelleA.Honeyford,KatieA.Clinton, andJennaMcWilliams 9 MakingAssessmentRelevanttoStudents,Teachers,andSchools . 139 JosephA.MartineauandVincentJ.Dean 10 AutomatedEssayScoringinaHighStakesTestingEnvironment . 167 MarkD.Shermis 11 AssessingChangeinLearners’CausalUnderstanding UsingSequentialAnalysisandCausalMaps . . . . . . . . . . . . 187 AllanC.Jeong ix x Contents 12 DevelopmentandUseofaToolforEvaluatingTeacher EffectivenessinGradesK-12 . . . . . . . . . . . . . . . . . . . . . 207 AlysiaD.RoehrigandEricChristesen 13 Epilogue:AchievingQuality21stCenturyAssessment . . . . . . 229 BetsyJaneBeckerandValerieJ.Shute SubjectIndex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 Contributors RussellG.Almond ResearchandDevelopment,EducationalTestingService, Princeton,NJ,USA,[email protected];[email protected] KingD.BeachIII InternationalEducationalResearchandDevelopment Consulting(IERDConsult),Tallahassee,FL,USA,[email protected] BetsyJaneBecker DepartmentofEducationalPsychologyandLearning Systems,FloridaStateUniversity,Tallahassee,FL,USA,[email protected] EricChristesen DepartmentofEducationalPsychologyandLearningSystems, FloridaStateUniversity,Tallahassee,FL,USA,[email protected] KatieA.Clinton NewMediaLiteracies,AnnenbergSchoolfor Communication&Journalism,UniversityofSouthernCalifornia,LosAngeles, CA,USA,[email protected] VincentJ.Dean OfficeofEducationalAssessment&Accountability,Michigan DepartmentofEducation,Lansing,MI,USA,[email protected] KrisEllington Accountability,Research,andMeasurement,FloridaDepartment ofEducation,Tallahassee,FL,USA,kris.ellington@fldoe.org JamesP.Gee DivisionofCurriculumandInstruction,MaryLouFultonCollege ofEducation,ArizonaStateUniversity,Tempe,AZ,USA,[email protected] DanielT.Hickey LearningSciencesProgram,IndianaUniversity,Bloomington, IN,USA,[email protected] MichelleA.Honeyford DepartmentofLiteracy,Culture,&LanguageEducation, IndianaUniversity,Bloomington,IN,USA,[email protected] AllanC.Jeong DepartmentofEducationalPsychologyandLearningSystems, FloridaStateUniversity,Tallahassee,FL,USA,[email protected] JosephA.Martineau OfficeofEducationalAssessment&Accountability, MichiganDepartmentofEducation,Lansing,MI,USA,[email protected] JennaM.McWilliams LearningSciencesProgram,IndianaUniversity, Bloomington,IN,USA,[email protected] xi