W I NAEP? HAT S The National Assessment of Educational Progress (NAEP) is a continuing and nationally representative measure of trends in academic achievement of U.S. elementary and secondary students in various subjects. For nearly four dec- ades, NAEP assessments have been conducted periodically in reading, mathematics, science, writing, U.S. history, civics, geography, and other subjects. By collecting and reporting information on student performance at the nation- al, state, and local levels, NAEP is an integral part of our nation's evaluation of the condition and progress of educa- tion. THE 2016–2017 NATIONAL ASSESSMENT GOVERNING BOARD The National Assessment Governing Board was created by Congress to formulate policy for NAEP. Among the Governing Board’s responsibilities are developing objectives and test specifications and designing the assessment methodology for NAEP. Members Honorable James Geringer Honorable Jeanette Nuñez Director of Policy and Public Sector Legislator Terry Mazany, Chair Strategies Florida House of Representatives President and CEO Environmental Systems Research Miami, Florida The Chicago Community Trust Institute (ESRI) Chicago, Illinois Cheyenne, Wyoming Father Joseph O’Keefe, S.J. Professor Lucille E. Davy, Vice Chair Doris R. Hicks Boston College Lynch School of President and CEO Principal and CEO Education Transformative Education Solutions, Dr. Martin Luther King, Jr. Charter Chestnut Hill, Massachusetts LLC School for Science and Technology Pennington, New Jersey New Orleans, Louisiana W. James Popham Emeritus Professor Alberto Carvalho Andrew Dean Ho University of California, Los Angeles Superintendent Professor Los Angeles, California Miami-Dade County Public Schools Harvard Graduate School of Education Miami, Florida Cambridge, Massachusetts B. Fielding Rolston Chairman Honorable Mitchell D. Chester Carol Jago Tennessee State Board of Education Commissioner of Elementary and Associate Director Kingsport, Tennessee Secondary Education California Reading and Literature Massachusetts Department of Project at UCLA Linda Rosen Elementary and Secondary Education Oak Park, Illinois CEO Malden, Massachusetts Change the Equation Tonya Matthews Washington, DC Frank K. Fernandes President and CEO Principal Michigan Science Center Cary Sneider Kaimuki Middle School Detroit, Michigan Associate Research Professor Honolulu, Hawaii Portland State University Tonya Miles Portland, Oregon Honorable Anitere Flores General Public Representative Senator Mitchellville, Maryland Ken Wagner Florida Senate Commissioner for Elementary and Miami, Florida Honorable Ronnie Musgrove Secondary Education Founding Member Rhode Island Department of Education Rebecca Gagnon Musgrove/Smith Law Providence, Rhode Island School Board Member Jackson, Mississippi Minneapolis Public Schools Chasidy White Minneapolis, Minnesota Dale Nowlin Eighth-Grade Teacher Teacher and Mathematics Department Brookwood Middle School Shannon Garrison Chair Brookwood, Alabama Fourth-Grade Teacher Bartholomew Consolidated School Solano Avenue Elementary School Corporation Joe Willhoft Los Angeles, California Columbus, Indiana Consultant Tacoma, Washington National Assessment Governing Board Terry Mazany Chair Lucille E. Davy Vice Chair William Bushaw Executive Director Lisa Stooksberry Deputy Executive Director National Assessment of Educational Progress Mathematics Project Wayne Martin Project Director John Olson Deputy Project Director Linda Wilson Consensus Coordinator Framework Author Developed for the National Assessment Governing Board under contract number ED–00–CO–0115 by the Council of Chief State School Officers, with subcontracts to the Council of Basic Education and the Association of State Supervisors of Mathematics and Grade 12 preparedness objectives developed under contract with Achieve, Inc. For further information, contact: National Assessment Governing Board 800 N. Capitol St., NW, Suite 825 Washington, DC 20002–4233 www.nagb.gov January 2017 Publication Note The 2017 NAEP Mathematics Framework is the same framework developed for the 1992 NAEP Mathematics Assessment for 4th and 8th grades, with minor modifications to clar- ify assessment objectives. For 12th grade, this 2017 framework is the same framework developed for the 2005 assessment and includes 2009 modifications to support NAEP reporting on academic preparedness for postsecondary endeavors. Continuity in the NAEP Mathematics Framework enables reporting of student achieve- ment trends over time. To reflect this continuity, this edition reflects updated dates and references to legislation, National Assessment Governing Board actions, and NAEP ac- tivities, including the 2017 transition to digital-based assessment. For sale by the U.S. Government Printing Office Superintendent of Documents, Mail Stop: SSOP, Washington, DC 20402-0328 T C ABLE OF ONTENTS List of Exhibits ........................................................................................................................ ii Acknowledgments .................................................................................................................. iii NAEP Mathematics Project Staff and Committees .............................................................. v Chapter One: Overview .......................................................................................................... 1 What Is an Assessment Framework? ................................................................................. 2 Need for a New Framework at Grade 12 ........................................................................... 2 Chapter Two: Framework for the Assessment ..................................................................... 5 Content Areas .................................................................................................................... 5 Item Distribution ............................................................................................................... 6 NAEP Mathematics Objectives Organization ................................................................... 6 Mathematical Content Areas ............................................................................................. 7 Chapter Three: Mathematical Complexity of Items .......................................................... 37 Low Complexity .............................................................................................................. 38 Moderate Complexity ...................................................................................................... 42 High Complexity ............................................................................................................. 46 Chapter Four: Item Formats ................................................................................................ 51 Selected-Response Items ................................................................................................. 51 Short Constructed-Response Items .................................................................................. 53 Extended Constructed-Response Items ........................................................................... 55 Scoring Constructed-Response Items .............................................................................. 57 Chapter Five: Design of Test and Items .............................................................................. 59 Balance of Content .......................................................................................................... 59 Balance of Mathematical Complexity ............................................................................. 60 Balance of Item Formats ................................................................................................. 60 Balance of Item Contexts ................................................................................................ 61 Sampling .......................................................................................................................... 62 Calculators ....................................................................................................................... 63 Manipulatives and Tools ................................................................................................. 65 Accessibility .................................................................................................................... 66 Item Families ................................................................................................................... 67 Bibliography ........................................................................................................................... 69 Appendix A: NAEP Mathematics Achievement-Level Descriptions ................................ 71 Mathematics Achievement-Levels Descriptions for Grade 4 ......................................... 71 Mathematics Achievement-Levels Descriptions for Grade 8 ......................................... 72 Mathematics Achievement-Levels Descriptions for Grade 12 ....................................... 73 NAEP 2017 MATHEMATICS FRAMEWORK i L E IST OF XHIBITS Exhibit 1. Comparison of 2005 and 2009–2017 mathematics frameworks ........................ 3 Exhibit 2. Percentage distribution of items by grade and content area ............................... 6 Exhibit 3. Number properties and operations ..................................................................... 9 Exhibit 4. Measurement .................................................................................................... 14 Exhibit 5. Geometry .......................................................................................................... 19 Exhibit 6. Data analysis, statistics, and probability .......................................................... 25 Exhibit 7. Algebra ............................................................................................................. 31 Exhibit 8. Percent of testing time at each level of complexity ......................................... 38 Exhibit 9. Percent of testing time by item formats ........................................................... 51 Exhibit 10. Percent distribution of items by grade and content area ................................ 59 Exhibit 11. Percent of testing time at each level of complexity ....................................... 60 Exhibit 12. Percent of testing time by item formats ......................................................... 60 NAEP 2017 MATHEMATICS FRAMEWORK ii A CKNOWLEDGMENTS The following people were the primary authors of the introductions to the content areas: • Roger Howe, Yale University (Number Properties and Operations, Geometry, and Algebra) • Richard Scheaffer, University of Florida (Data Analysis and Probability) • Mary Lindquist, Columbus State University (Measurement) NAEP 2017 MATHEMATICS FRAMEWORK iii NAEP M P S ATHEMATICS ROJECT TAFF AND C OMMITTEES MEMBERS OF THE BOARD’S GRADE 12 Sharif Shakrani MATHEMATICS PANEL Director, Education Policy Center Michigan State University Herbert Clemens East Lansing, Michigan Professor, Department of Mathematics The Ohio State University Linda Dager Wilson, Chair Columbus, Ohio Mathematics Consultant Washington, D.C. Mary Ann Huntley Assistant Professor, Mathematics University of Delaware NAEP GRADE 12 MATHEMATICS Newark, Delaware PROJECT ACHIEVE NAEP GRADE 12 MATHEMATICS PANEL Jeremy Kilpatrick Regents Professor, Mathematics Education Sue Eddins (Retired) University of Georgia Mathematics Teacher Athens, Georgia Illinois Mathematics and Science Academy Aurora, Illinois Mary Lindquist Fuller E. Callaway Professor, Emeritus William McCallum Columbus State University University Distinguished Professor of Lewisburg, West Virginia Mathematics University of Arizona Mary Jo Messenger (Retired) Tucson, Arizona Chair, Department of Mathematics River Hill High School Fabio Milner Clarksville, Maryland Professor of Mathematics Purdue University William Schmidt West Lafayette, Indiana University Distinguished Professor Department of Counseling, Educational William Schmidt Psychology and Special Education University Distinguished Professor Michigan State University Department of Counseling, Educational East Lansing, Michigan Psychology and Special Education Michigan State University East Lansing, Michigan NAEP 2017 MATHEMATICS FRAMEWORK v Lynn Steen Phoebe Winter Professor of Mathematics Project Director St. Olaf College State Education Assessment Center Northfield, Minnesota Council of Chief State School Officers Richmond, Virginia Norman Webb Senior Research Scientist 2005 NAEP MATHEMATICS PROJECT Wisconsin Center for Education Research STEERING COMMITTEE University of Wisconsin Madison, Wisconsin Eileen Ahearn Project Director REVIEWS RECEIVED ON THE DRAFT OF National Association of State Directors NAEP 12TH GRADE MATHEMATICS of Special Education OBJECTIVES Alexandria, Virginia Achieve, Inc. Charles Allan American Mathematical Society Mathematics Education Consultant Association of State Supervisors of Michigan Department of Education Mathematics Lansing, Michigan Thomas B. Fordham Institute State Mathematics Supervisors from various B. Marie Byers states National School Boards Association National Council of Teachers of Mathematics Hagerstown, Maryland State Testing Directors from various states Randy DeHoff Colorado State Board of Education 2009 NAEP MATHEMATICS SPECIFICATIONS 6th Congressional District – Littleton WORK GROUP Denver, Colorado Mary Lindquist M.B. “Sonny” Donaldson Fuller E. Callaway Professor Emeritus Superintendent Columbus State University Aldine Independent School District Lewisburg, West Virginia Houston, Texas Mary Jo Messenger (Retired) Janice Earle Chair, Department of Mathematics Senior Program Director River Hill High School National Science Foundation Clarksville, Maryland Arlington, Virginia Linda Dager Wilson, Chair Louis M. Fabrizio Mathematics Consultant Data, Research and Federal Policy Director Washington, D.C. North Carolina Department of Public Instruction Raleigh, North Carolina NAEP 2017 MATHEMATICS FRAMEWORK vi Bettye Forte Lee McCaskill Mathematics Consultant Principal Arlington, Texas Brooklyn Technical High School Brooklyn, New York Matt Gandal Vice President Barbara Montalto Achieve, Inc. Assistant Director of Mathematics Washington, D.C. Texas Education Agency Austin, Texas Alice Gill Associate Director, Educational Issues Judy Rohde American Federation of Teachers Mathematics Instructor Washington, D.C. John Glenn Middle School Maplewood, Minnesota M. Kathleen Heid Distinguished Professor of Education Wilfried Schmid (Mathematics Education) Professor of Mathematics The Pennsylvania State University Harvard University University Park, Pennsylvania Cambridge, Massachusetts Audrey Jackson Mary Frances Taymans, SND Assistant Principal Executive Director Claymont Elementary School Secondary Schools Department Parkway City Schools National Catholic Education Association Fenton, Missouri Washington, D.C. James M. Landwehr Zalman Usiskin Director, Data Analysis Research Professor of Education Avaya Labs Director, University of Chicago School Basking Ridge, New Jersey Mathematics Project Chicago, Illinois Sharon Lewis Director of Research Judy Walter Council of the Great City Schools Director of Research and Development Washington, D.C. Association for Supervision and Curriculum Development Dane Linn Alexandria, Virginia Director, Education Division National Governors Association Diana Wearne Washington, D.C. Associate Professor School of Education Eddie Lucero University of Delaware Principal Newark, Delaware Griegos Elementary School Albuquerque, New Mexico NAEP 2017 MATHEMATICS FRAMEWORK vii