DOCUMENT RESUME CS 013 542 ED 430 209 Reading Framework for the National Assessment of Educational TITLE Progress: 1992-2000. NAEP Reading Consensus Project. INSTITUTION Council of Chief State School Officers, Washington, DC. National Assessment Governing Board, Washington, DC. SPONS AGENCY PUB DATE 1999-00-00 NOTE 75p. RS-89175001 CONTRACT ED Pubs, P.O. Box 1398, Jessup, MD 20794-1398; Tel: AVAILABLE FROM 877-433-7827 (Toll Free). Guides PUB TYPE Non-Classroom (055) EDRS PRICE MF01/PC03 Plus Postage. Elementary Secondary Education; *Literacy; *Reading DESCRIPTORS Achievement; *Reading Skills; *Reading Tests; Student Development; Test Construction; *Test Content; *Test Format *National Assessment of Educational Progress; Reading IDENTIFIERS Management ABSTRACT This booklet presents the Reading Framework for the 1992, 1994, 1998, and 2000 National Assessments of Educational Progress (NAEP), adopted by the National Assessment Governing Board (NAGB). The Framework contains the rationale for the aspects of reading assessed and criteria for development of the assessment. Developed through a national consensus process as part of an effort to move assessment forward, the framework presented in the booklet is more consistent with contemporary knowledge about reading and more relevant to the needs of education decisionmakers than earlier assessments have been. After an overview and introduction, chapters of the (1) development of the Reading Framework (which discusses booklet are: Steering Committee guidelines, the nature of assessment, reading literacy, assessment and instruction, new methodologies, and the basis of the (2) design of the Reading Framework (which discusses a goal for Framework); reading literacy education; constructing, extending, and examining meaning; and constructing the assessment); and (3) special Studies and Background Information (which discusses the oral reading and response study, the portfolio study, the metacognition study, and information for education Contains 10 references; appendixes present reading achievement policymakers) . level descriptions, lists of Steering and Planning Committee members, and sample items. (RS) ******************************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. ******************************************************************************** S. 1 ef* 10 Reading Fra ework for ...... .. ' the National Assess ent . ° of Educational Progress: 2000 199 U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) :. -.4'..- ': ' . Ki This document has been reproduced as 7.. :*!?:::::. . : received from the person or organization originating it II; ::.::: . ::.:Z: 0 Minor changes have been made to .... 1.7..::!:: 4 improve reproduction quality. 11111. EAk :: N . ..!,:;%%:; So Points of view or opinions stated in this MR *AMIN :1':.:::: :. 7.? ::: document do not necessarily represent II ...... ; official OERI position or policy. ai . . 1 :'.:Z:t:... .: _ Ae :: .. . :LI: . ..... I : 801 A4-- " . .. . ..... ..... :ii. . ..,... . .. ..... S 4 ;' IIN ....M: 'O''. , ,,, ::: ...la% :........ ". r .714111111- t P Reading Consensus Project N National Assessment Governing Board U.S. Department of Education (1912Wiillnal What Is NAEP? The National Assessment of Educational Progress (NAEP) is a congressionally mandated project of the U.S. Department of Education's National Center for Education Statistics. It assesses what U.S. students should know and be able to do in geography, reading, writing, mathematics, science, U.S. history, the arts, civics, and other academic subjects. Since 1969, NAEP has surveyed the achievement of students at ages 9, 13, and 17 and, since the 1980s, in grades 4, 8, and 12. Measuring educational achievement trends over time is critical to measuring progress toward the National Education Goals. The National Assessment Governing Board The National Assessment Governing Board (NAGB) was created by Congress to formulate policy for NAEP. Among the Board's responsi- bilities are developing objectives and test specifications and designing the assessment methodology for NAEP. Members Edward H. Haertel Diane Ravitch Professor Senior Research Scholar Mark D. Musick, Chair School of Education New York University President Stanford University New York, New York Southern Regional Education Board Stanford, California Atlanta, Georgia Honorable Roy Romer Juanita Haugen (Member Designate) Michael T. Nettles, Vice Chair Local School Board President Former Governor of Colorado Professor of Education and Public Policy Pleasanton, California Denver, Colorado University of Michigan Carole Kennedy Ann Arbor, Michigan John H. Stevens Middle School Principal Executive Director Moses Barnes Columbia, Missouri Texas Business and Education Coalition Secondary School Principal Austin, Texas Honorable Nancy Kopp Fort Lauderdale, Florida Democratic State Legislator Adam Urbanski Melanie A. Campbell Maryland House of Delegates President Fourth-Grade Teacher Bethesda, Maryland Rochester Teachers Association, AFT Topeka, Kansas Rochester, New York Honorable William J. Moloney Honorable Wilmer S. Cody Commissioner of Education Deborah Voltz Kentucky Commissioner of Education State of Colorado Assistant Professor Frankfort, Kentucky Denver, Colorado Department of Special Education Edward Donley University of Louisville Mitsugi Nakashima Former Chairman Louisville, Kentucky President Air Products & Chemicals, Inc. Hawaii State Board of Education Marilyn A. Whirry Allentown, Pennsylvania Honolulu, Hawaii Twelfth-Grade English Teacher Honorable John M. Engler Mira Costa High School Debra Paulson Governor of Michigan Manhattan Beach, California Eighth-Grade Teacher (Mathematics) Lansing, Michigan El Paso, Texas Dennie Palmer Wolf Thomas H. Fisher Senior Research Associate Honorable Norma Paulus Director Harvard Graduate School of Education Former Superintendent of Student Assessment Services Cambridge, Massachusetts Public Instruction Florida Department of Education Oregon State Department of C. Kent McGuire (Ex-Officio) Tallahassee, Florida Education Assistant Secretary of Education Michael J. Guerra Salem, Oregon Office of Educational Research and Executive Director Improvement Honorable Jo Ann Pottorff National Catholic Education Association U.S. Department of Education Kansas House of Representatives Secondary School Department Washington, DC Wichita, Kansas Washington, DC 3 AVPA2U 5E-ST COPY S I D 4 A 1 N 4 1 1 1 1 M 1 1 1 1 I 1 P 1 1 1 M , 1 1 1 1 1 1 1 I . . / ; ; ; ! : 4 4 1 1 1 1 4 1 t 4 4 . :" : A r I I I t ' : ' ' ; : 1 ' " : ' : ' ' ' ' . . . ' ' . ' . : . " . . . . . : 4 1 1 4 E D National Assessment Governing Board Mark D. Musick Chair Michael T. Nettles Vice Chair Roy truby Executive Director Mary Crovo Project Officer Reading Consensus Project Ramsay W. Selden Director Barbara Kapinus Consensus Coordinator Reading Framework for the National Assessment of Educational Progress: 1992-2000. Developed for the National Assessment Governing Board under contract number RS 89175001 by the Council of Chief State School Officers. For further information, contact: National Assessment Governing Board 800 North Capitol Street, NW. Suite 825 Washington, DC 20002 http://www.nagb.org For sale by the U.S. Government Printing Office Superintendent of Documents, Mail Stop: SSOP, Washington, DC 20402-9328 Table of Contents Overview 1 Introduction 3 Chapter One: Development of the Reading Framework 5 Steering Committee Guidelines 7 Considerations and Principles 7 The Nature of Assessment 7 Reading Literacy 8 Assessment and Instruction 8 New Methodologies 8 The Basis of the Framework 9 Chapter Two: Design of the Reading Framework 11 A Goal for Reading Literacy Education 11 Types of Reading Situations 12 Scales 14 Constructing, Extending, and Examining Meaning 15 Forming an Initial Understanding 15 Developing an Interpretation 16 Personal Reflection and Response 16 Demonstrating a Critical Stance 17 Constructing the Assessment 20 Designing Items 20 Selecting Passages 21 Achievement Levels 22 Chapter Three: Special Studies and Background Information 23 Special Studies 23 Oral Reading and Response Study 23 24 Portfolio Study iii 26 Metacognition Study 27 Information for Education Policymakers References 29 Appendix A: Reading Achievement Level Descriptions 31 Appendix B: Steering and Planning Committees 39 Appendix C: Sample Items 45 7 iv Overview This document presents the Reading Framework for the 1992, 1994, 1998, and 2000 National Assessments of Educational Progress (NAEP), adopted by the National Assessment Governing Board (NAGB). The Framework contains the rationale for the aspects of reading assessed and criteria for development of the assessment. The NAEP in Reading uses a variety of assessment approaches, both conventional and innovative. Developed through a national consensus process as part of an effort to move assessment forward, the new NAEP in Reading is more consistent with con- temporary knowledge about reading and more relevant to the needs of educational decisionmakers than earlier assessments were. The new reading assessment has the following characteristics: It examines students' abilities to construct, extend, and examine the meaning of what they read through the use of items that elicit a variety of responses to both multiple-choice and open-ended tasks. It assesses student performance in different reading situa- tionsreading for literary experience, reading to be informed, and reading to perform a taskby using authentic, "real-life" texts. Students in grades 8 and 12 are permitted to choose from among different short stories. It includes special studies to examine other aspects of reading, including the reading fluency of students and their reading habits and practices. 8 1 Introduction Reading is the most important, fundamental ability taught in the nation's schools. It is vital to society and to the people within it. It is the door to knowledge and a capability that can liberate people both intellectually and personally. For 30 years the National Assessment of Educational Progress (NAEP) has been reporting the reading achievement of students in the United States. Known in recent years as The Nation's Report Card, NAEP reports provide descriptive information about student strengths and weaknesses in reading and a number of other sub- jects. The reports provide data that compare groups of students by race and ethnicity, gender, type of community, and region, as well as data that chart trends in achievement over time. Relationships between student achievement and school-related experiences such as homework and instruction are also reported. A significant change occurred in 1990 in how and for whom NAEP results are reported. The 1990 NAEP mathematics assess- ment collected information on a trial basis to provide "state report cards" that allow state-to-nation and state-to-state comparisons. In 1992 the NAEP reading assessment included state-level reporting of fourth-grade results on a trial basis. Reading was assessed again in 1994 at the national level, and at the state level in grade four. In 1998 NAEP assessed reading in grades four and eight at the state level. The decision to undertake state-by-state reporting of assessment data had its beginnings in 1984 when a majority of chief state school officers supported the development of an assessment system that would provide state-level information. In the following year, that group supported the expansion of NAEP as the most feasible vehicle for such an assessment system. In 1986 Georgia and Wyoming contracted with NAEP to conduct in-state assessments concurrently with the national assessment and to provide them with state-to-nation comparison data. In 1986 and 1987, several groups of southern states contracted with NAEP to conduct state-level assessments in mathematics. This effort was coordinated by the Southern Regional Education Board. 3 During this period, the governors of various other states ex- pressed a desire to obtain information from NAEP on the educa- tional achievement of students in their states. In 1987 a national study group chaired by Lamar Alexander, then governor of Tennes- see, made a series of recommendations about the future of NAEP. One of the most significant was that the assessment be expanded to provide state-by-state reporting. In 1988 Congress passed the Hawkins-Stafford Elementary and Secondary School Improvement Amendments (P.L. 100-297), which added a new dimension to NAEP: a voluntary trial state assessment in 1990 and 1992. The first trial was conducted in spring 1990, with 40 states and territories participating in a math- ematics assessment at grade eight. The trial continued in 1992 with mathematics assessments at grades four and eight and a reading assessment at grade four. In 1994 NAEP state-level assessments were given in reading at grade four. NAEP conducted a state-level reading assessment in grades four and eight in 1998. The decision to undertake state-by-state reporting is not without its critics, especially among reading educators. Some fear that state- by-state comparisons will be used to draw inappropriate inferences or to make unsupported cause-and-effect relationships. Others are concerned that the assessment design will not reflect appropriate educational goals. Still others fear that the various NAEP assess- ments will foster, and ultimately outline, a national curriculum. Reading educators in particular were apprehensive that the reading assessment would be insensitive to many theoretical and instruc- tional developments in the field and that it might oversimplify the complex set of behaviors that are integral to reading. The first report card related to America's national education goals was released in fall 1991. It contained many gaps, especially at the state level. Reading information was limited to what could be inferred from Scholastic Aptitude Test scores. There is clearly a need for an assessment that will allow states to determine the success with which they have met the goals established by the President and the nation's governors. The NAEP in Reading fills a critical gap in this area. 1 0 4