DOCUMENT RESUME ED 378 550 CS 011 970 AUTHOR Pinnell, Gay S.; And Others Listening to Children Read Aloud: Data from NAEP's TITLE Integrated Reading PerforMance Record (IRPR) at Grade 4 Educational Testing Service, Princeton, NJ. Center INSTITUTION for the Assessment of Educational Progress.; National Assessment of Educational Progress, Princeton, NJ. National Center for Education Statistics (ED), SPONS AGENCY Washington, DC. REPORT NO ISBN-0-88685-167-X; NAEP-23-FR-04; NCES-95-726 Jan 95 PUB DATE 98p.; For related documents, see CS 011 969-971. NOTE PUB TYPE Research!Technical (143) Reports EDRS PRICE MF01/PC04 Plus Postage. DESCRIPTORS *Grade 4; Intermediate Grades; Interviews; *Oral Reading; *Reading Achievement; *Reading Aloud to Others; *Reading Attitudes; Reading Instruction; Reading Research National Assessment of Educational Progress; *Reading IDENTIFIERS Fluency ABSTRACT Conducted as part of the 1992 Integrated Reading Performance Record (IRPR), a study investigated the oral reading proficiency of a subgroup of students participating in the 1992 reading assessment conducted by the National Assessment of Educational Progress (NAEP). Subjects, 1,136 fourth graders, read aloud one passage and were audiotaped as they responded to a series of questions about habits and attitudes related to both instructional and recreational reading. Subjects also completed measures of fluency and comprehension. Major findings were that (1) much can be learned and documented about children's abilities by listening to them read (2) 55% of the subjects were considered to be fluent, but only aloud; 13% could be described as consistently reading with appropriate phrasing and with at least minimal expressiveness; (3) oral reading fluenc,' demonstrated a significant relationship with reading comprehension; (4) fluent reading appeared to be related to certain literacy activities; (5) 57% of the students were at least 96% accurate in their oral reading of the passage; (6) 61% of the students read the passage at a rate of at least 100 words per minute; and (7) accuracy and rate displayed some relationship to reading fluency. (Contains 11 tables and two figures of data. Appendixes present the interview guide, and a description of the procedures and methods of the IRPR.) (RS) *********************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. *********************************************************************** NATIONAL CENTER FOR EDUCATION STATISTICS Listening to Children Read Aloud Data from NAEP's Integrated Reading Performance Record (IRPR) at Grade 4 U S DEPARTMENT OF EDUCATION Office of ecluCalonal Resea,-h and Innaovemr, EDUCATIONAL RESOURCES INFORMATION KCENTER (ERfCi This document has been reproduced as received from the person or organi7atinn originating it r Minor changes have been made to .mprovo ieproduction quality r'oints of view or opinions slated in this document do not necessarily represent official OERI position or policy. 111111)1 (mits) 1'1%1).11)d 1)% It...111144 1/4.)(.1 (If t I )111%.111))11.11 for I (11R,Ition (till) 11i). mid I wilt ti I (111t.iiitit1,11 kt.m..11( )1111 U Ill ITli .S. I)cl).11lnu II of f (111t..111))11 BEST COPY AVAILABLE What is The Nation's Report Card? THE NATION'S REPORT CARD, the National Assessment of qualified organizations. NAEP reports directly to the Commissioner, Educational Progress (NAEP), is the only nationally representative who is also responsible for providing continuing reviews, including and continuing assessment of what America's students know and can validation studies and solicitation of public comment, on NAEP's do in various subject areas. Since 1969, assessments have been conduct and usefulness. conducted periodically in reading, mathematics, science, writing, history/geography, and other fields. By making objective information In 1988, Congress created the National Assessment Governing Board on student performance available to policymakers at the national, (NAGB) to formulate policy guidelines for NAEP. The board is state, and local levels, NAEP is an integral part of our nation's responsible for selecting the subject areas to be assessed, which may evaluation of the condition and progress of education. Only include adding to those specified by Congress; identifying appropriate information related to academic achievement is collected under this achievement goals for each age and grade; developing assessment program. NAEP guarantees the privacy of individual students and objectives; developing test specifications; designing the assessment their families. methodology; developing guidelines and standards for data analysis and for reporting and disseminating results; developing standards NAEP is a congressionally mandated project of the National Center and procedures for interstate, regional, and national comparisons; for Education Statistics, the U.S. Department of Education. The improving the form and use of the National Assessment; and ensuring Commissioner of Education Statistics is responsible, by law, for that all items selected for use in the National Assessment are free from arrying out the NAEP project through competitive awards to racial, cultural, gender, or regional bias. The National Assessment Governing Board Mark D. Musick, Chair Michael J. Guerra President Executive Director William J. Moloney Southern Regional Education Board National Catholic Education Association Superintendent of Schools Atlanta, Georgia Secondary School Department Calvert County Public Schools Washington, DC Prince Frederick, Maryland Honorable William T. Randall, Vice Chair Commissioner of Education William (Jerry) Hume Mitsugi Nakashima State Department of Education Chairman Hawaii State Board of Education Denver, Colorado Basic American, Inc. Honolulu, Hawaii San Francisco, California Parris C. Battle Michael T. Nettles Education Specialist Christine Johnson Professor of Education & Public Policy Office of Grants Administration Director of Urban Initiatives University of Michigan Miami Springs, Florida Education Commission of the States Ann Arbor, Michigan Denver, Colorado Honorable Evan Bayh Honorable Edgar D. Ross Governor of Indiana John S. Lindley Senator Indianapolis, Indiana Director, Admin. Training & Development Christiansted, St. Croix Clark County School District U.S. Virgin Islands Mary R. Blanton Las Vegas, Nevada Attorney Marilyn A. Whirry Blanton Sc Blanton Jan B. Loveless 12th Grade English Teacher Salisbury, North Carolina Educational Consultant Mira Costa High School Jan B. Loveless & kssociates Manhattan Beach, California Linda R. Bryant Midland, Michigan, Dean of Students Sharon P. Robinson (ex-officio) Florence Reizenstein Middle School Marilyn McConachie Assistant Secretary Pittsburgh, Pennsylvania Local School Board Member Office of Educational Research Glenview High Schools and Improvement Honor,: le Naomi K. Cohen Glenview, Illinois U.S. Department of Education Former hepresentative Washington, DC State of Connecticut Honorable Stephen E. Merrill I larttord, Connecticut Governor of New Hampshire Concord, New Hampshire Charlotte A. Crabtree Roy Truby Professor ot Education Jason Millman Executive Director, NAGB University of California Prof. of Educational Research Methodology Washington, DC Los Angeles, California Cornell University Ithaca, New York Chester E. Finn, Jr. Founding Partner & Sr. Scholar Honorable Richard P. Mills The Edison Project Commissioner of Education Washington, DC State Department of Education Montpelier, Vermont 3 1 EDUCATION STATISTICS NATIONAL CENTER FOR Listening to Children Read Aloud Reading Performance Data from NAEP's Integrated Record (IRPR) at Grade 4 Gay S. Pinnell John J. Pikuiski Karen K. Wixson Jay R. Campbell Phillip B. Gough Alexandra S. Beatty January 1995 Report No. 23-FR-04 THE NATION'S Educational Testing Seri ice under contract Prepared REPORT Education Statistks %% it h the National ('enter for CARD Imprmernent Office of Educational Research and ..5. Department of Education U.S. Department of Education Richard W. Riley Secretary Office of Educational Research and Improvement Sharon P. Robinson Assistant Secretary National Center for Education Statistics Emerson J. Elliott Commissioner Education Assessment Division Cary W. Phillips .kssociate Commissioner FOR MORE INFORMATION: For ordering information on this report, write: Education Information Branch Office of Educational Research and Improvement U.S. Department of Education 555 New jersey Avenue, NW Washington, D.C. 20208-5641 or call 1-800-424-1616 (in the Washington, D.C. metropolitan area call 202-219-1651). Library of Congress, Catalog Card Number: 94-68492 ISBN: 0-88685- 167-X The work upon which this publication is baseJ was performed for the National Center for Education Statistics, Office of Educational Research and Improvement, by Educational Testing Service. Educational Testing Service is an equal opportunity, affirmative action employer. Educational Testing Service, FTS, and the EFS logo are registered trademarks of Educational Testing Service. ii a C 0 ; S 1 Introduction 1 Major Findings of the IRPR Oral Reading Study 2 Background on the Integrated Reading Performance Record 4 Developing the Integrated Reading Performance Record 6 Administration of the Integrated Reading Performance Record 7 The IRPR Reports 9 Chapter 1 Reading Fluently 11 Describing Oral Reading Fluency 13 Figure 1.1 Oral Reading Fluency Scale 15 Summary 16 The Oral Reading Fluency of Fourth-Grade Students Chapter 2 19 The Oral Reading Task 20 Results of the Fluency Analysis 21 Table 2.1 Fluency Levels and Average Proficiencies 22 Exploring the Relationship Between Fluency and Reading Experiences 24 Table 2.2 Fluency and Reading Outside of School 25 Table 2.3 Fluency and Using the Library for Enjoyment 27 Table 2.4 Fluency and Reading Self-Selected Books 28 Table 2.5 Fluency and Reading Aloud in Class 29 Summary 30 Ora! Reading Accuracy and Rate Chapter 3 33 Fourth-Graders' Accuracy with the IRPR Passage 34 Table 3.1 Fourth-Graders' Oral Reading Accuracy 36 Table 3.2 Fourth-Graders' Self-Corrections 38 Fourth-Graders' Reading Rate with the IRPR Passage 40 Figure 3.1 Reading Rate Distribution 41 Table 3.3 Oral Reading Rates and Average Proficiencies 43 Fluency's Association with Accuracy and Rate 43 Table 3.4 Fluency's Relationship to Accuracy and Rate 44 Table 3.5 Fluency and Accuracy 45 Table 3.6 Fluency and Rate 46 Summary 47 iii 6 Implications for Reading Instruction and Assessment Chapter 4 51 The Role of Fluency in Reading Instruction 53 Oral Reading Assessment in the Classroom 57 Implications for Large-Scale Assessments 58 Summary of Implications 59 Integrated Reading Performance Record Interview Guide Appendix A 61 Procedures and Methods Appendix B 77 Developing the IRPR 78 Figure B.1 IRPR Development Committee Members 78 The IRPR Design and Administration 79 Reporting Groups 79 Sample 81 Table B.1 Number of Students Eligible and Sampled 81 Table B.2 Comparison of IRPR and Main Assessment Students by Demographic Characteristics 83 Table B.3 Average Proficiency of IRPR and Main Assessment Students 84 Drawing Inferences from the Results 84 Appendix C Oral Reading Scoring Procedures 87 Rating Students' Oral Reading Fluency 87 Table C.1 Fluency Scoring Reliability Data 88 Coding Students' Deviations from Text 88 Table C.2 Deviation Coding Reliability Data 90 Acknowledgments 91 I.or sale h (het S ernmeni Printing ( e Super intenden( of I )ot unlent. '0.u! Stop SSOP. Washingion. DC 2(402.,:12s ISBN 0-16 045459-X iv In this, its 25th year of serving as the nation's only ongoing monitor of American students' academic achievement, the National Assessment of Educational Progress (NAEP) focuses this report on a relevant, but often oral reading proficiency. In ignored, aspect of reading development describing what students know and can do in reading, NAEP assessments regularly collect data on reading proficiency as well as contextual information regarding students instructional and background experiences. The investigation of fourth graders' oral reading abilities was conducted as a part of the 1992 Integrated Reading Performance Record (IRPR), a special study involving individual interviews with a subgroup of fourth graders in the 1992 NAEP reading assessment. As a result, NAEP for the first time can describe students' oral reading fluency in light of the accuracy and rate with which they read, and their overall literacy development. In responding to the longstanding connection between students' reading habits and proficiency, the IRPR coupled its oral fluency assessment with student interviews about the content, context, and frequency of their reading practices at home and at school. In addition, the IRPR also involved the collection of sample assignments and representative works from 1 8 students' reading classes. These results are presented in a companion report, Interviewing Children About their Literacy Experiences.' Major Findings of the IR The oral reading study conducted as a part of the IRPR represents NAEP's first, and one of the first ever, attempts to measure aspects of oral reading on a large-scale basis. For many years, oral reading assessments have been conducted informally in classrooms, where teachers depend on the information they gain from these observations to determine the status of students' reading development and individual needs. The IRPR study of oral reading provides a national data base that can be used to inform educators, parents, and researchers about how fourth graders are developing and how their oral reading abilities relate to their overall reading achievement. Describing Oral Reading Fluency. Perhaps the most significant finding from this study is that much can be learned about children's abilities by listening to them read aloud. The fluency scale developed for the IRPR to describe those aspects of oral reading that go beyond accuracy and rate may have wide applicability for reading educators. Fourth Graders' Oral Reading Fluency. In reading a portion of one narrative text, 55 percent of fourth graders were considered to be fluent. However, only 13 percent could be described as consistently reading with appropriate phrasing and with at least minimal expressiveness the highest degree of fluency rated. This was a passage they had read silently twice before. Those students who were rated as fluent in their oral reading demonstrated appropriate phrasing and adherence to the author's sentence structure. Students who were not rated as fluent read primarily in two- or one-word phrases with little or no recognition of the text's sentence structure. Oral Reading Fluency and Reading Proficiency. Another major finding from this study was that oral reading fluency demonstrated a significant relationship with reading comprehension. Increasingly higher levels of 'Campbell, J. R., Kapinus, B. A., Beatty, A. S. (1994). Interviewing Children About Their Literacy Experiences. National Center for Educational Statistics. Washington, DC: Government Printing Office. 2 9 fluency were associated with increasingly higher overall reading proficiency, as measured on the main NAEP reading assessment. Fluent reading also Oral Reading Fluency and Literacy Experiences. appeared to be related to certain literacy activities. For example, having read at least one book outside of school in the previous month was associated with higher oral reading fluency. Making use of the library to find recreational reading materials was also related to reading fluently. In addition, the fluent readers were more likely to say they had daily opportunities in class to read books they had chosen. Interestingly, reading aloud in class as a part of instruction demonstrated little connection to oral reading fluency. This may be due to the wide variety of oral reading activities that teachers may use and the likelihood that some are more effective than others with individual students. Oral Reading Accuracy. Two other aspects of oral reading accuracy and rate were measured in the IRPR oral reading study. The majority of students (57 percent) were at least 96 percent accurate in their oral reading of the passage presented to them. The relationship between reading accuracy and reading comprehension appeared to be dependent on the nature of students' deviations from the text. That is, the number of deviations students made in their oral reading that resulted in a meaning change was more directly related to their overall proficiency than simply the total number of deviations they made. There was also some indication that students made fewer self-corrections of their deviations from text when no meaning change occurred. Oral Reading Rate. Sixty-one percent of fourth graders read the IRPR passage at a rate of at least 100 words per minute. A consistent pattern was apparent in the relationship between proficiency and rate on average slower readers demonstrated lower reading proficiency. Both accuracy and rate displayed some Accuracy, Rate, and Fluency. relationship to reading fluency. While not all fluent readers were among the most accurate or the fastest of their peers, those readers who read fluently were, on average, at least 96 percent accurate and read the passage at an average rate of at least 126 words per minute. 3 4