The Impact of Reading Aloud to Children An Evaluation of the Words Alive Read Aloud Program October 2017 Presented by: Caster Family Center for Nonprofit and Philanthropic Research About Words Alive The mission of Words Alive is to open opportunities for life success by inspiring a commitment to reading. About the Caster Family Center for Nonprofit and Philanthropic Research The mission of the Caster Center is to provide research, evaluation and consulting services that build the leadership and strategic and evaluative-thinking capacity of local nonprofits, as well as to be the leading source of information, data and research on the local nonprofit sector. Project Team: Mary Jo Schumann, PhD, Director of Research Tessa Tinkler, PhD, Senior Research Associate Kim Hunt, PhD, Research Associate Azadeh Davari, MSc, Doctoral Research Assistant Meghan Kelly, MPA, Research Assistant Aya Tore, MA, Research Assistant Connelly Meschen, MArch, Administrative Coordinator Copyright © 2017 All Rights Reserved Caster Family Center for Nonprofit and Philanthropic Research The Nonprofit Institute School of Leadership and Education Sciences University of San Diego www.sandiego.edu/npresearch Acknowledgements: The Caster Center would like to acknowledge the Words Alive staff and volunteers, as well as the teachers and students (especially from Golden Hill Elementary) for their time and participation in the data collection process. TABLE OF CONTENTS Executive Summary ....................................................................................................................... i Overview and Background ............................................................................................................ 1 Description of Read Aloud ....................................................................................................... 2 Evaluation Objectives ............................................................................................................... 2 Methodology ................................................................................................................................. 3 Summary of Data Sources ....................................................................................................... 3 Methods of Analysis ................................................................................................................. 4 Read Aloud Program Impact On Children .................................................................................... 4 Presentation of Findings .......................................................................................................... 4 Building Foundational Literacy Skills ........................................................................................ 5 Improving Reading Comprehension ......................................................................................... 8 Increasing Reading Motivation and Engagement .................................................................. 12 Comparison of the Small Group and Large Group Model ...................................................... 16 Teacher and Volunteer Satisfaction ............................................................................................ 18 Implementation Lessons ............................................................................................................. 23 Teacher Feedback ................................................................................................................. 23 Volunteer Feedback ............................................................................................................... 23 Limitations ................................................................................................................................... 24 Conclusions and Recommendations .......................................................................................... 25 Appendix A: Logic Model for Words Alive Read Aloud Program ................................................ 26 EXECUTIVE SUMMARY Words Alive is a nonprofit organization focused on “The program has been amazing. creating opportunities for life success by inspiring a My students are so engaged commitment to reading. With numerous programs serving youth, teens, and families, Words Alive teaches during the read aloud and eager the value of reading and the importance of strong to ask questions that I often literacy skills in opening doors of opportunity for both wonder if it is the same class. children and their parents. They love to hear our volunteer One of the Words Alive literacy programs is called the read to them.” Read Aloud Program (Read Aloud). Through Read – Teacher Aloud, trained volunteer readers (hereafter referred to as volunteers) engage with students weekly in preschool through third grade classrooms to share the joy of reading. Read Aloud offers two different models to schools. In the large group model, volunteers visit classrooms for 30 minutes each week to conduct a large group read aloud session with 20-25 students. In the small group model, volunteers visit classrooms for 90 minutes each week to read to the large group, then facilitate discussions and activities related to the story with small rotating groups of 3-5 students. In both models, volunteers are provided with standards-based resources to guide their sessions. In January 2016, Words Alive commissioned the Caster Family Center for Nonprofit and Philanthropic Research (Caster Center) at the University of San Diego to evaluate the impact of Read Aloud on students from participating schools and the implementation of Read Aloud’s small group model at Golden Hill Elementary School. Between October 2016 and July 2017 the Caster Center research team designed and administered teacher surveys, analyzed a student reading motivation assessment, conducted a focus group with volunteers, and collected and analyzed volunteers’ weekly field notes. Overall, the findings indicate that Read Aloud had a positive impact on students. Key findings are highlighted below. Teachers felt Read Aloud… Impacted the development of students’ foundational literacy skills. of teachers reported Read Aloud impacted students’ 96% vocabulary development and knowledge of literary terms Impacted students’ reading comprehension skills. of teachers reported Read Aloud impacted students’ 88% ability to infer the central message of a story Increased students’ motivation and engagement in reading. of teachers reported Read Aloud 95% increased students’ interest in reading of teachers reported Read Aloud 89% increased students’ confidence in reading aloud Words Alive Read Aloud Program Evaluation | Caster Center, University of San Diego | i The small group model added significant value to Read Aloud. Teachers and “The connections [children] made from book volunteers from the small group model to book was so incredible to me… especially reported numerous benefits such as with the nonfiction. They started really allowing less vocal students to participate, believing some of the messages because reinforcing vocabulary, discussing the they saw them over and over. Being able to central message of the books, and making have that small group instruction time to talk curricular connections between texts, about [the message of the book] was students’ lives, and the larger world. invaluable. I saw a real huge growth in that Additionally, teachers who had the small small group… you wouldn’t have that group model rated the impact of the opportunity in the large group.” program slightly higher than teachers who – Volunteer had the large group model. The quality of the volunteer was integral to the success of Read Aloud. The majority of volunteers were highly engaged and many of them spent time outside of class planning activities, researching authors, and sharing additional props, photos, 95% and videos to help bring the books to life. However, teachers who of were paired with volunteers who had less teaching experience teachers rated the expressed a desire for their assigned volunteer to more easily engage with students. Similarly, volunteers from the small group model volunteers very indicated they would have benefited from more support in managing favorably and planning the small groups. Reading aloud to young children has been recognized as an essential practice to developing emerging literacy skills. Reading research has also identified specific strategies that adults can use when reading aloud to enhance its impact.1 The combination of dedicated, trained volunteers, a careful selection of diverse literature, and the use of standards-based curriculum guides seem to be essential components to Read Aloud’s success. This evaluation suggests that Words Alive has developed a read aloud model that expands children’s opportunities for essential literacy-skill building and also instills a love of reading. 1 Lane, H., & Wright, T. (2007). Maximizing the effectiveness of reading aloud. International Reading Words Alive Read Aloud Program Evaluation | Caster Center, University of San Diego | ii OVERVIEW AND BACKGROUND Developing strong literacy skills at an early age is critical for the long-term academic success of students.2 A 2011 study showed that students who do not read proficiently by the time they leave third grade are four times more likely to not receive a high school diploma.3 Yet, according to the National Assessment of Educational Progress in 2015, only 36% of fourth graders nationwide were reading proficiently.4 Words Alive is a nonprofit organization focused on creating opportunities for life success by inspiring a commitment to reading. With numerous programs serving youth, teens and families, Words Alive teaches the value of reading and the importance of strong literacy skills in opening doors of opportunity for both children and their parents. Words Alive focuses on three areas of influence with regard to literacy development: What pa rticipants do (habits) How pa rticipants What participants feel about reading know and themselves as (skills & abilities) readers (interest, attitudes & motivation) One of the Words Alive literacy programs is called the Read Aloud Program (Read Aloud). Through Read Aloud, trained volunteer readers (hereafter referred to as volunteers) engage with students weekly in preschool through third grade classrooms to share the joy of reading. Reading aloud to young children offers an opportunity to create a love of literature and has been shown to have a positive impact in many areas of literacy development, such as vocabulary, comprehension, and word-recognition skills.5 In January 2016, Words Alive commissioned the Caster Family Center for Nonprofit and Philanthropic Research (Caster Center) at the University of San Diego to conduct an evaluation of the impact of Read Aloud on participating students. This report summarizes the findings from data collected between October 2016 and July 2017. 2 Annie E. Casey Foundation. (2010). Early warning! Why reading by the end of third grade matters. KIDS COUNT Special Report. Baltimore, MD: Leila Fiester. Retrieved from www.aecf.org. 3 Annie E. Casey Foundation. (2011). Double jeopardy: How third-grade reading skills and poverty influence high school graduation. Baltimore, MD: Donald J. Hernandez. Retrieved from www.aecf.org. 4 U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Reading Assessment. 5 Lane, H., & Wright, T. (2007). Maximizing the effectiveness of reading aloud. International Reading Association, 668-675. Words Alive Read Aloud Program Evaluation | Caster Center, University of San Diego | 1 Description of Read Aloud Read Aloud operates in child development centers, Head Start classrooms, and elementary schools across the San Diego area. With a focus on supporting children in high-need areas, Read Aloud serves approximately 4,300 students in preschool through third grade each week. Read Aloud has two models: 1. Large Group Model: Utilizing the Read Aloud curriculum and pre-selected books, trained volunteers visit their assigned classrooms for 30 minutes each week to conduct a large group read aloud session with 20-25 students. 2. Small Group Model: Volunteers visit classrooms weekly for 90 minutes to read the book to the whole class and then facilitate discussions and activities with small rotating groups of 3-5 students at a time. Building upon the benefits of the Read Aloud large group model, the small group model is intended to allow students a chance to build a deeper understanding of the story and allow the volunteer to focus more on students’ developing literacy skills. Over the course of the 2016/17 academic year, this model was active in two elementary schools: Golden Hill Elementary and Monarch School. In both models, volunteers are provided with standards-based resources to guide their session and help students focus on the critical skills needed for literacy development. Additionally, all classrooms participate in book giveaway events twice per year, in which each student receives a new book. Evaluation Objectives Based on a logic model developed collaboratively with staff from Words Alive and the Caster Center, data were collected to determine the impact of Read Aloud on students (See Appendix A for logic model). The specific objectives of this study were to evaluate: è The process and implementation of the Read Aloud small group model at Golden Hill Elementary è The impact of Read Aloud on children’s foundational literacy skill development è The impact of Read Aloud on children’s reading comprehension è The impact of Read Aloud on children’s reading motivation è If the small group model added significant value to Read Aloud Words Alive Read Aloud Program Evaluation | Caster Center, University of San Diego | 2 METHODOLOGY The findings presented in this report are based on a comprehensive synthesis of multiple data sources. The Caster Center administered a teacher survey across all participating Read Aloud schools and collected in-depth data at Golden Hill Elementary, one of the schools implementing the small group model. Summary of Data Sources Data Source Participants Description Motivation to n=28 In October 2016 and again in July 2017, Words Alive Read Profile • 2 classrooms staff administered the MRP to two classes of third (MRP)6 • All 3rd grade graders participating in the Read Aloud small group students model. Questions were read aloud to small groups of students to ensure comprehension of the items. Questions measured each student’s perceived value for reading and self-concept as a reader. Volunteer n=9 From October 2016 through June 2017, volunteers Field Notes • 158 field notes participating in the Read Aloud small group model • 7 classrooms completed weekly field notes, submitted online. Field • Grades K-3 notes documented weekly activities and volunteers’ observations about students’ literacy skill development and engagement in the sessions. Volunteer n=4 In July 2017, the Caster Center conducted a focus Focus Group group with volunteers who participated in the Read Aloud small group model during the 2016/17 academic year. Volunteers were asked about the strengths and challenges of Read Aloud, the support they received from Words Alive, and the perceived impact of the program on students. Teacher n=75 In Spring 2017, Words Alive sent a survey to 122 Survey • 22 schools teachers (61% response rate) participating in Read • 22% Preschool Aloud. The Caster Center designed the survey which • 47% Grades K-1 inquired about teachers’ perceptions of Read Aloud • 31% Grades 2-3 and its impact on students. • 84% had 2+ years experience with Read Aloud • 13% small group • 87% large group 6 Malloy, J. A., Marinak, B. A., Gambrell, L. B., & Mazzoni, S. A. (2013). Assessing motivation to read. The Reading Teacher, 67(4), 273-282. Words Alive Read Aloud Program Evaluation | Caster Center, University of San Diego | 3 Methods of Analysis The qualitative data (open-ended teacher survey responses, volunteer field notes and volunteer focus group) were analyzed using content analysis, a method for identifying the themes in responses.7 The quantitative data (MRP and teacher survey) analysis included the following:8 • Descriptive statistics to summarize the data (i.e., frequencies, percentages) • Paired-sample t-tests to test for statistically significant differences between pre- and post-mean (i.e., average) MRP responses • Independent sample t-tests to test for statistically significant differences between mean (i.e., average) teacher survey responses across grade levels and program model (small group vs. large group) READ ALOUD PROGRAM IMPACT ON CHILDREN Based on data collected from students, teachers, and Words Alive volunteers, Read Aloud supported children’s foundational literacy skill development, reading comprehension, and motivation and engagement in reading. Presentation of Findings The sections that follow present data from teachers, volunteers, and students. Note that the following icons for teachers, volunteers, and students are used throughout the report to signify the data source. Teachers: Teacher survey results are displayed in bar charts. The data are presented in aggregate (i.e., total) and by grade level or program model when there were statistically significant differences. Verbatim teacher quotes from open-ended survey responses are also included. Volunteers: Field note excerpts from volunteers at Golden Hill Elementary are displayed in text boxes formatted to look like a notebook. The field note excerpts describe the types of activities volunteers engaged in with students and identify instances in which students demonstrated their skills and knowledge. The field notes were edited for readability, and children’s names were removed to protect their identity. Verbatim quotes from the volunteer focus group are also included. Students: Student MRP results are displayed in bar charts. 7 Qualitative data were analyzed using Dedoose qualitative software. 8 Quantitative data were analyzed using SPSS statistical software. Words Alive Read Aloud Program Evaluation | Caster Center, University of San Diego | 4 Building Foundational Literacy Skills For students to develop the ability to comprehend texts, they must have both the skills to decode letter and word sounds and sufficient background knowledge of common text structures, concepts of print, and vocabulary, to name a few.9 According to teachers and volunteers, Read Aloud played a role in supporting children’s foundational literacy skill development. Figure 1 shows that nine out of ten teachers reported that Read Aloud had an impact on expanding students’ knowledge of literary terms, vocabulary, and concepts of print. Eight out of ten teachers reported it had an impact on improving fluency and learning phonics. Figure 1: Read Aloud’s impact on students’ foundational literacy skill development (n=75) (Teachers who reported 3-5 ratings on a 1=No impact to 5=Very large impact scale) Concepts of print 25% 32% 41% 98% Knowledge of literary terms 19% 37% 40% 96% Vocabulary development 16% 43% 37% 96% Phonics 23% 37% 24% 84% Fluency 25% 28% 29% 82% Moderate impact Large impact Very large impact 9 Dole, J. A., Duffy, G. G., Roehler, L. R., & Pearson, P. D. (1991). Moving from the old to the new: Research on reading comprehension instruction. Review of Educational Research, 61, 239–264. Words Alive Read Aloud Program Evaluation | Caster Center, University of San Diego | 5