Abstract Title Page Not included in page count. Title: The Influence of Content-Focused Coaching® on Reading Comprehension Instruction and Student Achievement Author(s): Lindsay Clare Matsumura (University of Pittsburgh) Helen Garnier (UCLA/LessonLab Research Institute) Brian Junker (Carnegie Mellon University) Lauren Resnick (LRDC/University of Pittsburgh) Donna DiPrima Bickel (LRDC/University of Pittsburgh) Abstract Body Limit 5 pages single spaced. Background/context: Description of prior research and/or its intellectual context and/or its policy context. Reading is critical to children’s academic and future economic success. Yet reading skills continue to lag in the upper elementary grades. Recent NAEP results from 2004, for example, showed that 69% of fourth-graders scored below a proficient level in reading, and 36% of fourth- graders scored in the below basic category (NAEP, 2005). Focused instruction on the basic skills of phonemic awareness, phonics and decoding fluency are beginning to show results in the primary grades. But many schools and districts report stagnation or even declines once students enter fourth grade, where the focus of reading instruction should turn to comprehension in other content areas (Snow, 2002). One way that school districts across the country have responded to the challenge of improving students’ reading skills has been to hire additional, presumably more skilled, teachers as “instructional coaches” to work with teachers to improve their instructional practice (Neufeld & Roper, 2003; Richard, 2003; Russo, 2004). However, there is no agreed upon definition of the coaching job, and the actual tasks that coaches carry out vary widely. In some schools coaches work with individual and small groups of teachers to analyze and plan lessons, and they may do some actual in-class observation and provide some “at elbow” guidance for teachers. In many schools, however, coaches are an extra pair of hands whom principals ask to take on various administrative tasks (for example, overseeing testing, ordering supplies, setting up school programs, attending district meetings, etc.) or to provide additional instructional time/intervention to individual or small groups of students. Most coaches receive very little training and ongoing support in either the subject-matter content or pedagogical strategies they are meant to help teachers master. Moreover, in the absence of clear standards for coach qualifications, coaches may not be more expert in their content area than the teachers they are 2009 SREE Conference Abstract Template 1 intended to instruct (Allington, 2007; Frost & Bean, 2006; International Reading Association, 2004). Not surprisingly, attempts to study the effectiveness of coaching have mostly found that coaching programs have been so poorly or variably implemented that it is not possible to determine with confidence what effect well-implemented coaching would have on either teacher performance or student learning (e.g., Rivera, Burley & Sass; 2004; Wade, 1985). While some evidence exists that coaching might improve teacher learning under specific conditions (e.g., Joyce & Showers, 1988), there is virtually no evidence that coaching improves student learning. Purpose/objective/research question/focus of study: Description of what the research focused on and why. Our project implements and studies a structured coaching program (Content-Focused CoachingSM; CFC) that is designed to improve reading comprehension instruction in the upper elementary grades. The fundamental research question for our study is: 1) What is the effect of a well-defined instructional coaching program (CFC) on reading comprehension instruction and students’ reading achievement? Setting: Specific description of where the research took place. The study was conducted in an urban school district in Texas serving large numbers of Hispanic, English language learning students from low-income families. The schools in our study (N=29) were sampled from the 32 lowest achieving schools in the district. Prior to the beginning of the CFC intervention, one treatment and two comparison schools left the study, resulting in 29 schools total (15 treatment and 14 comparison). Population/Participants/Subjects: Description of participants in the study: who (or what) how many, key features (or characteristics). Teachers. Only 98 of the 177 teachers participating in the first year of the study (55%) remained at their school or grade for the second year of the study (CFC n=60 61%; Comparison n=38 39%). Two teachers from comparison schools moved to treatment (CFC) schools. These teachers were treated as new teachers in the study and are not included in the results reported in this paper. One teacher who left a treatment school to teach at a comparison school and so was no longer eligible to be in the study. Of the 98 teachers who remained, 58% teach fourth grade and 42% teach fifth grade. Over a third of the teachers (38%) have a Master’s degree. A small number of teachers (4%) hold National Board for Professional Teaching Standards certification. Eight-percent of the teachers hold temporary, provisional, or emergency certification. Sixty-percent of the teachers teach a designated ELL class. As a group, teachers have a wide range of teaching experience, ranging from 0 to 32 years and averaging about 10 years teaching generally and 9 years teaching reading. English is the language of instruction for the majority of teachers (53%). The remaining teachers teach lessons in both English and Spanish. The teachers in the comparison and intervention schools do not differ on any demographic characteristics. Comparisons between the 98 two-year longitudinal teachers and teachers who left their grade or school at Year 2 indicated no 2009 SREE Conference Abstract Template 2 differences in education, teaching experience, or certification backgrounds. Only one difference (p<.05) emerged in the language of instruction in the Year 1 classes: 73% of teachers who left their school or grade taught their classes in English only compared to 53% of the longitudinal teachers. Students. Nearly all of the students in the study schools (N=1714) are eligible for free or reduced-price lunch (91%). The large majority of students (80%) are Hispanic. The remaining students are African American (15%), white (3%) and Asian (2%). Forty-percent of the students are classified as Limited English Proficient (n=741), and 2% are immigrants from a non-English- speaking country. Differences in the student population between the treatment and comparison schools were included as covariates in the analyses. Intervention/Program/Practice: Specific description of the intervention, including what it was, how it was administered, and its duration. Content-Focused Coaching (CFC) is an intensive professional development program that aims to ameliorate the problems of implementing effective coaching in schools by, first, providing coaches with the knowledge and skills they need to work effectively with teachers and, second, helping create the organizational conditions in schools and districts that are posited to support effective coaching. The program was originally developed in mathematics (West & Staub, 2003), but was adapted by fellows from the University of Pittsburgh’s Institute for Learning to support literacy coaching in elementary schools (Staub & Bickel, 2003; Staub, West, & Bickel, 2003). For the purpose of the study, school district leaders and fellows from the University of Pittsburgh’s Institute for Learning (IFL) collaborated to hire the CFC coaches. CFC coaches engaged in three days of professional development per month over the course of two academic years. The professional development, led by the IFL fellows, is designed to increase coaches’ knowledge of the theory and research underlying effective reading comprehension, vocabulary, and writing instruction (Beck, McKeown, & Kucan, 2002; Block & Pressley, 2002; Hirsch, 2003, 2006; New Standards, 1999; Resnick, 1995; Resnick & Hall, 2001; Sinatra, Brown, & Reynolds, 2002; Willingham, 2006), with a special emphasis on the role of classroom talk and vocabulary instruction in supporting students’ understanding of texts The program also focuses on building coaches’ pedagogical skills so that they can model exemplary reading comprehension lessons in teachers’ classrooms and coaches’ expertise at working effectively with teachers in schools. Coaches engage in “coaching labs,” in which they observe other coaches and IFL trainers working with teachers to plan a future lesson, reflect together on an enacted lesson, and/or determine next steps for improving student learning. CFC-trained coaches, in turn, are expected to meet with teachers individually and in weekly grade-level team meetings to study the theory underlying effective reading comprehension instruction and plan rigorous reading comprehension lessons (i.e. to implement what they learned in the training). CFC coaches also are expected to model lessons, co-teach, observe reading comprehension lessons in teachers’ classrooms on a regular basis, and reflect on the effectiveness of these lessons (e.g., what students seemed to understand, what they seemed to 2009 SREE Conference Abstract Template 3 have difficulty comprehending, etc.) with teachers in post-lesson conferences. Improved quality of reading comprehension instruction, in turn, is posited to lead to improved student achievement in reading. Research Design: This study is a randomized-field trial. Data Collection and Analysis: Description of plan for collecting and analyzing data, including description of data. Teachers completed surveys at baseline and at the end of each study year focused on their participation in literacy coaching and instructional practice. Teachers also were observed teaching a reading comprehension lesson twice during each study year (in fall and spring). Student achievement was assessed on the Degrees of Reading Power (DRP) assessment and the state achievement test (Texas Assessment of Knowledge and Skills, TAKS). Repeated measures analyses of variance were used to identify and compare patterns of change in teachers’ participation in coaching activities, the content emphasized in coaching activities, and teachers’ perceptions of the usefulness of coaching for improving their practice. Frequencies also were computed to describe variation in teachers’ participation in specific coaching activities. Repeated measures analyses of variance also were used to investigate patterns of change over time in teachers’ self-reported and observed instruction. HLM analyses were used to estimate the relative effect of CFC on student achievement. Findings/Results: Results indicate that teachers’ in the CFC schools significantly and positively increased in the frequency of their participation in coaching focused on enacting instruction and understanding the theory underlying effective reading comprehension instruction. Teachers in the CFC schools also showed modest improvement in the quality of their observed and self-reported reading comprehension instruction. No difference between groups was detected for students’ reading achievement on the whole (N=1714 LEP and non-LEP students). Students designated as Limited English Proficient (n=741) in the CFC schools, however, had significantly higher scores on the TAKS. No difference between groups was detected on the DRP. Conclusions: The implications for education policy with regard to the use of coaching resources in urban districts, and challenges of conducting a multi-level intervention and a randomized-field trial in an urban district with high-levels of teacher (and student) mobility will be discussed. 2009 SREE Conference Abstract Template 4 Appendixes References Allington, R.L. (2006). President's Message: Reading specialists, reading teachers, reading coaches: A question of credentials. Reading Today, 23(4), 16-17. Ball, D.L., & Cohen, D.K. (1999). Developing practice, developing practitioners: Toward a practice-based theory of professional education. In G. Sykes & L. Darling-Hammond (Eds.), Teaching as the learning profession: Handbook of policy and practice (pp. 3-32). San Francisco: Jossey-Bass. Block. & Pressley, M. (2002). Comprehension instruction: Research-based best practices. New York, NY: Guilford. Beck, I.L., McKeown, M.G., & Kucan, L. (2002). Bringing words to life: Robust vocabulary instruction. New York: The Guilford Press. Beck, I.L., & McKeown, M.B. (2006). Improving comprehension with questioning the author: A fresh and expanded view of a powerful approach. New York: Scholastic Inc. Bickel, D.D., Nosal, M., & Moroch, G. (2005). Using Content-Focused CoachingSM to build a reflective learning community focused on high student achievement in elementary literacy: Challenges and supports. Presentation at the 37th Annual National Staff Development Council Conference. Philadelphia, PA. Coburn, C. (2003). Rethinking scale: Moving beyond numbers to deep and lasting change. Educational Researcher, 32(6), 3-12. Cohen, D.K., & Ball, D. L. (2001). Making change: Instruction and its improvement. Phi Delta Kappan, 83(1), 73-77. Costa, A., & Garmston, R. (2002). Cognitive Coaching: A foundation for Renaissance schools. Norwood, MA: Christopher-Gordon. Darling-Hammond, L. & McLaughlin, M. (1995). Policies that support professional development in an era of reform. Phi Delta Kappan, 76(8), 597-604. Duessen, T., Coskie, T., Robinson, L., & Autio, E. (2007). “Coach” can mean many things: Five categories of literacy coaches in Reading First. REL Northwest (REL 2007-No. 005). IES: US Department of Education. 2009 SREE Conference Abstract Template A–1 Garet, M., S., Porter, A.C., Desimone, L., Birman, B.F., & Yoon, K.S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915-945. Glennan, T.K., Jr. & Resnick, L. (2004). School districts as learning organizations: A strategy for scaling education reform. In T.K. Glennan, Jr., S.J. Bodilly, J. Galegher, and K. Kerr, (Eds.) Expanding the reach of education reforms: Collected essays by leaders in the scale-up of educational interventions. Santa Monica, CA: RAND. Hawley, W. D., & Valli, L. (1999). The essentials of effective professional development. A new consensus. In L. Darling-Hammond & G. Sykes (Eds.), Teaching as the Learning Profession. Handbook of Policy and Practice (pp. 127-150). San Francisco: Jossey-Bass. Hirsch, E.D., Jr. (2006). Building knowledge: The case for bringing content into the language arts block and for a knowledge-rich curriculum core for all children. American Educator, Spring 2006, 8-21, 28-29, 50-51. Hirsch, E.D., Jr. (2003). Reading comprehension requires knowledge – of words and the world. American Educator, Spring 2003, 10-29, 48. Joyce, B., & Showers, B. (1988). Student Achievement through staff development. New York: Longman. Lieberman, A. (1995). Practices that support teacher development. Phi Delta Kappan, 76(8), 591-596. Louis, K.S., Marks, H.M., & Kruse, S. (1996). Teachers’ professional community in restructuring schools. American Educational Research Journal, 33(4), 757-798. Matsumura, L.C., Garnier, H., Slater, S.C., & Boston, M.B. (In press). Measuring instructional interactions 'at-scale', Educational Assessment. Matsumura, L.C., Garnier, H., Bickel, D.D., Junker, B., & Resnick, L. (2007). Year 1 Report: Content-Focused CoachingSM for high-quality reading instruction. Annual Report submitted to the Institute for Education Sciences. Matsumura, L.C. (2005). Creating high-quality classroom assignments. Lanham, MD: Scarecrow Education. Michaels, S., O’Connor, M.C., & Hall, M.W. (2002). Accountable TalkSM: classroom conversation that works. CD-ROM Series, University of Pittsburgh. Neufeld, B., & Roper D. (2003) Coaching: A strategy for developing instructional capacity: Promises and practicalities. Washington, DC: Aspen Institute Program on Education and Annenberg Institute for School Reform. 2009 SREE Conference Abstract Template A–2 NEW STANDARDS (1999). Reading and writing grade by grade: Primary literacy standards for kindergarten through third grade. National Center on Education and the Economy and the University of Pittsburgh. O’Donnell, C. (2008). Defining, Conceptualizing, and Measuring Fidelity of Implementation and Its Relationship to Outcomes in K–12 Curriculum Intervention Research Review of Educational Research, Vol. 78, No. 1, 33-84 (2008) Pearson, P.D. & Gallagher, M. (1983). The instruction of reading comprehension. Contemporary Educational Psychology, 8(3), 317-344. Pearson, P.D., & Hamm, D. (2005). The assessment of reading comprehension: A review of practices- past, present, and future. In S.G. Paris & S.A. Stahl (Eds.), Children's reading comprehension and assessment (pp. 13-69). Mahwah NJ: Lawrence Erlbaum Associates. Raudenbush S W, Bryk, A S, Cheong Y K, Congdon R T. HLM 6: Hierarchical linear and nonlinear modeling. Lincolnwood, IL: Scientific Software International, 2004. Resnick, L.B. (1995). From aptitude to effort: A new foundation for our schools. Daedalus, 124(4). 55-62. Resnick, L.B., & Hall, M.W. (2001). The principles of learning: Study tools for educators. [CD- ROM, version 2.0]. Available: www.Instituteforlearning.org. Resnick, L.B., & Hall, M. W. (1998). Learning organizations for sustainable education reform. Daedalus, 127(4), 89-118. Resnick, L.B., & Spillane, J. P. (2006). From individual learning to organizational designs for learning. In Verschaffel, L., Dochy, F., Boekaerts, M., & Vosniadou, S. (Eds). Instructional psychology: Past, present and future trends. Sixteen essays in honor of Erik De Corte (Advances in Learning and Instruction Series) (pp.. 257-274). Oxford: Pergamon. Richard, A. (2003). Making our own road: The emergence of school-based staff developers in America’s public schools. New York: Edna McConnell Clark Foundation, 2003. Russo, A. (2004). School based coaching: A revolution in professional development or just a fad. Harvard Education Letter, 20(1-6). Roller, C. (2006). Reading and literacy coaches report on hiring requirements and duties survey. Newark, DE: International Reading Association. Schon, D.A.(1983). The reflective practitioner: How professional think in action. New York: Basic Books. 2009 SREE Conference Abstract Template A–3 Schon, D.A. (1987). Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. San Francisco: Jossey-Bass. Sinatra, G.M., Brown, K.J., & Reynolds, R. (2002). Implications of cognitive resource allocation for comprehension strategies instruction. In C.C. Block & M. Pressley (Eds.), Comprehension instruction: Research-based best practices (pp. 2-76). New York: Guilford Press. Snow, C. (2002). Reading for understanding: Toward a R&D program in reading comprehension. Santa Monica, CA: Rand Education. Spillane, J.P. & Resnick, L.B. (2008). Learning leadership: Kernel routines for instructional improvement. Unpublished manuscript. Northwestern University and University of Pittsburgh. Staub, F.C. & Bickel, D.D. (2003). Developing Content-Focused CoachingSM in elementary literacy: A case study on designing for scale. Paper presented at the biannual meeting of the European Association on Research and Instruction (EARLI), Padova, Italy. Staub, F.C., West, L. & Bickel, D.D. (2003). What is content-focused coaching? In L. West & F.C. Staub (Eds.), Content-Focused CoachingSM: Transforming mathematics lessons (pp. 1-17). Portsmouth, NH/Pittsburgh, PA: Heinemann/University of Pittsburgh. Stein, M.K., Smith, M.S., & Silver, E.A. (1999). The development of professional developers: Learning to assist teachers in new settings in new ways. Harvard Educational Review, 69(3), 237-269. Walpole, S., & McKenna, M.C. (2004). The literacy coach’s handbook: A guide to research- based practices. New York, NY: Guilford Press. West, L. & Staub, F.C. (2003). Content-focused coachingSM: Transforming mathematics lessons, Portsmouth, NH: Heinemann. Willingham, D.T. (2006). How knowledge helps: It speeds and strengthens reading comprehension, learning – and thinking. American Educator, Spring 2006, 30-37. 2009 SREE Conference Abstract Template A–4 Appendix B. Tables and Figures Not included in page count. Table 1 Insert Table Title Here Table Text Figure 1. Insert figure caption here 2009 SREE Conference Abstract Template B–1 APA Reference Style Examples Sample Citation: Journal Article Hypericum Depression Trial Study Group. (2002). Effect of Hypericum perforatum (St John’s Wort) in major depressive disorder: A randomized controlled trial. JAMA, 287, 1807– 1814. Sample Citation: Newsletter/Newspaper Article Brown, L. S. (1993, Spring). My research with oranges. The Psychology Department Newsletter, 3, 2. Sample Citation: Book American Psychiatric Association. (1990). Diagnostic and statistical manual of mental disorders (3rd ed.). Washington, DC: Author. Booth, W. C., Colomb, G. G., & Williams, J. M. (1995). The craft of research. Chicago: University of Chicago Press. Sample Citation: Chapter or Section in a Book Stephan, W. G. (1985). Intergroup relations. In G. Lindzey & E. Aronson (Eds.), The handbook of social psychology (3rd ed., Vol. 2, pp. 599–658). New York: Random House. Sample Citation: Web Page Dewey, R. A. (2004). APA Style Resources by Russ Dewey. Retrieved September 8, 2004, from http://www.psywww.com/resource/apacrib.htm 2009 SREE Conference Abstract Template