ELLs with Disabilities Report 17 Use of Chunking and Questioning Aloud to Improve the Reading Comprehension of English Language Learners with Disabilities N A T I O N A L C E N T E R O N EDUCATIONAL O U T C O M E S In collaboration with: Council of Chief State School Officers (CCSSO) National Association of State Directors of Special Education (NASDSE) Supported by: U.S. Office of Special Education Programs ELLs with Disabilities Report 17 Use of Chunking and Questioning Aloud to Improve the Reading Comprehension of English Language Learners with Disabilities Manuel Barrera • Kristi Liu • Martha Thurlow • Steve Chamberlain December 2006 All rights reserved. Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as: Barrera, M., Liu, K., Thurlow, M., & Chamberlain, S. (2006). Use of chunking and questioning aloud to improve the reading comprehension of English language learners with disabilities (ELLs with Disabilities Report 17). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. N A T I O N A L C E N T E R O N EDUCATIONAL O U T C O M E S The Pathways for Promoting the Success of English Language Learners with Disabilities in Standards-based Education project is supported by a grant (#H324D010023) from the Research to Practice Division, Office of Special Education Programs, U.S. Department of Education. Opinions expressed herein do not necessarily reflect those of the U.S. Department of Education or Offices within it. NCEO Core Staff Deb A. Albus Jane L. Krentz Jason R. Altman Kristi K. Liu Manuel T. Barrera Ross E. Moen Laurene L. Christensen Michael L. Moore Marjorie I. Cuthbert Rachel F. Quenemoen Cynthia L. Jiban Dorene L. Scott Christopher J. Johnstone Martha L. Thurlow, Director National Center on Educational Outcomes University of Minnesota • 350 Elliott Hall 75 East River Road • Minneapolis, MN 55455 Phone 612/626-1530 • Fax 612/624-0879 http://www.nceo.info The University of Minnesota is committed to the policy that all persons shall have equal access to its programs, facilities, and employment without regard to race, color, creed, religion, national origin, sex, age, marital status, disability, public assistance status, veteran status, or sexual orientation. This document is available in alternative formats upon request. Introduction Growing pressure from both the teaching profession (cf. Cochran-Smith & Zeichner, 2005; Levine, 2006; Walsh, Glaser, & Wilcox, 2006) and federal legislation in the No Child Left Behind Act is forcing a reexamination of instructional practices as a way to improve academic achievement of students who do not perform at grade level. Students with disabilities are re- ceiving increased attention as one such group (National Council on Disability, 2004a, 2004b; Thurlow, Anderson, Minnema, & Hall-Lande, 2005; Thurlow, Minnema, & Treat, 2004). A growing body of research documents that English language learners (ELLs) with disabilities are a group of students with an overall level of achievement on statewide assessments that is even farther below that of the larger group of students with disabilities (Albus, Barrera, Thurlow, Guven, & Shyyan, 2004; Albus & Thurlow, 2005; Albus, Thurlow, Barrera, Guven, & Shyyan, 2004; Liu, Barrera, Thurlow, Guven, & Shyyan, 2005; Liu, Thurlow, Barrera, Guven, & Shyyan, 2005) and that of ELLs. Despite this documentation of poor performance, there are few research studies on instructional strategies specifically aimed at improving the literacy of ELLs with disabilities. Clearly, teach- ers need increased support in creating accessible instruction on grade-level content standards so that ELLs with disabilities can achieve academically. The need for research to support instructional practices spans all grade levels but is particularly acute at the middle school level for a variety of reasons. First, many ELLs who enter public school in the United States for the first time in middle school or junior high have experienced inconsistent schooling in their primary language (McKeon, 1994). McKeon has noted that ap- propriate inclusion of students with disabilities becomes more challenging at the middle school level. Teachers must find ways to teach students who, along with limited English proficiency and some learning challenges created by the disability, may not yet have a knowledge base in areas to which their peers have already been exposed. Hence, achieving grade level standards in English will be doubly challenging. A second reason for the importance of research at the middle school level is that the curriculum makes greater cognitive demands on students, especially in the expectations of reading literacy for the content areas (Klingner, Artiles, & Barletta, 2006). Finally, as the age of students in- creases, so does the chance of school dropout (Mikow-Porto, Humphries, Egelson, O’Connell, & Teague, 2004). High school dropout rates are around 30% for students with disabilities (National Council on Disability, 2004a); this rate is significantly higher than the rates for students without disabilities. Forty percent or more of secondary students with disabilities do not receive a high school diploma (National Council on Disability, 2004a). Research studies targeting students in grades 6-9 can be important sources of information on ways to support the grade level content learning and school completion of these students. NCEO  Reading: The Gateway to Academic Success ELLs with disabilities struggle with reading and the reasons for their struggles are not well understood owing to little knowledge about the impact of disability on language development in either the first or second language (Klingner et al., 2006). Nevertheless, this difficulty in read- ing achievement historically has been a marker for students with learning disabilities (Bender, 2003). It may be challenging for ELLs with disabilities to learn in large general education classes (Vaughn, Klingner, & Bryant, 2001). Vaughn and her colleagues indicate that because students with the most severe reading difficulties need intense reading strategy instruction, they must have additional support services in a special education setting where there are sufficient time and resources to address student’s specific learning needs. Thus, it seems apparent that an important area of research attention is to examine the benefits of instruction in the specialized settings where small group or individualized services are provided to English language learn- ers with disabilities. Moreover, because of the need to adapt learning situations to the specific needs of these students, it seems imperative that any examination of individualized instruction should also examine the degree to which adjustments to instruction should be made. It should also examine how different approaches to the instruction of these learners may affect learner outcomes. To add to this limited, but growing knowledge base, this report provides details about a series of single-subject research studies. The studies examine how an instructional reading strategy identified by classroom teachers could be used to improve grade-level standards-based read- ing achievement among English language learners with learning disabilities in individualized instructional settings. A companion report (Barrera, Liu, Yan, Chamberlain, & Thurlow, 2006) examines a similar series of studies with a mathematics instructional strategy. Background Before discussing the relevant literature on the instructional strategy investigated in this research study, it is important to describe aspects of the research process that influenced the choice of strategy for this single-subject study. The research described in this report was developed based on previous input from practicing teachers during the 2002-2003 school years (Thurlow, Albus, Shyyan, Liu, & Barrera, 2004). At that time, small groups of teachers from many disciplines worked together to answer the question: What instructional strategies do you use or do you recommend for teaching grade-level standards-based content to middle school and junior high ELLs with disabilities? Through a structured brainstorming technique called Multi-Attribute Consensus Building (MACB; cf. Vanderwood, Ysseldyke, & Thurlow, 1993), teachers developed a list of reading strategies they recommended, defined the strategies in their own words, and weighted the strategies in terms of importance (see Thurlow et al., 2004 for a comprehensive list of strategies). These lists of recommended strategies served as a starting point for the single-  NCEO subject intervention described in this report as well as another single subject study focusing on a mathematics instructional strategy (Barrera et al., 2006). Procedures for the single-subject studies were developed using established research methods (cf. Tawney & Gast, 1984) and involved some of the reading strategies most highly weighted in MACB group sessions. Teacher-identified strategies were chosen because of their relatively strong support and the degree to which they could be “operationalized” into a specific procedure. Strategy Definition: Chunking and Questioning The reading instructional strategy that is examined in this single-subject intervention is “Chunk- ing and Questioning Aloud” (CQA). Teachers in MACB groups (Thurlow et al., 2004) described this strategy with two distinct parts: “The process of reading a story aloud to a group of students and stopping after certain blocks of text to ask the students specific questions about their com- prehension of the story and some key features of the text” (Thurlow et al., 2004, p. 34). Operationalizing Chunking and Questioning for Use in Research The chunking and questioning strategy nominated in our research study bears a strong resem- blance to what is commonly known in the teaching literature as a “Directed Reading Thinking Activity” or DRTA (Stauffer, 1969). Stauffer embedded the DRTA into a larger approach to teaching called the Language Arts Approach (or Language Experience Approach, cf. Stauffer, 1980) which involved extensive reading and writing of materials relating to a child’s experi- ences. According to Stauffer (1969), DRTA is a group problem solving approach to reading that teaches children comprehension skills through making predictions about the text and finding evidence to support or refute those predictions. The group-based approach provides an environment in which students behave as readers who think critically about texts. Students can then take the behaviors they practiced in the group setting and apply them to individual reading situations. In the DRTA, the teacher chooses a text at the student’s instructional level and divides it into chunks of varying lengths to maintain reader interest. The students then set the purpose for reading by making predictions about the individual chunks of text. Knowing the purpose for reading helps skilled readers determine how fast they should read a text of a particular diffi- culty. Students read a chunk at a time to determine whether their predictions about each chunk are correct. Finally, students use evidence from the text to prove or disprove their predictions in a group discussion. Other students respond and the teacher can guide student thinking by asking questions such as “Why do you think so?” or “Can you prove it?” Readers then have an opportunity to revise their predictions if necessary, set new predictions for the next chunk of text, and continue the process. NCEO  Stauffer conducted large-scale quantitative studies into the effectiveness of the Language Arts Approach (cf. Stauffer & Hammond, 1969). He found the Language Arts Approach to be effec- tive at improving reading comprehension as well as writing and spelling skills for students in the primary grades. However, the effects of the DRTA alone were not examined in his work. The DRTA and a teacher-led variant, Directed Listening and Thinking Aloud (DLTA) are widely recommended in the popular teaching literature. Organizations as diverse as the National Urban Alliance for Effective Education (n.d.); The Education Alliance (n.d.); The Northwest Educa- tional Laboratory (NWREL, n.d.); and the United Nations Educational, Scientific and Cultural Organization (UNESCO, n.d.) mention DRTA on lists of recommended instructional strategies. The Kansas state reading standards for 4th grade refer to DRTA as a way to meet benchmarks in reading comprehension and fluency (KSDE, 2003). Test preparation materials for the Florida Comprehensive Assessment Tests in grade 4 contain references to both DRTA and DLTA as potentially useful strategies for standards-based instruction (Florida Center for Instructional Technology, 2006). Many other grass-roots organizations and teacher bulletin boards on Web sites contain a description of how teachers have used DRTA and DLTA in their classes. However, on close examination it becomes apparent that the strategy has many variations that teachers use as they adapt DRTA to particular contexts. Some versions are more structured and teacher directed. Other activities use no text at all and involve little teacher input other than directing students to fill in a chart of what they know and want to know about the topic of a book (McIn- tosh & Bear, 1993). All of these various strategies and activities are called DRTA but the stated purposes vary greatly from activating prior knowledge, modeling, and reducing student reading anxiety to the teaching of prediction skills in reading fiction. Our review of research documenting the effects of DRTA found three specific research studies on DRTA that assisted us in structuring an intervention. One relatively recent article by Schorz- man and Cheek (2004) examined a teacher-directed version of DRTA in a general education classroom; this research may or may not have included students with disabilities. Schorzman and Cheek (2004) examined the use of DRTA in combination with a “Pre-reading plan” (cf. Langer & Nicholich, 1981) and graphic organizers (cf. Barron, 1969). Three middle school teachers in one school used a combination of strategies to teach reading and the results were compared to a control group of three other teachers at a different middle school in the district under different teaching conditions in which intensive test preparation was a focus for reading instruction. The findings of the study were mixed. The package of instructional strategies appeared to create significant pre-post student gains on a cloze test but not on a standardized reading test. Findings were complicated by issues such as logistical constraints on instructional time, teachers’ lack of willingness to change curricula for the control group setting, and a research design using intact classrooms that may have had some preexisting differences in student ability levels. In addition, standardized reading test scores may simply not be sensitive to relatively short instructional  NCEO interventions. The authors did not mention whether students with disabilities were included in the classrooms involved in the research study. A second study by Bauman, Russell and Jones (1992) examined the effectiveness of a think aloud strategy on the reading comprehension of 4th grade students. Students with disabilities appear to have been excluded from the study. Although the focus of the study was on using the think aloud procedure, the experimental design included one comparison group of students using DRTA and a control group that was taught via a teacher-led guided reading process that researchers called Directed Reading Activity (DRA). The study incorporated a pre-test–post- test design with the type of intervention as the independent variable and scores from reading assessment tasks (e.g., cloze exercises, comprehension monitoring activities and error detec- tion tasks) along with qualitative interview data as the dependent variable. Results of the study indicated that both the Think Aloud strategy and the DRTA strategy were better at increasing students’ reading comprehension skills than the traditional teacher-led method of teaching reading. However, the data were not conclusive as to which of the strategies were the most successful. On some measures it appeared that students in the Think Aloud group had greater comprehension skills while on other measures it appeared that students in the DRTA group had better comprehension skills. A third study by Draheim (1986) is a commonly referenced conference presentation on the use of the DRTA with college students. Draheim (1986) investigated the effects of four instructional strategies or combinations of strategies on student recall and use of main and surbordinate ideas in analytical essays about reading texts. The strategies included (1) Mapping, (2) Directed Reading Thinking Activity (DRTA), (3) DRTA + mapping, and (4) Reading for main ideas and underlining. This four-and-a-half week experimental study involved students in remedial education courses but did not specifically mention students with disabilities. Teachers in four sections of a remedial writing course each taught their students the use of a strategy for reading or studying text. Teachers were trained in using strategies with which they were not familiar and then were assigned to the experimental or control condition they preferred. After a period of guided instruction in the use of the instructional strategy, students were then asked to use the strategy independently to read and understand a piece of text and write an analytical es- say based on it. The researcher measured the number of main and subordinate ideas students recalled after reading, and coded the essays for evidence that the ideas had been transferred into writing. Students who were taught the DRTA plus mapping strategy could recall and use the largest number of main ideas in their writing. Students in the other conditions were able to recall main ideas but were less likely to use them in their writing. The available research provided guidance in five particular areas for structuring an intervention specifically aimed at studying the effectiveness of our CQA strategy. First, the studies reviewed (Bauman et al., 1992; Draheim, 1986; Schorzman & Cheek, 2004) were conducted with groups NCEO  of students who were largely not students with disabilities. Given the unique learning needs of ELLs with disabilities and their frequent need for individualized instruction, a single subject study seemed to be a more suitable research design. Single subject studies allow teachers to adapt instructional interventions to the unique needs of the student as well as to the demands of the instructional context. The results of such studies show learning under optimal conditions. Second, the studies reviewed did not isolate the use of the DRTA strategy; therefore, the results of the studies cannot speak with confidence to DRTA’s effectiveness at increasing student learn- ing outcomes. Schorzman and Cheek (2004) studied DRTA used simultaneously with another instructional strategy while Bauman et al. (1992) included DRTA as a comparison for a different strategy in which the researchers were most interested. Given the frequency with which this instructional strategy appears to be used by educators, well-designed research on the effects of strategies like DRTA alone seems appropriate. Third, the Schorzman and Cheek article (2004) suggested the importance of allowing teachers to have choice in the curriculum used for instruction as well as the importance of incorporat- ing extensive teacher training on the use of instructional strategies. Draheim (1986) found that teachers asked for choice in which strategy they implemented. Incorporating an element of teacher choice in the teaching of the instructional strategy seemed desirable. Fourth, Bauman et al. (1992) highlighted the importance of explicitly teaching students the components of the strategy while phasing out teacher guidance and increasing student respon- sibility for strategy use. The researchers followed a sequence of steps from teacher modeling to guided practice and independent practice to ensure that students could use the strategy in- dependently. Finally, both Schorzman and Cheek (2004) and Draheim (1986) incorporated the idea of training teachers on strategy definitions. Furthermore, Schorzman and Cheek (2004) created observation checklists for assessing teacher fidelity to basic components of the strategy while Bauman et al. (1992) included observations of researcher fidelity to the strategy in a setting where researchers acted as teachers for the duration of the study. Teachers may adapt instructional strategy use over time and it is important to know exactly what teachers are doing. Training helps ensure that all teachers have a common frame of reference on what a strategy includes and observing fidelity to the strategy enables researchers to describe adaptations teachers make in a particular instructional context. The findings from this review of research helped to inform the methodology we used in imple- menting this study. Because of the need for a clearly defined, stepwise method, we further ex- amined the practitioner literature for how the DRTA is operationalized, that is, defined clearly in steps for use in the classroom. We chose one form, a lesson plan by a practitioner, Padak  NCEO (2006), as being closest to what teachers in our earlier research (Thurlow et al., 2004) described as the CQA strategy. Padak (2006) describes the following process: 1. Students read title (and perhaps a bit of the reading) and make predictions about con- tent. 2. Students read the first chunk of text. They confirm, refine or reject their predictions and justify their actions using evidence from the text. Students then make new hypotheses. 3. Students read the next chunk and follow procedures in step two. This cycle continues until the entire text is completed. 4. Follow-up activities may be completed after the text is read. This process formed the basis for the CQA strategy reported here. Method Single subject research (also known as single case research) was the core methodology of this study. This method is considered experimental rather than correlational or descriptive, and its purpose is to document causal or functional relationships between independent and dependent variables as applied to research with individual subjects (Campbell & Stanley, 1963; Tawney & Gast, 1984). Single case research employs within- and between-subjects comparisons to control for major threats to internal validity, and requires systematic replication to enhance external validity (Martella, Nelson, & Marchand-Martella, 1999). Choosing a Strategy An additional feature of our research was to simulate the instructional assessment and planning process by providing training to participating teachers so that they could (1) identify a student’s academic needs from her or his IEP and observed needs in meeting state academic standards, and (2) choose the appropriate strategy for a student based on these identified student needs. The research team selected three reading instructional strategies derived from among the high- est supported strategies identified through the prior study using Multi-Attribute Consensus Building with classroom teachers (Thurlow et al., 2004). Factors used in choosing strategies consisted of: • attributed levels of importance, feasibility, and use from the previous study • research support within the research literature • specific treatment needs of students identified by teachers • prerequisite skill requirements; and • roles of teachers and students in employing each strategy. NCEO