897886 research-article20202020 EROXXX10.1177/2332858419897886Spencer et al.Multitiered Dual-Language Instruction AERA Open January-March 2020, Vol. 6, No. 1, pp. 1 –16 DOI: 10.1177/2332858419897886 https://doi.org/ Article reuse guidelines: sagepub.com/journals-permissions © The Author(s) 2020. http://journals.sagepub.com/home/ero Early Efficacy of Multitiered Dual-Language Instruction: Promoting Preschoolers’ Spanish and English Oral Language Trina D. Spencer University of South Florida Meghan Moran Northern Arizona University Marilyn S. Thompson Arizona State University Douglas B. Petersen Brigham Young University M. Adelaida Restrepo Arizona State University The purpose of this cluster randomized group study was to investigate the effect of multitiered, dual-language instruction on children’s oral language skills, including vocabulary, narrative retell, receptive and expressive language, and listening com- prehension. The participants were 3- to 5-year-old children (n = 81) who were learning English and whose home language was Spanish. Across the school year, classroom teachers in the treatment group delivered large-group lessons in English to the whole class twice per week. For a Tier 2 intervention, the teachers delivered small-group lessons 4 days a week, alternat- ing the language of intervention daily (first Spanish, then English). Group posttest differences were statistically significant, with moderate to large effect sizes favoring the treatment group on all the English proximal measures and on three of the four Spanish proximal measures. Treatment group advantages were observed on Spanish and English norm-referenced standard- ized measures of language (except vocabulary) and a distal measure of language comprehension. Keywords: ANOVA/MANOVA, bilingual/bicultural, early childhood, language comprehension/development, vocabulary Reading comprehension and academic achievement are need of effective interventions that are strategically and dependent on oral language skills (Catts, Fey, Tomblin, & intensely designed to prepare them for the academic lan- Zhang, 2002; Gough & Tunmer, 1986; Griffin, Hemphill, guage demands of school (Castro, Páez, Dickinson, & Frede, Camp, & Wolf, 2004; Storch & Whitehurst, 2002). Although 2011). The purpose of this study was to examine the effect of interventions to promote code-related skills have prolifer- an innovative instructional model designed specifically for ated, interventions to systematically teach oral language and young dual-language learners on children’s oral language its components, such as vocabulary, narratives, listening skills preparatory to their entrance into kindergarten. comprehension, and use of complex sentences (Cain & Oakhill, 2011; Elleman, Lindo, Morphy, & Compton, 2009; The Oral Language and Literacy Connection Mehta, Foorman, Branum-Martin, & Taylor, 2005; Verhoeven & van Leeuwe, 2008), remain largely unavail- Oral language is a unique and meaningful indicator of able to early-childhood educators (Zucker, Cabell, Justice, academic success (Barton-Hulsey, Sevcik, & Romski, 2017; Pentimonti, & Kaderavek, 2013). Spanish-speaking children Catts, Nielsen, Bridges, & Liu, 2016; Chaney, 1998; Clarke, entering English-only elementary schools are in particular Snowling, Truelove, & Hulme, 2010; Larney, 2002). Creative Commons Non Commercial CC BY-NC: This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage). Spencer et al. Specifically, vocabulary (Bleses, Makransky, Dale, Højen, chief MTSS attributes, which are (a) multiple tiers of & Ari, 2016; National Institute of Child Health and Human instruction and intervention, (b) students who need more Development, 2000), narrative ability (Griffin, et al., support transition to more intense arrangements of inter- 2004), listening comprehension (Catts, Adlof, & Ellis vention, (c) interventions are intensified by adjusting the Weismer, 2006), and the use of complex sentences (Craig, duration and frequency of intervention, and the expertness Connor, & Washington, 2003) are key contributors to of the interventionist, (d) educators other than classroom reading comprehension. Limited reading comprehension teachers assist in the delivery of targeted and intensive can be the direct result of limited academic English oral interventions, and (e) tiered placement is determined irre- language skills (Cain, Lemmon, & Oakhill, 2004; Catts spective of special education classification (Marston, et al., 2006; Dickinson, McCabe, Anastasopoulos, Peisner- 2005). Feinberg, & Poe, 2003; Hammer, Lawrence, & Miccio, MTSS has several advantages over the traditional gen- 2007). Many young children with typical language-learn- eral-special education dichotomy. Perhaps the greatest is ing ability may not produce or understand language on par that rather than focusing on what caused the delays, MTSS with academic expectations for a variety of reasons, delivers supplemental intervention to all who need it, not including economic, cultural, and linguistic diversity just those with the appropriate diagnosis. Despite the suc- (National Center for Education Statistics, 2017). The idea cess of MTSS for early reading intervention, language has that children with language differences must wait until been neglected. If the goal is to ensure all children receive their language difficulties evolve into reading difficulty what they need to succeed in school, then more systematic and poor academic performance to receive special, indi- language intervention should be considered for children vidualized help is problematic because with early identifi- with language differences. In the traditional system, chil- cation and intervention, their difficulties may be prevented dren who receive language supports experience no inter- (Catts, 1993; Catts et al., 2006). mediate step such as Tier 2 intervention. There is no With the adoption of higher language and reading stan- strategy for eliminating environmental confounds to lan- dards across states, expectations of what children are to guage delays and no way to prevent language-related dis- understand and produce linguistically in school have like- abilities. Students go straight from classroom instruction wise increased. Young children who have typical lan- to special education, and that pathway is only available to guage-learning abilities but who are far behind their peers students who have a disability. Nonetheless, a multitiered in English language development, for whatever reason, approach for language, one that affords an intermediate, have few options. The outdated dichotomous system of preventative step, is possible, especially in early child- general and special education cannot fully meet the needs hood (Carta & Young, 2019; Durán & Wackerle-Hollman, of children with typically developing language skills who 2019; Greenwood et al., 2013). are learning English. More research is needed to develop effective models of instruction that are strategically Dual-Language Approach to Intervention designed to facilitate and hasten the acquisition of English (Vaughn et al., 2006). Recent recommendations for creating powerful interven- tions for Spanish-speaking English learners include incorpo- rating children’s first language (L1) to facilitate development Multitiered Systems of Support of their second language (L2; Baker, 2000; Barnett, Yarosz, One model that may have utility for promoting English Thomas, Jung, & Blanco, 2007; Castro, Garcia, & Markos, language acquisition before children experience academic 2013; Collier & Thomas, 2017; Coltrane, 2003; MacSwan & failure, is multitiered system of supports (MTSS). The Rolstad, 2005; Restrepo, Morgan, & Thompson, 2013). idea of providing special services to children who are not Those who receive sustained dual-language instruction tend performing as expected, irrespective of ability status, is to be two to three years ahead of those who receive English- not new. In 2004, the reauthorized IDEA (Individuals with only instruction in terms of academic performance Disabilities Education Act) clearly outlined the concept of (Mahoney, MacSwan, & Thompson, 2005; Rolstad, response to intervention that has been shaped into the con- Mahoney, & Glass, 2005). Collier and Thomas (2017) temporary framework of MTSS. In general, MTSS is a argued that sustained L1 and L2 instruction engages socio- framework for identifying children with emerging diffi- cultural, linguistic, cognitive, and academic processes that culties so that timely differentiated and preventative lead to high academic achievement in children’s L2. instruction can be dispensed according to individual chil- Furthermore, they posit that when schools provide strong dren’s needs. As a conceptual basis for early identification dual-language programs, children from low–socioeconomic and prevention (Fuchs & Deshler, 2007), MTSS is a para- backgrounds overcome the negative effects of poverty. Such digmatic model, not a formula, method, or procedure. sentiments are echoed in the recent National Academies of Therefore, there are many effective ways to actualize the Sciences, Engineering, and Medicine (2017) report on 2 Multitiered Dual-Language Instruction promoting educational success of children learning English, language skills when they are assessed using distal to include recommendations for incorporating children’s L1 story comprehension and general language measures? and involving families in the promotion and retention of their home language. Because the curriculum was new, the extent to which The possibility of skills learned in one language transfer- preschool teachers perceived it to be feasible in their class- ring with minimal direct teaching to another language helps rooms was not known. Feasibility of an intervention can explain the facilitative effects seen in dual-language instruc- depend on how well teachers like it, its contextual fit to the tion research (Méndez, Crais, Castro, & Kainz, 2015; Miller, school system, how well teachers understand it and how to Heilmann, Nockerts, Iglesias, Fabiano, & Francis, 2006; deliver the lessons, and the extent to which teachers can Proctor, Carlo, August, & Snow, 2005; Restrepo et al., 2013; make reasonable modifications. Therefore, we also exam- Rolstad et al., 2005). That is, when children receive strategic ined the curriculum’s feasibility in a secondary research language instruction in L1, it is possible that their knowl- question: Research Question 3: To what extent is multi- edge and skills transfer to L2, and vice versa in some cases tiered dual-language instruction feasible? (Marian & Kaushanskaya, 2007). It is theorized that cross- language interactions will occur across structures that have a Method similar, underlying cognitive schema (MacWhinney, 1999). Schemas are the mental organization of prior experiences Setting and Participants (Anderson & Pearson, 1984), and such schemas can be This study was conducted in Head Start preschool class- expressed through narration (Stein & Glenn, 1979). Narrative rooms in a southwestern state. During the spring prior to the organization is very similar across English and Spanish, commencement of the study, the first author gave a presenta- which implies that the narrative schemas for the two lan- tion regarding the study to administrators of two Head Start guages are similar. This underlying similarity suggests that grantees (one urban and one rural). Once the administrators narrative structure will have linguistic reciprocity between volunteered to allow their centers to participate, the first and L1 and L2 (and vice versa). For example, Petersen, second authors visited each center to speak directly with Thompsen, Guiberson, and Spencer (2016) found that the teachers about the study. Head Start teachers who were inter- effects of an L2 intervention targeting narrative and linguis- ested in participating signed an informed consent form and tic structures transferred to typically developing children’s completed a demographic survey. When school started at the L1. In vocabulary programs, this transfer is evidenced by the beginning of August the next year, the research team col- faster acquisition of concepts from L1 instruction to L2 than lected parental permission for children to participate. Using when children receive the instruction only in L2 (English in the parent-completed forms at their sites, the teachers identi- the case of the United States; Perozzi, 1985; Perozzi & fied children from Spanish-speaking homes. All children for Chavez Sanchez, 1992). Moreover, Miller et al. (2006) whom Spanish was one of the languages spoken at home found that sentence complexity and story structure in L1 at were invited to participate. school entry predicted academic achievement in L2 in Spanish-English dual-language learners. These studies, cor- relational and causal, indicate that one language can facili- Teachers/Classrooms. In total, 25 classrooms were tate the acquisition of another language and that the stronger included in this study. The classrooms were randomly the child’s L1, the greater their acquisition in L2. assigned to treatment and control groups at the completion of the consenting and screening process, resulting in 12 classrooms in the treatment group and 13 in the control The Current Study group. One lead teacher and one teaching assistant pro- This study represents an early efficacy pilot study to vided instruction to 18 to 20 children (3-, 4-, and 5-year- determine the promise of a multitiered dual-language cur- olds) in each classroom. Although efforts were made to riculum for a large-scale efficacy trial. As such, it was par- recruit classrooms that had at least one teacher or teaching ticularly important to understand the extent to which assistant who spoke Spanish fluently, given the available measures of narrative, vocabulary, language comprehension, workforce and frequent turnover, three of the treatment and general language abilities could be affected. Therefore, classrooms and five of the control classrooms were without we addressed the following research questions: a Spanish-speaking teacher or teaching assistant. Children in 18 (nine in treatment and nine in control) of the class- Research Question 1: To what extent does multitiered rooms attended preschool Monday through Thursday. In dual-language instruction enhance preschoolers’ oral the remaining seven classrooms, children attended 5 days a language skills when they are assessed using proximal week. All the teachers reported using Creative Curriculum narrative retell and targeted vocabulary measures? (Dodge, Colker, & Heroman, 2002) as their core curricu- Research Question 2: To what extent does multitiered lum, which was complemented by Teaching Strategies dual-language instruction enhance preschoolers’ oral Gold (Heroman, Tabors, & Teaching Strategies, Inc., 3 TABLE 1 Teacher and Classroom Characteristics Characteristic Treatment Group Control Group Number of classrooms 12 13 Years teaching, mean (range) 10 (3 months to 20) 9 (3 months to 18) Highest level of education, number of teachers High school diploma 2 2 Associate’s degree 5 6 Bachelor’s degree 4 5 Graduate degree 1 0 Race/ethnicity, number of teachers (percentage) White 6 (50) 8 (62) Hispanic/Latino 6 (50) 4 (31) American Indian 0 (0) 1 (7) Language of instruction, number of classrooms English only 8 6 Mostly English 3 5 50/50 Bilingual 1 1 Type of classroom, number of classrooms Half-day 9 8 Full day 3 5 CLASS scores, mean Emotional support 6.03 6.07 Classroom organization 5.79 5.72 Instructional support 3.65 4.46 Note. CLASS = Classroom Assessment Scoring System. 2010). The Head Start programs completed Classroom to age expectations on English measures, indicating that Assessment Scoring System (CLASS; Pianta, La Paro, & they may benefit from a Tier 2 oral language intervention. Hamre, 2008) observations of all of their teachers during To select participants, we conducted a multistep process. September or October of the school year. These data are First, we examined the children’s English NLM Listening reported, along with additional information about the retell scores, and any child who earned a retell score of 8 or teachers and classrooms, in Table 1. higher in English was excluded. A retell score of 8 presup- poses the use of key story grammar features and places a Children. During the recruitment phase, the research team preschool student above the 20th percentile based on nor- went to each center during drop-off or pickup times and met mative data from 281 preschool students across the United with the parents or guardians of the children. The research- States (Petersen & Spencer, 2016). Second, children who ers explained the study to the parents in their preferred lan- earned an English retell score of 0 to 7 but scored within guage (Spanish or English). Consent was obtained from the the normal range on the English EV subtest of the CELF-P parents of 144 children of ages 3 to 5 years who were were also excluded. In other words, scores within age exposed to Spanish at home. Once signed consents were expectations for English on either screening measure dis- obtained, the research team administered screening mea- qualified children from being participants. Therefore, chil- sures to assess the children’s language skills in English and dren who displayed low English skills and low, moderate, Spanish. Screening involved the use of the Expressive or high Spanish language skills were included as partici- Vocabulary (EV) subtest of Clinical Evaluations Language pants. The screening process resulted in 43 children Fundamentals–Preschool (CELF-P; Semel, Wiig, & Secord, recruited for 12 treatment classrooms and 40 children in 13 2004; Wiig, Secord, & Semel, 2009), a norm-referenced test control classrooms. Shortly after the pretesting, 2 children of language, and the Narrative Language Measures (NLM) from the control group moved away from the area, which Listening retell subtest of the CUBED Assessment (Petersen resulted in 38 children in the control group. & Spencer, 2016). In five of the 12 treatment classrooms, more than three The goal for participant recruitment was to identify children qualified to be research participants (i.e., they could Spanish-speaking children who did not perform according potentially benefit from Tier 2 intervention). However, the 4 TABLE 2 Child Characteristics Child Characteristic Treatment Group (n = 43) Control Group (n = 38) Gender, number (percentage) Male 16 (37) 13 (34) Female 25 (58) 21 (55) Age in months, mean (range) 50 (39–59) 49 (37–59) Race/ethnicity, number (percentage) White 1 (2) 0 (0) Hispanic/Latino/a 39 (91) 38 (87) Primary language, number (percentage) English 0 (0) 0 (0) Spanish 31 (72) 27 (71) Both English and Spanish 7 (16) 6 (16) Number of children with IEP 0 0 Note. IEP = Individualized Education Program. The percentages do not add up to 100 within each variable and group due to incomplete demographic survey data. teachers were not able to feasibly provide the Tier 2 inter- delivery of the intervention, and collect data, they were vention to more than one group every day. Therefore, only not blind to assignment. three children in each class were able to receive the intended multitiered instruction, and the rest of the children (n = 10) Video Manual and Training received only large-group instruction in English. The teach- We created a video manual to explain the rationale and ers determined which children would receive the small- teaching procedures needed to deliver the multitiered lan- group instruction, and the researchers did not guide them in guage curriculum. The video manual consisted of 13 short making those choices. Although 10 of the 43 children who (5–15 minutes) modules that covered the active ingredients were identified as needing Tier 2 small-group support only of the program, its materials, and guidelines for delivering received large-group instruction, these 10 children were lessons. During a full-day group training prior to the begin- included as research participants because they received part ning of the school year, the modules were played one by one of the intended intervention and the control group received for the teachers, teaching assistants, and directors from the none of it. treatment group. Each teacher and teaching assistant prac- The parents completed a brief survey to report demo- ticed teaching a lesson to the other attendees. Question-and- graphic information about their children. Child character- answer sessions were interspersed throughout the day to istics are shown in Table 2. The parents also reported their address any questions or concerns. In addition to the train- highest level of education and annual family income. ing, the teachers were given their own flash drives with the Only 7% of the treatment group’s parents had attended video manual so that they could review any module at any college, with two of them having earned a college degree, time throughout the year. Once they began using the curricu- and 8% of the control group’s parents had attended col- lum, the RAs spent 1 to 2 weeks coaching the Head Start lege, with none having earned a college degree. Only 26% teachers and teaching assistants until they felt comfortable of the treatment group parents and 21% of the control delivering the lessons independently. group parents reported the family’s annual income to be more than $22,000. Research Design and General Procedures Because the 81 child participants were nested within Research Assistants classrooms, a cluster-randomized group study design was Research assistants (RAs) were responsible for all employed to investigate the effect of multitiered dual-lan- screening, data collection, and supporting teachers as they guage instruction on the children’s language skills. After the implemented the intervention. The RAs visited each class- children were screened and included as research partici- room once or twice a week to check in with the teachers pants, the RAs completed the pretesting (September). The and teaching assistants and to conduct fidelity observa- intervention consisted of three units of instruction (Units tions. The first author completed rigorous training with the A–C), with each unit lasting 8 to 10 weeks. Throughout the RAs prior to their participation in the study. Because they school year, the children in both control and intervention were all needed to observe fidelity, support teachers’ classrooms were administered several proximal and distal 5 Spencer et al. measures to examine the extent to which the multitiered cur- target vocabulary words, and complex sentences (e.g., with riculum affected important child outcomes. Dependent vari- coordinating and subordinating conjunctions). ables included narrative retells, receptive vocabulary, The Head Start teachers and teaching assistants worked listening comprehension, and general oral language abilities together to determine how they would deliver the compo- (e.g., understanding and use of grammar). Posttesting was nents of Puente de Cuentos. All the children in the class- completed at the end of the study (April/May); however, the rooms participated in the large-group activities, but the proximal measures (e.g., receptive vocabulary and narrative research participants received small-group lessons in addi- retells) were repeated four times across the year to ensure tion to the large-group lessons as their Tier 2 intervention. A that the participants’ skills were assessed both before and typical implementation consisted of two English large- after each of the three units of instruction. The Head Start group lessons, two Spanish small-group lessons, and two teachers and teaching assistants completed all of the inter- English small-group lessons each week. Spanish small- vention components by integrating them within the routine group lessons preceded English small-group lessons to of their classroom, although each teacher decided how and facilitate cross-language transfer. In the three treatment when to implement each component. classrooms that did not have a Spanish-speaking teacher or All the research activities, including assessments and teaching assistant, the children only received English large- intervention, took place in Head Start classrooms. In an and small-group lessons, each twice a week. In addition to effort to minimize noise and distractions, the RAs conducted the explicit, teacher-led instruction, the teachers embedded the assessments with individual children during scheduled several child-directed extension activities throughout their activities that were moderately quiet (e.g., drop-off and daily routine. pickup times, as the children finished snack time, and when Parents of the children who qualified for the Tier 2 Puente the class was at circle time). Although there were a large de Cuentos intervention in the classroom received a set of number of assessments that were administered to the chil- family engagement activities in Spanish. Each activity fea- dren individually and repeatedly, all of the assessments were tured one of the 72 stories from the Puente de Cuentos cur- extremely brief (most were under 5 minutes), and only one riculum and listed questions and suggestions on how to help assessment was completed at a time. their children to retell the story and to use the target words in Spanish. The control group was considered a “business as usual” Multitiered Dual-Language Narrative Curriculum condition. The center directors reported that teachers used The multitiered dual-language narrative curriculum is small-group instruction to differentiate individual students called Puente de Cuentos (“Bridge Made of Stories”). It fea- but most consistently delivered instruction in large groups. tures 36 English stories (three units of 12 stories each), with Because the teachers did not have access to a Spanish cur- 36 corresponding Spanish stories. Each story was written to riculum or a systematic Spanish program, instruction was include two target vocabulary words (e.g., rough/áspero). As completed in English with occasional directions or explana- the units progressed, coordinating and subordinating con- tions in Spanish (if the teacher spoke Spanish). junctions were folded into the stories and lessons. To accom- pany each story, a set of five illustrations was created. These Proximal Measures and Data Collection illustrations were simple line drawings with minimal color and few details. Photos of the target vocabulary words were Narrative Language Measures Listening. The NLM Listen- included in the materials so teachers could show how the ing is a subtest of the CUBED Assessment (Petersen & words could be used in contexts other than the stories. Spencer, 2016). To collect retell language samples in Eng- Additional information about the stories and lessons can be lish and Spanish using NLM Listening, the RAs read a brief found in the online supplemental material. story to a child, and the child retold the story. The RAs Stories served as the basis for language instruction in scored the children’s retells in real time, giving points for small-group and large-group arrangements. Lessons were each story grammar element and indicators of complex lan- scripted for teachers and adhered to a consistent format guage use (e.g., subordinating conjunctions such as because, across the three units. During each lesson, the teacher or when, after). At each assessment time point, the children teaching assistant read the featured story and then guided the were administered three of the NLM Listening parallel children through a series of activities designed to help them forms in a single session lasting 3 to 4 minutes. However, learn the meaning of target words and to retell the stories. only the retell with the highest score was used in the analysis Some activities required children to respond together as a and to identify the participants. The NLM Listening stories group to increase active responding, whereas other activities were unfamiliar to the children and children never heard the required children to respond individually. When individual same story twice. Because NLM Listening stories are simi- children retold the featured stories, they were prompted (and lar to those directly taught in Puente de Cuentos, this was supported) to use all of the story grammar elements, the considered a proximal outcome measure for this study. 6 Multitiered Dual-Language Instruction Receptive Picture Vocabulary Assessment. The researcher- completed a fidelity checklist that documented adherence (12 designed receptive picture vocabulary assessment measured items), responsiveness (3 items), and quality (9 items) of the the children’s mastery of the Spanish and English words tar- intervention (Dane & Schneider, 1998). The RAs recorded geted in the Puente de Cuentos curriculum. The children the fidelity of 21% of large-group lessons, 21% of Spanish were shown four different black-and-white line drawings small-group lessons, and 17% of English small-group les- and asked to point to the target word. sons. To yield a percent fidelity, the number of items com- pleted as intended or with high quality was divided by the total number of items on the checklist and multiplied by 100. Distal Measures and Data Collection Assessment of Story Comprehension. The Assessment of Intervention Logs. To capture information about the extent to Story Comprehension (ASC; Spencer & Goldstein, 2019) is which the children received the intended dose, the researchers a narrative-based, criterion-referenced assessment for pre- provided intervention and attendance logs for each classroom. schoolers. It is only in English. During its administration, the Dose for each type of teacher-directed lesson (i.e., large-group RAs read a short story to a child, then asked a series of factual English, small-group Spanish, or small-group English) was and inferential questions. Examiners wrote the children’s recorded, as well as how many extension activities were com- answers word for word on record forms and rated each pleted and for which words and concepts. answer for correctness and clarity on a 0 to 2 or 0 to 3 scale, yielding a total of 17 possible points. Six parallel forms were Implementation Survey. At the end of the school year, the administered, three at pre-intervention (September) and three Head Start teachers completed a short survey. This con- at postintervention (April/May). The highest score was used sisted of nine researcher-generated questions that probed for analysis. Because the ASC stories are significantly differ- the teachers’ perceptions about the modifications com- ent from the Puente de Cuentos stories and children answer pleted and needed, planned sustainment, and contextual fit factual and inferential questions instead of retelling stories, it of the Puente de Cuentos curriculum in Head Start settings. is considered a distal measure of language comprehension. Responses were rated using a Likert-type scale of 1 to 5. Clinical Evaluation of Language Fundamentals: Preschool. Results The CELF-P in English and Spanish (Semel et al., 2004; Wiig et al., 2009) includes three language subtests that measure gen- Descriptive statistics for the focal measures are shown in eral oral language proficiency. The Sentence Structure (SS) Table 3. Less than 1% of the scores were missing overall subtest requires children to point to pictures corresponding to a (18/2754 = 0.0065, or 0.65%), and all available data were spoken sentence. The Word Structure (WS) subtest requires an used in the multilevel model results that follow. Detailed expressive response that examines children’s grammatical abil- descriptions of the analyses are available in the online sup- ities. In the EV subtest, children label pictures of objects and plemental material. actions. The EV subtests of the English and Spanish versions were used for screening, but participants who qualified for Tier Proximal Child Outcomes 2 intervention also completed the SS and WS subtests in Eng- lish and Spanish as part of pretesting. Raw scores were calcu- We evaluated baseline equivalence across the treatment lated and used in the analysis. and control groups on pretest measures. As shown in Table 4, tests of pretest differences on these measures were nonsignifi- cant (gs = −.10 to .46), except for Spanish Vocabulary B, for Feasibility Measures and Data Collection which the treatment group had a significantly higher pretest Usage Rating Profile–Intervention. At the end of the inter- mean (g = .53). We proceeded to test the differences in post- vention phase, the classroom teachers and teaching assis- test scores adjusted for the respective pretest to control for any tants completed the Usage Rating Profile–Intervention baseline differences between groups. (URP-I; Chafouleas, Briesch, Riley-Tillman, & McCoach, 2009). The URP-I consists of 35 questions, each with NLM Listening English and Spanish. On the English and 6-point Likert scale responses regarding four intervention Spanish NLM posttests, tests of the estimated difference dimensions: acceptability, understanding, feasibility, and between groups on the adjusted means in the random-inter- system support. Because each dimension has a different cept analyses of covariance showed statistically significant number of items, we converted scores to percent so they differences in favor of the treatment group (see Table 5). The can be interpreted. 95% confidence intervals, although somewhat wide given the pilot study sample size, support the estimated positive Fidelity Checklists. The RAs monitored the fidelity of the effects for the treatment group. On the English NLM, the Puente de Cuentos lessons. During each observation, an RA effect size was large (g = .85), and the improvement index 7 TABLE 3 Descriptive Statistics for Pretest, Posttest, and Adjusted Posttest Scores by Treatment Group Treatment Group (N = 43) Control Group (N = 38) T C Measure Pretest, M (SD) Posttest, M (SD) Posttest, M a Pretest, M (SD) Posttest, M (SD) Posttest, M a adj adj E NLM 1.00 (1.93) 6.86 (5.41) 6.91 1.24 (2.10) 2.92 (3.77) 2.86 S NLM 5.51 (6.15) 10.62 (6.35) 10.09 3.55 (4.86) 6.50 (6.06) 7.08 E Vocab A 9.84 (3.75) 12.72 (4.47) 12.65 9.45 (4.18) 10.54 (3.66) 10.66 E Vocab B 8.21 (2.48) 11.81 (3.73) 11.73 8.32 (3.10) 9.47 (3.26) 9.48 E Vocab C 10.07 (3.59) 12.86 (4.30) 12.53 9.18 (3.97) 10.18 (4.31) 10.51 S Vocab A 13.09 (4.02) 14.86 (4.02) 14.38 11.29 (4.31) 11.16 (4.09) 11.80 S Vocab B 11.86 (3.81) 13.95 (3.95) 13.24 10.08 (2.74) 11.39 (3.04) 12.11 S Vocab C 9.58 (4.34) 11.93 (3.84) 11.55 8.34 (2.39) 9.32 (3.66) 9.75 ASC 1.14 (1.66) 4.23 (3.39) 4.24 1.19 (1.60) 2.71 (2.95) 2.66 E SS CELF 5.74 (4.69) 9.14 (4.30) 9.23 6.19 (3.21) 7.24 (3.67) 7.01 E WS CELF 2.60 (3.40) 6.30 (4.76) 6.18 2.45 (3.03) 4.24 (3.76) 4.40 E EV CELF 3.09 (3.66) 7.49 (5.16) 7.21 2.79 (4.34) 7.29 (5.85) 7.47 S SS CELF 9.65 (4.27) 14.19 (4.84) 13.76 8.45 (3.89) 10.71 (4.23) 10.84 S WS CELF 10.28 (5.51) 14.91 (5.84) 14.39 8.37 (5.40) 11.26 (6.32) 11.89 S EV CELF 17.98 (9.56) 18.88 (10.03) 17.22 13.11 (8.34) 14.45 (9.45) 16.66 Note. E = English; S = Spanish; NLM = Narrative Language Measure; Vocab = Puente de Cuentos Picture Vocabulary Assessment; ASC = Assess- ment of Story Comprehension; CELF = Clinical Evaluation of Language Fundamentals—Preschool; SS = Sentence Structure; WS = Word Structure; EV = Expressive Vocabulary; M = mean; SD = standard deviation. aAdjusted posttest means have been adjusted for group differences on the pretest and were used in conducting the analyses of covariance. TABLE 4 Unconditional Pretest ICCs and Tests of Baseline Equivalence (Random-Intercept ANOVAsa), with Hedges’ g Effect Sizes With Small- Sample Adjustment Measure Pretest, ICC Estimated M − M , γ (95% CI) p (for γ ) Hedges g (Effect Size) T C 01 01 E NLM .24 −0.19 (−1.39, 1.00) .74 −.10 S NLM .01 1.96 (−0.52, 4.43) .12 .35 E Vocab A .09 0.31 (−1.80, 2.41) .76 .08 E Vocab B .00 −0.12 (−1.36, 1.13) .85 −.04 E Vocab C .07 0.86 (−1.04, 2.75) .36 .23 S Vocab A .13 1.94 (−0.27, 4.15) .08 .46 S Vocab B .00 1.78 (0.30, 3.27) .02 .53 S Vocab C .07 1.26 (−0.52, 3.04) .16 .35 ASC .10 −0.09 (−0.97, 0.79) .83 −.05 E SS CELF .11 −0.51 (−2.75, 1.74) .64 −.12 E WS CELF .00 0.16 (−1.27, 1.59) .83 .05 E EV CELF .07 0.33 (−1.73, 2.38) .74 .08 S SS CELF .00 1.20 (−0.61, 3.02) .19 .29 S WS CELF .01 2.06 (−0.65, 4.77) .13 .37 S EV CELF .00 4.87 (0.88, 8.86) .02 .54 Note. N = 43, NC = 38. ICC = intraclass correlation coefficient; T = treatment group; C = control group; E = English; S = Spanish; NLM = Narrative T Language Measure; Vocab = Puente de Cuentos Picture Vocabulary Assessment; ASC = Assessment of Story Comprehension; CELF = Clinical Evaluation of Language Fundamentals–Preschool; SS = Sentence Structure; WS = Word Structure; EV = Expressive Vocabulary; ANOVA = analysis of variance; CI = confidence interval. aSolutions for S NLM, E Vocab B, S Vocab B, E WS CELF, S SS CELF, and S EV CELF are equivalent to general linear model–based ANOVAs, as the between-class random-intercept variance component estimate was 0 (or near 0). 8 TABLE 5 Unconditional Posttest ICCs and Tests of Postintervention Differences in Adjusted Means (Random-Intercept ANCOVAsa), with Hedges’ g Effect Sizes and Improvement Indexes Measure Posttest, ICC bEstimated M − M , γ (95% CI) p (for γ ) Hedges g (effect size) Improvement index adjT adjC 01 01 E NLM .21 4.05 (2.06, 6.05) <.01 .85 30% S NLM .02 3.01 (0.53, 5.50) .02 .48 18% E Vocab A .13 1.99 (0.32, 3.67) .02 .48 18% E Vocab B .26 2.25 (0.42, 4.09) .02 .63 24% E Vocab C .25 2.02 (0.39, 3.66) .02 .46 18% S Vocab A .08 2.58 (0.94, 4.22) <.01 .63 24% S Vocab B .22 1.12 (−0.60, 2.85) .19 .31 12% S Vocab C .26 1.80 (0.39, 3.22) .02 .48 18% ASC .10 1.59 (0.18, 2.99) .03 .49 19% E SS CELF .00 2.22 (0.60, 3.84) .01 .55 21% E WS CELF .09 1.78 (−0.12, 3.69) .07 .41 16% E EV CELF .09 −0.26 (−2.38, 1.86) .80 −.05 −2% S SS CELF .25 2.91 (0.23, 5.60) .03 .63 24% S WS CELF .05 2.50 (−0.27, 5.27) .07 .41 16% S EV CELF .00 0.56 (−3.00, 4.11) .75 .06 2% Note. N = 43, N = 38. ICC = intraclass correlation coefficient; T = treatment group; C = control group; E = English; S = Spanish; NLM = Narrative T C Language Measure; Vocab = Puente de Cuentos Picture Vocabulary Assessment; ASC = Assessment of Story Comprehension; CELF = Clinical Evaluation of Language Fundamentals–Preschool; SS = Sentence Structure; WS = Word Structure; EV = Expressive Vocabulary; ANCOVA = analysis of covariance; CI = confidence interval. aSolutions for E NLM, S NLM, and E SS CELF are equivalent to general linear model–based analyses of variance, as the between-class random-intercept variance component estimate was 0 (or near 0). bAdjusted posttest means were adjusted for group differences on the pretest and used in conducting the ANCOVAs. was 30%, indicating that an average student in the control the treatment group had a significantly higher pretest mean group would be expected to score about 30% higher if (g = .54). We evaluated differences in posttest scores receiving the intervention. The effect size for the Spanish adjusted for the respective pretest to control for any baseline NLM was moderately strong (g = .48), with an improve- differences between groups. ment index of 18%. Assessment of Story Comprehension. The random-intercept Receptive Picture Vocabulary Assessment. With the excep- analysis of covariance on the ASC adjusted posttest means tion of the posttest for Spanish Unit B, tests of the estimated was statistically significant, with a moderate effect size (g = difference between groups on the adjusted posttest means .49). The improvement index estimated that an average stu- for English and Spanish vocabulary were statistically sig- dent in the control group would be expected to score 19% nificant, favoring the treatment group (see Table 5). Effect higher on the ASC if receiving the intervention, which would sizes for these five measures (English Vocabulary A, B, and be a meaningful gain. C; Spanish Vocabulary A and C) were moderate (gs = .46 to .63). The improvement indices suggested that an average Clinical Evaluation of Language Fundamentals—Pre- student in the control group would be expected to score from school. Results for adjusted posttest differences between 18% to 24% higher on the vocabulary assessments if receiv- the treatment and control groups differed across the CELF-P ing the intervention. Although the vocabulary posttest for SS, WS, and EV subtests, but were very consistent for sub- Spanish B was not statistically significant, the effect size tests across English and Spanish. The treatment group was not trivial (g = .31), and the improvement index was clearly outperformed the control group on SS, evidenced by 12% in favor of the treatment group. statistically significant differences, moderate effect sizes (gs = .55 for English and .63 for Spanish), and improvement indices. An average student in the control group would be Distal Child Outcomes expected to score 21% higher on SS for English and 24% As shown in Table 4, tests of pretest differences on distal higher for Spanish. measures were not significant with small to moderate effect Differences in adjusted posttest means were not statisti- sizes (gs = −.12 to .37), except for Spanish EV, for which cally significant for WS in either language, but effect sizes 9 implemented by the middle of May. The small-group inter- vention portion of the log revealed that all the research par- ticipants were present for at least 85% of the Tier 2 lessons intended for them. Moreover, 90% or more of the target words and concepts were addressed through extension activ- ities in all the treatment classrooms. Implementation Survey. Mean ratings of all the teachers and teaching assistants who completed the implementation sur- vey are displayed in Table 6. Overall, they reported that they made few modifications during the study, but some had plans to make more. Most of the teachers planned to con- tinue using Puente de Cuentos after the study. The mean rat- ings indicate that there is a reasonable contextual fit between FIGURE 1 Teachers’ mean ratings on the Usage Rating Profile–Intervention. the intervention and their values, students, and setting. Discussion approached moderate (gs = .41), with improvement indices of 16% in support of intervention effects. The final two dis- The importance of building oral language skills is clear as tal measures—English and Spanish EV—did not evidence there is a strong link between oral language and reading any appreciable differences between treatment and control comprehension (Cain et al., 2004; Chaney, 1998; Clarke group adjusted means. et al., 2010; Dickinson & Tabors, 2001; Larney, 2002). Vocabulary and narrative skills are particularly important areas to develop early so that children can benefit more from Feasibility subsequent instruction and comprehension of what is read to Usage Rating Profile–Intervention. Mean percentage for them and what they read (Cain & Oakhill, 2011; Elleman each dimension of the URP-I is displayed in Figure 1. et al., 2009; Mehta et al., 2005; Verhoeven & van Leeuwe, Higher scores in acceptability, understanding, and feasibil- 2008). If oral language instructional efforts can incorporate ity suggest that the intervention was perceived as useful children’s L1 and produce meaningful improvements in and doable. The teachers and teaching assistants reported English, there is an added benefit of helping cultivate a bilin- Puente de Cuentos to be more acceptable than feasible, gual and biliterate society (Collier & Thomas, 2017). The although the scores for both characteristics were moder- purpose of this early-stage efficacy study was to examine the ately high. The teachers also reported having a good under- extent to which multitiered dual-language instruction standing of the curriculum. For systems support, the improved children’s Spanish and English language skills on teachers reported lower scores than for the other dimen- proximal and distal measures of vocabulary, narrative retells, sions, but because of the nature of the scale, higher scores language comprehension, and general language abilities. were not necessarily desired. Proximal Measures of Vocabulary and Narrative Retell Fidelity. After the Head Start teachers and teaching assis- tants felt comfortable delivering the lessons (1–2 weeks), the Consistent with prior English, oral narrative–based lan- RAs began assessing their intervention fidelity using the guage intervention studies that have focused on proximal fidelity checklists. The teachers and teaching assistants dem- outcomes (e.g., Spencer, Petersen, & Adams, 2015; Spencer, onstrated consistently high fidelity to the Puente de Cuentos Petersen, Slocum, & Allen, 2015; Spencer, Weddle, Petersen, procedures. For the small-group lessons in Spanish, the & Adams, 2017), we found statistically significant effects mean fidelity scores were 97%, 96%, and 98% for Units A, for narrative retells in English. Narrative retelling was the B, and C, respectively. For the small-group English lessons, most salient instructional activity in the Puente de Cuentos they were 97%, 96%, and 97%, respectively. For the large- instruction, with all the large-group and half of the small- group lessons, fidelity was slightly lower, with mean fidelity group lessons based on English stories. The teachers sup- scores of 91%, 97%, and 94% for the respective units. ported the children’s practice of each model story, English vocabulary, and English language complexity through retell- Intervention Logs. Based on a review of the intervention ing activities in every lesson. Only half of the small-group logs the teachers completed, very few teacher-directed les- lessons featured Spanish story retelling, which may account sons were omitted, with the exception of small-group les- for the differences in the effect sizes for English (g = .85) sons in Spanish in the three treatment classrooms without a and Spanish (g = .48) retell outcomes. Although improve- Spanish-speaking teacher. All planned lessons had been ment in the proximal narrative retell outcome was expected, 10