Sustained positive effects mdrc on graduation rates Building Knowledge To improve Social policy produced by new york city’s Small public High Schools of choice By Howard S. Bloom and Rebecca Unterman D uring the past decade, New improving academic progress and York City undertook a district- substantially improving graduation wide high school reform that is perhaps prospects, particularly for disadvantaged unprecedented in its scope, scale, and pace. students. This report included data for Between fall 2002 and fall 2008, the school over 21,000 students from four cohorts district closed 23 large failing high schools who entered ninth grade between fall F (with graduation rates below 45 percent),1 2005 and fall 2008.4 Findings in the report opened 216 new small high schools (with were based on a series of randomized different missions, structures, and student lotteries that were used when SSCs had E selection criteria), and implemented a more applicants than seats, in order to centralized high school admissions process determine who got offered an opportunity that assigns over 90 percent of the roughly to enroll in an SSC and who did not. SSC I 80,000 incoming ninth-graders each year lotteries are a little-known byproduct based on their school preferences. of the current district-wide high school R application processing system (HSAPS).5 At the heart of this reform are 123 small, These lotteries make it possible to academically nonselective, public high rigorously estimate the effects of enrolling B schools. Each with approximately 100 in an SSC versus enrolling in some other students per grade in grades 9 through 12, New York City public high school, based Y these schools were created to serve some of on a comparison of subsequent outcomes the district’s most disadvantaged students for lottery winners and lottery losers and C and are located mainly in neighborhoods a procedure that accounts for who among where large failing high schools had been them enroll in an SSC.6 closed.2 Hence, they provide a realistic I choice for students with widely varying In a large sample, like that used for the l academic backgrounds. MDRC researchers MDRC study, lottery winners and lottery call them “small schools of choice” (SSCs) losers are the same, on average, in all because of their small size and the fact that ways before they enter high school. o they do not screen students based on their Consequently, it is valid to attribute any P academic backgrounds.3 differences in their future academic outcomes to their access to an SSC. In June 2010, MDRC, a nonprofit, Because students who lose an SSC nonpartisan research organization, lottery attend over 200 high schools that released a report on the effectiveness vary widely in their size, age, structure, J a n u a r y of 105 of the 123 new SSCs, which academic programs, and effectiveness, 2 0 1 2 ddeemmoonnssttrraatteedd tthhaatt tthheeyy aarree mmaarrkkeeddllyy the MDRC report judged SSCs against the MDrC PoliCy Brief overall effectiveness of a diverse group of resources during start-up.8 By integrating a other high schools.7 The results released in demanding and comprehensive academic 2010 indicated that, on average, the 105 SSCs curriculum, personal attention to student studied increased student progress toward academic progress, and real-world experiences graduation during their first three years of with community partners, the new small high school and increased students’ four-year schools intended to prepare students for both graduation rates. college and career. Given data that were available at the time, In other cities, small high schools are often the first MDRC report could only follow fashioned by reconfiguring large existing the first student cohort (that entered in fall schools into smaller units in the same 2005) through four years of high school to buildings with the same teachers and students; examine its graduation rates. This policy in contrast, the typical SSC was created largely brief extends the analysis by a year, which “from scratch” with a principal, teachers, and adds information on high school graduation students who were new to the school. At the rates for the 2006 cohort and provides a fifth same time, many SSCs are located with other year of follow-up for the 2005 cohort. This SSCs in buildings that previously housed a information makes it possible to address the large public high school that was closed. following three questions: • Was the positive average effect of SSCs on SSCs are not just small in size. They are four-year graduation rates for the study’s purposefully organized around smaller first student cohort sustained through the educational units that are designed to give second cohort? students a better chance of being known by • Was this positive average effect sustained their teachers and other adults in the building. across subgroups of students with different For example, many SSCs have created special prior academic proficiency, family income, student advisory structures that provide race/ethnicity, and gender? common planning time for teachers to discuss • Was the average four-year graduation effect their students’ progress and problems. sustained after five years? SSCs received start-up funding from The answer to all three of these questions is yes. philanthropic organizations plus technical assistance and policy support from the New w Ha T ar e S Sc s ? York City Department of Education and SSCs were developed through a competitive intermediary organizations that were often proposal process that was designed to experienced in managing the launching of ensure that school founders met specified new schools — which helped to facilitate conditions and to stimulate innovative ideas school leadership development, staff hiring, from a range of stakeholders and institutions. and program start-up. Most SSCs began with SSCs emphasize academic rigor and strong only a ninth-grade class, adding a subsequent and sustained personal relationships among grade each year. Finally, SSCs received special students and faculty. In addition, most were allowances with respect to serving English 2 founded with community partners who offer language learners and special education students relevant learning opportunities inside students during their first two years of start-up.9 and outside the classroom and provide school The creation of SSCs was supported by a faculty with additional staffing support and consortium of funders, led by the Bill & January 2012 Melinda Gates Foundation, the Carnegie academic achievement.13 Prior research on Corporation of New York, and the open small schools has produced mixed results Society Institute. SSCs were implemented about their effectiveness. Some studies have in collaboration with the United Federation found that they improve student outcomes, of Teachers and the Council of School such as increasing levels of academic Supervisors and Administrators.10 Unlike achievement and lowering rates of school charter schools, which typically operate dropout.14 In addition, some studies suggest independently of the school district in which that these effects are most pronounced they are located, SSCs are directly responsible for disadvantaged students.15 on the other to the New York City Department of Education. hand, a major study of a large number of small schools initiatives suggests that they w Hy STu d y S Sc s ? do not improve student academic outcomes Almost all major urban school districts in the appreciably.16 In all of these cases however, the U.S. and all but four states have attempted nonexperimental nature of the research limits to create new small schools or attempted its ability to support causal conclusions. to transform large schools into campuses with “small learning communities.”11 This MDRC’s findings about SSCs are relevant movement had its beginnings in the 1960s, to current federal policy on high school when urban educators and community reform, particularly the U. S. Department of organizations began to implement smaller Education’s School Improvement Grants school structures as an alternative to the large (SIGs) for failing schools. Reforms funded by high schools that dominated the landscape, SIGs include school transformation, school many of which were failing. By the late 1990s, restart, school closing, and school turnaround. small schools had emerged as a national SSCs straddle several of these categories since reform strategy championed by such groups they are typically replacements for schools as the Coalition of Essential Schools and the that have closed and they operate as regular National Association of Secondary School public schools. However, it is important to Principals. This strategy proliferated through recognize that SSCs represent far more than district- and foundation-led initiatives in major just changes in school size and structure. cities like Chicago, Philadelphia, Boston, They also represent innovative ways to use oakland, and New York. In 2000, the Bill & these structural changes to leverage human, Melinda Gates Foundation launched a national financial, and curricular resources. campaign to improve failing urban high schools, with small schools as a key strategy.12 The coupling of small schools with a system of choice for all entering ninth- The common theory of change underlying graders also has relevance for the national both small schools and small learning debate about charter schools. on the one communities specifies that smaller hand, SSCs share many similarities with educational units promote stronger charters — they are small and personalized; relationships among students, among most began with the help of intermediary adults, and between students and adults. sponsors that resemble charter management These relationships, in turn, are expected organizations; the students and teachers 3 to increase student engagement and help are there by choice; and the curricula are teachers respond to students’ academic and demanding. on the other hand, SSCs are social needs, thereby increasing students’ overseen by the Department of Education MDrC PoliCy Brief Table 1. estimated average effects of SSCs on Four-Year Graduation Rates: Student Cohorts 1 and 2 TargeT conTrol p-value For eSTimaTed ouTcome (%) SSc group eSTimaTed eFFecT enrolleeS counTerparTS eFFecT graduaTion graduaTed From HigH ScHool 67.9 59.3 8.6 ** 0.000 local diploma granted 19.9 19.6 0.3 0.406 regents diploma granted 41.5 34.9 6.5 * 0.041 advanced regents diploma granted 6.2 4.7 1.6 0.154 college readineSS passed math regents at 75 or higher 23.3 22.5 0.8 0.682 passed english regents at 75 or higher 37.3 29.7 7.6 ** 0.002 NoTES: There are 13,064 student observations for cohorts one and two combined. There are no statistically significant differences between estimated effects for the two cohorts. Because of rounding error and the complex calculations involved, results by diploma type do not sum exactly to those for overall graduation rates. with unionized teachers and principals; The report found similar positive effects on the schools are nonselective and open to progress toward graduation in ninth grade students regardless of academic proficiency; for subgroups of students who varied in and the school choice process was designed terms of their reading and math proficiency to engage all entering ninth-graders, not just before entering high school, their eligibility the most motivated parents and children that for free or reduced-price lunches, and their might seek out charter schools. race/ethnicity and gender. H o w mu cH do S Sc s At two and three years after students entered H e l p STu d e nT S? high school, the study found that SSCs previous Study Findings: Because students increased their progress toward graduation by became part of the study sample in four roughly the same amount as they did during annual cohorts, MDRC’s first report on the the first year. And at four years after students effects of SSCs was based on four years entered high school, the study found that of follow-up for the first cohort of entering SSCs increased high school graduation rates ninth-graders, three years of follow-up for the by 6.8 percentage points (to 68.7 percent for second, two years of follow-up for the third, target SSC enrollees from 61.9 percent for and one year of follow-up for the fourth cohort. their control group counterparts). Based on data for all four cohorts, the report new Findings on graduation and college found that SSCs increased the percentage readiness for Two Student cohorts: Findings of students who were on track17 toward in Table 1 indicate that SSC effects on 4 graduation in ninth grade by 10.0 percentage four-year high school graduation rates are points (to 58.5 percent for “target SSC sustained through a second cohort of entering enrollees” from 48.5 percent for their control ninth-graders. Enrolling in an SSC instead group counterparts).18 of another type of New York City public high January 2012 school increased average four-year graduation subgroups examined that had samples that rates for the first two cohorts combined by 8.6 were large enough to support findings.21 percentage points (to 67.9 percent for target SSC enrollees from 59.3 percent for their Consider the results for subgroups defined control group counterparts).19 Almost all of by students’ prior academic proficiency, as this graduation effect reflects an increase in measured by their eighth-grade state test receipt of New York State Regents diplomas. scores in reading and math. New York State For this type of diploma, students must pass reports this proficiency in four levels. levels 1 a minimum of five Regents examinations and 2 (did not meet or partially met standards) (English language Arts, Mathematics, represent student performance that is below Science, Global History, and American grade level. levels 3 and 4 (fully met standards History) with a score of 65 points or above or met standards with distinction) represent and must pass all courses that are required performance that is at or above grade level. by the state. None of the SSC graduation Since very few sample members score in the effect appears to reflect an increase in receipt top category, no findings are reported for it. of local diplomas, which have less stringent The findings indicate that SSCs increased four- standards for scores on Regents examinations year graduation rates appreciably for students and are being phased out by New York State in levels 1, 2, and 3 of reading and math prior for the graduating class of 2012. likewise, proficiency. little if any of the SSC graduation effect reflects an increase in receipt of Advanced Although specific estimates vary somewhat Regents diplomas, which have the most (from 7.7 to 11.3 percentage points for reading stringent standards and are received by very and from 8.6 to 11.4 percentage points for few students who apply to SSCs. math), all of the effects are appreciable and statistically significant, indicating that every Table 1 also indicates that enrolling in an group benefited substantially from attending SSC increased students’ college readiness an SSC. For example the 11.4 percentage point in English but not in math, as measured gain in high school graduation rates produced by the percentage who scored 75 points or by SSCs for students with a level 1 eighth-grade higher on the corresponding New York State math proficiency score implies a 33 percent Regents examinations.20 This threshold increase (11.4/34.5) in graduation rates for this is used by the City University of New York subgroup of students with especially weak math to exempt students from taking remedial preparation. However, there is no clear pattern courses in these subjects. to the variation in results across proficiency subgroups; so while the impacts are statistically new Subgroup Findings on graduation significant, the reported differences between for Two Student cohorts: Adding a second subgroups are not statistically significant. cohort of students to the analysis of SSC Therefore, one should not conclude, for effects on four-year graduation rates instance, that SSCs worked differently for level 1 increased the sample size by enough to students than for level 2 students, even though make it possible to examine how these the point estimates differ. effects vary across subgroups of students 5 with diverse backgrounds. Findings in Table These subgroup findings by academic 2 indicate that SSCs increased four-year high proficiency are especially striking given school graduation rates appreciably for all the dramatic differences that exist in the MDrC PoliCy Brief Table 2. estimated effects of SSCs on Four-Year Graduation Rates for Student Subgroups: Student Cohorts 1 and 2 TargeT conTrol p-value For eSTimaTed STudenT cHaracTeriSTic SSc group eSTimaTed eFFecT enrolleeS counTerparTS eFFecT 8TH-grade reading proFiciency not meeting State learning Standards (level 1) 37.3 29.6 7.7 * 0.044 partially meeting State learning Standards (level 2) 66.7 55.5 11.2 ** <.0001 meeting State learning Standards (level 3) 81.1 69.8 11.3 ** 0.003 exceeding State learning Standards (level 4) -- -- -- 8TH-grade maTH proFiciency not meeting State learning Standards (level 1) 45.9 34.5 11.4 ** <.0001 partially meeting State learning Standards (level 2) 65.1 53.8 11.3 ** <.0001 meeting State learning Standards (level 3) 85.9 77.3 8.6 ** 0.001 exceeding State learning Standards (level 4) -- -- -- low-income STaTuS eligible for free/reduced-price lunch 66.9 55.7 11.2 ** 0.000 not eligible for free/reduced-price lunch 74.3 67.4 6.9 ** 0.001 race/eTHniciTy, By gender Black male 65.2 55.3 9.9 * 0.021 Black female 69.1 60.8 8.3 * 0.018 Hispanic male 61.3 53.7 7.7 0.065 Hispanic female 68.7 60.0 8.7 ** 0.032 other male 78.9 69.3 9.6 0.359 other female 85.4 78.4 7.0 0.409 Known/unKnown STaTuS Known 68.8 61.1 7.6 ** <.0001 unknown 67.0 56.9 10.1 ** <.0001 NoTES: Estimated subgroup effects are not statistically significantly different from each other within any category of subgroups. Because of analytic constraints, the total number of student observations varies across categories. Respectively, the number of student observations is 651, 8,223, 2,221, and 114 for reading proficiency levels 1 through 4; 2,612, 5,963, 2,768, and 53 for math proficiency levels 1 through 4; 9,841 and 2,409 for eligible and not eligible for free/reduced-price lunch; 1,708, 2,452, 1,866, 2,130, 283, and 261 for black males, black females, Hispanic males, Hispanic females, other males, and other females; and 6,389 and 4,820 for known and unknown. January 2012 Table 2. estimated effects of SSCs on Four-Year Graduation Rates for Student Subgroups: Student Cohorts 1 and 2 TargeT conTrol p-value For eSTimaTed STudenT cHaracTeriSTic SSc group eSTimaTed graduation prospects for each subgroup, as experienced a 7.6 percentage point increase eFFecT enrolleeS counTerparTS eFFecT measured by its control group counterpart’s in four-year graduation rates; the roughly 43 graduation rate. For example, rates range percent of students in the sample who were 8TH-grade reading proFiciency from 29.6 to 55.5 to 69.8 percent for levels 1, not known experienced a 10.1 percentage 2, and 3 in reading, respectively. Therefore, point increase. Both findings are statistically not meeting State learning Standards (level 1) 37.3 29.6 7.7 * 0.044 SSCs increase high school graduation rates significant although they are not statistically partially meeting State learning Standards (level 2) 66.7 55.5 11.2 ** <.0001 appreciably for students with extremely weak significantly different from each other. Thus, meeting State learning Standards (level 3) 81.1 69.8 11.3 ** 0.003 graduation prospects, modest prospects, contrary to what might be expected, students and stronger prospects. who make an effort to become known to their exceeding State learning Standards (level 4) -- -- -- SSC do not benefit more from attending it Findings for subgroups of students who were than students who do not make this effort. 8TH-grade maTH proFiciency and were not eligible for free or reduced- not meeting State learning Standards (level 1) 45.9 34.5 11.4 ** <.0001 price lunch tell a similar story. They indicate new Findings on Five-year graduation that SSCs increased graduation rates for rates for the Study’s First Student cohort: partially meeting State learning Standards (level 2) 65.1 53.8 11.3 ** <.0001 both subgroups (by 11.2 and 6.9 percentage Competing hypotheses exist about the likely meeting State learning Standards (level 3) 85.9 77.3 8.6 ** 0.001 points) even though their graduation effects of SSCs on five-year graduation prospects differed substantially. rates. For example, SSCs might be especially exceeding State learning Standards (level 4) -- -- -- effective at keeping struggling students low-income STaTuS Findings for subgroups of students by race/ engaged in school for a fifth year in order ethnicity and gender further reinforce the to meet their graduation requirements. If eligible for free/reduced-price lunch 66.9 55.7 11.2 ** 0.000 preceding results. They indicate that SSCs so, then SSC effects on five-year graduation increase graduation rates appreciably for rates might be greater than their effects on not eligible for free/reduced-price lunch 74.3 67.4 6.9 ** 0.001 all subgroups with a sample that is large four-year graduation rates. on the other race/eTHniciTy, By gender enough to detect these effects.22 Specifically, hand, SSC effects on four-year graduation SSCs increase graduation rates for black rates might reflect their ability to “speed Black male 65.2 55.3 9.9 * 0.021 males and females by 9.9 and 8.3 percentage up” graduation for students who otherwise Black female 69.1 60.8 8.3 * 0.018 points and for Hispanic males and females would need five years. In addition, due to by 7.7 and 8.7 percentage points.23 their small staffs, SSCs might not have the Hispanic male 61.3 53.7 7.7 0.065 capacity to provide the additional resources Hispanic female 68.7 60.0 8.7 ** 0.032 The final subgroup findings in the table needed to adequately assist students in their are based on whether or not students were fifth year. For either or both of these reasons, other male 78.9 69.3 9.6 0.359 known ahead of time to the SSC in whose SSC effects on five-year graduation rates other female 85.4 78.4 7.0 0.409 lottery they participated.24 A student can might be less than their effects on four-year become known to an SSC by contacting it graduation rates. Known/unKnown STaTuS in person or by telephone, by visiting it, by meeting with its representatives at a high Table 3 presents estimates of SSC effects on Known 68.8 61.1 7.6 ** <.0001 school fair, or in other ways. To promote graduation rates at five years after the study’s unknown 67.0 56.9 10.1 ** <.0001 informed school choice, the New York City first student cohort entered high school, as Department of Education encourages eighth- well as their four-year graduation rates (from NoTES: Estimated subgroup effects are not statistically significantly different from each other within any category of subgroups. Because of analytic graders and their parents to do all of these MDRC’s first report). These findings indicate constraints, the total number of student observations varies across categories. Respectively, the number of student observations is 651, 8,223, 2,221, and 114 for reading proficiency levels 1 through 4; 2,612, 5,963, 2,768, and 53 for math proficiency levels 1 through 4; 9,841 and 2,409 for eligible and things, which require initiative and effort. that between year four and year five, graduation not eligible for free/reduced-price lunch; 1,708, 2,452, 1,866, 2,130, 283, and 261 for black males, black females, Hispanic males, Hispanic females, rates for target SSC enrollees and their control 7 other males, and other females; and 6,389 and 4,820 for known and unknown. Findings in Table 2 indicate that the roughly group counterparts increased by almost exactly 57 percent of students in the subgroup the same amount. Hence, the effect of SSCs on sample who were known to their SSC graduation rates did not change. MDrC PoliCy Brief Table 3. estimated average effects of SSCs on Four-Year and Five-Year Graduation Rates: Student Cohort 1 TargeT conTrol p-value For eSTimaTed ouTcome (%) SSc group eSTimaTed eFFecT enrolleeS counTerparTS eFFecT Four-year graduaTion graduaTed From HigH ScHool 68.7 61.9 6.8* 0.013 local diploma granted 24.6 21.9 2.8 0.261 regents diploma granted 39.5 34.6 4.9 0.074 advanced regents diploma granted 4.4 5.5 -1.1 0.366 Five-year graduaTion graduaTed From HigH ScHool 75.2 68.1 7.1* 0.018 local diploma granted 27.3 24.4 3.0 0.265 regents diploma granted 44.9 40.0 4.9 0.081 advanced regents diploma granted 4.7 5.8 -1.1 0.368 NoTES: There are 5,363 and 5,428 student observations for the four-year and five-year analyses, respectively. The difference represents 65 students who reenrolled in a NYC public high school between their fourth and fifth years. Because of rounding error and the complex calculations involved, results by diploma type do not sum exactly to those for overall graduation rates Specifically, graduation rates increased from 1. The 105 SSCs studied serve about 68.7 percent in year four to 75.2 percent in year 45,000 students at full capacity. This five (or by 6.5 percentage points) for target SSC is equivalent in size to the entire high enrollees and from 61.9 percent in year four to school population of Houston, Texas, 68.1 percent in year five (or by 6.2 percentage the seventh largest school district in the points) for their control group counterparts. United States. The effect of SSCs on five-year graduation rates (a 7.1 percentage point increase) therefore was 2. The estimated 8.6 percentage point effect almost identical to the effect of SSCs on four- of SSCs on four-year graduation rates year graduation rates (a 6.8 percentage point is equivalent in magnitude to about 43 increase).25 In addition, at five years, the majority percent of the gap in graduation rates of the SSC impact comes from receiving Regents between white students and students of diplomas, as was the case at four years. color in New York City.26 Furthermore, this effect is relative to currently existing w H y a r e T H e S e schools, not to the 23 large failing high F i n d i n gS im p o rT a nT? schools that were closed, schools that had To place these findings in perspective, graduation rates that were much lower than consider: (1) the scale of the intervention the rates of the control group schools that evaluated, (2) the size of the effects it the SSCs were compared with in this study. produced, (3) the nature of the population it 8 served, (4) the diversity of subpopulations 3. SSC effects reflect a student population it benefited, (5) the institutional nature of that is 93.2 percent black or Hispanic, the intervention, and (6) the rigor of the 83.9 percent eligible for free or reduced- evaluation conducted. price lunch, 21.8 percent overage for January 2012 Table 3. estimated average effects of SSCs on Four-Year and Five-Year Graduation Rates: Student Cohort 1 TargeT conTrol p-value For eSTimaTed ouTcome (%) SSc group eSTimaTed grade, and 70.5 percent below grade level • Do SSCs increase college readiness and eFFecT enrolleeS counTerparTS eFFecT in reading and 63.9 percent below grade success in postsecondary education? level in math, based on state tests in • How were SSCs created? How do they Four-year graduaTion these subjects in eighth grade. operate? What obstacles do they face? graduaTed From HigH ScHool 68.7 61.9 6.8* 0.013 How are they dealing with these obstacles? local diploma granted 24.6 21.9 2.8 0.261 4. SSC effects were experienced by students What are their most important sources with widely varying graduation prospects, of financial and human resources? What regents diploma granted 39.5 34.6 4.9 0.074 by students who were and were not factors do they believe are most important advanced regents diploma granted 4.4 5.5 -1.1 0.366 eligible for free and reduced-price lunch, to their success? by students who were male and female, • How do SSC effects vary by school and Five-year graduaTion black and Hispanic, and by students who what factors predict this variation? graduaTed From HigH ScHool 75.2 68.1 7.1* 0.018 were known and not known to their SSCs. • What role does teacher selection, training, and quality play in producing SSC effects? local diploma granted 27.3 24.4 3.0 0.265 5. New York City’s SSCs were developed regents diploma granted 44.9 40.0 4.9 0.081 through a demanding proposal process co n c l uSi o n that was designed to ensure specified In summary, the present findings provide advanced regents diploma granted 4.7 5.8 -1.1 0.368 conditions and to stimulate innovation highly credible evidence that in a relatively NoTES: There are 5,363 and 5,428 student observations for the four-year and five-year analyses, respectively. The difference represents from a range of stakeholders and short period of time, with sufficient 65 students who reenrolled in a NYC public high school between their fourth and fifth years. Because of rounding error and the complex institutions. This differs markedly from organization and resources, an existing calculations involved, results by diploma type do not sum exactly to those for overall graduation rates many small schools in other jurisdictions school district can implement a complex that are created by reconfiguring existing high school reform that markedly improves schools. Furthermore, SSCs operate graduation rates for a large population of within the school district, in contrast low-income, disadvantaged students of color. to charter schools, which operate more independently. en d n oTeS 1 The closure of the 23 large high schools happened over time — as they no longer accepted entering ninth-graders 6. These findings are based on data for a and, thus, phased out over four years. very large sample that was created by a 2 Fifty SSCs are in the Bronx and 31 are in Brooklyn. Many series of naturally occurring randomized of them are in the city’s poorest neighborhoods. lotteries. Hence, they provide unusually 3 other new small secondary schools created by New York City during this time period include 38 general high schools rigorous evidence about the causal for grades 9-12, which screen students based on their effects of SSCs. And instead of relying academic backgrounds; 21 transfer schools that are designed on a single test score as a proxy for to help students who are overage for grade and under- credited; 33 middle/high schools for grades 6-12 or 7-12; and later life outcomes, the study measures one specialized high school for high-performing students. actual grade progression, high school 4 Bloom, levy Thompson, and Unterman (2010). graduation, and Regents diplomas 5 SSC lotteries are not public events, but rather the result of attained and scores achieved. the rules by which HSAPS assigns students to SSCs. of the 123 SSCs created in New York City, 105 had at least one lot- tery for one of the four student cohorts in the present study. wHaT are THe nexT STepS 6 Appendix A of Bloom, levy Thompson, and Unterman For reSearcH on SSc s? (2010) describes how HSAPS creates SSC lotteries, how over the next three years, with funding from an SSC lottery creates a treatment group of SSC lottery winners and a control group of SSC lottery losers, and the Bill and Melinda Gates Foundation, 9 how this information was used, together with information MDRC and its partners will expand and about the schools in which students enrolled, to estimate deepen their research on SSCs to address the effects of enrolling in an SSC relative to enrolling in some other New York City public high school. such questions as: MDrC PoliCy Brief Corresponding findings for the effects of winning an SSC SSCs worked better for the second cohort than the first. lottery are presented in Appendix B of the original report. 20 These findings on college readiness for cohorts one These findings are consistent with those for the effects of and two combined tell the same story as those for cohort enrolling in an SSC. The analysis in this policy brief uses one in MDRC’s first report. the same methodology. 21 To estimate SSC effects for student subgroups, the 7 Because MDRC’s study examines only one component of sample for each SSC lottery is stratified by the student New York City’s overall high school reform initiative (that characteristics that define each subgroup. Because not is, the small schools of choice), it does not attempt to de- all lotteries contain students from each stratum, the total termine the overall effect of these reforms. Neither does sample represented by each type of student characteristic it attempt to determine the effect of closing the 23 large (e.g., by reading proficiency or race/ethnicity) varies. failing high schools on students who would have attended 22 The sample for “other males” comprised only 283 stu- them had they not been closed. dent observations; for “other females,” the sample was 8 For example, a law firm provides internships and em- only 261 student observations. beds real-world examples in the curriculum for the Urban 23 The estimated SSC effect for Hispanic males is not sta- Assembly School for law & Justice, and the Urban Assem- tistically significant at the 0.05 level, although it is statisti- bly New York Harbor School is growing one of the largest cally significant at the 0.10 level, which is used by many oyster beds in the region. researchers. Furthermore, this finding is consistent with 9 As a result of a court decision, the New York City those for the other three subgroups of students of color Department of Education ended this practice. and is not statistically significantly different from them. 10 As described by Randi Weingarten at a forum on small Hence, the data suggest that SSCs increase four-year schools sponsored by the Alliance for Excellent Education graduation rates for Hispanic males, although the evi- in November of 2010, the United Federation of Teachers dence for this subgroup is weaker than that for the others. participated in the planning and approval process for 24 Bloom, levy Thompson, and Unterman (2010) de- many of the new small schools in New York City. scribe how each SSC lottery includes either students who 11 By 2008, 46 of 50 states and the District of Columbia were known to the SSC or students who were not known had received multimillion-dollar federal grants for small to it, but not both. lottery winners and losers are thus learning communities (U.S. Department of Education, perfectly matched in this regard. n.d.). 25 The slight discrepancy between the reported difference 12 While the Gates Foundation’s stated goal was to sup- in changes over time of graduation rates for target SSC port the creation of schools that possessed seven “Attri- enrollees and their control group counterparts and the butes of High Performing Schools,” it was the structural change over time in the estimated impact of SSCs on the focus on school size (not to exceed 100 students per graduation rate is due to rounding. grade) that became a key strategy of the foundation’s 26 MDRC computations based on 2011 data from the NYC grantmaking (Evan et al., 2006). Department of Education indicate a gap between citywide 13 See Finn and Voelkl (1993); lee and loeb (2000); Was- four-year graduation rates for white students and students ley et al. (2000); Klem and Connell (2004). of color equal to 21.0 percentage points for the fall 2005 cohort and 18.8 percentage points for the fall 2006 cohort. 14 For student achievement, see Haller, Monk, and Tien The average gap is thus 19.9 percentage points. (1993); Howley (1989); Howley and Huang (1991); lee and Smith (1997). For dropout rates, see Pittman and Haughwout (1987); McMullen, Sipe, and Wolf (1994). reFerenceS 15 See lee, Bryk, and Smith (1993); lee and Smith (1995); For all analyses in this brief, MDRC’s calculations use lee and Smith (1997). High School Application Processing System data from 16 See Evan et al. (2006). eighth-graders in 2004-2005 to 2006-2007, as well as data from New York City Department of Education atten- 17 The “on-track” measure was developed by the Consor- dance, course credits, Regents exam, and enrollment files tium on Chicago School Research. High school students from the 2005-2006 to 2009-2010 school years. who are on track to graduate by the end of their first year — meaning that they have earned at least 10 credits and Allensworth, Elaine, and John Q. Easton. 2005. The On- are failing no more than one core subject — are three and Track Indicator as a Predictor of High School Graduation. a half times more likely to graduate in four years than are Chicago: Consortium on Chicago School Research. other students (Allensworth and Easton, 2005). Bloom, Howard S., Saskia levy Thompson, and Rebecca 18 Target SSC enrollees are sample members for whom Unterman. 2010. Transforming the High School Experience: SSC effects are reported. Appendix A of Bloom, levy How New York City’s New Small Schools Are Boosting Stu- Thompson, and Unterman (2010) describes how this dent Achievement and Graduation Rates. New York: MDRC. group is defined and how findings for it are obtained. Evan, Aimee, Jamie Shkolnik, Mette Huberman, Becky 10 19 The estimated average effect of SSCs on four-year Smerdon, Barbara Means, Mengli Song, Karen Mitchell, graduation rates was 9.9 percentage points for the second Charles Storey, linda Shear, and Kazuaki Uekawa. 2006. cohort and 6.8 for the first cohort. Both of these estimates Evaluation of the Bill & Melinda Gates Foundation’s High are statistically significant, although the difference between School Grants Initiative. Washington, DC: American Insti- them is not. Hence, one cannot say with confidence that tutes for Research.