Abstract Title Page. Title: Teacher Layoffs, Teacher Quality and Student Achievement: The Implementation and Consequences of a Discretionary Reduction-in-force Policy Authors and Affiliations: Matthew A. Kraft Harvard Graduate School of Education SREE Spring 2013 Conference Abstract Template Abstract Body Layoffs, as painful as they are, should fall on the least-effective teachers when layoffs are absolutely unavoidable. - Arne Duncan, US Secretary of Education, March 22, 2011 . Background / Context: Personnel reductions are a common cost-cutting measure that firms employ in response to changes in market demand and macroeconomic downturns. In the public educator sector, where personnel costs represent between 60-80% of total expenditures, school districts commonly turn to reductions in force when faced with a major budget shortfall (Roza, 2007). Unlike private sector firms, employee layoffs in most U.S. public schools are governed by collective bargaining agreements between districts and local teachers’ unions. Beginning in the 1950’s, inverse- seniority layoff clauses became widely incorporated into union contracts as a means of protecting teachers from arbitrary and unlawful termination processes. Seniority-based layoffs replaced discriminatory and nepotistic practices with a simple and objective criterion available for all teachers. Today, the vast majority of collective bargaining agreements continue to prioritize seniority over any other layoff selection criteria.1 In recent years, the Great Recession has caused teacher layoffs policies to become of central importance as districts began implementing reductions in force to compensate for large decreases in local and state tax revenues. The implementation of longstanding “last hired, first fired” layoff policies has caused these practices to come under increasing criticism among policy organizations (e.g. National Center on Teacher Quality, 2010; The New Teacher Project, 2010) and in the media (e.g. “Our view,” 2011; Abramson, 2011). Goldhaber and Theobald (2010) find that those teachers who received RIF notices in Washington State had value-added scores that were no different on average than their peers whose jobs were not threatened. Simulations by Goldhaber and Theobald as well as Boyd, Lankford, Loeb, and Wyckoff (2011) demonstrate how a hypothetical layoff policy that prioritizes teacher effectiveness (as measured by value- added scores) would result in the selection of less-effective teachers than those who would lose their jobs under a “last in, first out” policy. Purpose / Objective / Research Question / Focus of Study: I present some of first evidence on the implementation and subsequent effect of discretionary layoff policies, by studying the 18th largest public school district in the nation, Charlotte-Mecklenburg Schools (CMS). In total, the CMS School Board approved the elimination of almost 2,000 employees, including more than 1,000 teaching positions, over two years (2008/09 and 2009/10). Candidates for layoffs were identified using a set of five general criteria: duplicative positions, enrollment trends, job performance, job qualifications, and length of service. Importantly, my dataset contains the primary measure of job performance used by principals and district officials, principal evaluation scores on a statewide rubric. I focus my analyses on three main objectives: 1) comparing the relative weight that CMS administrators and principals placed on a variety of RIF criteria when implementing layoffs, 2) estimating the causal 1 In a minority of states, reduction-in-force policies are regulated by the state through legislation or legal precedence. In five states where collective-bargaining is explicitly illegal, local school boards maintain the authority to determine reduction-in-force policies. SREE Spring 2013 Conference Abstract Template 1 effect of discretionary layoffs in CMS on student achievement in the following academic year, and 3) documenting how these negative effects are exacerbated when highly effective teachers are laid-off teachers. Setting Charlotte-Mecklenburg serves over 137,000 pre-k to 12th grade students across 180 schools and employs over 9,000 teachers. It is located in one of five “right-to-work” states where collective bargaining is explicitly illegal. Policies governing teacher contracts are determined by the North Carolina state legislature and subject to modifications made by the 115 Local Education Agencies. Local education agencies such as Charlotte Mecklenburg Schools are allowed to determine reduction-in-force policies unilaterally, but in accordance with federal and state fair labor practices, policies, and statutes. CMS Board policy requires that superintendents make a recommendation to the Board regarding the need for a reduction in force and the criteria to be used in the reduction. Subjects & Intervention: In March of 2009, Superintendent Gorman presented the CMS School Board with recommendations for employee reductions including approximately 450 classroom teaching positions in order to close an $87 million budgetary shortfall. This recommendation stipulates the broad procedures the Superintendent proposed to use in order to implement the layoffs. It outlines three key steps: 1) the district would allocate layoffs across schools based on projected enrollment trends, 2) principals would identify position categories (such as grade levels or subjects) that would be reduced, and 3) district officials would select which teacher(s) among those in the identified position categories would be laid off. The CMS School Board approved the Superintendent’s proposed process for carrying out reductions in force in 2009. The following year, the Board again approved the Superintendent’s proposal to eliminate the jobs of approximately 600 classroom teachers using a similar process. Notably, student performance was not included as a criterion to be used in the RIF process in either of the two years. Research Design & Analysis: In order to describe the relative importance of different teacher characteristics in the reduction-in-force process, I construct a teacher-year data set for all K-12 teachers employed by CMS in the 2008/09 and 2009/10 academic years. My final analytic sample includes 17,249 teacher-year records. I then fit a series of linear probability models where I regress an indicator of being laid off, RIF, on measures of district RIF criteria and other controls for teacher j in school s in year t: Here, the parameters of interest are the vector of coefficients, , that capture the estimated relationship between RIF criteria measures and the conditional probability of being laid off. I include the following covariates in my full model to restrict my comparisons to teachers with similar demographic characteristics who taught in similar schools in the same academic year: a vector of observable teacher demographic characteristics, ( ), a vector of school characteristics, ), and fixed effects for academic years, ( ). In additional models, I further SREE Spring 2013 Conference Abstract Template 2 restrict my comparisons to teachers within the same school by adding fixed effects for schools. In all models, I estimate standard errors clustered by school-year. I identify credible estimates of the effect of layoffs in CMS on student achievement by isolating plausibly exogenous variation in average grade-level achievement within a school-year and the number of teachers who were laid off in that school and grade in the previous year. This identification strategy rests on the assumption that principals, who selected the grades in their schools in which positions would be cut, did not make this choice based on any differential ability among rising cohorts of students at their school. Empirical tests lend strong support to this assumption. I implement this approach by constructing a continuous measure of the number of teachers laid off in each grade of a school in the previous year, RIF_GRADE. I then fit a series of models where students’ test scores in mathematics are modeled as a function of RIF_GRADE, control variables, and select sets of fixed effects. I specify my baseline education production function following Jackson & Bruegmann (2009): (II) where the outcome of interest, , is the standardized scaled scored on the state end-of-grade mathematics test for student i in grade g with teacher j in school s in year t. I include grade- specific cubic functions of students’ prior-year achievement, , in both mathematics and reading, as well as vectors of controls for observable student characteristics , the characteristics of a student’s peers with the same teacher ( ), and grade-by-year fixed effects ( ). The inclusion of school-by-year fixed effects ( ) account for any school-wide year- specific shocks to student achievement, such as the turnover of a principal or the adoption of a new curriculum, by restricting my comparison to the students within the same school in the same year. The estimated coefficient on RIF_GRADE captures the average effect of laying off a teacher in year t-1 on the academic achievement of students in year t in the grade in which the laid-off teacher taught. In this baseline model, my estimates are identified off of variation in the average achievement across grades within a given school-year. I estimate standard errors clustered at the teacher-level. I extend the analyses described above by interacting RIF_GRADE with a lagged measure of the average effectiveness of laid-off teachers in a school-grade-year. I use both standardized average principal evaluation scores and value-added estimates as my performance measures. I construct these average measures of effectiveness using all available data for a given teacher in the years t-n to t-1 where t-n is the first year the teacher appears in the data. This approach, described in detail by Chetty, Friedman, and Rockoff (2011), breaks any potential for correlated errors in measures of teacher effectiveness and student achievement outcomes. I estimate value- added scores for each teacher following the empirical Bayes approach described by Kane and Staiger (2008). Finally, I refit model (II) and include the interaction of RIF_GRADE with these measures of the average effectiveness of laid-off teachers. The addition of these interaction terms allows me to examine whether the loss of a more effective teacher had a larger negative effect on future student achievement than laying off a less effective teacher. Findings / Results: Results from descriptive regression analyses suggest that CMS used multiple RIF criteria when selecting teachers for layoffs including tenure status, licensure status and type, and job performance. I present averages of RIF criteria and school characteristics across RIFed teachers SREE Spring 2013 Conference Abstract Template 3 and non-RIFed teachers for my full analytic sample in Table 1. On average, RIFed teachers had accrued similar amounts of seniority and experience, but were rated almost 0.8 SD lower by principals (approximately a third of a point on a 4 point scale), and were 0.037 and 0.011 test- score standard deviations (SD) less effective than non-RIFed teachers as measured by value- added scores. Teachers who were hired after the beginning of the school year constitute over 22% of all RIFed teachers, but only 1% of all non-RIFed teachers. Differences in rates of licensure deficiency are also evident as 16% of RIFed teachers did not hold appropriate licenses while the comparable rate was only 3% for non-RIFed teachers. Results from descriptive regression analyses suggest that CMS used multiple RIF criteria when selecting teachers for layoffs including tenure status, licensure status and type, and job performance. Tenure status, evaluation scores, licensure status, and contract status were all important predictors of the probability of being laid off. I also find that principals were more likely to eliminate foreign language, arts, and physical education teachers than general- elementary or secondary teachers who taught core subjects. I examine the effect that teacher layoffs in 2008/09 had on student achievement in the following year by isolating plausibly exogenous variation in layoffs within a school across grade-levels. I present estimates for mathematics achievement from model (II) as well as results from a variety of robustness checks in Table 2. My baseline estimates indicate that, for each teacher who was laid off, student achievement decreased by three percent of a standard deviation in the following year in mathematics (column 1). This negative relationship is robust to a variety of alternative model specifications. I extend the analyses above by interacting the number of teachers laid off in a grade in the previous year with the average effectiveness of these laid-off teachers. This provides a test of any differential effect of layoffs across schools due to differences in the effectiveness of the RIFed teachers. In Table 3, I present estimates from three approaches of interacting RIF_GRADE with a measure of effectiveness; one with a linear term for lagged evaluation scores, one with evaluation scores binned by quartiles, and one with a linear term for lagged value-added scores. Overall, I find strong evidence that laying off highly-effective teachers as measured by principal evaluations exacerbated the negative effect of layoffs. Estimates using within-school-year variation across grades (column 1) show that laying off a teacher that was one standard deviation more effective than a teacher with an average evaluation score increased the negative effect of layoffs by 50%. These results remain unchanged when I include both school- by-grade and student fixed effects, although the results are attenuated when either set of fixed effects is added separately. Conclusions: Research has shown that “last hired, first fired” policies maximize the number of teachers subject to reductions in force by eliminating those teachers that are lowest on the pay scale first. Until now, advocates of effectiveness-based reduction-in-force policies could only point to simulated policy exercises as evidence of the potential benefits of a discretionary reduction-in- force policy. This study suggests that, while reductions in force negatively affect student achievement, districts have the potential to reduce these negative effects by concentrating layoffs among the lowest-performing teachers. Although replacing seniority with performance measures can minimize effects on student achievement, exchanging one inflexible criterion for another will not provide districts with any discretion in navigating a complex process aimed at preventing a variety of negative consequences. SREE Spring 2013 Conference Abstract Template 4 Appendices Appendix A. References Abramson, L. (2011, March 6). Pressure mounts to ax teacher seniority rules. National Public Radio. Boyd, D., Lankford, H., Loeb, S., & Wyckoff, J. (2010). Teacher layoffs: An empirical illustration of seniority vs. measures of effectiveness. CALDER Brief 12, July. Chetty, R., Friedman, J., & Rockoff, J. (2011). The long-term impacts of teachers: Teacher value-added and student outcomes in adulthood. NBER Working Paper No. 17699. Goldhaber, D., & Theobald, R. (2010). Managing the teacher workforce in austere times: The implications of teacher layoffs. Working Paper 2010-07. Center for Education Data & Research. Jackson, C.K., & Bruegmann, E. (2009). Teaching students and teaching each other: the importance of peer learning for teachers. American Economic Journal: Applied Economics, 1(4), 85-108. Kane, T. J., & Staiger, D. O. (2008). Estimating teacher impacts on student achievement: An experimental evaluation. NBER Working Paper No. 14607. National Center on Teacher Quality. (2010). Teacher layoffs: Rethinking “last-hired, first-fired” policies. February. Our view: Teacher layoffs based on only seniority fail kids [Editorial]. (2011, March 8). USA Today. Roza, M. (2007). Frozen assets: Rethinking teacher contracts could free billions for school reform. Education Sector Report. January. The New Teacher Project. (2010). A smarter teacher layoff system: How quality-based layoffs can help schools keep great teacher in tough economic times. The New Teacher Project. Policy Brief, March. Appendix B. Tables and Figures SREE Spring 2013 Conference Abstract Template 5 Table 1: Means and differences of select teacher and school characteristics across RIFed and Non-RIFed teachers. Full Analytic Sample RIFed Non-RIFed Difference Experience Average Experience (Years) 11.39 10.61 0.78 Average Seniority (Years) 6.63 6.56 0.08 Probationary Teacher 81.7% 37.6% 44.1% *** Returning Retired Teacher 20.4% 0.8% 19.6% *** Late Hire 22.4% 1.0% 21.4% *** Observations 17,249 Effectiveness Principal Evaluation Score (SD) -0.742 0.056 -0.798*** Observations 12,674 Mathematics Value-Added Score (test-score SD) -0.020 0.017 -0.037** Observations 2,508 Reading Value-Added Score (test-score SD) -0.007 0.004 -0.011+ Observations 2,483 Licensure Licensure Deficiency 16.5% 3.0% 13.5% *** Mathematics 13.8% 9.1% 4.7% *** English Language Arts 20.6% 16.8% 3.8% * Science 9.5% 7.8% 1.7% + Social Studies 12.5% 10.8% 1.7% Foreign Language 6.3% 3.3% 3.0% *** Arts 8.4% 6.6% 1.8% + Physical Education 5.7% 4.8% 0.9% Vocational Education 7.4% 5.4% 2.0% * Elementary Education 29.3% 48.0% -18.7% *** English as a Second Language 4.2% 3.9% 0.3% Special Education 12.3% 15.0% -2.7% * Observations 17,249 School Characteristics Elementary School 37% 52% -14.8% *** Middle School 20% 19% 1.1% High School 41% 27% 14.2% *** State Performance Index (SD) -0.073 0.09 -0.163** Average days absent per student per year (days) 8.643 7.829 0.814** % White Students 27.9% 31.4% -3.5% ** % African American Students 48.3% 43.2% 5.1% *** % Asian Students 4.7% 4.7% 0.0% % Hispanic Students 15.3% 16.2% -0.9% % Limited English Proficient Students 13.1% 14.2% -1.1% + % Special Eduation Students 11.4% 10.9% 0.5% Observations 17,249 Average Mathematics Achievement (student SD) -0.184 -0.066 -0.118*** Average Reading Achievement (student SD) -0.185 -0.064 -0.121*** Observations 12,652 p<0.001***, p<0.01**, p<0.05*, p<0.10 + Notes: P-values are derived from OLS regressions of the given characteristic on an indicator for layoff status in which standard errors are clustered by school-year. SREE Spring 2013 Conference Abstract Template 6 Table 2: Parameter estimates of the relationship between the number of teachers laid off in a student's grade in the previous year and student achievement in mathematics. School x Year FE School x Year FE School x Year FE & School x Year FE & & School x Grade FE School x Grade FE Student FE & Student FE (1) (2) (3) (4) RIF_GRADE -0.033** -0.013 -0.032** -0.015 (0.012) (0.011) (0.012) (0.012) Observations 239,600 239,600 239,600 239,600 p<0.001***, p<0.01**, p<0.05*, p<0.10 + Notes: Standard errors clustered by teacher are reported in parentheses. All regressions include grade-by- year fixed effects and school demographic controls. School demographic controls include average mathematics and reading achievement in the previous year, and percentages of students with missing test scores in the previous year, and who are male, African-American, Hispanic, Asian, Native American, multi- racial, limited English proficient, and have an independent education plan. SREE Spring 2013 Conference Abstract Template 7 Table 3: Parameter estimates of the relationship between the interaction of the number of teachers laid off in a student's grade in the previous year with the average effectiveness of these laid-off teachers, and student achievement in mathematics. School x Year FE School x Year FE School x Year FE & School x Year FE & & School x Grade FE School x Grade FE Student FE & Student FE (1) (2) (3) (4) Panel A: Evaluation Score - Linear RIF_GRADE -0.042* -0.013 -0.026 -0.016 (0.016) (0.014) (0.016) (0.015) RIF_GRADE*Evaluation Score -0.021* -0.005 -0.014 -0.020* (0.008) (0.008) (0.008) (0.008) Observations 223,805 223,805 223,805 223,805 Panel B: Evaluation Score - Quartiles RIF_GRADE -0.021 -0.01 -0.021 0.004 (0.016) (0.015) (0.016) (0.017) RIF_GRADE*Evaluation Score 2nd Quartile -0.027** -0.01 -0.015 -0.013 (0.010) (0.011) (0.011) (0.011) 3rd Quartile -0.007 0.002 -0.002 -0.014 (0.012) (0.012) (0.011) (0.011) Top Quartile -0.068** -0.026 -0.093*** -0.081*** (0.022) (0.020) (0.023) (0.025) Observations 223,805 223,805 223,805 223,805 Panel C: Value-Added - Linear RIF_GRADE 0.022 0.003 -0.005 -0.030 (0.077) (0.071) (0.070) (0.065) RIF_GRADE*Value-Added Score -0.032 -0.053 -0.023 -0.024 (0.040) (0.040) (0.037) (0.037) Observations 205,680 205,680 205,680 205,680 p<0.001***, p<0.01**, p<0.05*, p<0.10 + Notes: Standard errors clustered by teacher are reported in parentheses. Value-Added scores are re-standardized to make them comparable to standardized evaluation scores. All regressions include grade-by-year fixed effects and school demographic controls. School demographic controls include average mathematics and reading achievement in the previous year, and percentages of students with missing test scores in the previous year, and who are male, African- American, Hispanic, Asian, Native American, multi-racial, limited English proficient, and have an independent education plan. SREE Spring 2013 Conference Abstract Template 8