education policy analysis archives A peer-reviewed, independent, open access, multilingual journal Arizona State University Volume 20 Number 3 January 30, 2012 ISSN 1068-2341 Re-Examining Exit Exams: New Findings from the Education Longitudinal Study of 2002 Kate Shuster Claremont Graduate University Citation: Shuster, K. (2012.) Re-Examining exit exams: New findings from the education longitudinal study of 2002. Education Policy Analysis Archives, 20(3). Retrieved [date], from http://epaa.asu.edu/ojs/article/view/797 Abstract: Using the nationally representative, cohort-based data of the Education Longitudinal Study of 2002 (ELS:02), this study employs multiple regression to examine the effects of exit exams on student achievement and school completion. This study finds that exit exams as a whole do not have substantial effects on student achievement in mathematics, twelfth grade GPA, or school completion. Standards-based exams are a positive predictor of dropping out of school but lose their predictive power once GED recipients are coded as completing school. Exit exams do not affect GED seeking and acquisition. When exit exams are disaggregated by type and students are sorted by ninth grade GPA quartiles, end-of-course exams have some negative effects on mathematics test score gains. Students in the bottom two quartiles see reduced test score gains of 28% and 29% of a grade level equivalency (GLE). These effects disappear when students in North Carolina are coded as taking a different type of exam. Standards-based exams had a small positive effect, about 37% of a GLE, on the top quartile of students. Overall, the findings showed no results for school completion and mixed results for test score gains. The article concludes that policymakers looking to boost high school achievement would be better served by working to boost student accomplishments before high school. Keywords: high school exit examinations; high school graduation tests; accountability; graduation tests; minimum competency testing; education longitudinal study. Manuscript received: 07/15/2010 Revisions received: 06/01/2011 Accepted: 10/14/2011 Re-examining Exit Exams 2 Re-examinando los exámenes de egreso: Nuevos hallazgos del Estudio Longitudinal de Educación de 2002 Resumo: Usando datos de una muestra representativa nacional del Estudio Longitudinal de Educación del 2002 (ELS: 02), este estudio emplea regresión múltiple para examinar los efectos de los exámenes de egreso en el rendimiento escolar y finalización de la escuela. Este estudio revela que los exámenes de egreso como un todo no tienen efectos sustanciales en el rendimiento escolar en matemáticas, el promedio de notas (GPA) del duodécimo grado, o de la finalización de la escuela. Los exámenes basados en estándares son un predictor positivo de abandono escolar, pero pierden su poder predictivo, una vez que los estudiantes que obtienen los diplomas de exámenes de educación GED se codifican como completando la escolaridad. Exámenes de egreso no afectan la búsqueda y adquisición de diplomas de GED. Cuando los exámenes de egreso están desagregados por tipo y los estudiantes de noveno grado son clasificados por los cuartiles promedio (GPA) los exámenes tienen algunos efectos negativos sobre las mejorías en las calificaciones en las pruebas de matemáticas. Los estudiantes en los dos cuartiles más bajos pierden 28% y 29% de las mejorías en las calificaciones de las pruebas de equivalencia de nivel de grado (GLE). Estos efectos desaparecen cuando los estudiantes de Carolina del Norte están codificados como tomar otro tipo de examen. Exámenes basados en estándares tuvieron un pequeño efecto positivo, alrededor del 37% de un GLE con estudiantes en el cuartil superior. En general, los resultados mostraron que no hay resultados asociados para completar la escolaridad y los resultados son mixtos respecto a las mejorías en las notas de las pruebas. El artículo concluye que políticos que busquen mejorar los logros en la escuela secundaria obtendrían mejores resultados trabajando para aumentar los logros de los estudiantes antes de entrar a la escuela secundaria. Palabras clave: exámenes de egreso; pruebas de graduación de la escuela secundaria; responsabilidad; graduación. Re-analisando os exames de saída: Novas descobertas do Estudo Longitudinal da Educação de 2002 Resumo: Utilizando dados de uma amostra nacionalmente representativa do Estudo Longitudinal da Educação de 2002 (ELS: 02), este estudo utiliza regressão múltipla para examinar os efeitos de exames de saída no desempenho dos alunos e na conclusão da escolaridade. Este estudo revela que os exames de saída como um todo não têm efeitos significativos sobre o desempenho dos alunos em matemática, na média de pontos (GPA) no ano final do Ensino Médio ou na conclusão da escolaridade. Testes baseados em padrões são um indicador positivo de abandono escolar, mas perdem o seu poder preditivo, uma vez que os alunos que são aprovados nos testes para obter os diplomas de educação GED são codificados como concluindo a escolaridade. Exames de saída não afetam a busca e aquisição de diplomas GED. Quando os exames de saída são discriminados por tipo e os alunos de nono ano são classificados por quartis do GPA, os exames têm alguns efeitos negativos sobre as melhorias nos resultados dos testes de matemática. Alunos nos dois quartis mais baixos perderam 28% e 29% das melhorias nos resultados dos testes de equivalência do nível da série (GLE). Estes efeitos desaparecem quando os estudantes da Carolina do Norte são codificados como tendo feito outro tipo de exame. Testes baseados em padrões tiveram um pequeno efeito positivo, cerca de 37% de um GLE com estudantes do quartil superior. Em geral, os resultados não mostraram resultados para conclusão da escolaridade e resultados mistos sobre melhorias nos resultados nos testes. O artigo conclui que os políticos que buscam melhorar o desempenho da escola obteria melhores resultados trabalhando para aumentar o desempenho do aluno antes de entrar no ensino médio. Palavras-chave: exames de saída, testes de formatura do ensino médio; políticas de avaliação e responsabilidade; graduação. Education Policy Analysis Archives Vol. 20 No. 3 3 Introduction States administering high school exit exams enrolled 74% of all students and 83% of students of color in the 2009-2010 school year (Center on Education Policy, 2010). Although the degree of difficulty and material consequences of exit exams vary from state to state, these tests have a common policy rationale: the state’s desire for accountability. Any accountability that exit exams provide does not come cheaply. California’s 2003-2004 budget included $21 million for the administration of the California High School Exit Examination (CAHSEE). This number does not represent the exam’s total fiscal impact on the state. School districts assume additional costs as they prepare for the exam, administer it, and deal with the consequences for students who fail to pass. Even if all these costs were added together, CAHSEE’s total cost would still be relatively small in the scope of the state’s $55 billion budget for education. That does not mean it is insignificant: $21 million represented half the cost of California’s Intensive Algebra Instruction Academies and Elementary School Intensive Reading Program, discontinued in 2003-2004 as part of budget cuts and twice the 2003-2004 funding cuts for school and classroom library materials (O’Connell, 2003). States like California continue to implement and refine their exit examination policies while education researchers struggle to provide decisive answers to the question of what, exactly, such exams do. The current literature on exit exams has not kept pace with these tests’ evolving nature. Studies have relied on cohort data that do not account for the “second wave” of exit exams or state aggregate data that fail to apply important controls for student and state covariates. The present study uses newly available data from the Education Longitudinal Study of 2002 (ELS:02) and addresses key methodological issues. The opportunity costs and risks of exams like the CAHSEE must be subject to our most rigorous evaluations. Only then can policymakers make decisions that balance the need for meaningful high school diplomas with the need for access to education. This study addresses two major research questions: First, how do high school exit exams affect school completion? Second, how do high school exit exams affect student achievement? These two questions cut to the heart of the debate about high school exit exams. States have a profound social and economic interest in ensuring that as many of their students as possible receive high school diplomas. States also have an interest in securing and signaling the value of these diplomas. This leads to a delicate balancing act for policymakers setting standards – a kind of Goldilocks effect – where some standards are too high (forcing too many students out of school or setting unachievable benchmarks), some are too low (reducing the value of a diploma or lowering aggregate achievement), and the elusive “just right” standards require sophisticated research that is aligned with policy evaluation for decisions that try to maximize outcomes for students and societies. These policy questions do not occur in a vacuum. In an age of increasingly tight state budgets, policymakers must be conscious of the opportunity costs of their accountability decisions. Exit exams are expensive and time-consuming. If they do not provide substantial benefits, policymakers must rethink their accountability strategies and devise different mechanisms to monitor and incentivize school and student performance. Exit Examinations and School Completion American exit exams have changed dramatically in the last 20 years (Warren & Jenkins, 2005) as they have moved from minimum competency exams to more difficult standards-based assessments. This shift has led to some incommensurability in otherwise similar evaluations. There is Re-examining Exit Exams 4 some support for the claim that exit exams suppress graduation rates while increasing the number of students seeking a General Educational Development (GED) credential or diploma (Bishop, 2005; Dee & Jacob, 2006; Jacob, 2001; Papay, Murnane, & Willett, 2010; Reardon, 1996; Reardon, Atteberry, Arshan, & Kurlaender, 2009; Warren, Jenkins, & Kulick, 2006). Other studies (Catterall, 1987; Greene & Winters, 2004a; Griffin & Heidorn, 1996; Muller, 1998; Warren & Edwards, 2005; Warren & Jenkins, 2005) have found no relationship between exit exams and school completion. The most recent and definitive review of the literature to date (Holme, Richards, Jimerson, & Cohen, 2010) shows that easier exams do not affect school completion, while more difficult exams are associated with higher drop-out rates. Policy changes and methodological differences may explain some of the literature’s divergent findings. Until recently, longitudinal analyses were limited to use of the National Education Longitudinal Study 1988 (NELS:88) data – a set that does not account for the “second wave” of accountability measures (Dee & Jacob, 2006). Longitudinal studies are important in this area because the effects of graduation requirements vary over time as participating parties adjust their behavior (Lillard & DeCicca, 2001). A 2004 report by the Center for Education Policy (CEP) predicted that it might take “half a generation” (p. 26) before students show the full effects of a high school exit exam. Data tracking a cohort over time allow researchers to control for student and school-level variables that are known to have substantial effects on outcomes like school completion and test scores. Even when using appropriate data, researchers must navigate a number of methodological pitfalls. These include omitted variable bias (in particular, failure to control for prior academic achievement) and what Jacob (2001) has called the “endogeneity of the MCT [minimum competency test] policy variable.” High school exit exams are correlated with other characteristics of schools or states that may influence dropout rates in either direction. Even local or school-specific requirements may bias statewide samples (Lillard & DeCicca, 2001). We know that states with the highest dropout rates, lowest overall student achievement, higher unemployment rates, and highest proportion of minority students are the states most likely to have high school exit exams (Reardon, 1996; Warren & Kulick, 2007). Because exit exams are often introduced as part of larger standards- based reform and accountability measures, exit exam policies may seem to cause effects that are actually more closely related to other school or statewide variables (Reardon, 1996; Bishop, Mane, Bishop, & Moriarty, 2001). For example, Lillard and DeCicca (2001) found that higher state-mandated minimum course requirements were positively related to dropout rates. They estimated that if state course graduation requirements (CGRs) were increased by one standard deviation (about 2.5 CGRs), attrition rates would change by about one percent. This small change in attrition was large when considered in absolute terms. As the authors note, “from a base population of roughly 13 million 14-17 year old youth in 1990, these results suggest that between 104,000 and 208,000 more students will leave high school before graduating when CGRs increase by one standard deviation” (p. 465). This negative effect was strongest for minority students in the poorest quintile. The authors found no independent effects for high school exit exams. By contrast, Dee and Jacob’s (2006) analysis of data from the 2000 Census and the National Center for Education Statistics’ (NCES) Common Core of Data (CCD) found that Minnesota’s exit exam increased the dropout rate in poor and urban schools while nationwide exit exams significantly increased the probability of dropping out of high school for all students, and black students in particular. Reardon, et al. (2009) found that graduation rates in several large California school districts declined by 3.6 to 4.5 percentage points as a result of exit exam requirements. These effects were concentrated on low-achieving, female, and minority students. New research on the effects of Massachusetts’ exit exam (Papay, Murnane, & Willett, 2010) showed that low-income urban students Education Policy Analysis Archives Vol. 20 No. 3 5 who barely failed the mathematics exit exam had an eight percentage point lower graduation rate than similar students who barely passed. Earlier multi-state studies, such as that by Greene and Winters (2004a), found no such effects; however, the measures that study used to calculate state- level completion rates have been criticized for inaccuracy and a failure to control for observed and unobserved differences between states (Dee & Jacob, 2006; Warren, 2005; Warren, Jenkins, & Kulick, 2006). Interaction between school completion and academic achievement outcomes may bias achievement measures in studies not using cohort data. If exit exams induce low-achieving students to drop out, then the achievement effect of the test may be inflated (Jacob, 2001, p. 104). On the other hand, if exit exams have the opposite effect on dropout decisions, achievement effects may be suppressed or diminished. The most current research using a national sample (Current Population Survey, CCD, and GED exam data) to evaluate the relationship between exit exams and school completion was a 2006 study by Warren, Jenkins, and Kulick. The authors comprehensively revised previous models estimating school completion rates, finding that exit exams are associated with lower rates of school completion, especially in poor states with high percentages of racial and ethnic minorities. They argue that these findings are consistent with seemingly contradictory findings from analyses using the NELS:88 data because that survey did not include the era of more difficult exit exams and therefore was unable to distinguish between the effects of more and less difficult exams. Exit Exams and Academic Achievement The major claim in favor of exit exams is that they increase student achievement. In theory, exit exams provide a signal for distribution of rewards and consequences to succeeding and failing schools, teachers, and students (Bishop, Moriarty, & Mane, 2000). This signal should increase incentives for students to achieve by raising the value of a diploma and clearly articulating the conditions for its receipt. There is research that supports a relationship between exit exams and improved achievement (Bishop, 1996; Bishop, 1997; Bishop, 2005; Bishop, Moriarty, & Mane, 2000; Bishop, et al., 2001); again, there is evidence to the contrary (Reardon, et al., 2009; Grodsky, Warren, & Kalogrides, 2009; Jacob, 2001). When Holme, et al. (2010) reviewed the literature to date, they found that the available evidence did not show a link between easier or more difficult tests and improved student achievement; in fact, as they note, there is some evidence (e.g., Reardon, et al., 2009) that exit examinations reduce achievement among minority and low-achieving students. The reliability of the evidence turns on methodological issues like the nature of the data studied (statewide or cohort-based), the age of the sample (much of the research uses the NELS:88 data) and the array of controls applied (for example, whether controls for state education policy or prior achievement were used). Some research concludes that end-of-course exam systems have a greater effect than minimum competency exams on student achievement (Bishop 2005; Bishop, Moriarty, & Mane 2000). Comparative international studies support an especially strong relationship between end-of- course exams and student achievement (Bishop, 1996; Bishop, 1997; Bishop, 2005). Using NELS:88 data, Bishop, et al., 2001 found that end-of-course exams in New York State were significantly associated with score gains of 38% of a grade level equivalent (GLE) for B/B+ students and with roughly 50% of a GLE for A students. This number is derived from gains on a test score composite, averaging student gains on the four tests (science, math, social studies, and English). Math score gains in New York were significant only at the ten percent level on a one-tail test – a very weak threshold for significance in a sample greater than 11,000. Re-examining Exit Exams 6 Research from Grodsky, Warren, and Kalogrides (2009) challenges these findings. This study analyzed the relationship between exit exams and achievement using the long-term trend data from the National Assessment of Educational Progress (NAEP). Controlling for prior achievement, socioeconomic status, ethnicity, and a variety of state factors, they found no achievement effects in reading and math at the mean or for students in the 10th, 20th, 80th, or 90th percentiles of the achievement distribution. These results were constant when exit exams were disaggregated by relative difficulty. Methods Using the second follow-up to the Education Longitudinal Study of 2002 (ELS:02) data, this study employed multivariate stepwise regressions to predict school completion and academic achievement while controlling for a variety of background factors including student characteristics, family characteristics, family processes, state characteristics, average state achievement, prior student achievement, school characteristics, and school processes. The change in p-value of the F-statistic required to include a variable was .05, while a change of .10 in p-value was grounds for removal. Data The ELS:02 is an ongoing longitudinal survey of a nationally representative sample of students, tracking a cohort of students from their sophomore year through their postsecondary experiences. The base-year survey collected data from a variety of sources, including students, parents, and school administrators. Subsequent rounds of the ELS:02 followed up with students and administrators in 2004 and 2006. This study used the secure version of the ELS:02 to extract state and school information. Supplemental data were gathered to control for policy and economic conditions in students’ states of residency. States’ 2004 Education Week Quality Counts (Skinner & Staresina, 2004) ratings were used to control for state level education reform packages. School demographic data used to control for divergent school characteristics came from the CCD and Private School Survey (PSS) data linked to school codes and embedded in the secure ELS:02. State economic indicators represent select characteristics of each state and the District of Columbia in 2004. These variables (Table 1) control for state-specific economic conditions (Bishop, et al., 2001). Sample The ELS:02 base-year study sampled 750 public and private schools. Of 17,590 eligible selected sophomores, 15,360 completed a base-year questionnaire, as did 13,490 parents, 7,140 teachers, 740 principals, and 720 librarians (Ingels, Pratt, Wilson, Burns, Currivan, Rogers, & Hubbard-Bednasz, 2007). Cases were removed from the data set if they lacked base-year math test scores, follow-up math test scores, ninth grade GPA, or if their high school completion status was unknown at the time of the second follow-up survey. The final sample was composed of 12,520 students from 720 schools in 49 states (no students from North Dakota were included once the sample was cleaned for the purposes of this study) and the District of Columbia. As the number of dropouts in the overall sample fell below the 10% threshold that would be necessary for prediction as a dependent variable with the full sample, a special subsample of the larger data set was created to allow prediction of school completion outcomes. First, the 850 status dropouts were extracted from the main data set. Then a random sample of 3,380 diploma recipients Education Policy Analysis Archives Vol. 20 No. 3 7 was extracted from the remaining cases. This resulted in a subsample N of 4,230, where 20% were classified as status dropouts. This subsample was used to predict school completion. Variables Outcome measures. Two models evaluated the relationship between exit exams and school completion. The exit exam literature differs on whether GED recipients should be counted as dropouts. NCES defines GED recipients as completers but not as graduates, a classification that this study employs. This study was somewhat more interested in graduation than completion, but defined students as dropouts in two ways to test relationships between exams and these two outcomes. This approach permitted a test of the possibility that standards-based exams increased incentives for students to acquire their GEDs (Bishop, et al. 2001; Bishop, 2005). For the first analysis, a dummy variable differentiated between students with a high school diploma and those without. For the second analysis, an additional dummy variable was created where students with their GED were coded as high school completers. Two additional models examined the relationship between exit exams and academic achievement. The first academic achievement model predicted students’ standardized twelfth grade point average (GPA). The second model predicted students’ gain in math test scores from tenth to twelfth grade. Like previous analyses using similar (NELS:88) data (Bishop, et al., 2001), this study predicted score gains using item response theory (IRT) estimated number right scores. IRT estimated number right scores are overall criterion-referenced measures of status (Ingels, et al., 2007) that estimate the number of questions students would have answered correctly if they had responded to all 72 questions in the mathematics pool of questions. Participants’ math gain score was calculated by subtracting the base year math IRT estimated number right from the F1 math IRT estimated number right. Gain scores were subsequently standardized. State Characteristics. This study used a variety of state economic indicators to replicate the controls used by Bishop, Mane, Bishop, and Moriarty in their 2001 analysis of the NELS:88 data. This approach allowed consideration of the hypothesis that “new” (above minimum competency) exit exams change achievement and school completion outcomes for students in exit exam states. Table 1 describes these variables and their data sources. Additional controls were used to account for changes in state education policy. Five Education Week Quality Counts ratings for 2004 (Skinner & Staresina, 2004) were matched with corresponding state codes for inclusion in the study’s data set: standards and accountability, efforts to improve teacher quality, school climate, adequacy, and equity. The ratings were converted into numerical scores using a standard four point GPA scale. As a final set of controls for background characteristics that vary between states, this study used state 2003 NAEP scores in fourth and eighth grade reading and math. Student and school characteristics. This study controlled for sex (a dichotomous variable with female coded 2 and male coded 1), ethnicity (a dichotomous variable identifying students as White was used as a proxy) and ESL participation as well as student self-reports of time spent on homework and time spent watching television. While student self-reports of homework time are somewhat unreliable (Trautwien & Köller, 2003), they are useful as a measure of student perceptions of their own commitment to schooling (Cool & Keith, 1991). One additional student self-reported variable was drawn from the second follow-up interview. It represents the number of stressful life events the student experienced Re-examining Exit Exams 8 in the past two years. The survey and codebook do not define “stressful life event,” leaving the interpretation up to the students.. Status variables like socio-economic status (SES), family characteristics (Battin-Pearson, Newcomb, Abbott, Hill, Catalano, & Hawkins, 2000; Rumberger, 1983) and family educational expectations (Ensminger & Slusarcick, 1992) are strong predictors of school completion. This study used two family characteristics variables as controls: an ELS:02-generated SES composite variable and a parent-reported count of the number of student siblings that dropped out of high school. ELS:02 offered two versions of the SES composite variable: one that used occupation prestige values based on the 1961 Duncan index (SES1), and one that used the 1989 General Social Survey occupational prestige scores (SES2). This study uses SES2 based on its superior fit with contemporary occupational data (Nakao & Treas, 1994). SES2 uses five equally weighted, standardized components: father’s/guardian’s education, mother’s/guardian’s education, family income, father’s/guardian’s occupation, and mother’s/guardian’s occupation. There is a well-documented connection between school success and parental involvement in students’ lives and schooling (Fan & Chen, 2001; Jimerson, Egeland, Sroufe, & Carlson, 2000; Rumberger & Arellano, 2007; Steinberg, Lamborn, Dornbusch, & Darling, 1992). Sixteen variables in the ELS:02 base year parental survey measured parental involvement. Exploratory factor analysis using a principal components method was used to examine the relationships between these variables. Four factors emerged (variables and factor loading tables are in the appendix). The first factor, with an initial eigenvalue of 3.82 and a cumulative explained variance of 20.1%, was used in subsequent regression analysis. The variables with the highest loading on this factor included contacting the school about the program for the year (loading of .63), contacting the school about course selection (loading of .60), contacting the school about helping with homework (loading of .60), contacting the school about plans after high school (loading of .59), and contacting the school about fundraising/volunteer work (loading of .59). Given the very high eigenvalue of this first factor and its high explained variance, the remaining three factors were not used in regression analyses. Any study of the impact of exit exams must control for prior student achievement (Jacob, 2001). One major shortcoming of the ELS:02 data is the lack of eighth grade achievement outcomes to set pre-high school baselines. This does not mean that cohort data should not be used to study the impact of exit exams; instead, caution should be used when interpreting the results. This study used transcript derived ninth grade GPA, the earliest ELS:02 metric, to control for prior achievement. Education Policy Analysis Archives Vol. 20 No. 3 9 Table 1 State Economic Characteristics Variables Variable Data sources 2004 per capita Gathered from the U.S. Department of Commerce, Bureau of Economic income in home state Analysis, Survey of Current Business (www.bea.doc.gov/bea/regional/spi). This survey computes 2004 per capita income using population estimates from the Bureau of the Census 2004 ratio of college 2004 high school graduate, high school dropout and college graduate graduate earnings to earnings were gathered from the U.S. Census Bureau’s American high school graduate Community Survey data set earnings in home (http://www.census.gov/acs/www/index.html). state 2004 ratio of high Numbers reflect median earnings in 2004 inflation-adjusted dollars for the school graduate to population 25 years and over. Data are limited to the household population high school dropout and exclude the population living in institutions, college dormitories, and earnings in home other group quarters. state 2004 ratio of tuition 2004 mean weekly retailing wage rates were compiled from the Bureau of at 4-year public Labor Statistics’ Occupational Employment and Wage Estimates databases. colleges to weekly Annual wages in retail sales (BLS occupation code 41-2031) were extracted earnings in retail in from http://www.bls.gov/oes/oes_dl.htm, divided by 52, and rounded to home state two decimal places. 2004 tuition at 4-year public colleges was compiled using 2004 data from the National Postsecondary Student Aid Study (NPSAS) (http://nces.ed.gov/surveys/npsas). Data were filtered to reflect only public 4-year nondoctorate and public 4-year doctorate institutions. Mean tuition and fees are measured in 2004 dollars, reflecting the sampling procedure of the NPSAS. 2004 unemployment The 2004 unemployment rate was compiled from the Bureau of Labor rate in home state Statistics’ Local Area Unemployment Statistics tables (http://www.bls.gov/lau/lastrk04.htm). Employment rates for states represent the annual average ranking for 2004 as a percentage of the state’s labor force. School characteristics play an important role in student academic achievement and school completion (Goldschmidt & Wang, 1999; Rumsberger & Arellano, 2007). Including these variables allows for control of known characteristics of high and low performing schools, such as high rates of free and reduced price lunch eligibility and average class size. Researchers using the ELS:02 have little choice but to take administrator reports at face value even though these reports may be unreliable (Warren, 2005). Even unreliable reports are likely to have significant heuristic value. Re-examining Exit Exams 10 Administrator-reported variables used here included the percentage of tenth graders receiving ESL, the percentage of tenth graders receiving remedial math, the percentage of tenth graders receiving remedial English. To control for the effects of internal dropout prevention programs, this study used an administrator-reported dichotomous variable indicating the presence of a dropout program at the school. A final administrator-reported control was a continuous variable representing years of mathematics coursework required to graduate. School demographic variables include percentage of students eligible for free or reduced price lunch, the number of full time employee teachers, grade ten enrollment, percentage minority students, total school enrollment, and ratio of students to teachers. The CCD school type variable was recoded as a public/private dichotomy. School safety contributes to student outcomes (Boyd, 2004; Flannery & Singer, 1999; Noakes & Noakes, 2000). Sixteen school characteristics variables in the ELS:02 administrator survey measure school safety. Exploratory factor analysis using a principal components method was used to examine the relationships between these variables. Four factors emerged. Their loading tables and variable descriptions appear in the appendix. The first factor was extracted to use in regression analysis. The variables with the highest loading on this factor included how often the use of illegal drugs was a problem at school (loading of .70), how often students on drugs/alcohol at school was a problem (loading of .73), and how often the sale of drugs near school was a problem (loading of .71). Given the very high eigenvalue (5.78) of this first factor and its high explained variance (36.10%), the remaining three factors were not used in regression analyses. Additional literature (Cash 1993; Earthman, 2002; Lackney, 1994; Phillips, 1997) supports a connection between clean and well-maintained school facilities and student outcomes. Fourteen school characteristics variables evaluate school facilities. Exploratory factor analysis using a principal components method was used to examine the relationships between these variables. Six factors emerged. Their loading tables along with variable descriptions appear in the appendix. The first factor was extracted for use in subsequent regression analysis. The variables with the highest loading on this factor included trash on front hallway floors (loading of .63), graffiti on hallway walls/doors/ceiling (loading of .63), graffiti on bathroom walls and ceilings (loading of .57), and graffiti on bathroom staff doors/walls (loading of .57). Given the high eigenvalue of this first factor (2.94) and its high explained variance (17.32%), the remaining five factors were not used in regression analyses. State high school exit examinations. Information on state high school exit examinations in 2004 (Table 2) was drawn from three major sources: the CEP’s 2004 report on high school exit examinations (Center on Education Policy, 2004), Dee and Jacob’s 2006 paper assigning degrees of difficulty to high school exit examinations, and the 2004 Quality Counts ratings (Skinner & Staresina, 2004). Exams were coded by type and difficulty to account for their heterogeneous effects (Wößmann, 2005). Discrepancies in six states (Alaska, Arkansas, Maryland, New Mexico, North Carolina and Ohio) required further investigation to reconcile conflicting accounts in the major data sources this study used to characterize the content of 2004 state graduation exams. Alaska’s High School Graduation Qualifying Exam (HSGQE) was implemented for the graduating class of 2004 (Center on Education Policy, 2004), and tested at tenth grade levels (Skinner & Staresina, 2004), so was included in this study contrary to Dee and Jacob’s (2006) classification. Closer examination revealed that neither Arkansas (Greene & Winters, 2004b; Howell, 2008) nor Maryland (Center on Education Policy, 2004; Center on Education Policy, 2005) had a high