Vol. 30.1 Educational Research Quarterly 3 The Value of Extracurricular Support in Increased Student Achievement: An Assessment of a Pupil Personnel Model Including School Counselors and School Psychologists Concerning Student Achievement as Measured by an Academic Performance Index Greg S. Goodman I. Phillip Young Clarion University of University of California, Pennsylvania Davis This study examined two models of extra-curricular support for enhancing the academic achievement of students as measured by state mandated test scores. One management model includes the use of school counselors as enhancers of the educational process while the other model addresses the contribution of school psychologists. To differentiate between these separate although related models for serving students in a large Pacific coast state’s public school setting, this study included the structural equations modeling technique Analysis of Moment Structures (AMOS). Within the statistical model examined, student achievement on the API scores served as the dependent variable and student diversity as operationalized by using the number of students receiving free or reduced meals as the proxy for economic status was always included as a covariate. Findings include that the number of psychologists employed by a public school district demonstrate a significant and decisive impact on achievement. Since Sputnick I in 1957, a major concern of policy makers and educators has been and continues to be the improvement of student performance in the public school setting. Initial efforts in this area were encapsulated in the Elementary and Secondary Education Act (E.S.E.A.), and these reforms have been followed by a spate of additional influential legislations (i.e., Public Law 94-142). Most recently, this same thrust of 4 Educational Research Quarterly 2006 legislative activities continues to be perpetuated by the No Child Left Behind (N.C.L.B.) Act (2002). Constant across these legislative enactments throughout the past decades is the perceived value of extra curricular support beyond the classroom setting. Extra-curricular support was promoted initially as a means of identifying and of enticing gifted students to pursue careers in science and mathematics. Early on, school counselors played an almost exclusive role as a means of accomplishing these goals for identifying and enticing students into selective academic ventures. Although concerns about student achievement continue to be a mainstream concern during the last 50 years (Fullan, 1991), the focus of educational reform efforts has been recast to include other students beyond the most able. Most notably, within the 1970’s, emphasis was redirected from the most able students to the least able students. To address the needs of the least able students within the public school setting, Public Law 94-142 (U.S. Congress, Senate, 1975) was passed in 1975. Public Law 94-142, like E.S.E.A., required the services of additional professional personnel in the public school setting. To complement, as well expand the expertise of school counselors, PL 94-142 legislated that public school districts employ school psychologists. During the late 70’s and early 80’s, school psychologists were employed by public school districts, and these newly appointed personnel focused largely on the neediest students and became a mainstay in public education (Tindall, 1983). Following the commissioned report “A Nation at Risk” (National Commission on Excellence in Education, 1983), concerns about student achievement were redirected somewhat from individuals, as students, to school districts as organizations. To measure the performance of school districts, most states have legislated proficiency tests. Unlike achievement levels of individuals, as students, protected by the Family Educational Rights and Privacy Act (1966), achievement levels for school districts are public information and are reported by the popular Vol. 30.1 Educational Research Quarterly 5 press in most communities. As a source of public information, the academic performance of school districts is well publicized. For some states this information is labeled as a report card, while for other states this information is reported by an academic performance index. In all states using standardized measures, these results serve as a barometer of pupil performance for a variety of educational stakeholders including parents, educators, and policy makers. Pivotal in the process of enhancing student achievement within each school district is the direct effect of classroom instruction as provided by teachers. Teachers serve as the primary conduit for instruction and work directly with students in the classroom setting. However, with respect to district level indices for performance, any specific teacher serves only as part of the overall mosaic contributing to district indicators disclosing student achievement. Principals, on the other hand, are designated instructional leaders for their building and have influence beyond a single classroom. As an instructional leader, principals are responsible for aligning the school’s curriculum and for promoting instruction that is both centered on students and individualized (Barth, 1990). To accomplish these goals, principals must have open communication with students, parents, and teachers and draw, most importantly, from other extra- curricular personnel bearing on student achievement. However, teachers like principals, are a constant within every public school district across the United States. What can vary across school districts is the type of extra curricular support provided to assist teachers and principals in their attempts to enhance student achievement. The specific types of extra- curricular support examined in this study include the services of school counselors and of school psychologists as defined by a particular type of pupil personnel model. Advancements In the past, attempts to assess pupil personnel services in the public school setting have focused on processes rather than on outcomes. According to Fagan and Wise (2000), this 6 Educational Research Quarterly 2006 approach involves counting the number of student contacts, the number of student assessments, and/or other quantifiable involvements such as the number of times serving in IEP meetings. Generally, these global assessments involving counts have focused on the pupil services model as a collective and have ignored the individual contributions for different role incumbents (school counselors and school psychologists) comprising the model. Even though both school counselors and school psychologists provide a potential means for enhancing the achievement level of students, they do so from different perspectives (Goodman & Carey, 2004). For school counselors, the roles are framed well by the American Counseling Association (ACA, 1995). According to this learned society, school counselors should address academic development, career exploration, and personal/social enhancement for all students. In contrast to the general roles performed by school counselors for all students, school psychologists are more interventional and assessment driven in their approach (Thomas & Grimes, 2002). School psychologists provide individualized assessments for students through testing and other forms of evaluation. In fact some researchers have demonstrated that school psychologists spend over 50% of their time performing assessments and designing interventional strategies (Reschly, 1998). Unknown, however, within this particular body of literature is the impact of school counselors and/or school psychologists on outcomes of academic achievement as opposed to process measures within the public school setting. More specifically, the question remains unanswered as to what are the unique as well as the collective contribution of different pupil personnel providers; i.e., school counselors and school psychologists for the achievement levels of a public school district as measured by outcomes from a standardized proficiency assessment? An attempt to provide such information is the purpose of this study. Vol. 30.1 Educational Research Quarterly 7 Method The population for this study is all public school districts located in a large Pacific coast state. From this population (N=352), 150 public school districts were selected at random. This number of school districts was determined through a power analysis. Following procedures suggested by Cohen (1977), the probability of a type I error was set by an alpha level of .05. The probability of a type II error was set by a beta level of .20, and a medium effect size of .25 was determined. Collectively, these parameter constraints indicate that 150 public school districts are required given these researchers particular configuration of variables chosen for investigation in this study. Variables Investigated The dependent variable of interest in this study is a composite achievement measure for school districts and reported as an academic performance index (API) score. This API score is assessed through a group administered, standardized proficiency test battery and developed by the state department of education. Based on outcomes from these assessments, academic performance levels for a particular school district can range from a low of 200 to a high of 1,000 with the higher score implying greater academic achievement. Another variable assessed in our study was the social economic status levels of students enrolled within each school district. This variable was operationalized by using the number of students receiving free or reduced lunch as a proxy for economic status. Because social economic status of students is found to correlate significantly, as well as consistently, with student achievement in many other studies (Sweetland & Hoy, 2000), we, consistent with these researchers, used this variable as a control in our study. The variables used as predictors of student achievement in this study were the number of school counselors and the number of school psychologists. Student achievement as assessed for school districts via an academic performance index was regressed against these predictor variables while controlling for the social economic status of students. 8 Educational Research Quarterly 2006 For operational measures of our dependent variable (school district performance), the control variable (social economic status of students) as well as our independent variables (number of school counselors and school psychologists), we collected all information from a state database. As required by state statue within our targeted state, all public school districts must provide this information. As such, we were able to collect these data from a single source and did not have to rely on responses from individual school districts. Figure 1 To assess the impact of a pupil personnel model containing both school counselors and school psychologists on the student achievement of district level students, we used the structural equations modeling technique Analysis of Moment Structures (AMOS) as described by Arbuckle and Wothke (1999). Contained in Figure 1 is the depiction of the general pupil personnel model containing all variables of interest in this study. As can be observed in Figure 1, this model affords an assessment for the overall as well as unique effects associated with each variable of interest. For the overall model containing all variables, a total of 28% of the variance associated with student achievement at the local school district level can be accounted for given this specific Vol. 30.1 Educational Research Quarterly 9 configuration of variables (see Figure 1). This amount of variance is statistically significant at an alpha level of .05. That is, the extra curricular support as provided by a pupil personnel model containing both school counselors and school psychologists covaries with the student achievement levels of public school district students. By using a structural equation model for analyzing these results, we are able to provide important insights relative to this overall finding. For each of our independent variables (school counselors and school psychologists), we are able to estimate the direct as well as indirect contribution to student achievement at the district level. These effects are noted by appropriate path coefficients as provided in Figure 1. Although the number of counselors as well as the number of school psychologists employed by public school districts within our targeted state covary substantially (r=.91, as measured by a correlation coefficient, see Figure 1), each position accounts for a different amount of variance associated with the achievement of district level students as measured by the API results (see Beta coefficients in Figure 1). Without controlling for the social economic status of students within particular school districts as measured by direct effects (Beta coefficients), the standardized regression coefficient for school counselors is .05, while the standardized regression coefficient for school psychologists is .75 (see Figure 1). The direct effect for the number of school psychologists is both significantly different from zero and significantly different from the number of school counselors (the later not significantly different from zero but significantly different form school psychologists). Because academic performance of students as measured by a district level API results has been reported by others (Sweetland & Hoy, 2000) to be moderated by the social economic status of students, we control this potential source of variation within our structural equation model. Within our structural equation model, the percentage of variance (R2 ) associated with the social economic status of students accounted for the number of school counselors and the number of school 10 Educational Research Quarterly 2006 psychologists is found to be .73 (see Figure 1). This outcome indicates that past staffing practices for components of a pupil personnel model (school counselors and school psychologists) may be driven largely by the social economic status of students. Indeed, like other research, we found that social economic status of students to be an important factor bearing on the student achievement levels as measured by a district level performance index (Β=-1.01, p < .05, as measured by a standardized regression coefficient). School districts enrolling higher numbers of low social economic students are found to score lower on the API results than school districts enrolling lower numbers of economically disadvantaged students. To control for the social economic status of students enrolled in schools included in our working sample of school districts, we examined the indirect effects of school counselors and school psychologists on API outcomes. The indirect effect for the number of school counselors is insignificant (B=.18, p $ .05). In contrast, the indirect effect for the number of school psychologists is significant (B= .69, p < .05, see Figure 1). Discussion and Conclusion Student achievement issues are hardly novel news for stakeholders concerned about educational performance within the public school setting. These concerns have endured for decades and have shaped as well as reshaped many reform efforts. However, the focal point for reform efforts has shifted somewhat across time from individual students at both ends of the distribution (ESEA & PL 42-192, respectively) to the professional educational providers within the school setting. Even though individuals as students continue to receive specific attention relative to their particular needs (PL 94-142), more recently the performance of school districts has come under close scrutiny. Unlike the academic achievement of particular students shielded from public preview by the Family Rights and Accountability Act (1996), the academic performance of school districts is public information. To provide the public with information about school Vol. 30.1 Educational Research Quarterly 11 districts, state legislators have mandated the use of proficiency testing within our targeted state. By so doing, a standard yardstick exists for measuring outcomes associated with both educational performance as well as educational practice. Of these two outcome measures, this manuscript focuses on a particular type of educational practice: extra-curricular support as provided by school counselors and school psychologists. The specific educational practice examined in this study is the configuration of a pupil personnel model utilized by most school districts. Although the particular pupil personnel model utilized by any one school district can vary in many ways; i.e., by including school social workers, school nurses, and other mental health professionals, constant within any pupil personnel model is the engagement of school counselors and school psychologists. Both school counselors and school psychologists contribute to the overall educational enterprise but do so in different ways (Goodman & Carey, 2004). Using a district level index as the unit for analysis for measuring academic performance via a state proficiency examination within our targeted state, our data indicate that school psychologists, as opposed to school counselors, appear to be more influential between these two different extra curricular role incumbents in accounting for variance in student performance. This finding is consistent regardless of whether or not the social economic status of the students was considered within the analyses. However, by considering the social economic status of students in our analyses, these findings have important implication for practice in the field setting. Within the recent past as well as in the foreseeable future, most public school districts have endured and will continue to face a declining financial budget situation. To control costs and to contain expenditures, cuts in operation of a public school district may be needed. First to go within this deliberation process involving the balancing of a budget is maintenance personnel followed by extra curricular support service personnel. In the past, little information existed for guiding school 12 Educational Research Quarterly 2006 boards and school administrators about choosing among extra curricular support personnel. Indeed, previous studies in this area have fallen short in two areas. These studies have assessed global counts of process variables (number of times served) and have failed to differentiate within the program evaluation design the effects of different role incumbents (school counselors or school psychologists) especially as related to student achievement. By focusing on outcomes (student achievement) as opposed to focusing on processes (counts) (Fagan & Wise, 2000) and by differentiating between role incumbents within the pupil personnel model (school counselors as opposed to school psychologists), this study provides school board members and school administrators with important empirical information. To either reduce or to expand the pupil personnel services at the school district level, our data suggest that money may be best spent by protecting and expanding the intervention and assessment services provided by school psychologists. References American Counseling Association (1995). Code of ethics and standards of practice. Alexandria, Virginia: Author. Arbuckle, J.L. & Wothke, W. (1999). AMOS 4.0 user’s guide. Chicago, IL: Small Waters Corporation. Barth, R. (1990). Improving schools from within. San Francisco, Ca: Jossey-Bass. Cohen, J. (1977). Statistical power analysis for the behavioral sciences. New York: Academic Press. Elementary and Secondary Education Act. ESEA (Public Law 89-10). Fagan, T. K., & Wise, P. S. (2000). School psychology; Past, present, and future. Washington, DC: National Association of School Psychologists. Family Education Rights and Privacy Act (Public Law 93-380), Section 438, The Buckley Amendment. In T.B. Gutkin & C.R. Reynolds (Eds.). The handbook of school psychology, Second edition pp. 1046-1056. New York: John Wiley and Sons.