DOCUMENT RESUME ED 428 977 SO 028 225 AUTHOR Croxford, Linda; Cowie, Michael TITLE The Effectiveness of Grampian Secondary Schools. INSTITUTION Edinburgh Univ. (Scotland). Centre for Educational Sociology. PUB DATE 1996-01-00 NOTE 10p. AVAILABLE FROM Centre for Educational Sociology, The University of Edinburgh, 7 Buccleuch Place, Edinburgh EH8 9LW, Scotland. PUB TYPE Information Analyses (070) EDRS PRICE MF01/PC01 Plus Postage. DESCRIPTORS Comparative Education; Educational Planning; Educational Policy; Foreign Countries; Global Education; Outcomes of Education; Program Evaluation; *School Effectiveness; *Secondary Education; *Student Attitudes IDENTIFIERS Scotland ABSTRACT In response to the (Scottish) government's decision to publish annual educational examination results, on a school-by-school basis, a three-year development program was established by Grampian Education Authority (Scotland) and the Centre for Educational Sociology (CES) at the University of Edinburgh. In this report, the main findings of the development program with respect to two different types of indicators of school performance are described. The types of indicators include: young people's perceptions of their experiences at school, and the "added value" of schools in terms of their pupils' attainment. Key findings include: (1) the majority of young people who attended Grampian secondary schools found their schooling a worthwhile experience; (2) over three-quarters agreed that "school work was worth doing," "discipline was fair," and "my teachers helped me to do my (3) average attainment in public examinations has risen between 1993 best"; (4) although there are large differences between Grampian schools and 1994; in the average attainment of school leavers, most of the difference between schools are attributable to differences in pupil intakes; and (5) league tables based solely on published examination results, which do no control for pupil intake, give a misleading picture of the relative effectiveness of Grampian schools. Tables showing results of the survey are provided. (EH) ******************************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. ******************************************************************************** The Effectiveness of Grampian Secondary Schools. U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement PERMISSION TO REPRODUCE AND EDUCATIONAL RESOURCES INFORMATION DISSEMINATE THIS MATERIAL HAS CENTER (ERIC) BEEN GRANTED BY lifhis document has been reproduced as received from the person or organization Linda CroeorcL originating it. 0 Minor changes have been made to improve reproduction quality. o Points of view or opinions stated in this TO THE EDUCATIONAL RESOURCES document do not necessarily represent INFORMATION CENTER (ERIC) official OERI position or policy. 1' 2 BEST COPY AVAILABLE THE EFFECTIVENESS OF GRAMPIAN SECONDARY SCHOOLS January 1996 This report describes the results of a three-year development programme by Grampian Education Authority and the Centre for Educational Sociology (CES) at the University of In part, the development programme was a response .to the government's Edinburgh. decision to publish annual information on public examination results on a school-by- Equally important was the Authority's wish to base its policies for the school basis. continuous improvement of its schools on the best possible information, and to ensure that the views of parents, teachers and the public were informed by a proper understanding of the strengths and limitations of indicators of school quality. In this report we describe the main findings of the development programme with respect to two different types of indicator of school performance: young people's perceptions of their experiences at school, and the "added value" of schools in terms of their pupils' attainment. Some Key Findings The majority of young people who attended Grampian secondary schools found their >> schooling a worthwhile experience. Over three-quarters agreed that "school work was worth doing", "discipline was fair", and "my teachers helped me to do my best". Average attainment in public examinations has risen between 1992 and 1994. » Although there are large differences between Grampian schools in the average attainment of >> is attributable to differences in pupil school leavers, most of the difference between schools intakes. League tables based solely on published examination results, which do not control for pupil » intake, give a misleading picture of the relative effectiveness of Grampian schools. This report was written and produced by Dr Linda Croxford, Centre for Educational Sociology Contributions were also made by Bruce and Michael Cowie, Grampian Education Authority. Robertson, Grampian Education Authority; Richard Bisset, Kemnay Academy and Michael Taylor, Dyce Academy. CES CENTRE for EDUCATIONAL Grampian SOCIOLOGY 3 Regional Council PAGE 2 Young People's Perceptions of their Young people wanted their teachers 1. be to friendly and approachable, but also to be able to Experiences at School maintain an ordeily environment. Although 62 The survey asked young people a number of percent of respondents felt that "teachers could keep order in class", some 45 percent considered questions about their experiences of school, and "there were too many troublemakers in my asked them to give "an overall view". Their responses are shown in Table 1, in terms of the class". percentages of young people who agreed with The majority of school leavers had favourable each statement. Questions from the Grampian perceptions of their school and most young surveys were subsequently included in the 1993 Scottish School Leavers Survey; comparative people who attended schools in Grampian would figures for the whole of Scotland in 1993 are of echo the words youngster the who included in Table I. commented: Grampian school leavers favourable more generally their are in "When I was at school the teachers really perceptions of their school experiences than their encouraged you to do your best and were counterparts throughout Scotland. there for you when ever you had a problem More three-quarters of than from leavers They also wanted with school/home etc. Grampian schools agreed that "school work was you to do your best and they treated you worth doing", "discipline was fair" and "my like adults, so I think my school was really teachers helped me to do my best". One young worthwhile." person explained: It is important to note, however, that there is a disaffected minority who feel "my teachers "I liked the way that it was always easy to approach teachers. They always helped didn't care about me". Fortunately, there were you as much as possible. If there was comparatively few young people alienated by something you didn't understand they were their experience of school. There are, however, willing to explain things if you asked, a number of issues raised by the survey that giving up part of their lunch break to do schools will wish to reflect on and respond to. so." Table 1: Young people's perceptions of their experiences at school (Percentage agreed with statement) Scotland Grampian School work was worth doing 82 80 Discipline was fair 79 74 My teachers helped me do my best 75 71 Teachers listened to my ideas and views 72 66 If I had a problem there was always a teacher I could talk to 63 62 Teachers could keep order in class 62 67 My school dealt well with any bullying that went on 59 51 47 My school had a wide choice of after-school activities 38 Pupils respected the teachers 42 37 There was vandalism at my school during the school day 50 58 There were too many troublemakers in my class 45 44 Theft among pupils was common at my school 25 29 My teachers didn't care about me 16 16 Sources: Grampian - Grampian Surveys 1992, 1993, 1994, N (.100%) 11582 Scotland - Scottish School Leavers Survey 1993, N (r--100%) 3469 BEST COPY AVMLABLE 4 TrZFr 2.1 Measuring added value Value 2. Added It is found by The added value is an estimate. Examination results published by the SOED of a the SCE attainment Audit Unit can be described as "raw" results, estimating what hypothetical "average" pupil would be if: because they do not take account of differences In this report we compare the in pupil intake. we sent the same pupil to each school, "raw" results for Grampian schools with results i) based on the "added value" which does take ii) each school had the same social context, set same the school iii) each was in account of pupil intake. neighbourhood. In this study "added value" is taken to mean the i) We take account of each pupil's own family contribution made by the school to the progress background of each pupil, after taking account of differences in attainment attributable to: Research at CES and elsewhere has shown that a pupil's attainment is enhanced or reduced by the each pupil's own family background, of the disadvantage or advantage relative family's background circumstances in which the school social context, s/he lives while at school. the relative deprivation of the neighbourhood In estimating the added value of schools, we take in which the school is located, account of family background and estimate the attainment of a pupil with "average" family stability and change over three years, background characteristics in each school. By this means we can measure the effect if we sent gender. the same pupil to each school. We estimate added value in terms of the final ii) We take account of the school social context attainment of young people when they leave school. Our estimate of added value is different Pupils are affected not only by their own family from that of the SOED Audit Unit because it background, but also by the family background allows for all the progress made by pupils of the other pupils attending their school. during their time at school, not just between Standard Grade and Highers. Family background characteristics which affect pupil attainment A disadvantaged family background An advantaged family background lone or step parent natural parents two above-average number of siblings below-average number of siblings parents who left school at the earliest opportunity parents who had post-compulsory schooling parents without qualifications parents who gained advanced qualifications parents with unskilled or manual occupations parents with non-manual occupations father unemployed father in work mother at work mother full-time unpaid in the home living in rented accommodation living in a home owned by parents 5 PAGE 4 We find that the attainment of all pupils in a 2.2 Results school is enhanced if the school has many pupils from advantaged backgrounds, and conversely Rise in attainment of Grampian school leavers the attainment of all a school pupils is in depressed if the school has few pupils from The average SCE attainment of Grampian school advantaged backgrounds. leavers has risen over the three years from 1992 to 1994. This rise in attainment is the result of In estimating the added value of schools we improved added value in Grampian schools over estimate what the attainment of an "average" the three years. The majority of schools shared pupil would be if each school had an "average" in the rise. However, in two schools added value proportion of pupils from advantaged family did not increase over the three years to the same backgrounds (average social context). By this extent as in the rest of Grampian. means we can measure the effect if each school had the same social context. Some of the rise in the added value of schools can be attributed to the increased proportion of young people who have stayed on to post- iii) We take account of the effect of the relative deprivation of the neighbourhood in which the compulsory schooling, and thereby improved school is located their SCE qualifications. School policies for improvement may also have contributed to the Pupils may also be affected by aspects of rise. prosperity or deprivation in the neighbourhood surrounding their school. We use indicators of Female school leavers have higher attainment neighbourhood deprivation derived from the Part of the difference is caused by than males. 1991 census to estimate what the attainment of the greater tendency for girls than boys to stay on our "average" pupil would be if each school was However, the gender gap narrowed at school. located in an "average" neighbourhood. By this over three because slightly years the the means we can measure the effect if each school attainment of males increased more than that of was set in the same neighbourhood. females. The rise in attainment of females and males of "average" family background is shown in Figure 1. A Note of Caution Figure 1: Predicted attainment of 'average' Grampian school leaver, 1992-1994 Estimates of added value used in this study are not perfect measures of school effectiveness. The quality of our estimates depends on the quality of the measures we use. If we had measures of attainment on entry of to S1 secondary schooling, in addition to measures of family background, we could more accurately measure the boost given by schools to the attainment of each pupil between S1 and their SCE examinations. However, we know from Differences between schools research that of pupils' measures family background are correlated with S 1 attainment, By taking the average results over the three year and estimates of added value which control fully period we obtain stable estimates of the added for school intake do provide good estimates of value of schools. Figures 2 and 3 show results variation between schools. per school averaged over three years. PAGE 5 on the right of Figure 3 had above average added Figure 2: School differences before taking These significant differences between value. account of intake, 1992-1994 average suggest notable that there very schools is 3.00 achievement in some schools and scope for 2.00 further improvement in other schools. There is 1 00 no room for complacency. Nevertheless, the II I III I w differences between schools are very small in terms of the enhancement of pupil attainment by the quality of teaching and learning within each .4.00 school. Schools in order of difference in school social The effect of differences between differences shows Figure the 2 context Grampian schools in average examination results before taking account of differences in pupil There are large differences between Grampian (These differences are similar to the intake. characteristics of the family schools the in "raw" examination results used in published background of pupils attending each school, and league tables.) The horizontal line represents the pupils' inequalities serious in cause these Bars on the left Grampian average attainment. The attainment of an examination results. falling below the hand side line horizontal "average pupil" may be raised or lowered by two "raw" which schools the represent in or three Standard Grades by differences in examination results are below the Grampian school social context. Bars on the right hand side represent average. schools in which "raw" examination results are The effects of school social context were not the We found a above the Grampian average. same for all pupils. A school with a high social difference of five and a half Standard Grades in context has a greater effect in improving the the "raw" averages between schools with the whose own family of pupil attainment a highest and lowest results, before taking account background is relatively disadvantaged than of a of pupil intake. background family own whose is pupil Conversely, a school with low advantaged. Figure 3: School differences after taking account social context has a more depressing effect on of intake, 1992-1994 average the attainment of a pupil whose own family 3.00 background is relatively disadvantaged. A pupil 2.00 whose own family background relatively is advantaged is less affected by the social context of the school. .2.00 aco In this study we have not examined the process .4.00 Schools in order of difference influences context social which school by attainment. Other research has suggested that in Figure 3 shows the small difference between schools attended by pupils of advantaged family schools after taking account of pupil intake. It is favourable more pupils have backgrounds interesting to note that the ordering of schools in attitudes to academic study and that teachers There is a the two charts does not coincide. have higher expectations of their pupils. difference of just one Standard Grade in the attainment of an "average" school leaver in the The effect of neighbourhood deprivation school with the highest added value compared to the school with the lowest added value (after in the neighbourhood of the Urban deprivation taking account of differences in pupil intakes). pupil on effect detrimental school has a attainment. This is in addition to the effects of Statistical tests of significance show that the five school intake characteristics. However, although schools shown on the left of Figure 3 had below most indicators of local-area deprivation are average added value. The three schools shown PAGE 6 associated with low pupil attainment, we were therefore provides a base for future monitoring surprised to find that higher than average local by the successor authorities, and has implications unemployment rates are associated with higher for their policies. Few Grampian schools differ attainment. It may be that young people in areas from the regional average in their added value. of high unemployment feel a greater incentive to But is the regional average as high as it could their improve school and be? Is there scope for general improvement? An at stay on analogy might be: are the bottles half full or half qual ifications. empty? Consideration should now be given to how added value can be raised further. 3. Some Implications of this Report A second purpose has been to aid school self- Grampian Education Authority and its schools evaluation, development planning and have broken new ground with this study. The improvement. So far, the results for each school study is unique in Scotland and the UK, and has have been confidential to the Head Teacher of been a three-year development programme. The each school. The results have not been shared analysis has benefited from co-operation and between schools or schools. within We dialogue between Grampian Head Teachers, anticipate that now, at the culmination of the Advisers and Officers. This dialogue has led to development programme, Head Teachers will increased awareness of the assumptions and wish to share the results and discuss them with limitations of performance indicators. As well as sharing the results, we colleagues. need to share the awareness of the assumptions The majority of young people find their and limitations of the results. schooling worthwhile, but a small minority are alienated by their school experiences. The findings relating to the perceptions of school leavers of their school experiences should give differences among There substantial schools much encouragement. However, some are Grampian schools in SCE attainment, but aspects of these results will afford each school these differences are very poor indicators opportunity focus on the specific to indeed of the quality of schooling or of the improvement initiatives within the school; in improvements a school is making. particular, how can the school guard against the alienation felt by a minority of pupils? SCE differences among Grampian schools The findings regarding added value have pointed overwhelmingly reflect differences in school to the many factors that contribute to pupil intake. Pupils may attainment. suffer disadvantage because of their own family background, because There are very small differences between of the relative disadvantage of their peers and schools in their added value. In other words, because of neighbourhood also deprivation. there are very small differences between These findings raise questions about the role of schools in the quality of the contribution of schools in countering disadvantage. each school to the attainment of its pupils. We anticipate that schools these use will Most schools cannot be separated by ranking findings to identify potential under-achievement them using added value. among their pupils, and improve their ways of helping these pupils to learn. But the problem Nevertheless, significant there are needs the consideration of Grampian and its differences between some schools which successor authorities. Large differences between suggests there that room for no is schools in their social context are a major source complacency. of inequality within the region. This should form the basis for discussion of social equity and One purpose of the study has been to improve the nature of disadvantage in Grampian schools monitoring of schools by the authority. It and in the new Education Authorities. PAGE 7 About this study CES Briefings There were three annual postal surveys of This new series will provide regular information about the work of the Centre for Educational all school leavers from Education Authority Sociology. CES Briefings are designed to make schools in the region. The surveys in 1992, our research accessible to a wider audience, 1993 and 1994 related to leavers from and to enhance its value for policy and practice. 1993/4 and 1992/3 1991/2, sessions The following Briefings are also available, free The surveys described the respectively. of charge, from the CES: characteristics, experiences, attainment and Grampian of destinations post-school year-olds of "Participation 16-19 in No 1: school leavers. education and training: recent trends" by Paula Surridge and David Raffe. Tables showing many of the results of each to Head provided have been survey No 2: "Leaving home" by Gill Jones. Teachers, to enable them to compare the responses of leavers from their own school No. 3: "A Curriculum for AV" by Linda Croxford. to those for the whole of Grampian Region. Other CES Briefings will cover guidance, young Results for each school are confidential to people's incomes and other areas of our work. the school. Further Details Further Reading contact Linda Croxford, For further details McPherson, A. (1992) "Measuring Added Value Centre for Educational Sociology, Tel 0131 650 on Commission National Schools", in 4202, E-mail [email protected]. The views Education Briefing No 1, (London: National expressed are those of the author. Commission on Education). Monitoring School (1992) Willms, J.D. Performance, (Lewes: Falmer). 9 8 PAGE CES Centre for Educational Sociology The University of Edinburgh Telephone UK 0131 650 1000 7 Buccleuch Place or direct UK 0131 650 4186/7 Edinburgh EH8 9LW Fax UK 0131 668 3263 Scotland email [email protected] 10