The Diana Award Anti-Bullying Ambassadors Programme A Mixed Methods Evaluation of the Programme Pooja Kumari THE DIANA AWARD 120 MOORGATE, LONDON, EC2M 6UR 2 | P age Table of Contents Executive Summary ............................................................................................................... 4 Evaluation Design ................................................................................................................ 11 Aggregate Data Findings ..................................................................................................... 16 The September Cohort ........................................................................................................ 27 Findings from School Y, Primary School, Cheltenham .................................................. 32 Findings from School X, Middle School, Bedfordshire .................................................. 43 Findings from School Z, Secondary School, Wigan ........................................................ 54 Conclusion ............................................................................................................................. 65 APPENDIX I – KPIs ................................................................................................................ 68 APPENDIX II – Weighted Averages ................................................................................... 69 APPENDIX III – School Y Research Diary ......................................................................... 70 APPENDIX IV – Quantitative Data Collection .................................................................. 74 APPENDIX V – Qualitative Data Collection ...................................................................... 89 Bibliography .......................................................................................................................... 92 ACKNOWLEDGEMENTS The researcher would like to extend her sincere gratitude to all of the research participants who helped to conduct this evaluation, and her colleagues who assisted with proof-reading the final write-up. Throughout this study the following abbreviations will be used: ABA – Anti-Bullying Ambassador SABL – Staff Anti-Bullying Lead NON-ABA – Student who is not an Anti-Bullying Ambassador KPIs – Key Progress Indicators DfE – Department for Education SLT – Senior Leadership Team PSHE – Personal Social Health Education N – Sample size 3 | P age Executive Summary Introduction In 2011, The Diana Award Anti-Bullying Campaign launched the Anti-Bullying Ambassadors Programme. In 2013 the programme received funding from the Department for Education (DfE). The programme aims to equip students and staff with a holistic and peer-led approach to address bullying by providing young people, parents, and teachers with information about bullying and the tools to empower them to become advocates for change in relation to bullying behaviour within their schools. After receiving our training, students become Anti-Bullying Ambassadors (ABAs) in their schools. Throughout their role they help re-educate their peers on the facts behind bullying, lead on anti-bullying campaigns, promote a culture which celebrates and tolerates difference and help keep their peers safe both online and offline. Research Aims The research aimed to investigate the impact of the programme on the whole school community, from the students leading the programme, to the students benefitting from the programme, and any changes in the ‘culture’ of the school including impact on bullying incidences. Alongside this, in order to better understand impact of the programme, the research aimed to uncover the processes involved in change. Methodology A mixed methods approach is the most appropriate method of evaluating the impact of this programme. This allows the researcher to combine elements of qualitative and quantitative data collection and analysis for the purposes of ‘breadth and depth of understanding’ and corroboration (Johnson et al., 2007). This will allow the quantitative and qualitative approaches to triangulate findings and provide a comprehensive account of the impact of the programme (Bryman, 2009). The main elements of the research were:  Initial ‘ABA Pre-Training’ and ‘SABL Pre-Training’ surveys which are completed by all SABLs and ABAs prior to attending the training event.  ‘Training feedback’ surveys conducted with all ABAs and SABLs– administered immediately after the training events.  ‘ABA follow-up’ surveys administered to ABAs who were trained in this bid, in March 2016. 217 ABAs completed this survey.  ‘Staff Anti-Bullying Lead (SABL) follow-up’ survey administered to SABLs trained, in this bid in March 2016. 108 SABLs completed this survey.  Case study research with three schools who have not been trained previously – one primary, secondary and middle school entailing mixed methods research. This involved: 4 | P age o Pre and post year group surveys o Pre and post staff surveys o Pre and post semi-structured interviews with trained teachers o Pre and post semi-structured interviews with ABAs o Pre and post focus groups with NON-ABAs o Semi-structured interviews with one student who sought the help of an ABA Key Findings The key findings revealed that The Diana Award Anti-Bullying Ambassadors Programme has either reached or exceeded all of the KPIs (outlined in the appendix and hereafter referred to in italics) as agreed by the DfE Grant Funding Agreement 2015-16. The need The initial surveys conducted with ABAs and SABLs confirm the need for the programme. The majority (83%) of SABLs confirmed that bullying was ‘a slight’ problem in their schools and a fair proportion of ABAs confirmed experiencing bullying (60%). Of those ABAs who did not take action when they witnessed bullying, 38% claimed they were afraid, 23% claimed they ‘didn’t know what to do or who to talk to’, and 21% claimed it wasn’t ‘any of their business’. These findings illustrate the necessity of the programme in its potential for raising awareness of support networks available to students, and encouraging bystanders to take action when witnessing bullying. Training Feedback The training events were extremely popular amongst both teachers and ABAs. The ‘training feedback’ surveys completed by both ABAs and teachers – along with case study research – revealed that the training helped to equip them with the right knowledge, skills, and practical guidance in running anti-bullying campaigns in their schools.  100% of teachers claimed that the training event helped them to understand the signs of bullying and support students being bullied Similar levels of ABAs expressed the effectiveness of the training event:  97% claimed the training helped them to know how to run an Anti-Bullying Campaign well.  98% claimed the training equipped them with confidence to help people being bullied.  97% claimed the training equipped them with confidence to carry out Anti- Bullying tasks in school. The training also helped both teachers and ABAs to feel empowered to take action within their contexts. For example, 98% of ABAs and 100% of teachers reported that they felt more confident about making a difference in their school. These findings exceed the KPI targets 3, 4, 5, and 6 as training has helped to equip students and staff members with the relevant skills in order to tackle bullying. 5 | P age Implementing the scheme The case study schools implemented the scheme in very different ways. As a result, the level of knowledge about the presence of the ABAs and take-up of their services differed across the three case study schools. The aggregate case study surveys across year groups revealed that a high proportion of the sample were aware that the ABAs were active (86%) and students understood how to contact them (72%). Under a third of the sample (26%) had actually contacted the ABAs when witnessing or experiencing bullying. Although this represented a good proportion of students contacting the ABAs given that the ABAs were only introduced three months prior to the second round of data collection, it is clear that some schools need to consider ways of building trust between the student body and the ABAs. These findings confirm that the schools who have been trained are more engaged in effective Anti-Bullying work as highlighted by KPI target 2. Impact on bullying All data sources reveal that the Anti-Bullying Ambassadors Programme is having an impact on reducing the incidences of bullying taking place in the school. This research estimated that the Anti-Bullying Ambassadors Programme on average results in a 69% decrease in perceived bullying incidents in school. This figure is calculated by taking the weighted averages (see Appendix II) of the respondents who perceived reduced incidences of bullying since the introduction of the ABAs.  The ‘SABL follow-up’ survey revealed that 40% of SABLs believe that the Anti- Bullying Ambassador Programme has resulted in a decrease in incidences of bullying behaviour in the school.  The aggregate case study findings reveal that 58% of staff members across all three case study schools believe that the ABAs’ work has led to a reduction of bullying incidences.  Aggregate case study figures also reveal that 77% of year groups surveyed across all three case study schools believe that since the introduction of the ABAs, less bullying takes place in school.  On the other hand, the ‘ABA follow-up’ survey revealed that 78% of ABAs believe that their work has reduced bullying in the school. This contributes to reaching KPI target 7 which shows that young people along with ABAs believe that anti-bullying initiatives are having an impact on bullying. Qualitative research from case studies also confirm these findings: Bullying has gone down a lot since they’ve [ABAs] been introduced. Every day there used to be bullying. They’ve sorted out every problem some people have had, which makes bullying go down. NON-ABA, Male, School Y. Impact on ABAs Participation in the Anti-Bullying Ambassadors Programme has a clear impact on the ABAs in their personal development and the development of their soft skills. The most 6 | P age prominent themes emerging from the research is the development of ABAs confidence, problem-solving, and empathy skills.  97% of ABAs surveyed revealed that they feel more confident in helping other students tackle their bullying issues. Both teachers and ABAs confirmed these findings in the case study research. It is clear that through this role the ABAs developed important communication and problem- solving skills. 86% of ABAs highlighted their confidence and 65% highlighted that their perseverance skills had been developed through the role. Impact on Teachers It is clear that the Anti-Bullying Ambassadors Programme positively affected both SABLs running the programme and the staff body of the school. As explained above, almost every teacher agreed that the training provided them with greater knowledge and understanding of bullying. The SABL follow-up survey revealed that the majority of SABLs agreed they were:  88-89% are more confident in preventing, identifying, and addressing bullying; this exceeds KPI target 3. Furthermore the case study research highlighted the general consensus amongst SABLs that this role helped to develop their confidence in carrying out anti-bullying duties (KPI target 3) The Anti-Bullying Ambassador Programme also had a positive impact on the staff body as 58% of staff members across three case study schools believed that their confidence in preventing and tackling bullying has increased since the introduction of the ABAs. Impact on School Community This evaluation revealed the role of the Anti-Bullying Ambassadors Programme in contributing to the wider school life. Not only do ABAs provide the student body with a valuable service in addressing bullying, the programme also fostered positive peer relationships amongst the rest of the student body. For example, the ‘SABL follow-up’ survey revealed that:  63% of SABLs believed that ABAs encourage bystanders to stop bullying; revealing the positive impact ABAs are having on behaviour (KPI target 7)  80% of SABLs believed that ABAs have made bullying a comfortable subject to talk about; revealing the increased engagement in effective anti-bullying work (KPI target 2)  Furthermore, 50% of the SABLs expressed that as part of the anti-bullying work done in their school, school anti-bullying policies were reviewed (KPI target 2)  ‘ABA follow-up’ surveys revealed that 86% of ABAs believe that their anti-bullying work has helped other students feel safe in school and 85% of ABAs expressed that their anti-bullying work has helped to encourage other students to be kinder in their schools. This confirms KPI target 7 as ABAs believe their work has had an impact on safety, bullying, and behaviour. 7 | P age Recommendations This research highlighted the value of the Anti-Bullying Ambassadors Programme in contributing to reducing incidences of bullying, improving teacher confidence, and building a positive school environment. It is clear that there is scope to improve and strengthen this work further through continuing the programme with the following recommendations.  Anti-Bullying Ambassadors Programme should help support the ABAs build trust amongst the student body One finding which emerged from both quantitative and qualitative data is that although the ABAs are highly visible around school, there appeared to be a disconnect between this and the take-up of the ABAs’ services. High levels of visibility of the ABAs is very important around the school but the ABAs should also aim to build trust amongst the students in order to encourage more students to report experiencing or witnessing bullying. The Anti-Bullying Ambassadors Programme should consider ways of providing this support both online and offline.  Anti-Bullying Ambassadors Programme should provide targeted support to different types of schools through the Aftercare Process. This research revealed that the context in which the programme is translated into practice differs depending on the type of school. For example, as School Y is a primary school and as a result is smaller and has more resources at its disposal, perhaps the programme was able to launch successfully and establish itself. However the SABL commented that this happened successfully in the ‘Juniors’ section of the school and not the ‘Infants’. She expressed a desire to launch the programme in the ‘Infants’ section but alluded to the difficulty in doing this without any resources and support. The Diana Award Anti-Bullying Ambassadors Programme trains students from the age of eight onwards, therefore providing online guidance on how to talk about bullying with younger students could help primary school SABLs to translate effective anti-bullying work in the earlier years of the primary school. In contrast, the SABL in School Z (a secondary school) explained that the pressure around time and resources hindered the establishment of the programme in the school. The Anti-Bullying Ambassadors Programme should consider publishing some guidance and ‘top tips’ online, perhaps before the training event, in order to allow teachers to secure the resources and staff time necessary for leading the programme effectively.  Supporting SABLs to run the programme successfully The energy, enthusiasm, and time of the SABLs are very important for ensuring the longevity of the programme in schools. It is clear that the role they play is indeed a demanding one. The programme is already supporting teachers through 8 | P age organising ‘Regional Staff Champion Events’ where schools have been inviting local schools to showcase and share good Anti-Bullying practice, however other forms of support could also help to ease the demands on the SABL. For example, The Diana Award Anti-Bullying Ambassadors Programme could provide ongoing support via its social media channels or through webinars to help share advice and overcome common barriers faced by SABLs. Having support and buy-in from the staff body and SLT is very important for the establishment of the programme. The Anti-Bullying Ambassadors Programme could support SABLs by providing templates and guidance on presenting ‘the case’ for SLT and staff members to support the programme. All three SABLs expressed a high amount of enthusiasm, dedication, and commitment for the programme. However, they also believed that they weren’t doing enough to support the programme, and that they could always be doing more. The Anti-Bullying Ambassadors Programme could provide guidance for these SABLs in sharing the responsibility for the programme with another teacher.  The Diana Award Anti-Bullying Ambassadors Programme should support a longer time-frame for the evaluation. There was a 3-4 month difference between the data collection pre-and-post training. It is clear that for some case study schools, this did not provide sufficient time to fully establish the programme in their schools. It is important to track the process of the programme in establishing itself in the school as it will provide The Diana Award Anti-Bullying Programme with fresh insight into the best ways of implementing the scheme. 9 | P age Introduction The Anti-Bullying Ambassadors Programme is run by The Diana Award and it aims to equip students and staff with a holistic and peer-led approach to address bullying. The programme provides young people, parents, and their teachers with information about bullying to empower them to become advocates for change in relation to bullying behaviour within their schools. This is provided through regional training days and personalised visits where schools are trained alongside other schools in the local area. Ongoing support is also offered through opportunities to showcase the school’s work, providing resources, follow-up training, communication through feedback appointments, and networking opportunities with other schools. Programme Context Research has revealed that bullying can negatively affect students irrespective of their age and background (Tattum and Lane, 1989; Besage, 1989; Oliver and Candappa, 2003). Elliott and Kilpatrick (1994) have identified loss of self-confidence, truancy, lower levels of attainment, anxiety, and attempted suicide as side-effects of bullying. The negative long-term consequences of childhood bullying also result in depression, social isolation, and lower levels of self-esteem in adulthood (Olweus, 1991; Rivers, 1996; Elliot and Kilpatrick, 1994). Increasing attention is shifting away from individual protagonists in bullying to the wider peer group as perpetrators seldom act alone but are usually supported by their immediate group of ‘assistants’ (who help) and ‘reinforcers’ (who encourage), and ‘bystanders’ (who do nothing) (Cowie, 2014). Bullying episodes often escalate due to the inaction of bystanders (Padgett & Notar, 2014). The rationale behind peer-led initiatives in addressing bullying is rooted in the idea that bullying issues originate from peer groups thus pupils may be ‘well placed’ to help other pupil solve their problems (Cowie and Wallace, 2000). Furthermore, peers communicate with each other in the ‘vernacular’, absent of the authoritarian tone which teachers may exude (Topping, 1996). Therefore, peer-led anti-bullying programmes in schools “extend protectiveness friendship group and help to develop a school community founded on concern for others and empathy for others’ feelings” (Cowie, 2014). Successful peer-led anti-bullying programmes have demonstrated to students that school is a caring place to be (Naylor and Cowie, 1999) and teachers have reported improvements in school climate (Smith and Watson, 2004). The Diana Award Anti-Bullying Ambassadors Programme The Diana Award Anti-Bullying Ambassadors Programme was set up in 2011 and received funding from the Department for Education in 2013. In total the programme has trained 18,500 young people in over 4,000 schools and in the current funding year (April 2014-2015) the programme has trained over 3,147 young people in 273 schools. 10 | P age