A W O R L D B A N K S T U D Y Learning in the Face of Adversity THE UNRWA EDUCATION PROGRAM FOR PALESTINE REFUGEES Husein Abdul-Hamid, Harry Anthony Patrinos, Joel Reyes, Jo Kelcey, and Andrea Diaz Varela Learning in the Face of Adversity A WORLD BANK STUDY Learning in the Face of Adversity The UNRWA Education Program for Palestine Refugees Husein Abdul-Hamid, Harry Anthony Patrinos, Joel Reyes, Jo Kelcey, and Andrea Diaz Varela © 2016 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW, Washington, DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org Some rights reserved 1 2 3 4 18 17 16 15 World Bank Studies are published to communicate the results of the Bank’s work to the development com- munity with the least possible delay. The manuscript of this paper therefore has not been prepared in accordance with the procedures appropriate to formally edited texts. This work is a product of the staff of The World Bank with external contributions. The findings, inter- pretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be considered to be a limitation upon or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. Rights and Permissions This work is available under the Creative Commons Attribution 3.0 IGO license (CC BY 3.0 IGO) http:// creativecommons.org/licenses/by/3.0/igo. Under the Creative Commons Attribution license, you are free to copy, distribute, transmit, and adapt this work, including for commercial purposes, under the following conditions: Attribution—Please cite the work as follows: Abdul-Hamid, Husein, Harry Anthony Patrinos, Joel Reyes, Jo Kelcey, and Andrea Diaz Varela. 2016. Learning in the Face of Adversity: The UNRWA Education Program for Palestine Refugees. World Bank Studies. Washington, DC: World Bank. doi:10.1596/978-1-4648-0706-0. License: Creative Commons Attribution CC BY 3.0 IGO. Translations—If you create a translation of this work, please add the following disclaimer along with the attribution: This translation was not created by The World Bank and should not be considered an official World Bank translation. The World Bank shall not be liable for any content or error in this translation. Adaptations—If you create an adaptation of this work, please add the following disclaimer along with the attribution: This is an adaptation of an original work by The World Bank. Views and opinions expressed in the adaptation are the sole responsibility of the author or authors of the adaptation and are not endorsed by The World Bank. Third-party content—The World Bank does not necessarily own each component of the content contained within the work. The World Bank therefore does not warrant that the use of any third-party–owned individual component or part contained in the work will not infringe on the rights of those third parties. The risk of claims resulting from such infringement rests solely with you. If you wish to reuse a compo- nent of the work, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright owner. Examples of components can include, but are not lim- ited to, tables, figures, or images. All queries on rights and licenses should be addressed to the Publishing and Knowledge Division, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@ worldbank.org. ISBN (paper): 978-1-4648-0706-0 ISBN (electronic): 978-1-4648-0707-7 DOI: 10.1596/978-1-4648-0706-0 Cover photo: UNRWA Photo Gallery. Used with permission; further permission required for reuse. Cover design: Debra Naylor, Naylor Design, Inc. Library of Congress Cataloging-in-Publication Data has been requested. Learning in the Face of Adversity • http://dx.doi.org/10.1596/978-1-4648-0706-0 Contents Preface ix Foreword xi Acknowledgments xiii Executive Summary: The UNRWA Story xv Abbreviations xxix Chapter 1 The UNRWA Effect 1 Introduction 1 A Brief Introduction to UNRWA: Evidence from Test Scores 2 UNRWA Schools Continually and Consistently Outperform Public Schools by a Margin Equivalent to More than One Additional Year of Learning 3 At Baseline (without Controls) UNRWA Schools Increase Student Performance 4 Performance of UNRWA Students Is Higher than Their Peers despite Two Important Characteristics 7 Do UNRWA Schools Increase Student Performance? 8 Notes 16 Chapter 2 Supporting Teachers and Managing Schools 17 The Systems Approach 17 UNRWA Teachers Are Better Supported 17 School Autonomy and Accountability Represent Areas for Improvement for Both 22 UNRWA Schools Promote High-Quality Teaching and Classroom Time 24 Conclusion 31 Notes 31 Chapter 3 Learning and Resilience 33 Introduction 33 Education Has a Collective Meaning and Purpose for Palestine Refugees in the Face of Adversity 34 Learning in the Face of Adversity • http://dx.doi.org/10.1596/978-1-4648-0706-0 v vi Contents UNRWA Staff Understand and Are Able to Model a Positive Identity and Well-Being in the Midst of the Same Challenges Their Students Face 37 The School System Supports Students’ Competence in the Midst of Adversity through Academic Guidance and Socioemotional Sup port 38 Learning Is Supported by Many Actors Including Teachers, Students, Peers, and Family Members 42 UNRWA’s Close Partnership with the Refugee Community Creates Shared Accountability for Learning Outcomes 44 What Is Resilience in the UNRWA Education System? 45 Relevance of Findings 47 Notes 49 Appendix A Methodology 51 SABER 51 Stallings Classroom Observation 52 Qualitative Data Approach 53 Methodological Limitations 56 Notes 56 References 59 Boxes 1.1 Who Are the UNRWA Palestine Refugees? 1 1.2 Data Modeling Method Used in the Study 4 1.3 The Model 14 1.4 Oaxaca-Blinder 15 2.1 Systems Approach for Better Education Results (SABER) 17 2.2 What Matters in School Autonomy and Accountability? 23 2.3 Stallings Classroom Observation 25 3.1 Qualitative Data Approach 33 A.1 Interviewee Demographic Data 54 Figures ES.1 Sequential Mixed Methods Methodology Design xvi ES.2 Parents Are More Involved in UNRWA Schools xviii ES.3 Teachers at UNRWA Schools Are More Satisfied with Their Jobs xviii ES.4 Advantage in Test Scores for UNRWA over Public Schools, Controlling for Stated Factors xix ES.5 Cost per Student in Jordan’s UNRWA Schools and Public Schools xxx ES.6 More Interaction and Less Lecturing in UNRWA Schools xxiii Learning in the Face of Adversity • http://dx.doi.org/10.1596/978-1-4648-0706-0 Contents vii ES.7 Students Benefit from a Wide Range of Support That Helps Them Learn despite the Adversity (Noting the Number of Times a Topic Was Mentioned in Student Interviews) xxvi 1.1 Comparison of TIMSS Scores over Time 5 1.2 Comparison of PISA Scores between Public and UNRWA Schools for Jordan 6 1.3 UNRWA Advantage: Magnitude of the Difference in Mean Scores Relative to Public in NAfKE (Jordan) 6 1.4 Predicted Mean Score Difference between UNRWA Schools and Public Schools, Controlling for General School Characteristics 7 1.5 Explained and Unexplained Portions of TIMSS Scores Based on Regression Decomposition 15 2.1 Metrics for Classifying Students 18 2.2 International Comparisons in Teaching Time 25 2.3 Difference in Activities between Public and UNRWA Classrooms 26 2.4 Important Difference Regarding the Use of Materials by School Type 27 2.5 Percentage of Classes in Which Students Work on Measurable Activities 28 2.6 Percentage of Classes with Quizzes or Test Preparation 28 2.7 Incidence of Students Being Off-Task 29 2.8 Rating of Teacher Competency by Region and Subject 30 2.9 Overall Lesson Quality by Region and Subject 30 3.1 How Students Make Sense and Find Purpose through Education 35 3.2 The Type of Goals High-Performing Students Relate with Doing Well in School (Disaggregated by Gender and Noting the Number of Times It Was Mentioned in Student Interviews) 36 3.3 Types of Academic Guidance Provided to Students (Noting the Number of Times a Topic Was Mentioned in Stud ent Interviews) 40 3.4 Different Sources of Support on Learning Outcomes as Perceived by UNRWA Students (Noting the Number of Times It Was Mentioned in Student Interviews) 42 3.5 The Five Education Resilience Supportive Mechanisms Identified in UNRWA Schools 47 3.6 The Different Ways in Which Schools Support the Resilience Process (Noting the Number of Times It Was Mentioned in Student Interviews) 48 Tables ES.1 Comparison between UNRWA and Public Systems Policies in School Management and Autonomy xxi 1.1 Student Background Characteristics, Based on TIMSS 2007 Data 5 1.2 Comparison of Time Use and Level of Parental Support and Involvement 8 Learning in the Face of Adversity • http://dx.doi.org/10.1596/978-1-4648-0706-0 viii Contents 1.3 Difference in Performance between High and Low Values in Characteristics Related to Teacher and School 9 1.4 Results of the Regression Models (Jordan) 10 1.5 Results of Regression Models in the West Bank and Gaza 11 1.6 Comparison of Test Scores for TIMSS Controlling for Other Factors Using PSM 13 1.7 Comparison of Test Scores for PISA Controlling for Other Factors and Using PSM (Jordan) 13 1.8 Comparison of Test Scores for NAfKE Using PSM (Jordan) 14 2.1 Recruiting and Attracting High-Quality Teachers, UNRWA and Public Schools Comparison 19 2.2 Teacher Expectations and Guidance, UNRWA and Public Schools Comparison 20 2.3 Teacher Motivation, Accountability, and Performance, UNRWA and Public Schools Comparison 20 2.4 Minimum Training Requirements for Teachers, UNRWA and Public Schools Comparison 21 2.5 Opportunities for Teacher Career Development, UNRWA and Public Schools Comparison 21 2.6 Quality of School Principals, UNRWA and Public Schools Comparison 22 2.7 Management Layers and Accountability for Student Outcomes, UNRWA and Public Schools Comparison 24 A.1 Teacher Time Categorization 53 Learning in the Face of Adversity • http://dx.doi.org/10.1596/978-1-4648-0706-0