(Draft Report) Final Evaluation: ‘Conservation Education & Awareness Component of Mountain Areas Conservancy Project Consultant: Raja Muhammad Zarif Conservation Education & Awareness component of MACP TABLE OF CONTENTS Page Acknowledgement List of Acronyms Executive Summary ………………………………………………………… 5 Introduction ………………………………………………………… 6 1. Background ………………………………………………………… 6 2. WWF-P Task ………………………………………………………… 6 Evaluation Scope ………………………………………………………… 6 1. Objectives ………………………………………………………… 6 2. Methodology ……………………………………………………… 7 3. Analytical Approach ……………………………………………… 8 General Evaluation ………………………………………………………… 9 In-depth Evaluation Output 2 …………………………………………… 14 Evaluation Findings ………………………………………………………… 18 EC Support Project ………………………………………………………… 19 Analysis ………………………………………………………… 24 Emerging Lessons ………………………………………………………… 30 Conclusion Drawn ………………………………………………………… 32 Proposed Improvement in Implementing Strategy …………………… 33 Final Evaluation Report 1 Conservation Education & Awareness component of MACP ACKNOWLEDGEMENTS Consultant wishes to express appreciation to Energy & Environment Unit – UNDP Country Office Islamabad and IUNC - Pakistan – The World Conservation Union who entrusted and facilitated to undertake this evaluation study. Thanks are also extended to the National Project Manager, Mountain Areas Conservancy Project (MACP) who facilitated and shared his experiences and views that helped in the finalization of this document. The unforgettable cooperation extended by the IUCN, MACP and WWF-P Conservancy area staff, community leaders, members, villagers, teachers, students, youth and religious leaders during severe climatic condition is highly appreciated. Under prevailing circumstances and time constraints and without their support, this report would have been an uphill task. This report might not have been possible had there not been the unwavering and resolute support, guidance, and encouragement so graciously afforded by Dr. Ghulam Akbar, Director Environmental Education of WWF-Pakistan at Islamabad. Last but not the least, the support received from all those who have participated in administrative, logistic, travel, provision of literature, field discussions and observations, sharing of their experiences and views, and facilitation in any manner to embark on and finalize this study is highly appreciated and acknowledged. Final Evaluation Report 2 Conservation Education & Awareness component of MACP LIST OF ACRONYMS AKRSP Aga Khan Rural Support Program AKES, P Aga Khan Education Service, Pakistan APS Army Public School ACC Area Coordination Committee AFP Adventure Foundation Pakistan AMPS Al Mustafa Public Schools AKS Aga Khan School ANPW Annual Networking Planning Workshop BACIP Building and Construction Improvement Program BEAJ Baltistan Association of Environmental Journalists CCS Chitral Conservation Strategy CEC Commission of European Communities CMC Conservancy Management Community CEE Coordinator Environmental Education CIC Conservation Information Center CCHAs Community Controlled Hunting Areas CBO Community Based Organization DCC District Conservation Community DoE Directorate of Education HERP Hunza Education Resource Project HEC Hunza Education Committee ICDP Integrated Conservation & Development Project IUCN World Conservation Union LFA Log frame analysis LEAD Leadership for Environment and Development MT Master Trainers MACP Mountain Areas Conservancy Project MELGRD Ministry of Environment, Local government & Rural Dev. MoU Memorandum of Understanding NAs Northern Areas NWFP North West Frontier Province NRM Natural Resource Management NACS Northern Areas for Conservation Strategy NGO Non government organizations OHP Over Head Projector Final Evaluation Report 3 Conservation Education & Awareness component of MACP PAC Project Advisory Committee PCP Participatory Conservation Planning PRIF Pre Investment Feasibility PSC Project Steering Committee PDCN Professional Development Center North PDCC Professional Development Center Chitral PRA Participatory Rural Appraisal RNA Resource Need Assessment RPM Regional Project Manager SEE Society for Environmental Education SEC Serena Environmental Committee SMC School Management Committee ToP Terms of Partnership UNDP United Nations Development Program VCC Valley Conservation Committee VCP Valley Conservation Plan WCC Women Conservation Committee WED World Environment Day DFID Department for International Development EED Environmental Education Division EEP Environmental Education Programme EE Environment Education EEMS Environmental Education Model School EC European Commission EPA Environmental Protection Agency E&A Education and Awareness FIR First Investigation Report FES Fuel Efficient Stove GEF Global Environmental Facility GED Government Education Department GoP Government of Pakistan Final Evaluation Report 4 Conservation Education & Awareness component of MACP EXECUTIVE SUMMARY Conservation, Education, and Awareness component of MACP is a “non- ending” process that tends to educate and train key players and masses in environmental education with lasting impacts such that to manage the natural resources of the project areas in a sustainable manner. WWF–Pakistan has substantially contributed to build the knowledge and capacity of teachers, students, youth, community committee members, and religious leaders, to undertake activities related to conservation of biodiversity in the project areas. This has been effectively done by establishing village conservation committees, and by designing and implementing culturally appropriate strategies and involving women in the decision-making processes, besides making the concerned interest groups aware of the benefits of conservation and sustainable usage of biodiversity components in the on-going community development schemes. The overall evaluation rating of the performance of WWF-Pakistan was found “Good” in the perspective of accomplishing the demands of the mandate under OUTPUT 2. The overall impact of activities undertaken by the WWF- Pakistan on OUTPUTS 1,4, and 5 of the MACP documents was also adjudged as “Good”. These conclusions are drawn on the premise that WWF-P performed well in building the capacity of institutions and individuals at the community-based organizations to conserve biodiversity in sustainable manner and by concentrating on agencies and communities to pool-up financial and human resources for eco-development in conservancies. As for the evaluation of EC Support Project, the overall performance of WWF-P was assessed as “Very good” in terms of training and education material building but “Excellent” in capacity buildings of staff, departments and partners and involvement of schools and youth. Final Evaluation Report 5 Conservation Education & Awareness component of MACP INTRODUCTION Background An initiative of the United Nations Development Programme (UNDP), the Global Environmental Facility (GEF) and the Government of Pakistan (GoP), the Mountain Areas Conservation Project (MACP) was intended to empower the local stakeholder communities to use the natural resources in a sustainable manner without compromising the integrity of the natural environment. The World Wide Fund for Nature-Pakistan (WWF-P) is co-implementing one of the project components pertaining to education and awareness of the local communities under Output-2. The co-financing of WWF-P under EC project dealt with ancillary issues such as waste disposal, littering, pollution, soil erosion and management of community wood plots in addition to provision of support for the development of literacy training materials with conservation themes. WWF-P Task To accomplish the objectives of Output-2, WWF-P was entrusted to:  develop and implement a communication strategy;  implement a “Schools in Conservation” program;  operationalise informal awareness programmes; and  ensure the support of local mullacracy in the conservation and sustainable usage of biodiversity WWF-P requested UNDP and GEF - the principal financing agencies - at the end of five-year period for final evaluation of the Conservation Education and Awareness Component of MACP logically termed as Output-2 along with European Union (EU) funded project titled “Environmental Education Program in Support of MACP” - shortly termed as EC Support Project - as co-financing to meet the MACP objectives. EVALUATION SCOPE Objectives The final evaluation was meant to critically review all the systems put in place and analyze the favorable and unfavorable conditions and their implications on Final Evaluation Report 6 Conservation Education & Awareness component of MACP the effective and efficient implementation of the project keeping in view the capacity of implementing agencies and support given by the partners and stakeholders. As such, following are decisive objectives of the evaluation:  To asses the progress of Conservation Education Component and EC project vis-à-vis the outputs as enunciated in the project documents  To asses their effectiveness towards achievement of the project objectives and goals i.e. support to MACP in Component No. 1, 4 and 5  To identify gaps in implementation of the project activities  To asses sustainability of the project (i.e. the likelihood of the project interventions continuing after donor support discontinues)  To determine ways and means to improve the project design with special focus on the content and delivery of environmental education and awareness in mountain communities as a replication strategy Methodology Evaluation has long been important for funding agencies to assess the extent of actual change against stated objectives, and thus to judge whether or not development assistance has been successfully used. This can be done, by involving external experts evaluating against indicators that have been determined externally or through rigid, imposed monitoring procedures. Yet changes are afoot and organizations are increasingly using M&E for internal learning and continual improvement to their work. Evaluation of the Output-2 involves the assessment of change through processes that engross many people or groups, each of whom is affecting or affected by the impacts being assessed. It is a challenging process as different stakeholders examine their assumptions about what constitutes progress – and together deal with the contradictions and conflicts that can emerge. For managers of funded work, evaluation stimulates a shift from assessing impacts based on given perceptions of benefits to include the perceptions of the target population. Evaluation can, therefore, provide more comprehensive information on efficiency, relevance, sustainability, impact and effectiveness of work in progress. By learning from mistakes en route, it can lead to timely corrective action. By highlighting the successes of people’s efforts, it can increase motivation. The systematic and continual exchange of information can also strengthen working relationships. As the effectiveness of evaluation is based on sharing information, it requires careful identification of those who should share Final Evaluation Report 7 Conservation Education & Awareness component of MACP information and what information is worthwhile sharing. The following procedure was adopted for evaluation methodology: 1. Literature Review: As a first step and before undertaking field work, the review of the project documents and related literature was made to gain in depth knowledge about the theoretical aspects of approaches/ methodologies adopted for the implementation MACP. 2. Meeting Stakeholders: As second step meetings were held with members of valley conservation committees, social and religious leaders, villagers, allied departments, and non-governmental organization working in the area to have their independent views about the activities undertaken by the project. 3. Field Observation: The third step was impact assessment of activities undertaken by WWF-P through Participatory Rural Appraisal (PRA) approach to seek opinion of trained teachers, students, village youth, members of local institutions, field staff, project supervisors, line departments, and local NGOs. 4. Physical Verification: In the fourth step, the selected project areas were visited along with the WWF-P staff to study project level activities to gain information regarding implementation of Output-2 dealing with education and awareness of targeted communities. This evaluation report is based on the assessment of key parameters of four areas: (i) relevance and appropriateness; (ii) effectiveness; (iii) sustainability; (iv) efficiency; (v) impacts; and (vi) design improvement. Analytical Approach The analytical part of the report is based on: a) Effectiveness 1. Extent of achievement of the project objective 2. Extent of communities’ understanding and adopting E&A concepts 3. Actual and planned project benefits 4. Unforeseen benefits & results 5. Adequacy of project activities to realize objectives b) Relevance 1. Alignment of E&A objectives with MACP’s objectives 2. Changes, if any, in directions to better reflect E&A needs Final Evaluation Report 8 Conservation Education & Awareness component of MACP c) Sustainability 1. Extent of E&A support likely to continue project activities 2. Willingness of involved parties to continue E&A activities on their own d) Efficiency 1. Extent of cost-effectiveness of ways and means adopted for E&A 2. Appropriateness of step taken in each phase 3. Extent of appropriateness of interventions and methods used for E&A e) Impacts 1. For a discussions about conservation issues 2. Support to enhancement of MACP “Conservancy” concept 3. Apparent change in community’s perception 4. Local media coverage 5. Communities exercising sustainable practices 6. Reduction in fuel wood pressure on natural resources 7. Improvement in outlook of socio-economic conditions in the area 8. Reduction in hunting/poaching cases 9. Religious leaders support 10. Youth, school, villagers etc. participation in conservation activities 11. Activeness of institutions in Environmental Education (EE) 12. Institutionalization of EE at school level 13. Mechanism developed to integrate EE with school curricula f) Design Improvement 1. Better ways to improve project objectives 2. Improvement of mechanism to integrate EE with school curricula Evaluation Rating The Output-2 component of the MACP was evaluated on five-point rating scale (1=Excellent, 2=Very Good, 3=Good, 4=Satisfactory, and 5=Poor) based on quantitative and qualitative performance judgment of evaluator and target groups in evaluation process. The rating was developed for consistency while the evaluation parameters included: (a) Achievement of objectives and planned results (b) Attainment of outputs and activities (c) Implementation approach (d) Stakeholders involvement (e) Impacts (f) Monitoring and evaluation Final Evaluation Report 9