G L B AL P A R T N E R S H I P S Shared Objectives ● Joint Action ● Real Impact G L B AL P A R T N E R S H I P S Shared Objectives ● Joint Action ● Real Impact G L B AL P A R T N E R S H I P S Shared Objectives ● Joint Action ● Real Impact Independent External Evaluation In d e p e n d e n t E x t e r n a l E v a lu a t io n • J u n e 2 0 1 1 June 2011 Japan J a p a n A d m in is Administered Account for te re d A c c Selected IMF Activities (JSA) o u n t fo r S Office of Technical Assistance Management e le c te 700 19th Street NW d Washington, DC 20431 IM F USA A Tel.: 1-202-623-7646 ctiv Fax: 1-202-623-7106 itie Email: [email protected] s (J S I n t e r n a t i o n a l M o n e t a r y F u n d A ) Evaluation of the Japanese Technical Assistance Subaccount 2011 Independent Evaluation Final report Client: International Monetary Fund Rotterdam, June 29, 2011 Evaluation of the Japanese Technical Assistance Subaccount Client: International Monetary Fund Ferry Philipsen Murray Petrie Piero Ugolini Rotterdam, June 29, 2011 Evaluation of the Japanese Technical Assistance Subaccount 2 Evaluation of the Japanese Technical Assistance Subaccount Table of contents List of abbreviations 7  Executive Summary 11  Preface 21  1  Background and introduction 23  1.1 Background 23  1.2 Objectives of the evaluation 23  1.3 Evaluation criteria and evaluation questions 24  1.3.1  OECD DAC criteria 24  1.3.2  Accountability criteria 24  1.4 Approach and methodology 25  2  Japanese ODA policies and JSA 27  2.1 Overview of Japanese ODA policies 27  2.2 Japanese Subaccount 29  2.2.1  Overview 29  2.2.2  JSA-funded projects 32  2.2.3  The TAIMS Guidance Note 36  2.3 JSA Subaccount in the context of TA policy of IMF 38  3  Relevance 40  3.1 Evaluation questions 40  3.2 Relevance of FAD projects 40  3.3 Relevance of MCM projects 42  3.4 Relevance of STA projects 43  3.5 Relevance of INS and LEG projects 45  3.6 Overall assessment of Relevance 45  4  Effectiveness 47  4.1 Evaluation questions 47  4.2 Effectiveness of FAD projects 47  4.2.1  FAD’s Project Management Framework 47  4.2.2  Assessment of the effectiveness of FAD projects 51  4.3 Effectiveness of MCM projects 56  4.3.1  MCM’s Project Management Framework 56  4.3.2  Assessment of the effectiveness of MCM projects 58  4.4 Effectiveness of STA projects 61  4.4.1  STA’s Project Management Framework 61  4.4.2  Assessment of the effectiveness of STA projects 63  4.5 Effectiveness of INS and LEG projects 68  4.6 Overall assessment of Effectiveness 71  5  Sustainability 72  5.1 Evaluation questions 72  Evaluation of the Japanese Technical Assistance Subaccount 3 5.2 Sustainability of FAD projects 72  5.3 Sustainability of MCM projects 74  5.4 Sustainability of STA projects 75  5.5 Sustainability of INS and LEG projects 78  5.6 Overall assessment of Sustainability 80  6  Efficiency 81  6.1 Evaluation questions 81  6.2 Cost-effectiveness of the JSA projects 81  6.3 Efficiency of FAD projects 83  6.4 Efficiency of MCM projects 85  6.5 Efficiency of STA projects 86  6.6 Efficiency of INS and LEG projects 87  6.7 Overall assessment of Efficiency 88  7  Successful versus less successful JSA projects in the countries visited 89  7.1 Field visits 89  7.2 FAD projects 90  7.3 MCM projects 93  7.4 STA projects 96  8  Accountability 103  8.1 Consistency 103  8.1.1  Evaluation questions 103  8.1.2  Assessment of Consistency 103  8.2 Visibility 105  8.2.1  Evaluation questions 105  8.2.2  Assessment of Visibility 105  8.3 Support for IMF TA versus other avenues for providing development assistance107  8.3.1  Evaluation question 107  8.3.2  Assessment of Support for IMF TA 107  8.4 Management of JSA-funded activities 108  8.4.1  Evaluation questions 108  8.4.2  Assessment of Management of JSA-funded activities 108  9  Overall assessment of JSA-funded projects 111  9.1 Overall Assessment of JSA against OECD DAC criteria 111  9.1.1  Overall assessment of JSA-funded TA projects 111  9.1.2  Summary assessment of FAD projects 112  9.1.3  Summary assessment of MCM projects 113  9.1.4  Summary assessment of STA projects 114  9.1.5  Summary assessment of INS and LEG projects 115  9.2 Overall Assessment of JSA against Accountability criteria 117  10  Recent efforts to strengthen TA project management 120  10.1  General developments 120  10.2  Fiscal Affairs Department 121  4 Evaluation of the Japanese Technical Assistance Subaccount 10.3  Monetary and Capital Markets Department 121  10.4  Statistics Department 122  11  Recommendations 125  Annex A.1: Evaluation approach and methodology 135  Annex A.2: References 141  Annex A.3: List of meetings 145  A.3.1: List of meetings in Washington, D.C. 145  A.3.2: List of meetings in Cambodia 147  A.3.3: List of meetings in Nepal 148  A.3.4: List of meetings in the Philippines 148  A.3.5: List of meetings in Fiji 149  A.3.6: List of meetings in Lebanon 149  Annex A.4: Overview of JSA-funded projects 150  Annex A.5: JSA-funded projects and their ratings 154  Annex A.6: Questionnaires and responses 172  Evaluation of the Japanese Technical Assistance Subaccount 5 List of abbreviations ADB Asian Development Bank AFRITAC Africa Regional Technical Assistance Center AML Anti-Money Laundering APD Asia and Pacific Department ASYCUDA Automated System for Customs Data BCEAO Banque Central des Etats de l’ Ouest BCP Basel Core Principles BEAC Banque des Etats de l’ Afrique BoP Balance of Payments BSP Bangko Sentral ng Pilipinas (Philippines Central Bank) CAMC Central American Monetary Council CARTAC Caribbean Regional Technical Assistance Centre CDIS Coordinated Direct Investment Survey CMU Cash Management Unit CPI Consumer price Index CPM CAMC’s Monetary Policy Committee CV Curriculum Vitae DAB Da Afghanistan Bank ROSC Report on the Observance of Standards and Codes DCS Depository Corporations Survey DFID Department for International Development DSA Daily Subsistence Allowance EBC European Central Bank EC European Commission EMI European Monetary Institute EU European Union EP Evaluation Program FAD Fiscal Affairs Department FCGO Financial Comptroller General's Office FSAP Financial Sector Assessment Program FY Fiscal Year GDDS General Data Dissemination System GDP Gross Domestic product GS Government Statistician HIPC Heavily Indebted Poor Countries HMFS Harmonized Monetary and Financial Statistics HQ Headquarters ICP International Comparison Program IDF International Development Fund IFS International Financial Statistics IFMIS Integrated Financial Management Information System Evaluation of the Japanese Technical Assistance Subaccount 7