Information and Communication Technologies for Development Evaluation Written by a team of expert practitioners at the Independent Office of Eval- uation of International Fund for Agricultural Development (IFAD), this book gives an insight into the implications of new and emerging technolo- gies in development evaluation. Growing technologies such as big data analytics, machine learning and remote sensing present new opportunities for development practitioners and development evaluators, particularly when measuring indicators of the Sus- tainable Development Goals. The volume provides an overview of informa- tion and communication technologies (ICTs) in the context of evaluation, looking at the theory and practice, and discussing how the landscape may unfold. It also considers concerns about privacy, ethics and inclusion, which are crucial issues for development practitioners and evaluators working in the interests of vulnerable populations across the globe. Among the contri- butions are case studies of seven organizations using various technologies for data collection, analysis, dissemination and learning. This valuable insight into practice will be of interest to researchers, practition- ers and policymakers in development economics, development policy and ICT. Oscar A. García is the Director of the Independent Office of Evaluation (IOE) of the International Fund for Agricultural Development (IFAD). Before joining IFAD, Oscar served as the head of the advisory services at UNEP – Technology, Industry and Economics Division, Paris, providing guidance to the Partnership for Action on Green Economy. He was senior evaluation ad- visor for the United Nations Development Programme (UNDP) Evaluation Office overseeing programmatic evaluations in Africa, Asia, Latin America and the Caribbean. Mr. Garcia has more than 25 years of professional expe- rience, combining operational and managerial practice with results-based management, strategic planning and evaluation expertise. Prashanth Kotturi joined the IOE in October 2012 and is currently working as an Evaluation Analyst. Since then, he has worked in lead and support roles on a wide range of evaluations ranging from project evaluations, coun- try portfolio evaluations to corporate-level evaluations and evaluation syn- thesis. Before IOE, Prashanth has worked in the financial services industry and with microfinance institutions in his home country, India. Routledge Studies in Development Economics 142 The Service Sector and Economic Development in Africa Edited by Evelyn F. Wamboye and Peter J. Nyaronga 143 Macroeconomic Policy for Emerging Markets Lessons from Thailand Bhanupong Nidhiprabha 144 Law and Development Theory and Practice Yong-Shik Lee 145 Institutions, Technology and Development in Africa Jeffrey James 146 Urban Policy in Latin America Towards the Sustainable Development Goals? Edited by Michael Cohen, Maria Carrizosa and Margarita Gutman 147 Unlocking SME Finance in Asia Roles of Credit Rating and Credit Guarantee Schemes Edited by Naoyuki Yoshino and Farhad Taghizadeh-Hesary 148 Full and Productive Employment in Developing Economies Towards the Sustainable Development Goals Rizwanul Islam 149 Information and Communication Technologies for Development Evaluation Edited by Oscar A. García and Prashanth Kotturi For more information about this series, please visit www.routledge.com/ series/SE0266 Information and Communication Technologies for Development Evaluation Edited by Oscar A. García and Prashanth Kotturi First published 2020 by Routledge 52 Vanderbilt Avenue, New York, NY 10017 and by Routledge 2 Park Square, Milton Park, Abingdon, Oxon OX14 4RN Routledge is an imprint of the Taylor & Francis Group, an informa business © 2020 selection and editorial matter, Oscar A. García and Prashanth Kotturi; individual chapters, the contributors The right of Oscar A. García and Prashanth Kotturi to be identified as the authors of the editorial material, and of the authors for their individual chapters, has been asserted in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library Library of Congress Cataloging-in-Publication Data A catalog record has been requested for this book ISBN: 978-0-367-13714-4 (hbk) ISBN: 978-0-429-02823-6 (ebk) Typeset in Times New Roman by codeMantra Contents List of Figures, Tables and Boxes ix List of Contributors xi Acknowledgements xiii Introduction 1 OSCAR A. GARCíA AND PRASHANTH KOTTURI The Challenge of Evaluating the Sustainable Development Goals 2 ICTs with Promise for Evaluators 4 Privacy, Equity and Biases 5 ICT4Eval – The Publication 7 1 Evaluation and the Sustainable Development Goals: Opportunities and Constraints 9 MARCO SEGONE The Greatest Opportunity and the Greatest Challenge for the Global Evaluation Community 10 How Do We Assess Whether Development Interventions Are Relevant, and Are Having an Impact in Decreasing Inequality and Improving the Welfare of the Worst-Off Groups? 11 How Do We Carry Out Evaluation Given the Complexity of the SDGs? 14 How Can We Take Advantage of New Technologies to Address New Challenges? 18 How Can We Strengthen the Capacities of Governments, Civil Society Organizations, and Parliamentarians to Evaluate Whether Interventions Are Having Equitable Outcomes for Marginalized Populations? 20 Individual and Institutional Evaluation Capacities Enabled by a Supportive Environment 21 Fostering Demand for and Supply of Evaluation 23 Conclusion 24 vi Contents 2 Information and Communication Technologies for Evaluation (ICT4Eval): Theory and Practice 27 OSCAR A. GARCíA, JYRKI PULKKINEN, PRASHANTH KOTTURI ET AL. Data Collection: Faster, Cheaper, More Accurate 27 Data Analysis: The Machine Learning Revolution 29 Dissemination and Learning: Reaching a Global Audience 29 ICTs in Practice – the Case for Cautious Optimism 65 Sunk Costs vs. an Opportunity 65 Do It Yourself vs. Outsourcing 66 Mainstreaming ICTs into Operations 67 Evaluation 2.0: Turning Dilemmas to Dividends? 68 3 Big Data Analytics and Development Evaluation: Optimism and Caution 76 MICHAEL BAMBERGER Some Themes from the Big Data Literature 77 Demystifying Big Data 79 Defining Big Data and NIT 79 Big Data and Data Analytics 80 The Data Continuum 87 The NIT Ecology and the Linkages to Development Evaluation 88 Where Is the Big Data Revolution Headed? 89 Does Big Data Apply to Development Evaluation? Should Evaluators Care About It? 91 The Great Potential for Integrating Big Data into Development Evaluation 92 Big Data and Development Evaluation: The Need for Caution 96 Overcoming Barriers to Big Data Use in Evaluation 102 Future Scenarios for Development Evaluation in the Age of Big Data 104 New Skills Required for Evaluation Offices, Evaluators and Data Scientists 106 4 Technology, Biases and Ethics: Exploring the Soft Sides of Information and Communication Technologies for Evaluation (ICT4Eval) 111 LINDA RAFTREE Factors Affecting Information Technology Access and Use Among the Most Vulnerable 112 Contents vii Data and Technology Alone Cannot Ensure Inclusion 115 Inclusiveness of Access and Use Affect the Representativeness of Big Data 116 Bias in Big Data, Artificial Intelligence and Machine Learning 118 Protecting Data Subjects’ Rights in Tech-Enabled, Data-Led Exercises 121 Improving Data Privacy and Protection in the Development Sector 123 5 Technology and Its Implications for Nations and Development Partners 128 OSCAR A. GARCíA AND PRASHANTH KOTTURI Structural Transition and Pathways for Economic Development 128 Income - Who Has Technology Affected the Most? 133 A Luddite’s Nightmare or a Passing Phenomenon? 137 Geography - Implications for Sustainable Rural Development 138 Dealing with Disruptions and Moving Forward 139 Implications for Development Partners 142 Conclusions 146 OSCAR A. GARCíA AND PRASHANTH KOTTURI The Story This Far 146 Key Takeaways 148 Looking Ahead 149 Index 153 List of Figures, Tables and Boxes Figures 1.1 National average reduction in income poverty in a hypothetical country 13 1.2 Reduction in income poverty in a hypothetical country, by gender 14 1.3 The SDGs as a system of goals and targets 15 1.4 Links between the SDGs through targets: an aggregated picture 16 1.5 Links among Sustainable Development Goal 10 (inequality) and other goals 17 2.1 A spatial model represents the real world as a combination of layers or themes 31 2.2 Globally distributed GEF-supported PAs overlaid with sites of conservation importance 33 2.3 Forest change data used for global analysis, an example of a PA in Mexico 34 2.4 Economic valuation of carbon sequestrated at each GEF-supported project site 36 2.5 Geospatial analysis based on satellite data shows decreasing vegetation at the mouth of the Kagera River, Uganda 37 2.6 In a systematic review, 20% of the search delivered 80% of the evidence 48 2.7 Google search trends for the term “machine learning” 50 2.8 Nightlights classified as low (red), medium (yellow) and high (green) in northern Nigeria 58 2.9 Trained vs. test models 59 2.10 Satellite imagery and neural networks 60 2.11 Predictions at 500m by 500m resolution of food insecurity for a town in Mali, based on a 2018 Emergency Food Security Assessment 60 3.1 The components of NIT 80 3.2 The four stages of the data analytics cycle 82 3.3 The data continuum 88