Data use challenges and the potential of live data visualization tools A case study of health data-use workshops in Zambia Master’s Thesis Peter Mogaka Ogega November, 2017 Data use challenges and the potential of live data visualization tools “…data needs not only to be reviewed, it needs to be contextualized, synthesized, and it need to be prioritized.” – Zambia partner representative; on improving data use in Zambia. A case study of health data-use workshops in Zambia Peter Mogaka Ogega © Peter Mogaka Ogega 2017 Title Peter Mogaka Ogega http://www.duo.uio.no Print: Reprosentralen, University of Oslo II Abstract Zambia’s Health Management Information System (HMIS) is currently undergoing a strengthening process. One aim of the strengthening process is to improve data use. Data in Zambia´s HMIS receives limited usage for data-informed decision making. Earlier studies have identified some challenges to data use such as poor data quality, data availability etc. This thesis is an exploratory interpretive study on Zambia´s data-use workshops and how they contribute to data use in Zambia public health domain. It looked to identify how and for what purposes data is used, what are the key challenges to data use, and the prospects of introducing live data visualization tools to strengthen data-use workshops and data use. The study has involved users from all levels of Zambia´s HMIS in order to identify common thoughts and ideas on live data visualization tools; and data use at large. It mainly involved attending a data-use workshop, conducting interviews along with document analysis. The study identifies key challenges to data use as financial constraints, infrastructure constraints, poor data quality, limited data availability and accountability and lack of key data competencies by personnel. The implementation of live data visualizations tools, for the strengthening of DIMs and data use, was likely to impact both positively. However, while the study´s findings suggest that live data tools could potentially improve data use through improving data access, data availability and data quality, such measures would also be challenged by general financial and infrastructural constraints. In addition, I develop a model for the potential introduction of live data visualization tools. The model identifies reciprocal and determinant relationship between the challenges and the live data visualization tools. I II Acknowledgement All this work wouldn`t be possible on my own. I would therefore like to acknowledge the efforts of several people. First , I would like to thank my supervisor Terje Aksel Sanner for his guidance, support and light moments throughout the journey of writing this thesis . I would like to thank the Akros team and Zambian team ; for making my stay as comfortable as possible. Scott Russpatrick for informative talks, help and review, Natasha Mwelwa for being my ‘PA’ during m y visits , Albert for making sure I got the right materials for my research , Tabonga for her a ssistance, and T abo for helping me settle in well during my visits to Lusaka. I would l ike to thank my ‘study buddy’ and friend Anastasi a Parramore for the good ti mes, thoughts on various topics and the chit-chat. They were always interesting. Martine Wanjiku for helping me spellcheck , tremendous love and support . Lastly, I would like to thank my family for believing me, supporting me, and letting me make my own dec isions; including studying abroad. IV Table of content Table of Contents Data use challenges and the potential of live data visualization tools ........................................ I Abstract .................................................................................................................................... III Acknowledgement .................................................................................................................... IV Table of content ......................................................................................................................... V List of figures ........................................................................................................................ VIII List of tables ............................................................................................................................. IX Abbreviations ............................................................................................................................ X 1 INTRODUCTION .............................................................................................................. 1 1.1 Problem statement ....................................................................................................... 1 1.2 Research context .......................................................................................................... 2 1.3 Motivation .................................................................................................................... 3 1.3.1 Personal motivation .............................................................................................. 3 1.3.2 Practical motivation .............................................................................................. 3 1.3.3 Knowledge motivation ......................................................................................... 4 1.3.4 Assumptions ......................................................................................................... 4 1.4 Research Questions ...................................................................................................... 4 1.5 Thesis structure ............................................................................................................ 5 2 LITERATURE REVIEW ................................................................................................... 7 2.1 Data use in Low-resource context: data-use workshops and data visualization tools. 7 2.1.1 Data use workshops and Peer reviews ................................................................. 7 2.1.2 Scorecard .............................................................................................................. 9 2.1.3 Summary ............................................................................................................ 10 2.2 Conceptual framework ............................................................................................... 11 2.2.1 Data use and decision making ............................................................................ 11 2.2.2 District level data use and decision-making ....................................................... 14 2.2.3 Summary ............................................................................................................ 16 3 BACKGROUND .............................................................................................................. 18 3.1 Zambia overview ....................................................................................................... 18 3.2 Health System in Zambia ........................................................................................... 21 3.3 Health Information Systems Program (HISP) and District Health Information System 2 (DHIS2) ............................................................................................................................. 24 3.3.1 Health Information Systems Program ................................................................ 24 3.3.2 District Health Information System 2 ................................................................ 25 V 3.3.3 Visualization tools ............................................................................................ 26 3.3.4 DHIS2 in Zambia ............................................................................................. 28 3.4 MOH Cooperating Partners ..................................................................................... 28 3.4.1 UNICEF and Millennium Development Goals inititiative (MDGi ).................. 28 3.4.2 Akros ............................................................................................................... 30 3.5 HMIS strengthening project .................................................................................... 31 3.5.1 Quality Improvement ....................................................................................... 31 3.5.2 District Integrated Meeting .............................................................................. 32 3.6 Summary ................................................................................................................. 32 4 METHODS .................................................................................................................... 34 4.1 Research Strategy ................................................................................................... 34 4.1.1 Research approach ............................................................................................ 34 4.1.2 Reflexivity ....................................................................................................... 35 4.2 Research methods ................................................................................................... 37 4.2.1 Case study ........................................................................................................ 37 4.3 Data collection ........................................................................................................ 38 4.3.1 Access and study setting ................................................................................... 38 4.3.2 Data collection methods ................................................................................... 39 4.3.3 Sources for data collection ............................................................................... 44 4.4 Data analysis ........................................................................................................... 45 4.5 Limitations .............................................................................................................. 50 4.6 Ethical considerations .............................................................................................. 51 4.7 Summary ................................................................................................................. 52 5 EMPIRICAL FINDINGS .............................................................................................. 53 5.1 Capacity of Personnel .............................................................................................. 53 5.1.1 The participants ............................................................................................... 53 5.1.2 Computer literacy ............................................................................................. 54 5.1.3 Data presentation, review and analysis ............................................................. 54 5.1.4 DHIS2 Capability ............................................................................................. 55 5.1.5 Staffing ............................................................................................................ 56 5.2 Process/ Routine for DIMs ...................................................................................... 56 5.2.1 Standard DIM agenda/framework .................................................................... 56 5.2.2 Measure of success and Purpose of DIMs ......................................................... 58 5.2.3 District Scorecard ............................................................................................. 59 5.3 Data quality and Availability ................................................................................... 60 5.3.1 Data quality, availability, accessibility and use ................................................ 60 5.3.2 DHIS2 clean-up and Live DHIS2 .................................................................... 62 V I 5.3.3 Population data ................................................................................................... 62 5.4 Infrastructure .............................................................................................................. 63 5.4.1 Location of DIMs ............................................................................................... 63 5.4.2 Funding ............................................................................................................... 63 5.4.3 Internet connectivity and Electricity .................................................................. 64 5.5 Quality Improvement ................................................................................................. 64 5.6 Governance, Donors and Partners ............................................................................. 66 5.6.1 Funding ............................................................................................................... 67 5.6.2 Donor collaboration ............................................................................................ 68 6 RESULTS AND DISCUSSION ...................................................................................... 69 6.1 Results........................................................................................................................ 69 6.1.1 Data use .............................................................................................................. 69 6.1.2 Key challenges to data use ................................................................................. 70 6.2 Discussions ................................................................................................................ 74 6.2.1 Engaging data users and data producers ............................................................. 74 6.2.2 Improving data quality ....................................................................................... 75 6.2.3 Improving data availability ................................................................................ 76 6.2.4 Building capacity and competencies .................................................................. 77 6.2.5 Straightening organizations data demand and use infrastructure ....................... 78 6.2.6 Monitoring and evaluation, communicating data demand and use successes .... 79 6.2.7 Live data visualization tools ............................................................................... 79 6.3 Summary .................................................................................................................... 83 7 CONCLUSION AND FURTHER RESEARCH ............................................................. 85 7.1 Summary of the research process .............................................................................. 85 7.2 Reflections and conclusion ........................................................................................ 85 7.3 Further Research ........................................................................................................ 86 REFERENCES ......................................................................................................................... 88 APPPENDIX ............................................................................................................................ 92 Appendix A – DIM opening presentation ................................................................................ 93 Appendix B – Data quality reports ........................................................................................... 94 Appendix C – Informed consent .............................................................................................. 96 Appendix D – Interview guides ................................................................................................ 97 Appendix E – Rufunsa district scorecard ................................................................................. 98 Appendix F – DIM agenda ....................................................................................................... 99 V II List of figures Figure 1; Scorecard example .................................................................................................. 9 Figure 2; The conceptual framework ..................................................................................... 12 Figure 3; Map of Zambia and her neighboring countries. ................................................... 18 Figure 4; A Zambian Chief with a tablet used to track his chiefdom´s performance .............. 20 Figure 5; Lusaka province ................................................................................................... 21 Figure 6; a rural health center in Zambia ................................................................................. 22 Figure 7; a visualization of Zambia´s Health System .............................................................. 23 Figure 8; CC at work ................................................................................................................ 24 Figure 9; DHIS2 role out ..................................................................................................... 25 Figure 10; DHIS as a data warehouse .................................................................................... 26 Figure 11; dashboard example ............................................................................................... 27 Figure 12; RMNCH&N sample scorecard ............................................................................... 28 Figure 13; MDGi presence .................................................................................................. 29 Figure 14; QI committees structure .......................................................................................... 32 Figure 15; observation notes from the DIM ............................................................................. 43 Figure 16; data analysis notes and category mapping and relationships .................................. 48 Figure 17; data analysis - live data tools model relation and development ............................. 50 Figure 18; Rufunsa District´s facility presentations ................................................................. 57 Figure 19; district scorecard presentation and discussion ........................................................ 59 Figure 20; A section of Rufunsa District´s scorecard for, quarter 3 and 4 of 2016. ................ 60 Figure 21; QI project presentation ............................................................................................ 65 Figure 22; DC addressing the DIM .......................................................................................... 67 Figure 23; A model for synthesizing the potential of live data visualization tools use ........... 80 VI II List of tables Table 1; Summary of framework ............................................................................................. 17 Table 2; list of data sources ...................................................................................................... 44 Table 3: Data analysis categories, key findings and framework intervention relation ............ 49 Table 4; Participants and their designations ............................................................................. 54 Table 5; Challenges, interventions and their measures ............................................................ 72 I X