Aalto University School of Science Degree Programme in Computer Science and Engineering Taufik Akbar Sitompul Usability and User Experience Evalu- ation of EUDAT Services Use Case: Aalto Data Repository Master’s Thesis Espoo, September 30th, 2016 Supervisor: Assoc. Prof. Keijo Heljanko, Aalto University Advisors: Assoc. Prof. Antonella De Angeli, University of Trento Altti Ilari Maarala, M.Sc. (Tech.), Aalto University Aalto University School of Science ABSTRACT OF Degree Programme in Computer Science and Engineering MASTER’S THESIS Author: Taufik Akbar Sitompul Title: Usability and User Experience Evaluation of EUDAT Services Use Case: Aalto Data Repository Date: September 30th, 2016 Pages: 159 Major: Service Design and Engineering Code: T-110 Supervisor: Assoc. Prof. Keijo Heljanko Advisors: Assoc. Prof. Antonella De Angeli Altti Ilari Maarala, M.Sc. (Tech.) The amount of generated research data is growing exponentially. Following this trend, many universities and research institutes require their researchers to plan research data management before commencing a new research project. Research data is a valuable product of research process. Without proper management, the value of research data cannot be utilized properly. As the largest university in Finland in the field of engineering, Aalto University has defined its research data management policy. The policy aims to make re- search data management easier and it requires Aalto University to find the most suitable platforms to be used for its research data management. Currently Aalto University has four available options for its research data management platform and EUDAT services are considered as one of the best candidate for Aalto Uni- versity research data management platform. EUDAT is a pan European project which offers common data services and it is funded by European Union. However, it is still unclear how useful EUDAT services would be for researchers in Aalto University. This thesis evaluates the current state of usability and user experience of four EUDAT services: B2DROP, B2SHARE, B2FIND, and B2ACCESS. In addition to that, this thesis also evaluates the functionalities that are available on those services. There are two usability evaluation methods that are used to evaluate EUDAT services: feature inspection and usability testing. The results from those twoevaluationsshowthatB2DROPandB2FINDhavefewusabilityproblemsand the available functionalities are good enough for researchers in Aalto University. On the other hand, the results show that B2SHARE and B2ACCESS need to be improved significantly before it can be widely used by researchers in Aalto University. This mainly caused by existing usability problems with significant impact that need to be fixed first. Keywords: usability, user experience, EUDAT, research data, common data services, data management Language: English 2 Aalto-yliopisto Perustieteiden korkeakoulu DIPLOMITYO¨N Tietotekniikan koulutusohjelma TIIVISTELMA¨ Tekij¨a: Taufik Akbar Sitompul Ty¨on nimi: EUDAT-palvelujen k¨aytett¨avyyden ja k¨aytt¨okokemusten arviointi K¨aytt¨otapaus: Aalto-yliopiston datavarasto P¨aiv¨ays: 30. syyskuuta 2016 Sivum¨a¨ar¨a: 159 P¨a¨aaine: Service Design and Engineering Koodi: T-110 Valvoja: Professori Keijo Heljanko Ohjaajat: Professori Antonella De Angeli Diplomi-insin¨o¨ori Altti Ilari Maarala Luodun tutkimusdatan m¨a¨ar¨a kasvaa eksponentiaalisesti. T¨at¨a suuntausta seura- ten monet yliopistot ja tutkimuslaitokset vaativat tutkijoitaan suunnittelemaan tutkimusaineiston hallintaa ennen uuden tutkimuksen aloittamista. Tutkimusda- ta on arvokas tutkimusprosessin tuote. Ilman asianmukaista datan hallintaa t¨at¨a arvokasta dataa ei voida kunnolla hy¨odynt¨a¨a. Aalto-yliopisto on Suomen suurimpana insin¨o¨oritieteiden yliopistona linjannut tutkimusdatapolitiikkansa, jonka tarkoituksena on tehd¨a tutkimusdatan hallin- nasta helpompaa. T¨am¨a my¨os edellytt¨a¨a sen, ett¨a Aalto-yliopiston on l¨oydett¨av¨a sopivimmat alustat tutkimusdatan hallintaan. T¨all¨a hetkell¨a Aalto-yliopistolla on nelj¨a vaihtoehtoa datan hallinnan alustaksi ja EUDAT-palveluja pidet¨a¨an yh- ten¨a parhaimpana vaihtoehtoina Aalto-yliopiston tutkimusdatan hallintaan. EU- DATonyleiseurooppalainenprojekti,jokatarjoaayhteisi¨adatapalveluitajaseon Euroopan Unionin rahoittama. Viel¨a on kuitenkin ep¨aselv¨a¨a, kuinka hy¨odyllisi¨a EUDAT-palvelut olisivat Aalto-yliopiston tutkijoille. T¨am¨a tutkielma arvioi nelj¨an eri EUDAT-palvelun t¨am¨anhetkist¨a k¨aytett¨avyytt¨a ja niiden k¨aytt¨okokemuksia: B2DROP, B2SHARE, B2FIND ja B2ACCESS. Lis¨aksi tutkielma arvioi kyseisten palvelujen toiminnallisuutta. Kahta arvioin- timenetelm¨a¨a k¨aytet¨a¨an arvioimaan EUDAT-palveluja: ominaisuuksien tarkas- telu sek¨a k¨aytett¨avyyden testaus. N¨aiden kahden arvioinnin tulokset osoittavat, ett¨a B2DROP ja B2FIND -palveluilla on vain v¨ah¨an k¨aytett¨avyysongelmia ja nii- den toiminnallisuudet ovat tarpeeksi hyvi¨a Aalto-yliopiston tutkijoille. Tulokset osoittavat kuitenkin B2SHARE:n ja B2ACCESS:n tarvitsevan viel¨a merkitt¨av¨a¨a kehitt¨amist¨a ennen kuin Aalto-yliopiston tutkijat voivat k¨aytt¨a¨a niit¨a laajalti. Syyn¨a t¨ah¨an on p¨a¨aasiassa nykyiset k¨aytt¨oongelmat, joilla on merkitt¨avi¨a vaiku- tuksia ja jotka t¨aytyy ensin korjata. Asiasanat: k¨aytett¨avyys, k¨aytt¨okokemukset, EUDAT, tutkimusdata, yh- teiset datapalvelut, datan hallinta Kieli: Englanti 3 Acknowledgements This thesis would not have been completed without the contribution of many people. I would like to use this opportunity to express my gratitude to those who have directly or indirectly involved in completion of this master’s thesis. First of all, I would like to thank my supervisor, Assoc. Prof. Keijo Hel- janko for giving me an opportunity to work with a big project like EUDAT and supervising this thesis. Working on a big project like EUDAT has been a long and exhausting, but really rewarding experience for me. Special thanks to my two advisors, Assoc. Prof. Antonella De Angeli for her guidance on usability and user experience and Altti Ilari Maarala for proof-reading this thesis for many times. Thank you very much to thirteen researchers and six students of Aalto University who have participated as test users and pilot users, respectively. ManythankstodevelopersofEUDATfortheircooperationduringthesepast seven months. I also would like to thank my two colleagues, Farouk Salem and Asad Javed for teaching me to write this thesis in LaTeX. Last but not least, I would like to thank my family for their unconditional and infinite support during my master studies. Espoo, September 30th, 2016 Taufik Akbar Sitompul 4 Abbreviations EUDAT EuropeanAssociationforDatabasesinEducationand Training ACRIS Aalto Current Research Information System CC Creative Commons MIT Massachusetts Institute of Technology CDI Collaborative Data Infrastructure API Application Programming Interface CKAN Comprehensive Kerbal Archive Network CESSDA ConsortiumofEuropeanSocialScienceDataArchives CLARIN Common Language Resources and Technology Infras- tructure ENES European Network for Earth System GUI Graphical User Interface HCI Human-Computer Interaction ISO International Organization for Standardization UEM Usability Evaluation Method WebDAV Web Distributed Authoring and Versioning SQL Structured Query Language LAMP Linux, Apache, MySQL, and PHP CSC Center for Science OAI-PMH Open Archives Initiative Protocol for Metadata Har- vesting REST Representational State Transfer PID Persistent Identifier EPIC European Persistent Identifier Consortium XML Extensible Markup Language JSON JavaScript Object Notation SAML Security Assertion Markup Language LDAP Lightweight Directory Access Protocol SUS System Usability Scale 5 CSUQ Computer System Usability Questionnaire QUIS Questionnaire for User Interaction Satisfaction SCI School of Science ENG School of Engineering ARTS School of Arts, Design, and Architecture HIIT Helsinki Institute for Information Technology 6 Contents Acknowledgements 4 Abbreviations and Acronyms 5 1 Introduction 10 1.1 Aalto University research data management policy . . . . . . . 11 1.2 Introduction to EUDAT services . . . . . . . . . . . . . . . . . 14 1.3 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . 17 1.4 Thesis scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.5 Thesis structure . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2 Background 19 2.1 Introduction to usability and user experience . . . . . . . . . . 19 2.2 Usability evaluation methods . . . . . . . . . . . . . . . . . . 21 2.2.1 Usability testing . . . . . . . . . . . . . . . . . . . . . 21 2.2.2 Usability inspection . . . . . . . . . . . . . . . . . . . . 33 2.3 Combining usability evaluation methods . . . . . . . . . . . . 40 3 EUDAT services 42 3.1 B2FIND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.2 B2DROP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.3 B2SHARE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 3.4 B2ACCESS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4 Conducting usability evaluation 53 4.1 Selected usability evaluation methods . . . . . . . . . . . . . . 53 4.2 Feature inspection . . . . . . . . . . . . . . . . . . . . . . . . 53 4.3 Usability testing . . . . . . . . . . . . . . . . . . . . . . . . . . 55 4.3.1 Test plan . . . . . . . . . . . . . . . . . . . . . . . . . 55 4.3.2 Test users . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.3.3 Task scenario . . . . . . . . . . . . . . . . . . . . . . . 57 7 4.3.4 Test materials . . . . . . . . . . . . . . . . . . . . . . . 58 4.3.5 Pilot test . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.3.6 Test site and equipment . . . . . . . . . . . . . . . . . 60 4.3.7 Real Usability Test . . . . . . . . . . . . . . . . . . . . 61 4.3.8 Data analysis . . . . . . . . . . . . . . . . . . . . . . . 62 5 Results 64 5.1 Overall results . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 5.1.1 Simple and natural dialogue . . . . . . . . . . . . . . . 67 5.1.2 Speak the user’s language . . . . . . . . . . . . . . . . 67 5.1.3 Minimize the user’s memory load . . . . . . . . . . . . 69 5.1.4 Be consistent . . . . . . . . . . . . . . . . . . . . . . . 69 5.1.5 Provide feedback . . . . . . . . . . . . . . . . . . . . . 70 5.1.6 Provide clearly marked exits . . . . . . . . . . . . . . . 70 5.1.7 Provide shortcuts . . . . . . . . . . . . . . . . . . . . . 71 5.1.8 Provide good error messages . . . . . . . . . . . . . . . 72 5.1.9 Error prevention . . . . . . . . . . . . . . . . . . . . . 72 5.2 Results for B2FIND . . . . . . . . . . . . . . . . . . . . . . . . 73 5.3 Results for B2DROP . . . . . . . . . . . . . . . . . . . . . . . 78 5.4 Results for B2SHARE . . . . . . . . . . . . . . . . . . . . . . 82 5.5 Results for B2ACCESS . . . . . . . . . . . . . . . . . . . . . . 85 5.6 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 6 Conclusions 90 Bibliography 99 A Test Script 100 A.1 General introduction . . . . . . . . . . . . . . . . . . . . . . . 100 A.2 Introduction and background questionnaire for each EUDAT service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 B Test users’ profile 103 C List of test tasks for the test users 106 C.1 List of test tasks to be completed on B2FIND . . . . . . . . . 106 C.2 List of test tasks to be completed on B2DROP . . . . . . . . . 107 C.3 List of test tasks to be completed on B2SHARE . . . . . . . . 108 D Post-task questionnaires 109 D.1 Modified System Usability Scale (SUS) . . . . . . . . . . . . . 109 D.2 Modified Computer System Usability Questionnaire (CSUQ) . 110 8 D.3 Modified Questionnaire for User Interaction Satisfaction (QUIS)111 E List of problems and suggestions 113 E.1 B2FIND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 E.1.1 List of problems found on B2FIND . . . . . . . . . . . 114 E.1.2 List of proposed suggestions for B2FIND . . . . . . . . 115 E.2 B2DROP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 E.2.1 List of problems found on B2DROP . . . . . . . . . . . 118 E.2.2 List of proposed suggestions for B2DROP . . . . . . . 119 E.3 BSHARE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 E.3.1 List of problems found on B2SHARE . . . . . . . . . . 125 E.3.2 List of proposed suggestions for B2SHARE . . . . . . . 128 E.4 B2ACCESS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 E.4.1 List of problems found on B2ACCESS . . . . . . . . . 134 E.4.2 List of proposed suggestions for B2ACCESS . . . . . . 135 F Results from post-task questionnaires 136 F.1 Results from System Usability Scale (SUS) . . . . . . . . . . . 136 F.2 ResultsfromComputerSystemUsabilityQuestionnaire(CSUQ)142 F.3 Results from Questionnaire for User Interaction Satisfaction (QUIS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 9 Chapter 1 Introduction With millions of researchers and scientists pursuing their research across the globe, the amount of generated research data is growing exponentially1. Fol- lowing this trend, many universities and research institutes require their re- searcherstohavearesearchdatamanagementplanbeforecommencinganew research project2. Evidence of good practices of research data management has become an expected requirement for granting research funding3. Re- search data management is the procedure that covers research data lifecycle, includingcreating, storing, delivering, maintaining, archiving, andpreserving research data4. To have a better understanding of research data management, researchers also need to understand the nature of research data. Boston University stated that ”research data is data that is collected, observed, created for purposes of analysis to produce original research results”5. Research data can be in physical or digital format and it also comes in various types, such as text, image, audio, video, specimens, samples, and so on. Depending on the research purposes, research data can be classified into four categories, such as observational data, experimental data, simulation data, and derived or compiled data6. Different category and type of research data may require different research data management plan. The advancement of technology revolutionizes the way researchers cap- ture, store, manage, analyse, and visualize research data, thus making re- 1http://researchanalytics.thomsonreuters.com/m/pdfs/1003903-1.pdf 2https://blogs.ntu.edu.sg/lib-datamanagement/introduction 3http://www.dcc.ac.uk/events/workshops/introduction-research-data-management 4http://datalib.edina.ac.uk/mantra/datamanagementplans 5http://www.bu.edu/datamanagement/background/whatisdata 6https://www.nottingham.ac.uk/research/research-data-management/what-is- research-data.aspx 10