End-User Development of Mashups Using Live Natural Language Programming Doctoral Dissertation submitted to the Faculty of Informatics of the Università della Svizzera Italiana in partial fulfillment of the requirements for the degree of Doctor of Philosophy presented by Saeed Aghaee under the supervision of Prof. Cesare Pautasso February 2014 Dissertation Committee Prof. Walter Binder Università della Svizzera Italiana, Lugano, Switzerland Prof. Marc Langheinrich Università della Svizzera Italiana, Lugano, Switzerland Prof. Mary Beth Rosson Pennsylvania State University, PA, USA Prof. Fabio Casati Università degli Studi di Trento, Trento, Italy Dissertation accepted on 24 February 2014 Research Advisor PhD Program Directors Prof. Cesare Pautasso Prof. Stefan Wolf and Prof. Igor Pivkin i I certify that except where due acknowledgement has been given, the work presented in this thesis is that of the author alone; the work has not been sub- mitted previously, in whole or in part, to qualify for any other academic award; and the content of the thesis is the result of work which has been carried out since the official commencement date of the approved research program. Saeed Aghaee Lugano, 24 February 2014 ii To my mother. . . iii iv Abstract The emergence of the long-tail in the market of software applications is shifting the role of end-users from mere consumers to becoming developers of applica- tions addressing their unique, personal, and transient needs. On the Web, a popular form of such applications is called mashup, built out of the lightweight composition of Web APIs (reusable software components delivered as a service through the Web). To enable end-users to build mashups, there is a key prob- lem that must be overcome: End-users lack programming knowledge as well as the interest to learn how to master the complex set of Web technologies re- quired to develop mashups. End-User Development (EUD) is an emerging re- search field dealing with this type of problems. Its main goal is to design tools and techniques facilitating the development of software applications by non- programmers. In this dissertation, we designed and implemented NaturalMash, an EUD system that empowers end-users to develop mashups. NaturalMash adopts a novel hybrid end-user programming technique combining natural language pro- gramming with a what-you-see-is-what-you-get interface in a live programming environment. We followed an iterative user-centered design process, in which three formative evaluations drove the incremental design of our system. At the end of the process, we conducted a summative usability evaluation, whose re- sults suggest that the system is highly usable by non-programmers. Also, we proposed a novel benchmarking framework to evaluate mashup tools against each other. Using the framework, we conducted a comparative evaluation of 28 state-of-the-art mashup tools (NaturalMash included) against their expres- sive power. According to the results, our proposed system has a moderate yet competitive level of expressiveness. All in all, NaturalMash contributes a novel design featuring a unique combination of end-user programming techniques, a suitable metaphor, and the ability to enable an optimal learning experience. Our extensive evaluation results indicate that NaturalMash is located at a sweet spot along the classical trade-off between expressiveness and usability/learnability. v vi Acknowledgements My PhD has been a pleasant and fruitful journey which brought many life-lasting knowledge, experiences, lessons, memories, friendships, and of course, a pile of code and documents including this dissertation. As I am writing this, I am miles away from those who made this journey possible. So I am taking this opportunity to profusely thank them in writing. Foremost, I would like to express my greatest gratitude and appreciation to Prof. Cesare Pautasso, from whom I learned many things. Being always avail- able, he patiently guided me through ups and downs of my PhD, and believed in me when I was lost. It was a pleasure and a privilege to do my PhD under your supervision. I am sincerely grateful to Daniele Bonetta, Marcin Nowak, Dr. Achille Pe- ternier, Masiar Babazadeh, and Vasileios Triglianos for being friends, giving con- tinuous constructive feedback on my work, proofreading my papers, and partic- ipating in my user studies. I wish you luck in your lives and hope to see you again. I would like to thank Mauro Prevostini for giving me the opportunity to con- duct a usability test on high school students, Dr. Monica Landoni for her valuable input into the design of the formative evaluation of my work, and Prof. Antonella De Angeli for hosting me in University of Trento, Italy, to properly conduct a user study of my system. I am thankful to my best friends, Ehsan, Morteza, Lea, Marco, Andrea, and Silvia, who made the beautiful and wonderful city of Lugano socially pleasant for me. Your memory lingers on. I profusely thank my parents, Akram and Hassan, and my brothers, Farid and Amir, without whom I wouldn’t be where I am today. All I ever need is you by my side, so live forever. Last but not least, I wish to thank my love of life, Negin, for her loving care and support. I am so lucky to have you in my life. vii viii