S C IENCE (cid:127) (cid:127) Changing the planning for agile and lean software VT ONS TEC dFreovme lroopamdmenapt ping to continuous planning T SC VISI Disser tation HNO (cid:127) L IE S 132 O N T G Market uncertainties, increased competitiveness and the constant C H (cid:127)Y nreesepdo tnos isvheo artnedn addeavpetliovpem soenfttw cayrcel edse vceallol pfomr emnot raen flde pxilbanlen, ing E 13 GILHGIHHCRA E S E R 2 practices. Thus, creating a long-term future plan has become challenging for software development companies. This thesis investigates how planning has changed for agile and lean software development from roadmap-based planning towards continuous planning. This thesis provides empirical evidence of C how large and global software development companies are h a conducting planning. The empirical data were collected by n g conducting an initial inquiry consisting of both a questionnaire in g study and semi-structured interviews, and then, by conducting a t h multiple-case study involving three case companies. e p l The results show that planning practices have changed both in a n regard to their scope and schedule. The scope of planning in agile n i n and lean software development is not restricted to release planning g only; instead planning should be viewed from a wider perspective fo r involving also strategic, financial, business, and product planning. a What is more, the time frame of plans has shortened remarkably g i l from years down to months, weeks and days. The reasons for e a these changes are both internal and external, which are elaborated n d in more details in the thesis. l Changing the planning e a n for agile and lean s o f tw software development a r e . . . From roadmapping to continuous ISBN 978-951-38-8446-8 (Soft back ed.) ISBN 978-951-38-8445-1 (URL: http://www.vttresearch.com/impact/publications) planning ISSN-L 2242-119X ISSN 2242-119X (Print) ISSN 2242-1203 (Online) Tanja Suomalainen http://urn.fi/URN:ISBN:978-951-38-8445-1 VTT SCIENCE 132 Changing the planning for agile and lean software development From roadmapping to continuous planning Tanja Suomalainen Thesis for the degree of Doctor of Philosophy to be presented with due permission for public examination and criticism in auditorium IT116, at the University of Oulu, Linnanmaa, on the 9th of September 2016, at 12 noon. ISBN 978-951-38-8446-8 (Soft back ed.) ISBN 978-951-38-8445-1 (URL: http://www.vttresearch.com/impact/publications) VTT Science 132 ISSN-L 2242-119X ISSN 2242-119X (Print) ISSN 2242-1203 (Online) http://urn.fi/URN:ISBN:978-951-38-8445-1 Copyright © VTT 2016 JULKAISIJA – UTGIVARE – PUBLISHER Teknologian tutkimuskeskus VTT Oy PL 1000 (Tekniikantie 4 A, Espoo) 02044 VTT Puh. 020 722 111, faksi 020 722 7001 Teknologiska forskningscentralen VTT Ab PB 1000 (Teknikvägen 4 A, Esbo) FI-02044 VTT Tfn +358 20 722 111, telefax +358 20 722 7001 VTT Technical Research Centre of Finland Ltd P.O. Box 1000 (Tekniikantie 4 A, Espoo) FI-02044 VTT, Finland Tel. +358 20 722 111, fax +358 20 722 7001 Juvenes Print, Tampere 2016 Preface My dream of becoming a doctor of philosophy (PhD) was born many years ago. Already while writing my Master’s thesis, I realised that this is something that I wanted to do. The PhD journey has required both my passion and persistence, but there have also been a lot of people to thank for making my dream of a completing a PhD a reality. First of all, I would like to thank my principal supervisor, Professor Jouni Similä, who has over the years encouraged me to pursue the dream and believed that I would finalise this thesis one day. Thank you also for the valuable guidance and comments on the studies, papers, and the thesis itself. Secondly, I would like to thank my other supervisor, Professor Veikko Seppänen, for bringing speed and focus to the actual writing work. I am also really grateful for the endless discussions about drawing pictures and improving the content of the thesis, as well as your extremely fast replies to all of my questions. Thirdly, I want to thank Professor Pekka Abrahamsson for the conversations and advice that you have given to me along the way, which have calmed my mind in problematic situations. I would also like to thank the reviewers of this thesis, Professor Casper Lassenius of Aalto University, Finland, and Professor Pasi Tyrväinen of the University of Jyväskylä, Finland. I express my sincere thanks for the time and effort they have spent in reviewing my research and giving their constructive comments and recommendations, which have helped me to improve the quality of the thesis. I am also grateful to VTT for giving me the opportunity to work on various projects to collect the data for my thesis, as well as to write and thereafter publish the work. I am grateful to Dr Tua Huomo, who took me into the Cloud software program and gave me valuable advice at the beginning of the writing process. Also, I want to thank Dr Raija Kuusela for taking me to the N4S research programme and letting me work freely on my thesis while still believing that I could handle it on my own. I really appreciate that you have always been there for me, listening my troubles and excitement along the way and giving me guidance and support when needed. Furthermore, as this thesis is based on five papers, I wish to thank all my co-authors for their contributions. Especial thanks to colleague, co- author, friend and aunt Dr Maarit Tihinen for your friendship, help, and encouragement throughout. Also, I want to thank my longitudinal room-mate and 3 colleague Susanna Teppola for your support and putting up with me over the years. I also, I want to thank my colleague Kaisa Koskela-Huotari for helping me to organise some of the interviews from one of the case companies. Over the years, I have worked with many great people, both at VTT and in the case companies, and I wish to thank all of you. Since this thesis would not have been possible without the case companies involved, I offer all the case company representatives as well as all the interviewees who are part of this research much gratitude. Furthermore, there have been so many talented and great colleagues during my career who I have been privileged to work with, but as you are so many, it would be impossible to name you all here. I hope you will understand. Then to my loved ones, I wish to express my sincere gratitude for your loving support, understanding, and encouragement through the years. First of all, I want to thank my loving husband Lari Suomalainen just for being there for me, and loving me from the bottom of your heart. You have always tried to make my dreams come true and helped me achieve my goals. I am grateful that you understand me and my temper as well as changes in my mood so well. Then, thanks to my loveable children: Veera, Luka, and Viola, for bringing so much love and joy to my life. I also want to thank my mum and dad for their endless love and support in all areas of life. I also want to thank my mother-in-law Irma Suomalainen for taking care of my kids while I needed to work long hours, and my grandma Rauha Oikarinen for giving all the best instructions for life and providing me with the time to relax when I have needed it. Finally, I want to thank all my friends who have supported and encouraged me along the way. I want to thank my support group ‘baby greetings’, for understanding me as a woman and a mum and empathising the small and large joys and sorrows of life. In all, this would not have become true without all of you. Tanja Suomalainen Oulu, Finland, August 2016 4 Academic dissertation Supervisors Professor Jouni Similä University of Oulu Faculty of Information Technology and Electrical Engineering Department of Information Processing Science P.O. Box 3000, 90014 University of Oulu, Finland Professor Veikko Seppänen University of Oulu Martti Ahtisaari Institute, Oulu Business School, and Faculty of Information Technology and Electrical Engineering P.O. Box 3000, 90014 University of Oulu, Finland Reviewers Professor Casper Lassenius Aalto University Department of Computer Science P.O. Box 15400, 00076 Aalto, Finland Professor Pasi Tyrväinen University of Jyväskylä Department of Computer Science and Information Systems P.O. Box 35, 40014 Jyväskylä, Finland Opponent Professor Hannu Jaakkola Tampere University of Technology Department of Software Engineering P.O. Box 300, 28101 Pori, Finland 5 List of original publications This thesis is based on the following original publications which are referred to in the text as Papers I–V. The publications are reproduced with kind permission of the publishers. I Suomalainen, T., Tihinen, M., and Parviainen, P. (2009). Challenges for Product Roadmapping in Inter-company Collaboration. In: Proceedings of the Third International Conference on Software Engineering Approaches for Offshore and Outsourced Development (SEAFOOD). ETH Zurich, Switzerland on July 2–3, 2009. Springer, pp. 66–80. II Suomalainen, T., Salo, O., Abrahamsson, P., and Similä, J. (2011). Software Product Roadmapping in a Volatile Business Environment. The Journal of Systems and Software, Vol. 84, Issue 6, pp. 958–975. III Suomalainen, T., Kuusela, R., and Tihinen, M. (2015). Continuous Planning: An Important aspect of Agile and Lean Development. International Journal of Agile Systems and Management (IJASM), Vol. 8, No. 2, pp. 132–162. IV Suomalainen, T. (2015). Defining Continuous Planning through a Multiple- Case Study. In: Proceedings of the 16th International Conference of Product-Focused Software Process Improvement (PROFES). Bolzano, Italy on December 2–4, 2015. Springer, LNCS 9459, pp. 288–294. V Rodrıguez, P., Haghighatkhah, A., Lwakatare, L. E., Teppola, S, Suomalainen, T., Eskeli, J., Karvonen, T., Kuvaja, P., Verner, J. M., and Oivo, M. (2016). Continuous Deployment of Software Intensive Products and Services: A Systematic Mapping Study. The Journal of Systems and Software, in press. 6 Author’s contributions Paper I ‘Challenges for product roadmapping in inter-company collaboration’ describes product roadmapping from the inter-company collaboration perspective. Collaboration is defined as an activity in which two or more parties, e.g. companies, departments, customers, or agencies work together to create a mutual value and achieve a common goal. The paper discusses the challenges and opportunities that inter-company collaboration (that is joint R&D partnerships, customer–supplier relationships, and technology exchange agreements and licencing) sets for product roadmapping, including, for example, the most important activities to consider, the most typical problems and how those problems can be avoided. The empirical research was conducted as a questionnaire study and interviews, which were planned, carried out, and analysed by Suomalainen. She was also responsible for the planning and execution of related research and was the main author of the conference paper. Paper II ‘Software product roadmapping in a volatile business environment’ develops a framework for software product roadmapping, which is used to study the critical aspects of the product roadmapping process. The research findings presented in the paper aim at defining product roadmapping based on the current literature and empirical research, including an overview of roadmapping, stakeholders of the product roadmapping process and product roadmapping phases. The empirical research was conducted as a questionnaire study and interviews. Both the empirical and scientific part of the research were planned, executed, and analysed by Suomalainen. Suomalainen was the main author of the journal paper. Paper III ‘Continuous planning: an important aspect of agile and lean development’ defines continuous planning though a multiple-case study. The paper highlights the importance of continuous planning throughout an entire organisation, including the elements of continuous planning (that is organisational planning, strategic planning, and business planning) and their close interrelation. Organisational planning serves to define a plan’s organisational level and the time frames of a plan, strategic planning serves to set an overall plan of an organisation, and business planning serves to establish the budgeting frame of a plan. The empirical evidence of the paper is drawn from the experiences of three 7 case companies in terms of how they viewed and conducted continuous planning. The author was responsible for the planning and execution of related research. Also, the author conducted and analysed the case studies relating to cases A and B. Raija Kuusela conducted and analysed the case study relating to case C. Suomalainen was the main author of the journal paper, and Kuusela and Tihinen provided comments and feedback to the manuscript. Paper IV ‘Defining continuous planning through a multiple-case study’ presents the results of a multiple-case study in which the different levels of planning (that is strategic, financial, business, product, and team) along with their time frames are explored. The research findings presented in the paper reveal the practices of continuous planning among the case companies with their key activities. The empirical evidence of the paper is drawn from the experiences of three case companies, which were collected through several interviews. Suomalainen was responsible for planning, carrying out, and analysing the interviews. Suomalainen was also responsible for the planning and execution of the related work and she was the sole author of the conference paper. Paper V ‘Continuous deployment of software intensive products and services: a systematic mapping study’ defines the method and the main findings of a systematic mapping study about continuous deployment. The paper classified and analysed the literature related to continuous deployment in the software domain in order to scope the phenomenon, provided an overview of its state-of-the-art, investigated the scientific evidence in the reported results, and identified areas that were suitable for further research. Suomalainen was one of the participants of the research group conducting the systematic mapping study of the continuous deployment. During the analysis of the research results, factors relating to continuous deployment were divided among the researchers of the research group based on their interests and research topics. Suomalainen was responsible for the factors relating to fast and frequent release, including also continuous planning, as well as agile and lean software development in the continuous deployment context. Suomalainen was a co-author of the paper as the journal paper was written together by the research group. 8 Contents Preface .................................................................................................................. 3 Academic dissertation ......................................................................................... 5 List of original publications ................................................................................ 6 Author’s contributions ........................................................................................ 7 List of abbreviations .......................................................................................... 11 1. Introduction ................................................................................................. 13 1.1 Background and motivation .................................................................. 15 1.2 Research questions and scope ............................................................ 16 1.3 Structure of the thesis .......................................................................... 18 2. Background and related work .................................................................... 19 2.1 Agile and lean software development .................................................. 19 2.1.1 Agile software development ...................................................... 20 2.1.2 Lean software development ...................................................... 22 2.1.3 Planning in an agile–lean organisation ..................................... 23 2.2 Roadmapping ....................................................................................... 28 2.2.1 Roadmap structure ................................................................... 29 2.2.2 Roadmapping process .............................................................. 31 2.2.3 Roadmapping participants ........................................................ 32 2.3 Continuous planning ............................................................................ 34 2.3.1 Strategic planning ..................................................................... 36 2.3.2 Business and financial planning ................................................ 38 2.3.3 Portfolio planning ...................................................................... 40 2.3.4 Product planning ....................................................................... 41 2.3.5 Release planning ...................................................................... 43 2.4 Summary of the related work ............................................................... 45 3. Research design ......................................................................................... 49 3.1 Research approach .............................................................................. 49 3.2 Research methods ............................................................................... 51 9