The relationship between agile systems development methodologies and software process improvement models C Malambo 22639578 Dissertation submitted in partial fulfilment of the requirements for the degree Magister Scientiae in Computer Science at the Potchefstroom Campus of the North-West University Supervisor: Prof HM Huisman 11 May 2017 School of Natural Sciences NWU (PUK) ACKNOWLEDGEMENTS Firstly, I would like to thank The Almighty Lord for the guidance and wisdom bestowed upon me. It is through His Will and Grace that all things are possible, and I am a living testimony. Glory be to God. Secondly, I want to extend my gratitude to my family for always believing, and giving their unconditional support through the worst of times and the best of times. It is through their prayers, and at times harsh but honest reminders, that I have been able to accomplish my goals in life. I wish to also extend a warm “thank you” to my partner for the love and encouragement. To my closest friends, thank you for always driving each other to achieve beyond our expectations. A special “thank you” goes to my supervisor, Professor Magda Huisman, who not only guided me as a supervisor but took on the role of a mother, scolding me whenever I strayed off course. Baie dankie Prof, for the Whatsapp messages (strong language) - a stern reminder of the opportunity I had. The Scrum meetings we had in your office have paid dividends. You had the patience and belief in my abilities. I also want to thank The National Research Foundation for the financial assistance rendered, to afford my studies. To my manager and colleagues, “thank you” for the support given at all times. To everyone else that had an input, I wish to express a sincere “thank you” for all the support throughout my studies. This dissertation is dedicated to the Malambo clan. i School of Natural Sciences NWU (PUK) ABSTRACT Background: Agile systems development methodologies (ASDMs) have become more frequently deployed to develop software products at a faster pace as compared to their traditional counterparts. The nature of their agility and underlying principles and practices ensures that the software product is of a quality standard. Software Process Improvement Models (SPIMs) aid organizations in improving the process of software development and ultimately improve software quality. However, the relationship (or lack of) between ASDMs and SPIMs within software- developing organizations is little known. A comprehensive understanding of these two methodologies may assist organisations in producing better-quality software, enhanced information technology project management, and help with devising an organized software development framework. Objectives: The primary objective of this study is to investigate the relationship between ASDMs and SPIMs. The co-existence of these two aspects of the study may provide an understanding of the relationship, and results could be utilised as a guide for organisations to determine the best combination between process models and system development methodologies that can be implemented to achieve the highest possible level of maturity. Methods: The research was conducted by first reviewing existing literature from books, accredited journal entries and other sources of literature. A survey with the aid of a questionnaire was performed between June 2014 and December 2015. In total, 100 questionnaires were collected, and statistical analysis was performed on the data. Results: The study identified five agile methodologies and three different software-process improvement models. Each was subjected to an in-depth literature review. In addition, data concerning the use of agile methodologies and software process models was ii School of Natural Sciences NWU (PUK) collected from information technology professionals. The questionnaire gathered data on the respondent‟s job category, their organisations‟ sizes, and the outcomes of the last project in which they had participated. Conclusion: The study addressed the industry use of ASDMs and SPIMs, and the interrelations between them. Literature suggests that these two can co-exist and when used together could present greater benefit than when implemented in isolation. CMMi was the most combined process model, with various agile methods such as XP and Scrum. However, in practice, the situation seemed different. The results showed that combinations were rare. Key Terms: Agile systems development methodologies, relationship, Software Process Improvement models, survey, organisation, Process, Procedure, Project success. iii School of Natural Sciences NWU (PUK) TABLE OF CONTENTS LIST OF TABLES .................................................................................................................... XI LIST OF FIGURES ................................................................................................................. XIII ABBREVIATIONS .................................................................................................................. XIV CHAPTER 1 .............................................................................................................................. 1 INTRODUCTION ....................................................................................................................... 1 1.1 INTRODUCTION .................................................................................................... 1 1.2 BACKGROUND OF THE STUDY ......................................................................... 1 1.3 PROBLEM STATEMENT ...................................................................................... 2 1.4 RESEARCH OBJECTIVES ................................................................................... 3 1.5 RESEARCH METHODOLOGY.............................................................................. 4 1.6 EXPECTED RESEARCH CONTRIBUTION .......................................................... 5 1.7 CHAPTER OUTLINE ............................................................................................. 6 1.8 SUMMARY ............................................................................................................. 8 CHAPTER 2 .............................................................................................................................. 9 AGILE SYSTEMS DEVELOPMENT METHODOLOGIES (ASDMS) AND ................................ 9 SOFTWARE PROCESS IMPROVEMENT MODELS (SPIMS) .................................................. 9 2.1 LITERATURE REVIEW: INTRODUCTION ........................................................... 9 2.2 SYSTEMS DEVELOPMENT METHODOLOGIES ................................................ 9 2.2.1 Definition of a Systems Development Methodology .................................................. 10 2.2.2 Definition of Agile Systems Development Methodologies ........................................ 11 2.2.3 Agile Systems Development Methodologies .............................................................. 11 2.2.3.1 Agile Manifesto ............................................................................................................... 12 2.2.3.2 History on the Agile Manifesto ..................................................................................... 13 2.2.3.3 Principles of the Agile Manifesto ................................................................................. 13 2.2.4 Types of ASDMs ............................................................................................................. 14 iv School of Natural Sciences NWU (PUK) 2.2.4.1 Extreme Programming .................................................................................................. 15 2.2.4.1.1 Use of XP ............................................................................................................. 18 2.2.4.1.2 Strengths and weaknesses of XP ..................................................................... 18 2.2.4.2 Feature Driven Development ....................................................................................... 19 2.2.4.2.1 Use of FDD .......................................................................................................... 22 2.2.4.2.2 Strengths and weaknesses of FDD .................................................................. 22 2.2.4.3 Dynamic Systems Development Methodology .......................................................... 23 2.2.4.3.1 Uses of DSDM ..................................................................................................... 26 2.2.4.3.2 Strengths and Weaknesses of DSDM .............................................................. 27 2.2.4.4 Lean Software Development (LSD) ............................................................................ 27 2.2.4.4.1 Use of LSD ........................................................................................................... 30 2.2.4.4.2 Strengths and Weaknesses of LSD .................................................................. 31 2.2.4.5 Scrum ............................................................................................................................... 31 2.2.4.5.1 Use of Scrum ....................................................................................................... 33 2.2.4.5.2 Strengths and Weaknesses of Scrum .............................................................. 33 2.2.5 Comparison of agile and traditional methodologies................................................... 34 2.2.5.1 Comparison based on systems development approaches...................................... 34 2.2.5.2 Comparison based on systems development process model ................................ 35 2.2.5.3 Comparison based on systems development methods ........................................... 36 2.2.5.4 Comparison based on systems development techniques ....................................... 38 2.2.6 Summary .......................................................................................................................... 39 2.3 SOFTWARE PROCESS IMPROVEMENT MODELS (SPIMs) ........................... 39 2.3.1 Introduction ...................................................................................................................... 39 2.3.2 Capability Maturity Model Integration (CMMi) ............................................................ 41 2.3.2.1 CMMi Architecture Overview ........................................................................................ 41 2.3.2.2 CMMi: Maturity levels .................................................................................................... 42 2.3.2.3 CMMi Criticism ............................................................................................................... 46 v School of Natural Sciences NWU (PUK) 2.3.3 Software Process Improvement and Capability Determination (SPICE)................ 47 2.3.3.1 Process Dimension ........................................................................................................ 47 2.3.3.2 Capability dimension ..................................................................................................... 48 2.3.3.3 SPICE Criticism .............................................................................................................. 49 2.3.4 International Organization for Standardization (ISO) 9000-3 ................................... 50 2.3.4.1 ISO 9000-3 Guidelines/Clauses .................................................................................. 51 2.3.4.2 ISO 9000-3 Guideline Criticism ................................................................................... 53 2.3.5 Comparison of Software Process Improvement Frameworks ................................. 54 2.3.5.1 Comparing CMMi and ISO/IEC15504 SPICE ........................................................... 55 2.3.5.2 CMMi or SPICE .............................................................................................................. 57 2.3.6 Interrelations between ASDMs and SPIMs ................................................................. 57 2.3.6.1 CMMi vs. Agile Methods ............................................................................................... 58 2.3.7 Summary .......................................................................................................................... 62 CHAPTER 3 ............................................................................................................................ 63 RESEARCH METHODOLOGY AND DESIGN ........................................................................ 63 3.1 INTRODUCTION .................................................................................................. 63 3.2 RESEARCH PARADIGMS .................................................................................. 64 3.2.1 Positivist Research ......................................................................................................... 65 3.2.1.1 Characteristics of the positivist approach ................................................................... 66 3.2.1.2 Criticisms of Positivism ................................................................................................. 67 3.3 RESEARCH STRATEGIES ................................................................................. 67 3.3.1 Survey research strategy ............................................................................................... 68 3.3.1.1 Advantages of using the Survey research strategy .................................................. 69 3.3.1.2 Disadvantages of using the Survey research strategy ............................................. 69 3.4 DATA COLLECTION METHODS USED IN THIS STUDY ................................. 69 3.4.1 Questionnaires ................................................................................................................ 70 vi School of Natural Sciences NWU (PUK) 3.4.1.1 Type of Questionnaires ................................................................................................. 70 3.4.1.2 Advantages of Questionnaires ..................................................................................... 70 3.4.1.3 Disadvantages of Questionnaires ............................................................................... 71 3.4.2 Questionnaire design ..................................................................................................... 72 3.4.2.1 Section A: Organisation background information ..................................................... 72 3.4.2.2 Section B: Systems development methodologies used ........................................... 73 3.4.2.3 Section C: Software process improvement models .................................................. 75 3.4.2.4 Section D: Project outcome .......................................................................................... 76 3.4.3 The application of questionnaire strategies in the study ........................................... 76 3.5 DATA-ANALYSIS METHODS USED IN THE STUDY ....................................... 77 3.6 DATA-ANALYSIS TOOLS ................................................................................... 78 3.6.1 SPSS version 16.0 .......................................................................................................... 78 3.7 SUMMARY ........................................................................................................... 79 CHAPTER 4 ............................................................................................................................ 80 RESEARCH RESULTS ........................................................................................................... 80 4.1 INTRODUCTION .................................................................................................. 80 4.2 RESEARCH AIMS AND OBJECTIVES .............................................................. 81 4.3 SECTION A: Organisation Background Information ...................................... 81 4.3.1 Job category .................................................................................................................... 81 4.3.2 Core business area ......................................................................................................... 84 4.3.3 Organizational size ......................................................................................................... 85 4.3.4 Information systems departmental size ....................................................................... 86 4.3.5 IS investment in new applications ................................................................................ 87 4.3.6 IS investment into maintenance and support ............................................................. 87 4.3.7 IS investment to package customization ..................................................................... 89 vii School of Natural Sciences NWU (PUK) 4.3.8 Last IS project size ......................................................................................................... 89 4.3.9 Duration of last project ................................................................................................... 90 4.4 SECTION B: Systems development methodologies used (Research objective 1) .......................................................................................................... 91 4.4.1 Commercial systems development methodology usage .......................................... 91 4.4.2 Period of systems development methodology usage ................................................ 95 4.4.3 Motivation systems development methodology choice ............................................. 95 4.4.4 The proportion of projects developed by using systems development methodologies ................................................................................................................. 96 4.4.5 Proportion of employees who used systems development methodology regularly ............................................................................................................................ 97 4.4.6 Perceived use of systems development methodologies ........................................... 98 4.4.7 Sub-section B: Reasons for not using systems development methodologies ....... 99 4.4.7.1 Experience with the interaction of SDMs ................................................................... 99 4.4.7.2 Reasons for not adopting SDMs in last project ....................................................... 100 4.4.7.3 Factor analysis and reliability ..................................................................................... 104 4.5 SECTION C: Software Process Improvement Models Certification (Research objective 2) ...................................................................................... 107 4.5.1 Software Process Improvement Models (SPIMs) usage ........................................ 107 4.5.2 CMMi level of certification ............................................................................................ 107 4.5.3 Reason for certification ................................................................................................ 108 4.5.4 System development methodology role on certification ......................................... 109 4.5.5 SPI Procedures and Processes .................................................................................. 109 4.5.6 Descriptive statistics of processes and procedures ................................................ 113 4.5.7 Maturity level classification Algorithm ........................................................................ 114 4.5.8 Activity distribution per maturity level ......................................................................... 115 viii School of Natural Sciences NWU (PUK) 4.5.9 Reasons for non-SPIM certification............................................................................ 121 4.6 SECTION D: The project outcome (Objective 3) ............................................ 122 4.6.1 Last project outcome .................................................................................................... 123 4.6.2 Life-span of last project involved in ............................................................................ 124 4.6.3 Project success: process factors influencing project success ............................... 125 4.6.3.1 Factor analysis and reliability ..................................................................................... 129 4.6.4 Project success: End-product factors influencing project success ....................... 130 4.6.4.1 Factor analysis and reliability ..................................................................................... 133 4.6.5 Statistical analysis: Components of factor analysis for project success .............. 134 4.7 Combination of SDMs and SPIMs use in organisation ................................. 134 4.8 The Correlations between SDMs and Maturity levels ................................... 139 4.9 Relationship between process/product success factors and maturity levels ................................................................................................................... 141 4.10 CHAPTER SUMMARY ........................................................................................ 142 CHAPTER 5 .......................................................................................................................... 144 DISCUSSIONS ..................................................................................................................... 144 5.1 INTRODUCTION ................................................................................................. 144 5.2 THE DEMOGRAPHIC INFORMATION .............................................................. 144 5.3 RESEARCH OBJECTIVE 1: The use and effectiveness of ASDMs .............. 145 5.3.1 To what extent does IS department use standard system development methodologies? ............................................................................................................. 145 5.3.1.1 Reasons for not using systems development methodologies ............................... 145 5.4 RESEARCH OBJECTIVE 2: The use and effectiveness of SPIMs ............... 146 5.4.1 Software Process Improvement Model (SPIMs) usage .......................................... 146 5.4.2 SPI procedures and processes ................................................................................... 146 5.4.3 Factors contributing to non-SPIM certification .......................................................... 147 ix