www.it-ebooks.info Effective DevOps Building a Culture of Collaboration, Affinity, and Tooling at Scale Jennifer Davis & Katherine Daniels Boston www.it-ebooks.info Effective DevOps by Jennifer Davis and Katherine Daniels Copyright © 2015 Jennifer Davis and Katherine Daniels. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc. , 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles ( http://safaribooksonline.com ). For more information, contact our corporate/ institutional sales department: 800-998-9938 or [email protected] . Editor: Brian Anderson Proofreader: FILL IN PROOFREADER Production Editor: FILL IN PRODUCTION EDI‐ Indexer: FILL IN INDEXER TOR Interior Designer: David Futato Copyeditor: FILL IN COPYEDITOR Cover Designer: Karen Montgomery Illustrator: Rebecca Demarest February 2016: First Edition Revision History for the First Early Release Edition 2015-05-05: First Release 2015-07-22: Second Release See http://oreilly.com/catalog/errata.csp?isbn=9781491926307 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Effective DevOps, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. While the publisher and the author(s) have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author(s) disclaim all responsibil‐ ity for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. 978-1-491-92630-7 [FILL IN] www.it-ebooks.info Table of Contents 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2. What is Devops?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 The History of Devops 10 Developer as Operator 10 The Advent of Software Engineering 10 The Age of the Operating System 12 The Beginnings of a Global Community 13 The Age of Applications and the Web 14 Agile Infrastructure 16 The Beginning of DevopsDays 17 Foundational Devops Terminology and Concepts 17 Waterfall 18 Extreme Programming 19 Lean 19 ITIL 20 Agile 20 Community of Practice and Community of Interest 21 Blame Culture 22 Silos 23 Blamelessness 24 Retrospective 25 Organizational Learning 26 Post-Mortem 26 Devops: Adding it All Up 27 Common Devops Misconceptions 28 Devops only involves developers and system administrators. 28 Devops is a team. 28 iii www.it-ebooks.info Devops is a job title. 29 Devops is only relevant to web startups. 30 Devops is about the tools. 30 You need a devops certification. 30 Devops means doing all the work with half the people. 31 There is one “right way” (or “wrong way”) to do devops. 31 It will take X weeks/months to implement devops. 32 Devops is about automation. 32 Devops is a fad. 33 The Current State of Devops 34 The Devops Compact 34 What’s Next in this Book 36 3. Collaboration: Individuals Working Together. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Introduction 37 Individual Differences and Backgrounds 37 Goals 38 Backgrounds 38 Working Styles 41 Individual Growth 45 The Right Mindset 45 Organizational Pressure 50 Superstars and Superflocks 52 Negotiation Styles 53 From Competition to Collaboration 55 Communication 55 Why Communicate 55 What we Communicate 58 How we Communicate 58 Trust and Empathy 61 Developing Empathy 62 Developing Trust 64 4. Hiring: Choosing Individuals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Introduction and Audience 67 Determining your Hiring Needs 67 Position and Skills 67 Timeframe 68 Budget and Resources 69 Sourcing 70 Diversity 71 Interviewing 76 iv | Table of Contents www.it-ebooks.info Before the Interview 76 During the Interview 77 After the Interview 78 Onboarding 79 Retention 81 Compensation 82 Growth Opportunities 84 Workload 85 Culture and Atmosphere 87 Case Studies 91 Measuring Success 98 Troubleshooting 102 We aren’t getting enough candidates. 102 We aren’t getting diverse candidates. 104 Interviews are a waste of time for the team. 104 People aren’t accepting our offers. 105 Conclusion 105 5. Tools: Selection and Implementation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Introduction and Audience 107 Why Tools Matter 109 Why Tools Don’t Matter 112 Tool Ecosystem Overview 115 Configuration Management 115 Version Control 115 Infrastructure Automation 116 System Provisioning 116 Hardware Lifecycle Management 117 Continuous Integration 118 Test and Build Automation 118 Continuous Delivery 120 Application Deployment 120 Continuous Deployment 120 Metrics 120 Logging 121 Monitoring 121 Alerting 122 Events 123 Auditing your Tool Ecosystem 125 Communication 126 Moving Beyond the Basics 136 Optimization: Selection and Elimination of Tools 141 Table of Contents | v www.it-ebooks.info Version Control 142 Infrastructure Automation 146 Artifact Management 149 Work Visualization 149 Metrics 152 Improvements: Planning and Measuring Change 153 vi | Table of Contents www.it-ebooks.info CHAPTER 1 Introduction <section data-type="sect1"> <aside data-type="sidebar"> <h5>Early Release Edi‐ tion</h5> <p>This book is a work in progress – new chapters will be added as they are written. We welcome feedback – if you spot any errors or would like to suggest improve‐ ments, please let us know.</p> </aside> <h1>Who This Book Is For</h1> <p>This book is aimed primarily at managers and individual contributors in leader‐ ship roles who see friction within their organizations and are looking for concrete, actionable steps they can take towards implementing or improving a devops culture in their work environment. However, individual contributors of all levels who want practical suggestions for easing some of the pain points they face will find actionable takeaways.</p> <p>The audience is made up of a mix of professional roles, as devops is a professio‐ nal and cultural movement that stresses the iterative efforts to break down informa‐ tion silos, monitor relationships and repair when misunderstandings arise between teams in an organization. Many may be leaders within their organizations who have worked closely with developers or operations engineers.</p> <p>The book covers a wide range of devops skills and theory, including an introduc‐ tion to the basic ideas and concepts. It is assumed that you will have heard of the term devops and perhaps have a rudimentary understanding of devops, tools and pro‐ cesses used in the field.</p> <p>By the end of Effective Devops, our hope is that you will have a solid understand‐ ing of what having a devops culture means practically for your organization, how to encourage effective collaboration to help indivdiual contributors from different back‐ 7 www.it-ebooks.info grounds and teams deal with different goals and working styles to work together pro‐ ductively, how to help teams collaborate to maximize value between them while increasing employee satisfaction and balancing conflicting organizational goals, and how to choose tools and workflows for your organization that complement your organization.</p> </section> <section data-type="sect1"> <h1>How this Book Is Organized</h1> <p>This book is broken down into several parts, starting with an introductory chap‐ ter and then covering each of the pillars of devops.</p> <ul> <li>Chapter I, What is Devops</li> <li>Chapter II, Collaboration: Individuals Working Together</li> <li>Chapter III, Hiring: Choosing Individuals</li> <li>Chap‐ ter IV, Affinity: From Individuals to Teams</li> <li>Chapter V, Tools: Choosing and Using Them</li> <li>Chapter VI, Scale: Scaling Everything UP</li> </ul> </section> <section data-type="sect1"> <h1>Conventions Used in This Book</h1> </section> <section data-type="sect1"> <h1>Using Code Examples</h1> </section> <section data-type="sect1"> <h1>How to Contact Us</h1> <p>We have a web page for this book, where we list errata, examples, and any addi‐ tional information. You can access this page at http://effectivedevops.net.</p> <p>To comment or ask technical questions about this book, send email to bookques‐ [email protected].</p> </section> <section data-type="sect1"> <h1>Acknowledgements</h1> <p>Effective Devops would not have been possible without the help and guidance of many friends, colleagues, and family members.</p> </section> </section> 8 | Chapter 1: Introduction www.it-ebooks.info CHAPTER 2 What is Devops? What is devops? Some might define it as a software development method, while oth‐ ers might think that it is a set of tools and technologies such as configuration man‐ agement and continuous delivery. We would argue instead that devops is a cultural movement that seeks to improve both software development and the professional lives of the people involved in the field. In order to fully understand what we mean when we’re talking about devops, it is necessary to understand not only what the con‐ cept means and how it is used today, but also the history of how it came to be. No cultural movement exists in a vacuum. Technology is part of culture. We are born, accept what is and introduce new cultural aspects as we live. The way that our overall culture influences technology, and technology influences culture shapes the fabric of how we live our lives. Pre-1880, walking was the primary mode of transportation. Cities were compact with residences and workplaces intermingled. Streets were narrow and inconsistently paved. As automobiles were introduced, cities made decisions about the organization and infrastructure either in favor of automobiles or existing pedestrians. These days, some cities forgo infrastructure planning that factors in pedestrians at all creating busy roadways with no safe way to travel by foot. Walking shaped the city and how we worked, and the arrival of new technology changed the landscape accordingly. Devops is part of the cultural weave that shapes how we work and why. While devops does involve certain tools and technologies, an equally important part of our culture is our values, norms, and knowledge. Examining how people work, the technologies that we use, how technology influeces how we work, and how people influence tech‐ nology can help us make intentional decisions about the landscape of our industry. This chapter will delve into the evolution of software engineering as it pertains to the history of devops, define the terms and ideas closest to the movement, and address 9 www.it-ebooks.info