Praise for Cloud Native DevOps with Kubernetes Cloud Native DevOps is an essential guide to operating today’s distributed systems. A super clear and informative read, covering all the details without compromising readability. I learned a lot, and definitely have some action points to take away! —Will Thames, Platform Engineer, Skedulo The most encompassing, definitive, and practical text about the care and feeding of Kubernetes infrastructure. An absolute must-have. —Jeremy Yates, SRE Team, The Home Depot QuoteCenter I wish I’d had this book when I started! This is a must-read for everyone developing and running applications in Kubernetes. —Paul van der Linden, Lead Developer, vdL Software Consultancy This book got me really excited. It’s a goldmine of information for anyone looking to use Kubernetes, and I feel like I’ve levelled up! —Adam McPartlan (@mcparty), Senior Systems Engineer, NYnet I really enjoyed reading this book. It’s very informal in style, but authoritative at the same time. It contains lots of great practical advice. Exactly the sort of information that everybody wants to know, but doesn’t know how to get, other than through first-hand experience. —Nigel Brown, cloud native trainer and course author Cloud Native DevOps with Kubernetes Building, Deploying, and Scaling Modern Applications in the Cloud John Arundel and Justin Domingus BBeeiijjiinngg BBoossttoonn FFaarrnnhhaamm SSeebbaassttooppooll TTookkyyoo Cloud Native DevOps with Kubernetes by John Arundel and Justin Domingus Copyright © 2019 John Arundel and Justin Domingus. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com/safari). For more information, contact our corporate/insti‐ tutional sales department: 800-998-9938 or [email protected]. Acquisitions Editor: Rachel Roumeliotis Developmental Editors: Virginia Wilson and Nikki McDonald Production Editor: Nan Barber Copyeditor: Kim Cofer Proofreader: Amanda Kersey Indexer: Judith McConville Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Rebecca Demarest February 2019: First Edition Revision History for the First Edition 2019-01-24: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781492040767 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Cloud Native DevOps with Kubernetes, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the authors, and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. This work is part of a collaboration between O’Reilly and NGINX. See our statement of editorial inde‐ pendence (http://www.oreilly.com/about/editorial_independence.html). 978-1-492-04076-7 [LSI] Table of Contents Foreword by NGINX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix Foreword by Ihor Dvoretskyi. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii 1. Revolution in the Cloud. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 The Creation of the Cloud 2 Buying Time 3 Infrastructure as a Service 3 The Dawn of DevOps 3 Nobody Understands DevOps 5 The Business Advantage 5 Infrastructure as Code 6 Learning Together 6 The Coming of Containers 7 The State of the Art 7 Thinking Inside the Box 8 Putting Software in Containers 8 Plug and Play Applications 9 Conducting the Container Orchestra 10 Kubernetes 11 From Borg to Kubernetes 11 What Makes Kubernetes So Valuable? 11 Will Kubernetes Disappear? 13 Kubernetes Doesn’t Do It All 13 Cloud Native 14 The Future of Operations 16 vii Distributed DevOps 17 Some Things Will Remain Centralized 17 Developer Productivity Engineering 17 You Are the Future 18 Summary 19 2. First Steps with Kubernetes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Running Your First Container 21 Installing Docker Desktop 22 What Is Docker? 22 Running a Container Image 22 The Demo Application 23 Looking at the Source Code 23 Introducing Go 24 How the Demo App Works 24 Building a Container 25 Understanding Dockerfiles 25 Minimal Container Images 26 Running docker image build 26 Naming Your Images 27 Port Forwarding 27 Container Registries 28 Authenticating to the Registry 28 Naming and Pushing Your Image 28 Running Your Image 29 Hello, Kubernetes 29 Running the Demo App 30 If the Container Doesn’t Start 30 Minikube 31 Summary 31 3. Getting Kubernetes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Cluster Architecture 33 The Control Plane 34 Node Components 35 High Availability 35 The Costs of Self-Hosting Kubernetes 37 It’s More Work Than You Think 37 It’s Not Just About the Initial Setup 38 Tools Don’t Do All the Work for You 39 Kubernetes Is Hard 39 Administration Overhead 39 viii | Table of Contents Start with Managed Services 40 Managed Kubernetes Services 41 Google Kubernetes Engine (GKE) 41 Cluster Autoscaling 42 Amazon Elastic Container Service for Kubernetes (EKS) 42 Azure Kubernetes Service (AKS) 43 OpenShift 43 IBM Cloud Kubernetes Service 43 Heptio Kubernetes Subscription (HKS) 43 Turnkey Kubernetes Solutions 44 Stackpoint 44 Containership Kubernetes Engine (CKE) 44 Kubernetes Installers 44 kops 45 Kubespray 45 TK8 45 Kubernetes The Hard Way 45 kubeadm 46 Tarmak 46 Rancher Kubernetes Engine (RKE) 46 Puppet Kubernetes Module 46 Kubeformation 46 Buy or Build: Our Recommendations 47 Run Less Software 47 Use Managed Kubernetes if You Can 48 But What About Vendor Lock-in? 48 Use Standard Kubernetes Self-Hosting Tools if You Must 49 When Your Choices Are Limited 49 Bare-Metal and On-Prem 49 Clusterless Container Services 50 Amazon Fargate 50 Azure Container Instances (ACI) 51 Summary 51 4. Working with Kubernetes Objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Deployments 53 Supervising and Scheduling 54 Restarting Containers 54 Querying Deployments 55 Pods 55 ReplicaSets 56 Maintaining Desired State 57 Table of Contents | ix The Kubernetes Scheduler 58 Resource Manifests in YAML Format 59 Resources Are Data 59 Deployment Manifests 59 Using kubectl apply 60 Service Resources 60 Querying the Cluster with kubectl 63 Taking Resources to the Next Level 64 Helm: A Kubernetes Package Manager 64 Installing Helm 65 Installing a Helm Chart 65 Charts, Repositories, and Releases 66 Listing Helm Releases 67 Summary 67 5. Managing Resources. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 Understanding Resources 69 Resource Units 70 Resource Requests 70 Resource Limits 71 Keep Your Containers Small 72 Managing the Container Life Cycle 72 Liveness Probes 72 Probe Delay and Frequency 73 Other Types of Probes 73 gRPC Probes 74 Readiness Probes 74 File-Based Readiness Probes 75 minReadySeconds 75 Pod Disruption Budgets 76 Using Namespaces 77 Working with Namespaces 78 What Namespaces Should I Use? 78 Service Addresses 79 Resource Quotas 79 Default Resource Requests and Limits 80 Optimizing Cluster Costs 81 Optimizing Deployments 82 Optimizing Pods 83 Vertical Pod Autoscaler 84 Optimizing Nodes 84 Optimizing Storage 85 x | Table of Contents Cleaning Up Unused Resources 86 Checking Spare Capacity 88 Using Reserved Instances 88 Using Preemptible (Spot) Instances 89 Keeping Your Workloads Balanced 91 Summary 92 6. Operating Clusters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Cluster Sizing and Scaling 95 Capacity Planning 96 Nodes and Instances 98 Scaling the Cluster 101 Conformance Checking 102 CNCF Certification 103 Conformance Testing with Sonobuoy 104 Validation and Auditing 105 K8Guard 106 Copper 106 kube-bench 107 Kubernetes Audit Logging 107 Chaos Testing 107 Only Production Is Production 108 chaoskube 108 kube-monkey 109 PowerfulSeal 109 Summary 110 7. Kubernetes Power Tools. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Mastering kubectl 111 Shell Aliases 111 Using Short Flags 112 Abbreviating Resource Types 112 Auto-Completing kubectl Commands 113 Getting Help 113 Getting Help on Kubernetes Resources 114 Showing More Detailed Output 114 Working with JSON Data and jq 114 Watching Objects 115 Describing Objects 116 Working with Resources 116 Imperative kubectl Commands 116 When Not to Use Imperative Commands 117 Table of Contents | xi