PRESS Accelerating Digital Transformation with Containers and Kubernetes An Introduction to Cloud-Native Technology Author Steve Hoenisch Warning and Disclaimer Every efort has been made to make this book as complete and as accurate as possible, but no warranty or ftness is implied. The information provided is on an “as is” basis. The authors, VMware Press, VMware, and the publisher shall have neither liability nor responsibility to any person or entity with respect to any loss or damages arising from the information contained in this book. The opinions expressed in this book belong to the author and are not necessarily those of VMware. VMware, Inc. 3401 Hillview Avenue Palo Alto CA 94304 USA Tel 877-486-9273 Fax 650-427-5001 www.vmware.com. Copyright © 2018 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at http://www.vmware.com/go/patents. VMware is a registered trademark or trademark of VMware, Inc. and its subsidiaries in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies. 2 Contents Introduction 8 Organization of this Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8 Point of Departure: Cloud-Native Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9 Driving Digital Transformation with Containers and Kubernetes 13 The Business Value of Digital Transformation . . . . . . . . . . . . . . . . . 13 Cloud-Native Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 12-Factor Apps: A Methodology for Delivering Software as a Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 The Business Value of Kubernetes . . . . . . . . . . . . . . . . . . . . . . . . . . 17 An Example Use Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Demystifying Kubernete s 19 Platform vs. Runtime Environment . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Robust Open-Source Technology from a Google Production System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Defogging the Abstract Terminology of Kubernetes . . . . . . . . . 20 A Concise Overview of Kubernetes . . . . . . . . . . . . . . . . . . . . . . . . . 21 Just Another Fad in the Hype Cycle? . . . . . . . . . . . . . . . . . . . . . . . 24 Kubernetes in Production Environments . . . . . . . . . . . . . . . . . . . . 24 A Rapidly Maturing Ecosystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Kubernetes Won’t Solve All Your Problems . . . . . . . . . . . . . . . . . 25 Introduction to Cloud-Native Architectures and Practices 27 Microservices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27 Deconstructing the Monolith and Other Use Cases . . . . . . . . . . 29 Kubernetes for Cloud-Native and 12-Factor Applications . . . . . 30 Profle of a DevOps Engineer: Responsibilities and Skills . . . . . 32 Continuous Integration and Continuous Deployment . . . . . . . . 34 Container Technology in the Software-Defned Data Center 35 VMware vSphere and the SDDC . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Abstract and Automate: Network Virtualization . . . . . . . . . . . . . 36 Risk-Free Scale Out with Ease: Virtual Storage . . . . . . . . . . . . . . .37 Put a Lid on It: Security for Containers . . . . . . . . . . . . . . . . . . . . . 38 Linux Container Hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Securing Cloud Platforms with Lightwave . . . . . . . . . . . . . . . . . . 44 Managing Container Images with Harbor . . . . . . . . . . . . . . . . . . . . 51 Microservices Meets Micro- segmentation: Delivering Developer-Ready Infrastructure for Modern Application Development with NSX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 An Introduction to Cloud-Native Technology | 3 BOSH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 CFCN for Deploying and Operating Kubernetes . . . . . . . . . . . . . 67 Container Platforms and Services 70 High-Level Use Cases for Container Platforms . . . . . . . . . . . . . . . 70 Maturity of Container Adoption . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 Cloud Natives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Matching the Platform to the Project . . . . . . . . . . . . . . . . . . . . . . . . 71 Prescription and Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 vSphere Integrated Containers . . . . . . . . . . . . . . . . . . . . . . . . . . . . .73 VMware Pivotal Container Service . . . . . . . . . . . . . . . . . . . . . . . . . 92 Use Cases 101 Self-Service Infrastructure for Agile Development . . . . . . . . . . . 101 Replatforming Applications with PKS . . . . . . . . . . . . . . . . . . . . . .106 Deploying New Cloud-Native Apps with PKS . . . . . . . . . . . . . . .108 Exploiting the Power of Containers . . . . . . . . . . . . . . . . . . . . . . . .109 Running a Containerized App with Photon OS on Amazon Elastic Cloud Compute . . . . . . . . . . . . . .109 Using vSphere Integrated Containers to Solve Container Networking Problems . . . . . . . . . . . . . . . . . . . . . . . . . . .120 Providing Persistent Storage for Legacy Applications . . . . . . . . 124 Setting Up a Developer Sandbox . . . . . . . . . . . . . . . . . . . . . . . . . . 133 Deploying Jenkins by Using VIC . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 Optimizing Cloud-Native Apps with PCF and Developer-Ready Infrastructure from VMware . . . . . . . . . . . . . . . 147 Case Study: Optimizing Critical Banking Workloads . . . . . . . . . 158 Conclusion 162 Glossary 163 4 Author and Contributors Author and Editor Steve Hoenisch is a technology evangelist, writer, and editor who specializes in emerging technology and cloud-native solutions He’s written numerous infuential technical white papers and magazine articles on digital transformation, Kubernetes, containers, big data, Hadoop, storage platforms, security, and regulatory compliance A former newspaper editor with a master’s degree in linguistics, he has published articles in XML Journal, The Hartford Courant, and the Chicago Tribune He works in the Cloud-Native Apps business unit at VMware Contributors Ben Corrie has been a leading voice of technical innovation in the container space at VMware for three years Ben was the initiator of the research that led to the vSphere Integrated Containers product and as an architect on that product, Ben’s role now is to look 6 to 12 months ahead to help align VMware with the container challenges ahead Patrick Daigle is a Senior Technical Marketing Architect in Montreal, Canada As part of VMware’s Cloud-Native Applications business unit, he focuses on vSphere Integrated Containers and works with enterprises around the globe, explaining and demonstrating the benefts of VMware container solutions and how they can bring value to the business An Introduction to Cloud-Native Technology | 5 Contributors, cont Ning Ge is a Senior Product Marketing Manager in VMware’s Cloud-Native Apps business unit and works on VMware’s container solutions, such as VMware Pivotal Container Service and vSphere Integrated Containers Ning has over 7 years of experience in marketing enterprise software technologies and solutions, and her main area of focus includes container and cloud- native, middleware, and infrastructure technologies Ning has master’s degrees in both Business Administration and Communications Merlin Glynn is a product manager at VMware, where he builds products that help customers architect and deploy cloud-native applications Merlin has been building complex environments as an architect for over 20 years, focusing on solutions for large enterprises and the academic and scientifc community Previously, he architected some of the world’s largest supercomputers at IBM, which were regularly listed on the TOP500 list, and, at Pivotal, designed many next-generation Pivotal Cloud Foundry platforms for key enterprise customers Merlin is a certifed AWS Solutions Architect He enjoys volunteering for charities 6 Simone Morellato is currently a Director of Technical Product Management at VMware where he leads technical product management and marketing eforts for the company’s Cloud- Native Applications business unit Simone has more than 16 years of experience in storage, networking and infrastructure for both traditional and cloud applications Before joining VMware, Simone worked at Apcera, a container management platform, acquired by Ericsson He has also held leadership, marketing and technical presales roles at Cisco, Riverbed Technology, Astute Networks and Andiamo Systems (later acquired by Cisco) Tom Scanlan is a Senior Consultant in the CNA/ DevOps and Emerging technologies arenas He has 20 years experience across software engineering, systems and networking administration and consulting thereon Tom has had deep focus on enabling DevOps and multicloud architectures An Introduction to Cloud-Native Technology | 7 Introduction Digital transformation, the commoditization of IT, the Internet of things, the proliferation of mobile devices, the growing popularity of public clouds, big data, and other seismic technological changes are radically altering the way businesses are run. Innovative software applications are, for many businesses, a critical objective. Consumers, customers, and keeping ahead of the competition demand it. But one-time innovation is often not enough. The digital era calls for continuous innovation at an accelerated pace—and the kind of modern- ized data centers and software development technologies that make it possible. Container technology can help transform a company into a digital enterprise focused on delivering innovations at the speed of business. Containers package applications and their dependencies into a distribut- able image that can run almost anywhere, streamlining the development and deployment of software. By adopting containers, organizations can take a vital step toward remaking themselves into fexible, agile digital enterprises capable of accelerating the delivery of innovative products, services, and customer experiences. Enterprises can become the disrupters instead of the dis- rupted. But containers create technology management problems of their own, especially when containerized applications need to be deployed and managed at scale, and that’s when Kubernetes comes into play. Kuber- netes automates the deployment and management of containerized applications. More specifcally, Kubernetes orchestrates containerized applications to manage and automate resource utilization, failure han- dling, availability, confguration, desired state, and scalability. This book introduces you to containers and Kubernetes, explains their business value, explores their use cases, and illuminates how they can accelerate your organization’s digital transformation. Organization of this Book The chapters at the beginning of the book explain the business value container technology and examine how enterprises are modernizing their data centers to take advantage of cloud-native innovations. After briefy examining the architectural patterns, practices, processes, and pipelines that help propel you toward digital transformation, the book 8 considers the kind of infrastructure, virtualization technologies, systems, and security required by next-generation data centers. The chapters that follow become increasingly technical as they use two key products from VMware—VMware vSphere Integrated Containers and VMware Pivotal Container Service—to explain the architecture of cloud-native applications, the capabilities of Kubernetes, and the use cases for container technology. The fnal sections of the book turn to examples that demonstrate how to exploit the power of containers and Kubernetes to solve technical problems. Point of Departure: Cloud-Native Terminology Container technology comes with its own lexicon. If you’re familiar with the basic terminology around containers, Kubernetes, and cloud-native applications, you can skip this section. For plain-language descriptions of terminology in the cloud-native space, see the glossary at the end of the book. Containers Container: A portable format, known as an image, for packaging an application with instructions on how to run it as well as an environment in which the image is executed. When the container image is executed, it runs as a process on a computer or virtual machine with its own isolated, self-described application, fle system, and networking. A container is more formally known as an application container. The use of containers is increasing because they provide a portable, fexible, and predictable way of packaging, distributing, modifying, testing, and running applica- tions. Containers speed up software development and deployment. Docker is a widely used container format. Docker defnes a standard format for packaging and porting software, much like ISO containers defne a standard for shipping freight. As a runtime instance of a Docker image, a container consists of three parts: • A Docker image • An environment in which the image is executed • A set of instructions for running the image Containerized application: An application that has been packaged to run in a container. An Introduction to Cloud-Native Technology | 9 Kubernetes and Orchestration Kubernetes: A system that automates the deployment and manage- ment of containerized applications. As an application and its services run in containers on a distributed cluster of virtual or physical machines, Kubernetes manages all the moving pieces to optimize the use of com- puting resources, to maintain the desired state, and to scale on demand. On Kubernetes, a container (or a set of related containers) is deployed in a logical unit called a pod. In addition to scheduling the deployment and automating the management of containerized applications, a key bene- ft of Kubernetes is that it maintains the desired state—the state that an administrator specifes the application should be in. Cluster: Three or more interconnected virtual machines or physical computers that, in efect, form a single system. A computer in a cluster is referred to as a node. An application running on a cluster is typically a distributed application because it runs on multiple nodes. By inherently providing high availability, fault tolerance, and scalability, clusters are a key part of cloud computing. Orchestration: Because it can automatically deploy, manage, and scale a containerized application, Kubernetes is often referred to as an orches- tration framework or an orchestration engine. It orchestrates resource utilization, failure handling, availability, confguration,desired state, and scalability. Application Types and Architectural Patterns Microservices: A “modern” architectural pattern for building an appli- cation. A microservices architecture breaks up the functions of an application into a set of small, discrete, decentralized, goal-oriented pro- cesses, each of which can be independently developed, tested, deployed, replaced, and scaled. Cloud-native applications: Generally speaking, they are developed and optimized to run in a cloud as distributed applications. According to the Cloud Native Computing Foundation, cloud-native applications, which are also generally referred to as “modern” applications, are marked by the following characteristics: • Containerized for reproducibility, transparency, and resource isolation. 10