Cloud Native Using Containers, Functions, and Data to Build Next-Generation Applications Boris Scholl, Trent Swanson & Peter Jausovec Cloud Native Using Containers, Functions, and Data to Build Next-Generation Applications Boris Scholl, Trent Swanson, and Peter Jausovec BBeeiijjiinngg BBoossttoonn FFaarrnnhhaamm SSeebbaassttooppooll TTookkyyoo Cloud Native by Boris Scholl, Trent Swanson, and Peter Jausovec Copyright © 2019 Boris Scholl, Trent Swanson, and Peter Jausovec. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or [email protected]. Acquisitions Editor: Kathleen Carr Indexer: Ellen Troutman-Zaig Development Editor: Nicole Tache Interior Designer: David Futato Production Editor: Elizabeth Kelly Cover Designer: Karen Montgomery Copyeditor: Octal Publishing, Inc. Illustrator: Rebecca Demarest Proofreader: Rachel Monaghan September 2019: First Edition Revision History for the First Edition 2019-08-21: First Release 2019-12-10: Second Release 2020-02-07: Third Release See http://oreilly.com/catalog/errata.csp?isbn=9781492053828 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Cloud Native, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the authors, and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. 978-1-492-05382-8 [LSI] Table of Contents Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi 1. Introduction to Cloud Native. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Distributed Systems 1 Fallacies of Distributed Systems 1 CAP Theorem 3 The Twelve-Factor App 4 Availability and Service-Level Agreements 6 Summary 7 2. Fundamentals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Containers 9 Container Isolation Levels 11 Container Orchestration 13 Kubernetes Overview 13 Kubernetes and Containers 16 Serverless Computing 17 Functions 18 From VMs to Cloud Native 19 Lift-and-Shift 19 Application Modernization 20 Application Optimization 22 Microservices 22 Benefits of a Microservices Architecture 23 Challenges with a Microservices Architecture 25 Summary 27 iii 3. Designing Cloud Native Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Fundamentals of Cloud Native Applications 29 Operational Excellence 29 Security 31 Reliability and Availability 33 Scalability and Cost 34 Cloud Native versus Traditional Architectures 34 Functions versus Services 39 Function Scenarios 39 Considerations for Using Functions 39 Composite of Functions and Services 40 API Design and Versioning 42 API Backward and Forward Compatibility 44 Semantic Versioning 45 Service Communication 45 Protocols 46 Messaging Protocols 47 Serialization Considerations 48 Idempotency 49 Request/Response 49 Publisher/Subscriber 51 Choosing Between Pub/Sub and Request Response 53 Synchronous versus Asynchronous 54 Gateways 55 Routing 55 Aggregation 56 Offloading 57 Implementing Gateways 58 Egress 58 Service Mesh 59 Example Architecture 68 Summary 72 4. Working with Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Data Storage Systems 75 Objects, Files, and Disks 75 Databases 76 Streams and Queues 79 Blockchain 79 Selecting a Datastore 80 Data in Multiple Datastores 82 Change Data Capture 83 iv | Table of Contents Write Changes as an Event to a Change Log 86 Transaction Supervisor 86 Compensating Transactions 88 Extract, Transform, and Load 89 Microservices and Data Lakes 89 Client Access to Data 92 Restricted Client Tokens (Valet-Key) 93 Database Services with Fine-Grained Access Control 94 GraphQL Data Service 95 Fast Scalable Data 96 Sharding Data 96 Caching Data 97 Content Delivery Networks 98 Analyzing Data 100 Streams 100 Batch 100 Data Lakes on Object Storage 101 Data Lakes and Data Warehouses 101 Distributed Query Engines 102 Databases on Kubernetes 103 Storage Volumes 104 StatefulSets 105 DaemonSets 106 Summary 107 5. DevOps. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 What Is DevOps? 109 Collaboration 109 Automation 110 Lean Principles and Processes 110 Measurement 111 Sharing 111 Testing 112 Test Doubles 113 Test Automation Pyramid 113 When to Run Which Types of Tests 118 Testing Cadence 119 Testing in Production 120 Development Environments and Tools 123 Development Tools 124 Development Environments 126 Local Development Environments 126 Table of Contents | v Local Development with a Remote Cluster 127 Skaffold Development Workflow 128 Remote Cluster Routed to Local Development 129 Cloud Development Environments 130 CI/CD 130 Source Code Control 131 Build Stage (CI) 133 Test Stage (CI) 133 Deploy Stage (CD) 134 Release Stage (CD) 136 Post-Release Stage 137 Monitoring 138 Collecting Metrics 139 Observable Services 144 Configuration Management 149 Single-Environment Variable 151 Multiple-Environment Variables 151 Adding ConfigMap Data to a Volume 151 Storing Secrets 152 Deployment Configuration 153 Sample CI/CD Flows 155 Summary 158 6. Best Practices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 Moving to Cloud Native 159 Breaking Up the Monolith for the Right Reasons 159 Decouple Simple Services First 160 Learn to Operate on a Small Scale 160 Use an Anticorruption Layer Pattern 160 Use a Strangler Pattern 161 Come Up with a Data Migration Strategy 162 Rewrite Any Boilerplate Code 162 Reconsider Frameworks, Languages, Data Structures, and Datastores 163 Retire Code 163 Ensuring Resiliency 163 Handle Transient Failures with Retries 163 Use a Finite Number of Retries 164 Use Circuit Breakers for Nontransient Failures 164 Graceful Degradation 164 Use a Bulkhead Pattern 164 Implement Health Checks and Readiness Checks 165 Define CPU and Memory Limits for Your Containers 165 vi | Table of Contents Implement Rate Limiting and Throttling 165 Ensuring Security 165 Treat Security Requirements the Same as Any Other Requirements 165 Incorporate Security in Your Designs 166 Grant Least-Privileged Access 166 Use Separate Accounts/Subscriptions/Tenants 166 Securely Store All Secrets 166 Obfuscate Data 166 Encrypt Data in Transit 167 Use Federated Identity Management 167 Use Role-Based Access Control 167 Isolate Kubernetes Pods 167 Working with Data 167 Use Managed Databases and Analytics Services 168 Use a Datastore That Best Fits Data Requirements 168 Keep Data in Multiple Regions or Zones 168 Use Data Partitioning and Replication for Scale 168 Avoid Overfetching and Chatty I/O 169 Don’t Put Business Logic in the Database 169 Test with Production-like Data 169 Handle Transient Failures 169 Performance and Scalability 169 Design Stateless Services That Scale Out 170 Use Platform Autoscaling Features 170 Use Caching 170 Use Partitioning to Scale Beyond Service Limits 170 Functions 171 Write Single-Purpose Functions 171 Don’t Chain Functions 171 Keep Functions Light and Simple 171 Make Functions Stateless 171 Separate Function Entry Point from the Function Logic 171 Avoid Long-Running Functions 172 Use Queues for Cross-Function Communication 172 Operations 172 Deployments and Releases Are Separate Activities 172 Keep Deployments Small 172 CI/CD Definition Lives with the Component 172 Consistent Application Deployment 173 Use Zero-Downtime Releases 173 Don’t Modify Deployed Infrastructure 173 Use Containerized Build 173 Table of Contents | vii Describe Infrastructure Using Code 173 Use Namespaces to Organize Services in Kubernetes 173 Isolate the Environments 173 Separate Function Source Code 173 Correlate Deployments with Commits 174 Logging, Monitoring, and Alerting 174 Use a Unified Logging System 174 Use Correlation IDs 174 Include Context with Log Entries 174 Common and Structured Logging Format 174 Tag Your Metrics Appropriately 175 Avoid Alert Fatigue 175 Define and Alert on Key Performance Indicators 175 Continuous Testing in Production 175 Start with Basic Metrics 176 Service Communication 176 Design for Backward and Forward Compatibility 176 Define Service Contracts That Do Not Leak Internal Details 177 Prefer Asynchronous Communication 177 Use Efficient Serialization Techniques 177 Use Queues or Streams to Handle Heavy Loads and Traffic Spikes 178 Batch Requests for Efficiency 178 Split Up Large Messages 178 Containers 178 Store Images in a Trusted Registry 179 Utilize the Docker Build Cache 179 Don’t Run Containers in Privileged Mode 179 Use Explicit Container Image Tags 179 Keep Container Images Small 179 Run One Application per Container 180 Use Verified Images from Trusted Repositories 180 Use Vulnerability Scanning Tools on Images 180 Don’t Store Data in Containers 181 Never Store Secrets or Configuration Inside an Image 181 Summary 181 7. Portability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 Why Make Applications Portable? 183 The Costs of Portability 184 Data Gravity and Portability 185 When and How to Implement Portability 185 Standardized Interfaces 186 viii | Table of Contents