What Readers Are Saying About Seven Databases in Seven Weeks, Second Edition Choosing a database is perhaps one of the most important architectural decisions a developer can make. Seven Databases in Seven Weeks provides a fantastic tour of different technologies and makes it easy to add each to your engineering toolbox. ➤ Dave Parfitt Senior Site Reliability Engineer, Mozilla By comparing each database technology to a tool you’d find in any workshop, the authors of Seven Databases in Seven Weeks provide a practical and well-balanced survey of a very diverse and highly varied datastore landscape. Anyone looking to get a handle on the database options available to them as a data platform should read this book and consider the trade-offs presented for each option. ➤ Matthew Oldham Director of Data Architecture, Graphium Health Reading this book felt like some of my best pair-programming experiences. It showed me how to get started, kept me engaged, and encouraged me to experiment on my own. ➤ Jesse Hallett Open Source Mentor This book will really give you an overview of what’s out there so you can choose the best tool for the job. ➤ Jesse Anderson Managing Director, Big Data Institute We've left this page blank to make the page numbers the same in the electronic and paper books. We tried just leaving it out, but then people wrote us to ask about the missing pages. Anyway, Eddy the Gerbil wanted to say “hello.” Seven Databases in Seven Weeks, Second Edition A Guide to Modern Databases and the NoSQL Movement Luc Perkins with Eric Redmond and Jim R. Wilson The Pragmatic Bookshelf Raleigh, North Carolina Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and The Pragmatic Programmers, LLC was aware of a trademark claim, the designations have been printed in initial capital letters or in all capitals. The Pragmatic Starter Kit, The Pragmatic Programmer, Pragmatic Programming, Pragmatic Bookshelf, PragProg and the linking g device are trade- marks of The Pragmatic Programmers, LLC. Every precaution was taken in the preparation of this book. However, the publisher assumes no responsibility for errors or omissions, or for damages that may result from the use of information (including program listings) contained herein. Our Pragmatic books, screencasts, and audio books can help you and your team create better software and have more fun. Visit us at https://pragprog.com. The team that produced this book includes: Publisher: Andy Hunt VP of Operations: Janet Furlow Managing Editor: Brian MacDonald Supervising Editor: Jacquelyn Carter Series Editor: Bruce A. Tate Copy Editor: Nancy Rapoport Indexing: Potomac Indexing, LLC Layout: Gilson Graphics For sales, volume licensing, and support, please contact Contents Acknowledgments . . . . . . . . . . . vii Preface . . . . . . . . . . . . . . ix 1. Introduction . . . . . . . . . . . . . 1 It Starts with a Question 2 The Genres 3 Onward and Upward 8 2. PostgreSQL . . . . . . . . . . . . . 9 That’s Post-greS-Q-L 9 Day 1: Relations, CRUD, and Joins 10 Day 2: Advanced Queries, Code, and Rules 21 Day 3: Full Text and Multidimensions 36 Wrap-Up 50 3. HBase . . . . . . . . . . . . . . 53 Introducing HBase 54 Day 1: CRUD and Table Administration 55 Day 2: Working with Big Data 67 Day 3: Taking It to the Cloud 82 Wrap-Up 88 4. MongoDB . . . . . . . . . . . . . 93 Hu(mongo)us 93 Day 1: CRUD and Nesting 94 Day 2: Indexing, Aggregating, Mapreduce 110 Day 3: Replica Sets, Sharding, GeoSpatial, and GridFS 124 Wrap-Up 132 5. CouchDB . . . . . . . . . . . . . 135 Relaxing on the Couch 135 Contents • vi Day 1: CRUD, Fauxton, and cURL Redux 137 Day 2: Creating and Querying Views 145 Day 3: Advanced Views, Changes API, and Replicating Data 158 Wrap-Up 174 6. Neo4J . . . . . . . . . . . . . . 177 Neo4j Is Whiteboard Friendly 177 Day 1: Graphs, Cypher, and CRUD 179 Day 2: REST, Indexes, and Algorithms 189 Day 3: Distributed High Availability 202 Wrap-Up 207 7. DynamoDB . . . . . . . . . . . . . 211 DynamoDB: The “Big Easy” of NoSQL 211 Day 1: Let’s Go Shopping! 216 Day 2: Building a Streaming Data Pipeline 233 Day 3: Building an “Internet of Things” System Around DynamoDB 246 Wrap-Up 255 8. Redis . . . . . . . . . . . . . . 259 Data Structure Server Store 259 Day 1: CRUD and Datatypes 260 Day 2: Advanced Usage, Distribution 274 Day 3: Playing with Other Databases 289 Wrap-Up 303 9. Wrapping Up . . . . . . . . . . . . 305 Genres Redux 305 Making a Choice 309 Where Do We Go from Here? 309 A1. Database Overview Tables . . . . . . . . . 311 A2. The CAP Theorem . . . . . . . . . . . 315 Eventual Consistency 316 CAP in the Wild 317 The Latency Trade-Off 317 Bibliography . . . . . . . . . . . . 319 Index . . . . . . . . . . . . . . 321 Acknowledgments A book with the size and scope of this one is never the work of just the authors, even if there are three of them. It requires the effort of many very smart people with superhuman eyes spotting as many mistakes as possible and providing valuable insights into the details of these technologies. We’d like to thank, in no particular order, all of the folks who provided their time and expertise: Dave Parfitt Jerry Sievert Jesse Hallett Matthew Oldham Ben Rady Nick Capito Jesse Anderson Sean Moubry Finally, thanks to Bruce Tate for his experience and guidance. We’d also like to sincerely thank the entire team at the Pragmatic Bookshelf. Thanks for entertaining this audacious project and seeing us through it. We’re especially grateful to our editor, Jackie Carter. Your patient feedback made this book what it is today. Thanks to the whole team who worked so hard to polish this book and find all of our mistakes. For anyone we missed, we hope you’ll accept our apologies. Any omissions were certainly not intentional. From Eric: Dear Noelle, you’re not special; you’re unique, and that’s so much better. Thanks for living through another book. Thanks also to the database creators and committers for providing us something to write about and make a living at. From Luc: First, I have to thank my wonderful family and friends for making my life a charmed one from the very beginning. Second, I have to thank a handful of people who believed in me and gave me a chance in the tech industry at different stages of my career: Lucas Carlson, Marko and Saša Gargenta, Troy Howard, and my co-author Eric Redmond for inviting me on board to report erratum • discuss Acknowledgments • viii prepare the most recent edition of this book. My journey in this industry has changed my life and I thank all of you for crucial breakthroughs. From Jim: First, I want to thank my family: Ruthy, your boundless patience and encouragement have been heartwarming. Emma and Jimmy, you’re two smart cookies, and your daddy loves you always. Also, a special thanks to all the unsung heroes who monitor IRC, message boards, mailing lists, and bug systems ready to help anyone who needs you. Your dedication to open source keeps these projects kicking. report erratum • discuss