Distributed Network Data Alasdair Allan Kipp Bradford Published by Make Beijing • Cambridge • Farnham • Köln • Sebastopol • Tokyo Special Upgrade Offer If you purchased this ebook directly from oreilly.com, you have the following benefits: DRM-free ebooks—use your ebooks across devices without restrictions or limitations Multiple formats—use on your laptop, tablet, or phone Lifetime access, with free updates Dropbox syncing—your files, anywhere If you purchased this ebook from another retailer, you can upgrade your ebook to take advantage of all these benefits for just $4.99. Click here to access your ebook upgrade. Please note that upgrade offers are not available from sample content. Preface The gap between having an idea and being able to build a hardware solution is narrowing. The ability to prototype, build, and deploy simple sensor platforms is rapidly leading to an exponential growth in the amount of data available. Over the next few years, day-to-day computing will become increasingly invisible, dissipating out into the environment. This is already starting to happen, without you noticing, as the physical interfaces to the new smart devices look almost identical to their dumb counterparts. You will soon begin to move in a sea of data: your movements tracked and your environment measured and adjusted to your preferences, all without your intervention. At the O’Reilly Strata Conference on data science in Fall 2012 in New York, we gave the attendees a taste of the super-connected world that’s ahead of all of us. By instrumenting the conference environment with basic off-the-shelf sensors and mesh networking, we observed and reported, and gave the attendees a taste of their lives in a more measured and quantified world. This book will allow you to do likewise. Bill of Materials Before we get started building our sensor mote we’ll need to gather all the components we’ll need together in one place. In the hardware business this is generally called the “bill of materials.” Initially we’ll make use of the following components: Arduino Uno Breadboard DHT-22 PIR sensor Adafruit Electret Microphone 220Ω Resistor (×3) 10kΩ Resistor LED (×3) Jumper wire Later in the book we’ll expand our sensor mote to communicate over XBee (802.15.4) networking and if you want to follow along you’ll need to add the following additional components: XBee USB Explorer, $24.95 Arduino Wireless SD Shield, $19.95 XBee Series 1 Radio (×2), $22.95 Who Should Read this Book? This book provides an introduction to the topic of how to build and deploy a distributed sensor network. As part of that, we’ll make extensive use of the Arduino open-source electronics prototyping platform. This is hardware hacking for data scientists. If you’re a designer familiar with data visualization or a programmer with in interest in data, and want to learn about how to build simple sensor networks to gather data about your environment, then this book is for you. What You Should Already Know? This book is intended for data scientists who want to learn how to work with external hardware. It assumes some basic computing and programming knowledge, but no real expert knowledge is assumed. Little or no familiarity with the Arduino platform is expected. However, if you are totally unfamiliar with the Arduino platform, you might want to take a look at Getting Started with Arduino by Massimo Banzi (O’Reilly). What Will You Learn? This book will guide you through building a distributed sensor network and gathering data, and will show you how to do some simple real-time analysis and visualization. It will walk you through your first hardware prototypes, show you how to improve them, and teach you how to build a network of sensors and begin taking data. What’s In This Book? Chapter 1, Introduction to Arduino This chapter is intended for programmers new to Arduino. It will introduce you to the platform and walk you through the hardware equivalent of “Hello World”: the blinking LED. We’ll also discuss how to use the serial connection between the Arduino and your development machine. Chapter 2, Getting Started This chapter will walk you through wiring up a breadboard to prototype a circuit, and using the Arduino to read values from a sensor. Chapter 3, Adding Another Sensor This chapter provides a hands-on tutorial to adding a second sensor to the breadboard design—this one an infared motion detector (PIR). Chapter 4, Finishing the Breadboard This chapter is a guide to adding yet another sensor to your mote – a microphone. It also walks you through added diagnostic LEDs and modifying your Arduino sketch to output CSV through the serial console. Chapter 5, Moving from Breadboard to Prototype This chapter will explain how to use Fritzing, a program that lets us convert our circuit design into a direct graphical representation on the computer in preparation for building an actual circuit board. It also introduces theArduino prototyping shield and shows you how to turn your sketch into a professional circuit board. Chapter 6, Simplifying the Design After explaining the fundamental concepts of power and ground, this chapter takes a look at Arduino pins and how to use them for power, discussing the limitations of this approach as well as some particular use cases. Chapter 7, Building Point-to-Point XBee Networks This chapter discusses eliminating the USB cable and replacing it with a wireless connection to the Internet using XBees. Chapter 8, Building Many-to-Point XBee Networks This chapter provides a basic overview of how to handle multiple sensor platforms and request data using a single master multiple slave configuration, as well as a multi-master system, using zigbee protocols. Chapter 9, Visualizing with Processing This chapter will show you how to take the data your sensor network has collected and begin to visualize it using Processing. Chapter 10, Visualizing with LabVIEW This chapter gives you some more data visualization options with an introduction to LabVIEW. Chapter 11, Going Further This chapter provides a collection of pointers to more advanced material on the topics we covered in the book, and material covering some of those topics that we didn’t manage to talk about in this book. Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, and file extensions. Constant width Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords. Constant width bold Shows commands or other text that should be typed literally by the user. Constant width italic Shows text that should be replaced with user-supplied values or by values determined by context. NOTE This icon signifies a tip, suggestion, or general note. WARNING This icon signifies a warning or caution. Using Code Examples This book is here to help you get your job done. In general, if this book includes code examples, you may use the code in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission. We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Distributed Network Data by Alasdair Allan and Kipp Bradford (O’Reilly). Copyright 2013 Alasdair Allan and Kipp Bradford, 978-1-449-36026-9.” If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at [email protected]. Safari® Books Online Safari Books Online (www.safaribooksonline.com) is an on-demand digital library that delivers expert content in both book and video form from the world’s leading authors in technology and business. Technology professionals, software developers, web designers, and business and creative professionals use Safari Books Online as their primary resource for research, problem solving, learning, and certification training. Safari Books Online offers a range of product mixes and pricing programs for organizations, government agencies, and individuals. Subscribers have access to thousands of books, training videos, and prepublication manuscripts in one fully searchable database from publishers like O’Reilly Media, Prentice Hall Professional, Addison-Wesley Professional, Microsoft Press, Sams, Que, Peachpit Press, Focal Press, Cisco Press, John Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks, Packt, Adobe Press, FT Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, Course Technology, and dozens more. For more information about Safari Books Online, please visit us online. How to Contact Us Please address comments and questions concerning this book to the publisher: O’Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, CA 95472 800-998-9938 (in the United States or Canada) 707-829-0515 (international or local) 707-829-0104 (fax) We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at http://oreil.ly/distributed-network-data. To comment or ask technical questions about this book, send email to [email protected]. For more information about our books, courses, conferences, and news, see our website at http://www.oreilly.com. Find us on Facebook: http://facebook.com/oreilly Follow us on Twitter: http://twitter.com/oreillymedia Watch us on YouTube: http://www.youtube.com/oreillymedia Acknowledgments by Alasdair Allan Everyone has one book in them, however this isn’t mine. This isn’t my first book, but it’s my first with a coauthor, and that’s a very different experience. Books do not write themselves, and I’d like to thank my coauthor, Kipp Bradford, and my editors at O’Reilly, Julie Steele and Brian Jepson, for holding my hand through the process. I very much want to thank my wife Gemma Hobson for her continued support and encouragement, and for letting me fly to the States for the week before Christmas to camp out in my coauthor’s living room without complaint. Those small (and sometimes larger) sacrifices an author’s spouse routinely has to make don’t get any less inconvenient the second, or third, or the nth time around. I’m not sure why she lets me write, perhaps because I claimed to enjoy it so much. Thank you Gemma. Finally to my son Alex, who is just now learning to read, and won’t be reading these words for some time to come. Thank you. Acknowledgments by Kipp Bradford This is my first book. For all the projects I’ve done and stories I’d like to tell, this is the first time I’ve actually sat down to write something. Hopefully I have many more books