Data Science & Big Data Analytics ffirs.indd 2:43:16:PM/12/11/2014 Page i Data Science & Big Data Analytics Discovering, Analyzing, Visualizing and Presenting Data EMC Education Services ffirs.indd 2:43:16:PM/12/11/2014 Page iii Data Science & Big Data Analytics: Discovering, Analyzing, Visualizing and Presenting Data Published by John Wiley & Sons, Inc. 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright © 2015 by John Wiley & Sons, Inc., Indianapolis, Indiana Published simultaneously in Canada ISBN: 978-1-118-87613-8 ISBN: 978-1-118-87622-0 (ebk) ISBN: 978-1-118-87605-3 (ebk) Manufactured in the United States of America 10 9 8 7 6 5 4 3 2 1 No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permis- sion of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions. Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation warranties of fitness for a particular purpose. No warranty may be created or extended by sales or promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the services of a competent professional person should be sought. Neither the publisher nor the author shall be liable for damages arising herefrom. The fact that an organization or Web site is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or website may provide or recommendations it may make. Further, readers should be aware that Internet websites listed in this work may have changed or disappeared between when this work was written and when it is read. For general information on our other products and services please contact our Customer Care Department within the United States at (877) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com. Library of Congress Control Number: 2014946681 Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates, in the United States and other coun- tries, and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book. ffirs.indd 2:43:16:PM/12/11/2014 Page iv Credits Executive Editor Professional Technology and Strategy Director Carol Long Barry Pruett Project Editor Business Manager Kelly Talbot Amy Knies Production Manager Associate Publisher Kathleen Wisor Jim Minatel Copy Editor Project Coordinator, Cover Karen Gill Patrick Redmond Manager of Content Development Proofreader and Assembly Nancy Carrasco Mary Beth Wakefield Indexer Marketing Director Johnna VanHoose Dinse David Mayhew Cover Designer Marketing Manager Mallesh Gurram Carrie Sherrill ffirs.indd 2:43:16:PM/12/11/2014 Page v About the Key Contributors David Dietrich heads the data science education team within EMC Education Services, where he leads the curriculum, strategy and course development related to Big Data Analytics and Data Science. He co-au- thored the first course in EMC’s Data Science curriculum, two additional EMC courses focused on teaching leaders and executives about Big Data and data science, and is a contributing author and editor of this book. He has filed 14 patents in the areas of data science, data privacy, and cloud computing. David has been an advisor to several universities looking to develop academic programs related to data analytics, and has been a frequent speaker at conferences and industry events. He also has been a a guest lecturer at universi- ties in the Boston area. His work has been featured in major publications including Forbes, Harvard Business Review, and the 2014 Massachusetts Big Data Report, commissioned by Governor Deval Patrick. Involved with analytics and technology for nearly 20 years, David has worked with many Fortune 500 companies over his career, holding multiple roles involving analytics, including managing analytics and operations teams, delivering analytic con- sulting engagements, managing a line of analytical software products for regulating the US banking industry, and developing Software-as-a-Service and BI-as-a-Service offerings. Additionally, David collaborated with the U.S. Federal Reserve in develop- ing predictive models for monitoring mortgage portfolios. Barry Heller is an advisory technical education consultant at EMC Education Services. Barry is a course developer and cur- riculum advisor in the emerging technology areas of Big Data and data science. Prior to his current role, Barry was a consul- tant research scientist leading numerous analytical initiatives within EMC’s Total Customer Experience organization. Early in his EMC career, he managed the statistical engineering group as well as led the data warehousing efforts in an Enterprise Resource Planning (ERP) implementation. Prior to joining EMC, Barry held managerial and analytical roles in reliability engineering functions at medical diagnostic and technology companies. During his career, he has applied his quantitative skill set to a myriad of business applications in the Customer Service, Engineering, Manufacturing, Sales/Marketing, Finance, and Legal arenas. Underscoring the importance of strong executive stakeholder engagement, many of his successes have resulted from not only focusing on the technical details of an analysis, but on the decisions that will be resulting from the analysis. Barry earned a B.S. in Computational Mathematics from the Rochester Institute of Technology and an M.A. in Mathematics from the State University of New York (SUNY) New Paltz. Beibei Yang is a Technical Education Consultant of EMC Education Services, responsible for developing several open courses at EMC related to Data Science and Big Data Analytics. Beibei has seven years of experience in the IT industry. Prior to EMC she worked as a software engineer, systems manager, and network manager for a Fortune 500 company where she introduced new technologies to improve efficiency and encourage collaboration. Beibei has published papers to prestigious conferences and has filed multiple patents. She received her Ph.D. in computer science from the University of Massachusetts Lowell. She has a passion toward natural language processing and data mining, especially using various tools and techniques to find hidden patterns and tell stories with data. Data Science and Big Data Analytics is an exciting domain where the potential of digital information is maximized for making intelligent business decisions. We believe that this is an area that will attract a lot of talented students and professionals in the short, mid, and long term. ffirs.indd 2:43:16:PM/12/11/2014 Page vi Acknowledgments EMC Education Services embarked on learning this subject with the intent to develop an “open” curriculum and certification. It was a challenging journey at the time as not many understood what it would take to be a true data scientist. After initial research (and struggle), we were able to define what was needed and attract very talented professionals to work on the project. The course, “Data Science and Big Data Analytics,” has become well accepted across academia and the industry. Led by EMC Education Services, this book is the result of efforts and contributions from a number of key EMC organizations and supported by the office of the CTO, IT, Global Services, and Engineering. Many sincere thanks to many key contributors and subject matter experts David Dietrich, Barry Heller, and Beibei Yang for their work developing content and graphics for the chapters. A special thanks to subject matter experts John Cardente and Ganesh Rajaratnam for their active involvement reviewing multiple book chapters and providing valuable feedback throughout the project. We are also grateful to the following experts from EMC and Pivotal for their support in reviewing and improving the content in this book: Aidan O’Brien Joe Kambourakis Alexander Nunes Joe Milardo Bryan Miletich John Sopka Dan Baskette Kathryn Stiles Daniel Mepham Ken Taylor Dave Reiner Lanette Wells Deborah Stokes Michael Hancock Ellis Kriesberg Michael Vander Donk Frank Coleman Narayanan Krishnakumar Hisham Arafat Richard Moore Ira Schild Ron Glick Jack Harwood Stephen Maloney Jim McGroddy Steve Todd ffirs.indd 2:43:16:PM/12/11/2014 Page vii Jody Goncalves Suresh Thankappan Joe Dery Tom McGowan We also thank Ira Schild and Shane Goodrich for coordinating this project, Mallesh Gurram for the cover design, Chris Conroy and Rob Bradley for graphics, and the publisher, John Wiley and Sons, for timely support in bringing this book to the industry. Nancy Gessler Director, Education Services, EMC Corporation Alok Shrivastava Sr. Director, Education Services, EMC Corporation ffirs.indd 2:43:16:PM/12/11/2014 Page viii