Cognitive Computinng and BBiiggg DDaattaa AAnnaaaallyyyttiiccss Judith HHurwitz Marcia KKaufman Adriann Bowles Cognitive Computing and Big Data Analytics Published by John Wiley & Sons, Inc. 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright © 2015 by John Wiley & Sons, Inc., Indianapolis, Indiana Published simultaneously in Canada ISBN: 978-1-118-89662-4 ISBN: 978-1-118-89678-5 (ebk) ISBN: 978-1-118-89663-1 (ebk) Manufactured in the United States of America 10 9 8 7 6 5 4 3 2 1 No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley .com/go/permissions. Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifi cally disclaim all warranties, including without limitation warranties of fi tness for a particular purpose. No warranty may be created or extended by sales or promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the services of a competent professional person should be sought. Neither the publisher nor the author shall be liable for damages arising herefrom. The fact that an organization or Web site is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or website may provide or recommendations it may make. Further, readers should be aware that Internet websites listed in this work may have changed or disappeared between when this work was written and when it is read. For general information on our other products and services please contact our Customer Care Department within the United States at (877) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visitwww.wiley.com. Library of Congress Control Number: 2014951020 Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affi liates, in the United States and other countries, and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book. Executive Editor: Carol Long Professional Technology & Strategy Director: Project Editor: Tom Dinse Barry Pruett Technical Editors: Mike Kowolensko, Business Manager: Amy Knies James Kobielus, Al Nugent Associate Publisher: Jim Minatel Production Manager: Kathleen Wisor Project Coordinator, Cover: Patrick Redmond Copy Editor: Apostrophe Editing Services Proofreader: Jen Larsen, Word One Manager of Content Development & Assembly: Indexer: Johnna VanHoose Dinse Mary Beth Wakefi eld Cover Designer: Wiley Marketing Director: David Mayhew Cover Image: © iStock.com/Andrey Prokhorov Marketing Manager: Carrie Sherrill We would like to dedicate this book to the power of collaboration. We would like to thank the rest of the team at Hurwitz & Associates for their guidance and support: Dan Kirsch, Vikki Kolbe, and Tricia Gilligan. —The Authors To my husband Warren and my two children, Sara and David. I also dedicate this book to my parents Elaine and David Shapiro. —Judith Hurwitz To my husband Matt and my children, Sara and Emily for their support through this writing process. —Marcia Kaufman To Jeanne, Andrew, Chris, and James, whose unfailing love and support allowed me to disappear long enough to write. —Adrian Bowles About the Technical Editors Al Nugent is a managing partner at Palladian Partners, LLC. He is an expe- rienced technology leader and industry veteran of more than three decades. At Palladian Partners, he leads the organization’s technology assessment and strategy practices. Al has served as executive vice president, chief technology offi cer, senior vice president, and general manager of the Enterprise Systems Management business unit at CA Technologies. Previously, he was senior vice president and CTO at Novell, Inc., and has held CTO positions at BellSouth and Xerox. He is an independent member of the Board of Directors for Telogis and Adaptive Computing, and is an advisor to several early/mid-stage technology and healthcare startups. He is a co-author of Big Data For Dummies (John Wiley & Sons, 2013). James Kobielus is a big data evangelist at IBM and a senior program director of product marketing and Big Data analytics solutions. He is an industry veteran, a popular speaker, social media participant, and a thought leader in big data, Hadoop, enterprise data warehousing, advanced analytics, business intelligence, data management, and next best action technologies. Dr. Michael D. Kowolenkois currently an industrial fellow at the Center for Innovation Management Studies (CIMS) based at the N.C. State Poole College of Management. His research is focused on the interface of technology and business decision making. Prior to joining CIMS, he was a senior vice president at Wyeth Biotech Technical Operations and Product Supply (TO&PS), provid- ing strategic and operations leadership perspective to ongoing integrated and cross-functional global business decisions. iv About thee Authors Judith S. Hurwitz is president and CEO of Hurwitz & Associates, LLC, a research and consulting fi rm focused on emerging technology including Big Data, cogni- tive computing, cloud computing, service management, software development, and security and governance. She is a technology strategist, thought leader, and author. A pioneer in anticipating technology innovation and adoption, she has served as a trusted advisor to many industry leaders over the years. Judith has helped these companies make the transition to a new business model focused on the business value of emerging platforms. She was the founder of CycleBridge, a life science software consulting fi rm, and Hurwitz Group, a research and consulting fi rm. She has worked in various corporations including Apollo Computer and John Hancock. Judith has written extensively about all aspects of enterprise and distributed software. In 2011, she authored Smart or Lucky? How Technology Leaders Turn Chance into Success (Jossey Bass, 2011). Judith is a co-author on sixFor Dummies books, including Big Data For Dummies, Hybrid Cloud For Dummies, Cloud Computing For Dummies,Service Management For Dummies, andService Oriented Architecture For Dummies, 1st and 2nd Editions (all John Wiley & Sons). Judith holds B.S. and M.S. degrees from Boston University. She serves on several advisory boards of emerging companies. She is a member of Boston University’s Alumni Council. She was named a distinguished alumnus at Boston University’s College of Arts & Sciences in 2005. She is also a recipient of the 2005 Massachusetts Technology Leadership Council award. v vi About the Authors Marcia A. Kaufman is COO and principle analyst at Hurwitz & Associates, LLC, a research and consulting fi rm focused on emerging technology including Big Data, cognitive computing, cloud computing, service management, software development, and security and governance. She has authored major studies on advanced analytics and has written extensively on cloud infrastructure, Big Data, and security. Marcia has more than 20 years of experience in business strategy, industry research, distributed software, software quality, information management, and analytics. Marcia has worked within the fi nancial services, manufacturing, and services industries. During her tenure at Data Resources Inc. (DRI), she developed econometric industry models and forecasts. She holds an A.B. degree from Connecticut College in mathematics and economics and an M.B.A. degree from Boston University. Marcia is a co-author on six retail For Dummies books including Big Data For Dummies, Hybrid Cloud For Dummies,Cloud Computing For Dummies, Service Management For Dummies, andService Oriented Architecture For Dummies, 1st and 2nd Edition (all John Wiley & Sons). Dr. Adrian Bowles is the founder of STORM Insights, Inc., a research and advisory fi rm providing services for buyers, sellers, and investors in emerg- ing technology markets. Previously, Adrian founded the Governance, Risk Management & Compliance Roundtable for the Object Management Group, the IT Compliance Institute with 101 Communications, and Atelier Research. He has held executive positions at Ovum (Datamonitor), Giga Information Group, New Science Associates, and Yourdon, Inc. Adrian’s focus on cogni- tive computing and analytics naturally follows his graduate studies. (His fi rst natural language simulation application was published in the proceedings of the International Symposium on Cybernetics and Software.) Adrian also held academic appointments in computer science at Drexel University and SUNY- Binghamton, and adjunct faculty positions in the business schools at NYU and Boston College. Adrian earned his B.A. degree in psychology and M.S. degree in computer s cience from SUNY-Binghamton, and his Ph.D. degree in computer science from Northwestern University. Acknowleddggmmeennttss Writing a book on a topic as complex as cognitive computing required a tre- mendous amount of research. Our team read hundreds of technical articles and books on various aspects of technology underpinning of the fi eld. In addition, we were fortunate to reach out to many experts who generously spent time with us. We wanted to include a range of perspectives. So, we have many people to thank. We are sure that we have left out individuals who we met at conferences and provided insightful discussions on topics that infl uenced this book. We would also like to acknowledge the partnership and collaboration among the three of us that allowed this book to be written. We would also like to thank our editors at Wiley, including Carol Long and Tom Dinse. We appreciate the insights and assistance from our three technical editors, Al Nugent, James Kobielus, and Mike Kowolenko. The following people gave generously of their time and insights: Dr. Manny Aparicio; Avron Barr, Aldo Ventures; Jeff Cohen, Welltok; Dr. Umesh Dayal, Hitachi Data Systems; Stephen DeAngelis, Enterra; Rich Y. Edwards, IBM; Jeff Eisen, IBM; Tim Estes, Digital Reasoning; Sara Gardner, Hitachi Data Systems; Murtaza Ghadyali, Refl exis; Stephen Gold, IBM; Manish Goyal, IBM; John Gunn, Memorial Sloan Kettering Cancer Center; Sue Feldman, Synthexis; Dr. Fern Halper, TDWI; Dr. Kris Hammond, Narrative Science; Ed Harbor, IBM; Marten den Haring, Digital Reasoning; Dr. C. Martin Harris, Cleveland Clinic; Dr. Larry Harris; Dr. Erica Hauver, Hitachi Data Systems; Jeff Hawkins, Numenta and The Redwood Center for Theoretical Neuroscience; Rob High, IBM; Holly T. Hilbrands, IBM; Dr. Paul Hofmann, Space-Time Insight; Amir Husain, Sparkcognition, Inc.; Terry Jones, WayBlazer; Vikki Kolbe, Hurwitz & Associates; Michael Karasick, IBM; Niraj Katwala, Healthline Networks, Inc.; Dr. John Kelly, IBM; Natsuko Kikutake, Hitachi Consulting Co., LTD; Daniel Kirsch, Hurwitz & Associates; Jeff vii viii Acknowledgments Margolis, Welltok; D.J. McCloskey, IBM; Alex Niznik, Pfi zer; Vincent Padua, IBM; Tapan Patel, SAS Institute; Santiago Quesada, Repsol; Kimberly Reheiser, IBM; Michael Rhoden, IBM; Shay Sabhikhi, Cognitive Scale; Matt Sanchez, Cognitive Scale; Chandran Saravana, SAP; Manoj Saxena, Saxena Foundation; Dr. Candy Sidner, Worchester Polytechnic Institute; Dean Stephens, Healthline Networks, Inc.; Sridhar Sudarsan, IBM; David E. Sweenor, Dell; Wayne Thompson, SAS Institute; Joe Turk, Cleveland Clinic; and Dave Wilson, Hitachi Data Systems. —Judith Hurwitz —Marcia Kaufman —Adrian Bowles