NO MORE SECRETS vint.sogeti.com with Big Data Analytics N O M O R E S E C R E T S V IN T | V About Sogeti isio n Sogeti is a leading provider of professional technology services, specializing • In in Application Management, Infra structure Management, High-Tech sp Engineering and Testing. Working closely with its clients, Sogeti enables them iratio to leverage technological innovation and achieve maximum results. Sogeti •n N brings together more than 20,000 professionals in 15 countries and is present avig in over 100 locations in Europe, the us and India. For more information please a tio visit www.sogeti.com. n • T re n About vint ds It is an arduous undertaking to attempt to keep up on all developments in the it field. The state-of-the-art it opportunities are often very far from the workings of the core business. Sources that provide a deeper understanding, a pragmatic approach, and potential uses for these developments are few ISBN 978-90-75414-73-8 and far between. vint, a Sogeti research institute, provides a meaningful interpretation of the connection between business processes and new developments in it. A balance is struck between factual description and intended utilization in every report drawn up by vint for the investigations it carries out. vint uses Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen, Erik van Ommeren, Sandeep Sachdeva this approach to inspire organizations to consider and use new technology. VINT | Vision • Inspiration • Navigation • Trends No More Secrets with Big Data Analytics Jaap Bloem, Menno van Doorn, Sander Duivestein, Thomas van Manen, Erik van Ommeren, Sandeep Sachdeva VINT | Vision • Inspiration • Navigation • Trends Attribution-NonCommercial-ShareAlike 3.0 Uported (CC BY-NC-SA 3.0) 2013 The Sogeti Trend Lab VINT | Vision • Inspiration • Navigation • Trends Book production LINE UP boek en media bv, Groningen Table of Contents Foreword from the vint Board of Recommendation 5 Introduction The Future of Big Data Analytics 7 Part I Creating Clarity with Big Data 11 1 Digital data as the new industrial revolution 12 2 Total data management in each organization 15 3 Participate in our Big Data discussion at vint.sogeti.com 17 4 Why the word “Big”? 19 5 The importance of Big Data 22 6 Big Data is Extreme Information Management 24 7 Big Data in organizations 27 8 With Big Data from Big Science to Big Business 31 9 Big Data as new Data Science era 34 10 Closing the knowledge gap is essential 35 11 Big Data in hard cash 37 12 Summary 39 Part II Your Big Data Potential – The Art of the Possible 41 1 “The Art of the Possible” 42 2 Your Potential: The tension between Could Be and Is 46 Question 1: Why Big Data intelligence? 50 Question 2: What new insights can I expect? 56 Question 3: How will these insights help me? 59 Question 4: What skills do I need? 60 Question 5: How do Big Data pioneers organize data management and it processes? 62 Question 6: How can I merge my structured and unstructured data? 66 Question 7: Which new technologies should I be watching? 70 Question 8: What is looming on the horizon? 75 Question 9: What does this mean in organizational terms? 78 Question 10: How does this affect everyday life? 84 Part III Big Social – Predicting Behavior with Big Data 87 1 What’s next in Big Data? – Nine observations with both feet on the ground 88 2 Rhythms of human activity 93 3 More data for better answers 95 3 4 Total Data Management: the ‘Big Five’ social sources 97 5 How Web and Social Analytics became entwined 101 6 Toward Next-Generation Analytics 103 7 Data and algorithms instead of models 105 8 Social media as lens distortion 110 9 The toolbox is bursting at the seams 111 10 Start by listening attentively 114 11 The strength of Big Social Data 121 12 Summary and the organization of privacy 125 Eight central Big Social definitions 127 Part IV Privacy, Technology and the Law – Big Data for Everyone through Good Design 131 Introduction – Reaping the fruits of Big Data 132 1 An anatomy of Big Data anxiety 138 2 What is privacy? 151 3 Privacy by Design and the balance between pits and pets 166 4 Legislation in a state of flux 174 5 Seven privacy principles and your next step 182 No More Secrets Management Summary 187 Literature and Illustrations 191 Part i 192 Part ii 193 Part iii 195 Part iv 198 4 no more secrets with big data analytics Foreword from the Board of Recommendation vint Daily blog posts about Big Data and four reports on this new field preceded this book. In No More Secrets the authors combine their findings. Their focus is the new opportunities and challenges offered by Big Data Analytics. The insight that data is the new oil catches on and Big Data has become a prominent topic of discussion in boardrooms. Big Data Analytics is becoming good practice in ever more domains: this fact lingers most after reading No More Secrets. Some organizations are well under way, others are accelerating or have started experiments. The consequence of this burgeoning Big Data success now starts to play out: secrets, big and small, have no future. Searching for oil or the way dna rules our health used to be covered in mystery, as was con- sumer behavior. Thanks to advanced Big Data Analytics one secret after another is now being unraveled. Rooted in data-intensive science, Big Data Analytics now is being deployed every- where. The economy and everyday life are full of guiding examples. Big Data tech- niques help the discovery of buying patterns and the detection of fraud. The cases in this book on predicting human behavior from all digital traces we leave behind, incite reflection and inspiration. Based on real-time tweets the inflation rate can be deter- mined: one of those remarkable things that Big Data Analytics has made possible. No More Secrets may provide the basis for updating or refining your understanding of Big Data Analytics and for exploring new ground. The first Part sheds light on the Big Data phenomenon in general. Part ii presents ample suggestions for determining your specific Big Data potential. These you can readily apply to gain insight in what exactly makes your customers tick, the topic of Part iii. The triad of privacy, technology and the law concludes the book, while the introductory section after this foreword sketches out the actual context of Big Data along five leading themes. The warning that Big Data Analytics is not a technology toy but an integral part of strategy, marketing, hr or R&D may already be familiar. The authors go even further and urge you to create “magic moments,” since abandoning conventional thinking and patterns will cost at least as much energy as getting your unstructured data analysis right. Foreword 5 The answer to the fundamental question of what it means to live in a world where secrets of consumers and citizens, and mysteries of our Earth and life itself have been unraveled, became top of mind when Edward Snowden had his revelations published about the digital practices of secret services. These Big Data Analytics fully justify the title No More Secrets. Suddenly, Big Data itself had lost its final mystery as this Big Brotherhood of competing and cooperating agencies and companies for the sake of safety was exposed. In this book the authors decided to focus soleley on the fair commercial use and busi- ness impact of Big Data Analytics. We sincerely hope that the insights and ideas in No More Secrets will contribute to successful new strategies, innovative implementations, and faster and better business decisions. Board of Recommendation Sogeti vint H. Wesseling (Chairman of the Board), former Chief Information Officer Postnl H.W. Broeders, former Chief Executive Officer Capgemini Netherlands P. Dirix, Chief Operations Officer ProRail N. Jongerius, former Chief Information Officer sns Reaal D. Kamst, Program Director it@rwe2015 rwe T. van der Linden, Group Information Officer Achmea Prof. Dr. Ir. R. Maes, Professor Information and Communication Management Academy for I&M P. Morley, Chairman Artilium J. Muchez, Managing Director Morgan Clark & Company E. Schuchmann, Chief Information Officer Academic Medical Center Amsterdam P.W. Wagter, Chief Executive Officer Sogeti Netherlands J.P.E. van Waayenburg, Chief Operations Officer Sogeti Group A. van Zanen-Nieberg, General Director Governmental Audit Services, Ministry of Finance of The Netherlands 6 no more secrets with big data analytics Introduction The Future of Big Data Analytics While working on our Big Data reports and on this book, we met many marketeers and geeks, cios and managers, lawyers, activists, forerunners, followers and laggards in the emerging field of Big Data Analytics. Five main themes surfaced: acceleration, transformation, data ownership, privacy and Edward Snowden. 1 Big Data into gear Acceleration is an important one that Big Data projects have in common. The V of Velocity turns out to be the most popular of the defining Big Data triad of Volume, Variety and Velocity. Any Variety or Volume focus always leads to the question of how to conduct Big Data Analytics. Velocity however is less about technology and more about the possibilities, about performance and about business impact. Being able to rapidly process and analyze vast amounts of unstructured data is crucial. Consumers and citizens expect immediate response. They tweet their messages about what’s on their minds and want to be adequately served by webcare teams. The conclusion is obvious: as long as it can be faster, new Big Data technologies and applications continue to evolve. Time is money and acceleration is synonymous with competitive advantage. Real-time isn’t fast enough: predicting what will happen next is the real ambition, moving from predictive to prescriptive analysis. 2 Big Data transformation The ideal data scientist is a much discussed colleague. This all-rounder must embody all the competencies for an organization to become Big Data driven, a transforma- tion that requires direction and support from the top. For instance to start a statisti- cal agency using mobile phone data. Insights from location data and people moving around have proven to be an excellent alternative for charting consumer behavior. But every time Big Data comes in there will remain much resistance and conservatism to counter, since as Clay Shirky puts it: “Institutions will preserve the problem to which they are the solution.” bi experts for instance, may have many solutions for problems that they cherish: tra- ditional Business Intelligence and Big Data Analytics still are worlds apart. Introduction 7 3 Own your data and your future It could be a time bomb under many Big Data initiatives: consumers and citizens who rule their own data. This scenario was discussed by an international group of cios at the Sogeti Big Data Summit 2012 where Doc Searls, author of The Intention Economy: when customers take charge, hosted the meeting. Searls argues that individuals them- selves are in the best position to monetize their personal data, which might overthrow the relationship with organizations. Ultimately, Vendor Relationship Management would replace Customer Relationship Management. In his book Who Owns the Future? Jaron Lanier also pleads that people should make money from their own per- sonal data. For Lanier this could be a solution to the Big Data Analytics problem that people will lose their jobs in a world af advanced smart systems and devices. 4 Privacy in the picture How do we deal with privacy or what is left of it? When The Guardian started pub- lishing Edward Snowden’s revelations about the Big Data practices of secret services, it was fuel to an existing fire. The privacy issue always has been top of mind when per- sonal data were involved. We now follow the great privacy debate on a daily basis in the media as we are painfully aware of the fact that no secrets are safe since agencies and organizations have access to much more information than we would like. Organizations and governments of course have been collecting enormous amounts of data that can be related to an individual, but this kind of Personally Identifiable Information (pii) is protected throughout the world by privacy laws. However, in the digital age, legislation alone is insufficient by far and thanks to Big Data Analytics non-personal data also easily can lead to the right prospects. Few organizations seem to master the Privacy by Design maxim but the best advice to be trusted is: be trans- parent, comply and explain as much as you can. 5 Welcome to the No More Secrets era The question remains how much the Big Data future is influenced by the Snowden revelations which have put data protection and privacy at the center of our attention. All major U.S. online services were persuaded to participate in the largest ever moni- toring of data traffic: Big Data Analytics to the max. Friendly powers and foreign com- panies were tapped as smartphones and tablets were shamelessly searched. Encryp- tion and other security systems were cracked or had loopholes to circumvent them. A fundamental undermining of computer, data and network security, plus of privacy and data protection. No one was aware of the extent and depth of these operations. There are no secrets. In January 2010, Facebook ceo Mark Zuckerberg stated that the privacy era was completely over. In March 2012, cia chief David Petraeus conceded 8 no more secrets with big data analytics