Architect’s Guide to Implementing a Digital Transformation George Demarest - author Jim Scott - contributor Table of Contents Introduction 7 The four phases: 7 Phase 1. Experimentation 7 Phase 2. Implementation 7 Phase 3. Expansion 8 Phase 4. Optimization 8 Where Did These Phases Come From? 8 Digital Transformation, Open Source, and Cloud Economics 9 The Progression of Big Data Use Cases 10 Phase 1: Experimentation 13 Motivating Factors 13 Cost takeout 13 IT preparedness 13 Initiate IT-wide data strategy 13 Key Activities 14 Identify a team 14 Skills and tooling update 14 Identify systems or applications to ofload/migrate 14 Plan for growth 14 Use Cases 15 Platform bake-ofs 15 Legacy ofload 15 iii Table of Contents Data Warehouse Ofload 16 IT focused 19 Checklist to Progress to Phase II: Implementation 20 Summary 20 Phase 2: Implementation 23 Motivating Factors 23 Key Activities 24 Use Cases 24 Data lake 24 Marketing 26 Security and Fraud Detection 27 SaaS Architecture and Enterprise Application Replatforming 29 Checklist to Progress to Phase III: Expansion 30 Summary 31 Phase 3: Expansion 35 Motivating Factors 35 Key Activities 36 Considerations for Expansion 37 Use Cases 37 Data/Analytics Platform, Analytics as a Service 38 Marketing/Sales Suite 39 Security Suite 40 Operations 43 Checklist to Progress to Phase IV: Optimization 43 Summary 45 Phase 4 Optimization 51 Motivating Factors 52 iv Table of Contents Key Activities and Use Cases 52 A Note on Phase IV Use Cases 54 The MapR Converged Data Platform in Digital Transformation 56 Summary 62 Overall Summary Grid 69 v Introduction This document is meant to provide enterprise architects, IT architects, and other IT strategists some guidance as to how organizations can progress through the various stages of becoming a data-driven business. This guide describes four phases of the journey to- ward a digital transformation The four phases: Phase 1. Experimentation • Understand capabilities of the big data ecosystem • Develop basic skills in big data management and new application architectures • Create a pilot use case • Establish and maintain a working cluster Phase 2. Implementation • Develop the first production use cases • Committ dedicated resources to big data development and operations • Garner executive sponsorship • Develop a plan for a broader digital transformation 7 Introduction Phase 3. Expansion • Expand to multiple use cases across the company • Plan participation by multiple lines of business • Establish a dedicated command and control structure for digital transformation • Establish IT SLAs, ROI metrics, and growth plan for data-driven operations Phase 4. Optimization • Optimize and integrate apps on converged data platform • Establish digital business practices as the new normal supported by all key executive sponsors • Provide detailed business SLAs, revenue targets, and other financial targets • Normalize data lifecycle/governance, data monetization, microservice development Where Did These Phases Come From? These phases represent discernable patterns that we have observed from hundreds of engagements with MapR customers. While the phases are presented sequentially, individu- al experiences can vary significantly depending on the commitment of your organization, whether or not you have a motivated executive sponsor, external pressure from competi- tors in your industry, and other factors like budget, politics, and culture. MapR customer examples (both named and anonymized) will be used to illustrate key points of the big da- ta journey. The rapid growth of big data development is the result of major shifs in the broader information technology space. The combined impact of growing data volumes, accelerat- ing adoption of open source technologies, and the trend in shortening the application de- velopment cycle have all contributed to the big data phenomenon. The rate of growth in the volume, variety, and velocity of data is accelerating. In the fixed Internet of the 90s, there were 1 Billion Internet connections. With the mobile Internet of the 2000s, it grew to 6 8 Introduction billion. And by 2020, according to Cisco there will be a projected 50 billion connections with the Internet of Things. Digital Transformation, Open Source, and Cloud Economics Legacy systems were never designed to handle this scale of data, yet a true digital trans- formation demands that you find a way. What many IT executives are discovering is that legacy practices, and more importantly, legacy economics, are being challenged by new digital platforms that exploit open source tools, and cost efective distributed computing. Couple that with the ability to develop these new applications on premises or in the cloud on virtualized infrastructure brings with it new financial models for IT that could be termed cloud economics. More and more application architects and developers are asking questions like: • Does my application really need a RDBMS? • Is there a free or open source alternative to the commercial sofware I am using? • Can I run a mission-critical application without any commercial sofware? In our own customer base, there are dozens of examples of re-platforming of applica- tions or analytics from legacy platform such as mainframes, data warehouses/RDBMS, and premium storage arrays. The Hadoop/Spark ecosystem first grew in popularity because it provided an economical way to store massive amounts of data and do bulk processing on these large data sets. 9 Introduction This proliferation of new data, new tools, and new thinking ofers organizations tremen- dous opportunities to reach and serve customers through new application architecture. The growing utility and influence of machine learning, artificial intelligence, advanced ana- lytics, data engineering, statistical analysis, and data science vastly expand the lexicon of “business intelligence”. The impact of such a sea change in approach to data-driven busi- ness processes in the digital business suggests the ability to reimagine the role of IT in the business. The Progression of Big Data Use Cases While every organization has a unique entry point into the world of big data, we have been able to observe some fairly consistent patterns of use case development from our customer engagements. During the early stages of big data adoption, IT departments sen- sibly focus on easy wins that focus on cost savings, gaining valuable practical experience, and laying the groundwork for more ambitious and sophisticated projects. The graphic below roughly outlines how use case development progresses in most cus- tomer environments. There are, of course, exceptions. Typically, those exceptions stem from the customer having a critical use case or competitive situation that accelerates adoption of the technology. Use cases such as security analytics, fraud detection, market- ing use cases, or Internet of Things (IoT) projects can fall into this category. While use cases are a key measure of the maturity and sophistication of a digital trans- formation, this document presents a broader set of indicators that are meant to inform en- terprise architects, IT leadership, and application architects. While it is dificult to general- ize the economic benefits of big data or the MapR Converged Data Platform, we have hun- 10 Introduction dreds of examples of customers achieving significant — sometimes radical — costs savings, new revenue streams, and high returns on investment. See the IDC report. Finally, the fast pace of change in the big data ecosystem, the great strides being made in data engineering and data science, and the growing number of “data-driven” business and IT leaders mean that the phases described below will also evolve over time. This cur- rent version is based on conditions on the ground today and is based completely on real data and experience from MapR customers to date. 11