BELFER CENTER STUDY Artificial Intelligence and National Security Greg Allen Taniel Chan A study on behalf of Dr. Jason Matheny, Director of the U.S. Intelligence Advanced Research Projects Activity (IARPA) STUDY JULY 2017 Belfer Center for Science and International Affairs Harvard Kennedy School 79 JFK Street Cambridge, MA 02138 www.belfercenter.org Statements and views expressed in this report are solely those of the authors and do not imply endorsement by Harvard University, Harvard Kennedy School, the Belfer Center for Science and International Affairs, or IARPA. Design & Layout by Andrew Facini Cover photo and opposite page 1: Adobe Stock, Illustration Copyright 2017, President and Fellows of Harvard College Printed in the United States of America BELFER CENTER PAPER Artificial Intelligence and National Security Greg Allen Taniel Chan A study on behalf of Dr. Jason Matheny, Director of the U.S. Intelligence Advanced Research Projects Activity (IARPA) STUDY JULY 2017 Acknowledgments We would like to thank our advisors at Harvard Kennedy School and Harvard Business School, Dr. Joseph Nye Jr. and Dr. Gautam Mukunda, respectively, for their insight and dedication to the success of this project. We also thank Dr. Matthew Bunn and Dr. John Park, who provided timely feedback at multiple stages of this effort. We are grateful to the Belfer Center for Science and International Affairs, the Mossavar-Rahmani Center for Business and Government, and the Social Enterprise Initiative for their generous financial support, without which this research would not have been possible. We would also like to thank our client, Dr. Jason Matheny, who went out of his way to support this effort at every step. We thank the dozens of experts across government, industry, and academia who shared their time and expertise in being interviewed. We would especially like to thank Matt Daniels at the Department of Defense’s Office of Net Assessment. Matt’s brilliant insights can be found throughout this document. Finally, we would like to thank those individuals who took the time to review and provide feedback on early drafts of this document, including Dr. Edward Felten, Dr. Richard Danzig, Dr. Lynne Parker, Ambassador Richard Norland, and Dr. Randy Bryant. Without their assistance, this document would have been far weaker. Any remaining mistakes are ours alone. Project Overview Partially autonomous and intelligent systems have been used in military technology since at least the Second World War, but advances in machine learning and Artificial Intelligence (AI) represent a turning point in the use of automation in warfare. Though the United States military and intelligence communities are planning for expanded use of AI across their portfolios, many of the most transformative applications of AI have not yet been addressed. In this piece, we propose three goals for developing future policy on AI and national security: preserving U.S. technological leadership, supporting peaceful and commercial use, and mitigating catastrophic risk. By look- ing at four prior cases of transformative military technology—nuclear, aerospace, cyber, and biotech—we develop lessons learned and recommen- dations for national security policy toward AI. About the Authors Greg Allen is an Adjunct Fellow at the Center for A New American Security in Technology and National Security Program. Mr. Allen focuses on the intersection of Artificial Intelligence, cybersecurity, robotics, and national security. His writing and analysis has appeared in WIRED, Vox, and The Hill. Mr. Allen holds a joint MPP/MBA degree from the Harvard Kennedy School of Government and the Harvard Business School. Find him on twitter @gregory_c_allen Taniel Chan was the Associate Director of Strategy and Analytics for the NYC Department of Education and a financial and economic analyst at Goldman Sachs. Mr. Chan also worked for the White House National Economic Council on technology industry policy and workforce development. This fall, he will be join Bain & Company in London. Mr. Chan holds a joint MPP/MBA degree from the Harvard Kennedy School of Government and the Harvard Business School. Table of Contents Executive Summary .................................................................................................1 Introduction & Project Approach ............................................................................7 Part 1: The Transformative Potential of Artificial Intelligence ...........................12 Implications for Military Superiority ..............................................................................................................12 Implications for Information Superiority.......................................................................................................27 Implications for Economic Superiority ..........................................................................................................35 Part 2: Learning from Prior Transformative Technology Cases .........................42 Key Technology Management Aspects ..........................................................................................................42 Government Technology Management Approach .......................................................................................44 Government Management Approach “Scorecard” .......................................................................................45 AI Technology Profile: A Worst-case Scenario? ...........................................................................................46 Lessons Learned ............................................................................................................................................48 Part 3: Recommendations for Artificial Intelligence and National Security ....58 Preserving U.S. Technological Leadership .....................................................................................................58 Supporting Peaceful Use of AI Technology ..................................................................................................64 Mitigating Catastrophic Rsk ...........................................................................................................................67 Conclusion ..............................................................................................................70 Appendix: Transformative National Security Technology Case Studies..........71 Case Study #1: Nuclear Technology ..............................................................................................................71 Case Study #2: Aerospace Technology .........................................................................................................82 Case Study #3 Internet and Cyber Technology ............................................................................................91 Case Study #4 Biotechnology ......................................................................................................................100 Citations ..........................................................................................................................................................111 Executive Summary • Researchers in the field of Artificial Intelligence (AI) have demonstrated significant technical progress over the past five years, much faster than was previously anticipated. − Most of this progress is due to advances in the AI sub-field of machine learning. − Most experts believe this rapid progress will continue and even accelerate. • Most AI research advances are occurring in the private sector and academia. − Private sector funding for AI dwarfs that of the United States Government. • Existing capabilities in AI have significant potential for national security. − For example, existing machine learning technology could enable high degrees of automation in labor-intensive activities such as satellite imagery analysis and cyber defense. • Future progress in AI has the potential to be a transformative national security technology, on a par with nuclear weapons, aircraft, computers, and biotech. − Each of these technologies led to significant changes in the strategy, organization, priorities, and allocated resources of the U.S. national security community. − We argue future progress in AI will be at least equally impactful. Belfer Center for Science and International Affairs | Harvard Kennedy School 1 • Advances in AI will affect national security by driving change in three areas: military superiority, information superiority, and eco- nomic superiority. − For military superiority, progress in AI will both enable new capa- bilities and make existing capabilities affordable to a broader range of actors. ■ For example, commercially available, AI-enabled technology (such as long-range drone package delivery) may give weak states and non-state actors access to a type of long-range preci- sion strike capability. ■ In the cyber domain, activities that currently require lots of high-skill labor, such as Advanced Persistent Threat operations, may in the future be largely automated and easily available on the black market. − For information superiority, AI will dramatically enhance capabilities for the collection and analysis of data, and also the creation of data. ■ In intelligence operations, this will mean that there are more sources than ever from which to discern the truth. However, it will also be much easier to lie persuasively. ■ AI-enhanced forgery of audio and video media is rapidly improving in quality and decreasing in cost. In the future, AI-generated forgeries will challenge the basis of trust across many institutions. − For economic superiority, we find that advances in AI could result in a new industrial revolution. ■ Former U.S. Treasury Secretary Larry Summers has predicted that advances in AI and related technologies will lead to a dra- matic decline in demand for labor such that the United States “may have a third of men between the ages of 25 and 54 not working by the end of this half century.” 2 Artificial Intelligence and National Security