NUMBERS, FACTS AND TRENDS SHAPING THE WORLD FEBRUARY 8, 2017 BYLee Rainie and Janna Anderson FOR MEDIA OR OTHER INQUIRIES: Lee Rainie, Director, Pew Research Internet, Science and Technology Project Janna Anderson, Director, Elon University’s Imagining the Internet Center Dana Page, Senior Communications Manager 202.419.4372 www.pewresearch.org RECOMMENDED CITATION: Rainie, Lee and Janna Anderson, “Code-Dependent: Pros and Cons of the Algorithm Age. Pew Research Center, February 2017. Available at: http://www.pewinternet.org/2017/02/08/code-dependent-pros-and-cons-of-the-algorithm-age EMBARGOED COPY – NOT FOR PUBLICATION OR DISTRIBUTION UNTIL 00:00 a.m./p.m. EDT, MONTH DATE, YEAR 1 PEW RESEARCH CENTER About Pew Research Center Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping America and the world. It does not take policy positions. The center conducts public opinion polling, demographic research, content analysis and other data-driven social science research. It studies U.S. politics and policy; journalism and media; internet, science and technology; religion and public life; Hispanic trends; global attitudes and trends; and U.S. social and demographic trends. All of the center’s reports are available at www.pewresearch.org. Pew Research Center is a subsidiary of The Pew Charitable Trusts, its primary funder. For this project, Pew Research Center worked with Elon University’s Imagining the Internet Center, which helped conceive the research, collect, and analyze the data. © Pew Research Center 2016 www.pewresearch.org 2 PEW RESEARCH CENTER Code-Dependent: Pros and Cons of the Algorithm Age Algorithms are instructions for solving a problem or completing a task. Recipes are algorithms, as are math equations. Computer code is algorithmic. The internet runs on algorithms and all online searching is accomplished through them. Email knows where to go thanks to algorithms. Smartphone apps are nothing but algorithms. Computer and video games are algorithmic storytelling. Online dating and book-recommendation and travel websites would not function without algorithms. GPS mapping systems get people from point A to point B via algorithms. Artificial intelligence (AI) is naught but algorithms. The material people see on social media is brought to them by algorithms. In fact, everything people see and do on the web is a product of algorithms. Every time someone sorts a column in a spreadsheet, algorithms are at play, and most financial transactions today are accomplished by algorithms. Algorithms help gadgets respond to voice commands, recognize faces, sort photos and build and drive cars. Hacking, cyberattacks and cryptographic code-breaking exploit algorithms. Self-learning and self-programming algorithms are now emerging, so it is possible that in the future algorithms will write many if not most algorithms. Algorithms are often elegant and incredibly useful tools used to accomplish tasks. They are mostly invisible aids, augmenting human lives in increasingly incredible ways. However, sometimes the application of algorithms created with good intentions leads to unintended consequences. Recent news items tie to these concerns:  The British pound dropped 6.1% in value in seconds on Oct. 7, 2016, partly because of currency trades triggered by algorithms.  Microsoft engineers created a Twitter bot named “Tay” this past spring in an attempt to chat with Millennials by responding to their prompts, but within hours it was spouting racist, sexist, Holocaust-denying tweets based on algorithms that had it “learning” how to respond to others based on what was tweeted at it.  Facebook tried to create a feature to highlight Trending Topics from around the site in people’s feeds. First, it had a team of humans edit the feature, but controversy erupted when some www.pewresearch.org EMBARGOED COPY – NOT FOR PUBLICATION OR DISTRIBUTION UNTIL 00:00 a.m./p.m. EDT, MONTH DATE, YEAR 3 PEW RESEARCH CENTER accused the platform of being biased against conservatives. So, Facebook then turned the job over to algorithms only to find that they could not discern real news from fake news.  Cathy O’Neil, author of Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, pointed out that predictive analytics based on algorithms tend to punish the poor, using algorithmic hiring practices as an example.  Well-intentioned algorithms can be sabotaged by bad actors. An internet slowdown swept the East Coast of the U.S. on Oct. 21, 2016, after hackers bombarded Dyn DNS, an internet traffic handler, with information that overloaded its circuits, ushering in a new era of internet attacks powered by internet-connected devices. This after internet security expert Bruce Schneier warned in September that “Someone Is Learning How to Take Down the Internet.” And the abuse of Facebook’s News Feed algorithm and general promulgation of fake news online became controversial as the 2016 U.S. presidential election proceeded.  Researcher Andrew Tutt called for an “FDA for Algorithms,” noting, “The rise of increasingly complex algorithms calls for critical thought about how to best prevent, deter and compensate for the harms that they cause …. Algorithmic regulation will require federal uniformity, expert judgment, political independence and pre-market review to prevent – without stifling innovation – the introduction of unacceptably dangerous algorithms into the market.”  The White House released two reports in October 2016 detailing the advance of algorithms and artificial intelligence and plans to address issues tied to it, and it issued a December report outlining some of the potential effects of AI-driven automation on the U.S. job market and economy.  On January 17, 2017, the Future of Life Institute published a list of 23 Principles for Beneficial Artificial Intelligence, created by a gathering of concerned researchers at a conference at Asimolar, in Pacific Grove, California. The more than 1,600 signatories included Steven Hawking, Elon Musk, Ray Kurzweil and hundreds of the world's foremost AI researchers. The use of algorithms is spreading as massive amounts of data are being created, captured and analyzed by businesses and governments. Some are calling this the Age of Algorithms and predicting that the future of algorithms is tied to machine learning and deep learning that will get better and better at an ever-faster pace. While many of the 2016 U.S. presidential election post-mortems noted the revolutionary impact of web-based tools in influencing its outcome, XPrize Foundation CEO Peter Diamandis predicted that “five big tech trends will make this election look tame.” He said advances in quantum computing and the rapid evolution of AI and AI agents embedded in systems and devices in the Internet of Things will lead to hyper-stalking, influencing and shaping of voters, and hyper- personalized ads, and will create new ways to misrepresent reality and perpetuate falsehoods. www.pewresearch.org 4 PEW RESEARCH CENTER Seven major themes about the algorithm era INEVITABLE Theme 1 Algorithms will continue to spread everywhere ALGORITHMS  The benefits will be visible and invisible and can lead to greater human insight into the world  The many upsides of algorithms are accompanied by challenges Theme 2 Good things lie ahead  Data-driven approaches to problem-solving will expand  Code processes will be refined and improved; ethical issues are being worked out  “Algorithms don’t have to be perfect; they just have to be better than people”  In the future, the world may be governed by benevolent AI CONCERNS Theme 3 Humanity and human judgment are lost when data and predictive modeling become paramount  Programming primarily in pursuit of profits and efficiencies is a threat  Algorithms manipulate people and outcomes, and even “read our minds”  All of this will lead to a flawed yet inescapable logic-driven society  Some fear people could lose sophisticated decision-making capabilities and local intelligence  As code takes over complex systems, humans are left out of the loop  Solutions should include embedding respect for the individual Theme 4 Biases exist in algorithmically-organized systems  Algorithms reflect the biases of programmers and datasets  Algorithms depend upon data that is often limited, deficient or incorrect Theme 5 Algorithmic categorizations deepen divides  The disadvantaged are likely to be even more so  Algorithms create filter bubbles and silos shaped by corporate data collectors. They limit people’s exposure to a wider range of ideas and reliable information and eliminate serendipity Theme 6 Unemployment will rise  Smarter, more-efficient algorithms will displace many human work activities  Some seek a redefined global economic system to support humanity SOCIETAL Theme 7 The need grows for algorithmic literacy, transparency and oversight CHALLENGES  It starts with algorithm literacy – this goes beyond basic digital literacy  People call for accountability processes, oversight and transparency  Many are pessimistic about the prospects for policy rules and oversight PEW RESEARCH CENTER Analysts like Aneesh Aneesh of Stanford University foresee algorithms taking over public and private activities in a new era of “algocratic governance” that supplants “bureaucratic hierarchies.” Others, like Harvard’s Shoshana Zuboff, describe the emergence of “surveillance capitalism” that organizes economic behavior in an “information civilization.” To illuminate current attitudes about the potential impacts of algorithms in the next decade, Pew Research Center and Elon University’s Imagining the Internet Center conducted a large-scale canvassing of technology experts, scholars, corporate practitioners and government leaders. Some 1,302 responded to this question about what will happen in the next decade: www.pewresearch.org EMBARGOED COPY – NOT FOR PUBLICATION OR DISTRIBUTION UNTIL 00:00 a.m./p.m. EDT, MONTH DATE, YEAR 5 PEW RESEARCH CENTER Will the net overall effect of algorithms be positive for individuals and society or negative for individuals and society? The non-scientific canvassing found that 38% of these particular respondents predicted that the positive impacts of algorithms will outweigh negatives for individuals and society in general, while 37% said negatives will outweigh positives; 25% said the overall impact of algorithms will be about 50-50, positive-negative. [See “About this canvassing of experts” for further details about the limits of this sample.] Participants were asked to explain their answers, and most wrote detailed elaborations that provide insights about hopeful and concerning trends. Respondents were allowed to respond anonymously; these constitute a slight majority of the written elaborations. These findings do not represent all the points of view that are possible to a question like this, but they do reveal a wide range of valuable observations based on current trends. In the next section we offer a brief outline of seven key themes found among the written elaborations. Following that introductory section there is a much more in-depth look at respondents’ thoughts tied to each of the themes, beginning on page 30 of this report. All responses are lightly edited for style. Theme 1: Algorithms will continue to spread everywhere There is fairly uniform agreement among these respondents that algorithms are generally invisible to the public and there will be an exponential rise in their influence in the next decade. A representative statement of this view came from Barry Chudakov, founder and principal at Sertain Research and StreamFuzion Corp. He replied: “‘If every algorithm suddenly stopped working, it would be the end of the world as we know it.’ (Pedro Domingo’s The Master Algorithm). Fact: We have already turned our world over to machine learning and algorithms. The question now is, how to better understand and manage what we have done? “Algorithms are a useful artifact to begin discussing the larger issue of the effects of technology-enabled assists in our lives. Namely, how can we see them at work? Consider and assess their assumptions? And most importantly for those who don’t create algorithms for a living – how do we educate ourselves about the way they work, where they are in operation, what assumptions and biases are inherent in them, and how to keep them www.pewresearch.org 6 PEW RESEARCH CENTER transparent? Like fish in a tank, we can see them swimming around and keep an eye on them. “Algorithms are the new arbiters of human decision-making in almost any area we can imagine, from watching a movie (Affectiva emotion recognition) to buying a house (Zillow.com) to self-driving cars (Google). Deloitte Global predicted more than 80 of the world’s 100 largest enterprise software companies will have cognitive technologies – mediated by algorithms – integrated into their products by the end of 2016. As Brian Christian and Tom Griffiths write in Algorithms to Live By, algorithms provide ‘a better standard against which to compare human cognition itself.’ They are also a goad to consider that same cognition: How are we thinking and what does it mean to think through algorithms to mediate our world? “The main positive result of this is better understanding of how to make rational decisions, and in this measure a better understanding of ourselves. After all, algorithms are generated by trial and error, by testing, by observing, and coming to certain mathematical formulae regarding choices that have been made again and again – and this can be used for difficult choices and problems, especially when intuitively we cannot readily see an answer or a way to resolve the problem. The 37% Rule, optimal stopping and other algorithmic conclusions are evidence-based guides that enable us to use wisdom and mathematically verified steps to make better decisions. “The secondary positive result is connectivity. In a technological recapitulation of what spiritual teachers have been saying for centuries, our things are demonstrating that everything is – or can be – connected to everything else. Algorithms with the persistence and ubiquity of insects will automate processes that used to require human manipulation and thinking. These can now manage basic processes of monitoring, measuring, counting or even seeing. Our car can tell us to slow down. Our televisions can suggest movies to watch. A grocery can suggest a healthy combination of meats and vegetables for dinner. Siri reminds you it’s your anniversary. “The main negative changes come down to a simple but now quite difficult question: How can we see, and fully understand the implications of, the algorithms programmed into everyday actions and decisions? The rub is this: Whose intelligence is it, anyway? ... Our systems do not have, and we need to build in, what David Gelernter called ‘topsight,’ the ability to not only create technological solutions but also see and explore their consequences before we build business models, companies and markets on their strengths, and especially on their limitations.” www.pewresearch.org EMBARGOED COPY – NOT FOR PUBLICATION OR DISTRIBUTION UNTIL 00:00 a.m./p.m. EDT, MONTH DATE, YEAR 7 PEW RESEARCH CENTER Chudakov added that this is especially necessary because in the next decade and beyond, “By expanding collection and analysis of data and the resulting application of this information, a layer of intelligence or thinking manipulation is added to processes and objects that previously did not have that layer. So prediction possibilities follow us around like a pet. The result: As information tools and predictive dynamics are more widely adopted, our lives will be increasingly affected by their inherent conclusions and the narratives they spawn.” “The overall impact of ubiquitous algorithms is presently incalculable because the presence of algorithms in everyday processes and transactions is now so great, and is mostly hidden from public view. All of our extended thinking systems (algorithms fuel the software and connectivity that create extended thinking systems) demand more thinking – not less – and a more global perspective than we have previously managed. The expanding collection and analysis of data and the resulting application of this information can cure diseases, decrease poverty, bring timely solutions to people and places where need is greatest, and dispel millennia of prejudice, ill-founded conclusions, inhumane practice and ignorance of all kinds. Our algorithms are now redefining what we think, how we think and what we know. We need to ask them to think about their thinking – to look out for pitfalls and inherent biases before those are baked in and harder to remove. “To create oversight that would assess the impact of algorithms, first we need to see and understand them in the context for which they were developed. That, by itself, is a tall order that requires impartial experts backtracking through the technology development process to find the models and formulae that originated the algorithms. Then, keeping all that learning at hand, the experts need to soberly assess the benefits and deficits or risks the algorithms create. Who is prepared to do this? Who has the time, the budget and resources to investigate and recommend useful courses of action? This is a 21st-century job description – and market niche – in search of real people and companies. In order to make algorithms more transparent, products and product information circulars might include an outline of algorithmic assumptions, akin to the nutritional sidebar now found on many packaged food products, that would inform users of how algorithms drive intelligence in a given product and a reasonable outline of the implications inherent in those assumptions.” Theme 2: Good things lie ahead A number of respondents noted the many ways in which algorithms will help make sense of massive amounts of data, noting that this will spark breakthroughs in science, new conveniences and human capacities in everyday life, and an ever-better capacity to link people to the www.pewresearch.org 8 PEW RESEARCH CENTER information that will help them. They perform seemingly miraculous tasks humans cannot and they will continue to greatly augment human intelligence and assist in accomplishing great things. A representative proponent of this view is Stephen Downes, a researcher at the National Research Council of Canada, who listed the following as positive changes: “Some examples: Banks. Today banks provide loans based on very incomplete data. It is true that many people who today qualify for loans would not get them in the future. However, many people – and arguably many more people – will be able to obtain loans in the future, as banks turn away from using such factors as race, socio-economic background, postal code and the like to assess fit. Moreover, with more data (and with a more interactive relationship between bank and client) banks can reduce their risk, thus providing more loans, while at the same time providing a range of services individually directed to actually help a person’s financial state. “Health care providers. Health care is a significant and growing expense not because people are becoming less healthy (in fact, society-wide, the opposite is true) but because of the significant overhead required to support increasingly complex systems, including prescriptions, insurance, facilities and more. New technologies will enable health providers to shift a significant percentage of that load to the individual, who will (with the aid of personal support systems) manage their health better, coordinate and manage their own care, and create less of a burden on the system. As the overall cost of health care declines, it becomes increasingly feasible to provide single-payer health insurance for the entire population, which has known beneficial health outcomes and efficiencies. “Governments. A significant proportion of government is based on regulation and monitoring, which will no longer be required with the deployment of automated production and transportation systems, along with sensor networks. This includes many of the daily (and often unpleasant) interactions we have with government today, from traffic offenses, manifestation of civil discontent, unfair treatment in commercial and legal processes, and the like. A simple example: One of the most persistent political problems in the United States is the gerrymandering of political boundaries to benefit incumbents. Electoral divisions created by an algorithm to a large degree eliminate gerrymandering (and when open and debatable, can be modified to improve on that result).” A sampling of additional answers, from anonymous respondents:  “Algorithms find knowledge in an automated way much faster than traditionally feasible.” www.pewresearch.org EMBARGOED COPY – NOT FOR PUBLICATION OR DISTRIBUTION UNTIL 00:00 a.m./p.m. EDT, MONTH DATE, YEAR 9 PEW RESEARCH CENTER  “Algorithms can crunch databases quickly enough to alleviate some of the red tape and bureaucracy that currently slows progress down.”  “We will see less pollution, improved human health, less economic waste.”  “Algorithms have the potential to equalize access to information.”  “The efficiencies of algorithms will lead to more creativity and self-expression.”  “Algorithms can diminish transportation issues; they can identify congestion and alternative times and paths.”  “Self-driving cars could dramatically reduce the number of accidents we have per year, as well as improve quality of life for most people.”  “Better-targeted delivery of news, services and advertising.”  “More evidence-based social science using algorithms to collect data from social media and click trails.”  “Improved and more proactive police work, targeting areas where crime can be prevented.”  “Fewer underdeveloped areas and more international commercial exchanges.”  “Algorithms ease the friction in decision-making, purchasing, transportation and a large number of other behaviors.”  “Bots will follow orders to buy your stocks. Digital agents will find the materials you need.”  “Any errors could be corrected. This will mean the algorithms only become more efficient to humanity’s desires as time progresses.” Participants in this study were in substantial agreement that the abundant positives of accelerating code-dependency will continue to drive the spread of algorithms; however, as with all great technological revolutions, this trend has a dark side. Most respondents pointed out concerns, chief among them the final five overarching themes of this report; all have subthemes. Theme 3: Humanity and human judgment are lost when data and predictive modeling become paramount Advances in algorithms are allowing technology corporations and governments to gather, store, sort and analyze massive data sets. Experts in this canvassing noted that these algorithms are primarily written to optimize efficiency and profitability without much thought about the possible societal impacts of the data modeling and analysis. These respondents argued that humans are considered to be an “input” to the process and they are not seen as real, thinking, feeling, changing beings. They say this is creating a flawed, logic-driven society and that as the process evolves – that is, as algorithms begin to write the algorithms – humans may get left out of the loop, letting “the robots decide.” Representative of this view: www.pewresearch.org