A L L I S O N science / comPuters $29.99 us / $37.50 can “Pedro domingos demYstifies macHine learning and sHows How wondrous aaddVVaannccee PPrraaiissee ffoorr BEFORE AFTER and eXciting tHe future will Be.” —walter isaacson a BLURBS BLURBS lgorithms increasingly run our lives. They fi nd books, movies, jobs, and dates for us; manage 5/21 PE PDF KM our investments; and discover new drugs. More and more, these algorithms work by 5/21 PDF CG / “The holy grail of computer science is a machine that can teach itself, as we humans do, AUTHOR learning from the trails of data we leave in our newly MR 6/30 from experience. Machine learning could help us do everything from curing cancer to digital world. Like curious children, they observe AC building humanoid robots. Pedro Domingos demystifi es machine learning and shows how DUE EX. 7/15 us, imitate, and experiment. And in the world’s wondrous and exciting the future will be.” —walter isaacson, author of Steve Jobs CH top research labs and universities, the race is on to invent the ultimate learning algorithm: one capable NC “With terms like ‘machine learning’ and ‘big data’ regularly making headlines, there is of discovering any knowledge from data, and doing no shortage of hype-fi lled business books on the subject. There are also textbooks that KH anything we want, before we even ask. are too technical to be accessible. For those in the middle—from executives to college Machine learning is the automation of discovery— MR students—this is the ideal book, showing how and why things really work without the the scientifi c method on steroids. It enables intelligent heavy math. Unlike other books that proclaim a bright future, this one actually gives you LH robots and computers to program themselves. No what you need to understand the changes that are coming.” —Peter norVig, fi eld of science today is more important yet more TJK director of research, Google, and coauthor of Artifi cial Intelligence shrouded in mystery. Pedro Domingos, one of the Pedro domingos “Machine learning is the single most transformative technology that will shape our lives fi eld’s leading lights, lifts the veil for the fi rst time 6.25” x 9.5” is a professor of computer S: 1.13” to give us a peek inside the learning machines that over the next fi fteen years. This book is a must-read—a bold and beautifully written new B: 1.005” science at the University of Washington. He is a power Google, Amazon, and your smartphone. He framework for looking into the future.” —geoffreY moore, author of Crossing the Chasm How tHe Quest for BASIC-HC winner of the SIGKDD Innovation Award, the highest charts a course through machine learning’s fi ve major 4/C + PMS 8365C Metallic honor in data science. A fellow of the Association for “A delightful book by one of the leading experts in the fi eld. If you wonder how AI will schools of thought, showing how they turn ideas from the Advancement of Artifi cial Intelligence, he lives change your life, read this book.” —seBastian tHrun, research professor, Stanford, tHe ultimate neuroscience, evolution, psychology, physics, and FINISH: near Seattle. soft touch with interactive Google fellow, and inventor of the self-driving car statistics into algorithms ready to serve you. Step-by- illusion on the title, subtitle, learning macHine will step, he assembles a blueprint for the future universal author name on front cover “Starting with the audacious claim that all knowledge can be derived from data by a single learner—the Master Algorithm—and discusses what it only ‘master algorithm,’ Domingos takes the reader on a fast-paced journey through the brave means for you, and for the future of business, science, remake our world new world of machine learning. Writing breezily but with deep authority, Domingos is and society. the perfect tour guide from whom you will learn everything you need to know about this If data-ism is today’s rising philosophy, this book exciting fi eld, and a surprising amount about science and philosophy as well.” will be its bible. The quest for universal learning —duncan watts, principal researcher, Microsoft Research, is one of the most significant, fascinating, and and author of Six Degrees and Everything Is Obvious revolutionary intellectual developments of all time. A groundbreaking book, The Master Algorithm is the $29.99 US / $37.50 CAN ISBN 978-0-465-06570-7 essential guide for anyone and everyone wanting to 52999 understand not just how the revolution will happen, Jacket design by Jason Gabbert A Member of the Perseus Books Group but how to be at its forefront. www.basicbooks.com 9 780465 065707 More Advance Praise for The Master Algorithm “This is an incredibly important and useful book. Machine learning is already critical to your life and work, and will only become more so. Finally, Pedro Domingos has written about it in a clear and un- derstandable fashion.” —THOMAS H. DAVENPORT, distinguished professor, Babson College, and author of Competing on Analytics and Big Data at Work “Machine learning, known in commercial use as predictive analytics, is changing the world. This riveting, far-reaching, and inspiring book introduces the deep scientific concepts to even non-technical readers, and yet also satisfies experts with a fresh, profound perspective that reveals the most promising research directions. It’s a rare gem indeed.” —ERIC SIEGEL, founder, Predictive Analytics World, and author of Predictive Analytics “Machine learning is a fascinating world never before glimpsed by outsiders. Pedro Domingos initiates you to the mysterious languages spoken by its five tribes, and invites you to join in his plan to unite them, creating the most powerful technology our civilization has ever seen.” —SEBASTIAN SEUNG, professor, Princeton, and author of Connectome “[An] enthusiastic but not dumbed-down introduction to machine learning . . . lucid and consistently informative. . . . With wit, vision, and scholarship, Domingos describes how these scientists are creat- ing programs that allow a computer to teach itself. Readers . . . will discover fascinating insights.” —Kirkus Reviews 9780465065707-text.indd 1 7/16/15 12:44 PM 9780465065707-text.indd 2 7/16/15 12:44 PM The Master Algorithm How the Quest for the Ultimate Learning Machine Will Remake Our World Pedro Domingos A Member of the Perseus Books Group New York 9780465065707-text.indd 3 7/16/15 12:44 PM Copyright © 2015 by Pedro Domingos Published by Basic Books, A Member of the Perseus Books Group All rights reserved. Printed in the United States of America. No part of this book may be reproduced in any manner whatsoever without written permission except in the case of brief quotations embodied in critical articles and reviews. For information, address Basic Books, 250 West 57th Street, New York, NY 10107. Books published by Basic Books are available at special discounts for bulk purchases in the United States by corporations, institutions, and other organizations. For more information, please contact the Special Markets Department at the Perseus Books Group, 2300 Chest- nut Street, Suite 200, Philadelphia, PA 19103, or call (800) 810-4145, ext. 5000, or e-mail [email protected]. Library of Congress Cataloging-in-Publication Data Domingos, Pedro. The master algorithm : how the quest for the ultimate learning machine will remake our world / Pedro Domingos. pages cm Includes index. ISBN 978-0-465-06570-7 (hardcover)—ISBN 978-0-465-06192-1 (ebook) 1. Knowledge representation (Information theory) 2. Artificial intelligence—Social aspects. 3. Artificial intelligence—Philosophy. 4. Cognitive science—Mathematics. 5. Algorithms. I. Title. Q387.D66 2015 003'54—dc23 2015007615 10 9 8 7 6 5 4 3 2 1 9780465065707-text.indd 4 7/16/15 12:44 PM To the memory of my sister Rita, who lost her battle with cancer while I was writing this book 9780465065707-text.indd 5 7/16/15 12:44 PM 9780465065707-text.indd 6 7/16/15 12:44 PM The grand aim of science is to cover the greatest number of experimental facts by logical deduction from the smallest number of hypotheses or axioms. —Albert Einstein Civilization advances by extending the number of important operations we can perform without thinking about them. —Alfred North Whitehead 9780465065707-text.indd 7 7/16/15 12:44 PM 9780465065707-text.indd 8 7/16/15 12:44 PM Contents Prologue xi Chapter 1 The Machine-Learning Revolution 1 Chapter 2 The Master Algorithm 23 Chapter 3 Hume’s Problem of Induction 57 Chapter 4 How Does Your Brain Learn? 93 Chapter 5 Evolution: Nature’s Learning Algorithm 121 Chapter 6 In the Church of the Reverend Bayes 143 Chapter 7 You Are What You Resemble 177 Chapter 8 Learning Without a Teacher 203 Chapter 9 The Pieces of the Puzzle Fall into Place 235 Chapter 10 This Is the World on Machine Learning 263 Epilogue 291 Acknowledgments 295 Further Readings 297 Index 313 ix 9780465065707-text.indd 9 7/16/15 12:44 PM