2ND EDITION AAUUTTOOMMAATTEE LLEEAARRNN PPYYTTHHOONN.. OOOOVVVVEEEERRRR 222200000000,,,,000000000000 GGEETT SSTTUUFFFF DDOONNEE.. CCCCOOOOPPPPIIIIEEEESSSS SSSSOOOOLLLLDDDD 2ND EDITION TTHHEE BBOORRIINNGG SSTTUUFFFF AA WWIITTHH PPYYTTHHOONN SSUU TTTT If you’ve ever spent hours renaming files or updating • Search the web and download online content P R A C T I C A L P R O G R A M M I N G hundreds of spreadsheet cells, you know how tedious UUOO • Split, merge, watermark, and encrypt PDFs tasks like these can be. But what if you could have your F O R T O T A L B E G I N N E R S computer do them for you? • Send email responses and text notifications FFMM In this fully revised second edition of Automate the • Fill out online forms FF AA A L S W E I G A R T Boring Stuff with Python, you’ll learn how to use Python Step-by-step instructions walk you through each program, to write programs that do in minutes what would take WWTT and updated practice projects at the end of each chapter you hours to do by hand—no prior programming experi- challenge you to improve those programs and use your EE ence required. You’ll learn the basics of Python and newfound skills to automate similar tasks. II explore Python’s rich library of modules for performing TT specific tasks, like scraping data off websites, reading Don’t spend your time doing work a well-trained monkey TT PDF and Word documents, and automating clicking and could do. Even if you’ve never written a line of code, HH HH typing tasks. you can make your computer do the grunt work. Learn how in Automate the Boring Stuff with Python. The second edition of this international best-seller PPEE includes a brand-new chapter on input validation, ABOUT THE AUTHOR as well as tutorials on automating Gmail and Google YYBB Al Sweigart is a professional software developer who Sheets, and tips on automatically updating CSV files. You’ll learn how to create programs that effortlessly teaches programming to kids and adults. Sweigart TTOO has written several bestselling programming books perform useful feats of automation to: HH for beginners, including Invent Your Own Computer RR • Search for text in a file or across multiple files Games with Python, Cracking Codes with Python, and OO • Create, update, move, and rename files and folders Coding with Minecraft (all from No Starch Press). II NNNN GG COVERS PYTHON 3.X THE FINEST IN GEEK ENTERTAINMENT™ S W www.nostarch.com E I $39.95 ($53.95 CDN) G A R PYTHONPROGRAMMSHELVE IN: T IN G LAN G UAG ES/ AUTOMATE THE BORING STUFF WITH PYTHON A U T O M A T E T H E B O R I N G S T U F F W I T H P Y T H O N 2 N D E D I T I O N Practical Programming for Total Beginners by Al Sweigart San Francisco AUTOMATE THE BORING STUFF WITH PYTHON, 2ND EDITION. Copyright © 2020 by Al Sweigart. All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN-10: 1-59327-992-2 ISBN-13: 978-1-59327-992-9 Publisher: William Pollock Production Editor: Laurel Chun Cover Illustration: Josh Ellingson Interior Design: Octopod Studios Developmental Editors: Frances Saux and Jan Cash Technical Reviewers: Ari Lacenski and Philip James Copyeditors: Kim Wimpsett, Britt Bogan, and Paula L. Fleming Compositors: Susan Glinert Stevens and Danielle Foster Proofreaders: Lisa Devoto Farrell and Emelie Burnette Indexer: BIM Indexing and Proofreading Services For information on distribution, translations, or bulk sales, please contact No Starch Press, Inc. directly: No Starch Press, Inc. 245 8th Street, San Francisco, CA 94103 phone: 1.415.863.9900; [email protected] www.nostarch.com The Library of Congress Control Number for the first edition is: 2014953114 No Starch Press and the No Starch Press logo are registered trademarks of No Starch Press, Inc. Other product and company names mentioned herein may be the trademarks of their respective owners. Rather than use a trademark symbol with every occurrence of a trademarked name, we are using the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The information in this book is distributed on an “As Is” basis, without warranty. While every precaution has been taken in the preparation of this work, neither the author nor No Starch Press, Inc. shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in it. This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 United States License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/us/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA. For my nephew Jack About the Author Al Sweigart is a software developer and tech book author. Python is his favorite programming language, and he is the developer of several open source modules for it. His other books are freely available under a Creative Commons license on his website https://inventwithpython.com/. His cat now weighs 11 pounds. About the Tech Reviewer Philip James has been working in Python for over a decade and is a frequent speaker in the Python community. He speaks on topics ranging from Unix fundamentals to open source social networks. Philip is a Core Contributor to the BeeWare project and lives in the San Francisco Bay Area with his partner Nic and her cat River. BRIEF CONTENTS Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxv Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxvii PART I: PYTHON PROGRAMMING BASICS Chapter 1: Python Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 Chapter 2: Flow Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21 Chapter 3: Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .57 Chapter 4: Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77 Chapter 5: Dictionaries and Structuring Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .111 Chapter 6: Manipulating Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .129 PART II: AUTOMATING TASKS Chapter 7: Pattern Matching with Regular Expressions . . . . . . . . . . . . . . . . . . . . . . . . . .161 Chapter 8: Input Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .187 Chapter 9: Reading and Writing Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .201 Chapter 10: Organizing Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .231 Chapter 11: Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .249 Chapter 12: Web Scraping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .267 Chapter 13: Working with Excel Spreadsheets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .301 Chapter 14: Working with Google Sheets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .329 Chapter 15: Working with PDF and Word Documents . . . . . . . . . . . . . . . . . . . . . . . . . .347 Chapter 16: Working with CSV Files and JSON Data . . . . . . . . . . . . . . . . . . . . . . . . . .371 Chapter 17: Keeping Time, Scheduling Tasks, and Launching Programs . . . . . . . . . . . . . .389 Chapter 18: Sending Email and Text Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .415 Chapter 19: Manipulating Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .447 Chapter 20: Controlling the Keyboard and Mouse with GUI Automation . . . . . . . . . . . . . 473 Appendix A: Installing Third-Party Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .507 Appendix B: Running Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .511 Appendix C: Answers to the Practice Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .517 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 531 viii Brief Contents