Python Packages Chapman & Hall/CRC The Python Series About the Series Python has been ranked as the most popular programming language, and it is widely used in education and industry. This book series will offer a wide range of books on Python for students and professionals. Titles in the series will help users learn the language at an introductory and advanced level, and explore its many applications in data science, AI, and machine learning. Series titles can also be supplemented with Jupyter notebooks. Image Processing and Acquisition using Python, Second Edition Ravishankar Chityala, Sridevi Pudipeddi Python Packages Tomas Beuzen and Tiffany Timbers For more information about this series please visit: https://www.crcpress.com/Chapman- -HallCRC/book-series/PYTH Python Packages Tomas Beuzen Tiffany Timbers First edition published 2022 by CRC Press 6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742 and by CRC Press 4 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN CRC Press is an imprint of Taylor & Francis Group, LLC © 2022 Tomas Beuzen and Tiffany Timbers Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the conse- quences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and re- cording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, access www. copyright.com or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. For works that are not available on CCC please contact [email protected] Trademark notice: Product or corporate names may be trademarks or registered trade- marks and are used only for identification and explanation without intent to infringe. ISBN: 978-1-032-03825-4 (hbk) ISBN: 978-1-032-02944-3 (pbk) ISBN: 978-1-003-18925-1 (ebk) DOI: 10.1201/9781003189251 Publisher’s note: This book has been prepared from camera-ready copy provided by the authors. To you, the Reader. Never stop learning. You are capable of anything. Contents List of Figures xi List of Tables xiii Preface xv About the authors xix 1 Introduction 1 1.1 Why you should create packages . . . . . . . . . . . . . . . . 2 2 System setup 3 2.1 The command-line interface . . . . . . . . . . . . . . . . . . 3 2.2 Installing software . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2.1 Installing Python . . . . . . . . . . . . . . . . . . . . . 3 2.2.2 Install packaging software . . . . . . . . . . . . . . . . 4 2.3 Register for a PyPI account . . . . . . . . . . . . . . . . . . 5 2.4 Set up Git and GitHub . . . . . . . . . . . . . . . . . . . . . 5 2.5 Python integrated development environments . . . . . . . . . 6 2.5.1 Visual Studio Code . . . . . . . . . . . . . . . . . . . . 7 2.5.2 JupyterLab . . . . . . . . . . . . . . . . . . . . . . . . 8 2.5.3 RStudio . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.6 Developing with Docker . . . . . . . . . . . . . . . . . . . . . 11 2.6.1 Docker with Visual Studio Code . . . . . . . . . . . . 12 2.6.2 Docker with JupyterLab . . . . . . . . . . . . . . . . . 14 3 How to package a Python 17 3.1 Counting words in a text file . . . . . . . . . . . . . . . . . . 17 3.1.1 Developing our code . . . . . . . . . . . . . . . . . . . 17 3.1.2 Turning our code into functions . . . . . . . . . . . . . 20 3.2 Package structure . . . . . . . . . . . . . . . . . . . . . . . . 22 3.2.1 A brief introduction . . . . . . . . . . . . . . . . . . . 22 3.2.2 Creating a package structure . . . . . . . . . . . . . . 23 3.3 Put your package under version control . . . . . . . . . . . . 26 3.3.1 Set up local version control . . . . . . . . . . . . . . . 26 3.3.2 Set up remote version control . . . . . . . . . . . . . . 27 3.4 Packaging your code . . . . . . . . . . . . . . . . . . . . . . . 30 vii viii Contents 3.5 Test drive your package code . . . . . . . . . . . . . . . . . . 32 3.5.1 Create a virtual environment . . . . . . . . . . . . . . 32 3.5.2 Installing your package . . . . . . . . . . . . . . . . . 33 3.6 Adding dependencies to your package . . . . . . . . . . . . . 37 3.6.1 Dependency version constraints . . . . . . . . . . . . . 40 3.7 Testing your package . . . . . . . . . . . . . . . . . . . . . . 43 3.7.1 Writing tests . . . . . . . . . . . . . . . . . . . . . . . 43 3.7.2 Running tests . . . . . . . . . . . . . . . . . . . . . . . 45 3.7.3 Code coverage . . . . . . . . . . . . . . . . . . . . . . 48 3.8 Package documentation . . . . . . . . . . . . . . . . . . . . . 49 3.8.1 Writing documentation . . . . . . . . . . . . . . . . . 52 3.8.2 Writing docstrings . . . . . . . . . . . . . . . . . . . . 54 3.8.3 Creating usage examples . . . . . . . . . . . . . . . . . 59 3.8.4 Building documentation . . . . . . . . . . . . . . . . . 60 3.8.5 Hosting documentation online . . . . . . . . . . . . . . 69 3.9 Tagging a package release with version control . . . . . . . . 72 3.10 Building and distributing your package . . . . . . . . . . . . 74 3.10.1 Building your package . . . . . . . . . . . . . . . . . . 74 3.10.2 Publishing to TestPyPI . . . . . . . . . . . . . . . . . 78 3.10.3 Publishing to PyPI . . . . . . . . . . . . . . . . . . . . 79 3.11 Summary and next steps . . . . . . . . . . . . . . . . . . . . 79 4 Package structure and distribution 83 4.1 Packaging fundamentals . . . . . . . . . . . . . . . . . . . . . 83 4.2 Package structure . . . . . . . . . . . . . . . . . . . . . . . . 89 4.2.1 Package contents . . . . . . . . . . . . . . . . . . . . . 89 4.2.2 Package and module names . . . . . . . . . . . . . . . 92 4.2.3 Intra-package references . . . . . . . . . . . . . . . . . 93 4.2.4 The init file . . . . . . . . . . . . . . . . . . . . . . . . 94 4.2.5 Including non-code files in a package . . . . . . . . . . 97 4.2.6 Including data in a package . . . . . . . . . . . . . . . 98 4.2.7 The source layout . . . . . . . . . . . . . . . . . . . . 101 4.3 Package distribution and installation . . . . . . . . . . . . . 103 4.3.1 Package installation . . . . . . . . . . . . . . . . . . . 104 4.3.2 Building sdists and wheels . . . . . . . . . . . . . . . . 107 4.3.3 Packaging tools . . . . . . . . . . . . . . . . . . . . . . 108 4.3.4 Package repositories . . . . . . . . . . . . . . . . . . . 109 4.4 Version control . . . . . . . . . . . . . . . . . . . . . . . . . . 110 5 Testing 111 5.1 Testing workflow . . . . . . . . . . . . . . . . . . . . . . . . . 111 5.2 Test structure . . . . . . . . . . . . . . . . . . . . . . . . . . 114 5.3 Writing tests . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 5.3.1 Unit tests . . . . . . . . . . . . . . . . . . . . . . . . . 116 5.3.2 Test that a specific error is raised . . . . . . . . . . . . 120 Contents ix 5.3.3 Integration tests . . . . . . . . . . . . . . . . . . . . . 123 5.3.4 Regression tests. . . . . . . . . . . . . . . . . . . . . . 124 5.3.5 How many tests should you write . . . . . . . . . . . . 126 5.4 Advanced testing methods . . . . . . . . . . . . . . . . . . . 126 5.4.1 Fixtures . . . . . . . . . . . . . . . . . . . . . . . . . . 127 5.4.2 Parameterizations . . . . . . . . . . . . . . . . . . . . 129 5.5 Code coverage . . . . . . . . . . . . . . . . . . . . . . . . . . 132 5.5.1 Line coverage . . . . . . . . . . . . . . . . . . . . . . . 133 5.5.2 Branch coverage . . . . . . . . . . . . . . . . . . . . . 134 5.5.3 Calculating coverage . . . . . . . . . . . . . . . . . . . 134 5.5.4 Coverage reports . . . . . . . . . . . . . . . . . . . . . 136 5.6 Version control . . . . . . . . . . . . . . . . . . . . . . . . . . 138 6 Documentation 139 6.1 Documentation content and workflow . . . . . . . . . . . . . 139 6.2 Writing documentation . . . . . . . . . . . . . . . . . . . . . 143 6.2.1 README . . . . . . . . . . . . . . . . . . . . . . . . . 144 6.2.2 License . . . . . . . . . . . . . . . . . . . . . . . . . . 145 6.2.3 Contributing guidelines . . . . . . . . . . . . . . . . . 146 6.2.4 Code of conduct . . . . . . . . . . . . . . . . . . . . . 146 6.2.5 Changelog . . . . . . . . . . . . . . . . . . . . . . . . . 147 6.2.6 Examples . . . . . . . . . . . . . . . . . . . . . . . . . 148 6.2.7 Docstrings. . . . . . . . . . . . . . . . . . . . . . . . . 150 6.2.8 Application programming interface (API) reference . . 153 6.2.9 Other package documentation . . . . . . . . . . . . . . 153 6.3 Building documentation . . . . . . . . . . . . . . . . . . . . . 154 6.4 Hosting documentation online . . . . . . . . . . . . . . . . . 162 7 Releasing and versioning 165 7.1 Version numbering . . . . . . . . . . . . . . . . . . . . . . . . 165 7.2 Version bumping . . . . . . . . . . . . . . . . . . . . . . . . . 168 7.2.1 Manual version bumping. . . . . . . . . . . . . . . . . 168 7.2.2 Automatic version bumping . . . . . . . . . . . . . . . 169 7.3 Checklist for releasing a new package version . . . . . . . . . 172 7.3.1 Step 1: make changes to package source files. . . . . . 173 7.3.2 Step 2: document your changes . . . . . . . . . . . . . 173 7.3.3 Step 3: bump version number . . . . . . . . . . . . . . 174 7.3.4 Step 4: run tests and build documentation . . . . . . . 176 7.3.5 Step 5: tag a release with version control. . . . . . . . 177 7.3.6 Step 6: build and release package to PyPI . . . . . . . 178 7.4 Automating releases . . . . . . . . . . . . . . . . . . . . . . . 181 7.5 Breaking changes and deprecating package functionality . . . 181 7.6 Updating dependency versions . . . . . . . . . . . . . . . . . 183