Python for Finance The financial industry has adopted Python at a tremendous rate, with “ Python's readable P some of the largest investment banks and hedge funds using it to build syntax, easy integration y core trading and risk management systems. This hands-on guide helps with C/C++, and the both developers and quantitative analysts get started with Python, and wide variety of numerical t guides you through the most important aspects of using Python for quantitative finance. computing tools make h Using practical examples throughout the book, author Yves Hilpisch also it a natural choice for o shows you how to develop a full-fledged framework for Monte Carlo financial analytics. simulation-based derivatives and risk analytics, based on a large, realistic n It's rapidly becoming case study. Much of the book uses interactive IPython Notebooks, with the de-facto replacement topics that include: for a patchwork of f ■ Fundamentals: Python data structures, NumPy array handling, languages and tools o time series analysis with pandas, visualization with matplotlib, at leading financial high performance I/O operations with PyTables, date/time r information handling, and selected best practices institutions.” ■ Financial topics: Mathematical techniques with NumPy, SciPy, —Kirat Singh F Python cofounder, President and CTO and SymPy, such as regression and optimization; stochastics Washington Square Technologies for Monte Carlo simulation, Value-at-Risk, and Credit-Value-at- i Risk calculations; statistics for normality tests, mean-variance n portfolio optimization, principal component analysis (PCA), a and Bayesian regression ■ Special topics: Performance Python for financial algorithms, for Finance n such as vectorization and parallelization, integrating Python with Excel, and building financial applications based on Web c technologies e Yves Hilpisch is the founder and managing partner of The Python Quants, an analytics software provider and financial engineering group. Yves also lectures on mathematical finance and organizes meetups and conferences about Python for ANALYZE BIG FINANCIAL DATA Quant Finance in New York and London. H ilp i s c h PYTHON/FINANCE Twitter: @oreillymedia facebook.com/oreilly US $44.99 CAN $47.99 ISBN: 978-1-491-94528-5 Yves Hilpisch Python for Finance Yves Hilpisch Python for Finance by Yves Hilpisch Copyright © 2015 Yves Hilpisch. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://safaribooksonline.com). For more information, contact our corporate/ institutional sales department: 800-998-9938 or [email protected]. Editors: Brian MacDonald and Meghan Blanchette Indexer: Judith McConville Production Editor: Matthew Hacker Cover Designer: Ellie Volckhausen Copyeditor: Charles Roumeliotis Interior Designer: David Futato Proofreader: Rachel Head Illustrator: Rebecca Demarest December 2014: First Edition Revision History for the First Edition: 2014-12-09: First release See http://oreilly.com/catalog/errata.csp?isbn=9781491945285 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Python for Finance, the cover image of a Hispaniolan solenodon, and related trade dress are trademarks of O’Reilly Media, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc. was aware of a trademark claim, the designations have been printed in caps or initial caps. While the publisher and the author have used good faith efforts to ensure that the information and instruc‐ tions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intel‐ lectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. This book is not intended as financial advice. Please consult a qualified professional if you require financial advice. ISBN: 978-1-491-94528-5 [LSI] Table of Contents Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Part I. Python and Finance 1. Why Python for Finance?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 What Is Python? 3 Brief History of Python 5 The Python Ecosystem 6 Python User Spectrum 7 The Scientific Stack 8 Technology in Finance 9 Technology Spending 10 Technology as Enabler 10 Technology and Talent as Barriers to Entry 10 Ever-Increasing Speeds, Frequencies, Data Volumes 11 The Rise of Real-Time Analytics 12 Python for Finance 13 Finance and Python Syntax 14 Efficiency and Productivity Through Python 17 From Prototyping to Production 21 Conclusions 22 Further Reading 23 2. Infrastructure and Tools. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Python Deployment 26 Anaconda 26 Python Quant Platform 32 Tools 34 Python 34 iii IPython 35 Spyder 45 Conclusions 47 Further Reading 48 3. Introductory Examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Implied Volatilities 50 Monte Carlo Simulation 59 Pure Python 61 Vectorization with NumPy 63 Full Vectorization with Log Euler Scheme 65 Graphical Analysis 67 Technical Analysis 68 Conclusions 74 Further Reading 75 Part II. Financial Analytics and Development 4. Data Types and Structures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 Basic Data Types 80 Integers 80 Floats 81 Strings 84 Basic Data Structures 86 Tuples 87 Lists 88 Excursion: Control Structures 89 Excursion: Functional Programming 91 Dicts 92 Sets 94 NumPy Data Structures 95 Arrays with Python Lists 96 Regular NumPy Arrays 97 Structured Arrays 101 Vectorization of Code 102 Basic Vectorization 102 Memory Layout 105 Conclusions 106 Further Reading 107 iv | Table of Contents 5. Data Visualization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Two-Dimensional Plotting 109 One-Dimensional Data Set 110 Two-Dimensional Data Set 115 Other Plot Styles 121 Financial Plots 128 3D Plotting 132 Conclusions 135 Further Reading 135 6. Financial Time Series. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 pandas Basics 138 First Steps with DataFrame Class 138 Second Steps with DataFrame Class 142 Basic Analytics 146 Series Class 149 GroupBy Operations 150 Financial Data 151 Regression Analysis 157 High-Frequency Data 166 Conclusions 170 Further Reading 171 7. Input/Output Operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 Basic I/O with Python 174 Writing Objects to Disk 174 Reading and Writing Text Files 177 SQL Databases 179 Writing and Reading NumPy Arrays 181 I/O with pandas 183 SQL Database 184 From SQL to pandas 185 Data as CSV File 188 Data as Excel File 189 Fast I/O with PyTables 190 Working with Tables 190 Working with Compressed Tables 196 Working with Arrays 197 Out-of-Memory Computations 198 Conclusions 200 Further Reading 201 Table of Contents | v 8. Performance Python. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 Python Paradigms and Performance 204 Memory Layout and Performance 207 Parallel Computing 209 The Monte Carlo Algorithm 209 The Sequential Calculation 210 The Parallel Calculation 211 Performance Comparison 214 multiprocessing 215 Dynamic Compiling 217 Introductory Example 217 Binomial Option Pricing 218 Static Compiling with Cython 223 Generation of Random Numbers on GPUs 226 Conclusions 230 Further Reading 231 9. Mathematical Tools. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 Approximation 234 Regression 234 Interpolation 245 Convex Optimization 249 Global Optimization 250 Local Optimization 251 Constrained Optimization 253 Integration 255 Numerical Integration 256 Integration by Simulation 257 Symbolic Computation 257 Basics 258 Equations 259 Integration 260 Differentiation 261 Conclusions 262 Further Reading 263 10. Stochastics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 Random Numbers 266 Simulation 271 Random Variables 271 Stochastic Processes 274 Variance Reduction 287 vi | Table of Contents Valuation 290 European Options 291 American Options 295 Risk Measures 298 Value-at-Risk 298 Credit Value Adjustments 302 Conclusions 305 Further Reading 305 11. Statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 Normality Tests 308 Benchmark Case 309 Real-World Data 317 Portfolio Optimization 322 The Data 323 The Basic Theory 324 Portfolio Optimizations 328 Efficient Frontier 330 Capital Market Line 332 Principal Component Analysis 335 The DAX Index and Its 30 Stocks 336 Applying PCA 337 Constructing a PCA Index 338 Bayesian Regression 341 Bayes’s Formula 341 PyMC3 342 Introductory Example 343 Real Data 347 Conclusions 355 Further Reading 355 12. Excel Integration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357 Basic Spreadsheet Interaction 358 Generating Workbooks (.xls) 359 Generating Workbooks (.xslx) 360 Reading from Workbooks 362 Using OpenPyxl 364 Using pandas for Reading and Writing 366 Scripting Excel with Python 369 Installing DataNitro 369 Working with DataNitro 370 xlwings 379 Table of Contents | vii Conclusions 379 Further Reading 380 13. Object Orientation and Graphical User Interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 Object Orientation 381 Basics of Python Classes 382 Simple Short Rate Class 387 Cash Flow Series Class 391 Graphical User Interfaces 393 Short Rate Class with GUI 394 Updating of Values 396 Cash Flow Series Class with GUI 398 Conclusions 401 Further Reading 401 14. Web Integration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 Web Basics 404 ftplib 405 httplib 407 urllib 408 Web Plotting 411 Static Plots 411 Interactive Plots 414 Real-Time Plots 417 Rapid Web Applications 424 Traders’ Chat Room 426 Data Modeling 426 The Python Code 427 Templating 434 Styling 440 Web Services 442 The Financial Model 443 The Implementation 445 Conclusions 451 Further Reading 452 Part III. Derivatives Analytics Library 15. Valuation Framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455 Fundamental Theorem of Asset Pricing 455 A Simple Example 456 viii | Table of Contents The General Results 457 Risk-Neutral Discounting 458 Modeling and Handling Dates 458 Constant Short Rate 460 Market Environments 462 Conclusions 465 Further Reading 466 16. Simulation of Financial Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467 Random Number Generation 468 Generic Simulation Class 470 Geometric Brownian Motion 473 The Simulation Class 474 A Use Case 476 Jump Diffusion 478 The Simulation Class 478 A Use Case 481 Square-Root Diffusion 482 The Simulation Class 483 A Use Case 485 Conclusions 486 Further Reading 487 17. Derivatives Valuation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489 Generic Valuation Class 489 European Exercise 493 The Valuation Class 494 A Use Case 496 American Exercise 500 Least-Squares Monte Carlo 501 The Valuation Class 502 A Use Case 504 Conclusions 507 Further Reading 509 18. Portfolio Valuation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 511 Derivatives Positions 512 The Class 512 A Use Case 514 Derivatives Portfolios 515 The Class 516 A Use Case 520 Table of Contents | ix