SciPy Reference Guide Release 0.15.1 Written by the SciPy community January 18, 2015 CONTENTS 1 SciPy Tutorial 3 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Basic functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3 Special functions (scipy.special) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.4 Integration (scipy.integrate) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.5 Optimization (scipy.optimize) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.6 Interpolation (scipy.interpolate) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 1.7 Fourier Transforms (scipy.fftpack) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 1.8 Signal Processing (scipy.signal) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 1.9 Linear Algebra (scipy.linalg) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 1.10 Sparse Eigenvalue Problems with ARPACK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 1.11 Compressed Sparse Graph Routines (scipy.sparse.csgraph) . . . . . . . . . . . . . . . . . . 86 1.12 Spatial data structures and algorithms (scipy.spatial) . . . . . . . . . . . . . . . . . . . . . . 89 1.13 Statistics (scipy.stats) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 1.14 Multidimensional image processing (scipy.ndimage) . . . . . . . . . . . . . . . . . . . . . . . 114 1.15 File IO (scipy.io) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 1.16 Weave (scipy.weave) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 2 Contributing to SciPy 177 2.1 Contributing new code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 2.2 Contributing by helping maintain existing code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 2.3 Other ways to contribute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 2.4 Recommended development setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 2.5 SciPy structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 2.6 Useful links, FAQ, checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 3 API - importing from Scipy 185 3.1 Guidelines for importing functions from Scipy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 3.2 API definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 4 Release Notes 189 4.1 SciPy 0.15.0 Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 4.2 SciPy 0.14.0 Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 4.3 SciPy 0.13.2 Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 4.4 SciPy 0.13.1 Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 4.5 SciPy 0.13.0 Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 4.6 SciPy 0.12.1 Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 4.7 SciPy 0.12.0 Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 4.8 SciPy 0.11.0 Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 4.9 SciPy 0.10.1 Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 4.10 SciPy 0.10.0 Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 i 4.11 SciPy 0.9.0 Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 4.12 SciPy 0.8.0 Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 4.13 SciPy 0.7.2 Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 4.14 SciPy 0.7.1 Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 4.15 SciPy 0.7.0 Release Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248 5 Reference 255 5.1 Clustering package (scipy.cluster) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 5.2 K-means clustering and vector quantization (scipy.cluster.vq) . . . . . . . . . . . . . . . . . 255 5.3 Hierarchical clustering (scipy.cluster.hierarchy) . . . . . . . . . . . . . . . . . . . . . . 259 5.4 Constants (scipy.constants) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274 5.5 Discrete Fourier transforms (scipy.fftpack) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 5.6 Integration and ODEs (scipy.integrate) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 5.7 Interpolation (scipy.interpolate) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 5.8 Input and output (scipy.io) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386 5.9 Linear algebra (scipy.linalg) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398 5.10 Low-level BLAS functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446 5.11 Finding functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446 5.12 BLAS Level 1 functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447 5.13 BLAS Level 2 functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462 5.14 BLAS Level 3 functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474 5.15 Low-level LAPACK functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481 5.16 Finding functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481 5.17 All functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481 5.18 Interpolative matrix decomposition (scipy.linalg.interpolative) . . . . . . . . . . . . . 544 5.19 Miscellaneous routines (scipy.misc) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553 5.20 Multi-dimensional image processing (scipy.ndimage) . . . . . . . . . . . . . . . . . . . . . . . 563 5.21 Orthogonal distance regression (scipy.odr) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 617 5.22 Optimization and root finding (scipy.optimize) . . . . . . . . . . . . . . . . . . . . . . . . . . 626 5.23 Nonlinear solvers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 697 5.24 Signal processing (scipy.signal) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 699 5.25 Sparse matrices (scipy.sparse) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 815 5.26 Sparse linear algebra (scipy.sparse.linalg) . . . . . . . . . . . . . . . . . . . . . . . . . . 913 5.27 Compressed Sparse Graph Routines (scipy.sparse.csgraph) . . . . . . . . . . . . . . . . . . 942 5.28 Spatial algorithms and data structures (scipy.spatial) . . . . . . . . . . . . . . . . . . . . . . 953 5.29 Distance computations (scipy.spatial.distance) . . . . . . . . . . . . . . . . . . . . . . . 986 5.30 Special functions (scipy.special) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1001 5.31 Statistical functions (scipy.stats) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1055 5.32 Statistical functions for masked arrays (scipy.stats.mstats) . . . . . . . . . . . . . . . . . . 1393 5.33 C/C++ integration (scipy.weave) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1420 Bibliography 1425 Index 1439 ii SciPy Reference Guide, Release 0.15.1 Release 0.15.1 Date January 18, 2015 SciPy (pronounced “Sigh Pie”) is open-source software for mathematics, science, and engineering. CONTENTS 1 SciPy Reference Guide, Release 0.15.1 2 CONTENTS CHAPTER ONE SCIPY TUTORIAL 1.1 Introduction Contents • Introduction – SciPy Organization – Finding Documentation SciPy is a collection of mathematical algorithms and convenience functions built on the Numpy extension of Python. It adds significant power to the interactive Python session by providing the user with high-level commands and classes for manipulating and visualizing data. With SciPy an interactive Python session becomes a data-processing and system- prototyping environment rivaling sytems such as MATLAB, IDL, Octave, R-Lab, and SciLab. The additional benefit of basing SciPy on Python is that this also makes a powerful programming language available for use in developing sophisticated programs and specialized applications. Scientific applications using SciPy benefit from the development of additional modules in numerous niche’s of the software landscape by developers across the world. Everything from parallel programming to web and data-base subroutines and classes have been made available to the Python programmer. All of this power is available in addition to the mathematical libraries in SciPy. This tutorial will acquaint the first-time user of SciPy with some of its most important features. It assumes that the user has already installed the SciPy package. Some general Python facility is also assumed, such as could be acquired by working through the Python distribution’s Tutorial. For further introductory help the user is directed to the Numpy documentation. For brevity and convenience, we will often assume that the main packages (numpy, scipy, and matplotlib) have been imported as: >>> import numpy as np >>> import matplotlib as mpl >>> import matplotlib.pyplot as plt These are the import conventions that our community has adopted after discussion on public mailing lists. You will see these conventions used throughout NumPy and SciPy source code and documentation. While we obviously don’t require you to follow these conventions in your own code, it is highly recommended. 1.1.1 SciPy Organization SciPy is organized into subpackages covering different scientific computing domains. These are summarized in the following table: 3 SciPy Reference Guide, Release 0.15.1 Subpackage Description cluster Clustering algorithms constants Physical and mathematical constants fftpack Fast Fourier Transform routines integrate Integration and ordinary differential equation solvers interpolate Interpolation and smoothing splines io Input and Output linalg Linear algebra ndimage N-dimensional image processing odr Orthogonal distance regression optimize Optimization and root-finding routines signal Signal processing sparse Sparse matrices and associated routines spatial Spatial data structures and algorithms special Special functions stats Statistical distributions and functions weave C/C++ integration Scipy sub-packages need to be imported separately, for example: >>> from scipy import linalg, optimize Because of their ubiquitousness, some of the functions in these subpackages are also made available in the scipy namespace to ease their use in interactive sessions and programs. In addition, many basic array functions from numpy are also available at the top-level of the scipy package. Before looking at the sub-packages individually, we will first look at some of these common functions. 1.1.2 Finding Documentation SciPy and NumPy have documentation versions in both HTML and PDF format available at http://docs.scipy.org/, that cover nearly all available functionality. However, this documentation is still work-in-progress and some parts may be incomplete or sparse. As we are a volunteer organization and depend on the community for growth, your participation - everything from providing feedback to improving the documentation and code - is welcome and actively encouraged. Python’s documentation strings are used in SciPy for on-line documentation. There are two methods for reading them and getting help. One is Python’s command help in the pydoc module. Entering this command with no arguments (i.e. >>> help ) launches an interactive help session that allows searching through the keywords and modules available to all of Python. Secondly, running the command help(obj) with an object as the argument displays that object’s calling signature, and documentation string. The pydoc method of help is sophisticated but uses a pager to display the text. Sometimes this can interfere with the terminal you are running the interactive session within. A numpy/scipy-specific help system is also available under the command numpy.info. The signature and documentation string for the object passed to the help command are printed to standard output (or to a writeable object passed as the third argument). The second keyword argument of numpy.info defines the maximum width of the line for printing. If a module is passed as the argument to help than a list of the functions and classes defined in that module is printed. For example: >>> np.info(optimize.fmin) fmin(func, x0, args=(), xtol=0.0001, ftol=0.0001, maxiter=None, maxfun=None, full_output=0, disp=1, retall=0, callback=None) Minimize a function using the downhill simplex algorithm. Parameters ---------- func : callable func(x,*args) 4 Chapter 1. SciPy Tutorial SciPy Reference Guide, Release 0.15.1 The objective function to be minimized. x0 : ndarray Initial guess. args : tuple Extra arguments passed to func, i.e. ‘‘f(x,*args)‘‘. callback : callable Called after each iteration, as callback(xk), where xk is the current parameter vector. Returns ------- xopt : ndarray Parameter that minimizes function. fopt : float Value of function at minimum: ‘‘fopt = func(xopt)‘‘. iter : int Number of iterations performed. funcalls : int Number of function calls made. warnflag : int 1 : Maximum number of function evaluations made. 2 : Maximum number of iterations reached. allvecs : list Solution at each iteration. Other parameters ---------------- xtol : float Relative error in xopt acceptable for convergence. ftol : number Relative error in func(xopt) acceptable for convergence. maxiter : int Maximum number of iterations to perform. maxfun : number Maximum number of function evaluations to make. full_output : bool Set to True if fopt and warnflag outputs are desired. disp : bool Set to True to print convergence messages. retall : bool Set to True to return list of solutions at each iteration. Notes ----- Uses a Nelder-Mead simplex algorithm to find the minimum of function of one or more variables. Another useful command is source. When given a function written in Python as an argument, it prints out a listing of the source code for that function. This can be helpful in learning about an algorithm or understanding exactly what a function is doing with its arguments. Also don’t forget about the Python command dir which can be used to look at the namespace of a module or package. 1.2 Basic functions 1.2. Basic functions 5 SciPy Reference Guide, Release 0.15.1 Contents • Basic functions – Interaction with Numpy * Index Tricks * Shape manipulation * Polynomials * Vectorizing functions (vectorize) * Type handling * Other useful functions 1.2.1 Interaction with Numpy Scipy builds on Numpy, and for all basic array handling needs you can use Numpy functions: >>> import numpy as np >>> np.some_function() Rather than giving a detailed description of each of these functions (which is available in the Numpy Reference Guide or by using the help, info and source commands), this tutorial will discuss some of the more useful commands which require a little introduction to use to their full potential. To use functions from some of the Scipy modules, you can do: >>> from scipy import some_module >>> some_module.some_function() The top level of scipy also contains functions from numpy and numpy.lib.scimath. However, it is better to use them directly from the numpy module instead. Index Tricks There are some class instances that make special use of the slicing functionality to provide efficient means for array construction. This part will discuss the operation of np.mgrid , np.ogrid , np.r_ , and np.c_ for quickly constructing arrays. For example, rather than writing something like the following >>> concatenate(([3],[0]*5,arange(-1,1.002,2/9.0))) with the r_ command one can enter this as >>> r_[3,[0]*5,-1:1:10j] which can ease typing and make for more readable code. Notice how objects are concatenated, and the slicing syntax is (ab)used to construct ranges. The other term that deserves a little explanation is the use of the complex number 10j as the step size in the slicing syntax. This non-standard use allows the number to be interpreted as the number of points to produce in the range rather than as a step size (note we would have used the long integer notation, 10L, but this notation may go away in Python as the integers become unified). This non-standard usage may be unsightly to some, but it gives the user the ability to quickly construct complicated vectors in a very readable fashion. When the number of points is specified in this way, the end- point is inclusive. The “r” stands for row concatenation because if the objects between commas are 2 dimensional arrays, they are stacked by rows (and thus must have commensurate columns). There is an equivalent command c_ that stacks 2d arrays by columns but works identically to r_ for 1d arrays. 6 Chapter 1. SciPy Tutorial