Review

Array programming with NumPy

https://doi.org/10.1038/s41586-020-2649-2 Charles R. Harris1, K. Jarrod Millman2,3,4 ✉, Stéfan J. van der Walt2,4,5 ✉, Ralf Gommers6 ✉,
Pauli Virtanen7,8, David Cournapeau9, Eric Wieser10, Julian Taylor11, Sebastian Berg4,
Received: 21 February 2020
Nathaniel J. Smith12, Robert Kern13, Matti Picus4, Stephan Hoyer14, Marten H. van Kerkwijk15,
Accepted: 17 June 2020 Matthew Brett2,16, Allan Haldane17, Jaime Fernández del Río18, Mark Wiebe19,20,
Pearu Peterson6,21,22, Pierre Gérard-Marchant23,24, Kevin Sheppard25, Tyler Reddy26,
Published online: 16 September 2020
Warren Weckesser4, Hameer Abbasi6, Christoph Gohlke27 & Travis E. Oliphant6
Open access

Check for updates

Array programming provides a powerful, compact and expressive syntax for
accessing, manipulating and operating on data in vectors, matrices and
higher-dimensional arrays. NumPy is the primary array programming library for the
Python language. It has an essential role in research analysis pipelines in fields as
diverse as physics, chemistry, astronomy, geoscience, biology, psychology, materials
science, engineering, finance and economics. For example, in astronomy, NumPy was
an important part of the software stack used in the discovery of gravitational waves1
and in the first imaging of a black hole2. Here we review how a few fundamental array
concepts lead to a simple and powerful programming paradigm for organizing,
exploring and analysing scientific data. NumPy is the foundation upon which the
scientific Python ecosystem is constructed. It is so pervasive that several projects,
targeting audiences with specialized needs, have developed their own NumPy-like
interfaces and array objects. Owing to its central position in the ecosystem, NumPy
increasingly acts as an interoperability layer between such array computation
libraries and, together with its application programming interface (API), provides a
flexible framework to support the next decade of scientific and industrial analysis.

Two Python array packages existed before NumPy. The Numeric pack- that operate on it. Because of its inherent simplicity, the NumPy array
age was developed in the mid-1990s and provided array objects and is the de facto exchange format for array data in Python.
array-aware functions in Python. It was written in C and linked to stand- NumPy operates on in-memory arrays using the central processing
ard fast implementations of linear algebra3,4. One of its earliest uses was unit (CPU). To utilize modern, specialized storage and hardware, there
to steer C++ applications for inertial confinement fusion research at has been a recent proliferation of Python array packages. Unlike with
Lawrence Livermore National Laboratory5. To handle large astronomi- the Numarray–Numeric divide, it is now much harder for these new
cal images coming from the Hubble Space Telescope, a reimplementa- libraries to fracture the user community—given how much work is
tion of Numeric, called Numarray, added support for structured arrays, already built on top of NumPy. However, to provide the community with
flexible indexing, memory mapping, byte-order variants, more efficient access to new and exploratory technologies, NumPy is transitioning
memory use, flexible IEEE 754-standard error-handling capabilities, and into a central coordinating mechanism that specifies a well defined
better type-casting rules6. Although Numarray was highly compatible array programming API and dispatches it, as appropriate, to special-
with Numeric, the two packages had enough differences that it divided ized array implementations.
the community; however, in 2005 NumPy emerged as a ‘best of both
worlds’ unification7—combining the features of Numarray with the
small-array performance of Numeric and its rich C API. NumPy arrays
Now, 15 years later, NumPy underpins almost every Python library The NumPy array is a data structure that efficiently stores and accesses
that does scientific or numerical computation8–11, including SciPy12, multidimensional arrays17 (also known as tensors), and enables a wide
Matplotlib13, pandas14, scikit-learn15 and scikit-image16. NumPy is a variety of scientific computation. It consists of a pointer to memory,
community-developed, open-source library, which provides a mul- along with metadata used to interpret the data stored there, notably
tidimensional Python array object along with array-aware functions ‘data type’, ‘shape’ and ‘strides’ (Fig. 1a).
Independent researcher, Logan, UT, USA. 2Brain Imaging Center, University of California, Berkeley, Berkeley, CA, USA. 3Division of Biostatistics, University of California, Berkeley, Berkeley, CA,
1

USA. 4Berkeley Institute for Data Science, University of California, Berkeley, Berkeley, CA, USA. 5Applied Mathematics, Stellenbosch University, Stellenbosch, South Africa. 6Quansight, Austin,
TX, USA. 7Department of Physics, University of Jyväskylä, Jyväskylä, Finland. 8Nanoscience Center, University of Jyväskylä, Jyväskylä, Finland. 9Mercari JP, Tokyo, Japan. 10Department of
Engineering, University of Cambridge, Cambridge, UK. 11Independent researcher, Karlsruhe, Germany. 12Independent researcher, Berkeley, CA, USA. 13Enthought, Austin, TX, USA. 14Google
Research, Mountain View, CA, USA. 15Department of Astronomy and Astrophysics, University of Toronto, Toronto, Ontario, Canada. 16School of Psychology, University of Birmingham,
Edgbaston, Birmingham, UK. 17Department of Physics, Temple University, Philadelphia, PA, USA. 18Google, Zurich, Switzerland. 19Department of Physics and Astronomy, The University of
British Columbia, Vancouver, British Columbia, Canada. 20Amazon, Seattle, WA, USA. 21Independent researcher, Saue, Estonia. 22Department of Mechanics and Applied Mathematics, Institute
of Cybernetics at Tallinn Technical University, Tallinn, Estonia. 23Department of Biological and Agricultural Engineering, University of Georgia, Athens, GA, USA. 24France-IX Services, Paris,
France. 25Department of Economics, University of Oxford, Oxford, UK. 26CCS-7, Los Alamos National Laboratory, Los Alamos, NM, USA. 27Laboratory for Fluorescence Dynamics, Biomedical
Engineering Department, University of California, Irvine, Irvine, CA, USA. ✉e-mail: [email protected]; [email protected]; [email protected]

Nature | Vol 585 | 17 September 2020 | 357

Review
a Data structure d Vectorization g Example

0 1 2 data 0 1 2 3 4 5 6 7 8 9 10 11 0 1 1 1 1 2 In [1]: import numpy as np

3 4 5 data type 8-byte integer 3 4 1 1 4 5
x =
(4, 3)
8 bytes 3 × 8 = 24 bytes + → In [2]: x = np.arange(12)
6 7 8 shape per element to jump one 6 7 1 1 7 8
9 10 11 strides (24, 8) row down 9 10 1 1 10 11
In [3]: x = x.reshape(4, 3)

In [4]: x
b Indexing (view) e Broadcasting
Out[4]:
array([[ 0, 1, 2],
0 1 2 0 1 2 0 1 2 0 0 [ 3, 4, 5],
x[:,1:] → 3 4 5 with slices x[:,::2] → 3 4 5 with slices 3 3 6 [ 6, 7, 8],
6 7 8 6 7 8 with steps × →
6 6 12 [ 9, 10, 11]])
9 10 11 9 10 11
Slices are start:end:step, 9 9 18
any of which can be left blank
In [5]: np.mean(x, axis=0)
Out[5]: array([4.5, 5.5, 6.5])
c Indexing (copy) f Reduction
In [6]: x = x - np.mean(x, axis=0)
x[1,2] → 5 with scalars x[x > 9] → 10 11 with masks 0 1 2 3
sum
3 4 5 axis 1
12 In [7]: x
x 0 1
, 1 2 → x[0,1],x[1,2] → 1 5 with arrays 6 7 8 21 Out[7]:
9 10 11 30 array([[-4.5, -4.5, -4.5],
sum [-1.5, -1.5, -1.5],
1 1 0 1 1 1 0 4 3 with arrays
x → x → with broadcasting
axis 0 sum [ 1.5, 1.5, 1.5],
2 2 2 1 0 7 6 axis (0,1)
, , 18 22 26 66 [ 4.5, 4.5, 4.5]])

Fig. 1 | The NumPy array incorporates several fundamental array concepts. before performing the lookup. d, Vectorization efficiently applies operations
a, The NumPy array data structure and its associated metadata fields. to groups of elements. e, Broadcasting in the multiplication of two-dimensional
b, Indexing an array with slices and steps. These operations return a ‘view’ of arrays. f, Reduction operations act along one or more axes. In this example,
the original data. c, Indexing an array with masks, scalar coordinates or other an array is summed along select axes to produce a vector, or along two axes
arrays, so that it returns a ‘copy’ of the original data. In the bottom example, an consecutively to produce a scalar. g, Example NumPy code, illustrating some of
array is indexed with other arrays; this broadcasts the indexing arguments these concepts.

The data type describes the nature of elements stored in an array. statistics and trigonometry (Fig. 1d). Vectorization—operating on
An array has a single data type, and each element of an array occupies entire arrays rather than their individual elements—is essential to array
the same number of bytes in memory. Examples of data types include programming. This means that operations that would take many tens
real and complex numbers (of lower and higher precision), strings, of lines to express in languages such as C can often be implemented as
timestamps and pointers to Python objects. a single, clear Python expression. This results in concise code and frees
The shape of an array determines the number of elements along users to focus on the details of their analysis, while NumPy handles
each axis, and the number of axes is the dimensionality of the array. looping over array elements near-optimally—for example, taking
For example, a vector of numbers can be stored as a one-dimensional strides into consideration to best utilize the computer’s fast cache
array of shape N, whereas colour videos are four-dimensional arrays memory.
of shape (T, M, N, 3). When performing a vectorized operation (such as addition) on two
Strides are necessary to interpret computer memory, which stores arrays with the same shape, it is clear what should happen. Through
elements linearly, as multidimensional arrays. They describe the num- ‘broadcasting’ NumPy allows the dimensions to differ, and produces
ber of bytes to move forward in memory to jump from row to row, col- results that appeal to intuition. A trivial example is the addition of a
umn to column, and so forth. Consider, for example, a two-dimensional scalar value to an array, but broadcasting also generalizes to more com-
array of floating-point numbers with shape (4, 3), where each element plex examples such as scaling each column of an array or generating
occupies 8 bytes in memory. To move between consecutive columns, a grid of coordinates. In broadcasting, one or both arrays are virtually
we need to jump forward 8 bytes in memory, and to access the next row, duplicated (that is, without copying any data in memory), so that the
3 × 8 = 24 bytes. The strides of that array are therefore (24, 8). NumPy shapes of the operands match (Fig. 1d). Broadcasting is also applied
can store arrays in either C or Fortran memory order, iterating first over when an array is indexed using arrays of indices (Fig. 1c).
either rows or columns. This allows external libraries written in those Other array-aware functions, such as sum, mean and maximum,
languages to access NumPy array data in memory directly. perform element-by-element ‘reductions’, aggregating results across
Users interact with NumPy arrays using ‘indexing’ (to access sub- one, multiple or all axes of a single array. For example, summing an
arrays or individual elements), ‘operators’ (for example, +, − and × n-dimensional array over d axes results in an array of dimension n − d
for vectorized operations and @ for matrix multiplication), as well (Fig. 1f).
as ‘array-aware functions’; together, these provide an easily readable, NumPy also includes array-aware functions for creating, reshaping,
expressive, high-level API for array programming while NumPy deals concatenating and padding arrays; searching, sorting and counting
with the underlying mechanics of making operations fast. data; and reading and writing files. It provides extensive support for
Indexing an array returns single elements, subarrays or elements generating pseudorandom numbers, includes an assortment of prob-
that satisfy a specific condition (Fig. 1b). Arrays can even be indexed ability distributions, and performs accelerated linear algebra, using
using other arrays (Fig. 1c). Wherever possible, indexing that retrieves a one of several backends such as OpenBLAS18,19 or Intel MKL optimized
subarray returns a ‘view’ on the original array such that data are shared for the CPUs at hand (see Supplementary Methods for more details).
between the two arrays. This provides a powerful way to operate on Altogether, the combination of a simple in-memory array repre-
subsets of array data while limiting memory usage. sentation, a syntax that closely mimics mathematics, and a variety
To complement the array syntax, NumPy includes functions that of array-aware utility functions forms a productive and powerfully
perform vectorized calculations on arrays, including arithmetic, expressive array programming language.

358 | Nature | Vol 585 | 17 September 2020

 cesium PyChrono MDAnalysis eht-imaging iris
khmer PsychoPy Qiime2 FiPy deepchem
Application-specific
nibabel mne-python yellowbrick scikit-HEP
PyWavelets librosa SunPy QuTiP yt

Astropy Biopython NLTK

Astronomy Biology Linguistics
Domain-specific
QuantEcon cantera simpeg
Economics Chemistry Geophysics

scikit-learn scikit-image
Machine learning Image processing
Technique-specific
pandas, statsmodels NetworkX
Statistics Network analysis

SciPy Matplotlib
Foundation
Algorithms Plots

Python NumPy IPython / Jupyter

Language Arrays Interactive environments

New array implementations

NumPy API Array Protocols

Fig. 2 | NumPy is the base of the scientific Python ecosystem. Essential libraries and projects that depend on NumPy’s API gain access to new array
implementations that support NumPy’s array protocols (Fig. 3).

Telescope collaboration for radio interferometry imaging, analysis

Scientific Python ecosystem and simulation, relies on many lower-level components of the scientific
Python is an open-source, general-purpose interpreted programming Python ecosystem. In particular, the EHT collaboration used this library
language well suited to standard programming tasks such as cleaning for the first imaging of a black hole. Within eht-imaging, NumPy arrays
data, interacting with web resources and parsing text. Adding fast array are used to store and manipulate numerical data at every step in the
operations and linear algebra enables scientists to do all their work processing chain: from raw data through calibration and image recon-
within a single programming language—one that has the advantage of struction. SciPy supplies tools for general image-processing tasks such
being famously easy to learn and teach, as witnessed by its adoption as filtering and image alignment, and scikit-image, an image-processing
as a primary learning language in many universities. library that extends SciPy, provides higher-level functionality such
Even though NumPy is not part of Python’s standard library, it ben- as edge filters and Hough transforms. The ‘scipy.optimize’ module
efits from a good relationship with the Python developers. Over the performs mathematical optimization. NetworkX22, a package for com-
years, the Python language has added new features and special syntax plex network analysis, is used to verify image comparison consistency.
so that NumPy would have a more succinct and easier-to-read array Astropy23,24 handles standard astronomical file formats and computes
notation. However, because it is not part of the standard library, NumPy time–coordinate transformations. Matplotlib is used to visualize data
is able to dictate its own release policies and development patterns. and to generate the final image of the black hole.
SciPy and Matplotlib are tightly coupled with NumPy in terms of his- The interactive environment created by the array programming foun-
tory, development and use. SciPy provides fundamental algorithms for dation and the surrounding ecosystem of tools—inside of IPython or
scientific computing, including mathematical, scientific and engineer- Jupyter—is ideally suited to exploratory data analysis. Users can fluidly
ing routines. Matplotlib generates publication-ready figures and visu- inspect, manipulate and visualize their data, and rapidly iterate to refine
alizations. The combination of NumPy, SciPy and Matplotlib, together programming statements. These statements are then stitched together
with an advanced interactive environment such as IPython20 or Jupy- into imperative or functional programs, or notebooks containing both
ter21, provides a solid foundation for array programming in Python. The computation and narrative. Scientific computing beyond exploratory
scientific Python ecosystem (Fig. 2) builds on top of this foundation to work is often done in a text editor or an integrated development envi-
provide several, widely used technique-specific libraries15,16,22, that in ronment (IDE) such as Spyder. This rich and productive environment
turn underlie numerous domain-specific projects23–28. NumPy, at the has made Python popular for scientific research.
base of the ecosystem of array-aware libraries, sets documentation To complement this facility for exploratory work and rapid proto-
standards, provides array testing infrastructure and adds build sup- typing, NumPy has developed a culture of using time-tested software
port for Fortran and other compilers. engineering practices to improve collaboration and reduce error30. This
Many research groups have designed large, complex scientific librar- culture is not only adopted by leaders in the project but also enthusi-
ies that add application-specific functionality to the ecosystem. For astically taught to newcomers. The NumPy team was early to adopt
example, the eht-imaging library29, developed by the Event Horizon distributed revision control and code review to improve collaboration

Nature | Vol 585 | 17 September 2020 | 359

Review
Input NumPy Array Output
arrays API implementation arrays In [1]: import numpy as np
np.stack In [2]: import dask.array as da
np.reshape
NumPy
NumPy np.transpose NumPy In [3]: x = da.arange(12)
np.argmin
np.mean Dask In [4]: x = np.reshape(x, (4, 3))
np.std
Dask np.max Dask In [5]: x
np.cos CuPy Out[5]: dask.array<..., shape=(4, 3), ...>
np.arctan
np.log In [6]: np.mean(x, axis=0)
CuPy
PyData/ CuPy Out[6]: dask.array<..., shape=(3,), ...>
np.cumsum Sparse
np.diff
In [7]: x = x - np.mean(x, axis=0)
...

...
PyData PyData In [8]: x
Sparse Sparse Out[8]: dask.array<..., shape=(4, 3), ...>
...

...
NumPy array protocols

Fig. 3 | NumPy’s API and array protocols expose new arrays to the this case, Dask) and results in a new Dask array. Compare this code to the
ecosystem. In this example, NumPy’s ‘mean’ function is called on a Dask array. example code in Fig. 1g.
The call succeeds by dispatching to the appropriate library implementation (in

on code, and continuous testing that runs an extensive battery of auto- The community’s efforts to fill this gap led to a proliferation of new
mated tests for every proposed change to NumPy. The project also array implementations. For example, each deep-learning framework
has comprehensive, high-quality documentation, integrated with the created its own arrays; the PyTorch38, Tensorflow39, Apache MXNet40
source code31–33. and JAX arrays all have the capability to run on CPUs and GPUs in a
This culture of using best practices for producing reliable scientific distributed fashion, using lazy evaluation to allow for additional per-
software has been adopted by the ecosystem of libraries that build on formance optimizations. SciPy and PyData/Sparse both provide sparse
NumPy. For example, in a recent award given by the Royal Astronomi- arrays, which typically contain few non-zero values and store only those
cal Society to Astropy, they state: “The Astropy Project has provided in memory for efficiency. In addition, there are projects that build on
hundreds of junior scientists with experience in professional-standard NumPy arrays as data containers, and extend its capabilities. Distrib-
software development practices including use of version control, unit uted arrays are made possible that way by Dask, and labelled arrays—
testing, code review and issue tracking procedures. This is a vital skill referring to dimensions of an array by name rather than by index for
set for modern researchers that is often missing from formal university clarity, compare x[:, 1] versus x.loc[:, 'time']—by xarray41.
education in physics or astronomy”34. Community members explicitly Such libraries often mimic the NumPy API, because this lowers the
work to address this lack of formal education through courses and barrier to entry for newcomers and provides the wider community with
workshops35–37. a stable array programming interface. This, in turn, prevents disruptive
The recent rapid growth of data science, machine learning and arti- schisms such as the divergence between Numeric and Numarray. But
ficial intelligence has further and dramatically boosted the scientific exploring new ways of working with arrays is experimental by nature
use of Python. Examples of its important applications, such as the and, in fact, several promising libraries (such as Theano and Caffe) have
eht-imaging library, now exist in almost every discipline in the natu- already ceased development. And each time that a user decides to try a
ral and social sciences. These tools have become the primary software new technology, they must change import statements and ensure that the
environment in many fields. NumPy and its ecosystem are commonly new library implements all the parts of the NumPy API they currently use.
taught in university courses, boot camps and summer schools, and Ideally, operating on specialized arrays using NumPy functions or
are the focus of community conferences and workshops worldwide. semantics would simply work, so that users could write code once,
NumPy and its API have become truly ubiquitous. and would then benefit from switching between NumPy arrays, GPU
arrays, distributed arrays and so forth as appropriate. To support array
operations between external array objects, NumPy therefore added
Array proliferation and interoperability the capability to act as a central coordination mechanism with a well
NumPy provides in-memory, multidimensional, homogeneously typed specified API (Fig. 2).
(that is, single-pointer and strided) arrays on CPUs. It runs on machines To facilitate this interoperability, NumPy provides ‘protocols’ (or
ranging from embedded devices to the world’s largest supercomputers, contracts of operation), that allow for specialized arrays to be passed to
with performance approaching that of compiled languages. For most NumPy functions (Fig. 3). NumPy, in turn, dispatches operations to the
its existence, NumPy addressed the vast majority of array computa- originating library, as required. Over four hundred of the most popular
tion use cases. NumPy functions are supported. The protocols are implemented by
However, scientific datasets now routinely exceed the memory capac- widely used libraries such as Dask, CuPy, xarray and PyData/Sparse.
ity of a single machine and may be stored on multiple machines or in Thanks to these developments, users can now, for example, scale their
the cloud. In addition, the recent need to accelerate deep-learning and computation from a single machine to distributed systems using Dask.
artificial intelligence applications has led to the emergence of special- The protocols also compose well, allowing users to redeploy NumPy
ized accelerator hardware, including graphics processing units (GPUs), code at scale on distributed, multi-GPU systems via, for instance, CuPy
tensor processing units (TPUs) and field-programmable gate arrays arrays embedded in Dask arrays. Using NumPy’s high-level API, users
(FPGAs). Owing to its in-memory data model, NumPy is currently unable can leverage highly parallel code execution on multiple systems with
to directly utilize such storage and specialized hardware. However, millions of cores, all with minimal code changes42.
both distributed data and also the parallel execution of GPUs, TPUs These array protocols are now a key feature of NumPy, and are
and FPGAs map well to the paradigm of array programming: therefore expected to only increase in importance. The NumPy developers—
leading to a gap between available modern hardware architectures and many of whom are authors of this Review—iteratively refine and add
the tools necessary to leverage their computational power. protocol designs to improve utility and simplify adoption.

360 | Nature | Vol 585 | 17 September 2020

 leading part in interactive scientific computation, although to do so
Discussion will require sustained funding from government, academia and indus-
NumPy combines the expressive power of array programming, the try. But, importantly, for NumPy to meet the needs of the next decade
performance of C, and the readability, usability and versatility of Python of data science, it will also need a new generation of graduate students
in a mature, well tested, well documented and community-developed and community contributors to drive it forward.
library. Libraries in the scientific Python ecosystem provide fast imple-
mentations of most important algorithms. Where extreme optimiza- 1. Abbott, B. P. et al. Observation of gravitational waves from a binary black hole merger.
Phys. Rev. Lett. 116, 061102 (2016).
tion is warranted, compiled languages can be used, such as Cython43, 2. Chael, A. et al. High-resolution linear polarimetric imaging for the Event Horizon
Numba44 and Pythran45; these languages extend Python and trans- Telescope. Astrophys. J. 286, 11 (2016).
parently accelerate bottlenecks. Owing to NumPy’s simple memory 3. Dubois, P. F., Hinsen, K. & Hugunin, J. Numerical Python. Comput. Phys. 10, 262–267 (1996).
4. Ascher, D., Dubois, P. F., Hinsen, K., Hugunin, J. & Oliphant, T. E. An Open Source Project:
model, it is easy to write low-level, hand-optimized code, usually in C Numerical Python (Lawrence Livermore National Laboratory, 2001).
or Fortran, to manipulate NumPy arrays and pass them back to Python. 5. Yang, T.-Y., Furnish, G. & Dubois, P. F. Steering object-oriented scientific computations. In
Furthermore, using array protocols, it is possible to utilize the full Proc. TOOLS USA 97. Intl Conf. Technology of Object Oriented Systems and Languages
(eds Ege, R., Singh, M. & Meyer, B.) 112–119 (IEEE, 1997).
spectrum of specialized hardware acceleration with minimal changes 6. Greenfield, P., Miller, J. T., Hsu, J. & White, R. L. numarray: a new scientific array package
to existing code. for Python. In PyCon DC 2003 http://citeseerx.ist.psu.edu/viewdoc/download?d
NumPy was initially developed by students, faculty and researchers oi=10.1.1.112.9899 (2003).
7. Oliphant, T. E. Guide to NumPy 1st edn (Trelgol Publishing, 2006).
to provide an advanced, open-source array programming library for 8. Dubois, P. F. Python: batteries included. Comput. Sci. Eng. 9, 7–9 (2007).
Python, which was free to use and unencumbered by license servers and 9. Oliphant, T. E. Python for scientific computing. Comput. Sci. Eng. 9, 10–20 (2007).
software protection dongles. There was a sense of building something 10. Millman, K. J. & Aivazis, M. Python for scientists and engineers. Comput. Sci. Eng. 13, 9–12
(2011).
consequential together for the benefit of many others. Participating 11. Pérez, F., Granger, B. E. & Hunter, J. D. Python: an ecosystem for scientific computing.
in such an endeavour, within a welcoming community of like-minded Comput. Sci. Eng. 13, 13–21 (2011).
individuals, held a powerful attraction for many early contributors. Explains why the scientific Python ecosystem is a highly productive environment for
research.
These user–developers frequently had to write code from scratch 12. Virtanen, P. et al. SciPy 1.0—fundamental algorithms for scientific computing in Python.
to solve their own or their colleagues’ problems—often in low-level Nat. Methods 17, 261–272 (2020); correction 17, 352 (2020).
Introduces the SciPy library and includes a more detailed history of NumPy and SciPy.
languages that preceded Python, such as Fortran46 and C. To them,
13. Hunter, J. D. Matplotlib: a 2D graphics environment. Comput. Sci. Eng. 9, 90–95 (2007).
the advantages of an interactive, high-level array library were evident. 14. McKinney, W. Data structures for statistical computing in Python. In Proc. 9th Python in
The design of this new tool was informed by other powerful interactive Science Conf. (eds van der Walt, S. & Millman, K. J.) 56–61 (2010).
15. Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12,
programming languages for scientific computing such as Basis47–50,
2825–2830 (2011).
Yorick51, R52 and APL53, as well as commercial languages and environ- 16. van der Walt, S. et al. scikit-image: image processing in Python. PeerJ 2, e453 (2014).
ments such as IDL (Interactive Data Language) and MATLAB. 17. van der Walt, S., Colbert, S. C. & Varoquaux, G. The NumPy array: a structure for efficient
numerical computation. Comput. Sci. Eng. 13, 22–30 (2011).
What began as an attempt to add an array object to Python became
Discusses the NumPy array data structure with a focus on how it enables efficient
the foundation of a vibrant ecosystem of tools. Now, a large amount of computation.
scientific work depends on NumPy being correct, fast and stable. It is 18. Wang, Q., Zhang, X., Zhang, Y. & Yi, Q. AUGEM: automatically generate high performance
dense linear algebra kernels on x86 CPUs. In SC’13: Proc. Intl Conf. High Performance
no longer a small community project, but core scientific infrastructure.
Computing, Networking, Storage and Analysis 25 (IEEE, 2013).
The developer culture has matured: although initial development was 19. Xianyi, Z., Qian, W. & Yunquan, Z. Model-driven level 3 BLAS performance optimization
highly informal, NumPy now has a roadmap and a process for propos- on Loongson 3A processor. In 2012 IEEE 18th Intl Conf. Parallel and Distributed Systems
684–691 (IEEE, 2012).
ing and discussing large changes. The project has formal governance
20. Pérez, F. & Granger, B. E. IPython: a system for interactive scientific computing. Comput.
structures and is fiscally sponsored by NumFOCUS, a nonprofit that Sci. Eng. 9, 21–29 (2007).
promotes open practices in research, data and scientific computing. 21. Kluyver, T. et al. Jupyter Notebooks—a publishing format for reproducible computational
workflows. In Positioning and Power in Academic Publishing: Players, Agents and Agendas
Over the past few years, the project attracted its first funded develop- (eds Loizides, F. & Schmidt, B.) 87–90 (IOS Press, 2016).
ment, sponsored by the Moore and Sloan Foundations, and received 22. Hagberg, A. A., Schult, D. A. & Swart, P. J. Exploring network structure, dynamics, and
an award as part of the Chan Zuckerberg Initiative’s Essentials of Open function using NetworkX. In Proc. 7th Python in Science Conf. (eds Varoquaux, G.,
Vaught, T. & Millman, K. J.) 11–15 (2008).
Source Software programme. With this funding, the project was (and 23. Astropy Collaboration et al. Astropy: a community Python package for astronomy. Astron.
is) able to have sustained focus over multiple months to implement Astrophys. 558, A33 (2013).
substantial new features and improvements. That said, the develop- 24. Price-Whelan, A. M. et al. The Astropy Project: building an open-science project and
status of the v2.0 core package. Astron. J. 156, 123 (2018).
ment of NumPy still depends heavily on contributions made by gradu- 25. Cock, P. J. et al. Biopython: freely available Python tools for computational molecular
ate students and researchers in their free time (see Supplementary biology and bioinformatics. Bioinformatics 25, 1422–1423 (2009).
Methods for more details). 26. Millman, K. J. & Brett, M. Analysis of functional magnetic resonance imaging in Python.
Comput. Sci. Eng. 9, 52–55 (2007).
NumPy is no longer merely the foundational array library underlying 27. The SunPy Community et al. SunPy—Python for solar physics. Comput. Sci. Discov. 8,
the scientific Python ecosystem, but it has become the standard API for 014009 (2015).
tensor computation and a central coordinating mechanism between 28. Hamman, J., Rocklin, M. & Abernathy, R. Pangeo: a big-data ecosystem for scalable Earth
system science. In EGU General Assembly Conf. Abstracts 12146 (2018).
array types and technologies in Python. Work continues to expand on 29. Chael, A. A. et al. ehtim: imaging, analysis, and simulation software for radio
and improve these interoperability features. interferometry. Astrophysics Source Code Library https://ascl.net/1904.004 (2019).
Over the next decade, NumPy developers will face several challenges. 30. Millman, K. J. & Pérez, F. Developing open source scientific practice. In Implementing
Reproducible Research (eds Stodden, V., Leisch, F. & Peng, R. D.) 149–183 (CRC Press, 2014).
New devices will be developed, and existing specialized hardware will Describes the software engineering practices embraced by the NumPy and SciPy
evolve to meet diminishing returns on Moore’s law. There will be more, communities with a focus on how these practices improve research.
and a wider variety of, data science practitioners, a large proportion of 31. van der Walt, S. The SciPy Documentation Project (technical overview). In Proc. 7th Python
in Science Conf. (SciPy 2008) (eds Varoquaux, G., Vaught, T. & Millman, K. J.) 27–28 (2008).
whom will use NumPy. The scale of scientific data gathering will con- 32. Harrington, J. The SciPy Documentation Project. In Proc. 7th Python in Science
tinue to increase, with the adoption of devices and instruments such Conference (SciPy 2008) (eds Varoquaux, G., Vaught, T. & Millman, K. J.) 33–35 (2008).
as light-sheet microscopes and the Large Synoptic Survey Telescope 33. Harrington, J. & Goldsmith, D. Progress report: NumPy and SciPy documentation in 2009.
In Proc. 8th Python in Science Conf. (SciPy 2009) (eds Varoquaux, G., van der Walt, S. &
(LSST)54. New generation languages, interpreters and compilers, such as Millman, K. J.) 84–87 (2009).
Rust55, Julia56 and LLVM57, will create new concepts and data structures, 34. Royal Astronomical Society Report of the RAS ‘A’ Awards Committee 2020: Astropy
and determine their viability. Project: 2020 Group Achievement Award (A) https://ras.ac.uk/sites/default/files/2020-01/
Group%20Award%20-%20Astropy.pdf (2020).
Through the mechanisms described in this Review, NumPy is poised 35. Wilson, G. Software carpentry: getting scientists to write better code by making them
to embrace such a changing landscape, and to continue playing a more productive. Comput. Sci. Eng. 8, 66–69 (2006).

Nature | Vol 585 | 17 September 2020 | 361

Review
36. Hannay, J. E. et al. How do scientists develop and use scientific software? In Proc. 2009 56. Bezanson, J., Edelman, A., Karpinski, S. & Shah, V. B. Julia: a fresh approach to numerical
ICSE Workshop on Software Engineering for Computational Science and Engineering 1–8 computing. SIAM Rev. 59, 65–98 (2017).
(IEEE, 2009). 57. Lattner, C. & Adve, V. LLVM: a compilation framework for lifelong program analysis and
37. Millman, K. J., Brett, M., Barnowski, R. & Poline, J.-B. Teaching computational transformation. In Proc. 2004 Intl Symp. Code Generation and Optimization (CGO’04)
reproducibility for neuroimaging. Front. Neurosci. 12, 727 (2018). 75–88 (IEEE, 2004).
38. Paszke, A. et al. Pytorch: an imperative style, high-performance deep learning library. In
Advances in Neural Information Processing Systems 32 (eds Wallach, H. et al.) 8024–8035
(Neural Information Processing Systems, 2019). Acknowledgements We thank R. Barnowski, P. Dubois, M. Eickenberg, and P. Greenfield, who
39. Abadi, M. et al. TensorFlow: a system for large-scale machine learning. In OSDI’16: Proc. suggested text and provided helpful feedback on the manuscript. K.J.M. and S.J.v.d.W. were
12th USENIX Conf. Operating Systems Design and Implementation (chairs Keeton, K. & funded in part by the Gordon and Betty Moore Foundation through grant GBMF3834 and by
Roscoe, T.) 265–283 (USENIX Association, 2016). the Alfred P. Sloan Foundation through grant 2013-10-27 to the University of California,
40. Chen, T. et al. MXNet: a flexible and efficient machine learning library for heterogeneous Berkeley. S.J.v.d.W., S.B., M.P. and W.W. were funded in part by the Gordon and Betty Moore
distributed systems. Preprint at http://www.arxiv.org/abs/1512.01274 (2015). Foundation through grant GBMF5447 and by the Alfred P. Sloan Foundation through grant
41. Hoyer, S. & Hamman, J. xarray: N–D labeled arrays and datasets in Python. J. Open Res. G-2017-9960 to the University of California, Berkeley.
Softw. 5, 10 (2017).
42. Entschev, P. Distributed multi-GPU computing with Dask, CuPy and RAPIDS. In EuroPython Author contributions K.J.M. and S.J.v.d.W. composed the manuscript with input from
2019 https://ep2019.europython.eu/media/conference/slides/ others. S.B., R.G., K.S., W.W., M.B. and T.R. contributed text. All authors contributed
fX8dJsD-distributed-multi-gpu-computing-with-dask-cupy-and-rapids.pdf (2019). substantial code, documentation and/or expertise to the NumPy project. All authors
43. Behnel, S. et al. Cython: the best of both worlds. Comput. Sci. Eng. 13, 31–39 (2011). reviewed the manuscript.
44. Lam, S. K., Pitrou, A. & Seibert, S. Numba: a LLVM-based Python JIT compiler. In Proc.
Second Workshop on the LLVM Compiler Infrastructure in HPC, LLVM ’15 7:1–7:6 (ACM, 2015). Competing interests The authors declare no competing interests.
45. Guelton, S. et al. Pythran: enabling static optimization of scientific Python programs.
Comput. Sci. Discov. 8, 014001 (2015). Additional information
46. Dongarra, J., Golub, G. H., Grosse, E., Moler, C. & Moore, K. Netlib and NA-Net: building a Supplementary information is available for this paper at https://doi.org/10.1038/s41586-020-
scientific computing community. IEEE Ann. Hist. Comput. 30, 30–41 (2008). 2649-2.
47. Barrett, K. A., Chiu, Y. H., Painter, J. F., Motteler, Z. C. & Dubois, P. F. Basis System, Part I: Correspondence and requests for materials should be addressed to K.J.M., S.J.v.W. or R.G.
Running a Basis Program—A Tutorial for Beginners UCRL-MA-118543, Vol. 1 (Lawrence Peer review information Nature thanks Edouard Duchesnay, Alan Edelman and the other,
Livermore National Laboratory 1995). anonymous, reviewer(s) for their contribution to the peer review of this work.
48. Dubois, P. F. & Motteler, Z. Basis System, Part II: Basis Language Reference Manual Reprints and permissions information is available at http://www.nature.com/reprints.
UCRL-MA-118543, Vol. 2 (Lawrence Livermore National Laboratory, 1995). Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in
49. Chiu, Y. H. & Dubois, P. F. Basis System, Part III: EZN User Manual UCRL-MA-118543, Vol. 3 published maps and institutional affiliations.
(Lawrence Livermore National Laboratory, 1995).
50. Chiu, Y. H. & Dubois, P. F. Basis System, Part IV: EZD User Manual UCRL-MA-118543, Vol. 4 Open Access This article is licensed under a Creative Commons Attribution
(Lawrence Livermore National Laboratory, 1995). 4.0 International License, which permits use, sharing, adaptation, distribution
51. Munro, D. H. & Dubois, P. F. Using the Yorick interpreted language. Comput. Phys. 9, and reproduction in any medium or format, as long as you give appropriate
609–615 (1995). credit to the original author(s) and the source, provide a link to the Creative Commons license,
52. Ihaka, R. & Gentleman, R. R: a language for data analysis and graphics. J. Comput. Graph. and indicate if changes were made. The images or other third party material in this article are
Stat. 5, 299–314 (1996). included in the article’s Creative Commons license, unless indicated otherwise in a credit line
53. Iverson, K. E. A programming language. In Proc. 1962 Spring Joint Computer Conf. to the material. If material is not included in the article’s Creative Commons license and your
345–351 (1962). intended use is not permitted by statutory regulation or exceeds the permitted use, you will
54. Jenness, T. et al. LSST data management software development practices and tools. In need to obtain permission directly from the copyright holder. To view a copy of this license,
Proc. SPIE 10707, Software and Cyberinfrastructure for Astronomy V 1070709 (SPIE and visit http://creativecommons.org/licenses/by/4.0/.
International Society for Optics and Photonics, 2018).
55. Matsakis, N. D. & Klock, F. S. The Rust language. Ada Letters 34, 103–104 (2014). © The Author(s) 2020

362 | Nature | Vol 585 | 17 September 2020