Broadcasting

Pruthvish Rajput, Venus Patel

February 23, 2023

1 Computation on Arrays: Broadcasting

1.1 Introducing Broadcasting
Recall that for arrays of the same size, binary operations are performed on an element-by-element
basis:
[1]: import numpy as np

[2]: a = np.array([0, 1, 2])

b = np.array([5, 5, 5])
a

[2]: array([0, 1, 2])

[3]: a + 5

[3]: array([5, 6, 7])

[4]: M = np.ones((3, 3))

M

[4]: array([[1., 1., 1.],

[1., 1., 1.],
[1., 1., 1.]])

[5]: M + a

[5]: array([[1., 2., 3.],

[1., 2., 3.],
[1., 2., 3.]])

Here the one-dimensional array a is stretched, or broadcast across the second dimension in order
to match the shape of M.
[6]: a = np.arange(3)
b = np.arange(3)[:, np.newaxis]

print(a)

1
 print(b)

[0 1 2]
[[0]
[1]
[2]]

[7]: a + b

[7]: array([[0, 1, 2],

[1, 2, 3],
[2, 3, 4]])

The light boxes represent the broadcasted values: again, this extra memory is not actually
allocated in the course of the operation, but it can be useful conceptually to imagine that it
is.

1.2 Rules of Broadcasting

Broadcasting in NumPy follows a strict set of rules to determine the interaction between the two
arrays:
• Rule 1: If the two arrays differ in their number of dimensions, the shape of the one with fewer
dimensions is padded with ones on its leading (left) side.

2
 • Rule 2: If the shape of the two arrays does not match in any dimension, the array with shape
equal to 1 in that dimension is stretched to match the other shape.
• Rule 3: If in any dimension the sizes disagree and neither is equal to 1, an error is raised.
To make these rules clear, let’s consider a few examples in detail.

1.2.1 Broadcasting example 1

Let’s look at adding a two-dimensional array to a one-dimensional array:
[8]: M = np.ones((2, 3))
a = np.arange(3)
M,a

[8]: (array([[1., 1., 1.],

[1., 1., 1.]]),
array([0, 1, 2]))

Let’s consider an operation on these two arrays. The shape of the arrays are
• M.shape = (2, 3)
• a.shape = (3,)
We see by rule 1 that the array a has fewer dimensions, so we pad it on the left with ones:
• M.shape -> (2, 3)
• a.shape -> (1, 3)
By rule 2, we now see that the first dimension disagrees, so we stretch this dimension to match:
• M.shape -> (2, 3)
• a.shape -> (2, 3)
The shapes match, and we see that the final shape will be (2, 3):
[9]: M + a

[9]: array([[1., 2., 3.],

[1., 2., 3.]])

1.2.2 Broadcasting example 2

Let’s take a look at an example where both arrays need to be broadcast:
[10]: a = np.arange(3).reshape((3, 1))
b = np.arange(3)

Again, we’ll start by writing out the shape of the arrays:

• a.shape = (3, 1)
• b.shape = (3,)
Rule 1 says we must pad the shape of b with ones:

3
 • a.shape -> (3, 1)
• b.shape -> (1, 3)
And rule 2 tells us that we upgrade each of these ones to match the corresponding size of the other
array:
• a.shape -> (3, 3)
• b.shape -> (3, 3)
Because the result matches, these shapes are compatible. We can see this here:
[11]: a + b

[11]: array([[0, 1, 2],

[1, 2, 3],
[2, 3, 4]])

1.2.3 Broadcasting example 3

Now let’s take a look at an example in which the two arrays are not compatible:
[12]: M = np.ones((3, 2))
a = np.arange(3)
M,a

[12]: (array([[1., 1.],

[1., 1.],
[1., 1.]]),
array([0, 1, 2]))

This is just a slightly different situation than in the first example: the matrix M is transposed. How
does this affect the calculation? The shape of the arrays are
• M.shape = (3, 2)
• a.shape = (3,)
Again, rule 1 tells us that we must pad the shape of a with ones:
• M.shape -> (3, 2)
• a.shape -> (1, 3)
By rule 2, the first dimension of a is stretched to match that of M:
• M.shape -> (3, 2)
• a.shape -> (3, 3)
Now we hit rule 3–the final shapes do not match, so these two arrays are incompatible, as we can
observe by attempting this operation:
[13]: M + a

---------------------------------------------------------------------------
ValueError Traceback (most recent call last)

4
 /tmp/ipykernel_23560/3374645918.py in <module>
----> 1 M + a

ValueError: operands could not be broadcast together with shapes (3,2) (3,)

[14]: a[:, np.newaxis].shape

[14]: (3, 1)

[15]: M + a[:, np.newaxis]

[15]: array([[1., 1.],

[2., 2.],
[3., 3.]])

1.3 Broadcasting in Practice: Centering an array

• Imagine you have an array of 10 observations, each of which consists of 3 values. Using the
standard convention, we’ll store this in a 10 × 3 array:
[16]: X = np.random.random((10, 3))

We can compute the mean of each feature using the mean aggregate across the first dimension:
[17]: Xmean = X.mean(0)
Xmean

[17]: array([0.45634282, 0.45935642, 0.63206117])

And now we can center the X array by subtracting the mean (this is a broadcasting operation):

[18]: X_centered = X - Xmean

To double-check that we’ve done this correctly, we can check that the centered array has near zero
mean:
[19]: X_centered.mean(0)

[19]: array([-6.66133815e-17, 0.00000000e+00, -1.11022302e-16])

To within machine precision, the mean is now zero.

1.3.1 Plotting a two-dimensional function

One place that broadcasting is very useful is in displaying images based on two-dimensional func-
tions. If we want to define a function z = f (x, y), broadcasting can be used to compute the function
across the grid:
z = sin(x)10 + cos(10 + y ∗ x) ∗ cos(x)

5
[20]: # x and y have 50 steps from 0 to 5
x = np.linspace(0, 5, 50)
y = np.linspace(0, 5, 50)[:, np.newaxis]

z = np.sin(x) ** 10 + np.cos(10 + y * x) * np.cos(x)

[21]: %matplotlib inline

import matplotlib.pyplot as plt

[22]: plt.imshow(z, origin='lower', extent=[0, 5, 0, 5],

cmap='viridis')
plt.colorbar();

The result is a compelling visualization of the two-dimensional function.