Numpy
Numpy
It also has functions for working in domain of linear algebra, fourier transform, and matrices.
NumPy was created in 2005 by Travis Oliphant. It is an open source project and you can use it freely.
In Python we have lists that serve the purpose of arrays, but they are slow to process.
NumPy aims to provide an array object that is up to 50x faster than traditional Python lists.
Why is NumPy Faster Than Lists? NumPy arrays are stored at one continuous place in memory unlike
lists, so processes can access and manipulate them very efficiently.
This is the main reason why NumPy is faster than lists. Also it is optimized to work with latest CPU
architectures.
It is a table of elements (usually numbers), all of the same type, indexed by a tuple of positive integers.
In NumPy, dimensions are called axes. The number of axes is rank. NumPy’s array class is called
ndarray. It is also known by the alias array. Example:
In this example, we are creating a two-dimensional array that has the rank of 2 as it has 2 axes.
The first axis(dimension) is of length 2, i.e., the number of rows, and the second axis(dimension) is of
length 3, i.e., the number of columns. The overall shape of the array can be represented as (2, 3)
In [2]:
import numpy as np
Create a NumPy ndarray Object: NumPy is used to work with arrays. The array object in NumPy is
called ndarray.
[1 2 3 4 5]
<class 'numpy.ndarray'>
To create an ndarray, we can pass a list, tuple, set or any array-like object into the array() method, and it
will be converted into an ndarray:
In [3]:
import numpy as np
This minimize the necessity of growing arrays, an expensive operation. For example: np.zeros, np.ones,
np.full, np.empty, etc.
To create sequences of numbers, NumPy provides a function analogous to the range that returns
arrays instead of lists.
In [6]:
# Creating a 3X4 array with all zeros
c = np.zeros((3, 4))
print ("An array initialized with all zeros:\n", c)
Create Using arange() Function arange(): This function returns evenly spaced values within a given
interval. Step size is specified.
In [3]:
# Create a sequence of integers
# from 0 to 30 with steps of 5
import numpy as np
Create Using linspace() Function linspace(): It returns evenly spaced values within a given interval.
In [9]:
import numpy as np
42
# Create a list
list1 = [1, 2, 3, 4, 5]
[1 2 3 4 5]
In [6]:
# Type
import numpy as np
# Create a list
list1 = [1, 2, 3, 4, 5]
<class 'numpy.ndarray'>
[10 20 30 40]
[[ 1 2 3]
[ 4 5 6]
[ 7 8 9]
[10 11 12]]
In [9]:
import numpy as np
12
[[10 20 30]
[40 50 60]]
In [11]:
import numpy as np
[[[ 1 2 3]
[ 4 5 6]
[ 7 8 9]
[10 11 12]]
[[ 1 2 3]
[ 4 5 6]
[ 7 8 9]
[10 11 12]]
[[ 1 2 3]
[ 4 5 6]
[ 7 8 9]
[10 11 12]]
[[ 1 2 3]
[ 4 5 6]
[ 7 8 9]
[10 11 12]]]
NumPy Arrays provides the ndim attribute that returns an integer that tells us how many dimensions
the array have.
0
1
2
3
When the array is created, you can define the number of dimensions by using the ndmin argument.
Example Create an array with 5 dimensions and verify that it has 5 dimensions:
In [14]:
import numpy as np
[[[[[1 2 3 4]]]]]
number of dimensions : 5
Access Array Elements Array indexing is the same as accessing an array element.
The indexes in NumPy arrays start with 0, meaning that the first element has index 0, and the second
has index 1 etc.
# Print the sum of the third and fourth elements of the array
print(arr[2] + arr[3])
7
Access 2-D Arrays To access elements from 2-D arrays we can use comma separated integers
representing the dimension and the index of the element.
Think of 2-D arrays like a table with rows and columns, where the dimension represents the row and
the index represents the column.
In [17]:
import numpy as np
In [18]:
import numpy as np
Access 3-D Arrays To access elements from 3-D arrays we can use comma separated integers
representing the dimensions and the index of the element.
Example Access the third element of the second array of the first array:
In [19]:
import numpy as np
Negative Indexing Use negative indexing to access an array from the end.
In [20]:
import numpy as np
Note: The result includes the start index, but excludes the end index.
In [21]: # Example
# Slice elements from index 1 to index 5 from the following array:
import numpy as np
[2 3 4 5]
In [22]:
# Example
# Slice elements from index 4 to the end of the array
import numpy as np
# Print the sliced elements from index 4 to the end of the array
print(arr[4:])
[5 6 7]
In [23]:
# Example
# Slice elements from the beginning to index 4 (not included):
import numpy as np
# Print the sliced elements from the beginning to index 4 (not included)
print(arr[:4])
[1 2 3 4]
Negative Slicing
Use the minus operator to refer to an index from the end:
In [24]:
# Example
# Slice from the index 3 from the end to index 1 from the end:
import numpy as np
# Print the sliced elements from index 3 from the end to index 1 from the e
print(arr[-3:-1])
[5 6]
STEP
Use the step value to determine the step of the slicing:
In [25]:
# Example
# Return every other element from index 1 to index 5:
import numpy as np
[2 4]
In [26]:
# Example
# Return every other element from the entire array:
import numpy as np
[1 3 5 7]
# Slice the array to get elements from index 1 to index 4 (not included) of
print(arr[1, 1:4])
[7 8 9]
In [92]:
import numpy as np
# Slice the array to get the element at column index 2 from rows 0 to 1
print(arr[0:2, 2])
[3 8]
In [91]:
import numpy as np
[[2 3 4]
[7 8 9]]
strings - used to represent text data, the text is given under quote marks. e.g. "ABCD" integer - used to
represent integer numbers. e.g. -1, -2, -3 float - used to represent real numbers. e.g. 1.2, 42.42 boolean -
used to represent True or False. complex - used to represent complex numbers. e.g. 1.0 + 2.0j, 1.5 +
2.5j
Data Types in NumPy NumPy has some extra data types, and refer to data types with one character,
like i for integers, u for unsigned integers etc.
Below is a list of all data types in NumPy and the characters used to represent them.
Checking the Data Type of an Array The NumPy array object has a property called dtype that returns
the data type of the array:
In [90]:
# Example
# Get the data type of an array object:
import numpy as np
int64
In [89]:
import numpy as np
<U6
In [88]:
import numpy as np
In [87]:
import numpy as np
In [86]:
# Example
# Create an array with data type string:
import numpy as np
In [85]:
# Example
# Create an array with data type 4 bytes integer:
import numpy as np
[1 2 3 4]
int32
The astype() function creates a copy of the array, and allows you to specify the data type as a
parameter.
The data type can be specified using a string, like 'f' for float, 'i' for integer etc. or you can use the data
type directly like float for float and int for integer.
In [84]:
# Example
# Change data type from float to integer by using 'i' as parameter value:
import numpy as np
# Change the data type of the array to integer using 'i' as parameter value
newarr = arr.astype('i')
[1 2 3]
int32
In [83]:
# Example
# Change data type from integer to boolean:
import numpy as np
The copy owns the data and any changes made to the copy will not affect original array, and any
changes made to the original array will not affect the copy.
The view does not own the data and any changes made to the view will affect the original array, and
any changes made to the original array will affect the view.
COPY:
In [82]:
# Example
import numpy as np
[42 2 3 4 5]
[1 2 3 4 5]
The copy SHOULD NOT be affected by the changes made to the original array.
VIEW:
In [81]:
# Example
# Make a view, change the original array, and display both arrays:
import numpy as np
[42 2 3 4 5]
[42 2 3 4 5]
The view SHOULD be affected by the changes made to the original array.
In [80]:
# Example
# Make a view, change the view, and display both arrays:
import numpy as np
[31 2 3 4 5]
[31 2 3 4 5]
The original array SHOULD be affected by the changes made to the view
Every NumPy array has the attribute base that returns None if the array owns the data.
In [79]:
# Example
# Print the value of the base attribute to check if an array owns its data
import numpy as np
None
[1 2 3 4 5]
The copy returns None. The view returns the original array.
NumPy Array Shape
Shape of an Array The shape of an array is the number of elements in each dimension.
Get the Shape of an Array NumPy arrays have an attribute called shape that returns a tuple with each
index having the number of corresponding elements.
In [78]:
# Example
# Print the shape of a 2-D array:
import numpy as np
(2, 4)
The example above returns (2, 4), which means that the array has 2 dimensions, where the first
dimension has 2 elements and the second has 4.
In [77]:
import numpy as np
(4,)
In [76]:
import numpy as np
(4, 4, 3)
In [75]:
# Example
# Create an array with 5 dimensions using ndmin using a vector with values
import numpy as np
# Create a numpy array with 5 dimensions
arr = np.array([1, 2, 3, 4], ndmin=5)
[[[[[1 2 3 4]]]]]
shape of array : (1, 1, 1, 1, 4)
By reshaping we can add or remove dimensions or change number of elements in each dimension.
Reshape From 1-D to 2-D Example Convert the following 1-D array with 12 elements into a 2-D array.
In [74]:
import numpy as np
[ 1 2 3 4 5 6 7 8 9 10 11 12]
(12,)
[[ 1 2 3]
[ 4 5 6]
[ 7 8 9]
[10 11 12]]
(4, 3)
Reshape From 1-D to 3-D Example Convert the following 1-D array with 12 elements into a 3-D array.
The outermost dimension will have 2 arrays that contains 3 arrays, each with 2 elements:
In [73]:
import numpy as np
[ 1 2 3 4 5 6 7 8 9 10 11 12]
(12,)
[[[ 1 2]
[ 3 4]
[ 5 6]]
[[ 7 8]
[ 9 10]
[11 12]]]
(2, 3, 2)
arr3: [[1 2]
[3 4]
[5 6]
[7 8]]
Shape is: (4, 2)
New arr4 is: [[1 2 3 4]
[5 6 7 8]]
New shape is: (2, 4)
In [71]:
# 2 dimensional to 3 dimensional
import numpy as np
# Create a 2-D numpy array
arr3 = np.array([
[1, 2],
[3, 4],
[5, 6],
[7, 8]
])
arr3: [[1 2]
[3 4]
[5 6]
[7 8]]
Shape is: (4, 2)
New arr4 is: [[[1 2]
[3 4]]
[[5 6]
[7 8]]]
New shape is: (2, 2, 2)
In [70]:
# 3 DIMENSIONAL TO 2 DIMENSIONAL
import numpy as np
arr3: [[[1 2]
[3 4]]
[[5 6]
[7 8]]]
Shape is: (2, 2, 2)
New arr4 is: [[1 2]
[3 4]
[5 6]
[7 8]]
New shape is: (4, 2)
Flattening the arrays Flattening array means converting a multidimensional array into a 1D array.
arr3: [[[1 2]
[3 4]]
[[5 6]
[7 8]]]
Shape is: (2, 2, 2)
New arr4 is: [1 2 3 4 5 6 7 8]
New shape is: (8,)
In [68]:
# 2 DIMENSIONAL TO 1 DIMENSIONAL
import numpy as np
arr3: [[1 2]
[3 4]
[5 6]
[7 8]]
Shape is: (4, 2)
New arr4 is: [1 2 3 4 5 6 7 8]
New shape is: (8,)
Original array:
[[1 2 3]
[4 5 6]]
Flattened array:
[1 2 3 4 5 6]
Can We Reshape Into any Shape?: Yes, as long as the elements required for reshaping are equal in
both shapes.
We can reshape an 8 elements 1D array into 4 elements in 2 rows 2D array but we cannot reshape it
into a 3 elements 3 rows 2D array as that would require 3x3 = 9 elements.
Example Try converting 1D array with 8 elements to a 2D array with 3 elements in each dimension (will
raise an error):
Returns Copy or View? Example Check if the returned array is a copy or a view:
In [ ]: import numpy as np
[1 2 3 4 5 6 7 8]
Unknown Dimension
You are allowed to have one "unknown" dimension.
Meaning that you do not have to specify an exact number for one of the dimensions in the reshape
method.
Pass -1 as the value, and NumPy will calculate this number for you.
In [64]:
import numpy as np
[[[1 2]
[3 4]]
[[5 6]
[7 8]]]
In [63]:
import numpy as np
[[ 5 6]
[ 7 8]]
[[ 9 10]
[11 12]]]
Note: There are a lot of functions for changing the shapes of arrays in numpy flatten, ravel and also for
rearranging the elements rot90, flip, fliplr, flipud etc. These fall under Intermediate to Advanced section
of numpy.
As we deal with multi-dimensional arrays in numpy, we can do this using basic for loop of python.
In [62]: # Example
# Iterate on the elements of the following 1-D array:
import numpy as np
1
2
3
Iterating 2-D Arrays In a 2-D array it will go through all the rows.
In [61]:
# Example
# Iterate on the elements of the following 2-D array:
import numpy as np
[1 2 3]
[4 5 6]
In [60]:
# Example
# Iterate on each scalar element of the 2-D array:
import numpy as np
1
2
3
4
5
6
Iterating 3-D Arrays In a 3-D array it will go through all the 2-D arrays.
In [53]:
# Example
# Iterate on the elements of the following 3-D array:
import numpy as np
[[1 2 3]
[4 5 6]]
[[ 7 8 9]
[10 11 12]]
In [52]:
import numpy as np
1
2
3
4
5
6
7
8
9
10
11
12
Iterating on Each Scalar Element In basic for loops, iterating through each scalar of an array we need
to use n for loops which can be difficult to write for arrays with very high dimensionality.
In [54]: # Example
# Iterate through the following 3-D array:
import numpy as np
1
2
3
4
5
6
7
8
We can use op_dtypes argument and pass it the expected datatype to change the datatype of
elements while iterating.
NumPy does not change the data type of the element in-place (where the element is in array) so it
needs some other space to perform this action, that extra space is called buffer, and in order to enable
it in nditer() we pass flags=['buffered'].
In [55]:
# Example
# Iterate through the array as a string:
import numpy as np
np.bytes_(b'1')
np.bytes_(b'2')
np.bytes_(b'3')
Iterating With Different Step Size We can use filtering and followed by iteration.
In [56]: # Example
# Iterate through every scalar element of the 2D array skipping 1 element:
import numpy as np
1
3
5
7
Sometimes we require corresponding index of the element while iterating, the ndenumerate() method
can be used for those usecases.
In [57]:
# Example
# Enumerate on following 1D array elements:
import numpy as np
(0,) 1
(1,) 2
(2,) 3
In [58]: # Example
# Enumerate on following 2D array's elements:
import numpy as np
(0, 0) 1
(0, 1) 2
(0, 2) 3
(0, 3) 4
(1, 0) 5
(1, 1) 6
(1, 2) 7
(1, 3) 8
Joining NumPy Arrays Joining means putting contents of two or more arrays in a single array.
In SQL we join tables based on a key, whereas in NumPy we join arrays by axes.
We pass a sequence of arrays that we want to join to the concatenate() function, along with the axis. If
axis is not explicitly passed, it is taken as 0.
In [59]: # Example
# Join two arrays
import numpy as np
[1 2 3 4 5 6]
In [51]:
# Example
# Join two 2-D arrays along rows (axis=1):
import numpy as np
[[1 2 5 6]
[3 4 7 8]]
Joining Arrays Using Stack Functions Stacking is same as concatenation, the only difference is that
stacking is done along a new axis.
We can concatenate two 1-D arrays along the second axis which would result in putting them one over
the other, ie. stacking.
We pass a sequence of arrays that we want to join to the stack() method along with the axis. If axis is
not explicitly passed it is taken as 0.
In [50]:
# Example
import numpy as np
[[1 4]
[2 5]
[3 6]]
Stacking Along Rows NumPy provides a helper function: hstack() to stack along rows.
In [49]:
# Example
import numpy as np
[1 2 3 4 5 6]
Stacking Along Columns NumPy provides a helper function: vstack() to stack along columns.
In [48]:
# Example
import numpy as np
[[1 2 3]
[4 5 6]]
Stacking Along Height (depth) NumPy provides a helper function: dstack() to stack along height,
which is the same as depth.
In [47]: # Example
import numpy as np
[[[1 4]
[2 5]
[3 6]]]
Joining merges multiple arrays into one and Splitting breaks one array into multiple.
We use array_split() for splitting arrays, we pass it the array we want to split and the number of splits.
In [46]:
# Example
# Split the array in 3 parts:
import numpy as np
If the array has less elements than required, it will adjust from the end accordingly.
In [45]:
# Example
# Split the array in 4 parts:
import numpy as np
Note: We also have the method split() available but it will not adjust the elements when elements are
less in source array for splitting like in example above, array_split() worked properly but split() would
fail.
Split Into Arrays The return value of the array_split() method is an array containing each of the split as
an array.
If you split an array into 3 arrays, you can access them from the result just like any array element:
In [44]:
# Example
# Access the splitted arrays:
import numpy as np
[1 2]
[3 4]
[5 6]
Splitting 2-D Arrays Use the same syntax when splitting 2-D arrays.
Use the array_split() method, pass in the array you want to split and the number of splits you want to
do.
In [43]:
# Example
# Split the 2-D array into three 2-D arrays.
import numpy as np
[array([[1, 2],
[3, 4]]), array([[5, 6],
[7, 8]]), array([[ 9, 10],
[11, 12]])]
Let's look at another example, this time each element in the 2-D arrays contains 3 elements.
In [42]:
# Example
# Split the 2-D array into three 2-D arrays.
import numpy as np
[array([[1, 2, 3],
[4, 5, 6]]), array([[ 7, 8, 9],
[10, 11, 12]]), array([[13, 14, 15],
[16, 17, 18]])]
In addition, you can specify which axis you want to do the split around.
The example below also returns three 2-D arrays, but they are split along the row (axis=1).
In [41]:
# Example
# Split the 2-D array into three 2-D arrays along rows.
import numpy as np
[array([[ 1],
[ 4],
[ 7],
[10],
[13],
[16]]), array([[ 2],
[ 5],
[ 8],
[11],
[14],
[17]]), array([[ 3],
[ 6],
[ 9],
[12],
[15],
[18]])]
In [40]: # Example
# Use the hsplit() method to split the 2-D array into three 2-D arrays alon
import numpy as np
[array([[ 1],
[ 4],
[ 7],
[10],
[13],
[16]]), array([[ 2],
[ 5],
[ 8],
[11],
[14],
[17]]), array([[ 3],
[ 6],
[ 9],
[12],
[15],
[18]])]
Note: Similar alternates to vstack() and dstack() are available as vsplit() and dsplit()
NumPy Searching Arrays
Searching Arrays You can search an array for a certain value, and return the indexes that get a match.
In [39]:
# Example
# Find the indexes where the value is 4:
import numpy as np
(array([3, 5, 6]),)
In [38]:
# Example
# Find the indexes where the values are even:
import numpy as np
(array([1, 3, 5, 7]),)
In [37]: # Example
# Find the indexes where the values are odd:
import numpy as np
(array([0, 2, 4, 6]),)
Search Sorted There is a method called searchsorted() which performs a binary search in the array,
and returns the index where the specified value would be inserted to maintain the search order.
In [36]:
# Example
# Find the indexes where the value 7 should be inserted:
import numpy as np
Example explained: The number 7 should be inserted on index 1 to remain the sort order.
The method starts the search from the left and returns the first index where the number 7 is no longer
larger than the next value.
Search From the Right Side By default the left most index is returned, but we can give side='right' to
return the right most index instead.
In [35]:
# Example
# Find the indexes where the value 7 should be inserted, starting from the
import numpy as np
# Find the index where the value 7 should be inserted, starting from the ri
x = np.searchsorted(arr, 7, side='right')
Example explained: The number 7 should be inserted on index 2 to remain the sort order.
The method starts the search from the right and returns the first index where the number 7 is no longer
less than the next value.
Multiple Values To search for more than one value, use an array with the specified values.
In [34]:
# Example
# Find the indexes where the values 2, 4, and 6 should be inserted:
import numpy as np
[1 2 3]
The return value is an array: [1 2 3] containing the three indexes where 2, 4, 6 would be inserted in the
original array to maintain the order.
Ordered sequence is any sequence that has an order corresponding to elements, like numeric or
alphabetical, ascending or descending.
The NumPy ndarray object has a function called sort(), that will sort a specified array.
In [33]:
# Example
# Sort the array:
import numpy as np
[0 1 2 3]
Note: This method returns a copy of the array, leaving the original array unchanged.
You can also sort arrays of strings, or any other data type:
In [32]: # Example
# Sort the array alphabetically:
import numpy as np
In [31]:
# Example
# Sort a boolean array:
import numpy as np
Sorting a 2-D Array If you use the sort() method on a 2-D array, both arrays will be sorted:
In [30]:
# Example
# Sort a 2-D array:
import numpy as np
Filtering Arrays Getting some elements out of an existing array and creating a new array out of them is
called filtering.
If the value at an index is True that element is contained in the filtered array, if the value at that index is
False that element is excluded from the filtered array.
In [29]:
# Example
# Create an array from the elements on index 0 and 2:
import numpy as np
[41 43]
Because the new array contains only the values where the filter array had the value True, in this case,
index 0 and 2.
Creating the Filter Array In the example above we hard-coded the True and False values, but the
common use is to create a filter array based on conditions.
In [99]:
#Example
#Create a filter array that will return only values higher than 42:
import numpy as np
newarr = arr[filter_arr]
print(filter_arr)
print(newarr)
In [100]:
#Example
#Create a filter array that will return only even elements from the origina
import numpy as np
newarr = arr[filter_arr]
print(filter_arr)
print(newarr)
Creating Filter Directly From Array The above example is quite a common task in NumPy and NumPy
provides a nice way to tackle it.
We can directly substitute the array instead of the iterable variable in our condition and it will work just
as we expect it to.
In [28]: # Example
# Create a filter array that will return only values higher than 42:
import numpy as np
In [27]: # Example
# Create a filter array that will return only even elements from the origin
import numpy as np
In [1]:
import numpy as np
1. Operations on a single NumPy array We can use overloaded arithmetic operators to do element-
wise operations on the array to create a new array. In the case of +=, -=, *= operators, the existing array
is modified.
a = np.array([1, 2, 5, 3])
# transpose of array
a = np.array([[1, 2, 3], [3, 4, 5], [9, 6, 0]])
Original array:
[[1 2 3]
[3 4 5]
[9 6 0]]
Transpose of array:
[[1 3 9]
[2 4 6]
[3 5 0]]
NumPy – Unary Operators Many unary operations are provided as a method of ndarray class. This
includes sum, min, max, etc. These functions can also be applied row-wise or column-wise by setting
an axis parameter.
In [12]:
# Python program to demonstrate
# unary operators in numpy
import numpy as np
NumPy – Binary Operators These operations apply to the array elementwise and a new array is
created. You can use all basic arithmetic operators like +, -, /, etc. In the case of +=, -=, = operators, the
existing array is modified.
a = np.array([[1, 2],
[3, 4]])
b = np.array([[4, 3],
[2, 1]])
# add arrays
print ("Array sum:\n", a + b)
Array sum:
[[5 5]
[5 5]]
Array multiplication:
[[4 6]
[6 4]]
Matrix multiplication:
[[ 8 5]
[20 13]]
NumPy ufuncs NumPy provides familiar mathematical functions such as sin, cos, exp, etc. These
functions also operate elementwise on an array, producing an array as output.
Note: All the operations we did above using overloaded operators can be done using ufuncs like
np.add, np.subtract, np.multiply, np.divide, np.sum, etc.
In [14]:
# Python program to demonstrate
# universal functions in numpy
import numpy as np
# exponential values
a = np.array([0, 1, 2, 3])
print ("Exponent of array elements:", np.exp(a))
NumPy Sorting Arrays There is a simple np.sort() method for sorting Python NumPy arrays. Let’s
explore it a bit.
In [15]:
# Python program to demonstrate sorting in numpy
import numpy as np
a = np.array([[1, 4, 2],
[3, 4, 6],
[0, -1, 5]])
# sorted array
print ("Array elements in sorted order:\n",
np.sort(a, axis = None))
# Creating array
arr = np.array(values, dtype = dtypes)
print ("\nArray sorted by names:\n",
np.sort(arr, order = 'name'))
np.array(): Create an array. np.arange(): Return evenly spaced values within a given interval. np.zeros(),
np.ones(), np.full(): Create new arrays filled with zeros, ones, or a specified value, respectively. np.dot():
Dot product of two arrays. np.reshape(): Gives a new shape to an array without changing its data.
np.mean(), np.median(), np.std(): Compute the mean, median, and standard deviation of array
elements. np.linalg.inv(): Compute the (multiplicative) inverse of a matrix.