How Do I Build A Numpy Array From A Generator?

NumPy is a powerful library in Python for numerical operations, and it provides an efficient array object, numpy.ndarray, for working with large datasets. Often, data is generated using generators, and it becomes necessary to convert this data into NumPy arrays for further analysis. In this article, we'll explore some different methods to build a NumPy array from a generator.

How Do I Build A Numpy Array From A Generator?

Below, are the ways to Build A Numpy Array From A Generator in Python.

• Using numpy.fromiter()
• Using numpy.array()
• Using numpy.concatenate()

Numpy Array From A Generator Using numpy.fromiter()

In this example, the below code defines a generator function generator() that yields numbers from 1 to 10. It then uses NumPy's fromiter function to create an array from the generator, storing it in the variable arr. Finally, it prints the array and its data type.

#importing numpy library
import numpy as np

#generator function
def generator():
    n = 10
    for i in range(1,n+1):
        yield i

print(type(generator()))

if __name__ == "__main__":
    #calling the function and storing in our arrya
    arr = np.fromiter(generator(), dtype=int,count=-1)
    
    #Displaying the array
    print("Array : {}".format(arr))
    print(type(arr))

Output :

<class 'generator'>
Array : [ 1  2  3  4  5  6  7  8  9 10]
<class 'numpy.ndarray'>

Numpy Array From A Generator Using numpy.array()

In this example , below code defines a generator function generator() that yields numbers from 1 to 10. It then converts the generator output to a list and creates a NumPy array from that list using np.array(), storing it in the variable arr. Finally, it prints the array and its data type.

# importing numpy library
import numpy as np

# generator function


def generator():
    n = 10
    for i in range(1, n+1):
        yield i


print(type(generator()))

if __name__ == "__main__":
    # calling the function and storing in our arrya
    arr = np.array(list(generator()), dtype=int)

    # Displaying the array
    print("Array : {}".format(arr))
    print(type(arr))

Output :

<class 'generator'>
Array : [ 1  2  3  4  5  6  7  8  9 10]
<class 'numpy.ndarray'>

Numpy Array From A Generator Using numpy.concatenate()

In this example, below code defines a generator function generator() that yields numbers from 1 to 10. It then converts the generator output to a list (gen_list) and uses NumPy's np.concatenate() to concatenate the list and create an array (arr). Finally, it prints the array and its data type.

# importing numpy library
import numpy as np

# generator function
def generator():
    n = 10
    for i in range(1, n + 1):
        yield i

print(type(generator()))

if __name__ == "__main__":
    # calling the function and converting the 
    #generator to a list
    gen_list = list(generator())
    arr = np.concatenate([gen_list], dtype=int)

    # Displaying the array
    print("Array: {}".format(arr))
    print("Type of Array:", type(arr))

Output :

<class 'generator'>
Array: [ 1  2  3  4  5  6  7  8  9 10]
Type of Array: <class 'numpy.ndarray'>

Conclusion

In conclusion, building a NumPy array from a generator provides a memory-efficient way to handle large datasets by generating data on-the-fly. Utilizing the numpy.fromiter function or directly converting a generator expression to an array with numpy.array allows for seamless integration of generator-based data into NumPy workflows. This approach not only conserves resources but also enhances the overall efficiency and performance of numerical operations in Python.