How to Use Yield Keyword for Memory Efficient Python Code

We are given a task to understand how to use the yield keyword for memory-efficient Python code with the help of different examples with their code. In this article, we will see how to use the yield keyword for memory-efficient Python code.

What is the Yield Keyword?

The yield keyword in Python is used in the context of generator functions and is fundamental to creating iterators efficiently. It allows a function to pause its execution and yield a value to the caller without losing its state. When the function is called again, it resumes execution from where it left off.

Use Yield Keyword for Memory-Efficient Python Code

Below are some of the examples by which we can understand how to use the yield keyword for memory-efficient Python code:

Memory-Efficient Even Number Filtering

In this example, filter_even_numbers function utilizes the yield keyword to lazily generate even numbers from the input data, resulting in memory-efficient iteration and printing of the filtered values.

def filter_even_numbers(data):
    for value in data:
        if value % 2 == 0:
            yield value


my_data = [1, 2, 3, 4, 5]
result_generator = filter_even_numbers(my_data)

for value in result_generator:
    print(value)

2
4

Generating Lazy Sequences Efficiently

In this example, in below code The lazy_sequence generator function produces a sequence of doubled values up to the specified limit, demonstrating memory-efficient iteration when consumed in a for loop.

def lazy_sequence(limit):
    for i in range(limit):
        yield i * 2

# Using the generator
gen = lazy_sequence(5)

for value in gen:
    print(value)

0
2
4
6
8

Memory-Efficient Fibonacci Sequence Generation

In this example, the fibonacci_sequence generator function yields Fibonacci sequence values up to the specified limit, showcasing memory-efficient iteration for generating and printing the sequence.

def fibonacci_sequence(limit):
    a, b = 0, 1
    while a < limit:
        yield a
        a, b = b, a + b


fibonacci_gen = fibonacci_sequence(50)

for value in fibonacci_gen:
    print(value)

0
1
1
2
3
5
8
13
21
34

Memory-Efficient Task Parallelism

In this example, the parallel_tasks generator yields lambda functions representing parallel tasks, enabling memory-efficient execution of each task on a given input value (data in this case).

def parallel_tasks():
    yield lambda x: x * 2
    yield lambda x: x ** 2
    # Add more tasks as needed

# Using the generator for parallel tasks
tasks_gen = parallel_tasks()
data = 5

for task in tasks_gen:
    result = task(data)
    print(result)

10
25

Memory Efficiency Comparison: Yield vs. Generator

In this example, below code compares memory usage between a list-based approach (without yield) and a generator-based approach (with yield) for generating a sequence of squares up to a specified limit. The generator-based approach demonstrates superior memory efficiency, especially for large limits, as it generates values on-the-fly without storing the entire sequence in memory.

# Without Yield (List-Based Approach)
import sys


def generate_squares_list(limit):
    squares = [num ** 2 for num in range(limit)]
    return squares

# With Yield (Generator-Based Approach)


def generate_squares_yield(limit):
    for num in range(limit):
        yield num ** 2


# Memory Comparison

limit = 10**6  # Set a large limit for demonstration purposes

# Without Yield (List-Based Approach)
squares_list = generate_squares_list(limit)
memory_usage_list = sys.getsizeof(squares_list)

# With Yield (Generator-Based Approach)
squares_yield = generate_squares_yield(limit)
memory_usage_yield = sys.getsizeof(squares_yield)

print(f"Memory usage without yield: {memory_usage_list} bytes")
print(f"Memory usage with yield: {memory_usage_yield} bytes")

Memory usage without yield: 8697472 bytes
Memory usage with yield: 128 bytes

Conclusion

In conclusion, leveraging generators in Python is a powerful strategy for achieving memory-efficient code, particularly in situations where handling large datasets or dynamic data streams is essential. The lazy evaluation approach of generators allows for on-the-fly generation of values, reducing the overall memory footprint and enabling the processing of data in a scalable and efficient manner.