How to Use Yield Keyword for Memory Efficient Python Code
Last Updated :
18 Mar, 2024
We are given a task to understand how to use the yield keyword for memory-efficient Python code with the help of different examples with their code. In this article, we will see how to use the yield keyword for memory-efficient Python code.
What is the Yield Keyword?
The yield
keyword in Python is used in the context of generator functions and is fundamental to creating iterators efficiently. It allows a function to pause its execution and yield a value to the caller without losing its state. When the function is called again, it resumes execution from where it left off.
Use Yield Keyword for Memory-Efficient Python Code
Below are some of the examples by which we can understand how to use the yield keyword for memory-efficient Python code:
Memory-Efficient Even Number Filtering
In this example, filter_even_numbers
function utilizes the yield
keyword to lazily generate even numbers from the input data, resulting in memory-efficient iteration and printing of the filtered values.
Python3
def filter_even_numbers(data):
for value in data:
if value % 2 == 0:
yield value
my_data = [1, 2, 3, 4, 5]
result_generator = filter_even_numbers(my_data)
for value in result_generator:
print(value)
Generating Lazy Sequences Efficiently
In this example, in below code The lazy_sequence
generator function produces a sequence of doubled values up to the specified limit, demonstrating memory-efficient iteration when consumed in a for loop.
Python3
def lazy_sequence(limit):
for i in range(limit):
yield i * 2
# Using the generator
gen = lazy_sequence(5)
for value in gen:
print(value)
Memory-Efficient Fibonacci Sequence Generation
In this example, the fibonacci_sequence
generator function yields Fibonacci sequence values up to the specified limit, showcasing memory-efficient iteration for generating and printing the sequence.
Python3
def fibonacci_sequence(limit):
a, b = 0, 1
while a < limit:
yield a
a, b = b, a + b
fibonacci_gen = fibonacci_sequence(50)
for value in fibonacci_gen:
print(value)
Output0
1
1
2
3
5
8
13
21
34
Memory-Efficient Task Parallelism
In this example, the parallel_tasks
generator yields lambda functions representing parallel tasks, enabling memory-efficient execution of each task on a given input value (data
in this case).
Python3
def parallel_tasks():
yield lambda x: x * 2
yield lambda x: x ** 2
# Add more tasks as needed
# Using the generator for parallel tasks
tasks_gen = parallel_tasks()
data = 5
for task in tasks_gen:
result = task(data)
print(result)
Memory Efficiency Comparison: Yield vs. Generator
In this example, below code compares memory usage between a list-based approach (without yield
) and a generator-based approach (with yield
) for generating a sequence of squares up to a specified limit. The generator-based approach demonstrates superior memory efficiency, especially for large limits, as it generates values on-the-fly without storing the entire sequence in memory.
Python3
# Without Yield (List-Based Approach)
import sys
def generate_squares_list(limit):
squares = [num ** 2 for num in range(limit)]
return squares
# With Yield (Generator-Based Approach)
def generate_squares_yield(limit):
for num in range(limit):
yield num ** 2
# Memory Comparison
limit = 10**6 # Set a large limit for demonstration purposes
# Without Yield (List-Based Approach)
squares_list = generate_squares_list(limit)
memory_usage_list = sys.getsizeof(squares_list)
# With Yield (Generator-Based Approach)
squares_yield = generate_squares_yield(limit)
memory_usage_yield = sys.getsizeof(squares_yield)
print(f"Memory usage without yield: {memory_usage_list} bytes")
print(f"Memory usage with yield: {memory_usage_yield} bytes")
OutputMemory usage without yield: 8697472 bytes
Memory usage with yield: 128 bytes
Conclusion
In conclusion, leveraging generators in Python is a powerful strategy for achieving memory-efficient code, particularly in situations where handling large datasets or dynamic data streams is essential. The lazy evaluation approach of generators allows for on-the-fly generation of values, reducing the overall memory footprint and enabling the processing of data in a scalable and efficient manner.
Similar Reads
How to write memory efficient classes in Python?
Memory efficiency is a critical aspect of software development, especially when working with resource-intensive applications. In Python, crafting memory-efficient classes is essential to ensure optimal performance. In this article, we'll explore some different methods to write memory-efficient class
2 min read
How to Write Memory Efficient Loops in Python
When you are working on projects with limited resources or dealing with large amounts of data, it is important to be mindful of how much memory your Python code is using. In Python, loops are fundamental parts of programs, and making them use less memory can make your code run faster. This article e
3 min read
How to Yield Values from List in Python
In Python, we often deal with large lists where processing all elements at once might be inefficient in terms of memory usage. The yield keyword allows us to create iterators that yield values one at a time, instead of storing them all in memory. In this article, we will explore different methods to
3 min read
How to Make Python For Loops Faster?
While alternatives like list comprehensions and built-in functions can be faster sometimes you need to stick with for loops. Here are methods to speed up Python for loops focusing strictly on optimizing the loops themselves. In this article, we will explore several strategies you can use to make you
3 min read
Memory Efficient Python List Creation in Case of Generators
Creating lists in Python can sometimes be memory intensive, especially when dealing with large amounts of data. In such cases, generators offer a solution to reduce memory usage. Table of ContentWhat are Generators?Using List ComprehensionUsing Generator ExpressionUsing map()Using itertools Generato
3 min read
Python | Value summation of key in dictionary
Many operations such as grouping and conversions are possible using Python dictionaries. But sometimes, we can also have a problem in which we need to perform the aggregation of values of key in dictionary list. This task is common in day-day programming. Let's discuss certain ways in which this tas
6 min read
Python - Raise elements of tuple as power to another tuple
Sometimes, while working with records, we can have a problem in which we may need to perform exponentiation, i.e power of tuples. This problem can occur in day-day programming. Letâs discuss certain ways in which this task can be performed. Method #1: Using zip() + generator expression The combinati
8 min read
Get all Tuple Keys from Dictionary - Python
In Python, dictionaries can have tuples as keys which is useful when we need to store grouped values as a single key. Suppose we have a dictionary where the keys are tuples and we need to extract all the individual elements from these tuple keys into a list. For example, consider the dictionary : d
3 min read
Get Values from Dictionary Using For Loop - Python
In Python, dictionaries store data as key-value pairs and we are given a dictionary. Our task is to extract its values using a for loop, this approach allows us to iterate through the dictionary efficiently and retrieve only the values. For example, given d = {'a': 1, 'b': 2, 'c': 3}, we should extr
2 min read
Minimizing Dictionary Memory Usage in Python
We have a dictionary and we need to Minimizing Dictionary Memory Usage in Python. In this article, we will see some generally used methods for Minimizing Dictionary Memory Usage in Python. Example: Input : {'key1': 'value1', 'key2': 'value2', 'key3': 'value3'}Output : Original Dictionary Size: 184 b
3 min read