Python 3.6 Dictionary Implementation using Hash Tables
Last Updated :
16 Dec, 2022
Dictionary in Python is a collection of data values, used to store data values like a map, which, unlike other Data Types that hold only a single value as an element, Dictionary holds key:value pair. Key-value is provided in the dictionary to make it more optimized. Each key-value pair in a Dictionary is separated by a colon :, whereas each key is separated by a ‘comma’. To know more about dictionaries click here.
Based on a proposal by Raymond Hettinger the new dict() function has 20% to 25% less memory usage compared to python v.3.5 or less. It relies upon the order-preserving semantics proposed by Raymond Hettinger. This implementation makes the dictionaries more compact and provides a faster iteration over them.
The memory layout of dictionaries in earlier versions was unnecessarily inefficient. It comprised of a sparse table of 24-byte entries that stored the hash value, the key pointer, and the value pointer. The memory layout of dictionaries in version 3.5 and less were implemented to store in a single sparse table.
Example:
for the below dictionary:
d = {'banana':'yellow', 'grapes':'green', 'apple':'red'}
used to store as:
entries = [['--', '--', '--'],
[-5850766811922200084, 'grapes', 'green'],
['--', '--', '--'],
['--', '--', '--'],
['--', '--', '--'],
[2247849978273412954, 'banana', 'yellow'],
['--', '--', '--'],
[-2069363430498323624, 'apple', 'red']]
Instead, in the new dict() implementation the data is now being organized in a dense table referenced by a sparse table of indices as follows:
indices = [None, 1, None, None, None, 0, None, 2]
entries = [[2247849978273412954, 'banana', 'yellow']
[-5850766811922200084, 'grapes', 'green'],
[-2069363430498323624, 'apple', 'red']]
It is important to notice that in the new dict() implementation only the data layout has been changed and no changes are made in the hash table algorithms. Neither the collision statistics nor the table search order has been changed.
This new implementation of dict() is believed to significantly compress dictionaries for memory saving depending upon the size of the getdictionary. Small dictionaries gets the most benefit out of it.
For a sparse table of size t with n entries, the sizes are:
curr_size = 24 * t
new_size = 24 * n + sizeof(index) * t
In the above example banana/grapes/apple, the size of the former implementation is 192 bytes ( eight 24-byte entries) and the later implementation has a size of 90 bytes ( three 24-byte entries and eight 1-byte indices ). That shows around 58% compression in size of the dictionary.
In addition to saving memory, the new memory layout makes iteration faster. Now functions like Keys(), items(), and values can loop over the dense table without having to skip empty slots, unlike the older implementation. Other benefits of this new implementation are better cache utilization, faster resizing and fewer touches to the memory.
Similar Reads
Sparse Matrix in Python using Dictionary
A sparse matrix is a matrix in which most of the elements have zero value and thus efficient ways of storing such matrices are required. Sparse matrices are generally utilized in applied machine learning such as in data containing data-encodings that map categories to count and also in entire subfie
2 min read
How to maintain dictionary in a heap in Python ?
Prerequisites: Binary heap data structureheapq module in PythonDictionary in Python. The dictionary can be maintained in heap either based on the key or on the value. The conventions to be maintained are listed below: The key-value pair at index 'i' is considered to be the parent of key-value pair a
9 min read
Implementation of Hashing with Chaining in Python
Hashing is a data structure that is used to store a large amount of data, which can be accessed in O(1) time by operations such as search, insert and delete. Various Applications of Hashing are: Indexing in database Cryptography Symbol Tables in Compiler/Interpreter Dictionaries, caches, etc. Concep
3 min read
Bidirectional Hash table or Two way dictionary in Python
We know about Python dictionaries in a data structure in Python which holds data in the form of key: value pairs. In this article, we will discuss the Bidirectional Hash table or Two-way dictionary in Python. We can say a two-way dictionary can be represented as key â‡â‡’ value. One example of two-way
4 min read
Merging and Updating Dictionary Operators in Python 3.9
Python 3.9 is still in development and scheduled to be released in October this year. On Feb 26, alpha 4 versions have been released by the development team. One of the latest features in Python 3.9 is the merge and update operators. There are various ways in which Dictionaries can be merged by the
3 min read
How to convert a MultiDict to nested dictionary using Python
A MultiDict is a dictionary-like object that holds multiple values for the same key, making it a useful data structure for processing forms and query strings. It is a subclass of the Python built-in dictionary and behaves similarly. In some use cases, we may need to convert a MultiDict to a nested d
3 min read
Python | Pretty Print a dictionary with dictionary value
This article provides a quick way to pretty How to Print Dictionary in Python that has a dictionary as values. This is required many times nowadays with the advent of NoSQL databases. Let's code a way to perform this particular task in Python. Example Input:{'gfg': {'remark': 'good', 'rate': 5}, 'c
7 min read
Return Dictionary from a Function in Python
Returning a dictionary from a function allows us to bundle multiple related data elements and pass them back easily. In this article, we will explore different ways to return dictionaries from functions in Python. The simplest approach to returning a dictionary from a function is to construct it dir
3 min read
How to use a List as a key of a Dictionary in Python 3?
In Python, we use dictionaries to check if an item is present or not . Dictionaries use key:value pair to search if a key is present or not and if the key is present what is its value . We can use integer, string, tuples as dictionary keys but cannot use list as a key of it . The reason is explained
3 min read
Interesting Facts About Python Dictionary
Python dictionaries are one of the most versatile and powerful built-in data structures in Python. They allow us to store and manage data in a key-value format, making them incredibly useful for handling a variety of tasks, from simple lookups to complex data manipulation. There are some interesting
8 min read