Remove all Duplicates and Permutations in Nested List - Python
Last Updated :
28 Jan, 2025
Given a nested list the task is to remove all duplicates and permutations in that nested list. For example: we have the nested list : [[ -11, 0, 11], [ -11, 11, 0], [ -11, 0, 11], [-11, 2, -11], [-11, 2, -11], [-11, -11, 2]] then output will be {(-11, 0, 11), (-11, -11, 2)}
Using Map
We first sort each sublist to remove any permutations then use the map()function to apply sorted() to each sublist thus converting each sublist into a tuple (to make them hashable) and store them in a set to remove duplicates.
Python
li = [[-11, 0, 11], [-11, 11, 0], [-11, 0, 11],
[-11, 2, -11], [-11, -11, 2], [2, -11, -11]]
# Sorting and removing duplicates
out = set(map(lambda x: tuple(sorted(x)), li))
print(out)
Output{(-11, 0, 11), (-11, -11, 2)}
Explanation: sorted(x) ensures that permutations of the same elements are treated as equal and set removes duplicates to stores unique, sorted tuples.
Using Set Comprehension
In this method we sort each sublist and convert it into a tuple. By using set comprehension only unique sorted tuples are kept automatically removing duplicates and permutations.
Python
li = [[-11, 0, 11], [-11, 11, 0], [-11, 0, 11],
[-11, 2, -11], [-11, -11, 2]]
# Sorting and removing duplicates using set comprehension
out = {tuple(sorted(x)) for x in li}
print(out)
Output{(-11, 0, 11), (-11, -11, 2)} Using sort() and "not in" operator
This method involves manually sorting each sublist using the sort() function then iterating through the list to remove duplicates after that we append each unique sorted sublist into a result list by checking if it already exists using the "not in" operator.
Python
input = [[-11, 0, 11], [-11, 11, 0], [-11, 2, -11],
[-11, -11, 2], [2, -11, -11]]
# Sorting sublist elements and removing duplicates
res = []
for i in input:
i.sort()
res.append(i)
output = []
for i in res:
if i not in output:
output.append(i)
output = list(map(tuple, output))
print(tuple(output))
Output((-11, 0, 11), (-11, -11, 2))
Using operator.countOf() method
This method sorts each sublist and then uses the operator.countOf() method to check the frequency of each sorted sublist in the result list and if the count is zero then it means the sublist is unique and we append it to the result.
Python
import operator as op
input = [[-11, 0, 11], [-11, 11, 0], [-11, 2, -11],
[-11, -11, 2], [2, -11, -11]]
# Sorting sublist elements and removing duplicates using countOf
res = []
for i in input:
i.sort()
res.append(i)
output = []
for i in res:
if op.countOf(output, i) == 0:
output.append(i)
output = list(map(tuple, output))
print(tuple(output))
Output((-11, 0, 11), (-11, -11, 2))
Similar Reads
Python | Remove duplicates from nested list The task of removing duplicates many times in the recent past, but sometimes when we deal with the complex data structure, in those cases we need different techniques to handle this type of problem. Let's discuss certain ways in which this task can be achieved. Method #1 : Using sorted() + set()Â Th
5 min read
Python - Remove Duplicates from a list And Keep The Order While lists provide a convenient way to manage collections of data, duplicates within a list can sometimes pose challenges. In this article, we will explore different methods to remove duplicates from a Python list while preserving the original order.Using dict.fromkeys()dict.fromkeys() method creat
2 min read
Python | Remove all occurrences in nested list The task of removing an element generally doesn't pose any challenge, but sometimes, we may have a more complex problem than just removing a single element or performing removal in just a normal list. The problem can be removing all occurrences of the nested list. Let's discuss certain ways in which
5 min read
Removing Duplicates of a List of Sets in Python Dealing with sets in Python allows for efficient handling of unique elements, but when working with a list of sets, you may encounter scenarios where duplicates need to be removed. In this article, we will see how we can remove duplicates of a list of sets in Python. Remove Duplicates of a List of S
2 min read
Python | Remove duplicates in Matrix While working with Python Matrix, we can face a problem in which we need to perform the removal of duplicates from Matrix. This problem can occur in Machine Learning domain because of extensive usage of matrices. Let's discuss certain way in which this task can be performed. Method : Using loop This
2 min read
Python - Remove Duplicates from a List Removing duplicates from a list is a common operation in Python which is useful in scenarios where unique elements are required. Python provides multiple methods to achieve this. Using set() method is most efficient for unordered lists. Converting the list to a set removes all duplicates since sets
2 min read
Python | Remove duplicate lists in tuples (Preserving Order) Sometimes, while working with records, we can have a problem in which we need to remove duplicate records. This kind of problem is common in web development domain. Let's discuss certain ways in which this task can be performed. Method #1 : Using list comprehension + set() In this method, we test fo
7 min read
Python - Remove Equilength and Equisum Tuple Duplicates Sometimes, while working with Python tuples, we can have a problem in which we need to remove duplicates on basis of equal length and equal sum. This kind of problem can also be broken to accommodate any one of the required conditions. This kind of problem can occur in data domains and day-day progr
7 min read
Python | Remove duplicates based on Kth element tuple list Sometimes, while working with records, we may have a problem in which we need to remove duplicates based on Kth element of a tuple in the list. This problem has application in domains that uses records as input. Let's discuss certain ways in which this problem can be solved. Method #1: Using loop Th
8 min read
Python - Remove Kth Index Duplicates in Tuple Sometimes, while working with Python records, we can have a problem in which we need to remove all the tuples, which have similar Kth index elements in list of records. This kind of problem is common in day-day and web development domain. Let's discuss certain ways in which this task can be performe
7 min read