Python – Remove Duplicates from a List

Removing duplicates from a list is a common operation in Python which is useful in scenarios where unique elements are required. Python provides multiple methods to achieve this. Using set() method is most efficient for unordered lists. Converting the list to a set removes all duplicates since sets only store unique elements.

a = [1, 2, 2, 3, 4, 4, 5]

unique_list = list(set(a))
print(unique_list)

[1, 2, 3, 4, 5]

Explanation:

• set(a) removes duplicates but may reorder elements since sets are unordered.
• Wrap the result in list() to convert it back to a list.

Let’s explore some other methods to remove duplicates from a list.

Using List Comprehension

List comprehension combined with a enumerate() allows removing duplicates while preserving the original order. To remove duplicates while maintaining the original order we use a set to track seen elements to preserve order.

a = [1, 2, 2, 3, 4, 4, 5]

# Using list comprehension to remove duplicates
unique_list = [item for index, item in enumerate(a) if item not in a[:index]]
print(unique_list)

[1, 2, 3, 4, 5]

Explanation:

• seen.add(x) ensures each element is only added once.
• The list comprehension iterates over the original list and appends only unique elements.

Using dict.fromkeys()

Dictionaries in Python 3.7+ maintain insertion order making this method efficient way to remove duplicates while preserving order.

a = [1, 2, 2, 3, 4, 4, 5]

unique_list = list(dict.fromkeys(a))
print(unique_list)

[1, 2, 3, 4, 5]

Explanation:

• dict.fromkeys(a) creates a dictionary with list elements as keys automatically discarding duplicates.
• Convert the dictionary back to a list using list().

Using a Loop for Explicit Control

Using loop is useful for beginners as it follows step-by-step logic to remove duplicates. This method will remove the original order of elements.

a = [1, 2, 2, 3, 4, 4, 5]

unique_list = []

for x in a:
    if x not in unique_list:
        unique_list.append(x)

print(unique_list)

[1, 2, 3, 4, 5]

Explanation:

• Iterates over the original list and appends only elements not already in unique_list.
• This method preserves the order of elements.

chinmoy lenka

Follow

Improve

Article Tags :

• Python
• Python Programs
• Python list-programs
• python-list

Practice Tags :

• python
• python-list