How do you remove duplicates from a list?

You can remove duplicates from a list in several ways. Here are the most common methods:

1. Using set() (Fastest, but Unordered)
my_list = [1, 2, 2, 3, 4, 4, 5]
unique_list = list(set(my_list))
print(unique_list)  # Output: [1, 2, 3, 4, 5] (order may change)

Downside : This method does not preserve the original order.


2. Using a Loop (Preserves Order)
my_list = [1, 2, 2, 3, 4, 4, 5]
unique_list = []
for item in my_list:
    if item not in unique_list:
        unique_list.append(item)
print(unique_list)  # Output: [1, 2, 3, 4, 5]

* Keeps the original order.


3. Using Dictionary Keys (dict.fromkeys())
my_list = [1, 2, 2, 3, 4, 4, 5]
unique_list = list(dict.fromkeys(my_list))
print(unique_list)  # Output: [1, 2, 3, 4, 5]

* Preserves order (since Python 3.7+).
* Faster than a loop but slower than set().


4. Using List Comprehension with set() (Efficient)
my_list = [1, 2, 2, 3, 4, 4, 5]
seen = set()
unique_list = [x for x in my_list if not (x in seen or seen.add(x))]
print(unique_list)  # Output: [1, 2, 3, 4, 5]

* Preserves order and is faster than using a loop.

Best Choice?
  • Use set() if order doesn’t matter.
  • Use dict.fromkeys() if order matters and you want a simple approach.
  • Use list comprehension with a set for an optimized, order-preserving solution.