Beginner Python Exercises

Remove Duplicates from a List in Python (Keep Order)

Keep the first copy of every value in the original order, using a set to remember what you have seen.

  • Beginner
  • 2 min read
  • 1 runnable example
What you'll learn
  • Tracking seen values with a set
  • Building an ordered result
  • The dict.fromkeys shortcut
Input, then result
1
2
1
3
2
4
1
2
3
4
The highlighted repeats are skipped; order is kept.

Add a value only the first time

The problem

Data often contains repeats: the same email signed up twice, the same tag added again. Converting to a set removes them, but sets lose the original order, which matters for things like the order people signed up.

Think of it like this

Removing duplicates while keeping order is like a guest book at a party. As each guest arrives, you check the list of names already written. If the name is new, you add it; if it is already there, you skip it. The book keeps arrival order.

From a guest book to code
seen = set()📖 The names already written
value in seen🔎 Is this guest already in the book?
seen.add(value)✍️ Write a new guest's name
unique📋 Guests in arrival order: [1, 2, 3, 4]
Keep the first time each value appears, skip the repeats.
In one sentence

Removing duplicates while preserving order keeps the first occurrence of each value and drops later repeats.

Key points

  1. seen is a set that remembers which values have appeared.
  2. value in seen is a fast check for a value that was already kept.
  3. seen.add(value) returns None, so value in seen or seen.add(value) is False for new values and adds them at the same time.
  4. unique is [1, 2, 3, 4], in the order the values first appeared.

When should you use it?

  • Cleaning lists of emails, IDs or tags while keeping their order.
  • Practicing sets for fast membership checks.
  • Removing repeated events or log lines.

When not to use it

  • Order does not matter: set(values) is shorter.
  • Readability matters more than one line: list(dict.fromkeys(values)) or a normal loop is clearer than the seen.add trick.

Example code

Run code →
main.py
values = [1, 2, 1, 3, 2, 4]
seen = set()
# Add each value only on its first occurrence.
unique = [value for value in values if not (value in seen or seen.add(value))]
print(unique)
Get a Run button for this example

Result of running the code

[1, 2, 3, 4]

Code explanation

seen starts as an empty set and records each value the first time it appears. For every value, value in seen or seen.add(value) is True only when the value was seen before; for a new value, seen.add returns None, so the condition is False, the value is kept, and it is recorded at the same moment. The result keeps the first occurrence of each value in the original order.

Change the values and run the program in the CodeUtility online Python compiler without installing Python locally.

Practice exercises

Modify the runnable example with the exercises below to build understanding beyond copying the result.

  1. Change the input and predict the output before running.
  2. Handle empty or invalid input.
  3. Wrap the logic in a function and add more test cases.
Run in the Python compiler →

Common mistakes

Using a set when order matters

Sets do not keep insertion order.

Problem
unique = list(set(values))  # order is not guaranteed
Fix
unique = list(dict.fromkeys(values))

Finished this lesson?

Mark it as learned to track your progress.

Practice with a challenge

Use what this lesson covers to solve a real problem. Each challenge runs in your browser with instant tests and an AI review.

All challenges