Python - What Makes Sets Faster Than Lists ?

In Python, both sets and lists are built-in data structures used to store collections of items. While they share some similarities, they differ significantly in how they handle data and their performance characteristics, especially when it comes to operations like membership testing, adding and removing elements. A key reason why sets are faster than lists in many operations is due to their underlying implementation.

In this article, we will explore the key factors that make sets faster than lists in Python, focusing on time complexity, data structures.

How Sets Achieve Faster Operations

One of the most important performance factors in programming is time complexity, which describes the amount of time an algorithm takes to complete relative to the input size.

1. Membership Test

• List: Searching for an element in a list requires scanning through the list element by element. In the worst case, you need to check every single element, which results in a time complexity of O(n).
• Set: In contrast, sets use a hash table to store elements. When you perform a membership test (x in set), Python directly calculates the hash value of x and checks if it exists in the hash table, which happens in O(1) time on average.

2. Add or Remove Elements

• List: Adding or removing an element from a list typically requires shifting elements to maintain contiguous memory. In the worst case, adding or removing an element can take O(n) time if the operation occurs at the beginning or middle of the list.
• Set: In a set, adding or removing an element only involves calculating the hash of the element and placing it in the appropriate spot in the hash table, making both operations O(1) on average.

3. Iteration

• Both sets and lists take O(n) time to iterate over all elements because each element must be visited once.

Comparison between Sets and Lists:

Here’s the comparison of Sets and Lists in terms of data structures, time complexity, and common operations, presented in a table:

Practical Example: Speed Comparison

Let’s see a quick example comparing the speed of membership tests in sets and lists using Python's timeit module:

List time: 0.000372 | Set time: 0.000254

• a = range(10000) creates a range of numbers from 0 to 9999 (lazy generation).
• s = set(a) converts the range into a set, which stores unique elements using a hash table.
• lambda: 9999 in a and lambda: 9999 in s are anonymous functions checking if 9999 is present in the list a or the set s.
• timeit.timeit(...) runs the membership check (9999 in a or 9999 in s) 1000 times and returns the total time taken for all iterations.
• The time for list membership (t1) will be higher because lists require linear search (O(n)), while sets perform faster membership checks due to hashing (O(1)).
• The times for both list and set membership tests are printed, showing the performance difference between the two.