Iterables and Iterators in Python
Iterables, iterators, and iteration in Python
In this article, we will learn the differences between iteration, iterables, and iterators, how to identify iterables and iterators, and why it can be useful to be able to do so.
What is an Iterable?
Broadly speaking, an iterable is something that can be looped over. The process of looping over something, or taking each item of it, one after another, is iteration. For example, when we use a for loop to loop over a list, the process of looping over this list is iteration (or we are iterating over this list), and the list is the iterable. Tuples, dictionaries, strings, files, and generators are also iterables, as they can also be iterated over.
For example, we can use a for loop to loop over or iterate over a list as follows:
num_list = [1,2,3,4,5]for num in num_list:
print(num)But what makes something an iterable?
In other words, what makes something capable of being iterated over, such as in a for loop? To be an iterable, an object will have an iter() method. So if an object has an iter() method, then it is an iterable, and thus we can use a for loop to loop over it.
Technically, if an object has a getitem() method it can also be an iterable. However, we will focus on the iter() method here as that is the newer protocol for iteration in python.
If we use the dir() function on _numlist which will show us all the methods and attributes of that list object:
Notice how the list object, _numlist, has an iter() method, which makes it an iterable. Methods that have double leading and trailing underscores are hence called dunder methods (also known as magic methods or special methods). These methods are intended to be invoked indirectly, as most python operators have an associated magic method.
So what does this iter() method do?
An iter() method will return an iterator object. To loop over an iterable, such as a list, we don’t usually do that manually. For example, a for loop first calls the iter() method on our object which returns an iterator object. The for loop then loops over this iterator. All of this is done in the background.
So this brings us to iterators.
What is an Iterator?
A list is an iterable. But it is not an iterator. If we run the iter() method on our list, it will return an iterator. An iterator is an object with a state that remembers where it is during iteration. Iterators also know how to get their next value. They get their next value with a next() method. Thus, to know if an object is an iterator, we can check if it has a next() method.
We can first call the iter() function on num_list (the iter() function calls the iter() method on the object passed in), which returns an iterator, and then we will use the dir() function on the iterator to view its attributes and methods:
Notice how our iterator, _num_listiterator, has a next() method. A list object is not an iterator because it does not have a next() method. Thus calling the next() function (which would call the next() method on an object if it has one) on a list would result in a TypeError, since our list object is not an iterator and thus does not have a next() method. However, a list object is an iterable because it has an iter() method, which when called, returns an iterator that will have a next() method.
Notice how num_list_iterator also has an iter() method. As a result, iterators are also iterables, or self-iterables, since calling the iter() method on an iterator returns self.
What does the next() method do?
Calling the next() method on an iterator will return the next value in the iteration. Thus, calling the next() method on the iterator we created from _numlist, or _num_listiterator, will return the first value in our iterator. Calling the next() method on that same iterator will then return the second value in our iterator (remember that iterators have a state which remembers where it is during iteration). Once we’ve went through all the values of our iterator, calling the next() method again on our iterator will result in a StopIteration error, since our iterator has now been exhausted. Once an iterator is exhausted, we must make a new iterator via the iter() method of the iterable.
Note: Iterators can only go forward. Meaning we can only get the next value in an iterator via the next() method. To start from scratch, we would have to create a new iterator object by calling the iter() method on the iterable we would like to iterate over.
How does a for loop work?
A for loop works by doing the above in the background. Using a for loop to loop over a list (or other iterable) begins with it first creating an iterator object via the iter() method, then calling the next() method on that iterator, returning the values until it returns a StopIteration error (that it does not show to us), which lets it know that the iterator has been exhausted since there are no more values left in the iterator. This can be seen below:
Notice how instead of typing out the dunder methods, we can use the built-in iter() and next() functions, which call the iter() and next() methods for us. However, we could have also called the methods as follows:
num_list.__iter__()num_list_iterator.__next__()If you are familiar with while loops and try/except statements, then the above for loop is analogous to the following while loop:
Why is this useful to know?
Knowing whether an object is an iterable (or can be iterated over) can be very helpful while coding in python. For example, knowing if an object is an iterable would tell us if an object can be used in a for loop.
In the above example, we used a for loop to loop over _numlist, which was a list of integers starting at 1 and ending at 5, or [1,2,3,4,5]. Well, instead of creating _numlist as we did above and looping through it, we could check to see if we can use the range() function to create a range object that we can loop over using a for loop, giving us the same sequence of numbers to iterate over:
As we can see, the range object is an iterable since it has the iter() method! Therefore, it can be iterated over using a for loop, since the for loop will first create the iterator object, and then iterate over that iterator object using the next() method.
for num in range(1,6):
print(num)If you enjoy reading stories like these and want to support me as a writer, consider signing up to become a Medium member. It’s $5 a month, giving you unlimited access to stories on Medium. If you sign up using my link, I’ll earn a small commission.
Conclusion
In this tutorial, we discussed what iteration, iterables, and iterators all mean. We learned how an iterable is an object that has an iter() method, which will create an iterator object. We also learned that an iterator is an object that has a state which remembers where it is during iteration, and that they know how to get their next value via their next() method. Finally, we saw how being able to identify an object as an iterable helped us write more pythonic code.
Share This Article
Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.
Write for TDS
