One of the key features of Python is its support for iterable objects and iterators. In this article, we will take a closer look at what they are and how they can be used.

Iterable Object

An iterable object in Python is an object that can be iterated over using a loop or comprehension, such as lists, tuples, sets, and dictionaries.

numbers = [1, 2]
even_numbers = [number for number in numbers if number % 2 == 0]
for number in even_numbers:
    print(number)

In Python, every iterable will have a __iter__() method, which returns something called an iterator. The for loop and comprehension in the example makes use of this iterator under the hood (we will see the working of for loop later in this article). So what exactly is this "iterator"? 🥵

Iterator

An iterator is an object that represents a stream of data. It provides a way to access the elements of an iterable object one at a time. In Python, an iterator is an object that implements the iterator protocol.

Iterator Protocol

The iterator protocol consists of two methods: __iter__() and __next()__ . The iter method returns the iterator object itself, and the next method returns the next element in the stream. If there are no more elements in the stream, then the next will raise the StopIteration exception. let's verify if the iterables we know, do implement this protocol or not: 💻

print(f'Is iterable ? {"__iter__" in dir(numbers)}')

# The built-in function: iter(iterable) can also be used -> iter(numbers)
iterator = numbers.__iter__()
print(f'Class of iterator: {iterator.__class__.__name__}')

has_iter = '__iter__' in dir(iterator)
has_next = '__next__' in dir(iterator)
print(f'Does it implement the iterator protocol? {has_iter and has_next}')

# Is iterable ? True
# Class of iterator: list_iterator
# Does it implement the iterator protocol? True

So Python list is indeed an iterable as it has the __iter__() method, which returns an object implementing the iterator protocol (iterator).

Now let's explore what these protocol methods are doing. As per the definition, calling the iter method should return the iterator object itself and each next call should yield the next element from the iterable. Let's verify this: 💻

print(iterator is iterator.__iter__()) # True

# The built-in function: next(iterator) can also be used
print(iterator.__next__()) # 1
print(iterator.__next__()) # 2

Since we iterate all the elements in the list, the next call should now raise the StopIteration exception. Let's see: 💻

try:
    print(iterator.__next__())
except StopIteration:
    print('Iteration completed')

# Iteration completed

Custom Iterator

Now that we know the iterator protocol, let's create a custom iterator object by implementing the same. There is a built-in function: range(limit) in Python returns an iterator object which yields numbers in the range [0, limit). We will try to implement a similar interface where the sequence will be [1, limit]: 🤓

class MyRange:
    def __init__(self, limit):
        self.__limit = limit
        self.__current = 1

    def __iter__(self):
        return self

    def __next__(self):
        if self.__current <= self.__limit:
            current = self.__current
            self.__current += 1
            return current
        else:
            raise StopIteration

for num in MyRange(2):
    print(num) # 1 2

There is another, easier way to create an iterator in Python 😬 - The Generator Function. We will explore it in a dedicated article. 👍

Working of For-Loop

In Python, the for-loop is based on the iterator concept. When you use a for loop over an iterable, Python automatically creates an iterator object of the iterable and calls its next method until the StopIteration exception is raised. Now that we know the working of for loop, let's iterate over the numbers list without using for-loop: 😎

number_iterator = iter(numbers)
while 1:
    try:
        print(next(number_iterator))
    except StopIteration:
        pass

# 1 2

Use Cases

Iterators are more flexible and efficient in terms of memory and time as they can access the elements of a sequence one at a time, without having to access the entire sequence at once. This makes them an excellent choice in use cases such as:

Iterating over large datasets: When working with large datasets, iterators can be used to process data one item at a time, without having to load the entire dataset into memory at once. This can save memory and make your code more efficient. 🤌

Processing streams of data: Iterators can be used to process streams of data in real time, such as log files, sensor data, or social media streams. As new data becomes available, the iterator can be updated to include the new data and process it continuously.
Filtering and transforming data: Iterators can be used to filter and transform data in a flexible and memory-efficient way. For example, you can use a filtered iterator to remove certain elements from a sequence or use a map iterator to apply a function to each element in a sequence.
Custom data structures: Iterators can be used to implement custom data structures in Python. For example, you can create a custom iterator that generates Fibonacci numbers, or an iterator that generates prime numbers.
Lazy evaluation: Iterators allow for lazy evaluation of data, which means that items are generated only as they are needed. This can be useful for large, complex computations where intermediate results can be discarded to save memory.
Parallel processing: Iterators can be used in parallel processing to split large datasets into smaller chunks and process them in parallel. This can speed up computations on multi-core machines or distributed systems

Now that you have mastered the iterator concept, can you predict the output of this code? 😜✌️

languages = {"Python": 1, "JavaScript": 2, "Go": 3}
language_iter = iter(languages)
next(language_iter)
print(next(language_iter)) # ?
print(list(language_iter)) # ?

Well, that’s all for this article. Thanks for reading! 🙏

I hope you liked this article and if you have any questions or suggestions, please drop them in the comments. Happy Coding.. 👋

Iterators in Python