itertools and functools in Depth

Python’s itertools and functools libraries provide C-optimized functions for complex data manipulation. The primary advantage of itertools is that it operates lazily—consuming virtually zero memory overhead, no matter how massive the datasets involved are.

chain() treats several iterables as one long sequence, without building a combined list in memory. It is the lazy answer to a + b + c.

Chaining and Slicing (chain, islice)

Instead of concatenating lists with + (which copies all elements into a massive new list in memory), itertools.chain seamlessly glues iterables together lazily. Similarly, since you can’t slice a generator, itertools.islice acts as a lazy slicing window.

import itertools

# Chaining avoids creating a new combined list in memory
chained = itertools.chain([1, 2], [3, 4])
print(list(chained))

# Slicing an infinite generator
def counter():
    n = 0
    while True:
        yield n
        n += 1

# Slice from index 2 to 5
sliced = itertools.islice(counter(), 2, 5)
print(list(sliced))

[1, 2, 3, 4]
[2, 3, 4]

Combinatoric Iterators

Need to generate Cartesian products, permutations, or combinations? Writing nested for loops is messy and slow. itertools handles this instantly.

import itertools

letters = ['A', 'B', 'C']

# Cartesian product (equivalent to nested for loops)
prod = itertools.product(letters, repeat=2)
print("Product:", list(prod)[:3], "...")

# Order matters
perms = itertools.permutations(letters, 2)
print("Permutations:", list(perms))

# Order doesn't matter
combs = itertools.combinations(letters, 2)
print("Combinations:", list(combs))

Product: [('A', 'A'), ('A', 'B'), ('A', 'C')] ...
Permutations: [('A', 'B'), ('A', 'C'), ('B', 'A'), ('B', 'C'), ('C', 'A'), ('C', 'B')]
Combinations: [('A', 'B'), ('A', 'C'), ('B', 'C')]

Grouping Data

groupby() collapses consecutive equal keys into groups — the same behaviour as Unix uniq. That word consecutive is the whole gotcha: unsorted input produces several groups for the same key, so sort by the same key first.

Unlike SQL’s GROUP BY, Python’s itertools.groupby only groups consecutive matching elements. This means you must sort your data by the exact same key before you group it!

import itertools

data = [('A', 1), ('B', 2), ('A', 3)]

# Step 1: Sort by the key you want to group by
data.sort(key=lambda x: x[0])

# Step 2: Group it!
groups = {k: list(g) for k, g in itertools.groupby(data, key=lambda x: x[0])}

print(groups)

{'A': [('A', 1), ('A', 3)], 'B': [('B', 2)]}

Single Dispatch Polymorphism

@singledispatch turns one function into several implementations chosen by the type of the first argument. It is Python’s answer to a wall of isinstance() checks: register a handler per type and let the dispatcher pick.

Python doesn’t support method overloading out of the box like Java or C++ (defining the same function multiple times with different types). However, functools.singledispatch achieves exactly this for the first positional argument!

from functools import singledispatch

@singledispatch
def process(arg):
    return "default handling"

@process.register
def _(arg: int):
    return f"Handling integer: {arg * 2}"

@process.register
def _(arg: str):
    return f"Handling string: {arg.upper()}"

print(process(1.5))
print(process(42))
print(process("hello"))

default handling
Handling integer: 84
Handling string: HELLO