Learn › Python Programming

Decorators & Generators

Understand Python decorators for modifying function behavior and generators for memory-efficient iteration.

Understanding Decorators

Decorators are functions that modify the behavior of other functions without changing their source code. They take a function as an argument, wrap it with additional functionality, and return the modified function. Decorators are a powerful example of higher-order functions and the closure pattern in Python. They are widely used in frameworks like Flask and Django for routing, authentication, and caching.

import time
from functools import wraps

# Basic decorator
def timer(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        start = time.time()
        result = func(*args, **kwargs)
        end = time.time()
        print(f"{func.__name__} took {end - start:.4f} seconds")
        return result
    return wrapper

@timer
def slow_function():
    time.sleep(1)
    return "Done"

result = slow_function()  # "slow_function took 1.00xx seconds"
print(result)             # "Done"
print(slow_function.__name__)  # "slow_function" (preserved by @wraps)

Practical Decorator Patterns

Decorators can accept arguments by adding an additional layer of nesting, creating a decorator factory. This pattern is useful for creating configurable decorators that can be customized for different use cases. Decorators can also be stacked, with each decorator wrapping the result of the one below it. Class-based decorators are another option when you need to maintain state across calls.

from functools import wraps

# Decorator with arguments
def retry(max_attempts=3):
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            for attempt in range(1, max_attempts + 1):
                try:
                    return func(*args, **kwargs)
                except Exception as e:
                    print(f"Attempt {attempt} failed: {e}")
                    if attempt == max_attempts:
                        raise
        return wrapper
    return decorator

# Memoization decorator
def memoize(func):
    cache = {}
    @wraps(func)
    def wrapper(*args):
        if args not in cache:
            cache[args] = func(*args)
        return cache[args]
    return wrapper

@memoize
def fibonacci(n):
    if n < 2:
        return n
    return fibonacci(n - 1) + fibonacci(n - 2)

print(fibonacci(50))  # 12586269025 (computed instantly)

Generators and Yield

Generators are special functions that use the yield keyword to produce a sequence of values lazily, one at a time. Unlike regular functions that return a single value and lose their state, generators pause execution at each yield and resume where they left off. This makes them extremely memory-efficient for working with large datasets or infinite sequences. Generator expressions provide a concise syntax similar to list comprehensions but produce values on demand.

# Basic generator function
def countdown(n):
    while n > 0:
        yield n
        n -= 1

for num in countdown(5):
    print(num)  # 5, 4, 3, 2, 1

# Generator for Fibonacci sequence
def fibonacci_gen():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

fib = fibonacci_gen()
first_10 = [next(fib) for _ in range(10)]
print(first_10)  # [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]

# Generator expression
squares_gen = (x ** 2 for x in range(1000000))
print(sum(squares_gen))  # Memory-efficient sum

# Generator for reading large files
def read_chunks(filename, chunk_size=1024):
    with open(filename, "r") as f:
        while True:
            chunk = f.read(chunk_size)
            if not chunk:
                break
            yield chunk

Itertools for Advanced Iteration

The itertools module provides a collection of fast, memory-efficient tools for working with iterators. Functions like chain, cycle, islice, and combinations are building blocks for creating complex iteration patterns. These tools follow the iterator protocol and produce values lazily, making them suitable for processing large or infinite datasets. Combining itertools functions with generators allows you to build powerful data processing pipelines.

import itertools

# chain: combine multiple iterables
combined = list(itertools.chain([1, 2], [3, 4], [5, 6]))
print(combined)  # [1, 2, 3, 4, 5, 6]

# islice: slice an iterator
fib = (a := 0, b := 1) and (
    a for a, b in iter(lambda: None, None)
) # simplified version below:

def fib_gen():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

first_5 = list(itertools.islice(fib_gen(), 5))
print(first_5)  # [0, 1, 1, 2, 3]

# combinations and permutations
combos = list(itertools.combinations("ABC", 2))
print(combos)  # [('A', 'B'), ('A', 'C'), ('B', 'C')]

perms = list(itertools.permutations("AB", 2))
print(perms)   # [('A', 'B'), ('B', 'A')]

# groupby
data = sorted(["apple", "avocado", "banana", "blueberry", "cherry"])
for key, group in itertools.groupby(data, key=lambda x: x[0]):
    print(f"{key}: {list(group)}")
# a: ['apple', 'avocado']
# b: ['banana', 'blueberry']
# c: ['cherry']

← File Handling