Python
  • Master functools.wraps and decorator best practices
  • Create decorator factories and parameterized decorators
  • Use functools.lru_cache and functools.partial
  • Implement complex decorator patterns

Advanced Decorators and functools

Introduction

Decorators are one of Python's most powerful features, but we've only scratched the surface. In this lesson, we'll dive deep into advanced decorator patterns, explore the functools module's powerful utilities, and learn how to create sophisticated decorators that can transform your code in elegant ways.

The functools module provides essential tools for functional programming in Python, and understanding these tools will make you a more effective decorator author.

Mastering functools.wraps

We briefly mentioned functools.wraps in the previous lesson, but it's crucial for proper decorator implementation. Let's understand exactly what it does and why it's essential.

The Problem Without wraps

def my_decorator(func):
    def wrapper(*args, **kwargs):
        """This is the wrapper docstring"""
        return func(*args, **kwargs)
    return wrapper

@my_decorator
def important_function():
    """This function does something very important."""
    return "Important result"

print(important_function.__name__)  # "wrapper" - WRONG.
print(important_function.__doc__)   # "This is the wrapper docstring" - WRONG.

The decorator replaces the original function's metadata with the wrapper's metadata, breaking introspection.

The Solution: functools.wraps

import functools

def my_decorator(func):
    @functools.wraps(func)  # Preserves original function metadata
    def wrapper(*args, **kwargs):
        """This is the wrapper docstring"""
        return func(*args, **kwargs)
    return wrapper

@my_decorator
def important_function():
    """This function does something very important."""
    return "Important result"

print(important_function.__name__)  # "important_function" - CORRECT.
print(important_function.__doc__)   # "This function does something very important." - CORRECT.

What wraps Does

functools.wraps copies these attributes from the original function to the wrapper:

  • __name__ - Function name
  • __doc__ - Docstring
  • __module__ - Module where function is defined
  • __qualname__ - Qualified name
  • __annotations__ - Type annotations
  • __wrapped__ - Reference to the original function (Python 3.3+)

Decorator Factories and Parameterized Decorators

Sometimes you want your decorators to accept parameters. This requires a decorator factory - a function that returns a decorator.

Basic Decorator Factory

def repeat(times):
    """Decorator factory that creates a repeating decorator."""
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            results = []
            for _ in range(times):
                results.append(func(*args, **kwargs))
            return results
        return wrapper
    return decorator

@repeat(times=3)
def greet(name):
    return f"Hello, {name}!"

results = greet("Alice")
print(results)  # ['Hello, Alice!', 'Hello, Alice!', 'Hello, Alice!']

How It Works

  1. @repeat(times=3) calls repeat(3), which returns a decorator
  2. The decorator is then applied to the function
  3. When the function is called, it runs the specified number of times

Advanced Parameterized Decorator

def validate_types(**expected_types):
    """Validate function argument types."""
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            # Get function signature
            sig = inspect.signature(func)
            bound_args = sig.bind(*args, **kwargs)
            bound_args.apply_defaults()
            
            # Validate types
            for param_name, expected_type in expected_types.items():
                if param_name in bound_args.arguments:
                    actual_value = bound_args.arguments[param_name]
                    if not isinstance(actual_value, expected_type):
                        raise TypeError(f"Parameter '{param_name}' must be of type {expected_type.__name__}")
            
            return func(*args, **kwargs)
        return wrapper
    return decorator

@validate_types(name=str, age=int)
def create_person(name, age, email=None):
    return {"name": name, "age": age, "email": email}

# Works
person = create_person("Alice", 30, "alice@example.com")

# Raises TypeError
# person = create_person("Bob", "thirty")  # age should be int

functools.lru_cache: Function Result Caching

functools.lru_cache is a decorator that caches function results based on arguments. It's incredibly useful for expensive computations.

Basic Usage

import functools

@functools.lru_cache(maxsize=None)  # Unlimited cache
def fibonacci(n):
    """Calculate nth Fibonacci number with caching."""
    if n < 2:
        return n
    return fibonacci(n-1) + fibonacci(n-2)

# First call - computes
print(fibonacci(10))  # Calculates and caches

# Subsequent calls - uses cache
print(fibonacci(10))  # Instant result
print(fibonacci(5))   # Uses cached values

Cache Information

fibonacci.cache_info()
# CacheInfo(hits=2, misses=11, maxsize=None, currsize=11)

fibonacci.cache_clear()  # Clear the cache

Caching and Optimization Example: Web API Caching

import requests
import functools
import time

@functools.lru_cache(maxsize=100)
def get_weather(city, api_key):
    """Get weather data with caching."""
    url = f"https://api.weatherapi.com/v1/current.json?key={api_key}&q={city}"
    response = requests.get(url)
    return response.json()

# First call makes API request
weather1 = get_weather("London", "your_api_key")

# Second call uses cache (no API request)
weather2 = get_weather("London", "your_api_key")

print("API calls saved:", get_weather.cache_info().hits)

functools.partial: Partial Function Application

functools.partial creates a new function with some arguments already filled in. It's like creating a specialized version of a function.

Basic Usage

import functools

def multiply(x, y):
    return x * y

# Create specialized functions
double = functools.partial(multiply, 2)
triple = functools.partial(multiply, 3)

print(double(5))  # 10
print(triple(5))  # 15

With Keyword Arguments

def send_email(to, subject, body, from_addr="noreply@example.com"):
    print(f"From: {from_addr}")
    print(f"To: {to}")
    print(f"Subject: {subject}")
    print(f"Body: {body}")

# Create specialized email functions
send_support_email = functools.partial(send_email, 
                                     subject="Support Request",
                                     from_addr="support@example.com")

send_support_email(to="user@example.com", body="I need help!")

Caching and Optimization Example: Database Connections

import sqlite3
import functools

def create_connection(db_path, timeout=5.0, isolation_level=None):
    return sqlite3.connect(db_path, timeout=timeout, isolation_level=isolation_level)

# Create connection factories for different databases
connect_to_users = functools.partial(create_connection, "users.db")
connect_to_orders = functools.partial(create_connection, "orders.db", timeout=10.0)

# Use the specialized functions
users_conn = connect_to_users()
orders_conn = connect_to_orders()

functools.singledispatch: Function Overloading

functools.singledispatch enables function overloading based on the type of the first argument. This is Python's way of achieving polymorphism.

Basic Usage

from functools import singledispatch

@singledispatch
def process_data(data):
    """Default processor for unknown types."""
    raise TypeError(f"Don't know how to process {type(data)}")

@process_data.register
def _(data: str):
    """Process string data."""
    return f"String: {data.upper()}"

@process_data.register
def _(data: int):
    """Process integer data."""
    return f"Integer: {data * 2}"

@process_data.register
def _(data: list):
    """Process list data."""
    return f"List with {len(data)} items"

# Usage
print(process_data("hello"))    # "String: HELLO"
print(process_data(42))         # "Integer: 84"
print(process_data([1, 2, 3]))  # "List with 3 items"

Advanced Example: HTML Rendering

@singledispatch
def render_html(obj):
    """Default HTML renderer."""
    return f"<span>{str(obj)}</span>"

@render_html.register
def _(obj: str):
    return f"<p>{obj}</p>"

@render_html.register
def _(obj: int):
    return f"<strong>{obj}</strong>"

@render_html.register
def _(obj: list):
    items = "".join(f"<li>{render_html(item)}</li>" for item in obj)
    return f"<ul>{items}</ul>"

@render_html.register
def _(obj: dict):
    rows = "".join(f"<tr><td>{k}</td><td>{render_html(v)}</td></tr>" for k, v in obj.items())
    return f"<table>{rows}</table>"

# Usage
data = {
    "name": "Alice",
    "age": 30,
    "hobbies": ["reading", "coding", "gaming"]
}

print(render_html(data))
# <table><tr><td>name</td><td><p>Alice</p></td></tr>...

Advanced Decorator Patterns

Decorator Stacking and Order

def add_logging(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        print(f"Calling {func.__name__}")
        result = func(*args, **kwargs)
        print(f"{func.__name__} returned {result}")
        return result
    return wrapper

def add_timing(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        start = time.time()
        result = func(*args, **kwargs)
        end = time.time()
        print(f"{func.__name__} took {end - start:.2f} seconds")
        return result
    return wrapper

@add_logging
@add_timing
def complex_calculation(x, y):
    time.sleep(0.1)
    return x ** y

# Decorators apply from bottom to top
# Result: timing decorator wraps logging decorator
result = complex_calculation(2, 10)

Context Manager Decorators

from contextlib import contextmanager

@contextmanager
def database_transaction(connection):
    """Context manager for database transactions."""
    try:
        yield connection
        connection.commit()
    except Exception as e:
        connection.rollback()
        raise

def transactional(func):
    """Decorator that wraps function in a database transaction."""
    @functools.wraps(func)
    def wrapper(self, *args, **kwargs):
        with database_transaction(self.connection):
            return func(self, *args, **kwargs)
    return wrapper

class DatabaseService:
    def __init__(self, connection):
        self.connection = connection
    
    @transactional
    def transfer_money(self, from_account, to_account, amount):
        # This code runs within a transaction
        self.debit_account(from_account, amount)
        self.credit_account(to_account, amount)

Method Decorators for Classes

def requires_permission(permission):
    """Decorator factory for permission checking."""
    def decorator(method):
        @functools.wraps(method)
        def wrapper(self, *args, **kwargs):
            if not self.has_permission(permission):
                raise PermissionError(f"Permission '{permission}' required")
            return method(self, *args, **kwargs)
        return wrapper
    return decorator

class UserService:
    def __init__(self, user):
        self.user = user
    
    def has_permission(self, permission):
        return permission in self.user.permissions
    
    @requires_permission("admin")
    def delete_user(self, user_id):
        # Only admins can delete users
        pass
    
    @requires_permission("write")
    def update_profile(self, data):
        # Users with write permission can update profiles
        pass

Best Practices for Advanced Decorators

1. Always Use functools.wraps

# Good
@functools.wraps(func)
def wrapper(...):
    ...

# Bad - loses metadata
def wrapper(...):
    ...

2. Handle All Argument Types

# Good - flexible
def decorator(func):
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper

# Bad - brittle
def decorator(func):
    def wrapper(a, b, c=None):
        return func(a, b, c)
    return wrapper

3. Document Decorator Factories

def cache_with_ttl(ttl_seconds):
    """
    Create a caching decorator with time-to-live.
    
    Args:
        ttl_seconds: How long to cache results
        
    Returns:
        Decorated function with TTL caching
    """
    def decorator(func):
        cache = {}
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            key = (args, tuple(sorted(kwargs.items())))
            if key in cache:
                result, timestamp = cache[key]
                if time.time() - timestamp < ttl_seconds:
                    return result
            result = func(*args, **kwargs)
            cache[key] = (result, time.time())
            return result
        return wrapper
    return decorator

4. Consider Performance Impact

# Caching can speed up expensive operations
@functools.lru_cache(maxsize=128)
def expensive_computation(n):
    # Expensive calculation
    return result

# But don't cache everything - consider memory usage
# Use maxsize parameter to limit cache size

Key Points to Remember

  • functools.wraps is essential for preserving function metadata in decorators
  • Decorator factories enable parameterized decorators
  • functools.lru_cache provides automatic function result caching
  • functools.partial enables partial function application
  • functools.singledispatch provides function overloading by type
  • Advanced decorators can transform code behavior in sophisticated ways
  • Always consider performance implications of decorators

Decorators modify existing code, but metaprogramming takes this further by allowing code to create and modify itself at runtime. In the next lesson, we'll explore metaclasses, dynamic class creation, and other metaprogramming techniques that push Python's flexibility to its limits.