Advanced Python: Pushing the Boundaries of the Language

Introduction

Python is much more than just a beginner-friendly language. When mastered at an advanced level, it can handle high-performance computing, concurrent systems, and elegant metaprogramming.

In this blog, we’ll explore:
✔ Decorators & Metaclasses (Modifying functions & classes at runtime)
✔ Multi-threading vs Multiprocessing (Concurrency & parallelism)
✔ Async Programming (asyncio, aiohttp) (Non-blocking I/O)
✔ Custom Exceptions (Better error handling)
✔ Packaging (setuptools, poetry) (Building distributable packages)
✔ Profiling & Performance Tuning (cProfile, line_profiler)

Let’s unlock Python’s full potential! 🚀

---

1. Decorators & Metaclasses

Decorators (Modify Functions Dynamically)

def log_time(func):
    def wrapper(*args, **kwargs):
        import time
        start = time.time()
        result = func(*args, **kwargs)
        print(f"{func.__name__} took {time.time() - start:.2f}s")
        return result
    return wrapper

@log_time
def slow_function():
    time.sleep(1)

slow_function()  # Prints "slow_function took 1.00s"

Metaclasses (Modify Class Creation)

class SingletonMeta(type):
    _instances = {}
    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances:
            cls._instances[cls] = super().__call__(*args, **kwargs)
        return cls._instances[cls]

class Database(metaclass=SingletonMeta):
    pass

db1 = Database()
db2 = Database()
print(db1 is db2)  # True (only one instance exists)

Use Cases:

• Decorators: Logging, caching (@functools.lru_cache), access control

• Metaclasses: ORMs (like Django models), enforcing patterns (Singleton)

---

2. Multi-threading vs Multiprocessing

Multi-threading (I/O Tasks)

import threading

def fetch_url(url):
    print(f"Fetching {url}...")

threads = []
for url in ["url1", "url2"]:
    t = threading.Thread(target=fetch_url, args=(url,))
    t.start()
    threads.append(t)

for t in threads:
    t.join()  # Wait for all threads

Multiprocessing (CPU Tasks)

from multiprocessing import Pool

def compute_square(n):
    return n * n

with Pool(4) as p:  # 4 processes
    print(p.map(compute_square, [1, 2, 3]))  # [1, 4, 9]

---

3. Async Programming (asyncio, aiohttp)

Basic asyncio Example

import asyncio

async def fetch_data():
    print("Start fetching...")
    await asyncio.sleep(2)  # Non-blocking sleep
    print("Done fetching!")
    return {"data": 1}

async def main():
    task = asyncio.create_task(fetch_data())
    print("Do other work...")
    data = await task
    print(data)

asyncio.run(main())

aiohttp (Async HTTP Requests)

import aiohttp

async def fetch(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as response:
            return await response.text()

---

4. Custom Exceptions (Better Error Handling)

class APIError(Exception):
    """Base error for API failures"""
    def __init__(self, message, status_code=500):
        super().__init__(message)
        self.status_code = status_code

class NotFoundError(APIError):
    """Raised when a resource doesn't exist"""
    def __init__(self, resource):
        super().__init__(f"{resource} not found", 404)

# Usage
try:
    raise NotFoundError("User")
except APIError as e:
    print(f"Error {e.status_code}: {e}")

---

5. Packaging (setuptools, poetry)

setuptools (Traditional)

# setup.py
from setuptools import setup

setup(
    name="my_package",
    version="0.1",
    install_requires=["requests"],
    entry_points={"console_scripts": ["mycli=my_package.cli:main"]},
)

Install locally:

pip install -e .

poetry (Modern Alternative)

poetry init  # Creates pyproject.toml
poetry add requests  # Adds dependency
poetry build  # Creates distributable package

---

6. Profiling & Performance Tuning

cProfile (Identify Bottlenecks)

python -m cProfile -s cumtime my_script.py

line_profiler (Line-by-Line Analysis)

# Install: pip install line_profiler
# Add @profile decorator to target function
@profile
def slow_function():
    total = 0
    for i in range(1000000):
        total += i
    return total

Run with:

kernprof -l -v script.py

---

Conclusion

Mastering these advanced topics unlocks high-performance, scalable, and maintainable Python code.