Advanced Python: Pushing the Boundaries of the Language


Introduction
Python is much more than just a beginner-friendly language. When mastered at an advanced level, it can handle high-performance computing, concurrent systems, and elegant metaprogramming.
In this blog, we’ll explore:
✔ Decorators & Metaclasses (Modifying functions & classes at runtime)
✔ Multi-threading vs Multiprocessing (Concurrency & parallelism)
✔ Async Programming (asyncio
, aiohttp
) (Non-blocking I/O)
✔ Custom Exceptions (Better error handling)
✔ Packaging (setuptools
, poetry
) (Building distributable packages)
✔ Profiling & Performance Tuning (cProfile
, line_profiler
)
Let’s unlock Python’s full potential! 🚀
1. Decorators & Metaclasses
Decorators (Modify Functions Dynamically)
def log_time(func):
def wrapper(*args, **kwargs):
import time
start = time.time()
result = func(*args, **kwargs)
print(f"{func.__name__} took {time.time() - start:.2f}s")
return result
return wrapper
@log_time
def slow_function():
time.sleep(1)
slow_function() # Prints "slow_function took 1.00s"
Metaclasses (Modify Class Creation)
class SingletonMeta(type):
_instances = {}
def __call__(cls, *args, **kwargs):
if cls not in cls._instances:
cls._instances[cls] = super().__call__(*args, **kwargs)
return cls._instances[cls]
class Database(metaclass=SingletonMeta):
pass
db1 = Database()
db2 = Database()
print(db1 is db2) # True (only one instance exists)
Use Cases:
Decorators: Logging, caching (
@functools.lru_cache
), access controlMetaclasses: ORMs (like Django models), enforcing patterns (Singleton)
2. Multi-threading vs Multiprocessing
Feature | Multi-threading | Multiprocessing |
Best for | I/O-bound tasks | CPU-bound tasks |
GIL Impact | Limited by GIL | Bypasses GIL |
Memory | Shared memory | Separate memory |
Multi-threading (I/O Tasks)
import threading
def fetch_url(url):
print(f"Fetching {url}...")
threads = []
for url in ["url1", "url2"]:
t = threading.Thread(target=fetch_url, args=(url,))
t.start()
threads.append(t)
for t in threads:
t.join() # Wait for all threads
Multiprocessing (CPU Tasks)
from multiprocessing import Pool
def compute_square(n):
return n * n
with Pool(4) as p: # 4 processes
print(p.map(compute_square, [1, 2, 3])) # [1, 4, 9]
3. Async Programming (asyncio
, aiohttp
)
Basic asyncio
Example
import asyncio
async def fetch_data():
print("Start fetching...")
await asyncio.sleep(2) # Non-blocking sleep
print("Done fetching!")
return {"data": 1}
async def main():
task = asyncio.create_task(fetch_data())
print("Do other work...")
data = await task
print(data)
asyncio.run(main())
aiohttp
(Async HTTP Requests)
import aiohttp
async def fetch(url):
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
return await response.text()
4. Custom Exceptions (Better Error Handling)
class APIError(Exception):
"""Base error for API failures"""
def __init__(self, message, status_code=500):
super().__init__(message)
self.status_code = status_code
class NotFoundError(APIError):
"""Raised when a resource doesn't exist"""
def __init__(self, resource):
super().__init__(f"{resource} not found", 404)
# Usage
try:
raise NotFoundError("User")
except APIError as e:
print(f"Error {e.status_code}: {e}")
5. Packaging (setuptools
, poetry
)
setuptools
(Traditional)
# setup.py
from setuptools import setup
setup(
name="my_package",
version="0.1",
install_requires=["requests"],
entry_points={"console_scripts": ["mycli=my_package.cli:main"]},
)
Install locally:
pip install -e .
poetry
(Modern Alternative)
poetry init # Creates pyproject.toml
poetry add requests # Adds dependency
poetry build # Creates distributable package
6. Profiling & Performance Tuning
cProfile
(Identify Bottlenecks)
python -m cProfile -s cumtime my_script.py
line_profiler
(Line-by-Line Analysis)
# Install: pip install line_profiler
# Add @profile decorator to target function
@profile
def slow_function():
total = 0
for i in range(1000000):
total += i
return total
Run with:
kernprof -l -v script.py
Conclusion
Mastering these advanced topics unlocks high-performance, scalable, and maintainable Python code.
Subscribe to my newsletter
Read articles from Kolluru Pawan directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by
