The Gevent Pool: 5 Lessons Learned

If you run Celery tasks to perform API requests or other input/output operations and you want to increase throughput, then this is for you.

Greenlets

Gevent is a Python library that provides lightweight, cooperative concurrency via greenlets. Greenlets are the same thing as coroutines in the asyncio world: while one task waits for its result, it yields to another task to do its thing.

Both gevent and asyncio use an event loop for coordinating the execution of tasks. The difference between greenlet and asyncio is that greenlets do the switching between tasks magically, whereas asyncio makes you await explicitly.

In the gevent world, whenever your programme is blocked on a network request, it will automatically switch to another greenlet. This increases throughput for all input/output bound tasks, compared to the default prefork pool.

Monkey Patcher

This magic works because gevent monkey-patches Python’s socket library. Gevent replaces attributes of the standard library's modules with its own modules. This allows you to write traditional synchronous code without noticing how gevent takes over under the hood.

Getting Started With Gevent

Coming from prefork and getting started with gevent is more than just plug-and-play. Are you in the business of giving the gevent pool a try? Here are my top 5 lessons learned.

1. Monkey patching

The very first thing gevent needs to do in the lifecycle of a programme is to patch parts of the standard library with gevent-friendly functions. Celery takes care of this. Do not monkey-patch. If something goes wrong, Celery not monkey-patching is never the issue. It's a red herring, contrary to what people say on StackOverflow.

2. 3rd party SDKs and API wrappers

Third-party libraries are the single biggest cause for gevent related headaches. Some libraries don't play nicely with the patched socket library. The more third-party packages you rely on the more time you will spend on this. Package dependency hell is a thing.

3. Exceptions

When things do go wrong, be mindful of the exceptions you see. They can be misleading and point you towards components that are not at fault. If you come across any of the following messages in your stack trace, chances are high that it is not your ORM but a third-party SDK (see point 2):

• Module 'select' has no attribute 'epoll'

• SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async

• You cannot use AsyncToSync in the same thread as an async event loop

4. Things might work 1,000 times and then fail

This is the worst one and is difficult to reproduce. A task might run 1,000 times and then fail with a stack trace that doesn't make sense (see point 3). A good starting point is point 2.

5. 3rd party SDKs and API wrappers

I just want to reiterate points 2-4. In 9 out of 10 cases, it is a third-party API wrapper.