Should You Always Use Daemon Mode?

mod_wsgi is a great tool for deploying Python web applications with Apache. While it offers two modes embedded and daemon, understanding how they work is essential before deciding which one to use. Spoiler alert: daemon mode is the clear winner for most use cases. In this article, we’ll explain both modes, how they operate, and why daemon mode is your best bet for scalable and reliable deployments.

Note sure what WSGI or mod_WSGI mean? check out Understanding WSGI and mod_wsgi

Embedded Mode

In embedded mode, mod_wsgi runs your Python application directly within Apache’s worker processes. Essentially, the application becomes part of Apache itself, sharing resources and processes.

How Embedded Mode Works

• Apache receives an HTTP request.

• The worker process handling the request also loads and executes your Python application code.

• The response is returned to the client.

Client -> Apache Worker (Runs Python Application) -> Response to Client

• Applications run within Apache’s processes.

• All applications share the same memory space.

Daemon Mode

In daemon mode, mod_wsgi spawns dedicated processes to run your Python application. These processes are isolated from Apache’s main worker processes, providing better control and flexibility.

How Daemon Mode Works

• Apache receives an HTTP request.

• The request is passed to a separate process managed by mod_wsgi.

• The Python application processes the request and returns the response to Apache, which then delivers it to the client.

Client -> Apache -> mod_wsgi Daemon Process (Runs Python Application) -> Response to Client

• Applications run in isolated processes.

• Each application can have its own environment, dependencies, and configuration.

Why You Should Always Use Daemon Mode

The Downsides of Embedded Mode

• First, Apache handles both HTTP requests and runs the Python application, leading to resource bottlenecks.

• Since all Apache workers share the same memory, scaling up for traffic spikes becomes challenging. However, I'm not saying it will not scale but compared with Daemon mode you can scale way better and use fewer resources

• Also since the entire app deployed and managed by Apache, any error in your Python application can bring down the entire server.

The Advantages of Daemon Mode

• (Isolated Environments) Since each application runs in its own process, avoiding conflicts and improving stability.

• Because You can control the number of processes and threads independently of Apache this means better Scalability.

• Crashes or errors in the application won’t affect Apache’s ability to handle HTTP requests.

• Starting with daemon mode ensures you’re ready to scale when needed.

As Graham Dumpleton, the creator of mod_wsgi, puts it: “Friends don’t let friends use embedded mode.” Even for small applications, daemon mode sets you up for success.

Configuring mod_wsgi for Daemon Mode

Here’s how you can set up daemon mode for your application:

<VirtualHost *:80>
    ServerName example.com

    WSGIRestrictEmbedded On
    WSGIDaemonProcess myapp user=www-data processes=2 threads=25 python-home=/ .../env
    WSGIProcessGroup myapp
    WSGIScriptAlias / /path/to/app.wsgi

    <Directory /path/to/>
        Require all granted
    </Directory>
</VirtualHost>

• I will explain more about this configuration and talk about the best way to set up and tweak Apache and mod_wsgi, but I will leave this for another article

Quick tips for you:

• Always keep an eye on CPU and Memory utilization to adjust the number of processes and threads (More about this in another article)

• For CPU-intensive tasks, reduce the thread count. For I/O-heavy tasks, increase the number of threads.

• Always use a virtual environment to isolate dependencies to avoid conflicts

That's all for now. If you're interested in more advanced content about WSGI, Apache, or other DevOps topics, there are many articles in the DevOps series that you might find interesting.

Mostafa-DE Fayyad

Software Engineer