Django is inherently single-threaded when running under WSGI (Web Server Gateway Interface). This means that each request is handled sequentially in a single thread, which can become a bottleneck for applications handling real-time data, background tasks, or continuous monitoring*.*

However, Django provides several ways to manage parallel processing using processes and threads efficiently. In this guide, we will explore how to implement processes and threads in Django, particularly for handling PLC (Programmable Logic Controllers) and sensor-based systems.

🔹 Understanding Single-Threaded Django

Django processes requests sequentially using WSGI. This means:

• Each request is handled in a single worker process.

• Blocking operations (e.g., long-running scripts, waiting for sensor data) can freeze the server.

• Parallel execution is not possible within a single request-response cycle.

🛠 How Do We Overcome This?

To handle real-time tasks efficiently in Django, we use:

• Subprocesses (for running long-running external scripts).

• Threads (for background processing without blocking the main request cycle).

• Task Queues (Celery, Redis, etc.) for managing heavy computation asynchronously (recommended for production).

Implementing Processes in Django

Running External Scripts Using subprocess.Popen()

When handling real-time data from PLCs, we may need to run separate scripts that continuously generate or process data. Since Django is single-threaded, running such scripts within the request-response cycle would block the entire application.

Instead, we can use processes to handle these tasks separately:

import subprocess
def start_dummy_data(request, line_id):
    """Start the dummy data script for a given line."""
    process = subprocess.Popen(
        ["python", "manage.py", "generate_dummy_data", str(line_id)],
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE
    )
    return JsonResponse({"message": "Dummy data generation started."})

How it works:

• subprocess.Popen([...]) creates a new OS-level process that runs a Django management command.

• This script generates dummy measurement data for the specified line_id.

• The process is tracked using a dictionary (running_scripts), which stores the process reference.

What happens in the background:

• The process runs independently from Django’s main process.

• It executes manage.py with the appropriate mode's dummy data script.

• Output (stdout and stderr) is captured for debugging/logging purposes.

🛠 Why Use a Process Instead of a Thread?

• Processes run independently and do not block the Django application.

• They are better suited for CPU-heavy or long-running tasks.

• If one process crashes, it does not affect the main application.

🚨 Stopping a Running Process

We must ensure that long-running processes can be stopped when needed. To stop a process:

import os, signal
def stop_dummy_data(request, line_id):
    if line_id in running_scripts:
        process = running_scripts.pop(line_id, None)
        if process:
            process.terminate()  # Soft stop
            if process.poll() is None:
                os.kill(process.pid, signal.SIGKILL)  # Force stop if necessary
        return JsonResponse({"message": "Dummy data generation stopped."})
    return JsonResponse({"error": "No running script found!"}, status=400)

How this works:

• First, it checks if the process exists in running_scripts.

• It attempts to gracefully stop the process with process.terminate().

• If the process is still running, it forces termination using:

• Windows: taskkill /F /PID <PID>

• Linux/macOS: os.kill(<PID>, signal.SIGKILL)

Why use both terminate() and SIGKILL?

• terminate() sends a soft signal, allowing the script to clean up before exiting.

• If it does not respond, SIGKILL ensures immediate termination.

✅ This prevents zombie processes.
✅ Ensures the script stops reliably, even if it becomes unresponsive

Implementing Threads in Django

Running Background Tasks with threading.Thread()

In many real-time systems, we need to check sensor data and trigger alarms continuously. However, if this logic runs inside Django’s request cycle, it will block all other requests.

Instead, we use a background thread to monitor and process sensor data without blocking Django’s request handling:

import threading, time
def process_alarm():
    while True:
        check_and_trigger_alarm()  # Function that monitors sensor data
        time.sleep(1)  # Prevent high CPU usage
# Start background thread
threading.Thread(target=process_alarm, daemon=True).start()

🛠 Why Use a Thread Instead of a Process?

• Threads are lightweight and share memory with the main application.

• Best for I/O-bound tasks (e.g., monitoring sensor data, checking logs).

• Faster than processes when working with shared data.

However, Python’s Global Interpreter Lock (GIL) limits true parallelism in CPU-intensive tasks, making processes a better choice for heavy computation.

✅ Handling Real-Time Data Streaming in Django

3️⃣ Implementing SSE (StreamingHttpResponse)

When working with PLCs and sensors, we often need to stream real-time data to the frontend. Django supports Server-Sent Events (SSE), which allow continuous data streaming over a single HTTP connection.

from django.http import StreamingHttpResponse
import time, json

def event_stream():
    while True:
        data = fetch_latest_sensor_data()
        yield f"data: {json.dumps(data)}\n\n"
        time.sleep(2)  # Control update frequency

def sse_measurements(request):
    response = StreamingHttpResponse(event_stream(), content_type="text/event-stream")
    response["Cache-Control"] = "no-cache"
    return response

🛠 Why Use SSE Instead of Polling?

My requirements was just sending data from backend to frontend. frontend was not sending the data frequently hence i dropped the idea of using websockets which is full duplex and used sse instead.

• Efficient: Keeps a persistent connection without frequent HTTP requests.

• Low Latency: Sends updates instantly when new data is available.

• Scalable: Reduces server load compared to polling.

However, SSE is still synchronous in Django and may not scale well under heavy load. For large-scale real-time systems, consider WebSockets or an async framework like FastAPI.

🚀 Summary: Choosing Between Processes and Threads in Django

For long-running external scripts, processes (subprocess.Popen) are the best choice because they run independently and do not block Django’s main thread. These are particularly useful for handling data simulations, batch processing, or computationally intensive tasks.

On the other hand, threads (threading.Thread) are ideal for background tasks that need to run alongside Django without consuming excessive system resources. These are well-suited for monitoring and real-time alerts, where frequent updates are required but do not require intensive computation.

By understanding how to work around Django’s single-threaded nature, we can build efficient, real-time industrial applications without performance bottlenecks.

🔍 Final Takeaways

• Django is single-threaded under WSGI, which limits real-time processing.

• Use processes (subprocess.Popen) for independent scripts or heavy computation.

• Use threads (threading.Thread) for background tasks that run alongside Django.

• Use SSE (StreamingHttpResponse) for real-time data updates in Django.

By understanding how to work around Django’s single-threaded nature, we can build efficient, real-time industrial applications without performance bottlenecks.

Will keep updating the blogs as I learn more..