Supercharge Your Process Management Using Supervisor

If you are using a VPS (Virtual Private Server), you might need to run many small programs continuously, such as shell or python scripts to automate some boring tasks asynchronously. Usually, external packages come with a unit file that allows an init system like systemd to manage them, or they come as docker images that a container engine can handle. However, if the software is not packaged correctly, or if users do not want to interact with a low-level init system on their server, a lightweight alternative can be helpful and then the supervisor came into the picture.

Supervisor is a tool for managing and monitoring multiple long-running programs through a single interface. Let's see how to install Supervisor on a Linux server and learn how to manage Supervisor configurations.

Let's execute the following command to fetch the latest version of the packages and automatically upgrade the outdated packages.

sudo apt update && sudo apt upgrade

To install Supervisor:

sudo apt install supervisor

Once you've installed Supervisor, its service will automatically run. You can verify that it's running by executing the specified command:

sudo systemctl status supervisor

Output:

Output
● supervisor.service - Supervisor process control system for UNIX
     Loaded: loaded (/lib/systemd/system/supervisor.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2021-11-17 22:56:48 UTC; 5min ago

After successfully installing Supervisor, the next step is to add our desired programs to it. A best practice for working with Supervisor is to write a configuration file for every program it will handle.

Let's add a celery worker as our desired program. Firstly we have to create a particular config file for this task.

sudo touch /etc/supervisor/conf.d/backend_celery.conf

Now we have to add the following lines to that backend_celery.conf file:

[program:backend_celery]
command=/home/ubuntu/backend/django-backend/venv/bin/celery --app=your_project.celery:app worker -l info
directory=/home/ubuntu/backend/django-v3-backend
numprocs=1
stderr_logfile=/var/log/idle.err.log
stdout_logfile=/var/log/idle.out.log
autostart=true
autorestart=true
startsecs=10

Here autostart=true & autorestart=true define the automatic behavior of the script under certain conditions.

The autostart the option tells Supervisor that this program should be started when the system boots. Setting this to false will require a manual start following any system shutdown.

autorestart defines how Supervisor should manage the program in the event that it exits:

1. false tells Supervisor not to ever restart the program after it exits.

2. true tells Supervisor to always restart the program after it exits.

3. unexpected tells Supervisor to only restart the program if it exits with an unexpected error code (by default anything other than codes 0 or 2). To learn more about error codes, look into the errno command.

stderr_logfile=/var/log/idle.err.log
stdout_logfile=/var/log/idle.out.log

After creating and saving our configuration file, we need to let the Supervisor know about the new program by using the supervisorctl command. We start by asking Supervisor to check if there are any new or updated program configurations in the /etc/supervisor/conf.d directory. This is done using the reread command. If there are any changes, Supervisor will update its configuration accordingly. Then we use the update command to apply the changes and start any new programs or restart any existing programs with the updated configuration.

sudo supervisorctl reread

After running the reread command, you can then use the update command to apply the changes:

sudo supervisorctl update

Now, We can check its output by looking at the output log file:

sudo tail /var/log/idle.out.log

Output
Sat Nov 21 22:21:22 UTC 2021
Sat Nov 21 22:21:23 UTC 2021
Sat Nov 21 22:21:24 UTC 2021
Sat Nov 21 22:21:25 UTC 2021

Let's see the interactive mode of running supervisor with supervisorctl with no arguments:

sudo supervisorctl

Output:

backend_celery                 RUNNING   pid 5932, uptime 0:01:06
supervisor>

It looks awesome that our backend_celery program is working with a `supervisor`. As previously stated, Supervisor is considered lightweight compared to other tools available today, but it is still regularly maintained and can be a valuable asset for smaller-scale deployments. It is an easy-to-use and self-contained method of generating logs as part of a larger deployment. Additionally, the Supervisor requires minimal maintenance, making it an efficient option for managing processes in your application.