Using Redis Server for Shell Scripting in FreeBSD System Administration

The more I use Redis server, the more I enjoy it.
During the last year, I saw a lot of good system administrators using HashiCorp Consul to store information for their shell scripts.
As I am concerned, I am not a big fan of Consul because it can be a real gasworks to deploy and configure.
I prefer to use Redis server on my workstation to store configuration information.

Introduction

In the world of FreeBSD system administration, automation and scripting are essential tools for efficiently managing and maintaining systems.
Many administrators have adopted HashiCorp Consul for managing key-value pairs and coordinating their scripts, but Redis offers a simpler and more lightweight alternative.
Redis is a powerful in-memory data structure store that can easily be used for storing configuration data, managing locks, and tracking script execution status.
In this article, I will demonstrate how you can use Redis in conjunction with shell scripting to manage backup tasks on your FreeBSD system.

Why Redis Over Consul?

HashiCorp Consul is indeed a robust tool, especially when you need a full-fledged service discovery and configuration management system.
However, its complexity can be overkill for smaller tasks or personal use cases.
Redis, on the other hand, is much simpler to set up and use, making it a great alternative when your needs are more straightforward, such as storing key-value pairs for shell scripts.
Redis excels in scenarios where you need quick access to configuration data or need to manage state information for your scripts.
Its simplicity, speed, and versatility makes it an excellent choice for system administrators looking for a lightweight solution.

Adding the List of Users to Redis

Before running the backup script, we need to populate Redis with the list of users whose home directories need to be backed up.
This can be done easily using Redis CLI commands.
Here's how you can add users to the backup_users set in Redis:

Populating the User List in Redis

To add users to the Redis set, you can use the sadd command, which adds one or more members to a set stored at a given key.
In our case, the key is backup_users. Here’s how you can do it:

# Add individual users
redis-cli sadd backup_users myuser1
redis-cli sadd backup_users myuser2
redis-cli sadd backup_users myuser3

# Verify the users were added
redis-cli smembers backup_users

The sadd command adds user1, user2, and user3 to the backup_users set.
You can add as many users as you need by repeating the command with different usernames.

Managing the User List

You can manage the list of users directly from the Redis CLI or using scripts. To remove a user from the backup list, use the srem command:

redis-cli srem backup_users myuser3

To clear the entire list of users:

redis-cli del backup_users

These commands give you full control over the list of users to be backed up, allowing you to dynamically manage who gets backed up based on your needs.

Shell Scripting with Redis: A Practical Example

Let's walk through a practical example where we use Redis to manage a shell script that backs up user directories on a FreeBSD system.

The Scenario

We want to create a script that performs the following tasks:

1. Fetch a list of users whose home directories need to be backed up.

2. Back up each user’s home directory to a tarball (TGZ file).

3. Use a lock mechanism to ensure that the script doesn't run multiple times concurrently.

4. Store the status of the last backup (success or failure) in Redis.

Setting Up Redis on FreeBSD

First, you need to have Redis installed and running on your FreeBSD system.
You can install Redis using the FreeBSD package management system (pkg):

sudo pkg install redis

Ensure Redis starts on boot:

sudo sysrc redis_enable=yes

Start the Redis server:

sudo service redis start

The Shell Script

Here’s the shell script that achieves our goal:

#!/bin/sh

# Variables
REDIS_CLI="/usr/local/bin/redis-cli"
LOCK_KEY="backup_lock"
USERS_KEY="backup_users"
BACKUP_STATUS_KEY="backup_status"
BACKUP_DIR="/backup"

# Check if another instance of the script is running
lock_status=$($REDIS_CLI get $LOCK_KEY)
if [ "$lock_status" = "locked" ]; then
    echo "Backup script is already running. Exiting."
    exit 1
fi

# Set the lock
$REDIS_CLI set $LOCK_KEY "locked"

# Fetch the list of users from Redis
users=$($REDIS_CLI smembers $USERS_KEY)

# Perform backup
for user in $users; do
    user_home="/home/$user"
    if [ -d "$user_home" ]; then
        tar -czf $BACKUP_DIR/${user}_backup.tgz -C $user_home .
        if [ $? -eq 0 ]; then
            $REDIS_CLI hset $BACKUP_STATUS_KEY $user "success"
            echo "Backup for $user completed successfully."
        else
            $REDIS_CLI hset $BACKUP_STATUS_KEY $user "failure"
            echo "Backup for $user failed."
        fi
    else
        echo "User $user does not have a home directory. Skipping."
    fi
done

# Remove the lock
$REDIS_CLI del $LOCK_KEY

exit 0

Explanation of the Script

• Lock Mechanism: The script first checks if the backup_lock key in Redis is set to "locked".
  If it is, this means the script is already running, so it exits to prevent multiple instances from running simultaneously.
  If not, it sets the backup_lock key to "locked" to prevent other instances from starting.

• User List: The script fetches a list of users to back up from Redis using the smembers command, which retrieves all members of a Redis set stored under the backup_users key.

• Backup Process: For each user, the script checks if the user's home directory exists. If it does, it creates a tarball (tgz file) of the directory.
  The backup is stored in the /backup directory with the format {user}_backup.tgz.

• Backup Status: After each backup, the script updates the backup_status key in Redis with the result of the backup (either "success" or "failure").

• Lock Removal: Finally, the script removes the lock key in Redis, allowing future executions of the script.

Conclusion

Redis provides a lightweight and flexible solution for storing configuration data and managing state information in shell scripts.
While HashiCorp Consul is a powerful tool, Redis is often more than sufficient for many system administration tasks.
In this example, we used Redis to manage a backup script, demonstrating how it can be effectively employed to prevent race conditions, store configuration data, and track script execution status.
By integrating Redis into your shell scripts, you can simplify your system administration tasks and avoid the overhead of more complex tools like Consul. Redis's speed, simplicity, and ease of use makes it a valuable addition to any FreeBSD administrator’s toolkit.