Volume Management, Process Monitoring, and Automating Backups in Linux

In this blog, we will dive into three crucial aspects of Linux system administration: Volume Management & Disk Usage, Process Management & Monitoring, and Automating Backups with Shell Scripting. These are fundamental skills for any DevOps engineer or system administrator, as they help ensure that your system remains efficient, secure, and well-managed. Let’s break down each topic step by step.

Volume Management & Disk Usage: Ensuring Your Data Is Safe

In the world of system administration, managing disk space and volumes is one of the most essential tasks. A volume represents a disk or a partition on your system, and mounting it correctly is the key to managing your data efficiently.

Creating a Directory and Mounting a Volume

Let’s start by creating a directory and mounting a new volume on your system.

First, we need to create a directory to mount the new volume. This is done with the following command:

sudo mkdir -p /mnt/devops_data

This command creates a directory /mnt/devops_data, where we will mount the volume.

Next, we mount the new volume to the directory:

• If you have a physical volume, you can use the following:

  sudo mount /dev/sdb1 /mnt/devops_data

  Here, /dev/sdb1 refers to the device name of the new volume.

• Alternatively, if you are working locally, you can use:

  1. sudo dd if=/dev/zero of=/mnt/devops_data.img bs=1M count=100

dd: uility for low-level copying and conversion of raw data.

if=/dev/zero: specifies input file (if stands for "input file")

of=/mnt/devops_data.img: specifies the output file (of stands for "output file").

bs=1M: specifies the block size (bs stands for "block size"). I

count=100: Limits the number of blocks to write.

2. Create file system on that file.

   sudo mkfs.ext4 /mnt/devops_data.img

   

3. Mount the file as a loop device:

   sudo mount -o loop /mnt/devops_data.img /mnt/devops_data

   

4. Use df –h to verify the mount.

5. We can also use mount | grep devops_data.

   

Process Management & Monitoring: Ensuring Smooth Operation

Running processes are at the heart of any operating system. In Linux, managing and monitoring these processes is crucial for keeping everything running smoothly.

Running a Background Process and Monitoring It

Let’s start by running a background process. For this example, we’ll use the ping command to continuously ping Google's servers and log the results:

ping google.com > ping_test.log &

Here, the & at the end sends the command to the background, allowing you to continue using the terminal for other tasks.

Once the command is running in the background, you can monitor the process. A few commands you can use include:

• Using ps to list processes:

  ps aux | grep ping

This command shows you the details of the running ping process, including its process ID (PID).

• Using top or htop to monitor resources:

  • The top command shows system activity and processes in real-time.

• If you prefer a more interactive, user-friendly interface, use htop (which may need to be installed).

• 

Killing a Process

At some point, you may need to stop a process. To do this, identify the PID of the process from the ps or top output, then use the following command to kill it:

kill <PID>

Make sure to replace <PID> with the actual process ID of the ping command.

To verify that the process is gone, simply run:

ps aux | grep ping

You should see no results related to the ping process if it has been successfully terminated.

Automating Backups with Shell Scripting: Streamlining Your Backup Process

Backups are critical to any infrastructure, and automating the backup process ensures that you are always protected without requiring manual intervention. Let’s create a shell script that will back up a directory and automate it using cron.

Writing a Backup Script

Let’s say we want to back up a directory called /devops_workspace and store it in /backups. We want to name the backup with the current date, so the filename will be in the format backup_YYYY-MM-DD.tar.gz. The steps are given below:

1. Create a new script file:

   vim /path/to/backup.sh

   

2. Add the following content to the script:

   #!/bin/bash

   #Define the source and backup directory

   SOURCE_DIR="/devops_workspace"

   BACKUP_DIR="/backups"

   DATE=$(date +%F)

   BACKUP_FILE=”${BACKUP_DIR}/BACKUP_${DATE}.tar.gz”

   # Create the backup

   tar -czf “$BACKUP_FILE” -C “$SOURCE_DIR” .

   # Check if the backup was successful and display a message

   if [ $? -eq 0 ]; then

   echo -e "\e[32mBackup successful: $BACKUP_FILE\e[0m"

   else

   echo -e “\e[31mBackup failed.\e[0m“

   fi

   

3. Save the script and make it executable:

   chmod +x /path/to/backup.sh

   

This script uses tar to create a compressed .tar.gz file of the /devops_workspace directory and saves it in the /backups directory. After creating the backup, the script outputs a success message in green text using echo -e.

Scheduling the Script

Now that we have a backup script, let's automate its execution. We can use cron to schedule this script to run at specific intervals.

1. Open the crontab file for editing:

   crontab -e

   

2. Add a line to schedule the backup to run daily at 2AM:

   0 2 * * * /path/to/backup.sh

   

   This cron job will automatically run the backup script every day at 2AM, ensuring that your /devops_workspace directory is backed up regularly.

Conclusion

Mastering volume management, process monitoring, and automation are key components of effective system administration and DevOps practices. In this blog, we've learned how to mount a volume, monitor processes, and automate backups using shell scripting. These skills will make your system more resilient, help prevent data loss, and improve overall operational efficiency.