Day 2: Linux Shell Scripting & Automation

Initial Tasks:

✅ Install a Package: Install Git (sudo apt install git or sudo yum install git).
✅ Check Network Connectivity: Use ping google.com to verify internet connectivity.
✅ Fetch a Web Page: Run curl http://example.com to retrieve HTML content.
✅ Explore Processes: Use ps -ef to view running processes and find a specific process.
✅ Terminate a Process: Identify a process ID (PID) and kill it using kill -9 <PID>.
✅ Use top/htop: Monitor real-time CPU/memory usage.

LEARN:

Advanced Shell Scripting Knowledge: Click here to watch the Shell Scripting video by Sandip Das.

Challenge 1: Write a simple Bash script that prints “Hello DevOps” along with the current date and time

Goal: Learn how to print text and fetch the current date/time in Bash.

Answer:

#!/bin/bash
echo "Hello DevOps, the current date and time is: $(date)"

Explanation of the Script:

• echo → Prints the given message.

• $(date) → Fetches the current date and time.

---

Challenge 2: Create a script that checks if a website (e.g., https://www.learnxops.com) is reachable using curl or ping. Print a success or failure message.

Goal: Learn to check website availability using network commands.

Answer:

#!/bin/bash

URL="https://www.learnxops.com"
HOST="www.learnxops.com"

# Check website using curl
if curl -Is "$URL" --max-time 5 | grep "HTTP/"; then
    echo "✅ $URL is reachable (HTTP check passed)."
else
    echo "⚠️ HTTP check failed. Checking with ping..."

    # Check website using ping
    if ping -c 3 "$HOST" &> /dev/null; then
        echo "✅ $HOST is reachable via ping."
    else
        echo "❌ $HOST is unreachable via both HTTP and ping."
    fi
fi

Explanation of the Script:

1. URL="https://www.learnxops.com" & HOST="www.learnxops.com"

• Defines the website URL for curl and the hostname for ping.

2. curl -Is "$URL" --max-time 5 | grep "HTTP/"

• curl -Is → Sends a request and prints only the response headers.

• --max-time 5 → Limits the request timeout to 5 seconds.

• grep "HTTP/" → Checks if the response contains an HTTP status (e.g., 200 OK).

3. if condition (HTTP Check)

• If curl finds a valid HTTP response, prints:
  ✅ "$URL is reachable (HTTP check passed)."

• Otherwise, prints a warning and proceeds to ping.

4. ping -c 3 "$HOST" &> /dev/null

• ping -c 3 → Sends 3 packets to check reachability.

• &> /dev/null → Hides output, only showing success/failure.

5. Final if condition (Ping Check)

• If ping succeeds, prints:
  ✅ "$HOST is reachable via ping."

• If both curl and ping fail, prints:
  ❌ "$HOST is unreachable via both HTTP and ping."

---

Challenge 3: Write a script that takes a filename as an argument, checks if it exists, and prints the content of the file accordingly.

Goal: Work with file arguments and condition checking in Bash.

Answer:

#!/bin/bash

FILENAME="$1"

# Check if the file exists
if [ -f "$FILENAME" ]; then
    echo "📄 File '$FILENAME' exists. Here is its content:"
    cat "$FILENAME"
else
    echo "⚠️ File '$FILENAME' does not exist."
fi

Explanation:

• FILENAME="$1" → Stores the first argument as the filename.

• if [ -f "$FILENAME" ] → Checks if the file exists and is a regular file.

• cat "$FILENAME" → If the file exists, prints its content.

• Else condition → If the file does not exist, prints a warning.

---

Challenge 4: Create a script that lists all running processes and writes the output to a file named process_list.txt.

Goal: Learn to list processes and write output to a file.

Answer:

#!/bin/bash

OUTPUT_FILE="process_list.txt"
ps aux > "$OUTPUT_FILE"
echo "✅ Process list saved to $OUTPUT_FILE"

Explanation:

• OUTPUT_FILE="process_list.txt" → Defines the output file name.

• ps aux > "$OUTPUT_FILE" → Captures all running processes and writes them to the file.

• echo "✅ Process list saved to $OUTPUT_FILE" → Confirms successful execution.

---

Challenge 5: Write a script that installs multiple packages at once (e.g., git, vim, curl). The script should check if each package is already installed before attempting installation.

Goal: Automate package installation while avoiding redundant installs.

Answer:

#!/bin/bash

PACKAGES=("git" "vim" "curl")

for package in "${PACKAGES[@]}"; do
    if ! command -v "$package" &> /dev/null; then
        echo "📦 Installing $package..."
        sudo apt-get install -y "$package"
    else
        echo "✅ $package is already installed."
    fi
done

echo "Done!"

Explanation:

1. PACKAGES=("git" "vim" "curl")

• Defines a list of packages that need to be installed.

2. for package in "${PACKAGES[@]}"

• Loops through each package in the list.

3. command -v "$package" &> /dev/null

• command -v → Checks if the package is installed.

• &> /dev/null → Hides any output to keep it clean.

3. if ! command -v "$package" &> /dev/null; then

• If the package is not installed, proceed with installation.

4. echo "📦 Installing $package..."

• Prints a message to indicate the package installation.

5. sudo apt-get install -y "$package"

• sudo → Runs the command with administrative privileges.

• apt-get install -y → Installs the package without asking for confirmation.

7. else echo "✅ $package is already installed."

• If the package is already installed, prints a confirmation message.

8. echo " Done!"

• Displays a final message after completing the loop.

---

Challenge 6: Create a script that monitors CPU and memory usage every 5 seconds and logs the results to a file.

Goal: Learn how to capture system resource usage continuously.

Answer:

#!/bin/bash

LOG_FILE="system_monitor.log"

echo "Starting system monitoring... (Logs will be saved in $LOG_FILE)"
echo "Timestamp        | CPU (%) | Memory (%)" > "$LOG_FILE"

while true; do
    TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")

    CPU_USAGE=$(top -bn1 | grep "Cpu(s)" | awk '{print $2 + $4}')
    MEM_USAGE=$(free | awk '/Mem/ {printf "%.2f", $3/$2 * 100}')

    echo "$TIMESTAMP |  $CPU_USAGE   |   $MEM_USAGE" >> "$LOG_FILE"

    sleep 5
done

Explanation:

1. LOG_FILE="system_monitor.log"

• Defines the log file where system usage will be stored.

2. echo "Timestamp | CPU (%) | Memory (%)" > "$LOG_FILE"

• Writes the column headers to the log file.

3. while true; do

• Starts an infinite loop to continuously monitor system stats.

4. TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")

• Gets the current timestamp for logging.

5. CPU_USAGE=$(top -bn1 | grep "Cpu(s)" | awk '{print $2 + $4}')

• top -bn1 → Runs top once in batch mode.

• grep "Cpu(s)" → Extracts CPU usage details.

• awk '{print $2 + $4}' → Adds user and system CPU usage.

6. MEM_USAGE=$(free | awk '/Mem/ {printf "%.2f", $3/$2 * 100}')

• free → Gets memory usage details.

• awk '/Mem/ {printf "%.2f", $3/$2 * 100}' → Calculates used memory percentage.

7. echo "$TIMESTAMP | $CPU_USAGE | $MEM_USAGE" >> "$LOG_FILE"

• Logs CPU and memory usage with timestamp into the log file.

8. sleep 5

• Waits 5 seconds before running the next check.

---

Challenge 7: Write a script that automatically deletes log files older than 7 days from /var/log.

Goal: Learn to automate file cleanup.

Answer:

#!/bin/bash

#Better Version (If you want logging, flexibility, and automation) 
LOG_DIR="/var/log"
DAYS=7

find "$LOG_DIR" -type f -name "*.log" -mtime +$DAYS -exec rm -f {} \;
echo "Deleted log files older than $DAYS days from $LOG_DIR"

                 (   OR   )

#Simple version (If you need a quick cleanup)
find /var/log -type f -mtime +7 -exec rm -f {} \;

Explanation:

1. LOG_DIR="/var/log" → Specifies the directory where logs are stored.

2. DAYS=7 → Defines the age limit for deletion (logs older than 7 days will be removed).

3. find "$LOG_DIR" -type f -name "*.log" -mtime +$DAYS -exec rm -f {} \;

   • find "$LOG_DIR" → Searches in /var/log.

   • -type f → Only looks for files (not directories).

   • -name "*.log" → Targets log files ending with .log.

   • -mtime +$DAYS → Selects files older than 7 days.

   • -exec rm -f {} \; → Deletes each matched file.

4. echo "Deleted log files older than $DAYS days from $LOG_DIR"

• Prints a confirmation message after deletion.

---

Challenge 8: Automate user account creation – Write a script that takes the username as an argument, checks, if the user exists, gives the message “user already exists“ else creates a new user, adds it to a “devops“ group, and sets up a default home directory.

Goal: Automate user management in Linux.

Answer:

#!/bin/bash

if [ -z "$1" ]; then
    echo "❌ Usage: $0 <username>"
    exit 1
fi

USERNAME="$1"
GROUP="devops"

if id "$USERNAME" &>/dev/null; then
    echo "✅ User '$USERNAME' already exists."
else
    sudo useradd -m -s /bin/bash -G "$GROUP" "$USERNAME"

    if id "$USERNAME" &>/dev/null; then
        echo "User '$USERNAME' created successfully and added to the '$GROUP' group."
    else
        echo "❌ Failed to create user '$USERNAME'."
    fi
fi

Explanation:

1. if [ -z "$1" ]; then

• Checks if a username argument is provided.

• If not, prints usage instructions and exits.

2. USERNAME="$1" & GROUP="devops"

• Stores the username from the command-line argument.

• Sets the default group to devops.

3. if id "$USERNAME" &>/dev/null; then

• id "$USERNAME" → Checks if the user already exists.

• &>/dev/null → Suppresses the output.

• If the user exists, it prints a message and exits.

4. sudo useradd -m -s /bin/bash -G "$GROUP" "$USERNAME"

• sudo useradd → Runs with admin privileges to create a user.

• -m → Creates a home directory (/home/username).

• -s /bin/bash → Sets Bash as the default shell.

• -G "$GROUP" → Adds the user to the devops group.

5. if id "$USERNAME" &>/dev/null; then

• Checks if the user was created successfully and prints a confirmation message.

---

Challenge 9: Use awk or sed in a script to process a log file and extract only error messages.

Goal: Learn to filter and process text using awk and sed.

Answer:

#!/bin/bash

awk 'tolower($0) ~ /error|failed|critical/' /var/log/syslog > error_logs.txt

Explanation:

• awk ' tolower($0) ~ /error|failed|critical/' → Converts each line to lowercase and searches for "error", "failed", or "critical" (case-insensitive).

• > error_logs.txt → Saves output to a file.

---

Challenge 10: Set up a cron job that runs a script to back up (zip/tar) a directory daily.

Goal: Automate backups using cron jobs.

Answer:

crontab -e #run directly on terminal

Add the following line:

#run directly on terminal
0 2 * * * tar -czf /backup/home_backup_$(date +\%Y-\%m-\%d).tar.gz /home/user

Explanation:

1. 0 2 * * * → Schedule the Cron Job

• 0 → Minute (0th minute)

• 2 → Hour (2 AM)

• * * * → Runs daily (every day, every month, every weekday)

2. tar -czf → Create a Compressed Backup

• tar → Archive utility to bundle files

• -c → Create a new archive

• -z → Compress using gzip (.tar.gz format)

• -f → Specify the output file

3. /backup/home_backup_$(date +\%Y-\%m-\%d).tar.gz → Filename with Date

• /backup/home_backup_ → Backup location and file prefix

• $(date +\%Y-\%m-\%d) → Generates a date (YYYY-MM-DD)

• Example: /backup/home_backup_2025-02-27.tar.gz

4. /home/user → Directory to Backup

• Specifies the directory /home/user to be backed up

---

💡Bonus Challenge: Customize your Bash prompt to display the current user and working directory. (Hint: export PS1="\u@\h:\w$ "), try to make it permanent, so terminal closing and opening don’t default!

Goal: Personalize and persist Bash prompt settings.

Commands Used:

#Open the ~/.bashrc file using vi 
vi ~/.bashrc

#Add this line in bashrc file
export PS1="\u@\h:\w\$ "

#save and exit from the editor and apply the changes
source ~/.bashrc

Explanation:

• \u@\h:\w → Displays username, hostname, and working directory.

• $ (for normal users) or # (for root)

---

📚 Topics Covered Today

✅ Basic Shell Commands – Navigating directories (cd), listing files (ls), viewing and creating files (cat, touch).
✅ Filesystem Hierarchy – Understanding Linux directory structures (/etc, /var, /home, /bin, /usr).
✅ File Permissions & Ownership – Using chmod, chown, ls -l to manage read, write, execute (rwx).
✅ User & Group Management – Creating users (useradd), modifying groups (usermod, groupadd, passwd).
✅ File & Directory Operations – Creating, modifying, moving, and searching files (find, du, ls, awk, sort, grep).
✅ Process & System Monitoring – Checking running processes (ps), CPU/memory usage (top, free, htop), killing processes (kill, pkill).
✅ Automation & Scheduling – Setting up cron jobs (crontab -e), writing simple Bash scripts.
✅ Networking Basics – Checking connectivity (ping, curl, wget).
✅ Package Management – Installing, updating, and removing software (apt, yum, dnf, pacman).
✅ Disk Management – Checking disk usage (df -h, du -sh).
✅ Log File Analysis – Viewing system logs (tail -f /var/log/syslog), extracting errors (grep 'error' /var/log/syslog).