Log Analyzer and Report Generator

Rajat ChauhanRajat Chauhan
2 min read

Write a Bash script that automates the process of analyzing log files and generating a daily summary report. The script should perform the following steps:

  1. Input: The script should take the path to the log file as a command-line argument.

  2. Error Count: Analyze the log file and count the number of error messages. An error message can be identified by a specific keyword (e.g., "ERROR" or "Failed"). Print the total error count.

  3. Critical Events: Search for lines containing the keyword "CRITICAL" and print those lines along with the line number.

  4. Top Error Messages: Identify the top 5 most common error messages and display them along with their occurrence count.

  5. Summary Report: Generate a summary report in a separate text file. The report should include:

    • Date of analysis

    • Log file name

    • Total lines processed

    • Total error count

    • Top 5 error messages with their occurrence count

    • List of critical events with line numbers

#!/bin/bash

# Function to display usage information
usage() {
        echo "Usage: $0 /home/ubuntu/logs/days.log"
        exit 1
}

if [ $# -ne 1 ]; then
        usage
fi

log_file=$1

# Check if the log file is a valid regular file
if [ ! -f "$log_file" ]; then
        echo "Error: log file $log_file does not exist."
        exit 1
fi

# Variables defined
error_keyword="ERROR"
critical_keyword="CRITICAL"
date=$(date +"%Y-%m-%d")
summary_report="summary_report_$date.txt"
archive_dir="processed_logs"
total_lines=$(wc -l < "$log_file")

# Create summary report
{
        echo "Date of analysis: $date"
        echo "Log file name: $log_file"
        echo "Total lines in the log file: $total_lines"
} > "$summary_report"

# Total error count
error_count=$(grep -c "$error_keyword" "$log_file")
echo "Total error count: $error_count" >> "$summary_report"

# Get the list of critical events with line number
critical_events=$(grep -n "$critical_keyword" "$log_file")
echo "List of critical events with line number:" >> "$summary_report"
echo "$critical_events" >> "$summary_report"

# Extract error messages, count occurrences, sort by frequency, and display the top 5
top_five=$(grep -i "$error_keyword" "$log_file" | awk '{print $1,$2,$3,$4,$5}' | sort | uniq -c | sort -nr | head -n 5)
echo "Top 5 error messages with their occurrence count:" >> "$summary_report"
echo "$top_five" >> "$summary_report"

#grep -i Searches for lines containing the word error (case insensitive) in the log file.
#sort: Sorts the lines to prepare them for counting unique occurrences
#uniq -c: Counts the unique occurrences of each line
#sort -nr: Sorts the counted occurrences in numeric order (descending).
#head -n 5: Displays the top 5 lines

Output:

0
Subscribe to my newsletter

Read articles from Rajat Chauhan directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

Rajat Chauhan
Rajat Chauhan

Rajat Chauhan is a skilled Devops Engineer, having experience in automating, configuring, deploying releasing and monitoring the applications on cloud environment. • Good experience in areas of DevOps, CI/CD Pipeline, Build and Release management, Hashicorp Terraform, Containerization, AWS, and Linux/Unix Administration. • As a DevOps Engineer, my objective is to strengthen the company’s applications and system features, configure servers and maintain networks to reinforce the company’s technical performance. • Ensure that environment is performing at its optimum level, manage system backups and provide infrastructure support. • Experience working on various DevOps technologies/ tools like GIT, GitHub Actions, Gitlab, Terraform, Ansible, Docker, Kubernetes, Helm, Jenkins, Prometheus and Grafana, and AWS EKS, DevOps, Jenkins. • Positive attitude, strong work ethic, and ability to work in a highly collaborative team environment. • Self-starter, Fast learner, and a Team player with strong interpersonal skills • Developed shell scripts (Bash) for automating day-to-day maintenance tasks on top of that have good python scripting skills. • Proficient in communication and project management with good experience in resolving issues.