Day 10 Task: Log Analyzer and Report Generator

Scenario

You are a system administrator responsible for managing a network of servers. Every day, a log file is generated on each server containing important system events and error messages. As part of your daily tasks, you need to analyze these log files, identify specific events, and generate a summary report.

Task

Write a Bash script that automates the process of analyzing log files and generating a daily summary report. The script should perform the following steps:

1. Input: The script should take the path to the log file as a command-line argument.

2. Error Count: Analyze the log file and count the number of error messages. An error message can be identified by a specific keyword (e.g., "ERROR" or "Failed"). Print the total error count.

3. Critical Events: Search for lines containing the keyword "CRITICAL" and print those lines along with the line number.

4. Top Error Messages: Identify the top 5 most common error messages and display them along with their occurrence count.

5. Summary Report: Generate a summary report in a separate text file. The report should include:

   • Date of analysis

   • Log file name

   • Total lines processed

   • Total error count

   • Top 5 error messages with their occurrence count

   • List of critical events with line numbers

Sample Log File

A sample log file named sample_log.log has been provided in the same directory as this challenge file. You can use this file to test your script or use this

Script : log_analyzer.sh

#!/bin/bash

# Here we are showing how to use log file
if [ -z "$1" ]; then
  echo "Usage: $0 <path_to_log_file>"
  exit 1
fi

# Declare variable for first argument
log_file=$1
report_file="summary_report_$(date +%Y%m%d).txt" # Create file using custom timestamp

# Check if log file is exist or not
if [ ! -f "$log_file" ]; then
  echo "Log file not found!"
  exit 1
fi

# Counting lines in log file
total_lines=$(wc -l < "$log_file")
# Counting number of lines containing words ERROR & Failed
error_count=$(grep -cE "ERROR|Failed" "$log_file")
# Finds line include CRITICAL and line number using -n
critical_events=$(grep -n "CRITICAL" "$log_file")

# Declare an associative array to store error_messages
declare -A error_messages

# Read log file line by line
while IFS= read -r line; do
  if [[ "$line" =~ ERROR|Failed ]]; then
    error_message=$(echo "$line" | awk -F']' '{print $NF}')
    ((error_messages["$error_message"]++))
  fi
done < "$log_file"

# Sorting of errors
top_errors=$(for message in "${!error_messages[@]}"; do
  echo "${error_messages[$message]} $message"
done | sort -rn | head -n 5)

# Printing final outout data
echo "Date of analysis: $(date)" > "$report_file"
echo "Log file name: $log_file" >> "$report_file"
echo "Total lines processed: $total_lines" >> "$report_file"
echo "Total error count: $error_count" >> "$report_file"
echo "Top 5 error messages with their occurrence count:" >> "$report_file"
echo "$top_errors" >> "$report_file"
echo "List of critical events with line numbers:" >> "$report_file"
echo "$critical_events" >> "$report_file"

echo "Analysis complete. Report generated: $report_file"

Output

./log_analyzer.sh /home/ubuntu/sample_log.log 

Analysis complete. Report generated: summary_report_20240823.txt

$ cat summary_report*.txt
Date of analysis: Fri Aug 23 12:26:32 UTC 2024
Log file name: /home/ubuntu/sample_log.log
Total lines processed: 2001
Total error count: 13
Top 5 error messages with their occurrence count:
12  - Unexpected exception causing shutdown while sock still open
1  - Unexpected Exception: 
List of critical events with line numbers:

Follow for more update:)