How to Integrate Slack & Gmail Alerts with Wazuh SIEM: Step-by-Step Guide

Table of contents
- Introduction
- Part 1: Complete Slack Setup - Every Detail
- Part 2: Complete Gmail Setup - Every Detail
- Part 3: Wazuh Container Access and System Preparation
- Part 4: Complete Slack Integration - File by File
- Part 5: Complete Gmail Integration - File by File
- Part 6: Wazuh Configuration and Integration
- Part 7: Comprehensive Testing and Validation
- Part 8: Advanced Troubleshooting Guide
- Part 9: Performance Optimization and Production Setup
- Part 10: Final Validation and Success Confirmation
- Part 11: Accessing and Using Wazuh Dashboard
- Step 11.1: Dashboard Login and Setup
- Step 11.2: Initial Dashboard Configuration
- Step 11.3: Navigating Dashboard Sections
- Step 11.4: Monitoring Your Integration Alerts
- Step 11.5: Real-time Monitoring Setup
- Step 11.6: Verifying Integration Status
- Step 11.7: Dashboard Alert Correlation
- Step 11.8: Advanced Dashboard Features
- Step 11.9: Dashboard-Based Troubleshooting
- Step 11.10: Dashboard Best Practices
- Step 11.11: Dashboard Integration Verification
- Step 11.12: Dashboard Customization for Integration
- Conclusion
Introduction
This comprehensive guide will walk you through every single step of integrating Wazuh SIEM with Slack and Gmail notifications. We'll cover everything from creating accounts to troubleshooting, with detailed screenshots descriptions and exact navigation steps.
Part 1: Complete Slack Setup - Every Detail
Step 1.1: Creating Slack Workspace (If You Don't Have One)
If you don't have a Slack workspace:
Go to slack.com in your web browser
Click "Get Started" in the top right corner
Enter your email address and click "Continue"
Check your email for a verification code
Enter the 6-digit code from your email
Click "Create a Workspace"
Enter your workspace name (e.g., "Security Monitoring")
Enter a project or team name (e.g., "IT Security Team")
Add team members or skip this step
Your workspace URL will be something like: https://your-workspace.slack.com
Step 1.2: Creating Slack Channel for Wazuh Alerts
In your Slack workspace, look at the left sidebar
Under "Channels", click the "+" button next to "Channels"
Select "Create a channel"
Choose "Public" (so team members can see security alerts)
Name your channel: "wazuh-security-alerts" or any
Add a description: "Automated security alerts from Wazuh SIEM system"
Click "Create Channel"
You'll be automatically added to the channel
Step 1.3: Installing Incoming Webhooks App - Detailed Steps
In your Slack workspace, click on your workspace name in the top left
From the dropdown menu, select "Settings & administration"
Click "Manage apps"
This will open the Slack App Directory in a new tab
In the search bar at the top, type "Incoming Webhooks"
Click on "Incoming Webhooks" from the search results
You'll see the Incoming Webhooks app page
Click the green "Add to Slack" button
A new page will open asking "Post to Channel"
In the dropdown, select your "wazuh-security-alerts" channel
Click "Add Incoming Webhooks Integration"
Step 1.4: Getting Your Webhook URL - Critical Step
After clicking "Add Incoming Webhooks Integration":
You'll be redirected to a configuration page
Scroll down to find "Webhook URL"
You'll see a URL that looks like this:
https://hooks.slack.com/services/T1234567890/B1234567890/abcdefghijklmnopqrstuvwx
CRITICAL: Copy this ENTIRE URL - this is your webhook URL
Save it in a text file temporarily - you'll need it multiple times
Scroll down and you can customize:
Descriptive Label: "Wazuh SIEM Alerts"
Customize Name: "Wazuh Security Bot"
Customize Icon: Upload a security-related emoji or image
Click "Save Settings"
Step 1.5: Testing Your Slack Webhook
Before proceeding with Wazuh integration, test your webhook:
Open Command Prompt or Terminal
Run this command (replace YOUR_WEBHOOK_URL with your actual URL):
curl -X POST -H 'Content-type: application/json' --data '{"text":"๐ด Test alert from Wazuh SIEM setup - If you see this message, Slack integration is working correctly!"}' https://hooks.slack.com/services/YOUR/WEBHOOK/URL
Check your #wazuh-security-alerts channel
You should see your test message within 5 seconds
If you don't see it, double-check your webhook URL
Part 2: Complete Gmail Setup - Every Detail
Step 2.1: Verify Gmail Account Requirements
Make sure you have a Gmail account (not G Suite/Google Workspace)
Log into your Gmail account at gmail.com
Click on your profile picture in the top right
Click "Manage your Google Account"
In the left sidebar, click "Security"
Step 2.2: Enable 2-Factor Authentication (Required)
If 2FA is not already enabled:
On the Security page, look for "2-Step Verification"
If it shows "Off", click on "2-Step Verification"
Click "Get Started"
Enter your password when prompted
Choose your verification method:
Phone number (recommended): Enter your phone number
Authenticator app: Use Google Authenticator or similar
Follow the verification steps
Enter the code you receive via SMS or app
Click "Turn On" to enable 2-Step Verification
You'll see "2-Step Verification: On" on your Security page
Step 2.3: Generate App Password - Detailed Process
Still on the Security page, scroll down to find "App passwords"
Click "App passwords"
You may be asked to enter your password again
You'll see "App passwords" page with a dropdown
In the "Select app" dropdown, choose "Mail"
In the "Select device" dropdown, choose "Other (custom name)"
Type a descriptive name: "Wazuh SIEM Email Alerts"
Click "Generate"
CRITICAL: You'll see a 16-character password like:
abcd efgh ijkl mnop
Copy this password immediately - Google will only show it once
Remove all spaces from the password:
abcdefghijklmnop
Save this password in a secure location - you'll need it for configuration
Step 2.4: Important Email Addresses to Note
Write down these email addresses:
Sender Email: Your Gmail address (e.g., youremail@gmail.com)
Receiver Email: The email where you want to receive alerts (can be the same Gmail or different)
App Password: The 16-character password you just generated (no spaces)
Part 3: Wazuh Container Access and System Preparation
Step 3.1: Accessing Your Wazuh Container - Detailed Steps
Open Command Prompt (Windows) or Terminal (Mac/Linux)
Navigate to your Wazuh installation directory:
cd path/to/your/wazuh/single-node
List all running Docker containers:
docker ps
Look for a container with "wazuh.manager" in the name
The full name is usually:
single-node-wazuh.manager-1
Access the container:
docker exec -it single-node-wazuh.manager-1 /bin/bash
You should see a prompt like:
root@wazuh-manager:/var/ossec#
Verify you're in the right place:
pwd ls -la
You should see directories like: bin, etc, integrations, logs
Step 3.2: Understanding Wazuh Directory Structure
Current directory breakdown:
/var/ossec/
- Main Wazuh directory/var/ossec/etc/
- Configuration files/var/ossec/integrations/
- Integration scripts/var/ossec/logs/
- Log files/var/ossec/bin/
- Executable files
Step 3.3: Installing Required System Packages
Update the package manager:
yum update -y
Wait for this to complete (may take 2-3 minutes)
Install email-related packages:
yum install -y cyrus-sasl cyrus-sasl-plain cyrus-sasl-md5 postfix mailx python3 curl wget
Wait for installation to complete
Verify installations:
which postfix which mailx which python3 python3 --version
You should see paths and Python version 3.x
Part 4: Complete Slack Integration - File by File
Step 4.1: Creating Slack Configuration File
Navigate to integrations directory:
cd /var/ossec/integrations
Create the Slack configuration file:
cat > slack_config.json << 'EOF' { "webhook_url": "REPLACE_WITH_YOUR_WEBHOOK_URL" } EOF
Edit the file with your actual webhook URL:
nano slack_config.json
Replace
REPLACE_WITH_YOUR_WEBHOOK_URL
with your Slack webhook URL from Step 1.4 Press Ctrl+O to save, Enter to confirm, Ctrl+X to exitVerify the file:
cat slack_config.json
Step 4.2: Creating Main Slack Integration Python Script
Create the main Python script in multiple parts:(copy part by part as sometime inside the container the copy pasting of large files are not allowed).
Part 1 - Script Header and Imports:
cat > slack_integration.py << 'EOF'
#!/usr/bin/env python3
"""
Wazuh Slack Integration Script
Processes Wazuh alerts and sends formatted messages to Slack
"""
import sys
import json
import urllib.request
import urllib.parse
from datetime import datetime
# Configuration
SLACK_CONFIG_FILE = '/var/ossec/integrations/slack_config.json'
LOG_FILE = '/var/ossec/logs/slack_integration.log'
def log_message(message):
"""Log messages with timestamp for debugging"""
timestamp = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
try:
with open(LOG_FILE, 'a') as f:
f.write(f"{timestamp} - {message}\n")
except:
pass # Fail silently if logging fails
EOF
Part 2 - Alert Formatting Functions:
cat >> slack_integration.py << 'EOF'
def get_priority_info(level):
"""Return emoji and color based on alert level"""
level = int(level)
if level >= 12:
return "๐ด", "#FF0000", "CRITICAL"
elif level >= 8:
return "๐ ", "#FF6600", "HIGH"
elif level >= 5:
return "๐ก", "#FFCC00", "MEDIUM"
elif level >= 3:
return "๐ต", "#0066CC", "LOW"
else:
return "โช", "#808080", "INFO"
def format_timestamp(timestamp):
"""Format timestamp for better readability"""
try:
# Handle different timestamp formats
if 'T' in timestamp:
dt = datetime.fromisoformat(timestamp.replace('Z', '+00:00'))
else:
dt = datetime.strptime(timestamp, '%Y %b %d %H:%M:%S')
return dt.strftime('%Y-%m-%d %H:%M:%S UTC')
except:
return timestamp
def get_rule_category(rule_id, description):
"""Categorize alerts based on rule ID and description"""
rule_id = str(rule_id)
description = description.lower()
if 'authentication' in description or 'login' in description:
return "๐ Authentication"
elif 'file' in description or 'integrity' in description:
return "๐ File Monitoring"
elif 'network' in description or 'connection' in description:
return "๐ Network Activity"
elif 'privilege' in description or 'sudo' in description:
return "โก Privilege Escalation"
elif 'malware' in description or 'virus' in description:
return "๐ฆ Malware Detection"
else:
return "๐ก๏ธ Security Event"
EOF
Part 3 - Slack Message Formatting:
cat >> slack_integration.py << 'EOF'
def create_slack_message(alert_data):
"""Create formatted Slack message from alert data"""
try:
alert = json.loads(alert_data)
# Extract basic information
rule = alert.get('rule', {})
agent = alert.get('agent', {})
rule_id = rule.get('id', 'Unknown')
rule_level = rule.get('level', 0)
description = rule.get('description', 'No description available')
agent_name = agent.get('name', 'Unknown')
timestamp = alert.get('timestamp', 'Unknown')
location = alert.get('location', 'Unknown')
# Get priority information
emoji, color, priority = get_priority_info(rule_level)
formatted_time = format_timestamp(timestamp)
category = get_rule_category(rule_id, description)
# Create main message
message = {
"text": f"{emoji} Wazuh Security Alert - {priority} Priority",
"attachments": [
{
"color": color,
"title": f"{category} - Level {rule_level}",
"fields": [
{
"title": "Alert Description",
"value": description,
"short": False
},
{
"title": "Agent",
"value": agent_name,
"short": True
},
{
"title": "Rule ID",
"value": str(rule_id),
"short": True
},
{
"title": "Severity Level",
"value": f"{rule_level}/15",
"short": True
},
{
"title": "Location",
"value": location,
"short": True
},
{
"title": "Timestamp",
"value": formatted_time,
"short": False
}
],
"footer": "Wazuh SIEM Security Monitoring",
"footer_icon": "https://wazuh.com/favicon.ico",
"ts": int(datetime.now().timestamp())
}
]
}
# Add additional data if available
if 'data' in alert:
data = alert['data']
additional_fields = []
if 'srcip' in data:
additional_fields.append({
"title": "Source IP",
"value": data['srcip'],
"short": True
})
if 'srcuser' in data:
additional_fields.append({
"title": "Source User",
"value": data['srcuser'],
"short": True
})
if 'dstport' in data:
additional_fields.append({
"title": "Destination Port",
"value": data['dstport'],
"short": True
})
if additional_fields:
message["attachments"][0]["fields"].extend(additional_fields)
return message
except Exception as e:
log_message(f"Error formatting alert: {str(e)}")
# Return error message
return {
"text": f"๐ด Wazuh Alert Processing Error",
"attachments": [
{
"color": "#FF0000",
"text": f"Error: {str(e)}\nRaw data: {alert_data[:200]}..."
}
]
}
EOF
Part 4 - HTTP Sending Function:
cat >> slack_integration.py << 'EOF'
def send_to_slack(webhook_url, message):
"""Send message to Slack via webhook"""
try:
data = json.dumps(message).encode('utf-8')
req = urllib.request.Request(webhook_url, data=data)
req.add_header('Content-Type', 'application/json')
with urllib.request.urlopen(req, timeout=10) as response:
if response.status == 200:
log_message("โ
Successfully sent alert to Slack")
return True
else:
log_message(f"โ Slack API returned status: {response.status}")
return False
except urllib.error.URLError as e:
log_message(f"โ Network error sending to Slack: {str(e)}")
return False
except Exception as e:
log_message(f"โ Unexpected error sending to Slack: {str(e)}")
return False
def load_config():
"""Load Slack configuration from JSON file"""
try:
with open(SLACK_CONFIG_FILE, 'r') as f:
config = json.load(f)
return config.get('webhook_url')
except Exception as e:
log_message(f"โ Error loading Slack config: {str(e)}")
return None
EOF
Part 5 - Main Function:
cat >> slack_integration.py << 'EOF'
def main():
"""Main function"""
if len(sys.argv) != 2:
log_message("โ Invalid arguments. Usage: slack_integration.py <alert_json>")
print("Usage: slack_integration.py <alert_json>")
sys.exit(1)
alert_data = sys.argv[1]
log_message(f"๐จ Processing alert: {alert_data[:100]}...")
# Load webhook URL
webhook_url = load_config()
if not webhook_url:
log_message("โ Failed to load webhook URL from config")
sys.exit(1)
# Create and send message
message = create_slack_message(alert_data)
success = send_to_slack(webhook_url, message)
if success:
print("โ
Alert sent to Slack successfully")
sys.exit(0)
else:
print("โ Failed to send alert to Slack")
sys.exit(1)
if __name__ == "__main__":
main()
EOF
Set Permissions:
chmod 755 slack_integration.py
chown root:wazuh slack_integration.py
Step 4.3: Creating Slack Wrapper Script
cat > slack_wrapper.sh << 'EOF'
#!/bin/bash
# Wazuh Slack Integration Wrapper
# This script is called by Wazuh for each alert
ALERT_FILE=$1
LOG_FILE="/var/ossec/logs/slack_wrapper.log"
# Function to log messages
log_message() {
echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE"
}
# Validate input
if [ -z "$ALERT_FILE" ]; then
log_message "โ ERROR: No alert file provided"
exit 1
fi
if [ ! -f "$ALERT_FILE" ]; then
log_message "โ ERROR: Alert file does not exist: $ALERT_FILE"
exit 1
fi
log_message "๐จ Processing alert file: $ALERT_FILE"
# Read alert content and pass to Python script
ALERT_CONTENT=$(cat "$ALERT_FILE")
log_message "๐ Alert content: ${ALERT_CONTENT:0:200}..."
# Call Python integration script
python3 /var/ossec/integrations/slack_integration.py "$ALERT_CONTENT" >> "$LOG_FILE" 2>&1
EXIT_CODE=$?
if [ $EXIT_CODE -eq 0 ]; then
log_message "โ
Slack integration completed successfully"
else
log_message "โ Slack integration failed with exit code: $EXIT_CODE"
fi
exit $EXIT_CODE
EOF
# Set permissions
chmod 755 slack_wrapper.sh
chown root:root slack_wrapper.sh
Step 4.4: Testing Slack Integration
# Create test alert
cat > /tmp/test_slack_alert.json << 'EOF'
{
"timestamp": "2024-06-26T10:30:00.000Z",
"agent": {
"name": "test-server-01"
},
"rule": {
"description": "Test Slack integration - Manual verification",
"level": 5,
"id": 12345
},
"location": "manual-test",
"data": {
"srcip": "192.168.1.100"
}
}
EOF
# Test Python script directly
python3 /var/ossec/integrations/slack_integration.py "$(cat /tmp/test_slack_alert.json)"
# Test wrapper script
/var/ossec/integrations/slack_wrapper.sh /tmp/test_slack_alert.json
# Check logs
cat /var/ossec/logs/slack_integration.log
cat /var/ossec/logs/slack_wrapper.log
Expected Result: You should see a formatted alert message in your Slack channel within 10 seconds.
Part 5: Complete Gmail Integration - File by File
Step 5.1: Configuring Postfix for Gmail
Create main Postfix configuration:
# Backup existing config
cp /etc/postfix/main.cf /etc/postfix/main.cf.backup
# Create new configuration
cat > /etc/postfix/main.cf << 'EOF'
# Postfix Main Configuration for Gmail SMTP Relay
# Generated for Wazuh SIEM Email Integration
# Basic Postfix Settings
compatibility_level = 2
queue_directory = /var/spool/postfix
command_directory = /usr/sbin
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
mail_owner = postfix
myhostname = wazuh-server.localdomain
mydomain = localdomain
myorigin = $myhostname
inet_interfaces = loopback-only
inet_protocols = ipv4
mydestination = $myhostname, localhost.$mydomain, localhost
unknown_local_recipient_reject_code = 550
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
debug_peer_level = 2
sendmail_path = /usr/sbin/sendmail.postfix
newaliases_path = /usr/bin/newaliases.postfix
mailq_path = /usr/bin/mailq.postfix
setgid_group = postdrop
html_directory = no
manpage_directory = /usr/share/man
# Gmail SMTP Relay Configuration
relayhost = [smtp.gmail.com]:587
smtp_use_tls = yes
smtp_tls_security_level = encrypt
smtp_tls_note_starttls_offer = yes
smtp_tls_CAfile = /etc/pki/tls/certs/ca-bundle.crt
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_sasl_mechanism_filter = plain
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
# Additional security settings
smtp_tls_exclude_ciphers = export, low, medium
smtp_tls_mandatory_ciphers = high
smtp_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
EOF
Step 5.2: Creating Gmail Authentication
CRITICAL: Replace with YOUR credentials:
# Create authentication file (REPLACE WITH YOUR CREDENTIALS)
cat > /etc/postfix/sasl_passwd << 'EOF'
[smtp.gmail.com]:587 yourgmail@gmail.com:yourapppassword
EOF
# Example with real format:
# [smtp.gmail.com]:587 barsarun10@gmail.com:abcdefghijklmnop
Secure the authentication:
# Create hash database
postmap /etc/postfix/sasl_passwd
# Set secure permissions
chmod 600 /etc/postfix/sasl_passwd
chmod 600 /etc/postfix/sasl_passwd.db
# Verify files
ls -la /etc/postfix/sasl_passwd*
Step 5.3: Starting and Testing Postfix
# Start Postfix service
systemctl start postfix
systemctl enable postfix
# Check status
systemctl status postfix
# Test SMTP connectivity
telnet smtp.gmail.com 587
# Type 'quit' and press Enter to exit
# Send test email
echo "Subject: Wazuh Email Test - Configuration Verification
From: yourgmail@gmail.com
To: recipient@gmail.com
This is a test email from Wazuh SIEM email integration.
If you receive this, your email configuration is working correctly.
Configuration test performed at: $(date)
Server: $(hostname)
Postfix status: Active" | sendmail recipient@gmail.com
# Check mail queue
mailq
Step 5.4: Creating Email Integration Script
Create main email processing script:
cd /var/ossec/integrations
cat > email_integration << 'EOF'
#!/bin/bash
# Wazuh Email Integration Script
# Processes security alerts and sends formatted emails via Gmail
# Configuration
ALERT_FILE=$1
LOG_FILE="/var/ossec/logs/email_integration.log"
SENDER_EMAIL="yourgmail@gmail.com" # REPLACE WITH YOUR GMAIL
RECIPIENT_EMAIL="recipient@gmail.com" # REPLACE WITH RECIPIENT
MIN_LEVEL=7 # Only send emails for level 7+ alerts
# Logging function
log_message() {
echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE"
}
# Validate input
if [ -z "$ALERT_FILE" ]; then
log_message "โ ERROR: No alert file provided"
exit 1
fi
if [ ! -f "$ALERT_FILE" ]; then
log_message "โ ERROR: Alert file does not exist: $ALERT_FILE"
exit 1
fi
log_message "๐ง Processing email alert: $ALERT_FILE"
# Read and parse alert content
ALERT_CONTENT=$(cat "$ALERT_FILE")
log_message "๐ Alert content length: $(echo "$ALERT_CONTENT" | wc -c) characters"
# Extract alert information using Python for better JSON parsing
ALERT_INFO=$(python3 << EOF
import json
import sys
try:
alert = json.loads('''$ALERT_CONTENT''')
rule = alert.get('rule', {})
agent = alert.get('agent', {})
print(f"LEVEL:{rule.get('level', 0)}")
print(f"ID:{rule.get('id', 'Unknown')}")
print(f"DESCRIPTION:{rule.get('description', 'No description')}")
print(f"AGENT:{agent.get('name', 'Unknown')}")
print(f"TIMESTAMP:{alert.get('timestamp', 'Unknown')}")
print(f"LOCATION:{alert.get('location', 'Unknown')}")
# Extract additional data if available
if 'data' in alert:
data = alert['data']
print(f"SRCIP:{data.get('srcip', '')}")
print(f"SRCUSER:{data.get('srcuser', '')}")
print(f"DSTPORT:{data.get('dstport', '')}")
else:
print("SRCIP:")
print("SRCUSER:")
print("DSTPORT:")
except Exception as e:
print(f"ERROR:Failed to parse JSON: {str(e)}")
sys.exit(1)
EOF
)
# Check if parsing was successful
if echo "$ALERT_INFO" | grep -q "ERROR:"; then
ERROR_MSG=$(echo "$ALERT_INFO" | grep "ERROR:" | cut -d':' -f2-)
log_message "โ JSON parsing error: $ERROR_MSG"
exit 1
fi
# Extract parsed information
RULE_LEVEL=$(echo "$ALERT_INFO" | grep "LEVEL:" | cut -d':' -f2)
RULE_ID=$(echo "$ALERT_INFO" | grep "ID:" | cut -d':' -f2-)
DESCRIPTION=$(echo "$ALERT_INFO" | grep "DESCRIPTION:" | cut -d':' -f2-)
AGENT_NAME=$(echo "$ALERT_INFO" | grep "AGENT:" | cut -d':' -f2-)
TIMESTAMP=$(echo "$ALERT_INFO" | grep "TIMESTAMP:" | cut -d':' -f2-)
LOCATION=$(echo "$ALERT_INFO" | grep "LOCATION:" | cut -d':' -f2-)
SRCIP=$(echo "$ALERT_INFO" | grep "SRCIP:" | cut -d':' -f2-)
SRCUSER=$(echo "$ALERT_INFO" | grep "SRCUSER:" | cut -d':' -f2-)
DSTPORT=$(echo "$ALERT_INFO" | grep "DSTPORT:" | cut -d':' -f2-)
log_message "๐ Parsed - Level: $RULE_LEVEL, Rule: $RULE_ID, Agent: $AGENT_NAME"
# Check if alert level meets email threshold
if [ "$RULE_LEVEL" -ge "$MIN_LEVEL" ]; then
log_message "โ
Alert level $RULE_LEVEL meets email threshold ($MIN_LEVEL+)"
# Determine priority and formatting
if [ "$RULE_LEVEL" -ge 12 ]; then
PRIORITY="๐ด CRITICAL"
URGENCY="IMMEDIATE ACTION REQUIRED"
BORDER="๐จ๐จ๐จ๐จ๐จ๐จ๐จ๐จ๐จ๐จ๐จ๐จ๐จ๐จ๐จ๐จ๐จ๐จ๐จ๐จ"
elif [ "$RULE_LEVEL" -ge 10 ]; then
PRIORITY="๐ HIGH"
URGENCY="HIGH PRIORITY - PROMPT ATTENTION NEEDED"
BORDER="โ ๏ธโ ๏ธโ ๏ธโ ๏ธโ ๏ธโ ๏ธโ ๏ธโ ๏ธโ ๏ธโ ๏ธโ ๏ธโ ๏ธโ ๏ธโ ๏ธโ ๏ธโ ๏ธโ ๏ธโ ๏ธโ ๏ธโ ๏ธ"
elif [ "$RULE_LEVEL" -ge 8 ]; then
PRIORITY="๐ก MEDIUM"
URGENCY="MEDIUM PRIORITY - REVIEW REQUIRED"
BORDER="๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ"
else
PRIORITY="๐ต ELEVATED"
URGENCY="ELEVATED ALERT - MONITOR SITUATION"
BORDER="๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น"
fi
# Create email subject
EMAIL_SUBJECT="$PRIORITY Wazuh Security Alert - $DESCRIPTION"
# Create comprehensive email body
EMAIL_BODY="$BORDER
๐ก๏ธ WAZUH SECURITY INFORMATION AND EVENT MANAGEMENT SYSTEM
$BORDER
$URGENCY
๐ SECURITY ALERT SUMMARY
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
๐ Alert ID: $RULE_ID
๐ Severity Level: $RULE_LEVEL/15
๐ป Affected Agent: $AGENT_NAME
๐ Log Location: $LOCATION
โฐ Event Timestamp: $TIMESTAMP
๐ ALERT DESCRIPTION
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
$DESCRIPTION"
# Add source information if available
if [ ! -z "$SRCIP" ]; then
EMAIL_BODY="$EMAIL_BODY
๐ SOURCE INFORMATION
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
๐ฅ๏ธ Source IP Address: $SRCIP"
if [ ! -z "$SRCUSER" ]; then
EMAIL_BODY="$EMAIL_BODY
๐ค Source User: $SRCUSER"
fi
if [ ! -z "$DSTPORT" ]; then
EMAIL_BODY="$EMAIL_BODY
๐ Destination Port: $DSTPORT"
fi
fi
EMAIL_BODY="$EMAIL_BODY
๐ COMPLETE ALERT DATA
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
$ALERT_CONTENT
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
๐ก๏ธ SYSTEM INFORMATION
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
๐ง Email Generated: $(date '+%Y-%m-%d %H:%M:%S %Z')
๐ฅ๏ธ Wazuh Manager: $(hostname)
๐ฑ Optimized for Gmail App viewing
๐ Dashboard Access: https://your-wazuh-dashboard-url
โ ๏ธ INCIDENT RESPONSE REMINDER:
1. Acknowledge this alert in your ticketing system
2. Investigate the source and nature of the security event
3. Document findings and remediation actions taken
4. Update security policies if necessary
$BORDER"
# Send email with proper headers for Gmail
log_message "๐ค Sending email notification to $RECIPIENT_EMAIL"
(
echo "From: Wazuh Security System <$SENDER_EMAIL>"
echo "To: Security Operations Team <$RECIPIENT_EMAIL>"
echo "Subject: $EMAIL_SUBJECT"
echo "Reply-To: $SENDER_EMAIL"
echo "MIME-Version: 1.0"
echo "Content-Type: text/plain; charset=UTF-8"
echo "Content-Transfer-Encoding: 8bit"
echo "X-Priority: 1"
echo "X-MSMail-Priority: High"
echo "Importance: High"
echo "X-Mailer: Wazuh SIEM v4.x"
echo "X-Wazuh-Alert-Level: $RULE_LEVEL"
echo "X-Wazuh-Rule-ID: $RULE_ID"
echo ""
echo "$EMAIL_BODY"
) | sendmail "$RECIPIENT_EMAIL"
# Check if email was sent successfully
if [ $? -eq 0 ]; then
log_message "โ
Email sent successfully - Rule $RULE_ID, Level $RULE_LEVEL"
echo "โ
Email notification sent successfully"
else
log_message "โ Email sending failed - Rule $RULE_ID, Level $RULE_LEVEL"
echo "โ Failed to send email notification"
exit 1
fi
else
log_message "โน๏ธ Alert level $RULE_LEVEL below email threshold ($MIN_LEVEL), skipping email notification"
echo "โน๏ธ Alert level too low for email notification"
fi
log_message "๐ Email integration processing completed"
EOF
# Set permissions
chmod 750 email_integration
chown root:wazuh email_integration
CRITICAL: Edit the script to replace:
yourgmail@gmail.com
with your Gmail addressrecipient@gmail.com
with your recipient email address
nano email_integration
# Find and replace the email addresses, then save
Step 5.5: Creating Email Wrapper Script
cat > custom-email << 'EOF'
#!/bin/bash
# Wazuh Email Integration Wrapper
# This script is called by Wazuh for email notifications
ALERT_FILE=$1
LOG_FILE="/var/ossec/logs/email_wrapper.log"
# Logging function
log_message() {
echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE"
}
# Validate input
if [ -z "$ALERT_FILE" ]; then
log_message "โ ERROR: No alert file provided to email wrapper"
exit 1
fi
if [ ! -f "$ALERT_FILE" ]; then
log_message "โ ERROR: Alert file does not exist: $ALERT_FILE"
exit 1
fi
log_message "๐ง Email wrapper called with alert file: $ALERT_FILE"
# Verify email integration script exists
if [ ! -f "/var/ossec/integrations/email_integration" ]; then
log_message "โ ERROR: Email integration script not found"
exit 1
fi
# Call email integration script
log_message "๐ Calling email integration script"
bash /var/ossec/integrations/email_integration "$ALERT_FILE"
EXIT_CODE=$?
if [ $EXIT_CODE -eq 0 ]; then
log_message "โ
Email integration completed successfully"
else
log_message "โ Email integration failed with exit code: $EXIT_CODE"
fi
exit $EXIT_CODE
EOF
# Set permissions
chmod 750 custom-email
chown root:wazuh custom-email
Step 5.6: Testing Email Integration
# Create test alert for email (high level)
cat > /tmp/test_email_alert.json << 'EOF'
{
"timestamp": "2024-06-26T10:30:00.000Z",
"agent": {
"name": "production-server-01"
},
"rule": {
"description": "Test Gmail integration - Critical security alert simulation",
"level": 10,
"id": 99999
},
"location": "/var/log/secure",
"data": {
"srcip": "192.168.1.100",
"srcuser": "testuser",
"dstport": "22"
}
}
EOF
# Test email integration directly
/var/ossec/integrations/email_integration /tmp/test_email_alert.json
# Test wrapper script
/var/ossec/integrations/custom-email /tmp/test_email_alert.json
# Check logs
echo "=== Email Integration Log ==="
cat /var/ossec/logs/email_integration.log
echo "=== Email Wrapper Log ==="
cat /var/ossec/logs/email_wrapper.log
# Check mail queue
echo "=== Mail Queue Status ==="
mailq
Expected Result: You should receive a detailed email in your Gmail account within 2-3 minutes.
Part 6: Wazuh Configuration and Integration
Step 6.1: Configuring Wazuh Integrations
Edit the main Wazuh configuration file:
nano /var/ossec/etc/ossec.conf
Find the <ossec_config>
section and add these integration blocks (usually after the <global>
section):
<!-- Custom Slack Integration - All Alerts -->
<integration>
<name>custom-slack</name>
<hook_url>/var/ossec/integrations/slack_wrapper.sh</hook_url>
<alert_format>json</alert_format>
<level>1</level>
</integration>
<!-- Custom Email Integration - Critical Alerts Only -->
<integration>
<name>custom-email</name>
<hook_url>/var/ossec/integrations/custom-email</hook_url>
<alert_format>json</alert_format>
<level>7</level>
</integration>
Configuration Explanation:
<name>
: Custom integration name (must be unique)<hook_url>
: Path to the wrapper script<alert_format>
: Format of alert data (json recommended)<level>
: Minimum alert level to trigger integrationSlack: Level 1+ (all alerts for real-time monitoring)
Email: Level 7+ (critical alerts only to avoid spam)
Save and exit: Ctrl+O, Enter, Ctrl+X
Step 6.2: Creating Log Files
# Create log files for monitoring
touch /var/ossec/logs/slack_integration.log
touch /var/ossec/logs/slack_wrapper.log
touch /var/ossec/logs/email_integration.log
touch /var/ossec/logs/email_wrapper.log
# Set proper ownership and permissions
chown ossec:ossec /var/ossec/logs/slack_integration.log
chown ossec:ossec /var/ossec/logs/slack_wrapper.log
chown ossec:ossec /var/ossec/logs/email_integration.log
chown ossec:ossec /var/ossec/logs/email_wrapper.log
chmod 664 /var/ossec/logs/slack_integration.log
chmod 664 /var/ossec/logs/slack_wrapper.log
chmod 664 /var/ossec/logs/email_integration.log
chmod 664 /var/ossec/logs/email_wrapper.log
Step 6.3: Restarting Wazuh Services
# Stop all Wazuh services
echo "Stopping Wazuh services..."
/var/ossec/bin/wazuh-control stop
# Wait for clean shutdown
sleep 10
# Start all Wazuh services
echo "Starting Wazuh services..."
/var/ossec/bin/wazuh-control start
# Wait for startup
sleep 10
# Check service status
/var/ossec/bin/wazuh-control status
Expected output should show all services as "running":
wazuh-agentlessd is running...
wazuh-analysisd is running...
wazuh-authd is running...
wazuh-csyslogd is running...
wazuh-dbd not running...
wazuh-integratord is running...
wazuh-logcollector is running...
wazuh-maild not running...
wazuh-monitord is running...
wazuh-reportd not running...
wazuh-syscheckd is running...
wazuh-execd is running...
Step 6.4: Verifying Integration Loading
# Check if integrations loaded successfully
echo "=== Integration Loading Verification ==="
grep -i "integration" /var/ossec/logs/ossec.log | tail -10
# Look for our specific integrations
echo "=== Our Integration Status ==="
grep -E "custom-slack|custom-email" /var/ossec/logs/ossec.log | tail -5
# Check for integration errors
echo "=== Integration Error Check ==="
grep -i "error.*integration" /var/ossec/logs/ossec.log | tail -5
Expected output should include:
INFO: Enabling integration for: 'custom-slack'
INFO: Enabling integration for: 'custom-email'
Part 7: Comprehensive Testing and Validation
Step 7.1: File Verification Checklist
echo "=== COMPLETE FILE VERIFICATION ==="
# Check all Slack files
echo "--- Slack Integration Files ---"
ls -la /var/ossec/integrations/slack_integration.py
ls -la /var/ossec/integrations/slack_config.json
ls -la /var/ossec/integrations/slack_wrapper.sh
# Check all Email files
echo "--- Email Integration Files ---"
ls -la /var/ossec/integrations/email_integration
ls -la /var/ossec/integrations/custom-email
# Check system files
echo "--- System Configuration Files ---"
ls -la /etc/postfix/main.cf
ls -la /etc/postfix/sasl_passwd*
# Check log files
echo "--- Log Files ---"
ls -la /var/ossec/logs/slack_*.log
ls -la /var/ossec/logs/email_*.log
# Verify permissions
echo "--- Permission Verification ---"
stat -c "%a %U:%G %n" /var/ossec/integrations/slack_wrapper.sh
stat -c "%a %U:%G %n" /var/ossec/integrations/custom-email
Step 7.2: Service Status Verification
echo "=== SERVICE STATUS VERIFICATION ==="
# Check Wazuh services
echo "--- Wazuh Services ---"
/var/ossec/bin/wazuh-control status
# Check Postfix service
echo "--- Postfix Service ---"
systemctl status postfix --no-pager
# Check if integratord is running (handles integrations)
echo "--- Integration Service ---"
ps aux | grep integratord | grep -v grep
Step 7.3: Generating Test Alerts
Test 1: Low-level alert (Slack only):
echo "=== TEST 1: Low-level Alert (Slack Only) ==="
logger -p local0.info "Wazuh integration test - low level security event $(date)"
# Wait for processing
sleep 5
# Check Slack logs
echo "--- Slack Activity ---"
tail -3 /var/ossec/logs/slack_wrapper.log
Test 2: High-level alert (Both Slack and Email):
echo "=== TEST 2: High-level Alert (Both Platforms) ==="
# Generate multiple failed login attempts (creates high-level alert)
for i in {1..3}; do
logger -p auth.warning "sshd[1234]: Failed password for root from 192.168.1.100 port 22 ssh2"
sleep 2
done
# Wait for processing
sleep 15
# Check both integration logs
echo "--- Slack Activity ---"
tail -5 /var/ossec/logs/slack_wrapper.log
echo "--- Email Activity ---"
tail -5 /var/ossec/logs/email_integration.log
echo "--- Mail Queue ---"
mailq
Test 3: Critical security event:
echo "=== TEST 3: Critical Security Event ==="
# Generate privilege escalation attempt
logger -p auth.warning "sudo: hacker : user NOT in sudoers ; TTY=pts/1 ; PWD=/tmp ; USER=root ; COMMAND=/bin/cat /etc/shadow"
# Generate suspicious file access
logger -p local0.warning "File integrity monitoring: /etc/passwd was modified by unauthorized user"
# Wait for processing
sleep 20
# Check alert generation
echo "--- Recent Alerts ---"
tail -10 /var/ossec/logs/alerts/alerts.log | grep -E '"level":[0-9]*'
# Check both notification systems
echo "--- Slack Notifications ---"
tail -5 /var/ossec/logs/slack_wrapper.log
echo "--- Email Notifications ---"
tail -5 /var/ossec/logs/email_integration.log
Step 7.4: Real-time Monitoring Setup
Terminal 1 - Monitor all alerts:
docker exec -it single-node-wazuh.manager-1 tail -f /var/ossec/logs/alerts/alerts.log
Terminal 2 - Monitor Slack activity:
docker exec -it single-node-wazuh.manager-1 tail -f /var/ossec/logs/slack_wrapper.log
Terminal 3 - Monitor email activity:
docker exec -it single-node-wazuh.manager-1 tail -f /var/ossec/logs/email_integration.log
Terminal 4 - Monitor mail queue:
watch 'docker exec -it single-node-wazuh.manager-1 mailq'
Part 8: Advanced Troubleshooting Guide
Issue 1: Slack Integration Not Working
Symptom: No messages appearing in Slack channel
Detailed Diagnosis:
- Check integration loading:
grep -i "custom-slack" /var/ossec/logs/ossec.log
Expected: INFO: Enabling integration for: 'custom-slack'
- Verify webhook URL configuration:
cat /var/ossec/integrations/slack_config.json
Verify the webhook URL is complete and correct.
- Test webhook manually:
curl -X POST -H 'Content-type: application/json' \
--data '{"text":"๐ด Manual webhook test - if you see this, webhook is working"}' \
https://hooks.slack.com/services/YOUR/WEBHOOK/URL
- Check file permissions:
ls -la /var/ossec/integrations/slack*
Required permissions:
slack_
integration.py
: 755 (rwxr-xr-x)slack_
wrapper.sh
: 755 (rwxr-xr-x)slack_config.json
: 644 (rw-r--r--)
- Test Python script directly:
python3 /var/ossec/integrations/slack_integration.py '{"rule":{"id":"test","description":"Direct test","level":5},"agent":{"name":"test"},"timestamp":"2024-01-01","location":"test"}'
- Check Python dependencies:
python3 -c "import json, urllib.request; print('All modules available')"
Common Solutions:
# Fix permissions
chmod 755 /var/ossec/integrations/slack_integration.py
chmod 755 /var/ossec/integrations/slack_wrapper.sh
chmod 644 /var/ossec/integrations/slack_config.json
chown root:wazuh /var/ossec/integrations/slack_integration.py
chown root:root /var/ossec/integrations/slack_wrapper.sh
# Verify webhook URL format
nano /var/ossec/integrations/slack_config.json
# Ensure no extra characters, quotes, or spaces
# Restart Wazuh services
/var/ossec/bin/wazuh-control restart
Issue 2: Email Integration Not Working
Symptom: No emails being received
Detailed Diagnosis:
- Check email integration loading:
grep -i "custom-email" /var/ossec/logs/ossec.log
- Verify Postfix status:
systemctl status postfix
journalctl -u postfix -n 20
- Check authentication configuration:
cat /etc/postfix/sasl_passwd
ls -la /etc/postfix/sasl_passwd*
Verify:
Gmail address is correct
App password has no spaces
Files have 600 permissions
- Test SMTP connectivity:
telnet smtp.gmail.com 587
Should connect successfully. Type quit
to exit.
- Check mail queue:
mailq
postqueue -p
- Test manual email sending:
echo "Subject: Manual Test
From: yourgmail@gmail.com
To: recipient@gmail.com
Manual test email from Wazuh server.
Timestamp: $(date)" | sendmail recipient@gmail.com
- Check Postfix logs:
tail -20 /var/log/maillog
grep "smtp.gmail.com" /var/log/maillog
Common Solutions:
# Fix authentication file
echo "[smtp.gmail.com]:587 yourgmail@gmail.com:yourapppassword" > /etc/postfix/sasl_passwd
postmap /etc/postfix/sasl_passwd
chmod 600 /etc/postfix/sasl_passwd*
# Restart Postfix
systemctl restart postfix
# Clear stuck mail queue
postsuper -d ALL
# Fix integration permissions
chmod 750 /var/ossec/integrations/email_integration
chmod 750 /var/ossec/integrations/custom-email
chown root:wazuh /var/ossec/integrations/email_integration
chown root:wazuh /var/ossec/integrations/custom-email
Issue 3: No High-Level Alerts Generated
Symptom: Only receiving Slack notifications, no emails
Diagnosis:
- Check alert level distribution:
tail -100 /var/ossec/logs/alerts/alerts.log | grep -o '"level":[0-9]*' | cut -d':' -f2 | sort | uniq -c | sort -nr
- Verify email threshold configuration:
grep -A 5 "custom-email" /var/ossec/etc/ossec.conf
Should show <level>7</level>
- Generate forced high-level alerts:
# Multiple failed SSH attempts
for i in {1..5}; do
logger -p auth.warning "sshd[1234]: Failed password for root from 192.168.1.100 port 22 ssh2"
sleep 1
done
# Privilege escalation attempt
logger -p auth.warning "sudo: attacker : user NOT in sudoers ; TTY=pts/1 ; PWD=/home/attacker ; USER=root ; COMMAND=/bin/bash"
# Wait and check
sleep 30
grep -E '"level":[7-9]|"level":1[0-5]' /var/ossec/logs/alerts/alerts.log | tail -3
Issue 4: Permission Denied Errors
Symptom: "Permission denied" or "Command not found" errors
Complete Permission Fix:
# Navigate to integrations directory
cd /var/ossec/integrations
# Fix all permissions at once
chmod 755 slack_integration.py
chmod 755 slack_wrapper.sh
chmod 644 slack_config.json
chmod 750 email_integration
chmod 750 custom-email
# Fix ownership
chown root:wazuh slack_integration.py
chown root:root slack_wrapper.sh
chown root:root slack_config.json
chown root:wazuh email_integration
chown root:wazuh custom-email
# Verify all permissions
echo "=== Permission Verification ==="
stat -c "%a %U:%G %n" slack_*
stat -c "%a %U:%G %n" email_integration
stat -c "%a %U:%G %n" custom-email
# Make scripts executable
chmod +x slack_integration.py
chmod +x slack_wrapper.sh
chmod +x email_integration
chmod +x custom-email
Issue 5: Configuration Not Persistent
Symptom: Configuration lost after container restart
Solution - Create Persistent Volumes:
- Exit container and create backup directory:
exit # Exit from Wazuh container
mkdir -p ./wazuh-persistent-config
- Backup all configuration files:
# Backup integration files
docker cp single-node-wazuh.manager-1:/var/ossec/integrations/ ./wazuh-persistent-config/
# Backup main configuration
docker cp single-node-wazuh.manager-1:/var/ossec/etc/ossec.conf ./wazuh-persistent-config/
# Backup email configuration
docker cp single-node-wazuh.manager-1:/etc/postfix/ ./wazuh-persistent-config/
- Create restoration script:
cat > restore-wazuh-config.sh << 'EOF'
#!/bin/bash
echo "๐ Restoring Wazuh integration configuration..."
# Restore integration files
docker cp ./wazuh-persistent-config/integrations/ single-node-wazuh.manager-1:/var/ossec/
# Restore main configuration
docker cp ./wazuh-persistent-config/ossec.conf single-node-wazuh.manager-1:/var/ossec/etc/
# Restore email configuration
docker cp ./wazuh-persistent-config/postfix/ single-node-wazuh.manager-1:/etc/
# Fix permissions
docker exec single-node-wazuh.manager-1 bash -c "
cd /var/ossec/integrations
chmod 755 slack_integration.py slack_wrapper.sh
chmod 644 slack_config.json
chmod 750 email_integration custom-email
chown root:wazuh slack_integration.py email_integration custom-email
chown root:root slack_wrapper.sh slack_config.json
"
# Restart services
docker exec single-node-wazuh.manager-1 /var/ossec/bin/wazuh-control restart
docker exec single-node-wazuh.manager-1 systemctl restart postfix
echo "โ
Configuration restored successfully!"
EOF
chmod +x restore-wazuh-config.sh
- Add to docker-compose.yml for permanent persistence:
version: '3.8'
services:
wazuh.manager:
# ... existing configuration ...
volumes:
# ... existing volumes ...
# Add these lines for persistent integration configuration
- ./wazuh-persistent-config/integrations:/var/ossec/integrations
- ./wazuh-persistent-config/ossec.conf:/var/ossec/etc/ossec.conf
- ./wazuh-persistent-config/postfix:/etc/postfix
Part 9: Performance Optimization and Production Setup
Step 9.1: Alert Frequency Management
To prevent notification overload in production:
# Edit Wazuh configuration
nano /var/ossec/etc/ossec.conf
# Modify integration blocks for production use:
<!-- Production-Optimized Slack Integration -->
<integration>
<name>custom-slack</name>
<hook_url>/var/ossec/integrations/slack_wrapper.sh</hook_url>
<alert_format>json</alert_format>
<level>3</level>
<max_log>10</max_log>
</integration>
<!-- Production-Optimized Email Integration -->
<integration>
<name>custom-email</name>
<hook_url>/var/ossec/integrations/custom-email</hook_url>
<alert_format>json</alert_format>
<level>8</level>
<max_log>5</max_log>
</integration>
Configuration explanation:
level
: Minimum alert level (raised from 1 to 3 for Slack, 7 to 8 for email)max_log
: Maximum alerts per time period to prevent spam
Step 9.2: Log Rotation Setup
# Create log rotation configuration
cat > /etc/logrotate.d/wazuh-integrations << 'EOF'
/var/ossec/logs/slack_integration.log
/var/ossec/logs/slack_wrapper.log
/var/ossec/logs/email_integration.log
/var/ossec/logs/email_wrapper.log {
daily
rotate 7
compress
delaycompress
missingok
notifempty
create 664 ossec ossec
postrotate
# Signal processes to reopen log files if needed
/bin/kill -HUP `cat /var/ossec/var/run/wazuh-logcollector.pid 2>/dev/null` 2>/dev/null || true
endscript
}
EOF
# Test log rotation
logrotate -d /etc/logrotate.d/wazuh-integrations
Step 9.3: Health Monitoring Script
cat > /var/ossec/bin/integration_health_monitor.sh << 'EOF'
#!/bin/bash
# Wazuh Integration Health Monitor
# Run this script daily to monitor integration health
LOG_FILE="/var/ossec/logs/health_monitor.log"
TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
echo "========================================" >> $LOG_FILE
echo "Wazuh Integration Health Check: $TIMESTAMP" >> $LOG_FILE
echo "========================================" >> $LOG_FILE
# Check Wazuh service status
echo "1. Wazuh Services:" >> $LOG_FILE
/var/ossec/bin/wazuh-control status >> $LOG_FILE 2>&1
# Check Postfix status
echo -e "\n2. Email Service Status:" >> $LOG_FILE
systemctl status postfix --no-pager >> $LOG_FILE 2>&1
# Check integration file integrity
echo -e "\n3. Integration Files:" >> $LOG_FILE
ls -la /var/ossec/integrations/slack* /var/ossec/integrations/*email* >> $LOG_FILE 2>&1
# Check recent activity
echo -e "\n4. Recent Slack Activity (last 5):" >> $LOG_FILE
tail -5 /var/ossec/logs/slack_wrapper.log >> $LOG_FILE 2>&1
echo -e "\n5. Recent Email Activity (last 5):" >> $LOG_FILE
tail -5 /var/ossec/logs/email_integration.log >> $LOG_FILE 2>&1
# Check mail queue
echo -e "\n6. Mail Queue Status:" >> $LOG_FILE
mailq >> $LOG_FILE 2>&1
# Check for stuck emails
QUEUE_COUNT=$(mailq | grep -c "^[A-F0-9]")
if [ $QUEUE_COUNT -gt 0 ]; then
echo -e "\nโ ๏ธ WARNING: $QUEUE_COUNT emails stuck in queue!" >> $LOG_FILE
fi
# Check alert levels from last hour
echo -e "\n7. Alert Level Distribution (last hour):" >> $LOG_FILE
find /var/ossec/logs/alerts -name "alerts.log" -newermt "1 hour ago" -exec grep -o '"level":[0-9]*' {} \; | cut -d':' -f2 | sort | uniq -c | sort -nr >> $LOG_FILE 2>&1
echo -e "\nHealth check completed: $TIMESTAMP" >> $LOG_FILE
echo "========================================" >> $LOG_FILE
EOF
chmod +x /var/ossec/bin/integration_health_monitor.sh
# Create daily cron job
echo "0 9 * * * root /var/ossec/bin/integration_health_monitor.sh" >> /etc/crontab
Step 9.4: Custom Alert Rules for Better Detection
cat > /var/ossec/etc/rules/local_rules.xml << 'EOF'
<!-- Local Custom Rules for Enhanced Security Detection -->
<group name="local,syslog,">
<!-- Enhanced Authentication Monitoring -->
<rule id="100001" level="5">
<if_sid>5712</if_sid>
<description>Multiple SSH authentication failures detected</description>
<group>authentication_failed,pci_dss_10.2.4,pci_dss_10.2.5,</group>
</rule>
<rule id="100002" level="8" frequency="5" timeframe="300">
<if_matched_sid>100001</if_matched_sid>
<description>SSH brute force attack detected (5+ failures in 5 minutes)</description>
<group>authentication_failures,attack,pci_dss_11.4,</group>
</rule>
<!-- Privilege Escalation Detection -->
<rule id="100003" level="8">
<if_sid>5401</if_sid>
<match>sudo</match>
<regex>user NOT in sudoers</regex>
<description>Unauthorized sudo attempt detected</description>
<group>privilege_escalation,pci_dss_10.2.5,</group>
</rule>
<!-- File Integrity Monitoring -->
<rule id="100004" level="7">
<if_sid>554</if_sid>
<match>/etc/passwd|/etc/shadow|/etc/sudoers|/etc/hosts</match>
<description>Critical system file modification detected</description>
<group>file_integrity,system_audit,pci_dss_11.5,</group>
</rule>
<!-- Network Anomaly Detection -->
<rule id="100005" level="6">
<if_sid>4502</if_sid>
<regex>unusual.*connection|suspicious.*traffic</regex>
<description>Unusual network activity pattern detected</description>
<group>network_anomaly,ids,</group>
</rule>
<!-- Malware Detection -->
<rule id="100006" level="12">
<if_sid>554</if_sid>
<match>malware|virus|trojan|backdoor</match>
<description>Potential malware detected on system</description>
<group>malware,virus,pci_dss_5.1,</group>
</rule>
<!-- Service Monitoring -->
<rule id="100007" level="8">
<if_sid>2502</if_sid>
<match>sshd|httpd|nginx|mysql|postgresql</match>
<regex>stopped|failed|crashed</regex>
<description>Critical service failure detected</description>
<group>service_availability,system_error,</group>
</rule>
</group>
EOF
Part 10: Final Validation and Success Confirmation
Step 10.1: Complete System Test
Run comprehensive test to validate entire setup:
echo "๐ STARTING COMPLETE WAZUH INTEGRATION SYSTEM TEST"
echo "=================================================="
# Test 1: Generate low-level alert (Slack only)
echo "Test 1: Low-level alert (Slack notification only)"
logger -p local0.info "Wazuh integration test: User login detected $(date)"
sleep 5
# Test 2: Generate medium-level alert (Slack notification)
echo "Test 2: Medium-level alert (Slack notification)"
logger -p auth.warning "su: failed login attempt for user admin from pts/1"
sleep 5
# Test 3: Generate high-level alert (Both Slack and Email)
echo "Test 3: High-level alert (Both Slack and Email notifications)"
for i in {1..3}; do
logger -p auth.warning "sshd[2234]: Failed password for root from 203.0.113.100 port 22 ssh2"
sleep 2
done
sleep 10
# Test 4: Generate critical alert
echo "Test 4: Critical security alert (Both platforms with high priority)"
logger -p auth.crit "sudo: hacker : user NOT in sudoers ; TTY=pts/1 ; PWD=/tmp ; USER=root ; COMMAND=/bin/bash"
logger -p local0.crit "File integrity: /etc/passwd modified by unauthorized process"
sleep 15
echo "โณ Waiting for alert processing to complete..."
sleep 30
echo "๐ TESTING RESULTS:"
echo "=================="
# Check alert generation
echo "1. Recent alerts generated:"
tail -10 /var/ossec/logs/alerts/alerts.log | grep -E '"level":[0-9]*' | tail -5
# Check Slack integration
echo -e "\n2. Slack integration results:"
tail -10 /var/ossec/logs/slack_wrapper.log
# Check email integration
echo -e "\n3. Email integration results:"
tail -10 /var/ossec/logs/email_integration.log
# Check mail queue
echo -e "\n4. Email queue status:"
mailq
echo -e "\nโ
SYSTEM TEST COMPLETED"
echo "========================="
echo "Manual verification required:"
echo "- Check your Slack channel for 4 new alert messages"
echo "- Check your Gmail for 2 new security alert emails"
echo "- Verify message formatting and information completeness"
Step 10.2: Success Criteria Checklist
Your Wazuh integration is successful when ALL of these are true:
โ Slack Integration Success:
[ ] Messages appear in Slack channel within 30 seconds of alert generation
[ ] Messages are properly formatted with colors, emojis, and structured fields
[ ] Different alert levels show different priorities (๐ด๐ ๐ก๐ต)
[ ] All alert information is clearly displayed (agent, rule ID, description, timestamp)
[ ] No error messages in
/var/ossec/logs/slack_wrapper.log
โ Email Integration Success:
[ ] Emails arrive in Gmail within 2-3 minutes of high-level alerts
[ ] Emails are properly formatted with headers, priorities, and detailed information
[ ] Only high-level alerts (7+) trigger email notifications
[ ] Email subject lines clearly indicate priority level
[ ] Mail queue is empty (no stuck messages)
[ ] No error messages in
/var/ossec/logs/email_integration.log
โ System Health Success:
[ ] All Wazuh services show "running" status
[ ] Postfix service is active and running
[ ] Integration files have correct permissions and ownership
[ ] No integration errors in
/var/ossec/logs/ossec.log
[ ] Configuration survives container restarts
โ Alert Level Success:
[ ] Low-level alerts (1-6) trigger Slack notifications only
[ ] High-level alerts (7+) trigger both Slack and email notifications
[ ] Critical alerts (10+) are clearly marked with high priority indicators
[ ] Alert frequency controls prevent spam (if configured)
Step 10.3: Production Deployment Checklist
Before moving to production, ensure:
Security:
[ ] Gmail app password is stored securely
[ ] Slack webhook URL is not exposed in logs
[ ] File permissions are correctly set (750/755 for scripts, 600 for credentials)
[ ] Log files are properly rotated and managed
Reliability:
[ ] Configuration is backed up and persistent
[ ] Health monitoring script is scheduled
[ ] Alert thresholds are appropriate for your environment
[ ] Redundancy plans exist if notifications fail
Documentation:
[ ] Team members know how to interpret alerts
[ ] Escalation procedures are documented
[ ] Maintenance procedures are scheduled
[ ] Contact information is current in email templates
Step 10.4: Maintenance Schedule
Daily:
Monitor integration logs for errors
Check mail queue for stuck messages
Verify Slack connectivity
Weekly:
Run health monitoring script
Review alert patterns and adjust thresholds
Check disk space for log files
Monthly:
Test full integration with manual alerts
Review and update Gmail app password if needed
Update team contact information
Review alert rules effectiveness
Quarterly:
Backup complete configuration
Review overall security posture
Update integration scripts if needed
Train team on new procedures
You're absolutely right! I completely forgot to include the dashboard access section. Let me add that crucial part:
Part 11: Accessing and Using Wazuh Dashboard
Step 11.1: Dashboard Login and Setup
Access the Wazuh Dashboard:
Open your web browser and navigate to:
https://localhost:443
Or if accessing remotely:
https://your-server-ip:443
Accept the SSL certificate warning (since it's self-signed):
Click "Advanced" or "Show Details"
Click "Proceed to localhost (unsafe)" or "Accept Risk and Continue"
Login with default credentials:
Username: admin Password: SecretPassword
(These are the default credentials from your docker-compose.yml)
Important: Change these default credentials immediately after first login!
Step 11.2: Initial Dashboard Configuration
After successful login, you'll see the main dashboard:
Dashboard Overview:
Security Events: Real-time security event counter
Alert Level Distribution: Pie chart showing alert severity levels
Top Agents: Most active monitored systems
Alert Evolution: Timeline of security events
Change Default Password:
Click on the user icon (top right corner)
Select "Account Settings" or "Security"
Change the admin password to something secure
Example:
MySecureWazuh2024!
Step 11.3: Navigating Dashboard Sections
Main Dashboard Sections:
Security Events Overview:
Dashboard โ Overview โ Security Events
Shows real-time security events
Alert level distribution
MITRE ATT&CK framework mapping
Agents Management:
Dashboard โ Agents
View all connected agents
Agent status and configuration
Deploy new agents
Security Analytics:
Dashboard โ Security Analytics
Security Events: Detailed event analysis
Integrity Monitoring: File system changes
System Auditing: System-level security events
Vulnerability Detection: CVE analysis
Compliance & Auditing:
Dashboard โ Compliance
PCI DSS compliance monitoring
GDPR compliance tracking
HIPAA compliance reports
SOC 2 compliance metrics
Step 11.4: Monitoring Your Integration Alerts
View Integration-Generated Alerts:
Go to Security Events:
Dashboard โ Security Analytics โ Security Events
Filter for your test alerts:
In the search bar, type:
rule.id:12345
(your test rule ID)Or search for:
agent.name
:test-server-01
Time range: Last 1 hour
Verify Alert Levels:
Look for alerts with different severity levels
Level 1-6: Should appear in dashboard only
Level 7+: Should have triggered both Slack and email
Check Alert Details: Click on any alert to see:
Complete alert information
Rule description
Agent details
Timestamp
Raw log data
Step 11.5: Real-time Monitoring Setup
Set up real-time monitoring dashboards:
Create Custom Dashboard:
Dashboard โ Visualize โ Create Visualization
Add Security Metrics Widgets:
Alert Level Counter: Shows current alert levels
Top Rules Triggered: Most frequent security rules
Geographic Map: Attack sources by location
Timeline: Security events over time
Configure Refresh Interval:
Top right corner: Click the refresh icon
Set to "Auto-refresh every 30 seconds"
This shows real-time security events
Step 11.6: Verifying Integration Status
Check Integration Activity in Dashboard:
Monitor Recent Alerts:
Dashboard โ Security Analytics โ Security Events
Sort by "Most Recent"
Look for your generated test alerts
Verify alert details match your Slack/email notifications
Check Manager Logs:
Dashboard โ Management โ Logs
Select "Manager logs"
Look for integration-related messages:
INFO: Enabling integration for: 'custom-slack' INFO: Enabling integration for: 'custom-email'
Verify Agent Status:
Dashboard โ Agents
All agents should show "Active" status
Check "Last Keep Alive" timestamps
Step 11.7: Dashboard Alert Correlation
Cross-reference dashboard alerts with notifications:
Generate Test Alert:
# In your container or via SSH logger -p auth.warning "sshd[3456]: Failed password for admin from 192.168.1.200 port 22 ssh2"
Check Dashboard (within 30 seconds):
Go to Security Events
Look for the new alert
Note the Rule ID, Level, and Timestamp
Verify Notifications:
Slack: Should show the same alert details
Email: If level 7+, should receive email
Dashboard: Should display in real-time
Step 11.8: Advanced Dashboard Features
Useful Dashboard Features for Integration Monitoring:
Discover Tab:
Dashboard โ Discover
Raw log analysis
Custom field searches
Export alert data
Reporting:
Dashboard โ Management โ Reporting
Schedule automated reports
Email compliance reports
PDF security summaries
API Explorer:
Dashboard โ Dev Tools โ API Console
Test Wazuh API calls
Debug integration issues
Custom automation scripts
Step 11.9: Dashboard-Based Troubleshooting
Use dashboard to troubleshoot integration issues:
No Alerts Appearing:
Dashboard โ Management โ Configuration
Check if integrations are enabled
Verify configuration syntax
Missing High-Level Alerts:
Dashboard โ Security Events โ Advanced Filters
Filter by:
rule.level:>=7
Check if high-level alerts are being generated
Agent Connectivity Issues:
Dashboard โ Agents โ Agent Details
Check agent status
View agent configuration
Monitor agent logs
Step 11.10: Dashboard Best Practices
Optimize dashboard for security operations:
Create Dedicated SOC Dashboard:
Pin important visualizations
Set up alert thresholds
Configure automatic refreshes
Set Up Dashboard Alerts:
Dashboard โ Management โ Watcher
Create dashboard-based alerts
Set thresholds for critical events
Configure dashboard notifications
Monitor Dashboard Performance:
Keep dashboard queries optimized
Limit time ranges for better performance
Use filters to reduce data load
Step 11.11: Dashboard Integration Verification
Complete integration verification using dashboard:
# Generate comprehensive test
logger -p auth.warning "sshd[4567]: Failed password for root from 203.0.113.100 port 22 ssh2"
logger -p auth.warning "sshd[4567]: Failed password for root from 203.0.113.100 port 22 ssh2"
logger -p auth.warning "sshd[4567]: Failed password for root from 203.0.113.100 port 22 ssh2"
Then verify in dashboard:
Check Alert Appears (within 1 minute):
Dashboard โ Security Events
Look for new SSH authentication failure
Note the alert level and rule ID
Verify Alert Details:
Click on the alert
Check all fields are populated correctly
Verify source IP, agent name, timestamp
Cross-Reference with Notifications:
Dashboard Alert Time: Note exact timestamp
Slack Message Time: Should match within seconds
Email Received Time: Should match within minutes (if level 7+)
Step 11.12: Dashboard Customization for Integration
Customize dashboard for better integration monitoring:
Create Integration Status Widget:
Dashboard โ Visualize โ Create New Visualization
Widget Type: "Metric"
Query: Count of alerts per integration
Display: Integration success rate
Add Alert Level Distribution:
Widget Type: "Pie Chart"
Buckets: Split by alert level
Shows which levels trigger notifications
Create Notification Timeline:
Widget Type: "Line Chart"
X-axis: Timestamp
Y-axis: Alert count
Split: By notification channel (Slack/Email)
Expected Dashboard Results:
โ Dashboard Access: Successfully login and navigate โ Real-time Alerts: See security events appear within 30 seconds โ Alert Correlation: Dashboard alerts match Slack/email notifications โ Integration Status: No integration errors in manager logs โ Custom Dashboards: Monitoring widgets show integration activity
Dashboard URLs for Quick Access:
Main Dashboard:
https://localhost:443/app/wazuh#/overview
Security Events:
https://localhost:443/app/wazuh#/security-events
Manager Logs:
https://localhost:443/app/wazuh#/manager/logs
This dashboard integration allows you to visually confirm that your Slack and email notifications are working correctly and provides a centralized view of all security events alongside your notification channels!
Conclusion
๐ Congratulations! You have successfully implemented a comprehensive, enterprise-grade Wazuh SIEM notification system!
What you've accomplished:
๐ก๏ธ Complete Security Monitoring Integration:
Real-time Slack notifications for immediate awareness
Detailed Gmail alerts for critical incident documentation
Multi-tier alerting strategy preventing notification fatigue
Production-ready error handling and logging
๐ง Technical Excellence:
Custom Python integration with advanced formatting
Robust bash scripting with comprehensive error handling
Secure SMTP configuration with Gmail integration
Persistent configuration surviving system updates
๐ Operational Benefits:
24/7 automated security monitoring
Immediate threat notification across multiple channels
Detailed audit trails for compliance requirements
Scalable architecture supporting infrastructure growth
๐ Production Readiness:
Health monitoring and alerting
Log rotation and maintenance automation
Backup and restoration procedures
Performance optimization controls
Our Wazuh SIEM now provides enterprise-level security operations capabilities with intelligent, multi-channel alerting that ensures no critical security event goes unnoticed while maintaining operational efficiency through smart filtering and prioritization.
This implementation represents a complete, production-ready security monitoring solution that will serve as the foundation for organization's security operations center (SOC) activities.
Subscribe to my newsletter
Read articles from Sarun directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by
