Boost Productivity: Top 5 Python Scripts for a Streamlined Workflow
Here's a document outlining five production-level Python automation scripts that can enhance efficiency and reliability in various operational tasks. Each script is accompanied by a brief description, its purpose, and an example.
Production-Level Python Automation Scripts
1. Automated Database Backup Script
Description
This script automates the process of backing up a database at regular intervals, ensuring that data is securely stored and can be restored if necessary.
Purpose
To protect data by creating regular backups of the database and storing them in a specified location.
Example
import os
import time
import subprocess
from datetime import datetime
def backup_database(db_name, backup_dir):
timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
backup_file = os.path.join(backup_dir, f"{db_name}_{timestamp}.sql")
command = f"mysqldump -u username -p password {db_name} > {backup_file}"
try:
subprocess.run(command, shell=True, check=True)
print(f"Backup successful: {backup_file}")
except subprocess.CalledProcessError as e:
print(f"Error during backup: {e}")
if __name__ == "__main__":
while True:
backup_database('my_database', '/path/to/backup')
time.sleep(86400) # Wait for 24 hours
2. Automated Log Monitoring Script
Description
This script monitors log files for specific patterns (like error messages) and sends alerts when such patterns are detected.
Purpose
To proactively identify issues in the system by monitoring logs and alerting the relevant personnel.
Example
import time
import re
import smtplib
from email.mime.text import MIMEText
LOG_FILE = '/path/to/logfile.log'
ERROR_PATTERN = r'ERROR'
ALERT_EMAIL = 'alert@example.com'
def send_alert(message):
msg = MIMEText(message)
msg['Subject'] = 'Log Alert'
msg['From'] = 'monitor@example.com'
msg['To'] = ALERT_EMAIL
with smtplib.SMTP('smtp.example.com') as server:
server.sendmail(msg['From'], [msg['To']], msg.as_string())
def monitor_logs():
with open(LOG_FILE, 'r') as file:
file.seek(0, 2) # Move to the end of the file
while True:
line = file.readline()
if not line:
time.sleep(1)
continue
if re.search(ERROR_PATTERN, line):
send_alert(f"Detected error: {line.strip()}")
if __name__ == "__main__":
monitor_logs()
3. Automated File Synchronization Script
Description
This script synchronizes files between a local directory and a remote server, ensuring that the latest versions are always available.
Purpose
To keep files in sync between different environments, reducing the risk of discrepancies.
Example
import os
import subprocess
import time
LOCAL_DIR = '/path/to/local/dir'
REMOTE_DIR = 'user@remote_host:/path/to/remote/dir'
def sync_files():
command = f"rsync -avz {LOCAL_DIR}/ {REMOTE_DIR}"
try:
subprocess.run(command, shell=True, check=True)
print("Files synchronized successfully.")
except subprocess.CalledProcessError as e:
print(f"Error during synchronization: {e}")
if __name__ == "__main__":
while True:
sync_files()
time.sleep(3600) # Sync every hour
4. Automated Trivy Vulnerability Scanning Script
Description
This script automates the process of scanning Docker images for vulnerabilities using Trivy and logs the results.
Purpose
To ensure that Docker images are free from known vulnerabilities before deployment.
Example
import subprocess
import json
def run_trivy_scan(image_name):
command = f"trivy image --format json {image_name}"
result = subprocess.run(command, shell=True, capture_output=True, text=True)
return json.loads(result.stdout)
def parse_and_log_results(results):
with open('trivy_results.log', 'a') as log_file:
for result in results.get('Results', []):
for vulnerability in result.get('Vulnerabilities', []):
log_file.write(f"{vulnerability['PkgName']} - {vulnerability['Severity']}\n")
if __name__ == "__main__":
image_name = 'your_docker_image:latest'
results = run_trivy_scan(image_name)
parse_and_log_results(results)
5. Automated Email Reporting Script
Description
This script collects system metrics (like CPU and memory usage) and sends a daily email report to the system administrator.
Purpose
To keep system administrators informed about the health and performance of the systems.
Example
import psutil
import smtplib
from email.mime.text import MIMEText
def gather_system_metrics():
cpu_usage = psutil.cpu_percent()
memory = psutil.virtual_memory()
return f"CPU Usage: {cpu_usage}%\nMemory Usage: {memory.percent}%"
def send_report(report):
msg = MIMEText(report)
msg['Subject'] = 'Daily System Report'
msg['From'] = 'monitor@example.com'
msg['To'] = 'admin@example.com'
with smtplib.SMTP('smtp.example.com') as server:
server.sendmail(msg['From'], [msg['To']], msg.as_string())
if __name__ == "__main__":
report = gather_system_metrics()
send_report(report)
Conclusion
These five production-level Python automation scripts demonstrate how automation can streamline operations, enhance security, and improve system monitoring. Organizations can customize these scripts further to suit their specific needs and integrate them into their existing workflows.
Subscribe to my newsletter
Read articles from Siddhant Gahlot directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by
Siddhant Gahlot
Siddhant Gahlot
A dedicated DevOps and Cloud Enthusiast with a strong focus on automating and optimizing infrastructure and deployment processes. Experienced in implementing CI/CD pipelines, managing cloud resources, and enhancing operational efficiency. Committed to leveraging modern practices and solutions to build robust, scalable, and reliable systems. Continuously exploring new technologies and driving innovation in the dynamic field of DevOps and cloud computing.