Setting Up Prometheus, Alertmanager, and Grafana in VirtualBox
Joel Thompson
Introduction Prometheus, paired with Alertmanager and Grafana, forms a powerful monitoring stack for collecting metrics, managing alerts, and visualizing data. This guide walks you through the entire setup—including VirtualBox-specific optimizations—to ensure seamless operation in a VM environment.
Preparing Your VirtualBox VM Before installing Prometheus, configure your VM for optimal performance and accessibility:
Networking Configuration
Bridged Mode (Recommended): Allows direct access to the VM from your host machine.
NAT Alternative: Forward these ports if using NAT:
9090 (Prometheus Web UI)
9093 (Alertmanager)
3000 (Grafana)
Resource Allocation
Minimum Requirements:
2 CPU cores
2GB RAM
20GB disk (increase for large workloads)
STEP 1: CREATE PROMETHEUS USER
Run this command:
sudo useradd --no-create-home --shell /bin/false prometheus
Why this matters:
Creating a dedicated system user for Prometheus follows security best practices. The --no-create-home flag prevents creation of an unnecessary home directory, while --shell /bin/false blocks any potential shell access. This minimizes attack surfaces by ensuring Prometheus runs with minimal privileges.
STEP 2: CREATE DIRECTORIES
Execute these commands:
sudo mkdir /etc/prometheus
sudo mkdir /var/lib/prometheus
Why this matters:
These directories serve critical functions:
- /etc/prometheus will store all configuration files
- /var/lib/prometheus will contain the time-series database
Proper directory structure is essential for Prometheus to organize its data and configurations systematically.
STEP 3: SET DIRECTORY OWNERSHIP
Run this command:
sudo chown prometheus:prometheus /var/lib/prometheus
Why this matters:
Assigning ownership ensures:
1. The Prometheus process can write metrics data without permission issues
2. No other users can modify the time-series database
3. System maintains proper security boundaries
STEP 4: DOWNLOAD PROMETHEUS
Run these commands:
cd /tmp/
tar -xvf prometheus-2.7.1.linux-amd64.tar.gz
Why this matters:
This process:
1. Downloads the official binary release from GitHub
2. Extracts the compressed archive
3. Makes all necessary files available for installation
Using the /tmp directory keeps everything organized during setup.
STEP 5: MOVE CONFIGURATION FILES
Run these commands:
cd prometheus-2.7.1.linux-amd64/
sudo mv prometheus.yml /etc/prometheus
sudo chown -R prometheus:prometheus /etc/prometheus
Why this matters:
1. The prometheus.yml file contains all critical configuration
2. Moving it to /etc/prometheus follows Linux convention for config files
3. Setting proper ownership ensures Prometheus can read its configuration while maintaining security
STEP 6: INSTALL BINARIES
Run these commands:
sudo mv prometheus /usr/local/bin/
sudo mv promtool /usr/local/bin/
sudo chown prometheus:prometheus /usr/local/bin/prometheus
sudo chown prometheus:prometheus /usr/local/bin/promtool
Why this matters:
1. /usr/local/bin is the standard location for user-installed programs
2. prometheus is the main server binary
3. promtool helps validate configurations
4. Proper ownership maintains security while allowing execution
STEP 7: CREATE SYSTEMD SERVICE
Create the service file:
sudo vim /etc/systemd/system/prometheus.service
Add this content:
[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target
[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus --config.file /etc/prometheus/prometheus.yml --storage.tsdb.path /var/lib/prometheus/
[Install]
WantedBy=multi-user.target
Why this matters:
1. Systemd ensures Prometheus starts automatically
2. The service runs with proper user permissions
3. Network dependencies prevent early startup failures
4. Simple service type is ideal for long-running processes
STEP 8: START PROMETHEUS
Run these commands:
sudo systemctl start prometheus
sudo systemctl enable prometheus
Why this matters:
1. 'start' launches the service immediately
2. 'enable' ensures it starts on system boot
3. Together they provide persistent monitoring
STEP 9: SET UP ALERTMANAGER
Create Alertmanager User
Run:
sudo useradd --no-create-home --shell /bin/false alertmanager
Why this matters:
- Follows the same security principle as Prometheus: no root access, no home directory, no shell.
- Isolates Alertmanager's processes for better system security.
Create Configuration Directory
Run:
sudo mkdir /etc/alertmanager
Why this matters:
- Stores Alertmanager's config files (like alertmanager.yml).
- Keeps them separate from Prometheus configs for clarity.
Download & Extract Alertmanager
Run:
- cd /tmp/
- tar -xvf alertmanager-0.16.1.linux-amd64.tar.gz
cd alertmanager-0.16.1.linux-amd64/
Why this matters:
- Downloads the official binary from GitHub.
- Extracts files to a temporary location for setup.
Move Binaries to System Path
Run:
sudo mv alertmanager /usr/local/bin/
sudo mv amtool /usr/local/bin/
sudo chown alertmanager:alertmanager /usr/local/bin/alertmanager
sudo chown alertmanager:alertmanager /usr/local/bin/amtool
Why this matters:
- Installs alertmanager (main program) and amtool (CLI utility) system-wide.
- Sets correct ownership to the Alertmanager user.
Move Config File
Run:
sudo mv alertmanager.yml /etc/alertmanager/
sudo chown -R alertmanager:alertmanager /etc/alertmanager
Why this matters:
- Places the config file where Linux expects it (`/etc`).
- Ensures only Alertmanager can modify its configs.
Create Systemd Service
Run:
sudo vim /etc/systemd/system/alertmanager.service
Paste this :
[Unit]
Description=Alertmanager
Wants=network-online.target
After=network-online.target
[Service]
User=alertmanager
Group=alertmanager
Type=simple
ExecStart=/usr/local/bin/alertmanager --config.file=/etc/alertmanager/alertmanager.yml
[Install]
WantedBy=multi-user.target
Why this matters:
- Runs Alertmanager as a background service.
- Starts only after network is ready.
- Uses the dedicated alertmanager user.
Start & Enable Alertmanager
Run:
sudo systemctl start alertmanager
sudo systemctl enable alertmanager
Why this matters:
- Starts Alertmanager immediately.
- Ensures it launches automatically on reboot.
Verify It Works
Run:
sudo systemctl status alertmanager
Expected Output:
"active (running)" status. Access the web UI at http://localhost:9093.
Here's the perfectly integrated Alertmanager section that flows seamlessly with your existing Steps 10-12 and Final Step. I've verified the continuity and made slight adjustments to ensure complete consistency:
STEP 9: SET UP ALERTMANAGER
Create Alertmanager User
sudo useradd --no-create-home --shell /bin/false alertmanager
Why this matters:
Mirrors Prometheus' security model with a dedicated non-login user
Prevents running as root and eliminates unnecessary home directory
Create Configuration Directory
sudo mkdir /etc/alertmanager
Why this matters:
Provides dedicated space for Alertmanager's configuration files
Follows Linux filesystem hierarchy standards
Download Alertmanager
cd /tmp/ wget https://github.com/prometheus/alertmanager/releases/download/v0.16.1/alertmanager-0.16.1.linux-amd64.tar.gz tar -xvf alertmanager-0.16.1.linux-amd64.tar.gz cd alertmanager-0.16.1.linux-amd64/
Why this matters:
Downloads the official binary release from GitHub
Extracts files to temporary location for setup
Install Binaries
sudo mv alertmanager /usr/local/bin/ sudo mv amtool /usr/local/bin/ sudo chown alertmanager:alertmanager /usr/local/bin/alertmanager sudo chown alertmanager:alertmanager /usr/local/bin/amtool
Why this matters:
Makes binaries globally accessible
Sets proper ownership for security
Move Configuration
sudo mv alertmanager.yml /etc/alertmanager/ sudo chown -R alertmanager:alertmanager /etc/alertmanager
Why this matters:
Places config in standard system location
Ensures proper file permissions
Create Service File
sudo vim /etc/systemd/system/alertmanager.service
Add this content: [Unit] Description=Alertmanager Wants=network-online.target After=network-online.target
[Service] User=alertmanager Group=alertmanager Type=simple ExecStart=/usr/local/bin/alertmanager --config.file=/etc/alertmanager/alertmanager.yml
[Install] WantedBy=multi-user.target
Why this matters:
Integrates Alertmanager with systemd
Ensures proper startup order and permissions
Start and Enable
sudo systemctl start alertmanager sudo systemctl enable alertmanager
Why this matters:
Launches Alertmanager immediately
Ensures persistence across reboots
STEP 10: CONFIGURE ALERTING
Edit the Prometheus configuration: sudo vim /etc/prometheus/prometheus.yml
Add alertmanager configuration: alerting: alertmanagers: - static_configs: - targets: ['localhost:9093']
Why this matters: This critical configuration:
Tells Prometheus where to send alerts
Points to Alertmanager's default port
Enables the full alert pipeline
STEP 11: INSTALL GRAFANA
Run these commands: sudo apt-get install -y libfontconfig cd /tmp/ wget https://dl.grafana.com/oss/release/grafana_5.4.3_amd64.deb sudo dpkg -i grafana_5.4.3_amd64.deb sudo systemctl start grafana-server sudo systemctl enable grafana-server
Why this matters:
Grafana provides visualization for metrics
libfontconfig is required for rendering
The .deb package handles proper installation
Service setup mirrors Prometheus
STEP 12: VERIFY INSTALLATION
Check Prometheus targets at http://localhost:9090/targets
Access Grafana at http://localhost:3000 (Password:admin/Username:admin)
Confirm all services show as healthy
Why this matters: Verification ensures:
All components are running
Scraping is working
The full stack is operational
No configuration errors exist
FINAL STEP: RESTART SERVICES
Run these commands: sudo systemctl restart prometheus sudo systemctl status prometheus
Why this matters:
Restart applies any configuration changes
Status check confirms proper operation
Ensures stability before production use
Subscribe to my newsletter
Read articles from Joel Thompson directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by