Master SSH Access: Secure Your AWS Cloud

Introduction

In the world of server administration and cloud computing, secure remote access is of paramount importance. This blog post guides you through the entire process of setting up secure SSH (Secure Shell) access to a remote Linux server on AWS, using multiple SSH key pairs for authentication, and implementing additional security measures like fail2ban to protect against brute force attacks.

By the end of this guide, you'll have a comprehensive understanding of SSH key management, server configuration, and security best practices that you can apply to your own projects.

Understanding SSH from First Principles

What is SSH?

SSH (Secure Shell) is a cryptographic network protocol that enables secure communication between two computers over an unsecured network. Unlike its predecessors like Telnet, SSH encrypts all traffic, protecting against eavesdropping and man-in-the-middle attacks.

The Cryptographic Foundation of SSH

SSH security is built on public-key cryptography. This system uses a pair of keys:

Private Key: Kept secret on your local machine
Public Key: Shared with remote servers

These keys work together through asymmetric encryption:

Messages encrypted with the public key can only be decrypted with the corresponding private key
The private key is used to generate digital signatures that can be verified with the public key

This creates a secure system where:

The server knows it's talking to the authorized client (authentication)
All communication is encrypted (confidentiality)
Messages cannot be altered in transit (integrity)

sequenceDiagram
    Client->>Server: 1. Connection Request
    Server->>Client: 2. Server Identity
    Client->>Server: 3. Key Exchange
    Client->>Server: 4. Authentication with Private Key
    Server->>Client: 5. Authentication Verification
    Client->>Server: 6. Encrypted Session Begins

Project Overview: Remote Server Setup with Multiple SSH Keys

Let's implement these concepts by setting up a remote Linux server on AWS with secure SSH access using two separate SSH key pairs. This approach demonstrates how you can manage different authentication credentials for the same server.

Our Architecture

graph TD
    subgraph "AWS Cloud"
        EC2["Amazon Linux EC2 Instance"]
        SG["Security Group"]
        Auth["~/.ssh/authorized_keys"]
        F2B["fail2ban"]
    end

    subgraph "Local Machine"
        Key1["SSH Key Pair 1"]
        Key2["SSH Key Pair 2"] 
        Config["SSH Config File"]
        Client["SSH Client"]
    end

    SG -->|"Allows Port 22"| EC2
    Key1 -->|"Public Key"| Auth
    Key2 -->|"Public Key"| Auth
    Auth -->|"Authenticates"| EC2
    Client -->|"SSH Connection"| EC2
    Config -->|"Configures"| Client
    F2B -->|"Protects"| EC2

    style EC2 fill:#f9f,stroke:#333,stroke-width:2px
    style SG fill:#bbf,stroke:#333,stroke-width:1px
    style Key1 fill:#bfb,stroke:#333,stroke-width:1px
    style Key2 fill:#bfb,stroke:#333,stroke-width:1px
    style F2B fill:#f66,stroke:#333,stroke-width:1px

Step 1: Provisioning the AWS EC2 Instance

The first step is to create your virtual server in the AWS cloud. EC2 (Elastic Compute Cloud) provides scalable computing capacity in the AWS cloud.

sequenceDiagram
    participant User
    participant AWS as AWS Console
    participant EC2 as EC2 Instance

    User->>AWS: Create EC2 Instance
    AWS->>EC2: Launch Amazon Linux
    AWS->>EC2: Configure Security Group
    AWS->>User: Provide Initial Key Pair
    User->>EC2: Initial SSH Connection
    Note over User,EC2: Using AWS-provided key pair

Step-by-Step EC2 Setup:

Log in to your AWS Console and navigate to the EC2 dashboard
Launch a new instance with the following specifications:
- AMI: Amazon Linux (a Linux distribution optimized for AWS)
- Instance Type: t2.micro (free tier eligible)
- Security Group: Create a new one with port 22 (SSH) open
- Key Pair: Create or select an existing key pair for initial access

Connect to your instance using the AWS-provided key:

ssh -i ~/.ssh/aws_key.pem ec2-user@your-instance-ip

💡 Why Amazon Linux? Amazon Linux is optimized for AWS, includes AWS tools by default, and receives regular security updates directly from Amazon. This makes it an excellent choice for AWS-hosted servers.

Step 2: Understanding and Generating SSH Key Pairs

SSH key pairs are the cryptographic credentials that allow secure authentication without passwords. Let's generate two separate key pairs for our server:

graph LR
    A["ssh-keygen command"] --> B["~/.ssh/my_first_key (Private)"]
    A --> C["~/.ssh/my_first_key.pub (Public)"]
    A --> D["~/.ssh/my_second_key (Private)"]
    A --> E["~/.ssh/my_second_key.pub (Public)"]

    style B fill:#f96,stroke:#333
    style D fill:#f96,stroke:#333
    style C fill:#9f6,stroke:#333
    style E fill:#9f6,stroke:#333

Generating SSH Keys from First Principles:

The ssh-keygen utility creates a mathematical key pair using public key cryptography algorithms. When generating keys, consider:

Key Type: RSA is widely supported, but newer algorithms like Ed25519 offer better security with smaller key sizes
Key Size: For RSA keys, 4096 bits provides strong security
Passphrase: An optional extra layer of security that encrypts your private key

Creating Our Keys:

# Generate first key pair
ssh-keygen -t rsa -b 4096 -f ~/.ssh/my_first_key -C "first-key"

# Generate second key pair
ssh-keygen -t rsa -b 4096 -f ~/.ssh/my_second_key -C "second-key"

Each command creates:

A private key file (my_first_key or my_second_key)
A public key file with .pub extension

⚠️ Security Alert: Your private key files should NEVER be shared with anyone or committed to repositories. They should have permissions set to 600 (readable only by your user).

Step 3: Server-Side SSH Configuration

Now we'll configure our remote server to accept both SSH key pairs for authentication.

graph LR
    A["Local: Public Keys"] -->|"Copy to Server"| B["Server: ~/.ssh/authorized_keys"]
    B -->|"Permissions: 600"| C["SSH Server"]

    subgraph "Server Configuration"
        B
        D["~/.ssh directory<br/>Permissions: 700"]
    end

    style A fill:#bfb,stroke:#333
    style B fill:#bbf,stroke:#333
    style C fill:#f9f,stroke:#333
    style D fill:#bbf,stroke:#333

Understanding authorized_keys from First Principles:

The authorized_keys file contains a list of public keys that are allowed to authenticate. When an SSH client tries to connect:

The server reads authorized_keys
The client proves it has the corresponding private key
If proven, access is granted without a password

Adding Our Public Keys:

Display your public keys on your local machine:

cat ~/.ssh/my_first_key.pub
cat ~/.ssh/my_second_key.pub

Add to authorized_keys on the server:

# On your remote server
nano ~/.ssh/authorized_keys

# Paste both public keys, each on its own line
# Save and exit (Ctrl+X, Y, Enter in nano)

Set proper permissions:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

💡 Why these permissions? The SSH daemon is highly security-conscious and will refuse to use keys if the permissions are too open. These permission settings ensure only the owner can read or modify the keys.

Step 4: Configuring Your Local SSH Client

To simplify connections, we'll create an SSH config file that specifies connection details and key locations.

graph LR
    A["~/.ssh/config file"] -->|"Contains"| B["Host alias configuration"]
    B -->|"Specifies"| C["Connection details"]
    C -->|"Includes"| D["HostName (IP)"]
    C -->|"Includes"| E["User"]
    C -->|"Includes"| F["IdentityFile paths"]

    G["ssh roadmapsh-test-server command"] -->|"Uses"| A
    G -->|"Connects to"| H["Remote Server"]

    style A fill:#bbf,stroke:#333
    style G fill:#bfb,stroke:#333
    style H fill:#f9f,stroke:#333

SSH Config from First Principles:

The SSH config file allows you to:

Create shortcuts for complex connection commands
Specify different keys for different servers
Set server-specific options

Creating the Config File:

# On your local machine
nano ~/.ssh/config

Add these lines:

Host roadmapsh-test-server
    HostName your-instance-ip
    User ec2-user
    IdentityFile ~/.ssh/my_first_key
    IdentityFile ~/.ssh/my_second_key

Save and exit. Now you can connect with:

ssh roadmapsh-test-server

SSH will automatically try each key in order until one works.

💡 Pro Tip: You can include additional options like ServerAliveInterval 60 to keep connections alive or Compression yes to speed up connections over slow networks.

Step 5: Enhancing Security with fail2ban

fail2ban is a security tool that monitors log files and automatically blocks IP addresses that show malicious signs, such as multiple failed login attempts.

flowchart TD
    A["Server Logs"] -->|"Monitored by"| B["fail2ban"]
    B -->|"Detects"| C["Suspicious Activity"]
    C -->|"Triggers"| D["Ban Action"]
    D -->|"Updates"| E["Firewall Rules"]
    E -->|"Blocks"| F["Attacking IP"]

    style A fill:#bbf,stroke:#333
    style B fill:#f9f,stroke:#333
    style C fill:#f66,stroke:#333
    style F fill:#f66,stroke:#333

How fail2ban Works from First Principles:

fail2ban:

Constantly monitors log files like /var/log/secure
Uses regular expressions to detect patterns like failed login attempts
Maintains counters for each IP address
When thresholds are exceeded, it updates firewall rules to block the IP
After a configurable ban time, it removes the block

Installing and Configuring fail2ban:

Install fail2ban:

sudo yum update -y
sudo yum install fail2ban -y

Create a custom configuration:

sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo nano /etc/fail2ban/jail.local

Configure the SSH jail by ensuring these settings:

[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/secure
maxretry = 3
bantime = 3600  # 1 hour in seconds

Start and enable fail2ban:

sudo systemctl start fail2ban
sudo systemctl enable fail2ban

Verify it's working:
```
sudo fail2ban-client status sshd
```

💡 Understanding fail2ban jails: Each "jail" is a separate configuration for a specific service (like SSH). You can have different settings for different services, allowing fine-grained control over security policies.

Testing Our Setup

Let's verify that our setup works correctly:

sequenceDiagram
    participant User
    participant LocalSSH as Local SSH Client
    participant RemoteSSH as Remote SSH Server
    participant F2B as fail2ban

    User->>LocalSSH: ssh -i ~/.ssh/my_first_key ec2-user@IP
    LocalSSH->>RemoteSSH: Authenticate with first key
    RemoteSSH->>User: Successful login

    User->>LocalSSH: ssh -i ~/.ssh/my_second_key ec2-user@IP
    LocalSSH->>RemoteSSH: Authenticate with second key
    RemoteSSH->>User: Successful login

    User->>LocalSSH: ssh roadmapsh-test-server
    LocalSSH->>RemoteSSH: Try first key, then second key
    RemoteSSH->>User: Successful login

    User->>LocalSSH: Attempt with wrong key (3 times)
    LocalSSH->>RemoteSSH: Failed authentication
    RemoteSSH->>F2B: Log failed attempts
    F2B->>RemoteSSH: Ban IP after 3 failures

Test Cases:

Connect with the first key:

ssh -i ~/.ssh/my_first_key ec2-user@your-instance-ip

Connect with the second key:

ssh -i ~/.ssh/my_second_key ec2-user@your-instance-ip

Connect with the alias:
```
ssh roadmapsh-test-server
```
Test fail2ban (optional, and with caution): On a different machine, attempt to connect with incorrect credentials multiple times. Then check if your IP is banned:
```
sudo fail2ban-client status sshd
```

Security Best Practices

Throughout this tutorial, we've implemented several security best practices. Here's a summary:

Key Management Best Practices

Use strong keys: 4096-bit RSA or Ed25519 keys
Protect private keys: Set permissions to 600 and never share them
Use passphrases: Add an extra layer of protection to private keys
Rotate keys periodically: Generate new keys and remove old ones

Server Configuration Best Practices

Disable password authentication in /etc/ssh/sshd_config:
```
PasswordAuthentication no
```
Limit user access by specifying allowed users:
```
AllowUsers ec2-user
```
Change the default SSH port to reduce automated attacks:
```
Port 2222  # Example alternative port
```
Use security groups/firewalls to restrict IP ranges that can access SSH
Keep your system updated with regular security patches:
```
sudo yum update -y
```

Conclusion

You've now set up a secure remote server access system using multiple SSH keys and enhanced protection with fail2ban. This approach provides:

Strong authentication without passwords
Multiple access credentials that can be managed separately
Protection against brute force attacks
Simplified connection through SSH config

This knowledge forms the foundation of secure server administration and can be applied to any Linux-based server, not just AWS EC2 instances.

Additional Resources

Github repo [https://github.com/kaalpanikh/ssh-remote-server-setup]

Have you set up remote SSH access before? What other security measures do you implement on your servers? Share your experience in the comments below!

Securing the Cloud: Mastering SSH Access on AWS

Table of contents