How SSH Authentication Works: Hosts, Keys, and the Trust Behind Your Connection

If you've ever SSH’d into a server using a private key — whether for deployment automation, server maintenance, or accessing cloud infrastructure — you’ve probably wondered:

• Why does SSH ask you to trust the host the first time you connect?

• And what exactly happens when your private key is used for authentication?

SSH handles a lot behind the scenes, but these questions fall under one specific part of the process: authentication — when the client and server verify each other.

Let’s explore that in this post.

SSH ensures a two-way trust.

When you SSH into a server, two things need to happen:

1. You need to verify that you’re connecting to the real server (not an imposter).

2. The server needs to verify that you are who you say you are.

Verifying the Server: known_hosts

The known_hosts file lives in your local machine at ~/.ssh/known_hosts.

The first time you connect to a server, SSH retrieves its public host key and shows a message like:

The authenticity of host 'example.com (12.34.56.78)' can't be established.
Are you sure you want to continue connecting (yes/no)?

If you type yes, SSH saves the server’s host key to your known_hosts file.

From then on, every time you SSH into that server, SSH compares the saved public host key to the one the server presents. If it doesn’t match, SSH warns you that something might be wrong — a potential man-in-the-middle attack.

Wait, What is a Host Key?

A host key is a cryptographic key pair (a private key and a public key) that’s unique to the server. It acts as the server’s fingerprint.

The public part of the host key is what your SSH client receives and stores in known_hosts. The private part stays safely on the server.

Here’s what a known_hosts entry might look like:

65.2.169.169 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAABBBN4ROAhiruMNQLt2jf8sjTwIVAq953RJOS4n6w4oNou1xNkVyhbdLggoj+IESrXGkB4ZGi2LzVMhaTSrxerZ1HQ=

• 65.2.169.169 is the server’s IP address.

• ecdsa-sha2-nistp256 is the algorithm used to create the host key (in this case, ECDSA with NIST P-256 curve).

• The long string that follows is the server’s base64-encoded public host key.

The SSH client uses this to ensure that it’s still connecting to the same trusted server.

Verifying You: Public Key Authentication

• When you use key-based authentication, you keep a private key on your local machine (usually in ~/.ssh/id_rsa or ~/.ssh/id_ed25519).

• The matching public key is generated alongside your private key on your local machine and then copied to the server, where it’s stored inside the ~/.ssh/authorized_keys file for the user you’re connecting as.

Here’s what happens:

[You run ssh -i ~/.ssh/id_ed25519 <username>@<remote-host>]
            |
            v
[Your client sends ~/.ssh/id_ed25519.pub (public key) to the server]
            |
            v
[Server checks if public key from SSH client is in /home/<username>/.ssh/authorized_keys]
     |                        |
     |                        |
[No match]              [Match found]
     |                        |
     v                        v
[Authentication         [Server sends a
   denied ❌]            challenge (nonce)]
                                  |
                                  v
              [Client signs challenge with ~/.ssh/id_ed25519 (its private key)]
                                  |
                                  v
              [Server verifies signature using client's public key]
                         |                        |
                         |                        |
                [Invalid ❌]            [Valid ✅]
                         |                        |
                         v                        v
       [Authentication failed ❌]   [Authentication successful ✅]

At no point does your private key leave your machine. That’s what makes it secure.

Wrapping up

It’s all about mutual trust — you’re trusting the server to avoid impersonation and the server is trusting you to control access.

So the next time you SSH into a server, you’ll know exactly what that “Are you sure you want to continue connecting?” message really means.