SSH Simplified: Unfogging and Understanding SSH

Hello there, have you ever wanted to understand ssh and how it works ? This should be a friendly guide to help with that, and I’ll try keeping it as simple as possible. This article assumes that you are on a linux machine and have ssh installed, as it comes pre-installed with most distribution of linux anyways.

What is SSH?

SSH Protocol stands for SECURE SHELL Protocol, it’s a secure means of login and access from a client to a remote server across the internet or untrusted networks. It provides strong authentication and strong encryption of traffic for data transfer.

How SSH connections work.

For ssh to work both systems would need to have ssh installed on them. The ssh system is a client-server protocol, driven / initialized by the client. Linux systems have ssh pre-installed in them, and this installation runs an ssh daemon (sshd). The sshd is responsible for receiving connections on the remote servers from the client computers.

SSH uses a set of cryptographic keys for authentication and encryption of data to make it secure. These keys are generated in pairs :- A public key and a private key. If the names are any indicator (and they are 😅), the public key is to be shared while the private key is to be kept secret.

An interesting part of this which I didn’t figure out until later, is that the same keys are not used for authentication and encryption. The public and private (asymmetric) keys mentioned above are only used in authentication. A different agreed upon set of symmetric keys are used in encryption of traffic.

The magic of ssh from within can be explained pretty simply in this format.

• The public key is shared with a remote system (server), while the private key is kept on your local computer (client)

• The client triggers an open connection received by the servers sshd.

• The server generates and sends a random message to the client for verification.

• The client signs the message with it’s private key via it’s ssh-agent (more on this later) and returns it to the server.

• The server verifies the signature on the message from the client with the public key it has (as there must be correspondence between both keys).

This process then in fact confirms that the client is holding the private key to the servers public key, and is further used to negotiate the symmetric keys to be used in data encryption.

Generating the infamous cryptographic keys:

To generate ssh keys, the command ssh-keygen is used in this way:

ssh-keygen -t ed25519

T -t flag specifies the type of key to create, and it’s possible options as specified by the man page are:

dsa | ecdsa | ecdsa-sk | ed25519 | ed25519-sk | rsa

After entering the command, it ask if you want to save the keys using a custom file name.

Enter file in which to save the key (/home/username/.ssh/id_ed25519):

If not, it defaults to id_ALGORITHM in this case that would be id_ed25519. The entire path in the parenthesis would be the path (default) to your private keys.

I would recommend giving custom names to the key files for two reasons:

• due to the fact that these files overwrite: So say there already exists a file with the name id_ALGORITHM and a newer key is saved with the same name, the older one is overwritten.

  Why this is an issue is the problem that arises when you try to authenticate into a server you have previously set up ssh on. The authentication fail because the new private key you generated, and the previous public key already submitted to the server are not in correspondence and cannot be used to decrypt each other.

• to avoiding security compromises: You ideally do not want one key authenticating into multiple servers, It’s pretty simple really if that single private key is compromised, all servers that can be authenticated into should be considered compromised as well.

  On the other hand, having separate keys for say github ssh, staging server, prod server, mail server etc. ensures that if your private key to staging server gets leaked or compromised, the bad actor in this scenario has no access to your prod server, mail server, cant authenticate to your github e.t.c.

So give custom names to your keys, and not so obvious names like `this-is-my-production-server-ssh-keys`. We’ve all done it, I know 😅

Next it asks you for a passphrase, as an added layer of security. You can skip this if you wish by just pressing the enter key or you can set a passphrase for your ssh keys.

What’s important is remembering your passphrase if you set one, as you’ll be asked when you plan to authenticate with those keys. That’s it you have your ssh keys now.

Connecting to your remote server:

Firstly we need to get our public key up on to the remote server in the first place. To this we can use the ssh-copy-id command, which copies the specified public key to the destination server. Let’s copy the public key of the ssh key we just created.

ssh-copy-id -n -i ~/.ssh/id_ALGORITHM user@serverIP

• the -i flag specifies the path to the key file. It should always pickout the public key for the key file name you provide. PS: the ssh-copy-id file copies all keys if this flag is omitted which is a security risk.

• the -n flag is used to run a dry-run sample which displays the exact keys it intends to copy

• the serverIP is the IP of the remote server and the user is the username you’d be operating as on the server.

I highly recommend making a dry-run first then removing the -n flag if satisfactory.

Then we can connect from the client as the second step. To do this we just run the command ssh user@serverIP.

SSH typically uses port 22 which can be specified in the command as -p 22 but it’s so common it’s not necessary. Of course as well the port can be configured differently (good luck when you lock yourself out of the server 😅). No seriously, don’t do this unless you know your stuff && have a genuine reason to. Emphasis on &&.

SSH agents:

Ha, the bit that triggered this entire write up.

So I had just created a new ssh key for GitHub, but could not use it. I couldn’t push, pull, clone e.t.c. Fundamentally, I couldn’t even connect to the server even though I had my private key locally and had set up the servers public key (as GitHub’s GPG keys). A few minutes of googling later, I found out why.

SSH-agents is a tool (a program) that essentially helps manage your keys.

And as it would be I didn’t have this newly created key in there. I’ve created a bunch of keys in the past and never ran into this issue so I found this interesting. The only notable difference was me creating this particular key while the ssh service was down (sudo systemctl ssh status).

Okay so what does the ssh-agent tool do:

• manage your keys -

  • doesn’t store it physically on your device

  • doesnt allow them to be exported

• sign messages with your private key - does this over a socket where it takes the message and returns the signed message at no point actually handing over the keys to client programs, this allows it to be applicable with programs you don’t fully trust.

• caches the key passphrase, to prevent entering it each and every time the keys are requested.

• handles ssh tunneling and agent forwarding.

Working with ssh-agents:

To work with ssh-agents you need the ssh agent running. You can either do this by ensuring your ssh service is running, and if that doesn’t work, you can manually run an instance of the program itself with the command

eval “$(ssh-agent -s)“

The ssh-agents has a few main operations to it.

• Add a regular key pair (public and decrypted private keys)

• Add a constrained key pair (public and decrypted private keys)

• Add a key (regular or constrained) from a smart card (public key only)

• Remove a key

• List keys stored in the agent

• Sign a message with a key stored in the agent

• Lock or unlock the entire agent with a passphrase

and all of these except the signing of a message can be achieved with the ssh-add command.

ssh-add by default adds all your keys to the ssh-agent. It’s scans for standard expectation of keys and thus would pick default like

• ~/.ssh/id_rsa

• ~/.ssh/id_ed25519

• ~/.ssh/id_dsa

• ~/.ssh/id_ecdsa

but wait what happens with custom named keys like our id_ALGORITHM, well keys like these need to be added manually.

The ssh-add takes an argument which is the file path to the specific file to add, and defaults to adding all files if none is specified.

ssh-add ~/.ssh/id_ALGORITHM would add our newly generated key to the ssh-agent.

You can man the ssh-add command to learn more about it, but here’s a few flags to buttress the operations above:

• -d - delete a key

• -D - delete all keys

• -l - list fingerprints of all identities

• -L - list public keys of all identities

• -x - Lock the agent

• -X - Unlock the agent

Alright !, So I learnt a lot, did my best at thorough research to make this as comprehensive as possible. I hope this helped you understand a bit more about ssh in general.

I was speaking in Linux here, so if you’re on a different OS you might need to do more research, for example installing openSSH for windows users not on wsl, or adding an ssh key to the ssh-agent on mac which use a keychain and makes the process slightly different (add ssh key to ssh-agent for mac).

Many thanks to GitHub, ssh.com and other internet resources for informing this article.

Have a lovely week.

Mukhtar B. Abdulrazaq.