Building an HTTP Server from scratch

Omm PaniOmm Pani
12 min read

Table of contents

Introduction

I’ve used Express.js plenty of times. Spin up a server, add a few routes, call it a day. But I started wondering — what’s really happening under the hood?

Then I stumbled upon a tweet by Arpit Bhayani to “build an HTTP server from scratch” using any language.

This sparked a deep curiosity. How does a browser’s request turn into a response? What are protocols anyway? Why does everyone talk about TCP and ports? How do headers actually matter?

This blog is a personal deep dive — a build-first, learn-as-you-go journey to answer these questions:

  • 📜 What are protocols, really? And how does HTTP work?

  • 🧩 How to write a protocol parser to read HTTP requests line by line?

  • 🖧 How do TCP sockets work — from binding to ports to listening for connections?

  • 📦 How to read and write large chunks of data over a socket?

  • 📑 What role do headers like Content-Length or Content-Type play?

  • 👥 How can a server handle multiple client connections — using threads or event loops?

  • 📂 How can we serve static files from different paths?

  • 🔧 How to write custom route handlers like a mini Flask/Express?

We’ll build everything from scratch — no frameworks, just raw TCP sockets.

🧠 TL;DR

We built an HTTP server from scratch using Python and raw sockets.

✅ Parsed HTTP requests (method, path, headers, body) manually
✅ Wrote low-level code to handle TCP sockets and bind to ports
✅ Served static files with correct MIME types and 404 handling
✅ Parsed JSON POST bodies and handled routes like /api/echo
✅ Used threads to support multiple clients concurrently
✅ Built a Flask-style routing system using decorators
✅ Understood the HTTP protocol at the byte level — no magic, no black boxes

💡 Everything was built without using Express, Flask, or any web framework.

📜What are Protocols, Really? And What’s Special About HTTP?

When computers talk to each other, they need to speak the same language. That shared language is a protocol — a well-defined set of rules for communication.

Think of it like this:

Client ------------------> Server
      "GET /hello HTTP/1.1\r\n"
      "Host: example.com\r\n"
      "User-Agent: Chrome\r\n"
      "\r\n"

Both client and server must agree on:

  • What a request looks like

  • How it starts and ends

  • How to respond

  • What the headers and status codes mean

So What’s the HTTP Protocol?

HTTP (HyperText Transfer Protocol) is a text-based protocol built on top of TCP. It's the reason you can open a browser, hit https://google.com, and get back a webpage.

It’s stateless, human-readable, and follows a specific format:

GET /hello HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0

That's a full HTTP GET request. Every part has meaning:

  • GET: the method

  • /hello: the path

  • HTTP/1.1: the version

  • Followed by headers

  • A blank line \r\n\r\n signals the end of the headers

The response follows a similar structure:

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 1024

<html>...</html>

💡 Key Realization

At its core, HTTP is just formatted text over a TCP socket.

You send a string. You receive a string. That’s it. No magic — just a contract between client and server.

🔧 Code Preview: Raw HTTP in Action

Here’s a real example of how we read that raw HTTP request in Python using sockets:

def http_req_parser(data: str):
    lines = data.split("\r\n")
    request_line = lines[0]  # e.g. "GET /hello HTTP/1.1"
    method, path, version = request_line.split(" ")

    headers = {}
    for line in lines[1:]:
        if line == "":
            break  # end of headers
        key, value = line.split(":", 1)
        headers[key.strip()] = value.strip()

    return method, path, version, headers

This is our protocol parser — a simple way to break down a raw HTTP request into meaningful parts. We'll use this later to handle routes, headers, and more.

📎 Diagram: HTTP Request-Response Over TCP

Client                               Server
  |                                     |
  | -- TCP Connection (3-way handshake) |
  |                                     |
  | --------- HTTP Request ------------>|
  |    GET /hello HTTP/1.1              |
  |    Host: localhost:8080             |
  |    \r\n\r\n                         |
  |                                     |
  | <--------- HTTP Response -----------|
  |    HTTP/1.1 200 OK                  |
  |    Content-Type: application/json   |
  |    Content-Length: 23               |
  |                                     |
  |    {"message": "hello"}             |
  |                                     |

🛠 How I Parsed HTTP by Hand

Once we understand that HTTP is just text over a socket, the next step is:
“Can we break down this raw HTTP string ourselves — like a real web server would?”

🧾 A Raw HTTP Request Looks Like This:

POST /api/echo HTTP/1.1
Host: localhost:8080
Content-Type: application/json
Content-Length: 27

{"message":"You are awesome"}

So the parser’s job is to extract:

  • Method (POST)

  • Path (/api/echo)

  • Headers (Content-Type, Content-Length, etc.)

  • Body ({"message":"You are awesome"})

🔍 Step-by-Step Breakdown

Let’s say we already received this request as a string using a socket.

def parse_http_request(request_data):
    # Split headers from body
    header_part, body = request_data.split("\r\n\r\n", 1)

    lines = header_part.split("\r\n")
    request_line = lines[0]  # e.g., "POST /api/echo HTTP/1.1"
    method, path, version = request_line.split(" ")

    headers = {}
    for line in lines[1:]:
        if ':' in line:
            key, value = line.split(":", 1)
            headers[key.strip()] = value.strip()

    return method, path, version, headers, body

Now parse_http_request gives you back all the structured information. It’s the secret format used by browsers and servers😁😎!

💡 Debug Tip

When the client sends a POST request with a JSON body, the Content-Length header tells you how many bytes to expect in the body.
If you read fewer than that — you’ll miss the body. Read more, and you risk blocking.

That’s why after parsing headers, we use:

content_length = int(headers.get("Content-Length", 0))
body = conn.recv(content_length).decode()

📎 Diagram: Request Parser Mental Model

┌──────────────────────────────────────┐
│ POST /api/echo HTTP/1.1              │ ← Request Line
├──────────────────────────────────────┤
│ Host: localhost:8080                 │
│ Content-Type: application/json       │
│ Content-Length: 27                   │ ← Headers
├──────────────────────────────────────┤
│ {"message":"You are awesome"}        │ ← Body
└──────────────────────────────────────┘

We’re building our own mini-Express by understanding and extracting each layer manually.

✅We now have a working protocol parser, built from scratch.
It didn't rely on any web framework — just string manipulation and socket reads.

🌐 How Do TCP Sockets Actually Work?

After parsing the request manually, Let’s dig even deeper:

“How does a server even receive this request in the first place?”

For that Lets go into the world of TCP sockets — the backbone of all internet communication.

📦 What’s a TCP Socket?

A TCP socket is a two-way communication pipe between two computers.
When a browser hits localhost:8080, it’s actually opening a TCP connection to:

IP: 127.0.0.1
PORT: 8080

Once connected, they can send and receive raw bytes.

🔌 Creating a TCP Server in Python

Here’s the minimal code that creates a working TCP socket server:

import socket

server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(('localhost', 8080))   # IP + Port
server_socket.listen(5)                   # Allow up to 5 queued clients

print("Server listening on port 8080...")

while True:
    conn, addr = server_socket.accept()
    print(f"Connection from {addr}")

    request_data = conn.recv(1024).decode()
    print("Received data:")
    print(request_data)

    conn.sendall(b"HTTP/1.1 200 OK\r\n\r\nHello, world!")
    conn.close()

🧠 What Just Happened?

  • socket() creates a socket file descriptor

  • bind() attaches it to an IP and port

  • listen() makes it a server socket (ready to accept clients)

  • accept() waits (blocks!) for a new connection

  • recv() reads data from that client

  • sendall() sends bytes back

  • close() closes the connection

🚦 Why listen(5)?

That’s the size of the backlog — how many incoming connections the OS can queue while your server is busy handling one.

🖼 Diagram: TCP Socket Lifecycle

[Browser] ──TCP Connect──▶ [Your Server on 127.0.0.1:8080]
            ◀── HTTP Response ──

This is the exact mechanism behind every HTTP request.

No frameworks. No magic.
Just your server, a port, and the client.

✅At this point, we have a working low-level server that could:

  • Accept connections

  • Receive raw HTTP requests

  • Send back raw responses

📁 How Do We Serve Static Files Based on the Request Path?

So far, we’ve only returned hardcoded responses.

But browsers usually request:

  • /index.html

  • /styles.css

  • /app.js

  • /images/logo.png

“How do we translate a URL like /index.html into a real file on disk?”

Let’s build a basic static file server.

🗂️ Step 1: Map the URL to a File Path

We'll use Python's os module to safely join paths:

import os

def handle_request(request):
    method, path, *_ = request.split(' ')

    if path == '/':
        path = '/index.html'

    file_path = os.path.join('public', path.lstrip('/'))

    ...

✅ If the browser asks for /styles.css, we’ll look for public/styles.css on disk.

📄 Step 2: Read and Return the File

    try:
        with open(file_path, 'rb') as f:
            body = f.read()
        response = (
            "HTTP/1.1 200 OK\r\n"
            f"Content-Length: {len(body)}\r\n"
            "Content-Type: text/html\r\n"
            "\r\n"
        ).encode() + body
    except FileNotFoundError:
        response = (
            "HTTP/1.1 404 Not Found\r\n"
            "Content-Length: 0\r\n"
            "\r\n"
        ).encode()

    client_socket.sendall(response)

We’re sending:

  • Status line (e.g., 200 OK)

  • Headers: Content-Length, Content-Type

  • Body: raw file content (HTML, CSS, JS, etc.)

📦 Content-Type Magic

Right now, everything returns Content-Type: text/html. But we can detect the correct type:

import mimetypes

content_type, _ = mimetypes.guess_type(file_path)
content_type = content_type or "application/octet-stream"

Now .css gets text/css, .js gets application/javascript, etc.

📁 Directory Layout

Here’s our folder structure:

project/
├── server.py
└── public/
    ├── index.html
    ├── styles.css
    ├── app.js
    └── images/
        └── logo.png

Every request is resolved from the public/ folder.

✅ Takeaway

We just built:

  • 🗺️ URL-to-path mapping

  • 📁 File serving logic

  • 🧠 Content-Type detection

  • 🧼 Graceful 404 fallback

Our server now speaks HTML, CSS, JS — just like a real one

🔁 What If We Want to Handle Routes Like /api/echo?

Static files are great for websites. But APIs? They’re the backbone of dynamic apps.

When a client sends:

POST /api/echo HTTP/1.1
Content-Type: application/json
Content-Length: 27

{"message": "You are awesome"}

The browser expects the server to parse this request and send back a response like:

{"you sent": "You are awesome"}

So now the question is:

“How do we wire up custom routes and handlers like /api/echo?”

🛠 Defining Routes

We will introduce a global ROUTES dictionary that maps a path string to a handler function:

ROUTES = {}

def route(path):
    def decorator(func):
        ROUTES[path] = func
        return func
    return decorator

Then we can register routes like this:

@route('/api/echo')
def echo_handler(method, headers, body):
    if method == 'POST':
        data = json.loads(body)
        return http_response(200, json.dumps({"you sent": data.get("message")}), content_type='application/json')
    return http_response(405, 'Method Not Allowed')

🎉 Boom — we just mimicked Express-style app.post('/api/echo', handler) logic!

🧠 What Happened Behind the Scenes?

  1. We parsed the request line and headers manually.

  2. We matched the path to a function in ROUTES.

  3. We passed method, headers, and the raw body to that handler.

  4. The handler decided how to respond.

🧪 Full Flow for Custom API Route

pgsqlCopyEditPOST /api/echo ──▶ Server
                ├─ Matches route in ROUTES
                ├─ Parses headers & body
                ├─ Calls echo_handler()
                └─ Returns JSON response

✅ We created a micro version of how frameworks like Flask, Express, or FastAPI handle routes.
And it was just a few lines of code.

This gave us complete flexibility — we could define any number of custom APIs and return HTML, JSON, or plain text.

Let’s dive into one of the most powerful upgrades to our server:

👥 Can Our Server Handle Multiple Clients at the Same Time?

We’ve been serving one request at a time — and that’s fine... for a toy project.

But real servers deal with dozens, hundreds, thousands of concurrent clients.

That led to the question:

“What happens if a second client sends a request while we're still processing the first?”

⚙️ Enter Threads

We rewired our main connection loop to spin up a new thread for every client.

Here’s the change:

import threading

while True:
    client_socket, addr = server_socket.accept()
    thread = threading.Thread(target=handle_client, args=(client_socket,))
    thread.start()

💡 Now each request is handled independently. The main loop just keeps accepting clients!

🔍 What’s Happening Internally?

Imagine the server loop like this:

MAIN THREAD
  └─ Accepts Client 1
       └─ Spawns Thread A (handle_client)
  └─ Accepts Client 2
       └─ Spawns Thread B (handle_client)
  └─ ...

Each thread runs handle_client(), which:

  • reads the request,

  • parses it,

  • generates a response,

  • and sends it back.

🔥 The Result?

We can now:

✅ Accept concurrent requests
✅ Serve multiple browsers at once
✅ Run long tasks (like file reads or big JSON parsing) without blocking the whole server

⚠️ Real-World Note

Threads are powerful, but they have downsides:

  • 🧠 Context switching is expensive

  • 🧵 Too many threads can crash the process

  • ⚡ Python threads are subject to the GIL (Global Interpreter Lock)

In a production-grade HTTP server, you'd use async IO or process pools, but threads are a great first step.

✅ Takeaway

With just 3 lines, we added concurrency to our HTTP server — and took a giant leap toward real-world performance.

⭐️ Summary.

We started with a simple curiosity: "How does Express.js or Flask actually work under the hood?"

To answer that, we went deep — from raw sockets to routing logic.

Here’s a quick recap of what we uncovered:

🔌 We Learned About Protocols

  • A protocol is just a set of rules for how two systems communicate.

  • HTTP is a text-based application-layer protocol built on top of TCP.

  • Every browser request follows a structure: request line, headers, optional body.

🧠 We Parsed the HTTP Protocol Ourselves

  • We wrote our own HTTP request parser from scratch.

  • We manually split lines, extracted methods, paths, headers, and bodies.

  • This gave us a first-principles understanding of how real servers behave.

🌐 We Used TCP Sockets Like Pros

  • We created a TCP socket and bound it to a port.

  • Accepted connections using accept(), and read raw bytes from clients.

  • Understood what a socket really is — a communication endpoint.

📦 We Read and Wrote Real HTTP Data

  • We manually received request bytes and handled reading the full body.

  • We learned to use the Content-Length header to avoid partial reads.

  • We wrote our own HTTP response, line by line — status line, headers, body.

📁 We Served Static Files From Disk

  • We handled file paths using os.path safely.

  • Set appropriate Content-Type headers using file extensions.

  • Handled 404s and directory traversal protection manually.

🔁 We Built a Minimal Flask-Like Routing System

  • Used decorators to define custom route handlers like /api/echo.

  • Registered routes in a global ROUTES dictionary.

  • Parsed JSON from the body of POST requests and sent JSON responses back.

👥 We Handled Multiple Clients (The Easy Way)

  • With Python’s threading, we spun up a new thread for each connection.

  • This allowed us to serve multiple requests at once — like any real server.

🧰 Full Code

Here’s the final code — this is our entire server in under 150 lines:

👉 View full code as a GitHub Gist

https://gist.github.com/Omm-Pani/b122fe430e947582ffd68abb2e2fb092

This is a working server built from scratch that:

  • Accepts real HTTP requests

  • Parses them manually

  • Serves static files

  • Handles custom routes

  • Supports concurrent clients

All built without any framework.

🎯 Final Thoughts

What started as a weekend curiosity ended up becoming a systems deep dive.

I understood how servers actually work.

This project taught me about protocols, sockets, headers, parsing, threading, file I/O, and basic routing — all from first principles.

0
Subscribe to my newsletter

Read articles from Omm Pani directly inside your inbox. Subscribe to the newsletter, and don't miss out.

httpserverWeb Developmentwebwebdev

Written by

Omm Pani
Omm Pani