Building a web server from scratch with NodeJS Streams!

Reesav GuptaReesav Gupta
12 min read

Let's code a web server from scratch with NodeJS Streams!

In this guide, we'll explore the fundamentals of HTTP by creating a basic web server from scratch using Node.js. Along the way, we'll break down how HTTP requests and responses work and get hands-on with Node's stream API.

We'll start with a quick look at Node.js' built-in http module. Then, we'll examine the structure of HTTP communication. Finally, we'll build a custom web server using the net module, which allows us to handle raw TCP connections.

This project is purely for learning and isn't meant for real-world applications. Let's get started!

Node's built in http module

Node.js provides a built-in HTTP server that allows us to handle web requests without any external dependencies. This server can listen on any specified port and execute a callback function whenever a request is received. The callback receives two key objects: the Request Object, which contains details about the incoming request, and the Response Object, which is used to send data back to the client.

With just a few lines of code, we can create a basic web server that responds with a simple message. For example, a Node.js HTTP server listening on port 3000 can be set up to send "Hello World!" as a response to every request. This approach provides an easy way to understand how HTTP communication works while offering flexibility for building more advanced web applications.

import http from 'http'

const server = http.createServer()

server.on(
  'request',
  (
    req: http.IncomingMessage,
    res: http.ServerResponse<http.IncomingMessage>
  ) => {
    console.log(`request method: ${req.method}, request url: ${req.url}`)
    console.log(`request socket:`, req.socket)
    res.setHeader(`Content-Type`, `text/plain`)
    res.end(`hello world`)
  }
)

server.listen(3000, () => {
  console.log(`server is running on port 3000`)
})

When the server starts, it begins listening for incoming requests on port 3000. Each time a request is received, key details such as the HTTP method (GET, POST, PUT, DELETE, etc.) and the request URL (like / or /some/route) are logged to the console.

The Request Object (req) provides access to this data. It also includes a .socket property, which represents the lower-level TCP connection. While the built-in HTTP module abstracts away much of the complexity, we will soon build our own TCP server to handle connections directly and use raw sockets to create a custom web server.

In the example above, the fact that req.method and req.url are easily accessible means that Node.js has already parsed the raw request text into a structured object. To better understand how this process works, we will now explore the structure of an HTTP request and learn how to parse it manually.

Dissecting HTTP

Here is a sample http request

POST /articles/99/replies HTTP/1.1\r\n
Host: api.example.com\r\n
Accept: application/json, text/plain\r\n
Authorization: Bearer YWJjMTIzNDU2Nzg5MGFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6\r\n
Accept-Encoding: gzip, deflate\r\n
Content-Type: application/json\r\n
Content-Length: 35\r\n
User-Agent: PostmanRuntime/7.29.0\r\n
Referer: https://app.example.com\r\n
\r\n
{"message":"Great article! Thanks."}

A few observations:

An HTTP request consists of multiple lines, each separated by \r\n. Let's break down its structure:

  1. The Request Line
    The first line of the request is known as the request line. It consists of three space-separated parts:

    • Method: The HTTP method, such as POST, GET, PUT, or DELETE. (Defined in Section 9 of the HTTP/1.1 specification)
    • URL: The request path, e.g., /articles/99/replies.
    • Protocol Version: The HTTP version, such as HTTP/1.1.

  2. Request Headers
    Each subsequent line is a request header, formatted as Field: Value. Headers provide additional details about the request. (Defined in Section 14 of the HTTP/1.1 specification)

  3. End of Headers
    A blank line (\r\n) separates the headers from the request body.

  4. The Request Body
    The body contains the actual data being sent.

    • Its format should match the Content-Type header (e.g., JSON, form data).
    • Its length should match the Content-Length header.

For example, if a request body contains a JSON document that is 35 bytes long, the Content-Length header should also indicate 35. This ensures proper data handling by the server.

And this is the sample Http response:

HTTP/1.1 201 Created\r\n
Server: Apache/2.4.41 (Ubuntu)\r\n
Date: Wed, 26 Mar 2025 10:45:30 GMT\r\n
Content-Type: application/json\r\n
Content-Length: 165\r\n
Connection: keep-alive\r\n
\r\n
{
  "id": "a9x7y6b52q",
  "message": "Great article! Thanks.",
  "articleId": 99,
  "createdAt": "2025-03-26T10:45:30.123Z",
  "updatedAt": "2025-03-26T10:45:30.123Z"
}

again a few observations: Like HTTP requests, responses are also structured with lines separated by \r\n. Here’s how they are organized:

  1. The Status Line
    The first line of the response is called the status line, which consists of:

    • HTTP Version: The protocol version, e.g., HTTP/1.1.
    • Status Code: A three-digit HTTP status code, such as 201.
    • Reason Phrase: A brief explanation of the status, like Created.

  2. Response Headers
    Each subsequent line contains a response header, structured similarly to request headers as Field: Value.

  3. End of Headers
    A blank line (\r\n) separates the headers from the response body.

  4. The Response Body
    The body contains the actual data being sent back to the client.

    • Its format should match the Content-Type header (e.g., JSON, HTML).
    • Its length should match the Content-Length header.

For example, if the response body is a 165-byte JSON document, the Content-Length header should indicate 165 to ensure proper handling.

In the next section, we will build a TCP server that will allow us to observe an incoming HTTP request in real time, process it chunk by chunk, parse it, and send a response using Streams.

Reciveing and parsing a http request

Node.js includes a built-in net module that allows us to create a streaming TCP server. The term streaming means that the server can continuously send and receive data over time using Node’s Stream API.

Unlike an HTTP server, which emits a request event with request and response objects, a TCP server emits a connection event, providing a socket object. These socket objects are Duplex Streams, meaning they can be both read from and written to.

When an HTTP request is sent to our TCP server, reading from the socket gives us the raw text of the request. In Node.js, there are two primary ways to read from a readable stream:

  1. Subscribing to its data event.
  2. Calling its .read() method.

Next, let’s explore how we can use the data event to read from the socket.

import net from 'net'

const server = net.createServer()

server.on('connection', handleConnection)

server.listen(3000, () => {
  console.log(`running`)
})

function handleConnection(socket: net.Socket) {
  socket.on('data', (chunk) => {
    console.log(`chunk recieved : `, chunk.toString())
  })

  socket.write(
    `HTTP/1.1 200 OK\r\nServer: my-web-server\r\nContent-Length: 0\r\n\r\n`
  )
}

put this code in a file called server.ts and run it from your command line with bun run server.ts. In another terminal, use cURL to make a simple http request to your server:

curl -v localhost:3000/some/url

You should see a verbose output from cURL, showing you the request and response headers. In the Node terminal, you should see something like this:

Received chunk:
GET /some/url HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.54.0
Accept: */*

let's make a POST request with some data:

curl -v -X POST -d'hello=world' localhost:3000/some/url

This should give you the following output in your Node terminal:

chunk recieved :  POST /some/url HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.87.0
Accept: */*
Content-Length: 11
Content-Type: application/x-www-form-urlencoded

hello=world

When sending a long request body, the data is received in multiple chunks. However, a web server doesn’t always need to process the entire request body. In many cases, the request headers alone provide enough information to begin processing.

Since we are building a generic web server, we need a way to stop receiving request data from the socket as soon as we detect the sequence \r\n\r\n, which marks the end of the headers.

The best way to achieve this is by using the .read() method on the socket. This allows us to control when to stop pulling data from the stream. Below, we’ll see how to implement this behavior effectively.

import net from 'net'

const server = net.createServer()

server.on('connection', handleConnection)

server.listen(3000)

function handleConnection(socket: net.Socket) {
  // setup a readable event once we can start calling .read()
  socket.once('readable', () => {
    // setup a buffer to hold the incoming data
    let reqBuffer = Buffer.from('')

    // setup a temporary buffer to read in chunks
    let buf
    let reqHeader

    while (1) {
      // read data from socket
      buf = socket.read()
      // stop if there is no more data
      if (buf === null) break

      // concatenate the existing request buffer with the new data
      reqBuffer = Buffer.concat([reqBuffer, buf])

      // check if we reached \r\n\r\n, indicating the end of the header
      let marker = reqBuffer.indexOf('\r\n\r\n') //returns -1 if \r\n\r\n hasn't been recieved or doesn't contain in the reqBuffer

      if (marker !== -1) {
        // if we reached \r\n\r\n, there could be data after it. Take note.
        let remaining = reqBuffer.subarray(marker + 4) //remaining.toString() --> req.body

        // The header is everything we read, up to and not including \r\n\r\n

        reqHeader = reqBuffer.subarray(0, marker).toString()

        // This pushes the extra data we read back to the socket's readable stream
        socket.unshift(remaining)
        break
      }
    }
    // At this point, we've stopped reading from the socket and have the header as a string
    // If we wanted to read the whole request body, we would do this:

    reqBuffer = Buffer.from('')
    while ((buf = socket.read()) !== null) {
      reqBuffer = Buffer.concat([reqBuffer, buf])
    }

    let reqBody = reqBuffer.toString()

    //send a generic response
    socket.end(
      'HTTP/1.1 200 OK\r\nServer: my-custom-server\r\nContent-Length: 0\r\n\r\n'
    )
  })
}

The code is slightly longer because we need logic to determine when to stop reading from the stream. This ensures that we separate the request headers from the request body, allowing the developer using our web server to decide how to handle the body.

A key part of this process is the socket.unshift line. This method "puts back" any extra data that was read into the readable stream. By doing so, we ensure that the socket remains available for further reading if needed.

Now, here’s the complete implementation of our basic web server, incorporating everything we’ve learned so far. The server exposes a function called createWebServer(requestHandler), which takes a handler function in the format (req, res) => void, similar to Node's built-in HTTP server. The comments in the code explain each step in detail.

import net from 'net'

type RequestType = {
  method: string
  url: string
  httpVersion: string
  headers: {} | undefined
  socket: net.Socket
}

type ResponseType = {
  write(chunk: Uint8Array | string): void
  end(chunk?: Uint8Array | string): void
  setHeader: (key: string, value: string | number) => void
  setStatus(newStatus: number, newStatusText: string): void
  json(data: any): void
}

function createWebServer(
  requestHandler: (request: RequestType, response: ResponseType) => void
) {
  const server = net.createServer()

  server.on('connection', handleConnection)

  function handleConnection(socket: net.Socket) {
    socket.once('readable', () => {
      let reqBuffer = Buffer.from('')
      let buf
      let reqHeader

      while (true) {
        buf = socket.read()

        // Stop if there is no more data
        if (buf === null) break

        // Concatenate the existing request buffer with the new data
        reqBuffer = Buffer.concat([reqBuffer, buf])

        // Check if we reached \r\n\r\n, indicating the end of the header
        const marker = reqBuffer.indexOf('\r\n\r\n')

        if (marker !== -1) {
          // If we reached \r\n\r\n, there could be data after it. Take note.
          const remaining = reqBuffer.subarray(marker + 4)

          // The header is everything we read, up to and not including \r\n\r\n
          reqHeader = reqBuffer.subarray(0, marker).toString()

          // This pushes the extra data we read back to the socket's readable stream
          socket.unshift(remaining)
          break
        }
      }

      /* Request related business */

      // Start parsing the header
      const reqHeaders = reqHeader?.split('\r\n')

      // First line is special
      const reqLine = reqHeaders?.shift()?.split(' ')

      // Further lines are one header per line, build an object out of it.
      const headers = reqHeaders?.reduce((accumulator, currentHeader) => {
        const [key, value] = currentHeader.split(':')
        return {
          ...accumulator,
          [key.trim().toLowerCase()]: value.trim(),
        }
      }, {})

      // This object will be sent to the handleRequest callback.
      if (!reqLine) return

      const request: RequestType = {
        method: reqLine[0],
        url: reqLine[1],
        httpVersion: reqLine[2].split('/')[1],
        headers,
        // The user of this web server can directly read from the socket to get the request body
        socket,
      }

      /* Response related business */

      // Initial values
      let status = 200
      let statusText = 'OK'
      let headersSent = false
      let isChunked = false

      const responseHeaders = new Map<string, string | number>()

      responseHeaders.set('server', 'my-custom-server')

      function setHeader(key: string, value: string | number) {
        responseHeaders.set(key.toLowerCase(), value)
      }

      function sendHeaders() {
        // Only do this once :D
        if (!headersSent) {
          headersSent = true

          // Add the date header
          setHeader('date', new Date().toISOString())

          // Send the status line
          socket.write(`HTTP/1.1 ${status} ${statusText}\r\n`)

          // Send each following header
          Array.from(responseHeaders.keys()).forEach((headerKey) => {
            socket.write(`${headerKey}: ${responseHeaders.get(headerKey)}\r\n`)
          })

          // Add the final \r\n that delimits the response headers from body
          socket.write('\r\n')
        }
      }

      const response: ResponseType = {
        write(chunk: Uint8Array | string) {
          if (!headersSent) {
            // If there's no content-length header, then specify Transfer-Encoding chunked
            if (!responseHeaders.get('content-length')) {
              isChunked = true
              setHeader('transfer-encoding', 'chunked')
            }
            sendHeaders()
          }

          if (isChunked) {
            const size = chunk.length.toString(16)
            socket.write(`${size}\r\n`)
            socket.write(chunk)
            socket.write('\r\n')
          } else {
            socket.write(chunk)
          }
        },
        end(chunk?: Uint8Array | string) {
          if (!headersSent) {
            // We know the full length of the response, let's set it
            if (!responseHeaders.get('content-length')) {
              // Ensure we don't access `.length` of undefined
              setHeader('content-length', Buffer.byteLength(chunk || ''))
            }
            sendHeaders()
          }
          if (isChunked) {
            if (chunk) {
              const size = chunk.length.toString(16)
              socket.write(`${size}\r\n`)
              socket.write(chunk)
              socket.write('\r\n')
            }
            socket.end('0\r\n\r\n')
          } else {
            socket.end(chunk!)
          }
        },
        setHeader,
        setStatus(newStatus: number, newStatusText: string) {
          status = newStatus
          statusText = newStatusText
        },
        // Convenience method to send JSON through server
        json(data: any) {
          if (headersSent) {
            throw new Error('Headers sent, cannot proceed to send JSON')
          }
          const json = Buffer.from(JSON.stringify(data))
          setHeader('content-type', 'application/json; charset=utf-8')
          setHeader('content-length', json.length)
          sendHeaders()
          socket.end(json)
        },
      }
      // Send the request to the handler!
      requestHandler(request, response)
    })
  }

  return server // Ensure the server instance is returned
}

const webServer = createWebServer((req, res) => {
  // This is the same as our original code with the http module :)
  console.log(`${new Date().toISOString()} - ${req.method} ${req.url}`)
  res.setHeader('Content-Type', 'text/plain')
  res.end('Hello World!')
})

webServer.listen(3000)

As mentioned at the beginning of this post, this server is not meant for production use. Its sole purpose is to help understand streams and buffers in Node.js.

Most of the code should be self-explanatory. We parse the request and send the response based on the rules we covered earlier.

One new concept introduced here is Transfer-Encoding: chunked. This encoding is used when the server does not know the full length of the response in advance. Instead of sending a Content-Length header, the response is sent in chunks, each prefixed with its size in hexadecimal.

If you're curious, you can read more about chunked transfer encoding on Wikipedia.

Conclusion

So, we built a web server from scratch—no magic, just sockets, streams, and buffers doing their thing. Along the way, we dissected HTTP requests, wrangled raw data, and saw how Node.js handles networking under the hood.

Is this server production-ready? Absolutely not. But did we get a peek behind the curtain of how real servers work? You bet. Now, the next time you spin up an Express app, you’ll know what’s happening beneath all that abstraction. Happy coding!

0
Subscribe to my newsletter

Read articles from Reesav Gupta directly inside your inbox. Subscribe to the newsletter, and don't miss out.

TypeScriptwebserverNode.jsStreams

Written by

Reesav Gupta
Reesav Gupta

Final-year BTech CSE student who loves breaking things just to figure out how they work. Passionate about system design, cloud, and low-level engineering. If it runs on a server, I probably want to optimize it. Always building, always learning—because “works on my machine” isn’t good enough