In this post, I'll be sharing my journey during the process of an HTTP Server development from scratch. The purpose of this project is the understanding of how an HTTP server works under the hood. Curiosity and a desire to challenge myself are also big motivators.

I'll be following the “Build Your Own X” tutorial series. This is a great initiative that encourages learning by doing, and it’s an awesome resource that breaks things down step by step.

For this project, I've chosen C# as a language. Since I haven't used it recently, this project gives me a chance to relearn, practice, and improve my skills in this language.

What is an HTTP Server?

An HTTP Server is an application that is expecting a request from a client on a particular port and returns website components to that client. The server is said to be listening on this port.

A web browser requests a particular file to the server on one of its open ports using either HTTP/HTTPS over the internet. The server validate the request and retrieve the requested resource. If it exists, it returns the resource otherwise it returns a NotFound.

Binding to a port

For this stage, I'll be creating a Transmission Control Protocol (TCP) Server that listens on port 4221. TCP maintains the connection with the sender from before the first part of message is sent to after the final part is sent. TCP is used in conjunction with IP in order to maintain a connection between the sender and the target and to ensure packet order.

A port on the server is dedicated to listening to/expecting the request from the web browser/client.

Here is the code to configure the HTTP server with the address and port:

using System.Net;
using System.Net.Sockets;

TcpListener server = new TcpListener(IPAddress.Any, 4221);
server.Start();
server.AcceptSocket(); // wait for client

Responding with status code 200

For this stage, the server needs to respond to an HTTP request with a 200 status code response. Responses are the HTTP messages a server sends back in reply to a request. The response lets the client know what the outcome of the request was.

The HTTP 200 OK successful response status code indicates that a request has succeeded.

An HTTP response is made up of three parts, each separated by a CRLF (\r\n):

Status line.
Zero or more headers, each ending with a CRLF.
Optional response body.

Here's a breakdown of the 200 response:

// Status line
HTTP/1.1  // HTTP version
200       // Status code
OK        // Optional reason phrase
\r\n      // CRLF that marks the end of the status line

// Headers (empty)
\r\n      // CRLF that marks the end of the headers

// Response body (empty)

Here is the code to send the successful response status code:

using System.Net;
using System.Net.Sockets;
using System.Text;

// ...
Socket socket = server.AcceptSocket(); // wait for client

//Sends a 200 response string to the connected client.
socket.Send(Encoding.UTF8.GetBytes("HTTP/1.1 200 OK\r\n\r\n"));

Extracting the URL path

In this stage, the server needs to extract the URL path from an HTTP request, and respond with either a 200 or 404, depending on the path.

An HTTP request is made up of three parts, each separated by a CRLF (\r\n):

Request line.
Zero or more headers, each ending with a CRLF.
Optional request body.

Here's a breakdown of the request:

// Request line
GET                          // HTTP method
/index.html                  // Request target
HTTP/1.1                     // HTTP version
\r\n                         // CRLF that marks the end of the request line

// Headers
Host: localhost:4221\r\n     // Header that specifies the server's host and port
User-Agent: curl/7.64.1\r\n  // Header that describes the client's user agent
Accept: */*\r\n              // Header that specifies which media types the client can accept
\r\n                         // CRLF that marks the end of the headers

// Request body (empty)

In order to achieve this task on the server, the first step is to read the request sent by the client, then we need to split the request by CRLF (\r\n) to break it into parts. We will be using the first item that it will be the request line, which contains:

<method> <request-target> <protocol>

We need to separate the request line by spaces to extract the request-target, in order to get the requested URL path. I'll be using a dictionary to map valid paths with their corresponding responses. This allows us to validate the path and send the appropriate response with a 200 or 400 status code.

Here is the code to send the successful response status code:

// ...
Socket socket = server.AcceptSocket(); // wait for client

// Read request
byte[] httpRequest = new byte[1024];
socket.Receive(httpRequest);

string requestData = Encoding.UTF8.GetString(httpRequest);

string[] httpRequestParts = requestData.Split("\r\n");
string requestLine = httpRequestParts[0];
string[] requestLineParts = requestLine.Split(" ");
var (httpMethod, requestTarget, httpVersion) = (requestLineParts[0], requestLineParts[1], requestLineParts[2]);

Dictionary<string, string> responses = new Dictionary<string, string>{
  {"/", "HTTP/1.1 200 OK\r\n\r\n" }
};

if (responses.ContainsKey(requestTarget)){
    //Sends a response string to the connected client.
    socket.Send(Encoding.UTF8.GetBytes(responses[requestTarget]));
}
else{
    //Sends a 400 response string to the connected client.
    socket.Send(Encoding.UTF8.GetBytes("HTTP/1.1 404 Not Found\r\n\r\n"));
}

Responding with response body

In this stage, I'll implement the /echo/{str} endpoint, which accepts a string and returns it in the response body.

The response body is used to return content to the client. This content can be an entire web page, a file, a string, or anything else that can be represented by bytes. For this stage, the /echo/ endpoint must respond with a 200 OK status code, a Content-Type and Content-Length headers, and the given string as the response body.

To handle this on the server, the first step is to update how I get the requested URL path to read the endpoint properly. After that, I added a new entry to the dictionary of responses, where the key represents the /echo/ endpoint and the value is a response template.

This allows us to validate the path and generate the appropriate response for this endpoint using the response template. The template contains two placeholders markers: %length% and %message%. %length% will be replaced by the length of the given string, and %message% will be the given string. Once finished, the response built for the client will be sent.

// ...
var (httpMethod, requestTarget, httpVersion) = (requestLineParts[0], requestLineParts[1], requestLineParts[2]);

string urlPath = requestTarget.IndexOfAny("/".ToCharArray(), 1) == -1 ? requestTarget : requestTarget.Substring(0, requestTarget.IndexOfAny("/".ToCharArray(), 1));

Dictionary<string, string> responses = new Dictionary<string, string>
{
  {"/", "HTTP/1.1 200 OK\r\n\r\n" },
  {"/echo", "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: %length%\r\n\r\n%message%" }
};

if (responses.ContainsKey(urlPath))
{
  //Sends a response string to the connected client.
  if (urlPath == "/")
  {
    socket.Send(Encoding.UTF8.GetBytes(responses[urlPath]));
  }
  else if (urlPath == "/echo")
  {
    string data = requestTarget.Replace("/echo/", "");
    string responseTemplate = responses[urlPath];
    responseTemplate = responseTemplate.Replace("%length%", data.Length.ToString()); // Replacing content-length
    responseTemplate = responseTemplate.Replace("%message%", data);                  // Replacing response body
    socket.Send(Encoding.UTF8.GetBytes(responseTemplate));
  }
}
else
{
  //Sends a 400 response string to the connected client.
  socket.Send(Encoding.UTF8.GetBytes("HTTP/1.1 404 Not Found\r\n\r\n"));
}

Reading the header

In this stage, I'll implement the /user-agent endpoint, which reads the User-Agent header and returns it in the response body.

The User-Agent header describes the application, operating system, vendor, and/or version of the client's user agent. For this stage, the /user-agent endpoint must read the User-Agent header and respond with 200 OK status code, a Content-Type and Content-Length headers, and the User-Agent string as the response body.

To handle this on the server, I added a new entry to the dictionary of responses, where the key represents the /user-agent endpoint and the value is a response template.

Once the user-agent endpoint is validated, the next step is to search in the HTTP Request the User-Agent header in order to extract his value. Then, I replace the data in the template response. %length% will be replaced by the length of the user agent value, and %message% will be the user agent value. Once finished, the response built for the client will be sent.

// ...
Dictionary<string, string> responses = new Dictionary<string, string>
{
  {"/", "HTTP/1.1 200 OK\r\n\r\n" },
  {"/echo", "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: %length%\r\n\r\n%message%" },
  {"/user-agent" , "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: %length%\r\n\r\n%message%"}
};

if (responses.ContainsKey(urlPath))
{
  // ...
  else if (urlPath == "/user-agent")
  {
    string userAgentData = httpRequestParts.FirstOrDefault(item => item.StartsWith("user-agent") || item.StartsWith("User-Agent")) ?? "User-Agent: null";
    string userAgentValue = userAgentData.Split(":")[1].Trim();

    string responseTemplate = responses[urlPath];
    responseTemplate = responseTemplate.Replace("%length%", userAgentValue.Length.ToString()); // Replacing content-length
    responseTemplate = responseTemplate.Replace("%message%", userAgentValue);                  // Replacing response body
    socket.Send(Encoding.UTF8.GetBytes(responseTemplate));
  }

}
// ...

Concurrent connections

In this stage, I'll add support for concurrent connections. Currently, the server only handles one request and then exits. To manage multiple requests, I'll add a loop to accept incoming connections, and I'll use an asynchronous task responsible to handle each request.

Additionally, I refactored the code:

I moved the responses dictionary to the top.
I created a new function handleRequest, where all the logic will be performed. This function takes two parameters: the socket and the responses dictionary.

// ...

Dictionary<string, string> responses = new Dictionary<string, string>
{
    // ...
};  

TcpListener server = new TcpListener(IPAddress.Any, 4221);
server.Start();

while (true)
{
     Socket socket = server.AcceptSocket(); // wait for client
     _ = Task.Run(() => handleRequest(socket, responses));
}

void handleRequest(Socket socket, Dictionary<string, string> responses)
{
    Console.WriteLine($"Connection from {socket.RemoteEndPoint} has been established");

    // Read request
    byte[] httpRequest = new byte[1024];

    // ...  

    //Closes the socket to free up system resources
    socket.Close();
}

Returning a file

In this stage, I'll implement the /files/{filename} endpoint, which returns a requested file to the client, also I'll add a --directory flag that specifies the directory where the files are stored, as an absolute path.

In order to achieve this task on the server, the first step was to validate the --directory flag. I checked if the flag was provided to determinate the path. In case this flag wasn't set, a default path is used based on the OS ( C:\tmp\ for Windows, or \tmp\ for Linux). Then I validated that the directory exists. If the directory doesn't exist, the directory will be created automatically.

I added a new entry to the dictionary of responses, where the key represents the /files endpoint and the value is a response template.

Once the /files endpoint is validated, the next step is to extract the requested filename. I generate the full path of the file and validate if the file exists on the server and that the filename is not empty. If the file exists, I get all the information needed, such as the MIME type (based on the file extension) and the file content.

Then, I replace the placeholders in the response template. %mime_type% is replaced with the MIME type of the file, %length% is replaced with the length of the file content, and %message% is replaced with the file content. Once finished, the response built for the client will be sent.

Note: I use the MimeTypes class, which contains a dictionary where the keys are file extensions and the values are their corresponding MIME type. It also includes methods to get the MIME Type information.

If the file doesn't exist on the server, a 404response is sent.

// ...
Console.WriteLine("Logs from your program will appear here!");

string[] arguments = Environment.GetCommandLineArgs();

// If no additional arguments are passed, use a default temporary directory path based on the operating system.
string tmpDirectoryPath = arguments.Length == 1 ? RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ? @"C:\tmp\" : "/tmp/" : arguments[2];

if (!Directory.Exists(tmpDirectoryPath))
{
  Directory.CreateDirectory(tmpDirectoryPath);
  Console.WriteLine("The temporary directory was created successfully");
}

Dictionary<string, string> responses = new Dictionary<string, string>{
    // ...
    {"/files", "HTTP/1.1 200 OK\r\nContent-Type: %mime_type%\r\nContent-Length: %length%\r\n\r\n%message%"}
}

// ...
    else if (urlPath == "/files"){

      string fileNameRequested = requestTarget.Replace("/files/", "");
      string filePath = String.Concat(tmpDirectoryPath, fileNameRequested);

      if (File.Exists(filePath) && fileNameRequested.Length > 0){
        string fileExtension = Path.GetExtension(filePath);
        string mimeType = MimeTypes.GetMimeTypeOrDefault(fileExtension, "application/octet-stream");
        string fileContent = File.ReadAllText(filePath);  // Open the file to read from.

        string responseTemplate = responses[urlPath];
        responseTemplate = responseTemplate.Replace("%mime_type%", mimeType);
        responseTemplate = responseTemplate.Replace("%length%", fileContent.Length.ToString());
        responseTemplate = responseTemplate.Replace("%message%", fileContent);

        socket.Send(Encoding.UTF8.GetBytes(responseTemplate));

      }
      else{
        socket.Send(Encoding.UTF8.GetBytes("HTTP/1.1 404 Not Found\r\n\r\n"));
      } 
    }

// ...

Reading request body

In this stage, I'll add support for the POST method of the /files/{filename} endpoint, which accepts text from the client and creates a new file with that text.

Once the /files endpoint is validated, the next step is to extract the filename and generate the full file path. Then, I checked the HTTP method, determining if it was GET or POST.

If GET is detected, the server returns the content of the file.
If POST is detected, the server creates a new file with the text from the request body.

To handle the POST method, the first step is obtain the value of the Content-Length Header. Next, I extract the content of the request body. To avoid any additional data, I trim the content using the Content-Length value. Finally, I create the file, write the request body content into it, and respond with 201 Created status code.

// ...
else if (urlPath == "/files"){
      string fileNameRequested = requestTarget.Replace("/files/", "");
      string filePath = String.Concat(tmpDirectoryPath, fileNameRequested);

      if (httpMethod == "GET"){
        if (File.Exists(filePath) && fileNameRequested.Length > 0){
          string fileExtension = Path.GetExtension(filePath);

          // ...

          socket.Send(Encoding.UTF8.GetBytes(responseTemplate));
        }else{
          socket.Send(Encoding.UTF8.GetBytes("HTTP/1.1 404 Not Found\r\n\r\n"));
        }
      }
      else if (httpMethod == "POST"){
        string contentLengthHeader = httpRequestParts.Where(item => item.StartsWith("Content-Length") || item.StartsWith("content-length")).First();
        int contentLengthValue = int.Parse(contentLengthHeader.Split(":")[1].Trim()); // Obtain the Content-Lenght value

        string requestBody = httpRequestParts.Last(); // Obtain the request body
        requestBody = requestBody.Substring(0, contentLengthValue); // Avoid any additional request data
        File.WriteAllText(filePath, requestBody); // Create and Write the request body in the file

        socket.Send(Encoding.UTF8.GetBytes("HTTP/1.1 201 Created\r\n\r\n"));
      }
    }
// ...

If you're curious to see how everything came together, you can check out the full project on GitHub: HTTP Server (C#). The code is organized to follow each stage of the process, and I’ve tried to keep it clean and easy to follow. Feel free to explore, fork the repo, or open an issue if you have ideas or suggestions—I'd love to hear your thoughts!

Conclusion

Through this challenge, I built an HTTP server that's capable of handling simple GET and POST requests, serving files and handling multiple concurrent connections. I learnt about some key concepts such as TCP connections, HTTP headers, HTTP verbs, handling multiple connections, and file serving.

Step-by-Step Guide to Creating an HTTP Server in C# - Part 1