WTF is multipart/form-data in HTTP

Introduction

I've always wondered how files are sent to the server in an HTTP request. Whenever I need to create a UI for file upload on the frontend and send it to the backend using an API call, I set the content-type to multipart/form-data in my HTTP headers. But what is multipart/form-data and how does it actually send the data to the server? Let's dive deep into it...

Handling form data

What is form data? Form data is a format for sending data from a web form. Let's say you have a form in your application that a user needs to fill out and submit, which essentially makes a POST request. When you make a POST request, you need to encode the data that forms the body of the request in some way. HTML forms provide three methods of encoding.

• application/x-www-form-urlencoded (the default)

• multipart/form-data

• text/plain

text/plain is rarely used because it doesn't encode special characters and has issues with new lines (read more about it here). application/x-www-form-urlencoded is the default encoding when you use the <form/> tag in your HTML code unless you specify the enctype as multipart/form-data. It is recommended to use multipart/form-data when the form data includes complex data like files, binary data, or non-ASCII characters. Otherwise, application/x-www-form-urlencoded is fast and efficient for small and simple data but not suitable for file uploads.

Don't worry about having a <form/> tag in your code whenever you have to upload files in the frontend. You can always use let fd = new FormData() and append whatever data you wanna send.

Structure of multipart/form-data

Let's examine the structure of multipart/form-data by setting up a simple loop that listens for incoming connections on port 8002 using netcat (nc).

The command will continuously listen for connections on port 8002. When a connection is made, it doesn't send any data (printf '' sends an empty string), and the loop restarts, ready to accept a new connection. Now, let's make a connection and send a file to the server at port 8002 using the command curl -F secret=@tick.gif http://localhost:8002, where tick.gif is a file in my folder. This command sends the file in an HTTP request. The -F flag tells curl to send the data as a multipart/form-data request. Let's look at the data printed by our while loop.

We can see that the host is localhost:8002 and the Content-Type is multipart/form-data. After that, we see a boundary and the binary data of the file starts. Here's how the data is sent when there are two (name, value) pairs. For example, the form sends two things to the server: a name and an image.

--boundary
Content-Disposition: form-data; name="name"

John Doe
--boundary
Content-Disposition: form-data; name="file"; filename="photo.jpg"
Content-Type: image/jpeg

(binary data)
--boundary--

The form data is divided into multiple parts and separated by a boundary. Each part includes headers that describe the content, followed by the data (text, binary, etc.). Now you know how multipart/form-data actually sends the data.

Conclusion

multipart/form-data encoding is a good choice when your form includes complex data like files. For simple and small data, it is not as efficient as application/x-www-form-urlencoded because of the overhead from boundary markers, which are needed for sending files and mixed data types.