Speed Up File Uploads with Signed URLs

What are Signed URL’s?

Signed URLs are secure, time-limited links that let users upload or download files directly from cloud storage (like GCP or AWS) without a server processing the request in between.

A simply analogy would be - Imagine you're renting a locker at a gym. You get a special one-time key that lets you open it just once for the next 5 minutes. You can’t reuse it after that.

We’ll explore the problem with traditional file uploads, and how leveraging signed URLs benefits us in both performance and operating costs by minimizing upload latencies and network bandwidth.

Traditional approach for uploading files to cloud.

As the diagram explains, in the traditional approach, we first send the data or file to a specific API. After the API validates the user and the data, if the data is in the expected format, the code for uploading the files to the cloud will run , We can see the issue here: there's an extra layer involved in uploading files to the cloud.

The biggest disadvantage here is that the server has to do a lot of heavy lifting. All uploads must pass through the API, which increases the total latency from the client to the cloud. This latency can be even worse if the API is hosted in a different location from your storage. We also can't ignore the bandwidth issue if the file size is large.

Let’s have a look how Signed URL helps us,

A signed URL is a link used to perform specific tasks on the cloud, such as uploading or downloading. It is created using the credentials of your cloud provider and has an expiration attached to it for safety purposes, which we will discuss later in the blog. For example, if you are using AWS S3 or a GCP bucket, you would use their credentials to create a signed URL.

Let see a example, let’s see how in a web-app where user uploads an file from their browser to cloud using signed URL.

In this process, your backend will generate a signed URL, and you will have an API endpoint for that purpose. So, whenever a user clicks on upload after selecting a file, this API will be called first. It will return the signed URL, which will then be used to make another request from the frontend directly to this URL.

API example:

That's a signed URL:

• Valid only for a limited time

• So users can upload files directly to storage (S3/GCP) without hitting your server

• Grants specific access (upload / download)

Some things to remember:

• Signed URL can be used by anyone until its not expired, So be careful while creating one make sure to give quick expiry , Anyone with bad intention can use it to put anything fishy on your cloud.

• Signed URLs do not perform validation for you, such as checking the content and type of file you are uploading to the cloud. You need to be careful and validate the file yourself before sending it to the signed URL.

• Signed URL authorization occurs when it is used, not when it is created. Therefore, if the permission for upload is removed or changed between the creation time and expiration time, it won't work.