Uploading large files (up to 5TB) efficiently and securely from a React application to an AWS S3 bucket is a common use case. In this blog, I'll walk you through implementing this solution using pre-signed URLs, with AWS Lambda handling backend logic. This approach improves performance and security by bypassing the backend server for file transfer.

Overview

This use case demonstrates:

Setting up CORS for an S3 bucket.
Generating pre-signed URLs using AWS Lambda (Python).
Directly uploading files from a React frontend to S3.

Technologies Used

AWS Lambda: For generating pre-signed URLs.
AWS S3: Secure and scalable file storage.
React (TypeScript): Frontend for user interactions.
AWS API Gateway: To expose Lambda as an HTTP API.

Step 1: Configure CORS for the S3 Bucket

The first step is to enable Cross-Origin Resource Sharing (CORS) on your S3 bucket. This allows your React frontend to interact with the bucket securely. Use the following configuration:

[
  {
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["PUT"],
    "AllowedOrigins": ["*"],
    "ExposeHeaders": ["ETag"],
    "MaxAgeSeconds": 3000
  }
]

Explanation:

AllowedHeaders: Headers allowed in requests (e.g., Content-Type).
AllowedMethods: HTTP methods permitted (e.g., PUT for uploads).
AllowedOrigins: Allowed domains (* for all origins).
ExposeHeaders: Exposed response headers.
MaxAgeSeconds: Cache duration for preflight responses.

Step 2: Backend Logic to Generate Pre-Signed URL

Lambda Function to Generate Pre-Signed URL

Here’s a Python snippet for generating pre-signed URLs using boto3:

import mimetypes
import boto3
import logging
from botocore.exceptions import NoCredentialsError

s3 = boto3.client('s3')
bucket_name = "your-s3-bucket-name"
logger = logging.getLogger()

def send_api_endpoint(body, prefix):
    file_name = body.get('file_name')
    sub_folder = body.get('sub_folder')
    attachment_type = body.get('attachment_type')
    ticket_id = body.get('ticket_id')

    s3_object_key = f"{sub_folder}/{attachment_type}/{ticket_id}/{file_name}"

    def get_content_type(file_name):
        content_type, _ = mimetypes.guess_type(file_name)
        return content_type or "application/octet-stream"

    content_type = get_content_type(file_name)

    try:
        uri = s3.generate_presigned_url(
            'put_object',
            Params={
                'Bucket': bucket_name,
                'Key': s3_object_key,
                'ContentType': content_type
            },
            ExpiresIn=3600  # 1 hour
        )
        return {
            "statusCode": 200,
            "body": {
                "message": "Attachment Added Successfully",
                "uri": uri,
                "content_type": content_type
            }
        }
    except NoCredentialsError:
        logger.error("Credentials not available")
        return {"statusCode": 500, "body": {"message": "Credentials error"}}

Step 3: Frontend Logic in React

Request Pre-Signed URL

On the frontend, request the pre-signed URL from the backend:

const onFinish = (values) => {
  const filename = values.file.file.name;
  const data = {
    event_type: "ext_upload_access",
    file_name: filename,
    sub_folder: "ext_emails",
    attachment_type: "attachments",
    ticket_id: "12345", // Example ticket ID
  };

  fetch(process.env.REACT_APP_API_URL, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-key": process.env.REACT_APP_API_KEY,
    },
    body: JSON.stringify(data),
  })
    .then((response) => response.json())
    .then((res) => {
      if (res.statusCode === 200) {
        const { uri, content_type } = res.body;
        AddToS3(values, uri, content_type);
      } else {
        console.error("Failed to fetch pre-signed URL");
      }
    })
    .catch(console.error);
};

Upload File to S3

Once the pre-signed URL is retrieved, upload the file directly to S3:

const AddToS3 = (values, api_endpoint, content_type) => {
  const bigfile = values.file.file;
  fetch(api_endpoint, {
    method: "PUT",
    headers: {
      "Content-Type": content_type,
    },
    body: bigfile,
  })
    .then((response) => {
      if (response.ok) {
        alert("File uploaded successfully");
      } else {
        throw new Error("Upload failed");
      }
    })
    .catch((error) => console.error("Error uploading to S3:", error));
};

How It Works

Frontend Requests Pre-Signed URL: The React app sends a POST request with file details to the Lambda backend via API Gateway.
Lambda Generates URL: AWS Lambda generates a pre-signed URL using boto3.
Direct Upload to S3: The React app uses the pre-signed URL to upload the file directly to S3.

Benefits of This Approach

Improved Performance: File uploads bypass the backend, reducing server load.
Enhanced Security: Pre-signed URLs are time-bound and grant limited access.
Scalability: AWS S3 handles large file uploads (up to 5TB) seamlessly.

Conclusion

Uploading files to AWS S3 using pre-signed URLs is an efficient and secure way to handle large files in your React application. By integrating AWS Lambda and API Gateway, you can scale this solution while keeping your backend lightweight and secure.

You can now easily integrate this workflow into your projects! If you have questions or want to share your thoughts, feel free to leave a comment below. 🚀

serverless uploading to AWS S3