S3 Mountpoint Introduction

Mountpoint for S3 translates local file system API Calls to S3 Object API Calls. Presents Amazon S3 Objects as files in the local file system. Amazon S3 Mountpoint seamlessly integrates with Linux-based systems, allowing file-aware applications to connect directly to Amazon S3 buckets without the need for complex configurations or additional software.

In the realm of cloud storage solutions, Amazon Simple Storage Service (Amazon S3) stands tall as a reliable and scalable option for businesses of all sizes. Its flexibility, durability, and cost-effectiveness have made it a go-to choice for storing vast amounts of data securely in the cloud. However, integrating Amazon S3 with Linux/Windows server applications has often presented challenges, requiring additional tools and configurations. Enter Mountpoint for Amazon S3, an open-source file client that simplifies this process, enabling seamless connectivity between Application Servers and Amazon S3 buckets.

Key Characteristics

• Mountpoint for Amazon S3 boasts high performance and high throughput

• Scalability: Whether you're dealing with small files or large datasets, Amazon S3 Mountpoint scales effortlessly to accommodate your storage operation needs, providing seamless access to Amazon S3 directly from your Compute instance.

• Cost-Effective: By eliminating the need for additional hardware or proprietary solutions, Amazon S3 Mountpoint helps reduce infrastructure costs associated with data storage and management.

• Reliability and Durability: Leveraging the robust infrastructure of Amazon S3, Amazon S3 Mountpoint offers high reliability and durability, with built-in redundancy and data protection mechanisms to safeguard against data loss or corruption.

• Encryption and Security: Amazon S3 Mountpoint prioritizes data security by supporting encryption in transit and at rest, ensuring that sensitive data remains protected throughout the transfer process.

• Use case: Mountpoint for Amazon S3 is ideal for workloads that read large datasets (terabytes to petabytes in size). Common use cases include large-scale machine learning (ML) training, autonomous vehicle simulation, genomics analysis, and image rendering. While these workloads read large datasets over several compute instances, they write sequentially to a file from a single node. This means they do not need shared file system features such as locking.

Mountpoint for Amazon S3 serves as a bridge between Linux-based systems and Amazon S3 buckets, facilitating direct access to stored data without the need for complex configurations or additional software layers. By mounting S3 buckets as local filesystems, Mountpoint enables Linux applications to interact with S3 objects as if they were traditional files and directories, streamlining data management and enhancing compatibility.

Note: For applications that require shared file system features such as file locking and POSIX permissions, you can use Amazon FSx for Lustre with a data repository association to your S3 bucket.

How it works

Demo

1. Download the Mountpoint for the Amazon S3 package "mount-s3.rpm" .

    wget https://s3.amazonaws.com/mountpoint-s3-release/latest/x86_64/mount-s3.rpm
   

   

2. Install the package by using the following command:

    sudo yum install ./mount-s3.rpm
   

   

   

3. Check the Mount S3 Version

   

   I have uploaded some .jpg files onto my S3 Bucket name "mountpoint-bucket"

   

4. To use Mountpoint for Amazon S3, your host needs valid AWS credentials with access to the bucket or buckets that you would like to mount. For example, you can create a new AWS Identity and Access Management (IAM) user and role for this purpose. Make sure that this role has access to the bucket or buckets that you would like to mount.
   OR You can pass the IAM role to your Amazon EC2 instance with an instance profile. As shown below, I created a IAM Role that has access to S3 Bucket and have EC2 instance assume this role.

   

5. Run the mount-s3 command with s3 bucket ARN with instance mount directory

   run

   mkdir mntdir

   

   S3 BUCKET is MOUNTED using Mountpoint file client.

6. We can check the files (Objects) inside the S3 bucket directly from the mounted S3 mount point.

   

   from here, After you mount your bucket locally, you can use common Linux commands, such as cat or ls, to work with your S3 objects.

Unmount your bucket by using the umount command. This command unmounts your S3 bucket and exits Mountpoint.

Limitation of Mountpoint

\> Unlike FUSE, Mountpoint mounts cannot be added to your fstab file so do not persist after stopping your machine. They need to be re-mounted each time the machine is restarted.

\> Mountpoint for Amazon S3 is available only for Linux operating systems

\> Mountpoint cannot access files archived in S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive tiers.

Here in this demo, I have used Mountpoint to access the existing S3 bucket mounted on Linux VM which contained image files which I can perform action against and Copy or push to S3 to my local mount directory and seamlessly push new images to S3 bucket using mountpoint.

Hope this Blog helps understand Amazon S3 Mountpoint capabilities and give you insights about the amazing new ways to connect with S3 from your Linux(Application) Instance.

For more information on this, please follow Working with Mountpoint for Amazon S3

Thank you for the read. Hope you like it.
I appreciate your time.

Follow for more Azure and AWS Content. Happy Learning!

Regards,
Jineshkumar Patel