Spring boot: Scale file storage with Amazon S3

You should learn this first

---

If you already have a bucket, secret, access key, and bucket region then you can skip these steps

Let's create a sample s3 bucket.

A bucket is a container for objects stored in Amazon S3. You can store any number of objects in a bucket and can have up to 100 buckets in your account.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingBucket.html

Here is a guide link on how to create a bucket in Amazon S3.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html

How to create a secret and access key?

https://docs.aws.amazon.com/powershell/latest/userguide/creds-idc.html

Now we have bucket, secret, access key, and bucket region.

---

Let's create a spring boot application and add this dependency.

Gradle:
implementation 'com.amazonaws:aws-java-sdk-s3:1.12.402'

Maven:
<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk-s3</artifactId>
    <version>1.12.402</version>
</dependency>

https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-s3

2. Create the Configuration class.
You have access_key, secret_key and your-bucket-region. let's create a connection between your application and the Amazon S3.

@Configuration
public class AwsS3Client {

    @Bean
    public AmazonS3 getS3Client() {
        BasicAWSCredentials awsCreds = new BasicAWSCredentials("access_key"
              , "secret_key");

        return AmazonS3ClientBuilder.standard()
                .withCredentials(new AWSStaticCredentialsProvider(awsCreds))
                .withRegion(Regions.fromName("your-bucket-region"))
                .build();
    }

1. Create a utility class.

   consider

Bucket Name: student-reports
String cloud path = "/prod/student";

---

1. Upload file

/**
    cloudFilePath is Dir where file to be uploaded.
**/
private final AmazonS3 amazonS3client;

public void uploadToS3(String bucketName, String cloudPath, File file) {
    PutObjectRequest putObjectRequest = new PutObjectRequest(bucketName, cloudPath, file);
    amazonS3client.putObject(putObjectRequest);
}
// caller function
uploadToS3("student-reports", "/prod/student/report_10.csv", new File("/usr/Download/report_10.csv"));

The bucket is nothing but a directory and a cloud path is a folder structure inside the bucket or directory.

Folder structure created inside the bucket.

1. Check if the file exists.

private final AmazonS3 amazonS3client;

public boolean doesObjectExistInS3(String bucketName, String key) {
    return amazonS3client.doesObjectExist(bucketName, key);
}

doesObjectExistInS3("student-reports", "/prod/student/report_10.csv");

3. Delete file.

private final AmazonS3 amazonS3client;
public void deleteObject(String bucketName, String key) {
    amazonS3client.deleteObject(bucketName, key);
}
// caller function
deleteObject("student-reports", "/prod/student/report_10.csv");

1. Deleting Multiple Objects

public void deleteFilesFromS3(String bucketName, List<KeyVersion> keys) {
     try {
            // Delete the objects.
            DeleteObjectsRequest multiObjectDeleteRequest = new DeleteObjectsRequest(bucketName)
                .withKeys(keys)
                .withQuiet(false);

            // Verify that the objects were deleted successfully.
            DeleteObjectsResult delObjRes = amazonS3client.deleteObjects(multiObjectDeleteRequest);

            int successfulDeletes = delObjRes.getDeletedObjects().size();

        } catch (AmazonServiceException e) {
            // The call was transmitted successfully, but Amazon S3 couldn't process
            // it, so it returned an error response.
            e.printStackTrace();
        } catch (SdkClientException e) {
            // Amazon S3 couldn't be contacted for a response, or the client
            // couldn't parse the response from Amazon S3.
            e.printStackTrace();
        }
}

// caller function 
deleteFilesFromS3("student-reports". List.of(new KeyVersion("/prod/student/report_10.csv")));

or

String objkeyArr[] = {
  "/prod/student/report_10.csv", 
  "/prod/student/report_11.csv",
  "/prod/student/report_1.csv",
};

DeleteObjectsRequest multiObjectDeleteRequest = new DeleteObjectsRequest("student-reports")
  .withKeys(objkeyArr);

amazonS3client.deleteObjects(delObjReq);

5. Rename the object

You can't rename the s3 object.
Steps:

• Copy the file/object with a new name.

Delete old files/objects.

Eg: Renaming the object name from “/prod/student/report_10.csv” to “/prod/student/report_40.csv”

private final AmazonS3 amazonS3client;

public void renameObject(String sourceBucketName, String sourceKeyName, String destinationBucketName,
String  destinationKeyName){
    CopyObjectRequest copyObjRequest = new CopyObjectRequest(sourceBucketName, 
           sourceKeyName, destinationBucketName, destinationKeyName);

    amazonS3client.copyObject(copyObjRequest);
    amazonS3client.deleteObject(new DeleteObjectRequest(sourceBucketName, sourceKeyName));
}

// caller function
renameObject("student-reports", "/prod/student/report_10.csv", "student-reports", 
 "/prod/student/report_40.csv");

6. Copying, Moving the object

CopyObjectRequest copyObjRequest = new CopyObjectRequest(bucketName, 
           keyName, bucketName, destinationKeyName);

amazonS3client.copyObject(copyObjRequest);

7. Download the object

S3Object s3object = amazonS3client.getObject("student-reports", "/prod/student/report_10.csv");
S3ObjectInputStream objectInputStream = amazonS3client.getObjectContent();
FileUtils.copyInputStreamToFile(objectInputStream, new File("/usr/Download/report_10.csv"));

Here is the complete Utility class.

Now you can leverage AWSFileUploadService in your application.

Hope you enjoyed reading!
Your support means a lot. Feel free to like and share if you find it valuable. Thanks for your time! 🙏