AWS S3 CLI Uploader

Salutations Fellow Cloud Enthusiasts,

Welcome to today’s blog, where we’ll dive into the fascinating world of the Command Line Interface (CLI) and Bash scripting. If you’re on a journey to enhance your cloud computing skills, mastering these tools is a crucial step. Whether you’re just starting out or looking to sharpen your existing knowledge, this post will offer you practical insights through a project you can complete, using the Linux CLI and Bash scripting to interact with AWS services.

What to Expect.

In this post, I’ll walk you through:

1. The Project: How I used the CLI and Bash to create the CloudUploader, a tool for uploading files to cloud storage.

2. Lessons Learned: Key takeaways and best practices for working with the CLI and Bash in a cloud environment.

By the end of this entry, I hope you’ll feel inspired to take on your own projects and gain hands-on experience with these powerful tools. So, grab your preferred selection of caffeine (coffee for me), and let’s get started!

Getting Started: My Experience with the CLI and Bash

First things first, going into this project I already had played around with the CLI and Bash and was somewhat comfortable with both. So, my experience will be different from yours depending on your skill level with these tools. Regardless I still learned a lot and ran into my own problems, mainly with the bash portion of it since I'm still novice. Let me also start off by saying that no matter what level you're at, you need to be comfortable with solving problems on your own, and eventually get good at it. Think about it most System Administrators, Jr and Sr Developers, and even IT Support just get really good at identifying the problem, searching for possible solutions, and testing/implementing. As a Jr Developer or SysAdmin, solving problems independently and documenting them will fast-track your career. You are gonna get ahead a lot quicker and get that promotion or position a lot faster vs the other guy; when the time comes. Be a problem solver not a problem bringer.

Alright enough of that, lets get into it.

The CloudUploader Project: An Overview

The CloudUploader CLI is a Bash based tool created/used for seamless upload of files to a specified cloud storage service. In my case of course I used AWS S3 standard. It's also the capstone project for the 1st phase of the Learn to Cloud study guide. Phase 1 focuses on learning Linux CLI, Bash scripting, and computer networking with good resources for all 3 and obviously a project to top it off.

Learn more details or start your Learn to Cloud journey here: Learn to Cloud

Disclaimer: Prerequisites for this project. Get familiar with the basics of AWS Cloud and working in the console. I recommend studying the material found in the Cloud Practitioner exam. Also get familiar with the AWS docs as this will become your best friend and eventually go-to resource when working in the cloud.

Get familiar with the documentation here: AWS DOCS

Step 1: Provisioning Cloud Resources

Pretty straight forward, to start building this for yourself I would start with provisioning your cloud resources for this project. If you don't already have a free tier AWS account(most cloud providers have this tier) I highly recommend doing that first. There's plenty of tutorials and documentation on how to get started so I'll let you pick a resource for that.

After creating your very own AWS free tier account you will be logged in as "root" and free to start building, but wait a second! I seriously advise that you go ahead and create an admin account to work with as it is not best practice and generally unsafe to work as root(also best to set up Multi-Factor Authentication for both root and admin. DO NOT FORGET YOUR LOGIN CREDENTIALS FOR ROOT). If we keep security in mind first we don't have to deal with more headaches later.

Your next step after creating an admin account will be to create an S3 bucket that you want to use for uploading files. you'll be able to use and upload to more than just one S3 bucket, but for now let's just provision one. call it "cli-uploader" or "cloud-uploader-project" whatever makes sense to you.

Step 2: Installing and Configuring the AWS CLI

Next your going to want to download and install the AWS-CLI. To configure the AWS-CLI you will need IAM credentials(access key, secret key), now you can either use your admin account or create a specific IAM user for this project. Do not, I repeat, please do NOT use the root account for this. Your root account shouldn't even have access key credentials to begin with.

I chose to create an IAM user specifically for this project with only access to the CLI not the console, with admin privileges. Best practice here would be to give the user access to only what you need(least-privelege), but for this case I went ahead and gave admin rights.

Helpful links:

CLI Installation

Configure CLI

Step 3: Bash Scripting and Next Steps

Alright great! You have your cloud environment set up, AWS CLI installed and configured, you are ready to start that Bash script. For this part I recommend doing some of your own research and practicing with Bash before you start coding.

The big issue i ran into during this step was learning the syntax, I had already taken programing I & II in Java so was familiar with most programing concepts. To really get familiar with Bash syntax I followed a YouTube course from Gwyneth Peña S. as well as looking on GitHub for people who did the same project and seeing how they did it.

You can look at my GitHub to help you start but the fun part about this is your only limited to what you know. This makes this project scalable in a sense, since you can always work on bugs, add new features, and refactor as you learn throughout time. Hell if you wanted you can take this project all the way to the finish line; setting up a CICD pipeline, configuring Infrastructure as Code, and even containerize using Docker if you really want. Like I said you're only limited to what you know.

My GitHub

Engage with Me: Share Your Experience!

I’d love to hear from you!

• What’s your experience with CLI and Bash?

• Have you encountered any specific challenges while working with these tools?

Feel free to share your thoughts in the comments below or reach out to me directly. Engaging with fellow learners is a great way to enhance our collective knowledge, so don’t hesitate to join the conversation! If you enjoyed this blog, consider subscribing to my newsletter to stay up-to-date with AWS and IT in general. Let’s continue this journey of learning together!