S3 Programmatic Access with AWS-CLI
Introduction to Amazon S3
Amazon Simple Storage Service (Amazon S3) is a service offered by Amazon Web Services (AWS) that provides object storage through a web service interface. S3 is used for storing data such as text files, images, videos, backups, and more. It’s highly scalable, secure, and durable, making it ideal for a wide range of applications.
Task-01: Launching an EC2 Instance and Interacting with S3
Step 1: Launch an EC2 Instance
Open the AWS Management Console:
Go to the AWS Management Console at AWS Console.
Sign in to your AWS account.
Navigate to EC2 Dashboard:
- In the AWS Management Console, click on "Services" and select "EC2" under "Compute."
Launch an Instance:
- Click on the "Launch Instance" button.
Choose an Amazon Machine Image (AMI):
- Select the "Amazon Linux 2 AMI (HVM), SSD Volume Type."
Choose an Instance Type:
- Select the "t2.micro" instance type, which is eligible for the AWS free tier.
Configure Instance Details:
- Click "Next: Configure Instance Details." You can keep the default settings here and click "Next: Add Storage."
Add Storage:
- The default storage size is typically sufficient. Click "Next: Add Tags."
Add Tags:
- (Optional) Add tags to your instance to help manage and identify it later. Click "Next: Configure Security Group."
Configure Security Group:
Create a new security group or select an existing one.
Add a rule to allow SSH traffic from your IP address:
Type: SSH
Protocol: TCP
Port Range: 22
Source: My IP
Review and Launch:
Click "Review and Launch."
Click "Launch."
Select an existing key pair or create a new one. Make sure to download the key pair (.pem file) as you will need it to connect to your instance.
Step 2: Connect to Your EC2 Instance
Open your terminal (Mac/Linux) or Git Bash (Windows).
Connect using SSH:
ssh -i "your-key-pair.pem" ec2-user@your-ec2-public-dns
Replace
your-key-pair.pem
with the name of your key pair file andyour-ec2-public-dns
with the Public DNS (IPv4) of your instance, which can be found on the instance details page in the AWS console.
Step 3: Create an S3 Bucket and Upload a File
Create an S3 Bucket using the AWS Management Console:
In the AWS Management Console, go to "Services" and select "S3" under "Storage."
Click "Create bucket."
Enter a unique bucket name and select the region where you want the bucket to be created.
Click "Create bucket."
Upload a File to S3:
Select your newly created bucket.
Click "Upload."
Add files by dragging and dropping or selecting them from your file system.
Click "Upload" to start the upload process.
Step 4: Access the File from EC2 using AWS CLI
Install AWS CLI on EC2 (if not already installed):
sudo yum install aws-cli -y
Configure AWS CLI:
aws configure
- Enter your AWS Access Key ID, Secret Access Key, region, and output format when prompted.
List Buckets to Verify:
aws s3 ls
This command lists all the S3 buckets in your AWS account.
Download the File from S3 to EC2:
aws s3 cp s3://your-bucket-name/your-file-name .
Replace
your-bucket-name
with the name of your S3 bucket andyour-file-name
with the name of the file you uploaded. This command copies the file from S3 to your EC2 instance.
Task-02: Creating a Snapshot and Launching a New EC2 Instance
Step 1: Create a Snapshot of the EC2 Instance
Navigate to the EC2 Dashboard:
- In the AWS Management Console, go to "Services" and select "EC2."
Create a Snapshot:
Select "Volumes" from the left menu.
Choose the volume attached to your instance.
Click "Actions" and select "Create Snapshot."
Name and Create the Snapshot:
Enter a name and description for the snapshot.
Click "Create Snapshot."
Step 2: Launch a New EC2 Instance from the Snapshot
Navigate to the Snapshots Section:
In the EC2 dashboard, select "Snapshots" from the left menu.
Select the snapshot you created.
Create a Volume from the Snapshot:
Click "Actions" and select "Create Volume."
Specify the Availability Zone.
Click "Create Volume."
Attach the Volume to a New EC2 Instance:
Launch a new EC2 instance (same steps as in Task-01).
After launching, go to the "Volumes" section.
Select the volume created from the snapshot.
Click "Actions" and select "Attach Volume."
Choose the new EC2 instance to attach the volume.
Step 3: Verify the File Download on the New Instance
Connect to the New EC2 Instance via SSH:
ssh -i "your-key-pair.pem" ec2-user@your-new-ec2-public-dns
Replace
your-new-ec2-public-dns
with the Public DNS (IPv4) of your new instance.Download the File from S3:
aws s3 cp s3://your-bucket-name/your-file-name .
Verify File Contents:
cat your-file-name
This command displays the contents of the downloaded file, allowing you to verify that it is the same as the original file.
Useful AWS CLI Commands for S3
Here are some common AWS CLI commands you can use to interact with S3:
List Buckets:
aws s3 ls
Create a Bucket:
aws s3 mb s3://bucket-name
Replace
bucket-name
with your desired bucket name.Delete a Bucket:
aws s3 rb s3://bucket-name
Upload a File:
aws s3 cp file.txt s3://bucket-name
Replace
file.txt
with your file name andbucket-name
with your bucket name.Download a File:
aws s3 cp s3://bucket-name/file.txt .
Sync a Local Folder with S3:
aws s3 sync local-folder s3://bucket-name
Replace
local-folder
with your local directory andbucket-name
with your bucket name.List Objects in a Bucket:
aws s3 ls s3://bucket-name
Delete an Object:
aws s3 rm s3://bucket-name/file.txt
Generate a Pre-signed URL:
aws s3 presign s3://bucket-name/file.txt
This command generates a URL that provides temporary access to the object.
List Buckets using S3 API:
aws s3api list-buckets
Conclusion
By completing these tasks, you have gained hands-on experience with launching and connecting to EC2 instances, creating and managing S3 buckets, and using the AWS CLI to interact with S3. These are fundamental skills for managing AWS resources and are crucial for DevOps practices.
Subscribe to my newsletter
Read articles from Urvish Suhagiya directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by
Urvish Suhagiya
Urvish Suhagiya
Exploring the world of DevOps 🌐.