S3 Programmatic Access with AWS-CLI

Urvish SuhagiyaUrvish Suhagiya
5 min read

Introduction to Amazon S3

Amazon Simple Storage Service (Amazon S3) is a service offered by Amazon Web Services (AWS) that provides object storage through a web service interface. S3 is used for storing data such as text files, images, videos, backups, and more. It’s highly scalable, secure, and durable, making it ideal for a wide range of applications.

Task-01: Launching an EC2 Instance and Interacting with S3

Step 1: Launch an EC2 Instance

  1. Open the AWS Management Console:

    • Go to the AWS Management Console at AWS Console.

    • Sign in to your AWS account.

  2. Navigate to EC2 Dashboard:

    • In the AWS Management Console, click on "Services" and select "EC2" under "Compute."
  3. Launch an Instance:

    • Click on the "Launch Instance" button.
  4. Choose an Amazon Machine Image (AMI):

    • Select the "Amazon Linux 2 AMI (HVM), SSD Volume Type."
  5. Choose an Instance Type:

    • Select the "t2.micro" instance type, which is eligible for the AWS free tier.
  6. Configure Instance Details:

    • Click "Next: Configure Instance Details." You can keep the default settings here and click "Next: Add Storage."
  7. Add Storage:

    • The default storage size is typically sufficient. Click "Next: Add Tags."
  8. Add Tags:

    • (Optional) Add tags to your instance to help manage and identify it later. Click "Next: Configure Security Group."
  9. Configure Security Group:

    • Create a new security group or select an existing one.

    • Add a rule to allow SSH traffic from your IP address:

      • Type: SSH

      • Protocol: TCP

      • Port Range: 22

      • Source: My IP

  10. Review and Launch:

    • Click "Review and Launch."

    • Click "Launch."

    • Select an existing key pair or create a new one. Make sure to download the key pair (.pem file) as you will need it to connect to your instance.

Step 2: Connect to Your EC2 Instance

  1. Open your terminal (Mac/Linux) or Git Bash (Windows).

  2. Connect using SSH:

     ssh -i "your-key-pair.pem" ec2-user@your-ec2-public-dns
    

    Replace your-key-pair.pem with the name of your key pair file and your-ec2-public-dns with the Public DNS (IPv4) of your instance, which can be found on the instance details page in the AWS console.

Step 3: Create an S3 Bucket and Upload a File

  1. Create an S3 Bucket using the AWS Management Console:

    • In the AWS Management Console, go to "Services" and select "S3" under "Storage."

    • Click "Create bucket."

    • Enter a unique bucket name and select the region where you want the bucket to be created.

    • Click "Create bucket."

  2. Upload a File to S3:

    • Select your newly created bucket.

    • Click "Upload."

    • Add files by dragging and dropping or selecting them from your file system.

    • Click "Upload" to start the upload process.

Step 4: Access the File from EC2 using AWS CLI

  1. Install AWS CLI on EC2 (if not already installed):

     sudo yum install aws-cli -y
    
  2. Configure AWS CLI:

     aws configure
    
    • Enter your AWS Access Key ID, Secret Access Key, region, and output format when prompted.
  3. List Buckets to Verify:

     aws s3 ls
    

    This command lists all the S3 buckets in your AWS account.

  4. Download the File from S3 to EC2:

     aws s3 cp s3://your-bucket-name/your-file-name .
    

    Replace your-bucket-name with the name of your S3 bucket and your-file-name with the name of the file you uploaded. This command copies the file from S3 to your EC2 instance.


Task-02: Creating a Snapshot and Launching a New EC2 Instance

Step 1: Create a Snapshot of the EC2 Instance

  1. Navigate to the EC2 Dashboard:

    • In the AWS Management Console, go to "Services" and select "EC2."
  2. Create a Snapshot:

    • Select "Volumes" from the left menu.

    • Choose the volume attached to your instance.

    • Click "Actions" and select "Create Snapshot."

  3. Name and Create the Snapshot:

    • Enter a name and description for the snapshot.

    • Click "Create Snapshot."

Step 2: Launch a New EC2 Instance from the Snapshot

  1. Navigate to the Snapshots Section:

    • In the EC2 dashboard, select "Snapshots" from the left menu.

    • Select the snapshot you created.

  2. Create a Volume from the Snapshot:

    • Click "Actions" and select "Create Volume."

    • Specify the Availability Zone.

    • Click "Create Volume."

  3. Attach the Volume to a New EC2 Instance:

    • Launch a new EC2 instance (same steps as in Task-01).

    • After launching, go to the "Volumes" section.

    • Select the volume created from the snapshot.

    • Click "Actions" and select "Attach Volume."

    • Choose the new EC2 instance to attach the volume.

Step 3: Verify the File Download on the New Instance

  1. Connect to the New EC2 Instance via SSH:

     ssh -i "your-key-pair.pem" ec2-user@your-new-ec2-public-dns
    

    Replace your-new-ec2-public-dns with the Public DNS (IPv4) of your new instance.

  2. Download the File from S3:

     aws s3 cp s3://your-bucket-name/your-file-name .
    
  3. Verify File Contents:

     cat your-file-name
    

    This command displays the contents of the downloaded file, allowing you to verify that it is the same as the original file.

Useful AWS CLI Commands for S3

Here are some common AWS CLI commands you can use to interact with S3:

  • List Buckets:

      aws s3 ls
    
  • Create a Bucket:

      aws s3 mb s3://bucket-name
    

    Replace bucket-name with your desired bucket name.

  • Delete a Bucket:

      aws s3 rb s3://bucket-name
    
  • Upload a File:

      aws s3 cp file.txt s3://bucket-name
    

    Replace file.txt with your file name and bucket-name with your bucket name.

  • Download a File:

      aws s3 cp s3://bucket-name/file.txt .
    
  • Sync a Local Folder with S3:

      aws s3 sync local-folder s3://bucket-name
    

    Replace local-folder with your local directory and bucket-name with your bucket name.

  • List Objects in a Bucket:

      aws s3 ls s3://bucket-name
    
  • Delete an Object:

      aws s3 rm s3://bucket-name/file.txt
    
  • Generate a Pre-signed URL:

      aws s3 presign s3://bucket-name/file.txt
    

    This command generates a URL that provides temporary access to the object.

  • List Buckets using S3 API:

      aws s3api list-buckets
    

Conclusion

By completing these tasks, you have gained hands-on experience with launching and connecting to EC2 instances, creating and managing S3 buckets, and using the AWS CLI to interact with S3. These are fundamental skills for managing AWS resources and are crucial for DevOps practices.

2
Subscribe to my newsletter

Read articles from Urvish Suhagiya directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

Urvish Suhagiya
Urvish Suhagiya

Exploring the world of DevOps 🌐.