Image Labeling with Amazon Rekognition in Python

The idea is simple: You’ll upload a relevant image to an S3 bucket, connect via CLI in your terminal, and write a Python function to process that image to generate detected labels using Amazon libraries.

Tools needed:

→ Amazon Account

→ Local terminal

→ Favorite IDE for executing the Python file (Visual Studio Code)

Steps:

1. Create your AWS S3 bucket to storage the image you’ll work with

   1. AWS Management console: Navigate to S3 (virtual storage in the cloud) from the search bar.

   2. Click on “Create bucket”.

   3. Select a region, a bucket type, and a relevant unique name for the bucket.

   4. Click on “Create bucket” to confirm.

   5. The bucket will appear. Click on the bucket name.

   6. Click on “Upload” and select it from your computer.

   7. Click on “Upload” to confirm.

2. Install the AWS CLI (Command Line Interface) in your system

   1. Open your terminal of preference.

   2. Run the command based on your operating system.

       #Windows:
       msiexec.exe /i https://awscli.amazonaws.com/AWSCLIV2.msi
      
       #MacOS (using Homebrew, install it if you don't have it. It's quite useful):
       brew install awscli
      
       #Linux (using package manager):
       sudo apt-get install awscli
      

      Verify if it’s installed correctly by running the following:

       aws --version
      

   3. Set up a user account to use from CLI

      1. AWS Management console: Navigate to IAM from the search bar.

      2. Go to Users. Click on Create User.

      3. Enter a relevant name and click on Next.

      4. In the Permissions options section, choose “Attach policies directly”

      5. Choose “AdministratorAccess” policy and click on Next and Create User.

      6. Click on the new user and click on “create access key”.

      7. Choose the Command Line Interface(CLI), check the confirmation box and click Next.

      8. Write a clear description and purpose and continue with the creation.

      9. Copy the generated Access key and Secret Access Key to enter during the CLI set up to make the connection.

   4. Login to your CLI with your user credentials

      1. Go back to your terminal and write the command “aws configure”.

      2. Provide the requested information (Make sure to specify the region where the bucket is located).

Python Function

Open the IDE of your choice to write the Python function (.py file) that will do the work! In this case, we’ll use Visual Studio Code.

import boto3
import matplotlib.pyplot as plt
import matplotlib.patches as patches
from PIL import Image
from io import BytesIO

def detect_labels(photo, bucket):
    # Create a Rekognition client
    client = boto3.client('rekognition')

    # Detect labels in the photo
    response = client.detect_labels(
        Image={'S3Object': {'Bucket': bucket, 'Name': photo}},
        MaxLabels=10)

    # Print detected labels
    print('Detected labels for ' + photo)
    print()
    for label in response['Labels']:
        print("Label:", label['Name'])
        print("Confidence:", label['Confidence'])
        print()

    # Load the image from S3
    s3 = boto3.resource('s3')
    obj = s3.Object(bucket, photo)
    img_data = obj.get()['Body'].read()
    img = Image.open(BytesIO(img_data))

    # Display the image with bounding boxes
    plt.imshow(img)
    ax = plt.gca()
    for label in response['Labels']:
        for instance in label.get('Instances', []):
            bbox = instance['BoundingBox']
            left = bbox['Left'] * img.width
            top = bbox['Top'] * img.height
            width = bbox['Width'] * img.width
            height = bbox['Height'] * img.height
            rect = patches.Rectangle((left, top), width, height, linewidth=1, edgecolor='r', facecolor='none')
            ax.add_patch(rect)
            label_text = label['Name'] + ' (' + str(round(label['Confidence'], 2)) + '%)'
            plt.text(left, top - 2, label_text, color='r', fontsize=8, bbox=dict(facecolor='white', alpha=0.7))
    plt.show()

    return len(response['Labels'])

def main():
    photo = 'sailor-moon-cosmos.JPEG'
    bucket = 'rekognition-label-s3'
    label_count = detect_labels(photo, bucket)
    print("Labels detected:", label_count)

if __name__ == "__main__":
    main()

→ Change the value inside of def main(): for photo and bucket based on your own data.

👇🏻 The code is fairly intuitive, but let’s highlight the main parts:

→ The main functionality is inside the def detect_labels function, which takes only the photo name and the bucket name as inputs.

→ We’re creating an Amazon Rekognition client from the boto3 library to analyze the image with their main detect_labels method.

→ We use Matplotlib with .imshow and .Rectangle, for example, to help create the visualization that displays the rectangles around the detected labels.

→ We use PIL (Python Imaging Library) and BytesIO to work with the image data.

Let’s run the code using the terminal inside the IDE:

python name_of_file.py

If everything works correctly, you’ll see a list of all the detected labels in the same terminal, along with their confidence percentages (which we’re also asking to print, if you read the code carefully), and a pop-up with the rectangles added to the image from the S3 bucket around the detected objects.

Example:

SAILOR MOON CRYSTAL : ©Naoko Takeuchi/PNP/Kodansha/Toei Animation