Hey there! Guess what? Today is an extra special day! We're going to explore how to make the most of Amazon Textract in Salesforce using LWC and Apex. Don't worry, I'll guide you through the process with easy-to-follow, step-by-step instructions to set up your AWS for Textract services.

Remember when we learned how to access the device camera using LWC in my last blog post? Well, today we're taking it up a notch! We'll be integrating it with Amazon Textract services for an even more exciting experience. So, buckle up and let's have a blast exploring this together!

What is Amazon Textract?

Amazon Textract is this amazing AI buddy that can effortlessly pullout text and data from almost any document. Thanks to its smart machine-learning skills, Textract makes data entry a breeze, smooths out document workflows, and reveals some pretty useful insights. No more manual work or tricky coding is needed! Wave goodbye to those pesky, hard-to-read documents and say hello to a more efficient way with Amazon Textract.

More information on Amazon Textract can be found on the official site here.

Steps to configure IAM user in AWS

Using an IAM (Identity and Access Management) user instead of the root user for AWS services is highly recommended due to enhanced security and better access control. With IAM, you can create users with specific permissions, ensuring the principle of least privilege is followed. This minimizes the risk of unauthorized access or accidental changes to critical resources. Additionally, IAM users can be easily audited, helping you track and monitor activity within your AWS environment, while also making it simpler to enforce security policies and compliance requirements.

Log in to the AWS console. On the top left corner search for IAM. Select IAM.
On the next page, from the left panel select "Users" and from the right panel click on "Add User"
Give a username and click "Next".
On the next page, select "Attach policies directly" search for "textract" and select "AmazonTextractFullAccess" and press "Next".
Finally, click "Create User" on the next page.
Now, it will redirect to the user's table, and select the user which just got created.
Navigate to the tab "Security Credentials".
Scroll down and click "Create Access Keys".
On this screen select "Application running outside AWS"
Just press "Create Access Keys" on the next screen. Download the access key and access secret, as you won't be able to access this secret later.

Steps to configure Named Credential in Salesforce using this key and secret.

Create a new legacy Named Credential in Salesforce.
Add a label.
Add the URL as "https://textract.ap-south-1.amazonaws.com" , replace "ap-south-1" with your region.
Select these from the Authentication section:

Identity Type : Named Principal

Authentication Protocol : AWS Signature Version 4
Add your access key and access secret from the CSV file downloaded from AWS.
Add region as whatever region you have selected in AWS. For me it's "ap-south-1".
Add Service as "textract" and click "Save".

Apex class to call Amazon textract using named credential.

In the following example, I am calling "AnalyzeId" service of AWS textract, to analyze an Id dynamically and share the result. We can extend this use to analyze Ids, documents, passports etc.

public class AmazonTextractService {

    @AuraEnabled
    public static String callTextractService(String image){
        FormDataForAnalyzeId analyzeId = new FormDataForAnalyzeId();
        DocumentClass documentClass = new DocumentClass();
        documentClass.Bytes = image;
        analyzeId.DocumentPages = new List<DocumentClass>{documentClass};
        HttpRequest httpRequest = new HttpRequest();
        httpRequest.setEndpoint('callout:AmazonTextractNew');
        httpRequest.setMethod('POST');
        httpRequest.setHeader('Content-Type','application/x-amz-json-1.1');
        httpRequest.setHeader('X-Amz-Target','Textract.AnalyzeID');
        httpRequest.setBody(JSON.serializePretty(analyzeId));
        Http http = new Http();
        HttpResponse httpResponse = http.send(httpRequest);
        Map<String, Object> obj = (Map<String, Object>) JSON.deserializeUntyped(httpResponse.getBody());
        return JSON.serialize(obj.get('IdentityDocuments'));
    }

    public class FormDataForAnalyzeId{
        public List<DocumentClass> DocumentPages;
    }

    public class DocumentClass{
        public String Bytes;
    }
}

LWC Component to call this service

The LWC component is the same as in my previous blog, just some html and js level changes to incorporate responses coming from the textract service.

<template>
    <div class="spinner">
        <template lwc:if={isLoading}>
            <lightning-spinner alternative-text="Loading" variant="brand" size="large"></lightning-spinner>
        </template>
    </div>

    <div class="camera-wrapper">
        <lightning-button label="Start Video" class="slds-m-left_xx-small" onclick={initCamera}></lightning-button>
        <lightning-button label="Stop Video" class="slds-m-left_xx-small" onclick={stopCamera}></lightning-button>
        <lightning-button label="Capture Image" class="slds-m-left_xx-small" onclick={captureImage}></lightning-button>
    </div>
    <div class="slds-grid slds-gutters">
        <div class="slds-col">
            <video class="videoElement" autoplay></video>
            <canvas class="slds-hide canvas"></canvas>
        </div>
        <div class="slds-col">
            <img src="" class="slds-hide imageElement" alt="captured image"/>
        </div>
    </div>
    <div class="slds-card slds-p-around_xxx-small">
        <div class="slds-card__tile">
            Data from Amazon Textract
        </div>
        <div class="slds-card__body">
            <div class="slds-grid">
                <div class="slds-col slds-size_4-of-12 slds-text-title_bold">
                    Field Name
                </div>
                <div class="slds-col slds-size_4-of-12 slds-text-title_bold">
                    Field Value
                </div>
                <div class="slds-col slds-size_4-of-12 slds-text-title_bold">
                    Confidence
                </div>
            </div>
            <template for:each={identityDocumentFields} for:item="eachIdentityDocumentField">
                <div key={eachIdentityDocumentField.Type.Text} class="slds-grid">
                    <div class="slds-col slds-size_4-of-12">
                        {eachIdentityDocumentField.Type.Text}
                    </div>
                    <div class="slds-col slds-size_4-of-12">
                        {eachIdentityDocumentField.ValueDetection.Text}
                    </div>
                    <div class="slds-col slds-size_4-of-12">
                        {eachIdentityDocumentField.ValueDetection.Confidence}
                    </div>
                </div>
            </template>
        </div>
    </div>
</template>

import {LightningElement} from 'lwc';
import callTextractService from '@salesforce/apex/AmazonTextractService.callTextractService';

export default class CameraCapture extends LightningElement {
    videoElement;
    canvasElement;
    identityDocumentFields = [];
    isLoading = false;

    renderedCallback() {
        this.videoElement = this.template.querySelector('.videoElement');
        this.canvasElement = this.template.querySelector('.canvas');
    }


    async initCamera() {
        if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) {
            try {
                this.videoElement.srcObject = await navigator.mediaDevices.getUserMedia({video: true, audio: false});
            } catch (error) {
                console.error('Error accessing the camera: ', JSON.stringify(error));
            }
        } else {
            console.error('getUserMedia is not supported in this browser');
        }
    }

    async captureImage() {
        if(this.videoElement && this.videoElement.srcObject !== null) {
            this.canvasElement.height = this.videoElement.videoHeight;
            this.canvasElement.width = this.videoElement.videoWidth;
            const context = this.canvasElement.getContext('2d');
            context.drawImage(this.videoElement, 0, 0, this.canvasElement.width, this.canvasElement.height);
            const imageData = this.canvasElement.toDataURL('image/png');
            const imageElement = this.template.querySelector('.imageElement');
            imageElement.setAttribute('src', imageData);
            imageElement.classList.add('slds-show');
            imageElement.classList.remove('slds-hide');
            const base64Canvas = imageData.split(';base64,')[1];
            this.isLoading = true;
            callTextractService({image : base64Canvas}).then(response => {
                this.identityDocumentFields = (JSON.parse(response)[0]).IdentityDocumentFields;
                this.isLoading = false;
            }).catch(error => {
                console.error('Error From Apex call ',error);
                this.isLoading = false;
            })

        }
    }

    async stopCamera(){
        const video = this.template.querySelector(".videoElement");
        video.srcObject.getTracks().forEach((track) => track.stop());
        video.srcObject = null;
        this.hideImageElement();
    }

    hideImageElement(){
        const imageElement = this.template.querySelector('.imageElement');
        imageElement.setAttribute('src', "");
        imageElement.classList.add('slds-hide');
        imageElement.classList.remove('slds-show');
    }
}

In this part, the LWC requests access to the camera and snaps a picture. The captured image, in Blob format, is sent to the AWS Textract API. The API then returns the field names, values, and confidence scores based on a specified template format.

As for the text, even though the ID might be a bit hard to read for us humans, the amazing power of AI comes to the rescue! Amazon Textract skillfully interprets it and provides the correct response, along with a confidence score to show how sure it is. Pretty neat, right?

Demo

Use Cases

There can be multiple use cases of this, here are a few to start with.

Automated data entry: By integrating Textract with Salesforce, you can automatically extract data from scanned documents or images, such as invoices, receipts, and contracts, and populate the relevant Salesforce records. This reduces manual data entry, saves time, and minimizes errors.
Enhanced customer support: Textract can be used to analyze customer communications, such as emails or support tickets, and extract relevant information to help customer support agents quickly access the necessary details. This can lead to faster response times and improved customer satisfaction.
Document management: Textract can help organizations to better manage their documents by automatically extracting metadata from files, such as document type, date, or author. This information can be used to create a more organized and searchable document repository in Salesforce.
Intelligent workflows: By using Textract to extract data from documents, Salesforce can leverage its AI capabilities to create automated workflows, routing extracted information to the right teams or individuals for further action. This can improve process efficiency and reduce the time it takes to complete tasks.
Compliance and risk management: Textract can be used to identify and extract critical information from documents to ensure compliance with industry regulations or company policies. This information can be used to automatically flag non-compliant documents in Salesforce, enabling organizations to mitigate potential risks and maintain compliance.

Useful Links

This blog post provides step-by-step instructions on how to use Amazon Textract in Salesforce with LWC and Apex. It includes an example Apex class to call Amazon Textract, a Lightning Web Component (LWC) to capture an image and use Amazon Textract to analyze the image, and HTML and JavaScript to incorporate the responses from the Textract service. This LWC can be used for automated data entry, enhanced customer support, and document management.

Simplifying Amazon Textract OCR Usage in Salesforce: A Detailed Guide

Table of contents