Inside the Engine: Architecture of Amazon Augmented AI (A2I) and Creating Custom Human Review Workflows

---

In our previous article, we explored what Amazon Augmented AI (A2I) is and how it brings together human judgment and machine learning to create more trustworthy, accurate workflows. Now, let’s go deeper. In this post, we’ll explore the architecture behind Amazon A2I and guide you through creating your own custom human review loop.
Whether you’re an AI practitioner, a product manager, or just curious about how to balance automation with human intelligence, this guide will help you get started with building robust A2I-enabled applications.

What is Amazon A2I Architecture?

At its core, Amazon A2I provides the infrastructure for routing predictions from machine learning models to human reviewers based on configurable rules.
Here’s how a typical A2I setup looks:

Key Components:

• Machine Learning Model: This could be your custom model hosted on Amazon SageMaker or a pre-built AWS AI service like Textract, Rekognition, or Comprehend.

• Amazon A2I Human Review Workflow: The rules and conditions under which a prediction is flagged for human review.

• Human Task UI: A web-based form that reviewers use to inspect and validate predictions.

• Workforce: The team of people (private team, Mechanical Turk, or vendor-managed) who review the flagged predictions.

• Amazon S3: Stores input data, A2I output (approved/corrected predictions), and logs.

• Amazon CloudWatch: Monitors and logs activity, task status, errors, etc.

---

Pre-Built vs. Custom Workflows

Amazon A2I supports two types of workflows:

Pre-Built Workflows:

• Available for Amazon Textract, Rekognition, and Comprehend.

• Ready-to-use review UIs and logic.

• Ideal for quick deployments.

Custom Workflows:

• For use with SageMaker models or external AI systems.

• Fully customizable task UI, logic, and thresholds.

• Gives you maximum control.

Let’s now look at how to build your custom human review loop.

---

How to Build a Custom A2I Workflow (Step-by-Step)

Step 1: Prepare Your Model

• Use Amazon SageMaker or any other inference endpoint.

• Make sure your model returns prediction scores (confidence levels).

Step 2: Define a Human Review Workflow

• Use an A2I workflow definition in JSON format.

• Example: Trigger a review when confidence < 80%.

"FlowDefinition": {
  "FlowDefinitionName": "low-confidence-review",
  "HumanLoopConfig": {
    "WorkteamArn": "<your-workteam-arn>",
    "HumanTaskUiArn": "<your-ui-arn>",
    "TaskCount": 1,
    "TaskDescription": "Review low confidence predictions",
    "TaskTitle": "Human Review"
  },
  "OutputConfig": {
    "S3OutputPath": "s3://your-bucket/review-results/"
  },
  "RoleArn": "<your-iam-role>"
}

Step 3: Create a Human Task UI

This is a web form in HTML that your reviewers will see.

• Example: Display extracted text and ask, "Is this correct?"

• You can use AWS-provided templates or build your own.

Step 4: Set Up Your Workforce

Choose one:

• Private team (internal reviewers)

• Amazon Mechanical Turk (crowdsourced)

• Vendor-managed workforce

Step 5: Connect to A2I

In your application:

• Make a prediction using the model.

• Based on confidence, invoke the StartHumanLoop API.

response = a2i_runtime.start_human_loop(
  HumanLoopName='review-loop-001',
  FlowDefinitionArn='<your-flow-arn>',
  HumanLoopInput={
    'InputContent': json.dumps({"text": predicted_text})
  }
)

Step 6: Store and Retrieve Results

Once the reviewer finishes the task, A2I will store the corrected prediction in S3.
You can retrieve it for retraining or further automation.

---

Real-World Example: AI-Generated Product Descriptions

Imagine you’re using a GPT-style model to auto-generate product descriptions for an e-commerce site. But some descriptions sound odd or incorrect.

Using A2I:

• Automatically send these low-confidence generations to a human reviewer.

• The reviewer edits the text if needed.

• Final, corrected content is saved and published.

This way, your customers see only high-quality product information, and your AI learns from the feedback.

---

Best Practices

• Threshold Tuning: Start with conservative confidence thresholds, then adjust as your model improves.

• Reviewer Rotation: Avoid reviewer fatigue by rotating tasks.

• Logging: Use CloudWatch to track workflow performance.

• Auditing: Use random sampling to ensure consistent quality.

---

📆 Conclusion

Amazon Augmented AI is not just a tool—it’s a framework that empowers you to bring humans and machines together.

With a custom A2I workflow, you can:

• Improve prediction accuracy

• Build trust in your ML applications

• Save time and money

In the next article, we’ll explore how to use A2I feedback to retrain and improve your machine learning models over time.

Stay tuned—and happy building!