Monitoring Amazon Bedrock with CloudWatch

What is AWS bedrock?

AWS Bedrock is a managed service that provides easy access to pre-trained generative AI models for tasks like text and image generation. It enables developers to build and scale AI-powered applications without needing deep machine learning expertise.

Why integrate with CloudWatch?

Amazon Bedrock integrates with CloudWatch to provide comprehensive monitoring capabilities for generative AI applications.

Model Invocation Logging

Bedrock can log full request data, response data, and metadata for all model invocations to CloudWatch Logs or S3. This allows you to:

• Collect user prompts, model inputs/outputs, and invocation details

• Analyze usage patterns and performance

• Audit model interactions for compliance purposes

To enable invocation logging:

1. Set up a CloudWatch log group or S3 bucket destination

   

2. Create an IAM role with proper permissions

   

3. Enable logging in Bedrock console

4. Choose which data to log (text, images, embeddings)

CloudWatch Metrics

Bedrock sends real-time metrics to CloudWatch including:

• Invocation counts

• Latency

• Error rates

• Token usage

  • Content filtering events

    

These can be used to:

• Set up alarms and notifications

• Create dashboards

• Detect anomalies

• Compare performance across models

Live Tail and Log Insights

CloudWatch provides powerful tools to analyze Bedrock logs:

• Live Tail: Stream logs in real-time to detect issues

• Log Insights: Run queries to search and analyze log data

• Pattern recognition: Automatically identify trends and patterns

Sample Log Insights query:

textfields @timestamp, @message
| sort @timestamp desc
| limit 100

Cost Optimization

• Save frequently used queries to reduce query costs

• Create metrics from logs for more cost-effective monitoring

• Use CloudWatch dashboards to visualize key metrics

Securing Bedrock Data

CloudWatch Logs Data Protection can automatically detect and mask sensitive data in Bedrock logs:

• Enable at log group or account level

• Use pre-built managed data identifiers for PII, credentials, etc.

• Create custom data identifiers for business-specific needs

• Verify masking in real-time with Live Tail

• Set up alarms for non-compliant findings

To enable:

1. Go to log group settings

2. Enable data protection

3. Select data identifiers to mask

4. Choose audit log destination

This helps comply with regulations like GDPR, HIPAA, PCI DSS by protecting sensitive data before it reaches logging systems. By leveraging these CloudWatch capabilities, you can gain deep visibility into your Bedrock applications while maintaining strong security and compliance controls. The combination of comprehensive logging, real-time monitoring, and automated data protection provides a powerful toolkit for operating generative AI responsibly at scale