Building the future of cloud operations at any scale

The VP of Search, Observability & Cloud Operations, Nandini Ramani, gave a presentation at re:Invent 2024 that included important information about how AWS is influencing cloud operations going forward. To assist you improve the agility, efficiency, and security of your cloud operations, here is a roundup of the most important announcements.

1. Transform how you govern

• Use the Appropriate Instruments and Governance Structures First

  • Selecting technologies that offer uniform visibility throughout your cloud environment is important.

  • You should be able to safeguard and manage your cloud operations with the help of governance frameworks.

• Maintain Visibility and Stop Noncompliant Behavior

  • You should be able to identify undesirable activity and avoid compliance problems with the aid of tools.

  • Being visible is essential for seeing issues before they become more serious.

• Controls are Simple to Use and Scalable

  • Even in huge contexts, controls should be simple to implement.

  • Implement these rules widely to avoid any security threats or unauthorized changes.

• Avoid Drift and Boost Compliance and Security

  • Make sure that uniform policies are enforced to all of your cloud resources to avoid configuration fluctuations.

  • Boost security and compliance posture with proactive management and automation.

AWS Systems manager

• Increase Operational Effectiveness

  • Makes it easier to manage nodes in multicloud, hybrid, and EC2 settings.

  • Simplifies node management to increase overall effectiveness.

• Management and Remediation in One Place

  • Enables scaled management of all nodes with a single view.

  • Unmanaged nodes are simple to locate, diagnose, and repair.

• Connectivity to Amazon Q Developer

  • From anywhere in the AWS dashboard, take control of and see your nodes.

  • To manage your nodes and obtain insights fast, use Amazon Q Developer.

Declarative Policies

• Keep Yourself Consistent Throughout Changes

  • Even when new features, accounts, or APIs are added, policies remain in effect.

  • Guarantees that policies are consistently applied without modifications.

• Managed Controls and Supported Services

  • Supports VPC, EBS, and EC2 configurations at the moment.

  • The AWS Control Tower offers managed controls.

• Make Policy Implementation Easier

  • Standard configurations for AWS services are enforced with the aid of declarative policies.

  • By stopping non-compliant behavior, they guarantee compliance.

• Examples of Usage

  • Set up EC2 to only permit instance launches with particular AMIs.

  • Block all organization-wide public access in VPCs.

Resource Control Policies (RCPs)

• Use AWS Control Tower for deployment

  • For simpler maintenance, you can use AWS Control Tower to implement RCP-based controlled controls.

• Implement Access Limitations

  • Example: Even with individual bucket policies, enforce the rule that no one outside your company can access Amazon S3 buckets.

  • Gives authority over who has access to particular resources.

• Additional to SCPs (Service Control Policies)

  • SCPs control the IAM roles and users maximum permissions.

  • RCPs manage your organization’s maximum permissions on AWS resources.

• Preventive Measures to Protect Data

  • RCPs assist in establishing a data boundary throughout your AWS environment.

  • They limit large-scale external access to AWS resources.

2. Transform how you observe

• Essential to the Effectiveness of Business

  • To function effectively at any size, observability is necessary.

  • It offers the visibility required to take prompt action and make wise choices.

• New AWS Capabilities

  • In order to improve visibility into databases, containers, networks, infrastructure, and applications, AWS added additional functionality.

• Quick Issue Resolution

  • Helps identify root causes of problems faster, improving response times.

  • Increases overall operational efficiency.

Reports for AWS Fault Injection Service experiments

• Conduct experiments with fault injection

  • To test disaster recovery and failover plans, create realistic failure scenarios.

  • Helps practice handling failures and improve system resilience.

• Automatic Experiment Reports

  • AWS FIS now generates reports for experiments, saving time and effort in resilience testing.

  • Reports from your Amazon CloudWatch Dashboard combine activities and record app reactions.

Monitor network flows in Amazon Cloudwatch

• Monitor Network Performance

  • To keep tabs on your AWS workloads network performance almost instantly, use flow monitors.

  • Tracks the performance of AWS services (S3, RDS, DynamoDB) and compute instances (EC2, EKS).

• Detect and Troubleshoot Issues

  • Identify and assign network issues that are impacting your workloads as soon as possible.

  • Helps identify problems by providing TCP-based measurements for packet loss, latency, and general network health.

Amazon Cloudwatch Database Insights

• Support for Amazon Aurora

  • Amazon Aurora PostgreSQL and Amazon Aurora MySQL are now supported.

  • Gives these database engines improved observability.

• Solving Integrated Database Issues

  • Designed to speed up database troubleshooting for developers, DBAs, and DevOps engineers.

  • Provides a consolidated view of operating system, database, and application logs and metrics.

Enhanced observability for ECS in Amazon Cloudwatch Container Insights

• Enhanced Observability for ECS

  • Offers detailed metrics at the cluster and container levels for Amazon ECS operating on EC2 and Fargate.

  • Speeds up problem isolation and troubleshooting.

• Visual Drill-Down for Issue Detection

  • To find problems like memory leaks, simply dig up and down through the levels of the container.

  • Minimizes mean time to resolution by prompt problem identification.

3. Transform how you analyze

• Simplified Data Analysis

  • It can take a lot of time to analyze performance issues from raw data.

  • To effectively evaluate data without moving it between systems, you require scalable storage and query engines.

• New Features in CloudWatch and OpenSearch

  • AWS enhances Amazon CloudWatch and Amazon OpenSearch’s search and analytics features.

  • Better observability and analysis are made possible by zero-ETL integrations, which enable smooth data processing.

Application transaction spans in Amazon Cloudwatch

• Complete Visibility into Application Transactions

  • Explains application transaction spans in detail, which monitor user-application component interactions.

  • Helps in the understanding of application performance and end-user impact by developers and on-call engineers.

• Improvements to the Interactive Visual Editor and Logs Insights

  • To respond to inquiries about performance, use a visual editor.

  • Analysis possibilities are improved by improvements to Logs Insights queries.

• Advanced Features in CloudWatch Logs

  • Comprises metric extraction for transaction spans, data masking, and forwarding through subscription filters.

Zero-ETL for Amazon Cloudwatch logs and Amazon OpenSearch service

• Zero-ETL Integration

  • With zero-ETL connectivity, CloudWatch and OpenSearch now provide consumers the best of both services.

  • OpenSearch SQL and the Piped Processing Language (PPL) are available to CloudWatch clients.

• Selected Dashboards to Help with Quicker Troubleshooting

  • Innovative dashboards for offered logs, such as AWS WAF, CloudTrail, and Amazon VPC, allow debugging more quickly.

• Seamless Data Analysis

  • CloudWatch Logs can now be easily analyzed by OpenSearch users without requiring duplicate data.

Conclusion

• Improved Control and Governance

  • Declarative Policies and enhanced node management in Systems Manager are two examples of new features that increase security and governance.

  • Policies for resource control help in controlling access to various AWS resources.

• Improved Observability

  • Improved ECS observability, database insights for Amazon Aurora, and new CloudWatch features for network traffic monitoring.

  • Application transaction spans assist analysis and debugging.

• Integrated Analytics and Data Management

  • Operational and security data analysis is revolutionized by Zero-ETL integrations and OpenSearch improvements.

  • ETL pipeline management, context construction, and data copying are all eliminated by streamlined services.

• Unified and Simplified Experience

  • You can concentrate on innovation and providing your clients with better applications thanks to AWS more integrated and connected experience.

References

• Building the future of cloud operations at any scale