Beyond the Basics: Exploring AWS RDS, ElastiCache, and Aurora Brilliance

AWS RDS

• RDS stands for Relational Database Service.

• It is a managed database service for database that support SQL as query language.

  • PostgreSQL.

  • MariaDB.

  • MySQL.

  • Oracle.

  • Microsoft SQL Server.

  • Aurora (AWS Proprietary Database).

Advantage of Deploying Database on EC2 vs Using RDS.

• RDS is a managed Service:

  • Automatic provisioning (Setup), OS Patching.

  • Continuous backups and restore to specific timestamps (Point in time restore).

  • Monitoring Dashboards.

  • Read Replicas for improving read performance.

  • Multi AZ setup for disaster recovery.

  • Scaling (horizontal and vertical).

  • Storage backed by EBS (gp2 or IO1)

  • Note: We do not have SSH shell feature to RDS.

AWS RDS - Auto Scaling Storage

• Helps you increase storage on your RDS DB instance dynamically.

• When RDS detect you are running out of database storage it automatically increases.

• You have to set Maximum Storage Threshold.

• You have to configure when and how storage scaling works.

RDS Replicas for Read Scalability

• Up to 15 read replicas in same AZ, Multi AZ, or Multi Region.

• Replication is ASYNC so reads are eventually consistent.

• Replicas can be converted into independent read/write database.

• Automatic Read redirection not supported so application need to update connection string to read replicas.

Use case of Read Replicas

• You have production database and now you want to perform some analytics on that data.

• But It not recommended to perform analytics on production database.

• So create Read replica and consume data from it for analytics.

• Read replica only support SELECT statement not support (UPDATE, CREATE, DELETE).

RDS Read Replicas - Network Cost

• In AWS there’s a network cost when data goes from one AZ to another.

• If data replicate in same region no cost involve.

RDS Multi AZ (Disaster Recovery)

• SYNC replication.

• We use standby RDS DB instance to overcome failure.

• As part of the failover process, the DNS entry associated with the primary database DNS name is dynamically updated to point to the virtual IP address of the standby RDS server.

• We can also use Read Replica as Standby database for recovery.

RDS Single AZ to Multi AZ

• A snapshot of the primary database is taken.

• Restore a new database from the snapshot.

• The new database act as standby and ASYNC with the primary database.

AWS RDS Custom

• Let’s talk about RDS custom and how this is different from AWS RDS.

• RDS custom helps you to customize and automate your database according to you, It is not managed by RDS automatically.

• With the help of it, you can connect with database underlying EC2 using SSH or session manager, configure settings, install patch and updates.

• RDS: Entire database and OS managed by AWS.

• RDS Custom: Full Admin access to the underlying OS and database.

Magician of the party “AMAZON AURORA” (Important)

• It is also a database managed service provided by AWS.

• Aurora DB connected with both MYSQL and PostgreSQL using drivers provided by Aurora.

• Up to 5x the throughput of MySQL and 3x the throughput of PostgreSQL

• Aurora storage automatically grows in increment of 10GB up to 128tb.

• Aurora gives 15 replicas and high replication speed compare to MYSQL.

• Aurora cost 20% more than RDS.

Aurora High Availability and Read Scaling

• 6 copies of your data across 3 AZ.

  • 4 copies out of 6 need for writes.

  • 3 copies out of 6 need for reads.

  • self healing with peer to peer replication.

  • storage is distribute across 100 volumes.

• One Aurora Instance become Master.

• Automated failover for master in less than 30 seconds, new replica replace the master.

• Master + 15 Aurora Read Replicas serve reads.

• Support for cross Region Replication.

High Level Design of Aurora working at Scale

• We have writer Endpoint pointed towards Primary Aurora DB instance or Master Node or Write Node.

• If any failure happened to Master Node then Aurora can promote any Replica as new Master and this endpoint updated to new master.

• Load balancer get associated with Reader endpoint which distribute all reader traffic to multiple replicas.

• This system provide high availability, Automatic scalability, handling failovers, Backups and Recovery.

Aurora Custom Endpoints

• Custom endpoints help us to define the subset of the database replicas which means this endpoint help us to access some specific DB replicas.

• This is generally used for performing Analytics on database, for accessing data for analytics we create separate database replicas.

Aurora Serverless

• Automated database instantiation and autoscaling based on usage.

• Good for less frequent usage apps or services.

• No capacity planning and pay per usage per second.

• You can also pause and restart when you need.

Aurora MULTI-MASTER

• You can create multiple Master or Writer node to increase write availability.

Aurora Global Scaling

• 1 Primary Region (Read)

• Up to 5 secondary (read-only) Regions (replication lag is less than 1 second).

• Up to 16 replication in each secondary region.

• Help’s in maintaining low latency.

• Typically cross-region replication takes less than 1 seconds

Aurora Machine Learning (Cool 😎)

• This allows you to use machine learning models to make predictions on data stored in your database without the need to move the data to a separate machine learning service.

• With Aurora ML, you can make real-time predictions on new data as it is added to the database.

• Aurora ML supports a SQL interface for invoking machine learning inference.

SELECT * FROM comments, aws_comprehend.detect_sentiment(comments.comment_text, 'en') as s

• You can use AWS SageMaker (Any model) , Amazon Comprehend (Sentimental analysis)

Backup, Restore and Cloning

Backups

• Automated Backups

  • Daily Backups of the database during backup window.

  • 1 to 35 days retention.

  • Restore from the last 5 minutes to oldest backup.

• Manual DB Snapshot

  • Manually triggered by User.

  • Retention of backup as long as you want.

Manual Restore

• Restore MySQL RDS from S3

  • Create Backup of your on premises Database.

  • Store backup file on AWS S3

  • Restore backup file on new RDS instance running MYSQL

• Restore MySQL Aurora Cluster from s3

  • Create backup of your database using Percona XtraBackup.

  • Store backup file on S3.

  • Restore the backup on new Aurora MySQL Cluster.

Aurora Database Cloning

• This allow you to create a new Aurora DB Cluster from the existing one.

• Faster than snapshot and restore.

• Initially this database uses the same data and new data automatically added to it and it is self managed cluster so need to worry about storage.

RDS and Aurora Security

• Database Master and Replicas encrypted using AWS KMS (Key Management Service).

• If Master is not encrypted then Replicas also not.

• To encrypt the un-encrypted database you have create database snapshot then restore and encrypt it.

• IAM Roles, Username and Password, security groups all help in securing your cluster.

• No SSH is enabled for managed service RDS, Aurora except for RDS Custom.

Amazon RDS Proxy

• It help in implementing features like Pooling, Caching, Authentication.

• Pooling: RDS Proxy maintains a pool of established database connections. Instead of creating a new connection for each incoming request, RDS Proxy efficiently manages and reuses existing connections.

• Creating and Shutting connections to database are very expensive tasks so, it helps in reducing load on database.

• RDS Proxy handles the details of connection creation, management, and reusing on behalf of your application.

• Caching: When a query is cached, RDS Proxy can quickly respond to similar queries without the need to execute them against the database.

• Reduce failover by 66%

• Supported by Both RDS and Aurora.

ElastiCache

• Similar to RDS, ElastiCache helped you to manage in memory database like Redis or Memcached.

• Elasticache take care of maintaining OS, patches, updates, etc

• It cache the DB queries and reduce the load on server for read intensive task.

• We can store User session details in ElastiCache and when user access other services we’ll retrieve session detail from cache and verify user to give access to service.

• Redis: Provide data replication means high availability.

• Memcached: Provide Sharding means high performance.

• You can access to Redis using token/user/password and Memcached using SASL (Simple Authentication and Secure Layer used to authenticate client against server) authentication.

Thank you very much guys for reading and I love you gain lot of new knowledge from this blog post.