Day 15 of 100 Days of AWS: Database Basics Part II

Hi folks! Welcome to Day 15 of 100 Days of AWS🎯, where we will cover the complete AWS cloud from beginner to professional. Today, we will expand our horizons in the AWS cloud by understanding Different types of services provided by aws for storing the data produced by your system for different purposes namely transactional, Reporting, and analysis purpose. and how to choose the right solution for businesses demand to attain high Availability, performance, cost effective services. Let’s get started🚀!

Day 14 Overview;

On Day 14 of the "100 Days of AWS" series we focused on understanding the basics of databases in the AWS cloud. The article begins by explaining the importance of data in the digital age and the necessity of storing it in databases for efficient management. It distinguishes between SQL and NoSQL databases, highlighting their structures, use cases, and examples. The article also covers different data types: structured, semi-structured, and unstructured, and discusses the role of Database Management Systems (DBMS) in organizing and retrieving data. Additionally, it outlines the types of datastores offered by AWS, including self-managed, SQL, and NoSQL options, each catering to specific data management needs. The day concludes with a wrap-up of the key points and a preview of the next day's focus on individual AWS database services. if you want to dive deep into this topic or revise it, feel free to click here.

Self-Managed Datastore Services,

On AWS, a self-managed datastore refers to deploying and managing your own database infrastructure using services like EC2 and EBS rather than fully managed services. They are preferred if you need full control over your infrastructure, the specific requirement you’re looking for is not met in the fully managed datastore, and you have expertise in managing it.

pros,

• Cost-effective: You only pay for the raw resources you use Ex, EC2 for compute and EBS for storage.

• Flexibility: You have complete control over your DB configuration, performance tuning, and data location. including the ability to migrate to other cloud vendors or on-prem.

• Scalability: You can scale beyond the limit of managed services by adding more compute and storage resources as needed.

• Data Protection: You have full control over your backups and disaster recovery strategies.

• Features: You can upgrade and patch the database software whenever needed.

cons,

• Increased Management Overhead: You are responsible for all Database management, including Patching, backups, Monitoring, and performance tuning.

• Requires Expertise: you need Expertise in Database Management and AWS Infrastructure.

1. EC2,

   • Uses EC2 for compute capacity, and EBS for data storage.

2. ECS,

   • For running as a container.

3. EKS,

   • Kubernetes Managed by AWS – AWS manages the control plane, but worker nodes and datastore can be self-managed.

   • Self-Managed Datastore Options – Can use self-hosted ETCD, RDS, MySQL, PostgreSQL, or other DBs.

   • Complexity & Maintenance – Requires setting up backups, scaling, and monitoring manually.

   • Data Persistence – Unlike managed AWS datastore (like RDS), self-managed stores require HA and DR strategies.

Fully Managed Databases and Analytics services,

For some organizations, Database management is undifferentiated and heavy lifting. Your data is your most valuable asset. and you want to invest resources in getting value from your data to create innovative and undifferentiated experiences. AWS removes the heavy lifting of undifferentiated database management tasks by providing automatic failover, backup and recovery, industry compliance, automated patching and more with built-in best practices.

AWS takes care of availability, storage durability, and disaster recovery, offloading the overload burden from the organization. Above all, AWS has invested in its global infrastructure so you can deploy your application in a new Region within minutes. With AWS, database operations transform from being a resource-intensive burden into a strategic advantage while maintaining the security, availability, and performance your applications require.

Fully managed Relational/SQL Datastore Services;

Relational databases store data with predefined schemas and relationships between them. These databases are designed to support ACID transactions and maintain referential integrity and strong data consistency. For more information about ACID transactions, click here.

Some of the Industrial sectors using Relational Databases for their data processes are Traditional Applications, Enterprise Resource Planning(ERP), Customer Relationship Management(CRM), e-commerce, generative AI use cases (such as chatbots with retrieval-augmented generation, Similarity search, recommendation systems, and more.)

1. RDS Relational Database Service,

   • Amazon RDS is a web service that makes it easier to set up, operate, and scale a relational database in the AWS Cloud.

   • It provides cost-efficient, resizable capacity for an industry-standard relational database and manages common database administration tasks. By eliminating tedious manual processes, Amazon RDS frees you to focus on your application and your users.

   • supports Database engines that are most widely used in all industries such as IBM Db2, MariaDB, Microsoft SQL Server, MySQL, Oracle Database, and PostgreSQL

   • can get high availability with a primary DB instance and a synchronous secondary DB instance that you can fail over to when problems occur. You can also use read replicas to increase read scaling.

2. Aurora,

   • Aurora is a relational database management system (RDBMS) built for the cloud with full MySQL and PostgreSQL compatibility. provides unparalleled high performance and availability at global scale for PostgreSQL, MySQL, and DSQL.

   • Aurora has 5x the throughput of MySQL and 3x of PostgreSQL with full PostgreSQL and MySQL compatibility. and gives you the performance and availability of commercial-grade databases at one-tenth the cost.

   • Aurora is designed for up to 99.999% multi-Region availability. With Aurora DSQL, Aurora provides virtually unlimited scale in and across regions with no infrastructure management.

3. Aurora Serverless v2,

   • Aurora Serverless v2 is an on-demand, autoscaling configuration for Amazon Aurora. Aurora Serverless v2 helps to automate the processes of monitoring the workload and adjusting the capacity for your databases.

   • Capacity is adjusted automatically based on application demand. and You're charged only for the resources that your DB clusters consume.

4. Redshift,

   • Redshift is a Cloud-based data warehouse service from AWS. designed to store and analyze large amounts of data. popular choice for analytics and big data.

   • it is built on a modified version of PostgreSQL and has its dialect of SQL. It uses massively parallel processing (MPP) to quickly execute complex queries.

   • it can be used for: Analyzing large amounts of data, Large-scale data migrations, Querying and managing huge amounts of structured data, and Near real-time analysis of data.

No SQL Datastore Services;

1. Amazon Dynamo DB,

   • The lightning-fast king of Key Value at AWS. Amazon DynamoDB is a highly scalable, serverless, NoSQL database service provided by AWS that supports key-value and document data models, enabling developers to build modern, scalable applications.

   • removes traditional scalability limitations on data storage while maintaining low latency and predictable performance.

   • For globally distributed applications, DynamoDB global tables is a multi-Region, multi-active database with a 99.999% availability SLA and increased resilience.

   • The DynamoDB data model concepts include tables, items, and attributes.

2. Amazon Document DB,

   • Amazon DocumentDB (with MongoDB compatibility) is a fast, reliable, and fully managed database service. Amazon DocumentDB makes it easy to set up, operate, and scale MongoDB-compatible databases in the cloud.

   • With Amazon DocumentDB, you can run the same application code and use the same drivers and tools that you use with MongoDB.

   • With Amazon DocumentDB, you can increase read throughput to support high-volume application requests by creating up to 15 replica instances. Amazon DocumentDB replicas share the same underlying storage, lowering costs and avoiding the need to perform writes at the replica nodes. This capability frees up more processing power to serve read requests and reduces the replica lag time—often down to single-digit milliseconds.

3. Amazon Key spaces,

   • Amazon Keyspaces (for Apache Cassandra) is a scalable, highly available, and managed Apache Cassandra compatible database service. With Amazon Keyspaces, you can run your Cassandra workloads on AWS using the same Cassandra application code and developer tools that you use today.

4. Amazon Neptune,

   • Amazon Neptune is a fast, reliable, fully managed graph database service that makes it easy to build and run applications that work with highly connected datasets.

   • The core of Neptune is a purpose-built, high-performance graph database engine. This engine is optimized for storing billions of relationships and querying the graph with milliseconds latency.

   • Neptune powers graph use cases such as recommendation engines, fraud detection, knowledge graphs, drug discovery, and network security.

5. Amazon Elastic cache,

   • Amazon ElastiCache is a fully managed in-memory data store and cache service that speeds up web applications. It's compatible with Valkey, Redis OSS, and Memcached.

   • Gaming, Ad-Tech, E-Commerce, Healthcare, Financial Services, IoT, Media streaming, Session stores, Leaderboards, and Machine learning (ML).

6. Amazon Open search,

   • OpenSearch is a fully open-source search and analytics engine for use cases such as log analytics, real-time application monitoring, and clickstream analysis.

7. Amazon Quantum Ledger Database (QLDB),

   

   • Amazon Quantum Ledger Database (Amazon QLDB) is a fully managed ledger database that provides a transparent, immutable, and cryptographically verifiable transaction log owned by a central trusted authority. You can use Amazon QLDB to track all application data changes, and maintain a complete and verifiable history of changes over time.

8. Amazon Timestream,

   

   • With Amazon Timestream for LiveAnalytics, you can easily store and analyze sensor data for IoT applications, metrics for DevOps use cases, and telemetry for application monitoring scenarios such as clickstream data analysis. Amazon Timestream for InfluxDB is a managed time-series database engine that makes it easy for application developers and DevOps teams to run InfluxDB databases on AWS for real-time time-series applications using open-source APIs.

when to choose which Service;

• As discussed above, AWS offers a growing number of database options (15+) with diverse data models to support a variety of workloads. These include relational, key-value, document, in-memory, graph, time series, vector, and wide-column.

• Choosing the right database, or multiple databases, requires you to make a series of decisions based on your organizational needs. This image above will help you ask the right questions, provide a clear path for implementation, and help you migrate from your existing database. rather than lift and shift migration. you can efficiently rearchitect your application as per cloud standard and reap the maximum benefit of it.

Day 15 wrap-up;

In conclusion, understanding the various database services offered by AWS is crucial for selecting the right solution to meet your business needs. Whether you opt for self-managed datastores for greater control or fully managed services for ease of use and scalability, AWS provides a wide range of options to support different data models and workloads. By carefully evaluating your organizational requirements and leveraging AWS's diverse offerings, you can ensure high availability, performance, and cost-effectiveness in your data management strategy. As you continue our journey through the "100 Days of AWS" series, we’ll gain deeper insights into optimizing your cloud infrastructure and harnessing the full potential of AWS services.

summary & Key points;

Up next on Day 16;

• Application Integration services for seamless working of the application parts.