Day 10:AWS Storage

Types of Storage Services

Elastic Block Store (EBS) - Block Storage

• Block Storage: Data is divided into fixed-size blocks, which the operating system directly manages.

• Usage: Ideal for virtual hard drives attached to EC2 instances.

• Characteristics:

  • Single write volume support.

  • Operated by the operating system.

  • Suitable for applications requiring a virtual hard drive with performance consistency.

AWS Elastic File Storage (EFS) - File Storage

• File Storage: Data is stored with metadata in a hierarchical file system structure.

• Usage: Perfect for file sharing where multiple users or VMs need concurrent access.

• Characteristics:

  • Supports multiple connections via a network share.

  • Multiple reads and file locks during writes.

  • Ideal for applications needing shared access to files.

Amazon Simple Storage Service (S3) - Object Storage

• Object Storage: Data is managed as objects, each with a unique ID and associated metadata.

• Usage: Best for storing files without worrying about infrastructure details.

• Characteristics:

  • Scales indefinitely with no file or storage limits.

  • Supports multiple reads and writes without locking.

  • Not designed for high IOPS (Input/Output Operations Per Second).

Introduction to S3 (Simple Storage Service)

S3 is an object-based storage service that offers unlimited storage. Unlike file and block storage, S3 organizes data as objects rather than files or blocks. Key components include:

• S3 Buckets: Containers that hold objects. Each bucket name must be unique across the entire S3 namespace.

• Objects: Consist of:

  • Key: The name of the object.

  • Value: The data itself.

  • Version ID: Identifies different versions of the same object (if versioning is enabled).

  • Metadata: Additional information about the object.

S3 Storage Classes

AWS provides various S3 storage classes, each balancing cost, retrieval time, and durability:

• S3 Standard: Default class with high availability (99.99%) and durability (11 nines). Replicated across multiple Availability Zones (AZs).

• S3 Intelligent-Tiering: Uses machine learning to move data between access tiers based on usage patterns.

• S3 Standard-IA (Infrequent Access): Cheaper for less frequently accessed data. Additional retrieval fees apply.

• S3 One-Zone-IA: Lower cost for data stored in a single AZ. Suitable for non-critical data.

• S3 Glacier: For long-term cold storage with retrieval times from minutes to hours. Cost-effective for archival data.

• S3 Glacier Deep Archive: Lowest-cost option for data that is rarely accessed, with retrieval times of up to 12 hours.

AWS Snow Family

The AWS Snow Family provides physical devices to move data in and out of the cloud when internet transfer is impractical:

• Snowcone: A small device with 8TB of storage, ideal for edge locations or small data transfers.

• Snowball Edge: Available in two versions:

  • Storage Optimized: 80TB of storage.

  • Compute Optimized: 39.5TB of storage with additional compute capabilities.

• Snowmobile: A massive 100PB storage and compute container transported by semi-trailer for large-scale data migrations.

Additional Storage Services

• AWS Storage Gateway: Hybrid cloud storage service that extends on-premises storage to AWS:

  • File Gateway: Connects local storage to AWS S3.

  • Volume Gateway: Caches local drives to S3, providing continuous backup.

  • Tape Gateway: Uses virtual tapes for cost-effective long-term storage.

• AWS Backup: A fully managed service to centralize and automate backups across AWS services like EC2, EBS, RDS, DynamoDB, and EFS.

• CloudEndure Disaster Recovery: Provides continuous replication of your machines to a staging area in AWS, enabling fast recovery from data center failures.

• Amazon FSx: Provides feature-rich file systems:

  • Amazon FSx for Windows File Server: Supports SMB protocol for Windows server integration.

  • Amazon FSx for Lustre: Uses Lustre file system for high-performance computing on Linux servers.

Conclusion

AWS offers a diverse range of storage services to meet various needs, from high-performance block storage to cost-effective archival solutions. Understanding these services and their use cases will help you choose the right storage solution for your applications and data requirements.