AWS Cloud Data Migration Services

Ready to move Data to the cloud?
Choosing the right approach is critical to easily and efficiently migrate your On-Prem Data to AWS Cloud.

Here I have provided some of the AWS Services that will be useful to determine various approaches from which you choose the appropriate method to Migrate your Data to AWS Cloud.

1. Hybrid cloud storage

• AWS Storage Gateway

  • AWS Storage Gateway is a set of hybrid cloud storage services that provide on-premises access to virtually unlimited cloud storage.

• AWS Direct Connect

  • AWS Direct Connect establishes a dedicated connection from an on-premises network to one or more VPCs in the same region in AWS.
  • AWS Direct Connect can reduce network costs, increase bandwidth throughput, and provide a more consistent network experience than internet-based connections. It uses industry-standard 802.1q VLANs to connect to Amazon VPC using private IP addresses.
  • AWS Direct Connect lets you establish 1 Gbps or 10 Gbps dedicated network connections (or multiple connections) between AWS networks and one of the AWS Direct Connect locations.
    

2. Online data transfer

• AWS DataSync

  • AWS DataSync is a secure, online service that automates and accelerates moving data between on premises and AWS Storage services.
  • DataSync can copy data between Network File System (NFS) shares, Server Message Block (SMB) shares, Hadoop Distributed File Systems (HDFS), self-managed object storage, AWS Snowcone, Amazon Simple Storage Service (Amazon S3) buckets, Amazon Elastic File System (Amazon EFS) file systems, Amazon FSx for Windows File Server file systems, Amazon FSx for Lustre file systems, Amazon FSz for OpenZFS file systems, and Amazon FSx for NetApp ONTAP file systems.
    

• AWS Transfer Family

  • Seamlessly migrate, automate, and monitor your file transfer workflows in and out of Amazon S3 & Amazon EFS.
  • AWS Transfer Family supported Protocols : SFTP, FTPS, and FTP
  • User Guide AWS Transfer Family User Guide
    

• Amazon S3 Transfer Acceleration

  • Amazon S3 Transfer Acceleration is a bucket-level feature that enables fast, easy, and secure transfers of files over long distances between your client and an S3 bucket.
  • Transfer Acceleration takes advantage of the globally distributed edge locations in Amazon CloudFront. As the data arrives at an edge location, the data is routed to Amazon S3 over an optimized network path.
  • Amazon S3 Transfer Acceleration Speed Comparison Tool

• Amazon Kinesis Data Firehose

  • Amazon Kinesis Data Firehose is an extract, transform, and load (ETL) service that reliably captures, transforms, and delivers streaming data to data lakes, data stores, and analytics services.
    

• APN Partner Products

  • AWS Partner Network (APN) Blog
  • Category: Migration & Transfer Services

3. Offline data transfer

• AWS Snow Family (AWS Snowcone, Snowball Edge, Snowmobile)
  • Can Move 8 terrabytes to 100 petabytes of data to & from AWS.
    

4. Unmanaged cloud data migration tools

rsync
Customers use rsync, an open source tool, along with 3rd party file system tools to copy data directly into S3 buckets.

S3 command line interface
Customers use the Amazon S3 CLI to write commands to move data directly into S3 buckets.

S3 Glacier command line interface Customers use the Amazon S3 Glacier CLI to move data into S3 Glacier vaults.

The common cloud data migration challenge

The daunting realities of data transport apply to most projects. How do you gracefully move from your current location to your new cloud, with minimal disruption, cost and time? What is the smartest way to actually move your GB, TB, or PB of data?

It's a basic underlying problem: how much data can move how far how fast? For a best-case scenario use this formula:

Number of days = (Total Bytes)/(Megabits per second * 125 * 1000 * Network Utilization * 60 seconds * 60 minutes * 24 hours)

For example, if you have a T1 connection (1.544Mbps) and 1TB (1024 1024 1024 * 1024 bytes) to move in or out of AWS the theoretical minimum time it would take to load over your network connection at 80% network utilization is 82 days.

Customers can approach this in two ways:
One, They can use very basic unmanaged migration tools to move their data or they select one of AWS's suite of services noted above.

As a general rule of thumb, for best results we suggest:

Connection Data Scale Method Less than 10 Mbps Less than 500 GB Unmanaged More than 10 Mbps More than 500 GB Managed service

Thank you for the read. Hope this will be useful. Happy Learning.

Like and Follow for more Azure and AWS Content.

Regards,
Jineshkumar Patel