AWS EFS migration with AWS DataSync

AWS DataSync is a managed data transfer service that makes it faster and simpler to move data between on-premises storage and Amazon S3, Amazon Elastic File System (EFS), and Amazon FSx for Windows File Server. This service automates and accelerates copying data over the internet or AWS Direct Connect.

Here's a step-by-step tutorial on how to use AWS DataSync to transfer data from an unencrypted AWS EFS to another encrypted EFS in the same region and account.

Prerequisites

• Two AWS EFS file systems set up in your AWS account, one unencrypted (source) and one encrypted (destination).

  

• Necessary permissions in AWS IAM for creating and managing DataSync tasks.

Step 1: Set Up AWS DataSync

1. Go to the AWS Management Console: Log in to your AWS account and navigate to the DataSync service.

2. Create a DataSync Agent: DataSync agents are not required when transferring data between AWS services, but make sure that the regions of your EFS file systems are supported by DataSync.

Step 2: Configure Source and Destination

1. Configure Source Location (Unencrypted EFS):

   • Select “Amazon EFS” for location type.

   • Provide the details of your unencrypted EFS file system.

     

2. Configure Destination Location (Encrypted EFS):

   • Select “Amazon EFS” for location type.

   • Provide the details of your encrypted EFS file system.

     

Step 3: Create a Data Transfer Task

1. Create a New Task:

   • In the DataSync console, choose “Create task”.

   • Select the source and destination locations you just configured.

2. Configure Task Settings:

   • You can specify options like data validation, metadata copying, and scheduling.

     

     

     

3. Launch the Task:

   • Once the task is configured, review the settings, and then launch the task to start the data transfer.

Step 4: Monitor the Transfer

• Check Progress: You can monitor the progress directly in the DataSync console. It provides information about the status, data transferred, and transfer speed.

Step 5: Validation and Cleanup

• Validate Data Transfer: Ensure that all files are transferred correctly and completely.

• Cleanup Resources: If you no longer need the DataSync resources or the source EFS, consider cleaning them up to avoid unnecessary charges.

Additional Notes

• Data Transfer Cost: Be aware of the costs associated with data transfer using DataSync.

• Encryption: Since the destination EFS is encrypted, AWS handles the encryption of data in transit and at rest.

AWS DataSync handles the heavy lifting in data transfer scenarios, making it a reliable choice for moving data between EFS file systems, especially when dealing with large datasets or the need for regular data synchronization.

Advantages

Transferring data from an unencrypted AWS Elastic File System (EFS) to an encrypted EFS using AWS DataSync can be a good choice depending on your specific requirements and context. Here are some factors to consider:

1. Enhanced Security: Moving data to an encrypted EFS adds an additional layer of security. Encryption at rest protects your data from unauthorized access, which is crucial for sensitive or confidential information.

2. Compliance Requirements: If your organization has compliance requirements that mandate data encryption, migrating to an encrypted EFS can help meet these standards.

3. Automated and Efficient: Using AWS DataSync for the transfer automates the process, ensuring a more efficient and error-free operation compared to manual methods.

4. Data Integrity: DataSync ensures data integrity during transfer. It checks for consistency in files transferred, which is critical when handling important data.

Considerations

1. Cost Implications: AWS DataSync incurs charges for data transfer. It's important to evaluate the cost implications, especially if dealing with large volumes of data.

2. Performance Impact: While AWS DataSync is optimized for high-speed data transfer, the performance may vary based on your network conditions, the size of the data, and the configurations of your EFS instances.

3. One-time Transfer vs. Ongoing Need: If this is a one-time transfer, AWS DataSync is a great tool. However, if you require ongoing synchronization between two EFS instances, you might need to set up a regular sync task which could have additional implications, both in terms of cost and management.

4. Network and Resource Utilization: Ensure your network and AWS resources can handle the data transfer load, especially if the EFS is in use during the migration process.

Alternatives

• If the primary goal is to encrypt data at rest in EFS, AWS also provides an option to enable encryption on an existing EFS file system without needing to transfer data to a new file system. This could be a simpler solution if the only objective is to encrypt the existing data.

Using AWS DataSync to transfer data from an unencrypted to an encrypted EFS is a good choice when you need efficient, secure, and reliable data transfer, and if the added security benefits align with your organization's requirements. However, always weigh the benefits against the costs and the specific needs of your use case.

References:

1. AWS DataSync

2. Getting started with AWS DataSync

3. How AWS DataSync transfers work

4. Configuring AWS DataSync transfers with Amazon EFS

5. Encrypt at rest existing EFS