The journey of migrating a common service from one Virtual Private Cloud (VPC) to another can be a daunting task. It’s like moving your entire house from one neighbourhood to another but with the added complexity of ensuring that your precious belongings (in this case, your service) are safe and secure during the move. In this article, we will walk you through the journey of migrating a common service from VPC 1 to VPC 2, highlighting the key steps that we took to ensure a smooth transition.

Let’s call our original VPC “Rohan” and our new VPC “Gondor”.

Step 1: Decoupling the Tightly Coupled Common Service

Our common service was tightly coupled with our vertical services in Rohan, making it difficult to move without disrupting the entire ecosystem. It was like trying to remove a brick from a wall without bringing the whole thing crashing down.

In order to proceed, two tasks were necessary. The first was to establish a uniform format for the request body. The second involved moving all current data to this new format.

We also implemented some changes in the mapping query to help with the transition and ensure compatibility with Gondor. These changes allowed us to move the common service without disrupting the rest of our system.

When decoupling the vertical services and the common service, it was necessary to make changes to the common service. However, it was important to ensure that these changes did not cause any disruptions to the existing functionality of the system. To achieve this, we took a backward-compatible approach to the changes.

This means that any changes made to the common service were designed to be compatible with the existing interfaces, APIs, and workflows used by the vertical services. This allowed the vertical services to continue using the common service without any interruptions.

Step 2: Taking Care of Security

When migrating a common service from one VPC to another, it’s crucial to ensure that security is taken care of. There are several ways to establish secure communication between VPCs, including VPC peering, transit gateway, and public call (with API Gateway in the picture).

VPC peering is a straightforward method of connecting two VPCs within the same region. It allows for secure communication between VPCs over the Amazon network, without the need for a VPN connection. However, VPC peering has some limitations, such as the fact that it only supports communication between VPCs within the same region.
Public call is another option, where the service is exposed publicly, and requests are made to the public endpoint. However, this approach has significant security implications and should be avoided wherever possible. It requires robust security measures, such as API Gateway, to ensure that only legitimate requests are accepted and that sensitive data is protected.
Transit Gateway is a service that allows for centralized management of VPCs, enabling secure communication between VPCs across different accounts and regions. It acts as a hub that connects multiple VPCs and can be used to route traffic securely between them. Transit Gateway has several advantages over other options, such as VPC peering, including:

Scalability: Transit Gateway can handle a much larger number of VPCs than VPC peering, making it an excellent choice for the future plans of Gondor.
Simplified management: Transit Gateway provides a central point of management for all VPCs, simplifying configuration and troubleshooting.
Security: Transit Gateway provides additional security features, such as network segmentation and access control lists, to ensure that traffic between VPCs is secure.

For these reasons, we chose Transit Gateway as the method for establishing secure communication between Rohan and Gondor during our migration. It provided us with a scalable, secure, and easy-to-manage solution that ensured the integrity of our services.

Step 3: Pragmatics of API-gateway

One of the most effective ways to secure your service is by implementing a filter in the API Gateway that can remove any potentially spoofed headers from incoming requests.

We implemented such a filter in the API Gateway to ensure that only legitimate requests were accepted, and any requests with spoofed headers were filtered out. This involved removing all incoming headers in the request. We used a special token header value to look into the application configuration of the API Gateway and find out the required headers to set and their values. This made the entire process more secure and ensured that the migrated common service would work seamlessly across VPCs.

In addition to this, we also implemented IP whitelisting to restrict access to the service only to authorized sources. This further ensured that requests were only accepted from trusted sources, and helped prevent attacks from malicious actors.

Furthermore, we added several important features like quota and t**ime-to-live (TTL) in the API Gateway to improve the service’s reliability and availability. Quotas can limit the number of requests a user can make within a specific time frame, while TTL can ensure that data is not stored for longer than necessary, improving the performance and security of the service.

In conclusion, it’s evident that securing the migrated common service across VPCs was crucial to maintaining the integrity of the service. By implementing an API Gateway with robust filtering and security features, we were able to create a safe end-to-end request journey from Gondor to Rohan and Rohan to Gondor. With these measures in place, we can be confident that our common service is secure, reliable, and available across VPCs.

YOU SHALL NOT PASS!

Just as Gandalf stood guard at the bridge of Khazad-dûm to prevent the Balrog from passing, the filter in the API Gateway stood guard to prevent unauthorized requests from passing through. 🧙‍♂️

Step 4: Making Changes in Vertical Services

To ensure that our vertical services were able to access the common service after it was migrated, we had to make changes to their configuration. Each vertical service had to be updated to point to the Gondor where the common service was now hosted. This involved modifying the network settings of each service to ensure that requests were being routed to the correct destination.

Step 5: Testing and Validation

Testing and validation are critical components of any migration project. It’s essential to ensure that all aspects of the system are functioning properly and securely before and after the migration. Without proper testing, there is a risk of introducing errors, vulnerabilities, and other issues that could potentially compromise the security and performance of the system.

We followed a rigorous testing and validation process to ensure that the system was functioning as expected. We conducted unit tests, manual tests, and system tests to verify that all components of the system were working correctly and that there were no issues or errors. We also conducted security testing to identify and address any vulnerabilities or potential threats to the system.

After the migration, we continued to monitor the system closely and perform regular testing and validation to ensure that everything was still functioning correctly. This allowed us to identify and address any issues or problems that may have arisen due to the migration.

Step 6: Releasing the Changes

In the final stage of the migration process, we released the changes during a designated window to minimize business impact. This involved meticulously coordinating with all stakeholders to ensure a smooth transition and minimize any potential disruptions to our services.

It was like launching a rocket into space — a momentous occasion that required careful planning, coordination, and execution. 🚀

Conclusion

In conclusion, the migration of our common service to a new VPC was no easy feat, but with careful planning, execution, and the use of robust security measures, we were able to make the transition with minimal disruption. Not only did this help us to streamline our operations and improve our security posture, but it also gave me a newfound confidence in the ability to tackle even the most complex challenges. We can now move forward with the knowledge that we have a solid foundation for our services and can continue to innovate and improve our offerings.

An Odyssey of VPCs