Importing CloudFormation templates into a CDK application

Introduction

In this brief blog post, we will look into how we can integrate an existing AWS CloudFormation template into an AWS CDK (Cloud Development Kit) application, and some of the use cases where we might want to use this approach. We will also talk about proxy objects and how they work when importing resources.

Before we start

👋🏼 Hi, I'd love to connect with like-minded professionals and talk about all things AWS. Reach out to me on LinkedIn and feel free to share your thoughts on this blog post, too.

https://www.linkedin.com/in/imduchy

Why import a CloudFormation template into a CDK application?

At first glance, importing a CloudFormation template into a CDK application might sound odd, but there are several scenarios where it makes perfect sense. Two of the most common use cases I came across are:

1. Deploying a template that someone from within or outside of our organization shared with us, as a part of our CDK application. For example, an SaaS provider sharing resources required to integrate with their platform, or an internal operations team sharing a template that deploys a VPC with on-premise connection.

2. Migrating from CloudFormation to CDK without the need to rewrite all of our existing infrastructure.

It allows us to deploy our stack(s) as a unified application, regardless of whether resources are defined in CloudFormation or CDK, using a single command (cdk deploy), instead of managing and deploying stacks using two deployment mechanisms.

We start with a CloudFormation template

Imagine we have deployed a CloudFormation stack to create resources for our business application. It’s a simple static website hosted with an S3 bucket and CloudFront distribution.

AWSTemplateFormatVersion: "2010-09-09"

Parameters:
  Environment:
    Type: String

Resources:
  WebsiteBucket:
    Type: AWS::S3::Bucket
    Properties:
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        BlockPublicPolicy: true
        IgnorePublicAcls: true
        RestrictPublicBuckets: true

  CloudFrontDistribution:
    Type: AWS::CloudFront::Distribution
    Properties:
      DistributionConfig:
        Origins:
          - DomainName: !GetAtt WebsiteBucket.DomainName
            Id: S3Origin
            S3OriginConfig: {}
            OriginAccessControlId: !Ref OriginAccessControl
      Enabled: true
      DefaultRootObject: index.html
      # ... left out for simplicity

  OriginAccessControl:
    Type: AWS::CloudFront::OriginAccessControl
    Properties:
      # ...

  WebsiteBucketPolicy:
    Type: AWS::S3:BucketPolicy
    Properties:
      # ...

After the initial POC, the team decided to switch to CDK to take advantage of its capabilities.

“So what do we do with the existing CloudFormation templates?”

Instead of dedicating time to rewriting our existing infrastructure as code, we want to keep delivering new features for our users. We also want to keep our deployment pipeline simple and deploy all our stacks and resources using a unified approach.

Import resources to CDK

The cloudformation-include.CfnInclude module allows us to import resources from a CloudFormation template, and use them in our CDK application as if they were originally defined using L1 constructs. The module essentially adds an AWS CDK API wrapper to the resources defined in the template.

Let’s take a look at how it works. First, we implement a CDK Stack for our static website and then import the CloudFormation template.

from aws_cdk import Stack
from aws_cdk import aws_cloudfront as cloudfront
from aws_cdk import aws_s3 as s3
from aws_cdk import cloudformation_include as cfn_include

class StaticWebsiteStack(Stack):
    def __init__(self, scope, construct_id, **kwargs):
        super().__init__(scope, construct_id, **kwargs)

        self._bucket: s3.IBucket = None
        self._distribution: cloudfront.IDistribution = None

        template = cfn_include.CfnInclude(
            scope=self,
            id="StaticWebsite",
            # Point to the location where we store our CloudFormation template
            template_file="./libs/website.template.yaml",
            # To keep logical IDs of the resources, the `preserve_logical_ids`
            # flag must be set to `True` (default value). If set to `False`,
            # CDK will generate unique logical IDs and try to redeploy them.
            preserve_logical_ids=True,
            # Here we can provide parameters that will be passed to the
            # CloudFormation template.
            parameters={
                "Environment": "Dev",
            },
        )

        # Get the bucket using its logical ID and `get_resource` method 
        cfn_bucket = template.get_resource(logical_id="WebsiteBucket")
        # Transform the L1 construct (CfnBucket) to IBucket (proxy object)
        self._bucket = s3.Bucket.from_bucket_name(
            scope=self,
            id="WebsiteBucket",
            bucket_name=cfn_bucket.ref,
        )

        # Get the distribution using its logical ID and `get_resource` method 
        cfn_distribution = template.get_resource(logical_id="CloudFrontDistribution")
        # Transform the L1 construct (CfnDistribution) to IDistribution (proxy object)
        self._distribution = cloudfront.Distribution.from_distribution_attributes(
            scope=self,
            id="CloudFrontDistribution",
            distribution_id=cfn_distribution.ref,
            domain_name=cfn_distribution.get_att("DomainName").to_string(),
        )

    @property
    def bucket_arn(self):
        return self._bucket.bucket_arn

    @property
    def distribution_domain_name(self):
        return self._distribution.distribution_domain_name

We have referenced the S3 bucket and CloudFront distribution using get_resource method and transformed them to L2 constructs which grant us access to some of the convenience methods such as S3 bucket permission grants. Straightforward.

But there’s an important aspect of importing resources I want to go through.

Proxy objects

If you look carefully at the previous code example, you will notice that the static methods from_bucket_name and from_distribution_attributes don’t return instances of type s3.Bucket and cloudfront.Distribution. Instead, they return instances of s3.IBucket and cloudfront.IDistribution (notice the “I”) respectively. They are what AWS calls proxy objects.

L2 constructs created from L1 constructs are proxy objects that refer to the L1 resource, similar to those created from resource names, ARNs, or lookups. Modifications to these constructs do not affect the final synthesized AWS CloudFormation template (since you have the L1 resource, however, you can modify that instead).

Reference: https://docs.aws.amazon.com/cdk/v2/guide/cfn_layer.html#develop-customize-unescape

There are a couple of points I want to touch on because they might be confusing. I will talk specifically about S3 buckets but the same applies to other resource types too.

• Both s3.Bucket and s3.IBucket are considered L2 constructs

• You can transform s3.CfnBucket (L1) into s3.IBucket → creating a proxy object

• But, you can not transform s3.CfnBucket into s3.Bucket

Modify resources imported using CfnInclude

The resources imported using the CfnInclude module will become a part of (will be managed by) our CDK app and can be accessed using L1 constructs. Because we have access to these L1 constructs, we can modify their attributes.

template = cfn_include.CfnInclude(
    scope=self,
    id="StaticWebsite",
    template_file="./libs/website.template.yaml",
    preserve_logical_ids=True,
    parameters={
        "Environment": "Dev",
    },
)

cfn_bucket: s3.CfnBucket = template.get_resource(logical_id="WebsiteBucket")
# ✅ This will work perfectly fine because we're using an L1 construct
cfn_bucket.bucket_name = "updated-through-l1-construct"

But we can't modify the S3 bucket through its proxy object. In the following code example, we transform the s3.CfnBucket into the s3.IBucket construct (proxy object) and try to modify the bucket’s name. This code will throw an error.

template = cfn_include.CfnInclude(
    scope=self,
    id="StaticWebsite",
    template_file="./libs/website.template.yaml",
    preserve_logical_ids=True,
    parameters={
        "Environment": "Dev",
    },
)

cfn_bucket: s3.CfnBucket = template.get_resource(logical_id="WebsiteBucket")
self._bucket: s3.IBucket = s3.Bucket.from_cfn_bucket(cfn_bucket)
# ❌ AttributeError: property 'bucket_name' of '_BucketBaseProxy' object has
# no setter
self._bucket.bucket_name = "updated-through-proxy-object"

To sum up, if we import resources from a CloudFormation template using the CfnInclude module, we have access to L1 constructs which we can use to modify the underlying resources. However, the underlying resources can’t be modified through their proxy (L2) objects.

Modify resources using lookup methods

Contrary to resources imported using CfnInclude module, resources imported using lookup methods will not become a part of our CDK app—CDK won’t manage them. Therefore, any changes made to the proxy objects will not affect the existing resources. These proxy objects also don’t hold a reference to an L1 construct.

imported_bucket: s3.IBucket = s3.Bucket.from_bucket_name(
    self,
    "S3BucketFromMyAwsAccount",
    bucket_name="cdk-hnb659fds-assets-111122223333-eu-central-1"
)
# ❌ AttributeError: property 'bucket_name' of '_BucketBaseProxy' object has
# no setter
imported_bucket.bucket_name = "updated-through-proxy-object"

# This will throw an error too because `imported_bucket` is a proxy object
# without a reference to an L1 construct (no default_child).
imported_cfn_bucket = imported_bucket.node.default_child
# ❌ AttributeError: 'NoneType' object has no attribute 'bucket_name'
imported_cfn_bucket.bucket_name = "updated-through-l1-construct"

So, to reference an already existing resource from our AWS account, we can use a static lookup method (assuming the resource supports it) to create a proxy object. This proxy object, however, can’t be used to modify the underlying resource because CDK doesn’t manage it. If we want to bring an existing resource into our application, we can use cdk import instead.

Use convenience methods exposed by proxy objects

Although we can’t use proxy objects to modify the underlying resource, we can use their convenience (L2) methods such as S3 permission grants.

In the next screenshot, you can see some of the L2 methods the S3 proxy object exposed for creating an EventBridge rule that would get triggered on various CloudTrail data events.

Deploy a stack with imported resources

We can now reference the StaticWebsiteStack stack in our app.py file and deploy the template as a part of our CDK application.

⚠️ If we already have an active stack deployed from the CloudFormation template, we don’t want CDK to create a new stack. Instead, we want CDK to take over the existing stack and manage its resources. We can do that by explicitly setting the stack_name attribute to the name of the existing CloudFormation stack.

from aws_cdk import App

from libs.static_website import StaticWebsiteStack

app = App()

static_website_stack = StaticWebsiteStack(
    app,
        "StaticWebsiteStack",
        # Setting the `stack_name` attribute to the name of the existing
        # CloudFormation stack.
        stack_name="StaticWebsiteBucketManual",
)

When we run cdk diff, CDK will see that there is already an active stack and compare the synthesized template against the deployed template. The first time we try to deploy our stack, CDK will create the BootstrapVersion parameter (that’s expected) and our output should look similarly to this:

We can now execute cdk deploy and CDK will take care of importing the resources from the template and deploying the changes against the existing stack. That’s it, easy!

Conclusion

We discussed how the CfnInclude module allows you to import, manage, and deploy CloudFormation templates using CDK. We mentioned some of the most common use cases for using this approach. And last but not least, we talked about proxy objects and their limitations.

✋🏼 Before you go

Thank you for reading all the way here. If you enjoyed this blog post, please feel free to share it or share feedback with me. You can reach out and connect with me on my LinkedIn http://linkedin.com/in/imduchy.