3½ ways to workaround missing CloudFormation support

Are you following the Infrastructure as Code approach using CloudFormation? If so, I bet you encountered a situation where CloudFormation misses support for a service’s latest features. I run into those issues weekly! So what can we do about it?

Do you prefer listening to a podcast episode over reading a blog post? Here you go!

For quite some time, a Custom Resource backed by a Lambda function was the way to go. A couple of weeks ago, I learned about new creative ways from Soenke (co-author of this blog post and co-founder of superluminar). Before we present you with the workarounds, let me issue one warning: If you start using workarounds, you will face issues when switching to native CloudFormation in the future. If you can not replace your workaround resource without downtime (e.g., datastores, network config), I strongly advise against using any of the workarounds. Better wait for native CloudFormation support or go with Terraform.

Custom Resource

A custom resource has to support three actions: create, update, delete. The steps work like this:

• The create event contains the ResourceProperties defined in your CloudFormation template. Create the resource and return at least a PhysicalResourceId to identify the resource later.
• The update event contains the PhysicalResourceId from the create step, the OldResourceProperties, and the new ResourceProperties defined in your template. If possible, update the existing resource (identified by PhysicalResourceId) using the new ResourceProperties. If the resource can not be updated, create a new resource, and return a new PhysicalResourceId. In the case of a replacement, you receive a delete event during the update cleanup phase for the old PhysicalResourceId.
• Finally, the delete event contains the PhysicalResourceId and ResourceProperties defined in your template. Delete the resource identified by the PhysicalResourceId.

You can either implement the custom resource using a Lambda function or use an SNS topic with a subscriber of your choice. If you are interested in the details, check out this Custom Resource to manage the IAM Password Policy.

Resource Providers & CloudFormation Registry

Resource Providers are the evolution of custom resources. The benefits are:

• A well-defined schema of the resource that is machine-readable
• You can share them with other AWS accounts
• All features of CloudFormation are supported, such as drift detection and resource import
• Resource Provider code execution is managed by AWS as well as IAM credentials

Sharing Resource Providers with other accounts is more cumbersome than it should be at the moment. Development effort is higher compared to a custom resource. The best examples are the official CloudFormation resource types.

SSM Automation documents

Systems Manager Automation provides a way to call arbitrary AWS APIs with a YAML/JSON language. One advantage of this approach is that one does not need to maintain or update language runtimes like Javascript or Python.

SSM Automation also provides basic features as conditions and a library of standard operating procedures such as creating a CloudFormation stack or executing other AWS-provided SSM documents.

CloudFormation cannot trigger SSM Automation documents directly, but EventBridge events can trigger them. A practical example is setting up GuardDuty Delegated Administrator - a feature that has no CloudFormation support at the time of writing this.

It’s also possible to execute automation documents across AWS accounts and regions. This comes in handy when administering medium-sized or bigger multi-region/account setups. For example, enabling GuardDuty for existing accounts.

Important detail: The underlying AWS Client SDK is only updated periodically, so it’s impossible to use newer features and releases of AWS APIs.

CloudWatch Synthetics

Some AWS services or features don’t provide APIs at all. Currently, examples are:

• Billing features like setting the VAT ID, tax inheritance, or currency
• AWS Control Tower
• Partially Amazon Connect and AWS Chatbot

Automating these features is nearly impossible, but as a last resort, it can be done with headless browser frameworks like Puppeteer/Selenium.

AWS provides a service called CloudWatch Synthetics to monitor websites. It uses Puppeteer/Selenium under the hood, so it can be used to automatically login into the AWS Management Console and automate anything.
A big advantage is that synthetics canaries are completely managed by AWS, so one does not need a process to install or keep runtimes and libraries up to date.

One example is activating Control Tower (which provides no API currently) via Cloudformation. The canary is started right away after creation since StartCanaryAfterCreation is set to true.

There are some limitations as well:

• The AWS Management Console is no stable API. If something gets changed in the UI, e.g., new design or HTML/CSS changes, the automation could break.
• It’s not easy to get the result of the automation since the API call does not wait until the canary is finished. This could be circumvented by using a custom resource Lambda function with Puppeteer/Selenium installed instead of Synthetics.
• If more control over the start conditions of the canary was necessary, it would need a CloudFormation custom resource as a shim to trigger the StartCanary API (see example).

Summary

Let me end with a summary. If there is no API, CloudWatch Synthetics is your only option. If there is an API, you can choose between SSM Automation, Custom Resource, and Resource Registry. Custom resources and SSM provide a fast track. Resource Providers are the way to implement a resource type in CloudFormation properly.