3½ ways to workaround missing CloudFormation support
Are you following the Infrastructure as Code approach using CloudFormation? If so, I bet you encountered a situation where CloudFormation misses support for a service’s latest features. I run into those issues weekly! So what can we do about it?
Do you prefer listening to a podcast episode over reading a blog post? Here you go!
For quite some time, a Custom Resource backed by a Lambda function was the way to go. A couple of weeks ago, I learned about new creative ways from Soenke (co-author of this blog post and co-founder of superluminar). Before we present you with the workarounds, let me issue one warning: If you start using workarounds, you will face issues when switching to native CloudFormation in the future. If you can not replace your workaround resource without downtime (e.g., datastores, network config), I strongly advise against using any of the workarounds. Better wait for native CloudFormation support or go with Terraform.
- The create event contains the
ResourcePropertiesdefined in your CloudFormation template. Create the resource and return at least a
PhysicalResourceIdto identify the resource later.
- The update event contains the
PhysicalResourceIdfrom the create step, the
OldResourceProperties, and the new
ResourcePropertiesdefined in your template. If possible, update the existing resource (identified by
PhysicalResourceId) using the new
ResourceProperties. If the resource can not be updated, create a new resource, and return a new
PhysicalResourceId. In the case of a replacement, you receive a delete event during the update cleanup phase for the old
- Finally, the delete event contains the
ResourcePropertiesdefined in your template. Delete the resource identified by the
You can either implement the custom resource using a Lambda function or use an SNS topic with a subscriber of your choice. If you are interested in the details, check out this Custom Resource to manage the IAM Password Policy.
Resource Providers are the evolution of custom resources. The benefits are:
- A well-defined schema of the resource that is machine-readable
- You can share them with other AWS accounts
- All features of CloudFormation are supported, such as drift detection and resource import
- Resource Provider code execution is managed by AWS as well as IAM credentials
Sharing Resource Providers with other accounts is more cumbersome than it should be at the moment. Development effort is higher compared to a custom resource. The best examples are the official CloudFormation resource types.
Looking for a new challenge?
CloudFormation cannot trigger SSM Automation documents directly, but EventBridge events can trigger them. A practical example is setting up GuardDuty Delegated Administrator - a feature that has no CloudFormation support at the time of writing this.
It’s also possible to execute automation documents across AWS accounts and regions. This comes in handy when administering medium-sized or bigger multi-region/account setups. For example, enabling GuardDuty for existing accounts.
Important detail: The underlying AWS Client SDK is only updated periodically, so it’s impossible to use newer features and releases of AWS APIs.
Some AWS services or features don’t provide APIs at all. Currently, examples are:
- Billing features like setting the VAT ID, tax inheritance, or currency
- AWS Control Tower
- Partially Amazon Connect and AWS Chatbot
Automating these features is nearly impossible, but as a last resort, it can be done with headless browser frameworks like Puppeteer/Selenium.
AWS provides a service called CloudWatch Synthetics to monitor websites. It uses Puppeteer/Selenium under the hood, so it can be used to automatically login into the AWS Management Console and automate anything.
A big advantage is that synthetics canaries are completely managed by AWS, so one does not need a process to install or keep runtimes and libraries up to date.
One example is activating Control Tower (which provides no API currently) via Cloudformation. The canary is started right away after creation since
StartCanaryAfterCreation is set to
There are some limitations as well:
- The AWS Management Console is no stable API. If something gets changed in the UI, e.g., new design or HTML/CSS changes, the automation could break.
- It’s not easy to get the result of the automation since the API call does not wait until the canary is finished. This could be circumvented by using a custom resource Lambda function with Puppeteer/Selenium installed instead of Synthetics.
- If more control over the start conditions of the canary was necessary, it would need a CloudFormation custom resource as a shim to trigger the StartCanary API (see example).
Let me end with a summary. If there is no API, CloudWatch Synthetics is your only option. If there is an API, you can choose between SSM Automation, Custom Resource, and Resource Registry. Custom resources and SSM provide a fast track. Resource Providers are the way to implement a resource type in CloudFormation properly.