Prevent CloudFormation Change Sets from piling up
Recently, I’ve stumbled upon a problem when using
aws cloudformation deploy within deployment pipelines (Jenkins, GitLab CI, …) that I wanted to share with you.
Usually, I’m using the AWS CLI to deploy CloudFormation stacks.
aws cloudformation package --template-file example.yml --output-template-file output.yml --s3-bucket example
aws cloudformation package command packages the template
example.yml and uploads dependencies like nested stack templates to S3.
aws cloudformation deploy command creates or updates the stack
example. The option
--no-fail-on-empty-changeset makes sure the command does not throw an error in case the template has not changed, which is very likely when using a deployment pipeline.
It is essential to know that the pipeline creates a change set whenever executing
aws cloudformation deploy. Deploying a stack without making any changes to the template or the parameters leads to a failed change set:
The submitted information didn't contain changes. Submit different information to create a change set.
aws cloudformation deploy command does not clean up those failed change sets. This leads to change sets piling up. And one day or another your deployment pipeline will fail with the following error message:
An error occurred (LimitExceededException) when calling the CreateChangeSet operation: ChangeSet limit exceeded for stack ...
You have reached the maximum number of change sets for a stack.
How to fix that? It would be great if
aws cloudformation deploy would clean up failed change sets automatically. Keep an eye on #4534 for progress on that.
For now, adding the following bash script to your deployment pipeline might help. The
cleanup() function list all stacks starting with
prefix, fetches the change sets for each stack, and deletes change sets in status
# cleanup (region, prefix)
The following command executes the function
cleanup() for region
eu-west-1 and stack name prefix
cleanup eu-west-1 example-
aws cloudformation deploy leaves behind waste that you need to deal with.