6 unknown CloudFormation features you should know about

Michael Wittig ā€“ 15 Apr 2019

I was recently invited to a CloudFormation workshop with a group of early CloudFormation users. I soon realized that the group had a good understanding of the basics, so I started to introduce more advanced features.

6 CloudFormation features you should know about

Today, I would like to share with you six CloudFormation features that have inspired the workshop participants most.

cfn-lint

Writing CloudFormation templates is error prone. We should use every tool available to make our lives easier. I use the following tools:

  1. yamllint Locally ensures that our yaml is correct
  2. cfn-lint Locally checks if the template makes sense
  3. aws cloudformation validate-template Sends the template to AWS for validation

yamllint and cfn-lint are integrated into my editor, and I execute aws cloudformation validate-template before I git commit.

Creation policy

A CloudFormation creation policy is helpful if you provision an EC2 instance, typically by using user data. By default, when CloudFormation creates and EC2 instance it will not wait for the operating system and application to be ready. With a creation policy, you can ask CloudFormation to wait for an external signal.

In the following snippet, you find a CreationPolicy that instructs CloudFormation to wait for a signal with a timeout of 10 minutes (PT10M). The /opt/aws/bin/cfn-signal command sends the signal to CloudFormation to indicate that the user data script finished successfully (--exit-code 0).

#[...]
Resources:
VirtualMachine:
Type: 'AWS::EC2::Instance'
CreationPolicy:
ResourceSignal:
Timeout: PT10M
Properties:
#[...]
UserData:
'Fn::Base64': !Sub |
#!/bin/bash -ex
# run some commands
/opt/aws/bin/cfn-signal --exit-code 0 --resource VirtualMachine --region ${AWS::Region} --stack ${AWS::StackName}

Keep in mind that the logical id VirtualMachine of the resource needs to be referenced in cfn-signal using --resource VirtualMachine.

You can even improve this snippet to let CloudFormation know if the user data script failed using #!/bin/bash -e, trap, and --exit-code 1:

#[...]
Resources:
VirtualMachine:
Type: 'AWS::EC2::Instance'
CreationPolicy:
ResourceSignal:
Timeout: PT10M
Properties:
UserData:
'Fn::Base64': !Sub |
#!/bin/bash -ex
trap '/opt/aws/bin/cfn-signal --exit-code 1 --resource VirtualMachine --region ${AWS::Region} --stack ${AWS::StackName}' ERR
# run some commands
/opt/aws/bin/cfn-signal --exit-code 0 --resource VirtualMachine --region ${AWS::Region} --stack ${AWS::StackName}

Check out the following links if you are interested in real-world examples:

  • EC2 instance: CreationPolicy and cfn-signal
  • Auto Scaling Group: CreationPolicy and cfn-signal (Keep in mind that the CreationPolicy is attached to the auto scaling group while the cfn-signal is added to the user data in the launch configuration)

Update policy

A CloudFormation update policy triggers a rolling update to make changes to an auto scaling group. A rolling update will execute your change in small batches (e.g., instance by instance).

The following CloudFormation snippet demonstrates a rolling update that waits for signals in the same way you learned about in the creation policy:

#[...]
Resources:
LaunchConfiguration:
Type: 'AWS::AutoScaling::LaunchConfiguration'
Properties:
# [...]
AutoScalingGroup:
Type: 'AWS::AutoScaling::AutoScalingGroup'
Properties:
# [...]
LaunchConfigurationName: !Ref LaunchConfiguration
UpdatePolicy:
AutoScalingRollingUpdate:
PauseTime: PT10M
WaitOnResourceSignals: true

Check out the following links if you are interested in real-world examples:

A update policy can do even more.

Deletion policy

A CloudFormation deletion policy can prevent serious data loss. If CloudFormation deletes a resource, you can instruct CloudFormation to perform a backup first. This works for the following resource types:

  • AWS::RDS::DBInstance (enabled by default)
  • AWS::RDS::DBCluster (enabled by default)
  • AWS::EC2::Volume
  • AWS::ElastiCache::CacheCluster (only Redis)
  • AWS::ElastiCache::ReplicationGroup
  • AWS::Redshift::Cluster
  • AWS::Neptune::DBCluster

The following snippet shows a deletion policy:

#[...]
Resources:
Volume:
DeletionPolicy: Snapshot
Type: 'AWS::EC2::Volume'
Properties:
#[...]

Check out the following links if you are interested in real-world examples:

But what about other resource types? You can also instruct CloudFormation not to delete a resource at all. We use this to prevent deletions of KMS keys which can also cause data loss. Why is it dangerous to delete KMS keys? You can delete them even if they are in use. Sometimes it is now obvious that the key is still used in a snapshot. Once the key is deleted, the snapshot cannot be restored anymore.

#[...]
Resources:
Key:
DeletionPolicy: Retain
Type: 'AWS::KMS::Key'
Properties:
#[...]

Check out the following links if you are interested in real-world examples:

Update replace policy

The CloudFormation update replace policy is similar to the deletion policy. You can use it to prevent data loss. If you update an existing stack, CloudFormation figures out what resources need to be updated. For some attributes, an update requires replacement of the resource. Therefore, CloudFormation creates the new resource and when successful, deletes the old one. If this resource was your database, you are in troubles.

The following resource types can be backed up before they are replaced:

  • AWS::RDS::DBInstance
  • AWS::RDS::DBCluster
  • AWS::EC2::Volume
  • AWS::ElastiCache::CacheCluster (only Redis)
  • AWS::ElastiCache::ReplicationGroup
  • AWS::Redshift::Cluster
  • AWS::Neptune::DBCluster

The following snippet shows an update replace policy.

#[...]
Resources:
Volume:
UpdateReplacePolicy: Snapshot
Type: 'AWS::EC2::Volume'
Properties:
#[...]

Check out the following links if you are interested in real-life examples:

But what about other resource types? As with the deletion policy, you can also instruct CloudFormation to not delete a resource at all during a replacement.

#[...]
Resources:
Key:
UpdateReplacePolicy: Retain
Type: 'AWS::KMS::Key'
Properties:
#[...]

cfn-init

Writing shell scripts to configure and install en EC2 instance is error prone. Sometimes, you need a tool that is easier to use than a script but not as complex as Chef/Puppet/Ansible/ā€¦

cfn-init solves that problem. cfn-init is executed in the user data script and fetches configuration information from your CloudFormation stack. The configuration is attached as metadata to the resource and supports:

  • Execution of commands
  • Creation of files
  • Fetching of external sources (tar, tar+gzip, tar+bz2, and zip)
  • Creation of users and groups
  • Installation of packages (yum, rpm, gems)
  • Management of services

The following CloudFormation snippet shows how you can install the package awslogs.

#[...]
Resources:
VirtualMachine:
Type: 'AWS::EC2::Instance'
Metadata:
'AWS::CloudFormation::Init':
config:
packages:
yum:
awslogs: []
Properties:
#[...]
UserData:
'Fn::Base64': !Sub |
#!/bin/bash -ex
/opt/aws/bin/cfn-init -v --resource VirtualMachine --region ${AWS::Region} --stack ${AWS::StackName}

Check out the following links if you are interested in real-world examples:

The user data script only runs once. If you update the stack and change the metadata, nothing happens. cfn-hup is a way ro run cfn-init regularly to notice and apply updates to the metadata.

Summary

CloudFormation is a great service about which we have written more than 40 articles. There are many features to discover such as the numerous policies that you can attach to resources: CreationPolicy, DeletionPolicy, UpdatePolicy, and UpdateReplacePolicy. Besides that, the tooling around CloudFormation has also evolved in the past years. cfn-lint saves you time by spotting failures in your templates early.

The Reddit user coinclink mentioned another hidden feature that I use often: aws cloudformation package and aws cloudformation deploy.

Michael Wittig

Michael Wittig

Iā€™m the author of Amazon Web Services in Action. I work as a software engineer, and independent consultant focused on AWS and DevOps.

You can contact me via Email, Twitter, and LinkedIn.

Briefcase icon
Hire me
Cover of Rapid Docker on AWS

New book: Rapid Docker on AWS

A rapid way to get your web application up and running on AWS. Made for web developers and DevOps engineers who want to dockerize their web applications and run their containers on Amazon Web Services. Prior knowledge of Docker and AWS is not required.

Buy icon
Buy now
Marbot Logo

Incident Management for Slack

Team up to solve incidents with our chatbot marbot. Never miss a critical alert. Escalate alerts from your AWS infrastructure among your team members. Strong integrations with all parts of your AWS infrastructure: CloudWatch, Elastic Beanstalk, RDS, EC2, ...

Slack icon
Try for free
šŸ“š Rapid Docker on AWS
A rapid way to get your web application up and running on AWS. Learn how to package your application into Docker containers. Learn more.