CloudFormation cfn-init pitfall: Auto scaling and throttling error rate exceeded

Michael Wittig – 11 Aug 2022

cfn-init is a little helper to install and configure EC2 instances managed with CloudFormation. Lately, I was running into issues when starting a more significant amount of EC2 (let’s say 50) during an auto scaling event. This blog post will teach you why the error happens and how to avoid it.

Throttling error rate exceeded

Introducing cfn-init

cfn-init configuration is added as metadata to a resource using the AWS::CloudFormation::Init key. The following example configures cfn-init to

  • create/update the file /etc/sample.conf.
  • enable & start the service sample (also restarts the service if /etc/sample.conf is changed).

cfn-init is usually executed in the user data script.

Type: 'AWS::EC2::Instance'
content: !Sub |
mode: '000400'
owner: root
group: root
enabled: true
ensureRunning: true
- '/etc/sample.conf'
# [...]
'Fn::Base64': !Sub |
#!/bin/bash -ex
/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource VirtualMachine --region ${AWS::Region}

The pitfall

The way cfn-init is implemented is this:

  1. Call the CloudFormation API DescribeStackResource to read the metadata.
  2. Validate and parse the metadata.
  3. Apply the configuration to the EC2 instance.

Unfortunately, the CloudFormation API has notorious low API rate limits, and cfn-init does not retry in the case of a rate exceeded error. Therefore, when many EC2 instances run cfn-init more or less at the same time, you will see the following error:

2022-06-02 07:13:14,838 [DEBUG] Response: 400***&Version=2010-05-15 [headers: {'x-amzn-RequestId': '***', 'Content-Type': 'application/json', 'Content-Length': '124', 'Date': 'Thu, 02 Jun 2022 07:13:14 GMT', 'Connection': 'close'}]
2022-06-02 07:13:14,838 [DEBUG] Response error: b'{""Error"":{""Code"":""Throttling"",""Message"":""Rate exceeded"",""Type"":""Sender""},""RequestId"":""***""}'

Solving the issue

How can we solve the issue?

  1. Do not use cfn-init at all.
  2. Load the metadata from a file.

To load the metadata from a file and not the CloudFormation API, create a file (e.g., metadata.json) like this:

"AWS::CloudFormation::Init": {
"config": {
"files": {
"/etc/sample.conf": {
"content": "[main]\nregion=eu-west-1\n",
"mode": "000400",
"owner": "root",
"group": "root"
"services": {
"sysvinit": {
"sample": {
"enabled": true,
"ensureRunning": true,
"files": [

And invoke cfn-init like this:

/opt/aws/bin/cfn-init -v metadata.json

I hope this article will help you avoid the pitfall.

Michael Wittig

Michael Wittig

I’ve been building on AWS since 2012 together with my brother Andreas. We are sharing our insights into all things AWS on cloudonaut and have written the book AWS in Action. Besides that, we’re currently working on bucketAV, HyperEnv for GitHub Actions, and marbot.

Here are the contact options for feedback and questions.