Rapid CloudFormation: cfn-modules

Today, we release a new open source project to make your CloudFormation live easier. We promise rapid CloudFormation with cfn-modules. Our modules provide common building blocks to automate your infrastructure with plain CloudFormation templates.

Why cfn-modules?

We started with aws-cf-templates in 2015. Three years later, we believe that we have learned enough to come up with a new approach to use CloudFormation more efficient.

Modular

Reusing CloudFormation templates is hard. Most often, templates are initially copied and then modified.

Two problems arise. First, updates to the copy are not applied to the original. Second, updates to the original are not applied to the copy. In essence: we do not learn from each other!

By using an easy to use package manager (npm) you can install and update cfn-modules to spin up complex infrastructure in minuted that just works.

Production ready

All modules are production-ready. If no other limitations are documented, they are:

• Highly available
  • no single point of failure
• Scalable
  • increase or decrease the capacity based on utilization
• Secure
  • using the latest operating systems and software components
  • follow the least privilege principle (e.g., IAM policies and Security Groups)
  • backups enabled
  • encryption at-rest enabled
  • encryption in-transit enabled and preferred
• Operations friendly
  • logging enabled
  • alerting enabled
  • updatable

Open source

All modules are licensed under Apache-2.0. Commercial use is allowed.

Prerequisites

• AWS CLI installed (install)
• npm 3.x installed (install Node.js 8.x)

Getting started

cfn-modules are installed and updated with the package manager npm. The module catalog contains all available modules. Let’s start with a simple example: An EC2 instance launched into a VPC.

Install Node.js 8.x if npm is not installed on your system

Install the modules using npm:

npm i @cfn-modules/vpc
npm i @cfn-modules/ec2-instance-amazon-linux

Use the modules as nested stacks in your CloudFormation template. The vpc module comes with no required parameters. The ec2-instance-amazon-linux module comes with the required VpcModule parameter to make the connection with the vpc module. The UserData parameter is optional. Use it to install additional software like the Apache HTTP Server. Create a file named example.yml with the following content:

---
AWSTemplateFormatVersion: '2010-09-09'
Resources:
  Vpc:
    Type: 'AWS::CloudFormation::Stack'
    Properties:
      Parameters:
        S3Endpoint: 'false' # speed up the example
        DynamoDBEndpoint: 'false' # speed up the example
        FlowLog: 'false' # speed up the example
        NatGateways: 'false' # speed up the example
      TemplateURL: './node_modules/@cfn-modules/vpc/module.yml'
  Instance:
    Type: 'AWS::CloudFormation::Stack'
    Properties:
      Parameters:
        VpcModule: !GetAtt 'Vpc.Outputs.StackName' # reference the vpc module
        UserData: |
          yum install -y httpd24
          service httpd start
          echo "cfn-modules" > /var/www/html/index.html
        IngressTcpPort1: '80' # open up port 80 to the world
      TemplateURL: './node_modules/@cfn-modules/ec2-instance-amazon-linux/module.yml'
Outputs:
  Url:
    Value: !Sub 'http://${Instance.Outputs.PublicIpAddress}'

Upload the CloudFormation template and the dependencies to S3 with the aws cloudformation package command.

Install AWS CLI if aws is not installed on your system

If you use cfn-modules the first time, create an S3 bucket to store the artifacts first (otherwise, skip this step). Choose a unique bucket name, e.g. cfn-modules-$Name-$Region.

In the following command, replace $Name with a unique name (e.g. your initials or company name), and replace $Region with your AWS default region (e.g. us-east-1) to create an S3 bucket:

aws s3 mb s3://cfn-modules-$Name-$Region

Now you can upload all artifacts to S3:

aws cloudformation package --template-file example.yml --s3-bucket cfn-modules-$Name-$Region --output-template-file packaged.yml

Finally, you can create a CloudFormation stack with aws cloudformation deploy:

aws cloudformation deploy --template-file packaged.yml --stack-name ec2-example --capabilities CAPABILITY_IAM

Creating the stack will take ~10 minutes. You can find the URL to the demo page in the stack outputs:

aws cloudformation describe-stacks --stack-name ec2-example --query "Stacks[0].Outputs"

After you have finished testing delete the stack to avoid unwanted costs.

aws cloudformation delete-stack --stack-name ec2-example
aws cloudformation wait stack-delete-complete --stack-name ec2-example

Fin. Check out our examples next.

Examples

• Auto Scaling Group (singleton)
  • SSM
• EC2
  • Mount EBS volume
  • Mount EFS file system
  • Connect to MySQL
  • Connect to Postgres
  • SSH bastion
  • SSM
• Serverless
  • Cron
  • Auto IAM policies
  • Image resize
  • SQS queue

Check out all examples if you need more.

Modules

• alerting
• Auto Scaling Group (singleton)
  • asg-singleton-amazon-linux2
• client-sg
• dynamodb-table
• ebs-volume
• EC2 instance
  • ec2-instance-amazon-linux
  • ec2-instance-amazon-linux2
• efs-file-system
• kms-key
• Lambda
  • lambda-event-source-cron
  • lambda-event-source-sqs-queue
  • lambda-function
• RDS
  • rds-mysql
  • rds-postgres
• Route53
  • route53-hosted-zone-private
  • route53-hosted-zone-public
• s3-bucket
• sqs-queue
• ssh-bastion
• vpc

Check out the module catalog to browse all modules.

Feedback

I’m looking forward to your feedback! @hellomichibye or michael@widdix.de.