Fargate is ready for prime time, and we share our CloudFormation templates

The recent AWS Fargate Price Reduction (up to 50%) is the last piece in the puzzle to call Fargate a reasonable choice for running Docker workloads on AWS.

The CIO perspective is as simple as this: you provide the Docker image and scaling rules, Fargate deploys and runs your Docker containers for you.

However, the reality is more complicated. AWS is a provider of building blocks. It’s up to you to compose them to systems that survive reality. It turns out that composing the building blocks is not as easy. Many pitfalls are waiting for you such as invalid parameter combinations and you need deep understanding of the components. We worked on a couple of consulting projects where we used Fargate to run the workloads of our clients. We codified our experience into CloudFormation templates that we want to share with you today.

We maintain two open source projects where we share our CloudFormation templates. Free Templates for AWS CloudFormation are composable and very opinionated but easy to use and CloudFormation beginner friendly. cfn-modules are highly modularized and flexible but require more CloudFormation experience.

In this post, I demonstrate to deploy the same application using both CloudFormation open source projects. The application consists of two Docker images:

• Varnish cache: Docker image eeacms/varnish:4.1-6.2
• Nginx with a simple “Hello World” app: Docker image widdix/hello:v1

We make use of the ambassador pattern where traffic arrives on one container (ambassador container) that forwards to the other container (app container). The following diagram illustrates the high-level architecture.

Let’s see how you can deploy this.

Free Templates for AWS CloudFormation

Free Templates for AWS CloudFormation are composable. Common aspects are split into templates to allow sharing of resources such as a VPC.

VPC

Let’s start by creating the VPC (do not modify the parameters):

Wait until the status changes to CREATE_COMPLETE.

ESC Cluster for Fargate

Next, you create an ESC Cluster for Fargate.

The figure shows how the previously created vpc stack is passed in as the ParentVPCStack parameter (do not modify the parameters, acknowledge that AWS CloudFormation might create IAM resources):

Wait until the status changes to CREATE_COMPLETE.

Expand the Outputs section and open the URL output value in your browser. You receive a 503 error for now.

Fargate service

Finally, you create the Fargate service that runs the Docker images.

The figure shows how the Docker images are configured using parameters (do not modify the parameters):

Wait until the status changes to CREATE_COMPLETE and reload the URL in your web browser. Rember that you can find the URL in the URL output value. Now it works.

Don’t forget to delete all three stack in reverse order once you are done with the demo.

Let’s see how you can deploy this with cfn-modules.

cfn-modules

cfn-modules are installed and updated with the package manager npm.

Install Node.js 8.x if npm is not installed on your system

Create a fresh demo directory and change into the newly created directory.

mkdir cfn-modules-fargate-alb-ambassador-pattern-example
cd cfn-modules-fargate-alb-ambassador-pattern-example/

Install the modules using npm:

npm i @cfn-modules/alerting@1.0.0
npm i @cfn-modules/vpc@1.1.0
npm i @cfn-modules/ecs-cluster@1.0.0
npm i @cfn-modules/alb@1.0.3
npm i @cfn-modules/alb-listener@1.0.0
npm i @cfn-modules/ecs-alb-target@1.0.2
npm i @cfn-modules/fargate-service@1.1.1

Create a new file to store your CloudFormation template:

touch example.yml

You use the modules as nested stacks in your CloudFormation template. Let’s look at the first three modules that provide some basic functionality:

• alerting: Central SNS topic that receives alerts from other modules and forwards them to your team via email, HTTP, or HTTPS.
• vpc: AWS VPC using three availability zones with public and private subnets, VPC endpoints for DynamoDB and S3, Flow Logs, and NAT gateways.
• ecs-cluster: ECS cluster.

See how we modules link to each other using parameters (e.g., AlertingModule) and the CloudFormation stack names (e.g., !GetAtt 'Alerting.Outputs.StackName').

AWSTemplateFormatVersion: '2010-09-09'
Description: 'cfn-modules: Fargate ALB with ambassador pattern example'
Resources:
  Alerting:
    Type: 'AWS::CloudFormation::Stack'
    Properties:
      Parameters:
        Email: 'email@domain.com' # replace with your email address to receive alerts
        # HttpsEndpoint: 'https://api.marbot.io/v1/endpoint/xyz' # or uncommnet and receive alerts in Slack using marbot.io
      TemplateURL: './node_modules/@cfn-modules/alerting/module.yml'
  Vpc:
    Type: 'AWS::CloudFormation::Stack'
    Properties:
      Parameters:
        AlertingModule: !GetAtt 'Alerting.Outputs.StackName' # link to alerting module
      TemplateURL: './node_modules/@cfn-modules/vpc/module.yml'
  Cluster:
    Type: 'AWS::CloudFormation::Stack'
    Properties:
      TemplateURL: './node_modules/@cfn-modules/ecs-cluster/module.yml'

Next, you will create the ALB setup with the following modules:

• alb: Application load balancer shell.
• alb-listener: ALB listener to listen on port 80 (we use HTTP, but HTTPS is also supported if you pass a CertificateArn).
• ecs-alb-target: ECS ALB target that connects the ALB with the Fargate service.

Resources:
  [...]
  Alb:
    Type: 'AWS::CloudFormation::Stack'
    Properties:
      Parameters:
        VpcModule: !GetAtt 'Vpc.Outputs.StackName' # link to vpc module
        AlertingModule: !GetAtt 'Alerting.Outputs.StackName' # link to alerting module
      TemplateURL: './node_modules/@cfn-modules/alb/module.yml'
  AlbListener:
    Type: 'AWS::CloudFormation::Stack'
    Properties:
      Parameters:
        AlbModule: !GetAtt 'Alb.Outputs.StackName' # link to alb module
      TemplateURL: './node_modules/@cfn-modules/alb-listener/module.yml'
  Target:
    Type: 'AWS::CloudFormation::Stack'
    Properties:
      Parameters:
        AlbModule: !GetAtt 'Alb.Outputs.StackName'
        AlbListenerModule: !GetAtt 'AlbListener.Outputs.StackName'
        VpcModule: !GetAtt 'Vpc.Outputs.StackName'
        AlertingModule: !GetAtt 'Alerting.Outputs.StackName'
      TemplateURL: './node_modules/@cfn-modules/ecs-alb-target/module.yml'

Any finally, you define the Docker images in the fargate-service module.

Resources:
  [...]
  Service:
    Type: 'AWS::CloudFormation::Stack'
    Properties:
      Parameters:
        ClusterModule: !GetAtt 'Cluster.Outputs.StackName'
        VpcModule: !GetAtt 'Vpc.Outputs.StackName'
        TargetModule: !GetAtt 'Target.Outputs.StackName'
        AlertingModule: !GetAtt 'Alerting.Outputs.StackName'
        AmbassadorImage: 'eeacms/varnish:4.1-6.2'
        AmbassadorPort: '6081'
        AmbassadorEnvironment1Key: 'BACKENDS'
        AmbassadorEnvironment1Value: 'localhost'
        AmbassadorEnvironment2Key: 'BACKENDS_PORT'
        AmbassadorEnvironment2Value: '80'
        AppImage: 'widdix/hello:v1'
        AppPort: '80'
      TemplateURL: './node_modules/@cfn-modules/fargate-service/module.yml'
Outputs:
  Url:
    Value: !Sub 'http://${Alb.Outputs.DnsName}'

If you merge all the code snipets together in example.yml, you can deploy the template. You can also download the example on GitHub.

Install AWS CLI if aws is not installed on your system

If you use cfn-modules the first time, create an S3 bucket to store the artifacts first (otherwise, skip this step). Choose a unique bucket name, e.g. cfn-modules-$Name-$Region.

In the following command, replace $Name with a unique name (e.g. your initials or company name), and replace $Region with your AWS default region (e.g. us-east-1) to create an S3 bucket:

aws s3 mb s3://cfn-modules-$Name-$Region

Now you can upload all artifacts to S3:

aws cloudformation package --template-file example.yml --s3-bucket cfn-modules-$Name-$Region --output-template-file packaged.yml

Finally, you can create a CloudFormation stack with aws cloudformation deploy:

aws cloudformation deploy --template-file packaged.yml --stack-name fargate-alb-ambassador-pattern-example --capabilities CAPABILITY_IAM

Creating the stack will take about 15 minutes. You can find the URL to the “hello world” page in the stack outputs:

aws cloudformation describe-stacks --stack-name fargate-alb-ambassador-pattern-example --query "Stacks[0].Outputs[?OutputKey=='Url'].OutputValue" --output text

Open the URL in your web browser.

Don’t forget to delete the stack once you are done with the demo:

aws cloudformation delete-stack --stack-name fargate-alb-ambassador-pattern-example

Check out the modules to learn more about additional parameters that are available.

• alerting
• vpc
• ecs-cluster
• alb
• alb-listener
• ecs-alb-target

Summary and Fargate limitations

Fargate is the easiest way to run Docker workloads on AWS. We recommend using CloudFormation to put the AWS building blocks together. We maintain two open source projects with production-ready templates for you to use:

• Free Templates for AWS CloudFormation are composable and very opinionated but easy to use and CloudFormation beginner friendly.
• cfn-modules are highly modularized and flexible but require more CloudFormation experience.

Keep in mind the following limitations of Fargate:

• No support for EFS file systems
• No support for EBS volumes
• No support for encryption of data at rest