AWS Velocity Series: Containerized ECS based app infrastructure
EC2 Container Service (ECS) is a highly scalable, fast, container management service that makes it easy to run, stop, and manage Docker containers on a cluster of Amazon EC2 instances. To run an application on ECS, you need the following components:
ECS cluster: EC2 instances running Docker and the ECS agent
ECS service: Managed Docker containers on the ECS cluster
ECS provides the ability to run a production-ready application on EC2 with reduced responsibilities and increased deployment speed compared to the EC2 based approach. By production-ready, I mean:
Highly available: no single point of failure
Scalable: increase or decrease the number of instances/containers based on load
Frictionless deployment: deliver new versions of your application automatically without downtime
Secure: patching operating systems and libraries frequently, follow the least privilege principle in all areas
Operations: provide tools like logging, monitoring and alerting to recognize and debug problems
The overall architecture will consist of a load balancer, forwarding requests to containers running on multiple EC2 instances, distributed among different availability zones (data centers).
Most of our clients use AWS to reduce time-to-market following an agile approach. But AWS is only one part of the solution. In this article series, I show you how we help our clients to improve velocity: the time from idea to production. Discover all posts!
Let’s start with the needed infrastructure. The ECS cluster and the service.
The ECS cluster is a fleet of EC2 instances with the ECS agent and Docker installed. The ECS cluster is responsible for scheduling the work (containers) to the EC2 instances.
Your Docker containers will run on those EC2 instances. You don’t need to care about the ECS cluster too much. We provide you a free and production-ready CloudFormation template. Please setup the ECS cluster now if you want to setup the scenario. AWS charges will likely occur! The ECS cluster need to run in a VPC, so if you don’t have a VPC stack based of our Free Templates for AWS CloudFormation (https://github.com/widdix/aws-cf-templates/tree/master/vpc) create a VPC stack first.
This first step was easy. Now you learn to use the ECS cluster with ECS services.
The ECS service is responsible for launching Docker containers in the cluster. The service also makes sure that failed containers are replaced, and it also takes care about performing rolling updates for you. You also need a load balancer to route traffic to the containers. The following ECS service template is based on free and production-ready CloudFormation template.
Description:'The email address of the admin who receives alerts.'
Alerts are triggered by a CloudWatch Alarm which can send an alert to an SNS topic. You can subscribe to this topic via an email address to receive the alerts. Let’s create an SNS topic and two alarms in the Resources section:
# A SNS topic is used to send alerts via Email to the value of the AdminEmail parameter
# This alarm is triggered, if the load balancer responds with 5XX status codes
AlarmDescription:'Load balancer responds with 5XX status codes.'
# This alarm is triggered, if the backend responds with 5XX status codes
AlarmDescription:'Load balancer target responds with 5XX status codes.'
Let’s recap what you implemented: A load balancer with a firewall rule that allows traffic on port 80. In the case of 5XX status codes, you will receive an Email. But the load balancer alone is not enough. Now it’s time to add the ECS service.
I already talked about the ECS service. It will take care of your containers. To be more precise, it will take care of your tasks that run in the ECS cluster. One task can contain one or multiple Docker containers. Three resources are needed:
A Task Definition that describes the Docker containers (similar to a Docker Compose file, or a Kubernetes Deployment)
An IAM Role for your container, so you don’t need to pass in static credentials when you want to interact with AWS from within your container. If you don’t want to make AWS API calls from your container the role is not needed.
The ECS Service itself.
To make things parameterizable, you also need to add a few parameters to the Parameters section:
DeploymentConfiguration:# This is the configuration for the rolling update
ContainerPort:3000# The image exposes the app on port 3000
Let’s recap what you implemented: A Task definition to define the containers that are managed by the service, an IAM role that is accessible inside the containers, and the ECS service that used the task definition to launch tasks (a bunch of containers) in the cluster. Logs from the containers are already shipped to CLoudWatch Logs by the awslogs log driver and are visible in the Log Group that is part of the ECS cluster template.
Two things are missing:
Scalability of containers
Alerting if containers have issues
Let’s tackle those issues step by step.
Auto scaling of containers
If you auto scale the number of containers, your ECS cluster must be able to auto scale as well. if you use our free template on GitHub, the cluster will auto scale as well
Auto Scaling works similar compared to the EC2 based approach. To scale based on the load you need to add:
Scaling Policies to define what should happen if the system should scale up/down
CloudWatch Alarms to trigger a Scaling Policy based on a metric such as CPU utilization
Additionally, you need a so-called Scalable Target. The Scalable Target can be an ECS service, but it could also be a Spot Fleet or an EMR Instance Group.
Again, you have to add those resources to the Resources section of your template:
Are you part of a highly motivated DevOps team? Use marbot, a friendly chatbot, to forward all kind of alerts from your AWS infrastructure to Slack. Alerts are escalated across your team automatically allowing you to focus on your daily work.