Monitor your AWS account to detect suspicious behavior in real time
You can track every change made to your AWS account with CloudTrail. Did you know that you can also monitor your AWS account in near real time with custom rules specific to your use case?
By combining CloudTrail, S3, SNS, and Lambda, you can run a piece of code to check the API activity in your account. Because of the reporting frequency of CloudTrail, this will happen approximately every 5 minutes. This post explains how to deploy a solution to monitor your EC2 instance tags for suspicious behavior.
The following figure shows how this works on a high level.
Let’s look at a concrete example.
CloudTrail is recording a lot of API activity. Your job is to determine which activities are suspicious. Here are a few ideas:
- A security group was changed to open a port to the outside world (0.0.0.0/0).
- An IAM user was created outside of regular business hours.
- An EC2 instance was started without following your company’s tag schema (for example, you may mark technical ownership, cost ownership, and so on).
The example that follows implements the idea of EC2 instance tag monitoring.
Special offer: cloudonaut t-shirt
Do you love our blog posts and podcast episodes? Unlock our weekly videos and online events by subscribing to cloudonaut plus.
Special offer: Join cloudonaut plus before November 30th, and we will send you a cloudonaut t-shirt for free.Subscribe now!
Each time CloudTrail has new data for you, a Lambda function is triggered. The Lambda function needs to do the following:
- Understand the input data generated from SNS.
- Download the compressed CloudTrail files from an S3 bucket.
- Uncompress the files.
- Iterate through the API activities, looking for EC2 tag-related events:
- Alert violations of the tag schema.
Fortunately, the code is implemented already so that we won’t dive into Node.js code this time. Instead, we’ll focus on deploying this solution.
Deploying Lambda is possible almost entirely with CloudFormation. A few steps are required to prepare everything you need:
- Choose an AWS region you want to monitor (referenced as
$regionin the following).
- Create an SNS topic in
$region, and subscribe to the topic via email. The system will send alerts to this endpoint.
- Download the code by running
wget https://github.com/widdix/aws-tag-watch/archive/master.zipin your terminal.
unzip master.zipin your terminal.
- Change dir by running
npm installin your terminal to install Node.js dependencies.
config.json, and set
alertTopicArnto the ARN of your SNS topic from step 1.
./bundle.shin your console.
aws-tag-watch.zipto S3 (the bucket must be in
- Create a CloudFormation stack based on
Now your AWS account in
$region is monitored. Whenever you run a new EC2 instance or change the tags of an existing EC2 instance, the Lambda function will check whether you’re sticking to the tag schema.
Raising an alert via email isn’t that helpful if you are working on a team. You may want to look at OpsGenie, which integrates nicely with SNS.
This blog post has been translated into German: Beobachte deinen AWS Account in echtzeit um verdächtige Aktivitäten aufzuspüren.