AWS Security Monitoring in 2023: Untangle the chaos

AWS security monitoring is a set of practices, tools, and processes designed to detect and respond to security threats and vulnerabilities within the Amazon Web Services (AWS) cloud environment. Sounds easy? In this blog post, I share how I use a variety (but not all) of AWS services to detect issues quickly and alert someone in charge to solve them.

Available AWS services

AWS is confusing. Multiple services are competing for your attention when you try to understand the basics of security monitoring. Let me untangle the chaos by grouping the related AWS services/capabilities into three groups (visually and textually). After that, I will present you my selection of AWS services.

I came up with the following three groups:

1. Sources of information: Provide raw security events or data that we can analyze. Usually, the volume is very high and overwhelming. Low-level findings also fall in this category.
2. Best practices and anomaly detection: Create higher-level findings by analyzing raw security events and data using predefined best practices or machine learning to detect unusual patterns.
3. Aggregation: Correlate and aggregate findings with all kinds of information and present them in a human-friendly way.

Some AWS services belong to multiple groups. E.g., AWS Security Hub checks for best practices and aggregates events in a human-friendly way.

The following sources of information can be used to monitor security-related activity in your AWS Account:

• AWS API (current configuration of each resource can be requested)
• AWS Config configuration item
• AWS CloudTrail management and data event
• Amazon VPC Flow Logs
• Amazon Route 53 DNS query logs
• AWS Health event
• Amazon Inspector vulnerability finding
• Amazon Macie sensitive data finding
• Amazon ECR Basic scanning finding
• AWS IoT Device Defender Detect finding

The following AWS services/capabilities check the above sources against best practices or to detect anomalies. The following table lists the services and the sources that they use.

Last but not least, AWS aggregates all the information in different services. The following table lists the services and the sources that they use.

Selecting the right AWS services

Many AWS services/capabilities overlap in what information they analyze and the checks they perform. Examples:

• AWS Trusted Advisor, AWS Config Rules / Conformance Packs, and AWS Security Hub Security standards all check if you follow predefined best practices in your AWS account.
• Amazon Detective, AWS CloudTrail Insight, and Amazon GuardDuty analyze your CloudTrail data to find suspicious activity.

If you mindlessly enable all the services, you pay for the same check multiple times. But the issue worsens: Each service creates a finding, but all refer to the same problem. For example, if you make AWS Trusted Advisor, AWS Config Rules / Conformance Packs, AWS Security Hub Security standards, and Amazon Macie create an S3 bucket public, a firework of findings. They all tell you that the bucket is now public. But how do you manage this information overload? Amazon invented another service to aggregate this information again: Amazon Detective.

I suggest that you enable the following AWS services/capabilities in your a delegated admin AWS Account (aka security account, a feature of AWS organizations) to check all your AWS accounts in one place:

• AWS Config with a retention period of 1 year in each region you use.
• AWS Security Hub with the AWS Foundational Security Best Practices (FSBP) standard enabled in each region you use (AWS Security Hub requires AWS Config).
• Optional: Amazon GuardDuty in each region you use.
• Optional: Amazon Inspector in each region you use.

By default, GuardDuty and Inspector send findings to Security Hub. Therefore, Security Hub is your central place to work with findings. A finding can be NEW, acknowledged (NOTIFIED), SUPPRESSED, or RESOLVED. To end goal is to keep the number of findings low.

You can disable AWS Security Hub controls if you disagree with the “best practice.”

Last, we must alert someone in charge to handle the finding.

Incident Response

A finding is the beginning, not the end, of the security incident response process. Once a finding is created, the process starts:

1. Alert the right person.
2. Analyse finding (research additional data).
3. Fix issue.
4. Resolve finding.

To alert the right person, you have to understand how Security Hub publishes information about findings. Security Hub publishes an event to EventBridge whenever a finding is created or updated. If you followed my advice to use a delegated admin AWS Account, the event is published in the source (aka member) and delegated admin accounts. I recommend creating an EventBridge rule to listen to new findings in each AWS member account (not the delegated admin). Assuming that you use AWS accounts to isolate workloads, there should be a relationship between the AWS account and a team in charge that can be alerted when a new finding arrives. Remember to create an EventBridge rule in each region you use.

The following Terraform snippet creates an EventBridge rule connected to an SNS topic:

resource "aws_cloudwatch_event_rule" "security_hub_finding" {
  name          = "security-hub-finding"
  description   = "Findings (severity >= high) from AWS SecurityHub."
  event_pattern = <<JSON
{
  "source": [ 
    "aws.securityhub"
  ],
  "detail-type": [
    "Security Hub Findings - Imported"
  ],
  "detail": {
    "findings": {
      "Severity": {
        "Normalized": [{"numeric": [">=", 70]}]
      },
      "Workflow": {
        "Status": [
          "NEW"
        ]
      },
      "RecordState": [
        "ACTIVE"
      ]
    }
  }
}
JSON
}

resource "aws_cloudwatch_event_target" "security_hub_finding" {
  rule      = aws_cloudwatch_event_rule.security_hub_finding.name
  target_id = "sns"
  arn       = "YOUR SNS TOPIC ARN"
}

There is one issue with this approach. As long as the status is NEW, you will receive an EventBridge event per finding every day. If this feels too spammy, write a Lambda function to set the status to NOTIFIED. Alternatively, you can use marbot, our AWS Monitoring chatbot. marbot creates the EventBridge rule for you and sets the status to NOTIFIED after you receive an alert in Slack or Microsoft Teams about the new finding.

Summary

Many AWS services and capabilities are relevant when talking about AWS Security Monitoring. They overlap in what information they analyze and the checks they perform. If you carefully select the right AWS services, you can avoid duplicate findings and costs while observing all the security-relevant sources. Use AWS Security Hub as your central place for AWS Security Monitoring. Optionally, use GuardDuty and Inspector to feed additional insights into Security Hub. Last, use EventBridge to forward Security Hub findings to the right team.