Security Iceberg: AWS Security Hub the right way

This is a warning about AWS Security Hub. Organizations that use AWS Security Hub to monitor and mitigate risks pay too much attention to the visible part of the AWS security iceberg, namely the findings. These organizations tend to overlook or underestimate the invisible part of the iceberg where critical risks are hidden.

Below, I go into the details of my observations and outline possible countermeasures to ensure you are using AWS Security Hub in the right way.

Overwhelming amount of findings

The AWS Security Hub comes with controls for the following standards:

• CIS AWS Foundations: 48 controls
• Payment Card Industry Data Security Standard (PCI DSS): 49 controls
• AWS Foundational Security Best Practices: 162 controls

It is not uncommon that you will end up with hundreds or even thousands of findings after enabling one or multiple standards for multiple AWS accounts with running workloads. Working through a huge list of findings to evaluate, suppress, or fix the issue is demotivating and uses up a lot of energy and attention.

I’ve seen teams spend weeks or months fixing all the findings generated by AWS Security Hub. Don’t get me wrong, fixing the findings will improve overall security. My point is that you may be overlooking important areas where security can be significantly enhanced by spending the same time and money.

A few ideas to avoid getting overwhelmed by the number of findings generated by the AWS Security Hub.

1. Do not enable all security standards at once. For example, start with the CIS AWS Foundations standard.
2. Enable and monitor the AWS Security Hub at the beginning of a project before creating resources in the AWS account.
3. Disable irrelevant or outdated controls. Also, disable controls generating a lot of minor findings.

Superficial security controls

Before the AWS Security Hub was a thing, I worked on a similar tool many years ago. The challenge when designing and implementing controls is abstraction and generalization. To evaluate the risk of a configuration, context is needed.

Let me illustrate this with an example. A Lambda function assumes an IAM role with the following policy attached.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:CreateBucket",
        "s3:DeleteBucket",
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}

A generic control cannot tell whether the IAM policy follows the least privilege principle because it does not know the Lambda function. The IAM policy could either be fine or add an unnecessary risk of accidental or malicious data deletion.

That’s why the controls focus on generic risks that are easy to detect. So it is crucial that you ask the question: Which parts of the iceberg are not covered by AWS Security Hub? Let me give you a few examples:

• Do resource- and identity-based policies follow the least privilege principle.
• Are the security group rules as strict as possible?
• Is the ALB publicly accessible by intention?
• Should objects be protected from being manipulated or deleted by enabling S3 Object Lock?
• What’s the data classification of the data stored in a database cluster? Is encryption with a customer-managed KMS key required?
• Does that Lambda function (Node.js) contain a package with a critical vulnerability?

Be aware that the AWS Security Hub does not check all relevant aspects of cloud security. Also, it is impossible to evaluate rules based on context information like the data classification, for example. That does not mean the service is terrible; it is just not possible to go into the details with generic controls.

Some might argue, that other AWS services or 3rd parties could jump in and provide advanced controls. For example, the IAM Access Analyzer compares CloudTrail logs with IAM polices to make suggestions on how to improve the policy. However, the IAM Access Analyzer comes with the same limitation, it does not know about your scenario. Instead, it makes guesses based on incomplete/expensive CloudTrail logs. In my opinion, all those tools try to solve a problem, that cannot be solved without any context information about the scenario.

Some ideas on avoiding blind spots when using the AWS Security Hub.

1. Be aware that the AWS Security Hub does not cover all relevant aspects of cloud security.
2. Ask AWS security specialists to review AWS accounts regularly.
3. Validate whether IAM and network configuration follows the least privilege principle.

False-positives

We use the AWS Security Hub to evaluate the security risks within our AWS accounts. Every day, marbot -our chatbot- forwards a Security Hub finding to one of our Slacks channels. The finding tells us that there is an EC2 instance with a public IP address running.

Running an EC2 instance with a public IP address should raise a red flag. However, we have a good reason to start an EC2 instance with public IP once a day for 5 minutes. Thus, the finding is a false positive. The problem with false positives is that we get used to them. I close the warning every day, and it has happened to me that I have overlooked an important warning.

So, watch out for false positives and mark them as SUPPRESSED, which means you reviewed the finding and do not think any action is needed. In our case, a new finding gets generated every day because we are launching a new instance. That’s why automatically suppressing the finding is on our TODO list.

A few ideas to ensure you do not get bothered by false positives and avoid alert fatigue.

1. Mark false-postives as SUPPRESSED (see Setting the workflow status for findings) manually.
2. Use a Lambda function to mark false positives as SUPPRESSED in case new resources result in findings regularly.

Summary

Think of AWS security as an iceberg. The easy-to-find risks look out of the water, and AWS Security Hub will easily detect them. But under the water, the iceberg continues; critical vulnerabilities are hiding there. Therefore, here are a few tips for dealing with the AWS Security Hub.

1. Avoid generating an overwhelming amount of findings. For example, do not enable all standards and controls at once.
2. Be aware that the AWS Security Hub does not cover all relevant aspects of cloud security. The generic controls do not guarantee that you are following the least-privilege principle, for example.
3. Reduce the noise generated by false positives to avoid alert fatigue. Mark findings as SUPPRESSED manually or automatically.