Managing application secrets: SSM Parameter Store vs. Secrets Manager

Many applications interact with external or internal systems like databases or REST APIs. When your application talks to another system, it usually authenticates with a secret, e.g., an API key, username + password, or a certificate. This leads to the question: How can we safely make the secret available to our application? In this blog post, I compare two options provided by AWS: Parameter Store and Secrets Manager.

When our application starts, the needed secrets are fetched at runtime from a secure system. But we need to authenticate with the secure system before we can get the secrets. Does this lead to a chicken and egg problem? Not if you use an AWS service where you can authenticate with AWS credentials. Luckily, AWS provides short-lived credentials to our application via the Metadata service (e.g., EC2 & Fargate) or environment variables (e.g., Lambda). With the short-lived AWS credentials, we can reach out to fetch the secrets.

AWS provides two services to store and retrieve secrets: AWS Secrets Manager and AWS SSM Parameter Store. I compare both services in the following.

Secrets Manager

AWS Secrets Manager is purpose build to store and retrieve secrets. We can restrict access to secrets with identity-based IAM policies (attached to IAM users and roles) and resource-based IAM policies attached to the secret directly. This allows you to share secrets with other AWS accounts as well.

A great feature that is poorly implemented is secret rotation. Once you define a secret, you also have to develop a process to rotate the secret from time to time (AWS recommends rotating secrets every 30 days). Rotation can be done in different ways. My preferred approach is this:

1. Create a new secret (e.g., add database user) and store the value
2. Update the latest label to point to the new secret version
3. Wait for all applications to retrieve the new secret version (e.g., retrieve the secret every 12 hours)
4. Delete the old secret (e.g., remove database user)

Unfortunately, not all systems support this kind of rotation. Sometimes we can only generate a single API key. If you still want to rotate the secret, you will risk a short downtime. AWS provides a set of Lambda function templates to rotate secrets for RDS, Redshift, and DocumntDB. But keep in mind that the rotation logic runs in your AWS account, so you are responsible if things go wrong.

Another handy feature of Secrets Manager are multi-region secrets to replicate secrets into multiple regions.

Finally, I want to mention AWS provided caching libs to lower the costs and improve performance.

Parameter Store

Parameter Store is part of AWS Systems Manager. Parameters can be secrets or plain-text values. By default, Parameter Store does not support resource-based IAM policies. By using a KMS customer-managed CMK for encryption of the secret, you can add a resource-based IAM policy. The resource-based IAM policy of the KMS key controls access to the key and, therefore, to the parameter value.

A useful feature of Parameter Store are parameter policies to:

• Delete a parameter at a specific date
• Send a notification if a parameter expires
• Send a notification if a parameter was not changed for n days

The notifications are delivered via Amazon EventBridge as well as a notification if a parameter changes. I used the parameter change notification in the past to trigger pipelines if a parameter value changed.

Summary

The following table shows the differences between Secrets Manager and Parameter Store.