Review: Aurora Serverless v2

I was excited when AWS announced Aurora Serverless at re:Invent 2017. Disappointment followed shortly after. Even after Aurora Serverless became a generally available service in August 2018, it missed important features like multi-AZ deployments and read replication. Unfortunately, the innovative service never achieved a breakthrough. Therefore, I used Aurora Serverless in exceptional cases only. Four years later, AWS is making a fresh start with Aurora Serverless v2. Reason enough to take a closer look at the new service.

Before we start, let me explain why Aurora Serverless is a big deal.

Do you prefer listening to a podcast episode over reading a blog post? Here you go!

Why is Aurora Serverless a big deal?

Scaling relational databases was complicated. With Aurora, AWS provides a scalable and highly available storage layer for relational databases. No more need to provision storage upfront. The storage layer grows on-demand. Also, the storage layer replicates among availability zones.

But how to scale compute capacity of a relational database? One option is to add additional machines to the cluster, acting as read replicas. However, that works when scaling capacity for read requests only. The other option is to scale vertically by switching to a database machine with more CPU and memory. However, doing so causes a short downtime.

With Aurora Serverless, you can vertically scale a database machine. The CPU and memory capacity automatically adapts to the current workload in the background without causing downtime. That’s a game-changer because you no longer need to provision database machines based on load peaks. Think about the enormous cost savings for workloads with high load peaks and significant idle periods.

In summary, Aurora Serverless brings a relational database closer to DynamoDB’s pay-per-request pricing model.

Let’s dive into the review next.

Multi-AZ and Read Replication

When reviewing Aurora Serverless v1, I could not believe AWS offered a database service that does not span multiple availability zones. Also, the missing support for read replicas was a bummer. Luckily, AWS addressed both criticisms.

Aurora Serverless v2 comes with support for multi-AZ and read replication. You start by creating an Aurora cluster to configure and provision the storage layer. Next, you add database instances to the cluster. A database instance can either be a provisioned instance, db.t4g.medium for example, or a serverless instance. It is even possible to mix provisioned and serverless instances within a cluster. One of the instances becomes the write node, all other instances act as read nodes.

So, Aurora Serverless v2 is the breakthrough I’ve been waiting for for years.

Unfortunately, there are some aspects of the new service that I do not like. I will discuss them next.

Migrating from Aurora Serverless v1 to v2

Even though Aurora Serverless v1 did not support multi-AZ deployments, we used it in some scenarios where we could not resist because of the ability to scale the database down to 0. Although AWS brags about its customer obsession whenever possible, there is no way to upgrade from Aurora Serverless v1 to v2 without downtime and with little effort. However, AWS allows us, customers, to switch from provisioned instances to serverless instances without downtime in minutes. I’m sure it’s just a coincidence that this is also the direction that generates more revenue for AWS. More on the pricing model of Aurora Serverless v2 later.

To migrate from Aurora Serverless v1 to v2, you need to create a snapshot of v1 and launch a v2 cluster based on the snapshot. Most likely, you will need to stop write requests during this time.

Serverless without Data API!?

I prefer using DynamoDB when building a serverless application. However, many developers are familiar with relational databases, and not everyone is expecting web-scale traffic. So, there are arguments for adding a relational database to a serverless architecture. But, connecting a Lambda function with a relational database is tricky. Managing a pool of database connections from Lambda is the first challenge. The second challenge is avoiding too many database connections when scaling out the Lambda execution environments.

In summary, long-lived database connections are challenging when using Lambda. That’s why Aurora Serverless v1 introduced a Data API, which offers an HTTPS endpoint to insert, update, delete, or query data. The Data API was a natural fit for serverless applications, and I enjoyed using it for some projects.

Unfortunately, Aurora Serverless v2 does not provide a Data API. Also, I could not get any information on whether releasing the Data API feature is planned for the future.

Using an RDS Proxy solves one part of the problem: the proxy allows a Lambda to share database connections between different executions, environments, and functions. However, you still have to manage the database connections for an execution environment.

Aurora Serverless v2 is not a good fit for tiny workloads

Aurora Serverless v1 was a good fit for workloads, with little traffic and massive idle periods. For example, Michael is running a small application for accounting, which he logs into once a day for a few minutes. Or another example, one of our consulting clients runs an application that gets used once a week for about an hour. Aurora Serverless v1 was able to pause the compute layer. The next incoming request resumed the database automatically within about 60 seconds.

Unfortunately, Aurora Serverless v2 does not support pausing the compute layer. The baseline costs for Aurora Serverless v2 are about $43 per month. Therefore, it is not a good fit for tiny workloads.

Aurora Serverless v2 is too expensive!

In general, I think Aurora Serverless v2 is too expensive. Let me explain why with a chart. The following chart shows the monthly costs for the compute layer of an Aurora cluster running in us-east-1. I’m calculating with 720 hours per month and a db.r6g.large, which equals 8 Serverless ACUs. The chart shows the costs depending on the idle period in percent. Watch out where the lines of Serverless and Provisioned are crossing.

So using Aurora Serverless v2 makes sense for workloads that are idling for more than 77% of the time compared to on-demand instances. Or even worse, only for workloads idling more than 96% of the time compared to reserved instances with a three-year term and all-upfront payment.

That’s a showstopper for most scenarios, in my opinion. I’m afraid Aurora Serverless will fail not because of technical obstacles but because of the pricing model this time.

Service Maturity Table

Last but not least, I would like to evaluate the service maturity of Aurora Serverless v2.

Our maturity score for Aurora Serverless v2 is 8.5 on a scale from 0 to 10. In my opinion, Aurora Serverless v2 is ready for production workloads. However, the very high price is why I will decide against Aurora Serverless v2. Also, I did not migrate one of our workloads from Aurora Serverless v1 to v2 because of missing CloudFormation support.