Builder's Diary Vol. 4: Serverless Software Engineering

Get insights into the day-to-day challenges of builders. In this issue, Florian Dröge and Lars Hüper from our partner tecRacer share insights into crafting Serverless applications that last.

If you prefer a video or podcast instead of reading, here you go.

Do you prefer listening to a podcast episode over reading a blog post? Here you go!

Find a written excerpt of the interview with Florian Dröge and Lars Hüper in the following.

cloudonaut: What does a typical Serverless project at tecRacer look like?

Lars Hüper: Our team focuses on Serverless projects. Our projects are diverse. For example, we modernize applications by replacing an MS SQL database with a cloud-native database like Aurora Serverless. We also work on large projects where we build complex Serverless applications. An example we will surely mention more often is the new development of software for pension schemes. In this project, we build up an internal development team on behalf of the customer for years. We assume the role of architects, developers, and coaches in this project. The pension scheme application we are building has a time horizon of over 20 years.

cloudonaut: That’s amazing; the fact that you have accompanied projects for many years is outstanding. Most Serverless projects we have worked on were proofs-of-concept. What’s your favorite technology stack for building Serverless applications?

Florian Dröge: We prefer writing TypeScript. This applies to Infrastructure as Code with the AWS CDK, as well as to the Lambda-based backend and also the frontend. Doing so allows developers to quickly make changes to all parts of the stack, which increases agility a lot. Despite this, most developers focus on specific areas to deepen their expertise.

On top of that, we use the common Serverless building blocks:

• AWS Lambda
• Amazon API Gateway and AWS AppSync
• Amazon SNS and EventBridge
• Amazon DynamoDB and Aurora Serverless
• Amazon S3

cloudonaut: What are the challenges of developing complex systems using serverless technologies? And how do you overcome them?

Lars Hüper: A significant advantage of the Serverless approach is that it’s straightforward for a developer to get up and running and ship code right away. However, I would like to issue a warning. It is essential to understand the problem you are trying to solve before writing code and assembling building blocks.

Therefore, we ask questions like “How does the process work right now?” and “Why do you want to digitize the process?”. On top of that, we focus on understanding the business domain. For example, we’ve learned much about pension schemes in the past few years.

In other words, we are following the domain-driven design (DDD) approach:

1. We start with event storming, a workshop to learn about the domain quickly.
2. We design the system with input from domain experts.
3. We start implementing and shipping software.

We realized that it is crucial to go back to the first step from time to time. Therefore, we conduct event-storming workshops from time to time.

cloudonaut: I am impressed with how you craftmanship your software. How do you ensure you can replace services or the whole cloud provider over time?

Florian Dröge: As I said, we are planning a life cycle of more than 20 years for the pension scheme application. As a result, we expect to replace some of the services we use today over time.

We strive for Clean Architecture as Robert C. Martin (Uncle Bob) described. An architecture following this approach is represented by four circles.

1. Entities the business rules and objects.
2. Use Cases the application-specific business rules.
3. Interface Adapters the converts that transform the data into different formats.
4. Frameworks and Drivers the database, the frontend framework, …

According to the dependency rule of Clean Architecture, dependencies may only point from the outside to the inside.

So, our core business logic does not even know that it runs on Serverless or that we use DynamoDB to persist data. This pattern allows us to replace services when needed. For example, we replaced DynamoDB with Aurora Serverless in a specific scenario. We are not reinventing the wheel but are applying software engineering principles to Serverless development. And that’s what we are missing in the documentation and examples provided by AWS.

_cloudonaut: That’s great! You mentioned that you replaced DynamoDB with Aurora Serverless as the access pattern changed during the project. Please share more insights into how Serverless architectures evolve.

Florian Dröge: The first thing that comes to my mind is that at the beginning of a project, developers create a lot of DynamoDB tables to persist data. As the understanding of the access pattern improves over time, we aim for the single-table or other NoSQL patterns. On top of that, as our understanding of the data queries grows, we rethink whether we are using the best tool for the job. So we consider using specialized databases like Amazon Neptune, a graph database, for example.

Lars Hüper: Let me add another example. We’ve observed that SNS was used to implement component-to-component communication in a “remote procedure call”-like pattern. Therefore, we revisited that area with a focus on following the event-driven design. Don’t get me wrong, there are occasions where RPC-like calls between Lambda functions are fine, but only in a very limited scope. Outside the scope, the event-driven design allows for loose coupling of components.

Florian Dröge: One more thing, we noticed that we needed to hand over some parameters like the name of a global S3 bucket to many Lambda functions. Passing those parameters through our whole infrastructure code was cumbersome. Therefore, we added AWS AppConfing to the mix. We use AppConfig to manage global parameters that all or most of our Lambda functions require. For local parameters, we still use environment variables.

Lars Hüper: We also encountered difficulties deploying our CDK code. We realized that splitting your infrastructure into many small stacks caused troubles due to missing dependency declarations. Therefore, we are now using larger stacks and bundling them together using nested stacks, which simplifies dependency management for us.

cloudonaut: Thanks a lot, Florian and Lars, for sharing your insights into developing Serverless applications following domain-driven design and clean architecture. I learned a lot from both of you.

Open Position: Cloud Consultant AWS Serverless Development
Would you like to join Florian and Lars's team to engineer advanced Serverless applications? tecRacer is hiring a Cloud Consultant focusing on AWS Serverless. Apply now!