GitHub process automation: A bot to build pull requests authorized by core maintainers

Michael Wittig – 24 Aug 2018

We are maintaining multiple Open Source projects where we focus on CloudFormation templates. To ensure that the templates are of high quality, we leverage automated testing and pull requests. We could use the CodeBuild GitHub integration to execute yamllint, cfn-lint, and also aws cloudformation validate-template to make sure that a pull request is not breaking the templates. That’s a great way to allow contributions while ensuring high quality. You can also use Travis CI and others to run the static tests on each commit / pull request.

However, the mentioned tools only inspect the templates. They don’t create an actual CloudFormation stack. To be 100% sure that a pull request is not breaking any template, we have to create some CloudFormation stacks with the changes from the pull request. Creating a CloudFormation stack causes AWS costs (the actual AWS resources cause the costs, not CloudFormation). Usually, we spent 500 - 1000 USD per month to run automated tests on our templates.

The problem: If we create stacks automatically for every pull request, someone could open hundreds of pull requests and waste our money.

The solution: Let only core committers trigger the tests by commenting on the pull request.

You might come up with other processes based on GitHub Pull Requests that can be automated. E.g., you might want to push each change to a replication of the GitHub repository to CodeCommit. Read on if you are interested in the implementation using serverless technologies. You can find the full source code on GitHub.

Serverless architecture

GitHub bot architecture

The diagram was created with Cloudcraft - Visualize your cloud architecture like a pro.

GitHub triggers an HTTPS request for every comment made in a repository to an endpoint that you can define (aka webhook). You can create an HTTPS endpoint with API Gateway and forward the HTTPS request to a Lambda function.

The Lambda function inspects the GitHub data and kicks off a CodeBuild build.

To provide feedback to GitHub users, we watch CodeBuild with a CloudWatch Event Rule. Whenever a build status changes we trigger another Lambda function and signal the information back to GitHub.

The high-level architecture is described. Now it’s time to turn the plan into reality.

Serverless implementation

We recently released a new open source project to speed up CloudFormation projects in a modular way. I will use cfn-modules in the following implementation. Check out the details if you are interested: cfn-modules: Rapid CloudFormation. I will use cfn-modules in the following implementation.

First, you set up the project.

I assume that you have Node.js and the AWS CLI installed.

Create a new directory:

mkdir github-bot
cd github-bot/

cfn-modules relies on npm to install and update modules. Initialize the project with the following command:

npm init -y

Now it’s time to start adding the first modules.


You should always monitor your AWS resources. Let’s start with that.

Andreas and Michael Wittig

Hej, Andreas & Michael here!

We launched the cloudonaut blog in 2015. Since then, we have published 325 articles: small tips and tricks, best practices, and service reviews. We enjoy writing about all things AWS a lot.

Do you like our blog posts and podcast episodes? Have you learned something new? Consider supporting us create in-depth and independent AWS content. Please help us with a monthly or one-time payment through GitHub Sponsors.

Start supporting us today!

Run the following command to install the first cfn-module:

npm i @cfn-modules/alerting -P

Create a template.yml file with the following content:

AWSTemplateFormatVersion: '2010-09-09'
Description: 'GitHub bot'
Type: 'AWS::CloudFormation::Stack'
mail: ''
#HttpsEndpoint: '' # check out if you look for something better than email alerts!
TemplateURL: './node_modules/@cfn-modules/alerting/module.yml'

The alerting module provides an SNS topic with a topic policy that allows most AWS resources to send messages for alerting purposes.

Next, you have to define the CodeBuild project that executes the actual tests.


I use CodeBuild in an unusual way. Usually, CodeBuild downloads the source for you. In this case, I use a dummy CodeCommit repository to satisfy this requirement. But the actual code for testing is downloaded with git clone and the commit information are passed in via an environment variables (GITHUB_OWNER, GITHUB_REPO, GITHUB_SHA).

# [...]
Repository: # we only need this because the CodeBuild project cannot be created without a source.
Type: 'AWS::CodeCommit::Repository'
RepositoryName: !Sub '${AWS::StackName}-dummy'
Type: 'AWS::IAM::Role'
Version: '2012-10-17'
- Effect: Allow
Service: ''
Action: 'sts:AssumeRole'
ManagedPolicyArns: ['arn:aws:iam::aws:policy/AdministratorAccess'] # depending on your use case you can narrow down this policy
Type: 'AWS::Logs::LogGroup'
LogGroupName: !Sub '/aws/codebuild/${AWS::StackName}'
RetentionInDays: 14
DependsOn: LogGroup
Type: 'AWS::CodeBuild::Project'
Description: 'github-bot'
Image: 'aws/codebuild/nodejs:8.11.0'
Name: !Ref 'AWS::StackName'
ServiceRole: !GetAtt 'Role.Arn'
BuildSpec: |
version: 0.2
commands: []
- 'git clone "${GITHUB_OWNER}/${GITHUB_REPO}.git"'
- 'cd "${GITHUB_REPO}"'
- 'git reset --hard "${GITHUB_SHA}"'
- 'git submodule update --init --recursive'
- 'cd test/ && npm install && cd ../'
- 'cd test/ && npm test && cd ../'
Location: !GetAtt 'Repository.CloneUrlHttp'
TimeoutInMinutes: 60

Milestone completed: you can run tests for every commit on GitHub.

API Gateway

Luckily, cfn-modules provide an easy way to create an API Gateway backed by a Lambda function.

npm i @cfn-modules/lambda-event-source-webhook -P
npm i @cfn-modules/lambda-function -P

Append the following to the template.yml:

# [...]
Type: 'AWS::CloudFormation::Stack'
LambdaModule: !GetAtt 'WebhookFunction.Outputs.StackName'
AlertingModule: !GetAtt 'Alerting.Outputs.StackName'
HttpMethod: POST
TemplateURL: './node_modules/@cfn-modules/lambda-event-source-webhook/module.yml'
Type: 'AWS::CloudFormation::Stack'
AlertingModule: !GetAtt 'Alerting.Outputs.StackName'
Handler: 'webhook.handler'
Runtime: 'nodejs8.10'
EnvironmentVariable1: !Ref Project
TemplateURL: './node_modules/@cfn-modules/lambda-function/module.yml'
Type: 'AWS::IAM::Policy'
Roles: [!GetAtt 'WebhookFunction.Outputs.RoleName']
PolicyName: 'codebuild'
Version: '2012-10-17'
- Effect: Allow
Action: 'codebuild:StartBuild'
Resource: !GetAtt 'Project.Arn'

Find the source code of the Lambda function to trigger a CodeBuild build on GitHub. Store the webhook.js inside a new folder called lambda-src or check out the complete project at once.

Milestone completed: you can now receive GitHub webhooks.

CloudWatch Event Rule

To provide feedback to the contributor we use GitHub status API to update the status of the pull request. For example, a failed build will look like this in the GitHub Pull Request UI:

GitHub bot feedback

To provide feedback, you can listen to CodeBuild state changes using a CloudWatch Event Rule. Append the following to the template.yml:

# [...]
Type: 'AWS::CloudFormation::Stack'
AlertingModule: !GetAtt 'Alerting.Outputs.StackName'
Handler: 'rule.handler'
Runtime: 'nodejs8.10'
EnvironmentVariable1: !Ref Project
TemplateURL: './node_modules/@cfn-modules/lambda-function/module.yml'
Type: 'AWS::Lambda::Permission'
Action: 'lambda:InvokeFunction'
FunctionName: !GetAtt 'RuleFunction.Outputs.Name'
Principal: ''
SourceArn: !GetAtt 'Rule.Arn'
Type: 'AWS::Events::Rule'
- 'aws.codebuild'
- 'CodeBuild Build State Change'
- Arn: !GetAtt 'RuleFunction.Outputs.Arn'
Id: rule

Find the source code of the Lambda function to update GitHub status on GitHub. Store the rule.js inside a new folder called lambda-src or clone the complete project at once.

Finally, append the following to the template.yml to easily get the API Gateway endpoint URL.

Value: !GetAtt 'WebhookEventSource.Outputs.Url'

Find the full source code on GitHub including deployment information.


You used an API Gateway to receive GitHub webhooks and start CodeBuild builds. Asynchronously, in the background, you listen for build changes with a CloudWatch Event Rule and signal the build changes back to GitHub using the status API. With this pattern, you can implement many custom processes on top of GitHub to improve reliability and security of your software development and delivery capabilities.

Michael Wittig

Michael Wittig

I'm an independent consultant, technical writer, and programming founder. All these activities have to do with AWS. I'm writing this blog and all other projects together with my brother Andreas.

In 2009, we joined the same company as software developers. Three years later, we were looking for a way to deploy our software—an online banking platform—in an agile way. We got excited about the possibilities in the cloud and the DevOps movement. It’s no wonder we ended up migrating the whole infrastructure of Tullius Walden Bank to AWS. This was a first in the finance industry, at least in Germany! Since 2015, we have accelerated the cloud journeys of startups, mid-sized companies, and enterprises. We have penned books like Amazon Web Services in Action and Rapid Docker on AWS, we regularly update our blog, and we are contributing to the Open Source community. Besides running a 2-headed consultancy, we are entrepreneurs building Software-as-a-Service products.

We are available for projects.

Feedback? Questions? Drop me a line: Email, Twitter, LinkedIn.

Briefcase icon
Hire me