Deploying Self-Hosted Runners for GitHub Enterprise Server on AWS: A Guide to Efficient CI/CD

GitHub Actions is a continuous integration and continuous deployment (CI/CD) platform provided by GitHub. It allows you to automate your software development workflows by building, testing, and deploying code directly from your GitHub repository. Many companies, especially in regulated industries or dealing with sensitive data, choose GitHub Enterprise Server to host their code repositories and CI/CD pipelines on-premises.

However, it’s important to note that GitHub Enterprise Server does not come with built-in GitHub-hosted runners, so it is necessary to deploy self-hosted runners on your own infrastructure to run GitHub Actions workflows. Learn how to deploy self-hosted runners for GitHub Enterprise Server on AWS in the following.

GitHub Enterprise Server does not support GitHub-managed runners

The GitHub documentation states it clearly:

GitHub-hosted runners are not currently supported on GitHub Enterprise Server. You can see more information about planned future support on the GitHub public roadmap.

The linked issue on GitHub’s roadmap was created in July 2020. Up until now, there is no indicator that GitHub is planning to start working on the feature in the near future.

So we’re on our own. GitHub Enterprise Server supports self-hosted runners only.

Challenges of self-hosted GitHub runners

Deploying self-hosted GitHub runners for GitHub Enterprise Server presents a few key challenges:

• Security: Since self-hosted runners operate within the company’s infrastructure, extra care must be taken to secure them and ensure they cannot be misused as an entry point for malicious actors. Proper isolation, access controls, and monitoring are crucial.
• High availability: Depending on the workload, companies may need to deploy multiple self-hosted runners and implement strategies for high availability to ensure continuous service and efficient job execution.
• Scalability: As the number of concurrent jobs or workload increases, companies may need to implement auto-scaling mechanisms to dynamically provision and deprovision self-hosted runners to handle the demand efficiently.
• Cost efficiency: Provisioning self-hosted runners can lead to underutilized resources and higher costs if the workload is not consistent or predictable. Companies need to carefully plan and manage their runner infrastructure to optimize resource utilization and control costs.
• Maintenance: Self-hosted runners require regular updates and maintenance to keep them compatible with the latest GitHub Actions versions and to apply security patches or bug fixes.

Over the years, I’ve been implementing different approaches and improved the solution step by step. Here is the architecture that I currently think is the best way to deploy self-hosted runners.

• Configure GitHub webhooks to get notified when a GitHub job is waiting for a runner.
• Launch EC2 instance on-demand and register them as just-in-time runners.
• Terminate EC2 instance after the GitHub job finished.

This approach offloads the scalability challenge to AWS, as we just start EC2 instances when we need them. Also, each GitHub jobs runs on its own virtual machine, which provides a solid isolation boundary and therefore increases security.

How to deploy self-hosted GitHub runners on AWS?

Michael and I built a simple to use solution to deploy self-hosted GitHub runners on AWS: HyperEnv for GitHub Actions Runner. With it’s 2.0.0 release HyperEnv supports GitHub Free, Pro, Team, Enterprise Cloud and Enterprise Server. Here is how to deploy HyperEnv to your AWS account.

1. Go to the AWS Marketplace and subscribe to HyperEnv for GitHub Actions Runner.
2. Create a CloudFormation stack based on the provided template.
3. Install a private GitHub app to a GitHub organization.
4. Configure the GitHub workflows to run on the self-hosted runners.

For a more detailed explanation, please refer to the HyperEnv setup guide.

Summary

GitHub Actions allows you to automate workflows directly from GitHub repositories, but GitHub Enterprise Server requires self-hosted runners which present challenges around security, availability, scalability, cost, and maintenance. A solution like HyperEnv for GitHub Actions Runner can help deploy self-hosted runners on AWS by launching EC2 instances on-demand when jobs are triggered, providing isolation and auto-scaling capabilities.