Connect GitHub Actions with AWS VPC
GitHub Actions is my preferred CI/CD solution. I’m using GitHub Actions to build and deploy applications on AWS. However, GitHub Actions does not have access to private subnets, which is required in the following scenarios:
- Execute database migrations for RDS (Relational Database Service).
- Run load or integration tests against internal ALBs or NLBs (Elastic Load Balancing).
- Seed ElasticSearch, OpenSearch, or ElastiCache with data.
In the following, I will demonstrate how to access a VPC (Virtual Private Cloud) from GitHub Actions with the help of HyperEnv for GitHub Actions Runner, a solution I built recently.
By default, GitHub Actions executes jobs on machines provided by GitHub, so-called GitHub-hosted runners. Each GitHub-hosted runner comes with the runner application and other preinstalled tools and is available with Ubuntu Linux, Windows, or macOS operating systems.
A GitHub-hosted runner is connected to the Internet but cannot access any private networks, like a VPC on AWS. Therefore, reaching resources like an RDS database, an internal ALB, an ElasticSearch/OpenSearch domain, or an ElastiCache instance from within a running job is impossible.
There are two options to access private networks from GitHub Actions.
First, establish a tunnel between the GitHub-hosted runner and the private network, for example, by using VPN or SSH.
Second, run GitHub Actions inside your VPC.
How is that possible? GitHub Actions not only provides GitHub-hosted runners but also supports self-hosted runners. As illustrated in the following figure, an EC2 instance acting as a self-hosted runner launched in a VPC allows all jobs running on the machine to connect to resources in private subnets such as RDS, ElasticSearch, OpenSearch, ElastiCache, and more.
But how do you deploy a self-hosted runner on AWS? I’ve previously written about self-hosted GitHub runners on AWS. The tricky part is to come up with a scalable and cost-efficient solution.
That’s why I built HyperEnv for GitHub Actions Runner. The following figure illustrates the solution.
- GitHub sends a webhook event when starting a job.
- The API Gateway receives the event.
- A Lambda function validates the event and sends a message to SQS.
- Another Lambda function reads the message from SQS and launches an EC2 instance in an VPC of your choice.
- The EC2 instance starts and registers the GitHub runner.
- The GitHub runner executes the job.
- The EC2 instance terminates itself.
Deploy HyperEnv for GitHub Actions to your AWS account to enable GitHub Actions jobs to connect with RDS, ElasticSearch, OpenSearch, ElastiCache running in your VPC.