🎉 We are launching a new weekly show: Hot off the Cloud

🎉 We are launching a new weekly show

Private subnets are broken on AWS

Andreas WittigUpdated 23 Aug 2016

tl;dr
Think twice if you’re planning to use a VPC architecture containing a private subnet that includes EC2 instances that need to connect to the public Internet, especially if you have to do many requests or transfer a lot of data.

Being able to build a private network in the cloud with Virtual Private Cloud (VPC) is a key feature of AWS. But private subnets are broken on AWS.

A typical VPC setup consists of public and private subnets, as shown in the following figure. An EC2 instance in the private subnet running an application can’t connect to the Internet by default. But often there is a need to do that for the following reasons:

  • The EC2 instance needs to call the AWS API (DynamoDB, Route 53, and so on).
  • The EC2 instance needs to download packages from a public repository (yum, apt, and so on) to be provisioned or to install updates.

NAT Gateway

Traffic to 0.0.0.0/0 from the private subnet is routed to a single EC2 instance: the NAT instance. You can configure this with the help of the route table attached to the private subnet.


Looking for a new challenge?

  • tecRacer

    Cloud Consultant • AWS Migrations

    tecRacer • Premier AWS Consulting Partner • Germany, Austria, Portugal, and Switzerland
    Assessment Transformation Change Management
  • DEMICON

    Senior Lead Full Stack Developer

    DEMICON • AWS Advanced Consulting Partner • Remote
    AWS JavaScript/TypeScript Angular React

Unfortunately, all traffic from the private subnet to the Internet has to pass a single EC2 instance. This is a problem because

  • The NAT instance becomes a single point of failure.
  • The NAT instance can only be scaled vertically.

There are best practices for building an HA setup for your NAT instance. For example, the article High Availability for Amazon VPC NAT Instances: An Example. The limit for scaling vertically is sufficient for most use cases. But is using a private subnet to prevent access from the Internet to your app server worth all the trouble?

Probably not. If you want an app server that can’t be reached from the Internet but that can connect to other services over the Internet, you can use the following setup (illustrated in the next figure):

  • App server running with a public IP address
  • App server running in a public subnet with a route to the Internet gateway
  • App server running in a security group that prohibits all incoming traffic
  • ACL attached to the subnet allowing only incoming high ports from the public Internet

Public Subnet

If you need to, you can create an additional public subnet that allows connections from the public Internet (for example, to run an SSH bastion host).

Become a cloudonaut supporter

Andreas Wittig

Andreas Wittig ( Email, Twitter, or LinkedIn )

We launched the cloudonaut blog in 2015. Since then, we have published 360 articles, 49 podcast episodes, and 48 videos. It's all free and means a lot of work in our spare time. We enjoy sharing our AWS knowledge with you.

Please support us

Have you learned something new by reading, listening, or watching our content? With your help, we can spend enough time to keep publishing great content in the future. Learn more

$
Amount must be a multriply of 5. E.g, 5, 10, 15.

Thanks to Alan Leech, Alex DeBrie, ANTHONY RAITI, Christopher Hipwell, Jaap-Jan Frans, Jason Yorty, Jeff Finley, Jens Gehring, jhoadley, Johannes Grumböck, Johannes Konings, John Culkin, Jonas Mellquist, Juraj Martinka, Kamil Oboril, Ken Snyder, Markus Ellers, Ross Mohan, Ross Mohan, sam onaga, Satyendra Sharma, Shawn Tolidano, Simon Devlin, Thorsten Hoeger, Todd Valentine, Victor Grenu, and all anonymous supporters for your help! We also want to thank all supporters who purchased a cloudonaut t-shirt.