👉 AWS Debug Games (Beta) - Prove your AWS expertise by solving tricky challenges.

👉 AWS Debug Games - Prove your AWS expertise.

Private subnets are broken on AWS

Andreas WittigUpdated 23 Aug 2016

tl;dr
Think twice if you’re planning to use a VPC architecture containing a private subnet that includes EC2 instances that need to connect to the public Internet, especially if you have to do many requests or transfer a lot of data.

Being able to build a private network in the cloud with Virtual Private Cloud (VPC) is a key feature of AWS. But private subnets are broken on AWS.

A typical VPC setup consists of public and private subnets, as shown in the following figure. An EC2 instance in the private subnet running an application can’t connect to the Internet by default. But often there is a need to do that for the following reasons:

  • The EC2 instance needs to call the AWS API (DynamoDB, Route 53, and so on).
  • The EC2 instance needs to download packages from a public repository (yum, apt, and so on) to be provisioned or to install updates.

NAT Gateway

Traffic to 0.0.0.0/0 from the private subnet is routed to a single EC2 instance: the NAT instance. You can configure this with the help of the route table attached to the private subnet.


Looking for a new challenge?

  • DEMICON

    Cloud Operations Lead

    DEMICON • AWS Advanced Consulting Partner • Remote (Europe)
    service-delivery-management hiring devops platform

Unfortunately, all traffic from the private subnet to the Internet has to pass a single EC2 instance. This is a problem because

  • The NAT instance becomes a single point of failure.
  • The NAT instance can only be scaled vertically.

There are best practices for building an HA setup for your NAT instance. For example, the article High Availability for Amazon VPC NAT Instances: An Example. The limit for scaling vertically is sufficient for most use cases. But is using a private subnet to prevent access from the Internet to your app server worth all the trouble?

Probably not. If you want an app server that can’t be reached from the Internet but that can connect to other services over the Internet, you can use the following setup (illustrated in the next figure):

  • App server running with a public IP address
  • App server running in a public subnet with a route to the Internet gateway
  • App server running in a security group that prohibits all incoming traffic
  • ACL attached to the subnet allowing only incoming high ports from the public Internet

Public Subnet

If you need to, you can create an additional public subnet that allows connections from the public Internet (for example, to run an SSH bastion host).

Become a cloudonaut supporter

Andreas Wittig

Andreas Wittig ( Email Twitter LinkedIn Mastodon )

We launched the cloudonaut blog in 2015. Since then, we have published 365 articles, 68 podcast episodes, and 68 videos. It's all free and means a lot of work in our spare time. We enjoy sharing our AWS knowledge with you.

Please support us

Have you learned something new by reading, listening, or watching our content? With your help, we can spend enough time to keep publishing great content in the future. Learn more

$
Amount must be a multriply of 5. E.g, 5, 10, 15.

Thanks to Alan Leech, Alex DeBrie, Christopher Hipwell, Jason Yorty, Jeff Finley, jhoadley, Johannes Konings, John Culkin, Jonathan Deamer, Juraj Martinka, Ken Snyder, Markus Ellers, Oriol Rodriguez, Ross Mohan, sam onaga, Satyendra Sharma, Simon Devlin, Todd Valentine, Victor Grenu, and all anonymous supporters for your help! We also want to thank all supporters who purchased a cloudonaut t-shirt.