EFS with TLS behind a proxy

Andreas Wittig – 19 Jun 2019

Encryption of data at rest and in transit is the new normal. Or as Werner Vogels (Amazon, CTO) says: “Dance like nobody’s watching. Encrypt like everyone is.” The Amazon Elastic File System (EFS) supports both: encryption at rest and encryption in transit.

EFS Encryption In Transit

As often, the tooling for the cloud does not deal very well with the fact that there isn’t unrestricted Internet connectivity from every subnet. This article explains how to set up an encrypted connection from an EC2 instance - running in a subnet without a route to an Internet Gateway or NAT gateway - to EFS. That includes scenarios, where your EC2 instance is required to use a proxy for HTTP and HTTPS connections.

How does encryption in transit from EC2 to EFS work? First, the efs-utils tool configures and starts stunnel to establish a TLS connection between the instance and the file system. Next, the tool creates an NFS mount. The following figure illustrates the architecture.

Use efs-util to create EFS mount with TLS

How to mount an EFS file system with TLS behind a proxy or from a subnet without Internet connectivity? First of all, you need to install the efs-utils tool on your EC2 instance.

yum install -y amazon-efs-utils

Next, you need to mount your EFS file system. All you need is the ID of your file system and the following command. Note, the -o tls parameter tells the efs-util to establish a TLS connection to EFS.

mount -t efs -o tls fs-222121ea:/ efs

Unfortunately, the command will fail when your EC2 instance is not running in a subnet with a route to an Internet gateway or NAT gateway.

Looking for a new challenge?

  • tecRacer

    Cloud Consultant • AWS Migrations

    tecRacer • Premier AWS Consulting Partner • Germany, Austria, Portugal, and Switzerland
    Assessment Transformation Change Management

    Senior Lead Full Stack Developer

    DEMICON • AWS Advanced Consulting Partner • Remote
    AWS JavaScript/TypeScript Angular React

mount.nfs4: Connection reset by peer
Failed to initialize TLS tunnel for fs-222121ea

Why is that? The efs-util uses stunnel to establish a TLS connection between your EC2 instance and EFS. By default, stunnel uses OCSP (Online Certificate Status Protocol ) to validate the certificate used by EFS. Doing so requires an Internet connection. Even worse, stunnel does not support an HTTP/HTTPS proxy for OCSP.

As of Jul 23, 2019, Amazon no longer enables OCSP by default.

But there is a workaround for the problem: disabling OCSP. To do so, open the file /etc/amazon/efs/efs-utils.conf with the editor of your choice.

vi /etc/amazon/efs/efs-utils.conf

Search for the configuration parameter stunnel_check_cert_validity and set the value from true to false.

# Copyright 2017-2018 Amazon.com, Inc. and its affiliates. All Rights Reserved.
# Licensed under the MIT License. See the LICENSE accompanying this file
# for the specific language governing permissions and limitations under
# the License.

logging_level = INFO
logging_max_bytes = 1048576
logging_file_count = 10
# mode for /var/run/efs in octal
state_file_dir_mode = 750

dns_name_format = {fs_id}.efs.{region}.amazonaws.com
stunnel_debug_enabled = false

# Validate the certificate hostname on mount. This option is not supported by certain stunnel versions.
stunnel_check_cert_hostname = true

# Use OCSP to check certificate validity. This option is not supported by certain stunnel versions.
stunnel_check_cert_validity = false

# Define the port range that the TLS tunnel will choose from
port_range_lower_bound = 20049
port_range_upper_bound = 20449

enabled = true
poll_interval_sec = 1
unmount_grace_period_sec = 30

Afterward, try to mount the EFS again.

mount -t efs -o tls fs-222121ea:/ efs

Works now!

In my opinion, it is fine to disable the certification validation as the risk of a man-in-the-middle attack is very low inside your VPC. The authenticity provided by TLS is not needed, because it is already provided by the VPC service.

AWS is aware of the problem. You might want to check out the GitHub issue discussing the stunnel limitation.

Become a cloudonaut supporter

Andreas Wittig

Andreas Wittig ( Email, Twitter, or LinkedIn )

We launched the cloudonaut blog in 2015. Since then, we have published 360 articles, 50 podcast episodes, and 48 videos. It's all free and means a lot of work in our spare time. We enjoy sharing our AWS knowledge with you.

Please support us

Have you learned something new by reading, listening, or watching our content? With your help, we can spend enough time to keep publishing great content in the future. Learn more

Amount must be a multriply of 5. E.g, 5, 10, 15.

Thanks to Alan Leech, Alex DeBrie, ANTHONY RAITI, Christopher Hipwell, Jaap-Jan Frans, Jason Yorty, Jeff Finley, Jens Gehring, jhoadley, Johannes Grumböck, Johannes Konings, John Culkin, Jonas Mellquist, Juraj Martinka, Kamil Oboril, Ken Snyder, Markus Ellers, Ross Mohan, Ross Mohan, sam onaga, Satyendra Sharma, Shawn Tolidano, Simon Devlin, Thorsten Hoeger, Todd Valentine, Victor Grenu, and all anonymous supporters for your help! We also want to thank all supporters who purchased a cloudonaut t-shirt.