#s3 | Amazon S3 (1)
The Simple Storage Service (S3) provides a fault tolerant and highly scalable storage service which is a great fit for many scenarios.
Here are our toughts about the latest AWS announcements as well as our lessons learned. Unbelievable, re:Invent starts in 5 days. Therefore it is no surprise that AWS announces new features at a high pace. On the one hand, we are having difficulty keep...Read more
Builder's Diary Vol. 2: Serverless ETL with Airflow and Athena
Get insights into the day-to-day challenges of builders. In this issue, Peter Reitz from our partner tecRacer talks about how to build Serverless ETL (extract, transform and load) pipelines with the help of Amazon Managed Workflows for Apache Airflow (M...Read more
Show Me Your Architecture Vol. 1: Scanning S3 buckets for malware
Through the AWS documentation, books like AWS in Action or AWS trainings you can gain theoretical knowledge. But beyond that, it is very valuable to learn directly from practice. In this series, we inspect real-life AWS architectures. We start with Andr...Read more
Enabling S3 Versioning is not a backup strategy
Here are three reasons why enabling S3 Versioning is not a backup strategy. Instead, you should consider AWS Backup for S3, which AWS released on February 18th, 2022. AWS Backup enables you to control and automate managing backups centrally. To do so, A...Read more
Video Hosting on AWS
How to embed a video into your website? There is an alternative that looks and feels much more professional than embedding a YouTube video. Read on to learn how to host video-on-demand with the help of Amazon Web Services (AWS). First of all, deliverin...Read more
Unboxing S3 Object Lambda
Amazon S3 Object Lambda offers a way to execute a Lambda function when someone wants to download a file (GetObject) from an S3 bucket. You can implement whatever logic you wish and return any data as the response via the WriteGetObjectResponse API. Keep...Read more
All you need to know about encrypting S3 buckets
Dance like nobody’s watching, encrypt like everyone is. Are you encrypting your data stored on Amazon Simple Storage Service (S3)? No, this video explains why and how to do so. Yes, this video helps you to avoid common pitfalls when doing so. After wat...Read more
Does your VPC endpoint allow access to half of the Internet?
Are you using VPC endpoints to enable private connections between your VPC and AWS services? Drop everything and check the policy attached to your VPC Endpoint for S3. You might have allowed access to half of the Internet - assuming that half of the Int...Read more
Unboxing AWS DataSync
Your toolbox should contain AWS DataSync, a service to synchronize data between all kinds of locations. Copy data between S3, EFS, and FSx. On top of that, DataSync works with on-premises locations like NFS, SMB, and more. In this week’s video Andreas...Read more
How do you choose the best storage option on AWS?
Choosing storage service is critical when designing a cloud architecture. Read on to learn about the characteristics, limitations, typical use cases, and a decision tree for the following options to store data on AWS: Instance Store provides low latenc...Read more
How to analyze and reduce S3 storage usage?
S3 is an object store, not a file system. Object storage comes with significant advantages: unlimited storage capacity, high availability, and durability. However, there are some disadvantages too. For example, it is cumbersome to calculate the storage ...Read more
ETL with a Glue Python Shell Job: Load data from S3 to Redshift
Gaining valuable insights from data is a challenge. After collecting data, the next step is to extract, transform, and load (ETL) the data into an analytics platform like Amazon Redshift. Luckily, there is a platform to build ETL pipelines: AWS Glue. I...Read more
How we run our blog cloudonaut.io
Now and then you ask us: How do you run cloudonaut.io? Today, I want to share some insights with you about the work and technology behind the scenes of our blog. How do we come up with new topics? How do we host this website? How do we survive to be li...Read more
How to avoid S3 data leaks?
Not a week goes by without a frightening announcement that an organization has leaked confidential data from Amazon S3 accidentally. Most often, the root cause of a security breach is a misconfiguration of S3 access control. Do you prefer listening to...Read more
Download YouTube videos with AWS Lambda and store them on S3
Recently, I was faced with the challenge to download videos from YouTube and store them on S3. Sounds easy? Remember than Lambda comes with a few limitations: 512 MB of disk space available at /tmp 3008 MB of memory 15 minutes maximum execution time ...Read more
Own your analytics data: Replacing Google Analytics with Amazon QuickSight
Making decisions based on data is compelling. Especially, when you wrestle with the most straightforward decisions like me. Google Analytics is the de-facto standard for tracking your customers while clicking through your websites and applications. I’ve...Read more
Three simple rules to avoid data leaking from S3
Reviewing AWS accounts with a focus on security is part of my day-to-day job. My most common finding: unwanted public read or write access to S3 buckets. Why is that? Because there are three different ways to define who can access your S3 buckets: IAM p...Read more
Encrypting sensitive data stored on S3
S3 comes with a bunch of features to encrypt your data at rest. Data at rest means inactive data stored physically on disk. Before we dive into encrypting data at rest, I want to highlight that there is also data in use and data in transit. If the data...Read more
Rich Social Sharing with single page applications hosted on S3 and delivered via CloudFront
You undoubtedly heard about single page applications (SPA) written with frameworks like Angular or React. One of the benefits of this approach is the possibility to host the static files (HTML, js, CSS, etc.) on a simple storage solution like S3 and put...Read more
Cleaning up an S3 bucket with the help of Athena
Imagine your basement or attic would provide unlimited capacity for storing stuff. Sounds great? Maybe at first. But imagine how many stuff would pile up over the years if you were not forced to clean up your stuff because of limited storage space every...Read more