cloudonaut
Home Blog Podcast YouTube Games Books Logout
Subscribe Logout

Unboxing S3 Object Lambda

Michael Wittig – 05 May 2021

Amazon S3 Object Lambda offers a way to execute a Lambda function when someone wants to download a file (GetObject) from an S3 bucket. You can implement whatever logic you wish and return any data as the response via the WriteGetObjectResponse API. Keep in mind that the Lambda function must finish within 60 seconds and is called synchronously.

Unboxing S3 Object Lambda

How S3 Object Lambda works

The following figure shows the needed components:

S3 Object Lambda

Let’s look at the parts more closely:

  • S3 Object Lamda Access Point: References your Lambda function and your S3 Access Point. Block Public Access is always enabled.
  • Lambda function: Invoked when a GetObject request is made to the S3 Object Lambda Access Point.
  • S3 Access Point: Takes care of the non GetObject requests (such as ListObjects) and serves the pre-signed URL of the original file passed into the Lambda function.
  • S3 Bucket: Stores the original file (in fact, the original file could be non-existent).

In the following video, I go into more details and share code with you:

  • Introducing S3 Object Lambda
  • Use Cases
  • Demo: Generating content
  • Understanding Pricing
  • Pitfall: Keep the Lambda function running

Pricing

You pay for the following:

  • S3 Get Object API (as usual)
  • Lambda invocation
  • Lambda GB-second
  • $0.005 per GB returned via WriteGetObjectResponse

Source Code

---
AWSTemplateFormatVersion: '2010-09-09'
Description: 'S3 Object Lambda Demo'
Resources:
  Bucket:
    Type: 'AWS::S3::Bucket'
    Properties: {}
  FunctionLogGroup:
    Type: 'AWS::Logs::LogGroup'
    Properties:
      LogGroupName: !Sub '/aws/lambda/${Function}'
      RetentionInDays: 14
  FunctionRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service: 'lambda.amazonaws.com'
          Action: 'sts:AssumeRole'
      Policies:
      - PolicyName: 's3-object-lambda'
        PolicyDocument:
          Statement:
          - Effect: Allow
            Action: 's3-object-lambda:WriteGetObjectResponse'
            Resource: '*'
  FunctionPolicy:
    Type: 'AWS::IAM::Policy'
    Properties:
      Roles:
      - !Ref FunctionRole
      PolicyName: lambda
      PolicyDocument:
        Statement:
        - Effect: Allow
          Action:
          - 'logs:CreateLogStream'
          - 'logs:PutLogEvents'
          Resource: !GetAtt 'FunctionLogGroup.Arn'
  Function:
    Type: 'AWS::Lambda::Function'
    Properties:
      Code:
        ZipFile: |
          const AWS = require('aws-sdk');
          const s3 = new AWS.S3({apiVersion: '2006-03-01'});
          exports.handler = async (event) => {
              console.log(JSON.stringify(event));
              await s3.writeGetObjectResponse({
                RequestRoute: event.getObjectContext.outputRoute,
                RequestToken: event.getObjectContext.outputToken,
                Body: 'hello'
              }).promise()
              return {};
          };
      Handler: 'index.handler'
      MemorySize: 128
      Role: !GetAtt 'FunctionRole.Arn'
      Runtime: 'nodejs12.x'
      Timeout: 60
  AccessPoint:
    Type: 'AWS::S3::AccessPoint'
    Properties:
      Bucket: !Ref Bucket
  LambdaAccessPoint:
    Type: 'AWS::S3ObjectLambda::AccessPoint'
    Properties: 
      Name: !Ref 'AWS::StackName'
      ObjectLambdaConfiguration: 
        SupportingAccessPoint: !Sub 'arn:${AWS::Partition}:s3:${AWS::Region}:${AWS::AccountId}:accesspoint/${AccessPoint}'
        TransformationConfigurations:
        - Actions:
          - GetObject
          ContentTransformation:
            AwsLambda:
              FunctionArn: !GetAtt 'Function.Arn'
Outputs:
  BucketName:
    Value: !Ref Bucket
  LambdaAccessPointArn:
    Value: !GetAtt 'LambdaAccessPoint.Arn'

To invoke the S3 Object Lambda Access Point, run with your ARN: aws s3api get-object --bucket arn:aws:s3-object-lambda:us-east-1:123456789123:accesspoint/s3-object-lambda --key file.txt outfile.txt

Michael Wittig

Michael Wittig

I’ve been building on AWS since 2012 together with my brother Andreas. We are sharing our insights into all things AWS on cloudonaut and have written the book AWS in Action. Besides that, we’re currently working on bucketAV, HyperEnv for GitHub Actions, and marbot.

Here are the contact options for feedback and questions.

Further reading