© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.

SVS407-R

Understanding AWS Lambda

event source mapping

Chris McPeek Jacob Cravinho

(he/him) (he/him)
Principal Solutions Architect Senior Solutions Architect
AWS AWS

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is an event source connector

An event source connector is a class of event processor

services that transform and route events to consumers

•Examples: Amazon EventBridge Pipes and AWS Lambda ESM

•Connects passive event sources with downstream targets

Queue Event source Event

connector target

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is Lambda event source mapping (ESM)
An event source mapping is a Lambda resource that
reads items from stream- and queue-based services and
invokes a function with batches of records

•Processing high-volume data via batch for high efficiency

•Scaling and concurrency management
•Manages polling interactions

Lambda

Queue / Stream
ESM
Lambda
function
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lambda invocation models
Synchronous Asynchronous Poll-based
Amazon Amazon
Amazon
SNS SQS
API Gateway

Amazon Amazon
S3 Kinesis

/order
reqs Changes

event event

AWS Lambda
Synchronous
Lambda Lambda Lambda
function function function
event

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How Lambda ESM works
3. If the Lambda returns successfully
1. Lambda Service polls the queue / stream then the Lambda service advances to
the next set of records and repeats #1

2. The ESM synchronously invokes the

Lambda function with the batch of 4. If the Lambda errors, the
records behavior depends on the event
source configuration

Lambda

Queue / Stream
ESM
Lambda
function

On-failure
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. destination
Error handling – ReportBatchItemFailures
1. Report batch item failure (3,6)
(3)

Lambda

Stream

2. Stream
2. Stream checkpoints
checkpoints to
to 8 7 6 5 4 3 2 1
sequence numbernumber
lowest sequence from (3)

3. Poll records from

updated checkpoint

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Error handling – BisectBatchOnFunctionError

Lambda

8 7 6 5 4 3 2 1 8 7 6 5 4 3 2 1 4 3 2 1

= Record #3 found.
Retry based on MaxRetry
and record age settings

If sent to failure dest. If

4 3 configured or discarded.

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Service integration
breakdown

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Services

Amazon DocumentDB Amazon Managed Streaming

for Apache Kafka (Amazon
MSK)
Amazon DynamoDB

Self-managed Apache Kafka

Amazon Kinesis

Amazon Simple Queue

Amazon MQ Service (Amazon SQS)

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Whiteboarding

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Resources: serverlessland.com

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Continue your AWS serverless learning
Learn at your Increase your Earn AWS
own pace knowledge Serverless badge

Expand your serverless Use our Ramp-Up Guides Demonstrate your

skills with our learning plans to build your serverless knowledge by achieving
on AWS Skill Builder knowledge digital badges

https://s12d.com/serverless-learning

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Best practices, for everyone
Batch processing Config management

Observability Secrets handling

Powertools for
AWS Lambda REST/GraphQL API Idempotency

Python | TypeScript | Java | .NET BYO middleware

Input/Output validation

Self-documented schemas Feature flags

Caching Streaming Data extraction

*feature set may vary across languages

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Check out these other sessions
SVS216: Serverless data processing with AWS Lambda and Apache Kafka (Builders session)
Wednesday (Dec. 4) @ 8:30a – Mandalay Bay Surf B*

SVS406: Scale streaming workloads with AWS Lambda (Chalk talk)

Thursday (Dec. 5) @ 4:00p – MGM Premier 309*

SVS305: Handle scaling with AWS Lambda (Builders session)

Wednesday (Dec. 4) @11:30a – Mandalay Bay Surf B*
Thursday (Dec. 5) @ 3:30p - MGM Terrace 151*

SVS321: AWS Lambda and Apache Kafka for real-time data processing applications (Breakout
session)
*Times and locations are subject to change;
Look for YouTube after re:Invent! check session catalog for up-to-date information

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you! Please complete the session
survey in the mobile app

Chris McPeek Jacob Cravinho

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.