© 2024, Amazon Web Services, Inc. or its affiliates.

All rights
reserved.
A P I 4
0 2

Building advanced
workflows with AWS
Step Functions
Eric Johnson
Principal Developer Advocate,
Serverless Amazon Web Services

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
About me
• Eric Johnson
 Email: [email protected]

 X: @edjgeek

 LinkedIn: https:/
/www.linkedin.com/in/singledigit/

• Principal Developer
Advocate
• Serverless geek
• Previously
 Solutions architect

 Software developer

 Designer
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Agenda (as an AWS Step Functions
workflow)
Step Functions first

? Which workflow
type?
Standard Express

Reducing cost

Improved data flow

Managing failures Step Functions at

scale

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Manage
d

Connected by
events Th
e

serverle
ss
event-driven pyramid

Manu
al

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 The serverless
challenge
Question: When your application
is powered by multiple connected
services, how do you build, track,
inspect, visualize, and orchestrate
those connections?

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Answer: AWS Step Functions
A S E R V E R L E S S , L O W - C O D E V I S U A
L W O R K F L O W S E R V I C E

• Pay-per-use
• Scales
automatically
• Fully managed

• Drag and drop or

ASL
• Built-in error
handling

• Integrates with
over
200 AWS services

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
“Step Functions Step Functions first

first, Step ? Which workflow type?

Functions always.”
Standard Express

Reducing cost

Improved data flow

Managing failures Step Functions at scale

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
From code to workflow
app.js

An AWS Lambda function that queries

Amazon DynamoDB has multiple lines
of code

Lambd Dynamo
a DB

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
From code to workflow
app.js

const AWS = require('aws-sdk');

const docClient = new AWS.DynamoDB.DocumentClient();
An AWS Lambda function that queries
Amazon DynamoDB has multiple lines
of code

Lambd Dynamo
a DB

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
From code to workflow
app.js

const AWS = require('aws-sdk');

const docClient = new AWS.DynamoDB.DocumentClient();
An AWS Lambda function that queries
var params = {
"TableName": "reinvent2022!", Amazon DynamoDB has multiple lines
"Key": {
"PK": {"S": "Wardrobe"},
of code
"SK": {"S": ”shoes"}
}
}

Lambd Dynamo
a DB

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
From code to workflow
app.js

const AWS = require('aws-sdk');

const docClient = new AWS.DynamoDB.DocumentClient();
An AWS Lambda function that queries
var params = {
"TableName": "reinvent2022!", Amazon DynamoDB has multiple lines
"Key": {
"PK": {"S": "Wardrobe"},
of code
"SK": {"S": ”shoes"}
}
}

async function queryItems(){ try Lambd Dynamo

{
const data = await docClient.getItem(params).promise() return
a DB
data
} catch (err)
{ return err
}
}

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
From code to workflow
app.js

const AWS = require('aws-sdk');

const docClient = new AWS.DynamoDB.DocumentClient();
An AWS Lambda function that queries
var params = {
"TableName": "reinvent2022!", Amazon DynamoDB has multiple lines
"Key": {
"PK": {"S": "Wardrobe"},
of code
"SK": {"S": ”shoes"}
}
}

async function queryItems(){ try Lambd Dynamo

{
const data = await docClient.getItem(params).promise() return
a DB
data
} catch (err)
{ return err
}
}
exports.handler = async (event, context) => { try {
const data = await queryItems() return { body:
JSON.stringify(data) }
} catch (err) {
return { error: err }
}
}

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
From code to workflow
Even a single-task “workflow”
app.js
adds value with built-in error
handling, catch, retry,
const AWS = require('aws-sdk');
const docClient = new AWS.DynamoDB.DocumentClient(); observability, reduction of custom
code, and centralized logging of
var params = {
"TableName": "reinvent2022!",
each workload
"Key": {
"PK": {"S": "Wardrobe"},
"SK": {"S": ”shoes"}
}
}

async function queryItems(){ try

{
const data = await docClient.getItem(params).promise() return
data
} catch (err)
{ return err
}
}
exports.handler = async (event, context) => { try {
const data = await queryItems() return { body:
JSON.stringify(data) }
} catch (err) {
return { error: err }
}
}

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
From code to workflow
Even a single-task “workflow”
app.js
adds value with built-in error
handling, catch, retry,
const AWS = require('aws-sdk');
const docClient = new AWS.DynamoDB.DocumentClient(); observability, reduction of custom
code, and centralized logging of
var params = {
"TableName": "reinvent2022!",
each workload
"Key": {
"PK": {"S": "Wardrobe"},
"SK": {"S": ”shoes"}
}
}

async function queryItems(){ try

{
const data = await docClient.getItem(params).promise() return
data
} catch (err)
GetItem
{ return err
}
}
exports.handler = async (event, context) => { try {
const data = await queryItems() return { body:
JSON.stringify(data) }
} catch (err) {
return { error: err }
}
}

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
From code to workflow
Even a single-task “workflow”
app.js
adds value with built-in error
handling, catch, retry,
const AWS = require('aws-sdk');
const docClient = new AWS.DynamoDB.DocumentClient(); observability, reduction of custom
code, and centralized logging of
var params = {
"TableName": "reinvent2022!",
each workload
"Key": {
"PK": {"S": "Wardrobe"},
"SK": {"S": ”shoes"}
}
}

async function queryItems(){ try

{
Retry
const data = await docClient.getItem(params).promise() return
data
} catch (err)
GetIte
{ return err m
}
}
exports.handler = async (event, context) => { try {
const data = await queryItems() return { body:
JSON.stringify(data) }
} catch (err) {
return { error: err }
}
}

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
From code to workflow
Even a single-task “workflow”
app.js
adds value with built-in error
handling, catch, retry,
const AWS = require('aws-sdk');
const docClient = new AWS.DynamoDB.DocumentClient(); observability, reduction of custom
code, and centralized logging of
var params = {
"TableName": "reinvent2022!",
each workload
"Key": {
"PK": {"S": "Wardrobe"},
"SK": {"S": ”shoes"}
}
}

async function queryItems(){ try

{
Retry
const data = await docClient.getItem(params).promise() return
data
} catch (err)
GetItem
{ return err
}
} Catch error
exports.handler = async (event, context) => { try {
const data = await queryItems() return { body: DLQ
JSON.stringify(data) }
} catch (err) {
return { error: err }
}
}

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
From code to workflow
Even a single-task “workflow”
app.js
adds value with built-in error
handling, catch, retry,
const AWS = require('aws-sdk');
const docClient = new AWS.DynamoDB.DocumentClient(); observability, reduction of custom
code, and centralized logging of
var params = {
"TableName": "reinvent2022!",
each workload
"Key": {
"PK": {"S": "Wardrobe"},
"SK": {"S": ”shoes"}
}
}

async function queryItems(){ try

{
const data = await docClient.getItem(params).promise() return
data
} catch (err)
{ return err
}
}
exports.handler = async (event, context) => { try {
const data = await queryItems() return { body:
JSON.stringify(data) }
} catch (err) {
return { error: err }
}
}

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
From code to
workflow
Drill down to trace the
execution path of every
workload request

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
From code to workflow

Examine the input and

output of
each task for each request

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Breaking apart a
“Lambda-lith”
Amazon API Gateway
routes all requests to a
single Lambda function that
runs the appropriate code
/create based on its route
/update
/delete configuration
/*
• Security permissions
Clien API
Routing
DynamoD applied to the whole
t Gateway B
to logic in
function • Performance setting
applied to the whole
• Duration and space
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
limits applied to
reserved.
the whole
Micro Lambda
App-delete.js
const AWS = require('aws-sdk');
const docClient = new AWS.DynamoADpB.pD-
oucupmdeanttCel.iejnst();
arams = {
Ta bleName: 'your-table-name',
const
varInp const docClient
AWS
dexName: = = new AWS.DynamaopDBp.-
require('aws-sdk');
'some-index',
• E a c h C RUD operation
becomes
Ke yConditionExpression:
var params
const
uemaAWS
DcocndexName:
T rconst
Ex pressionAttributeValues:
'#name =in:value',
= { 's require('aws-sdk');
entte= omnt();e-
C.lije dex
s { ':value': 'shoes' ',
},
docClient = new
ableName: 'your-table-name', AWS.DynamoDB.DocumentClient();
Ex pressionAttributeNames:
I { '#name': 'name' }
/create } K var params = {
E TableName: 'your-table-name',
E eyConditionE
asyn c function xpr y es si on : '# na m e
tr {} a s ep ar a t e
queryItems(){
IndexName: 'some-index',
KeyConditionExpression: '#name = :value',
c onst data = await
ExpressionAttributeValues: { ':value': 'shoes' },
r docClient.query(params).pr
asy = ExpressionAttributeNames: { '#name': 'name' }
} c omise() :v al ue' , xpressionAttributeValues:
/update L a m b da function
t }eturn
r atch (err)
data
{ ':value': 'shoes' }, xpressionAttributeNames: { '#name': 'name' }
} { eturn err
async function queryItems(){ try
} } { • API Gateway routes
const data = await docClient.query(params).promise() return
Clien API DynamoD
expo
tr }
}
}
request to a
data
nc (err)
catch function queryItems(){
t Gateway /delete
B c rts.handler = async
{ specific Lambda
(event, context) => { y {
ry { err
return
onst data = await queryItems()
function
r exp {}body: JSON.stringify(data) } atch
} c eturn t }
r (err) {const data = await docClient.query(params).promise()
} eturn return
{ error: err }
data
exports.handler = async (event, context) => {
} } catchtry(err)
{
{ return err data = await queryItems() return
const
} { body: JSON.stringify(data) }
} } catch (err) {
return { error: err }
}
orts.handler = async (event, context) => { ry {
© 2024, Amazon Web Services, Inc. or its affiliates. All rights }
const data = await queryItems() return
reserved.
{ body: JSON.stringify(data) } catch (err) {
 API
Gateway

The “REST”
easy
Combine with API Gateway and Step Functions
Synchronous
Express Workflows to create a low-latency, scalable API
backend
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Order manager service as a Lambda
function
Version 1
Each operation invokes a Lambda
function
cancel
Application grows more complex
API
• Update Dynamo over time, performing multiple tasks
• Resum DB SFN
Gatewa
y e to handle increasingly complex
business logic
complete • Update Dynamo
API • Resum DB SFN
Gatewa
e
y
Results
mak • Update Dynamo
API e • Resum DB SFN • Tightly coupled code base
Gatewa
e
y
• Sanitize order • Slower release cadence
create • Update
API DynamoDB • Poor discoverability
Gatewa • Resume SFN
y
• Additional complexity

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Order manager service as a
workflow
Version 2
A single API Gateway endpoint
runs
a Step Functions workflow

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Step Functions first

? Which workflow type?

Choosing Standard

Reducing cost
Express

Standard or Improved data flow

Express Managing failures Step Functions at scale

Workflows

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Expres
s
Standar vs throughput
• High-

• Cost-effective
d
• Long-lasting
•
. • At least once
Asynchrono • Synchronous
us • Asynchronous
• Exactly once • Short duration

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Real-life
example

An ecommerce workflow

1,000 as a
Standard 1,000
as an Express

I create an
Amazon
CloudWatch
dashboard
to display the
average
execution times
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Cost per 1,000 executions

Expres
Standar s
vs.
d
$0.42 $0.01
*

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Cost per 1,000 executions
C H A R G E D B A S E D O N T H E N U M B E R
O F S T A T E T R A N S I T I O N S

Total cost for a Standard Workflow

V Standard Workflow billing

is
$0.025 per 1,000 state
transitions

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Cost per 1,000 executions
C H A R G E D B A S E D O N T H E N U M B E R
O F S T A T E T R A N S I T I O N S

Total cost for a Standard Workflow

# transitions * (# of executions * $0.000025)

V Standard Workflow billing
is
$0.025 per 1,000 state
transitions

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Cost per 1,000 executions
C H A R G E D B A S E D O N T H E N U M B E R
O F S T A T E T R A N S I T I O N S

Total cost for a Standard Workflow

# transitions * (# of executions * $0.000025)

V Standard Workflow billing
is
$0.025 per 1,000 state
transitions

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Cost per 1,000 executions
C H A R G E D B A S E D O N T H E N U M B E R
O F S T A T E T R A N S I T I O N S

Total cost for a Standard Workflow

# transitions * (# of executions * $0.000025)

V Standard Workflow billing
is
$0.025 per 1,000 state
transitions

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Cost per 1,000 executions
C H A R G E D B A S E D O N T H E N U M B E R
O F S T A T E T R A N S I T I O N S

Total cost for a Standard Workflow

# transitions * (# of executions *
$0.000025)
Total cost = 17 * 0.025 =
V Standard Workflow billing
is
$0.42* $0.025 per 1,000 state
transitions

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Cost per 1,000 executions
C H A R G E D B A S E D O N T H E N U M B E R
O F S T A T E T R A N S I T I O N S

Total cost for a Standard Workflow

# transitions * (# of executions *
$0.000025)
Total cost = 17 * 0.025 = $0.42*
V Standard Workflow billing
is
$0.025 per 1,000 state
Total cost = $0.42 per 1,000 transitions
executions

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E
R O F R E Q U E S T S A N D D U R A T I O N

Total cost for an Express Workflow

V Express Workflows charge

based on the number of
requests and duration (to
the nearest 100 ms)

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E
R O F R E Q U E S T S A N D D U R A T I O N

Total cost for an Express Workflow

(Execution cost + duration cost) * # of requests

V Express Workflows charge
based on the number of
requests and duration (to
the nearest 100 ms)

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E
R O F R E Q U E S T S A N D D U R A T I O N

Total cost for an Express Workflow

(Execution cost + duration cost) * # of

requests
Duration cost = (execution duration in ms /
V Express Workflows charge
based on the number of
100) * requests and duration (to
memory cost the nearest 100 ms)

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W SA P P R O X I M A T E P R I C E P E R 1 0 0
M S F O R D I F F E R E N T M E M O R Y S I Z E S

Price per 100 ms Price per 100 ms Price per 100

Memory
for first 1,000 for next 4,000 ms for
GB-hours GB-hours additional GB-
hour
64 MB $0.0000001042 $0.0000000521 $0.0000000285

128 MB $0.0000002083 $0.0000001042 $0.0000000570

192 MB $0.0000003125 $0.0000001563 $0.0000000856

256 MB $0.0000004167 $0.0000002083 $0.0000001141

Each additional 64 MB $0.0000001042 $0.0000000521 $0.0000000285

Step Functions pricing page:
https://aws.amazon.com/step-functions/pricing/

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W SA P P R O X I M A T E P R I C E P E R 1 0 0
M S F O R D I F F E R E N T M E M O R Y S I Z E S

Price per
Price per 100
100 ms Price per 100 ms Price per 100
Memory
for for
ms firstfirst
1,000 for next 4,000 ms for
GB-hours
1,000 GB-hours GB-hours additional GB-
64 $0.00000010 hour
MB 42
64 MB $0.0000001042 $0.0000000521 $0.0000000285

128 MB $0.0000002083 $0.0000001042 $0.0000000570

192 MB $0.0000003125 $0.0000001563 $0.0000000856

256 MB $0.0000004167 $0.0000002083 $0.0000001141

Each additional 64 MB $0.0000001042 $0.0000000521 $0.0000000285

Step Functions pricing page:
https://aws.amazon.com/step-functions/pricing/

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E
R O F R E Q U E S T S A N D D U R A T I O N

Total cost for an Express Workflow

(Execution cost + duration cost) * # of

requests
Duration cost = (11,300 ms /
V Express Workflows charge
based on the number of
100) * requests and duration (to
$0.0000001042 = the nearest 100 ms)
Execution cost = $0.000001 per request
$0.0000117746

($0.000001 + $0.0000117746) * 1,000

= $0.01

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Cost per 1,000 executions

Expres
Standar s
vs.
d
$0.42 $0.01
*

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Standard Workflow
cost
$450.0 1 million executions =
0

$12.77
$400.0
0
$350.0
0

Express
$300.0
0
$250.0
0

vs
$200.0
- 500,00 1,000,00 1,500,00
0
0 0 0
$150.0 Execution
0 s
$100.0

Standard .
0
$50.00
$-
Express Workflow
1 million executions = $15.0 cost
0

$420 $10.0
0
$5.0
0

$- - 200,00 400,00 600,000 800,000 1,000,000

0 0 1,200,000
Execution
s
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
 Standard vs. Express Workflow: 1 million
Standard Workflow
executions cost
$450.0

$350.00
0
1 million executions = $12.77
$450.00 $400.0

Express
$400.00 0
$300.00
$150.00
$250.00 $350.0
$200.00 0
$100.00
$50.00
$-
$300.0
0
- 500,000 1,000,000 1,500,000

$250.0
0

Standard $200.0

1 million executions =
0

$150.0
$420 $10.00
$15.00
Express Workflow cost

0
- 200,00 400,00 600,00 800,00 1,000,00 1,200,00
0 0 0 0 0 0
$100.0 Series Series
$5.00
0 1 2
$-
$50.0 -
0
200,000
$-
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
400,000
So why choose Standard?
Star
t

Which
? workflow type?
• The workload takes more than 5
minutes $.workload.duration > 5
• The workload uses callback minutes
(.waitForTaskToken) or (.sync) patterns $.workload.contains(.sy
nc)
$.workload.contains(.WaitForTaskTok
• The workload requires exactly-once en)
defaul
$.Execution.semantics ==
execution Exactly-one
t

Run as Standard Run as Express

En
d

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
So why choose Standard?
START

Which
?
• The workload takes more than 5 minutes workflow

$.workload.duration > 5 minutes

type?
Standard or
ck (.waitForTaskToken)
Express $.workload.contains(.sync)
• The workload uses

How about
$.workload.contains(.WaitForTaskT
callba or (.sync) ok
patterns ctly-once execution
$.Execution.semantics ==
en)

both?
Exactly-one
• The workload requires exa
default

Run as Standard

Run as Express

End
Pattern: The
nester Expres
Standar s
&
d

Create nested
workflows that
combine
workflow types
to optimize for
cost
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
How about both?

What if these sequences sometimes

took longer than 5 minutes?

The workflow must run as a

Standard Workflow

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
How about both?

What if these sequences sometimes

took longer than 5 minutes?

The workflow must run as a

Standard Workflow

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
How about both?

What if these sequences sometimes

took longer than 5 minutes?

The workflow must run as a

Standard Workflow

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Nested Parent Standard

workflo Workflow

ws Place the idempotent states

into
a nested Express Workflow

.StartExecuti
on

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E R O F R E Q
U E S T S A N D D U R A T I O N

Parent
Standard Workflow

Total cost for nest workflow Nested

Express
Workflow

This new workflow combination runs

1,000
times, costing a total of 30 cents
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E R O F R E Q
U E S T S A N D D U R A T I O N

Parent
Standard Workflow

Total cost for nest Nested

workflow Express
Workflow
Cost of parent
workflow
V

This new workflow combination runs

1,000
times, costing a total of 30 cents
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E R O F R E Q
U E S T S A N D D U R A T I O N

Parent
Standard Workflow

Total cost for nest Nested

workflow Express
Workflow
Cost of parent workflow
(14 * 1,000) * $0.000025 =
$0.30
V

This new workflow combination runs

1,000
times, costing a total of 30 cents
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E R O F R E Q
U E S T S A N D D U R A T I O N

Parent
Standard Workflow

Total cost for nest Nested

workflow Express
Workflow
Cost of parent workflow Number of

(14 * 1,000) * $0.000025 = $0.30

state
transitions V

This new workflow combination runs

1,000
times, costing a total of 30 cents
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E R O F R E Q
U E S T S A N D D U R A T I O N

Parent
Standard Workflow

Total cost for nest Nested

workflow Express
Workflow
Cost of parent workflow Number of

(14 * 1,000) * $0.000025 = $0.30

state
transitions V
+
Cost of nested workflow
Duration cost = (1,300 ms / 100) * $0.0000001042 =
$0.000001 Execution cost = $0.000001
($0.000001 + $0.000001) x 1,000 = $0.0002

This new
w
o
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved. r
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E R O F R E Q
U E S T S A N D D U R A T I O N

Parent
Standard Workflow

Total cost for nest Nested

workflow Express
Workflow
Cost of parent workflow Number of

(14 * 1,000) * $0.000025 =

$0.30
state
transitions
Averag
V
e
+
Cost of nested workflow duratio
n * $0.0000001042 =
Duration cost = (1,300 ms / 100)
$0.000001 Execution cost = $0.000001
($0.000001 + $0.000001) x 1,000 = $0.0002

T
h
i
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
s
reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E R O F R E Q
U E S T S A N D D U R A T I O N

Parent
Standard Workflow

Total cost for nest Nested

workflow Express
Workflow
Cost of parent workflow Number of

(14 * 1,000) * $0.000025 =

$0.30
state
transitions
Averag
Price per
100 ms for
V
e
first 1,000
+
Cost of nested duratio
GB-hours
n * $0.0000001042 =
Duration cost = (1,300 ms / 100)
workflow
$0.000001 Execution cost = $0.000001
($0.000001 + $0.000001) x 1,000 = $0.0002

T
h
i
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
s
reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E R O F R E Q
U E S T S A N D D U R A T I O N

Parent
Standard Workflow

Total cost for nest Nested

workflow Express
Workflow
Cost of parent workflow Number of

(14 * 1,000) * $0.000025 =

$0.30
state
transitions
Averag
Price per
100 ms for
V
e
first 1,000
+
Cost of nested duratio
GB-hours
workflow n * $0.0000001042 =
Duration cost = (1,300 ms / 100)
$0.000001 Execution cost = $0.000001
($0.000001 + $0.000001) x 1,000 = $0.0002

Total = $0.30 per 1,000

executions This new workflow combination runs
1,000
times, costing a total of 30 cents
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
 Step Functions first

? Which workflow type?

Standard Express

Reducing Reducing cost

cost Improved data flow

Managing failures Step Functions at scale

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Reducing workflow cost

Expres
sReduce execution
Standar vs duration Reduce
d state
Reduce . memory GB usage

transitions

Reduce Manage
the state
number of payloads
states natively

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Understanding total cost

Get Item

Invoke
Lambda

Put Event

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Understanding total cost

Get Item
A GetItem operation using DynamoDB read
capacity units
+

Invoke
Lambda

Put Event

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Understanding total cost

Get Item
A GetItem operation using DynamoDB read
capacity units
+

1 state
Invoke
Lambda transition
+

Put Event

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Understanding total cost

Get Item
A GetItem operation using DynamoDB read
capacity units
+

1 state
Invoke
Lambda transition
+
A Lambda
invocation cost
incurred based
Put Event
on duration
and
memory
configuration

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Understanding total cost

Get Item
A GetItem operation using DynamoDB read
capacity units
+

1 state
Invoke
Lambda transition
+
A Lambda
invocation
1 statecost
incurred based
transition
Put Event
on duration
+
and
memory
configuration

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Understanding total cost

Get Item
A GetItem operation using DynamoDB read
capacity units
+

1 state
Invoke
Lambda transition
+
A Lambda
invocation
1 statecost
incurred based
transition
Put Event
on duration
+
and
An eventPut
memory
operation
configuration
contributing to
overall usage
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
and cost
reserved.
Step Functions intrinsic functions

Array JSON data Encoding and

s manipulation decoding

Math String Unique identifier

operations operations generation

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Reducing cost: Standard Workflows
U S E I N T R I N S I C F U N C T I O N S I N S T E A D O F D A T T R A N S F O R M A T
C O M P U T E S E R V I C E S T O P E R F O R M A I O N S

Generating a unique
ID

Array JSON data Unique String Math Encoding

s manipulatio identifier operation operation and
n generation s s decoding
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Reducing cost: Standard Workflows
U S E I N T R I N S I C F U N C T I O N S I N S T E A D O F D A T T R A N S F O R M A T
C O M P U T E S E R V I C E S T O P E R F O R M A I O N S

Generating a unique
ID
Lambda example
Star
t

Generate ID

Array JSON data Unique String Math Encoding

s manipulatio identifier operation operation and
n generation s s decoding
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Reducing cost: Standard Workflows
U S E I N T R I N S I C F U N C T I O N S I N S T E A D O F D A T T R A N S F O R M A T
C O M P U T E S E R V I C E S T O P E R F O R M A I O N S

Generating a unique
ID
Lambda example
Star
t

const { v4: uuidv4 } = require('uuid');

exports.handler = async (event) => {
Generate ID const uuid = uuidv4();); return
JSON.stringify({ uuid });
};

Array JSON data Unique String Math Encoding

s manipulatio identifier operation operation and
n generation s s decoding
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Reducing cost: Standard Workflows
U S E I N T R I N S I C F U N C T I O N S I N S T E A D O F D A T T R A N S F O R M A T
C O M P U T E S E R V I C E S T O P E R F O R M A I O N S

Generating a unique
ID
Intrinsic
Lambda example example
Star Star
t t

const { v4: uuidv4 } = require('uuid');

exports.handler = async (event) => {
Generate ID const uuid = uuidv4();); return Generate ID
JSON.stringify({ uuid }); States.UUID()
};

Array JSON data Unique String Math Encoding

s manipulatio identifier operation operation and
n generation s s decoding
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Reducing cost: Standard Workflows
U S E I N T R I N S I C F U N C T I O N S I N S T E A D O F D A T T R A N S F O R M A T
C O M P U T E S E R V I C E S T O P E R F O R M A I O N S

Generating a unique
ID
Intrinsic
example
Star
t

No invocation Generate ID
delays No
States.UUID()
invocation cost
Less code to
manage

Array JSON data Unique String Math Encoding

s manipulatio identifier operation operation and
n generation s s decoding
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Reducing
state
transitions
and duration
with the
callback
pattern

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Reducing
state
transitions
and duration
with the
callback
pattern

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Reducing
state
transitions
and duration
with the
callback
pattern

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Emit and
wait:
milestone 1

Emit and
Catc wait:
h milestone 2

Catc Emit and

h wait:
milestone 3

TimedOut
Catc
h
The “emit and wait”
Emit milestone events that invoke external
decoupled
Emit: timeout
microservices and emit error events if they don’t
respond

En
d

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Dynamo
Emit and
DB
wait:
milestone 1

Emit and
Catc wait:
h milestone 2

Catc Emit and

h wait:
milestone 3

Catc
TimedOut h

Emit: timeout

En
d

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t
Milestone 1
TaskToken:
Rul Dynamo
Emit and 12345678 e DB
wait:
milestone 1

Emit and
Catc wait:
h milestone 2

Catc Emit and

h wait:
milestone 3

Catc
TimedOut h

Emit: timeout

En
d

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Milestone 1 Save to DynamoDB
Star
TaskToken: TaskToken:
t
Milestone 1 12345678 12345678
TaskToken:
Rul Lambd Dynamo
Emit and 12345678 e a DB
wait:
milestone 1

Emit and
Catc wait:
h milestone 2

Catc Emit and

h wait:
milestone 3

Catc
TimedOut h

Emit: timeout

En
d

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Milestone 1 Save to DynamoDB
Star
TaskToken: TaskToken:
t
Milestone 1 12345678 12345678
TaskToken:
Rul Lambd Dynamo
Emit and 12345678 e a DB
wait:
milestone 1

Emit and
Catc wait:
h milestone 2

Catc Emit and App.js

h wait:
milestone 3 stepfunctions.sendTaskSuccess({
“taskToken” :event.taskToken,
Catc “ExecutionArn” : event.arn
TimedOut h })

Emit: timeout

En
d

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Emit and
wait:
milestone 1
Milestone 2
TaskToken:
Emit and Rul
Catc wait: 12345678 e
Dynamo
DB
h milestone 2

Catc Emit and

h wait:
milestone 3

Catc
TimedOut h

Emit: timeout

En
d

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Emit and Milestone 2 Save to DynamoDB

wait: TaskToken: TaskToken:
milestone 1
Milestone 2 12345678 12345678
TaskToken:
Emit and Rul
Catc wait: 12345678 e
Lambd
a
Dynamo
DB
h milestone 2

Catc Emit and

h wait:
milestone 3

Catc
TimedOut h

Emit: timeout

En
d

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Emit and Milestone 2 Save to DynamoDB

wait: TaskToken: TaskToken:
milestone 1
Milestone 2 12345678 12345678
TaskToken:
Emit and Rul
Catc wait: 12345678 e
Lambd
a
Dynamo
DB
h milestone 2

Catc Emit and

h wait:
milestone 3

Catc
TimedOut h
Set a time limit to wait for task
token "HeartbeatSeconds": 20
Emit: timeout

En
d

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Emit and Milestone 2 Save to DynamoDB

wait: TaskToken: TaskToken:
milestone 1
Milestone 2 12345678 12345678
TaskToken:
Emit and Rul
Catc wait: 12345678 e
Lambd
a
Dynamo
DB
h milestone 2

Catc Emit and

h wait:
milestone 3

Catc
TimedOut h
Set a time limit to wait for task
token "HeartbeatSeconds": 20
Emit: timeout

En
d

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Remove polling
with callback
pattern

Standard
Workflow
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Remove polling
with callback
pattern
"Resource": "arn:aws:states:::sqs:sendMessage.waitForTaskToken"

"Resource": "arn:aws:states:::sqs:sendMessage.waitForTaskToken"

Standard
Workflow
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E
R O F R E Q U E S T S A N D D U R A T I O N

Parent
Standard
Workflow

Total cost for callback-optimized nested

Nested
workflow Express
Workflow

This new workflow combination

runs
1,000 times, costing a total of 20
cents
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E
R O F R E Q U E S T S A N D D U R A T I O N

Parent
Standard
Workflow

Total cost for callback-optimized nested

Nested
workflow Express
Workflow
Cost of parent
workflow
V

This new workflow combination

runs
1,000 times, costing a total of 20
cents
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E
R O F R E Q U E S T S A N D D U R A T I O N

Parent
Standard
Workflow

Total cost for callback-optimized nested

Nested
workflow Express
Workflow
Cost of parent workflow
(8 * 1,000) * $0.000025 =
$0.20
V

This new workflow combination

runs
1,000 times, costing a total of 20
cents
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E
R O F R E Q U E S T S A N D D U R A T I O N

Parent
Standard
Workflow

Total cost for callback-optimized nested

Nested
workflow Express
Number of Workflow
Cost of parent workflow
(8 * 1,000) * $0.000025 =
$0.20
state
transitions V

This new workflow combination

runs
1,000 times, costing a total of 20
cents
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E
R O F R E Q U E S T S A N D D U R A T I O N

Parent
Standard
Workflow

Total cost for callback-optimized nested

Nested
workflow Express
Number of Workflow
Cost of parent workflow
(8 * 1,000) * $0.000025 =
$0.20
state
transitions V
+
Cost of nested workflow
Duration cost = (1,300 ms / 100) * $ 0.0000001042 =
$0.000001 Execution cost = $0.000001
($0.000001 + $0.000001) x 1,000 = $0.0002

T
h
i
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
s
reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E
R O F R E Q U E S T S A N D D U R A T I O N

Parent
Standard
Workflow

Total cost for callback-optimized nested

Nested
workflow Express
Number of Workflow
Cost of parent workflow
state
(8 * 1,000) * $0.000025 = transitions
$0.20 Averag
V
e
+
Cost of nested workflow duratio
Duration cost = (1,300 ms / 100) n* $ 0.0000001042 =
$0.000001 Execution cost = $0.000001
($0.000001 + $0.000001) x 1,000 = $0.0002

T
h
i
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
s
reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E
R O F R E Q U E S T S A N D D U R A T I O N

Parent
Standard
Workflow

Total cost for callback-optimized nested

Nested
workflow Express
Number of Workflow
Cost of parent workflow

$0.20
state
(8 * 1,000) * $0.000025 = transitions
Averag
Price per
100 ms for
V
e
first 1,000
+
Cost of nested duratio
GB-hours
Duration cost = (1,300 ms / 100) n* $ 0.0000001042 =
workflow
$0.000001 Execution cost = $0.000001
($0.000001 + $0.000001) x 1,000 = $0.0002

T
h
i
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
s
reserved.
Cost per 1,000 executions
E X P R E S S W O R K F L O W S C H A R G E B A S E D O N N U M B E
R O F R E Q U E S T S A N D D U R A T I O N

Parent
Standard
Workflow

Total cost for callback-optimized nested

Nested
workflow Express
Number of Workflow
Cost of parent workflow

$0.20
state
(8 * 1,000) * $0.000025 = transitions
Averag
Price per
100 ms for
V
e
first 1,000
+
Cost of nested duratio
GB-hours
Duration cost = (1,300 ms / 100) n* $ 0.0000001042 =
workflow
$0.000001 Execution cost = $0.000001
($0.000001 + $0.000001) x 1,000 = $0.0002

Total = $0.20 per 1,000

executions This new workflow combination
runs
1,000 times, costing a total of 20
cents
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
HTTP API integrations

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
HTTP API integrations

Data
warehous
e

Inventor
y
service

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
HTTP API integrations

Data
warehous
e

Inventor
y
service

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
HTTP API integrations

Data
warehous
e

Authorization
required

Inventor
y
service

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
HTTP API integrations

Data
warehous
e

Authorization
required

Inventor
y
service

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
HTTP API private integrations

Virtual private cloud

(VPC)

Resource
endpoints

Amazon
Data
EventBridge
warehous
connection
e

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Step Functions first

? Which workflow type?

Standard Express

Improved data Reducing cost

flow Improved data flow

Managing failures Step Functions at scale

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
JSONata support for data
transformation
JSONata is a powerful
query and expression
language to select and
transform data in your
workflows

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Simplified task
state State
input

InputPat State
h input

Parameter
Task or s
sub- Argument
workflow Task or s
ResultSelect sub-
or workflow Outpu
t
ResultPat
h

OutputPat State
h output

JSONPat State JSONat

output
h © 2024, Amazon Web Services, Inc. or its affiliates. All rights
a
reserved.
JSONata syntax
{
“FullName”: “{% $states.input.FirstName & ‘ ‘ & $states.input.LastName %}”
}

Wrap the JSONata string in

{ % % } Drop the use of .$ in
the key name

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
JSONata reserved variables

{
“$states”: {
“input”: “ra to the state”,
“result”: w from the task if successful”,
inp
“errorOutput”: “Results from task if errored”,
ut
“context”: “the context object”
} “Re
sult
}
s

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
JSONata native functions
N O N E X H A U S T I V E ( N O TE V E N
C L O S E )

String functions Number Numeric Object

functions aggregation functions
$string() $join() functions
$number() $keys()
$length() $match() $sum()
$abs() $lookup()
$substring() $replace() $max()
$floor() $spread()
$substringBefore $eval() $min()
$ceil()
() $merge()
$base64encode() $round() $average()
$substringAfter() Array $sift()
$base64decode() $power()
$uppercase() functions $each()
$encodeUrlComponen $sqrt()
$lowercase() $count() $error()
t()
$random()
$trim() $append() $assert()
$encodeUrl()
$formatNumber
$pad() $sort() $type()
$decodeUrlComponen ()
t() $reverse()
$contains() $formatBase()
$decodeUrl() $formatInteger( $shuffle()
$split()
) $distinct()
$parseInteger()
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
$zip()
reserved.
Step Functions JSONata

$partition - partition a large array

$range - generate an array of values.
$hash - calculate the hash value of a given
input.
$random - return a random number n where 0
≤n<1
$uuid - generate a uuid
$parse - deserialize JSON strings

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Step Functions JSONata example

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Step Functions JSONata example

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Step Functions JSONata example

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Step Functions JSONata example

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Step Functions JSONata example

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Step Functions JSONata example

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Step Functions JSONata example
[ [
{ {
"owner": { "id": "monorepo",
"S": "user2.amazon.com" "oldOwner": "user2.amazon.com",
}, "newOwner": "user8.amazon.com"
"id": { },
"S": "monorepo" {
}, "id": "Video",
"url": { "oldOwner": "user2.amazon.com",
"S": "https://aws.amazon.com/monorepo" "newOwner": "user8.amazon.com"
}, },
"updatedAt": { {
"S": "2024-11-06T00:49:08.696Z" "id": "vid315",
}, "oldOwner": "user2.amazon.com",
"timestamp": { "newOwner": "user8.amazon.com"
"S": "03/Feb/2021:10:09:33 +0000" }
} ]
}
]

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Preserving data without the Map
state
Preserving data across
states requires passing
the data into each state:
{
"Comment": "A simple ASL
workflow", "StartAt": "StateA",
"States": {
"StateA":
{ "Type":
"Pass",
"Next": "StateB",
Step Functions doesn’t yet "Parameters"
"var":
: {"xyz"
"var":
"xyz"
}
have
}
"StateB":
,
{ "Type":
the concept of a variable "Pass",
"Next":
"var":
"xyz"
"StateC",
}
"Paramet
}, ers": {
"StateC":
"var.$": "var":
{ "Type":
"$.var" "xyz"
"Pass",
"End": true,
"Paramet
ers": {
"var.$":
} "$.var"
} }
}

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Preserving data without the Map
state
Preserving data across
states requires passing
the data into each state:
{
"Comment": "A simple ASL
workflow", "StartAt": "StateA",

has
"States": {
"StateA":
{ "Type":
"Pass",
"Next": "StateB",
Step Functions doesn’t yet "Parameters"
"var":
: {"xyz"
"var":
"xyz"
haveconcept of a
the
} }
"StateB":
,
{ "Type":
variable "Pass",
"Next":
"var":
"xyz"
"StateC",
}
"Paramet
}, ers": {
"StateC":
"var.$": "var":
{ "Type":
"$.var" "xyz"
"Pass",
"End": true,
"Paramet
ers": {
"var.$":
} "$.var"
} }
}

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Step Functions Workflow
Variables
Assign here

Available here

Workflow variables add the ability

to
assign and reference variables in
ASL.

Supported by JSONPath and

JSONata

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Workflow Variable assignment
"Set variables":
{ "Type": "Pass",
"Next": "Query",
"Assign": {
"oldOwner": ”{% $states.input.oldOwner %}",
"newOwner": ”{% $states.input.newOwner %}",
"table": "S12D"
}
}

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Workflow Variable reference
"Query": { "Type": Syntax:
"Task",
"Parameters": { $<variable_name>
"TableName": ”{% $table %}",
"IndexName": "OwnerIndex",
"KeyConditionExpression": "#owner = :owner",
"Limit": 10,
"ExpressionAttributeNames": {"#owner": "owner"},
"ExpressionAttributeValues": {
":owner": {"S": ”{% $oldOwner %}"}
}
},
...
}

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Workflow Variable reference
"Query": { "Type": Syntax:
"Task",
"Parameters": { $<variable_name>
"TableName": ”{% $table %}",
"IndexName": "OwnerIndex",
"KeyConditionExpression": "#owner = :owner",
"Limit": 10,
"ExpressionAttributeNames": {"#owner": "owner"},
"ExpressionAttributeValues": {
":owner": {"S": ”{% $oldOwner %}"}
}
},
...
}

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Workflow evaluation order
S U C C E S S

State input

Previou
$states.inp
InputPat s
ut
h variable
Parameter s
s
Task or
Assig Variable
sub- n s
workflow
ResultSelect
or

ResultPath

OutputPath

State
© 2024, Amazon Web Services, Inc. or its affiliates. All rights output
reserved.
State workflow variable evaluation
order
F A I L U R E

State input

Previou
$states.inp
InputPat s
ut
h variable
Parameter s
s
Task or Failure
catch
Assig Variable
sub- n
block s
workflow

ResultPat
h

State
© 2024, Amazon Web Services, Inc. or its affiliates. All rights output
reserved.
Workflow evaluation order
S U C C E
S S

State
input

Argument
s
Task or
sub-
workflow Output Variables

State
output

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Workflow Variable
scope
Assig
n

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Workflow Variable
scope
Not
available

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Workflow Variable
scope
Assig
n

Availabl
e

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Workflow Variable
scope
Assig
n

Availabl
e

Paralle Outer
l scope

Child Availabl Availabl

scope e e

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Workflow Variable
scope
Assig
n

Availabl
e

Inline Outer
Map scope
Available
Child Availabl
scope e

Availabl
e

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Workflow Variable
scope
Assig
n

Availabl
e

Distributed Outer
Map scope
Not
available
Child Not
availabl
scope
e

Not
availabl
e

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Workflow Variable
scope

Paralle
l

Child Assig
scope n Outer
scope

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Workflow Variable
scope Sibling
scopes

Parallel

Not
Child Assig
availabl
scope n Outer
e
scope
Availabl
Not
e
availabl
e

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Workflow Variable
scope Sibling
scopes

Parallel

Not
Child Assig
availabl
scope n Outer
e
scope
Availabl
Not
e
availabl
e

Not
available

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Step Functions first

? Which workflow type?

Standard Express

Managing Reducing cost

failures Improved data flow

Managing failures Step Functions at scale

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
“fails, all
Everythi
the
ng
time.”
Dr. Werner
Vogels VP and
CTO Amazon.com

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Catc
h

Catch

Catc

The “saga”
h

continues
Use inherent error handling to roll back
sequential
system failures for long-running transactions

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Book hotel

Catc
h
Book flight
Catc
h Success
Book case
car Catc

The “saga”
h
Cancel
car

Cancel
flight
continues
Use inherent error handling to roll back
Cancel
hotel sequential
system failures for long-running transactions
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Book hotel

Catc
h
Book flight
Catc
h Failure
Book cases
car Catc

The “saga”
h
Cancel
car

Cancel
flight
continues
Use inherent error handling to roll back
Cancel
hotel sequential
system failures for long-running transactions
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Book hotel

Catc
h
Book flight
Catc
h Failure
Book cases
car Catc

The “saga”
h
Cancel
car

Cancel flight continues

Use inherent error handling to roll back
Cancel
hotel sequential
system failures for long-running transactions
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Book hotel

Catc
h
Book flight
Catc
h Failure
Book cases
car Catc

The “saga”
h
Cancel
car

Cancel
flight
continues
Use inherent error handling to roll back
Cancel
hotel sequential
system failures for long-running transactions
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Get
circuit
status

Is
? circuit
closed
?
$.CircuitStatus==“OPE
N” Run

Catc
h
Lambd
a The “circuit breaker”
Update
Prevent a caller service from retrying another callee
circuit service call that has previously caused repeated
status
timeouts or failures
Circui Circui
X t t
open close
d
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Get
circuit
status

Is
? circuit
closed
?
Success
$.CircuitStatus==“OPE cases
N” Run

Catc
h
Lambd
a The “circuit breaker”
Update
Prevent a caller service from retrying another callee
circuit service call that has previously caused repeated
status
timeouts or failures
Circui Circui
X t t
open close
d
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Get
circuit
status

Is
? circuit
{“CircuitStatus”:“CLOSE
D”}
closed
?
Success
$.CircuitStatus==“OPE cases
N” Run

Catc
h
Lambd
a The “circuit breaker”
Update
Prevent a caller service from retrying another callee
circuit service call that has previously caused repeated
status
timeouts or failures
Circui Circui
X t t
open close
d
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Get
circuit
status

Is
? circuit
{“CircuitStatus”:“CLOSE
D”}
closed
?
Success
$.CircuitStatus==“OPE cases
N” Run

Catc
h
Lambd
a The “circuit breaker”
Update
Prevent a caller service from retrying another callee
circuit service call that has previously caused repeated
status
timeouts or failures
Circui Circui
X t t
open close
d
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Get
circuit
status

Is
? circuit
closed
?
Success
$.CircuitStatus==“OPE cases
N” Run

Catc
h
Lambd
a The “circuit breaker”
Update
Prevent a caller service from retrying another callee
circuit service call that has previously caused repeated
status
timeouts or failures
Circui Circui
X t t
open close
d
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Get
circuit
status

Is
? circuit
{“CircuitStatus”:“CLOSE
D”}
closed
?
Success
$.CircuitStatus==“OPE cases
N” Run

Catc
h
Lambd
a The “circuit breaker”
Update
Prevent a caller service from retrying another callee
circuit service call that has previously caused repeated
status
timeouts or failures
Circui Circui
X t t
open close
d
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Get
circuit
status

Is
? circuit
{“CircuitStatus”:“CLOSE
D”}
closed
?
Success
$.CircuitStatus==“OPE cases
N” Run

Catc
h
Lambd
a The “circuit breaker”
Update
Prevent a caller service from retrying another callee
circuit service call that has previously caused repeated
status
timeouts or failures
Circui Circui
X t t
open close
d
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Get
circuit
status

Is
? circuit
closed
?
$.CircuitStatus==“OPE
N” Run

Catc
h
Lambd
a The “circuit breaker”
Update
Prevent a caller service from retrying another callee
circuit service call that has previously caused repeated
status
timeouts or failures
Circui Circui
X t t
open close
d
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Get
circuit
status

Is
? circuit
{“CircuitStatus”:“OPE
N”}
closed
?
$.CircuitStatus==“OPE
N” Run

Catc
h
Lambd
a The “circuit breaker”
Update
Prevent a caller service from retrying another callee
circuit service call that has previously caused repeated
status
timeouts or failures
Circui Circui
X t t
open close
d
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Get
circuit
status

Is
? circuit
{“CircuitStatus”:“OPE
N”}
closed
?
$.CircuitStatus==“OPE
N” Run

Catc
h
Lambd
a The “circuit breaker”
Update
Prevent a caller service from retrying another callee
circuit service call that has previously caused repeated
status
timeouts or failures
Circui Circui
X t t
open close
d
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Learning from
failures

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Learning from failures
Star
t

Invoke Lambda

Process .ts files

Organize object
to .mp4
Persist metadata

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Learning from failures
Star
t

Invoke Lambda

Process .ts files

Organize object
to .mp4
Persist metadata

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Learning from failures
Star
t

Invoke Lambda

Catc
h

Run ECS task

Process .ts files

Organize object
to .mp4
Persist metadata
Identify points of failure in your
workflows and enhance
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Learning from failures
Star
t

Invoke Lambda

Catc
h

Run ECS task

Process .ts files

Organize object
to .mp4
Persist metadata
Identify points of failure in your
workflows and enhance
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Is video duration more

? than 20 minutes?

YE N
S O

Run ECS task Invoke Lambda

The compute
Organize object
maestro
Persist metadata
Orchestrate multiple compute
substrates in a single workflow
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Multiple serverless compute
substrates Star
t

Is video duration more

? than 20 minutes?
• Convert multiple .ts files into a
YE N single .mp4
S O
• Amazon ECS for longer-running tasks
Run ECS task Invoke Lambda
• AWS Lambda for short duration

• Same code (minus the Lambda event

Organize object
handler)
- exports.handler = async (event) => {

Persist metadata + async function main() {

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Redrive a workflow from failed task
Star
t

Is video duration
? more than 20
minutes?
YE N
S O

No need to wait for long-running tasks to Run ECS task Invoke Lambda
rerun
No cost incurred for previous
successfully completed tasks Organize object

Billed only for each state transition after the Persist metadata
failed state Redriv
e Emit:
Processing
completed

En
d

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Test 1 state at a time
Star
t

Is video duration
? more than 20
minutes?
YE N
S O

Run ECS task Invoke Lambda

Organize object

Persist metadata

Emit:
Processing
completed

En
d

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Test 1 state at a time
Start

Is video duration more

? than 20 minutes?

YES

No need to run the entire workflow to test a single NO

state with the TestState API

Run ECS task

Invoke Lambda

Persist metadata
Organize object

Emit: Processing completed

End
Test 1 state at a time
START

Is video duration more

? than 20 minutes?

YES

NO

No need to run the

entire workflow to test
a single state

Run ECS task

Invoke Lambda
with the Test API.

Organize object

Persist metadata
 Step Functions first

? Which workflow type?

Standard Express

Step Functions at Reducing cost

scale
Managing failures Step Functions at scale

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Parallel invocations –
lots of independent
workflows doing their
own thing

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Parallel
invocations
Same ASL definition executing at the same time
with different data

Each process is independent

Failure in one process doesn’t impact another

For example, remediation workflow for noncompliant

resource Limitation: account quote (mostly soft limits)

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 {“uid”:”1
”}
Parallel state Star
t

executes a fixed Parallel

state

number of branches Look up ZIP

Look
up

in parallel
numbe
r

Final state

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 {“uid”:”1
”}

Parallel Star
t

state
Executes multiple branches of steps Parallel
state
using the same input Look
Look up ZIP
up
Map state output is an array with numbe
r
1 element for each branch result 9021 1
0

Final state [ 90210,

1]

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 {“uid”:”1
”}

Parallel Star
t

state
Wait until all branches terminate (reach a
terminal state) before processing the next Parallel
state
state Look
Look up ZIP
up
If any branch fails because of an unhandled numbe
r
error or by transitioning to a Fail state, the 9021 Erro
entire parallel state is considered to have 0 r
failed, and all its branches are stopped

Final state Erro

r

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 {“uid”:”1
”}

Parallel Star
t

state
Catch errors to allow other branches to
Parallel
state
Look Look up
complete up phone
addres number
s Erro
All execution results are aggregated into 9021 r
the final state 0
Catch

Final state
[ 90210,
Error ]
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Dynamic
parallelism
Execute the same steps for multiple entries of an
Map
state

input array Put

MaxConcurrency controls the number of iterations Remov

e
that will messag
run in parallel e
Publish

Set to 1 for single-threaded

integrations; set to 0 for maximum
parallelism
En
d

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

GetNewBlogPosts

Map state
For each
post
Scrape
metadat
a

SaveToGitHub

The big “fan”

Rebuild
(fan-out)
fronten
d Deliver a workload to multiple destinations

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

GetNewBlogPosts

Map state
For each
post
Scrape
metadat
a

SaveToGitHub

The big “fan”

Rebuild
(fan-out)
fronten
d Deliver a payload to multiple destinations

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Scans RSS feed at

GetNewBlogPosts
https://aws.amazon.com/blogs/compute/feed
Returns an array of new compute blog post URLs
Map state
For each
post
Scrape
metadat
a

SaveToGitHub

The big “fan”

Rebuild
(fan-out)
fronten
d Deliver a payload to multiple destinations

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Scans RSS feed at

GetNewBlogPosts
https://aws.amazon.com/blogs/compute/feed
Returns an array of new compute blog post
URLs
{
Map state "body": [
"https://aws.amazon.com/blogs/compute/integrating-amazon-memorydb-for-redis-with-java-based-aws-lambda/",
For each "https://aws.amazon.com/blogs/compute/lifting-and-shifting-a-web-application-to-aws-serverless-part-2/",
post “https://aws.amazon.com/blogs/compute/lifting-and-shifting-a-web-application-to-aws-serverless-part-1” ,
……
]
Scrape }
metadat
a

SaveToGitHub

The big “fan”

Rebuild
(fan-out)
fronten
d Deliver a payload to multiple destinations

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Scans RSS feed at

GetNewBlogPosts
https://aws.amazon.com/blogs/compute/feed
Returns an array of new compute blog post
URLs
{
Map state "body": [
"https://aws.amazon.com/blogs/compute/integrating-amazon-memorydb-for-redis-with-java-based-aws-lambda/",
For each "https://aws.amazon.com/blogs/compute/lifting-and-shifting-a-web-application-to-aws-serverless-part-2/",
post “https://aws.amazon.com/blogs/compute/lifting-and-shifting-a-web-application-to-aws-serverless-part-1” ,
……
]
Scrape }
metadat
a {
“title”:”Lifting and shifting a web application to AWS Serverless: Part 1”, “Author”:”Marcia
Villalba”,
SaveToGitHub “desc”:” Customers migrating to the cloud often want to get the benefits of serverless architecture.”,
“link”:https://aws.amazon.com/blogs/compute/lifting-and-shifting-a-web-application-to-aws-serverless-part-1,
“thumbnail”:”/thumb1.png”,

The big “fan” -

“tags”:[”serverless”,”stepFunctions”]
"date": "Sep 14, 2022",
}

Rebuild
(fan
Deliver a payload to multiple
destinations out
fronten
d
)
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Scans RSS feed at

GetNewBlogPosts
https://aws.amazon.com/blogs/compute/feed
Returns an array of new compute blog post
URLs
{
Map state "body": [
"https://aws.amazon.com/blogs/compute/integrating-amazon-memorydb-for-redis-with-java-based-aws-lambda/",
For each "https://aws.amazon.com/blogs/compute/lifting-and-shifting-a-web-application-to-aws-serverless-part-2/",
post “https://aws.amazon.com/blogs/compute/lifting-and-shifting-a-web-application-to-aws-serverless-part-1” ,
……
]
Scrape }
metadat
a {
“title”:”Lifting and shifting a web application to AWS Serverless: Part 1”, “Author”:”Marcia
Villalba”,
SaveToGitHub “desc”:” Customers migrating to the cloud often want to get the benefits of serverless architecture.”,
“link”:https://aws.amazon.com/blogs/compute/lifting-and-shifting-a-web-application-to-aws-serverless-part-1,
“thumbnail”:”/thumb1.png”,

The big “fan” -

“tags”:[”serverless”,”stepFunctions”]
"date": "Sep 14, 2022",
}

Rebuild
(fan
Deliver a payload to multiple
destinations out
fronten
d
)
End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

GetNewBlogPosts

Map state
For each
post
Scrape
metadat
a

SaveToGitHub The scatter-gather

Broadcasts a payload to multiple destinations
Rebuild (scatter) and then aggregates the responses
fronten
d
back for the next steps (gather)

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

GetNewBlogPosts

Map state
For each
post
Scrape
metadat
a

SaveToGitHub The scatter-gather

Broadcasts a payload to multiple destinations
Rebuild (scatter) and then aggregates the responses
fronten
d
back for the next steps (gather)

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

GetNewBlogPosts

Map state
For each
post
Scrape [
{
metadat “title”:”Lifting and shifting a web application to AWS Serverless: Part 1”, “Author”:”Marcia
a Villalba”,
“desc”:” Customers migrating to the cloud often want to get the benefits of serverless architecture.”,
“link”:https://aws.amazon.com/blogs/compute/lifting-and-shifting-a-web-application-to-aws-serverless-part-1,
“thumbnail”:”/thumb1.png”,
“tags”:[”serverless”,”stepFunctions”]
"date": "Sep 14, 2022",

The scatter-
}
,
SaveToGitHu {
“title”:”Lifting and shifting a web application to AWS Serverless: Part 2”, “Author”:”Marcia
b Villalba”,

gather
“desc”:” Customers migrating to the cloud often want to get the benefits of serverless architecture part 2.”,
ons
“link”:https://aws.amazon.com/blogs/compute/lifting-and-shifting-a-web-application-to-aws-serverless-part-2,
“thumbnail”:”/thumb1.png”,
Rebuild “tags”:[”serverless”,”stepFunctions”]
"date": "Sep 16, 2022",
(scatter) or
fronten
Broadcasts a payload to multiplethe next
}
d ]

destinati and then aggregates the

End
responses back f steps (gather)

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Distributed Map
state
Coordinate large-scale parallel workloads

Iterate over millions of Amazon S3 objects, such as logs,

images, or
JSON or CSV files

Up to 10,000 parallel executions

Invoke Lambda or Amazon ECS/AWS Fargate

technology for large-scale on-demand serverless
compute

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Calculate chunks
A serverless GIF
generator
Convert an .mp4 from
SaveListToS3 Amazon S3 into multiple
GIFs for timeline scrubbing
Distributed Map
state
Item source

Generate GIF

Export
location

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 {
"key": ”Movie1l.mp4",
Star "bucket": ”SourceBucket"
}
t

Calculate chunks Using a Distributed Map state for this

workload
• Not limited to 40 concurrent executions
SaveListToS3 • Imports and exports directly
from/to Amazon S3
Distributed Map
state
Item source

Generat
e GIF

Export
location

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Calculate Using a Distributed Map state for this

[
chunks {
"Key":
•
workload
Not limited to 40 concurrent
"jumpstartingsDevWFFinal.mp4",
"start": 0,
executions
SaveListToS "end": 29,
"length": 1592.882958, • Imports and exports directly
3 "tsCreated": 1665052810845
}, from/to Amazon S3
Distributed Map {
"Key":
state
"jumpstartingsDevWFFinal.mp4",
Item "start": 30,
source "end": 59,
"length": 1592.882958,
Generat "tsCreated":
e GIF 1665052810845
},
……
]

Export
location

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Calculate chunks Using a Distributed Map state for this

•
workload
Not limited to 40 concurrent
executions
SaveListToS3 • Imports and exports directly
from/to Amazon S3
Distributed Map
state {
Item source Bucket” SourceBucket”,
Key:”movie1.json”
}
Generat
e GIF

Export
location

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Calculate chunks Using a Distributed Map state for this

•
workload
Not limited to 40 concurrent
executions
SaveListToS3 • Imports and exports directly
from/to Amazon S3
Distributed Map
state
Item source
{
"Key": " Movie1l.mp4 ",
Generat "start": 0,
e GIF "end": 29,
"length": 1592.882958,
"tsCreated":
} 1665052810845

Export
location

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Calculate chunks Using a Distributed Map state for this

•
workload
Not limited to 40 concurrent
executions
SaveListToS3 • Imports and exports directly
from/to Amazon S3
Distributed Map
state
Item source

Generat
e GIF

{
“Bucket”:”DestinationBucket”
Export body:“base64 encoded gif”
location }

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Calculate chunks Using a Distributed Map state for this

•
workload
Not limited to 40 concurrent
executions
SaveListToS3 • Imports and exports directly
from/to Amazon S3
Distributed Map
state
Item source

Generat
e GIF

Export
location

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Star
t

Calculate chunks Using a Distributed Map state for this

•
workload
Not limited to 40 concurrent
executions
SaveListToS3 • Imports and exports directly
from/to Amazon S3
Distributed Map
state
Item source

Generat
e GIF

Export
location

End

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
 Step Functions Distributed
Map
Each iteration runs as a
separate child
workflow with its own
execution history and
payload limits

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Without Step Functions Distributed
Map state
P R O C E S S A
R O M
L A R G E
D Y N A M O D B
D A T A S E T F

An application polls GitHub each

1 day to persist traffic data for a
number of repositories
GitHu API Gateway
b DynamoDB

A Map state concurrently processes

each data item 2

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Without Step Functions Distributed
Map state
P R O C E S S A
R O M
L A R G E
D Y N A M O D B
D A T A S E T F

Scan all data

Map
state
Invoke
Lambd
a

Publish
A separate application polls GitHub each day to
persist view data for a number of repositories

A Map state to concurrently process each

En
d data item

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Without Step Functions Distributed
Map state
P R O C E S S A
R O M
L A R G E
D Y N A M O D B
D A T A S E T

[
F

{
"Count": 2900,
"Items": [
{
"Views": {
"S": "8"
}, "SK":
Scan all {
data "S":
"2023
-02-
11T0
Map 0:00:
state 00Z"
… }, "PK":
Invoke ] {
Lambd "S":
"Repo
a s-
aws-
sampl
Publish es/aw
s- A separate application polls GitHub each day to
step-
functi
ons-
persist view data for a number of repositories
twitch
-
series
" A Map state to concurrently process each data
En }
d } item
,

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Step Functions Distributed
Map state
E A C HI T E R A T I O N
W O R K F L O W
R U N SA S A
E X E C U T I O N
C H I L D

Scan
for
parent
For each Distributed Map
parent

Scan
for
children

The payload
For each Inline
child Map

Organize data

Write to
divider
reporting
SaaS
Overcome payload limits but break workloads
down into multiple child workflows

En
d

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Step Functions Distributed
Map state
E A C HI T E R A T I O N
W O R K F L O W
R U N SA S A
E X E C U T I O N
[
C H I L D

"aws-samples/serverless-patterns", "aws-
Scan samples/serverless-coffee",
for "aws-samples/php-examples-for-aws-lambda",
"aws-samples/contact-form-processing-with-synchronous-express-workflows", "aws-
parent samples/aws-serverless-workshop-innovator-island",
"awslabs/aws-lambda-powershell-runtime", "aws-
For each Distributed Map samples/sessions-with-aws-sam",
parent "aws-samples/eventbridge-scheduler-blog-prerequisites", "aws-
samples/amazon-s3-to-aws-lambda-examples",
Scan "aws-samples/decoupling-with-amazon-eventbridge", "aws-
samples/aws-lambda-efs-samples”
for ….
children ]

The payload
For each Inline
child Map

Organize data

Write to
divider
reporting
SaaS
Overcome payload limits but break workloads
down into multiple child workflows

En
d

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Step Functions Distributed
Map state
E A C HI T E R A T I O N
W O R K F L O W
R U N SA S A
E X E C U T I O N
C H I L D

Scan
for
parent
For each Distributed "aws-samples/serverless-patterns”
parent Map

Scan
for
children

The payload
For each Inline
child Map

Organize data

Write to
divider
reporting
SaaS
Overcome payload limits but break workloads
down into multiple child workflows

En
d

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Step Functions Distributed
Map state
E A C HI T E R A T I O N
W O R K F L O W
R U N SA S A
E X E C U T I O N
C H I L D

Scan
for
parent
For each Distributed Map
parent

Scan {
"Count": 90,
for "Items": [
children {
"Views": {

The ivid
For each Inline "S": "8"
child Map }, "SK":
{
"S":
Organize data "2023

payload d er
-02-
11T0
0:00:
00Z" reak workloads
Write to … }, "PK":
reporting …
}
{
"S":
down
SaaS "Repo
s- Overcome payload limits
aws-
sampl
es/aw
but b into multiple child
s-
step-
workflows
functi
En ons-
d twitch
-
series
"
}
© 2024, Amazon Web Services, Inc. or its affiliates. All rights }
reserved. ,
The serverless
workflows
collection

Discover, deploy, and

share Step Functions
workflows

https://s12d.com/workflo
ws

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Discover workflow examples
1. Workflow 2. IaC 3. ASL
visual template definition

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Discover workflow examples
1. Workflow 2. IaC 3. ASL
visual template definition

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Resources

https://s12d.com/api402-
24
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Continue your AWS serverless
learning
Learn at Increase Earn AWS
your own your serverless
pace knowledg badges
e

Expand your Use AWS Ramp-Up Demonstrate your

serverless skills with Guides knowledge by
learning plans on to build your achieving digital
AWS Skill Builder serverless badges
knowledge

https://s12d.com/serverless-
learning
© 2024, Amazon Web Services, Inc. or its affiliates. All rights
reserved.
Check out these other
sessions
API201: The AWS Step Functions workshop
Monday, December 2 | 3:00 PM | Caesars Forum, Level 1, Summit 228

GAM303: Upgrade your game storyboards with multimodal

Wednesday,
prompt December 4 | 12:30 PM | MGM Grand, Level 1,
chaining
Grand 123

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.
Thank Please complete the
session survey in the
mobile app

you!
Eric Johnson
[email protected]
m
@edjgeek
https:/
/www.linkedin.co
m/in/singledigit/

© 2024, Amazon Web Services, Inc. or its affiliates. All rights

reserved.