Design and Simulate Scenarios for

Automated Driving Applications

Linghui Zhang, MathWorks Mark Corless, MathWorks

1
Share the EXPO experience
#MATLABEXPO

2
Develop Automated Driving Applications
with MATLAB, Simulink, & RoadRunner

Virtual Worlds Algorithms Software

Scenes Sensors Detection Localization Tracking Code
C/C++
GPU HDL

Scenarios Dynamics Planning Decision & Controls Architectures

AUTOSAR
ROS DDS

Development Platform
Analyze Simulate Design Deploy Integrate Test

3
 Develop virtual worlds for automated driving applications

Scenes Dynamics

Tracking &
Scenarios Planning Controls Metrics
Fusion

Sensors

Highway Lane Change Updated

4
 Develop algorithms for automated driving applications

Scenes Scenes
Dynamics
Tracking & Metrics
Scenarios Scenarios Planning Controls
Fusion
Metrics

Sensors

Surround Vehicle Sensor Fusion Highway Lane Change Planner and Controller
5
 Develop software for automated driving applications

Scenes Scenes
Dynamics
Tracking & Metrics
Scenarios Scenarios Planning Controls
Fusion
Metrics

Sensors Code Code Code

Surround Vehicle Sensor Fusion Highway Lane Change Planner and Controller
6
 Develop scenarios for automated driving applications

Planning

Code

Set map-aware vehicle paths, Highway Lane Change Planner with

scenario logic, conditions and goals RoadRunner Scenario
7
Develop Scenarios for Automated Driving Applications
with RoadRunner Scenario

Design and Simulate Scenarios

Interface with OpenSCENARIO

Simulate with MATLAB, Simulink, and CARLA

8
Develop Scenarios for Automated Driving Applications
with RoadRunner Scenario

Design and Simulate Scenarios

Interface with OpenSCENARIO

Simulate with MATLAB, Simulink, and CARLA

9
 Interactively design scenes with RoadRunner

▪ Author realistic roads

and intersections
▪ Import/export
OpenDRIVE
▪ Import HD maps
▪ Import Geographic
Information System
(GIS) files
▪ Export to common
driving simulation
environments

RoadRunner, RoadRunner Asset Library, RoadRunner Scene Builder

10
 Interactively design scenarios with RoadRunner Scenario

▪ Add various vehicles

▪ Author trajectories
▪ Specify actions and logic
▪ Parameterize variations

Scenario Edit Tool

RoadRunner Scenario
11
Simulate map-aware paths and scenario logic
Follow lanes when no path is specified Speed actions

Lane change actions Lateral offset actions

12
 Design actor paths and trajectories

▪ Cubic interpolation
▪ Clothoid interpolation
▪ EuroNCAP
(clothoid-arc-clothoid)

Route Timing Tool

RoadRunner Scenario
13
 Programmatically vary scenario parameters

MATLAB, gRPC, and

Command-line APIs
• Define scenario variables in
editor
• Set variables programmatically
from API
• Run simulations
• Export to OpenSCENARIO

Programmatic Scenario Interfaces

RoadRunner Scenario
14
Develop Scenarios for Automated Driving Applications
with RoadRunner Scenario

• Design paths and scenario logic

Design and Simulate Scenarios • Relocate scenarios to different scenes
• Programmatically vary parameters

Interface with OpenSCENARIO

Simulate with MATLAB, Simulink, and CARLA

15
 Export scenarios to OpenSCENARIO V1.x and V2.0

OpenSCENARIO
V1.x

https://github.com/esmini/esmini

OpenSCENARIO
V2.0

MathWorks is an ASAM Member

and actively participates in the
OpenSCENARIO 2.0
Implementers Forum
Export to ASAM OpenSCENARIO
RoadRunner Scenario
16
 Import and edit trajectories from OpenSCENARIO V1.x

• Import trajectories from

OpenSCENARIO V1.x
• Interactive edit
trajectories
• Relocate trajectories in
different scenes
• Extract the path for use
with scenario logic

Import Trajectories from ASAM OpenSCENARIO Files

RoadRunner Scenario
17
 Migrate trajectories from Driving Scenario Designer (DSD)
to RoadRunner Scenario
Automated Driving RoadRunner
Toolbox Scenario

Design actor Simulate

trajectories scenario

Export Import
trajectories trajectories

OpenSCENARIO 1.0

18
Develop Scenarios for Automated Driving Applications
with RoadRunner Scenario

• Design paths and scenario logic

Design and Simulate Scenarios • Relocate scenarios to different scenes
• Programmatically vary parameters

• Export to OpenSCENARIO v2.0

Interface with OpenSCENARIO • Export to OpenSCENARIO v1.x
• Import trajectories from OpenSCENARIO v1.0

Simulate with MATLAB, Simulink, and CARLA

19
Simulate scenarios with actor behaviors in multiple simulators

RoadRunner Scenario connects with

actors in MATLAB, Simulink, and CARLA

Actors can read scenario states

• Action commands (path, speed, lane
change, lateral offset)
• Pose and velocity of all actors in the
scenario
• Dimensions of all actors
• Map lanes and lane boundaries

Actors write scenario states

• Their pose and velocity for each scenario Built-in
simulation step Actors

20
 Design actor behaviors in MATLAB

Interface with RoadRunner scenario through

MATLAB APIs with Automated Driving Toolbox

▪ Connect to scenario simulation

▪ Read world state from the scenario
▪ Read actor specific supervisory actions from
scenario
▪ Write actor states to the scenario
▪ Report errors, warnings to the scenario

Simulate RoadRunner Scenarios with Actors Modeled in MATLAB

RoadRunner Scenario, Automated Driving ToolboxTM
21
 Simulate with speed action follower designed in MATLAB

▪ Design speed action follower behavior in

MATLAB
▪ Associate MATLAB behavior with actor
in RoadRunner Scenario
▪ Simulate and visualize results

Speed Action Follower with RoadRunner Scenario

RoadRunner Scenario, Automated Driving ToolboxTM
22
 Design actor behaviors in Simulink

Interface with RoadRunner Scenario using

blocks from Automated Driving Toolbox

▪ RoadRunner Scenario
– Establish a model’s interface with scenario

▪ RoadRunner Scenario Reader

– Read the world state: Actor pose, velocity,
color, supervisory actions

▪ RoadRunner Scenario Writer

– Write an actor’s state to scenario
– Report errors, warnings to scenario

Simulate RoadRunner Scenarios with Actors Modeled in Simulink

RoadRunner Scenario, Automated Driving ToolboxTM
23
 Simulate with trajectory follower designed in Simulink

Path Reference
Action Pose
Vehicle
Controls Dynamics New
Speed Action Self
Pose
Actor Poses

▪ Explore built-in trajectory following

behavior with linear velocity
▪ Design actor behavior in Simulink which
includes controls and dynamics
▪ Simulate and compare results

Trajectory Follower with RoadRunner Scenario

RoadRunner Scenario, Automated Driving ToolboxTM
24
 Simulate with lane change planner designed in Simulink

Reference
New Self
Path
Pose
All Actor Planner
Poses

Self Pose

▪ Design ego actor to implement planner

▪ Define trajectories and logic for target actors
▪ Visualize possible and selected ego trajectories

Highway Lane Change Planner with RoadRunner Scenario

RoadRunner Scenario, Automated Driving ToolboxTM, Navigation ToolboxTM
25
 Simulate with actor behaviors designed in CARLA
RoadRunner CARLA
Scenario

Design scene Build game

Design Design
scenario CARLA actor

Simulate Simulate
scenario actor

Cosimulate Actors with CARLA

RoadRunner Scenario
26
Develop Scenarios for Automated Driving Applications
with RoadRunner Scenario

• Design paths and scenario logic

Design and Simulate Scenarios • Relocate scenarios to different scenes
• Programmatically vary parameters

• Export to OpenSCENARIO v2.0

Interface with OpenSCENARIO • Export to OpenSCENARIO v1.x
• Import trajectories from OpenSCENARIO v1.0

• Author actor behaviors in MATLAB

Simulate with
• Author actor behaviors in Simulink
MATLAB, Simulink, and CARLA • Author actor behaviors in CARLA

27
Partner with MathWorks to extend scenario workflows

Extend verification workflows

with MATLAB & Simulink

RoadRunner Scenario

Extend actor design workflows

with MATLAB & Simulink

Engage with MathWorks engineers through proof-of-concept projects or Consulting Services to

extend scenario workflows
28
 Partner with MathWorks to extend scenario workflows

Generate Generate Automate

Verification
scenarios from scenario testing
workflows recorded data variations scenarios

RoadRunner Scenario

Integrate Extend actor design workflows

Integrate Integrate
planning & with MATLAB & Simulink
sensors external C/C++,
Actor design Python, ROS
controls code
Workflows

Engage with MathWorks engineers through proof-of-concept projects or Consulting Services to

extend scenario workflows
29
 Partner with MathWorks to extend workflows for tractor trailer

Engage with MathWorks

engineers through
proof-of-concept projects or
Consulting Services to
extend scenario workflows

30
Develop Scenarios for Automated Driving Applications
with RoadRunner Scenario

• Design paths and scenario logic

Design and Simulate Scenarios • Relocate scenarios to different scenes
• Programmatically vary parameters

• Export to OpenSCENARIO v2.0

Interface with OpenSCENARIO • Export to OpenSCENARIO v1.x
• Import trajectories from OpenSCENARIO v1.0

• Author actor behaviors in MATLAB

Simulate with
• Author actor behaviors in Simulink
MATLAB, Simulink, and CARLA • Author actor behaviors in CARLA

31
Thank you

© 2022 The MathWorks, Inc. MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See mathworks.com/trademarks
for a list of additional trademarks. Other product or brand names may be trademarks or registered trademarks of their respective holders.

32