Autonomous UAV Development and Evaluation

with MATLAB® and Simulink®

Julia Antoniou Andrew Grabowski

0
Autonomous UAV Development and Evaluation
MATLAB® Simulink®

▪ Integrated workflows enabled

by MATLAB and Simulink

▪ Tools to design UAV systems

and autonomous applications

▪ Select appropriate methods for

your UAV development tasks

▪ Evaluating systems through

closed-loop simulations with
sensor models
1
Autonomous UAV Development and Evaluation
MATLAB® Simulink®

▪ Integrated workflows enabled

by MATLAB and Simulink

▪ Tools to design UAV systems

and autonomous applications

▪ Select appropriate methods for

your UAV development tasks

▪ Evaluating systems through

closed-loop simulations with
sensor models
2
Increase in autonomous UAV usage

Mapping & Inspections & Delivery & Security &

Surveying Monitoring Transport Defense
3
Challenges in developing autonomous UAV systems & applications

Complexity of advanced
autonomous algorithms

Need of end-to-end workflows

Ensuring system quality and

reducing flight risk

4
Solutions for developing autonomous UAV systems & applications

Robust tools and features for

designing and testing UAV systems
and algorithms

Integrated development environment

that covers development from ideas
to production

Extensive verification and validation

tools to evaluate design quality
through virtual testing

5
Integrated workflows for developing UAV applications

MATLAB® Simulink®
System Architecture

6
Integrated workflows for developing UAV applications

MATLAB® Simulink®
System Architecture
Model UAV

7
Integrated workflows for developing UAV applications

MATLAB® Simulink®
System Architecture
Model UAV

Design Algorithms
Perception
Planning &
Decision
Control

8
 Integrated workflows for developing UAV applications

MATLAB® Simulink®
System Architecture
Model UAV
Verification & Validation

Design Algorithms
Perception
Planning &
Decision
Control

DO-178

9
 Integrated workflows for developing UAV applications

MATLAB® Simulink® Simulate with Sensor Models

System Architecture
Model UAV
Verification & Validation

Cuboid Gazebo Unreal Engine

Design Algorithms
Perception
Planning &
Decision
Control

DO-178

10
 Integrated workflows for developing UAV applications

MATLAB® Simulink® Simulate with Sensor Models

System Architecture
Model UAV
Verification & Validation

Cuboid Gazebo Unreal Engine

Deploy to Hardware

Design Algorithms Implement

Perception
Planning & PX4® NVIDIA® Jetson®
Decision
Control

DO-178

11
 Integrated workflows for developing UAV applications

MATLAB® Simulink® Simulate with Sensor Models

System Architecture
Model UAV
Verification & Validation

Cuboid Gazebo Unreal Engine

Deploy to Hardware

Design Algorithms Implement

Perception
Planning & PX4® NVIDIA® Jetson®
Decision
Control
Connect
DO-178
Connect

UAV Ground Control Station

12
 Integrated workflows for developing UAV applications

MATLAB® Simulink® Simulate with Sensor Models

System Architecture
Model UAV
Verification & Validation

Cuboid Gazebo Unreal Engine

Deploy to Hardware

Design Algorithms Implement

Perception
Planning & PX4® NVIDIA® Jetson®
Decision
Control
Connect
DO-178 Analyze Data
Connect

UAV Ground Control Station

13
 Integrated workflows for developing UAV applications

MATLAB® Simulink® Simulate with Sensor Models

System Architecture
Model UAV
Verification & Validation

▪ System Composer™ for Cuboid Gazebo Unreal Engine

designing and analyzing Deploy to Hardware

system and software
Design Algorithms
architecture Implement
Perception
Planning & PX4® NVIDIA® Jetson®
Decision
Control
▪ Simulink integration and Connect
requirementAnalyze
DO-178
allocation
Datafor
traceability Connect

UAV Ground Control Station

Link
14
 Integrated workflows for developing UAV applications

MATLAB® Simulink® Simulate with Sensor Models

System Architecture
Model UAV
Verification & Validation

Cuboid Gazebo Unreal Engine

Deploy to Hardware

Design Algorithms Implement

Perception
Planning & PX4® NVIDIA® Jetson®
Decision
Control
Connect
DO-178 Analyze Data
Connect

UAV Ground Control Station

15
 UAV Plant Modeling: Selecting the appropriate fidelity

High-Fidelity Approximate
Building UAV Programming UAV

Simscape Multibody, Aerospace Blockset, UAV Toolbox 16

 UAV Plant Modeling: Selecting the appropriate fidelity

High-Fidelity Approximate
Building UAV Programming UAV

Link

Simscape Multibody, Aerospace Blockset, UAV Toolbox 17

 UAV Plant Modeling: Selecting the appropriate fidelity

High-Fidelity Approximate
Building UAV Programming UAV

Link Link

Simscape Multibody, Aerospace Blockset, UAV Toolbox 18

 UAV Plant Modeling: Selecting the appropriate fidelity

High-Fidelity Approximate
Building UAV Programming UAV

Link Link Link

Simscape Multibody, Aerospace Blockset, UAV Toolbox 19

 Integrated workflows for developing UAV applications

MATLAB® Simulink® Simulate with Sensor Models

System Architecture
Model UAV
Verification & Validation

Cuboid Gazebo Unreal Engine

Deploy to Hardware

Design Algorithms Implement

Perception
Planning & PX4® NVIDIA® Jetson®
Decision
Control
Connect
DO-178 Analyze Data
Connect

UAV Ground Control Station

20
Autonomous UAV Algorithm Development
Design Algorithms
Perception
Planning &
Decision
Control

Map Data

Endpoint

Startpoint
Planned Path

21
 Autonomous UAV algorithm design with robust capabilities

Perception Planning & Control

Decision
Awareness
Self

Link Link Link

Awareness
Situational

Link Link Link

Sensor Fusion and Tracking Toolbox, Lidar Toolbox, Navigation Toolbox, Computer Vision Toolbox, Deep Learning Toolbox 22
 Autonomous UAV algorithm design with robust capabilities

Perception Planning & Control

Decision

Link Link
Define UAV missions with waypoint UAV motion planning with
and trajectory-following algorithms advanced path planners

UAV Toolbox, Navigation Toolbox 23

 Autonomous UAV algorithm design with robust capabilities

Perception Planning & Control

Decision

Link Link
Trajectory tracking controller with nonlinear Train policies for trajectory generation
model predictive control (MPC) using reinforcement learning algorithms
Model Predictive Control Toolbox, Reinforcement Learning Toolbox 24
 Integrated workflows for developing UAV applications

MATLAB® Simulink® Simulate with Sensor Models

System Architecture
Model UAV
Verification & Validation

Cuboid Gazebo Unreal Engine

Deploy to Hardware

Design Algorithms Implement

Perception
Planning & PX4® NVIDIA® Jetson®
Decision
Control
Connect
DO-178 Analyze Data
Connect

UAV Ground Control Station

25
 Tracking and automating verification and validation activities

Requirements Traceability Test Management & Automation Evaluate Completeness

Link Link Link

Simulink Requirements, Simulink Test, Simulink Coverage, Simulink Check 26

 Example: Automating UAV testing with requirements linking

Automating test execution and evaluation

Requirements linking for traceability

Link
UAV Toolbox, Simulink Requirements, Simulink Test 27
 Integrated workflows for developing UAV applications

MATLAB® Simulink® Simulate with Sensor Models

System Architecture
Model UAV
Verification & Validation

Cuboid Gazebo Unreal Engine

Deploy to Hardware

Design Algorithms Implement

Perception
Planning & PX4® NVIDIA® Jetson®
Decision
Control
Connect
DO-178 Analyze Data
Connect

UAV Ground Control Station

29
 Integrated simulations with sensor models

Cuboid Unreal Engine®

Performance Photorealistic

Rapidly author scenarios and Realistic graphics to test autonomous

generate sensor data algorithms in closed-loop simulations
Link Link
UAV Toolbox 30
 Integrated simulations with sensor models

Cuboid Unreal Engine®

Performance Photorealistic

Rapidly author scenarios and Realistic graphics to test autonomous

generate sensor data algorithms in closed-loop simulations
Link Link
UAV Toolbox 31
 Integrated simulations with sensor models

Cuboid Unreal Engine®

Performance Photorealistic

Rapidly author scenarios and Realistic graphics to test autonomous

generate sensor data algorithms in closed-loop simulations
Link Link
UAV Toolbox 32
 Example: Build 3D map using simulation Lidar point cloud data

Detect loop-closures
Execute simulation Extract and match features
Create pose graph
Obtain sensor data Register and align point cloud
Optimize poses

UAV Toolbox, Lidar Toolbox 33

 Create 3D scenes for UAV simulations

Link Link

Design 3D scenes for simulating and testing autonomous algorithms

RoadRunner, RoadRunner Asset Library, UAV Toolbox Interface for Unreal Engine Projects 34
 Automatic code generation for hardware implementation

MATLAB® Simulink® Simulate with Sensor Models

System Architecture Deploy flight controls to
Model UAV autopilot hardware
Verification & Validation

Cuboid Gazebo Unreal Engine

Link

Deploy to Hardware

Design Algorithms Implement

Perception
Planning & PX4® NVIDIA® Jetson®
Decision
Control
Connect
CPU GPU Deploy autonomous
DO-178 Analyze Data algorithms to onboard
FPGA ROS Connect computers
UAV Link Control Station
Ground

UAV Toolbox, Simulink Coder, Embedded Coder, GPU Coder 35

 Connecting to UAV hardware through MAVLink protocol

MATLAB® Simulink® Simulate with Sensor Models

Compute Board

System Architecture
MAVLink MAVLink

MAVLink
Remote UAV
Model UAV
Autopilot Host Machine
Verification & Validation

UAV Cuboid Gazebo Unreal Engine

MAVLink Connectivity
Deploy to Hardware
Message Blocks

Design Algorithms Implement

Perception
Planning & PX4® NVIDIA® Jetson®
Decision
Control
Connect
DO-178 Analyze Data
Connect
Link
UAV Ground Control Station

UAV Toolbox 36
 Post-flight data analysis

MATLAB® Simulink® Simulate with Sensor Models

Flight Log Analysis Payload Data Analysis
System Architecture
Model UAV
Verification & Validation

Cuboid Gazebo Unreal Engine

Deploy to Hardware

Design Algorithms Implement

Perception
Planning & PX4® NVIDIA® Jetson®
Decision
Control Link Connect
DO-178 Analyze Data
Connect Link Link

UAV Ground Control Station

UAV Toolbox, Computer Vision Toolbox, Deep Learning Toolbox 37

 Integrated Workflows for Developing UAV Applications

MATLAB® Simulink® Simulate with Sensor Models

System Architecture
Model UAV
Verification & Validation

Cuboid Gazebo Unreal Engine

Deploy to Hardware

Design Algorithms Implement

Perception
Planning & PX4® NVIDIA® Jetson®
Decision
Control
Connect
DO-178 Analyze Data
Connect

UAV Ground Control Station

39
 Key Takeaways

Integrated development workflows

from prototyping to productization
with MATLAB and Simulink

Robust tools/features for

autonomous UAV design and
simulations with sensor models
Call To Action:
• Download presentation file
Quality through verification & validation
and investigate linked
examples and pages tools for traceability, test completeness,
• Contact us for to learn and test management/automation
more details or for trials

40
Thank you

© 2021 The MathWorks, Inc. MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See mathworks.com/trademarks
for a list of additional trademarks. Other product or brand names may be trademarks or registered trademarks of their respective holders.

41
Q&A

42