OFFensive Swarm-Enabled Tactics (OFFSET)

Dr. Timothy H. Chung

Program Manager
Tactical Technology Office
Defense Advanced Research Projects Agency

Briefing for the Naval Counter-Improvised Threat Knowledge Network

March 18, 2021

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited.

Why Are Swarms Hard: Complexity of Swarms

Number

Agent Complexity Collective Complexity

1000

100
Highly capable platform Leader-follower Fluid, distributed
Simple platform with multiple mission formations maneuvers
with one mission systems, able to make Dynamic, resilient
complex decisions in
10 Static comms
system networking
unstructured
environments Scripted, rule- Adaptive collaboration,
based behaviors flexible coordination

Heterogeneity Human-Swarm Interaction

Identical platforms Heterogeneous Simple, fixed functions Freestyle intuitive

and payloads in a platforms and payloads or remotely operated human-swarm
single domain in multiple domains interactions

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 2

 OFFensive Swarm-Enabled Tactics (OFFSET) Vision
Objective Capability
Discover innovative technologies to enable Seize key urban terrain
large-scale teams of air and ground robots
to support small-unit forces operating in
complex urban environments

4-6 hours

Approx. 8 square city blocks

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 3

 OFFSET Urban Mission-focused Approach
Vignette 1 Vignette 2 Objective Capability

Isolate an urban objective Conduct an urban raid Seize key urban terrain

15-30 minutes 1-2 hours 4-6 hours

Approx. 2 square city blocks Approx. 4 square city blocks Approx. 8 square city blocks

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 4

 Introducing OFFSET: Swarms in Urban Environments

*footage at 4x speed for presentation purposes

Highlights from recently completed OFFSET FX-4 @ Joint Base Lewis-McChord, Washington
https://www.darpa.mil/news-events/2020-09-18

FX-4 – Field Experiment 4 DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 5
 Accelerating Swarm Capabilities through Swarm Tactics

Swarm Tactics Swarm Systems Architectures Scalable Swarm Simulators Modular Swarm Testbeds

OFFSET Focus

Swarm
Mission Warfighting Needs
“Game”
Rapid Integration Swarm Ecosystem
(RISE)
Swarm
Tactics OFFSET Swarm Systems Integrators
“Plays”

Command and Control of

Swarm
Primitives Aggregate Swarm Tactics
“Behaviors” (CCAST)

Swarm
Algorithms OFFSET Impact: Software Repository of swarm tactics code,
“Skills”
user tutorials, and developer documentation

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 6

 Swarm Tactics Exchange: Repositories of Swarm Software

Swarm Tactics Count Evolution RISE Swarm Tactics Exchange

90 • Examine Objective • Platform test • Artifact secure
RBBN NGMS • Follow Route • Building scan • Building perimeter scan
80 Waypoint navigation/POI Outdoor exploration
• Hold Position • •
70 • Hover Above Current Location • Tornado scan • GPS to indoor explore
• Inward Scan • Hover • GPS to indoor non-explore
Number of Swarm Tactics

60 • Move To • Ring around POI • Indoor ramp traversal

• Multi-agent inward scan • Overhead Scan • Leader follower (follower)
50 • Overhead scan • Multi-agent inward scan • Leader follower (leader)
• Ring around POI • Examine Object • Platform Test
40
• UAV Test POI • Safe Land • Approach area
30 • RADAR scan for ST-MTRI • Outdoor to indoor transition • Breach Building
• Safe Land • Indoor exploration • Waypoint Navigation/POI
20 • Sector split leader follower • Artifact handler • Approach target area leader
follower
10

0 CCAST Swarm Tactics Exchange

• Add To Replacements • Interact • Return To Launch
19

19

19

20

20

20
0

1
/2

/2

/2

/2

/2

/2

/2

/2

/2

/2
0/

0/

1/

1/

0/

1/
31

29

31

30

31

30

31

31

30

31
Amass Interact Go To Revive at Medic
/3

/3

/3

/3

/3

/3
• • •
1/

2/

3/

4/

5/

6/

7/

8/

9/

1/
10

11

12

10

11

12
• April Tag Scan • Join Quick Reaction Force • Route Nav
• Area Survey • Land In Area • Split
• Cordon Area • Land In Place • Stage In Area
Cordon Building • Look At • Stop
• Explore • Maintain Proximity to AT • Stop/Estop
• Explore Building • Patrol • Surveil Ground
• Flock To LLA • Process Obstacles • Surveil Object
• Follow Object • Proximity • Surveil Area
• Gather Ground Imagery • Relieve agent • Transition To Region
• Go To • Rendezvous at Point • Watch Entry Points in Zone
Sketch gestures Virtual reality icons • Go To – Complex • Report Objects • Designate Quick Reaction Force
for NGMS RISE for RBBN CCAST
swarm tactics swarm tactics

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 7

 Swarm Tactics: Spotlights

Overhead Scan Multiagent Inward

Scan
parameters = [ parameters = [
Altitude
Starting Altitude
Cell Size
Level Count
Agent Count]
Distance Between Levels
Agent Count
Outward]
Description: Fly UAVs over Description: Use multiple UAV
area to find artifacts agents to scan enclosed area
specified by path stroke.

Surveil Area Surveil Object

parameters = [ parameters = [
Polygon Building (spatial
Agent(s) database)
Concluding Action] Agent(s)
Concluding Action]

Description: Command Description: Command

wildcard agent(s) to travel to agent(s) to travel to and surveil
user input area an object in the spatial database

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 8

 Swarm tactics-enabling Swarm Systems Architectures

Swarm Tactics Swarm Systems Architectures Scalable Swarm Simulators Modular Swarm Testbeds

Fact 1: real-world missions requires composability

Different approaches, different assumptions, different context
Swarm
Tactics
“Plays” Fact 2: adopting new innovation requires flexibility
Novel ideas don’t come in just one shape and size (or interface)

OFFSET relies on composable and flexible

swarm autonomy
Swarm swarm tactics | swarm primitives | swarm algorithms
Primitives
“Behaviors”

OFFSET Impact: Swarm Sprinter-exercised design

patterns for how to contribute novel swarm tech
Swarm through flexible architecture interfaces
Algorithms
“Skills”

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 9

 Swarm Systems Architectures: Layers of Swarm Autonomy

“Open architecture” means open to diverse and flexible pathways for integrating novel technologies

Swarm
Tactics
“Plays”

Swarm
Primitives
“Behaviors”

Swarm
Algorithms
“Skills”

35 Swarm Sprinter efforts across five Swarm Sprints

Swarm Sprint Thrust Areas:
Swarm Tactics Swarm Autonomy
Human-Swarm Teaming Virtual Environment
Physical Testbed Applications of AI DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 10
 Swarm Sprint Thrust Areas:
Swarm Sprinters: Spotlights Swarm Tactics Swarm Autonomy Human-Swarm Teaming
Virtual Environment Physical Testbed Applications of AI

ENHANCE: Evolving Neural Architectures with Human

Augmented Novelty for Complex Environments

ACCIPITER: Aerobatic Control and Collaboration

for Improved Performance In Tactical Evasion and
Reconnaissance

HiveXL: automated 80-drone launch,

recovery, charging, and transport

Rotor Mounted
Bearing+Elevation
Sensor

Swarm Sprint 1 Swarm Sprint 2 Swarm Sprint 3 Swarm Sprint 4 Swarm Sprint 5

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 11

 Swarm tactics-enabling Scalable Swarm Simulators

Swarm Tactics Swarm Systems Architectures Scalable Swarm Simulators Modular Swarm Testbeds

fidelity Level 3 higher fidelity

accelerate transfer to physical
• full sensor models
• full-rotor physics
systems; increased confidence in
• full environment and robustness of swarm tactics

Heavyweight
Level 2

• cameras only
• approx. collisions

Level 1
Lightweight
• simplified sensors
OFFSET Virtual Environments
higher scalability • simplified agents
100s+ agents for swarm tactics
prototyping; machine learning and
simulation analysis applications Featherweight

scalability

OFFSET Impact: Two extensible virtual environments for integrated

autonomous systems simulation + development + testing

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 12

 Scalable Swarm Simulators: Spotlights

Large-scale swarm mission scenario play Temporal coverage sensor modeling Multi-host swarm tactic execution

Variable faster-than-real-time simulation Field test swarm mission rehearsal Multi-robot environment interactions

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 13

 Swarm tactics-enabling Modular Swarm Testbed

Swarm Tactics Swarm Systems Architectures Scalable Swarm Simulators Modular Swarm Testbeds

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 14

 Modular Swarm Testbeds: Agile Swarm Experimentation

2. Rapidly instrument large-scale urban test sites

1. Design scenario for offensive urban operations
FX-4 Infrastructure Measures
• 5300+ meters of Ethernet
• 600 meters of optical fiber
• 811 Unique AprilTags
• 83 Raspberry Pi Nodes
• 26 network switches
• 14 USG team members

OFFSET FX-4 Scenario Laydown

Joint Base Lewis-McChord “Leschi Town” CACTF
Nested AprilTags
for multi-scale
3. Orchestrate and capture perception

real-time assessments of
“Swarm Under Test”

Dynamic AprilTags:
E-ink screens for
evolving scenarios
Screenshot of networked orchestration
software frontend (a.k.a. Mole)

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 15

 Program Schedule, by Vignette and Scope

Operational
Vignette 1 Vignette 2 Vignette 3
Context

Representative
Mission Isolate an urban objective Conduct an urban raid Seize key urban terrain

Mission
Duration 15-30 minutes 1-2 hours 4-6 hours

Area of
Operations Approx. 2 square city blocks Approx. 4 square city blocks Approx. 8 square city blocks

Swarm Size 50 100 250

FX Experiment FX-1 FX-2 FX-3 FX-4 FX-5 FX-6

Oct 22-25, June 5-13, Dec 7-19, Aug 1-12, Jun 27-Jul 14,
Date Jan 2021
2018 2019 2019 2020 2021

Cancelled

Location
Joint Base Joint Base
Camp Roberts Fort Benning Camp Shelby Lewis- Camp Shelby Lewis-
McChord McChord

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 16

 (Ongoing) OFFSET Lessons Learned

OFFSET Urban Scenario Reference Dataset

What has OFFSET created?
• Library of collaborative autonomy software
• Swarm virtual environments as bridge to real-world
• Unique swarm datasets

What are (actionable) OFFSET lessons learned?

• Define point-of-departure swarm autonomy capabilities
High-resolution 3D photogrammetric mesh • Seek applications needing high + fast “swarm power”

What comes next after OFFSET?

• Urban maneuver-capable platforms
• Still need agile and adaptable robotic systems
Interior 3D mesh scans
of all 33 building
Exterior building scan
• Swarm logistics enablers
interiors in-play
videos (by drone) for all • On-the-move and/or federated swarm deployment
40 buildings in-play
• Swarm replenishment, recharge, and recovery technologies
• Counter swarm/autonomy
Scenario walkthrough
videos (outdoor/indoor)
of entire test range

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 17

 www.darpa.mil

[email protected]
www.darpa.mil/work-with-us/offensive-swarm-enabled-tactics

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 18

 Physical Testbed: Northrop Grumman Swarm Agents
Specification Overview Uvify IFO-S Capabilities
• PX4 Flight Stack • CSI/MIPI Camera • Collision Avoidance • Camera Feed & AprilTag
• NVIDIA Jetson Nano • TFMini LiDAR • Static and dynamic Detection – forward &
• Intel RealSense D435 • Rajant DX2 • Scanning Tactics downward
• GPS with Helical • Waypoint Navigation • Depth sensing via
Antenna • Follow Route Tactic RealSense and TFMini
• POI LIDAR
• Hover/Loiter

Specification Overview AION R1 w. TX2 (ATX) Capabilities

• ArduRover Flight Stack • Embedded Wheel • Collision Avoidance • Indoor Capabilities
• NVIDIA TX2 Encoder • Static and dynamic • Building Breaching
• Intel RealSense • RPLiDAR • Scanning Tactics Tactic
D435/D415 • Rajant ES1 • Area Scan • SLAM
• Here2 GPS • Waypoint Navigation • GPS-denied capable
• Depth sensing via • Camera Feed & AprilTag
RealSense and RPLiDAR Detection

Specification Overview AION VTOL Capabilities

• ArduPilot Flight Stack • Downward CSI/MIPI • Overhead Scan and SA • Downward Camera Feed
• NVIDIA Jetson Nano Camera • Waypoint Navigation & AprilTag Detection
• Here2 GPS • Rajant Dual Band • Follow Route Tactic
mini radio • POI
• Hover/Loiter

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 19

 Physical Testbed: Raytheon BBN Swarm Agents (1of2)
Specification Overview 3DR Solo Capabilities
• Electro Optical camera • Raspberry Pi 3B+ • Surveil Building • AprilTag Detection –
– Pi Camera v2 co-processor • Watch points forward or downward
• FCC-approved radios • Raspberry Pi Cam • Path planning • Flock
• Custom expansion (down EO sensor • Identify points of • Rendezvous
board • USB LTE modem ingress/interest • Battery Swap
• Surveil Area

Specification Overview AION R1 w. TX2 (ATX) Capabilities

• ArduRover Control • Backward-facing • Explore Building • Join Quick Reaction Force
Software webcam • Enter building • AprilTag Detection
• Nvidia Jetson TX2 co- • 2D spinning lidar • Cordon Area • Rendezvous
processor • USB LTE modem • Distribute in circle
• Intel RealSense D435 • Cordon Building
(forward EO sensor) • Depth sensing via
RPLiDAR

Specification Overview Uvify IFO-S Capabilities

• PX4 Flight Control • Raspberry Pi Cam • Surveil Building • Flock
Software (downfacing EO • Watch points • Move to target point
• Nvidia Jetson Nano sensor) • Path planning • Rendezvous
• Intel RealSense D435 • USB LTE modem • Identify points of • Maintain Proximity
(forward EO sensor) ingress/interest • Depth sensing via
• Surveil Area RealSense

20
 Physical Testbed: Raytheon BBN Swarm Agents (1of2)
Specification Overview Modal AI VOXL m500 Capabilities
• VOXL Flight Deck with • Stereo image sensor • Surveil Building • AprilTag Detection –
PX4 • Tracking image • Watch points forward or downward
• GPS/Compass sensor • Path planning • Flock
• PWM breakout board • 4K image sensor • Identify points of • Rendezvous
ingress/interest • Battery Swap
• Surveil Area

Specification Overview mRo Nano Talon EVO Capabilities

• ArduRover Control • Backward-facing • Combat Air Patrol • Explore
Software webcam • Surveil Building • AprilTag Detection
• Nvidia Jetson TX2 co- • 2D spinning lidar • Watch points
processor • USB LTE modem • Path planning
• Intel RealSense D435 • Identify points of
(forward EO sensor) ingress/interest

Specification Overview JHU Custom Fixed Wings Capabilities

• mRo Control Zero F7 • Basler Dart (EO • Multi-Agent Fixed-Wing • Multi-Agent Fixed-Wing
• Intel NUC Camera) Building Scan Aerobatic Decoy
• Ardupilot/ACCIPITER • RFD900U telemetry
radio

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited. 21