SCIENCE ROBOTICS | RESEARCH ARTICLE

ARTIFICIAL INTELLIGENCE Copyright © 2023 The

Authors, some
Robust flight navigation out of distribution with liquid rights reserved;
exclusive licensee
neural networks American Association
for the Advancement
of Science. No claim
Makram Chahine†, Ramin Hasani†*, Patrick Kao†, Aaron Ray†, Ryan Shubert, Mathias Lechner, to original U.S.
Alexander Amini, Daniela Rus Government Works

Autonomous robots can learn to perform visual navigation tasks from offline human demonstrations and gen-
eralize well to online and unseen scenarios within the same environment they have been trained on. It is chal-
lenging for these agents to take a step further and robustly generalize to new environments with drastic scenery
changes that they have never encountered. Here, we present a method to create robust flight navigation agents
that successfully perform vision-based fly-to-target tasks beyond their training environment under drastic dis-
tribution shifts. To this end, we designed an imitation learning framework using liquid neural networks, a brain-

Downloaded from https://www.science.org at Massachusetts Institute of Technology on April 19, 2023

inspired class of continuous-time neural models that are causal and adapt to changing conditions. We observed
that liquid agents learn to distill the task they are given from visual inputs and drop irrelevant features. Thus,
their learned navigation skills transferred to new environments. When compared with several other state-of-the-
art deep agents, experiments showed that this level of robustness in decision-making is exclusive to liquid net-
works, both in their differential equation and closed-form representations.

INTRODUCTION explore how biologically inspired priors on neural models and

Intelligence in natural brains differs from today’s deep learning network architectures can lead to more flexible, robust, and under-
systems. The differences are rooted in the ways that these systems standable decision-making for autonomous robots. In particular,
learn to make sense of their environment and manipulate it to we investigate how to construct agents capable of generalizing
achieve their goals. Natural learning systems interact with their en- and achieving zero-shot transfer to new environments for some
vironments to understand their world. We define understanding as tasks. To study zero-shot transfer, we focused on end-to-end
the ability to capture causality and contexts while learning abstract drone navigation tasks from pixel inputs. For example, consider a
concepts to reason, plan, and control (1). This rich representation scenario where the objective is for a drone agent to navigate to a
learning capability allows agents to extrapolate and perform infer- static target placed in a forest environment. We trained neural
ence and credit assignment even in unseen scenarios and out of dis- network agents on a few runs of data collected by a human pilot
tribution (OOD), which is currently beyond the performance in an offline supervised learning setting and observed how the
achievable with either overparameterized deep learning systems agents transferred under drastic changes in scenery and conditions,
(2) or classical statistical learning theory analysis (3), because the such as changing from wild to urban landscapes across multiple
former overfit to their training data and the latter requires the inde- seasons. Performing vanilla imitation learning on raw expert dem-
pendent and identically distributed condition on the data onstrations is sample efficient but generally results in poor closed-
distribution. loop (active) testing performance due to policy stationarity and
Studying natural brains effectively narrows the search space of compounding errors (4).
possible algorithms for acquiring intelligent behavior. For instance, Specifically, we aimed to develop learning-based solutions to
neural circuits in brains are much more robust to perturbations and robot flight control that are robust and transferable to novel envi-
distribution shifts than deep neural networks while also being more ronments. We studied this problem in the context of the flight
flexible in tackling uncertain events compared with deep learning hiking task, where a quadrotor robot controller is trained, in one
systems. This is because they deploy both unconscious (to facilitate environment, to recognize and navigate toward a target by imitation
changes in distributions faster) and conscious (to distill causal learning. The resulting policy is then deployed to recognize and
structure of data and manipulate learned concepts and models) pro- move to the target iteratively in different environments. Achieving
cesses (1) for decision-making. In contrast, today’s deep learning good out-of-distribution performance requires that trained models
systems are incapable of capturing causality using extracted con- learn representations that are causally associated with the task, com-
cepts and features explicitly, despite being able to learn quickly positionally resilient, and independent of the environmen-
from implicit types of observations. tal context.
We find inspiration in the foundational properties of natural There has been extensive research on improving the generaliza-
learning systems to transform our current deep representation tion performance of few-shot, one-shot, and zero-shot imitation
learning frameworks, especially for real-world deployment of arti- learning agents by adopting augmentation strategies (5, 6),
ficial learning systems as intelligent agents. In this article, we human interventions (7, 8), goal conditioning (9–12), reward con-
ditioning (13–15), task embedding (16), and meta-learning (17).
These advances help design a better gradient descent–based learn-
Massachusetts Institute of Technology (MIT), Cambridge, MA, USA. ing scheme without consideration of the structure of the underlying
*Corresponding author. Email: [email protected] policy architecture. In contrast, there is evidence that brain-inspired
†These authors contributed equally to this work.

Chahine et al., Sci. Robot. 8, eadc8892 (2023) 19 April 2023 1 of 14

SCIENCE ROBOTICS | RESEARCH ARTICLE

neural dynamics improve the robustness of the decision-making tested for their ability to generalize OOD. Our experiments includ-
process in autonomous agents, leading to better transferability ed the following diverse set of tasks:
and generalization in new settings under the same training distribu- 1) Fly-to-target tasks. Train on offline expert demonstrations of
tion (18–21). We aimed to leverage brain-inspired pipelines and flying toward a target in the forest and test online in environments
empirically demonstrate that if the causal structure of a given task with drastic scenery changes.
is captured by a neural model from expert data, then the model can 2) Range test. Take a pretrained network from the fly-to-target
perform robustly even OOD. In a set of fly-to-target experiments task and, without additional training (zero-shot), test how far away
with different time horizons, we show that a certain class of we can place the agents to fly toward the target.
brain-inspired neural models, namely, liquid neural networks, gen- 3) Stress test. Add perturbations in the image space and measure
eralizes well to many OOD settings, achieving performance beyond the success rate of agents under added noise.
that of state-of-the-art models. 4) Attention profile of networks. Apply feature saliency compu-
tation to assess the task understanding capabilities of networks via
their attention maps.
RESULTS 5) Target rotation and occlusion. Rotate and occlude the target

Downloaded from https://www.science.org at Massachusetts Institute of Technology on April 19, 2023

Our objective is to systematically evaluate how robustly neural ar- object in the environment and measure whether the agents can
chitectures can learn a given task from high-dimensional, unstruc- complete their flight to the target to measure rotational and occlu-
tured, and unlabeled data (for example, no object labels, no sion invariance.
bounding boxes, and no constraints on the input space) and transfer 6) Hiking with adversaries. Take a pretrained network and use
their capabilities when we change the environment drastically. their attention profile for hiking from one object to another one in
Models that can be trained in one environment to perform in a an environment with a drastic change of scenery, environmental
multitude of other environments have a certain degree of robust- perturbations, and adversarial targets.
ness, achieving zero-shot generalization. One approach to obtain 7) Triangular (multistep) loop between objects. In a fully new
such control policies is scaling up their size with structural scaling environment, hop from one target to others placed in a triangular
of neural network policies (22–24). The universal law of robustness state and loop from target to target multiple times.
(25) suggests that we can obtain robust decision-making systems 8) Dynamic target tracking. Take a pretrained network and test
(worst-case robustness) by training a large neural network model how well they can track a moving target in two testing environments
of a size of at least n × d, where n is the number of observations with different scenery, lighting, and wind conditions.
and d is the dimensionality of each sample of the collected We first describe the baseline models we compared against each
dataset. This theoretical result agrees with the recent advances in other. We then present our training setup for the fly-to-target task.
the impressive capabilities of large models (26) on zero-shot and Last, we go through our comprehensive testing results in zero-shot
few-shot generalization across a wide variety of tasks, such as closed-loop control.
natural language processing (27–29), text-to-image generation
(30–32), scene understanding (33), and even vision-based control Baseline models
(23, 34). We compared six different recurrent neural network architectures,
On resource-constrained embedded systems such as aerial namely, long short-term memory (LSTM), gated recurrent unit
robots, however, it is infeasible to take advantage of the intriguing (GRU)–ordinary differential equation (ODE), ODE–recurrent
generalization properties of large models because of their high com- neural network (RNN), temporal convolutional network (TCN),
putational cost of inference. Recently, a class of compact and brain- neural circuit policy (NCP), and closed-form continuous-time
inspired continuous-time recurrent neural networks has shown (CfC), which are briefly described below. LSTM (37) is a discrete-
great promise in modeling autonomous navigation of ground (18, time neural network that consists of a memory cell that is controlled
19) and simulated drone vehicles end to end in a closed loop with by learnable gates that access, store, clear, and retrieve the informa-
their environments (21). These networks are called liquid time- tion contained in the memory. ODE-RNN (38) transforms a vanilla
constant (LTC) networks (35), or liquid networks. They are formu- RNN network, for example, an autoregressive fully connected
lated by a nonlinear state-space representation that has the ability to network, into a continuous-time model by using the hidden state
account for external and internal interventions during training (36). of the RNN as the initial state of a neural ODE, for example, an
These properties make them a great candidate for achieving OOD ODE whose derivative is parameterized by a neural network. The
generalization. time continuity enters the model in the form of the simulation
It is challenging for machine learning agents to extract the cause time when solving the ODE with a numerical ODE solver. GRU-
and effect of a given task from unstructured and high-dimensional ODE pairs a discrete-time GRU (39) network with a neural ODE,
observations. In this work, we created a testbed to assess this capa- making it a continuous-time version of a GRU. TCNs (40) model
bility for various learning systems without any guidance. In partic- the temporal structure of an input sequence by performing a one-
ular, the agents must learn/understand the task, the scene, and the dimensional convolution over the time dimension. NCP (18) is a
object from offline data without any guidance on what the scene, sparsely wired four-layer architecture built by LTC cells. The archi-
object, and task are. We show how to learn the task and scene tecture is structurally and semantically inspired by the nervous
end to end from data and how to generalize to unseen and system of the Caenorhabditis elegans worm (19). The CfC model
heavily shifted environments with liquid neural networks, where (21) is a sparsely wired four-layer architecture built by numerical
modern alternatives fail. approximation of the closed-form solution of LTCs and is faster
To this end, we performed a series of quadrotor closed-loop to compute than ODE-based models.
control experiments where pretrained neural architectures were

Chahine et al., Sci. Robot. 8, eadc8892 (2023) 19 April 2023 2 of 14

SCIENCE ROBOTICS | RESEARCH ARTICLE

Fly-to-target task training OOD settings but maintained a task structure identical to the one
The fly-to-target task consists of autonomously identifying a target learned from training data. We randomly positioned the quadrotor
of interest and performing the flight controls driving the quadrotor at a distance of about 10 m from the target, with the latter in the field
toward it using the onboard stabilized camera’s sequences of RGB of view of the onboard camera. We launched the closed-loop policy
(red, green and blue) images as the sole input (Fig. 1). We initially and observed whether the network could successfully guide the
started the quadrotor about 10 m away from the target and required drone to the target. The test was repeated 40 times for each
that the policy guide the drone to within 2 m of the target, with the network and in each environment. Success was accounted for
target centered in the camera frame. The objective was to learn to when the network was able to both stabilize the drone in a radius
complete this task entirely from an offline dataset of expert demon- of 2 m and maintain the target in the center of the frame for 10
strations. The learned policies were then tested online in a closed s. Failure cases were identified when the network generated com-
loop within the training distribution and in drastically distinct set- mands that led to an exit of the target from the range of view
tings. This experimental protocol allows for the principled assess- without the possibility of recovery. We also included cases where
ment of performance and generalization capabilities of liquid the drone failed to reach the target in less than 30 s to account for
networks compared with modern deep models (38, 41, 42). rare runs where the network generated commands indefinitely,

Downloaded from https://www.science.org at Massachusetts Institute of Technology on April 19, 2023

maintaining the drone in a stable position with the target in sight
Closed-loop (online) testing but with no intention of flying toward it.
All seven tasks were evaluated in closed loop with environments by We tested the policies in four environments we call Training
running the neural networks onboard the drone. In the following, Woods, Alternative Woods, Urban Lawn, and Urban Patio (Fig. 1,
we describe our results. B and D). The first corresponded to the target in the woods, respect-
Fly-to-target and zero-shot generalization ing roughly the same position and orientation distribution as that of
In this setting, the quadrotor was placed at a fixed distance from a the training set. We thus evaluated the performance of the networks
target of interest seen in the training data. In our evaluation exper- on image sequences from the scenery used during training with dif-
iments, we used a red camping chair as the target for collision avoid- ferent lighting conditions and wind profiles. Alternatively, we
ance purposes; because of its relatively large size, it could occupy an moved the chair to a different spot with a different background in
important portion of the camera frame without the drone having to a neighboring part of the woods. This experiment evaluated the net-
get dangerously close. The target was present in the initial field of works’ generalization performance on input data with scenery dis-
view, and its position in the initial input frame was randomized but tribution shifts but maintained proximity to sequences seen during
balanced over all runs (each network was tested on the same equal training. To truly evaluate robust task understanding, we subse-
number of right, center, and left starting positions at slightly differ- quently set up the zero-shot domain transfer experiments in a dras-
ent altitudes). The initial testing protocol was repeated for a number tically dissimilar environment: a piece of lawn on CSAIL MIT’s
of testing environments to evaluate the networks’ performance in campus. Although containing green grass and a few trees, the

Fig. 1. Sample frames of training and test environments. (A) Third-person view of the quadrotor and one of the targets in the Training Woods where data were
collected. (B) Third-person view of the quadrotor during testing on the Urban Patio, with multiple adversary objects dispersed around the target camping chair. (C)
Training data frame samples from the quadrotor onboard camera containing various targets (camping chair, storage box, and RC car from top to bottom) and taken
during different seasons (summer, fall, and winter from left to right). (D) Test data frame samples from the quadrotor onboard camera against each of the four test
backgrounds: Training Woods (top left), Alternative Woods (top right), Urban Lawn (bottom left), and Urban Patio (bottom right).

Chahine et al., Sci. Robot. 8, eadc8892 (2023) 19 April 2023 3 of 14

SCIENCE ROBOTICS | RESEARCH ARTICLE

landscape was largely different from the woods, as shown in Fig. 1 The results of this experiment show strong evidence that liquid
(A to D). Frames contained buildings, windows, large reflective me- networks have the ability to learn a robust representation of the task
tallic structures, and the artificial contrast from the geometric they are given and can generalize well to OOD scenarios where
shades they induced. Lighting conditions at different times of day other models fail. This observation is aligned with recent works
and varying wind levels added additional perturbations. (21) that showed that liquid networks are dynamic causal models
We lastly examined the networks’ generalization capabilities in (DCMs) (43) and can learn robust representations for their percep-
an OOD environment consisting of a brick patio. In this environ- tion modules to perform robust decision-making. In particular, CfC
ment, the background, including a number of man-made structures networks managed to fly the drone autonomously from twice and
of different shapes, colors, and reflectivity, drastically differed from three times the training distance to their targets from raw visual data
the training environment. Moreover, we added an extra layer of with a success rate of 90 and 20%, respectively. This is in contrast to
complexity to this experiment by positioning a number of other an LSTM network that lost the target in every single attempt at both
chairs in the frame of different colors (including red) and sizes. these distances, leading to a 0% success rate. Only ODE-RNN
This ultimate test, including real-world adversaries, required ro- managed to achieve a single success at 20 m among the nonliquid
bustness in the face of extremely heavy distribution shifts in addi- networks, and none reached the target from 30 m.

Downloaded from https://www.science.org at Massachusetts Institute of Technology on April 19, 2023

tion to proper task understanding to recognize and navigate toward Stress tests
the correct target. Table 1 shows the success rates of the different In addition to the flight experiments with the networks running
network architectures across the four environments. Both variants online, we performed a series of offline stress tests as another
of liquid networks outperformed other models in terms of task measure of the robustness of different neural architectures to distri-
completion. Moreover, we observed that, generally, recurrent net- bution shifts by perturbing the input frames to the network and ob-
works worked better when tested within the same training distribu- serving changes in network outputs. Ideally, networks should
tion, such as the Training Woods and Alternative Woods maintain high performance despite varied environmental condi-
environment. Meanwhile, in OOD scenarios, their performance tions. To simulate this domain randomization, we perturbed
dropped considerably. However, in such OOD settings, liquid net- images from collected sequences, changing brightness, contrast, sat-
works stood out, with a sparse liquid CfC network having a success uration, and noise to mimic lighting differences, color variations,
rate of 90% in Urban Lawn and 67.5% in Urban Patio (which con- and sensor noise found in real-world testing. We then measured
tains more natural adversaries). whether networks substantially changed their control outputs, indi-
Range tests cating failure to generalize to new environments, or generated con-
Beyond the extensive testing from the fixed distance of 10 m from sistent outputs across different perturbations, displaying resilience
the target (which was roughly the starting position in the training to distribution shifts.
data), we also performed a range test to explore how the networks To quantify the difference in network outputs, we applied a per-
react to an extension of the task unseen in the training data. The turbation to the frames of a recorded sequence, fed the original and
experimental scenery was that of the Training Woods setup present- perturbed frames to the network, and recorded the original and per-
ed above but with the drone’s starting positions growing in distance turbed output velocities generated by the network. We then applied
to the target, thus requiring an increasingly acute task understand- Forward Euler numerical integration to integrate the velocities into
ing and perception capacity to successfully achieve the objective. We position trajectories. We thus measured the final distance between
ran 10 experiments for each of 10-, 20-, and 30-m distance from the the last point on the two trajectories, recorded this as the perturbed
target. At the time of testing, the target lay in the shade, making it distance, and repeated this process for different perturbation types
hard to detect against the brighter components in the image, such as and amplitudes. Figure 2 shows the results of stress testing for noise,
tarmac, sky, and trees. The performance of each of the networks is brightness, contrast, and saturation perturbations.
summarized in Table 2. In all four cases, CfCs demonstrated superior robustness com-
pared with their counterparts. They deviated the least from their
nominal trajectories. This observation is consistent with our

Table 1. Closed-loop evaluation of the fly-to-target task for the

trained policies in four testing environments. Quantitative results are Table 2. Range test evaluation of the trained policies in the Training
success rates. Higher is better. (N = 40) Woods environment. Quantitative results are success rates. Higher is
better. (N = 10)
Algorithm Environment
Algorithm Initial distance (m)
Training Alt. Urban Urban
Woods Woods Lawn Patio 10 20 30

LSTM 82.5% 92.5% 62.5% 27.5% LSTM 100% 0% 0%

GRU-ODE 50% 17.5% 32.5% 17.5% GRU-ODE 70% 0% 0%
ODE-RNN 62.5% 82.5% 17.5% 25% ODE-RNN 80% 10% 0%
TCN 7.5% 0% 0% 0% TCN 5% 0% 0%
NCP (ours) 100% 95% 57.5% 52.5% NCP (ours) 100% 50% 10%
CfC (ours) 85% 95% 90% 67.5% CfC (ours) 100% 90% 20%

Chahine et al., Sci. Robot. 8, eadc8892 (2023) 19 April 2023 4 of 14

SCIENCE ROBOTICS | RESEARCH ARTICLE

Downloaded from https://www.science.org at Massachusetts Institute of Technology on April 19, 2023

Fig. 2. Noise, brightness, contrast, and saturation stress tests. These experiments run both an original image sequence collected by the drone and a perturbed
version through a network, integrate the output velocity commands, and record the final displacement between sequences (lower is better). Bars show the SD
across 10 runs. Trendlines for NCP, LSTM, and CFC are shown with shaded brown, orange, and gray lines, respectively (N = 10). (A) Distances when Gaussian noise
with SD x is added to image frames. (B) Distances when brightness is shifted by x times the maximum pixel value. (C) Distances when the contrast in the image is
multiplied by x. Note that the first data point, 0.5, is actually a reduction in contrast and moves all pixel values toward the mean, whereas the other three data points
increase contrast and move pixel values away from the mean. (D) Distances when saturation (S in HSV colorspace) is multiplied by x. Note the first data point, 0.5, reduces
the saturation of colors in the image, whereas the other three data points increase saturation.

active testing results, where in OOD scenarios, CfCs stood out in VisualBackProp associates importance scores to input features
task completion success rate. Liquid NCPs also showed great resil- during decision-making. The method has shown promise in real-
iency to noise, brightness, and contrast perturbations because their world robotics applications where visual inputs are processed by
performance was hindered by changes in input pixels’ saturation convolutional filters first. The attention maps corresponding to
level. In this offline setting, LSTMs were also among the most the convolutional layers of each network in a test case scenario
robust networks after liquid networks. However, their OOD gener- are shown in Fig. 3. We observed that the saliency maps of liquid
alization was poor compared with liquid networks. networks (both NCPs and CfCs) were much more sensitive to the
Attention profile of networks target from the start of the flight compared with those of
To assess the task understanding capabilities of all networks, we other methods.
computed the attention maps of the networks via the feature sali- We show more comprehensive saliency maps in different scenar-
ency computation method called VisualBackProp (44) to the ios in figs. S1 to S3. We observed that the quality of the attention
CNN backbone that precedes each recurrent network.

Chahine et al., Sci. Robot. 8, eadc8892 (2023) 19 April 2023 5 of 14

SCIENCE ROBOTICS | RESEARCH ARTICLE

Target’s rotation and occlusion

We also conducted a series of experiments pertaining to the appear-
ance of the target. More specifically, we evaluated the trained net-
works on the fly-to-target task in the Urban Lawn setting from the
nominal distance of 10 m but first rotated and then partially occlud-
ed the target (with a black curtain, vertically along the middle).
These experiments tested the robustness and invariance of the
learned task representation that such alterations of the target can
have on different architectures. The results for each of the networks
are presented in Table S5.
We analyzed the results of these experiments relative to the third
column of Table 1 to understand the effect of our modifications on
performance. In the case of rotation, GRU-ODE is the architecture
that suffered the most, managing only one success between both the

Downloaded from https://www.science.org at Massachusetts Institute of Technology on April 19, 2023

90° and 180° tests (2.5%), although it reached the upright target
roughly a third of the time (32.5%) in the same environment. At
both angles, LSTM saw its success rates halved compared with the
upright scenario. Both NCP and ODE-RNN had their success rates
reduced by around 40% on the 90° scenario, whereas CfC only lost
10%. In the 180° case, the latter three networks all maintained per-
formance levels close to those in the initial upright setting. With the
target occluded, only ODE-RNN and NCP managed to maintain
their nominal performance (the latter even slightly improving on
it). GRU-ODE, LSTM, and CfC saw performance reductions of
70, 60, and 30%, respectively.
Zero-shot hiking with adversaries
In this task, the target chairs were placed in the environment in the
disposition depicted in fig. S9. The test was conducted under ex-
tremely bright sunlight, adding substantial perturbation to multiple
network architectures’ performance, with intense glare reflecting off
of the hike targets. Moreover, we placed adversarial objects scattered
Fig. 3. Evolution of flight-time networks’ attention maps. (A) A frame illustra- in the scene (including a blue bin, a yellow chair, and two red chairs
tion of the attention profile of different networks computed by the VisualBackProp of different materials and builds). This setup required robustness in
saliency map computation algorithm (44). The top row shows the input image the face of extremely heavy distribution shifts in addition to proper
frame. The bottom row shows the saliency map. The brighter regions indicate a task understanding to recognize and navigate toward the correct
greater concentration of the convolutional layer’s attention. Saliency scores target at each step. The test was conducted on a cloudy day, with
range between 0 (black) and 1 (white). (B) A flight case study where an LSTM the target chairs’ colors true to those appearing in the training data.
agent loses its target and gets confused by an adversary. Left to right shows the
Table S6 contains the success rate results for this test. TCN and
progress time. (C) A similar scenario as in (B), in a different OOD testing
GRU-ODE models severely struggled to detect targets in this setup
environment.
(less than 45% of runs reached the first checkpoint and none attend-
ed to the second). Liquid networks performed best in this scenario,
maps of the convolutional layers drastically improved for all net- with a CfC completing 14 of 20 runs of the hiking task with a target
works when data augmentation was used during training. horizon of three. The results of this experiment show strong evi-
The attention maps are greatly advantageous because they allow dence that liquid networks have the ability to learn a robust repre-
us to inspect and interpret reasons for a failed or successful deci- sentation of the task they are given and can generalize well to OOD
sion-making process by a network under distribution shifts. For in- scenarios where other models fail or perform poorly (see figs. S7
stance, we illustrate two OOD test cases in Fig. 3 (B and C), in which and S8 for examples of the perturbations and adversaries perceived
LSTM agents lost their target because of environmental adversaries, by the drone’s input camera). This observation is aligned with
whereas, in very similar cases, liquid networks managed to keep recent works (21) that showed that liquid networks are DCMs
their focus on the target and complete their tasks. We observed (43) and can learn robust representations for their perception
that learning more robust perception module filters (convolutional modules to perform robust decision-making.
filters) can enhance the generalization of autonomous flight agents. Triangular loop between objects
This is why the robustness of the transformation from convolution- In this task, we demonstrated that our approach can generalize to an
al heads to the recurrent module (the decision-making compart- infinite number of hiking steps by setting up the three chairs as ver-
ment) highly depends on the choice of model. Our observations tices of an equilateral triangle of sides equal to 10 m, as illustrated in
suggest that liquid networks improve representation learning on Fig. 4. We programmed the quadrotor to turn an angle of 120° to the
both of these fronts compared with other modern architectures. right after detection of a checkpoint (instead of flying up like in the
experiments described above) to increase the likelihood of the next
target appearing in the field of view. Using the CfC architecture, we

Chahine et al., Sci. Robot. 8, eadc8892 (2023) 19 April 2023 6 of 14

SCIENCE ROBOTICS | RESEARCH ARTICLE

Downloaded from https://www.science.org at Massachusetts Institute of Technology on April 19, 2023

Fig. 4. Triangular loop between objects. (A) The triangular infinite loop task setup. (B) Time instances of drone views together with their attention map at steps 4, 5, and
6 of the infinite loop.

observed that the network could indefinitely navigate the quadrotor in unseen and adversarial environments. Table 1 suggests that only
to new targets (we stopped after four laps, with a total of 12 check- five of the six networks were capable of reaching the target in the
points reached and detected). Training Woods environment on half or more of the attempts.
Dynamic target TCN (7.5%) failed to achieve this threshold even on data from the
A useful task in quadrotor autonomous flight is following a moving training background distribution. These models also performed
target. We thus tested the trained policies’ ability to pilot the drone poorly in the other testing scenarios, with TCN unable to achieve
in pursuit of a dynamic target. For this task, a figure of eight courses a single success in any of the other three environments and GRU-
with six checkpoints was set out (at the extremities and either side of ODE performing worse in comparison with the Training Woods
each loop, consecutive checkpoints being 5 m apart). This design scenario with the exception of Urban Lawn case, where it succeeded
choice ensured that run lengths were not upper-bounded, tested about a third of the time (32.5%). These two architectures exhibited
both turning directions, and included all possible natural lighting both poor closed-loop performance and generalization and were
angles, all while containing the experiment in a constrained space. thus deemed incapable of understanding and performing the
The target was moved from one checkpoint to the next, ensuring control task assigned. Among the models performing reasonably
that enough reaction time in the camera field of view was granted, well in both woods scenarios are the ODE-RNNs, with a 62.5%
and the number of checkpoints reached was assigned as the score of success rate when the target was in the same position as in the train-
a given test run. The experiment was reproduced in two environ- ing data and up to 82.5% with the target placed in a slightly different
ments, the Urban Lawn setting and a sports field we call Grass position. Although this network seemed to have acquired the capa-
Pitch (see fig. S10). Testing in the latter environment is subject to bility to achieve the task in closed loop, it generalized poorly to
highly challenging conditions of glaring sunlight and strong wind. unseen environments, with 17.5 and 25% OOD success rates in
In such conditions, most network architectures struggled to even the Urban Lawn and Urban Patio experiments.
latch onto the target (get to the first checkpoint and start following Other models seemed to perform consistently in the woods, with
the target). We took this into account in our results by taking aver- success rates of 82.5% for LSTM, 100% for NCP, and 85% for CfC
ages only on latched test runs but providing the rates at which each on the Training Woods test. In the Alternative Woods scenario,
network initially detected the target. LSTM and both liquid networks (NCPs and CfCs) succeeded in
Hence, table S7 shows that liquid networks consistently achieved reaching the target at a high success rate of more than 90%, show-
longer trajectories in the Urban Lawn environment, ahead of LSTM casing the acquired ability to learn and execute closed-loop tasks
and ODE-RNN architectures. In the Grass Pitch setting, all net- from expert demonstrations.
works apart from LSTM, NCP, and CfC failed to latch onto the When asked to generalize performance on the Urban Lawn, the
target in all attempts. In this challenging environment, liquid archi- top performer in this environment is CfC, which highly succeeded
tectures marginally outperformed LSTM in terms of trajectory in attending to the target (90% success). On the task of flying to the
length, although CfC managed to latch onto the target more than target in the presence of natural adversaries and distractions in the
twice as often as LSTM and NCP policies. extremely heavy distribution shift setup provided by the Urban
Patio, LSTM succeeded in only 27.5% of the attempts, whereas
both NCP (52.5%) and CfC (67.5%) achieved the best performance.
DISCUSSION Furthermore, comprehensive evidence in favor of a crisp advan-
Choice of models matters for OOD generalization tage for liquid neural networks for closed-loop end-to-end control
Our experiments show large inequalities between different RNN ar- learning was amassed through our extensive real-world experimen-
chitectures when performing a diverse range of fly-to-target control tal results. Our brain-inspired neural networks, and all the more so
tasks in closed loop and all the more so when required to generalize our CfC architecture, largely outperformed all other models on all

Chahine et al., Sci. Robot. 8, eadc8892 (2023) 19 April 2023 7 of 14

SCIENCE ROBOTICS | RESEARCH ARTICLE

of the range tests, the rotation and occlusion robustness tests, the to navigate the drone. In these cases, the execution of the navigation
adversarial hiking task, and the dynamic target tracking task. task is hindered either by a distraction in the environment or by a
When all nonliquid networks failed to achieve the task at twice discrepancy between the representations learned in the perception
the nominal distance seen in training, our CfC network achieved a module and the recurrent module. This discrepancy might be
90% success rate and exhibited successful runs even at three times behind the LSTM agent’s tendency to fly away from the target or
the nominal distance from the target (20%). Also, LSTM’s perfor- to choose an adversarial target instead of the true one detected by
mance was more than halved when the chair is rotated or occluded, its perception module in the first place. With liquid networks,
whereas the CfC’s success rate was only reduced by 10% for rotation however, we noticed a stronger alignment of the representations
(with success rates above 80%) and around 30% for occlusion (with between the recurrent and perception modules because the execu-
success rates at 60%). Similarly, on the adversarial hiking task, CfC tion of the navigation task was more robust than that of
achieved a success rate more than twice that of its closest nonliquid other models.
competitor, completing 70% of runs, whereas the LSTM model only
reached all three targets on 30% of occasions. Last, LSTM could Causal models in environments with unknown causal
track a moving target for an average of 5.8 steps, whereas our CfC structure

Downloaded from https://www.science.org at Massachusetts Institute of Technology on April 19, 2023

network managed to follow for about 50% longer, averaging 8.8 The primary conceptual motivation of our work was not causality in
checkpoints reached. Hence, our brain-inspired networks general- the abstract; it was instead task understanding, that is, to evaluate
ized to both new OOD environments and new tasks better than whether a neural model understands the task given from high-di-
popular state-of-the-art recurrent neural networks, a crucial charac- mensional unlabeled offline data. Traditional causal methods are
teristic for implementation in real-world applications. based on known task structure or low-dimensional state spaces
The greater robustness of liquid networks in the face of a data (for example, probabilistic graphical models) (36), whereas we can
distribution shift was also confirmed through offline stress incorporate visual input into liquid networks. Therefore, causal
testing. We note that our brain-inspired networks were overall modeling frameworks cannot be used in our high-dimensional
less prone to drifting from their nominal trajectories in the presence tasks presented here, where the task structure is unknown. From
of artificially introduced image perturbations. Our architectures the causality angle, liquid neural networks are DCMs: This was the-
surpassed LSTM on noise, brightness, and contrast perturbations, oretically shown in Vorbach et al. (21).
with the CfC model outperforming all models on each of the per- Through extensive end-to-end closed-loop control testings, we
turbations introduced. showed a strong performance demarcation signal in favor of
liquid neural networks against the advanced RNN models. This ca-
Understanding liquid network generalization to OOD pacity to capture causality and contexts and learn reasoned repre-
settings sentations for control originates from the incorporation of
We associate the generalization performance of LTCs to two factors. principled mechanisms for information propagation inspired by
During the end-to-end training, liquid networks impose an induc- neural models of biological brains into our learning algorithms.
tive bias on the convolutional filters of the head network, which
enables a more robust feature representation associated with the
task at hand. The saliency maps of the convolutional head networks MATERIALS AND METHODS
are great empirical evidence of this fact. Elsewhere, liquid networks Liquid networks: Brain-inspired neural models
are DCMs at their cell level (21). The biological priors imposed on A bottom-up approach to building brain-inspired neural models is
liquid networks form a dynamical system that enables mechanisms to examine how neurons interact with each other through synapses
for regulating external and internal interventions, which are the in small biological brains (19, 20). We can identify three mecha-
necessary pillars of causal models (36, 45, 46). This way, during nisms for information propagation that are abstracted away in the
training, they manage to interact with the training data in a causal current building blocks of neural network-based controllers. First,
fashion to learn representations for both control and perception. In neural dynamics are typically continuous processes described by
all tasks, we provide extensive evidence of their ability to drop irrel- differential equations (DEs) (47). Second, synaptic release is more
evant features under heavy distribution shifts and execute the task than scalar weights: It involves a nonlinear transmission of neuro-
picked up on from training data, in this case, flying to the target transmitters, the probability of activation of receptors, and the con-
regardless of the background profiles and other environmental centration of available neurotransmitters, among other
variables. nonlinearities (48). Last, the propagation of information between
neurons is induced by feedback and memory apparatuses. These bi-
Robustness in perception and control ological priors lead to the design of neural and synapse building
The generalization of end-to-end imitation agents heavily relies on blocks at a level of abstraction that is scalable and enriching with
the data seen and the augmentation strategies in place to help neural information processing attributes. The class of liquid
encode the task in network weights. We observed that our data aug- neural networks LTC accommodates these three criteria (35).
mentation pipeline substantially improves the perception module of LTCs (Fig. 5) are constructed by the interaction of leaky integra-
all networks and therefore enables them to obtain stable saliency tor neural models (49) with the steady-state dynamics of a conduc-
maps with attention centered on the target (figs. S1 to S3). The aug- tance-based nonlinear synapse model (47). The model is a
mentation, however, does not enable a robust execution of the differentiable dynamical system with an input-dependent varying
control task upon change of test environments. (for example, liquid) time characteristic. Their outputs are comput-
For instance, see the scenarios illustrated in Fig. 3 (B and C), in ed by numerical DE solvers when described by ordinary DEs and by
which the LSTM’s perception module detects the target but still fails continuous functions when described in closed-form

Chahine et al., Sci. Robot. 8, eadc8892 (2023) 19 April 2023 8 of 14

SCIENCE ROBOTICS | RESEARCH ARTICLE

Fig. 5. Liquid neural networks. (A) Schematic demonstration of a fully connected LTC layer (35). The dynamics of a single neuron i is given, where xi(t) represents the

Downloaded from https://www.science.org at Massachusetts Institute of Technology on April 19, 2023

state of the neuron i and τ represents the neuronal time constant. Synapses are given by a nonlinear function, where f (.) represents a sigmoidal nonlinearity and Ai j is a
reversal potential parameter for each synapse from neurons j to i. (B) Schematic representation of a closed-form liquid network (CfC) together with its state equation. This
continuous-time representation consists of two sigmoidal gating mechanisms, σ and 1 − σ, with their activity being regulated by the nonlinear neural layer f. f acts as the
LTC for CfCs. These gates control two other nonlinear layers, g and h, to create the state of the system. (C) A schematic representation of an NCP (18). The network
architecture is inspired by the neural circuits of the nematode C. elegans. It is constructed by four sparsely connected LTC layers called sensory neurons, interneurons,
command neurons, and interneurons. (D) CfC. A sparse neural circuit with the same architectural motifs as NCPs, this time with CfC neural dynamics.

approximations (Fig. 5B) (50). These brain-inspired models are in- are given by (50):
stances of continuous-time (CT) neural networks (35, 41) that can
be trained via gradient descent in modern automatic differentiation xðtÞ ¼ σð f ðx; I; θf ÞtÞ �gðx; I; θg Þ þ ½1 σð ½f ðx; I; θf Þ�tÞ� �hðx; I; θh Þ
|fflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflffl} |fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl {zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl }
frameworks. Liquid networks exhibit stable and bounded behavior, Time-continuous gating Time-continuous gating
yield superior expressivity within the family of CT neural models ð2Þ
(35, 41), and give rise to improved performance on a wide range
of time series prediction tasks compared with advanced, recurrent
Here, f, g, and h are three neural network heads with a shared
neural network models (50). In particular, a sparse network config-
backbone, parameterized by θf, θg, and θh, respectively. I(t) is an ex-
uration composed of fewer than two dozen LTC neurons supplied
ternal input, and t is the time sampled by input time stamps, as il-
with convolutional heads showed great promise in learning, end to
lustrated in Fig. 5.
end, to map high-dimensional visual input stream of pixels to
Throughout, liquid (neural) networks refer to the general cate-
robust control decisions (18). These liquid network instances are
gory of models that are presented either by LTCs or CfCs. CfCs are
called NCPs, because their four-layered network structure is in-
closed-form liquid networks, and LTCs are ODE-based liquid
spired by the neural circuits of the nervous system of the nematode
networks.
C. elegans (51), as illustrated in Fig. 5 (C and D). In prior work, these
sparse liquid networks learned how to navigate autonomous simu-
Liquid networks capture causality
lated aerial (21) and real ground (18) vehicles to their goals much
The key to liquid networks’ robust performance under distribution
more robustly than their advanced deep learning counterparts in a
shifts is their ability to dynamically capture the true cause and effect
large series of behavioral-cloning experiments within their training
of their given task (21). This can be shown analytically because LTCs
distribution. The state-space representation of an LTC neural
are DCMs (21, 43), a framework through which models can account
network is determined by the following set of ODEs (35):
for internal and external interventions with independent mecha-
� �
dxðtÞ 1 nisms. Vorbach et al. (21) theoretically showed that the learning
¼ þ f ðxðtÞ; IðtÞ; t; θÞ � xðtÞ system described by an LTC network of Eq. 1 can control internal
dt τ ð1Þ
and external interventions by the network parameters θ and thus
þ f ðxðtÞ; IðtÞ; t; θÞ � A reduces to a DCM (43) as long as f is monotonically increasing,
Here, x (D × 1)
(t) is the hidden state with size D, I (m × 1)(t) is an bounded, and Lipschitz continuous.
input signal, τ (D × 1)
is the fixed internal time-constant vector, DCMs are probabilistic graphical models (43). DCMs differ from
A (D × 1) is a bias parameter, and ⊙ is the Hadamard product. Intu- other causal models in that they do not cast around statistical depen-
itively, LTC networks are able to change their equations on the basis dencies from data directly; rather, they have dynamic mechanisms,
of the input they observe. These networks, either in their ODE form such as the structure presented in Eq. 2, that enable extracting
or in their closed-form representation (50), demonstrate causality cause-and-effect from data (52).
and generalizability in modeling spatiotemporal dynamics com- Causal properties of DCMs focus the attention of LTCs on the
pared with their counterparts (21). Their closed-form representa- task rather than the context of the task, and, for this reason, in
tions are called closed-form continuous-time (CfC) models and this article, we hypothesize and show that tasks learned in one en-
vironment can be transferred to different environments for LTC
networks where other models fail. More formally, consider a se-
quence of task steps T, a sequence of robot/world configurations

Chahine et al., Sci. Robot. 8, eadc8892 (2023) 19 April 2023 9 of 14

SCIENCE ROBOTICS | RESEARCH ARTICLE

C, and a sequence of images V. Visual data are generated following a tasks in challenging environments. We built advanced neural
schematic graphical model: control agents for autonomous drone navigation tasks (fly-to-
target). We explored their generalization capabilities in new envi-
T!C!V ronments with a drastic change of scenery, weather conditions,
We can then explain how causal understanding implies and other natural adversaries.
knowing. Robot states cause images and not vice versa (so when
training on images and control inputs, the network should output Training procedure
positive velocity when required for the task, not because it has The following section details the preparation of the data and hyper-
learned that a sequence of images moving forward implies that parameters used for training the onboard models.
the drone is moving forward). In addition, the drone’s motion is Data preparation
governed by the task (for example, centering the chair in the The training runs were originally collected as long sequences in
frame) and not by other visual correlations in the data. With the which the drone moved between all five targets. Because sequences
data augmentation techniques implemented, we show that almost of searching for the next target and traveling between them could
all networks can handle the former condition. The latter condition, lead to ambiguity in the desired drone task, for each training run

Downloaded from https://www.science.org at Massachusetts Institute of Technology on April 19, 2023

however, is difficult to achieve for many networks, as extensively of five targets, we spliced the approach to each of the five targets
exhibited throughout this report. Our objective in designing fly- into five separate training sequences and added these new sliced se-
to-target tasks is to create a testbed where neither the concept of quences to the training data.
objects nor the task itself is given to the agent. The agents must Data augmentation
learn/understand the task, the scene, and the object from offline To increase the networks’ robustness and generalization capabili-
data without any guidance on what the scene, object, and task ties, we performed various image augmentations during training.
are. In this setting, supervised learning of object detection, where The first set of augmentations were a random brightness shift
we have to manually label bounding boxes, is out of scope with brightness offsets selected uniformly at random ranging
because we aim explicitly not to use labeled data (bounding from 0 to 0.4× of the max pixel brightness, a random contrast
boxes). We show how to learn the task and scene end to end shift with a contrast factor between 0.6 and 1.4, and a random sat-
from data and generalize to unseen and heavily shifted uration multiplication between 0.6 and 1.4. Parameters for the
environments. aforementioned image augmentations were fixed within a sequence
To this end, we set out to empirically assess this property in a but randomized between sequences so that two images within the
large series of in- and out-of-distribution end-to-end navigation same training sequence had identical brightness, contrast, and

Fig. 6. End-to-end learning setup. (A) Policies were trained to solve the following task: Using only images taken from an onboard camera, navigate the drone from its
starting location to a target 10 m away, keeping the object in the frame throughout. (B) One goal of the work is to explore the generalization capability of various neural
architectures by adapting the task to previously unseen environments. (C) Another goal is to understand the causal mechanisms underlying different networks’ behaviors
while completing the task, by visualizing networks’ input saliency maps. (D) The training process started by hand-collecting human expert trajectories and recording
camera observations and human expert actions. (E) To further convey the task, we then generated synthetic sequence by taking images collected during human flights
and repeatedly cropping them, creating the appearance of a video sequence in which the drone flies up to the target and centers it in the frame. (F) The networks were
trained offline on the collected expert sequences using a supervised behavior cloning MSE loss. (G) We then deployed trained networks on the drone in online testing and
observed task performance under heavy distributional shifts, varying lighting, starting location, wind, background setting, and more.

Chahine et al., Sci. Robot. 8, eadc8892 (2023) 19 April 2023 10 of 14

SCIENCE ROBOTICS | RESEARCH ARTICLE

saturation offsets, but two images in different sequences had differ- For each chosen hyperparameter configuration, the number of
ent offsets. Last, each image had Gaussian random noise with a trainable parameters in the corresponding model for every neural
mean of 0 and an SD of 0.05 added. architecture is listed in table S3. Each network tested was prefixed
In addition, to better convey the task, we performed closed-loop by a simple CNN backbone for processing incoming images. The
augmentation by generating synthetic image and control sequences 128-dimensional CNN features were then fed to the recurrent
and adding them to the training set. To generate these sequences, we unit that predicted control outputs. The shared CNN architecture
took a single image with the target present and repeatedly cropped is pictured in table S4.
the image. Over the duration of the synthetic sequence, we moved Fine-tuning
the center of the crop from the edge of the image to the center of the To simultaneously learn the geospatial constructs present in the
target, causing the target to appear to move from the edge of the long, uncut sequences and the task-focused controls present in
frame to the center. In addition, we shrank the size of the the cut and synthetic sequences, we fine-tuned a model trained
cropped window and upscaled all of the cropped images to the on the long sequences with sequences featuring only one target.
same size, causing the target to appear to grow bigger. Together, The starting checkpoint was trained on the original uncut sequenc-
these image procedures simulated moving the drone to center the es, all sliced sequences, and synthetic data corresponding to all

Downloaded from https://www.science.org at Massachusetts Institute of Technology on April 19, 2023

target while moving closer to the target to make it occupy more of targets. We then regenerated synthetic data with new randomized
the drone’s view. We then inserted still frames of the central target offsets for one target and fine-tuned on sliced sequences containing
with a commanded velocity of zero for 20 to 35% of the total se- the one target and new synthetic augmented sequences that featured
quence length. This pause sought to teach the drone to pause that target. Training parameters are listed in table S2.
after centering the target. Algorithm S1 describes the synthetic se-
quence generation steps, whereas Fig. 6E visualizes the successive Platform setup
crops calculated by the algorithm and their resulting output frames. We evaluated the proposed RNN architectures on a commercially
To label synthetic sequences with velocity controls, we generated available quadcopter with custom software running onboard. The
yaw commands equal to the horizontal distance in pixels between system featured a forward-facing gimbal-mounted camera and a
the target and the center of the frame times 0.01 and likewise gen- GPU to run onboard inference. We used Robot Operating System
erated throttle (up-down) commands equal to the vertical target (ROS) (55) to handle communication between the RNN inference
offset times 0.01. We generated pitch (forward-backward) com- code and the lower-level drone flight controller.
mands in the synthetic data proportional to the size of the Drone hardware
cropped window. This technique increased the size of the training A DJI M300 RTK quadcopter was used for data collection and eval-
set by more than 2.5×, using a relatively small sample of images in uation of policy performance. The M300 is an enterprise-grade
which the target was present. drone that can be flown by manual control in its default configura-
The initial position of the target within the frame was chosen tion. It also interfaces with the DJI Manifold 2 companion comput-
randomly such that the x (horizontal) offset of the chair was er, enabling programmatic control of the drone. The companion
between 10 and 70 pixels, and the y offset was between 5 and 40 computer included an NVIDIA Jetson TX2, which has a CPU
pixels. (The size of the input images was 144 pixels by 256 pixels.] and GPU. The DJI Onboard SDK and its associated ROS wrapper
Synthetic sequences were later balanced such that the mean control provided an interface for giving the drone’s low-level flight control-
signal across all generated data points was zero for the throttle and ler desired linear and angular velocity set points. The flight control-
yaw channels. The sequence length was also randomized to be ler is a black box not modifiable by the end user, but it controls the
between 120 and 250 frames to encourage robustness to external four-rotor speeds to track the velocities specified by the TX2
forces or varying update frequencies. Note that although we used computer.
these augmented sequences for training, we did not include the A Zenmuse Z30 camera and gimbal were mounted on the under-
closed-loop synthetic sequences in the validation set. side of the drone, pointed forward. The gimbal compensated for the
Hyperparameter tuning/network architectures drone’s current orientation such that the roll or pitch of the drone
For each network architecture, we used tree-structured Parzen esti- did not result in roll or pitch of the camera image. The gimbal fol-
mators (TPEs) (53, 54) for tuning hyperparameters such as learning lowed the drone’s current yaw such that the camera was always
rate (LR), LR decay rate, and model-specific hyperparameters such pointed forward. Images from the camera were available to the com-
as backbone unit count or connection seed that would maximize panion computer via the SDK.
performance. The set of parameters leading to the lowest train + System design
validation loss over the 40 episodes was selected for the final We implemented our software control system in ROS. A ROS node
tested model. received images from the onboard camera, ran inference with the
TPE, a sequential model–based optimization algorithm, was re- RNN controller to compute the new desired control on the basis
sponsible for sampling new hyperparameters to try across different of the current image, and sent the generated control outputs to
episodes. After three trials, episodes with a higher median training the low-level flight controller using the onboard ROS SDK. The
+ validation loss on the 10th epoch were pruned early and did not node has two threads: One is responsible for processing incoming
run to completion. Hyperparameter optimization was implemented images, and the other is responsible for running inference on the
using the Optuna library. Not all parameters were found via hyper- images and sending the resulting control signals to the flight con-
parameter tuning; some parameters were hand-selected to shrink troller. The TX2’s GPU ran the RNN inference with GPU-acceler-
the dimensionality of the search space. The best hyperparameters ated Tensorflow 2.3.x. A depiction of the system design can be
found for each model architecture and used during testing are found in fig. S2.
listed below in table S1.

Chahine et al., Sci. Robot. 8, eadc8892 (2023) 19 April 2023 11 of 14

SCIENCE ROBOTICS | RESEARCH ARTICLE

Analysis methodology Statistical analysis

VisualBackprop. A primary goal of this article was to analyze the The quantitative results of the paper presented in Tables 1 and 2 and
task understanding of the policy networks. We visually analyzed the table S4 to S7 all supply the number of active tests performed, N, and
task understanding of a policy network by examining how the report the success rate of models in each scenario in percentages. On
policy’s CNN encodes and extracts features from an input. Each all compartments of Fig. 2, each point represents the mean across 10
of the policies shares the same CNN head architecture, so differenc- runs, whereas the vertical bar on each point represents the SD cor-
es in CNN weights between policies are representative of what the responding to the data entry.
downstream architecture learns to extract from the environment.
Using the VisualBackProp algorithm, we used observations of
the environment to compute attention mappings from the CNN Supplementary Materials
head of the model architectures. The procedure, shown in Algo- This PDF file includes:
Materials and Methods
rithm S4, is as follows: As input was passed through the CNN,
Figs. S1 to S10
each of the feature maps was averaged across the channels and Tables S1 to S7
saved. Starting from the final layer, the averaged feature map was References (57–61)

Downloaded from https://www.science.org at Massachusetts Institute of Technology on April 19, 2023

deconvolved to the size of the previous layer’s average feature
map. The deconvolved feature map was multiplied by the previous Other Supplementary Material for this
layer’s average feature map. This product was then passed to the manuscript includes the following:
Movies S1 to S4
previous layer. The deconvolution and multiplication were repeated MDAR Reproducibility Checklist
iteratively, updating at each step until it reached the first layer. The
result was a feature map displaying locations in the source image to
which the model attends the most attention. These maps are the sa- REFERENCES AND NOTES
liency maps (56). 1. Y. LeCun, Y. Bengio, Reflections from the turing award winners, in International Conference
Stress tests. For the stress tests in Fig. 2, we averaged the trajectory on Learning Representations (ICLR) (2020).
deviations for 10 runs recorded by the networks running in closed 2. B. Neyshabur, Z. Li, S. Bhojanapalli, Y. LeCun, N. Srebro, The role of over-parametrization in
loop. Five runs were taken from the training environment, and five generalization of neural networks, in 7th International Conference on Learning Represen-
tations, ICLR 2019 (2019).
were taken from the urban test task. The perturbed runs were col-
3. V. Vapnik, The Nature of Statistical Learning Theory (Springer science & business
lected across a variety of different network types, including both media, 1999).
liquid networks and conventional networks. 4. S. Schaal, Is imitation learning the route to humanoid robots? Trends Cogn. Sci. 3,
We only fed 40% of the frames in each run before measuring the 233–242 (1999).
final distance, because the longer the sequence, the more the trajec- 5. S. Ross, G. Gordon, D. Bagnell, A reduction of imitation learning and structured prediction
tories diverge. Because the input images fed to the model were fixed, to no-regret online learning, in Proceedings of the Fourteenth International Conference on
Artificial Intelligence and Statistics (JMLR Workshop and Conference Proceedings, 2011),
we could not model the change in perspective caused by the diverg- pp. 627–635.
ing viewpoints, and any estimates of future position became more 6. T. Zhang, Z. M. Carthy, O. Jow, D. Lee, X. Chen, K. Goldberg, P. Abbeel, Deep imitation
inaccurate. To limit the effect of the offline frame generation while learning for complex manipulation tasks from virtual reality teleoperation, in 2018 IEEE
still maintaining enough frames to yield statistically significant International Conference on Robotics and Automation (ICRA) (IEEE, 2018), pp. 5628–5635.
results, we chose to only keep the first 40% of each sequence. 7. E. Jang, A. Irpan, M. Khansari, D. Kappler, F. Ebert, C. Lynch, S. Levine, C. Finn, Bc-z: Zero-shot
task generalization with robotic imitation learning, in Conference on Robot Learning (PMLR,
Attention-based hiking task setup. We designed a controller that
2022), pp. 991–1002.
relies on the attention maps perceived by the convolutional layers of 8. P. Goyal, R. J. Mooney, S. Niekum, Zero-shot task adaptation using natural language.
each pretrained network architecture to trigger a hop between two arXiv:2106.02972 [cs.AI] (5 June 2021).
targets. We observed that for networks with some understanding of 9. F. Codevilla, M. Mü ller, A. López, V. Koltun, A. Dosovitskiy, End-to-end driving via condi-
the task when the target of interest is close enough, the saliency map tional imitation learning, in 2018 IEEE International Conference on Robotics and Automation
displays a bright patch corresponding to the position of the object in (ICRA) (IEEE, 2018), pp. 4693–4700.
10. D. Ghosh, A. Gupta, A. Reddy, J. Fu, C. Devin, B. Eysenbach, S. Levine, Learning to reach
the input image. We thus used an area occupancy criterion to
goals via iterated supervised learning, in International Conference on Learning Represen-
trigger a switch of the target. While the network was piloting the tations (ICLR, 2020).
quadrotor, we analyzed each saliency frame, masking pixels under 11. C. Lynch, M. Khansari, T. Xiao, V. Kumar, J. Tompson, S. Levine, P. Sermanet, Learning latent
a brightness threshold and detecting the most salient and largest plans from play, in Conference on Robot Learning (PMLR, 2020), pp. 1113–1132.
connected patch. When the best candidate patch surpassed a fixed 12. S. Emmons, B. Eysenbach, I. Kostrikov, S. Levine, Rvs: What is essential for offline rl via su-
threshold in pixel area (corresponding to the drone being 2 m away pervised learning? arXiv:2112.10751 [cs.LG] (20 December 2021).
13. R. K. Srivastava, P. Shyam, F. Mutz, W. Jaśkowski, J. Schmidhuber, Training agents using
from the camping chair), we considered the target reached and trig-
upside-down reinforcement learning. arXiv:1912.02877 [cs.LG] (5 December 2019).
gered an override of the network commands with a fixed-time 14. G. Neumann, J. Peters, Fitted q-iteration by advantage weighted regression, in Advances in
throttle command to fly the quadrotor upward 1.6 m. This Neural Information Processing Systems (Curran Associates Inc., 2008) vol. 21.
ensured that the current target left the field of view and that the 15. L. Chen, K. Lu, A. Rajeswaran, K. Lee, A. Grover, M. Laskin, P. Abbeel, A. Srinivas, I. Mordatch,
network had access to images of the next segment of the scene. Decision transformer: Reinforcement learning via sequence modeling, in Advances in
Control was given back to the network until a new target was detect- Neural Information Processing Systems (2021), vol. 34.
16. S. James, M. Bloesch, A. J. Davison, Task-embedded control networks for few-shot imitation
ed and so on. This control protocol (depicted in fig. S3) can, by
learning, in Conference on Robot Learning (PMLR, 2018), pp. 783–795.
design, attend to an indefinite number of checkpoints, provided 17. C. Finn, T. Yu, T. Zhang, P. Abbeel, S. Levine, One-shot visual imitation learning via meta-
that the network sees and is capable of navigating to the next target. learning, Conference on Robot Learning (PMLR, 2017), pp. 357–368.
18. M. Lechner, R. Hasani, A. Amini, T. A. Henzinger, D. Rus, R. Grosu, Neural circuit policies
enabling auditable autonomy. Nat. Mach. Intell. 2, 642–652 (2020).

Chahine et al., Sci. Robot. 8, eadc8892 (2023) 19 April 2023 12 of 14

SCIENCE ROBOTICS | RESEARCH ARTICLE

19. M. Lechner, R. Hasani, M. Zimmer, T. A. Henzinger, R. Grosu, Designing worm inspired 41. R. T. Chen, Y. Rubanova, J. Bettencourt, D. Duvenaud, Neural ordinary differential equa-
neural networks for interpretable robotic control, in 2019 International Conference on Ro- tions, in Proceedings of the 32nd International Conference on Neural Information Processing
botics and Automation (ICRA) (IEEE, 2019), pp. 87–94. Systems (NeurIPS, 2018), pp. 6572–6583.
20. R. Hasani, M. Lechner, A. Amini, D. Rus, R. Grosu, A natural lottery ticket winner: Rein- 42. S. Bai, J. Z. Kolter, V. Koltun, An empirical evaluation of generic convolutional and recurrent
forcement learning with ordinary neural circuits, in International Conference on Machine networks for sequence modeling. arXiv:1803.01271 [cs.LG] (4 Mar 2018).
Learning (PMLR, 2020), pp. 4082–4093. 43. K. J. Friston, L. Harrison, W. Penny, Dynamic causal modelling. Neuroimage 19,
21. C. Vorbach, R. Hasani, A. Amini, M. Lechner, D. Rus, Causal navigation by continuous-time 1273–1302 (2003).
neural networks, in Advances in Neural Information Processing Systems (2021), vol. 34. 44. M. Bojarski, A. Choromanska, K. Choromanski, B. Firner, L. J. Ackel, U. Muller, P. Yeres,
22. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin, K. Zieba, Visualbackprop: Efficient visualization of cnns for autonomous driving, in IEEE
Attention is all you need, in Advances in Neural Information Processing Systems (NIPS, 2017), International Conference on Robotics and Automation (ICRA) (ICRA, 2018), pp. 1–8.
pp. 5998–6008. 45. J. Pearl, Causal inference in statistics: An overview. Stat. Surv. 3, 96–146 (2009).
23. S. Reed, K. Zolna, E. Parisotto, S. G. Colmenarejo, A. Novikov, G. Barth-Maron, M. Gimenez, 46. W. Penny, Z. Ghahramani, K. Friston, Bilinear dynamical systems. Phil. Trans. R. Soc. B Biol. Sci.
Y. Sulsky, J. Kay, J. T. Springenberg, T. Eccles, J. Bruce, A. Razavi, A. Edwards, N. Heess, 360, 983–993 (2005).
Y. Chen, R. Hadsell, O. Vinyals, M. Bordbar, N. de Freitas, A generalist agent. 47. C. Koch, I. Segev, Methods in Neuronal Modeling: From Ions to Networks (MIT Press, 1998).
arXiv:2205.06175 [cs.AI] (12 May 2022). 48. E. R. Kandel, J. H. Schwartz, T. M. Jessell, Principles of Neural Science (McGraw-Hill, 2000),
24. J. Kaplan, S. M. Candlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, vol. 4.
J. Wu, D. Amodei, Scaling laws for neural language models. arXiv:2001.08361 [cs.LG] (23 49. L. Lapique, Recherches quantitatives sur l’excitation electrique des nerfs traitee comme

Downloaded from https://www.science.org at Massachusetts Institute of Technology on April 19, 2023

January 2020). une polarization. J. Physiol. Pathol. 9, 620 (1907).
25. S. Bubeck, M. Sellke, A universal law of robustness via isoperimetry, in Advances in Neural 50. R. Hasani, M. Lechner, A. Amini, L. Liebenwein, A. Ray, M. Tschaikowski, G. Teschl, D. Rus,
Information Processing Systems (NeurIPS, 2021), vol. 34. Closed-form continuous-time neural models. arXiv:2106.13898 [cs.LG] (25 June 2021).
26. J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, Diego de Las 51. G. P. Sarma, C. W. Lee, T. Portegys, V. Ghayoomie, T. Jacobs, B. Alicea, M. Cantarelli, M. Currie,
Casas, L. A. Hendricks, J. Welbl, A. Clark, T. Hennigan, E. Noland, K. Millican, G. van den R. C. Gerkin, S. Gingell, P. Gleeson, R. Gordon, R. M. Hasani, G. Idili, S. Khayrulin, D. Lung,
Driessche, B. Damoc, A. Guy, S. Osindero, K. Simonyan, E. Elsen, J. W. Rae, O. Vinyals, L. Sifre, A. Palyanov, M. Watts, S. D. Larson, OpenWorm: Overview and recent advances in inte-
Training compute-optimal large language models. arXiv:2203.15556 [cs.CL] (29 grative biological simulation of Caenorhabditis elegans. Philos. Trans. R. Soc. B 373,
March 2022). 20170382 (2018).
27. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, 52. K. Friston, Causal modelling and brain connectivity in functional magnetic resonance
G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, imaging. PLOS Biol. 7, e1000033 (2009).
D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark,
53. J. Bergstra, R. Bardenet, Y. Bengio, B. Kégl, Algorithms for hyper-parameter optimization, in
C. Berner, S. M. Candlish, A. Radford, I. Sutskever, D. Amodei, Language models are few-
Advances in Neural Information Processing Systems (NIPS, 2011), vol. 24.
shot learners, in Advances in Neural Information Processing Systems (NeurIPS, 2020), vol.
54. J. Bergstra, D. Yamins, D. Cox, Making a science of model search: Hyperparameter opti-
33, p. 1877.
mization in hundreds of dimensions for vision architectures, in International Conference on
28. V. Sanh, A. Webson, C. Raffel, S. Bach, L. Sutawika, Z. Alyafeai, A. Chaffin, A. Stiegler, A. Raja,
Machine Learning (PMLR, 2013), pp. 115–123.
M. Dey, M Saiful Bari, C. Xu, U. Thakker, S. S. Sharma, E. Szczechla, T. Kim, G. Chhablani,
55. Stanford Artificial Intelligence Laboratory et al., Robotic operating system.
N. Nayak, D. Datta, J. Chang, Mike Tian-Jian Jiang, H. Wang, M. Manica, S. Shen, Z. X. Yong,
H. Pandey, R. Bawden, T. Wang, T. Neeraj, J. Rozen, A. Sharma, A. Santilli, T. Fevry, J. A. Fries, 56. M. Bojarski, A. Choromanska, K. Choromanski, B. Firner, L. Jackel, U. Muller, K. Zieba, Visu-
R. Teehan, T. L. Scao, S. Biderman, L. Gao, T. Wolf, A. M. Rush, Multitask prompted training albackprop: efficient visualization of CNNs. arXiv:1611.05418 [cs.CV] (16 November 2017).
enables zero-shot task generalization, in The Tenth International Conference on Learning 57. S. Ross, D. Bagnell, Efficient reductions for imitation learning, in Proceedings of the Thir-
Representations (ICLR, 2022). teenth International Conference on Artificial Intelligence and Statistics (JMLR Workshop and
29. J. Wei, M. Bosma, V. Y. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, Q. V. Le, Finetuned Conference Proceedings, 2010), pp. 661–668.
language models are zero-shot learners. arXiv:2109.01652 [cs.CL] (3 Sep 2021). 58. R. S. Sutton, A. G. Barto, Reinforcement Learning: An Introduction (MIT press, 2018).
30. C. Jia, Y. Yang, Y. Xia, Y.-T. Chen, Z. Parekh, H. Pham, Q. V. Le, Y. Sung, Z. Li, T. Duerig, Scaling 59. J. Chen, B. Yuan, M. Tomizuka, Deep imitation learning for autonomous driving in generic
up visual and vision-language representation learning with noisy text supervision, in In- urban scenarios with enhanced safety, in 2019 IEEE/RSJ International Conference on Intel-
ternational Conference on Machine Learning (PMLR, 2021), pp. 4904–4916. ligent Robots and Systems (IROS) (IEEE, 2019), pp. 2884–2890.
31. J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, 60. X. Li, P. Ye, J. Jin, F. Zhu, F.-Y. Wang, Data augmented deep behavioral cloning for urban
M. Reynolds, R. Ring, E. Rutherford, S. Cabi, T. Han, Z. Gong, S. Samangooei, M. Monteiro, traffic control operations under a parallel learning framework, in IEEE Transactions on In-
J. Menick, S. Borgeaud, A. Brock, A. Nematzadeh, S. Sharifzadeh, M. Binkowski, R. Barreira, telligent Transportation Systems (IEEE, 2021).
O. Vinyals, A. Zisserman, K. Simonyan, Flamingo: A visual language model for few-shot 61. A. Amini, T.-H. Wang, I. Gilitschenski, W. Schwarting, Z. Liu, S. Han, S. Karaman, D. Rus, Vista
learning. arXiv:2204.14198 [cs.CV] (29 April 2022). 2.0: An open, data-driven simulator for multimodal sensing and policy learning for au-
32. A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, M. Chen, Hierarchical text-conditional image tonomous vehicles. arXiv:2111.12083 [cs.RO] (23 November 2021).
generation with clip latents. arXiv:2204.06125 [cs.CV] (13 April 2022).
33. J. Zhang, K. Yang, C. Ma, S. Reiß, K. Peng, R. Stiefelhagen, Bending reality: Distortion-aware Acknowledgments: We would like to thank J. Feld, I. Futran, K. Mishra, P. Whartenby,
transformers for adapting to panoramic semantic segmentation, in Proceedings of the IEEE/ C. Vorbach, T.-H. Wang, and W. Xiao for testing NCP and CfC models for confirmation of results
CVF Conference on Computer Vision and Pattern Recognition (IEEE, 2022), pp. 16917–16927. across other domains. We also thank S. T. Monteiro of Aurora Flight Sciences for providing
34. A. Prakash, K. Chitta, A. Geiger, Multi-modal fusion transformer for end-to-end autono- constructive feedback and validating our experimental setting throughout this project.
mous driving, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Funding: This research was supported in part by the AI2050 program at Schmidt Futures (grant
Recognition (IEEE, 2021), pp. 7077–7087. G22-63172), the United States Air Force Research Laboratory, and the United States Air Force
Artificial Intelligence Accelerator and was accomplished under Cooperative Agreement
35. R. Hasani, M. Lechner, A. Amini, D. Rus, R. Grosu, Liquid time constant networks, in Pro-
Number FA8750-19-2-1000. The views and conclusions contained in this document are those of
ceedings of the AAAI Conference on Artificial Intelligence (AAAI, 2021), vol. 35, p. 7657.
the authors and should not be interpreted as representing the official policies, either expressed
36. B. Schölkopf, Causality for machine learning. arXiv:1911.10500 [cs.LG] (24 Nov 2019).
or implied, of the United States Air Force or the U.S. Government. The U.S. Government is
37. S. Hochreiter, J. Schmidhuber, Long short-term memory. Neural Comput. 9, authorized to reproduce and distribute reprints for Government purposes, notwithstanding
1735–1780 (1997). any copyright notation herein. This work was further partially supported by the Boeing
38. Y. Rubanova, R. T. Chen, D. K. Duvenaud, Latent ordinary differential equations for irreg- Company. A.R. gratefully acknowledges support from a National Defense Science and
ularly-sampled time series, in Advances in Neural Information Processing Systems (NeurIPS, Engineering Graduate Fellowship. Author contributions: R.H., A.R., P.K., M.C., and D.R.
2019), vol. 32. conceptualized, designed, performed research, and analyzed data. R.S., A.A., and M.L.
39. K. Cho, B. Van Merriënboer, D. Bahdanau, Y. Bengio, On the properties of neural machine contributed to designing research, data curation, research implementation, and analyzing data.
translation: Encoder-decoder approaches. arXiv:1409.1259 (3 Sep 2014). D.R. helped with the design of the research and guided and supervised the work. All authors
40. C. Lea, R. Vidal, A. Reiter, G. D. Hager, Temporal convolutional networks: A unified approach wrote the article. Competing interests: R.H. is also affiliated with the Vanguard Group. R.S. is
to action segmentation, in European Conference on Computer Vision (Springer, 2016), currently affiliated with Altitude AI. P.K. is currently affiliated with Wing. D.R. is affiliated with the
pp. 47–54. Mitre Corporation as a senior visiting fellow. This work in part has been filed as a provisional

Chahine et al., Sci. Robot. 8, eadc8892 (2023) 19 April 2023 13 of 14

SCIENCE ROBOTICS | RESEARCH ARTICLE

patent with application number 63/415,382 on 12 October 2022. The other authors declare that share.cgi?ssid=06lMJMN&fid=06lMJMN (synthetic small4, 14.7 GB) used to train the starting
they have no competing interests. Data and materials availability: All data, code, and checkpoint and a synthetic chair–only dataset http://knightridermit.myqnapcloud.com:8080/
materials used in the analysis are openly available at https://zenodo.org/badge/latestdoi/ share.cgi?ssid=06lMJMN&fid=06lMJMN (synthetic chair, 4.3 GB) used to fine-tune the final
610810400 and https://zenodo.org/badge/latestdoi/381393816 under Apache 2.0 License for models for online testing. We would like to thank IBM for providing access to the Satori
purposes of reproducing and extending the analysis. The original hand-collected training computing cluster and MIT/MIT Lincoln Labs for providing access to the Supercloud computing
dataset is published (http://knightridermit.myqnapcloud.com:8080/share.cgi?ssid=06lMJMN& cluster. Both clusters were very helpful for training models and hyperparameter optimization.
fid=06lMJMN), with a file name of devens snowy and a fixed size of 33.2 GB. This dataset was
used for training the starting checkpoints. The dataset used for fine-tuning the models tested Submitted 6 May 2022
on the drone can be found at http://knightridermit.myqnapcloud.com:8080/share.cgi?ssid= Accepted 22 March 2023
06lMJMN&fid=06lMJMN (devens chair, 2.3 GB) and has a subset of the full dataset containing Published 19 April 2023
only runs with the chair target. We have also included the exact synthetic datasets we used for 10.1126/scirobotics.adc8892
our experiments. We have both a full dataset http://knightridermit.myqnapcloud.com:8080/

Downloaded from https://www.science.org at Massachusetts Institute of Technology on April 19, 2023

Chahine et al., Sci. Robot. 8, eadc8892 (2023) 19 April 2023 14 of 14

 Robust flight navigation out of distribution with liquid neural networks
Makram Chahine, Ramin Hasani, Patrick Kao, Aaron Ray, Ryan Shubert, Mathias Lechner, Alexander Amini, and Daniela
Rus

Sci. Robot., 8 (77), eadc8892.

DOI: 10.1126/scirobotics.adc8892

Downloaded from https://www.science.org at Massachusetts Institute of Technology on April 19, 2023

View the article online
https://www.science.org/doi/10.1126/scirobotics.adc8892
Permissions
https://www.science.org/help/reprints-and-permissions

Use of this article is subject to the Terms of service

Science Robotics (ISSN ) is published by the American Association for the Advancement of Science. 1200 New York Avenue NW,
Washington, DC 20005. The title Science Robotics is a registered trademark of AAAS.
Copyright © 2023 The Authors, some rights reserved; exclusive licensee American Association for the Advancement of Science. No claim
to original U.S. Government Works