Current Robotics Reports (2021) 2:265–272

https://doi.org/10.1007/s43154-021-00062-5

SPACE ROBOTICS (Y GAO, SECTION EDITOR)

A Review of NASA Human-Robot Interaction in Space

Kimberly Hambuchen 1 & Jessica Marquez 2 & Terrence Fong 2

Accepted: 18 June 2021 / Published online: 7 August 2021

# This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply 2021

Abstract
Purpose of Review This review provides an overview of the motivation, challenges, state-of-the-art, and recent research for
human-robot interaction (HRI) in space. For context, we focus on NASA space missions, use cases, and systems (both flight and
research). However, the discussion is broadly applicable to all activities in space that require or make use of human-robot teams.
Recent Findings To date, HRI in space has largely been limited to remote interaction between humans on Earth and robots in
space. This interaction is associated with telerobotic operations—from direct (manual) control to intermittent, supervisory
control. Recent work, however, has begun to address a wide range of human-robot arrangements (co-located, remote, 1:1,
groups, etc.). In addition, researchers have been studying human-robot teaming theory and system design, efficient interaction
methods, and human-robot communication.
Summary We begin by describing NASA’s use of robots in space for both deep space science and human exploration. We then
describe several aspects of HRI that are important for space missions, with emphasis on factors that are critical or unique for the
space environment. Next, we provide a brief overview of HRI associated with space systems, including technology demonstra-
tions. Finally, we conclude with a short survey of recent research, which will affect human-robot interaction for both Artemis
missions and future missions to Mars.

Keywords Remote and supervised control . Proximate interaction . Distributed collaboration and coordination . Common
human-system interfaces . Space robotics

Introduction Artemis missions seek to achieve sustainable human presence

beyond low Earth orbit (LEO) [4]. The success of both current
NASA has a successful history of using robotic systems to and future human space exploration missions depends on ef-
enable past and current missions, such as the building and fective integration of humans and robotic technology [5].
maintenance of the International Space Station (ISS) and ex- NASA’s experience with in-space robots includes robotic
ploration of the surface of Mars [1–3]. Future NASA missions arms, rovers, in-flight free-flyers, and anthropomorphic sys-
will become even more reliant on robotic systems to achieve tems. The majority of NASA’s robotic flight experience to
mission objectives, leading to successful outcomes. Many of date is with large robotic manipulators on the Space Shuttle
these missions will be uncrewed; however, the upcoming and the ISS, and with planetary exploration rovers on Mars.
To prepare for future human exploration missions, NASA has
This article is part of the Topical Collection on Space Robotics explored human-robot teams in space, achieving technology
demonstrations of intra-vehicular robotic systems in space, in-
* Kimberly Hambuchen cluding the Robonaut 2 humanoid [6, 7] and free-flyers, specif-
[email protected] ically the “Smart Synchronized Position Hold, Engage, Reorient,
Experimental Satellites” (SPHERES) [8] and Astrobee [9].
Jessica Marquez
[email protected] Additionally, astronaut crews have experimented with in-flight
teleoperation of an external free-flyer, Autonomous
Terrence Fong
[email protected]
Extravehicular Activity (EVA) Robotic Camera (AERCam)
Sprint robot [10], and a surface rover [11].
1
NASA Johnson Space Center, Houston, TX, USA Artemis missions will rely on the Gateway, a small outpost
2
NASA Ames Research Center, Moffett Field, CA, USA
orbiting the Moon that provides vital support for a sustainable,
266 Curr Robot Rep (2021) 2:265–272

long-term human return to the lunar surface, as well as a stag- reduced (or zero) gravity environments, and operation on oth-
ing point for deep space exploration [12]. While some opera- er planetary bodies with associated issues due to radiation,
tions concepts for the Gateway can be taken directly from temperature, illumination, dust, etc. [15]. There may be one
lessons learned on the ISS, it is anticipated that the Gateway or more robots, with differing capabilities, that astronauts need
will be uncrewed for a large portion of every year. This pro- to manage at the same time [16•].
vides an opportunity to begin using robotic systems to main- Non-collocated human-robot teams will experience com-
tain and manage the Gateway while humans are not present. munication latencies from seconds to hours round trip, with
These intravehicular robots (IVR) will mostly be interacting network bandwidths ranging from a few hundred bits per sec-
with ground-based human support in the near term, but in the ond to a few megabits per second. Additionally, these teams
long term, opportunities exist to have IVR and crew interact may intermittently be unable to communicate. Robot
within the Gateway. Additionally, it is expected that astro- team members may be located in habitats (both in-
nauts on the lunar surface will use lunar rovers to increase orbit and surface) or on planetary surfaces. Human team
the surface distance which they can explore [13]. These ro- members may be on Earth, in orbiting habitats, or col-
vers, unlike the Apollo Lunar Rover, will be robotic systems located with the robotic systems.
capable of being supervised remotely by ground operators, As such, there are various challenges in both the design of
and possibly managing their own operations while human teams and procedures for mission scenarios, along with the
crew are residing within them [14]. Future Artemis missions operations of humans and robots in a safe, efficient, and ef-
may also include robotic systems constructing lunar surface fective manner. Many mission objectives cannot be met with-
habitats and executing in situ resource processing. While most out robots; however, integrating robots into spaceflight oper-
interaction with these systems will occur with ground-based ations inherently introduces an additional risk to astronauts
humans, opportunities will exist to test human-robot interac- and spacecraft safety. Missteps could lead to failed mission
tion paradigms and methods to advance technology for Mars- objectives, injury to crew or vehicle, or in the worst case, loss
based HRI. of crew or mission. As such, NASA has to carefully consider
The new paradigms that NASA faces for Artemis and be- how best to integrate robotic operations into spaceflight. For
yond will allow for flexibility, while at the same time require example, NASA needs to better understand collocated
unique methods for humans and robotic systems to interact. human-robot teams, how humans and robots can safely inter-
Future space missions will likely rely on teams composed of act, not just preventing physical collisions but also potentially
humans and robots with complementary capabilities intended negative social psychological effects on crew members.
to promote mission success. Tasks that are considered dull, NASA also still needs to develop methods to assess team
dirty, or dangerous can be transferred to autonomous robots, cohesiveness and effectiveness during space missions based
thus relieving human crewmembers to perform more complex on human-robot team composition and task allocations [17].
tasks or those requiring real-time modifications due to contin- Additionally, successful robotic integration into spaceflight is
gencies. Additionally, due to the limited number of astronauts compounded by the lack of formal standards (de facto or oth-
anticipated to crew planetary exploration missions, as well as erwise) for HRI [19].
their constrained schedules, ground-control personnel will For both collocated and non-collocated human-robot
likely need to remotely supervise and assist robots when pos- teams, effective collaboration is essential for operational
sible; therefore, NASA’s concept of human-robot interaction implementation. In the context of spaceflight operations,
(HRI) encompasses not only colocated teams, but also human- effective collaboration centers around communication
robot teams distributed across both time and space. Such between the human-robot team and adequate human-
human-robot teams can perform a wide range of tasks in space system performance. Significant progress needs to be
or on planetary surfaces. These include in situ resource utili- made for robots to have the capacity to communicate
zation, assembly of large structures or systems, food produc- the types of multi-modal cues and signals that humans
tion, reconnaissance, and sample collection/return. constantly express to each other, including information
about state, and intentions, without confusion. Similarly,
robots that can understand the state of astronauts (e.g.,
HRI Challenges in Space one that is fatigued or cognitively overloaded) and cor-
rectly shift work allocations may enable more efficient
Due to the unique circumstances of deep space missions, collaborations. NASA also must understand how to de-
many methods developed for terrestrial HRI cannot be easily sign systems where humans can easily and seamlessly
ported to space-based HRI. Spaceflight present unique chal- transition between direct control and supervisory control
lenges for human-robot interaction and collaboration, includ- of robots (or robot teams) without significantly affecting
ing high communication latencies and limited bandwidth be- human performance, such as loss of situation awareness,
tween non-collocated robots and humans, operation in high workload, or misplaced trust [18].
Curr Robot Rep (2021) 2:265–272 267

Since the coordination of a distributed team of humans and and balances. Ground crew operators routinely support EVA
robots is complex, the development of tools and techniques and EVR as the second robotics operator. In addition to the
that allow each team member to operate effectively with each base equipment provided at the RWS, operators rely on addi-
other is critical [20]. HRI methods that enable humans (both tional camera views accessible on laptop computers posi-
ground control and astronauts) and robots to communicate tioned near the RWS and visual display cues, such as targets
clearly about their capabilities, intent, state, and accomplish- overlaid onto video feeds. Ground can download up to six
ments will increase the likelihood of mission success, as well distinct video feeds, and can use overlays as well to perform
as increasing the capability of human-robot team members to contact operations (e.g., grappling and grasping).
coordinate and collaborate to solve problems. Lastly, given
the diversity of human-robot teaming scenarios in space, it is
essential that a variety of HRI tools and techniques be devel- Mars Surface
oped. In particular, the types, frequency, and criticality of
interactions will vary depending on whether humans and ro- Surface exploration robots, Sojourner, Phoenix, the Mars
bots interact in proximity, remotely without significant time Exploration Rovers (MER), and Mars Science Laboratory
delays, or remotely with significant time delays [21]. (MSL), have been used extensively for over a decade but their
operations are open-loop, where ground operators must send
sequences of commands and wait for data from those com-
Human-robot Teams at NASA mands to be received [24, 25]. Real-time commands are not
possible due to the communication latency and the intermit-
NASA currently operates multiple robots in space. Systems in tent communication availability between Earth and Mars.
low Earth orbit (LEO) provide the most direct interaction; as MER operations are constrained primarily by rover capabili-
the communication latency is low (1–1.5-s round trip), band- ties and a mission operation strategy that must accommodate
width can be significant (> 1 Mbps), and availability is high 6- to 44-min round-trip communication latencies plus the ab-
(i.e., mostly continuous with well-defined loss-of-signal pe- sence of a continuous communications link. These constraints
riods). Additionally, humans are continuously in LEO aboard mandate a multi-step human interaction process for the ro-
the ISS, so astronauts onboard ISS can always provide more ver(s) to execute activities on the surface of Mars. Beginning
information to ground control to increase situation awareness. with receipt of a downlink from Mars and assessment of its
NASA also operates robotic rovers on the Martian surface. content, science activities are planned, refined, validated,
However, communication latency to Mars can vary from 6- reviewed, and then approved. This results in the generation
to 44-min round trip, and there are no humans on the surface of command sequences that must be resolved and integrated,
to assist with situation awareness. Along with these robotic validated, and reviewed before commands are transmitted to
systems, NASA has demonstrated other robots in space for the spacecraft.
future mission needs [22]. McCurdy describes the iterative process that evolved the
tools used by the Mars robot operators to plan and execute
Low Earth orbit commands [26]. After tactical processes were implemented,
they found that they were not able to support the deadlines that
Optimized robotics operations are essential for spaceflight op- the tasks demanded. Planning tools were developed to meet
erations, including ISS maintenance and extravehicular robot- the necessary timelines, but these too suffered inefficiencies
ics (EVR), and it has been mainly focused on onboard and [27]. After applying human-centered principles, several inter-
ground-based telerobotics [23]. Telerobotic operation onboard face issues were identified. McCurdy found that a large num-
includes the proper setup and configuration of the onboard ber of tools designed by different groups for differing pur-
robotic workstation (RWS) and subsequent actuation and ex- poses were inconsistent and non-cohesive, leading to steep
ecution of arm operations. The RWS for the Space Station learning curves as well as performance issues. Additionally,
Remote Manipulator System (SSRMS) consists of a laptop the primary timeline tool was based on outdated legacy soft-
computer, translational and rotational hand controllers, three ware and developed with a different user population in mind.
video monitors, and a display and control panel (DCP). This, in turn, had a significant impact that depended on the
Although the workstation can be operated by a single operators’ ability to make or miss an uplink deadline. Through
crewmember, in practice, RWS activities are conducted with process improvements and tools, the operations schedule
two crewmembers. One person acts as the primary controller, shifted from 7 to 5 days a week. As a result of this analysis,
and the second crewmember is dedicated to controlling cam- McCurdy and his team were able to design and develop new
eras, tracking procedure steps, and confirming the direction of tools for the next Mars mission, Phoenix Mars Lander, and
motion. Astronauts report that this allows them to increase further improved for Mars Science Laboratory. Future opera-
their situation awareness and maintain a system of checks tions of Mars 2020 mission Perseverance rover aim to
268 Curr Robot Rep (2021) 2:265–272

significantly decrease the time required to plan and develop control personnel to run checks and balances on every com-
execution commands as well as integrate new onboard mand sent. The modifications done on R2 while onboard ISS
autonomy. allow the robot to become more interactive and autonomous
[30]. While R2 was onboard the ISS, technology development
In-Space Technology Demonstrations was continuous on Earth to increase the interactiveness of the
robot [31].
Free-flying intra-vehicular space robots can be used when
humans are present to off-load routine work, to increase crew
productivity, and to handle contingencies. Such robots can Research and Development
also be used during quiescent periods (i.e., when spacecraft
are unmanned) to perform spacecraft caretaking (mainte- NASA continues to investigate human-robot interaction in
nance, monitoring, etc.). From 2011 to 2012, NASA conduct- order to enable future mission operations, funding research
ed a series of tests with the “Smart Synchronized Position internally and externally through academic institutions.
Hold Engage and Reorient Experimental Satellites” (Smart Ultimately, basic research will lead to in-space technology
SPHERES) free-flying robots on the ISS [8]. These tests in- demonstrations, an essential first step for mission operations
cluded video surveys of the ISS Kibo laboratory to demon- deployments. The following sections highlight current and
strate that ground control can interact with a semi-autonomous recent research funded by NASA to advance technology read-
robot to perform a task that would normally be performed by iness for human-robot interaction in space. While this list is
astronauts. not exhaustive, the authors have chosen to highlight research
During Summer 2013, NASA tested a crew-centric “sur- focused on advancing human-robot interaction for lunar and
face telerobotics” concept of operations using the ISS as a Mars exploration missions, whether that interaction is co-
proxy for a human spacecraft orbiting the Moon [28, 29]. located or distributed across time and space.
Over the course of three test sessions, Expedition 36 astro-
nauts on the ISS remotely operated NASA’s “K10” planetary Human-Robot Teaming
rover in an outdoor terrain located at the NASA Ames
Research Center. The astronauts used a Space Station Ma et al. surveyed various ways to build and construct human-
Computer (Lenovo Thinkpad laptop), supervisory control robot teams, particularly for future space missions [32]. The
(command sequencing with interactive monitoring), survey defined communication, coordination, and collabora-
teleoperation (discrete commanding), and Ku-band satellite tion as the cornerstones for human-robot teamwork. The sur-
communications to operate K10 for a combined total of 11 vey also identified several key components of teaming, in-
h. These tests indicate that command sequencing with inter- cluding agent abilities, taskwork, metrics, and peer-to-peer
active monitoring is an effective strategy for surface interactions.
telerobotics, where astronauts would operate rovers on a plan- The “Peer to Peer Human-Robot Interaction” (P2P-HRI)
etary surface (Moon, Mars, etc.) from orbit. project explored how human-robot teams could operate inde-
Robonaut 2 (R2) is a humanoid robot designed to operate pendently of ground control during future space exploration
in a human environment and to execute tasks that would nor- missions [33, 34]. P2P-HRI focused on (1) developing natural
mally be managed by astronauts [6, 7]. R2 was stationed on interaction mechanisms, (2) reducing the workload associated
the ISS from February 2012 to May 2018 as a payload to with robot control, and (3) maximizing the work that the
conduct initial testing for future caretaking robots for human humans and robots can accomplish together. A key aspect of
space habitats. R2 operations are managed by two methods: the research was to study how humans and robots could work
telepresence teleoperation by crew on the ISS and ground as “equal” partners, i.e., that their roles should be as balanced
supervisory control. Telepresence operation presents an op- as possible. The project provided some evidence that it is
portunity to take advantage of the humanoid form of R2 for possible to compensate for failures and limitations of robot
a natural operation environment; however, since telepresence autonomy through interaction and teamwork.
operation is a method of “joysticking,” this method is not Gombolay et al. have recently begun research into multiple
useful for any situation where latency exists between the hu- topics for increasing the performance level of these teams,
man and robot. Additionally, this method requires a crew including inferring human goals from suboptimal demonstra-
member to devote their time to operating the robot, as opposed tion, characterizing performance-workload relationship, and
to handling other mission-critical tasks. Supervisory control robot explanation of internal state [35]. The work is primarily
from the ground requires more process and interfaces, but it applicable to co-located human-robot teams that work togeth-
decreases the need for highly valuable crew time to assist the er in a real-time, interactive manner, which is relevant for
robot in completing its tasks. Initially, the interaction with R2 future IVR that work with co-located crew. There is, however,
on ISS was tightly coordinated and dependent on ground- some work on providing decision support for robot
Curr Robot Rep (2021) 2:265–272 269

teleoperators in terms of providing suggestions for handling Efficient Interaction Methods

failures.
Two NASA research institutes have been developed to Chernova et al. have developed remote manipulation inter-
research how deep space habitats should be designed for au- faces that are robust to high latency environments, researching
tonomous operations, including how to advance human-au- how to leverage depth data in remote robot teleoperation in-
tonomy/robot interaction. The Habitats Optimized for terfaces for general object manipulation [43, 44] and develop-
Missions of Exploration (HOME) STRI is researching how ing temporal models for robot classification of human
to integrate humans as a critical link in a smart spacecraft interruptibility for modeling availability of collocated human
architecture, and how to make the entire system (humans in- crew members [45].
cluded) self-sufficient, such that there is minimal need for A key challenge for using robots to support human explo-
ground-control humans to continuously interact with the sys- ration is orienting remote personnel about the robot’s opera-
tem [36, 37]. The Resilient Extra-Terrestrial Habitats (RETH) tions as latency and communication constraints make real-
STRI seeks to develop and demonstrate transformative smart time monitoring impractical. Communication bandwidth will
autonomous habitats and related technologies that will adapt, be limited, making it essential to downlink high-priority infor-
absorb, and rapidly recover from expected and unexpected mation early. Schreckenghost, Milam, and Fong studied how
disruptions to deep space habitat systems without fundamental “any time summaries” can enable humans with different per-
changes in function or sacrifices in safety [38]. One of the spectives to understand the operational situation at any time
research thrusts for this STRI is to develop and validate ge- without relying on continuous vigilance monitoring [46].
neric, robust, and scalable methods for detection and diagno- When adjustable autonomy is used to flexibly allocate
sis of anticipated and unanticipated faults that incorporates an tasks among a human-robot team, performance metrics are
automated active learning framework with robots- and needed to evaluate the team productivity and success [47,
humans-in-the-loop. 48]. In particular, these metrics can be used to ensure that
Gervits et al. have studied how a shared mental model humans and robots are used effectively, as well as to improve
(SMM) can be applied to human-robot teaming in space, in- the efficiency of operations.
cluding both co-located and widely distributed teams [39, 40]. Ensuring human safety is one of the most important con-
A SMM is a construct that contains the knowledge and func- siderations for human-robot teaming in space. This does not
tions necessary for successful teamwork, including monitor- simply involve preventing collisions between humans and ro-
ing goal and task states, evaluating performance of teammates, bots operating within a shared space, but also requires consid-
inferring beliefs and intentions, tracking task focus, and eration of all possible ways in which harm could come to a
adapting behavior as a result of this knowledge. For space person, ranging from physical contact to adverse psychologi-
applications, a SMM can help coordinate human-robot activ- cal effects. Lasota, Fong, and Shah defined what safe HRI
ity across multiple spatial ranges, time scales, task ordering, entails and presented a survey of potential methods of ensur-
interaction modalities, etc. ing safety during HRI [49•].
Reig et al. have studied different ways a human-robot team Real-time remote robotic science operations will be a key
can recover from failure, particularly when multiple robots are part of future planetary exploration [50]. In contrast to current
involved [41]. Real-world human-robot interaction is messy, high-latency telerobotic science missions, real-time remote
and failures are bound to occur. When these failures happen, a robotic science operations require highly effective coordina-
robot’s immediate response can have critical and lasting ef- tion and collaboration between science teams, who define the
fects on the human’s perceptions of the robot. Multi-robot science plan and analyze real-time data collected as a result of
systems have a number of options for how to recover from executing that plan; robot operators, who perform navigation
failures in ways that repair trust and other aspects of human- of robots and ensure system functionality; and planetary ro-
robot relationships. vers, which explore remote locations. Moreover, real-time
McGuire et al. examined the open problem of how robots science operations rely on diverse, emerging information
can seek assistance to recover from failures [42]. Their work and require fast, effective decision-making, often in a matter
focused on the use of indicators of opportunity (physiological of minutes or even seconds. Therefore, operations software
state measurements, contextual indicators, etc.) as predictors that supports efficient monitoring of science data and planning
of the performance of each potentially available assistant. is essential.
Recent experiments suggest that reinforcement learning can
achieve better performance than a non-adaptive assistant se- Human-Robot Communication
lection policy.
Szafir et al. have developed methods for communicating robot
status at a glance and scalable robot interface technologies that
support both proximal and distal operation [51, 52].
270 Curr Robot Rep (2021) 2:265–272

Tellex et al. have developed methods to increase the speed success. We have presented NASA’s solutions to human-
and accuracy of inferring human intention from natural lan- robotic teaming in current and past mission scenarios, future
guage. These methods are intended to increase the number of mission scenarios that will need efficient and safe HRI, and
robots that a single astronaut could supervise [53]. challenges that NASA currently anticipates as barriers for
Efficient and accurate interpretation of instructions is par- successful missions involving human-robot teaming. Current
ticularly important for space missions involving robotic part- technology and research developments addressing NASA’s
ners where communication or interaction between humans unique circumstances in HRI have been described.
and robots is intermittent and bandwidth limited since the
robot may not always have the ability to request a clarification
when performing the task. Arkin et al. have developed trans- Declarations
parent and computationally efficient models for verifiable
grounded language communication [54]. Conflict of Interest The authors declare no competing interests.
Szafir et al. have developed a variety of novel systems for
Human and Animal Rights and Informed Consent This article does not
HRI with free-flying robots [55, 56]. These systems include
contain any studies with human or animal subjects performed by any of
the use of signaling via lights and movement, as well as con- the authors.
sideration of co-located and remote human-robot team
configurations.
Cha et al. have surveyed the use of non-verbal signals for
human-robot interaction. With space robots being increasing- References
ly utilized for tasks that require them to not only operate in
close proximity to humans but to interact with them as well, Papers of particular interest, published recently, have been
there has been great interest in the communication challenges highlighted as:
associated with the varying degrees of interaction in these • Of importance
environments. The success of such inter-actions depends on
robots’ ability to convey information about their knowledge, 1. Ticker RL, Cepollina F, Reed BB. AIAA Space 2015 Conference
intent, and actions to co-located humans [57–59]. and Exposition. In: NASA’s in-space robotic servicing; 2015. p.
4644. https://doi.org/10.2514/6.2015-4644.
2. Callen P. Robotic transfer and interfaces for external ISS payloads.
3rd Annual ISS Research and Development Conference; 2014.
Conclusion https://ntrs.nasa.gov/api/citations/20140008717/downloads/
20140008717.pdf.
While some descriptions of HRI in uncrewed missions have 3. Rankin A, Maimone M, Biesiadecki J, Patel N, Levine D, Toupet
O. Driving curiosity: Mars rover mobility trends during the first
been presented, this paper does not address all of the chal-
seven years. In: 2020 IEEE Aerospace Conference: IEEE; 2020.
lenges that NASA currently has for deep space robotic mis- p. 1–19.
sions to extreme environments. Dynamic surface environ- 4. Smith M, Craig D, Herrmann N, Mahoney E, Krezel J, McIntyre N,
ments, such as found on “ocean worlds” (Europa, et al. The Artemis Program: an overview of NASA’s activities to
Enceladeus, etc.), present challenges that go beyond known return humans to the moon. In: 2020 IEEE Aerospace Conference:
IEEE; 2020. p. 1–10.
methods for remote human-robot interaction, particularly in
5. Ferketic J, Goldblatt L, Hodgson E, Murray S, Wichowski R,
the presence of high communications latency. Additionally, Bradley A, Erkorkmaz C. Toward human–robot interface standards
this article does not address on-orbit servicing, assembly, and II: A closer examination of common elements in human-robot in-
manufacturing missions, such as in-space assembly of large teractions across the space enterprise. In: Proceedings of the 2006
AIAA Space Conference. https://doi.org/10.2514/6.2006-7388.
space structures or science observatories (e.g., radio tele-
6. Diftler MA, Mehling JS, Abdallah ME, Radford NA, Bridgwater
scopes), for which the requirements for HRI are not currently LB, Sanders AM, et al. Robonaut 2-the first humanoid robot in
known. Finally, this paper does not delve into human interac- space. In: 2011 IEEE international conference on robotics and au-
tion with “wearable robotics,” such as exoskeletons or other tomation: IEEE; 2011. p. 2178–83.
intelligent augmentation mechanisms. Many ground-based 7. Diftler MA, Ahlstrom TD, Ambrose RO, Radford NA, Joyce CA,
De La Pena N, et al. Robonaut 2—initial activities on-board the
methods being developed for the interaction of humans and
ISS. In: 2012 IEEE Aerospace Conference: IEEE; 2012. p. 1–12.
augmentation robotics could readily be applied to NASA’s 8. Fong T, Micire M, Morse T, Park E, Provencher C, To V, Wheeler
needs. DW, Mittman D, Torres RJ, Smith E. Smart SPHERES: a
In conclusion, NASA has learned a great deal about how telerobotic free-flyer for intravehicular activities in space. In
humans and robots interact in space. However, there are many Proceedings of AIAA Space. AIAA-2013-5338 2013; 2013.
https://doi.org/10.2514/6.2013-5338.
challenges that lie ahead for deep space missions that require 9. Smith T, Barlow J, Bualat M, Fong T, Provencher C, Sanchez H,
even complex robotic systems and humans, whether collocat- Smith E. Astrobee: a new platform for free-flying robotics on the
ed or remote, to interact as team members to achieve mission International Space Station. In: Proceedings of 13th international
Curr Robot Rep (2021) 2:265–272 271

symposium on artificial intelligence, robotics, and automation in 27. Norris JS, Powell MW, Vona MA, Backes PG, Wick JV. Mars
space; 2016. https://ntrs.nasa.gov/citations/20160007769. exploration rover operations with the science activity planner. In:
10. Fredrickson SE, Duran S, Howard N, Wagenknecht JD. Proceedings of the 2005 IEEE International Conference on
Application of the mini AERCam free flyer for orbital inspection. Robotics and Automation: IEEE; 2005. p. 4618–23.
In: Spacecraft Platforms and Infrastructure, vol. 5419: International 28. Fong T, Bualat M, Burns J, Hopkins J, Pratt W. Testing astronaut-
Society for Optics and Photonics; 2004. p. 26–35. controlled telerobotic operation of rovers from the International
11. Bualat M, Schreckenghost D, Pacis E, Fong T, Kalar D, Beutter B. Space Station as a precursor to lunar missions. IAC-14-A3-2A-7.
(2014) Results from testing crewcontrolled surface telerobotics on In: Proceedings of the 65th International Astronautical Congress.
the International Space Station. iSAIRAS - International Toronto, Canada; 2014. https://ntrs.nasa.gov/api/citations/
Symposium on Artificial Intelligence, Robotics and Automation 20190028898/downloads/20190028898.pdf.
in Space, June 17–19, 2014, Montreal, Quebec. 29. Bualat M, Schreckenghost D, Pacis E, Fong T, Kalar D, Beutter B.
12. Crusan JC, Smith RM, Craig DA, Caram JM, Guidi J, Gates M, Results from testing crew-controlled surface telerobotics on the
et al. Deep space gateway concept: extending human presence into International Space Station. In International Symposium on
cislunar space. In: 2018 IEEE Aerospace Conference: IEEE; 2018. Artificial Intelligence, Robotics, and Automation in Space, June
p. 1–10. 17, 2014' 2014. https://ntrs.nasa.gov/citations/20150007985.
13. Mueller RP, Connolly JC, Whitley RJ. NASA human spaceflight 30. Ahlstrom T, Diftler M, Berka R, Badger J, Yayathi S, Curtis A,
ArchitectureTeam: lunar surface exploration strategies. GLEX- Joyce C. Robonaut 2 on the International Space Station: Status
2012.02.P.17.x12620; 2012. https://ntrs.nasa.gov/api/citations/ update and preparations for IVA mobility. In: Proceedings of
20120008182/downloads/20120008182.pdf. AIAA Space; 2013. https://doi.org/10.2514/6.2013-5340.
14. Harrison DA, Ambrose R, Bluethmann B, Junkin L. Next genera- 31. Farrell L, Strawser P, Hambuchen K, Baker W, Badger J.
tion rover for lunar exploration. In: 2008 IEEE aerospace confer- Supervisory control of a humanoid robot in microgravity for ma-
ence: IEEE; 2008. p. 1–14. nipulation tasks. In: 2017 IEEE/RSJ International Conference on
15. Fong T, Rochlis Zumbado J, Currie N, Mishkin A, Akin DL. Space Intelligent Robots and Systems (IROS): IEEE; 2017. p. 3797–802.
telerobotics: unique challenges to human–robot collaboration in 32. Ma LM, Fong T, Micire MJ, Kim YK, Feigh K. Human-robot
space. Rev Hum Factors Ergon. 2013;9(1):6–56. teaming: concepts and components for design. In: Field and service
16.• Marquez J. Unique considerations for human-robotic interaction in robotics. Cham: Springer; 2018. p. 649–63.
human spaceflight. In: Kanki B, Clervoy J-F, Sandal G, editors. 33. Fong T, Scholtz J, Shah JA, Fluckiger L, Kunz C, Lees D, et al. A
Space Safety and Human Performance: Elsevier; 2017. This book preliminary study of peer-to-peer human-robot interaction. In: 2006
chapter provides information unique to NASA for integrating IEEE International Conference on Systems, Man and Cybernetics,
humans and robotics into human spaceflight missions. vol. 4: IEEE; 2006. p. 3198–203.
34. Fong T, Nourbakhsh I. Peer-to-peer human-robot interaction for
17. IJtsma M, Ma L, Pritchett A, Feigh K. Computational methodology
space exploration. In: Proceedings of the AAAI Fall Symposium:
for the allocation of work and interaction in human-robot teams. J
The Intersection of Cognitive Science and Robotics: From
Cogn Eng Decis Mak. 2019;13(4):221–41.
Interfaces to Intelligence; 2004. https://www.aaai.org/Papers/
18. Marquez JJ, Feary M, Rochlis JZ, Billman D. Evidence report: risk
Symposia/Fall/2004/FS-04-05/FS04-05-014.pdf.
of inadequate design of human and automation/robotic integration.
35. Banerjee S, Gombolay M, Chernova S. A tale of two suggestions:
Houston: NASA Johnson Space Center; 2013.
action and diagnosis recommendations for responding to robot fail-
19. Ferketic J, Goldblatt L, Hodgson E, Murray S, Wichowski R, ure. In: 2020 29th IEEE international conference on robot and
Bradley A, et al. Toward human-robot interface standards: Use of human interactive communication (RO-MAN): IEEE. p. 398–405.
standardization and intelligent subsystems for advancing human- 36. Bales G, Kong Z. Neurophysiological and behavioral correlates of
robotic competency in space exploration: SAE Technical Paper; human-Multiagent task performance. Manuscript; 2020.
2006. 37. Kintz JR, Clark TK. NASA human research program investigators’
20. Fong T, Nourbakhsh I. Interaction challenges in human-robot space workshop: Galveston; 2020. p. 27–30.
exploration. ACM Interact. 2005;12(2):42–5. 38. Maghareh A, Lenjani A, Dyke SJ, Marais K, Whitaker D, Bobet A,
21. Ferketic J, Goldblatt L, Hodgson E, Murray S, Wichowski R, Ramirez J, Modiriasari A, Theinat A. Resilience-oriented design of
Bradley A, et al. Toward human–robot interface standards II: a extraterrestrial habitat systems. In: AIAA propulsion and energy
closer examination of common elements in human-robot interac- 2019 forum; 2019. p. 3972. https://doi.org/10.2514/6.2019-3972.
tions across the space enterprise. In: Proceedings of the 2006 AIAA 39. Gervits F, Fong TW, Scheutz M. Shared mental models to support
Space Conference; 2006. distributed human-robot teaming in space. In: 2018 AIAA SPACE
22. Fong T, Berka R, Bualat M, Diftler M, Micire M, Mittman D, and astronautics forum and exposition; 2018. p. 5340. https://doi.
SunSpiral V, Provencher C. The Human Exploration Telerobotics org/10.2514/6.2018-5340.
Project. GLEX-2012.01.2.4x12180. In: Proceedings of IAF/AIAA 40. Gervits F, Thurston D, Thielstrom R, Fong T, Pham Q, Scheutz M.
Global Space Exploration Conference. Washington, DC; 2012. Toward genuine robot teammates: improving human-robot team
23. Chang M, Marquez JJ. Human-automation allocations for current performance using robot shared mental models. In: Proceedings
robotic space operations: space station remote manipulator system. of the 19th international conference on autonomous agents and
NASA TM -2018-220042. Moffett Field: NASA Ames Research MultiAgent systems; 2020. p. 429–37. https://dl.acm.org/doi/pdf/
Center; 2018. 10.5555/3398761.3398815.
24. Grotzinger JP, Crisp J, Vasavada AR, Anderson RC, Baker CJ, 41. Reig S, Carter E, Fong T, Forlizzi J, Steinfeld A. Flailing, hailing,
Barry R, et al. Mars Science Laboratory mission and science inves- prevailing: perceptions of multi-robot failure recovery strategies. To
tigation. Space Sci Rev. 2012;170(1-4):5–6. appear in proceedings of the ACM conference on human-robot
25. Carsten J, Rankin A, Ferguson D, Stentz A. Global planning on the interaction; 2021. https://dl.acm.org/doi/10.1145/3434073.
Mars Exploration Rovers: software integration and surface testing. J 3444659.
Field Rob. 2009;26(4):337–57. 42. McGuire S, Furlong PM, Fong T, Heckman C, Szafir D, Julier SJ,
26. McCurdy M. Planning tools for Mars surface operations: human- et al. Everybody needs somebody sometimes: validation of adap-
computer interaction lessons learned. In: 2009 IEEE Aerospace tive recovery in robotic space operations. IEEE Robot Autom Lett.
conference: IEEE; 2009. p. 1–12. 2019;4(2):1216–23.
272 Curr Robot Rep (2021) 2:265–272

43. Kent D, Saldanha C, Chernova S. A comparison of remote robot ACM/IEEE international conference on human-robot interaction;
teleoperation interfaces for general object manipulation. In: 2018. p. 78–86. https://dl.acm.org/doi/10.1145/3171221.3171251.
Proceedings of the 2017 ACM/IEEE international conference on 52. Walker M, Hooman H, Lee J, Szafir D. Communicating robot mo-
human-robot interaction; 2017. p. 371–9. https://dl.acm.org/doi/ tion intent with augmented reality. In: Proceedings of the 2018
abs/10.1145/2909824.3020249. ACM/IEEE international conference on human-robot interaction;
44. Kent D, Saldanha C, Chernova S. Leveraging depth data in remote 2018. p. 316–24. https://dl.acm.org/doi/abs/10.1145/3171221.
robot teleoperation interfaces for general object manipulation. Int J 3171253.
Robot Res. 2020;39(1):39–53. 53. Arumugam D, Karamcheti S, Gopalan N, Williams EC, Rhee M,
45. Kent D, Chernova S. Human-centric active perception for autono- Wong LL, et al. Grounding natural language instructions to seman-
mous observation. In: Proceedings of the IEEE international con- tic goal representations for abstraction and generalization. Auton
ference on robotics and automation; 2020. https://doi.org/10.1109/ Robot. 2019;43(2):449–68.
ICRA40945.2020.9197201. 54. Arkin J, Park D, Roy S, Walter MR, Roy N, Howard TM, et al.
46. Schreckenghost DL, Milam T, Fong T. Techniques and tools for Multimodal estimation and communication of latent semantic
summarizing performance of robots operating remotely. In: 14th knowledge for robust execution of robot instructions. Int J Robot
international conference on space operations; 2016. p. 2310. https:// Res. 2020;5:0278364920917755.
doi.org/10.2514/6.2016-2310. 55. Szafir D, Mutlu B, Fong T. Communicating directionality in flying
47. Schreckenghost D, Fong T, Utz H, Milam T. Measuring robot per- robots. In: 2015 10th ACM/IEEE International Conference on
formance in real-time for NASA robotic reconnaissance operations. Human-Robot Interaction (HRI): IEEE; 2015. p. 19–26.
In: Proceedings of the 9th workshop on performance metrics for 56. Szafir D, Mutlu B, Fong T. Designing planning and control inter-
intelligent systems; 2009. p. 194–202. https://dl.acm.org/doi/10. faces to support user collaboration with flying robots. Int J Robot
1145/1865909.1865950. Res. 2017;36(5-7):514–42.
48. Schreckenghost D, Milam T, Fong T. Measuring performance in 57. Cha E, Fitter NT, Kim Y, Fong T, Matarić M. Generating expres-
real time during remote human-robot operations with adjustable sive light signals for appearance-constrained robots. In:
autonomy. IEEE Intell Syst. 2010;23(5):36–45. International Symposium on Experimental Robotics. Cham:
49.• Lasota P, Fong T, Shah J. A survey of methods for safe human- Springer; 2018. p. 595–607.
robot interaction. Foundation and Trends in Robotics. 2017;5(4). 58. Cha E, Fitter NT, Kim Y, Fong T, Matarić MJ. Effects of robot
https://doi.org/10.1561/2300000052. This book discusses safety sound on auditory localization in human-robot collaboration. In:
implications of human robot interaction and methods to Proceedings of the 2018 ACM/IEEE international conference on
increase safe HRI for both the physical and mental well-being human-robot interaction; 2018. p. 434–42. https://dl.acm.org/doi/
of the humans. 10.1145/3171221.3171285.
50. Kim H, Park YW, Baker E, Adams J, Fong T. Design issues for 59. Cha E, Kim Y, Fong T, Mataric M. 2018. A survey of nonverbal
real-time remote robotic science operations support tools: observa- signaling methods for non-humanoid robots. Found Trends Robot.
tions from the field. In: 14th international conference on space 2018;6(4):211–323.
operations; 2016. p. 2476.
51. Hedayati H, Walker M, Szafir D. Improving collocated robot Publisher’s Note Springer Nature remains neutral with regard to jurisdic-
teleoperation with augmented reality. In: Proceedings of the 2018 tional claims in published maps and institutional affiliations.