2018 NDIA GROUND VEHICLE SYSTEMS ENGINEERING AND TECHNOLOGY

SYMPOSIUM
MODELING & SIMULATION, TESTING AND VALIDATION (MSTV) TECHNICAL SESSION
AUGUST 7-9, 2018 - NOVI, MICHIGAN

HUMAN WORKLOAD MODELING FOR

AUTONOMOUS GROUND VEHICLES
Vlad L. Pop, Ph.D.
W. Stuart Michelson
Human Systems Engineering
Georgia Tech Research Institute
Atlanta, GA

ABSTRACT
Accurate models of operator workload in highly automated ground
vehicles could inform interface design decisions, predict performance impacts of
new systems, and evaluate existing systems. This paper summarizes an existing
methodology for modeling human operator workload, demonstrates its
application to automated ground vehicles, and discusses its value in development,
certification, and acquisition of autonomous military ground systems.

INTRODUCTION human operators as AGS are implemented into the

Autonomous Ground Systems (AGS) play a sensor, C4I, and support grids.
significant role in the DoD’s Third Offset Previous research we have worked on [4] as well
Strategy. As technology matures and more as other studies [5-6] have demonstrated that one
automation-enabled vehicles are fielded the role of of the major consequences of automation is
automation in these ground systems becomes more changes to operator workload. As new automation
complex. Automation has the potential to decrease is implemented operator workload changes
operator workload, increase efficiency, and depending on how much monitoring of the
maintain high levels of safety, as we have seen in automation is required, how much cross-checking
aviation [1]. However, automation can also of the automation is required, how much crew
introduce new cognitive demands in the form of coordination is required, and how much time or
knowledge requirements, data management tasks, effort for task management is required [4]. System
and attentional demands [2]. A large volume of designers can optimize operator workload by
human factors research has focused on ensuring that automation is implemented where it
understanding how operators interact with is most beneficial to the operator, reduces
automation [3]. This research has demonstrated complexity, and does not overwhelm the operator
that introducing new automation without regard [7]. In order to achieve optimal workload in AGS
for human operators can lead to unforeseen designers must be able to assess the operator
problems that risk the potential gains of new workload levels associated with different
technologies [1]. In order to support the step configurations of automation and human operators
increase in performance required for the Third involved in the performance of missions.
Offset Strategy it is critical to assess the impact on The purpose of this paper is to summarize an
existing methodology for modeling human
 Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

operator workload, demonstrate its application to and organized according to system task sequence
automated ground vehicles, and discuss the value to build the structure of the task network model
for development, certification, and acquisition of [14].
autonomous military ground systems. Once the structure of the task network model is
built IMPRINT can be used to evaluate system
WORKLOAD MODELING performance using a detailed analysis of operator
As military systems become increasingly more workload. Workload is analyzed at each subtask,
complex they require system operators to process for each operator, for each action, with each
increasingly larger amounts of information. In specific equipment interface. IMPRINT uses a
order to understand the performance impacts of workload evaluation method based on Multiple
these complex systems the Human Research and Resource Theory [15]. Workload is evaluated for
Engineering Directorate of the U.S. Army each Visual, Auditory, Cognitive, and
Research Laboratory (ARL) has developed, Psychomotor (VACP) dimension [16].
applied, and validated the Improved Performance Validation studies have shown that the VACP
Research Integration Tool (IMPRINT) [8]. method has good predictive validity, providing
IMPRINT was originally developed to assess the workload measures that correlate with subjective
mental workload associated with different workload ratings from real operators, and
configurations of soldiers and complex equipment predicting the same performance differences
[7]. It has since been used in a variety of systems observed in real systems [7].
and demonstrated to be useful for identifying peak The VACP scale, included in Table 1, contains
levels of workload that indicate which tasks different 7.0 point interval scales with verbal
should be reallocated, redesigned, or automated anchors for each mental resource. Each task within
[9]. The tool has been successfully used to the task network model is given separate visual,
determine function allocations in U.S. Navy auditory, cognitive, and psychomotor ratings
destroyers [10], determine the number of operators depending on the demands they place on each
needed in Special Operations command stations component. For instance, if a task requires
[11], determine the crew size needed for the U.S. operators to identify whether a system is ready by
Army automated artillery system [12], determine detecting a light, it is given a visual workload
the performance effects of the U.S. Army Land rating of 1.0, but if a task requires operators to
Warrior integrated fighting system [13], and most identify whether a system is ready by reading, it is
recently used by the authors of this paper to given a visual workload rating of 5.9.
determine function allocation among crew Once all elements within the task network have
members for new capabilities in the C-130H. workload ratings the model is ready to run. The
IMPRINT models human performance using a model structure identifies the tasks that must be
task network architecture approach. Human accomplished for each mission, the sequence of
behavior within complex military systems is subtasks that are performed, which operators
organized by the missions that operators perform. perform them, and the workload demands
Each mission is then decomposed into smaller imposed. The workload profile for each mission
elements according to the tasks that must be can then be analyzed to identify aspects of the
accomplished. Tasks are continued to be broken system design that are complex, overwhelming, or
down into subtasks until all human-system degrade performance.
interaction is described as a closed-loop function. IMPRINT has been used throughout the lifecycle
Once all behavior is broken down into small of a wide array of complex military systems and
elements tasks and subtasks are linked together may be a valuable tool for evaluating AGS.

Human Workload Modeling for Autonomous Ground Vehicles

Page 2 of 7
 Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

Visual Auditory
1.0 Visually Register/Detect (detect occurrence of image) 1.0 Detect/Register Sound (detect occurrence of sound)
3.7 Visually Discriminate (detect visual differences) 2.0 Orient to Sound (general orientation/attention)
4.0 Visually Inspect/Check (discrete inspection/static condition) 4.2 Orient to Sound (selective orientation/attention)
5.0 Visually Locate/Align (selective orientation) 4.3 Verify Auditory Feedback (detect anticipated sound)
5.4 Visually Track/Follow (maintain orientation) 4.9 Interpret Semantic Content (speech)
5.9 Visually Read (symbol) 6.6 Discriminate Sound Characteristics (detect auditory differences)
7.0 Visually Scan/Search/Monitor (continuous/serial inspection) 7.0 Interpret Sound Patterns (pulse rates, etc.)

Cognitive Psychomotor
1.0 Automatic (simple association) 1.0 Speech
1.2 Alternative Selection 2.2 Discrete Actuation (button, toggle, trigger)
3.7 Sign/Signal Recognition 2.6 Continuous Adjustive (flight control, sensor control)
4.6 Evaluation/Judgment (consider single aspect) 4.6 Manipulative
5.3 Encoding/Decoding, Recall 5.8 Discrete Adjustive (rotary, thumbwheel, lever position)
6.8 Evaluation/Judgment (consider several aspects) 6.5 Symbolic Production (writing)
7.0 Estimation, Calculation, Conversion 7.0 Serial Discrete Manipulation (keyboard entries)

Table 1: The Visual Auditory Cognitive Psychomotor (VACP) Workload Scale.

without necessarily requiring auditory activity,

APPLICATION TO GROUND VEHICLES thus is receives a VACP scale rating of 0. For
The application of IMPRINT to automated cognitive activity, Manual Driving requires the
ground vehicles is discussed using a detailed driver to evaluate and judge several aspects,
example. In order to keep this paper Distribution including own speed, speed of other vehicles, and
A and releasable to the public, the example is the intention of other drivers, thus is receives a
focused on evaluating the capabilities of cognitive workload of 6.8. Finally, for
automated systems that are currently psychomotor activity, Manual Driving requires the
commercially available in passenger vehicles. driver to continuously adjust the steering wheel
Nevertheless, all the methods, analyses, and and pedals, thus it receives a psychomotor rating
applications discussed are the same across AGS. of 2.6 on the VACP scale.
In this example we apply IMPRINT to compare Once the baseline workload for manual driving is
manual driving, Tesla Autopilot, and Cadillac assed in each VACP dimension we move on to
Super Cruise™ for the task of maintaining a lane adding the automation subtasks. We begin with
during Interstate highway driving. We begin with Engaging the Automation. For visual workload,
a task network approach and decompose the task both Tesla Autopilot and Cadillac Super Cruise™
of using automation into the following subtasks: require the driver to visually detect a steering
Manually Driving, Engaging Automation, wheel icon on the dash (1.0 VACP rating), while
Confirming Engagement, “Driving” with also still manually driving (7.0 VACP rating), thus
Automation, Automation Disengagement, and the total visual workload is 8.0 for both systems.
Manual Disengagement. No auditory activity is required for engagement,
Once the task is decomposed we apply the so auditory workload remains at 0. For cognitive
VACP workload scale to each subtask. We begin workload, the driver had to make additional
with Manual Driving. Manual Driving requires the decisions about the automation in addition to the
driver to continuously scan the road and monitor evaluations and judgements while manually
the position of the vehicle in relation to the lane driving (6.8 VACP rating). With Tesla Autopilot
markings as well as other vehicles on the road, the driver must make a simple association between
thus it receives a visual workload of 7.0 on the the steering wheel icon on the dash and engaging
VACP scale. Manual Driving can be performed the automation (1.0 VACP rating), thus the total

Human Workload Modeling for Autonomous Ground Vehicles

Page 3 of 7
 Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

cognitive workload was 7.8. With Cadillac Super Tesla Autopilot the driver must confirm that the
Cruise™ the driver must make an alternative system is engaged by detecting a visual difference
selection, first engaging the radar cruise control, in the color of the steering wheel icon on the right
then engaging Super Cruise, thus a 1.2 VACP of the speedometer, resulting in a visual workload
rating is given in addition to the 6.8 VACP rating rating of 3.7. With Cadillac Super Cruise™ the
for manual driving, for a total of 8.0 VACP rating. driver must confirm that the system is engaged by
For psychomotor workload, both Tesla Autopilot visually registering that the top of the steering
and Cadillac Super Cruise™ require the driver to wheel has now been illuminated green, resulting in
perform a discrete actuation (2.2 VACP rating) via a visual workload rating of 1.0.
a button or knob to activate the automation while Figure 1 also shows a difference in workload
also continuing to adjust the steering wheel and while “Driving” with Automation across the two
pedals (2.6 VACP rating), thus both systems systems. Although both systems decreased
receive a total psychomotor workload rating of workload levels in comparison to manual driving,
4.8. Tesla Autopilot requires drivers to maintain their
Once both systems are evaluated in all VACP hands on the steering wheel, thus imposing more
dimensions for the first subtask of Engaging psychomotor workload on drivers than the
Automation the same process is applied to the next Cadillac Super Cruise™ system which is a hands
subtask of Confirming Engagement, and then so free system.
on to “Driving” with Automation, Automation
Disengagement, and Manual Disengagement. The DISCUSSION
end result of all VACP ratings, for all subtasks, for Our goal for this paper was to summarize the
each system is included in Table 2. The results are IMPRINT approach to human workload modeling
then graphed to view workload for each dimension and discuss a detailed example of how it can be
and in total, as demonstrated in Figure 1. applied to inform design decisions in AGS. Our
Looking at Figure 1 we can first see some trends example compared manual driving, Tesla
across both systems. In general both Tesla Autopilot, and Cadillac Super Cruise™ while
Autopilot and Cadillac Super Cruise™ briefly maintaining a lane on an Interstate highway. The
increase driver workload during engagement and results demonstrated that IMPRINT can be used to
disengagement tasks. This is to be expected since evaluate the performance impacts of new ground
the automation must be engaged while the driver vehicle automation in several ways. First, this
is still also maintaining manual control of the methodology can be used to evaluate system
vehicle. On the other hand, once automation is design decisions. In our example we found greater
engaged, both Tesla Autopilot and Cadillac Super visual workload with the system indicating that
Cruise™ decrease driver workload in comparison the automation is engaged by changing the color
to manual driving. Overall this type of analysis of an icon versus the additional presence of light
shows where the addition of automation is or image. Second, this methodology can be used to
beneficial to operator workload and where it is identify when the addition of automation benefits
not; however, the analysis also reveals some operators and when it does not. In our example we
differences between the two systems. found increased workload during automation
Looking at Figure 1 again, we see that Tesla engagement and automation disengagement but
Autopilot induces more workload on the operator decreased workload while automation was in use.
during engagement and disengagement than Finally, this methodology can be used to make
Cadillac Super Cruise™ does. This difference in objective comparisons across systems. For
workload stems from each system’s design. With example we could quantify the difference in

Human Workload Modeling for Autonomous Ground Vehicles

Page 4 of 7
 Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

Figure 1: Graphical VACP Workload Ratings for Each Automation Subtask

Tesla Autopilot Cadillac Super Cruise™

Task Vis. Aud. Cog. Psy. Tot. Vis. Aud. Cog. Psy. Tot.
Engaging Automation 8.0 0.0 7.8 4.8 20.6 8.0 0.0 8.0 4.8 20.8
Confirming Engagement 10.7 1.0 8.0 2.6 22.3 8.0 0.0 8.0 2.6 18.6
"Driving” with Automation 4.0 0.0 4.6 2.6 11.2 4.0 0.0 4.6 0.0 8.6
Automation Disengagement 8.0 1.0 10.5 2.6 22.1 8.0 1.0 10.5 2.6 22.1
Manual Disengagement 10.7 0.0 8.0 2.6 21.3 8.0 0.0 8.0 2.6 18.6
Table 2: VACP Workload Ratings for Each Automation Subtask.

Human Workload Modeling for Autonomous Ground Vehicles

Page 5 of 7
 Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

operator workload between the hands-on and in highly automated systems can successfully
hands-off automated driving system. inform user interface design decisions, predict
The work described in this paper highlights how performance impacts of new systems, and evaluate
IMPRINT can be used to inform design decisions existing systems. IMPRINT shows great potential
in AGS, but this is only the beginning of this for valuable applications in the AGS domain and
application. Our example was purposefully simple we hope to see it further developed.
and only had drivers engage the vehicle
automation while performing a single baseline REFERENCES
task of maintaining a lane on the highway in order [1] Durso, F. T., Feigh, K., Fischer, U., Morrow,
to show how the model computes workload across D., Mosier, K., Pop, V. L., Sullivan, K.,
concurrent tasks by Visual, Auditory, Cognitive, Blosch, J., & Wilson, J. (2011). Automation
and Psychomotor dimensions. In reality the value input features from the modern cockpit: toward
of IMPRINT is the ability to model the potential a human-automation relationship taxonomy.
overlap between the wide varieties of driving tasks Washington, DC.
that occur for different types of driving missions, [2] (Amalberti, R., & Sarter, N. B. (2000).
in different environmental conditions. The Cognitive engineering in the aviation domain -
application of IMPRINT to AGS should model Opportunities and challenges. In N. B. Sarter
VACP workload demands of automation tasks and R. Amalberti (Eds.), Cognitive engineering
across the variety of secondary automotive tasks in the aviation domain. Mahwah, NJ:
used in National Highway Traffic Safety Lawrence Erlbaum Associates. (pp. 1-9).
Administration research, including vehicle device
oriented tasks such as manually tuning the radio, [3] Sheridan, T. B., & Parasuraman, R. (2006).
navigation, communication, and entertainment, Human-automation interaction. Review of
portable device tasks including cell phones and human factors and ergonomics (Vol. 1, pp. 89–
tablets, and non-device oriented tasks including 129). Santa Monica, CA: Human Factors and
eating, drinking, grooming, and attending to Ergonomics Society.
passengers [17]. [4] Durso, F. T., Stearman, E. J., Morrow, D. G.,
The value of IMPRINT in the AGS domain Mosier, K. L., Fischer, U., Pop, V. L., & Feigh,
spans beyond system design. IMPRINT is utilized K. M. (2015). Exploring relationships of human
throughout the lifecycle of a wide array of automation interaction consequences on pilots:
complex military systems. As AGS are developed Uncovering Subsystems. Human Factors,
and actual performance data becomes available the 57(3), p. 397-406.
task network model within IMPRINT can be [5] Durso, F. T., & Alexander, A. (2010).
augmented with performance data for each Managing workload, performance, and
individual task for mean time, standard deviation, situation awareness in aviation systems. In E.
distribution curve, and completion rate [18]. As a Salas & D. Maurino (Eds.) Human factors in
result, these types of expanded task network aviation (pp. 217–247). London: Elsevier.
models have demonstrated to provide useful, [6] Parasuraman, R., Sheridan, T. B., & Wickens,
valid, and accurate predictions of the situations C. D. (2008). Situation awareness, mental
and circumstances in which human errors will workload, and trust in automation: Viable,
occur within a system [19]. empirically supported cognitive engineering
A wide variety of previous work in other constructs. Journal of Cognitive Engineering
transportation domains has demonstrated that and Decision Making, 2, 140–160.
accurate quantitative models of operator workload

Human Workload Modeling for Autonomous Ground Vehicles

Page 6 of 7
 Proceedings of the 2018 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

[7] Mitchell, D.K. (2000). Mental workload and analysis of the land warrior system (Document
ARL workload modeling tools. Aberdeen Number ARL-CR-291). Aberdeen Proving
Proving Ground, MD, US Army Research Ground, MD: U.S. Army Research Laboratory.
Laboratory, Human Research & Engineering [14] Leiden, K., Laughery, K. R., Keller, J. W.,
Directorate: 35. French, J. W., Warwick, W. & Wood, S.D.
[8] Improved Performance Research Integration (2001). A Review of Human Performance
Tool, IMPRINT (2009). [Computer Software]. Models for the Prediction of Human Error.
Available from the U.S. Army Research Boulder, CO: Micro Analysis and Design, Inc.
Laboratory website at: [15] Wickens, CD. (1991). Processing resources
http://www.arl.army.mil/www/default.cfm?pag and attention. In D.L. Damos (Ed.), Multiple
e=3200 Task Performance (pp. 3-34). Washington, DC:
[9] Cain, B. (2007). A review of the mental Taylor & Francis.
workload literature (Report Number RTO-TR [16] McCrasken, J. H., & Aldrich, T. B. (1984).
HFM-121-Part-II). Toronto, Canada: Defence Analysis of selected LHX mission functions:
Research and Development, Human System Implications for operator workload and system
Integration Section. automation goals (Technical Note ASI479-024-
[10] Archer, R.D., & Lockett, J.F. III (1997). 84). Fort Rucker, AL: Army Research Institute
WinCrew - a tool for analyzing performance, Aviation Research and Development Activity.
mental workload and function allocation among [17] Angell, L., Auflick, J., Austria, P.A.,
operators. Proceedings of the First International Kochhar, D., Tijerina, L., Biever, W.,
Conference on Allocation of Functions, Diptiman, T., Hogsett, J., & Kiger, S. (2006).
Galway, Ireland. Driver Workload Metrics Task 2 Final Report.
[11] Malkin, F.J., Allender, L.E., Kelley, T.D., National Highway Traffic Safety
O’Brien, P., & Graybill, S. (1997). Joint base Administration. Report No. DOT HS 810 635.
station variant 1 MOS-workload-skill [18] Wong, D. T. (2010). Validating Human
requirements analysis (Document Number Performance Models of the Future. Orion crew
ARL-TR-1441). Aberdeen Proving Grounds, exploration vehicle, 54th Annual Meeting
MD: Army Research Laboratory. Human Factors and Ergonomics Society, San
[12] Beideman, L.R., Munro, I., & Allender, L.E. Francisco, CA, pp. 1002–1006.
(2001). IMPRINT modeling for selected [19] Pop, V. L. (2015). Using task network
crusader research issues. Aberdeen Proving modeling to predict human error. Georgia
Ground, MD: U.S. Army Research Laboratory. Institute of Technology, Atlanta, GA.
[13] Adkins, R., Murphy, W., Hemenway, M.,
Archer, R., & Bayless (1996). HARDMAN III

Human Workload Modeling for Autonomous Ground Vehicles

Page 7 of 7