Lightweight Detection of Physical Unit Inconsistencies without

Program Annotations
John-Paul Ore, Carrick Detweiler, Sebastian Elbaum
Computer Science and Computer Engineering—University of Nebraska
Lincoln, Nebraska, USA 68588-0150
jore,carrick,[email protected]
ABSTRACT 1 INTRODUCTION
Systems interacting with the physical world operate on quantities Systems that interact with the physical world operate on quantities
measured with physical units. When unit operations in a program measured in physical units. Consider a mobile robot that perpetu-
are inconsistent with the physical units’ rules, those systems may ally perceives the world through depth sensors, lasers, cameras, and
suffer. Existing approaches to support unit consistency in programs gyroscopes, and interacts with the world through its actions. The
can impose an unacceptable burden on developers. In this paper, we robot collects measurements as it senses and acts, and transforms
present a lightweight static analysis approach focused on physical them into distances (meters) and angles (radians). The system also
unit inconsistency detection that requires no end-user program integrates these measurements with others such as time (seconds)
annotation, modification, or migration. It does so by capitalizing to derive measures like the robot’s velocity (meters-per-second).
on existing shared libraries that handle standardized physical units, To operate correctly, this kind of system must adhere to both
common in the cyber-physical domain, to link class attributes of the type semantics of the programming language and the unit
shared libraries to physical units. Then, leveraging rules from di- semantics of the physical world. Consider the simple code snip-
mensional analysis, the approach propagates and infers units in pet in Figure 1 belonging to the ‘Romeo’ robot [30]. The expres-
programs that use these shared libraries, and detects inconsistent sion on line 191 calculates the distance between the current posi-
unit usage. We implement and evaluate the approach in a tool, ana- tion and the goal. Normally this kind of distance function would
lyzing 213 open-source systems containing +900, 000 LOC, finding add meters-squared to meters-squared, but this code incorrectly
inconsistencies in 11% of them, with an 87% true positive rate for a adds meters to meters-squared. The code compiles without com-
class of inconsistencies detected with high confidence. An initial plaint as both variables have the same programming type. Yet the
survey of robot system developers finds that the unit inconsisten- inconsistency in how the units are combined in the code constitutes
cies detected by our tool are ‘problematic’, and we investigate how a fault that will go undetected by the type system, likely to manifest
and when these inconsistencies occur. later as incorrect behavior.
The consequences of such unit inconsistencies in systems inter-
CCS CONCEPTS acting with the physical world exhibit a range of severities, from
mild to occasionally catastrophic [31]. There does not seem to exist,
•Software and its engineering → Software testing and debug- however, an authoritative estimate of how frequently unit inconsis-
ging; tencies occur or with what severity. Still, the related work in type
systems indicates that these kinds of problems have been nagging
KEYWORDS system developers for a long time. As early as 1978, Karr and Love-
physical units; program analysis; static analysis; unit consistency; man [16] advocated for the design of programming languages with
dimensional analysis; type checking; robotic systems support for unit types.
There are four kinds of approaches to detecting unit type incon-
ACM Reference format:
sistencies: full native programming language support, migration to
John-Paul Ore, Carrick Detweiler, Sebastian Elbaum. 2017. Lightweight specialized type libraries, annotation-based approaches, and min-
Detection of Physical Unit Inconsistencies without Program Annotations. ing a program for contradictory variable type usage. Specialized
In Proceedings of 26th International Symposium on Software Testing and language support for units is built into Fortress [1] and more
Analysis, Santa Barbara, CA, USA, July 2017 (ISSTA’17), 11 pages. recently F# [18]. The type library boost::units [29] offers static
DOI: 10.1145/3092703.3092722 checking of unit type consistency but requires code migration for
all variables involved in expressions with physical units. Anno-
tation based methods [8, 13, 15, 32] commonly use dataflow and
constraint solvers to reason about the units of unknown expres-
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
sions. Approaches that mine variable type usage within a program
for profit or commercial advantage and that copies bear this notice and the full citation to infer unit inconsistencies, such as UniFi [12], are annotation-free
on the first page. Copyrights for components of this work owned by others than ACM but require the program to contain unit usages that together are
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or a contradictory. The first three of these require system developers
fee. Request permissions from [email protected]. to incur an annotation or migration burden, and the last requires
ISSTA’17, Santa Barbara, CA, USA examples of contradictory usage to occur within a single program.
© 2017 ACM. 978-1-4503-5076-1/17/07. . . $15.00
DOI: 10.1145/3092703.3092722

341
ISSTA’17, July 2017, Santa Barbara, CA, USA John-Paul Ore, Carrick Detweiler, Sebastian Elbaum

Figure 1: Code snippet from SoftBank’s Romeo robot [30] containing a unit inconsistency detected by our tool, subsequently
acknowledged by the developers and patched. package: ros-aldebaran source: https://git.io/v6Xll, fixed source: https://git.io/v6xkH

In this work we seek to quickly detect unit inconsistencies with- 2 BACKGROUND AND MOTIVATING
out annotation or migration burdens, and without depending on EXAMPLES
the program to contain self-contradictory type usage. We present
This section presents background on the nature of physical units
an approach that requires a one-time effort of building a mapping
and examples of unit inconsistencies. We first discuss how units
from attributes in shared libraries to units (instead of annotating
are governed by the rules of dimensional analysis.
every program that uses the shared library). Fortunately, middle-
ware for cyber-physical systems often abstracts out commonly used
2.1 Background
quantities with physical units into shared libraries because these
quantities are exchanged between components [19]. In addition Physical Unit Representation. Physical phenomena are quanti-
to the mapping, the proposed approach uses dimensional analysis, fied in terms of units, such as meters-per-second or furlongs-
[6] rules governing how physical quantities may be correctly com- -per-fortnight. More formally, units form an abelian group2 , and
bined, compared, and manipulated. As our approach analyzes a we extend the convention for units used by Jiang and Su [15] that
program, the mapping enables the automatic decoration of program models units as types and defines a simplified unit type language:
variables with physical units, and applies rules from dimensional ut ::= meter | kiloдram | second | ampere | kelvin | mole | radian |
analysis to detect inconsistent usage of physical units. To keep our
deдree | quaternion | candela | unity | ut 1 ∗ ut 2 | ut −1 | δ (1)
approach lightweight and practical, we accept design choices that
compromise soundness and completeness. The work is contextu- The set of units we consider are the seven base units of the
alized and inspired in part by our observations and experiences International System of Units (SI) [25]. The operator ‘∗’ means
with robotic systems where manipulating physical units is common, multiplication, unity is identity, ut −1 is a unit’s inverse, and δ
usually supported by libraries, and often challenging. However, this represents the unknown unit. We also include radian, deдree, and
approach generalizes to systems that analyze, interpret, and reason quaternion because they are familiar to developers, even though
about data with physical units, ranging from embedded systems to they are equivalent to unity with dimensionless units meter-per-
physical simulations of driverless cars, given that the system uses -meter [22]. The seven base units can be combined to represent
attributes in shared libraries that specify physical units. The key other physical quantities and these combinations are called derived
contributions of this work are: units. For example, the Newton is the SI unit of force and is a derived
unit. One Newton can be expressed in terms of its equivalent
• A novel approach for detecting unit inconsistencies in ex-
base units, (kilogram ∗ meter) ∗ (second ∗ second)−1 , or
isting programs without extra effort from developers. It in-
equivalently kд m s −2 . The unknown unit δ is useful in expressing
tegrates a one-time mapping of shared libraries’ attributes
and tracking uncertainty in units. The grammar ut generates the
to physical units with a lightweight static approach imple-
set of all possible unit assignments.
menting dimensional analysis.
Consistent Unit Operations with Dimensional Analysis. Ev-
• A lightweight static analysis tool implementing the ap-
ery base unit in the SI system corresponds to a base dimension. For
proach for systems built on the Robot Operating System
example, the base unit meter has a base dimension of lenдth. Other
(ROS).1 Available at http://nimbus.unl.edu/tools.
measurements of length, like furlongs or smoots, have different
• An initial evaluation of our tool on a corpus of 213 sys-
units than meter but the same dimension lenдth. All quantities
tems containing +900, 000 LOC, wherein 24 systems (11%)
with units have a corresponding dimension. The rules governing
contain unit inconsistencies, with a true positive rate of
how quantities with units can be manipulated is called dimensional
87% for a class of unit inconsistencies detected with ‘high
analysis [6, 17]. Based on dimensional analysis, we define rules
confidence’.
for addition, comparison, and assignment that are consistent only
• A validation of the kinds of unit inconsistencies detected
when satisfying the following:
by our tool, where 56% of inconsistencies were deemed
Addition/Subtraction:
problematic and 34% potentially problematic by surveyed
ut 1 {+, −}ut 2 → {consistent} ⇔ (ut 1 = ut 2 ) (2)
developers of such systems.
Comparison:
ut 1 {<, >, ≤, ≥, =, ,}ut 2 → {consistent} ⇔ (ut 1 = ut 2 ) (3)
1 ROS is “maybe the most popular robotic middleware” [19], and has +3000 citations,
+2500 systems, and nine million package downloads/month. We chose ROS deliber- 2 Abelian groups are finite or infinite sets with a binary operation (for units, multipli-
ately for impact. An initial investigation of similar cyber-physical middleware indicates cation) that satisfy associativity, commutativity, closure, and have identity and inverse
straightforward applicability to Orocos [7], OpenRTM [2], MOOS [4], and Yarp [21]. elements.

342
Lightweight Detection of Physical Unit Inconsistencies without Program Annotations ISSTA’17, July 2017, Santa Barbara, CA, USA

units for torque. Adding the square of force to the square of

torque is not consistent by Eq. 2. We call this addition of inconsis-
tent units. In the developer’s defense, this calculation might behave
as intended given input that implicitly normalizes these values.
Figure 2: Inconsistent assignment. However, adding quantities with dissimilar units is generally de-
package:ros-planning source:https://git.io/v6XlV void of physical meaning. Without explanation, this code might be
considered a bewildering hack that works on one particular system,
in one particular circumstance. If we assume this code is inten-
tional, then the unit inconsistency reveals the existence of latent
assumptions about the physical system. These assumptions hinder
code re-use, since system developers must duplicate the system and
Figure 3: Inconsistent comparison. environment or risk unintended behavior.
package:ros-teleop source:https://git.io/v6Xld
These examples illustrate how unit inconsistencies—multiple
units, comparison/addition of different units—can result in pro-
For ut 1,2 ∈ ut. Essentially, dimensional analysis specifies that you grams that are difficult to understand and maintain, incorrect, or
can only add or compare quantities with the same dimension. We hard to reuse. These problems are discussed in more detail in our
extend this notion of consistency to programming languages: study in Section 5 that shows these inconsistencies lurk in at least
Assignment: 11% of the 213 systems we analyzed.
(ut 1 ← ut 2 ) → {consistent} ⇔ (ut 1 = ut 2 ) (4)
This specifies that assignment is only consistent when the units 3 APPROACH
of a variable being assigned a new value are the same as the units This section describes the approach, including how we perform a
being assigned. Operations that violate Eqs. 2-4 are called unit one-time mapping from class attributes in shared libraries to units,
inconsistencies in this paper. We now explore three code examples an algorithm that utilizes this mapping to detect unit inconsisten-
to illustrate different kinds of units inconsistencies. cies, an analysis of the algorithm’s complexity, and a discussion of
the limitations of this approach. First, we describe requirements
2.2 Motivating Examples guiding the design decisions that shape our approach.
The code snippet shown in Figure 2 shows an assignment in line
3.1 Requirements
465 with the variable cmd vel.linear.x being assigned the value
returned by the function drive cmds.getOrigin().getX(). The The goal is to enable fast, lightweight, low-burden, meaningful unit
Figure shows unit decorations to aid understanding. Both the vari- inconsistency detection. More specifically, the requirements for an
able and the value returned by the function have the data type approach that meets this goal include:
float64, but they represent quantities with different units. The • run on systems that manipulate standard physical units.
variable cmd vel.linear is part of a structure called Twist, de- • execute sufficiently fast to be part of a build process.
clared in a shared library geometry msgs, that is specified to have • impose a minimal burden on system developers.
• detect inconsistencies that can lead to real problems.
units meters-per-second, while the function getX() on the right-
• yield a low-enough false-positive rate to justify the value
hand-side (RHS) returns meters. Because the specified units are
of its findings3 .
different than the units being assigned, this code does not satisfy
Eq. 4 and is therefore unit inconsistent. We call this kind of unit in- We pursued these requirements through a series of design and
consistency assignment of multiple units. As is, this code implicitly implementation iteration cycles in which we explored the tradeoffs
converts from one unit to another. At best, this inconsistency will between precision, recall, speed, and scalability. We now turn to
make the code harder to maintain and understand. At worst, this the details of the approach resulting from these iterations.
implicit conversion might lead to unintended system behavior.
A second example is shown in Figure 3 line 65 where system 3.2 One-time Mapping from Class Attributes in
developers compare two variables’ magnitudes. The comparison Shared Program Libraries to Units
is between twist.linear.y and twist.angular.z. The Twist The goal of mapping is to assign physical units to physical attributes
data structure is defined in geometry msgs, a shared library. The in shared libraries. By physical attributes we mean attributes, struc-
variable linear.y has units meters-per-second while the vari- tures, and class function return values found in shared libraries that
able angular.z has units radians-per-second. This comparison represent quantities measured in physical units. Rather than anno-
does not satisfy Eq. 3 and is therefore inconsistent, and we call this tating physical attributes at the point they are defined in shared
comparison of inconsistent units. The system developer might have libraries, this approach instead decouples this ‘mapping’ between
a reason to make this comparison, but such choices in code are physical attributes and units from the shared libraries. By decou-
suspicious and should be conspicuously documented and justified, pling the relationship between shared class attributes and physical
especially for shared code. units from the shared libraries, system developers do not need an-
Figure 4 shows a third example of unit inconsistency on line notated copies of those libraries. Further, this avoids the reliance
1094 in an addition expression. This sums the squares of three on unit-aware type libraries, compilers, or languages—all of which
quantities: force.x, force.y, and torque.z. The problem with
this expression is that the units for force are different than the 3 Both Bessey and Hovemeyer et al. used < 20% as a baseline [5, 14]

343
ISSTA’17, July 2017, Santa Barbara, CA, USA John-Paul Ore, Carrick Detweiler, Sebastian Elbaum

Figure 4: Inconsistent units during addition of force and torque in distance metric. package:eband local planner source: https://git.io/v6X8T

hinder re-use. When compared to individual system developers System (ROS). ROS is a widely-adopted middleware to enable rapid
annotating program variables with physical units at declaration, development in robotic systems used by both academic and profes-
this approach requires a single effort that can be broadly reused to sional developers, including industrial automation at Boeing [27]
enable unit inconsistency detection in every system that uses those and autonomous driving at BMW [26].
shared libraries. This approach has larger benefits at larger scales. The first step to identifying shared libraries with physical at-
Overall, the purpose of mapping is to achieve the same effect as if tributes in ROS was observing that ROS’s component-based ar-
the entire user base of the shared libraries were to agree to apply chitecture defines data structures with physical units in shared
physical unit annotations in the shared libraries. libraries so components can exchange standardized data repre-
More formally, the mapping is a binary relation between two senting sensor measurements and actuator commands. We looked
sets: the set of physical attributes PHYS ATTRIB (where physical at these shared libraries and found physical attributes for navi-
attributes are identified by fully qualified names (FQNs) in the gation, geometric relationships, and sensor values. The shared
shared libraries) and the set of unit types ut : library for navigation was nav msgs, for geometric relationships
Rmapping ⊆ (PHYS ATTRIB × ut ) (5) the shared library was geometry msgs, and for sensor values the
shared library was sensor msgs. We also determined the most
Mapping Process. The mapping process involves four tasks: frequently used libraries across the systems we studied, identifying
• identifying shared libraries containing physical attributes. additional libraries not in the standard code location of the ROS
• determining the physical unit for each physical attribute. physical libraries. This was an iterative process, first finding one
• finding the FQN for each physical attribute.
prospective shared library and then successively completing all
• encoding the mapping from FQN to physical unit in a
the steps in the mapping process for that library before finding
structured form that can be used programmatically.
another shared library. Within these libraries, we found a vari-
The first task, identifying the shared libraries, is a manual process ety of physical attributes such as geometry msgs::Twist.linear.y,
that requires knowledge or analysis of the target domain. Identify- sensor msgs::Imu.angular velocity covariance, and
ing these libraries might be performed incrementally, to find the nav msgs::Odometry.linear.x.
most commonly reused libraries within a corpus and then iden- In the second step, we associate each physical attribute in the
tifying physical attributes within that set. Or it might involve shared libraries with units. In the case of Odometry::linear.x, the
identifying a particular physical attribute and then examining the documentation specifies this as ‘velocity in free space’. Velocity has
shared library that contains that attribute. The second task involves dimensions of length-per-time and the SI system is specified as
determining a physical attribute’s units that might come from: 1) the default units in ROS [10], therefore we assigned to it the units
knowing the physical unit at design time; 2) finding source code meters-per-second.
comments or program documentation; 3) the class or variable name; The third step of finding the FQN names of the physical at-
and, 4) examining how this variable is used in context. The third tributes in shared libraries was straightforward, and involved copy-
task is simply recording the FQN for the shared class attribute. The ing the full name of the shared library, along with the names of
fourth task, encoding the mapping, involves creating a lookup table the structures containing the physical attribute. An example is
from FQN to physical units, and choosing some structured form nav msgs::Odometry.linear.x, where nav msgs is the shared li-
that can be read by an implementation. This lookup table is an brary, and Odometry.linear.x is the structure containing the phys-
implementation of the binary relation Rmapping . ical attribute x.
Cost. The upfront effort to create the external mapping is slightly The fourth step involved creating the encoding of the mapping,
more than applying in-line annotations to physical attributes in with a record or table entry for each pair of FQNs and corresponding
shared libraries, because of the effort to encode the mapping in an units for each physical attribute in the shared library. An example
external data structure that can be used programmatically. This of one entry is the FQN nav msgs::Odometry.linear.x with the
additional effort is justified by the benefits mentioned above. Com- physical units (meters ∗ (seconds)−1 ).
pared to annotating attributes in shared libraries, an external map- We repeated this process for other physical attributes, and col-
ping introduces no reliance on unit-aware type libraries, compilers, lected a mapping of 246 total physical attributes (class attributes
or languages. When compared to annotating programs that use or function return values) from 82 classes across 7 shared libraries.
shared libraries, the single effort to create the mapping is much less These physical attributes mapped to 17 distinct derived units. Fi-
than the repeated effort by every system developer to separately nally, we encoded the FQN of the physical attributes and its corre-
annotate program variable declarations for those shared libraries. sponding physical unit to create the mapping.
Mapping Example. To help illustrate the mapping process, we The effort to build the mapping for ROS was aided by the fact
now provide an example instantiated within the cyber-physical that two of the co-authors are proficient ROS users. Overall, we
domain, specifically, with a mapping built for the Robot Operating

344
Lightweight Detection of Physical Unit Inconsistencies without Program Annotations ISSTA’17, July 2017, Santa Barbara, CA, USA

completed the core mapping for ROS within 3-4 days. An initial Algorithm 1 Lightweight physical unit inconsistency detection
investigation of similar cyber-physical middleware like Orocos [7], over program P
OpenRTM [2], MOOS [4], and Yarp [21] indicates that a mapping
Input: Program P and unit mapping Rmapping.
for these domains would require a similar effort.
Output: Set of unit inconsistencies.
Again note that this mapping is a one-time effort per shared 1: function LightweightDetectUnitInconsistency(P, Rmapping )
library that then enables unit inconsistency detection in all systems 2: UI ←∅ ⊲ Unit Inconsistencies
that build on these shared libraries. 3: St at e ← ∅
We now present an algorithm for unit inconsistency detection 4: sor t ed F unct ions ← Preprocess(P)
utilizing this mapping. 5: for f unct ion ∈ sor t ed F unct ions do
6: for st at ement ∈ f unct ion do
3.3 Algorithm for Lightweight Detection of 7: DecorateWithUnits(st at ement, St at e, Rmapping )
Unit Inconsistencies 8: EvaluateExpressions(st at ement )
9: U I ← U I ∪ DetectExpInconsistency(st at ement )
This section describes the algorithm LightweightDetectUnit-
10: St at e ← St at e ∪ TransferFunction(st at ement )
Inconsistency (Algorithm 1), that uses the mapping described
11: U I ← U I ∪ DetectMultipleUnitInconsistencies(St at e)
in Sec. 3.2. Some functions of Algorithm 1 that require further
12: return U I
explanation are described in the text below.
As fitting our requirements, this approach seeks the simplest 13: function TransferFunction(st at ement )
analysis still capable of detecting meaningful unit inconsistencies. 14: newU nit s ← getRHSUnits(st at ement )
Our analysis is semi-flow-sensitive (a simplified forward dataflow), 15: if newU nit s = ∅ then
path-insensitive, context-insensitive, and intra-procedural. Note 16: return ∅
that although the analysis is intra-procedural, for some function 17: if isAssignment(st at ement ) then
calls the approach can determine the units of the return value 18: return {(getLHSVar(st at ement ), newU nit s)}
because of the order the functions are analyzed. In these cases, the 19: else if isReturn(st at ement ) then
approach applies the units returned by the function at its call point. 20: return {(f unct ionN ame, newU nit s)}
A dataflow analysis is often defined using states, a transfer func- 21: return ∅
tion, a lattice, and a join operation. The states represent knowledge
at entry/exit points of blocks, a transfer function calculates changes
to the state during that block, the lattice represents all possible bottom-up. If the call graph contains a cycle, an edge of the cycle
abstract states arranged in a power-set hierarchy, and the join func- is removed from the call graph until no cycles are found. If the
tion calculates the state at the entry to a block by ‘joining’ the topological sort yields a partial order, the approach breaks ties
states that flow into that block in the control flow graph. In con- arbitrarily and examines only the first ordering for simplicity. The
trast, our analysis has only one single state, State, that enters and output is an ordered list of functions.
exits every statement. State is a set of tuples representing vari- DecorateWithUnits. In line 7, this function traverses a state-
able unit assignments, {(var , {units}), ...} where var ∈ VAR, the ment’s Abstract Syntax Tree (AST) and applies unit decorations
set of program variables and {units} ⊂ ut, the unit type language to variables, when possible. We assume the existence of a relation
of Equation 1. A power-set lattice representation of the abstract between the set of program variables VAR and the set of physical
state is a poor fit because physical units form an abelian group, and attributes PHYS ATTRIB:
therefore we instead use a unit type language (Equation 1). State- RtypeOf ⊆ (VAR × PHYS ATTRIB) (6)
ments are analyzed sequentially without regard to control flow. At
a program point, the units of a variable in State are the union of: This relation is commonly provided by a compiler front end, and
1) any units specified by the mapping because the variable is of in our tool this is provided by CPPCheck [20]. Using the composi-
a type that belongs to a shared library and represents a physical tion of this relation with the mapping from Eq. 5 we have:
class attribute; 2) previous unit assignments. The transfer function RunitsOf ≡ (Rmapping ◦ RtypeOf ) ⊆ (VAR × ut ) (7)
from before a statement (the ‘in’ state) to after the statement (the
‘out’ state) is the union of: 1) the previous state; 2) the evaluation Where RunitsOf is the composition of the relations in Eq. 5 and
of the units resulting from the RHS expression of assignment and Eq. 6 linking program variables to units.
return statements. Since there is only one state, the join operation Program variables can be decorated with units from either a
is unnecessary. prior assignment statement listed in State or when the variable’s
Overview. Algorithm 1 takes as input a program P and relation type is found in RunitsOf . The function DecorateWithUnits first
Rmapping from Sec. 3.2. During the loop in lines 5-10, the algorithm checks for units in State and if no units are found, checks RunitsOf .
processes each program statement once. It detects unit inconsisten- If neither structure yields units, then the variable is decorated with
cies in two ways: 1) within a statement for addition/comparison δ , the unknown unit. An example of unit decoration using RunitsOf
inconsistencies; and 2) by analyzing variables in the final version is shown in the dotted boxed of Figure 5. These variables can be
of State for multiple unit assignments to one variable. decorated because their variable type belongs to the shared library
Preprocess. In line 4, the algorithm preprocesses program P by geometry msgs that declares a class WrenchStamped with physi-
constructing a context-insensitive call graph (without alias analysis) cal class attributes included in Rmapping . The composed relation
and performing a reverse topological sort, to analyze functions RunitsOf connects variable force.x to the units kg m s−2 .

345
ISSTA’17, July 2017, Santa Barbara, CA, USA John-Paul Ore, Carrick Detweiler, Sebastian Elbaum

EvaluateExpressions. This function visits a statement’s AST and

attempts to resolve the units of expressions using the unit resolution
rules shown in Table 1. It works from the leaves up, matching
expressions to unit resolution rules and decorating the interior
nodes of the AST with units. It continues to apply unit resolution
rules in a loop until no changes are made. These rules apply when
variables or expressions with units are combined and manipulated.
Note an important difference between the rule for multiplication
and the one for addition: during multiplication, if one operand has
known units but the other is δ , the unknown unit, we pessimistically
assume the result is unknown; during addition, if one operand is
known and the other is δ , we optimistically assume the result is the
known unit. The reason multiplication is pessimistic is that there is
only one way for multiplication to yield the same units, and many
ways for the result to be different. Multiplication only yields the
same units when multiplied by a scalar with unity as the unit, and
assuming that every unknown variable involved in multiplication
is a scalar leads to many false positives. The reason addition is
optimistic is that the resulting sum must have the same units as Figure 5: Example of a statement’s AST from the code
the known operand or be inconsistent, and we cannot conclude the in Figure 4 with the shared class fully qualified name
sum is inconsistent because δ is unknown. WrenchStamped::wrench omitted for simplicity. Figure shows
An example of how the function EvaluateExpressions works unit decoration of variables by the relation RunitsOf (dotted
is shown in Figure 5. The units in the dotted boxes were applied boxes), and evaluation of expressions’ units toward the root
in DecorateWithUnits, and the three multiplications near the by unit resolution rules in Table 1 (solid boxes).
bottom of the AST match the multiplication rule in Table 1. By
the multiplication rule, we add the exponents of the units of the confidence if at least two units without δ are assigned to a program
operands, yielding the unit decorations on the three ‘∗’ symbols. variable.
Next, the rules match the ‘+’ symbol up the tree, and apply the 3.4 Termination and Complexity
addition resolution rule, yielding the union of the operands’ units.
Preprocessing requires a linear pass over the program to construct
This function continues to apply unit resolution rules until no more
the context-insensitive call graph, and topologically sorting the call
changes can be made. This function only adds additional unit deco-
graph is O (|V | + |E|) with a worst case O (|E 2 |) when detecting and
rations and does not detect unit inconsistency in the expressions,
removing cycles. The loop in lines 5-10 analyzes each statement
which happens in the next function.
once and is linear in the size of the input program AST. Decorating
DetectExpInconsistency. This function applies the unit consis-
a statement’s variables with units, evaluating expressions, detecting
tency tests from Eq. 2 (addition) and Eq. 3 (comparison) to expres-
expression unit inconsistencies, and the transfer function are linear
sions within a single statement. This function scans a statement’s
in the size of a statement’s AST. After the loop, detecting multiple
AST looking for inconsistencies like those in Figures 1,3,4, and 5.
unit inconsistencies requires a linear scan of State, and State is as
The example in Figure 5 shows an unit inconsistency detected while
large as the number of program variables. Putting it all together,
evaluating an addition expression.
the worst case for the algorithm is quadratic in time and space.
As shown in Table 1, the unit inconsistency detection has a
Termination is guaranteed because the while loop within Evalu-
‘confidence’ that can be either high or low if the expression contains
ateExpressions applies unit resolution rules at most h times where
δ , the unknown unit. Figure 5 shows the detection of inconsistent
h is the height of a statement’s AST.
addition of kg2 m2 s−4 to kg2 m4 s−4 with high confidence.
TransferFunction. The transfer function in this analysis can only
add new information to the state, and only for assignment or return 3.5 Limitations
statements. For assignment statements, the function getRHSunits While designed to be as fast and lightweight as possible while
at line 14 simply returns the units decorating the ‘=’, and otherwise detecting useful inconsistencies, this design has limitations in ap-
returns the empty set. In line 10, State is updated as the union of plicability, soundness, and completeness. The mapping process
State and the output of TransferFunction. can only apply units to physical attributes that are identified and
DetectMultipleUnitInconsistencies. Scanning State at line 11 correctly assigned units beforehand, and the number of program
of Algorithm 1 can reveal assignment of multiple units inconsisten- variables that can be labeled with unit information might be lim-
cies. This kind of inconsistency comes from two sources 1) variables ited, bounding applicability. This approach is unsound because it
assigned units contrary to their specification in the Rmapping ; and 2) includes infeasible sets of variable-unit assignments in State as it
variables assigned different units at different points in the program. ignores control flow. This approach is incomplete because State
When State contains multiple units for a variable this function misses some variable-unit assignments in loops and because it is
reports inconsistencies with either low or high confidence, based not path-sensitive. Also, as described, the approach is semi-flow-
on the presence of δ in a variable’s units. This function reports high sensitive and intra-procedural, only supports the mapped libraries,

346
Lightweight Detection of Physical Unit Inconsistencies without Program Annotations ISSTA’17, July 2017, Santa Barbara, CA, USA

Table 1: Unit resolution rules used in Algorithm 1 function EvaluateExpressions on line 8, and the inconsistency rules are
used to detect addition/comparison inconsistencies in function DetectExpInconsistency on line 9.

EXPRESSION CONDITION RESULT INCONSISTENT CONFIDENCE

ut 1 {∗, ÷}ut 2 ut 3 =add/subtract exponents of ut 1 and ut 2
ut 1 {∗, ÷}δ ut 1 ∗ δ (pessimistic)
ut 1 {+, −}ut 2 ut 1 = ut 2 ut 1
ut 1 {+, −}ut 2 ut 1 , ut 2 ut 1 ∪ ut 2 Yes, by Eq. 2 high
ut 1 {+, −}δ ∗ ut 2 ut 1 , ut 2 ut 1 ∪ δ ∗ ut 2 Yes, by Eq. 2 low
ut 1 {+, −}δ ut 1 (optimistic)
pow (ut 1, n) n ∈R multiple each ut 1 exponent by n
sqr t (ut 1 ) divide each ut 1 exponent by 2
sqr t (δ ) δ
ut 1 {<, >, ≥, ≤, ,}ut 2 ut 1 = ut 2 none
ut 1 {<, >, ≥, ≤, ,}ut 2 ut 1 , ut 2 none Yes, by Eq. 3 high
ut 1 {<, >, ≥, ≤, ,}δ ∗ ut 2 ut 1 , ut 2 none Yes, by Eq. 3 low
ut 1 {<, >, ≥, ≤, ,}δ none
{f loor, ceil, (f )abs }(ut 1 ) ut 1
{min, max }(ut 1, ut 2 ) ut 1 ∪ ut 2
{min, max }(ut 1, δ ) ut 1
(Bool ean) ? ut 1 : ut 2 ut 1 ∪ ut 2 (ternary operator)

and does not handle complex dynamic concerns like dynamic dis- Table 2: ROS Open-Source Repositories
patch, pointers, or Standard Template Library data structures. Fur- CORPUS SOURCE # of REPOSITORIES
ther, the approach does not attempt symbolic analysis that could Total ROS.org “Indigo” Projects Links 2416
reason about statements like (U N IT n ) (−n) . We expect to keep ex- Live Git Repos 649 of 2416
amining these design decisions, interleaving the incorporation of Git REPOS with C++ FILES 436 of 649
stronger analyses with an assessment of their impact in the overall REPOS with C++ FILES AND ROS UNITS 213 of 436
cost-effectiveness and performance of the approach.

We provide our prototype as a software artifact that can be

downloaded from http://nimbus.unl.edu/tools.
4 PROTOTYPE IMPLEMENTATION 5 EVALUATION
Our implementation detects unit inconsistencies in C++ code writ-
The goal of the evaluation is to assess the ability of the tool to detect
ten for ROS. The architecture follows the approach, is implemented
unit inconsistencies that may cause problems for real systems. To
in 3300 lines of python, and can be run from the command-line.
achieve that goal we employ a methodology that captures two
Our tool utilizes CPPCheck as a C++ preprocessor and parser
perspectives: 1) we examine the results of running our tool on a
[20], invoked with default parameters and includes directories:
corpus of publicly available robotic systems and then hand-label
cppcheck --dump -I ../include myfile.cpp. The dump op-
the results as True and False Positives; and 2) we conduct a small
tion generates an XML file containing: 1) every program statement
survey of robotic system researchers to gauge whether they deem
as a separate abstract syntax tree; 2) a token list; and 3) a symbol
the unit inconsistencies detected by our tool to be ‘problematic.’
database including functions, variables, classes, and all scopes. CP-
Finally we discuss threats to validity.
PCheck can explore multiple compilation configurations (different
#define values), but in the reported results we only consider the 5.1 Analysis of Robotic Software Corpus
default system configuration. To evaluate the effectiveness of our tool we exercised it on a wide
We use a visitor pattern in each statement’s AST to apply units range of publicly available robotic software, specifically systems
and evaluate expressions with unit resolution rules. During im- designed to work with the robotic middleware ROS.
plementation, we realized that radian and quaternion require The maintainers of ROS publish a list of public software reposito-
special handling: during multiplication, radian and quaternion ries using ROS in academic and industrial robots. The list, published
act as unity since their units are meters-per-meter; during ad- at http://www.ros.org/browse/list.php, includes projects at various
dition, they are ‘coherent units of measure’ [22], meaning that stages of development, and for a wide variety of purposes: mobile
they cannot be added to a dissimilar unit, even though they are robot navigation, collision detection libraries for robotic arms, dri-
dimensionless. vers for depth cameras, control software for flying robots—a diverse
An example inconsistency message for the code in Figure 4 reads: set.
Addition of inconsistent units on line 1094 with high Table 2 shows statistics about this corpus. At the time we gath-
confidence. Attempting to add kg2 m2 s−4 to kg2 m4 s−4 . We ered this corpus, there were 2416 projects linked from the ‘Indigo’
consolidate error messages to report only the first unit inconsis- version of ROS. Of these 2416 links, 649 were linked to live Git repos-
tency for a particular variable. itories. ROS supports C++, Python, and a few projects with LISP

347
ISSTA’17, July 2017, Santa Barbara, CA, USA John-Paul Ore, Carrick Detweiler, Sebastian Elbaum

and Java, but the majority are in C++, so we focused on those. Of

the 649 live Git repositories, 436 contained C++ files. Of these 436,
we found 213 repositories with systems containing shared libraries
with physical attributes. For this initial work, we proceed with
all ROS geometry, navigation, transform, sensor, and time li- Figure 6: False positive during cross-product.
braries. The list of the 213 systems with GitHub download links package:pr2-navigation source:https://git.io/v6xS7
and version numbers is available at http://nimbus.unl.edu/tools.
Scale and Speed. We ran our tool on 213 systems containing ROS
physical units, analyzing 934,124 non-blank non-commented lines
of C/C++ as reported by CLOC [9]. Analyzing all systems took
108 minutes (61 minutes to parse the files with CPPCheck and 47
minutes to perform our analysis), with an average analysis time
of 31 seconds per system, when running on a MacBook Pro (‘early
2015’) with a 2.9 GHz Intel Quad Core i5 processor, and 16GB of Figure 7: False positive with constant.
memory. We only utilized a single core during evaluation, although package:crazyflie ros source:https://git.io/v6x7T
this could be easily parallelized since the files and analysis are
independent.
Effectiveness. We individually examined each inconsistency re-
The overall TP rate for ‘low confidence’ unit inconsistencies is
ported by the tool, reviewing the source code surrounding each
37.45%, with about 50% more low confidence TP (64) than high
reported line, and labeled each one as either ‘True Positive’ (TP)
confidence TP (40). The low TP rate is caused mainly by the large
or ‘False Positive’ (FP). Note that labeling inconsistencies as TP
number of variables and constants with implicit units not found
or FP lets us calculate precision, but the number of ‘False Nega-
in the mapping. Figure 7 shows an example of a constant with
tives’ (FN) is unknown and therefore we cannot calculate recall.
implicit units. The constant 10000 in line 104 is part of a expres-
This labeling process required several rounds of iterations as the
sion with units δ ∗ seconds that is then assigned to m thrust. The
analysis of some inconsistencies led us to question and re-analyze
units assigned at line 104 flow to the usage of m thrust in an as-
previous labels. The process converged towards the recognition of
signment statement on line 106. Since the units for msg.linear.z
TP caused by addition/comparison of inconsistent units and assign-
are meters-per-second and it is assigned units of δ ∗ seconds,
ment of multiple unit inconsistencies. We observe that multiple
our tool reports a low confidence unit inconsistency. In this case,
unit inconsistencies have distinct causes: 1) variable re-use (like
the source of the FP is the constant 10000. Constants are basically
temp variables); 2) disagreement between the units defined in the
devoid of units and we plan to assess them in more detail in the
mapping (from the documentation) and the actual units used; 3)
future.
the previous two combined—variables with units from the map-
ping used as temporary variables. We currently do not distinguish 5.2 Survey
between these causes but believe they could be separated auto- We conducted a survey to obtain an initial assessment of whether
matically by observing whether the units come directly from the these kinds of unit inconsistencies are problematic to robotic soft-
mapping and whether they are assigned only one kind of unit in ware developers. Specifically, some unit inconsistencies, like vari-
the program. able reuse and using a physical attribute to store a quantity against
Our results are summarized in Table 3. The overall TP rate, com- its specification, might be poor programming style, but also might
puted as T P% = 100 ∗ T P/(T P + F P ), for ‘high confidence’ unit not warrant a high-priority bug report. Therefore we wanted to
inconsistencies is 87.0%. This includes the three types, variables assess the severity of these kinds of inconsistencies. Our survey
assigned multiple units, addition of inconsistent units, and compar- instrument consists of eight questions, each showing a code exam-
ison of inconsistent units. As we noted earlier, for one of the cases ple similar to those in Figures 2-4, drawn from unit inconsistencies
where we contacted the authors of the code for clarification, shown detected by our tool. For each code example, we asked “Is the unit
in Figure 1, the inconsistency was acknowledged as a fault and inconsistency on line [X] problematic (e.g., cause failures, increase
developers patched the code immediately after our inquiry. Within cost of maintenance, make code more difficult to understand, or in-
the ‘high confidence’ TP, we found a TP rate of 84.6% for variables troduce interoperability problems)?”, with a choice of responses:
assigned multiple units. The False Positives with ‘high confidence’ ‘yes’, ‘maybe’, and ‘no’. After each question, the respondents could
all detect redundant implementations of vector cross-products and add an open-ended explanation. The order of the questions was
outer-products that are already provided by the ROS API, where randomized for each respondent.
meters-squared intentionally equals meters. Figure 6 contains The target population for our survey included either heads of
one such case in line 90, which is frequently used and deemed cor- academic robotics research labs or their senior research associates.
rect by system developers. In general, our tool handles vectors like These labs publish regularly in top robotic conferences and use
any other quantity and detects inconsistent addition, comparison, ROS extensively. We sent our survey to ten labs and received ten
and assignment. We believe we could modify the tool to detect and responses from six of the labs. We recognize the sample population
ignore this special case, but we would have to be careful not to size is small and may not generalize, but at this stage of the work
blind our tool to unintentional assignments of meters-squared to we wanted quick, initial feedback before attempting a larger, more
meters, therefore for now we accept these kinds of FP. nuanced study that might be justified in the future.

348
Lightweight Detection of Physical Unit Inconsistencies without Program Annotations ISSTA’17, July 2017, Santa Barbara, CA, USA

Table 3: Classification of unit inconsistencies found by our tool. Note: this table presents precision and not recall, because
recall requires false negatives (FN) that are unknown in our corpus.
High Confidence Low Confidence
INCONSISTENCY TYPE TP FP TP% TP FP TP%
TOTAL 40 6 87.0% 64 107 37.4%
ASSIGNMENT OF MULTIPLE UNITS 33 6 84.6% 55 83 39.9%
ADDITION OF INCONSISTENT UNITS 5 0 100% 9 20 31.0%
COMPARISON INCONSISTENT UNITS 2 0 100% 0 4 0%

Table 4: Summary of survey responses to whether unit in- be incorrect. We believe that identifying both of these kinds of unit
consistencies found by our tool are ‘problematic.’ inconsistencies has value to system developers.
Question # YES MAYBE NO At the end of the survey we let respondents write an open-ended
‘overall’ feedback to these kinds of unit inconsistencies. The most
1 6 2 2
critical respondent stated ‘Overall a lot of unit inconsistencies will
2 8 1 1
happen for control or optimization reasons and sometimes … cannot
3 3 5 2
be avoided,’ while the most laudatory stated ’This tool is amazing!
4 9 0 1
At the very worst, it find out questionable programming practice that
5 6 3 1
needs additional documentation. Most of the time, it finds bugs or
6 (Figure 2) 2 8 0
hacky heuristics.’
7 (Figure 4) 7 3 0
Overall, is spite of the limited size our population, and that this
8 4 5 1
population does not represent industrial system developers, most
TOTALS 45 27 8 responses affirm our assertion that the kinds of unit inconsistencies
% 56% 34% 10% detected by this approach are problematic.

Our results are shown in detail in Table 4. Overall, 56% of re- 5.3 Threats to Validity
sponses indicate that ‘yes’, these unit inconsistencies are problem- Self-labeling. One significant threat to the validity of our findings
atic. The ‘yes’ responses included explanations from ‘The addition is our reliance on self-labeled TP and FP. For high confidence TP,
of different units means nothing in real world’ to ‘just bad program- it was relatively easy to see when and how the physical units are
ming.’ This fits with our assessment that many unit inconsistencies inconsistent, and for high confidence FP there were only a few
require attention or at least special explicit justification. mathematical corner cases like the cross and outer vector product
The ‘maybe’ responses (34%) included explanations, such as ‘If that required careful consideration but we ended up confident about
the angular radius is unity, then OK, otherwise could lead to error’, those classifications. Low confidence FPs were more straightfor-
identifying a special case when the code could be correct, or ‘I don’t ward to identify and often were attributable to a single variable
know when you’d like to compute this.’ Several ‘maybe’ responses with unknown units, such as the code example in Figure 7. Low con-
indicated the possibility of a special circumstance when the unit fidence TPs were more time consuming to identify because these
inconsistency might not be problematic. In these cases our tool unit inconsistencies involved partial information, and we assumed
indicates a possible constraint on the circumstances under which the low-confidence inconsistencies were FP until proven to be a TP.
the code behaves correctly, and for the unit inconsistencies detected We mitigated the previous threat to internal validity by review-
by our tool, these special circumstances were never mentioned in ing the results multiple times and discussing examples with other
the code comments, to our knowledge. system developers, and ultimately by combining the analysis of
Of ‘no’ responses (not problematic), half came from one respon- the code corpus with the preliminary survey. The respondents clas-
dent (4 of 8), who explained: ‘The problem I see is that the proposed sified most examples as either ‘yes’ or ‘maybe’ problematic, and
method will get hung up in hacks that actually are workable solutions provided many compelling justifications for their responses. To a
and it might be impossible for the average coder to fix these issues.’ We large degree, these responses matched our expectations, but clearly
contend that detecting ‘workable’ ‘hacks’ is still valuable, especially further study is needed to better understand the larger body of
for junior developers lacking the hard-earned experience necessary inconsistencies found.
to recognize them in the first place. False Negatives. We cannot measure recall because the number of
Questions 6 and 7 from Table 4 of the survey are also presented false negatives in the software corpus is unknown, but we intend to
in this work as Figure 2 and 4. Notice for question 6 that most address this threat by seeding faults to better evaluate our approach.
respondents said ‘maybe’, and this code example shows unit in- To our knowledge, there exists no dataset of labeled physical unit
consistency by assignment to a data type with a different physical inconsistencies, although this work identifies an initial set of TP.
unit specification, which is perhaps more an issue of code main- Cost-effectiveness. We recognize that this approach’s cost effec-
tenance and reuse since it only uses a technically incorrect data tiveness will vary across systems depending at least partly on the
container. However in question 7 (Figure 4) most respondents said extent of physical unit usage, their degree of manipulation, and
‘yes problematic’, and this code example contains addition of incon- their standardization. Still, we argue that this threat to external
sistent units, which is perhaps more concerning because it might validity is mitigated by the fact that most physical systems regularly

349
ISSTA’17, July 2017, Santa Barbara, CA, USA John-Paul Ore, Carrick Detweiler, Sebastian Elbaum

incorporate sensors and actuators that utilize standardized middle- 7 CONCLUSIONS

ware libraries shared across platforms, all using some convention In this paper we presented a novel approach to detect physical unit
for data structures containing physical units. inconsistencies in programs including cyber-physical, scientific
Generality. Similarly, we recognize that we built the mapping for computing, and embedded systems. The approach detects useful
a particular robotic domain, namely ROS, but other mappings to a inconsistencies without modifying the program or shared libraries
variety of middleware systems would help validate the generality with annotations.
of the approach. We are currently exploring other middleware, and The approach is unique in that the mapping from physical at-
will likely next target Orocos [7], OpenRTM [2], MOOS [4], and tributes to units enables unit inconsistency detection without anno-
Yarp [21]. Still, ROS is by far the most commonly adopted robotic tation, and also unique in that it favors speed over precision while
middleware, and has a growing user base. still detecting meaningful unit inconsistencies. We implemented
Comparison to Other Tools. A further threat to external validity the approach in a prototype tool and evaluated it on a corpus of 213
is that we do not directly compare our tool to other unit inconsis- systems, detecting inconsistencies in more than 11% of the systems.
tency detection tools. We do discuss these and other approaches The tool has an 87% true positive rate for a class of inconsistencies
in related work. A practical empirical challenge is that other tools it can detect with high confidence, and a preliminary survey of
target different languages [12], require annotation [15], or code developers analyzing the tool’s findings provides encouragement
migration [29]. One way to address this threat would be to compile for further development and study.
a benchmark of unit inconsistencies, but that is beyond the scope In the future, we will pursue several research avenues. First, with
of this effort. the given approach and toolset in place, we are well positioned
to detect many more types of unit inconsistency. One kind of
6 RELATED WORK inconsistencies that we have started to examine consists of those
In this section we continue a discussion of related work started in where the resulting units are not defined in SI. For example, in one
Section 1, where we identified several kinds of approaches for unit of the systems we analyzed we found resulting units like (meters,
inconsistency detection. 1.5) which does not have a real physical meaning. Although having
Since the late 1970s, researchers have proposed programming such units seems odd, we need to investigate whether they warrant
language extensions and tool support to enable unit consistency enough attention to take action. Second, as is common with this
checks, such as work by Gehani [11], who proposed extending class of static analysis approaches, we will attempt to optimize the
Pascal, Hilfinger’s Ada package [13], Wand and O’Keefe’s simply- soundness, precision, and performance tradeoffs. In particular, we
typed lambda calculus [34], Novak’s work with unit conversion [23], will explore improving path sensitivity, interprocedural analysis,
Delft’s extension for Java [33], Roşu’s and Feng dynamic approach and extending the scope of analysis since we have seen cases where
in C [28], Umrigar’s compiler [32], Antoniu’s spreadsheet checker they could reduce the number of false positives and move some of
XeLda [3], Jiang and Su’s unit annotations in Osprey [15], and the findings from low to high confidence. This may require us to
Schabel and Watanabe’s boost::units for C++ [29]. More recently, integrate the approach with richer analysis frameworks. Third, we
unit consistency as envisioned by Kennedy [17] has been built into intend to analyze the significance of these kinds of inconsistencies:
F#. This implementation seems to fully realize unit consistency, but their impact on systems at runtime, how much developer time is
does not appear to have been widely adopted4 . Like this work, we consumed with alternate methods such as manual unit annotations
detect inconsistent units during addition and assignment, but unlike including constants, and the root causes of these inconsistencies.
this work our approach works without requiring extra annotations Fourth, we will extend the tool to other common cyber-physical
which often lower the barriers for tool adoption. Like these efforts libraries by introducing additional mappings, which will let us
we are concerned with unit consistency, but unlike these systems evaluate a larger number of systems. From an empirical perspective,
we are not proposing language extensions but rather seeking to we will also perform a more extensive survey of the tool findings,
leverage one mapping effort to enable unit inconsistency detection and submit more of the findings to developers’ for evaluation. Last,
without requiring developers to modify their programs. we would like to move beyond the detection of inconsistencies, to
One of the more similar efforts is the tool UniFi [12] that infers providing recommendations on how to fix the unit inconsistencies.
dimensions automatically by mining a program for contradictory
variable type usage, much in the same manner as Lackwit [24], but
applied to unit inconsistency detection. Like those tools, we propa-
gate an abstract type through assignment and detect inconsistent ACKNOWLEDGEMENTS
usage. Unlike their work we apply abstract types from outside the We deeply appreciate the time and input from members of The
program and can detect inconsistencies without requiring the pro- Robotics Institute at Carnegie Mellon University, The Correll Lab
gram to contain contradictory variable type usage. For example, our at the University of Colorado, The Dunbabin Lab at Queensland
tool can detect the addition of inconsistent units in Figure 4 even University of Technology, the NIMBUS lab at the University of
if these variables were used only once in this program, whereas Nebraska-Lincoln, the Autonomous Space Robotics Laboratory
UniFi would not detect this inconsistency. at the University of Toronto, and the Robotics Algorithms & Au-
tonomous Systems Lab at Virginia Tech. the RAAS Lab at Virginia
4 Assessing levels and rates of tool adoption is difficult in a large and diverse com- Tech. This work was supported in part by NSF awards #1638099
munity. However, we note that not one system among the 213 we explored uses any
programming languages with unit support like F#, and only 3 use the boost:units and #1526652, USDA-NIFA #2013-67021-20947, and USDA-NIFA
library extension requiring annotation. #2017-67021-25924

350
Lightweight Detection of Physical Unit Inconsistencies without Program Annotations ISSTA’17, July 2017, Santa Barbara, CA, USA

REFERENCES [17] Andrew Kennedy. 1996. Programming languages and dimensions. Number 391.
[1] Eric Allen, David Chase, Joe Hallett, Victor Luchangco, Jan-Willem Maessen, PhD Thesis, University of Cambridge.
Sukyoung Ryu, Guy L Steele Jr, Sam Tobin-Hochstadt, Joao Dias, Carl Eastlund, [18] Andrew Kennedy. 2010. Types for units-of-measure: Theory and practice. In
and others. 2005. The Fortress language specification. Sun Microsystems 139 Central European Functional Programming School. Springer, 268–305.
(2005), 140. [19] Gergely Magyar, Peter Sinčák, and Zoltán Krizsán. 2015. Comparison study of
[2] Noriaki Ando, Takashi Suehiro, and Tetsuo Kotoku. 2008. A software platform robotic middleware for robotic applications. In Emergent Trends in Robotics and
for component based RT-system development: OpenRTM-Aist. In International Intelligent Systems. Springer, 121–128.
Conference on Simulation, Modeling, and Programming for Autonomous Robots. [20] Daniel Marjamäki. 2013 (accessed 1 February 2017). Cppcheck: a tool for static
Springer, 87–98. C/C++ code analysis. http://cppcheck.sourceforge.net/
[3] Tudor Antoniu, Paul A Steckler, Shriram Krishnamurthi, Erich Neuwirth, and [21] Giorgio Metta, Paul Fitzpatrick, and Lorenzo Natale. 2006. YARP: Yet another
Matthias Felleisen. 2004. Validating the unit correctness of spreadsheet programs. robot platform. International Journal of Advanced Robotic Systems 3, 1 (2006), 8.
In Proceedings of the 26th International Conference on Software Engineering. IEEE [22] IM Mills, Barry N Taylor, and AJ Thor. 2001. Definitions of the units radian,
Computer Society, 439–448. neper, bel and decibel. Metrologia 38, 4 (2001), 353.
[4] Michael R Benjamin, Henrik Schmidt, Paul M Newman, and John J Leonard. 2010. [23] Gordon S. Novak. 1995. Conversion of units of measurement. IEEE Transactions
Nested autonomy for unmanned marine vehicles with MOOS-IvP. Journal of on Software Engineering 21, 8 (1995), 651–661.
Field Robotics 27, 6 (2010), 834–875. [24] Robert O’Callahan and Daniel Jackson. 1997. Lackwit: A program understanding
[5] Al Bessey, Ken Block, Ben Chelf, Andy Chou, Bryan Fulton, Seth Hallem, Charles tool based on type inference. In In Proceedings of the 19th International Conference
Henri-Gros, Asya Kamsky, Scott McPeak, and Dawson Engler. 2010. A few billion on Software Engineering. Citeseer.
lines of code later: using static analysis to find bugs in the real world. Commun. [25] International Bureau of Weights, Measures, Barry N Taylor, and Ambler Thomp-
ACM 53, 2 (2010), 66–75. son. 2001. The international system of units (SI). (2001).
[6] Percy Williams Bridgman. 1922. Dimensional Analysis. Yale University Press. [26] Open Source Robotic Foundation. 2016. Automated Driving
[7] Herman Bruyninckx. 2001. Open robot control software: the OROCOS project. with ROS at BMW. (2016). http://www.osrfoundation.org/
In Robotics and Automation, 2001. Proceedings 2001 ICRA. IEEE International michael-aeberhard-bmw-automated-driving-with-ros-at-bmw
Conference on, Vol. 3. IEEE, 2523–2528. [27] ROS Industrial Consortium. 2016. Current Members - ROS Industrial. (2016).
[8] Satish Chandra and Thomas Reps. 1999. Physical type checking for C. In ACM http://rosindustrial.org/ric/current-members
SIGSOFT Software Engineering Notes, Vol. 24. ACM, 66–75. [28] Grigore Rosu and Feng Chen. 2003. Certifying measurement unit safety policy.
[9] Al Danial. 2016. Count Lines Of Code. (2016). https://github.com/AlDanial/cloc In Automated Software Engineering, 2003. Proceedings. 18th IEEE International
[10] Open Source Robotics Foundation. 2010 (accessed 27 July 2016). ROS Enhancement Conference on. IEEE, 304–309.
Proposal 103. http://www.ros.org/reps/rep-0103.html [29] Matthias Christian Schabel and Steven Watanabe. 2008. Boost. Units 1, 0 (2008),
[11] Narain Gehani. 1977. Units of measure as a data attribute. Computer Languages 2003–2010.
2, 3 (1977), 93–111. [30] SoftBank. 2016. Romeo, the research robot from Aldebaran. (2016). http:
[12] Sudheendra Hangal and Monica S Lam. 2009. Automatic dimension inference and //projetromeo.com
checking for object-oriented programs. In Proceedings of the 31st International [31] Arthur G Stephenson, Daniel R Mulville, Frank H Bauer, Greg A Dukeman, Peter
Conference on Software Engineering. IEEE Computer Society, 155–165. Norvig, LS LaPiana, PJ Rutledge, D Folta, and R Sackheim. 1999. Mars climate
[13] Paul N Hilfinger. 1988. An Ada package for dimensional analysis. ACM Transac- orbiter mishap investigation board phase I report, 44 pp. NASA, Washington, DC
tions on Programming Languages and Systems (TOPLAS) 10, 2 (1988), 189–203. (1999).
[14] David Hovemeyer, Jaime Spacco, and William Pugh. 2005. Evaluating and tuning [32] Zerksis D Umrigar. 1994. Fully static dimensional analysis with C++. ACM
a static analysis to find null pointer bugs. In ACM SIGSOFT Software Engineering SIGPLAN Notices 29, 9 (1994), 135–139.
Notes, Vol. 31. ACM, 13–19. [33] André Van Delft. 1999. A Java extension with support for dimensions. Software
[15] Lingxiao Jiang and Zhendong Su. 2006. Osprey: a practical type system for Prac. Experience 29, 7 (1999), 605–616.
validating dimensional unit correctness of C programs. In Proceedings of the 28th [34] Mitchell Wand and Patrick O’Keefe. 1991. Automatic Dimensional Inference.. In
international conference on Software engineering. ACM, 262–271. Computational Logic-Essays in Honor of Alan Robinson. 479–483.
[16] Michael Karr and David B Loveman III. 1978. Incorporation of units into pro-
gramming languages. Commun. ACM 21, 5 (1978), 385–391.

351