Preventing Errors Before They Happen

The Checker Framework

http://CheckerFramework.org/
Twitter: @CheckerFrmwrk
Live demo: http://CheckerFramework.org/live/

Werner Dietl, University of Waterloo

Michael Ernst, University of Washington
Motivation

java.lang.NullPointerException
Cost of software failures
$312 billion per year global cost of software bugs (2013)
$300 billion dealing with the Y2K problem

$440 million loss by Knight Capital Group Inc. in 30

minutes in August 2012
$650 million loss by NASA Mars missions in 1999; unit
conversion bug
$500 million Ariane 5 maiden flight in 1996; 64-bit to
16-bit conversion bug
Software bugs can cost lives

2011: Software caused 25% of all medical device recalls

Outline
● Verification approach: Pluggable type-checking
● Tool: Checker Framework
● How to use it
● Creating a custom type system
Java's type system is too weak
Type checking prevents many errors
int i = "hello";

Type checking doesn't prevent enough errors

System.console().readLine();
Collections.emptyList().add("one");
Java's type system is too weak
Type checking prevents many errors
int i = "hello";

Type checking doesn't prevent enough errors

NullPointerException
System.console().readLine();
Collections.emptyList().add("one");
Java's type system is too weak
Type checking prevents many errors
int i = "hello";

Type checking doesn't prevent enough errors

System.console().readLine();
UnsupportedOperationException
Collections.emptyList().add("one");
Some errors are silent
Date date = new Date();
myMap.put(date, "now");
date.setSeconds(0); // round to minute
myMap.get(date);
Some errors are silent
Date date = new Date();
myMap.put(date, "now");
date.setSeconds(0); // round to minute
myMap.get(date);

Corrupted map
Some errors are silent
dbStatement.executeQuery(userInput);
Some errors are silent
dbStatement.executeQuery(userInput);

SQL injection attack

Initialization, data formatting, equality tests, …

SQL injection attack
Goal: don’t execute user input as a SQL command
private String wrapQuery(String s) {
return "SELECT * FROM User WHERE userId=’" + s + "’";
}

If a user inputs his name as: ’ or ’x’=’x

the SQL query is: … WHERE userID=’’ or ’x’=’x’

To prevent errors: sanitize user data before use

Vulnerable code

void op(String in) {

…
executeQuery(in);
}
…
op(userInput);
Vulnerable code
Where is the defect?
void op(String in) {
…
executeQuery(in);
}
…
op(userInput);
Vulnerable code
Where is the defect?
void op(String in) {
…
executeQuery(in);
}
…
op(userInput);
Vulnerable code
Where is the defect?
void op(String in) {
…
executeQuery(in);
}
Can’t decide without specification!
…
op(userInput);
Specification 1: untainted parameter

void op(@Untainted String in) {

…
executeQuery(in);
}
…
op(userInput);
Specification 1: untainted parameter

void op(@Untainted String in) {

…
executeQuery(in);
}
…
op(userInput); // error
Specification 2: tainted parameter

void op(@Tainted String in) {

…
executeQuery(in);
}
…
op(userInput);
Specification 2: tainted parameter

void op(@Tainted String in) {

…
executeQuery(in); // error
}
…
op(userInput);
Demo: Preventing SQL injection
Goal: don’t execute user input as a SQL command
private String wrapQuery(String s) {
return "SELECT * FROM User WHERE userId=’" + s + "’";
}
If a user inputs his name as: ’ or ’x’=’x
the SQL query is: … WHERE userID=’’ or ’x’=’x’

@Tainted = might be untrusted user input

@Untainted = sanitized, safe to use
Verification approach:
Pluggable Type Checking
1. Design a type system to solve a specific problem
2. Write type qualifiers in code (or, use type inference)
@Immutable Date date = new Date();
date.setSeconds(0); // compile-time error
3. Type checker warns about violations (bugs)
% javac -processor NullnessChecker MyFile.java

MyFile.java:149: dereference of possibly-null reference bb2

allVars = bb2.vars;
^
Optional Type Checking
No errors
Source Compiler Executable

Fix bugs
Change types Errors
Optional Type Checking
No errors
Source Compiler Executable

Guaranteed
Fix bugs Optional behavior
Change types Errors
Type Checker

Fix bugs
Add/change
Warnings
annotations
Optional Type Checking
No errors
Source Compiler Executable

Guaranteed
Fix bugs Optional
Optional behavior
Change types Errors Optional
Type
TypeChecker
TypeChecker
Checker

Fix bugs
Add/change
Warnings
annotations
Prevent null pointer exceptions
Type system that statically guarantees that:
the program only dereferences
known non-null references

Types of data:
@NonNull reference is never null
@Nullable reference may be null
Null pointer exception

String op(Data in) {

return "transform: " + in.getF();
}
…
String s = op(null);
Null pointer exception
Where
Whereisisthe
thedefect?
defect?
String op(Data in) {
return "transform: " + in.getF();
}
…
String s = op(null);
Null pointer exception
Where
Whereisisthe
thedefect?
defect?
String op(Data in) {
return "transform: " + in.getF();
}
…
String s = op(null);
Null pointer exception
Where
Whereisisthe
thedefect?
defect?
String op(Data in) {
return "transform: " + in.getF();
}
Can’t decide without specification!
…
String s = op(null);
Specification 1: non-null parameter

String op(@NonNull Data in) {

return "transform: " + in.getF();
}
…
String s = op(null);
Specification 1: non-null parameter

String op(@NonNull Data in) {

return "transform: " + in.getF();
}
…
String s = op(null); // error
Specification 2: nullable parameter

String op(@Nullable Data in) {

return "transform: " + in.getF();
}
…
String s = op(null);
Specification 2: nullable parameter

String op(@Nullable Data in) {

return "transform: " + in.getF();
} // error
…
String s = op(null);
Nullness demo
● Detect errors
● Guarantee the absence of errors
● Verify the correctness of optimizations
Benefits of type systems
● Find bugs in programs
○ Guarantee the absence of errors
● Improve documentation
○ Improve code structure & maintainability
● Aid compilers, optimizers, and analysis tools
○ E.g., could reduce number of run-time checks

● Possible negatives:
○ Must write the types (or use type inference)
○ False positives are possible (can be suppressed)
The Checker Framework
A framework for pluggable type checkers
“Plugs” into the OpenJDK or OracleJDK compiler

javac -processor MyChecker …

Standard error format allows tool integration

 Ant, Maven, Gradle integration
<presetdef name="jsr308.javac">
<javac fork="yes"
executable="${checkerframework}/checker/bin/${cfJavac}" >
<!-- JSR-308-related compiler arguments -->
<compilerarg value="-version"/>
<compilerarg value="-implicit:class"/>
</javac>
<dependencies>
</presetdef>
... existing <dependency> items ...
<!-- annotations from the Checker Framework:
nullness, interning, locking, ... -->
<dependency>
<groupId>org.checkerframework</groupId>
<artifactId>checker-qual</artifactId>
<version>1.9.7</version>
</dependency>
</dependencies>
Eclipse, IntelliJ, NetBeans integration
 Comparison: other nullness tools
Null pointer errors False Annotations
Found Missed warnings written
Checker Framework 9 0 4 35
FindBugs 0 9 1 0
Jlint 0 9 8 0
PMD 0 9 0 0
Eclipse, in 2017 0 9 8 0
Intellij (@NotNull 0 9 1 0
default), in 2017 3 6 1 925 + 8
Checking the Lookup program for file system searching (4kLOC)
Live demo: http://CheckerFramework.org/live/
Example type systems
Null dereferences (@NonNull)
>200 errors in Google Collections, javac, ...
Equality tests (@Interned)
>200 problems in Xerces, Lucene, ...
Concurrency / locking (@GuardedBy)
>500 errors in BitcoinJ, Derby, Guava, Tomcat, ...
Fake enumerations / typedefs (@Fenum)
problems in Swing, JabRef
String type systems
Regular expression syntax (@Regex)
56 errors in Apache, etc.; 200 annos required
printf format strings (@Format)
104 errors, only 107 annotations required
Method signature format (@FullyQualified)
28 errors in OpenJDK, ASM, AFU
Compiler messages (@CompilerMessageKey)
8 wrong keys in Checker Framework
Security type systems
Command injection vulnerabilities (@OsTrusted)
5 missing validations in Hadoop
Information flow privacy (@Source)
SPARTA detected malware in Android apps

It’s easy to write your own type system!

Checkers are usable
● Type-checking is familiar to programmers
● Modular: fast, incremental, partial programs
● Annotations are not too verbose
○ @NonNull: 1 per 75 lines
○ @Interned: 124 annotations in 220 KLOC revealed 11 bugs
○ @Format: 107 annotations in 2.8 MLOC revealed 104 bugs
○ Possible to annotate part of program
○ Fewer annotations in new code
● Few false positives
● First-year CS majors preferred using checkers to not
● Practical: in use in Silicon Valley, on Wall Street, etc.
What a checker guarantees
The program satisfies the type property. There are:
○ no bugs (of particular varieties)
○ no wrong annotations
● Caveat 1: only for code that is checked

○ Native methods (handles reflection!)

○ Code compiled without the pluggable type checker
○ Suppressed warnings
■ Indicates what code a human should analyze
Checking part of a program is still useful
● Caveat 2: The checker itself might contain an error
Practicality

Testing Pluggable
Type Systems
Built-in Type
Systems

Formal
Verification

Guarantees
Demo: regular expression errors
@Regex = valid regular expression
OK: “colou?r”
NOT: “1) first point”
@Regex(2) = has 2+ capturing groups
OK: “((Linked)?Hash)?Map”
OK: “(http|ftp)://([^/]+)(/.*)?”
NOT: “(brown|beige)”
Regular Expression Example
public static void main(String[] args) {
String regex = args[0];
String content = args[1];
Pattern pat = Pattern.compile(regex);
Matcher mat = pat.matcher(content);
if (mat.matches()) {
System.out.println("Group: " + mat.group(1));
}
}
Regular Expression Example
public static void main(String[] args) {
String regex = args[0];
String content PatternSyntaxException
= args[1];
Pattern pat = Pattern.compile(regex);
Matcher mat = pat.matcher(content);
IndexOutOfBoundsExceptionon
if (mat.matches()) {
System.out.println("Group: " + mat.group(1));
}
}
Fixing the Errors
Pattern.compile only on valid regex
Matcher.group(i) only if > i groups
...
if (!RegexUtil.isRegex(regex, 1)) {
System.out.println("Invalid: " + regex);
System.exit(1);
}
...
Since Java 5: declaration annotations
Only for declaration locations:
@Deprecated
class Foo {
@Getter @Setter private String query;
@SuppressWarnings("unchecked")
void foo() { … }
}
But we couldn’t express
A non-null reference to my data

An interned string

A non-null List of English strings

A non-empty array of English strings

With Java 8 Type Annotations we can!
A non-null reference to my data
@NonNull Data mydata;
An interned String
@Interned String query;
A non-null List of English Strings
@NonNull List<@English String> msgs;
A non-empty array of English strings
@English String @NonEmpty [] a;
Java 8 extends annotation syntax
Annotations on all occurrences of types:
@Untainted String query;
List<@NonNull String> strings;
myGraph = (@Immutable Graph) tmp;
class UnmodifiableList<T>
implements @Readonly List<T> {}
Stored in classfile
Handled by javac, javap, javadoc, …
Java 6 & 7 compatibility
(or avoid dependency on Checker Framework)

Annotations in comments:

List</*@NonNull*/ String> strings;

(Requires use of jsr308-langtools compiler.)

Annotating external libraries
When type-checking clients, need library spec.
Can write manually or automatically infer
Two syntaxes:
● As separate text file (stub file)

● Within its .jar file (from annotated partial

source code)
Checker Framework facilities
● Full type systems: inheritance, overriding, ...
● Generics (type polymorphism)
○ Also qualifier polymorphism
● Qualifier defaults
● Pre-/post-conditions
● Warning suppression
Brainstorming new type checkers
What runtime exceptions to prevent?
What properties of data should always hold?
What operations are legal and illegal?
Type-system checkable properties:
● Dependency on values

● Not on program structure, timing, ...

Example: Nullness Checker
What runtime exceptions to prevent?

What properties of data should always hold?

What operations are legal and illegal?

Example: Nullness Checker
What runtime exceptions to prevent?
NullPointerException
What properties of data should always hold?

What operations are legal and illegal?

Example: Nullness Checker
What runtime exceptions to prevent?
NullPointerException
What properties of data should always hold?
@NonNull references always non-null
What operations are legal and illegal?
Example: Nullness Checker
What runtime exceptions to prevent?
NullPointerException
What properties of data should always hold?
@NonNull references always non-null
What operations are legal and illegal?
Dereferences only on @NonNull references
Example: Regex Checker
What runtime exceptions to prevent?

What properties of data should always hold?

What operations are legal and illegal?

Example: Regex Checker
What runtime exceptions to prevent?
PatternSyntaxException,
IndexOutOfBoundsException
What properties of data should always hold?

What operations are legal and illegal?

Example: Regex Checker
What runtime exceptions to prevent?
PatternSyntaxException,
IndexOutOfBoundsException
What properties of data should always hold?
Whether a string is a regex and number of groups
What operations are legal and illegal?
Example: Regex Checker
What runtime exceptions to prevent?
PatternSyntaxException,
IndexOutOfBoundsException
What properties of data should always hold?
Whether a string is a regex and number of groups
What operations are legal and illegal?
Pattern.compile with non-@Regexp, etc,
New type system
What runtime exceptions to prevent?
1 NumberFormatException
What properties of data should always hold?
2 @ValidNumber vs. @Unknown
What operations are legal and illegal?
3 Integer.valueOf only with @ValidNumber
New type system
What runtime exceptions to prevent?
1 Class/MethodNotFoundException
What properties of data should always hold?
2 @ClassName, @MethodName

What operations are legal and illegal?

3 Class.forName only on @ClassName
New type system
What runtime exceptions to prevent?
1 UnsupportedOperation from Lists
What properties of data should always hold?
2 @Mutable vs @UnknownMutable

What operations are legal and illegal?

3 .add .remove only on @Mutable Lists
New type system
What runtime exceptions to prevent?
1 Invalid quoting of strings
What properties of data should always hold?
2 @JSTainted @HTMLTainted etc

What operations are legal and illegal?

3 println, dislpayHTML
New type system
What runtime exceptions to prevent?
1 Unknown serializer exception
What properties of data should always hold?
2 @JsonSerializable vs.
@UnknownSerializable
What operations are legal and illegal?
3 .toJson only on @JsonSerializable
Building a checker is easy
Example: Ensure encrypted communication
void send(@Encrypted String msg) {…}
@Encrypted String msg1 = ...;
send(msg1); // OK
String msg2 = ....;
send(msg2); // Warning!
Building a checker is easy
Example: Ensure encrypted communication
void send(@Encrypted String msg) {…}
@Encrypted String msg1 = ...;
send(msg1); // OK
String msg2 = ....;
send(msg2); // Warning!
The complete checker:
@Target(ElementType.TYPE_USE)
@SubtypeOf(Unqualified.class)
public @interface Encrypted {}
Encrypted Checker Demo
Let’s build it!
Testing infrastructure
jtreg-based testing as in OpenJDK

Lightweight tests with in-line expected errors:

String s = "%+s%";
//:: error: (format.string.invalid)
f.format(s, "illegal");
Defining a type system
1. Qualifier hierarchy
○ defines subtyping
2. Type introduction rules
○ types for expressions
3. Type rules
○ checker-specific errors
4. Flow-refinement
○ better types than the programmer wrote
Defining a type system
1. Qualifier hierarchy
○ subtyping, assignments @UnknownRegex

@SubtypeOf(UnknownRegex.class) @Regex
public @interface Regex {
Defining a type system
Data d = new Data();
2. Type introduction rules
○ types for expressions
@ImplicitFor( trees = {
Tree.Kind.NEW_CLASS,
Tree.Kind.NEW_ARRAY, ... })
@DefaultQualifierInHierarchy
@DefaultForUnannotatedCode({
DL.PARAMETERS, DL.LOWER_BOUNDS })
Defining a type system
synchronized(xxx) {
3. Type rules }
○ checker-specific errors
void visitSynchronized(SynchronizedTree node) {
ExpressionTree expr = node.getExpression();
AnnotatedTypeMirror type =
getAnnotatedType(expr);
if (!type.hasAnnotation(NONNULL))
checker.report(Result.failure(...), expr);
}
Defining a type system
if (x != null) {
4. Flow-refinement x.f = …; // valid
○ better types than the programmer wrote

if (ElementUtils.matchesElement(method,
IS_REGEX_METHOD_NAME,
String.class, int.class)) {
…
}
Dataflow Framework
Goal: Compute properties about expressions
● More accurate types than the user wrote
● Foundation for other static analyses
○ e.g. by Google error-prone and Uber NullAway
Dataflow Framework user provides
● What are we tracking?
● What do operations do?
● What are intermediate results?
Dataflow Framework does all the work!
Tips
● Start by type-checking part of your code
● Only type-check properties that matter to you
● Use subclasses (not type qualifiers) if possible
● Write the spec first (and think of it as a spec)
● Avoid warning suppressions when possible
● Avoid raw types such as List; use List<String>
 Verification Bug-finding
• Goal: • Goal:
prove that no bug exists find some bugs at low cost
• Specifications: • Specifications:
user provides infer likely specs
• False negatives: • False negatives:
none acceptable
• False positives: • False positives:
user suppresses warnings heuristics focus on most
important bugs
• Downside: user burden • Downside: missed bugs
Checker Framework Community
Open source project:
https://github.com/typetools/checker-framework

● Monthly release cycle

● >12,800 commits, 75 authors
● Welcoming & responsive community
Checker Framework plans
More type systems:
● Immutability
● Index-out-of-bounds
● Optional<T> type
● Determinism
● Signed vs. unsigned numbers
Type inference
Combined static & dynamic enforcement
More at JavaOne 2017
Using Type Annotations to Improve Your Code
BoF3048, tonight, 18:30 to 19:15
Moscone West - Room 2014
Pluggable type-checking improves code
Checker Framework for creating type checkers
● Featureful, effective, easy to use, scalable
Prevent bugs at compile time
Create custom type-checkers
Improve your code!

http://CheckerFramework.org/