The Rust Programming

Language for Game Tooling

Dan Olson
Principal Software Architect
 Who am I?
• Working in games since 2004, Treyarch since 2008.

• Core Engine team.

• Focused on data pipeline and infrastructure tooling.

The Rust Programming Language For Game Tooling

 What is Rust?
• Started by Mozilla in 2006, stable release in 2015.

• Currently supported by Amazon, Facebook, Microsoft, and others.

• Focused on security and performance.

The Rust Programming Language For Game Tooling

 Outline
• The case for Rust.

• Survey of several interesting uses for Rust.

• Integrating Rust at Treyarch.

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
• Easy to read. hasher = hashlib.md5()
with open(filename, 'rb') as f:
• Easy to write. hasher.update(f.read())
print(hasher.hexdigest())
• Errors are handled.

• Performs well.

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
• Easy to read. 😀 hasher = hashlib.md5()
with open(filename, 'rb') as f:
• Easy to write. 😀 hasher.update(f.read())
print(hasher.hexdigest())
• Errors are handled.

• Performs well.

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
• Easy to read. 😀 hasher = hashlib.md5()
with open(filename, 'rb') as f:
• Easy to write. 😀 hasher.update(f.read())
print(hasher.hexdigest())
• Errors are handled. 😐

• Performs well.

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
• Easy to read. 😀 hasher = hashlib.md5()
with open(filename, 'rb') as f:
• Easy to write. 😀 hasher.update(f.read())
print(hasher.hexdigest())
• Errors are handled. 😐

• Performs well. 🤔

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
int file_descript = open(filename, O_RDONLY);
• Easy to read. if(file_descript < 0) exit(-1);

• Easy to write. unsigned long file_size =

get_size_by_fd(file_descript);
• Errors are handled.
char* file_buffer = mmap(0, file_size,
PROT_READ, MAP_SHARED, file_descript, 0);
• Performs well. MD5((unsigned char*) file_buffer, file_size,
result);
munmap(file_buffer, file_size);

print_md5_sum(result);
Source: https://stackoverflow.com/a/1220177/69283

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
int file_descript = open(filename, O_RDONLY);
• Easy to read. 😐 if(file_descript < 0) exit(-1);

• Easy to write. unsigned long file_size =

get_size_by_fd(file_descript);
• Errors are handled.
char* file_buffer = mmap(0, file_size,
PROT_READ, MAP_SHARED, file_descript, 0);
• Performs well. MD5((unsigned char*) file_buffer, file_size,
result);
munmap(file_buffer, file_size);

print_md5_sum(result);
Source: https://stackoverflow.com/a/1220177/69283

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
int file_descript = open(filename, O_RDONLY);
• Easy to read. 😐 if(file_descript < 0) exit(-1);

• Easy to write. 😢 unsigned long file_size =

get_size_by_fd(file_descript);
• Errors are handled.
char* file_buffer = mmap(0, file_size,
PROT_READ, MAP_SHARED, file_descript, 0);
• Performs well. MD5((unsigned char*) file_buffer, file_size,
result);
munmap(file_buffer, file_size);

print_md5_sum(result);
Source: https://stackoverflow.com/a/1220177/69283

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
int file_descript = open(filename, O_RDONLY);
• Easy to read. 😐 if(file_descript < 0) exit(-1);

• Easy to write. 😢 unsigned long file_size =

get_size_by_fd(file_descript);
• Errors are handled. 😬
char* file_buffer = mmap(0, file_size,
PROT_READ, MAP_SHARED, file_descript, 0);
• Performs well. MD5((unsigned char*) file_buffer, file_size,
result);
munmap(file_buffer, file_size);

print_md5_sum(result);
Source: https://stackoverflow.com/a/1220177/69283

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
int file_descript = open(filename, O_RDONLY);
• Easy to read. 😐 if(file_descript < 0) exit(-1);

• Easy to write. 😢 unsigned long file_size =

get_size_by_fd(file_descript);
• Errors are handled. 😬
char* file_buffer = mmap(0, file_size,
PROT_READ, MAP_SHARED, file_descript, 0);
• Performs well. 😀 MD5((unsigned char*) file_buffer, file_size,
result);
munmap(file_buffer, file_size);

print_md5_sum(result);
Source: https://stackoverflow.com/a/1220177/69283

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
• Easy to read. let data = std::fs::read(filename)?;
let hash = md5::compute(&data);
• Easy to write. println!("{:x}", hash);

• Errors are handled.

• Performs well.

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
• Easy to read. 😀 let data = std::fs::read(filename)?;
let hash = md5::compute(&data);
• Easy to write. 😀 println!("{:x}", hash);

• Errors are handled.

• Performs well.

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
• Easy to read. 😀 let data = std::fs::read(filename)?;
let hash = md5::compute(&data);
• Easy to write. 😀 println!("{:x}", hash);

• Errors are handled. 😐

• Performs well.

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
• Easy to read. 😀 let data = std::fs::read(filename)?;
let hash = md5::compute(&data);
• Easy to write. 😀 println!("{:x}", hash);

• Errors are handled. 😐

• Performs well. 😀

The Rust Programming Language For Game Tooling

 Dan’s Rust Sales Pitch
• Efficiency of writing code: closer to Python.

• Efficiency of running code: closer to C++.

• Large, centralized ecosystem of “crates”, or community libraries (https://crates.io/).

• Integrated build + package + test tool (“cargo”).

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
q:\>cargo new md5sum
Created binary (application) `md5sum` package
q:\>cd md5sum
q:\md5sum>cargo add md5
Adding md5 v0.7.0 to dependencies
q:\md5sum>cargo run -- src/main.rs
Compiling md5 v0.7.0
Compiling md5sum v0.1.0 (Q:\md5sum)
Finished dev [unoptimized + debuginfo] target(s) in 1.66s
Running `target\debug\md5sum.exe src/main.rs`
4911739566caf58bf40be5b6d6a19262

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
q:\>cargo new md5sum
Created binary (application) `md5sum` package
q:\>cd md5sum
q:\md5sum>cargo add md5
Adding md5 v0.7.0 to dependencies
q:\md5sum>cargo run -- src/main.rs
Compiling md5 v0.7.0
Compiling md5sum v0.1.0 (Q:\md5sum)
Finished dev [unoptimized + debuginfo] target(s) in 1.66s
Running `target\debug\md5sum.exe src/main.rs`
4911739566caf58bf40be5b6d6a19262

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
q:\>cargo new md5sum
Created binary (application) `md5sum` package
q:\>cd md5sum
q:\md5sum>cargo add md5
Adding md5 v0.7.0 to dependencies
q:\md5sum>cargo run -- src/main.rs
Compiling md5 v0.7.0
Compiling md5sum v0.1.0 (Q:\md5sum)
Finished dev [unoptimized + debuginfo] target(s) in 1.66s
Running `target\debug\md5sum.exe src/main.rs`
4911739566caf58bf40be5b6d6a19262

The Rust Programming Language For Game Tooling

 Code comparison: md5sum
q:\>cargo new md5sum
Created binary (application) `md5sum` package
q:\>cd md5sum
q:\md5sum>cargo add md5
Adding md5 v0.7.0 to dependencies
q:\md5sum>cargo run -- src/main.rs
Compiling md5 v0.7.0
Compiling md5sum v0.1.0 (Q:\md5sum)
Finished dev [unoptimized + debuginfo] target(s) in 1.66s
Running `target\debug\md5sum.exe src/main.rs`
4911739566caf58bf40be5b6d6a19262

The Rust Programming Language For Game Tooling

 Dan’s Rust Sales Pitch
• Efficiency of writing code: closer to Python.

• Efficiency of running code: closer to C++.

• Large, centralized ecosystem of “crates”, or community libraries (https://crates.io/).

• Integrated build + package + test tool (“cargo”).

• Static, compile-time validation of common memory problems.

• Static, compile-time validation of common multithreading problems.

The Rust Programming Language For Game Tooling

 Case study: Treyarch Image Packer
• Rust version deployed in 2018.

• Heavily multithreaded.

• Active development throughout its lifetime.

• Total “crash” issues encountered: 2.

The Rust Programming Language For Game Tooling

 Dan’s Rust Sales Pitch
• Efficiency of writing code: closer to Python.

• Efficiency of running code: closer to C++.

• Large, centralized ecosystem of “crates”, or community libraries (https://crates.io/).

• Integrated build + package + test tool (“cargo”).

• Static, compile-time validation of common memory problems.

• Static, compile-time validation of common multithreading problems.

The Rust Programming Language For Game Tooling

 Rust for Game Tools
• Error Handling

• Multithreading

• Parsing Text

• Command Line Interfaces

• Parsing Debug Info

• C ABI compatibility

• Web Applications

• GUIs
The Rust Programming Language For Game Tooling
 1/8 - Error Handling
• Result – holds the success or failure state of an operation.

• Panic – instant program failure for unrecoverable errors.

• ? Operator – pass a failed Result up the callstack.

let file = std::fs::read(path)?;

• Use the anyhow crate to add context to errors.

The Rust Programming Language For Game Tooling

 1/8 - Error Handling

Error: The system cannot find the file specified.

let file = std::fs::read(path)?;
(os error 2)

let file = std::fs::read(path)

.with_context(|| format!("Reading contents of {:?}", path))?;

Error: Reading contents of "test.txt"

Caused by:
The system cannot find the file specified. (os error 2)

The Rust Programming Language For Game Tooling

 2/8 - Multithreading
• Using the rayon crate, multithreading is quick, easy, and safe.

file_names file_names
.iter() .par_iter()
.map(|x| hash_file(x)) .map(|x| hash_file(x))
.collect(); .collect();

The Rust Programming Language For Game Tooling

 2/8 - Multithreading
• Using the rayon crate, multithreading is quick, easy, and safe.

file_names
.par_iter()
.map(|x| hash_file(x, &mut count))
.collect();

The Rust Programming Language For Game Tooling

 3/8 - Parsing Text
• Use the serde crate for generic serialization/deserialization.

#[derive(Deserialize)]
struct Config {
string: String,
number: i32,
list: Vec<String>,
}

let config: Config = serde_json::from_str(&text)?;

let config: Config = serde_yaml::from_str(&text)?;

The Rust Programming Language For Game Tooling

 4/8 - Command Line Interfaces
• Use the structopt crate to create command line interfaces.
sourcehash
/// Generate
0.1.0a hash of all source and include files specified
Generate
by one aor
hash
more
of.vcxproj
all source
files.
and include files specified by one or
more
#[derive(StructOpt)]
.vcxproj files
struct SourceHash {
USAGE: /// One or more vcxproj files contributing to the hash.
sourcehash.exe
#[structopt(parse(from_os_str))]
[FLAGS] [files]...
files: Vec<PathBuf>,
FLAGS:
-h,#[structopt(long)]
--help Prints help information
-V,verbose:
--version
bool, Prints version information
} --verbose

ARGS: let options = SourceHash::from_args();

The Rust Programming Language For Game Tooling

<files>... One or more vcxproj files contributing to the hash

 5/8 - Parsing Debug Info
gz_header_s - 80 bytes, 12 padding
• Use pdb (win) and gimli (elf) crates to (optimal size should be 72 bytes, 4 padding)
struct gz_header_s
inspect debug info. {
int text; // 4 bytes
<padding> ; // 4 bytes
• This tool is now open-sourced! uLong time; // 8 bytes
int xflags; // 4 bytes
int os; // 4 bytes
• https://github.com/Activision/structpack Bytef* extra; // 8 bytes
uInt extra_len; // 4 bytes
uInt extra_max; // 4 bytes
Bytef* name; // 8 bytes
uInt name_max; // 4 bytes
<padding> ; // 4 bytes
Bytef* comment; // 8 bytes
uInt comm_max; // 4 bytes
int hcrc; // 4 bytes
int done; // 4 bytes
<padding> ; // 4 bytes
};

The Rust Programming Language For Game Tooling

 6/8 - C ABI compatibility
• Bind Rust code to other languages (e.g. python, nodejs, C, wasm).
/// Formats the sum of two numbers as string.
#[pyfunction]
fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
Ok((a + b).to_string())
}

/// A Python module implemented in Rust.

#[pymodule]
fn string_sum(py: Python, m: &PyModule) -> PyResult<()> {
m.add_function(wrap_pyfunction!(sum_as_string, m)?)?;

Ok(())
}
Source: https://crates.io/crates/pyo3
The Rust Programming Language For Game Tooling
 7/8 - Web Applications
• There are lots and lots of crates for web apps. I like rouille for quick, simple ones.
rouille::start_server("0.0.0.0:80", move |request| {
Response::text("hello world")
});
Source: https://docs.rs/rouille/3.1.1/rouille/
• But tide or actix-web might be better for more substantial apps.
#[async_std::main]
async fn main() -> tide::Result<()> {
let mut app = tide::new();
app.at("/orders/shoes").post(order_shoes);
app.listen("127.0.0.1:8080").await?;
Ok(())
}
Source: https://crates.io/crates/tide
The Rust Programming Language For Game Tooling
 8/8 - GUIs
• Native Rust: iced, druid, egui

• C++ Bindings: ritual (Qt), relm (Gtk), imgui-rs, web-view

ui.heading("My egui Application");

ui.horizontal(|ui| {
ui.label("Your name: ");
ui.text_edit_singleline(&mut name);
});
ui.add(egui::Slider::new(&mut age, 0..=120).text("age"));
if ui.button("Click each year").clicked() {
age += 1;
}
ui.label(format!("Hello '{}', age {}", name, age));

Source: https://crates.io/crates/egui

The Rust Programming Language For Game Tooling

 Integrating Rust at Treyarch
• Started work in late 2017.

• 3 major tools, around 20 smaller one-off tools.

• Around 120K LOC

• 27 individual contributors.

• Extremely high stability.

The Rust Programming Language For Game Tooling

 Rust Downsides
• Steep learning curve… very different from C++.

• Complicated language… lots of corners.

• Relies heavily on ecosystem.

• Compile times as bad as or worse than C++.

Managing these downsides is key to successful integration!

The Rust Programming Language For Game Tooling

 Integration Tips
• Communicate!

• Keep Rust siloed off until there is a critical mass of experience.

▪ Find an unmaintained tool and make it better!

• Start with the best workflow.

▪ vscode + rust_analyzer + cargo-edit + rustfmt + clippy

• Bad Rust code has the same safety guarantees as good Rust code!

• Initial hurdles are high, but large productivity gains after they are cleared.

• Provide time and space to both learn and teach.

▪ Presentations, courses, examples, code reviews.

The Rust Programming Language For Game Tooling

 Learning Resources
• Dive into code!
▪ https://leetcode.com/
▪ https://adventofcode.com/

• Dive into a book!

▪ “The Rust Programming Language” – Klabnik & Nichols
▪ “Programming Rust” – Blandy & Orendorff

• Dive into online resources!

▪ https://www.rust-lang.org/learn
▪ https://play.rust-lang.org/

The Rust Programming Language For Game Tooling

Thank you!
 We’re Hiring!

Gameplay Engineers (mid/senior)

UI Engineers (mid/senior)
Online Engineers (mid/senior)
Senior Test Automation Engineers

And many more!

https://careers.treyarch.com/