The Rust Programming
Language for Game Tooling
Dan Olson
Principal Software Architect
Who am I?
• Working in games since 2004, Treyarch since 2008.
• Core Engine team.
• Focused on data pipeline and infrastructure tooling.
The Rust Programming Language For Game Tooling
What is Rust?
• Started by Mozilla in 2006, stable release in 2015.
• Currently supported by Amazon, Facebook, Microsoft, and others.
• Focused on security and performance.
The Rust Programming Language For Game Tooling
Outline
• The case for Rust.
• Survey of several interesting uses for Rust.
• Integrating Rust at Treyarch.
The Rust Programming Language For Game Tooling
Code comparison: md5sum
• Easy to read. hasher = hashlib.md5()
with open(filename, 'rb') as f:
• Easy to write. hasher.update(f.read())
print(hasher.hexdigest())
• Errors are handled.
• Performs well.
The Rust Programming Language For Game Tooling
Code comparison: md5sum
• Easy to read. 😀 hasher = hashlib.md5()
with open(filename, 'rb') as f:
• Easy to write. 😀 hasher.update(f.read())
print(hasher.hexdigest())
• Errors are handled.
• Performs well.
The Rust Programming Language For Game Tooling
Code comparison: md5sum
• Easy to read. 😀 hasher = hashlib.md5()
with open(filename, 'rb') as f:
• Easy to write. 😀 hasher.update(f.read())
print(hasher.hexdigest())
• Errors are handled. 😐
• Performs well.
The Rust Programming Language For Game Tooling
Code comparison: md5sum
• Easy to read. 😀 hasher = hashlib.md5()
with open(filename, 'rb') as f:
• Easy to write. 😀 hasher.update(f.read())
print(hasher.hexdigest())
• Errors are handled. 😐
• Performs well. 🤔
The Rust Programming Language For Game Tooling
Code comparison: md5sum
int file_descript = open(filename, O_RDONLY);
• Easy to read. if(file_descript < 0) exit(-1);
• Easy to write. unsigned long file_size =
get_size_by_fd(file_descript);
• Errors are handled.
char* file_buffer = mmap(0, file_size,
PROT_READ, MAP_SHARED, file_descript, 0);
• Performs well. MD5((unsigned char*) file_buffer, file_size,
result);
munmap(file_buffer, file_size);
print_md5_sum(result);
Source: https://stackoverflow.com/a/1220177/69283
The Rust Programming Language For Game Tooling
Code comparison: md5sum
int file_descript = open(filename, O_RDONLY);
• Easy to read. 😐 if(file_descript < 0) exit(-1);
• Easy to write. unsigned long file_size =
get_size_by_fd(file_descript);
• Errors are handled.
char* file_buffer = mmap(0, file_size,
PROT_READ, MAP_SHARED, file_descript, 0);
• Performs well. MD5((unsigned char*) file_buffer, file_size,
result);
munmap(file_buffer, file_size);
print_md5_sum(result);
Source: https://stackoverflow.com/a/1220177/69283
The Rust Programming Language For Game Tooling
Code comparison: md5sum
int file_descript = open(filename, O_RDONLY);
• Easy to read. 😐 if(file_descript < 0) exit(-1);
• Easy to write. 😢 unsigned long file_size =
get_size_by_fd(file_descript);
• Errors are handled.
char* file_buffer = mmap(0, file_size,
PROT_READ, MAP_SHARED, file_descript, 0);
• Performs well. MD5((unsigned char*) file_buffer, file_size,
result);
munmap(file_buffer, file_size);
print_md5_sum(result);
Source: https://stackoverflow.com/a/1220177/69283
The Rust Programming Language For Game Tooling
Code comparison: md5sum
int file_descript = open(filename, O_RDONLY);
• Easy to read. 😐 if(file_descript < 0) exit(-1);
• Easy to write. 😢 unsigned long file_size =
get_size_by_fd(file_descript);
• Errors are handled. 😬
char* file_buffer = mmap(0, file_size,
PROT_READ, MAP_SHARED, file_descript, 0);
• Performs well. MD5((unsigned char*) file_buffer, file_size,
result);
munmap(file_buffer, file_size);
print_md5_sum(result);
Source: https://stackoverflow.com/a/1220177/69283
The Rust Programming Language For Game Tooling
Code comparison: md5sum
int file_descript = open(filename, O_RDONLY);
• Easy to read. 😐 if(file_descript < 0) exit(-1);
• Easy to write. 😢 unsigned long file_size =
get_size_by_fd(file_descript);
• Errors are handled. 😬
char* file_buffer = mmap(0, file_size,
PROT_READ, MAP_SHARED, file_descript, 0);
• Performs well. 😀 MD5((unsigned char*) file_buffer, file_size,
result);
munmap(file_buffer, file_size);
print_md5_sum(result);
Source: https://stackoverflow.com/a/1220177/69283
The Rust Programming Language For Game Tooling
Code comparison: md5sum
• Easy to read. let data = std::fs::read(filename)?;
let hash = md5::compute(&data);
• Easy to write. println!("{:x}", hash);
• Errors are handled.
• Performs well.
The Rust Programming Language For Game Tooling
Code comparison: md5sum
• Easy to read. 😀 let data = std::fs::read(filename)?;
let hash = md5::compute(&data);
• Easy to write. 😀 println!("{:x}", hash);
• Errors are handled.
• Performs well.
The Rust Programming Language For Game Tooling
Code comparison: md5sum
• Easy to read. 😀 let data = std::fs::read(filename)?;
let hash = md5::compute(&data);
• Easy to write. 😀 println!("{:x}", hash);
• Errors are handled. 😐
• Performs well.
The Rust Programming Language For Game Tooling
Code comparison: md5sum
• Easy to read. 😀 let data = std::fs::read(filename)?;
let hash = md5::compute(&data);
• Easy to write. 😀 println!("{:x}", hash);
• Errors are handled. 😐
• Performs well. 😀
The Rust Programming Language For Game Tooling
Dan’s Rust Sales Pitch
• Efficiency of writing code: closer to Python.
• Efficiency of running code: closer to C++.
• Large, centralized ecosystem of “crates”, or community libraries (https://crates.io/).
• Integrated build + package + test tool (“cargo”).
The Rust Programming Language For Game Tooling
Code comparison: md5sum
q:\>cargo new md5sum
Created binary (application) `md5sum` package
q:\>cd md5sum
q:\md5sum>cargo add md5
Adding md5 v0.7.0 to dependencies
q:\md5sum>cargo run -- src/main.rs
Compiling md5 v0.7.0
Compiling md5sum v0.1.0 (Q:\md5sum)
Finished dev [unoptimized + debuginfo] target(s) in 1.66s
Running `target\debug\md5sum.exe src/main.rs`
4911739566caf58bf40be5b6d6a19262
The Rust Programming Language For Game Tooling
Code comparison: md5sum
q:\>cargo new md5sum
Created binary (application) `md5sum` package
q:\>cd md5sum
q:\md5sum>cargo add md5
Adding md5 v0.7.0 to dependencies
q:\md5sum>cargo run -- src/main.rs
Compiling md5 v0.7.0
Compiling md5sum v0.1.0 (Q:\md5sum)
Finished dev [unoptimized + debuginfo] target(s) in 1.66s
Running `target\debug\md5sum.exe src/main.rs`
4911739566caf58bf40be5b6d6a19262
The Rust Programming Language For Game Tooling
Code comparison: md5sum
q:\>cargo new md5sum
Created binary (application) `md5sum` package
q:\>cd md5sum
q:\md5sum>cargo add md5
Adding md5 v0.7.0 to dependencies
q:\md5sum>cargo run -- src/main.rs
Compiling md5 v0.7.0
Compiling md5sum v0.1.0 (Q:\md5sum)
Finished dev [unoptimized + debuginfo] target(s) in 1.66s
Running `target\debug\md5sum.exe src/main.rs`
4911739566caf58bf40be5b6d6a19262
The Rust Programming Language For Game Tooling
Code comparison: md5sum
q:\>cargo new md5sum
Created binary (application) `md5sum` package
q:\>cd md5sum
q:\md5sum>cargo add md5
Adding md5 v0.7.0 to dependencies
q:\md5sum>cargo run -- src/main.rs
Compiling md5 v0.7.0
Compiling md5sum v0.1.0 (Q:\md5sum)
Finished dev [unoptimized + debuginfo] target(s) in 1.66s
Running `target\debug\md5sum.exe src/main.rs`
4911739566caf58bf40be5b6d6a19262
The Rust Programming Language For Game Tooling
Dan’s Rust Sales Pitch
• Efficiency of writing code: closer to Python.
• Efficiency of running code: closer to C++.
• Large, centralized ecosystem of “crates”, or community libraries (https://crates.io/).
• Integrated build + package + test tool (“cargo”).
• Static, compile-time validation of common memory problems.
• Static, compile-time validation of common multithreading problems.
The Rust Programming Language For Game Tooling
Case study: Treyarch Image Packer
• Rust version deployed in 2018.
• Heavily multithreaded.
• Active development throughout its lifetime.
• Total “crash” issues encountered: 2.
The Rust Programming Language For Game Tooling
Dan’s Rust Sales Pitch
• Efficiency of writing code: closer to Python.
• Efficiency of running code: closer to C++.
• Large, centralized ecosystem of “crates”, or community libraries (https://crates.io/).
• Integrated build + package + test tool (“cargo”).
• Static, compile-time validation of common memory problems.
• Static, compile-time validation of common multithreading problems.
The Rust Programming Language For Game Tooling
Rust for Game Tools
• Error Handling
• Multithreading
• Parsing Text
• Command Line Interfaces
• Parsing Debug Info
• C ABI compatibility
• Web Applications
• GUIs
The Rust Programming Language For Game Tooling
1/8 - Error Handling
• Result – holds the success or failure state of an operation.
• Panic – instant program failure for unrecoverable errors.
• ? Operator – pass a failed Result up the callstack.
let file = std::fs::read(path)?;
• Use the anyhow crate to add context to errors.
The Rust Programming Language For Game Tooling
1/8 - Error Handling
Error: The system cannot find the file specified.
let file = std::fs::read(path)?;
(os error 2)
let file = std::fs::read(path)
.with_context(|| format!("Reading contents of {:?}", path))?;
Error: Reading contents of "test.txt"
Caused by:
The system cannot find the file specified. (os error 2)
The Rust Programming Language For Game Tooling
2/8 - Multithreading
• Using the rayon crate, multithreading is quick, easy, and safe.
file_names file_names
.iter() .par_iter()
.map(|x| hash_file(x)) .map(|x| hash_file(x))
.collect(); .collect();
The Rust Programming Language For Game Tooling
2/8 - Multithreading
• Using the rayon crate, multithreading is quick, easy, and safe.
file_names
.par_iter()
.map(|x| hash_file(x, &mut count))
.collect();
The Rust Programming Language For Game Tooling
3/8 - Parsing Text
• Use the serde crate for generic serialization/deserialization.
#[derive(Deserialize)]
struct Config {
string: String,
number: i32,
list: Vec<String>,
}
let config: Config = serde_json::from_str(&text)?;
let config: Config = serde_yaml::from_str(&text)?;
The Rust Programming Language For Game Tooling
4/8 - Command Line Interfaces
• Use the structopt crate to create command line interfaces.
sourcehash
/// Generate
0.1.0a hash of all source and include files specified
Generate
by one aor
hash
more
of.vcxproj
all source
files.
and include files specified by one or
more
#[derive(StructOpt)]
.vcxproj files
struct SourceHash {
USAGE: /// One or more vcxproj files contributing to the hash.
sourcehash.exe
#[structopt(parse(from_os_str))]
[FLAGS] [files]...
files: Vec<PathBuf>,
FLAGS:
-h,#[structopt(long)]
--help Prints help information
-V,verbose:
--version
bool, Prints version information
} --verbose
ARGS: let options = SourceHash::from_args();
The Rust Programming Language For Game Tooling
<files>... One or more vcxproj files contributing to the hash
5/8 - Parsing Debug Info
gz_header_s - 80 bytes, 12 padding
• Use pdb (win) and gimli (elf) crates to (optimal size should be 72 bytes, 4 padding)
struct gz_header_s
inspect debug info. {
int text; // 4 bytes
<padding> ; // 4 bytes
• This tool is now open-sourced! uLong time; // 8 bytes
int xflags; // 4 bytes
int os; // 4 bytes
• https://github.com/Activision/structpack Bytef* extra; // 8 bytes
uInt extra_len; // 4 bytes
uInt extra_max; // 4 bytes
Bytef* name; // 8 bytes
uInt name_max; // 4 bytes
<padding> ; // 4 bytes
Bytef* comment; // 8 bytes
uInt comm_max; // 4 bytes
int hcrc; // 4 bytes
int done; // 4 bytes
<padding> ; // 4 bytes
};
The Rust Programming Language For Game Tooling
6/8 - C ABI compatibility
• Bind Rust code to other languages (e.g. python, nodejs, C, wasm).
/// Formats the sum of two numbers as string.
#[pyfunction]
fn sum_as_string(a: usize, b: usize) -> PyResult<String> {
Ok((a + b).to_string())
}
/// A Python module implemented in Rust.
#[pymodule]
fn string_sum(py: Python, m: &PyModule) -> PyResult<()> {
m.add_function(wrap_pyfunction!(sum_as_string, m)?)?;
Ok(())
}
Source: https://crates.io/crates/pyo3
The Rust Programming Language For Game Tooling
7/8 - Web Applications
• There are lots and lots of crates for web apps. I like rouille for quick, simple ones.
rouille::start_server("0.0.0.0:80", move |request| {
Response::text("hello world")
});
Source: https://docs.rs/rouille/3.1.1/rouille/
• But tide or actix-web might be better for more substantial apps.
#[async_std::main]
async fn main() -> tide::Result<()> {
let mut app = tide::new();
app.at("/orders/shoes").post(order_shoes);
app.listen("127.0.0.1:8080").await?;
Ok(())
}
Source: https://crates.io/crates/tide
The Rust Programming Language For Game Tooling
8/8 - GUIs
• Native Rust: iced, druid, egui
• C++ Bindings: ritual (Qt), relm (Gtk), imgui-rs, web-view
ui.heading("My egui Application");
ui.horizontal(|ui| {
ui.label("Your name: ");
ui.text_edit_singleline(&mut name);
});
ui.add(egui::Slider::new(&mut age, 0..=120).text("age"));
if ui.button("Click each year").clicked() {
age += 1;
}
ui.label(format!("Hello '{}', age {}", name, age));
Source: https://crates.io/crates/egui
The Rust Programming Language For Game Tooling
Integrating Rust at Treyarch
• Started work in late 2017.
• 3 major tools, around 20 smaller one-off tools.
• Around 120K LOC
• 27 individual contributors.
• Extremely high stability.
The Rust Programming Language For Game Tooling
Rust Downsides
• Steep learning curve… very different from C++.
• Complicated language… lots of corners.
• Relies heavily on ecosystem.
• Compile times as bad as or worse than C++.
Managing these downsides is key to successful integration!
The Rust Programming Language For Game Tooling
Integration Tips
• Communicate!
• Keep Rust siloed off until there is a critical mass of experience.
▪ Find an unmaintained tool and make it better!
• Start with the best workflow.
▪ vscode + rust_analyzer + cargo-edit + rustfmt + clippy
• Bad Rust code has the same safety guarantees as good Rust code!
• Initial hurdles are high, but large productivity gains after they are cleared.
• Provide time and space to both learn and teach.
▪ Presentations, courses, examples, code reviews.
The Rust Programming Language For Game Tooling
Learning Resources
• Dive into code!
▪ https://leetcode.com/
▪ https://adventofcode.com/
• Dive into a book!
▪ “The Rust Programming Language” – Klabnik & Nichols
▪ “Programming Rust” – Blandy & Orendorff
• Dive into online resources!
▪ https://www.rust-lang.org/learn
▪ https://play.rust-lang.org/
The Rust Programming Language For Game Tooling
Thank you!
We’re Hiring!
Gameplay Engineers (mid/senior)
UI Engineers (mid/senior)
Online Engineers (mid/senior)
Senior Test Automation Engineers
And many more!
https://careers.treyarch.com/
