smart-pointers-in-rust
smart-pointers-in-rust
SMART
POINTERS
IN RUST
TIM McNAMARA
accelerant.dev
A catalogue record for this book is available from the National Library of
New Zealand. Kei te pātengi raraunga o Te Puna Mātauranga o Aotearoa te
whakarārangi o tēnei pukapuka.
The Accelerated Guide to
Smart Pointers
in Rust
Tim McNamara
3
Contents
4
9. Recap ...................................................................................................................... 70
10. Cheat Sheet ........................................................................................................ 71
11. Afterword .............................................................................................................. 73
12. About Tim McNamara .................................................................................... 74
5
Defining smart pointers
6
You may have also heard the term fat pointer being used as a
synonym for smart pointer, but for the purposes of this document
we’ll consider these two terms to be distinct. They’re often used
interchangeably because a fat pointer includes some metadata
about the referent along side the memory address. That metadata is
typically the length. This provides some extra capabilities over a raw
pointer – specifically it’s possible to deduce what a valid memory
access would be without needing to interpret any bytes. [Sidenote:
This contrasts with text strings in C, which require the application to
check whether the next byte is NULL ( 0x0 ) whenever it accesses
the data.] However, because &T and &mut T are not considered
smart pointers in the Rust ecosystem, and because they contain a
length field when referring to dynamically-sized types (DSTs), we’ll
avoid the use of the term fat pointer.
7
Understanding Rust
8
abstraction because they are a compile-time construct that the
compiler “boils away” during the build process.
As we begin our journey, it’s essential first to grasp the concepts of
ownership, borrowing and lifetimes. If you’ve skipped these
concepts so far, please do take the time to read through the next
few sections because gaining an understanding of what is
happening will be very beneficial to you want to undesrstand how
some of the smart pointer types behave.
9
Ownership
fn main() {
let s1 = String::from("hello");
let s2 = s1;
println!("{s1}");
}
10
fn main() {
let s1 = String::from("hello");
takes_ownership(s1);
println!("{s1}");
}
fn takes_ownership(text: String) {
println!("I have taken ownership of: {text}");
}
11
Borrowing
fn main() {
let s1 = String::from("hello");
let only_ascii_bytes = only_ascii_bytes(&s1);
if only_ascii_bytes {
println!("Thank goodness for Unicode!");
}
println!("{s1}");
}
12
fn main() {
let mut s1 = String::from("hello");
append_world(&mut s1);
println!("{s1}");
}
13
Lifetimes
fn main() {
let s1 = String::from("hello");
let s2 = String::from("world");
let result = find_longest(&s1, &s2);
println!("{result}");
}
match t1.len().cmp(&t2.len()) {
Greater => &t1,
Equal => "",
Less => &t2,
}
}
14
In this example, we have a function, find_longest() that takes
references to two String values and returns a reference to the
longest one. The fragment t1.len().cmp(&t2.len()) compares the
lengths of t1 and t2 , returning a std::cmp::Ordering , which is
then matched against.
If you try to compile this code, you’ll get an error because Rust
cannot determine whether the lifetime of the returned reference,
which is itself a borrow, should be tied to s1 and s2 . To fix this,
can add lifetime annotations to indicate that s1 and s2 have the
same lifetime.
Here’s the modified code with lifetime annotations:
fn main() {
let s1 = String::from("hello");
let s2 = String::from("world");
let result = find_longest(&s1, &s2);
println!("{result}");
}
match t1.len().cmp(&t2.len()) {
Greater => &t1,
Equal => "",
Less => &t2,
}
}
15
Adding lifetime annotations does not change the lifetimes of the
references. They are a way to express the relationships between the
lifetimes of different references, helping the compiler verify that
your code doesn’t create any dangling references.
Let’s look at another example to understand how lifetimes work, this
time using structs:
struct Person<'a> {
name: &'a str,
}
impl<'a> Person<'a> {
fn new(name: &'a str) ‑> Person<'a> {
Person { name }
}
fn greet(&self) {
println!("Hello, my name is {}.", self.name);
}
}
fn main() {
let name = String::from("Alice");
let person = Person::new(&name);
person.greet();
}
16
demonstrates ownership, borrowing, and lifetimes through a Book
and Author types in a library.
struct Author<'a> {
name: &'a str,
}
struct Book<'a> {
title: &'a str,
author: Author<'a>,
publication_year: i32,
}
impl<'a> Author<'a> {
fn new(name: &'a str) ‑> Author<'a> {
Author { name }
}
}
impl<'a> Book<'a> {
fn new(
title: &'a str,
author: Author<'a>,
publication_year: i32
) ‑> Book<'a> {
Book {
title,
author,
publication_year,
}
}
fn display(&self) {
println!(
"{} ({}) by {}",
self.title, self.publication_year,
self.author.name
);
}
}
fn main() {
let author_name = "Maya Angelou";
let author = Author::new(&author_name);
17
let book = Book::new(&book_title, author, 1969);
book.display();
book2.display();
}
[playground]
In our library example, we define two structs, Author and Book ,
both with a lifetime parameter 'a . Re-using lifetime parameter
names, particularly 'a , is common and does not imply that that the
lifetimes are necessarily shared between the contexts using that
name. Think of it being similar to a variable, but for lifetimes.
The new() methods for both the Author and Book structs both
make use of a lifetime parameter to ensure that the instances and
their respective fields have compatible lifetimes.
In the Book struct’s impl block, we also define a display()
method to print out the book’s information. The display() method
borrows the Book instance immutably, which is itself borrowing an
Author immutably, which is also borrowing a String immutably.
The original instances remain accessible after the method call.
One consideration with borrowing is that adding a borrow places a
constraint on an owner. The value’s owner, that is the variable that is
bound to the value, is not able to become invalid to access until
after the lifetimes of all of the references to the value have ended.
18
Defining smart pointers,
again
So, what are smart pointers? Smart pointers are data structures
that act like pointers but have additional features, such as
automatic memory management, enabling shared ownership and
interior mutability. Unlike raw pointers, smart pointers implement
traits that allow them to provide these extra features, making them
safer and more convenient to use.
To explain why they exist, it might be worthwhile to consider writing
Rust without smart pointers. The following example shows how easy
it is to avoid Rust’s ownership system.
fn main() {
let x = 42;
let ptr = &x as *const _;
drop(x);
19
then deleted with drop() , which is shorthand for std::mem::drop()
and made available in local scope via the implicit prelude in all Rust
code. ptr now points to invalid memory. That is, according to
Rust’s lifetime rules, the value assigned to x has been dropped.
However, when we go to dereference ptr later on within the
unsafe block and then print the result, it’s surprising to notice this
doesn’t generate an error at runtime. Instead, 42 is printed to the
terminal. This ability to circumvent Rust’s ownersip system makes
raw pointers very dangerous, which is why the unsafe keyword is
necessary to dereference them.
20
Why use them?
21
Automatic memory management
fn main() {
{
// Heap‑allocate an i32 value
let x = Box::new(42);
} // x goes out of scope when its block ends,
// and the memory is deallocated
}
22
Prevent data races
fn main() {
let counter = Arc::new(Mutex::new(0));
let mut handles = vec![];
for _ in 0..10 {
let counter = Arc::clone(&counter);
let handle = thread::spawn(move || {
let mut num = counter.lock().unwrap();
*num += 1;
});
handles.push(handle);
}
[playground]
23
In this example, we use Arc<T> for shared ownership across
multiple threads and Mutex<T> to ensure exclusive access to the
data, preventing data races.
24
Add super powers to pointers
use std::rc::Rc;
struct Secret(u32);
fn main() {
let data = Rc::new(Secret(1234));
let data_clone = Rc::clone(&data);
25
Simplify code
26
Stdlib’s smart pointers
27
Box<T>
Allocating very large objects on the stack can cause stack overflow.
Box<T> is the mechanism that Rust provides to allocate objects on
the heap. Here’s an example of doing so:
28
fn main() {
// Allocate a large array on the heap
let data = Box::new([0; 1024 * 1024]);
enum Sizes {
S,
M,
L,
XL([0; 1024]),
XXL([0; 1024 * 1024])
}
29
enum Sizes {
S,
M,
L,
XL(Box<[0; 1024]>),
XXL(Box<[0; 1024 * 1024]>)
}
struct List(Option<Box<Item>>);
To see these two types in action, let’s expand the code out into a
fully working example. In the next code example, you’ll see an
append() method implemented, which traverses the list and
updates the last item to point to whatever’s being appended.
#[derive(Debug)]
struct List(Option<Box<Item>>);
30
#[derive(Debug)]
struct Item(i32, Option<Box<Item>>);
impl List {
fn append(&mut self, value: i32) {
let mut current = &mut self.0;
fn main() {
let mut l = List(None);
l.append(1);
l.append(2);
println!("{l:?}")
}
[playground]
By using Box<T> , we can create a recursive data structure that
stores its elements on the heap, allowing us to build linked lists of
arbitrary length.
31
Rc<T>
use std::rc::Rc;
fn main() {
let data = Rc::new("Hello, world!");
let data_clone1 = Rc::clone(&data);
let data_clone2 = Rc::clone(&data);
32
Clone 2: Hello, world!
Reference count: 3
use std::rc::Rc;
fn main() {
let data = Rc::new("Hello, world!");
let data_clone1 = Rc::clone(&data);
let data_clone2 = Rc::clone(&data);
take_ownership(data_clone2);
fn take_ownership(data: Rc<&str>) {
println!("Data in function: {}", data);
}
33
When we run this code, the output shows the reference count before
and after passing data_clone2 to the function:
use std::rc::Rc;
struct Node {
value: i32,
next: Option<Rc<Node>>,
}
fn main() {
let node1 = Rc::new(Node { value: 1, next: None });
let node2 = Rc::new(Node { value: 2, next:
Some(Rc::clone(&node1)) });
let node3 = Rc::new(Node { value: 3, next:
Some(Rc::clone(&node2)) });
34
We create three nodes that reference each other using Rc. The
next field of each node is an Option<Rc<Node>> , which allows for
the possibility of no next node (i.e., None ). This is similar to
implementing a linked list in Section 6.1.3, except that there can
now be much richer networks expressed than what is available with
references and Box<T> .
35
Arc<T>
36
use std::sync::Arc;
use std::thread;
fn main() {
let data = Arc::new("Hello, world!");
let mut handles = vec![];
for _ in 0..3 {
let data_clone = Arc::clone(&data);
let handle = thread::spawn(move || {
println!("Data in thread: {}", data_clone);
});
handles.push(handle);
}
37
simultaneously. To achieve thread-safe mutation, you can use
Arc<T> in combination with other concurrency primitives like
Mutex<T> or RwLock<T> .
38
RefCell<T>
use std::cell::RefCell;
fn main() {
let data = RefCell::new(42);
{
let mut data_ref_mut = data.borrow_mut();
*data_ref_mut += 1;
}
39
In this example, we create a RefCell<T> to manage an integer
value. We then borrow a mutable reference to the data using the
borrow_mut method, modify the data, and release the mutable
reference when it goes out of scope. Finally, we borrow an
immutable reference to the data using the borrow method and
print the value.
40
Runtime borrow checking and the potential
for panics
use std::cell::RefCell;
fn main() {
let data = RefCell::new(42);
41
Mutex<T>
The standard case for a mutex is when you wish to enable multiple
threads to be able to modify and/or read some value. Here’s an
example demonstrating how that works with Rust’s Mutex<T> type:
fn main() {
let counter = Arc::new(Mutex::new(0));
let mut handles = vec![];
for _ in 0..10 {
let counter_clone = Arc::clone(&counter);
let handle = thread::spawn(move || {
let mut num = counter_clone.lock().unwrap();
*num += 1;
});
handles.push(handle);
}
42
handle.join().unwrap();
}
[playground]
In this example, we create a Mutex<T> to protect an integer value,
which is wrapped in an Arc<T> to allow for shared ownership across
multiple threads. We then spawn 10 threads, each incrementing the
counter by 1. By using the lock() method, we ensure that each
thread has exclusive access to the counter when incrementing its
value.
43
the data at a time and preventing data races. However, mutual
exclusion can be heavy-handed. If you have a read-heavy workflow,
you may wish to consider using RwLock<T> .
44
RwLock<T>
45
managing separate read and write locks outweighs the potential
performance benefits. On the other hand, use RwLock<T> when you
have a read-heavy workload and allowing multiple concurrent
readers can lead to improved performance and resource utilization.
It’s important to note that the performance characteristics of
Mutex<T> and RwLock<T> can also be platform-dependent, so it’s a
good idea to benchmark and profile your specific use case to
determine the most suitable choice for your application.
46
Building your own smart
pointers
Now that you’ve taken a look at some the of the types that are
available to you, it would be helpful to understand a little more
about how they’re implemented.
47
Drop
The Drop trait provides a way to run custom code when a value is
about to go out of scope, allowing you to clean up resources
associated with the value. This is particularly useful when managing
resources like file handles, sockets, or heap-allocated memory.
Implementing the Drop trait for a type allows you to define a
drop() method that will be called automatically when an instance
of the type is no longer needed.
48
How it relates to resource management and
smart pointers
struct CustomResource {
name: String,
}
fn main() {
let resource = CustomResource {
name: String::from("Resource 1"),
};
49
} // Both resources are automatically dropped and cleaned
up here.
When a value implementing the Drop trait goes out of scope, Rust
automatically calls its drop() method. This call to drop() is
implicit. It does not appear in the program’s source code unless you
wish to customize the default behavior.
struct Outer {
inner: Inner,
}
struct Inner {
data: String,
}
50
fn drop(&mut self) {
println!("Dropping Outer");
}
}
fn main() {
let outer = Outer {
inner: Inner {
data: String::from("Some data"),
},
};
println!("Outer and Inner created.");
} // Inner is dropped first, followed by Outer.
[playground]
use std::mem;
struct CustomResource {
51
name: String,
}
fn main() {
let resource = CustomResource {
name: String::from("Resource 1"),
};
println!("Custom resource created.");
[playground]
Implementing Drop
52
• Avoid using std::mem::drop() explicitly: In most cases, you
should let Rust automatically call the drop() method when a
value goes out of scope. Only use std::mem::drop when you
absolutely need to release resources early, and be aware of
potential double-free errors and undefined behavior.
• Do not rely on the order of drop calls: Although Rust guarantees a
specific drop order for nested structures, it’s best not to rely on
this behavior, as it can make your code more fragile and harder to
refactor.
53
Deref
In some sense, the Deref trait is the thing that enables smart
pointers to exist so seamlessly within the Rust language.
When you call a method on an objet of type T that implements
Deref<Target =
U> , then your value can call methods from U directly.
[Sidebar: This functionality can actually be abused to create
something that feels a little like sub-typing in Rust. This is generally
known as an anti-pattern that will confuse your users.]
To understand its usefulness, consider how nice it is to be able to
call the methods implemented for the &str type from a String .
Without this “auto-deref” behavior, working with Rust wiould be
much more tedious.
A related characteristic of smart pointers is “deref coercion”, more
formally known as dereference coercion. Dereference coercion
makes it easier to work with smart pointers by allowing them to be
used in the same way as regular references. This reduces the need
for explicit dereferencing and thus removes some of the syntatic
noise in our programs.
54
trait Deref<T> {
type Target: ?Sized;
use std::ops::Deref;
struct Portal<T>(T);
impl<T> Portal<T> {
fn new(value: T) ‑> Portal<T> {
Portal(value)
}
}
fn main() {
let p = Portal::new(5);
assert_eq!(*p, 5);
}
[playground]
In this example, we define a custom smart pointer Portal<T> and
implement the Deref trait, allowing for easy access to the underlying
value using the dereference operator ( * ).
55
Understanding dereference coercion
fn print_value(value: &i32) {
println!("Value: {}", value);
}
Let’s say that we wanted to print a value of that’s wrapped with our
custom Portal<T> smart pointer defined at Section 7.2.1. Rust
allows us to pass it to print_value() directly—which only accepts
an &i32 —without explicitly dereferencing it:
fn main() {
let p = Portal::new(5);
print_value(&Portal);
}
struct Person {
name: String,
}
impl Person {
fn name(&self) ‑> &str {
&self.name
}
}
56
fn main () {
let person = Person { name: String::from("Alice") };
let boxed_person = Box::new(person);
57
DerefMut
To implement the DerefMut trait for a custom pointer type, you need
to define the deref_mut() method, which returns a mutable
reference to Self::Target .
struct Portal<T> {
data: T,
}
58
fn main() {
let mut p = Portal::<i32> {data: 42} ;
*p = 0;
println!("{}", 100 + *p );
}
[playground]
In this example, we implement both the Deref and DerefMut traits
for a Portal type. Within the main() function, the DerefMut type is
exercised with the expression *p = 0 , which sets the internal data
field of p to 0 .
3. How DerefMut interacts with the Deref trait (47:40 - 48:00)
The DerefMut trait builds upon the Deref trait to provide mutable
access to the inner data. When you have a mutable reference to a
type that implements both Deref and DerefMut, Rust will
automatically apply dereference coercion for mutable methods,
providing a consistent interface for working with the inner data.
fn main() {
let mut portal = Portal { data: "Wow.
".to_string() };
59
DerefMut best practices
Here is some general advice for making use of DerefMut, and for
building smart pointers generally.
• Deref and DerefMut enable something to be called a “smart
pointer”: You should consider implementing the Deref and
DerefMut traits for your custom types when you want to provide a
consistent and ergonomic interface for working with the inner
data of your wrapper type. Smart pointers are dapper wrappers, if
you forgive the terrible wording. Implementing these traits allows
for dereference coercion, which simplifies code and makes it
more readable.
• Don’t confuse your users: When implementing both Deref and
DerefMut traits, it’s crucial to ensure consistent behavior between
them. This means that if your Deref implementation provides
access to a specific field of a struct, the DerefMut
implementation should also provide mutable access to the same
data. This ensures that users of your type can rely on a
predictable interface.
• Overuse is abuse: Avoid implementing the Deref and DerefMut
traits for cases where the relationship between the types is not
clear or does not represent a pointer-like behavior. Doing so may
lead to confusing and error-prone code.
• For use with genuine ownership only: Be especially cautious when
implementing the Deref trait for types with interior mutability, e.g.
using RefCell<T> or Mutex<T> . This can lead to subtle bugs or
race conditions if not handled correctly.
• Be conservative: Make sure that your Deref and DerefMut
implementations do not introduce any side effects or unexpected
behavior, as they will be implicitly called by the compiler through
dereference coercion. That means that your types users will find it
difficult to track down a specific call site where an error was
introduced.
• Testing helps: Test your Deref and DerefMut implementations
thoroughly to ensure they provide the expected behavior in
60
various situations, especially when dealing with edge cases, such
as empty or invalid data.
61
Extension topics
Many people want to know more than what’s on the surface of what
the standard libary can offer them. This section is for readers who
want to explore more.
62
Cyclic data structures
Rust—well, safe Rust—does not like cycles. It would much prefer you
to represent everything as a hierarchy.
It turns out that Arc<T> or Rc<T> are available to implement a
cyclic data structure, such as a cyclic linked list or a tree with cycles.
This is because Arc<T> and Rc<T> offer shared ownership via their
use of reference counting to keep track of the number of references
to an object. Cyclic references will not cause memory leaks as long
as all references are managed by Arc<T> or Rc<T> .
63
Rc<T> from scratch
use std::cell::Cell;
use std::marker::PhantomData;
use std::ops::Deref;
use std::ptr::NonNull;
struct RcInner<T> {
count: Cell<usize>,
data: T,
}
impl<T> Rc<T> {
pub fn new(data: T) ‑> Self {
let inner = Box::new(RcInner {
count: Cell::new(1),
data,
});
Rc {
inner: unsafe {
NonNull::new_unchecked(Box::into_raw(inner))
},
_marker: PhantomData,
}
}
64
inner.count.get()
}
}
inner.count.set(inner.count.get() + 1);
Rc {
inner: self.inner,
_marker: PhantomData,
}
}
}
if inner.count.get() == 1 {
drop(inner);
let _free = unsafe {
Box::from_raw(self.inner.as_ptr())
};
} else {
inner.count.set(inner.count.get() ‑ 1);
}
}
}
fn main() {
let a = Rc::new(123);
let b = 456;
let a_prime = Rc::clone(&a);
65
println!("a + b = {}", b + *a_prime)
}
[playground]
I won’t go into much detail explaining this code. Unpicking it is half
of the fun! There is one concept that is completely new though, and
that’s PhantomData<T> .
66
PhantomData<T>
struct ExternalData<T> {
ptr: *const u8,
_marker: PhantomData<T>,
}
67
_marker: PhantomData<&'a U>,
}
In this case, the _marker field is used to tell the Rust compiler that
the lifetime of u_ref is tied to the lifetime 'a . Without this marker,
the Rust compiler might not be able to determine the correct
lifetime relationships between the different references.
Let’s revisit the definition of our custom Rc<T> implementation at
Section 8.2.
struct RcInner<T> {
count: Cell<usize>,
data: T,
}
68
compiler would have difficulty inferring the lifetime relationship
between the different references, which could lead to unsafe code
or memory leaks.
By including the _marker field with PhantomData in our Rc<T>
implementation, Rust’s borrow checker can correctly reason about
the lifetime of the reference and ensure that it does not outlive the
lifetime of the data it points to. This helps to ensure memory safety
and prevent bugs caused by incorrect lifetime assumptions.
69
Recap
70
Cheat Sheet
Box<T>
• Allocates memory on the heap
• Points to a single value of type T
• Automatically deallocates memory when it goes out of scope
• Provides ownership and move semantics
Rc<T>
• Points to a value of type T shared among multiple owners
• Keeps track of the number of owners and deallocates memory
when the count reaches 0
• Useful for scenarios where shared ownership is required in
single-threaded environments
Arc<T>
• Similar to Rc<T> , but for thread-safe shared ownership in
multithreaded environments
• Uses atomic reference counting to ensure safe sharing among
multiple threads
RefCell<T>
• Provides interior mutability, allowing mutation of immutable
values
• Uses runtime borrow checking to enforce rules for shared
mutable access
• Useful for scenarios where immutable values need to be mutated,
but ownership cannot be transferred
71
Mutex <T >
• Provides concurrency-safe shared mutable access to values of
type T
• Uses locks to enforce exclusive access, preventing race
conditions and data races
• Useful for scenarios where shared mutable access is required in
multithreaded environments
Drop trait
• Provides a method for custom cleanup when a value goes out of
scope
• Useful for scenarios where resource management is required
Deref trait
• Allows a type to be dereferenced like a pointer
• Useful for scenarios where a pointer-like interface is required for
a custom type
Interior mutability
• Allows mutation of immutable values through the use of smart
pointers like RefCell<T>
• Useful for scenarios where values need to be mutated but
ownership cannot be transferred
Common pitfalls
• Lifetime issues, including dangling pointers and use-after-free
errors
• Memory leaks and resource management issues
• Concurrency issues, including deadlocks and race conditions
• Use Rust’s static analysis and testing tools to prevent common
pitfalls
72
Afterword
Tim McNamara
May 2023
73
About Tim McNamara
74