16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved.

Lock-free by Example
(one very complicated example)

Tony Van Eerd

CppCon, September 2014
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 1

Guide to Threaded Coding

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 2

Guide to Threaded Coding

Use Locks
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 3

Guide to Threaded Coding

1. Forget what you learned in Kindergarten
(ie stop Sharing)
2. Use Locks
3. Measure
4. Measure
5. Change your Algorithm
6. GOTO 1

∞. Lock-free

Lock-free coding is the last thing you want to do.

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 5

Guide to Threaded Coding

Use Locks
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 6

Guide to Threaded Coding

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 7

Guide to Threaded Coding

MACROS are EVIL
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 9

NOTE:
CAS = compare_exchange
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 10

NOTE:
CAS = compare_exchange

Not my coding style/structure

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 11

NOTE:
CAS = compare_exchange

Not my coding style/structure

Remember to lower the audience’s expectations:

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 12

NOTE:
CAS = compare_exchange

Not my coding style/structure

Remember to lower the audience’s expectations:

I’m no Paul McKenney

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 13

… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 14

… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 15

… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 16

… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 17

… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 18

… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 19

Multi-Producer Multi-Consumer Queue

… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 20

MPMC Queue

… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 21

MPMC Queue

… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 22

SPSC
MPMC Queue SPMC
MPSC
MPMC

… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 23

MPMC Queue

… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 24

Bottleneck

… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 25

… …
… …
… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 26

… 0 …
… X X X X X X X X X X X …
… 0 …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 27

… X X X X …
… X X X X X …
… X X X …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 28

… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 29

… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 30

… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 31

… …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 32

tail

… X X X X X X X X …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 33

class Queue
{
int buffer[some_size];
size_t head;
size_t tail;
};

tail

… X X X X X X X X …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 34

void push(int val)

{
buffer[tail++] = val;
}

tail

… X X X X X X X X V …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 35

void push(int val)

{
buffer[tail++] = val;
}

A
tail

… X X X X X X X X * …
head
B
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 36

void push(int val)

{
buffer[tail++] = val;
}
Possible Outcomes?

A
tail

… X X X X X X X X * …
head
B
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 37

void push(int val)

{
buffer[tail++] = val;
}

tail

… X X X X X X X X A B …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 38

void push(int val)

{
buffer[tail++] = val;
}

tail

… X X X X X X X X B A …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 39

void push(int val)

{
buffer[tail++] = val;
}

tail

… X X X X X X X X B …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 40

void push(int val)

{
buffer[tail++] = val;
}

tail

… X X X X X X X X A …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 41

void push(int val)

{
buffer[tail++] = val;
}

tail

…
head
X X X X X X X X A
? …
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 42

void push(int val)

{
buffer[tail++] = val;
}

tail

… X X X X X X X X A …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 43

void push(int val) class Queue A

{ {
buffer[tail++] = val; int buffer[some_size];
} size_t head;
size_t tail;
};
B
tail

… X X X X X X X X A …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 44

void push(int val) class Queue A

{ {
buffer[tail++] = val; int buffer[some_size];
} size_t head;
size_t tail;
};
B
tail

… X X X X X X X X A …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 45

void push(int val) class Queue A

{ {
buffer[tail++] = val; atomic<int> buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};
B
tail

… X X X X X X X X …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 46

void push(int val) class Queue A

{ {
buffer[tail++] = val; atomic<int> buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};
B
tail

… X X X X X X X X …
head
reserved!
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 47

void push(int val) class Queue

{ {
buffer[tail++] = val; int buffer[SZ]; A
} atomic<size_t> head;
atomic<size_t> tail;
};
B
tail

… X X X X X X X X …
head
reserved!
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 48

void push(int val) class Queue

{ {
buffer[tail++] = val; int buffer[SZ]; A
} atomic<size_t> head;
atomic<size_t> tail;
};
B
tail

… X X X X X X X X A B …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 49

void push(int val) class Queue

{ {
buffer[tail++] = val; int buffer[SZ]; A
} atomic<size_t> head;
atomic<size_t> tail;
};
B
tail

… X X X X X X X X B A …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 50

void push(int val) class Queue

{ {
buffer[tail++] = val; int buffer[SZ]; A
} atomic<size_t> head;
atomic<size_t> tail;
};
B
tail

…
head
X X X X X X X X B A …
?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 51

void push(int val) class Queue

{ {
buffer[tail++] = val; int buffer[SZ]; A
} atomic<size_t> head;
atomic<size_t> tail;
};
B
tail

…
head
X X X X X X X X …
?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 52

void push(int val) class Queue

{ {
buffer[tail++] = val; int buffer[SZ]; A
} atomic<size_t> head;
atomic<size_t> tail;
};
B
tail

…
head
X …
?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 53

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ]; A
} atomic<size_t> head;
atomic<size_t> tail;
};
B
tail

…
head
X …
?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 54

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ]; A
} atomic<size_t> head;
atomic<size_t> tail;
};
B
tail

… X …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 55

A B
void push(int val) class Queue X...
{ {
buffer[tail++] = val; int buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};

tail

… X B …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 56

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};

tail

… X …
head head < tail ?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 57

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};

tail

… X …
head head < tail ?
not atomic!
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 58

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};

if (atomic_less(head, tail))
{
tail

}; …
do_something();
X …
head head < tail ?
not atomic!
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 59

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};

if (atomic_less(head, tail))
{ THEN
tail

}; …
do_something();
X …
head head < tail ?
not atomic!
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 60

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};

if (atomic_less(head, tail))
{ THEN
tail

}; …
do_something();
X …
head head < tail ?
not atomic!
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 61

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ];
} atomic<size_t> head;

THEN
if (atomic_less(head, tail))
{ THEN
tail
};
atomic<size_t> tail;

}; …is a 4-letterX word

do_something();
…
head head < tail ?
not atomic!
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 62

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};

if (atomic_less(head, tail))
{ THEN
tail

}; …
do_something();
X …
head head < tail ?
not atomic!
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 63

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};

tail

… X …
head head < tail ?

read head THEN read tail

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 64

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};

tail

… X …
head head < tail ?

don’t assume STATE

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 65

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};

tail

… X …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 66

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};

tail

… X …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 67

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};

tail

… X …
head

don’t assume STATE

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 68

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};

tail

… X …
head

every STATE is a good STATE

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 69

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};

tail

… X …
head

no “temporary suspension” of invariants

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 70

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ];
} atomic<size_t> head;
atomic<size_t> tail;
};

tail

… X …
head

no “temporary suspension” of invariants

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 71

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ]; A
} atomic<size_t> head;
atomic<size_t> tail;
};
B
tail

… X …
head head < tail ?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 72

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ]; A
} atomic<size_t> head;
atomic<size_t> tail;
};
B
tail

… X …
head head < tail ?

ensure tail is always increasing

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 73

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ]; A
} atomic<size_t> head;
atomic<size_t> tail;
};
B
tail

… X …
head head < tail ?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 74

void push(int val) class Queue X...

{ {
buffer[tail++] = val; int buffer[SZ]; A
} atomic<size_t> head;
atomic<size_t> tail;
};
B
tail

… X B …
head head < tail ?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 75

void push(int val) class Queue {

{ int buffer[SZ];
size_t tmp = tail.e++; atomic<size_t> head;
buffer[tmp] = val; struct { atomic<size_t> s;
if(tmp = tail.s) { atomic<size_t> e;
tail.s = ??? } tail;
} };
}
tail

… X B …
head head < tail.s ?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 76

void push(int val) class Queue {

{ int buffer[SZ];
size_t tmp = tail.e++; atomic<size_t> head;
buffer[tmp] = val; struct { atomic<size_t> s;
if(tmp = tail.s) { atomic<size_t> e;
tail.s = ??? } tail;
} };
}
tail

… X A B …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 77

void push(int val) class Queue {

{ int buffer[SZ];
size_t tmp = tail.e++; atomic<size_t> head;
buffer[tmp] = val; struct { atomic<size_t> s;
if(tmp = tail.s) { atomic<size_t> e;
tail.s = ??? } tail;
} };
}
tail
Compromise…
… X A B …
Queue of int -> head
Queue of int != 0
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 78

void push(int val) { class Queue {

size_t tmp = tail.e++; int buffer[SZ];
buffer[tmp] = val; atomic<size_t> head;
if(tmp = tail.s) { struct { atomic<size_t> s;
do atomic<size_t> e;
tail.s++; } tail;
while (buffer[tail.s]); };
}
tail
}
… X A ? 0 …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 79

void push(int val) { class Queue {

size_t tmp = tail.e++; int buffer[SZ];
buffer[tmp] = val; atomic<size_t> head;
if(tmp = tail.s) { struct { atomic<size_t> s;
do atomic<size_t> e;
tail.s++; } tail;
while (buffer[tail.s]); };
}
tail
}
… X A ? 0 …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 80

void push(int val) { class Queue {

size_t tmp = tail.e++; atomic<int> buffer[SZ];
buffer[tmp] = val; atomic<size_t> head;
if(tmp = tail.s) { struct { atomic<size_t> s;
do atomic<size_t> e;
tail.s++; } tail;
while (buffer[tail.s]); };
}
tail
}
… X A ? 0 …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 81

void push(int val) { class Queue {

size_t tmp = tail.e++; atomic<int> buffer[SZ];
buffer[tmp] = val; atomic<size_t> head;
B if(tmp = tail.s) {
THEN struct { atomic<size_t> s;
do atomic<size_t> e;
A tail.s++; } tail;
while (buffer[tail.s]); };
}
tail
}
… X A B 0 …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 82

void push(int val) { class Queue {

size_t tmp = tail.e++; atomic<int> buffer[SZ];
buffer[tmp] = val; atomic<size_t> head;
B if(tmp = tail.s) {
THEN struct { atomic<size_t> s;
do atomic<size_t> e;
A CAS(tail.s,tmp,tmp+1); } tail;
while (buffer[++tmp]); };
}
tail
}
… X A B 0 …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 83

void push(int val) { class Queue {

size_t tmp = tail.e++; atomic<int> buffer[SZ];
buffer[tmp] = val; atomic<size_t> head;
bool r; struct { atomic<size_t> s;
do atomic<size_t> e;
B r = CAS(tail.s,tmp,tmp+1); } tail;
A
while (r && buffer[++tmp]); };
}
tail

… X A B 0 …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 84

void push(int val) { class Queue {

size_t tmp = tail.e++; atomic<int> buffer[SZ];
buffer[tmp] = val; atomic<size_t> head;
bool r; struct { atomic<size_t> s;
do atomic<size_t> e;
r = CAS(tail.s,tmp,tmp+1); } tail;
while (r && buffer[++tmp]); };
}
tail

…
head
X A B 0 …
?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 85

void push(int val) { class Queue {

size_t tmp = tail.e++; atomic<int> buffer[SZ];
buffer[tmp] = val; atomic<size_t> head;
bool r; struct { atomic<size_t> s;
do atomic<size_t> e;
r = CAS(tail.s,tmp,tmp+1); } tail;
while (r && buffer[++tmp]); };
}
tail

… X 0 B C … 0

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 86

void push(int val) { class Queue {

size_t tmp = tail.e++; atomic<int> buffer[SZ];
buffer[tmp] = val; atomic<size_t> head;
bool r; struct { atomic<size_t> s;
do atomic<size_t> e;
r = CAS(tail.s,tmp,tmp+1); } tail;
while (r && buffer[++tmp]); };
}
tail

… X 0 B C … 0

head

!= lock-free
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 87

void push(int val) { class Queue {

size_t tmp = tail.e++; atomic<int> buffer[SZ];
buffer[tmp] = val; atomic<size_t> head;
bool r; struct { atomic<size_t> s;
do atomic<size_t> e;
r = CAS(tail.s,tmp,tmp+1); } tail;
while (r && buffer[++tmp]); };
}
tail

… X 0 B C … 0

head
An algorithm is lock-free if at all times at least one thread is
guaranteed to be making progress. != lock-free
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 88

void push(int val) { class Queue {

size_t tmp = tail.e++; atomic<int> buffer[SZ];
buffer[tmp] = val; atomic<size_t> head;
bool r; struct { atomic<size_t> s;
do atomic<size_t> e;
r = CAS(tail.s,tmp,tmp+1); } tail;
while (r && buffer[++tmp]); };
}
tail

… X 0 B C … 0

head
If I suspended a certain thread at the worst time, for a long time or forever,
do bad things!=happen?
lock-free
Yes -> not lockfree.
16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 89
16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 90
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 91

void push(int val) { class Queue {

size_t tmp = tail.e++; atomic<int> buffer[SZ];
buffer[tmp] = val; atomic<size_t> head;
bool r; struct { atomic<size_t> s;
do atomic<size_t> e;
r = CAS(tail.s,tmp,tmp+1); } tail;
while (r && buffer[++tmp]); };
}
tail

… X 0 B C … 0

head
An algorithm is lock-free if at all times at least one thread is
guaranteed to be making progress. != lock-free
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 92

tail

… X 0 B C … 0

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 93

tail

… X 0 B C … 0

head

don’t want to wait

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 94

tail

… X 0 B C … 0

head

don’t wait
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 95

tail

… 0 B C … 0

head

don’t wait! (?)

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 96

tail

… 0 B C … 0

head

come back later (when?)

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 97

tail

… 0 0 0 … 0

head

spin?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 98

tail

… 0 A’ A’’ … 0

head

pop order ?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 99

tail

… 0 0 0 … 0

head

pop order ?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 100

tail

… A A’ A’’ … 0

head

pop order ?
16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 101
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 102

…
tail X…
X X X X X X X X 0 ✓
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 103

tail

… X X X X X X X X 0 …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 104

CAS(buffer[tail], 0, val)

A
tail

… X X X X X X X X 0 …
head
B
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 105

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tmp

… X X X X X X X X B …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 106

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tmp

… X X X X X X X X 0 …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 107

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) ); if it fails
THEN try again

tmp

… X X X X X X X X B …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 108

do read tail
tmp = tail.load(); THEN read buffer

while ( ! CAS(buffer[tmp], 0, val) );

tmp

… X X X X X X X X B …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 109

do read tail
tmp = tail.load(); THEN read buffer

while ( ! CAS(buffer[tmp], 0, val) );

tmp

… X X X X X X X X B …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 110

do read tail
tmp = tail.load(); THEN read buffer

while ( ! CAS(buffer[tmp], 0, val) );

tmp

… X X X X X X X X 0 …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 111

do read tail
tmp = tail.load(); THEN read buffer

while ( ! CAS(buffer[tmp], 0, val) );

tmp

… X X X X X X X X 0 …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 112

do read tail
tmp = tail.load(); THEN read buffer

while ( ! CAS(buffer[tmp], 0, val) );

tmp tail

… 0 0 0 0 0 0 0 0 0 0 0 X X X X 0 0 0 …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 113

do read tail
tmp = tail.load(); THEN read buffer

while ( ! CAS(buffer[tmp], 0, val) );

tmp tail

… 0 0 0 0 0 0 0 0 A 0 0 X X X X 0 0 0 …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 114

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tmp tail

… 0 0 0 0 0 0 0 0 0 0 0 X X X X 0 0 0 …
head
trailing zeros ?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 115

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tmp tail

… 0 0 0 0 0 0 0 0 0 0 0 X X X X 0 0 0 …
head
pop() ?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 116

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tmp tail

… 0 0 0 0 0 0 0 0 0 0 0 X X X X 0 0 0 …
head
pop() ?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 117

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tmp tail

… Compromise…
0 0 0 0 0 0 0 0 0 0 0 X X X X 0 0 0 …
head
pop() ?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 118

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tmp tail

… 0 0 0 0 0 0 0 0 0 0 0 X X X X 0 0 0 …
head
pop() ?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 119

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tmp tail

… 1 1 1 1 1 1 1 1 1 1 1 X X X X 0 0 0 …
head
pop() ?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 120

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tmp tail

… 1 1 1 1 1 1 1 1 1 1 1 X X X X 0 0 0 …
head
circular ?
16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 121
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 122

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tail

… X X X X X X X X …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 123

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tail

- - - X X X X X X X X - - - -

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 124

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tail

1 1 1 X X X X X X X X 0 0 0 0

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 125

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tmp tail

… 1 1 1 1 1 1 1 1 1 1 1 X X X X 0 0 0 …
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 126

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tail tmp

X X X 1 1 1 1 1 1 1 1 1 1 1 X

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 127

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tail tmp

1 1 1 X X X X 1 1 1 1 1 1 1 1

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 128

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tail tmp

Compromise…
1 1 1 X X X X 1 1 1 1 1 1 1 1

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 129

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tail tmp

1 1 1 X X X X 1 1 1 1 1 1 1 1

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 130

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tail tmp

2 2 2 X X X X 1 1 1 1 1 1 1 1

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 131

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tail tmp

5 5 5 X X X X 4 4 4 4 4 4 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 132

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tail tmp

5 5 5 X X X X 4 4 4 4 4 4 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 133

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );

tail tmp

5 5 5 X X X X 4 4 4 4 4 4 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 134

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 0, val) );
tmp
tail

5 5 5 X X X X 4 4 4 4 4 4 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 135

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], 4, val) );
tmp
tail

5 5 5 X X X X 4 4 4 4 4 4 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 136

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], gen(tmp), val) );
tmp
tail

5 5 5 X X X X 4 4 4 4 4 4 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 137

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], gen(tmp), val) );
tmp
tail

Compromise…
5 5 5 X X X X 4 4 4 4 4 4 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 138

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], gen(tmp), val) );
tmp
tail

5 5 5 X X X X 4 4 4 4 4 4 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 139

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], gen(tmp), val) );
4tmp
4tail

5 5 5 X X X X 4 4 4 4 4 4 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 140

class index {
size_t value; // gen | idx
size_t generation();
operator size_t(); do
index& operator++(); // % tmp = tail.load();
//etc while ( ! CAS(buffer[tmp], gen(tmp), val) );
};
4tmp
4tail

5 5 5 X X X X 4 4 4 4 4 4 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 141

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], gen(tmp), val) );
snapshot
4tmp
4tail

5 5 5 X X X X 4 4 4 4 4 4 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 142

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], gen(tmp), val) );
4tmp
4tail

5 5 5 X X X X X X X X X X 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 143

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], gen(tmp), val) );
4tmp
4tail

5 5 5 5 5 5 5 5 5 X X X X 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 144

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], gen(tmp), val) );
4tmp
7tail

X X X X X X X 7 7 7 7 7 X X X

head
All states are valid states for all lines of code (*)
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 145

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], gen(tmp), val) );
tmp
tail

5 5 5 X X X X 4 4 4 4 4 4 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 146

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], gen(tmp), val) );
tmp
tail

5 5 5 X X X X A 4 4 4 4 4 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 147

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], gen(tmp), val) );
tail++; //???
tmp
tail

5 5 5 X X X X A 4 4 4 4 4 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 148

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], gen(tmp), val) );
tail++; //???
tmp
tail

5 5 5 5 5 5 5 5 5 X X X X 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 149

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], gen(tmp), val) );
tail++; // yes!
tmp
tail

5 5 5 5 5 5 5 5 5 X X X X 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 150

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], gen(tmp), val) );
tail++; // yes!
tmp tail
? ? ? ? ? ? ?

5 5 5 5 5 5 5 5 5 X X X X 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 151

do
tmp = tail.load();
while ( ! CAS(buffer[tmp], gen(tmp), val) );
tail++; // yes!
tmp tail
?? ?

5 5 5 5 5 5 5 5 5 X X X X 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 152

do
spinlock ? tmp = tail.load();
while ( ! CAS(buffer[tmp], gen(tmp), val) );
tail++; // yes!
tmp tail
?? ?

5 5 5 5 5 5 5 5 5 X X X X 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 153

do {
tmp = tail.load();
while (buffer[tmp] != gen(tmp))
tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
tail++; // yes!
tmp tail
?? ?

5 5 5 5 5 5 5 5 5 X X X X 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 154

do {
tmp = tail.load();
while (buffer[tmp] != gen(tmp))
tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
tail++; // yes!
tmp tail
?? ? ?

Sorry Herb…
5 5 5 5 5 5 5 5 5 X X X X 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 155

do {
tmp = tail.load(memory_order_relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))
tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
tail++; // yes!
tmp tail
?? ?

5 5 5 5 5 5 5 5 5 X X X X 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 156

do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))
tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);
tmp tail
?? ?

5 5 5 5 5 5 5 5 5 X X X X 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 157

do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))
tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);
Is tail up to date “now”?
tmp tail
?? ?

5 5 5 5 5 5 5 5 5 X X X X 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 158

do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))
tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);
Is tail up to date “now”?
tail
?? ?

5 5 5 5 5 5 5 5 5 X X X X 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 159

do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))
tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);
Is tail up to date “now”?
tail tmp
?
5 5 5 5 5 5 5 5 5 X X X X 4 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 160

do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))
tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);
Is tail up to date “now”?
tail tmp
?
5 5 5 5 5 5 5 5 5 X X X X A 4

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 161

do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))
tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);
Is tail up to date “now”?
tail
tmp

5 5 5 5 5 5 5 5 5 X X X X A B

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 162

do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))
tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);
Is tail up to date “now”?
“meh” tail
tmp

5 5 5 5 5 5 5 5 5 X X X X A B

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 163

do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))
tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1, relaxed);
Is tail up to date “now”?
“meh” tail
tmp

5 5 5 5 5 5 5 5 5 X X X X A B

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 164

push(val) do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))

? tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);
tail
tmp

5 5 5 5 5 5 5 5 5 X X X X A B

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 165

push(val) do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))

? tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);
tail
tmp

5 5 5 5 5 5 5 5 5 X X X X A B

head
All states are valid states for all lines of code?
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 166

push(val) do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))

? tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);

X X X X X X 4 X X X X X X X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 167

push(val) do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))

? tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);

X X X X X X X X X X X X X X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 168

push(val) do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))

? tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);

X X X X X X X X X X X X X X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 169

push(val) do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))

? tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);
tail
?? ? ?

X X X X X X X X X X X X X X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 170

push(val) do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))

? tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);
tail
?? ? ?

X X X X X X X X X X X X X X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 171

push(val) do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))

? tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);
tail
?? ? ?

X X X X X X X X X X X X X X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 172

do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))
(worse?) spinlock ?
tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);
tail
?? ? ?

X X X X X X X X X X X X X X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 173

do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))
(worse?) spinlock ?
tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);
tail
?? ? ?

Compromise…
X X X X X X X X X X X X X X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 174

do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))
(worse?) spinlock ?
tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val) );
CAS(tail, oldtail, tmp+1);
tail
?? ? ?

X X X X X X X X X X X X X X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 175

do {
tmp = oldtail = tail.load(relaxed);
while (buffer[tmp].load(relaxed) != gen(tmp))
(worse?) spinlock ?
tmp++;
} while ( ! CAS(buffer[tmp], gen(tmp), val | odd(tmp
CAS(tail, oldtail, tmp+1);
4tail
?? ? ?

X X X X X X X X X X X X X X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 176

do {
tmp = oldtail = tail.load(relaxed);
tmp = find_tail(tmp);
(worse?) spinlock ?
} while ( ! CAS(buffer[tmp], gen(tmp), val | odd(tmp
CAS(tail, oldtail, tmp+1);

4tail
?? ? ?

X X X X X X X X X X X X X X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 177

do {
tmp = oldtail = tail.load(relaxed);
tmp = find_tail(tmp);
(worse?) spinlock ?
} while ( ! CAS(buffer[tmp], gen(tmp), val | odd(tmp)
CAS(tail, oldtail, tmp+1);

tmp

5 5 5 5 X X X X 4 4 4 4 4 4 4

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 178

do {
tmp = oldtail = tail.load(relaxed);
tmp = find_tail(tmp);
(worse?) spinlock ?
} while ( ! CAS(buffer[tmp], gen(tmp), val | odd(tmp)
CAS(tail, oldtail, tmp+1);

tmp

5 5 5 5 5 X X X X 4 4 4 4 4 4

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 179

do {
tmp = oldtail = tail.load(relaxed);
tmp = find_tail(tmp);
(worse?) spinlock ?
} while ( ! CAS(buffer[tmp], gen(tmp), val | odd(tmp)
CAS(tail, oldtail, tmp+1);

tmp

5 5 5 5 5 5 X X X X 4 4 4 4 4

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 180

do {
tmp = oldtail = tail.load(relaxed);
tmp = find_tail(tmp);
(worse?) spinlock ?
} while ( ! CAS(buffer[tmp], gen(tmp), val | odd(tmp)
CAS(tail, oldtail, tmp+1);

tmp

5 5 5 5 5 5 5 X X X X 4 4 4 4

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 181

do {
tmp = oldtail = tail.load(relaxed);
tmp = find_tail(tmp, &oldtail);
(worse?) spinlock ?
} while ( ! CAS(buffer[tmp], gen(tmp), val | odd(tmp)
CAS(tail, oldtail, tmp+1);

unlikely, however…
exponential back-off?
tmp

5 5 5 5 5 5 5 5 X X X X 4 4 4

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 182

do {
tmp = oldtail = tail.load(relaxed);
tmp = find_tail(tmp, &oldtail);
“fullish” if (tmp == FULL) …???;
} while ( ! CAS(buffer[tmp], gen(tmp), val | odd(tmp)
CAS(tail, oldtail, tmp+1);

tmp

X X X X X X X X X X X X 4 X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 183

do {
tmp = oldtail = tail.load(relaxed);
tmp = find_tail(tmp, &oldtail);
“fullish” if (tmp == FULL) wait_for_space();
} while ( ! CAS(buffer[tmp], gen(tmp), val | odd(tmp)
CAS(tail, oldtail, tmp+1);

tmp

X X X X X X X X X X X X 4 X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 184

do {
tmp = oldtail = tail.load(relaxed);
tmp = find_tail(tmp, &oldtail);
“fullish” if (tmp == FULL) { wait_for_space(); continue;}
} while ( ! CAS(buffer[tmp], gen(tmp), val | odd(tmp)
CAS(tail, oldtail, tmp+1);

tmp

X X X X X X X X X X X X 4 X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 185

do {
{ tmp = oldtail = tail.load(relaxed);
unique_lock lock(mutex); tmp = find_tail(tmp, &oldtail);
“fullish” if (tmp == FULL) { wait_for_space(); continue;}
while (still_fullish())
cond_full.wait(lock);
} while ( ! CAS(buffer[tmp], gen(tmp), val | odd(tmp)
} CAS(tail, oldtail, tmp+1);

tmp

X X X X X X X X X X X X 4 X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 186

Lock-free by Example
(one very complicated example)

Tony Van Eerd

CppCon, September 2014
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 187

do {
{ tmp = oldtail = tail.load(relaxed);
unique_lock lock(mutex); tmp = find_tail(tmp, &oldtail);
“fullish” if (tmp == FULL) { wait_for_space(); continue;}
while (still_fullish())
cond_full.wait(lock);
} while ( ! CAS(buffer[tmp], gen(tmp), val | odd(tmp)
} CAS(tail, oldtail, tmp+1);

tmp

X X X X X X X X X X X X 4 X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 188

do {
{ tmp = oldtail = tail.load(relaxed);
unique_lock lock(mutex); tmp = find_tail(tmp, &oldtail);
“fullish” if (tmp == FULL) { wait_for_space(); continue;}
while ( ! …find_tail… )
cond_full.wait(lock);
} while ( ! CAS(buffer[tmp], gen(tmp), val | odd(tmp)
} CAS(tail, oldtail, tmp+1);

tmp

X X X X X X X X X X X X 4 X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 189

do {
{ tmp = oldtail = tail.load(relaxed);
unique_lock lock(mutex); tmp = find_tail(tmp, &oldtail);
“fullish” if(tmp == FULL)wait_for_space(&tmp,&oldtail);
while ( ! …find_tail… )
cond_full.wait(lock);
} while ( ! CAS(buffer[tmp], gen(tmp), val | odd(tmp)
} CAS(tail, oldtail, tmp+1);

tmp

X X X X X X X X X X X X 4 X X

All states are valid states for all lines of code?

 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 190

{
unique_lock lock(mutex);

}
“fullish”
while ( ! …find_tail… )
cond_full.wait(lock);
? who calls notify()?

tmp

X X X X X X X X X X X X 4 X X
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 191

{
unique_lock lock(mutex);
“fullish”
while ( ! …find_tail… )
cond_full.wait(lock);
}

tmp

X X X X X X X X X X X X 4 X X

int pop() { int pop() {

}
...
cond_full.notify();
? }
...
unique_lock lock(mutex);
cond_full.notify();
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 192

{
unique_lock lock(mutex);
“fullish”
while ( ! …find_tail… )
cond_full.wait(lock);
}

X X X X X X X X X X X X X X X

int pop() { int pop() {

}
...
cond_full.notify();
? }
...
unique_lock lock(mutex);
cond_full.notify();
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 193

{
unique_lock lock(mutex);
“fullish”
while ( ! …find_tail… )
cond_full.wait(lock);
}

5 5 5 5 5 5 5 5 5 5 5 5 4 4 4

int pop() { int pop() {

}
...
cond_full.notify();
? }
...
unique_lock lock(mutex);
cond_full.notify();
16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 194
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 195

{
unique_lock lock(mutex);
“fullish”
while ( ! …find_tail… )
cond_full.wait(lock);
}

5 5 5 5 5 5 5 5 5 5 5 5 4 4 4

int pop() { int pop() {

}
...
cond_full.notify();
? }
...
unique_lock lock(mutex);
cond_full.notify();
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 196

waiting = true; I’m

{ waiting!
unique_lock lock(mutex);
“fullish”
while ( ! …find_tail… )
cond_full.wait(lock);
}

5 5 5 5 5 5 5 5 5 5 5 5 4 4 4

int pop() { int pop() {

}
...
cond_full.notify();
? }
...
unique_lock lock(mutex);
cond_full.notify();
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 197

waiting = true; I’m int pop() {

{ waiting! …CAS(buffer[x], val, gen);//4
unique_lock lock(mutex); if (waiting) {
“fullish”
unique_lock lock(mutex);
while ( ! …find_tail… ) cond_full.notify();
cond_full.wait(lock); }
} }
tmp

X X X X X X X X X X X X 4 X X

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 198

waiting = true; I’m int pop() {

{ waiting! …CAS(buffer[x], val, gen);//4
unique_lock lock(mutex); if (waiting) {
“fullish”
unique_lock lock(mutex);
while ( ! …find_tail… ) cond_full.notify();
cond_full.wait(lock); }
} }
waiting = false;
tmp

X X X X X X X X X X X X 4 X X

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 199

waiting++; I’m int pop() {

{ waiting! …CAS(buffer[x], val, gen);//4
unique_lock lock(mutex); if (waiting) {
“fullish”
unique_lock lock(mutex);
while ( ! …find_tail… ) cond_full.notify();
cond_full.wait(lock); }
} }
waiting--;
tmp

X X X X X X X X X X X X 4 X X

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 200

waiting++; I’m int pop() {

{ waiting! …CAS(buffer[x], val, gen);//4
unique_lock lock(mutex); if (waiting) {
“fullish”
unique_lock lock(mutex);
while ( ! …find_tail… ) cond_full.notify();
cond_full.wait(lock); }
} }
waiting--;
tmp

X X X X X X X X X X X X 4 X X
rarely always
head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 201

waiting++; I’m int pop() {

{ waiting! …CAS(head, oldhead, tmp+1);
unique_lock lock(mutex); if (waiting) {
“fullish”
unique_lock lock(mutex);
while ( ! …find_tail… ) cond_full.notify();
cond_full.wait(lock); }
} }
waiting--;
tmp

X X X X X X X X X X X X 4 X X

head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 202

{ int pop() {
unique_lock lock(mutex); …CAS(head, oldhead, tmp+1);
if (waiting++ == 0) if (oldhead.waitbit()) {
“fullish”
head.set_waitbit(); unique_lock lock(mutex);
while ( ! …find_tail… ) cond_full.notify();
cond_full.wait(lock); }
if (--waiting == 0) }
head.clear_waitbit();
} tmp

X X X X X X X X X X X X 4 X X

*head
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 203

{ int pop() {
unique_lock lock(mutex); …CAS(head, oldhead, tmp+1);
if (waiting++ == 0) if (oldhead.waitbit()) {
“fullish”
head.set_waitbit(); unique_lock lock(mutex);
while ( ! …find_tail… ) cond_full.notify();
cond_full.wait(lock); }
if (--waiting == 0) }
head.clear_waitbit();
} tmp

X X X X X X X X X X X X 4 X X

*head
NOTE: waiting is NOT atomic
16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 204
16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 205
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 206

Looking Back
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 207

Looking Back
push()
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 208

Looking Ahead
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 209

X X X X X X X X X X X X 4 X X
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 210

X X X X X X X X X X X X 4 X X
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 211

2X ->

X X X X X X X X X X X X X X X X X X X X X X X X X X X

X X X X X X X X X X X X @ X X
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 212

…
…
ref++ 4X ->

X X X X X X X X X X X X X X X X X X X X X X X X X X X

ref++ 2X ->

X X X X X X X X X X X X X X X X X X X X X X X X X X X

< 32

X X X X X X X X X X X X @ X X
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 213

…
…
+ Structures, not just ints!

ref++ 4X ->

X X X X X X X X X X X X X X X X X X X X X X X X X X X

ref++ 2X ->

X X X X X X X X X X X X X X X X X X X X X X X X X X X

< 32

X X X X X X X X X X X X @ X X
16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 214
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 215

“The Problem with Threads”

http://ptolemy.eecs.berkeley.edu/

http://ptolemy.eecs.berkeley.edu/publications/papers/06/problemwithThreads/

“A part of the Ptolemy Project experiment was to see whether effective software engineering practices
could be developed for an academic research setting. We developed a process that included a code maturity
rating system (with four levels, red, yellow, green, and blue), design reviews, code reviews, nightly builds,
regression tests, and automated code coverage metrics. The portion of the kernel that ensured a
consistent view of the program structure was written in early 2000, design reviewed to yellow, and code
reviewed to green. The reviewers included concurrency experts, not just inexperienced graduate students
(Christopher Hylands (now Brooks), Bart Kienhuis, John Reekie, and myself were all reviewers). We wrote
regression tests that achieved 100 percent code coverage. The nightly build and regression tests ran on a
two processor SMP machine, which exhibited different thread behavior than the development machines, which
all had a single processor. The Ptolemy II system itself began to be widely used, and every use of the
system exercised this code. No problems were observed until the code deadlocked on April 26, 2004,
four years later.”
 16 September 2014 Copyright BlackBerry, Ltd. All Rights Reserved. 216

All states are valid states for all lines of code!