C++ Coroutines

a negative overhead abstraction

Gor Nishanov • [email protected]
CppCon 2015 C++ Coroutines 9/25/2015 2
 000100 IDENTIFICATION DIVISION.
000200 PROGRAM-ID. HELLOWORLD.
000300*
000400 ENVIRONMENT DIVISION.
000500 CONFIGURATION SECTION.
000600 SOURCE-COMPUTER. RM-COBOL.
000700 OBJECT-COMPUTER. RM-COBOL.
000800
001000 DATA DIVISION.
001100 FILE SECTION.
001200
100000 PROCEDURE DIVISION.
100100
100200 MAIN-LOGIC SECTION.
100300 BEGIN.
100400 DISPLAY " " LINE 1 POSITION 1 ERASE EOS.
100500 DISPLAY "Hello world!" LINE 15 POSITION 10.
100600 STOP RUN.
100700 MAIN-LOGIC-EXIT.
100800 EXIT.

CppCon 2015 C++ Coroutines 9/25/2015 3

CppCon 2015 C++ Coroutines 9/25/2015 4
CppCon 2015 C++ Coroutines 9/25/2015 5
CppCon 2015 C++ Coroutines 9/25/2015 6
CppCon 2015 C++ Coroutines 9/25/2015 7
 

Melvin Conway

Joel Erdwinn

CppCon 2015 C++ Coroutines

image credits: wikipedia commons, Communication of the ACM vol.6 No.7 July 1963

9/26/2015 8
 

Melvin Conway

Joel Erdwinn

CppCon 2015 C++ Coroutines

image credits: wikipedia commons, Communication of the ACM vol.6 No.7 July 1963

9/25/2015 9
 Subroutine Coroutine

1 S A 1 S A
Basic Symbol Basic Symbol Basic Name
Reducer 2 C Y 2 C Y Reducer Reducer

S A Write to
3 S Y A A
Tape

S Y
output token
S A
S A read token

S A
C Y S Y read token

S A
Subroutine
C Y
Subroutine CY C
Coroutine Subroutine
Subroutine
CppCon 2015 C++ Coroutines 9/25/2015 10
100 cards per minute!

CppCon 2015 C++ Coroutines 9/25/2015 11

9/25/2015 CppCon 2015 C++ Coroutines 12
 Async state machine

Connecting Reading

Failed Completed

9/25/2015 CppCon 2015 C++ Coroutines 13

 Trivial if synchronous
int tcp_reader(int total)
{
char buf[4 * 1024];
auto conn = Tcp::Connect("127.0.0.1", 1337);
for (;;)
{
auto bytesRead = conn.Read(buf, sizeof(buf));
total -= bytesRead;
if (total <= 0 || bytesRead == 0) return total;
}
}

9/25/2015 CppCon 2015 C++ Coroutines 14

 std::future<T> and std::promise<T>
shared_state<T>

atomic<long> refCnt;
mutex lock;
variant<empty, T, exception_ptr> value;
conditional_variable ready;

promise<T>
future<T> intrusive_ptr<shared_state<T>>
intrusive_ptr<shared_state<T>>
set_value(T)
wait() set_exception(exception_ptr)
T get()

9/25/2015 CppCon 2015 C++ Coroutines 15

future<int> tcp_reader(int64_t total) { N4399 Working Draft,
struct State { Technical Specification for C++
Extensions for Concurrency
char buf[4 * 1024];
int64_t total;
Tcp::Connection conn;

.then
explicit State(int64_t total) : total(total) {}
};
auto state = make_shared<State>(total);
return Tcp::Connect("127.0.0.1", 1337).then(
[state](future<Tcp::Connection> conn) {
state->conn = std::move(conn.get());
return do_while([state]()->future<bool> {
if (state->total <= 0) return make_ready_future(false);
return state->conn.read(state->buf, sizeof(state->buf)).then(
[state](future<int> nBytesFut) {
auto nBytes = nBytesFut.get()
if (nBytes == 0) return make_ready_future(false);
state->total -= nBytes;
return make_ready_future(true);
});
}); future<void> do_while(function<future<bool>()> body) {
}); return body().then([=](future<bool> notDone) {
} return notDone.get() ? do_while(body) : make_ready_future(); });
9/25/2015 } CppCon 2015 C++ Coroutines 16
 Forgot something
int tcp_reader(int total)
{
char buf[4 * 1024];
auto conn = Tcp::Connect("127.0.0.1", 1337);
for (;;)
{
auto bytesRead = conn.Read(buf, sizeof(buf));
total -= bytesRead;
if (total <= 0 || bytesRead == 0) return total;
}
}

9/25/2015 CppCon 2015 C++ Coroutines 17

future<int> tcp_reader(int64_t total) {
struct State {
char buf[4 * 1024];
int64_t total;
Tcp::Connection conn;
explicit State(int64_t total) : total(total) {}
};
auto state = make_shared<State>(total);
return Tcp::Connect("127.0.0.1", 1337).then(
[state](future<Tcp::Connection> conn) {
state->conn = std::move(conn.get());
return do_while([state]()->future<bool> {
.then
if (state->total <= 0) return make_ready_future(false);
return state->conn.read(state->buf, sizeof(state->buf)).then(
[state](future<int> nBytesFut) {
auto nBytes = nBytesFut.get()
if (nBytes == 0) return make_ready_future(false);
state->total -= nBytes;
return make_ready_future(true);
}); // read
}); // do_while
}); // Tcp::Connect
}
9/25/2015 CppCon 2015 C++ Coroutines 18
future<int> tcp_reader(int64_t total) {
struct State {
char buf[4 * 1024];
int64_t total;
Tcp::Connection conn;
explicit State(int64_t total) : total(total) {}
};
auto state = make_shared<State>(total);
return Tcp::Connect("127.0.0.1", 1337).then(
[state](future<Tcp::Connection> conn) {
state->conn = std::move(conn.get());
return do_while([state]()->future<bool> {
.then
if (state->total <= 0) return make_ready_future(false);
return state->conn.read(state->buf, sizeof(state->buf)).then(
[state](future<int> nBytesFut) {
auto nBytes = nBytesFut.get()
if (nBytes == 0) return make_ready_future(false);
state->total -= nBytes;
return make_ready_future(true);
}); // read
}); // do_while
}).then([state](future<void>){return make_ready_future(state->total)});
}
9/25/2015 CppCon 2015 C++ Coroutines 19
 Hand-crafted async state machine (1/3)
class tcp_reader ③
① ②
{ Connecting Reading
char buf[64 * 1024];
Tcp::Connection conn; ④ ⑤
promise<int> done;
Failed Completed
int total;

explicit tcp_reader(int total): total(total) {}

② void OnConnect(error_code ec, Tcp::Connection newCon);

③ void OnRead(error_code ec, int bytesRead);
④ void OnError(error_code ec);
⑤ void OnComplete();

public:
① static future<int> start(int total);
};
int main() {
cout << tcp_reader::start(1000 * 1000 * 1000).get(); }
9/25/2015 CppCon 2015 C++ Coroutines 20
 Hand-crafted async state machine (2/3)
future<int> tcp_reader::start(int total) {
auto p = make_unique<tcp_reader>(total);
auto result = p->done.get_future();
Tcp::Connect("127.0.0.1", 1337,
[raw = p.get()](auto ec, auto newConn) {
raw->OnConnect(ec, std::move(newConn));
});
p.release();
return result;
}

void tcp_reader::OnConnect(error_code ec,

Tcp::Connection newCon)
{
if (ec) return OnError(ec);
conn = std::move(newCon);
conn.Read(buf, sizeof(buf),
[this](error_code ec, int bytesRead)
{ OnRead(ec, bytesRead); });
}
9/25/2015 CppCon 2015 C++ Coroutines 21
 Hand-crafted async state machine (3/3)
void tcp_reader::OnRead(error_code ec, int bytesRead) {
if (ec) return OnError(ec);
total -= bytesRead;
if (total <= 0 || bytesRead == 0) return OnComplete();
conn.Read(buf, sizeof(buf),
[this](error_code ec, int bytesRead) {
OnRead(ec, bytesRead); });
}

void OnError(error_code ec) {

auto cleanMe = unique_ptr<tcp_reader>(this);
done.set_exception(make_exception_ptr(system_error(ec)));
}

void OnComplete() {
auto cleanMe = unique_ptr<tcp_reader>(this);
done.set_value(total);
}

9/25/2015 CppCon 2015 C++ Coroutines 22

 Async state machine

Connecting Reading

Failed Completed

9/25/2015 CppCon 2015 C++ Coroutines 23

 Trivial
auto tcp_reader(int total) -> int
{
char buf[4 * 1024];
auto conn = Tcp::Connect("127.0.0.1", 1337);
for (;;)
{
auto bytesRead = conn.Read(buf, sizeof(buf));
total -= bytesRead;
if (total <= 0 || bytesRead == 0) return total;
}
}

9/25/2015 CppCon 2015 C++ Coroutines 24

 Trivial
auto tcp_reader(int total) -> future<int>
{
char buf[4 * 1024];
auto conn = await Tcp::Connect("127.0.0.1", 1337);
for (;;)
{
auto bytesRead = await conn.Read(buf, sizeof(buf));
total -= bytesRead;
if (total <= 0 || bytesRead == 0) return total;
}
}

9/25/2015 CppCon 2015 C++ Coroutines 25

 What about perf?
int main() {
printf("Hello, world\n");
}

Coroutines Hand-Crafted Hello

MB/s 495 (1.3x) 380 0

Binary size 25 (0.85x) 30 9

(Kbytes)

Visual C++ 2015 RTM. Measured on Lenovo W540 laptop. Transmitting & Receiving 1GB over loopback IP addr
9/25/2015 CppCon 2015 C++ Coroutines 26
 Coroutines are closer to the metal

Handcrafted
State
Machines

Coroutines
I/O Abstractions
(Callback based) I/O Abstraction
(Awaitable based)

OS / Low Level Libraries

Hardware

9/25/2015 CppCon 2015 C++ Coroutines 28

 How to map high level call to OS API?
conn.Read(buf, sizeof(buf),
[this](error_code ec, int bytesRead)
{ OnRead(ec, bytesRead); });

template <class Cb>

void Read(void* buf, size_t bytes, Cb && cb);

Windows: WSARecv(fd, ..., OVERLAPPED*) Posix aio: aio_read(fd, ..., aiocbp*)

OVERLAPPED aiocbp

Function Function
Object Object

9/25/2015 CppCon 2015 C++ Coroutines 29

struct OverlappedBase : os_async_context { os_async_ctx
virtual void Invoke(std::error_code, int bytes) = 0; OVERLAPPED /aiocbp
virtual ~OverlappedBase() {} Function
Object
static void io_complete_callback(CompletionPacket& p) {
auto me = unique_ptr<OverlappedBase>(static_cast<OverlappedBase*>(p.overlapped));
me->Invoke(p.error, p.byteTransferred);
}
};
After open associate a socket handle with a threadpool and a callback
ThreadPool::AssociateHandle(sock.native_handle(), &OverlappedBase::io_complete_callback);

template <typename Fn> struct CompletionWithCount : OverlappedBase, private Fn

{
CompletionWithCount(Fn fn) : Fn(std::move(fn)) {}

void Invoke(std::error_code ec, int count) override { Fn::operator()(ec, count); }

};
template <typename Fn> unique_ptr<OverlappedBase> make_handler_with_count(Fn && fn) {
return std::make_unique<CompletionWithCount<std::decay_t<Fn>>(std::forward<Fn>(fn));
}
9/25/2015 CppCon 2015 C++ Coroutines 30
 conn.Read(buf, sizeof(buf),
[this](error_code ec, int bytesRead)
{ OnRead(ec, bytesRead); });

template <typename F>

void Read(void* buf, int len, F && cb) {
return Read(buf, len, make_handler_with_count(std::forward<F>(cb)));
}

void Read(void* buf, int len, std::unique_ptr<detail::OverlappedBase> o)

{
auto error = sock.Receive(buf, len, o.get());
if (error) {
if (error.value() != kIoPending) {
o->Invoke(error, 0);
return;
}
}
o.release();
}

9/25/2015 CppCon 2015 C++ Coroutines 31

 await conn.Read(buf, sizeof(buf));

?
9/25/2015 CppCon 2015 C++ Coroutines 32
 Awaitable – Concept of the Future<T>

Present
T
.await_ready()
F<T> → bool

Present
T
.await_suspend(cb)
F<T> x Fn → void

Present

.await_resume()
T F<T> → T

await expr-of-awaitable-type

9/25/2015 CppCon 2015 C++ Coroutines 33

 await <expr>
Expands into an expression equivalent of

{
auto && tmp = <expr>;
if (!await_ready(tmp)) {
await_suspend(tmp, <coroutine-handle>);
suspend
resume
}
return await_resume(tmp);
}

9/25/2015 CppCon 2015 C++ Coroutines 34

 Overlapped Base from before
struct OverlappedBase : os_async_context
{
virtual void Invoke(std::error_code, int bytes) = 0;
virtual ~OverlappedBase() {}

static void io_complete_callback(CompletionPacket& p) {

auto me = static_cast<OverlappedBase*>(p.overlapped);
auto cleanMe = unique_ptr<OverlappedBase>(me);

me->Invoke(p.error, p.byteTransferred);
}
};

9/25/2015 CppCon 2015 C++ Coroutines 35

Overlapped Base for awaitable
struct AwaiterBase : os_async_context
{
coroutine_handle<> resume; sizeof(void*)
std::error_code err; no dtor
int bytes;

static void io_complete_callback(CompletionPacket& p) {

auto me = static_cast<AwaiterBase*>(p.overlapped);
me->err = p.error;
me->bytes = p.byteTransferred;
me->resume(); mov rcx, [rcx]
} jmp [rcx]
};

9/25/2015 CppCon 2015 C++ Coroutines 36

 await conn.Read(buf, sizeof(buf));

?
9/25/2015 CppCon 2015 C++ Coroutines 37
auto Connection::Read(void* buf, int len) {
struct awaiter: AwaiterBase {
Connection* me;
void* buf;
awaiter(Connection* me, void* buf, int len) : me(me), buf(buf) { bytes = len; }

bool await_ready() { return false; }

void await_suspend(coroutine_handle<> h) {
this->resume = h;
auto error = me->sock.Receive(buf, bytes, this);
if (error.value() != kIoPending)
throw system_error(err);
struct AwaiterBase : os_async_context {
} coroutine_handle<> resume;
std::error_code err;
int bytes;
int await_resume() {
if (this->err) throw system_error(err); static void io_complete_callback(CompletionPacket& p){
auto me = static_cast<AwaiterBase*>(p.overlapped);
return bytes; me->err = p.error;
} me->bytes = p.byteTransferred;
me->resume();
}; }
return awaiter{ this, buf, len }; };

}
9/25/2015 CppCon 2015 C++ Coroutines 38
 Trivial
auto tcp_reader(int total) -> future<int>
{
char buf[4 * 1024];
auto conn = await Tcp::Connect("127.0.0.1", 1337);
for (;;)
{
auto bytesRead = await conn.Read(buf, sizeof(buf));
total -= bytesRead;
if (total <= 0 || bytesRead == 0) return total;
}
}

9/25/2015 CppCon 2015 C++ Coroutines 39

 Can we make it better?

50% I/O completes synchronously

50% I/O with I/O pending error

SetFileCompletionNotificationModes(h,
FILE_SKIP_COMPLETION_PORT_ON_SUCCESS);

9/25/2015 CppCon 2015 C++ Coroutines 40

 SetFileCompletionNotificationModes(h,
FILE_SKIP_COMPLETION_PORT_ON_SUCCESS);

Take advantage of synchronous completions

void Read(void* buf, int len, std::unique_ptr<detail::OverlappedBase> o)

{
auto error = sock.Receive(buf, len, o.get());
if (error) {
if (error.value() != kIoPending) {
o->Invoke(error, 0);
return;
}
}
o.release();
}

9/25/2015 CppCon 2015 C++ Coroutines 41

 SetFileCompletionNotificationModes(h,
FILE_SKIP_COMPLETION_PORT_ON_SUCCESS);

Take advantage of synchronous completions

void Read(void* buf, int len, std::unique_ptr<detail::OverlappedBase> o)

{
auto error = sock.Receive(buf, len, o.get());

if (error.value() != kIoPending) {
o->Invoke(error, len);
return;
}

o.release();
}

9/25/2015 CppCon 2015 C++ Coroutines 42

 SetFileCompletionNotificationModes(h,
FILE_SKIP_COMPLETION_PORT_ON_SUCCESS);

Take advantage of synchronous completions

SuperLean.exe!improved::tcp_reader::OnRead(std::error_code ec, int bytesRead) Line 254
SuperLean.exe!improved::detail::CompletionWithSizeT<<lambda_ee38b7a750c7f550b4ee1dd60c2450c1> >::Invoke(std::error_code ec, int count) Line 31
SuperLean.exe!improved::tcp_reader::OnRead(std::error_code ec, int bytesRead) Line 254
SuperLean.exe!improved::detail::CompletionWithSizeT<<lambda_ee38b7a750c7f550b4ee1dd60c2450c1> >::Invoke(std::error_code ec, int count) Line 31
SuperLean.exe!improved::tcp_reader::OnRead(std::error_code ec, int bytesRead) Line 254
SuperLean.exe!improved::detail::CompletionWithSizeT<<lambda_ee38b7a750c7f550b4ee1dd60c2450c1> >::Invoke(std::error_code ec, int count) Line 31
SuperLean.exe!improved::tcp_reader::OnRead(std::error_code ec, int bytesRead) Line 254
void Read(void* buf, int len, std::unique_ptr<detail::OverlappedBase>
SuperLean.exe!improved::detail::CompletionWithSizeT<<lambda_ee38b7a750c7f550b4ee1dd60c2450c1> o)Line
>::Invoke(std::error_code ec, int count) 31
{
SuperLean.exe!improved::tcp_reader::OnRead(std::error_code ec, int bytesRead) Line 254
SuperLean.exe!improved::detail::CompletionWithSizeT<<lambda_ee38b7a750c7f550b4ee1dd60c2450c1>
>::Invoke(std::error_code ec, int count) Line 31
auto error = sock.Receive(buf, len, o.get()); >::Invoke(std::error_code ec, int count) Line
SuperLean.exe!improved::tcp_reader::OnRead(std::error_code ec, int bytesRead) Line 254
SuperLean.exe!improved::detail::CompletionWithSizeT<<lambda_ee38b7a750c7f550b4ee1dd60c2450c1> 31
SuperLean.exe!improved::tcp_reader::OnRead(std::error_code ec, int bytesRead) Line 254
SuperLean.exe!improved::detail::CompletionWithSizeT<<lambda_ee38b7a750c7f550b4ee1dd60c2450c1> >::Invoke(std::error_code ec, int count) Line 31
if (error.value() != kIoPending) {
SuperLean.exe!improved::tcp_reader::OnRead(std::error_code ec, int bytesRead) Line 254
SuperLean.exe!improved::detail::CompletionWithSizeT<<lambda_ee38b7a750c7f550b4ee1dd60c2450c1> >::Invoke(std::error_code ec, int count) Line 31
o->Invoke(error, len);
SuperLean.exe!improved::tcp_reader::OnRead(std::error_code ec, int bytesRead) Line 254
return;
SuperLean.exe!improved::detail::CompletionWithSizeT<<lambda_ee38b7a750c7f550b4ee1dd60c2450c1>
SuperLean.exe!improved::tcp_reader::OnRead(std::error_code ec, int bytesRead) Line 254
>::Invoke(std::error_code ec, int count) Line 31

}
SuperLean.exe!improved::detail::CompletionWithSizeT<<lambda_ee38b7a750c7f550b4ee1dd60c2450c1>
SuperLean.exe!improved::tcp_reader::OnRead(std::error_code ec, int bytesRead) Line 254
>::Invoke(std::error_code ec, int count) Line 31

SuperLean.exe!improved::detail::CompletionWithSizeT<<lambda_ee38b7a750c7f550b4ee1dd60c2450c1>
SuperLean.exe!improved::tcp_reader::OnRead(std::error_code ec, int bytesRead) Line 254
o.release();
SuperLean.exe!improved::detail::CompletionWithSizeT<<lambda_ee38b7a750c7f550b4ee1dd60c2450c1>
Stack
>::Invoke(std::error_code ec, int count) Line 31

>::Invoke(std::error_code ec, int count) Line 31

SuperLean.exe!improved::tcp_reader::OnRead(std::error_code ec, int bytesRead) Line 254
}
SuperLean.exe!improved::detail::CompletionWithSizeT<<lambda_ee38b7a750c7f550b4ee1dd60c2450c1>
SuperLean.exe!improved::tcp_reader::OnRead(std::error_code ec, int bytesRead) Line 254
SuperLean.exe!improved::detail::CompletionWithSizeT<<lambda_ee38b7a750c7f550b4ee1dd60c2450c1>
Overflow
>::Invoke(std::error_code ec, int count) Line 31

>::Invoke(std::error_code ec, int count) Line 31

SuperLean.exe!improved::tcp_reader::OnRead(std::error_code ec, int bytesRead) Line 254
SuperLean.exe!improved::detail::CompletionWithSizeT<<lambda_ee38b7a750c7f550b4ee1dd60c2450c1> >::Invoke(std::error_code ec, int count) Line 31
SuperLean.exe!improved::tcp_reader::OnRead(std::error_code ec, int bytesRead) Line 254
SuperLean.exe!improved::detail::CompletionWithSizeT<<lambda_ee38b7a750c7f550b4ee1dd60c2450c1> >::Invoke(std::error_code ec, int count) Line 31
SuperLean.exe!improved::detail::io_complete_callback(CompletionPacket & p) Line 22
SuperLean.exe!CompletionQueue::ThreadProc(void * lpParameter) Line 112 C++
9/25/2015 CppCon 2015 C++ Coroutines 43
Need to implement it on the use side

void tcp_reader::OnRead(std::error_code ec, int bytesRead) {

if (ec) return OnError(ec);

total -= (int)bytesRead;
if (total <= 0 || bytesRead == 0) return OnComplete();
bytesRead = sizeof(buf);

conn.Read(buf, bytesRead,
[this](std::error_code ec, int bytesRead) {
OnRead(ec, bytesRead); }) ;
}

9/25/2015 CppCon 2015 C++ Coroutines 44

Now handling synchronous completion

void tcp_reader::OnRead(std::error_code ec, int bytesRead) {

do {
if (ec) return OnError(ec);
total -= (int)bytesRead;
if (total <= 0 || bytesRead == 0) return OnComplete();
bytesRead = sizeof(buf);
} while (
conn.Read(buf, bytesRead,
[this](std::error_code ec, int bytesRead) {
OnRead(ec, bytesRead); }));
}

9/25/2015 CppCon 2015 C++ Coroutines 45

Let’s measure the improvement (handwritten)

MB/s Executable size

Handcrafted Coroutine Handcrafted Coroutine

25

Original 380 495 30 25

Synchr Completion. Opt 485 30

9/25/2015 CppCon 2015 C++ Coroutines 46

 SetFileCompletionNotificationModes(h,
auto Connection::Read(void* buf, int len) { FILE_SKIP_COMPLETION_PORT_ON_SUCCESS);
struct awaiter: AwaiterBase {
Connection* me;
void* buf;
awaiter(Connection* me, void* buf, int len) : me(me), buf(buf) { bytes = len; }

bool await_ready() { return false; }

void await_suspend(coroutine_handle<> h) {
this->resume = h;
auto error = me->sock.Receive(buf, bytes, this);
if (error.value() == kIoPending) return;
if (error) throw system_error(err);
struct AwaiterBase : os_async_context {
return; coroutine_handle<> resume;
} std::error_code err;
int bytes;

int await_resume() { static void io_complete_callback(CompletionPacket& p){

auto me = static_cast<AwaiterBase*>(p.overlapped);
if (this->err) throw system_error(err); me->err = p.error;
return bytes; me->bytes = p.byteTransferred;
me->resume();
} }
}; };

return awaiter{ this, buf, len };

} 9/25/2015 CppCon 2015 C++ Coroutines 48
auto Connection::Read(void* buf, int len) {
struct awaiter: AwaiterBase {
Connection* me;
void* buf;
awaiter(Connection* me, void* buf, int len) : me(me), buf(buf) { bytes = len; }

bool await_ready() { return false; }

bool await_suspend(coroutine_handle<> h) {
this->resume = h;
auto error = me->sock.Receive(buf, bytes, this);
if (error.value() == kIoPending) return true;
if (error) throw system_error(err);
struct AwaiterBase : os_async_context {
return false; coroutine_handle<> resume;
} std::error_code err;
int bytes;

int await_resume() { static void io_complete_callback(CompletionPacket& p){

auto me = static_cast<AwaiterBase*>(p.overlapped);
if (this->err) throw system_error(err); me->err = p.error;
return bytes; me->bytes = p.byteTransferred;
me->resume();
} }
}; };

return awaiter{ this, buf, len };

} 9/25/2015 CppCon 2015 C++ Coroutines 49
 await <expr>
Expands into an expression equivalent of

{
auto && tmp = <expr>;
if (!await_ready(tmp)) {
await_suspend(tmp, <coroutine-handle>);
suspend
resume
}
return await_resume(tmp);
}

9/25/2015 CppCon 2015 C++ Coroutines 50

 await <expr>
Expands into an expression equivalent of

{
auto && tmp = <expr>;
if (!await_ready(tmp) &&
await_suspend(tmp, <coroutine-handle>) {
suspend
resume
}
return await_resume(tmp);
}

9/25/2015 CppCon 2015 C++ Coroutines 51

Let’s measure the improvement (coroutine)

MB/s Executable size

Handcrafted Coroutine Handcrafted Coroutine

25

Original 380 495 30 25

Synchr Completion. Opt 485 1028 30 25

9/25/2015 CppCon 2015 C++ Coroutines 52

 Can we make it better?

9/25/2015 CppCon 2015 C++ Coroutines 54

 Getting rid of the allocations
class tcp_reader {
std::unique_ptr<detail::OverlappedBase> wo;
…

tcp_reader(int64_t total) : total(total) {

wo = detail::make_handler_with_count(
[this](auto ec, int nBytes) {OnRead(ec, nBytes); });
…
}

void OnRead(std::error_code ec, int bytesRead) {

if (ec) return OnError(ec);
do {
total -= (int)bytesRead;
if (total <= 0 || bytesRead == 0) return OnComplete();
bytesRead = sizeof(buf);
} while (conn.Read(buf, bytesRead, wo.get()));
}

9/25/2015 CppCon 2015 C++ Coroutines 55

Let’s measure the improvement (handcrafted)

MB/s Executable size

Handcrafted Coroutine Handcrafted Coroutine

25

Original 380 495 30 25

Synchr Completion. Opt 485 1028 30 25
Prealloc handler 690 1028 28 25

9/25/2015 CppCon 2015 C++ Coroutines 56

 Coroutines are popular!
Python: PEP 0492
DART 1.9
Future<int> getPage(t) async { async def abinary(n):
var c = new http.Client(); if n <= 0:
try { return 1
var r = await c.get('http://url/search?q=$t'); l = await abinary(n - 1)
print(r); r = await abinary(n - 1)
return r.length();C#
return l + 1 + r
} finally { async Task<string> WaitAsynchronouslyAsync()
await c.close(); {
} await Task.Delay(10000); HACK (programming language)
} return "Finished";
}
async function gen1(): Awaitable<int> {
C++17
$x = await Batcher::fetch(1);
future<string> WaitAsynchronouslyAsync()
{ $y = await Batcher::fetch(2);
await sleep_for(10ms); return $x + $y;
return "Finished“s; }
}
9/25/2015 CppCon 2015 C++ Coroutines 58
 Generalized Function POF
Coroutine
Designer Monadic*
await - suspend

User Task
await

Generator
yield

Async
Compiler Generator
image credits: Три богатыря и змей горыныч does not care await + yield

9/25/2015 CppCon 2015 C++ Coroutines 59

 Design Principles
• Scalable (to billions of concurrent coroutines)
• Efficient (resume and suspend operations comparable in cost
to a function call overhead)
• Seamless interaction with existing facilities with no overhead
• Open ended coroutine machinery allowing library designers to
develop coroutine libraries exposing various high-level
semantics, such as generators, goroutines, tasks and more.
• Usable in environments where exceptions are forbidden or not
available

9/25/2015 CppCon 2015 C++ Coroutines 60

Coroutines – a negative overhead abstraction
• Proposal is working through C++ standardization committee
(C++17?)
• Experimental implementation in VS 2015 RTM
• Clang implementation is in progress
• more details:
• CppCon 2014 presentation on coroutines
http://github.com/cppcon
• http://www.open-
std.org/JTC1/SC22/WG21/docs/papers/2015/n4499.pdf
• Pre-Kona mailing (P0054, P0055, P0056)
9/25/2015 CppCon 2015 C++ Coroutines 61
 Thank you!

Kavya Kotacherry, Daveed Vandevoorde, Richard Smith, Jens Maurer,

Lewis Baker, Kirk Shoop, Hartmut Kaiser, Kenny Kerr, Artur Laksberg, Jim
Radigan, Chandler Carruth, Gabriel Dos Reis, Deon Brewis, Jonathan
Caves, James McNellis, Stephan T. Lavavej, Herb Sutter, Pablo Halpern,
Robert Schumacher, Viktor Tong, Geoffrey Romer, Michael Wong, Niklas
Gustafsson, Nick Maliwacki, Vladimir Petter, Shahms King, Slava
Kuznetsov, Tongari J, Lawrence Crowl, Valentin Isac
and many more who contributed

9/26/2015 CppCon 2015 C++ Coroutines 62

 Questions?

9/25/2015 CppCon 2015 C++ Coroutines 63