An introduction to

node.js & Riak

Francisco Treacy
May 31st, 2010 @ ams.rb
Amsterdam
 `whoami`

Building a web-based
collaborative e-reading
platform

J2EE veteran

Trumpetless wannabe trumpet player

@frank06 on the interwebs

Let’s look back, shall we?

How did we learn to program?

Does this sound familiar?
printf("What is your name?");
gets(string);
printf("Your name is: %s", string);

...it is how we learn how to program

So then we go build our first webapp...

<?php
$query = "SELECT astronaut FROM spacecraft";
$result = mysql_query($query);

while($result) { ... }
?>

(with the first language we come across)

Let’s imagine a bunch of people
trying to access your website
(run that code) at the same time
 It’s like hosting a party
and provide only one toilet

© hopechurchokc / flickr
 The cost of I/O
L1 cache: 3 cycles

L2 cache: 14 cycles

RAM: 250 cycles

Disk: 41,000,000 cycles

Network: 240,000,000 cycles

The cost of I/O

L1 L2 RAM Disk Network

 The cost of I/O

In other words, reaching RAM is like going from

here to the Red Light District.

Accessing the network is like going to the moon.

So... what is your program still
doing while MySQL goes fetch
Neil Armstrong?

$result = mysql_query($query);
But we have threads!
(to do other stuff while we wait)
Programming with threads
Race conditions

Coarse-grained locks – more blocking

Fine-grained locks – more complexity

Risk for deadlocks

Hard to reason about and therefore to get

right

Context switching overhead

Programming with threads
A look at Apache and Nginx
Memory vs concurrent connections

http://blog.webfaction.com/a-little-holiday-present
A look at Apache and Nginx
Apache uses 1 thread per connection

Nginx doesn’t use threads

it runs an event loop

small memory allocation per connection

1 thread per connection is

limiting for massive concurrency
 Event loops

Especially relevant when dealing with I/O

All code runs in a single thread

No need for multithreading – no locks!

 Event loops

window.onload = function() {
alert("Apollo 11 landed!")
}

Rings a bell?
 Javascript!
Fortunately, Google has been
developing a brand new
Javascript VM for Chrome...

It’s robust, blazingly fast, and open-source:

V8
 The node way

A set of bindings to Google’s V8 Javascript VM

A purely evented, non-blocking infrastructure

that makes it super simple to build highly
concurrent programs

Ability to handle thousands of concurrent

connections with minimal overhead on a
single process
 The node way
Why should I care?

I love Ruby

I can use EventMachine for async

And, after all, my Rails app doesn’t have

youporn’s traffic
 The node way
“Libraries like eventmachine will never be
truly intuitive to use, because event-
driven I/O is enough of a fundamental
shift that it requires deep language
integration. Javascript, it turns out, is a
fundamentally event-driven language
because of its origins in the browser”
Adam Wiggins – Heroku
 The node way

"Threads should be used by experts only"

Javascript is perfect for event loops with first

class function objects and closures

It is arguably the most popular programming

language

Full Javascript stack?

Full Javascript stack
Client-side

$.ajax({
url: '/api/feedme',
success: function(data) {
$('.result').html(data);
}
});
 Full Javascript stack
Client-side – even better
var socket = new io.Socket('localhost');
socket.connect();
socket.send('some data');
socket.addEvent('message', function(data){
$('.result').html(data);
});

WebSockets through Socket.IO-node

 Full Javascript stack
Server-side – node.js

http.createServer(function (request, response) {

response.writeHead(200, {'Content-Type': 'text/
html'});
response.end('I am back!');
}).listen(8000);
Full Javascript stack
Database

?
 Introducing Riak

Content-agnostic key/value store

REST API embraces HTTP

Javascript Map/Reduce

Distributed, “assume that failures will happen”

Linearly scalable
 Introducing Riak
What is scalable anyway?
(other than a buzzword)

€ throughput – cost predictability

Both up and down; less headaches for

operations and development
 Brief Riak overview
db.save('astronauts', 'neil', {
name: "Neil Armstrong",
retired: true,
daysInSpace: 8,
missions: ['Apollo 11', 'Gemini 8']
})()

riak-js – http://github.com/frank06/riak-js
Available for node.js and jQuery
 Brief Riak overview
var map = function(v, keydata, args) {
var result = [],
doc = Riak.mapValuesJson(v)[0]

doc.missions.forEach(function(mission) {
if (mission.match(new RegExp
(args.mission))) result.push(doc)
})
return result;
}

Map/Reduce jobs can be written in Javascript

and submitted via the HTTP interface
 Brief Riak overview
db.mapReduce({
inputs: "astronauts",
query: [{ map:
{ source: map,
arg: { mission: "Apollo" } }
}]
})(function(results) { // ... });

Bring the computation to the data – map

phases are executed in parallel

Aggregation happens in one node

 Brief Riak overview
db.get('astronauts', 'neil', {r: 2})(
function(neil) {
$('.result').html(neil)
})

Tunable N/R/W values to tweak CAP behaviour

Eventual consistency: Brief sacrifices of consistency in

failure conditions

“Choose your own fault tolerance/performance tradeoff”

Brief Riak overview
CAP theorem
Consistency: Reads and writes reflect a
globally consistent system state

Availability: System is available for reads

and writes

Partition tolerance: System can handle

the failure of individual parts
 Brief Riak overview
CAP theorem

No real-world data store can serve

completely consistent data while
being 100% available and handling
disconnected networks
 Internet

NGINX

Rails Rails Rails Rails Node

Collaborative platform for studying

Tens of thousands of books alongside with user

generated content

Highlighting, note-taking, sharing

Web-based, use it anywhere: laptop, phone, iPad

HTML5 + Javascript + node.js + Riak (and Rails!)

Expect a beta release in October

Riak is developer- and ops-friendly: it scales
down to your laptop as easily as up to a cluster – 
especially during exams period!

Allows us to store multimedia assets just as JSON

Lucene-like search coming soon

Node is used for Ajax calls and WebSocket

Rails for the rest (it’s convenient and mature)

There are caveats though
Cutting-edge technologies are not bug-free

Riak still has some rough edges (some in terms of

performance)

node is approaching its first stable version

Async JS code can get “boomerang-shaped”

There are caveats though
Boomerang-shaped code

db.save(bucket, doc, content)(function(response, meta) {

db.get(bucket, doc)(function(response2) {
assert.equal(response2, content);
db.remove(bucket, doc)(function() {
db.get(bucket, doc)(null, function(r, meta3) {
assert.equal(404, meta3.statusCode);
db.get(bucket, other)(function() {
// ...
})
});
});
});
});
 Step or flow-js
Address boomerang-shaped code

Step(
function readSelf() {
fs.readFile(__filename, this);
},
function capitalize(err, text) {
if (err) {
throw err;
}
return text.toUpperCase();
},
function showIt(err, newText) {
sys.puts(newText);
}
);
 CoffeeScript
is a new language inspired by Javascript and Ruby

grade: (student) ->

if student.excellent_work
"A+"
else if student.okay_stuff
if student.tried_hard then "B" else "B-"
else
"C"

eldest: if 24 > 21 then "Liz" else "Ike"

 CoffeeScript
compiles down to Javascript
var eldest, grade;
grade = function(student) {
if (student.excellent_work) {
return "A+";
} else if (student.okay_stuff) {
if (student.tried_hard) {
return "B";
} else {
return "B-";
}
} else {
return "C";
}
};
eldest = 24 > 21 ? "Liz" : "Ike";

And can be run on node!

There’s no doubt about it.

Javascript as a platform
is serious stuff.

Full JS stacks will become

more and more popular.
 Maybe it’s about time
we started teaching this?

puts("Enter your spacecraft:")

gets(function(s) {
puts("You’re flying your " + s)
})
 Go fetch a beer –
see you in a bit for a
demo and questions!
http://nodejs.org

http://howtonode.org

http://nodejs.org/cinco_de_node.pdf

http://www.slideshare.net/marcusf/nonblocking-io-
event-loops-and-nodejs

http://dailyjs.com/2010/05/26/realie

http://wiki.basho.com

http://jashkenas.github.com/coffee-script/

http://widescript.com

http://franciscotreacy.com