Introduct On: Asdsaddssd Asdasd

1.
Introducton
Thanks for vstng my book ste!
My deal book s short, comprehensve and nterestng.
In order to learn Node, you need to learn how to wrte code - not how to use a 3rd
party lbrary or a set of spells. I wll recommend a few lbrares, but the focus s on
wrtng code yourself.
Whenever I cover an API, I wll do what I can to nclude examples and snppets
that I have found useful myself. There are many parts that I would lke to expand
on, but I have to balance wrtng wth a lot of other projects.
Update: Jan 2012
Frst of all, thank you for everyone who's read the book, commented, followed me
on Gthub/Twtter or lnked to the book ste! You keep me motvated to wrte, so
thank you.
The update ncludes:
Coverage of Comet and Socket.o (Chapter 13)
Coverage of basc OOP patterns n JS (Chapter 6)
Added chapter summares at the top of each chapter
Sngle page verson of the book (e.g. for convertng to PDF or some other format)
I've started workng on a follow up book, whch wll cover more advanced topcs -
such as testng, ES6 and some topcs related to sngle page applcatons.
To make tme for wrtng that book, I'm droppng the planned chapters on MVC.
Ths topc has gotten some pretty good coverage elsewhere onlne; and I'd prefer
to workng on more advanced topcs next.
About me
H, I'm Mkto (mxu).
In short: my blog, my Node.js tlng wndow manager for X11 and my Gthub and
Twtter, Lnkedn.
Please leave comments and correctons usng Dsqus. I'll update the book,
though I'm generally too busy to offer personal assstance.
Ths book s avalable for free, but what I've wrtten remans mne. Ask me f you
A book about usng Node.js
- Mixu's Node book http://book.mixu.net/node/single.html
1 of 91 9.9.2014 03:2
sadasda
asdsaddssd
asdasd
dasd
want to do somethng wth t.
2. What s Node.js?
In ths chapter, I:
descrbe the Node.js event loop and the premse behnd asynchronous
I/O
go through an example of how context swtches are made between V8
and Node
Node - or Node.js, as t s called to dstngush t from other "nodes" - s an event-
drven I/O framework for the V8 JavaScrpt engne. Node.js allows Javascrpt to
be executed on the server sde, and t uses the wcked fast V8 Javascrpt engne
whch was developed by Google for the Chrome browser.
The basc phlosophy of node.js s:
Non-blockng I/O - every I/O call must take a callback, whether t s to retreve
nformaton from dsk, network or another process.
Bult-n support for the most mportant protocols (HTTP, DNS, TLS)
Low-level. Do not remove functonalty present at the POSIX layer. For example,
support half-closed TCP connectons.
Stream everythng; never force the bufferng of data.
Node.js s dfferent from clent-sde Javascrpt n that t removes certan thngs,
lke DOM manpulaton, and adds support for evented I/O, processes, streams,
HTTP, SSL, DNS, strng and buffer processng and C/C++ addons.
Let's skp the borng general buzzword bngo ntroducton and get to the meat of
the matter - how does node run your code?
The Event Loop - understandng how Node executes Javascrpt
code
The event loop s a mechansm whch allows you to specfy what happens when a
partcular event occurs. Ths mght be famlar to you from wrtng clent-sde
Javascrpt, where a button mght have an onClck event. When the button s
clcked, the code assocated wth the onClck event s run. Node smply extends
ths dea to I/O operatons: when you start an operaton lke readng a fle, you can
pass control to back to Node and have your code run when the data has been
read. For example:
// read the file /etc/passwd, and call console.log on the returned data
fs.readFile('/etc/passwd', function(err, data){
console.log(data);
});
You can thnk of the event loop as a smple lst of tasks (code) bound to events.
When an event happens, the code/task assocated wth that event s executed.
2 of 91 9.9.2014 03:2
Remember that all of your code n Node s runnng n a sngle process. There s
no parallel executon of Javascrpt code that you wrte - you can only be runnng a
sngle pece of code at any tme. Consder the followng code, n whch:
We set a functon to be called after 1000 mllseconds usng setTmeout() and then 1.
start a loop that blocks for 4 seconds. 2.
What wll happen?
// set function to be called after 1 second
setTieout(function() {
console.log('Tieout ran at ' ! new "ate().toTie#tring());
}, 1$$$);
// store the start tie
%ar start & new "ate();
console.log(''nter loop at( '!start.toTie#tring());
// run a loop for ) seconds
%ar i & $;
// increent i while (current tie * start tie ! )$$$ s)
while(new "ate().getTie() * start.getTie() ! )$$$) {
i!!;
}
console.log(''+it loop at( '
!new "ate().toTie#tring()
!'. ,an '!i!' iterations.');
Because your code executes n a sngle process, the output looks lke ths:
'nter loop at( -$($)(.$ /0T!$1$$ (''#T)
'+it loop at( -$($)(.) /0T!$1$$ (''#T). ,an 12--314 iterations.
Tieout ran at -$($)(.) /0T!$1$$ (''#T)
Notce how the setTmeout functon s only trggered after four seconds. Ths s
because Node cannot and wll not nterrupt the whle loop. The event loop s only
used to determne what do next when the executon of your code fnshes, whch
n ths case s after four seconds of forced watng. If you would have a
CPU-ntensve task that takes four seconds to complete, then a Node server
would not be able to do respond to other requests durng those four seconds,
snce the event loop s only checked for new tasks once your code fnshes.
Some people have crtczed Node's sngle process model because t s possble
to block the current thread of executon lke shown above.
However, the alternatve - usng threads and coordnatng ther executon -
requres somewhat ntrcate codng to work and s only useful f CPU cycles are
the man bottleneck. In my vew, Node s about takng a smple dea (sngle-
process event loops), and seeng how far one can go wth t. Even wth a sngle
process model, you can move CPU-ntensve work to other background
processes, for example by settng up a queue whch s processed by a pool of
workers, or by load balancng over multple processes. If you are performng
CPU-bound work, then the only real solutons are to ether fgure out a better
algorthm (to use less CPU) or to scale to multple cores and multple machnes
3 of 91 9.9.2014 03:2
(to get more CPU's workng on the problem).
The premse of Node s that I/O s the man bottleneck of many (f not most) tasks.
A sngle I/O operaton can take mllons of CPU cycles, and n tradtonal,
non-event-loop-based frameworks the executon s blocked for that tme. In Node,
I/O operatons such as readng a fle are performed asynchronously. Ths s
smply a fancy way of sayng that you can pass control back to the event loop
when you are performng I/O, lke readng a fle, and specfy the code you want to
run when the data s avalable usng a callback functon. For example:
console.log('setTieout at '!new "ate().toTie#tring());
}, 1$$$);
re5uire('fs').readFile('/etc/passwd', function(err, result) {
console.log(result);
} );
Here, we are readng a fle usng an asynchronous functon, fs.readFle(), whch
takes as arguments the name of the fle and a callback functon. When Node
executes ths code, t starts the I/O operaton n the background. Once the
executon has passed over fs.readFle(), control s returned back to Node, and the
event loop gets to run.
When the I/O operaton s complete, the callback functon s executed, passng
the data from the fle as the second argument. If readng the fle takes longer than
1 second, then the functon we set usng setTmeout wll be run after 1 second -
before the fle readng s completed.
In node.js, you arent supposed to worry about what happens n the backend: just
use callbacks when you are dong I/O; and you are guaranteed that your code s
never nterrupted and that dong I/O wll not block other requests.
Havng asynchronous I/O s good, because I/O s more expensve than most code
and we should be dong somethng better than just watng for I/O. The event loop
s smply a way of coordnatng what code should be run durng I/O, whch
executes whenever your code fnshes executng. More formally, an event loop s
an entty that handles and processes external events and converts them nto
callback nvocatons.
By makng calls to the asynchronous functons n Nodes core lbrares, you
specfy what code should be run once the I/O operaton s complete. You can
thnk of I/O calls as the ponts at whch Node.js can swtch from executng one
request to another. At an I/O call, your code saves the callback and returns
control to the Node runtme envronment. The callback wll be called later when
the data actually s avalable.
Of course, on the backend - nvsble to you as a Node developer - may be thread
polls and separate processes dong work. However, these are not explctly
exposed to your code, so you cant worry about them other than by knowng that
I/O nteractons e.g. wth the database, or wth other processes wll be
4 of 91 9.9.2014 03:2
asynchronous from the perspectve of each request snce the results from those
threads are returned va the event loop to your code. Compared to the
non-evented multthreaded approach (whch s used by servers lke Apache and
most common scrptng languages), there are a lot fewer threads and thread
overhead, snce threads arent needed for each connecton; just when you
absolutely postvely must have somethng else runnng n parallel and even then
the management s handled by Node.js.
Other than I/O calls, Node.js expects that all requests return quckly; e.g.
CPU-ntensve work should be splt off to another process wth whch you can
nteract as wth events, or by usng an abstracton such as WebWorkers (whch
wll be supported n the future). Ths (obvously) means that you cant parallelze
your code wthout another process n the background wth whch you nteract wth
asynchronously. Node provdes the tools to do ths, but more mportantly makes
workng n an evented, asynchronous manner easy.
Example: A look at the event loop n a Node.js HTTP server
Lets look at a very smple Node.js HTTP server (server.js):
%ar http & re5uire('http');
%ar content & '*htl6*bod76*p68ello 9orld*/p6*script t7pe&:te+t/;a%ascript:'
!'6alert(<8i=:);*/script6*/bod76*/htl6';
http.create#er%er(function (re5uest, response) {
response.end(content);
}).listen(3$3$, 'localhost');
console.log('#er%er running at http(//localhost(3$3$/.');
You can run ths code usng the followng command:
node ser%er.;s
In the smple server, we frst requre the http lbrary (a Node core lbrary). Then we
nstruct that server to lsten on port 8080 on your computer (localhost). Fnally, we
wrte a console message usng console.log().
When the code s run, the Node runtme starts up, loads the V8 Javascrpt engne
whch runs the scrpt. The call to http.createServer creates a server, and the
lsten() call nstructs the server to lsten on port 8080. The program at the
console.log() statement has the followng state:
>?3 engine running ser%er.;s@
>Aode.;s runtie@
After the console message s wrtten, control s returned to the runtme. The state
of the program at that tme s stored as the executon context server.js.
>Aode.;s runtie (waiting for client re5uest to run callbacB) @
The runtme check wll check for pendng callbacks, and wll fnd one pendng
callback - namely, the callback functon we gave to http.createServer(). Ths
means that the server program wll not ext mmedately, but wll wat for an event
to occur.
of 91 9.9.2014 03:2
When you navgate your browser to http://localhost:8080/, the Node.js runtme
receves an event whch ndcates that a new clent has connected to the server. It
searches the nternal lst of callbacks to fnd the callback functon we have set
prevously to respond to new clent requests, and executes t usng V8 n the
executon context of server.js.
>?3 engine running the callbacB in the ser%er.;s conte+t@
>Aode.;s runtie@
When the callback s run, t receves two parameters whch represent the clent
request (the frst parameter, request), and the response (the second parameter).
The callback calls response.end(), passng the varable content and nstructng
the response to be closed after sendng that data back. Callng response.end()
causes some core lbrary code to be run whch wrtes the data back to the clent.
Fnally, when the callback fnshes, the control s returned back to the Node.js
runtme:
>Aode.;s runtie (waiting for client re5uest to run callbacB)@
As you can see, whenever Node.js s not executng code, the runtme checks for
events (more accurately t uses platform-natve APIs whch allow t to be
actvated when events occur). Whenever control s passed to the Node.js runtme,
another event can be processed. The event could be from an HTTP clent
connecton, or perhaps from a fle read. Snce there s only one process, there s
no parallel executon of Javascrpt code. Even though you may have several
evented I/O operatons wth dfferent callbacks ongong, only one of them wll
have t's Node/Javascrpt code run at a tme (the rest wll be actvated whenever
they are ready and no other JS code s runnng).
The clent (your web browser) wll receve the data and nterpret t as HTML. The
alert() call n the Javascrpt tag n the returned data wll be run n your web
browser, and the HTML contanng Hello World wll be dsplayed. It s mportant
to realze that just because both the server and the clent are runnng Javascrpt,
there s no specal connecton - each has ts own Javascrpt varables, functons
and context. The data returned from the clent request callback s just data and
there s no automatc sharng or bult-n ablty to call functons on the server
wthout ssung a server request.
However, because both the server and the clent are wrtten n Javascrpt, you
can share code. And even better, snce the server s a persstent program, you
can buld programs that have long-term state - unlke n scrptng languages lke
PHP, where the scrpt s run once and then exts, Node has ts own nternal HTTP
server whch s capable of savng the state of the program and resumng t quckly
when a new request s made.
3. Smple messagng applcaton
In ths chapter, I:
specfy a smple messagng applcaton that uses long pollng
buld a long pollng server usng Node and
! of 91 9.9.2014 03:2
buld a smple messagng clent usng jQuery
Lets jump rght n and do somethng wth Node.js. We wll be mplementng a
smple chat-type applcaton usng long pollng. In our example, we wll use
smple, manual technques to get a server up and runnng quckly. Routng, fle
servng and error handlng are topcs whch we wll expand upon n the later
chapters.
Long pollng s a smple technque for readng data from a server. The clent
browser makes a normal request, but the server delays respondng f t does not
have any new data. Once new nformaton becomes avalable, t s sent to the
clent, the clent does somethng wth the data and then starts a new long pollng
request. Thus the clent always keeps one long pollng request open to the server
and gets new data as soon as t s avalable.
Request-response Long pollng Sockets
The dfference between request-response (smple pollng), long pollng and sockets
To mplement long pollng, we need two thngs:
Some sort of data payload. In our case, ths wll be a chat message. 1.
Some way of knowng whch messages are new to our clent. In our case, we wll use
a smple counter to know whch messages are new.
2.
The clent wll be a smple HTML page whch uses jQuery to perform the long
pollng calls, whle the server wll be a Node.js server.
There are three cases we need to handle:
Case 1: New messages are avalable when the clent polls. The server should check
t's message lst aganst the counter receved from the clent. If the server has
messages that are newer than the counter, the server should return those messages
up to the current state as well as the current count.
1.
Case 2: No new messages are avalable when the clent polls. The server should store
the clent request nto the lst of pendng requests, and not respond untl a new
message arrves.
2.
Case 3: A clent sends a new message. The server should parse the message, and
add t to the message lst and release all pendng requests, sendng the message to
them.
3.
" of 91 9.9.2014 03:2
These are llustrated below:
3.1 Buldng a smple server
Lets start by gettng the server to respond to HTTP requests. We wll requre a
number of Node.js lbrares:
%ar http & re5uire('http'),
url & re5uire('url'),
fs & re5uire('fs');
In addton, we need storage for the messages as well as pendng clents:
%ar essages & >CtestingC@;
%ar clients & >@;
We can create a server usng http.createServer(). Ths functon takes a callback
functon as an argument, and calls t on each request wth two parameters: the
frst parameter s the request, whle the second parameter s the response. Refer
to nodejs.org for more nformaton on the http API. We wll get nto more detal n
the later chapters.
Lets create a smple server whch returns Hello World:
http.create#er%er(function (re5, res) {
res.end(C8ello worldC);
console.log('#er%er running.');
If you run the code above usng node server.js, and make a request by pontng
your browser to http://localhost:8080/, you wll get a page contanng Hello
World.
Ths s not partcularly nterestng, however, we have now created our frst server.
Lets make the server return a fle - whch wll contan our clent code. The man
reason for dong ths s that browsers enforce a same-orgn polcy for securty
# of 91 9.9.2014 03:2
reasons whch makes long pollng complcated unless the clent comes from the
same URL as we wll be usng for the long pollng.
Ths can be done usng the FS API:
fs.readFile('./inde+.htl', function(err, data) {
res.end(data);
});
We wll read the fle usng asynchronous functon fs.readFle. When t completes,
t runs the nner functon, whch calls res.end() wth the content of the fle. Ths
allows us to send back the content of the ndex.html fle n the same drectory as
server.js.
3.2 Wrtng the clent
Now that we have the capablty to serve a fle, lets wrte our clent code. The
clent wll smply be an HTML page whch ncludes jQuery and uses t to perform
the long pollng requests. We wll have a smple page wth a sngle text area,
whch wll contan the messages we have receved from the server:
*htl6
*head6
*script src&Chttp(//code.;5uer7.co/;5uer7D1.2.).in.;sC6*/script6
*script6
// client code here
*/script6
*/head6
*bod76
*te+tarea id&CoutputC st7le&Cwidth( 1$$E; height( 1$$E;C6
*/te+tarea6
*/bod76
*/htl6
jQuery provdes a number of AJAX functons, whch allow us to make HTTP
requests from the browser. We wll use the getJSON() functon, whch makes a
HTTP GET call and parses the resultng data from the JSON format. The frst
argument s the URL to get, and the second parameter s the functon whch
handles the returned response.
// Flient code
%ar counter & $;
%ar poll & function() {
G.getH#IA('/poll/'!counter, function(response) {
counter & response.count;
%ar ele & G('Joutput');
ele.te+t(ele.te+t() ! response.append);
poll();
});
}
poll();
We mantan a global counter, whch starts at zero and s passed to n the URL to
9 of 91 9.9.2014 03:2
the server. The frst request wll be to /poll/0, wth subsequent requests
ncrementng that counter to keep track of whch messages we have already
receved.
Once the message s receved, we update the counter on the clent sde, append
the message text to the textarea wth the ID #output, and fnally ntate a new
long pollng request by callng poll() agan. To start the pollng for the frst tme, we
call poll() at the end of code.
3.3 Implementng long-pollng on the server sde
Now that we have mplemented the clent, lets add the code to mplement long
pollng on the server sde. Instead of respondng to all requests wth the contents
of ndex.html, we need to parse the request URL and determne what we want to
do.
// parse K,L
%ar urlMparts & url.parse(re5.url);
console.log(urlMparts);
if(urlMparts.pathnae && '/') {
// file ser%ing
fs.readFile('./inde+.htl', function(err, data) {
res.end(data);
});
} else if(urlMparts.pathnae.substr($, .) && '/poll') {
// polling code here
}
We are usng the url API to parse the request URL, then we refer to the one of the
parts of the url, the pathname whch corresponds to the part that comes after the
server IP/doman name. Snce the clent polls the /poll locaton, we check
whether the frst fve characters of the pathname match that address before
executng the poll code.
The long pollng code on the server sde s smple.
%ar count & urlMparts.pathnae.replace(/>N$DO@P/, '');
console.log(count);
if(essages.length 6 count) {
res.end(H#IA.stringif7( {
count( essages.length,
append( essages.slice(count).;oin(CQnC)!CQnC
}));
} else {
clients.push(res);
}
We take the URL, and remove all non-numerc characters usng a regular
expresson. Ths gves us the counter value from the clent: /poll/123 becomes
smply 123. Then we check whether the messages array s longer than the
counter value, and f t s, we wll mmedately return by usng Response.end().
10 of 91 9.9.2014 03:2
Because we are sendng data as JSON, we create an object wth the "count" and
"append" propertes and encode t nto a strng usng JSON.strngfy. Ths JSON
message contans the current count on the server sde (whch s the same as
messages.length) and all the messages startng from count (usng the slce
functon) joned together (wth newlnes separatng the messages).
If the count s greater than the current number of messages, then we do not do
anythng. The clent request wll reman pendng, and we wll store the Response
object nto the clents array usng push(). Once ths s done, our server goes back
to watng for a new message to arrve, whle the clent request remans open.
3.4 Implementng message recevng and broadcastng on the
server sde
Fnally, lets mplement the message recevng functonalty on the server sde.
Messages are receved va the HTTP GET requests to the /msg/ path, for
example: /msg/Hello%20World. Ths allows us to skp wrtng more clent code for
makng these requests (easy, but unnecessary).
} else if(urlMparts.pathnae.substr($, .) && '/sg/') {
// essage recei%ing
%ar sg & unescape(urlMparts.pathnae.substr(.));
essages.push(sg);
while(clients.length 6 $) {
%ar client & clients.pop();
client.end(H#IA.stringif7( {
count( essages.length,
append( sg!CQnC
}));
}
res.end();
}
We decode the url-encoded message usng unescape(), then we push the
message to the messages array. After ths, we wll notfy all pendng clents by
contnuously pop()ng the clents array untl t s empty. Each pendng clent
request receves the current message. Fnally, the pendng request s termnated.
3.5 Concluson and further mprovements
Try runnng the code n Node and sendng messages usng your browser:
By navgatng to http://localhost:8080/, you can open the clent
To send messages, smply open http://localhost:8080/msg/Your+message+here,
replacng Your+message+here wth the message you want to send.
If you open several clent wndows, they wll all receve the messages you send.
There are several ways n whch ths smple server could be mproved:
Frst, the messages are not persstent - closng the server emptes out the messages
array. You could add persstence by wrtng the messages to a database when they
arrve, or even more smply by usng setInterval to save the messages to a fle. You
11 of 91 9.9.2014 03:2
wll then need to load the messages from the fle when the server s restarted.
Second, the clent s extremely smple: t does not do anythng wth the messages
themselves. You could mplement an mproved nterface for dsplayng the messages
by wrtng clent-sde Javascrpt that dynamcally adds the new messages to a lst. If
you want to mplement more complcated functonalty, then the message format
should be mproved wth new functonalty, such as the name of the user that sent the
message.
Thrd, the server-sde could be mproved wth addtonal functonalty such as support
for multple channels and user ncknames. These are best mplemented as separate
classes, such as a Channel class and a User class. You wll learn about mplementng
classes usng prototypal nhertance n the chapter on Objects, and we wll cover more
Node.js functonalty n the subsequent chapters. We wll also go further wth ths type
of applcaton n the later secton of the book, where we dscuss Comet applcatons.
For now, ths bref example should gve you a basc understandng of how a long
pollng Node.js HTTP server can be mplemented, and how you can respond to
clent requests. After coverng some more fundamental technques, we wll cover
more advanced ways of structurng your code that help you n wrtng more
complex applcatons.
4. V8 and Javascrpt gotchas
In ths chapter, I:
explan why you need a self varable sometmes along wth the rules
surroundng the this keyword
explan why you mght get strange results from for loops along wth
the bascs of the varable scope n Javascrpt
show a couple of other mnor gotchas that I found confusng
There are bascally two thngs that trp people up n Javascrpt:
The rules surroundng the "ths" keyword and 1.
Varable scope rules 2.
In ths chapter, I'll examne these JS gotchas and a couple of V8-related
surprses. If you're feelng pretty confdent, then feel free to skm or skp ths
chapter.
4.1 Gotcha #1: ths keyword
In object-orented programmng languages, the this keyword s used to refer to
the current nstance of the object. For example, n Java, the value of this always
refers to the current nstance:
public class Founter {
pri%ate int count & $;
public %oid increent(int %alue) {
this.count !& %alue;
}
12 of 91 9.9.2014 03:2
}
In Javascrpt - whch s a prototype-based language - the this keyword s not
fxed to a partcular value. Instead, the value of this s determned by how the
functon s called [1]:
Executon Context Syntax of functon call Value of ths
Global n/a global object (e.g. window)
Functon Method call:
7Ib;ect.foo();
7Ib;ect
Functon Baseless functon call:
foo();
global object (e.g. window)
(undefined n strct mode)
Functon Usng call:
foo.call(conte+t, 7Rrg);
conte+t
Functon Usng apply:
foo.appl7(conte+t, >7Rrgs@);
conte+t
Functon Constructor wth new:
%ar newFoo & new Foo();
the new nstance
(e.g. newFoo)
Evaluaton n/a value of this n parent context
Callng the method of an object
Ths s the most basc example: we have defned an object, and call object.f1():
%ar ob; & {
id( CRn ob;ectC,
f1( function() {
console.log(this);
}
};
ob;.f1();
As you can see, this refers to the current object, as you mght expect.
Callng a standalone functon
Snce every functon has a " this " value, you can access this even n functons
that are not propertes of an object:
function f1() {
console.log(this.to#tring());
console.log(this && window);
}
f1();
In ths case, this refers to the global object, whch s "DomWndow" n the
browser and "global" n Node.
13 of 91 9.9.2014 03:2
Manpulatng ths va Functon.apply and Functon.call
There are a number of bult-n methods that all Functons have (see the Mozlla
Developer Docs for detals). Two of those bult-n propertes of functons allow us
to change the value of " this " when callng a functon:
Functon.apply(thsArg[, argsArray]): Calls the functon, settng the value of this to
thsArg and the arguments of the functon the values of argsArray.
1.
Functon.call(thsArg[, arg1[, arg2[, ...]]]): Calls the functon, settng the value of this to
thsArg, and passng the arguments arg1, arg2 ... to the functon.
2.
Let's see some examples:
function f1() {
console.log(this);
}
%ar ob;1 & { id( CFooC};
f1.call(ob;1);
%ar ob;- & { id( CSarC};
f1.appl7(ob;-);
As you can see, both call() and apply() allow us to specfy what the value of this
should be.
The dfference between the two s how they pass on addonal arguments:
function f1(a, b) {
console.log(this, a, b);
}
f1.call(ob;1, 'R', 'S');
f1.appl7(ob;-, > 'R', 'S' @);
Call() takes the actual arguments of call(), whle apply() takes just two arguments:
thsArg and an array of arguments.
Stll wth me? OK - now let's talk about the problems.
Context changes
As I noted earler, the value of this s not fxed - t s determned by how the
functon s called. In other words, the value of this s determned at the tme the
functon s called, rather than beng fxed to some partcular value.
Ths causes problems (pun ntended) when we want to defer callng a functon.
For example, the followng won't work:
%ar ob; & {
id( C+7TC,
printUd( function() {
14 of 91 9.9.2014 03:2
console.log('The id is '! this.id ! ' '! this.to#tring());
}
};
setTieout(ob;.printUd, 1$$);
Why doesn't ths work? Well, for the same reason ths does not work:
%ar ob; & {
id( C+7TC,
}
};
%ar callbacB & ob;.printUd;
callbacB();
Snce the value of this s determned at call tme - and we are not callng the
functon usng the "object.method" notaton, " this " refers to the global object --
whch s not what we want.
In "setTmeout(obj.prntId, 100);", we are passng the value of obj.prntId, whch s
a functon. When that functon later gets called, t s called as a standalone
functon - not as a method of an object.
To get around ths, we can create a functon whch mantans a reference to obj,
whch makes sure that this s bound correctly:
%ar ob; & {
id( C+7TC,
}
};
setTieout(function() { ob;.printUd() }, 1$$);
%ar callbacB & function() { ob;.printUd() };
callbacB();
A pattern that you wll see used frequently s to store the value of this at the
begnnng of a functon to a varable called self , and then usng self n callback
n place of this :
%ar ob; & {
ites( >CaC, CbC, CcC@,
process( function() {
%ar self & this; // assign this to self
this.ites.for'ach(function(ite) {
// here, use the original %alue of this=
self.print(ite);
});
},
print( function(ite) {
1 of 91 9.9.2014 03:2
console.log('P' ! ite ! 'P');
}
};
ob;.process();
Because self s an ordnary varable, t wll contan the value of this when the
frst functon was called - no matter how or when the callback functon passed to
forEach() gets called. If we had used " this " nstead of "self" n the callback
functon, t would have referred to the wrong object and the call to prnt() would
have faled.
4.2 Gotcha #2: varable scope and varable evaluaton strategy
C and C-lke languages have rather smple varable scope rules. Whenever you
see a new block, lke { ... }, you know that all the varables defned wthn that
block are local to that block.
Javascrpt's scope rules dffer from those of most other languages. Because of
ths, assgnng to varables can have trcky sde effects n Javascrpt. Look at the
followng snppets of code, and determne what they prnt out.
Don't clck "run" untl you've decded on what the output should be!
Example #1: A smple for loop
for(%ar i & $; i * .; i!!) {
console.log(i);
}
Example #2: a setTmeout call nsde a for loop
for(%ar i & $; i * .; i!!) {
console.log(i);
}, 1$$);
}
Example #3: Delayed calls a functon
%ar data & >@;
for (%ar i & $; i * .; i!!) {
data>i@ & function foo() {
console.log(i);
};
}
data>$@(); data>1@(); data>-@(); data>1@(); data>)@();
Example #1 should be pretty smple. It prnts out 0, 1, 2, 3, 4. However, example
1! of 91 9.9.2014 03:2
#2 prnts out 5, 5, 5, 5, 5. Why s ths?
Lookng at examples #1 to #3, you can see a pattern emerge: delayed calls,
whether they are va setTmeout() or a smple array of functons all prnt the
unexpected result 5.
Varable scope rules n Javascrpt
Fundamentally, the only thng that matters s at what tme the functon code s
executed. setTmeout() ensures that the functon s only executed at some later
stage. Smlarly, assgnng functons nto an array explctly lke n example #3
means that the code wthn the functon s only executed after the loop has been
completed.
There are three thngs you need to remember about varable scope n Javascrpt:
Varable scope s based on the nestng of functons. In other words, the poston of the
functon n the source always determnes what varables can be accessed:
nested functons can access ther parents varables:
%ar a & CfooC;
function parent() {
%ar b & CbarC;
function nested() {
console.log(a);
console.log(b);
}
nested();
}
parent();
1.
non-nested functons can only access the topmost, global varables:
%ar a & CfooC;
function parent() {
%ar b & CbarC;
}
function nested() {
console.log(a);
console.log(b);
}
parent();
nested();
2.
1.
Defnng functons creates new scopes:
and the default behavor s to access prevous scope:
%ar a & CfooC;
function grandparent() {
%ar b & CbarC;
function parent() {
1.
2.
1" of 91 9.9.2014 03:2
function nested() {
console.log(a);
console.log(b);
}
nested();
}
parent();
}
grandparent();
but nner functon scopes can prevent access to a prevous scope by
defnng a varable wth the same name:
%ar a & CfooC;
function grandparent() {
%ar b & CbarC;
function parent() {
%ar b & Cb redefined=C;
function nested() {
console.log(a);
console.log(b);
}
nested();
}
parent();
}
grandparent();
2.
Some functons are executed later, rather than mmedately. You can emulate ths
yourself by storng but not executng functons, see example #3.
3.
What we would expect, based on experence n other languages, s that n the for
loop, callng a the functon would result n a call-by-value (snce we are
referencng a prmtve an nteger) and that functon calls would run usng a copy
of that value at the tme when the part of the code was passed over (e.g. when
the surroundng code was executed). Thats not what happens, because we are
usng a closure/nested anonymous functon:
A varable referenced n a nested functon/closure s not a copy of the value of the
varable t s a lve reference to the varable tself and can access t at a much
later stage. So whle the reference to s vald n both examples 2 and 3 they refer
to the value of at the tme of ther executon whch s on the next event loop
whch s after the loop has run whch s why they get the value 5.
Functons can create new scopes but they do not have to. The default behavor
allows us to refer back to the prevous scope (all the way up to the global scope);
ths s why code executng at a later stage can stll access . Because no varable
exsts n the current scope, the from the parent scope s used; because the
parent has already executed, the value of s 5.
1# of 91 9.9.2014 03:2
Hence, we can fx the problem by explctly establshng a new scope every tme
the loop s executed; then referrng back to that new nner scope later. The only
way to do ths s to use an (anonymous) functon plus explctly defnng a varable
n that scope.
We can pass the value of from the prevous scope to the anonymous nested
functon, but then explctly establsh a new varable j n the new scope to hold that
value for future executon of nested functons:
Example #4: Closure wth new scope establshng a new varable
for(%ar i & $; i * .; i!!) {
(function() {
%ar ; & i;
setTieout( function() { console.log(;); }, .$$Pi);
})();
}
Resultng n 0, 1, 2, 3, 4. Let's look at the expresson "(functon() { ... }) ()":
( ... ) - The frst set of round brackets are smply wrappers around an expresson.
( functon() { ... } ) - Wthn that expresson, we create a new anonymous functon.
( functon() { ... } ) () - Then we take the result of that expresson, and call t as a
functon.
We need to have that wrappng anonymous functon, because only functons
establsh new scope. In fact, we are establshng fve new scopes when the loop
s run:
each teraton establshes t's own closure / anonymous functon
that closure / anonymous functon s mmedately executed
the value of s stored n j wthn the scope of that closure / anonymous functon
setTmeout() s called, whch causes "functon() { console.log(j); }" to run at a later
pont n tme
When the setTmeout s trggered, the varable j n console.log(j) refers to the j defned
n closure / anonymous functon
In Javascrpt, all functons store a herarchcal chan of all parent varable objects,
whch are above the current functon context; the chan s saved to the functon at
ts creaton. Because the scope chan s stored at creaton, t s statc and the
relatve nestng of functons precsely determnes varable scope. When scope
resoluton occurs durng code executon, the value for a partcular dentfer such
as s searched from:
frst from the parameters gven to the functon (a.k.a. the actvaton object) 1.
and then from the statcally stored chan of scopes (stored as the functons nternal
property on creaton) from top (e.g. parent) to bottom (e.g. global scope).
2.
Javascrpt wll keep the full set of varables of each of the statcally stored chans
accessble even after ther executon has completed, storng them n what s
19 of 91 9.9.2014 03:2
called a varable object. Snce code that executes later wll receve the value n
the varable object at that later tme, varables referrng to the parent scope of
nested code end up havng unexpected results unless we create a new scope
when the parent s run, copy the value from the parent to a varable n that new
scope and refer to the varable n the new scope.
For a much more detaled explanaton, please read Dmtry Soshnkovs detaled
account of ECMA-262 whch explans these thngs n full detal; n partcular about
Scope chans and Evaluaton strateges.
When you are teratng through the contents of an array, you should use
Array.forEach(), as t passes values as functon arguments, avodng ths problem.
However, n some cases you wll stll need to use the "create an anonymous
functon" technque to explctly establsh new scopes.
4.3 Other mnor gotchas
You should also be aware of the followng gotchas:
Object propertes are not terated n order (V8)
If youve done clent-sde scrptng for Chrome, you mght have run nto the
problems wth teratng through the propertes of objects. Whle other current
Javascrpt engnes enumerate object propertes n nserton order, V8 orders
propertes wth numerc keys n numerc order. For example:
%ar a & {CfooC(CbarC, C-C(C-C, C1C(C1C};
for(%ar i in a) {
console.log(i);
};
Produces the followng output: 1 2 foo where as n Frefox and other browsers t
produces: foo 2 1. Ths means that n V8, you have to use arrays f the order of
tems s mportant to you, snce the order of propertes n an object wll not be
dependent on the order you wrte (or nsert) them n. Ths s techncally correct, as
ECMA-262 does not specfy enumeraton order for objects. To ensure that tems
reman n the order you want them to be n, use an array:
%ar a & >
{ Be7( 'foo', %al( 'bar'},
{ Be7( '-', %al( '-' },
{ Be7( '1', %al( '1' }
@;
for(%ar i in a) {
console.log(a>[email protected])
};
Arrays tems are always ordered consstently n all complant mplementatons,
ncludng V8.
20 of 91 9.9.2014 03:2
Comparng NaN wth anythng (even NaN) s always false
You cannot use the equalty operators (==, ===) to determne whether a value s
NaN or not. Use the bult-n, global sNaN() functon:
console.log(AaA && AaA);
console.log(AaA &&& AaA);
console.log(isAaA(AaA));
The man use case for sNaN() s checkng whether a converson from strng to
nt/float succeeded:
console.log(CUnput is 1-1 D C, =isAaA(parseUnt(C1-1C, 1$)));
console.log(CUnput is abc D C, =isAaA(parseUnt(CabcC, 1$)));
Floatng pont precson
Be aware that numbers n Javascrpt are floatng pont values, and as such, are
not accurate n some cases, such as:
console.log($.1 ! $.-);
console.log($.1 ! $.- && $.1);
Dealng wth numbers wth full precson requres specalzed solutons.
5. Arrays, Objects, Functons and JSON
In ths chapter, I go through a number of useful ECMA5 functons for stuatons
such as:
Searchng the content of an Array
Checkng whether the contents of an Array satsfy a crtera
Iteratng through the propertes (keys) of an object
Acceptng varable number of arguments n functons
Ths chapter focuses on Arrays, Objects and Functons. There are a number of
useful ECMAScrpt 5 features whch are supported by V8, such as
Array.forEach(), Array.ndexOf(), Object.keys() and Strng.trm().
If you haven't heard of those functons, t's because they are part of ECMAScrpt
5, whch s not supported by Internet Explorer versons pror to IE9.
Typcally when wrtng Javascrpt for executon on the clent sde you have to force
yourself to the lowest common denomnator. The ECMAScrpt 5 addtons make
wrtng server sde code ncer. Even IE s fnally addng support for ECMA 5 - n
IE9.
21 of 91 9.9.2014 03:2
Arrays vs. Objects
You have the choce between usng arrays or objects for storng your data n
Javascrpt. Arrays can also be used as stacks:
%ar arr & > 'a', 'b', 'c'@;
arr.push('d'); // insert as last ite
console.log(arr); // >'a', 'b', 'c', 'd'@
console.log(arr.pop()); // reo%e last ite
console.log(arr); // >'a', 'b', 'c'@
Unshft() and shft() work on the front of the array:
%ar arr & > 'a', 'b', 'c'@;
arr.unshift('1'); // insert as first ite
console.log(arr); // >'1','a', 'b', 'c'@
console.log(arr.shift()); // reo%e first ite
console.log(arr); // >'a', 'b', 'c'@
Arrays are ordered - the order n whch you add tems (e.g. push/pop or
shft/unshft) matters. Hence, you should use arrays for storng tems whch are
ordered.
Objects are good for storng named values, but V8 does not allow you to specfy
an order for the propertes (so addng propertes n a partcular order to an object
does not guarantee that the propertes wll be terated n that order). Objects can
also be useful for values whch need to be looked up quckly, snce you can
smply check for whether a property s defned wthout needng to terate through
the propertes:
%ar ob; & { hasMthing( true, id( 1-1 };
if(ob;.hasMthing) {
console.log('true', ob;.id);
}

Workng wth Arrays
Arrays are very versatle for storng data, and can be searched, tested, and have
functons appled to them n V8 usng the followng ECMAScrpt 5 functons:
Searchng the content of an Array
Array.sArray(array) Returns true f a varable s an array, false f t s not.
ndexOf(searchElement[,
fromIndex])
Returns the frst (least) ndex of an element wthn the array
equal to the specfed value, or -1 f none s found. The search
can optonally begn at fromIndex.
22 of 91 9.9.2014 03:2
lastIndexOf(searchElement[,
fromIndex])
Returns the last (greatest) ndex of an element wthn the
array equal to the specfed value, or -1 f none s found.The
array s searched backwards, startng at fromIndex.
The inde+If() and lastUnde+If() functons are very useful for searchng an
array for a partcular value, f necessary. For example, to check whether a
partcular value s present n an array:
function process(arg%) {
if(arg%.inde+If('help')) {
console.log('This is the help te+t.');
}
}
process(>'foo', 'bar', 'help'@);
However, be aware that ndexOf() uses the strct comparson operator (===), so
the followng wll not work:
%ar arr & >C1C, C-C, C1C@;
// #earch the arra7 of Be7s
console.log(arr.inde+If(-)); // returns D1
Ths s because we defned an array of Strngs, not Integers. The strct equalty
operator used by ndexOf takes nto account the type, lke ths:
console.log(- && C-C); // true
console.log(- &&& C-C); // false
%ar arr & >C1C, C-C, C1C@;
// #earch the arra7 of Be7s
console.log(arr.inde+If(-)); // returns D1
console.log(arr.inde+If(C-C)); // returns 1
Notably, you mght run nto ths problem when you use ndexOf() on the return
value of Object.keys().
%ar looBup & { 1-( { foo( 'b'}, 11( { foo( 'a' }, 1)( { foo( 'c' }};
console.log(Ib;ect.Be7s(looBup).inde+If(1-) 6 D1); // false
console.log(Ib;ect.Be7s(looBup).inde+If(''!1-) 6 D1); // true
Applyng functon to every tem n an Array
flter(callback[,
thsObject])
Creates a new array wth all of the elements of ths array for whch the
provded flterng functon returns true. If a thsObject parameter s
provded to flter, t wll be used as the ths for each nvocaton of the
callback. IE9
forEach(callback[,
thsObject])
Calls a functon for each element n the array.
23 of 91 9.9.2014 03:2
map(callback[,
thsObject])
Creates a new array wth the results of callng a provded functon on
every element n ths array.
filter() , ap() and for'ach() all call a callback wth every value of the array.
Ths can be useful for performng varous operatons on the array. Agan, the
callback s nvoked wth three arguments: the value of the element, the ndex of
the element, and the Array object beng traversed. For example, you mght apply
a callback to all tems n the array:
%ar naes & >'a', 'b', 'c'@;
naes.for'ach(function(%alue) {
console.log(%alue);
});
// prints a b c
or you mght flter based on a crteron:
%ar ites & > { id( 1 }, { id( -}, { id( 1}, { id( ) }@;
// onl7 include ites with e%en id's
ites & ites.filter(function(ite){
return (ite.id E - && $);
});
console.log(ites);
// prints > {id( - }, { id( )} @
If you want to accumulate a partcular value - lke the sum of elements n an array
- you can use the reduce() functons:
reduce(callback[,
ntalValue])
Apply a functon smultaneously aganst two values of the array
(from left-to-rght) as to reduce t to a sngle value. IE9
reduceRght(callback[,
ntalValue])
Apply a functon smultaneously aganst two values of the array
(from rght-to-left) as to reduce t to a sngle value. IE9
reduce() and reduce,ight() apply a functon aganst an accumulator and each
value of the array. The callback receves four arguments: the ntal value (or value
from the prevous callback call), the value of the current element, the current
ndex, and the array over whch teraton s occurrng (e.g.
arr.reduce(functon(prevousValue, currentValue, ndex, array){ ... }).
Checkng whether the contents of an Array satsfy a crtera
every(callback[,
thsObject])
Returns true f every element n ths array satsfes the provded
testng functon.
some(callback[,
thsObject])
Returns true f at least one element n ths array satsfes the
provded testng functon.
soe() and e%er7() allow for a condton to be specfed whch s then tested
aganst all the values n the array. The callback s nvoked wth three arguments:
the value of the element, the ndex of the element, and the Array object beng
24 of 91 9.9.2014 03:2
traversed. For example, to check whether a partcular strng contans at least one
of the tokens n an array, use soe() :
%ar t7pes & >'te+t/htl', 'te+t/css', 'te+t/;a%ascript'@;
%ar string & 'te+t/;a%ascript; encoding&utfD3';
if (t7pes.soe(function(%alue) {
return string.inde+If(%alue) 6 D1;
})) {
console.log('The string contains one of the content t7pes.');
}
ECMA 3 Array functons
I'd just lke to remnd you that these exst:
sort([compareFuncton]) Sorts the elements of an array.
concat(value1, value2, ..., valueN) Returns a new array comprsed of ths array joned wth
other array(s) and/or value(s).
jon(separator) Jons all elements of an array nto a strng.
slce(begn[, end] Extracts a secton of an array and returns a new array.
splce(ndex [,howMany][,element1[,
...[, elementN]]]
Adds and/or removes elements from an array.
reverse Reverses the order of the elements of an array -- the frst
becomes the last, and the last becomes the frst.
These functons are part of ECMAScrpt 3, so they are avalable on all modern
browsers.
%ar a & > 'a', 'b', 'c' @;
%ar b & > 1, -, 1 @;
console.log( a.concat(>'d', 'e', 'f'@, b) );
console.log( a.;oin('= ') );
console.log( a.slice(1, 1) );
console.log( a.re%erse() );
console.log( ' DDD ');
%ar c & a.splice($, -);
console.log( a, c );
%ar d & b.splice(1, 1, 'foo', 'bar');
console.log( b, d );
Workng wth Objects
Objects are useful when you need to have named propertes (lke a hash), and
you don't care about the order of the propertes. The most common basc
operatons nclude teratng the propertes and values of an Object, and workng
wth arrays of Objects.
Ib;ect.Be7s(ob;) Returns a lst of the ownPropertes of an object that are enumerable.
2 of 91 9.9.2014 03:2
hasIwnVropert7(prop) Returns a boolean ndcatng whether the object has the specfed
property. Ths method can be used to determne whether an object has
the specfed property as a drect property of that object; unlke the n
operator, ths method does not check down the object's prototype chan.
prop in ob;ectAae The n operator returns true f the specfed property s n the specfed
object. It s useful for checkng for propertes whch have been set to
undefned, as t wll return true for those as well.
You can use ths to count the number of propertes n an object whch you are
usng as a hash table:
// returns arra7 of Be7s
%ar Be7s & Ib;ect.Be7s({ a( 'foo', b( 'bar'});
// Be7s.length is -
console.log(Be7s, Be7s.length);
Iteratng through the propertes (keys) of an object
An easy way to terate through the keys s to use Object.keys() and then apply
Array.forEach() on the array:
%ar group & { 'Rlice'( { a( 'b', b( 'c' }, 'Sob'( { a( 'd' }};
%ar people & Ib;ect.Be7s(group);
people.for'ach(function(person) {
%ar ites & Ib;ect.Be7s(group>person@);
ites.for'ach(function(ite) {
%ar %alue & group>person@>ite@;
console.log(person!'( '!ite!' & '!%alue);
});
});
Iteratng objects n alphabetcal order
Remember that object propertes are not necessarly retreved n order, so f you
want the keys to be n alphabetcal order, use sort():
%ar ob; & { +( '1', a( '-', b( '1'};
%ar ites & Ib;ect.Be7s(ob;);
ites.sort(); // sort the arra7 of Be7s
console.log(ite ! '&' ! ob;>ite@);
});
Sortng arrays of objects by property
The default sort functon compares the tems n the array as strngs, but you can
pass a custom functon to sort() f you want to sort an array of objects by a
property of the objects:
2! of 91 9.9.2014 03:2
%ar arr & >
{ ite( 'W7lophone' },
{ ite( 'Farrot' },
{ ite( 'Rpple'}
@;
arr & arr.sort(function (a, b) {
return a.ite.localeFopare(b.ite);
});
console.log( arr );
The code above uses the comparator parameter of sort() to specfy a custom sort,
and then uses Strng.localCompare to return the correct sort order nformaton.
Checkng whether a property s present, even f t s false
There are multple ways of checkng whether a property s defned:
%ar ob; & { a( C%alueC, b( false };
// none+istent properties
console.log( ==ob;.none+istent );
console.log( 'none+istent' in ob; );
console.log( ob;.hasIwnVropert7('none+istent') );
// e+isting properties
console.log( ==ob;.a );
console.log( 'a' in ob; );
console.log( ob;.hasIwnVropert7('a') );
The expresson !!obj.propertyname takes the value of the property (or undefned)
and converts t to a Boolean by negatng t twce (!true == false, !!true == true).
The n keyword searches for the property n the object, and wll return true even f
the value of the property s zero, false or an empty strng.
// different results when the %alue e%aluates to false
console.log( ==ob;.b );
console.log( 'b' in ob; );
console.log( ob;.hasIwnVropert7('b') );
The hasOwnProperty() method does not check down the object's prototype chan,
whch may be desrable n some cases:
// different results when the propert7 is fro an ob;ect higher up in the protot7pe chain
console.log( ==ob;.to#tring );
console.log( 'to#tring' in ob; );
console.log( ob;.hasIwnVropert7('to#tring') );
2" of 91 9.9.2014 03:2
(Note: All objects have a toStrng method, derved from Object).
Flterng an array of objects
function atch(ite, filter) {
%ar Be7s & Ib;ect.Be7s(filter);
// true if an7 true
return Be7s.soe(function (Be7) {
return ite>Be7@ && filter>Be7@;
});
}
%ar ob;ects & > { a( 'a', b( 'b', c( 'c'},
{ b( '-', c( '1'},
{ d( '1', e( ')'},
{ e( 'f', c( 'c'} @;
ob;ects.for'ach(function(ob;) {
console.log(',esult( ', atch(ob;, { c( 'c', d( '1'}));
});
Substtutng some() wth every() above would change the defnton of match() so
that all key-value pars n the flter object must match.
Workng wth Functons
Defnng new functons:
function do#oething() { return 'do#oething'; }
%ar do#oething'lse & function() { return 'do#oething'lse'; };
console.log( do#oething() );
console.log( do#oething'lse() );
Order of functon defnton wthn a scope does not matter, but when defnng a
functon as a varable the order does matter.
// define the functions after calling the=
%ar do#oething'lse & function() { return 'do#oething'lse'; };
function do#oething() { return 'do#oething'; }
Functons are objects, so they can have propertes attached to them.
function do#oething() { return do#oething.%alue ! .$; }
%ar do#oething'lse & function() { return do#oething'lse.%alue ! 1$$; };
do#oething.%alue & 1$$;
do#oething'lse.%alue & 1$$;
2# of 91 9.9.2014 03:2
Call and apply
The value of the ths keyword s determned by how the functon was called. For
the detals, see the secton on ths scope and call() and apply() n the prevous
chapter.
Functon.call Calls a functon wth a gven ths value and arguments provded ndvdually.
Functon.apply Apples the method of another object n the context of a dfferent object (the
callng object); arguments can be passed as an Array object.
As you can see, both call() and apply() allow us to specfy what the value of ths
should be.
The dfference between the two s how they pass on addonal arguments:
function f1(a, b) {
console.log(this, a, b);
}
f1.call(ob;1, 'R', 'S'); // The %alue of this is changed to ob;1
f1.appl7(ob;-, > 'R', 'S' @); // The %alue of this is changed to ob;-
The syntax of call() s dentcal to that of apply(). The dfference s that call() uses
the actual arguments passed to t (after the frst argument), whle apply() takes
just two arguments: thsArg and an array of arguments.
Varable number of arguments
Functons have a arguments property:
Property:
arguments
The arguments property contans all the parameters passed to the
functon
whch contans all the arguments passed to the functon:
%ar do#oething'lse & function(a, b) {
console.log(a, b);
console.log(arguents);
};
do#oething'lse(1, -, 1, 'foo');
Usng apply() and arguments:
function sallest(){
return 0ath.in.appl7( 0ath, arguents);
}
console.log( sallest(OOO, 3OO, OOOOO) );
29 of 91 9.9.2014 03:2
The arguments varable avalable n functons s not an Array, through t acts
mostly lke an array. For example, t does not have the push() and pop() methods
but t does have a length property:
function test() {
console.log(arguents.length);
console.log(arguents.concat(>'a', 'b', 'c'@)); // causes an error
}
test(1, -, 1);
To create an array from the arguments property, you can use Array.prototype
combned wth Functon.call:
function test() {
// Freate a new arra7 fro the contents of arguents
%ar args & Rrra7.protot7pe.slice.call(arguents); // returns an arra7
console.log(args.length);
console.log(args.concat(>'a', 'b', 'c'@)); // worBs
}
test(1, -, 1);
Workng wth JSON data
The JSON functons are partcularly useful for workng wth data structures n
Javascrpt. They can be used to transform objects and arrays to strngs.
JSON.parse(text[, revver]); Parse a strng as JSON, optonally transform the produced value
and ts propertes, and return the value.
JSON.strngfy(value[,
replacer [, space]]);
Return a JSON strng correspondng to the specfed value,
optonally ncludng only certan propertes or replacng property
values n a user-defned manner.
JSON.parse() can be used to convert JSON data to a Javascrpt Object or Array:
// returns an Ib;ect with two properties
%ar ob; & H#IA.parse('{ChelloC( CworldC, CdataC( > 1, -, 1 @ }');
console.log(ob;.data);
JSON.strngfy() does the opposte:
%ar ob; & { hello( 'world', data( > 1, -, 1 @ };
console.log(H#IA.stringif7(ob;));
The optonal space parameter n JSON.strngfy s partcularly useful n producng
readable output from complex object.
30 of 91 9.9.2014 03:2
The revver and replacer parameters are rarely used. They expect a functon
whch takes the key and value of each value as an argument. That functon s
appled to the JSON nput before returnng t.
6. Objects and classes by example
In ths chapter, I:
cover OOP n Javascrpt by example
pont out a few caveats and recommended solutons
I'm not coverng the theory behnd ths, but I recommend that you start by learnng
more about the prototype chan, because understandng the prototype chan s
essental to workng effectvely wth JS.
The concse explanaton s:
Javascrpt s an object-orented programmng language that supports delegatng
nhertance based on prototypes.
Each object has a prototype property, whch refers to another (regular) object.
Propertes of an object are looked up from two places:
the object tself (Obj.foo), and 1.
f the property does not exst, on the prototype of the object
(Obj.prototype.foo).
2.
Snce ths lookup s performed recursvely (e.g. Obj.foo, Obj.prototype.foo,
Obj.prototype.prototype.foo), each object can be sad to have a prototype chan.
Assgnng to an undefned property of an object wll create that property on the object.
Propertes of the object tself take precedence over propertes of prototypes.
New objects are created usng a constructor, whch s a regular functon nvoked usng
new
The new constructor call (e.g. new Foo()):
creates a new object, 1.
sets the prototype of that object to Foo.prototype and 2.
passes that as this to the constructor. 3.
The delegatng nhertance mplemented n Javascrpt s dfferent from "classcal"
nhertance: t s based on run tme lookups from the prototype property rather than
statcally defned class constructs. The prototype chan lookup mechansm s the
essence of prototypal nhertance.
There are further nuances to the system. Here are my recommendatons on what
to read:
ECMA-262-3 n detal. Chapter 7.1. OOP: The general theory from Dmtry Soshnkov
ECMA-262-3 n detal. Chapter 7.2. OOP: ECMAScrpt mplementaton from Dmtry
Soshnkov
Detals of the Object Model from Mozlla
31 of 91 9.9.2014 03:2
Let's look at some appled patterns next:
Class pattern
// Fonstructor
function Foo(bar) {
// alwa7s initialiTe all instance properties
this.bar & bar;
this.baT & 'baT'; // default %alue
}
// class ethods
Foo.protot7pe.fooSar & function() {
};
// e+port the class
odule.e+ports & Foo;
Instantatng a class s smple:
// constructor call
%ar ob;ect & new Foo('8ello');
Note that I recommend usng function Foo() { ... } for constructors nstead of
%ar Foo & function() { ... } .
The man beneft s that you get better stack traces from Node when you use a
named functon. Generatng a stack trace from an object wth an unnamed
constructor functon:
%ar Foo & function() { };
Foo.protot7pe.bar & function() { console.trace(); };
%ar f & new Foo();
f.bar();
... produces somethng lke ths:
Trace(
at >ob;ect Ib;[email protected] (/hoe//nt/booB/code/$2Moop/constructors.;s(1(11)
at Ib;ect. (/hoe//nt/booB/code/$2Moop/constructors.;s(4(1)
at 0odule.Mcopile (odule.;s()1-(-2)
at Ib;ect..;s (odule.;s().$(1$)
at 0odule.load (odule.;s(1.1(11)
at Function.Mload (odule.;s(11$(1-)
at Rrra7.$ (odule.;s()4$(1$)
at '%ent'itter.MticBFallbacB (node.;s(1O-()$)
... whle usng a named functon
function SaT() { };
SaT.protot7pe.bar & function() { console.trace(); };
%ar b & new SaT();
b.bar();
... produces a stack trace wth the name of the class:
Trace(
at SaT.bar (/hoe//nt/booB/code/$2Moop/constructors.;s(11(11)
at Ib;ect. (/hoe//nt/booB/code/$2Moop/constructors.;s(1.(1)
32 of 91 9.9.2014 03:2
at 0odule.Mcopile (odule.;s()1-(-2)
at Ib;ect..;s (odule.;s().$(1$)
at 0odule.load (odule.;s(1.1(11)
at Function.Mload (odule.;s(11$(1-)
at Rrra7.$ (odule.;s()4$(1$)
at '%ent'itter.MticBFallbacB (node.;s(1O-()$)
To add prvate shared (among all nstances of the class) varables, add them to
the top level of the module:
// Vri%ate %ariable
%ar total & $;
// Fonstructor
function Foo() {
// access pri%ate shared %ariable
total!!;
};
// '+pose a getter (could also e+pose a setter to aBe it a public %ariable)
Foo.protot7pe.getTotalIb;ects & function(){
return total;
};
Avod assgnng varables to prototypes
If you want to defne a default value for a property of an nstance, defne t n the
constructor functon.
Prototypes should not have propertes that are not functons, because prototype
propertes that are not prmtves (such as arrays and objects) wll not behave as
one would expect, snce they wll use the nstance that s looked up from the
prototype. Example for Dmtry Sosnkov's ste:
%ar Foo & function (nae) { this.nae & nae; };
Foo.protot7pe.data & >1, -, 1@; // setting a nonDpriiti%e propert7
Foo.protot7pe.show"ata & function () { console.log(this.nae, this.data); };

%ar foo1 & new Foo(Cfoo1C);
%ar foo- & new Foo(Cfoo-C);

// both instances use the sae default %alue of data
foo1.show"ata(); // Cfoo1C, >1, -, 1@
foo-.show"ata(); // Cfoo-C, >1, -, 1@

// howe%er, if we change the data fro one instance
foo1.data.push());

// it irrors on the second instance
foo1.show"ata(); // Cfoo1C, >1, -, 1, )@
foo-.show"ata(); // Cfoo-C, >1, -, 1, )@
Hence prototypes should only defne methods, not data.
If you set the varable n the constructor, then you wll get the behavor you
expect:
33 of 91 9.9.2014 03:2
function Foo(nae) {
this.nae & nae;
this.data & >1, -, 1@; // setting a nonDpriiti%e propert7
};
Foo.protot7pe.show"ata & function () { console.log(this.nae, this.data); };
%ar foo1 & new Foo(Cfoo1C);
%ar foo- & new Foo(Cfoo-C);
foo1.data.push());
foo1.show"ata(); // Cfoo1C, >1, -, 1, )@
foo-.show"ata(); // Cfoo-C, >1, -, 1@
Don't construct by returnng objects - use prototype and new
For example, constructon pattern whch returns an object s terrble (even though
t was ntroduced n "JavaScrpt: The Good Parts"):
function Vhone(phoneAuber) {
%ar that & {};
// Xou are constructing a custo ob;ect on e%er7 call=
that.getVhoneAuber & function() {
return phoneAuber;
};
return that;
};
// or
function Vhone() {
// Xou are constructing a custo ob;ect on e%er7 call=
return {
getVhoneAuber( function() { ... }
};
};
Here, every tme we run Phone(), a new object s created wth a new property.
The V8 runtme cannot optmze ths case, snce there s no ndcaton that
nstances of Phone are a class; they look lke custom objects to the engne snce
prototypes are not used. Ths leads to slower performance.
It's also broken n another way: you cannot change the prototype propertes of all
nstances of Phone, snce they do not have a common ancestor/prototype object.
Prototypes exsts for a reason, so use the class pattern descrbed earler.
Avod mplementng classcal nhertance
I thnk classcal nhertance s n most cases an antpattern n Javascrpt. Why?
There are two reasons to have nhertance:
to support polymorphsm n languages that do not have dynamc typng, lke C++. The
class acts as an nterface specfcaton for a type. Ths provdes the beneft of beng
able to replace one class wth another (such as a functon that operates on a Shape
that can accept subclasses lke Crcle). However, Javascrpt doesn't requre you to do
ths: the only thng that matters s that a method or property can be looked up when
called/accessed.
1.
34 of 91 9.9.2014 03:2
to reuse code. Here the theory s that you can reuse code by havng a herarchy of
tems that go from an abstract mplementaton to a more specfc one, and you can
thus defne multple subclasses n terms of a parent class. Ths s sometmes useful,
but not that often.
2.
The dsadvantages of nhertance are:
Nonstandard, hdden mplementatons of classcal nhertance. Javascrpt doesn't have
a bultn way to defne class nhertance, so people nvent ther own ones. These
mplementatons are smlar to each other, but dffer n subtle ways.
1.
Deep nhertance trees. Subclasses are aware of the mplementaton detals of ther
superclasses, whch means that you need to understand both. What you see n the
code s not what you get: nstead, parts of an mplementaton are defned n the
subclass and the rest are defned pecemeal n the nhertance tree. The
mplementaton s thus sprnkled over multple fles, and you have to mentally
recombne those to understand the actual behavor.
2.
I favor composton over nhertance:
Composton - Functonalty of an object s made up of an aggregate of dfferent
classes by contanng nstances of other objects.
Inhertance - Functonalty of an object s made up of t's own functonalty plus
functonalty from ts parent classes.
If you must have nhertance, use plan old JS
If you must mplement nhertance, at least avod usng yet another nonstandard
mplementaton / magc functon. Here s how you can mplement a reasonable
facsmle of nhertance n pure ES3 (as long as you follow the rule of never
defnng propertes on prototypes):
function Rnial(nae) {
this.nae & nae;
};
Rnial.protot7pe.o%e & function(eters) {
console.log(this.nae!C o%ed C!eters!C.C);
};
function #naBe() {
Rnial.appl7(this, Rrra7.protot7pe.slice.call(arguents));
};
#naBe.protot7pe & new Rnial();
#naBe.protot7pe.o%e & function() {
console.log(C#lithering...C);
Rnial.protot7pe.o%e.call(this, .);
};
%ar sa & new #naBe(C#a7 the V7thonC);
sa.o%e();
Ths s not the same thng as classcal nhertance - but t s standard,
understandable Javascrpt and has the functonalty that people mostly seek:
3 of 91 9.9.2014 03:2
chanable constructors and the ablty to call methods of the superclass.
Or use utl.nherts() (from the Node.js core). Here s the full mplementaton:
%ar inherits & function (ctor, superFtor) {
ctor.superM & superFtor;
ctor.protot7pe & Ib;ect.create(superFtor.protot7pe, {
constructor( {
%alue( ctor,
enuerable( false
}
});
};
And a usage example:
%ar util & re5uire('util');
function Foo() { }
util.inherits(Foo, '%ent'itter);
The only real beneft to utl.nherts s that you don't need to use the actual
ancestor name n the Chld constructor.
Note that f you defne varables as propertes of a prototype, you wll experence
unexpected behavor (e.g. snce varables defned on the prototype of the
superclass wll be accessble n subclasses but wll also be shared among all
nstances of the subclass).
As I ponted out wth the class pattern, always defne all nstance varables n the
constructor. Ths forces the propertes to exst on the object tself and avods
lookups on the prototype chan for these varables.
Otherwse, you mght accdentally defne/access a varable property defned n a
prototype. Snce the prototype s shared among all nstances, ths wll lead to the
unexpected behavor f the varable s not a prmtve (e.g. s an Object or an
Array). See the earler example under "Avod settng varables as propertes of
prototypes".
Use mxns
A mxn s a functon that adds new functons to the prototype of an object. I prefer
to expose an explct mxn() functon to ndcate that the class s desgned to be
mxed nto another one:
function Foo() { }
Foo.protot7pe.bar & function() { };
Foo.protot7pe.baT & function() { };
// i+in D augent the target ob;ect with the Foo functions
Foo.i+in & function(destIb;ect){
>'bar', 'baT'@.for'ach(function(propert7) {
destIb;ect.protot7pe>propert7@ & Foo.protot7pe>propert7@;
});
};
3! of 91 9.9.2014 03:2
odule.e+ports & Foo;
Extendng the Bar prototype wth Foo:
%ar Foo & re5uire('./foo.;s');
function Sar() {}
Sar.protot7pe.5wert7 & function() {};
// i+in Foo
Foo.i+in(Sar);
Avod curryng
Curryng s a shorthand notaton for creatng an anonymous functon wth a new
scope that calls another functon. In other words, anythng you can do usng
curryng can be done usng a smple anonymous functon and a few varables
local to that functon.
Function.protot7pe.curr7 & function() {
%ar fn & this;
%ar args & Rrra7.protot7pe.slice.call(arguents);
return function() {
return fn.appl7(this, args.concat(Rrra7.protot7pe.slice.call(arguents, $)));
};
}
Curryng s ntrgung, but I haven't seen a practcal use case for t outsde of
subvertng how the this argument works n Javascrpt.
Don't use curryng to change the context of a call/the this argument. Use the
"self" varable accessed through an anonymous functon, snce t acheves the
same thng but s more obvous.
Instead of usng curryng:
function foo(a, b, c) { console.log(a, b, c); }
%ar bar & foo.curr7('8ello');
bar('9orld', '=');
I thnk that wrtng:
function foo(a, b, c) { console.log(a, b, c); }
function bar(b, c) { foo('8ello', b, c); }
bar('9orld', '=');
s more clear.
7. Control flow
In ths chapter, I:
dscuss nested callbacks and control flow n Node
ntroduce three essental async control flow patterns:
3" of 91 9.9.2014 03:2
Seres - for runnng async tasks one at a tme
Fully parallel - for runnng async tasks all at the same tme
Lmtedly parallel - for runnng a lmted number of async
tasks at the same tme
walk you through a smple mplementaton of these control flow
patterns
and convert the smple mplementaton nto a control flow lbrary that
takes callback arguments
When you start codng wth Node.js, ts a bt lke learnng programmng the frst
tme. Snce you want everythng to be asynchronous, you use a lot of callbacks
wthout really thnkng about how you should structure your code. Its a bt lke
beng overexcted about the f statement, and usng t and only t to wrte complex
programs. One of my frst programs n prmary school was a text-based adventure
where you would be presented wth a scenaro and a choce. I wrote code untl I
reached the maxmum level of nestng supported by the compler, whch probably
was 63 nested f statements.
Learnng how to code wth callbacks s smlar n many ways. If that s the only tool
you use, you wll create a mess.
Enlghtenment comes when you realze that ths:
as7nc1(function(input, result1) {
as7nc-(function(result-) {
as7nc1(function(result1) {
as7nc)(function(result)) {
as7nc.(function(output) {
// do soething with output
});
});
});
});
})
ought be wrtten as:
7Librar7.do#tuff(input, function(output){
// do soething with output
});
In other words, you can and are supposed to thnk n terms of hgher level
abstractons. Refactor, and extract functonalty nto ts own module. There can be
any number of callbacks between the nput that matters and the output that
matters, just make sure that you splt the functonalty nto meanngful modules
rather than dumpng t all nto one long chan.
Yes, there wll stll be some nested callbacks. However, more than a couple of
levels of nestng would should be a code smell - tme to thnk what you can
abstract out nto separate, small modules. Ths has the added beneft of makng
testng easer, because you end up havng smaller, hopefully meanngful code
modules that provde a sngle capablty.
3# of 91 9.9.2014 03:2
Unlke n tradonal scrptng languages based on blockng I/O, managng the
control flow of applcatons wth callbacks can warrant specalzed modules whch
coordnate partcular work flows: for example, by dealng wth the level
concurrency of executon.
Blockng on I/O provdes just one way to perform I/O tasks: sequentally (well, at
least wthout threads). Wth Node's "everythng can be done asynchronously"
approach, we get more optons and can choose when to block, when to lmt
concurrency and when to just launch a bunch of tasks at the same tme.
Let's look at the most common control flow patterns, and see how we can take
somethng as abstract as control flow and turn t nto a small, sngle purpose
module to take advantage of callbacks-as-nput.
7.2 Control flow: Specfyng executon order
If youve started readng some of the tutorals onlne, youll fnd a bewlderng
number of dfferent control-flow lbrares for Node. I fnd t qute confusng that
each of these has ts own API and termnology - talkng about promses, steps,
vows, futures and so on. Rather than endorse any partcular control flow soluton,
lets drll down to the bascs, look at some fundamental patterns and try to come
up wth a smple and undramatc set of terms to descrbe the dfferent optons we
have for control flow n Node.
As you already know, there are two types of API functons n Node.js:
asynchronous, non-blockng functons - for example: fs.readFle(flename, [encodng],
[callback])
1.
synchronous, blockng functons - for example: fs.readFleSync(flename, [encodng]) 2.
Synchronous functons return a result:
%ar data & fs.readFile#7nc('/etc/passwd');
Whle asynchronous functons receve the result va a callback (after passng
control to the event loop):
fs.readFile#7nc('/etc/passwd', function(err, data) { Y } );
Wrtng synchronous code s not problematc: we can draw on our experence n
other languages to structure t approprately usng keywords lke f, else, for, whle
and swtch. Its the way we should structure asynchronous calls whch s most
problematc, because establshed practces do not help here. For example, wed
lke to read a thousand text fles. Take the followng nave code:
for(%ar i & 1; i *& 1$$$; i!!) {
fs.readFile('./'!i!'.t+t', function() {
// do soething with the file
});
}
doMne+tMpart();
Ths code would start 1000 smultaneous asynchronous fle reads, and run the
39 of 91 9.9.2014 03:2
do_next_part() functon mmedately. Ths has several problems: frst, wed lke to
wat untl all the fle reads are done untl gong further. Second, launchng a
thousand fle reads smultaneously wll quckly exhaust the number of avalable
fle handles (a lmted resource needed to read fles). Thrd, we do not have a way
to accumulate the result for do_next_part().
We need:
a way to control the order n whch the fle reads are done
some way to collect the result data for processng
some way to restrct the concurrency of the fle read operatons to conserve lmted
system resources
a way to determne when all the reads necessary for the do_next_part() are completed
Control flow functons enable us to do ths n Node.js. A control flow functon s a
lghtweght, generc pece of code whch runs n between several asynchronous
functon calls and whch take care of the necessary housekeepng to:
control the order of executon, 1.
collect data, 2.
lmt concurrency and 3.
call the next step n the program. 4.
There are three basc patterns for ths.
7.2.1 Control flow pattern #1: Seres - an asynchronous for loop
Sometmes we just want to do one thng at a tme. For example, we need to do
fve database queres, and each of those queres needs data from the prevous
query, so we have to run one after another.
A seres does that:
// Rs7nc tasB (sae in all e+aples in this chapter)
function as7nc(arg, callbacB) {
console.log('do soething with Q''!arg!'Q', return 1 sec later');
setTieout(function() { callbacB(arg P -); }, 1$$$);
}
// Final tasB (sae in all the e+aples)
function final() { console.log('"one', results); }
// R siple as7nc series(
%ar ites & > 1, -, 1, ), ., 2 @;
%ar results & >@;
function series(ite) {
if(ite) {
as7nc( ite, function(result) {
results.push(result);
return series(ites.shift());
});
} else {
return final();
}
}
40 of 91 9.9.2014 03:2
series(ites.shift());
Bascally, we take a set of tems and call the seres control flow functon wth the
frst tem. The seres launches one async() operaton, and passes a callback to t.
The callback pushes the result nto the results array and then calls seres wth the
next tem n the tems array. When the tems array s empty, we call the fnal()
functon.
Ths results n seral executon of the asynchronous functon calls. Control s
passed back to the Node event loop after each async I/O operaton s completed,
then returned back when the operaton s completed.
Characterstcs:
Runs a number of operatons sequentally
Only starts one async operaton at a tme (no concurrency)
Ensures that the async functon complete n order
Varatons:
The way n whch the result s collected (manual or va a stashng callback)
How error handlng s done (manually n each subfuncton, or va a dedcated,
addtonal functon)
Snce executon s sequental, there s no need for a fnal callback
Tags: sequental, no-concurrency, no-concurrency-control
7.2.2 Control flow pattern #2: Full parallel - an asynchronous,
parallel for loop
In other cases, we just want to take a small set of operatons, launch them all n
parallel and then do somethng when all of them are complete.
A fully parallel control flow does that:
}
%ar ites & > 1, -, 1, ), ., 2 @;
%ar results & >@;
as7nc(ite, function(result){
if(results.length && ites.length) {
final();
}
})
});
41 of 91 9.9.2014 03:2
We take every tem n the tems array and start async operatons for each of the
tems mmedately. The async() functon s passed a functon that stores the result
and then checks whether the number of results s equal to the number of tems to
process. If t s, then we call the fnal() functon.
Snce ths means that all the I/O operatons are started n parallel mmedately, we
need to be careful not to exhaust the avalable resources. For example, you
probably don't want to start 1000's of I/O operatons, snce there are operatng
system lmtatons for the number of open fle handles. You need to consder
whether launchng parallel tasks s OK on a case-by-case bass.
Characterstcs:
Runs a number of operatons n parallel
Starts all async operatons n parallel (full concurrency)
No guarantee of order, only that all the operatons have been completed
Varatons:
The way n whch the result s collected (manual or va a stashng callback)
How error handlng s done (va the frst argument of the fnal functon, manually n
each subfuncton, or va a dedcated, addtonal functon)
Whether a fnal callback can be specfed
Tags: parallel, full-concurrency, no-concurrency-control
7.2.3 Control flow pattern #3: Lmted parallel - an asynchronous,
parallel, concurrency lmted for loop
In ths case, we want to perform some operatons n parallel, but keep the number
of runnng I/O operatons under a set lmt:
}
%ar ites & > 1, -, 1, ), ., 2 @;
%ar results & >@;
%ar running & $;
%ar liit & -;
function launcher() {
while(running * liit ZZ ites.length 6 $) {
%ar ite & ites.shift();
as7nc(ite, function(result) {
runningDD;
if(ites.length 6 $) {
launcher();
} else if(running && $) {
42 of 91 9.9.2014 03:2
final();
}
});
running!!;
}
}
launcher();
We start new async() operatons untl we reach the lmt (2). Each async()
operaton gets a callback whch stores the result, decrements the number of
runnng operatons, and then check whether there are tems left to process. If yes,
then laucher() s run agan. If there are no tems to process and the current
operaton was the last runnng operaton, then fnal() s called.
Of course, the crtera for whether or not we should launch another task could be
based on some other logc. For example, we mght keep a pool of database
connectons, and check whether "spare" connectons are avalable - or check
server load - or make the decson based on some more complcated crtera.
Characterstcs:
Runs a number of operatons n parallel
Starts a lmted number of operatons n parallel (partal concurrency, full concurrency
control)
No guarantee of order, only that all the operatons have been completed
7.3 Buldng a control flow lbrary on top of these patterns
For the sake of smplcty, I used a fxed array and named functons n the control
flow patterns above.
The problem wth the examples above s that the control flow s ntertwned wth
the data structures of our specfc use case: takng tems from an array and
populatng another array wth the results from a sngle functon.
We can wrte the same control flows as functons that take arguments n the form:
series(>
function(ne+t) { as7nc(1, ne+t); },
function(ne+t) { as7nc(-, ne+t); },
function(ne+t) { as7nc(), ne+t); },
function(ne+t) { as7nc(., ne+t); },
function(ne+t) { as7nc(2, ne+t); }
@, final);
E.g. an array of callback functons and a fnal() functon.
The callback functons get a next() functon as ther frst parameter whch they
should call when they have completed ther async operatons. Ths allows us to
use any async functon as part of the control flow.
43 of 91 9.9.2014 03:2
The fnal functon s called wth a sngle parameter: an array of arrays wth the
results from each async call. Each element n the array corresponds the values
passed back from the async functon to next(). Unlke the examples n prevous
secton, these functons store all the results from the callback, not just the frst
argument - so you can call next(1, 2, 3, 4) and all the arguments are stored n the
results array.
Seres
Ths converson s pretty straghtforward. We pass an anonymous functon whch
pushes to results and calls next() agan: ths s so that we can push the results
passed from the callback va arguments mmedately, rather than passng them
drectly to next() and handlng them n next().
function series(callbacBs, last) {
%ar results & >@;
function ne+t() {
%ar callbacB & callbacBs.shift();
if(callbacB) {
callbacB(function() {
results.push(Rrra7.protot7pe.slice.call(arguents));
ne+t();
});
} else {
last(results);
}
}
ne+t();
}
// '+aple tasB
%ar dela7 & 0ath.floor(0ath.rando() P . ! 1) P 1$$; // rando s
console.log('as7nc with Q''!arg!'Q', return in '!dela7!' s');
setTieout(function() { callbacB(arg P -); }, dela7);
}
function final(results) { console.log('"one', results); }
series(>
@, final);
Full parallel
Unlke n a seres, we cannot assume that the results are returned n any
partcular order.
Because of ths we use callbacks.forEach, whch returns the ndex of the callback
- and store the result to the same ndex n the results array.
44 of 91 9.9.2014 03:2
Snce the last callback could complete and return t's result frst, we cannot use
results.length, snce the length of an array always returns the largest ndex n the
array + 1. So we use an explct result_counter to track how many results we've
gotten back.
function fullVarallel(callbacBs, last) {
%ar results & >@;
%ar resultMcount & $;
callbacBs.for'ach(function(callbacB, inde+) {
callbacB( function() {
results>inde+@ & Rrra7.protot7pe.slice.call(arguents);
resultMcount!!;
if(resultMcount && callbacBs.length) {
last(results);
}
});
});
}
// '+aple tasB
%ar dela7 & 0ath.floor(0ath.rando() P . ! 1) P 1$$; // rando s
setTieout(function() { callbacB(arg P -); }, dela7);
}
fullVarallel(>
@, final);
Lmted parallel
Ths s a bt more complcated, because we need to launch async tasks once
other tasks fnsh, and need to store the result from those tasks back nto the
correct poston n the results array. Detals further below.
function liited(liit, callbacBs, last) {
%ar results & >@;
%ar running & 1;
%ar tasB & $;
function ne+t(){
runningDD;
if(tasB && callbacBs.length ZZ running && $) {
last(results);
}
while(running * liit ZZ callbacBs>tasB@) {
%ar callbacB & callbacBs>tasB@;
(function(inde+) {
callbacB(function() {
results>inde+@ & Rrra7.protot7pe.slice.call(arguents);
ne+t();
});
4 of 91 9.9.2014 03:2
})(tasB);
tasB!!;
running!!;
}
}
ne+t();
}
// '+aple tasB
%ar dela7 & 0ath.floor(0ath.rando() P . ! 1) P 1$$$; // rando s
%ar result & arg P -;
console.log(',eturn with Q''!arg!'Q', result '!result);
callbacB(result);
}, dela7);
}
liited(1, >
@, final);
We need to keep two counter values here: one for the next task, and another for
the callback functon.
In the fully parallel control flow we took advantage of [].forEach(), whch returns
the ndex of the currently runnng task n t's own scope.
Snce we cannot rely on forEach() as tasks are launched n small groups, we
need to use an anonymous functon to get a new scope to hold the orgnal ndex.
Ths ndex s used to store the return value from the callback.
To llustrate the problem, I added a longer delay to the return from async() and an
addtonal lne of loggng whch shows when the result from async s returned. At
that moment, we need to store the return value to the rght ndex.
The anonymous functon: (functon(ndex) { ... } (task)) s needed because f we
ddn't create a new scope usng an anonymous functon, we would store the result
n the wrong place n the results array (snce the value of task mght have
changed between callng the callback and returnng back from the callback). See
the chapter on Javascrpt gotchas for more nformaton on scope rules n JS.
7.4 The fourth control flow pattern
There s a fourth control flow pattern, whch I won't dscuss here: eventual
completon. In ths case, we are not nterested n strctly controllng the order of
operatons, only that they occur at some pont and are correctly responded to.
4! of 91 9.9.2014 03:2
In Node, ths can be mplemented usng EventEmtters. These are dscussed n
the chapter on Node fundamentals.
8. An overvew of Node: Modules and npm
In ths chapter, I:
dscuss modules and process-related globals n Node
recommend npm for package management, and show how:
t makes nstallng your own dependences easer
t allows you to fetch external gt repostores as
dependences and
t provdes a standard vocabulary for runnng scrpts, such
as start and test
Node.js has a good amount of functonalty bult n. Let's look at the Table of
Contents for the API documentaton and try to group t nto manageable chunks
(talc = not covered here):
Fundamentals
Globals
STDIO
Tmers
Modules
Events
Buffers
Streams
C/C++ Addons
Network I/O
HTTP
HTTPS
URL
Query Strngs
Net
UDP/Datagram
DNS
Fle system I/O
Fle System
Path
Process I/O and V8 VM
Process
VM
Chld Processes
Cluster
Termnal/console
REPL
Readlne
TTY
Testng & debuggng
Asserton Testng
Debugger
Utltes
Msc
Crypto
TLS/SSL
Strng Decoder
ZLIB
OS
Ill go through the parts of the Node API that youll use the most when wrtng web
applcatons. The rest of the API s best looked up from nodejs.org/ap/.
Fundamentals
The current chapter and
Chapter 9.
Network I/O
HTTP and HTTPS are covered n
Chapter 10.
Fle system I/O
The fle system module s
covered n Chapter 11.
Process I/O and V8 VM
Covered n Chapter
TODO.
Termnal/console
REPL s dscussed n Chapter
TODO.
Testng and debuggng
Coverage TODO.
4" of 91 9.9.2014 03:2
8.1 Node.js modules
Let's talk about the module system n Node.
Modules make t possble to nclude other Javascrpt fles nto your applcatons. In
fact, a vast majorty of Nodes core functonalty s mplemented usng modules
wrtten n Javascrpt - whch means you can read the source code for the core
lbrares on Gthub.
Modules are crucal to buldng applcatons n Node, as they allow you to nclude
external lbrares, such as database access lbrares - and they help n organzng
your code nto separate parts wth lmted responsbltes. You should try to
dentfy reusable parts n your own code and turn them nto separate modules to
reduce the amount of code per fle and to make t easer to read and mantan
your code.
Usng modules s smple: you use the re5uire() functon, whch takes one
argument: the name of a core lbrary or a fle system path to the module you want
to load. Youve seen ths before n the smple messagng applcaton example,
where I used re5uire() to use several core modules.
To make a module yourself, you need to specfy what objects you want to export.
The exports object s avalable n the top level scope n Node for ths purpose:
e+ports.funcnae & function() {
return [8ello 9orld\;
};
Any propertes assgned to the exports object wll be accessble from the return
value of the requre() functon:
%ar hello & re5uire([./hello.;s\);
console.log(hello.funcnae()); // Vrint <8ello 9orld:
You can also use module.exports nstead of exports:
function funcnae() { return [8ello 9orld\; }
odule.e+ports & { funcnae( funcnae };
Ths alternatve syntax makes t possble to assgn a sngle object to exports (such
as a class). Weve prevously dscussed how you can buld classes usng
prototypal nhertance. By makng your classes separate modules, you can easly
nclude them n your applcaton:
// in class.;s(
%ar Flass & function() { Y }
Flass.protot7pe.funcnae & function() {...}
odule.e+ports & Flass;
Then you can nclude your fle usng requre() and make a new nstance of your
class:
// in another file(
%ar Flass & re5uire([./class.;s\);
4# of 91 9.9.2014 03:2
%ar ob;ect & new Flass(); // create new instance
Sharng varables between modules
Note that there s no global context n Node. Each scrpt has t's own context, so
ncludng multple modules does not pollute the current scope. %ar foo & 'bar';
n the top level scope of another module wll not defne foo n other modules.
What ths means s that the only way to share varables and values between node
modules s to nclude the same module n multple fles. Snce modules are
cached, you can use a shared module to store common data, such as
confguraton optons:
// in config.;s
%ar config & {
foo( 'bar'
};
odule.e+ports & config;
In a dfferent module:
// in ser%er.;s
%ar config & re5uire('./config.;s');
console.log(config.foo);
However, Node module has a number of varables whch are avalable by default.
These are documented n the API docs: globals and process.
Some of the more nterestng ones are:
__flename The flename of the code beng executed.
__drname The name of the drectory that the currently executng scrpt resdes n.
process A object whch s assocated wth the currently runnng process. In
addton to varables, t has methods such as process.ext, process.cwd
and process.uptme.
process.argv. An array contanng the command lne arguments. The frst element wll
be 'node', the second element wll be the name of the JavaScrpt fle.
The next elements wll be any addtonal command lne arguments.
process.stdn,
process.stout,
process.stderr.
Streams whch correspond to the standard nput, standard output and
standard error output for the current process.
process.env An object contanng the user envronment of the current process.
requre.man When a fle s run drectly from Node, re5uire.ain s set to ts odule.
The code below wll prnt the values for the current scrpt:
console.log('MMfilenae', MMfilenae);
console.log('MMdirnae', MMdirnae);
console.log('process.arg%', process.arg%);
console.log('process.en%', process.en%);
if(odule &&& re5uire.ain) {
console.log('This is the ain odule being run.');
49 of 91 9.9.2014 03:2
}
requre.man can be used to detect whether the module beng currently run s the
man module. Ths s useful when you want to do somethng else when a module
s run standalone. For example, I make my test fles runnable va node
filenae.;s by ncludng somethng lke ths:
// if this odule is the script being run, then run the tests(
if (odule &&& re5uire.ain) {
%ar nodeunitMrunner & re5uire('nodeunitDrunner');
nodeunitMrunner.run(MMfilenae);
}
process.stdn, process.stdout and process.stderr are brefly dscussed n the next
chapter, where we dscuss readable and wrtable streams.
Organzng modules
There are three ways n whch you can requre() fles:
usng a relatve path: foo = requre('./lb/bar.js');
usng an absolute path: foo = requre('/home/foo/lb/bar.js')
usng a search: foo = requre('bar')
The frst two are easy to understand. In the thrd case, Node starts at the current
drectory, and adds ./node_modules/, and attempts to load the module from that
locaton. If the module s not found, then t moves to the parent drectory and
performs the same check, untl the root of the flesystem s reached.
For example, f requre('bar') would called n /home/foo/, the followng locatons would be
searched untl a match a found:
/home/foo/node_modules/bar.js
/home/node_modules/bar.js and
/node_modules/bar.js
Loadng modules n ths way s easer than specfyng a relatve path, snce you
can move the fles wthout worryng about the paths changng.
Drectores as modules
You can organze your modules nto drectores, as long as you provde a pont of
entry for Node.
The easest way to do ths s to create the drectory ./node_modules
/mymodulename/, and put an ndex.js fle n that drectory. The ndex.js fle wll be
loaded by default.
Alternatvely, you can put a package.json fle n the mymodulename folder,
specfyng the name and man fle of the module:
{
0 of 91 9.9.2014 03:2
CnaeC( C7odulenaeC,
CainC( C./lib/foo.;sC
}
Ths would cause the fle ./node_modules/mymodulename/lb/foo.js to be returned
from re5uire('7odulenae') .
Generally, you want to keep a sngle ./node_modules folder n the base drectory
of your app. You can nstall new modules by addng fles or drectores to
./node_modules. The best way to manage these modules s to use npm, whch s
covered brefly n the next secton.
8.2 npm
npm s the package manager used to dstrbute Node modules. I won't cover t n
detal here, because the Internet does that already.
npm s awesome, and you should use t. Below are a couple of use cases.
8.2.1 Installng packages
The most common use case for npm s to use t for nstallng modules from other
people:
np search pacBagenae
np %iew pacBagenae
np install pacBagenae
np outdated
np update pacBagenae
Packages are nstalled under ./node_modules/ n the current drectory.
8.2.2 Specfyng and nstallng dependences for your own app
npm makes nstallng your applcaton on a new machne easy, because you can
specfy what modules you want to have n your applcaton by addng a
package.json fle n the root of your applcaton.
Here s a mnmal package.json:
{ CnaeC( CodulenaeC,
CdescriptionC( CFoo for barC,
C%ersionC( C$.$.1C,
Crepositor7C( {
Ct7peC( CgitC,
CurlC( Cgit(//github.co/i+u/odulenae.gitC },
CdependenciesC( {
CunderscoreC( C1.1.+C,
CfooC( Cgit!ssh(//git]github.co(i+u/foo.gitJ$.).1C,
CbarC( Cgit!ssh(//git]github.co(i+u/bar.gitJasterC
},
Cpri%ateC( true
}
1 of 91 9.9.2014 03:2
Ths makes gettng the rght versons of the dependences of your applcaton a lot
easer. To nstall the dependences specfed for your applcaton, run:
np install
8.2.3 Loadng dependences from a remote gt repostory
One of my favorte features s the ablty to use gt+ssh URLs to fetch remote gt
repostores. By specfyng a URL lke gt+ssh://gthub.com:mxu/nwm.gt#master,
you can nstall a dependency drectly from a remote gt repostory. The part after
the hash refers to a tag or branch on the repostory.
To lst the nstalled dependences, use:
np ls
8.2.4 Specfyng custom start, stop and test scrpts
You can also use the "scrpts" member of package.json to specfy actons to be
taken durng varous stages:
{ CscriptsC (
{ CpreinstallC ( C./configureC,
CinstallC ( CaBe ZZ aBe installC,
CtestC ( CaBe testC,
CstartC( Cscripts/start.;sC,
CstopC( Cscripts/stop.;sC
}
}
In the example above, we specfy what should happen before nstall and durng
nstall. We also defne what happens when npm test, npm start and npm stop are
called.
9. Node fundamentals: Tmers,
EventEmtters, Streams and Buffers
In ths chapter, I cover the fundamentals - the essental buldng blocks of Node
applcatons and core modules.
The buldng blocks of Node applcatons are:
Streams Readable and wrtable streams an alternatve way of nteractng wth
(fle|network|process) I/O.
Buffers Buffers provde a bnary-frendly, hgher-performance alternatve to strngs by
exposng raw memory allocaton outsde the V8 heap.
Events Many Node.js core lbrares emt events. You can use EventEmtters to
mplement ths pattern n your own applcatons.
Tmers setTmeout for one-tme delayed executon of code, setInterval for perodcally
repeatng executon of code
C/C++ Addons Provde the capablty to use your own or 3rd party C/C++ lbrares
2 of 91 9.9.2014 03:2
from Node.js
Note that I wll not cover C/C++ addons, as ths goes beyond the scope of the
book.
9.1 Tmers
The tmers lbrary conssts of four global functons:
setTmeout(callback, delay,
[arg], [...])
Schedule the executon of the gven callback after delay
mllseconds. Returns a tmeoutId for possble use wth
clearTmeout(). Optonally, you can also pass arguments to the
callback.
setInterval(callback, delay,
[arg], [...])
Schedule the repeated executon of callback every delay
mllseconds. Returns a ntervalId for possble use wth
clearInterval(). Optonally, you can also pass arguments to the
callback.
clearTmeout(tmeoutId) Prevents a tmeout from trggerng.
clearInterval(ntervalId) Stops an nterval from trggerng.
These functons can be used to schedule callbacks for executon. The setTmeout
functon s useful for performng housekeepng tasks, such as savng the state of
the program to dsk after a partcular nterval. The same functons are avalable n
all major browsers:
// setting a tieout
console.log('Foo');
}, 1$$$);
// #etting and clearing an inter%al
%ar counter & $;
%ar inter%al & setUnter%al( function() {
console.log('Sar', counter);
counter!!;
if (counter 6& 1) {
clearUnter%al(inter%al);
}
}, 1$$$);
Whle you can set a tmeout or nterval usng a strng argument (e.g.
setTieout([long,epeatedTasB\, .$$$) , ths s a bad practce snce the strng
has to be dynamcally evaluated (lke usng the eval() functon, whch s not
recommended). Instead, use a varable or a named functon as nstead of a
strng.
Remember that tmeouts and ntervals are only executed when the executon s
passed back to the Node event loop, so tmngs are not necessarly accurate f
you have a long-runnng blockng task. So a long, CPU-ntensve task whch takes
longer than the tmeout/nterval tme wll prevent those tasks from beng run at
ther scheduled tmes.
3 of 91 9.9.2014 03:2
9.2 EventEmtters
event.EventEmtter s a class whch s used to provde a consstent nterface for
emttng (trggerng) and bndng callbacks to events. It s used nternally n many
of the Node core lbrares and provdes a sold foundaton to buld event-based
classes and applcatons.
Usng EventEmtters n your own objects
To create a class whch extends EventEmtter, you can use utls.nhert():
%ar '%ent'itter & re5uire([e%ents\).'%ent'itter;
%ar util & re5uire([util\);
// create the class
%ar 07Flass & function () { Y }
// augent the protot7pe using util.inherits
util.inherits(07Flass, '%ent'itter);
07Flass.protot7pe.whate%er & function() { Y }
Addng lsteners
EventEmtters allow you to add lsteners - callbacks - to any arbtrarly named
event (except newLstener, whch s specal n EventEmtter). You can attach
multple callbacks to a sngle event, provdng for flexblty. To add a lstener, use
'%ent'itter.on(e%ent, listener) or '%ent'itter.addListener(e%ent,
listener) - they both do the same thng:
%ar ob; & new 07Flass();
ob;.on([soee%ent\, function(arg1) { Y });
You can use '%ent'itter.once(e%ent, listener) to add a callback whch wll
only be trggered once, rather than every tme the event occurs. Ths s a good
practce, snce you should keep the number of lsteners to a mnmum (n fact, f
you have over 10 lsterners, EventEmtter wll warn you that you need to call
emtter.setMaxLsteners).
Trggerng events
To trgger an event from your class, use '%ent'itter.eit(e%ent, >arg1@,
>arg-@, >...@) :
07Flass.protot7pe.whate%er & function() {
this.eit([soee%ent\, [8ello\, [9orld\);
};
The emt functon takes an unlmted number of arguments, and passes those on
to the callback(s) assocated wth the event. You can remove event lsteners usng
'%ent'itter.reo%eListener(e%ent, listener) or
'%ent'itter.reo%eRllListeners(e%ent) , whch remove ether one lstener, or
all the lsteners assocated wth a partcular event.
How EventEmtters work
Now that you have seen the API exposed by EventEmtters, how would
4 of 91 9.9.2014 03:2
somethng lke ths be mplemented? Whle there are many thngs to take care of,
the smplest "EventEmtter" would be an object hash contanng functons:
%ar #iple'' & function() {
this.e%ents & {};
};
#iple''.protot7pe.on & function(e%entnae, callbacB) {
this.e%ents>e%entnae@ ^^ (this.e%ents>e%entnae@ & >@);
this.e%ents>e%[email protected](callbacB);
};
#iple''.protot7pe.eit & function(e%entnae) {
%ar args & Rrra7.protot7pe.slice.call(arguents, 1);
if (this.e%ents>e%entnae@) {
this.e%ents>e%[email protected]'ach(function(callbacB) {
callbacB.appl7(this, args);
});
}
};
// '+aple using the e%ent eitter
%ar eitter & new #iple''();
eitter.on('greet', function(nae) {
console.log('8ello, ' ! nae ! '=' );
});
eitter.on('greet', function(nae) {
console.log('9orld, ' ! nae ! '=' );
});
>'foo', 'bar', 'baT'@.for'ach(function(nae) {
eitter.eit('greet', nae);
});
It's really pretty smple - though the Node core EventEmtter class has many
addtonal features (such as beng able to attach multple callbacks per event,
removng lsteners, callng lsteners only once, performance-related mprovements
etc.).
EventEmtters extremely useful for abstractng event-based nteractons, and f
ths ntroducton seems a bt too theoretcal - don't worry. You'll see EventEmtters
used n many dfferent stuatons and wll hopefully learn to love them.
EventEmtters are also used extensvely n Node core lbrares.
One thng that you should be aware of s that EventEmtters are not prvleged n
any way - despte havng Event n ther name, they are executed just lke any
other code - emttng events does not trgger the event loop n any specal way. Of
course, you should use asynchronous functons for I/O n your callbacks, but
EventEmtters are smply a standard way of mplementng ths knd of nterface.
You can verfy ths by readng the source code for EventEmtters on Gthub.
9.3 Streams
Weve dscussed the three man alternatves when t comes to controllng
executon: Sequental, Full Parallel and Parallel. Streams are an alternatve way
of accessng data from varous sources such as the network (TCP/UDP), fles,
chld processes and user nput. In dong I/O, Node offers us multple optons for
of 91 9.9.2014 03:2
accessng the data:
Synchoronous Asynchronous
Fully buffered readFleSync() readFle()
Partally buffered (streamng) readSync() read(), createReadStream()
The dfference between these s how the data s exposed, and the amount of
memory used to store the data.
Fully buffered access
// Full7 buffered access
>1$$ 0b file@
D6 1. >allocate 1$$ 0b buffer@
D6 -. >read and return 1$$ 0b buffer@
Fully buffered functon calls lke readFleSync() and readFle() expose the data as
one bg blob. That s, readng s performed and then the full set of data s returned
ether n synchronous or asynchronous fashon.
Wth these fully buffered methods, we have to wat untl all of the data s read, and
nternally Node wll need to allocate enough memory to store all of the data n
memory. Ths can be problematc - magne an applcaton that reads a 1 GB fle
from dsk. Wth only fully buffered access we would need to use 1 GB of memory
to store the whole content of the fle for readng - snce both readFle and
readFleSync return a strng contanng all of the data.
Partally buffered (streamng) access
// #treas (and partiall7 buffered reads)
>1$$ 0b file@
D6 1. >allocate sall buffer@
D6 -. >read and return sall buffer@
D6 1. >repeat 1Z- until done@
Partally buffered access methods are dfferent. They do not treat data nput as a
dscrete event, but rather as a seres of events whch occur as the data s beng
read or wrtten. They allow us to access data as t s beng read from dsk/network
/other I/O.
Partally buffered methods, such as readSync() and read() allow us to specfy the
sze of the buffer, and read data n small chunks. They allow for more control (e.g.
readng a fle n non-lnear order by skppng back and forth n the fle).
Streams
However, n most cases we only want to read/wrte through the data once, and n
one drecton (forward). Streams are an abstracton over partally buffered data
access that smplfy dong ths knd of data processng. Streams return smaller
parts of the data (usng a Buffer), and trgger a callback when new data s
avalable for processng.
Streams are EventEmtters. If our 1 GB fle would, for example, need to be
! of 91 9.9.2014 03:2
processed n some way once, we could use a stream and process the data as
soon as t s read. Ths s useful, snce we do not need to hold all of the data n
memory n some buffer: after processng, we no longer need to keep the data n
memory for ths knd of applcaton.
The Node stream nterface conssts of two parts: Readable streams and Wrtable
streams. Some streams are both readable and wrtable.
Readable streams
The followng Node core objects are Readable streams:
Fles fs.createReadStream(path,
[optons])
Returns a new ReadStream object (See Readable Stream).
HTTP (Server)
http.ServerRequest
The request object passed when processng the
request/response callback for HTTP servers.
HTTP (Clent)
http.ClentResponse
The response object passed when processng the response
from an HTTP clent request.
TCP net.Socket Construct a new socket object.
Chld process chld.stdout The stdout ppe for chld processes launched from Node.js
Chld process chld.stderr The stderr ppe for chld processes launched from Node.js
Process process.stdn A Readable Stream for stdn. The stdn stream s paused by
default, so one must call process.stdn.resume() to read from
t.
Readable streams emt the followng events:
Event:
data
Emts ether a Buffer (by default) or a strng f setEncodng() was used.
Event:
end
Emtted when the stream has receved an EOF (FIN n TCP termnology). Indcates
that no more 'data' events wll happen.
Event:
error
Emtted f there was an error recevng data.
To bnd a callback to an event, use stream.on(eventname, callback). For example,
to read data from a fle, you could do the followng:
%ar fs & re5uire('fs');
%ar file & fs.create,ead#trea('./test.t+t');
file.on('error', function(err) {
console.log(''rror '!err);
throw err;
});
file.on('data', function(data) {
console.log('"ata '!data);
});
file.on('end', function(){
console.log('Finished reading all of the data');
});
" of 91 9.9.2014 03:2
Readable streams have the followng functons:
pause() Pauses the ncomng 'data' events.
resume() Resumes the ncomng 'data' events after a pause().
destroy() Closes the underlyng fle descrptor. Stream wll not emt any more events.
Wrtable streams
The followng Node core objects are Wrtable streams:
Fles fs.createWrteStream(path,
[optons])
Returns a new WrteStream object (See Wrtable
Stream).
HTTP (Server) http.ServerResponse
HTTP (Clent) http.ClentRequest
TCP net.Socket
Chld process chld.stdn
Process process.stdout A Wrtable Stream to stdout.
Process process.stderr A wrtable stream to stderr. Wrtes on ths stream are
blockng.
Wrtable streams emt the followng events:
Event:
dran
After a wrte() method returned false, ths event s emtted to ndcate that t s safe
to wrte agan.
Event:
error
Emtted on error wth the excepton excepton.
Wrtable streams have the followng functons:
wrte(strng,
encodng='utf8')
Wrtes strng wth the gven encodng to the stream.
end() Termnates the stream wth EOF or FIN. Ths call wll allow queued
wrte data to be sent before closng the stream.
destroy() Closes the underlyng fle descrptor. Stream wll not emt any more
events. Any queued wrte data wll not be sent.
Lets read from stdn and wrte to a fle:
%ar file & fs.create9rite#trea('./out.t+t');
process.stdin.on('data', function(data) {
file.write(data);
});
process.stdin.on('end', function() {
file.end();
});
process.stdin.resue(); // stdin in paused b7 default
# of 91 9.9.2014 03:2
Runnng the code above wll wrte everythng you type n from stdn to the fle
out.txt, untl you ht Ctrl+d (e.g. the end of fle ndcator n Lnux).
You can also ppe readable and wrtable streams usng
readableStream.ppe(destnaton, [optons]). Ths causes the content from the
read stream to be sent to the wrte stream, so the program above could have
been wrtten as:
process.stdin.pipe(fs.create9rite#trea('./out.t+t'));
process.stdin.resue();
9.4 Buffers - workng wth bnary data
Buffers n Node are a hgher-performance alternatve to strngs. Snce Buffers
represent raw C memory allocaton, they are more approprate for dealng wth
bnary data than strngs. There are two reasons why buffers are useful:
They are allocated outsde of V8, meanng that they are not managed by V8.
Whle V8 s generally hgh performance, sometmes t wll move data
unnecessarly. Usng a Buffer allows you to work around ths and work wth the
memory more drectly for hgher performance.
They do not have an encodng, meanng that ther length s fxed and accurate.
Strngs support encodngs such as UTF-8, whch nternally stores many foregn
characters as a sequence of bytes. Manpulatng strngs wll always take nto
account the encodng, and wll transparently treat sequences of bytes as sngle
characters. Ths causes problems for bnary data, snce bnary data (lke mage
fles) are not encoded as characters but rather as bytes - but may concdentally
contan byte sequences whch would be nterpreted as sngle UTF-8 characters.
Workng wth buffers s a bt more complcated than workng wth strngs, snce
they do not support many of the functons that strngs do (e.g. ndexOf). Instead,
buffers act lke fxed-sze arrays of ntegers. The Buffer object s global (you dont
have to use requre() to access t). You can create a new Buffer and work wth t
lke an array of ntegers:
// Freate a Suffer of 1$ b7tes
%ar buffer & new Suffer(1$);
// 0odif7 a %alue
buffer>$@ & -..;
// Log the buffer
console.log(buffer);
// outputs( *Suffer ff $$ $$ $$ $$ $$ )a 4b $3 1f6
Note how the buffer has ts own representaton, n whch each byte s shown a
hexadecmal number. For example, ff n hex equals 255, the value we just wrote
n ndex 0. Snce Buffers are raw allocatons of memory, ther content s whatever
happened to be n memory; ths s why there are a number of dfferent values n
the newly created buffer n the example.
Buffers do not have many predefned functons and certanly lack many of the
features of strngs. For example, strngs are not fxed sze, and have convenent
9 of 91 9.9.2014 03:2
functons such as Strng.replace(). Buffers are fxed sze, and only offer the very
bascs:
new Buffer(sze)
new Buffer(str, encodng='utf8')
new Buffer(array)
Buffers can be created: 1) wth a fxed sze, 2) from an
exstng strng and 3) from an array of octets
buffer.wrte(strng, offset=0,
encodng='utf8')
Wrte a strng to the buffer at [offset] usng the gven
encodng.
buffer.sBuffer(obj) Tests f obj s a Buffer.
buffer.byteLength(strng,
encodng='utf8')
Gves the actual byte length of a strng. Ths s not the
same as Strng.prototype.length snce that returns the
number of characters n a strng.
buffer.length The sze of the buffer n bytes.
buffer.copy(targetBuffer, targetStart=0,
sourceStart=0,
sourceEnd=buffer.length)
Does a memcpy() between buffers.
buffer.slce(start, end=buffer.length) Returns a new buffer whch references the same
memory as the old, but offset and cropped by the start
and end ndexes.
Modfyng the new buffer slce wll modfy memory n
the orgnal buffer!
buffer.toStrng(encodng, start=0,
end=buffer.length)
Decodes and returns a strng from buffer data
encoded wth encodng begnnng at start and endng
at end.
However, f you need to use the strng functons on buffers, you can convert them
to strngs usng buffer.toStrng() and you can also convert strngs to buffers usng
new Buffer(str). Note that Buffers offer access to the raw bytes n a strng, whle
Strngs allow you to operate on charaters (whch may consst of one or more
bytes). For example:
%ar buffer & new Suffer('87%__ p_i%__='); // create a buffer containing </ood da7=: in Finnish
%ar str & '87%__ p_i%__='; // create a string containing </ood da7=: in Finnish
// log the contents and lengths to console
console.log(buffer);
console.log('Suffer length(', buffer.length);
console.log(str);
console.log('#tring length(', str.length);
If you run ths example, you wll get the followng output:
*Suffer )3 4O 42 c1 a) c1 a) -$ 4$ c1 a) 2O 42 c1 a) c1 a) -16
Suffer length( 13
87%__ p_i%__=
#tring length( 11
Note how buffer.length s 18, whle strng.length s 13 for the same content. Ths s
!0 of 91 9.9.2014 03:2
because n the default UTF-8 encodng, the a wth dots character s represented
nternally by two characters (c3 a4 n hexadecmal). The Buffer allows us to
access the data n ts nternal representaton and returns the actual number of
bytes used, whle Strng takes nto account the encodng and returns the number
of characters used. When workng wth bnary data, we frequently need to access
data that has no encodng - and usng Strngs we could not get the correct length
n bytes. More realstc examples could be, for example, readng an mage fle
from a TCP stream, or readng a compressed fle, or some other case where
bnary data wll be accessed.
10. Node.js: HTTP, HTTPS
Ths chapter covers the HTTP and HTTPS modules.
Just a remnder: the coverage s lmted to common use cases; I wll go nto more
depth n the chapter on Controllers about specfc topcs such as routng and
sessons.
10.1 HTTP server
HTTP server
Methods
http.createServer([requestLstener])
server.lsten(port, [hostname],
[callback])
server.lsten(path, [callback])
server.close()
Events
request
connecton
close
checkContnue
upgrade
clentError
Server request
Methods
setEncodng(encodng=null)
pause()
resume()
Events
data
end
close
Propertes
method
url
headers
tralers
httpVerson
Server response
Methods
wrteContnue()
wrteHead(statusCode,
[reasonPhrase], [headers])
setHeader(name, value)
getHeader(name)
removeHeader(name)
wrte(chunk, encodng='utf8')
addTralers(headers)
end([data], [encodng])
Propertes
statusCode
!1 of 91 9.9.2014 03:2
connecton
Creatng an HTTP server s smple: after requrng the http module, you call
createServer, then nstruct the server to lsten on a partcular port:
%ar ser%er & http.create#er%er(function(re5uest, response) {
// ,ead the re5uest, and write bacB to the response
});
ser%er.listen(3$3$, 'localhost');
The callback functon you pass to http.createServer s called every tme a clent
makes a request to the server. The callback should take two parameters - a
request and a response - and send back HTML or some other output to the clent.
The request s used to determne what should be done (e.g. what path on the
server was requested, what GET and POST parameters were sent). The
response allows us to wrte output back to the clent.
The other API functons related to startng the HTTP server and recevng
requests and closng the server are:
http.createServer(requestLstener) Returns a new web server object. The requestLstener s a
functon whch s automatcally added to the 'request'
event.
server.lsten(port, [hostname],
[callback])
Begn acceptng connectons on the specfed port and
hostname. If the hostname s omtted, the server wll
accept connectons drected to any IPv4 address
(INADDR_ANY).
To lsten to a unx socket, supply a flename nstead of
port and hostname.
Ths functon s asynchronous. The last parameter
callback wll be called when the server has been bound to
the port.
server.on([eventname], [callback]) Allows you to bnd callbacks to events such as "request",
"upgrade" and "close".
server.close() Stops the server from acceptng new connectons.
The server object returned by http.createServer() s an EventEmtter (see the
prevous chapter) - so you can also bnd new request handlers usng server.on():
// create a ser%er with no callbacB bound to 're5uest'
%ar ser%er & http.create#er%er().listen(3$3$, 'localhost');
// bind a listener to the 're5uest' e%ent
ser%er.on('re5uest', function(re5, res) {
// do soething with the re5uest
});
The other events that the HTTP server emts are not partcularly nterestng for
!2 of 91 9.9.2014 03:2
daly use, so let's look at the Request and Response objects.
10.1.1 The Server Request object - http.ServerRequest
The frst parameter of the request handler callback s a ServerRequest object.
ServerRequests are Readable Streams, so we can bnd to the "data" and "end"
events to access the request data (see further below).
The Request object contans three nterestng propertes:
request.method The request method as a strng. Read only.
Example: 'GET', POST, 'DELETE'.
request.url Request URL strng.
Example: /, /user/1, /post/new/?param=value
request.headers A read only object, ndexed by the name of the header (converted to
lowercase), contanng the values of the headers.
Example: see below.
request.url The most mportant property s request.url, whch s used to
determne whch page was requested and what to do wth the request.
In the smple messagng applcaton example, we brefly saw how ths property
was used to determne what to do wth clent requests. We wll go nto further
depth when we dscuss routng n web applcatons n the chapter on Controllers.
Checkng request.method tells us what HTTP method was used to retreve the
page, the most common of whch are GET and POST. GET requests retreve a
resource, and may have addtonal parameters passed as part of request.url.
POST requests are generally the result of form submssons and ther data must
be read from the request separately (see below).
request.headers allows us read-only access to the headers. HTTP cookes are
transmtted n the headers, so we need to parse the headers to access the
cookes. User agent nformaton s also passed n the request headers.
You can access all of ths nformaton through the frst parameter of your request
handler callback:
%ar ser%er & http.create#er%er(function(re5uest, response) {
console.log(re5uest);
});
ser%er.listen(3$3$, 'localhost');
Once you pont your browser to http://localhost:8080/, ths wll prnt out all the
dfferent propertes of the current HTTP request, whch ncludes a number of more
advanced propertes:
{
!3 of 91 9.9.2014 03:2
socBet( { Y },
connection( { Y },
http?ersion( '1.1',
coplete( false,
headers(
{
host( 'localhost(3$3$',
connection( 'BeepDali%e',
'cacheDcontrol'( 'a+Dage&$',
'userDagent'( '0oTilla/..$ (W11; Linu+ +32M2)) ...',
accept( 'application/+l,application/+htl!+l ...',
'acceptDencoding'( 'gTip,deflate,sdch',
'acceptDlanguage'( 'enDK#,en;5&$.3',
'acceptDcharset'( 'U#ID33.OD1,utfD3;5&$.4,P;5&$.1'
},
trailers( {},
readable( true,
url( '/',
ethod( '/'T',
statusFode( null,
client( { Y },
http?ersion0a;or( 1,
http?ersion0inor( 1,
upgrade( false
}
10.1.1.1 Parsng data
There are several dfferent formats through whch an HTTP server can receve
requests. The most commonly used formats are:
HTTP GET - passed va request.url
HTTP POST requests - passed as data events
cookes - passed va request.headers.cookes
10.1.1.2 Parsng GET requests
The request.url parameter contans the URL for the current request. GET
parameters are passed as a part of ths strng. The URL module provdes three
functons whch can be used to work wth URLs:
url.parse(url#tr, parse`uer7#tring & false): Parses a URL strng and returns an
object whch contans the varous parts of the URL.
url.forat(urlIb;): Accepts a parsed URL object and returns the strng. Does the
reverse of url.parse().
url.resol%e(fro, to): Resolves a gven URL relatve to a base URL as a browser
would for an anchor tag.
The url.parse() functon can be used to parse a URL:
%ar url & re5uire('url');
%ar urlMparts & url.parse(re5.url, true);
By passng true as the second parameter (parseQueryStrng), you get an
addtonal "query" key that contans the parsed query strng. For example:
!4 of 91 9.9.2014 03:2
%ar url & re5uire('url');
console.log( url.parse(
'http(//user(pass]host.co(3$3$/p/a/t/ha5uer7&stringJhash', true
));
Returns the followng object:
{
href( 'http(//user(pass]host.co(3$3$/p/a/t/ha5uer7&stringJhash',
protocol( 'http(',
host( 'user(pass]host.co(3$3$',
auth( 'user(pass',
hostnae( 'host.co',
port( '3$3$',
pathnae( '/p/a/t/h',
search( 'a5uer7&string',
5uer7( { 5uer7( 'string' },
hash( 'Jhash',
slashes( true
}
Of these result values, there are three are most relevant for data prosessng n
the controller: pathname (the URL path), query (the query strng) and hash (the
hash fragment).
10.1.1.3 Parsng POST requests
Post requests transmt ther data as the body of the request. To access the data,
you can buffer the data to a strng n order to parse t. The data s accessble
through the data events emtted by the request. When all the data has been
receved, the end event s emtted:
function parseVost(re5, callbacB) {
%ar data & '';
re5.on('data', function(chunB) {
data !& chunB;
});
re5.on('end', function() {
callbacB(data);
});
}
POST requests can be n multple dfferent encodngs. The two most common
encodngs are: applcaton/x-www-form-urlencoded and multpart/form-data.
applcaton/x-www-form-urlencoded
nae&Hohn!"oeZgender&aleZfail7&.Zcit7&BentZcit7&iaiZother&abcE$"E$RdefZnicBnae&HE-2"
applcaton/x-www-form-urlencoded data s encoded lke a GET request. It's the
default encodng for forms and used for most textual data.
The QueryStrng module provdes two functons:
5uer7string.parse(str, sep&\Z\, e5&\&\): Parses a GET query strng and returns an
object that contans the parameters as propertes wth values. Example:
! of 91 9.9.2014 03:2
qs.parse(a=b&c=d) would return {a: b, c: d}.
5uer7string.stringif7(ob;, sep&\Z\, e5&\&\): Does the reverse of
querystrng.parse(); takes an object wth propertes and values and returns a strng.
Example: qs.strngfy({a: b}) would return a=b.
You can use querystrng.parse to convert POST data nto an object:
%ar 5s & re5uire('5uer7string');
%ar data & '';
re5.on('data', function(chunB) {
data !& chunB;
});
re5.on('end', function() {
%ar post & 5s.parse(data);
console.log(post);
});
multpart/form-data
FontentDT7pe( ultipart/forDdata; boundar7&RaS$1+
DDRaS$1+
FontentD"isposition( forDdata; nae&CsubitDnaeC
Larr7
DDRaS$1+
FontentD"isposition( forDdata; nae&CfilesC; filenae&Cfile1.t+tC
FontentDT7pe( te+t/plain
... contents of file1.t+t ...
DDRaS$1+DD
multpart/form-data s used for bnary fles. Ths encodng s somewhat complcated to decode, so
I won't provde a snppet. Instead, have a look at how t's done n:
felxge's node-formdable
vsonmeda's connect-form
10.1.2 The Server Response object - http.ServerResponse
The second parameter of the request handler callback s a ServerResponse
object. ServerResponses are Wrtable Streams, so we wrte() data and call end()
to fnsh the response.
10.1.2.1 Wrtng response data
The code below shows how you can wrte back data from the server.
%ar http & re5uire('http'),
url & re5uire('url');
%ar ser%er & http.create#er%er().listen(3$3$, 'localhost');
ser%er.on('re5uest', function(re5, res) {
%ar urlMparts & url.parse(re5.url, true);
switch(urlMparts.pathnae) {
case '/'(
case '/inde+.htl'(
res.write('*htl6*bod768ello=*/bod76*/htl6');
breaB;
!! of 91 9.9.2014 03:2
default(
res.write('KnBnown path( ' ! H#IA.stringif7(urlMparts));
}
res.end();
});
Note that response.end() must be called on each response to fnsh the response
and close the connecton.
10.1.2.2 Common response headers
Some common uses forHTTP headers nclude:
Specfyng the content type (content-type)
Redrectng requests (locaton)
Specfyng the flename and sze of a download (content-dsposton, content-length)
Settng cookes (set-cooke)
Specfyng compresson (content-encodng)
Controllng cachng (cache-control, expres, etag)
I wll only cover the frst two use cases, snce the focus here s on how to use the
HTTP API rather than on HTTP 1.1 tself.
Headers and wrte()
HTTP headers have to be sent before the request data s sent (that's why they
are called headers).
Headers can be wrtten n two ways:
Explctly usng response.write8ead(statusFode, >reasonVhrase@, >headers@). In ths
case, you have to specfy all the headers n one go, along wth the HTTP status code
and an optonal human-readable reasonPhrase.
Implctly: the frst tme response.write() s called, the currently set mplct headers are
sent.
The API has the detals:
response.wrteHead(statusCode,
[reasonPhrase], [headers])
Sends a response header to the request. The status
code s a 3-dgt HTTP status code, lke 404. The last
argument, headers, are the response headers.
Optonally one can gve a human-readable
reasonPhrase as the second argument.
response.statusCode When usng mplct headers (not callng
response.wrteHead() explctly), ths property controls
the status code that wll be send to the clent when the
headers get flushed.
response.setHeader(name, value) Sets a sngle header value for mplct headers. If ths
header already exsts n the to-be-sent headers, t's
value wll be replaced. Use an array of strngs here f you
!" of 91 9.9.2014 03:2
need to send multple headers wth the same name.
response.getHeader(name) Reads out a header that's already been queued but not
sent to the clent. Note that the name s case nsenstve.
Ths can only be called before headers get mplctly
flushed.
response.removeHeader(name) Removes a header that's queued for mplct sendng.
Generally, usng mplct headers s smpler snce you can change the ndvdual
headers up untl the pont when the frst call to response.wrte() s made.
Settng the content/type header
Browsers expect to receve a content-type header for all content. Ths header
contans the MIME type for the content/fle that s sent, whch s used to determne
what the browser should do wth the data (e.g. dsplay t as an mage).
In our earler examples, we dd not set a content-type header, because the
examples dd not serve content other than HTML and plantext. However, n order
to support bnary fles lke mages and n order to send formatted data such as
JSON and XML back, we need to explctly specfy the content type.
Usually, the mme type s determned by the server based on the fle extenson:
%ar ap & {
'.ico'( 'iage/+Dicon',
'.htl'( 'te+t/htl',
'.;s'( 'te+t/;a%ascript',
'.;son'( 'application/;son',
'.css'( 'te+t/css',
'.png'( 'iage/png'
};
%ar e+t & '.css';
if(ap>e+t@) {
console.log('FontentDt7pe', ap>e+t@);
}
There are ready-made lbrares that you can use to determne the mme type of a
fle, such as:
bentomas/node-mme
creatonx/smple-mme
To set the content-type header from a flename:
%ar e+t & re5uire('path').e+tnae(filenae);
if(ap>e+t@) {
res.set8eader('FontentDt7pe', ap>e+t@);
}
We wll look at how to read fles n the next chapter.
!# of 91 9.9.2014 03:2
Redrectng to a dfferent URL
Redrects are performed usng the Locaton: header. For example, to redrect to
/ndex.html:
res.statusFode & 1$-;
res.set8eader('Location', '/inde+.htl');
res.end();
10.2 HTTP clent
There s also a HTTP clent API, whch allows you to make HTTP requests and
read content from other webstes.
HTTP clent
Methods
http.request(optons, callback)
http.get(optons, callback)
Clent request
Methods
wrte(chunk, encodng='utf8')
end([data], [encodng])
abort()
setTmeout(tmeout, [callback])
setNoDelay(noDelay=true)
setSocketKeepAlve(enable=false,
[ntalDelay])
Events
response
socket
upgrade
contnue
Clent response
Methods
setEncodng(encodng=null)
pause()
resume()
Events
data
end
close
Propertes
statusCode
httpVerson
headers
tralers
10.2.1 Issung a smple GET request
http.get(options, callbacB) A convnence method to make HTTP GET requests.
http.get() returns a http.ClentRequest object, whch s a Wrtable Stream.
The callback passed to http.get() wll receve a http.ClentResponse object when
the request s made. The ClentResponse s a Readable Stream.
To send a smple GET request, you can use http.get . You need to set the
followng optons:
host: the doman or IP address of the server
!9 of 91 9.9.2014 03:2
port: the port (e.g. 80 for HTTP)
path: the request path, ncludng the query strng (e.g. 'ndex.html?page=12')
To read the response data, you should attach a callback to the 'data' and 'end'
events of the returned object. You wll most lkely want to store the data
somewhere from the 'data' events, then process t as a whole on the 'end' event.
The followng code ssues a GET request to www.google.com/, reads the 'data'
and 'end' events and outputs to the console.
%ar options & {
host( 'www.google.co',
port( 3$,
path( '/'
};
%ar re5 & http.get(options, function(response) {
// handle the response
%ar resMdata & '';
response.on('data', function(chunB) {
resMdata !& chunB;
});
response.on('end', function() {
console.log(resMdata);
});
});
re5.on('error', function(e) {
console.log(C/ot error( C ! e.essage);
});
To add GET query parameters from an object, use the querystrng module:
%ar 5s & re5uire('5uer7string');
%ar options & {
port( 3$,
path( '/'!'a'!5s.stringif7({5( 'hello world'})
};
// .. as in pre%ious e+aple
As you can see above, GET parameters are sent as a part of the request path.
10.2.2 Issung POST, DELETE and other methods
http.re5uest(options,
callbacB)
Issue an HTTP request. Host, port and path are specfed n the
optons object parameter. Calls the callback wth an
http.ClentRequest object wth the new request.
To ssue POST, DELETE or other requests, you need to use http.re5uest and
set the method n the optons explctly:
%ar opts & {
port( 3$,
ethod( 'VI#T'
path( '/',
headers( {}
"0 of 91 9.9.2014 03:2
};
To send the data along wth the POST request, call req.wrte() wth the data you
want to send along wth the request before callng req.end(). To ensure that the
recevng server can decode the POST data, you should also set the
content-type.
There are two common encodngs used to encode POST request data:
applcaton/x-www-form-urlencoded
// VI#T encoding
opts.headers>'FontentDT7pe'@ & 'application/+DwwwDforDurlencoded';
re5.data & 5s.stringif7(re5.data);
opts.headers>'FontentDLength'@ & re5.data.length;
and applcaton/json:
// H#IA encoding
opts.headers>'FontentDT7pe'@ & 'application/;son';
re5.data & H#IA.stringif7(re5.data);
opts.headers>'FontentDLength'@ & re5.data.length;
Makng a request s very smlar to makng a GET request:
%ar re5 & http.re5uest(opts, function(response) {
response.on('data', function(chunB) {
resMdata !& chunB;
});
response.on('end', function() {
callbacB(resMdata);
});
});
re5.on('error', function(e) {
console.log(C/ot error( C ! e.essage);
});
// write the data
if (opts.ethod =& '/'T') {
re5.write(re5.data);
}
re5.end();
Note, however, that you need to call req.end() after http.request(). Ths s
because http.ClentRequest supports sendng a request body (wth POST or other
data) and f you do not call req.end(), the request remans "pendng" and wll most
lkely not return any data before you end t explctly.
10.2.3 Wrtng a smple HTTP proxy
Snce the HTTP server and clent expose Readable/Wrtable streams, we can
wrte a smple HTTP proxy smply by ppe()ng the two together.
The ServerRequest s Readable, whle the ClentRequest s Wrtable. Smlarly,
the ClentResponse s Readable, whle the ServerResponse s Wrtable.
%ar ser%er & http.create#er%er(function(sre5, sres) {
%ar urlMparts & url.parse(sre5.url);
"1 of 91 9.9.2014 03:2
%ar opts & {
host( 'google.co',
port( 3$,
path( urlMparts.pathnae,
ethod( sre5.ethod,
headers( sre5.headers
};
%ar cre5 & http.re5uest(opts, function(cres) {
sres.write8ead(cres.statusFode, cres.headers);
cres.pipe(sres); // pipe client to ser%er response
});
sre5.pipe(cre5); // pipe ser%er to client re5uest
});
ser%er.listen(3$, '$.$.$.$');
The code above causes any requests made to http://localhost:80/ to be proxed to
Google. You can pass queres too, such as http://localhost:80/search?q=Node.js
10.3 HTTPS server and clent
The HTTPS server and clent API s almost dentcal to the HTTP API, so pretty
much everythng sad above apples to them. In fact, the clent API s the same,
and the HTTPS server only dffers n that t needs a certfcate fle.
The HTTPS server lbrary allows you to serve fles over SSL/TLS. To get started,
you need to have a SSL certfcate from a certfcate authorty or you need to
generate one yourself. Of course, self-generated certfcates wll generally trgger
warnngs n the browser.
10.3.1 Confguraton: generatng your own certfcate
Here s how you can generate a self-sgned certfcate:
openssl genrsa Dout pri%ateBe7.pe 1$-)
openssl re5 Dnew DBe7 pri%ateBe7.pe Dout certre5uest.csr
openssl +.$O Dre5 Din certre5uest.csr DsignBe7 pri%ateBe7.pe Dout certificate.pe
Note that ths certfcate wll trgger warnngs n your browser, snce t s
self-sgned.
10.3.2 Startng the server
To start the HTTPS server, you need to read the prvate key and certfcate. Note
that readFile#7nc s used n ths case, snce blockng to read the certfcates
when the server starts s acceptable:
// 8TTV#
%ar https & re5uire('https');
// read in the pri%ate Be7 and certificate
%ar pB & fs.readFile#7nc('./pri%ateBe7.pe');
%ar pc & fs.readFile#7nc('./certificate.pe');
%ar opts & { Be7( pB, cert( pc };
// create the secure ser%er
%ar ser% & https.create#er%er(opts, function(re5, res) {
console.log(re5);
"2 of 91 9.9.2014 03:2
res.end();
});
// listen on port ))1
ser%.listen())1, '$.$.$.$');
Note that on Lnux, you may need to run the server wth hgher prvleges to bnd to port 443.
Other than needng to read a prvate key and certfcate, the HTTPS server works lke the HTTP
server.
11. Fle system
Ths chapter covers the fle system module.
The fle system functons consst of fle I/O and drectory I/O functons. All of the
fle system functons offer both synchronous (blockng) and asynchronous
(non-blockng) versons. The dfference between these two s that the
synchronous functons (whch have Sync n ther name) return the value drectly
and prevent Node from executng any code whle the I/O operaton s beng
performed:
%ar data & fs.readFile#7nc('./inde+.htl', 'utf3');
// wait for the result, then use it
console.log(data);
Asynchronous functons return the value as a parameter to a callback gven to
them:
fs.readFile('./inde+.htl', 'utf3', function(err, data) {
// the data is passed to the callbacB in the second arguent
console.log(data);
});
The table below lsts all the asynchronous functons n the FS API. These
functons have synchronous versons as well, but I left them out to make the
lstng more readable.
Read & wrte a fle (fully
buffered)
fs.readFle(flename,
[encodng], [callback])
fs.wrteFle(flename,
data, encodng='utf8',
[callback])
Read & wrte a fle (n parts)
fs.open(path, flags,
[mode], [callback])
fs.read(fd, buffer, offset,
length, poston, [callback])
fs.wrte(fd, buffer, offset,
length, poston, [callback])
fs.fsync(fd, callback)
fs.truncate(fd, len,
[callback])
fs.close(fd, [callback])
Drectores: read, create & delete
fs.readdr(path, [callback])
fs.mkdr(path, mode,
[callback])
fs.rmdr(path, [callback])
Fles: nfo
fs.stat(path, [callback])
Readable streams
fs.ReadStream
Wrtable streams
fs.WrteStream
"3 of 91 9.9.2014 03:2
fs.lstat(path, [callback])
fs.fstat(fd, [callback])
fs.realpath(path,
[callback])
Event: 'open'
fs.createReadStream(path,
[optons])
Event: 'open'
fle.bytesWrtten
fs.createWrteStream(path,
[optons])
Fles: rename, watch changes &
change tmestamps
fs.rename(path1, path2,
[callback])
fs.watchFle(flename,
[optons], lstener)
fs.unwatchFle(flename)
fs.watch(flename,
[optons], lstener)
fs.utmes(path, atme,
mtme, callback)
fs.futmes(path, atme,
mtme, callback)
Fles: Owner and permssons
fs.chown(path, ud, gd,
[callback])
fs.fchown(path, ud, gd,
[callback])
fs.lchown(path, ud, gd,
[callback])
fs.chmod(path, mode,
[callback])
fs.fchmod(fd, mode,
[callback])
fs.lchmod(fd, mode,
[callback])
Fles: symlnks
fs.lnk(srcpath, dstpath,
[callback])
fs.symlnk(lnkdata, path,
[callback])
fs.readlnk(path, [callback])
fs.unlnk(path, [callback])
You should use the asynchronous verson n most cases, but n rare cases (e.g.
readng confguraton fles when startng a server) the synchronous verson s
more approprate. Note that the asynchronous versons requre a bt more
thought, snce the operatons are started mmedately and may fnsh n any order:
fs.readFile('./file.htl', function (err, data) {
// ...
});
fs.readFile('./other.htl', function (err, data) {
// ..
});
These fle reads mght complete n any order dependng on how long t takes to
read each fle. The smplest soluton s to chan the callbacks:
fs.readFile('./file.htl', function (err, data) {
// ...
fs.readFile('./other.htl', function (err, data) {
// ...
});
});
However, we can do better by usng the control flow patterns dscussed n the
chapter on control flow.
11.1 Fles: readng and wrtng
Fully buffered reads and wrtes are farly straghtforward: call the functon and
pass n a Strng or a Buffer to wrte, and then check the return value.
Recpe: Readng a fle (fully buffered)
fs.readFile('./inde+.htl', 'utf3', function(err, data) {
"4 of 91 9.9.2014 03:2
// the data is passed to the callbacB in the second arguent
console.log(data);
});
Recpe: Wrtng a fle (fully buffered)
fs.writeFile('./results.t+t', '8ello 9orld', function(err) {
if(err) throw err;
console.log('File write copleted');
});
When we want to work wth fles n smaller parts, we need to open(), get a fle
descrptor and then work wth that fle descrptor.
fs.open(path, flags, >ode@, >callbacB@) supports the followng flags:
'r' - Open fle for readng. An excepton occurs f the fle does not exst.
'r+' - Open fle for readng and wrtng. An excepton occurs f the fle does not exst.
'w' - Open fle for wrtng. The fle s created (f t does not exst) or truncated (f t
exsts).
'w+' - Open fle for readng and wrtng. The fle s created (f t does not exst) or
truncated (f t exsts).
'a' - Open fle for appendng. The fle s created f t does not exst.
'a+' - Open fle for readng and appendng. The fle s created f t does not exst.
mode refers to the permssons to use n case a new fle s created. The default s
0666.
Recpe: Openng, wrtng to a fle and closng t (n parts)
fs.open('./data/inde+.htl', 'w', function(err, fd) {
if(err) throw err;
%ar buf & new Suffer('bbbbbQn');
fs.write(fd, buf, $, buf.length, null, function(err, written, buffer) {
if(err) throw err;
console.log(err, written, buffer);
fs.close(fd, function() {
console.log('"one');
});
});
});
The read() and wrte() functons operate on Buffers, so n the example above we
create a new Buffer() from a strng. Note that bult-n readable streams (e.g.
HTTP, Net) generally return Buffers.
Recpe: Openng, seekng to a poston, readng from a fle and
closng t (n parts)
fs.open('./data/inde+.htl', 'r', function(err, fd) {
if(err) throw err;
%ar str & new Suffer(1);
fs.read(fd, buf, $, buf.length, null, function(err, b7tes,ead, buffer) {
if(err) throw err;
console.log(err, b7tes,ead, buffer);
fs.close(fd, function() {
" of 91 9.9.2014 03:2
console.log('"one');
});
});
});
11.2 Drectores: read, create & delete
Recpe: Readng a drectory
Readng a drectory returns the names of the tems (fles, drectores and others)
n t.
%ar path & './data/';
fs.readdir(path, function (err, files) {
if(err) throw err;
files.for'ach(function(file) {
console.log(path!file);
fs.stat(path!file, function(err, stats) {
console.log(stats);
});
});
});
fs.stat() gves us more nformaton about each tem. The object returned from
fs.stat looks lke ths:
{ de%( -11),
ino( )3$2)O2O,
ode( 11133,
nlinB( 1,
uid( 3.,
gid( 1$$,
rde%( $,
siTe( .-4,
blBsiTe( )$O2,
blocBs( 3,
atie( 0on, 1$ Ict -$11 -1(-)(11 /0T,
tie( 0on, 1$ Ict -$11 -1(-)(11 /0T,
ctie( 0on, 1$ Ict -$11 -1(-)(11 /0T }
atme, mtme and ctme are Date nstances. The stat object also has the followng
functons:
stats.sFle()
stats.sDrectory()
stats.sBlockDevce()
stats.sCharacterDevce()
stats.sSymbolcLnk() (only vald wth fs.lstat())
stats.sFIFO()
stats.sSocket()
The Path module has a set of addtonal functons for workng wth paths, such as:
path.normalze(p) Normalze a strng path, takng care of '..' and '.' parts.
path.jon([path1],
[path2], [...])
Jon all arguments together and normalze the resultng path.
"! of 91 9.9.2014 03:2
path.resolve([from ...],
to)
Resolves to to an absolute path.
If to sn't already absolute from arguments are prepended n rght to
left order, untl an absolute path s found. If after usng all from paths
stll no absolute path s found, the current workng drectory s used as
well. The resultng path s normalzed, and tralng slashes are
removed unless the path gets resolved to the root drectory.
fs.realpath(path,
[callback])
fs.realpathSync(path)
Resolves both absolute (/path/fle) and relatve paths (../../fle) and
returns the absolute path to the fle.
path.drname(p) Return the drectory name of a path. Smlar to the Unx drname
command.
path.basename(p, [ext]) Return the last porton of a path. Smlar to the Unx basename
command.
path.extname(p) Return the extenson of the path. Everythng after the last '.' n the last
porton of the path. If there s no '.' n the last porton of the path or the
only '.' s the frst character, then t returns an empty strng.
path.exsts(p,
[callback])
path.exstsSync(p)
Test whether or not the gven path exsts. Then, call the callback
argument wth ether true or false.
Recpe: Creatng and deletng a drectory
fs.Bdir('./newdir', $222, function(err) {
if(err) throw err;
console.log('Freated newdir');
fs.rdir('./newdir', function(err) {
if(err) throw err;
console.log(',eo%ed newdir');
});
});
Usng Readable and Wrtable streams
Readable and Wrtable streams are covered n Chapter 9.
Recpe: Readng a fle and wrtng to another fle
%ar file & fs.create,ead#trea('./data/results.t+t', {flags( 'r'} );
%ar out & fs.create9rite#trea('./data/results-.t+t', {flags( 'w'});
file.on('data', function(data) {
console.log('data', data);
out.write(data);
});
file.on('end', function() {
console.log('end');
out.end(function() {
console.log('Finished writing to file');
test.done();
});
"" of 91 9.9.2014 03:2
});
You can also use ppe():
%ar file & fs.create,ead#trea('./data/results.t+t', {flags( 'r'} );
%ar out & fs.create9rite#trea('./data/results-.t+t', {flags( 'w'});
file.pipe(out);
Recpe: Appendng to a fle
%ar file & fs.create9rite#trea('./data/results.t+t', {flags( 'a'} );
file.write('8'LLI=Qn');
file.end(function() {
test.done();
});
Practcal example
Snce Node makes t very easy to launch multple asynchronous fle accesses,
you have to be careful when performng large amounts of I/O operatons: you
mght exhaust the avalable number of fle handles (a lmted operatng system
resource used to access fles). Furthermore, snce the results are returned n a
asynchronous manner whch does not guarantee completon order, you wll most
lkely want to coordnate the order of executon usng the control flow patterns
dscussed n the prevous chapter. Lets look at an example.
Example: searchng for a fle n a drectory, traversng recursvely
In ths example, we wll search for a fle recursvely startng from a gven path. The
functon takes three arguments: a path to search, the name of the fle we are
lookng for, and a callback whch s called when the fle s found.
Here s the nave verson: a bunch of nested callbacks, no thought needed:
function findFile(path, searchFile, callbacB) {
if(err) { return callbacB(err); }
fs.stat(path!'/'!file, function() {
if(stats.isFile() ZZ file && searchFile) {
callbacB(undefined, path!'/'!file);
}
} else if(stats.is"irector7()) {
findFile(path!'/'!file, searchFile, callbacB);
}
});
});
});
}
findFile('./test', 'needle.t+t', function(err, path) {
if(err) { throw err; }
console.log('Found file at( '!path);
});
"# of 91 9.9.2014 03:2
Splttng the functon nto smaller functons makes t somewhat easer to
understand:
// checB for a atch, gi%en a stat
function is0atch(err, stats) {
if(stats.isFile() ZZ file && searchFile) {
callbacB(undefined, path!'/'!file);
stat"irector7(path!'/'!file);
}
}
// launch the search
stat"irector7(path, is0atch);
}
// ,ead and stat a director7
function stat"irector7(path, callbacB) {
fs.stat(path!'/'!file, callbacB);
});
});
}
findFile('./test', 'needle.t+t', function(err, path) {
if(err) { throw err; }
console.log('Found file at( '!path);
});
The functon s splt nto smaller parts:
fndFle: Ths code starts the whole process, takng the man nput arguments as well
as the callback to call wth the results.
sMatch: Ths hdden helper functon takes the results from stat() and apples the "s a
match" logc necessary to mplement fndFle().
statDrectory: Ths functon smply reads a path, and calls the callback for each fle.
I admt ths s farly verbose.
PathIterator: Improvng reuse by usng an EventEmtter
However, we can accomplsh the same goal n a more reusable manner by
creatng our own module (pathterator.js), whch treats drectory traversal as a
stream of events by usng EventEmtter:
%ar fs & re5uire('fs'),
'%ent'itter & re5uire('e%ents').'%ent'itter,
util & re5uire('util');
%ar VathUterator & function() { };
// augent with '%ent'itter
"9 of 91 9.9.2014 03:2
util.inherits(VathUterator, '%ent'itter);
// Uterate a path, eitting 'file' and 'director7' e%ents.
VathUterator.protot7pe.iterate & function(path) {
%ar self & this;
this.stat"irector7(function(fpath, stats) {
if(stats.isFile()) {
self.eit('file', fpath, stats);
self.eit('director7', fpath, stats);
self.iterate(path!'/'!file);
}
});
};
// ,ead and stat a director7
VathUterator.protot7pe.stat"irector7 & function(path, callbacB) {
if(err) { self.eit('error', err); }
%ar fpath & path!'/'!file;
fs.stat(fpath, function (err, stats) {
if(err) { self.eit('error', err); }
callbacB(fpath, stats);
});
});
});
};
odule.e+ports & VathUterator;
As you can see, we create a new class whch extends EventEmtter, and emts
the followng events:
error - functon(error): emtted on errors.
fle - functon(flepath, stats): the full path to the fle and the result from fs.stat
drectory - functon(drpath, stats): the full path to the drectory and the result from
fs.stat
We can then use ths utlty class to mplement the same drectory traversal:
%ar VathUterator & re5uire('./pathiterator.;s');
%ar pi & new VathUterator();
pi.on('file', function(file, stats) {
if(file && searchFile) {
callbacB(undefined, file);
}
});
pi.on('error', callbacB);
pi.iterate(path);
}
Whle ths approach takes a few lnes more than the pure-callback approach, the
result s a somewhat ncer and extensble (for example - you could look for
multple fles n the fle callback easly).
If you end up wrtng a lot of code that terates paths, havng a PathIterator
#0 of 91 9.9.2014 03:2
EventEmtter wll smplfy your code. The callbacks are stll there - after all, ths s
non-blockng I/O va the event loop - but the nterface becomes a lot easer to
understand. You are probably runnng fndFle() as part of some larger process -
and nstead of havng all that fle travelsal logc n the same module, you have a
fxed nterface whch you can wrte your path traversng operatons aganst.
Usng a specalzed module: async.js
Encapsulatng the I/O behnd a EventEmtter helps a bt, but we can do one better
by usng fjacob's async.js. Ths s a FS-specfc lbrary that encapsulates fle
system operatons behnd a chanable nterface. The fndFle() functon can be
wrtten usng async.js lke ths:
%ar as7nc & re5uire('as7nc;s');
as7nc.readdir(path)
.stat()
.filter(function(file) {
return file.stat && searchFile;
})
.to#tring(callbacB);
}
The technques behnd async.js consst essentally of an object that acts lke a
seres (from the chapter on Control Flow), but allows you to specfy the operatons
usng a chanable nterface.
My pont - f there s any - s that coordnatng asynchronous I/O control flow n
Node s somewhat more complcated than n scrptng envronments/languages
where you can rely on I/O blockng the executon of code. Ths s most obvous
when you are dealng wth the fle system, because fle system I/O generally
requres you to perform long sequences of asynchronous calls n order to get
what you want.
However, even wth fle I/O, t s possble to come up wth solutons that abstract
away the detals from your man code. In some cases you can fnd doman
specfc lbrares that provde very conscse ways of expressng your logc (e.g.
async.js) - and n other cases you can at least take parts of the process, and
move those nto a separate module (e.g. PathIterator).
12. Comet and Socket.o deployment
In ths chapter, I:
present an overvew of the technques to mplement Comet
ntroduce Socket.o and a basc applcaton
dscuss the complextes of real-world deployment wth Socket.o
So... do you want to buld a chat? Or a real-tme multplayer game?
In order to buld a (soft-) real-tme app, you need the ablty to update nformaton
#1 of 91 9.9.2014 03:2
quckly wthn the end user's browser.
HTTP was not desgned to support full two-way communcaton. However, there
are multple ways n whch the clent can receve nformaton n real tme or almost
real tme:
Technques to mplement Comet
Perodc pollng. Essentally, you ask the server whether t has new data every n
seconds, and dle meanwhle:
Flient( Rre we there 7eta
#er%er( Ao
Flient( >9ait a few seconds@
#er%er( Ao
Flient( >9ait a few seconds@
... (repeat this a lot)
#er%er( Xes. 8ere is a essage for 7ou.
The problem wth perodc pollng s that: 1) t tends to generate a lot of requests
and 2) t's not nstant - f messages arrve durng the tme the clent s watng,
then those wll only be receved later.
Long pollng. Ths s smlar to perodc pollng, except that the server does not
return the response mmedately. Instead, the response s kept n a pendng state
untl ether new data arrves, or the request tmes out n the browser. Compared to
perodc pollng, the advantage here s that clents need to make fewer requests
(requests are only made agan f there s data) and that there s no "dle" tmeout
between makng requests: a new request s made mmedately after recevng
data.
#er%er( >9ait for b1$ seconds@
#er%er( Ao
Ths approach s slghtly better than perodc pollng, snce messages can be
delvered mmedately as long as a pendng request exsts. The server holds on to
the request untl the tmeout trggers or a new message s avalable, so there wll
be fewer requests.
However, f you need to send a message to the server from the clent whle a long
pollng request s ongong, a second request has to be made back to the server
snce the data cannot be sent va the exstng (HTTP) request.
Sockets / long-lved connectons. WebSockets (and other transports wth socket
semantcs) mprove on ths further. The clent connects once, and then a
permanent TCP connecton s mantaned. Messages can be passed n both ways
through ths sngle request. As a conversaton:
#er%er( >9ait for until we're there@
#2 of 91 9.9.2014 03:2
If the clent needs to send a message to the server, t can send t through the
exstng connecton rather than through a separate request. Ths effcent and fast,
but Websockets are only avalable n newer, better browsers.
Socket.o
As you can see above, there are several dfferent ways to mplement Comet.
Socket.o offers several dfferent transports:
Long pollng: XHR-pollng (usng XMLHttpRequest), JSONP pollng (usng JSON wth
paddng), HTMLFle (forever Iframe for IE)
Sockets / long-lved connectons: Flash sockets (Websockets over plan TCP sockets
usng Flash) and Websockets
Ideally, we would lke to use the most effcent transport (Websockets) - but fall
back to other transports on older browsers. Ths s what Socket.o does.
Wrtng a basc applcaton
I almost left ths part out, snce I can't really do justce to Socket.o n one secton
of a full chapter. But here t s: a smple example usng Socket.o. In ths example,
we wll wrte smple echo server that receves messages from the browser and
echoes them back.
Let's start wth a package.json:
{
CnaeC( CsiosapleC,
CdescriptionC( C#iple #ocBet.io appC,
C%ersionC( C$.$.1C,
CainC( Cser%er.;sC,
CdependenciesC( {
CsocBet.ioC( C$.3.+C
},
Cpri%ateC( CtrueC
}
Ths allows us to nstall the app wth all the dependences usng np install .
In server.js:
%ar fs & re5uire('fs'),
http & re5uire('http'),
sio & re5uire('socBet.io');
%ar ser%er & http.create#er%er(function(re5, res) {
res.write8ead(-$$, { 'FontentDt7pe'( 'te+t/htl'});
res.end(fs.readFile#7nc('./inde+.htl'));
});
ser%er.listen(3$$$, function() {
console.log('#er%er listening at http(//localhost(3$$$/');
});
#3 of 91 9.9.2014 03:2
// Rttach the socBet.io ser%er
io & sio.listen(ser%er);
// store essages
%ar essages & >@;
// "efine a essage handler
io.socBets.on('connection', function (socBet) {
socBet.on('essage', function (sg) {
console.log(',ecei%ed( ', sg);
essages.push(sg);
socBet.broadcast.eit('essage', sg);
});
// send essages to new clients
essages.for'ach(function(sg) {
socBet.send(sg);
})
});
Frst we start a regular HTTP server that always respondes wth the content of
"./ndex.html". Then the Socket.o server s attached to that server, allowng
Socket.o to respond to requests drected towards t on port 8000.
Socket.o follows the basc EventEmtter pattern: messages and connecton state
changes become events on socBet . On "connecton", we add a message handler
that echoes the message back and broadcasts t to all other connected clents.
Addtonally, messages are stored n memory n an array, whch s sent back to
new clents so that the can see prevous messages.
Next, let's wrte the clent page (ndex.html):
*htl6
*head6
*st7le t7pe&Cte+t/cssC6
Jessages { padding( $p+; listDst7leDt7pe( none;}
Jessages li { padding( -p+ $p+; borderDbotto( 1p+ solid Jccc; }
*/st7le6
*script src&Chttp(//code.;5uer7.co/;5uer7D1.4.1.in.;sC6*/script6
*script src&C/socBet.io/socBet.io.;sC6*/script6
*script6
G(function(){
%ar socBet & io.connect();
socBet.on('connect', function () {
socBet.on('essage', function(essage) {
G('Jessages').append(G('*li6*/li6').te+t(essage));
});
socBet.on('disconnect', function() {
G('Jessages').append('*li6"isconnected*/li6');
});
});
%ar el & G('Jchatsg');
G('Jchatsg').Be7press(function(e) {
if(e.which && 11) {
e.pre%ent"efault();
socBet.send(el.%al());
G('Jessages').append(G('*li6*/li6').te+t(el.%al()));
el.%al('');
}
});
#4 of 91 9.9.2014 03:2
});
*/script6
*/head6
*bod76
*ul id&CessagesC6*/ul6
*hr6
*input t7pe&Cte+tC id&CchatsgC6
*/bod76
*/htl6
BTW, "/socket.o/socket.o.js" s served by Socket.o, so you don't need to have a
fle placed there.
To start the server, run node ser%er.;s and pont your browser to
http://localhost:8000/. To chat between two users, open a second tab to the same
address.
Addtonal features
Check out the Socket.o webste (and Gthub for server, clent) for more
nformaton on usng Socket.o.
There are two more advanced examples on Gthub.
I'm gong to focus on deployment, whch has not been covered n depth.
Deployng Socket.o: avodng the problems
As you can see above, usng Socket.o s farly smple. However, there are several
ssues related to deployng an applcaton usng Socket.o whch need to be
addressed.
Same orgn polcy, CORS and JSONP
The same orgn polcy s a securty measure bult n to web browsers. It restrcts
access to the DOM, Javascrpt HTTP requests and cookes.
In short, the polcy s that the protocol (http vs https), host (www.example.com vs
example.com) and port (default vs e.g. :8000) of the request must match exactly.
Requests that are made from Javascrpt to a dfferent host, port or protocol are
not allowed, except va two mechansms:
Cross-Orgn Resource Sharng s a way for the server to tell the browser that a
request that volates the same orgn polcy s allowed. Ths s done by addng a
HTTP header (Access-Control-Allow-Orgn:) and apples to requests made from
Javascrpt.
JSONP, or JSON wth paddng, s an alternatve technque, whch reles on the
fact that the <scrpt> tag s not subject to the same orgn polcy to receve
fragments of nformaton (as JSON n Javascrpt).
# of 91 9.9.2014 03:2
Socket.o supports these technques, but you should consder try to set up you
applcaton n such a way that the HTML page usng Socket.o s served from the
same host, port and protocol. Socket.o can work even when the pages are
dfferent, but t s subject to more browser restrctons, because dealng wth the
same orgn polcy requres addtonal steps n each browser.
There are two mportant thngs to know:
Frst, you cannot perform requests from a local fle to external resources n most
browsers. You have to serve the page you use Socket.o on va HTTP.
Second, IE 8 wll not work wth requests that 1) volate the same orgn polcy
(host/port) and 2) also use a dfferent protocol. If you serve your page va HTTP
(http://example.com/ndex.html) and attempt to connect to HTTPS
(https://example.com:8000), you wll see an error and t wll not work.
My recommendaton would be to only use HTTPS or HTTP everywhere, and to try
to make t so that all the requests (servng the page and makng requests to the
Socket.o backend) appear to the browser as comng from the same host, port
and protocol. I wll dscuss some example setups further below.
Flashsockets support requres TCP mode support
Flash sockets have ther own authorzaton mechansm, whch requres that the
server frst sends a fragment of XML (a polcy fle) before allowng normal TCP
access.
Ths means that from the perspectve of a load balancer, flash sockets do not look
lke HTTP and thus requre that your load balancer can operate n tcp mode.
I would serously consder not supportng flashsockets because they ntroduce yet
another hurdle nto the deployment setup by requrng TCP mode operaton.
Websockets support requres HTTP 1.1 support
It's farly common for people to run ngnx n order to serve statc fles, and add a
proxy rule from ngnx to Node.
However, f you put ngnx n front of Node, then the only connectons that wll work
are long pollng -based. You cannot use Websockets wth ngnx, because
Websockets requre HTTP 1.1 support throughout your stack to work and current
versons of ngnx do not support ths.
I heard from the ngnx team on my blog that they are workng on HTTP 1.1
support (and you may be able to fnd a development branch of ngnx that
supports ths), but as of now the versons of ngnx that are ncluded n most Lnux
dstrbutons do not support ths.
Ths means that you have to use somethng else. A common opton s to use
HAProxy n front, whch supports HTTP 1.1 and can thus be used to route some
#! of 91 9.9.2014 03:2
requests (e.g. /socket.o/) to Node whle servng other requests (statc fles) from
ngnx.
Another opton s to just use Node ether to serve all requests, or to use a Node
proxy module n conjucton wth Socket.o, such as node-http-proxy.
I wll show example setups next.
Sample deployments: scalng
Sngle machne, sngle stack
Ths s the smplest deployment. You have a sngle machne, and you don't want
to run any other technologes, lke Ruby/Python/PHP alongsde your applcaton.
>#ocBet.io ser%er at (3$@
The beneft s smplcty, but of course you are now taskng your Node server wth
a lot of work that t wouldn't need to do, such as servng statc fles and
(optonally) SSL termnaton.
The frst step n scalng ths setup up s to use more CPU cores on the same
machne. There are two ways to do ths: use a load balancer, or use node cluster.
Sngle machne, dual stack, node proxes to second stack
In ths case, we add another technology - lke Ruby - to the stack. For example,
the majorty of the web app s wrtten n Ruby, and real-tme operatons are
handled by Node.
For smplcty, we wll use Node to perform the routng.
>#ocBet.io ser%er at (3$@
DD6 >,ub7 at (1$$$ (not accessible directl7)@
To mplement the route, we wll smply add a proxy from the Socket.o server to
the Ruby server, e.g. usng node-http-proxy or bouncy. Snce ths mostly
app-specfc codng, I'm not gong to cover t here.
Alternatvely, we can use HAProxy:
>8RVro+7 at (3$@
DD6 >,ub7 at (1$$$@
DD6 >#ocBet.io ser%er at (3$$$@
Requests startng wth /socket.o are routed to Socket.o lstenng on port 8000,
and the rest to Ruby at port 3000.
To start HAProxy, run sudo hapro+7 Df path/to/hapro+7.cfg
The assocated HAProxy confguraton fle can be found here for your clonng and
forkng convnence.
#" of 91 9.9.2014 03:2
I've also ncluded a smple test server that lstens on ports :3000 (http) and :8000
(websockets). It uses the same ws module that Socket.o uses nternally, but wth
a much smpler setup.
Start the test server usng node ser%er.;s , then run the tests usng node
client.;s . If HAProxy works correctly, you wll get a "It worked" message from
the clent:
G node client.;s
Testing 8TTV re5uest
,ecei%ed 8TTV response( VIA/. 'I0.
Testing 9# re5uest
,ecei%ed 9# essage a
,ecei%ed 9# essage b
,ecei%ed 9# essage c
,ecei%ed 9# essage d
,ecei%ed 9# essage e
#uccessfull7 sent and recei%ed . essages.
Aow waiting . seconds and sending a last 9# essage.
Ut worBed.
Sngle machne, SSL, dual stack, statc assets served separately
Now, let's offload some servces to dfferent processes and start usng SSL for
Socket.o.
Frst, we wll use ngnx to serve statc fles for the applcaton, and have ngnx
forward to the second technology, e.g. Ruby.
Second, we wll start usng SSL. Usng SSL wll ncrease the complexty of the
setup somewhat, snce you cannot route SSL-encrypted requests based on ther
request URI wthout decodng them frst.
If we do not termnate SSL frst, we cannot see what the URL path s (e.g.
/socket.o/ or /rubyapp/ ). Hence we need to perform SSL termnaton before
routng the request.
There are two optons:
Use Node to termnate SSL requests (e.g. start a HTTPS server).
Use a separate SSL termnator, such as stunnel, stud or specalzed hardware
Usng Node s a neat soluton, however, ths wll also ncrease the overhead per
connecton n Node (SSL termnaton takes memory and CPU tme from
Socket.o) and wll requre addtonal codng.
I would prefer not to have to mantan the code for handlng the request routng n
the Node app - and hence recommend usng HAProxy.
Here we wll use stunnel (alternatve: stud) to offload ths work. Ngnx wll proxy to
Ruby only and Socket.o s only accessble behnd SSL.
>Agin+ at (3$@
DD6 >,ub7 at (1$$$@
## of 91 9.9.2014 03:2
>#tunnel at ())1@
DD6 >8RVro+7 at ()$$$@
DD6 >#ocBet.io at (3$$$@
DD6 >Agin+ at (3$@
DD6 >,ub7 at (1$$$@
Traffc comes n SSL-encrypted to port 443, where Stunnel removes the
encrypton, and then forwards the traffc to HAProxy.
HAProxy then looks at the destnaton and routes requests to /socket.o/ to Node
at port 8000, and all other requests to Ruby/Ngnx at port 3000.
To run Stunnel, use stunnel path/to/stunnel.conf .
The assocated HAProxy and Stunnel confguraton fles can be found here for
your clonng and forkng convnence.
To make connectons to port 443 over SSL, run the connecton tests for the
testng tool usng node client.;s https . If your HAProxy + Stunnel setup works
correctly, you wll get a "It worked" message from the clent.
Multple machnes, SSL, dual stack, statc assets
In ths scenaro, we are deployng to multple machnes runnng Socket.o servers;
addtonally, we have multple machnes runnng the second stack, e.g. Ruby.
For smplcty, I'm gong to assume that a sngle machne s gong to lsten to
ncomng requests and handle load balancng and SSL decrypton for the other
servers.
Non-SSL traffc uses a slghtly dfferent HAProxy confguraton, snce non-SSL
connectons to Socket.o are assumed to be unwanted.
SSL traffc s frst de-encrypted by Stunnel, then forwarded to HAproxy for load
balancng.
>8RVro+7 at (3$@
DD6 ,ound robin to ,ub7 pool
>#tunnel at ())1@
DD6 >8RVro+7 at ()$$$@
DD6 ,ound robin to ,ub7 pool
DD6 #ource UV based sticBiness to #ocBet.io pool
The confguraton here s essentally the same as n the prevous scenaro (you
can use the same confg), but nstead of havng one backend server n each pool,
we now have multple servers and have to consder load balancng behavor.
Load balancng strategy and handshakes
HAproxy s confgured wth the same (URL-based) routng as n the prevous
example, but the traffc s balanced over several servers.
Note that n the confguraton fle, two dfferent load balancng strateges are used.
For the second (non-Socket.o) stack, we are usng round robn load balancng.
#9 of 91 9.9.2014 03:2
Ths assumes that any server n the pool can handle any request.
Wth Socket.o, there are two optons for scalng up to multple machnes:
Frst, you can use source IP based stcky load balancng. Source IP based
stckness s needed because of the way Socket.o handles handshakes:
unknown clents (e.g. clents that were handshaken on a dfferent server) are
rejected by the current (0.8.7) verson of Socket.o.
Ths means that:
n the event of a server falure, all clent sessons must be re-establshed, snce even f
the load balancer s smart enough to drect the requests to a new Socket.o server, that
server wll reject those requests as not handshaken.
1.
load balancng must be stcky, because for example round robn would result n every
connecton attempt beng rejected as "not handshaken" - snce handshakes are
mandatory but not synchronzed across servers.
2.
dong a server deploy wll requre all clents to go through a new handshake, meanng
that deploys are ntrusve to the end users.
3.
Example wth four backend servers behnd a load balancer dong round robn:
>client@ D6 /handshaBe D6 >load balancer@ D6 >ser%er J1@ Xour new #ession id is 1
>client@ D6 /VI#T data (sess id &1) D6 >load balancer@ D6 >ser%er J-@ KnBnown session id, please reconnect
>client@ D6 /handshaBe D6 >load balancer@ D6 >ser%er J1@ Xour new #ession id is -
>client@ D6 /VI#T data (sess id &-) D6 >load balancer@ D6 >ser%er J)@ KnBnown session id, please reconnect
Ths means that you have to use stcky load balancng wth Socket.o.
The second alternatve s to use the Stores mechansm n Socket.o. There s a
Reds store whch synchronzes n memory nformaton across multple servers va
Reds.
Unfortunately, the stores n Socket.o are only a partal soluton, snce stores rely
on a pub/sub API arrangement where all Socket.o servers n the pool receve all
messages and mantan the state of all connected clents n-memory. Ths s not
desrable n larger deployments, because the memory usage now grows across
all servers ndependently of whether a clent s connected to a partcular server
(related ssue on GtHub).
Hopefully, n the future, Socket.o (or Engne.o) wll offer the ablty to wrte a
dfferent knd of system for synchronzng the state of clents accross multple
machnes. In fact, ths s beng actvely worked on n Engne.o - whch wll form
the bass for the next release of Socket.o. Untl then, you have to choose
between these two approaches to scale over multple machnes.
Thank you for readng my book!
If you've made t ths far, thank you.
If you lked the book, follow me on Twtter or on Gthub. I love seeng that I've had
90 of 91 9.9.2014 03:2
some knd of postve mpact.
I'd lke to talk at developer-centrc Node-related events n SF, so get n touch f
you're arrangng one.
91 of 91 9.9.2014 03:2

Introduct On: Asdsaddssd Asdasd

Uploaded by

Copyright:

Available Formats

Introduct On: Asdsaddssd Asdasd

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Introduct On: Asdsaddssd Asdasd

Uploaded by

Copyright:

Available Formats

1.

You might also like