Overview of NodeJS Architecture

15 mins read
NodeJS is a powerful and efficient JavaScript runtime built on Chrome's V8
JavaScript engine. It's designed to execute JavaScript code outside of a web
browser, allowing developers to create server-side applications with
JavaScript. But how does NodeJS actually work under the hood?

Core components

At its core, NodeJS operates on an event-driven, non-blocking I/O model. This

architecture is what makes NodeJS lightweight and efficient, particularly suited
for data-intensive real-time applications that run across distributed devices.
Let's break this down:

1. Event loop: The heart of NodeJS is the event loop. This is a clever
mechanism that allows NodeJS to perform non-blocking I/O operations,
despite JavaScript being single-threaded. When NodeJS needs to
perform an operation, like reading from a file or making a network
request, instead of blocking the entire process and waiting for the
operation to complete, it simply registers a callback and moves on to
the next task. When the I/O operation finishes, the callback is added to a
queue. The event loop constantly checks this queue for new events.
When it finds one, it dequeues the event and executes its associated
callback. This approach allows NodeJS to handle many operations
concurrently without the need for multi-threading.
2. Libuv: Another crucial component of NodeJS architecture is libuv. This
is a multi-platform support library that handles the thread pool,
asynchronous I/O, and the event loop. It's what allows NodeJS to
interface with the operating system, file system, and network in an
asynchronous manner.
3. Built-in modules: NodeJS also includes a rich set of built-in modules
that provide essential functionality. These modules, like 'fs' for file
system operations, 'http' for creating HTTP servers, and 'crypto' for
cryptographic functions, are all built on top of JavaScript but interface
with C++ bindings for performance-critical operations.
This unique architecture allows NodeJS to excel in scenarios where traditional
blocking I/O models struggle. It's particularly well-suited for applications that
require real-time updates, streaming, or handling a large number of
concurrent connections. By leveraging this non-blocking model, NodeJS can
achieve high throughput with a small memory footprint, making it an excellent
choice for scalable network applications.

Understanding this architecture is crucial for developers to write efficient

NodeJS applications and to fully harness the power of asynchronous
programming. With this foundation, let's delve deeper into those core
components of NodeJS and learn about them in details in the following
sections.
Event Loop
15 mins read
There is a big debate and some misunderstanding when it come to JavaScript
- is it synchronous or asynchronous? Let’s start by defining the terms:

Synchronous: Tasks are performed one after another, and each operation
must be finished before the next one starts. Due to this, it can lead to idle
waiting periods and inefficient use of time and resources.

Asynchronous: Tasks can start before other tasks finish. The program can
handle other operations during waiting periods, leading to better utilization of
time/resources and improved performance.

JavaScript is synchronous, blocking and single threaded. That means that JS

runs code line by line and waits for each operation to complete before starting
the next operation. It has a single thread meaning it cannot run two things in
parallel. However, you might have heard that JS can handle asynchronous
tasks - how can it do that if it is sync, blocking and single threaded? This is
where the event loop and JS runtime come into play. JS does not run in
vacuum so it needs a runtime: either the browser runtime or in NodeJS
runtime. It is the features of the runtime, specifically the event loop, that gives
JS its asynchronous capabilities. Even though JS by itself is synchronous, in real
life scenarios it is asynchronous and non-blocking - let’s explore that further.

Imagine you're at a busy restaurant with only one waiter. This waiter is
responsible for taking orders, delivering food, and processing payments. While
it might sound chaotic, the waiter can manage multiple tasks by not sticking
around waiting for each action to complete. This analogy is very similar to how
NodeJS operates.

The call stack is like a to-do list for our waiter. Every time a function is called,
it's added to the top of this stack. The function is then executed, and once
finished, it is removed from the stack. This ensures tasks are processed one at
a time, maintaining order. What if an task in the waiter’s to-do list takes time?
The waiter will start it, write down what he needs to do when the task is done,
and move on to the next task. Similarly, NodeJS will call an async method,
register the callback (the function that needs to be executed when the task is
done), and move on to process other requests.

Now, consider the event loop as the restaurant manager who ensures the
waiter isn't overwhelmed. When the waiter initiates a task that might take a
while, this task is handed over to a different person like the cook, allowing the
waiter to continue with other orders. When the cook finishes his order, the
manager will coordinate the task that needs to be done by the waiter (picking
up the food and serving it) and ask the waiter to process the task whenever he
is free. This analogy holds true for NodeJS.

The Event Loop is a C program that coordinates the execution of synchronous

and asynchronous code in NodeJS and it is what gives JS its async capabilities.
The event loop is alive as long as the NodeJS application is running. It works in
iterations (or loops) where in every iteration, it goes through different queues
that holds different type of callbacks and process them in a specific order.

One very important point to note is that the Event Loop doesn’t start
processing callbacks until all synchronous code is executed. When Node goes
through a function call it gets added to the callstack. If the call is synchronous
it blocks the main thread until it is executed completely and popped off the
stack. If it is an asynchonous call like a file read operation or a network call, the
callback is registered and the async task is started in the background and the
async function is popped off the stack.

When an async task completes, its callback gets added to the appropriate
queue of tasks that needs to be handled by the main thread. The event loops
observes the queue and the main thread call stack to coordinate when the call
back will be executed. The event loop continuously checks if the call stack is
empty and only then, it takes control and start looping through the queue and
process the callbacks by pushing them onto the call stack. This cycle ensures
that asynchronous tasks are handled promptly without blocking the main
thread.
Event Queues
15 mins read
In the previous section we mentioned that Node places callbacks in queues
and that there are different type of queues. A queue is a popular data
structure in programming that operates on the principle of FIFO (First In, First
Out). Similar to how queues in real life work like in a supermarket, where the
first customer entering the queue at the checkout counter will be the first
customer leaving the store. Event Queues is an ordered list of asynchronous
operations where an asynchronous operation is inserted at the end of the
queue and is removed from the front of the queue. This differs from the stack
data structure which operates on the LIFO (Last In, First Out) principle, where
elements are both inserted and removed from the end of the stack.

There are 5 queues that get checked by the event loop in every iteration of the
loop: timer, I/O, check, close, and microtask.We will explore and discuss each of
these in the following modules.

Timer Queue
15 mins read

NodeJS incorporates a Timer module that allows for asynchronous execution

of code after a specified delay. This module includes functions
like setTimeout() and setInterval(), which are essential for handling time-
based operations in NodeJS applications. It's important to note that these
timer functions are not native to JavaScript itself, but rather are provided by
the NodeJS runtime environment.

When a timer function is called, NodeJS doesn't block the execution of other
code. Instead, it registers the callback function and continues with the next
operation. Once the specified time has elapsed, the callback is placed in the
timer queue, ready to be processed by the event loop. This mechanism
enables NodeJS to manage time-related tasks efficiently without impeding the
execution of other parts of the application.
Examples

• setTimeout (timer module):

console.log('Start');
setTimeout(() => {
console.log('Timer callback (event queue)');
}, 0);

console.log('End');

// Output:
// Start
// End
// Timer callback (event queue)
I/O Queue
15 mins read

I/O operations are tasks that require interaction with external devices or
systems, such as reading and writing files or making network requests. These
operations are handled asynchronously by NodeJS, which manages them after
the synchronous code has finished executing. Since JavaScript itself does not
have direct access to the computer's internal systems, it delegates these tasks
to NodeJS to be managed in the background. Once an I/O operation is
complete, it is placed in the I/O callback queue. From there, the event loop
transfers it to the call stack for execution, ensuring that the task is processed
without blocking the main thread.

Examples

• File Read Operation:

const fs = require('fs');

fs.readFile('example.txt', 'utf8', (err, data) => {

if (err) {
console.error('Error reading file:', err);
return;
}
console.log('File content:', data);
});

console.log('This message is logged first, before the file content is

read.');

In this example, the readFile function initiates an asynchronous

operation to read the content of example.txt. While NodeJS handles the
file reading in the background, the rest of the code continues to
execute. Once the file read operation is complete, the callback function
is placed in the I/O queue and eventually executed.
• HTTP Request Operation:
const https = require('https');

https.get('<https://www.example.com>', (response) => {

let data = '';

response.on('data', (chunk) => {

data += chunk;
});

response.on('end', () => {
console.log('Received data:', data);
});
}).on('error', (err) => {
console.error('Error with request:', err);
});

console.log('This message is logged first, before the HTTP response is

received.');

Here, an asynchronous HTTP GET request is made using the

NodeJS https module. While the request is processed in the background,
the main thread remains unblocked. When the response data is fully
received, the callback is placed in the I/O queue, ready to be executed
by the event loop.
Check Queue
15 mins read

The check queue, also known as the immediate queue, is responsible for
executing callback functions immediately after all callbacks in the I/O queue
have been processed. The setImmediate function is used to add functions to
this queue, ensuring they run after the I/O operations have completed.

Example

• setImmediate :

const fs = require('fs');

setImmediate(() => {
console.log('setImmediate');
});

fs.readFile('example.txt', 'utf8', () => {

console.log('readFile');
});

console.log('Hello World!');

In this example, the setImmediate function is called first, adding its callback to
the check queue. However, the program continues to execute
the readFile operation, which is asynchronous and handed over to NodeJS,
while the main thread continues to run the rest of the code.

Next, theconsole.log statement is executed immediately, logging Hello

World! to the console.

Once the readFile operation completes, its callback is placed in the I/O
queue. The event loop then processes the I/O queue before moving on to the
check queue. This means the readFile callback will execute first,
logging readFile to the console. After that, the setImmediate callback in the
check queue is processed, logging setImmediate last.
Close Queue
15 mins read

The close queue is reserved for functions associated with close event listeners.
These events occur when a resource, such as a stream or an HTTP server, is
closed, signaling that no further events will be emitted from that resource.

Examples include:

• The close event on a stream, which is triggered when the stream has
been closed, indicating that no more data will be sent or received.
• The close event on an HTTP server, which is emitted when the server
shuts down.

The close queue is the last queue in the event loop and is processed after all
other queues are processed.

Example

• Stream close:

const fs = require('fs');

const readableStream = fs.createReadStream('example.txt');

readableStream.close();

readableStream.on('close', () => {
console.log('This is from the readableStream close event callback');
});
Microtask Queue
15 mins read

Last, but definitely not least, is the Microtask queue. The Microtask queue is a
special type of queue that handles functions scheduled to run at the earliest
opportunity, ahead of other asynchronous operations. The Microtask queue is
processed at the start of the event loop iteration, but also between phases of
the event loop. This queue is divided into two main parts:

1. process.nextTick Queue:

This queue holds callbacks from process.nextTick function. These

callbacks are given the highest priority and are processed before going
to the Promise Queue.

2. Promise Queue: The second part of the microtask queue deals with
promises. Unlike typical asynchronous operations managed by the I/O
or Timer queues, promises are initially stored as pending in JavaScript
memory. Once the promise operation is completed, NodeJS updates the
promise's state and places the associated callback in the microtask
queue.

As we mentioned earlier, the microtask queue is processed at the start of the

event loop iteration starting with the nextTick queue and then the Promise
queue. However due to the priority given to the microtask queue it is
constantly checked after each phase of the event loop, that means after every
queue is processed and before moving to the next queue, the Microtask
queue is checked first and if there are any callbacks in the queue, they get
processed. Also the microtask is checked between each callback execution
in the timer and check queues.
Examples

1. Using process.nextTick:

console.log('Start');

process.nextTick(() => {
console.log('nextTick callback');
});

console.log('End');

In this example, the function passed to process.nextTick is added to the

microtask nextTick queue and gets executed after the two console log
statements but before any other asynchronous tasks.

2. Using Promises:

console.log('Start');
process.nextTick(() => {
console.log('nextTick callback');
});
Promise.resolve().then(()=> console.log("promise.resolve callback");
console.log('End');

In this example, the callback function passed to the then function is

added to the microtask Promise queue and gets executed after the two
console log statements and after the process.nextTick callback.
Order of Execution
15 mins read

Now that we know the different types of queues, let’s understand how the
Event Loop cycles through these queues. It goes through different phases
within every phase a queue is processed.

• Microtask Phase: Handles callbacks from process.nextTick and

promises
• Timers Phase: Handles the execution
of setTimeout and setInterval callbacks.
• I/O Callbacks Phase: Processes callbacks for I/O operations like
fileRead and network operations.
• Poll Phase: Retrieves new I/O events and add completed operations
callbacks to the I/O queue.
• Check Phase: Executes setImmediate callbacks.
• Close Callbacks Phase: Handles closed events like socket.on('close',
...).

The microtask queue runs after each phase of the event loop, before the
event loop moves on to the next phase and between each callback
execution in the timer and check queues. This is why microtasks have a
higher priority than most of the event loop's phases.

The event loop continues to cycle through these queues as long as there are
tasks to process, maintaining NodeJS's efficient non-blocking behavior.

Here is an example to illustrate the order of execution for the different types
of queues:
const fs = require('fs');

console.log('Start');
// Timers Phase: setTimeout callback
setTimeout(() => {
console.log('setTimeout');

// Adding to the Microtask Queue from within setTimeout

process.nextTick(() => {
console.log('nextTick inside setTimeout');
});

Promise.resolve().then(() => {
console.log('Promise inside setTimeout');
});
}, 0);
// Check Phase: setImmediate callback
setImmediate(() => {
console.log('setImmediate');
// Adding to the Microtask Queue from within setImmediate
process.nextTick(() => {
console.log('nextTick inside setImmediate');
});
Promise.resolve().then(() => {
console.log('Promise inside setImmediate');
});
});
// I/O Callbacks Phase: File read (simulating an I/O operation)
fs.readFile(__filename, () => {
console.log('I/O callback');
// Adding to the Microtask Queue from within I/O callback
process.nextTick(() => {
console.log('nextTick inside I/O callback');
});

Promise.resolve().then(() => {
console.log('Promise inside I/O callback');
});
});
// Close Callbacks Phase: simulating a close callback
const readable = fs.createReadStream(__filename);
readable.close(() => {
console.log('close callback');
});
console.log('End');
Output:

Start
End
setTimeout
nextTick inside setTimeout
Promise inside setTimeout
setImmediate
nextTick inside setImmediate
Promise inside setImmediate
close callback
I/O callback
nextTick inside I/O callback
Promise inside I/O callback

1. Synchronous code:
• console.log('Start'); is executed first as part of the main script.

• console.log('End'); follows immediately.

2. Timers Phase (setTimeout):

• The setTimeout callback is executed: console.log('setTimeout');.
• Inside this setTimeout, we add microtasks:
• process.nextTick adds console.log('nextTick inside
setTimeout'); to the microtask queue.

• A promise adds console.log('Promise inside

setTimeout'); to the microtask queue.

• These microtasks are executed immediately after

the setTimeout callback before moving on to the next phase.
3. Check Phase (setImmediate):
• The setImmediate callback is
executed: console.log('setImmediate');.
• Inside this setImmediate, we add microtasks:
 • process.nextTick adds console.log('nextTick inside
setImmediate'); to the microtask queue.

• A promise adds console.log('Promise inside

setImmediate'); to the microtask queue.

• These microtasks are executed immediately after

the setImmediate callback.
4. Close Callbacks Phase:
• The close callback for the readable stream is
executed: console.log('close callback');.
5. I/O Callbacks Phase:
• The file read operation's callback is executed: console.log('I/O
callback');.

• Inside this callback, microtasks are added:

• process.nextTick adds console.log('nextTick inside I/O
callback');.

• A promise adds console.log('Promise inside I/O

callback');.

• These microtasks are executed immediately after the I/O callback.

You might be wondering why the setImmediate callback was executed before
the I/O callback - this is because the event loop goes through the I/O queue
first but, if it is empty, it polls to check if any I/O operation is completed. If
there is, it gets added to the I/O queue but will only be processed in the next
iteration. The event loop will then proceed to the check queue (after visiting
the microtask queue first) processes the setImmediate callback, hence we get the
console log statement from setImmediate first, then the callback from the close
queue. The event loop then starts a new iteration and go through the exact
same order. However, now that all queues except the I/O queue are empty, it
will only process the callback from the I/O queue, printing the log statement
and adding callbacks to the microtask queue which get processed next. If in
any iteration the I/O queue is not empty it gets processed first before the
check queue.
What is Libuv?
15 mins read
When an asynchronous operation is initiated, such as a database query or file
read, the operation is offloaded to the libuv library, which handles the
operation using the underlying OS mechanisms. Once the operation
completes, the callback associated with it is pushed to the appropriate queue.
The event loop then ensures these callbacks are executed in the right order.
Now you might be wondering what is Libuv?

You might be surprised that NodeJS, a JS runtime, is built on top of libraries

that use C and C++! As fascinating as this is, it is not surprising since JS is not
designed to handle low level operations like reading files, handling network
requests or interacting with databases in an asynchronous manner. These
tasks are inherently system-level operations that involve working directly with
the operating system's kernel and hardware. Writing this kind of low-level,
performance-critical code directly in JavaScript (an interpreted language)
would be inefficient and impractical. C, on the other hand, is a compiled
language that offers fine-grained control over system resources and memory,
making it ideal for this purpose. Libuv is an open-source, cross-platform
library written in C. It is crucial for handling asynchronous operations in
NodeJS by abstracting away the logic required for non-blocking operations.
Libuv achieves this by using a thread pool and the event loop that we
discussed above, which is part of Libuv.

How Libuv Works

Libuv's primary goal is to ensure that JavaScript's main thread is not blocked
by time-consuming tasks. It does this by offloading these tasks to its thread
pool or by using OS kernel (native async mechanisms).
Thread Pool
20 mins read

When JavaScript's single main thread encounters an asynchronous task, it delegates this
task to libuv. Libuv then handles the task using either native async mechanisms or the
thread pool. The default thread pool has 4 threads, meaning it can handle up to 4
asynchronous operations concurrently. If a fifth operation is initiated, it will have to wait
for one of the existing threads to become available.

Types of Tasks Using the Thread Pool

The thread pool in libuv is used for both I/O-intensive and CPU-intensive tasks.

1. I/O-Intensive Tasks:

• DNS: dns.lookup(), dns.lookupService().

• Async File System: All file system APIs except fs.FSWatcher() and those that
are explicitly synchronous.

2. CPU-Intensive Tasks:

• Crypto: crypto.pbkdf2(), crypto.scrypt(), crypto.randomBytes(), crypto.r

andomFill(), crypto.generateKeyPair() .

• Zlib: All zlib APIs except those that are explicitly synchronous.

Adjusting the Thread Pool Size

You can increase the thread pool size to handle more concurrent asynchronous
operations using the process.env object:

process.env.UV_THREADPOOL_SIZE = 6;

However, increasing the thread pool size indefinitely won't necessarily speed up task
execution. This is because libuv tries to distribute threads across the available CPU cores.
For example, with 8 CPU cores, if you increase the thread pool size to 16, each core will
have to handle 2 threads. This increases the CPU's task-switching overhead, potentially
slowing down task completion. Hence, the optimal thread pool size is limited by the
number of available CPU cores.

Network Requests and the Event Loop

Network requests, such as HTTP requests, do not use the thread pool. Instead, they rely
on the event loop and the underlying operating system for non-blocking I/O. When a
network request is made, the event loop handles it without blocking the main thread.
Once the response is ready, the event loop processes the result and executes the
callback. For database queries, they are run in the DB process. From Node's perspective
these are just network calls, and would not block the thread, and would behave like an
HTTP request.