MODULE - 5

CHAPTER 6

Using MongoDB

In this chapter, you’ll learn about MongoDB. The goal is to get rid of the in-memory list
of issues and start using a MongoDB database to add and retrieve issues. To achieve this,
we will need to install MongoDB, learn about how to add to and list records from the
database directly, and then modify the server code to use MongoDB to persist the list of
issues.

MongoDB Basics
This section is an introductory section, where we will not be modifying the application.
We’ll look at the following core concepts in this section: MongoDB, documents, and
collections. Then, we’ll install MongoDB and explore these concepts with examples using
the mongo shell while reading and writing to the database.

Documents
MongoDB is a document database, which means that the equivalent of a record is a
document, or an object. In a relational database, one has to use rows and columns,
whereas in a document database, an entire object can be written as a document.
For simple objects, this may seem no different from a relational database. But let’s
say you have objects with nested objects (called embedded documents) and arrays. Now,
when using a relational database, you typically need multiple tables. For example, an
Invoice object may be stored in a combination of an invoice table and invoice_lines
table in a relational database. In MongoDB, you store the entire Invoice object as one
document.
A document is a data structure composed of field and value pairs. The values of fields
may include other documents, arrays, and arrays of documents. MongoDB documents
are similar to JSON objects, so it is easy to think of them as JavaScript objects. Compared
to a JSON object, a MongoDB document has support not only for the primitive data
types boolean, numbers, and strings, but also other common data types such as dates,
timestamps, regular expressions, and binary data. You can even store JavaScript functions
as document fields.
You will see examples of documents when you explore the mongo shell later in this
chapter.

© Vasan Subramanian 2017 93

V. Subramanian, Pro MERN Stack, DOI 10.1007/978-1-4842-2653-7_6
Chapter 6 ■ Using MongoDB

Collections
A collection is like a table in a relational database. It is a set of documents, and you
access each document via the collection. Just like in a relational database, you can have a
primary key and indexes on the collection. But there are a few differences.
A primary key is mandated in MongoDB, and it has the reserved field name _id.
Even if you don’t supply an _id field when creating a document, MongoDB creates this
field and auto-generates a unique key for every document. More often than not, the auto-
generation is used as is, since it is convenient and guaranteed to produce unique keys
even when multiple clients are writing to the database simultaneously. MongoDB uses a
special data type called the ObjectId for the primary key.
The _id field is automatically indexed. Apart from this, indexes can be created
on other fields, and this includes fields within embedded documents and array fields.
Indexes are used to efficiently access a subset of documents in a collection.
Unlike a relational database, MongoDB does not require you to define a schema
for a collection. The only requirement is that all documents in a collection must have a
unique _id, but the actual documents may have completely different fields. In practice,
though, all documents in a collection do have the same fields. Although a flexible
schema may seem very convenient for schema changes during the initial stages of an
application, this can cause problems if you don’t have some kind of schema checking in
the application code.

Query Language
Unlike the universal English-like SQL in a relational database, the MongoDB query
language is made up of methods to achieve various operations. The main methods for
read and write operations are the CRUD methods. Other methods include aggregation,
text search, and geospatial queries.
All methods operate on a collection and take parameters as JavaScript objects that
specify the details of the operation. Each method has its own specification. For example,
to insert a document, the only parameter you need is the document itself. For querying,
the parameters are a match specification and a list of fields to return.
Unlike relational databases, there is no method that can operate on multiple
collections at once. All methods operate on only one collection at a time. If there is a need
to combine the result of multiple collections, each collection has to be queried separately
and manipulated by the client. In a relational database, you can use joins to combine
tables using fields that are common to the tables, so that the result includes the contents
of both tables. You can’t do this in MongoDB and many other NoSQL databases. This lets
NoSQL databases scale by using shards, or multiple servers to distribute documents part
of the same collection.
Also, unlike relational databases, MongoDB encourages denormalization, that is,
storing related parts of a document as embedded subdocuments rather than as separate
collections (tables) in a relational database. Take an example of people (name, gender,
etc.) and their contact information (primary address, secondary address etc.). In a
relational database, you would have separate tables for People and Contacts, then join the
two tables when you need all of the information together. In MongoDB, on the other hand,
you store the list of contacts within the same People document, thus avoiding a join.

94
 Chapter 6 ■ Using MongoDB

Installation
MongoDB can be installed easily on OS X, Windows, and most distributions based
on Linux. The installation instructions are different for each operating system and
have a few variations depending on the OS flavor as well. Please install MongoDB by
following the instructions at the MongoDB website (https://docs.mongodb.com/
manual/installation/ or search for “mongodb installation” in your search engine).
Choose version 3.2 or higher, preferably the latest, as some of the examples use features
introduced only in version 3.2. Most installation options let you install the server, the
shell, and tools all in one. Check that this is the case; if not, you may have to install them
separately.
After installation, ensure that you have started MongoDB server (the name of the
daemon or service is mongod), if it is not already started by the installation process. Test
the installation by running the mongo shell like this:

$ mongo

On a Windows system, you may need to add .exe to the command. The command
may require a path depending on your installation method. If the shell starts successfully,
it will also connect to the local MongoDB server instance. You should see the version of
MongoDB printed on the console, the database it is connecting to (the default is test),
and a command prompt, like this:

MongoDB shell version: 3.2.4

connecting to: test
>

If, instead, you see an error message, revisit the installation and server starting
procedure.

The mongo Shell

The mongo shell is an interactive JavaScript shell, very much like the Node.js shell. In this
interactive shell, there are a few non-JavaScript conveniences apart from the full power
of JavaScript. In this section, you’ll look at the basic operations that are possible via the
shell, those that are most commonly used. For a full reference of all you can do with the
shell, take a look at the MongoDB documentation.
By default, the shell connects to a database called test. At any time, to know the
current database, use the special command db like this:

> db

It should print the current database, which is by default, test. To connect to another
database, say a playground database, do this:

> use playground

95
Chapter 6 ■ Using MongoDB

Note that a database does not have to exist to connect to it. The first document
creation will initiate the database creation if it doesn’t exist. The same applies to
collections: the first creation of a document in a collection creates the collection. You can
see the proof of this by listing the databases and collections in the current database:

> show databases

> show collections

You will see that playground is not listed in the databases, and the collections
list is empty. Let’s create an employees collection by inserting a document. To insert
a document, you use the insert() method on the collection, which is referred to by a
property with the name of the database, of the special variable db:

> db.employees.insert({name: {first: 'John', last: 'Doe'}, age: 44});

Now, if you list the databases and collections, you will find both playground and
employees listed. Let’s also make sure that the first employee record has been created. To
list the contents of a collection, you need to use the find() method of the collection:

> db.employees.find();
{ "_id" : ObjectId("57b1caea3475bb1784747ccb"), "name" : { "first" : "John",
"last" : "Doe" }, "age" : 44 }

You can see that _id was automatically generated and assigned. If you wanted a
prettier, indented listing of employees, you should use the pretty() method on the
results of find() like this:

> db.employees.find().pretty()
{
        "_id" : ObjectId("57b1caea3475bb1784747ccb"),
        "name" : {
                "first" : "John",
                "last" : "Doe"
        },
        "age" : 44
}

Now, insert a few more documents with different names and ages. Add a middle
name for someone, like this:

> db.employees.insert({name: {first: 'John', middle: 'H', last: 'Doe'}, 

age: 22});

96
 Chapter 6 ■ Using MongoDB

This is what a flexible schema lets you do: you can enhance the schema whenever
you discover a new data point that you need to capture, without having to explicitly
modify the schema. In this case, it is implicit that any employee document where the
middle field under name is missing indicates an employee without a middle name. If,
on the other hand, you added a field that didn’t have an implicit meaning when absent,
you’d either have to handle the absence in the code, or run a migration to modify all
documents and add the field with a default value.
Note that MongoDB automatically generated the primary key for each of the
documents, which is displayed as ObjectId("57b1caea3475bb1784747ccb") in the
find() output. Just to reiterate, the ObjectId is a special data type, which is why it is
displayed like that. You can convert an ObjectId to and from strings, which you’ll see a
little later.
The insert() method can take in an array when inserting multiple documents
together. The variations insertOne() and insertMany() were introduced in version 3.2 to
make it more explicit whether the parameter is a single document or an array.
To retrieve only some documents that you’re interested in, you need to supply a filter
to find(). The filter specification is an object where the property name is the field that you
want to filter on, and the value is its value that you want to match. Say you want to find all
employees aged 44; this is what you would do:

> db.employees.find({age: 44});

The output would be similar to the output of find() without filters as described
previously. The filter is actually a shortcut for age: {$eq: 44}, where $eq is the operator.
Other operators for comparison are available, such as $gt for greater than. If you need to
compare and match fields within embedded documents, you can refer to field using the
dot notation (which will require you to specify quotes around the field name). If there are
multiple field specifications, all of them have to match.
A second parameter can be passed to restrict the fields that are returned. This is
called the projection. The format of this specification is an object with one or more field
names as the key and the value as 0 or 1, to indicate exclusion or inclusion. Unfortunately,
you cannot combine 0s and 1s: you can only start with nothing and include all the fields
using 1s, or start with everything and exclude fields using 0s. The _id field is an exception;
it is always included unless you specify a 0. The following will find employees whose first
name is John, aged 44 or more, and print only their first names and ages:

> db.employees.find({'name.first': 'John', age: {$gte: 44}}, 

{'name.first': 1, age: 1})

The method findOne() is a variation that returns a single document rather than an
cursor that can be iterated over.

97
Chapter 6 ■ Using MongoDB

In order to update a document, you first need to specify the filter that matches the
document to update, and then specify the modifications. The filter specification is the
same as for a read operation. Typically, the filter is the ID of the document so that you are
sure you update one and only one document. You can replace the entire document by
supplying the document as the second parameter. If you want to only change a few fields,
you do it by using the $set operator, like this:

> db.employees.update({_id: ObjectId("57b1caea3475bb1784747ccb")}, 

{$set: {age: 44}})

You identify the document to modify using its primary key, _id. In order to generate
the special data type ObjectId, you need to use the mongo shell’s built-in function
ObjectId() and pass it a string representation of the ID. The update results in a message
like this (the output may vary depending on the version of MongoDB you have installed):

WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })

The update() method can take more options, one of which is the upsert option.
When this is set to true, MongoDB will find the document based on the filter, but if it
doesn’t find one, it will create one based on the document supplied. Of course, this isn’t
useful if you are using the object ID to identify the object. It’s useful if the key to search for
is a unique key (like the employee number). Also, the second parameter in this case must
be the entire document; it can’t be a patch specification using $set.
The variations updateMany() and updateOne() were introduced in version 3.2 to
make it explicit as to the intention of the update. Use of these variations is recommended
over the plain update() since even if the filter matches multiple or single documents, the
update will affect one or many documents depending on which method was called.
To delete a document, use the remove method with a filter, just as in the find
method:

> db.employees.remove({"_id" : ObjectId("57b1caea3475bb1784747ccb")})

WriteResult({ "nRemoved" : 1 })

If you think a field is often used to filter the list, you should create an index on
the field to make the search more efficient. Failing this, MongoDB searches the entire
database for a match. To create an index on the age field, do this:

> db.employees.createIndex({age: 1})

98
 Chapter 6 ■ Using MongoDB

Shell Scripting
A mongo shell script is a regular JavaScript program, with all the collection methods
available as built-ins. One difference from the interactive shell is that you don’t have
the convenience commands such as use and the default global variable db. You must
initialize them within the shell script programmatically, like this:

var db = new Mongo().getDB("playground");

Add a few more statements in a script file, the same that you typed in the mongo shell
for inserting and reading documents. To execute the script, supply it as a parameter to the
mongo shell like this (if you have saved the file as test.mongo.js):

$ mongo test.mongo.js

EXERCISE: MONGO SHELL

1. Does the mongo shell support ES2015 features? Hint: Try out a
few ES2015 features in the mongo shell.
2. Write a simple statement to retrieve all employees who have
middle names. Hint: Look up the MongoDB documentation for
query operators.
3. Is the filter specification a JSON? Hint: Think about date objects
and quotes around field names.
4. Say an employee’s middle name was set mistakenly, and you
need to remove it. Write a statement to do this. Hint: Look up
the MongoDB documentation for update operators.
5. During index creation, what did the 1 indicate? What other
valid values are allowed? Hint: Look up the MongoDB indexes
documentation.
Answers are available at the end of the chapter.

Schema Initialization
Since MongoDB does not enforce a schema, there is really no such thing as a schema
initialization as you may do in other databases. The only thing you really want to do is
create indexes that will prove useful for often used filters in the application. While we’re
at it, let’s also initialize the database with some sample records to ease your testing.

99
Chapter 6 ■ Using MongoDB

Let’s create a mongo shell script called init.mongo.js and place it in a new scripts
directory in the project directory. We need to write the initialization statements, which
should include setting up the db variable; removing all existing issues, if any; inserting a
few sample records; and creating indexes. Listing 6-1 shows the contents of the script.

Listing 6-1. init.mongo.js: DB Initialization Script

db = new Mongo().getDB('issuetracker');

db.issues.remove({});

db.issues.insert([
  {
    status: 'Open', owner: 'Ravan',
    created: new Date('2016-08-15'), effort: 5, 
completionDate: undefined,
    title: 'Error in console when clicking Add',
  },
  {
    status: 'Assigned', owner: 'Eddie',
    created: new Date('2016-08-16'), effort: 14, 
completionDate: new Date('2016-08-30'),
    title: 'Missing bottom border on panel',
  },
]);

db.issues.createIndex({ status: 1 });

db.issues.createIndex({ owner: 1 });
db.issues.createIndex({ created: 1 });

The only notable thing we did differently from what I discussed in the previous
section is use an array to insert multiple records at once in the insert method. The array
of documents is a copy of what we had in server.js, except that we removed the field
called id. Run this script from the command line like this:

$ mongo scripts/init.mongo.js

It should run without any errors. To check the effect of the script, open up the
mongo shell, list all documents using find(), and list all indexes using getIndexes().
The find() should return the two documents that you inserted, with auto-generated
ObjectIDs in the _id field. And, getIndexes() should list four indexes: the three that we
created on status, owner, and created, and the auto-created index on _id.

100
 Chapter 6 ■ Using MongoDB

EXERCISE: SCHEMA INITIALIZATION

1. We let MongoDB generate the _id primary key. What are the
pros and cons of generating the primary key ourselves instead?
2. Are there any other indexes that may be useful? Hint: What if
you needed a search input field in the application?
Answers are available at the end of the chapter.

MongoDB Node.js Driver

This is the Node.js driver that lets you connect and interact with the MongoDB server. It
is, as you probably guessed, an npm module. It provides methods very similar to what you
saw in the mongo shell, but not exactly the same.
Another option is to use the mongoose client, which is an object-document-mapper
as discussed in Chapter 1. But since I want you to explore the nuts and bolts of how the
native driver works, which works similar to the shell commands, let’s use the native driver
itself.
To start, let’s install the driver:

$ npm install mongodb --save

To connect to the database from a Node.js program, you call the connect method
on the MongoClient object provided by the module. The mongodb module exports many
functions and objects, of which MongoClient gives you the ability to act as a client,
mainly, the connect method. The parameter to the connect function is a URL-like string
starting with mongodb:// followed by the server name, and then the database name
separated by a /. You can optionally include a port after the server name, separated by a
:, but if you’re using the default port, this can be skipped.
Once you acquire a connection, to get a handle to any collection, you need to call
its collection() method. To this method, you supply the name of the collection as the
parameter to indicate which collection. Then, the CRUD operation methods can be called
on the handle to the collection. For example, to connect to a database called playground
and fetch some documents from a collection called employees, you do the following:

...
const MongoClient = require('mongodb').MongoClient;

MongoClient.connect('mongodb://localhost/playground', function(err, db) {

  db.collection('employees').find().toArray(function(err, docs) {
    console.log('Result of find:', docs);
    db.close();
  });
});
...

101
Chapter 6 ■ Using MongoDB

In the above, a find() returns a cursor which you could iterate over. Calling
toArray() on the cursor runs through all the documents and makes an array out of
them. It calls the callback when the array is ready to be processed, passing the array as a
parameter to the callback.
To insert a document, you need to use the insertOne() method on the collection,
and pass it one parameter: the object to be inserted. The result of an insert contains
various things, one of which is the new _id that has been generated, in a property called
insertedId.
Note that all calls to the driver are asynchronous calls, which means that you don’t
get the result of the call as a return value to the function call. In the above example, you
supplied a callback to each of the MongoDB driver methods. When the result is ready,
these callbacks will be called. The MongoDB driver documentation gives you three
different paradigms for dealing with this asynchronous nature: the callbacks paradigm,
one using promises, and another using the co module and generator functions. Let’s
explore these three options, and one more option using the async module, which is not
mentioned in the driver documentation.
Let’s do all this in a JavaScript program called trymongo.js so that you have a ready
test if and when required. Let’s initialize this file with a test wrapper that we will use to
exercise each of the paradigms. We’ll use a command line argument to the program to
call a different function for each of the paradigms. Command line arguments to Node.js
programs are available in the array process.argv. The first two elements in the array are
node and the name of the program. All user arguments follow after these two. The initial
contents of the file are shown in Listing 6-2.

Listing 6-2. trymongo.js: Initial Contents, Command Line Arguments Handling

'use strict';
const MongoClient = require('mongodb');

function usage() {
  console.log('Usage:');
  console.log('node', __filename, '<option>');
  console.log('Where option is one of:');
  console.log('  callbacks   Use the callbacks paradigm');
  console.log('  promises    Use the Promises paradigm');
  console.log('  generator   Use the Generator paradigm');
  console.log('  async       Use the async module');
}

if (process.argv.length < 3) {
  console.log("Incorrect number of arguments");
  usage();
} else {
  if (process.argv[2] === 'callbacks') {
    testWithCallbacks();
  } else if (process.argv[2] === 'promises') {
    testWithPromises();

102
 Chapter 6 ■ Using MongoDB

  } else if (process.argv[2] === 'generator') {

    testWithGenerator();
  } else if (process.argv[2] === 'async') {
    testWithAsync();
  } else {
    console.log("Invalid option:", process.argv[2]);
    usage();
  }
}

We gave a name and associated a function with each of the paradigms: Callbacks,
Promises, Generator, and Async. We will fill in the functions in the following subsections.

Callbacks
The conventional and oldest way to deal with asynchronous calls is to provide a callback
to handle the result of the operation. As seen in the above example, it is minimal
JavaScript. You pass a callback to the method, with the first parameter of the callback
expecting any errors, and the second (or more) parameters expecting the result of the
operation.
Callbacks are easy to understand and reason about. They work in conventional ES5
JavaScript as well. They have very little constructs that you need to learn. The callback
paradigm is not particular to the MongoDB driver. There are many APIs, including the
core Node.js library, that follow this paradigm.
Listing 6-3 shows how to get a database connection, use the connection to insert into
a collection, and then use the result of the insert operation to retrieve the object. As you
can see, one problem with this paradigm is that it can get deeply nested and complicated,
depending on the depth of the chain: the result of one operation being passed to the next.

Listing 6-3. trymongo.js, testWithCallbacks: Callbacks Paradigm

function testWithCallbacks() {
  MongoClient.connect('mongodb://localhost/playground', function(err, db) {
    db.collection('employees').insertOne({id: 1, name: 'A. Callback'}, 
function(err, result) {
      console.log("Result of insert:", result.insertedId);
      db.collection('employees').find({id: 1}).toArray(function(err, docs) {
        console.log('Result of find:', docs);
        db.close();
      });
    });
  });
}

103
Chapter 6 ■ Using MongoDB

It can get even more deeply nested when you also have to handle errors (or other)
conditions: you’ll soon find yourself writing the same set of statements multiple times
(imagine db.close() in every error condition). This is often referred to as callback hell.
The only remedy if you want to stick to the callback paradigm is to split each small piece
of code into its own function and pass that function as a parameter to a call, chaining the
callback along.

Promises
Using ES2015 promises, the nesting can be avoided, and the chaining can become
seemingly sequential. The above can be written as shown in Listing 6-4, using the
promises paradigm.

Listing 6-4. trymongo.js, testWithPromises: Promises Paradigm

function testWithPromises() {
  let db;
  MongoClient.connect('mongodb://localhost/playground').then(connection => {
    db = connection;
    
return db.collection('employees')....insertOne({id: 1, name: 'B. Promises'});

  }).then(result => {
    console.log("Result of insert:", result.insertedId);
    return db.collection('employees').find({id: 1}).toArray();

  }).then(docs => {
    console.log('Result of find:', docs);
    db.close();

  }).catch(err => {
    console.log('ERROR', err);
  });
}

The result of every call is a promise, on to which you attach a then, which returns
another promise, and so on. Finally, there is a catch block that consumes all errors.
Assuming all calls throw errors, you’ll find that error handling isn’t needed in each
individual block: just one final catch() for errors at any stage is enough.

Generator and co Module

ES2015 introduces generator functions, which can be exited temporarily and called again.
The temporary exits are done using the yield statement. Between multiple calls, the
function retains the execution state. These functions are declared using an asterisk after
the function keyword, like function*().

104
 Chapter 6 ■ Using MongoDB

A module called co takes advantage of ES2015 generators and promises to make

asynchronous calls look sequential. It achieves this by asking you to sequence the
asynchronous calls within one function. Then, the co module makes multiple calls to this
function, where each asynchronous step temporarily exits the function.
To try this paradigm, let’s first install the co module:

$ npm install co

We did not use --save because we will not be using this module other than for trying
out this paradigm. The previous example can be rewritten using the co module as shown
in Listing 6-5.

Listing 6-5. trymongo.js, testWithGenerator: Generator Paradigm with co Module

function testWithGenerator() {
  const co = require('co');
  co(function*() {
    const db = yield MongoClient.connect('mongodb://localhost/playground');

    const result = yield db.collection('employees')...insertOne({id: 1, 

name: 'C. Generator'});
    console.log('Result of insert:', result.insertedId);

    const docs = yield db.collection('employees').find({id: 1}).toArray();

    console.log('Result of find:', docs);

    db.close();
  }).catch(err => {
    console.log('ERROR', err);
  });
}

As you can see, every asynchronous call was preceded by the keyword yield. This
causes a temporary return from the function, after which the function can be resumed
where it left off, if called again. The co module does the repeated calling, which is why we
needed to wrap the function around co().

The async Module

Yet another way to manage callbacks is by using the async module, though this method
is not mentioned in the MongoDB driver documentation. To start, let’s install the
async module (again without --save because we will not be using this paradigm in the
application, only trying it out in this section.)

$ npm install async

105
Chapter 6 ■ Using MongoDB

Apart from many other useful utilities for managing asynchronous calls, this module
provides a method called waterfall, which lets you pipe the result of one asynchronous
call to another. This method takes an array of functions to run. Each function is passed
the results of the previous function, and a callback (which takes an error and results as its
parameters). Each function in the array must call this callback when it is done. The results
are passed through as a waterfall from one function to the next, that is, the outputs of one
are passed to the next.
Since all the driver methods follow the same callback convention of error and results,
it’s easy to pass the callbacks through the waterfall. Listing 6-6 demonstrates this.

Listing 6-6. trymongo.js, testWithAsync: Async Paradigm

function testWithAsync() {
  const async = require('async');
  let db;
  async.waterfall([
    next => {
      MongoClient.connect('mongodb://localhost/playground', next);
    },
    (connection, next) => {
      db = connection;
      db.collection('employees').insertOne({id: 1, name: 'D. Async'}, next);
    },
    (insertResult, next) => {
      console.log('Insert result:', insertResult.insertedId);
      db.collection('employees').find({id: 1}).toArray(next);
    },
    (docs, next) => {
      console.log('Result of find:', docs);
      db.close();
      next(null, 'All done');
    }
  ], (err, result) => {
    if (err)
      console.log('ERROR', err);
    else
      console.log(result);
  });
}

Only in the last function did we explicitly call the callback with a null error and
a string as the final result. In all of the other function calls, we just passed the callback
through to the MongoDB driver method. The driver function calls the callback when the
results are available. We also named the callback next, just to be clear that it will be the
next function in the array that will be called when done.

106
 Chapter 6 ■ Using MongoDB

Choosing any one of the paradigms is a matter of taste and familiarity. All of them are
valid choices; it is really up to you as to which one you pick. For the purpose of the Issue
Tracker application, I am going to pick the promises paradigm, mainly because it does
not depend on any other module. Also, it lets you perform parallel tasks if you choose to
do so for certain operations.

Reading from MongoDB

Let’s first modify the List API to read from the database instead of the in-memory list of
issues on your server. Since we’ve initialized the database with a set of initial issues, you
should be able to test this easily.
To start, we have to include the MongoDB driver that we have already installed, in
server.js. Next, we need to connect to the MongoDB server and keep the connection
open for future calls to the database. We do this rather than acquire a connection in every
request because creating a connection takes a bit of time. Also, the MongoDB driver
keeps a pool of connections automatically and reuses them if you reuse the connection
handle. So, we’ll just save the connection in a global variable called db for later use.
We’ll also start the Express server only once we get the connection. Let’s call the
database issuetracker. Listing 6-7 shows the connection acquisition and modified
server startup section in server.js.

Listing 6-7. server.js: Modified Initialization Sequence with MongoDB connection

...
const MongoClient = require('mongodb').MongoClient;
...
let db;
MongoClient.connect('mongodb://localhost/issuetracker').then(connection => {
  db = connection;
  app.listen(3000, () => {
    console.log('App started on port 3000');
  });
}).catch(error => {
  console.log('ERROR:', error);
});
...

Now, we can modify the endpoint handler for /api/issues to read from the
database. All we need to do is call a find() on the issues collection, convert it to an
array, and return the documents returned by this call. Listing 6-8 shows the modified
endpoint handler.

107
Chapter 6 ■ Using MongoDB

Listing 6-8. server.js: List API Modified to Use MongoDB

app.get('/api/issues', (req, res) => {
  db.collection('issues').find().toArray().then(issues => {
    const metadata = { total_count: issues.length };
    res.json({ _metadata: metadata, records: issues })
  }).catch(error => {
    console.log(error);
    res.status(500).json({ message: `Internal Server Error: ${error}` });
  });
});

■■Caution Never skip the catch block when using promises. If you do so, any runtime
error within any of the blocks will not be caught and will be silently ignored. If you don’t
have a catch block, you may fail to “catch” errors in your code, even those such as a typo in
one of the variable names.

Since we have changed the field name id to _id, the front-end code referring to id
also needs to change. There are two places in App.jsx where this has to be done. Listing
6-9 shows the changed lines, with the change highlighted in bold.

Listing 6-9. App.jsx: Front-End Changes, Replacing id with _id

...
    <td>{props.issue._id}</td>
...
  const issueRows = props.issues.map(issue =><IssueRow 
key={issue._id} issue={issue} />)
...

Finally, now that even the List API can return a non-successful HTTP Status code,
let’s make sure that we handle that in the front end, just like we did for the Create API, by
looking at response.ok. The modified method loadData() is show in Listing 6-10.

Listing 6-10. App.jsx, IssueList loadData(): Error Handling

loadData() {
  fetch('/api/issues').then(response => {
    if (response.ok) {
      response.json().then(data => {
        console.log("Total count of records:", data._metadata.total_count);
        data.records.forEach(issue => {
          issue.created = new Date(issue.created);
          if (issue.completionDate)
            issue.completionDate = new Date(issue.completionDate);
        });

108
 Chapter 6 ■ Using MongoDB

        this.setState({ issues: data.records });

      });
    } else {
      response.json().then(error => {
        alert("Failed to fetch issues:" + error.message)
      });
    }
  }).catch(err => {
    alert("Error in fetching data from server:", err);
  });
}

Now, the changes can be tested. Refresh the browser, and the two issues that we
initialized using the mongo shell script should be displayed. The only change is that the
ID is now a long string instead of what used to be 1 and 2. Of course, adding a new issue
won’t work, which we’ll deal with in the next section.

EXERCISE: READING FROM MONGODB

1. We are saving the connection in a global variable. What happens
when the connection is lost? Restart MongoDB server to see
what happens. Does the connection still work?
2. Shut down the MongoDB server, refresh the browser, and then
start the server and see what happens. Try the sequence again;
this time wait for a much longer time before starting the server.
What happens? Hint: Look up the documentation of MongoDB
driver, look for the tutorial on connection failures.
3. We used toArray() to convert the entire list of issues into an
array. What if the list is too big, say, a million documents? How
would you deal with this?
Answers are available at the end of the chapter.

Writing to MongoDB
You saw how to write to MongoDB using the insert method on a collection. We’ll use that
method to create a new record in the Create API. We will need to, as in the MongoDB driver
trial, read back the object that was just created and return it as the result of the API call.
The find() call to read back the object is guaranteed to return a single document
since you are using the _id as the filter criterion. In such a case, it’s recommended that
you do not use toArray(); instead, you use the next() method on the cursor returned by
find(). Another alternative is to use the findOne() method. This is just to prevent a large
array being created in memory inadvertently and also to make it clear that you only want
one document and not an array of one document.

109
Chapter 6 ■ Using MongoDB

The modified Create API is shown in Listing 6-11.

Listing 6-11. server.js: Modified Create API That Writes to MongoDB

app.post('/api/issues', (req, res) => {
  const newIssue = req.body;
  newIssue.id = issues.length + 1;
  newIssue.created = new Date();
  if (!newIssue.status)
    newIssue.status = 'New';

  const err = validateIssue(newIssue)

  if (err) {
    res.status(422).json({ message: `Invalid request: ${err}` });
    return;
  }

  db.collection('issues').insertOne(newIssue).then(result =>
    db.collection('issues').find({ _id: result.insertedId }).limit(1).next()
  ).then(newIssue => {
    res.json(newIssue);
  }).catch(error => {
    console.log(error);
    res.status(500).json({ message: `Internal Server Error: ${error}` });
  });
});

Since MongoDB will now generate the ID for each issue, we removed the line
where we assigned the field id based on the array length. We also need to remove the
validation for the field id. All we have to do is remove the corresponding property in the
issueFieldType array. Finally, we can also get rid of the in-memory array, now that we
are saving the list in a database. These two changes are shown in Listing 6-12.

Listing 6-12. server.js: Clean up Validations and Remove In-Memory Array

...
const issueFieldType = {
  id: 'required',
  status: 'required',
...
const issues = [
  {
    id: 1, status: 'Open', owner: 'Ravan',
    created: new Date('2016-08-15'), effort: 5, completionDate: undefined,
    title: 'Error in console when clicking Add',
  },

110
 Chapter 6 ■ Using MongoDB

  {
    id: 2, status: 'Assigned', owner: 'Eddie',
    created: new Date('2016-08-16'), effort: 14, completionDate: new
Date('2016-08-30'),
    title: 'Missing bottom border on panel',
  },
];
...

Testing this set of changes will show that new issues can be added, and even on a
restart of the Node.js server, or the database server, the newly added issues are still there. As
a cross-check, use the mongo shell to look at the collection after every change from the UI.

EXERCISE: WRITING TO MONGODB

1. Could we have just added the _id to the passed-in object, and
returned that instead of doing a find() for the inserted object?
2. We could have used insert() instead of insertOne(). What
would be the difference? Why did we choose to use the latter?
Hint: Look up the MongoDB driver documentation.
Answers are available at the end of the chapter.

Summary
In this chapter, we looked at the installation and use of MongoDB, both via the mongo
shell as well as from Node.js code via the native driver. We explored a few methods, the
pillars of MongoDB via the shell as well as a test program in Node.js. We then used these
learnings to insert and read a document from the database, thus making the issue list
persistent.
You should now have a good understanding of the fundamentals of MongoDB and
the paradigms that it uses. We did not go deeper into other operations such as update
and delete, or getting summary reports via aggregates. We’ll do all of that later, when we
implement features in the Issue Tracker that require them to be used.
In the next chapter, we’ll take a break from implementing features. Instead, we’ll start
getting organized. We’ll modularize the code and use tools to improve our productivity
since the project is starting to get bigger.

111
Chapter 6 ■ Using MongoDB

Answers to Exercises
Exercise: Mongo Shell
1. Yes, the mongo shell does support some ES2015 features.
Notably, you can use arrow functions, string interpolation,
and const variables.
2. This can be done using the $exists operator like this:
> db.employees.find({"name.middle": {$exists: true}})

3. The filter specification is not a JSON document, because it

is not a string. It is a regular JavaScript object, which is why
you are able to skip the quotes around the property names.
You will also be able to have real Date objects as field values,
unlike a JSON string.
4. The $unset operator in an update can be used to unset a field
(which is actually different from setting it to null). Here is an
example:
> db.employees.update(({_id: ObjectId("57b1caea3475bb1784747ccb")},
{"name.middle": {$unset: null}})
Although we supplied null as the value for $unset, this value
is ignored. It can be anything.
5. The 1 indicates an ascending sort order for traversing the index.
-1 is used to indicate a descending sort order. This is useful only
for compound indexes, because a simple index on one field can
be used to traverse the collection in either direction.

Exercise: Schema Initialization

1. Creating our own _id would make the display and
identification of the issue more user friendly, because it can be
a continuous number like 1, 2, 3, and so on. The ObjectId is
harder to refer to. On the other hand, generating a continuous
number is more work, and may not scale well when there are
a large number of documents and simultaneous inserts from
multiple clients. Read the MongoDB tutorial called Create
an Auto-Incrementing Sequence Field for a more detailed
discussion.
2. A search bar is quite helpful when searching for issues. A text
index (an index based on the words) on the title field would
be useful in this case. Find out more about text indexes in the
MongoDB tutorial. We’ll implement a text index towards the
end of the book.

112
 Chapter 6 ■ Using MongoDB

Exercise: Reading from MongoDB

1. The connection object is in fact a connection pool. It
automatically figures out the best thing to do: reuse an
existing TCP connection, reestablish a new connection when
the connection is broken, etc. Using a global variable (at least,
reusing the connection object) is the recommended usage. If
you don’t like global variables, you can create a module and
encapsulate the connection within it’s namespace.
2. If the database is unavailable for a short period (30 seconds),
the driver waits and reconnects when the database is available
again. If the database is unavailable for a longer period, the
read throws an error. The driver is also unable to reestablish
a connection when the database is restored. The application
server needs to be restarted in this case.
If you’d rather throw an error if the database is unreachable,
set bufferMaxEntries to 0 when creating the connection. This
is perhaps the preferred approach for a single server. When
you have a replica set, the default behavior works better.
3. One option is to use limit() on the result to limit the return
value to a maximum number of records. For example,
find().limit(100) returns the first 100 documents. If you
were to paginate the output in the UI, you could also use the
skip() method to specify where to start the list.
Another option is to use forEach() or stream() to stream the
result back to the client, without occupying too much memory
on the server to hold the entire array. This assumes that the
client can handle large results.

Exercise: Writing to MongoDB

1. Adding the _id and returning the object passed in would
have worked, so long as you know for a fact that the write was
a success and the object was written to the database as is. In
most cases, this would be true, but it’s a good practice to get
the results from the database, as that is the ultimate truth.
Since we are querying on the _id (which is automatically
indexed), the extra call is very minimal in terms of latency
added to the query.
2. The result from an insert() is slightly different from
insertOne(). Instead of insertedId, it returns an array
of insertedIds. You use insertOne() since insert() is
deprecated in favor of insertOne() and insertMany(), and
the usage is unambiguous.

113
CHAPTER 7

Modularization and Webpack

In this chapter, we’ll take a break from regular coding and adding features. Instead, we’ll
get a bit organized so that the application can grow bigger, yet be manageable. The focus
will be on development tools.
The goal of this chapter is to be able to split the code into multiple files, both on the
server and the client side. Despite this, we should be able to continue the development
process as before: automatically restart or rebuild whenever files change, except that this
would be any file. Further, we’ll even get the browser to refresh automatically when files
change during development. Finally, we’ll add checks to verify that the code we write
follows some standards and good practices, and catches possible bugs earlier than the
testing cycle.

Server-Side Modules
You saw in the Hello World chapter how to include modules in a Node.js file after
installing the module using npm install. But what about your own modules? How do
you write a file that can be included in another file?
It’s surprisingly simple to do this. There is a special variable that Node.js looks for
called module.exports within any file. If that file is used in a require statement, Node.js
just uses the value of this module.exports variable, and returns the same in the require
call. Thus, anything that you want to be available to be exported, you can just add to the
module.exports object.
Let’s aim to separate out code related to the Issue object (or model) into its own file.
To start with, let’s create a directory called server where we will keep all server-side files,
since there is going to be more than one file going forward. Let’s move server.js into
this directory. We’ll also create a new file called issue.js and cut/paste all of the issue
validation-related code into this file. We need to add 'use strict'; at the beginning of
the file to let Node.js accept const declarations. Finally, we define module.exports as an
object with the property validateIssue set to the function validateIssue.

© Vasan Subramanian 2017 115

V. Subramanian, Pro MERN Stack, DOI 10.1007/978-1-4842-2653-7_7
Chapter 7 ■ Modularization and Webpack

The contents of the new issue.js file are shown in Listing 7-1.

Listing 7-1. issue.js: New File for the Issue Model

'use strict';

const validIssueStatus = {
  New: true,
  Open: true,
  Assigned: true,
  Fixed: true,
  Verified: true,
  Closed: true,
};

const issueFieldType = {
  status: 'required',
  owner: 'required',
  effort: 'optional',
  created: 'required',
  completionDate: 'optional',
  title: 'required',
};

function validateIssue(issue) {
  for (const field in issueFieldType) {
    const type = issueFieldType[field];
    if (!type) {
      delete issue[field];
    } else if (type === 'required' && !issue[field]) {
      return `${field} is required.`;
    }
  }

  if (!validIssueStatus[issue.status])
    return `${issue.status} is not a valid status.`;

  return null;
}

module.exports = {
  validateIssue: validateIssue
};

■■Note The first line, 'use strict';, is important. Without it, the behavior of const and
let are different in Node.js version 4.5. Ensure that you have not missed this line.

116
 Chapter 7 ■ Modularization and Webpack

To use this new module in server.js, we must include the module that we just
created, using the (by now familiar) require statement. When you refer to your own
modules rather than modules installed via npm, you need to tell Node.js the path of the
module’s file rather than just the name. So, in this case, we must use './issue.js' rather
than a plain 'issue'. The changes are shown in Listing 7-2.

Listing 7-2. server.js: Changes to Using issue.js Module

...
const MongoClient = require('mongodb').MongoClient;
const Issue = require('./issue.js');
...
app.post('/api/issues', (req, res) => {
...
   const err = Issue.validateIssue(newIssue);
...

We could have chosen to assign module.exports directly to validateIssue (like

module.exports = validateIssue), in which case, the return value of the require
statement in server.js would have been the function itself, and we could use it as is (like
const validateIssue = require('./issue.js');). But we chose to use an object to
enclose the exported function. The expectation is that we may export more things related
to the Issue object later, and they can all be inside this object.
Finally, to let nodemon watch all the files in the new server directory, we need to
modify the start command in package.json. Now that we have a directory for all of the
server-side code, the -w argument can take the directory server as the value instead of a
single file. Further, the starting point is server/server.js rather than simply server.js.
This change is listed in Listing 7-3.

Listing 7-3. package.json: Changes to Start Script

...
    "start": "nodemon -w server.js server.js",
    "start": "nodemon -w server server/server.js",
...

If you restart the server (by pressing Ctrl-C in the console where it was started, and
running npm start again), you should be able to test your changes. You need to do this
because package.json itself changed, and it needed to be reloaded. The application
should function as before, because we have only refactored the code. We have not
introduced any functional changes.

Introduction to Webpack
Traditionally, one would split client-side JavaScript code into multiple files, and include
them all (or whichever are required) using <script> tags in the main HTML file. This
is less than ideal because the dependency management is done by the developer, by
maintaining a certain order of files in the HTML file. Further, when the number of files
becomes large, this becomes unmanageable.

117
Chapter 7 ■ Modularization and Webpack

Tools such as webpack and browserify provide alternatives that let you define
dependencies as you would in a Node.js application using require or equivalent
statements. They automatically figure out not just your own dependent modules, but
also third-party libraries. Then, they put together these individual files into one or more
bundles of pure JavaScript that has all the required code that can be included in the
HTML file.
The only downside is that this requires a build step. But then, we already have a
build step to transform JSX and ES2015 into plain JavaScript. It’s not much of a change in
habit to let the build step also create a bundle based on multiple files. Both webpack
and browserify are good tools and can be used to achieve the goals. But I chose webpack,
because it is simpler to get all that we want done, which includes separate bundles for
third-party libraries and our own modules. It has a single pipeline to transform, bundle,
and watch for changes and generate new bundles as fast as possible.
If you choose Browserify instead, you will need other task runners such as gulp or
grunt to automate watching and include multiple transforms. This is because Browserify
does only one thing: bundle. In order to combine bundle and transform (using babel) and
watch for file changes, you need something that puts all of them together, and gulp is one
such utility. In comparison, webpack (with a little help from loaders, which we’ll explore
soon) can not only bundle, but can also do many more things such as transforms and
watching for changes to files. You don’t need additional task runners to use webpack.
Note that webpack can also handle other static assets such as CSS files. It can even
split the bundles such that they can be loaded asynchronously. We will not be exercising
those aspects of webpack; instead, we’ll focus on the goal of being able to modularize the
client-side code, which is mainly JavaScript at this point in time.

Using Webpack Manually

To get used to what webpack really does, we’ll use webpack from the command-line just
like we did for the JSX transform using babel command line. Let’s first install webpack:

$ npm install --save-dev webpack

We installed webpack locally because, just like with babel, we’ll eventually move all
commands into commands defined in package.json, so that they can be run using npm
run. We don’t need webpack globally. Let’s see what webpack does. You can run it on the
client-side JavaScript file App.js and produce a bundle called app.bundle.js like this:

$ node_modules/.bin/webpack static/App.js static/app.bundle.js

118
 Chapter 7 ■ Modularization and Webpack

You can see that webpack creates app.bundle.js, which is not very different from
App.js itself. Note also that we didn’t run it against the React file App.jsx, because
webpack cannot handle JSX natively. What webpack did in this bundling is hardly
interesting. We did it just to make sure we’ve installed it correctly and are able to run it.
To start the modularization exercise, let’s split App.jsx into two files by separating
out one component, IssueAdd. Let’s create a new file called IssueAdd.jsx under the src
directory, and move the entire IssueAdd class to this file. To include the new file,
we could use the require style imports and exports as in the server-side modules. But
ES2015 supports a new style with import statements, which are easier to read than
require statements. We could not take advantage of this style for the server side-code
because Node.js does not support this style natively as of the latest version at the time of
writing this book.
Using the new ES2015 style to export a class is as simple as prefixing export before
the class definition. Further, you can add the keyword default if you are exporting a
single class, and you want it to be the result of an import statement directly (or a top-level
export).
Listing 7-4 shows the changes to the class definition. The rest of the new file is the
original contents of the class unchanged.

Listing 7-4. IssueAdd.jsx: New File, Class Contents Moved Here from App.jsx
...
export default class IssueAdd extends React.Component {
  ...
}

To import this class in App.jsx, we need to use an import statement right at the
beginning of the file. This, as well as the removal of the IssueAdd class, is shown in
Listing 7-5

Listing 7-5. App.jsx: Move IssueAdd Class and Import It

...
import IssueAdd from ‘./IssueAdd.js’;
...
class IssueAdd extends React.Component {
  constructor() {
    super();
    this.handleSubmit = this.handleSubmit.bind(this);
  }
  ...
}
...

119
Chapter 7 ■ Modularization and Webpack

Let’s run the JSX transformation using npm run compile (or let the babel watch
automatically recompile upon detecting changes). This will transform both files, since
we’d specified the entire src directory as input source to babel. You’ll see that two files,
App.js and IssueAdd.js, are generated. Now, when you run webpack with the same
command as before, you will notice that it automatically includes IssueAdd.js in the
bundle, without you ever telling it do so. Here is a sample result of a webpack run:

Time: 88ms
        Asset     Size  Chunks             Chunk Names
app.bundle.js  11.7 kB       0  [emitted]  main
   [0] ./static/App.js 7.06 kB {0} [built]
   [1] ./static/IssueAdd.js 2.9 kB {0} [built]

You could create many more such files, and webpack does not need to be told
about any of them. That’s because it looks at the import statements within each of the
JavaScript files and figures out the dependency tree. Finally, we need to change the script
file referenced in index.html to the new app.bundle.js instead of App.js. This change is
shown in Listing 7-6.

Listing 7-6. index.html: Replace App.js Script with app.bundle.js

...
<script src="/app.bundle.js"></script>
...

At this point, you should test the application to see if it works as before. There will be
some residual temporary files, App.js and IssueAdd.js, in the static directory, which
you will not require any longer, so they can be removed.

Transform and Bundle

In the previous step, we had to manually transform JSX files, and then use webpack to
bundle them together. The good news is that webpack is capable of combining these two
steps, obviating the need for intermediate files. But it can’t do that on its own; it needs
some helpers called loaders. All transforms and file types other than pure JavaScript
require loaders in webpack. These are separate modules.
In this case, we need the babel loader to handle JSX transforms, so let’s install it:

$ npm install --save-dev babel-loader

Using loaders in the command line is cumbersome, especially if you want to pass
parameters to the loaders (babel, in this case, needs presets react and es2015). You
can instead use a configuration file to give these parameters to webpack. The default
configuration file that webpack looks for is called webpack.config.js, so let’s create that
file.

120
 Chapter 7 ■ Modularization and Webpack

Webpack loads this configuration file as a module and gets its parameters from this
module. Everything has to be under one exported object. Since we won’t be transforming
this file, let’s use the module.exports syntax rather than the ES2015 export syntax.
Inside the exported object, webpack looks for various properties. We’ll use the properties
entry and output to replace what we did with the command line parameters until now.
The entry point is now App.jsx rather than App.js. The output property takes path and
filename as two subproperties.
To add the babel loader, we need to define the configuration property
module.loaders and supply it with an array of loaders. We will have only one element,
the babel loader in this array. A loader specification includes a test (regular expression) to
match files, the loader to apply (in this case, babel-loader), and finally a set of options to
the loader specified by the property query.
The final contents of the configuration file are shown in Listing 7-7.

Listing 7-7. webpack.config.js: Configuration File for webpack

module.exports = {
  entry: './src/App.jsx',
  output: {
    path: './static',
    filename: 'app.bundle.js'
  },
  module: {
    loaders: [
      {
        test: /\.jsx$/,
        loader: 'babel-loader',
        query: {
          presets: ['react','es2015']
        }
      },
    ]
  }
};

The option for babel loader is an array of presets, very similar to the babel command
line:

...
        query: {
          presets: ['react','es2015']
        }
...

The import statement in App.jsx referred to IssueAdd.js, but this file will no longer
be created because webpack will transform IssueAdd.jsx to IssueAdd.js on the fly. So,
for the dependency tree to be built correctly, we will have to import the pretransformed
file with the jsx extension, as shown in Listing 7-8.

121
Chapter 7 ■ Modularization and Webpack

Listing 7-8. App.jsx: Import the jsx File Instead of js

...
import IssueAdd from './IssueAdd.jsx';
...

Now, webpack can be run on the command line without any command line
parameters, like this:

$ node_modules/.bin/webpack

You will notice that it takes quite a while to complete. You don’t want to wait this
long every time you modify a source file, do you? Nor do you want to run this command
over and over again. Fortunately, just like babel, webpack has a watch mode, which looks
for changes and processes only the changed files. So, run this command instead:

$ node_modules/.bin/webpack --watch

Modify some text (create a syntax error, for instance) and ensure that it rebuilds
the bundle on every change, and note how long it takes to do this. Watch how errors are
reported, too. Now is a good time to modify the npm script commands for building and
watching for changed files. Let’s replace the compile and watch script specifications in
package.json, as shown in Listing 7-9.

Listing 7-9. package.json: New Compile and Watch Scripts

...
    "compile": "webpack",
    "watch": "webpack --watch",
...

Now that you know what webpack is capable of, let’s organize your files into one
file per class: IssueList, IssueAdd, and IssueFilter. Let the stateless components
IssueTable and IssueRow remain with IssueList in the same file. The entry file, App.jsx,
will only import the other classes and mount the main component. This will create a
two-level hierarchy of imports: App.jsx will import IssueList.jsx, which in turn will
import IssueAdd.jsx and IssueFilter.jsx.
To start with, let’s move the placeholder class IssueFilter to its own file. This is
shown in in Listing 7-10.

Listing 7-10. IssueFilter.jsx: Move Class IssueFilter Here and Add Export
export default class IssueFilter extends React.Component {
  render() {
    return (
      <div>This is a placeholder for the Issue Filter.</div>
    )
  }
}

122
 Chapter 7 ■ Modularization and Webpack

Similarly, let’s take the class IssueAdd from the file App.jsx and create a new file
called IssueAdd.jsx. Listing 7-11 shows the partial contents of the new file: the class as
such is unchanged, except that in the class declaration you have the keywords export
default.

Listing 7-11. IssueAdd.jsx: New File, Move Class IssueAdd Here, and Add Export
export default class IssueAdd extends React.Component {
  constructor() {
    super();
    this.handleSubmit = this.handleSubmit.bind(this);
  }
  ...
}

Listing 7-12 shows another change, but here we extract three classes (IssueRow,
IssueTable, and IssueList) from App.jsx. The new file created is called IssueList.jsx.
Since this file needs the other extracted classes, IssueAdd and IssueFilter, we also need
import statements for them in addition to the classes that are moved into this file. Only
the class IssueList is exported, so we need to add the keywords export default only
to that class declaration. The other two classes are for internal use only, so they are not
exported.

Listing 7-12. IssueList.jsx: Move Classes IssueList, IssueTable, and Issue Row Here, and
Add Import and Export
import IssueAdd from './IssueAdd.jsx';
import IssueFilter from './IssueFilter.jsx';

const IssueRow = (props) => (

  <tr>
    ...
  </tr>
)

function IssueTable(props) {
  const issueRows = props.issues.map(issue => <IssueRow 
key={issue._id} issue={issue} />)
  return (
    ...
  );
}

123
Chapter 7 ■ Modularization and Webpack

export default class IssueList extends React.Component {

  constructor() {
    super();
    this.state = { issues: [] };

    this.createIssue = this.createIssue.bind(this);
  }
  ...
}

Now that most of the contents of App.jsx have been moved out to their individual
files, we’re left with just the rendering of the component. The contents of this file are
shown in Listing 7-13.

Listing 7-13. App.jsx: Complete Contents After Moving All Classes Out
import IssueList from './IssueList.jsx';

const contentNode = document.getElementById('contents');

ReactDOM.render(<IssueList />, contentNode);   // Render the component inside 
the content Node

If you run npm run watch, you will find that all the compilations are done and the
bundle is ready for you to test the application. If not, just start the npm start and npm
run watch commands in different consoles, and then test the application to ensure that it
behaves the same as before.

EXERCISE: TRANSFORM AND BUNDLE

1. Save any JSX file with only a spacing change, while running npm
run watch. Does webpack rebuild the bundle? Why not?

2. Why did we separate the mounting of the component and the

component into different files? Hint: Think about what other
pages you’ll need in the future.
3. What would happen if you did not use the default keyword
while exporting a class? Hint: Revisit how you had choices
for validateIssue to be exported. Also look up the Babel
documentation and/or Mozilla Developer Network (MDN)
documentation on JavaScript export statement.
Answers are available at the end of the chapter.

124
 Chapter 7 ■ Modularization and Webpack

Libraries Bundle
For the server side, we started by importing third-party libraries, and then we created
modules from our own code. For the client-side code, we have only used modules of our
own code until now.
But we do have library dependencies, notably React itself. We included them as
scripts from a CDN. In this section, we’ll use webpack to create a bundle that includes
these libraries. If you remember, I discussed that npm is used not only for server-side
libraries, but also client-side ones. What’s more, webpack understands this and can deal
with client-side libraries installed via npm.
So, let’s first install, using npm, the client-side libraries that we have used until now.
This is the same list as the list of <script>s in index.html.

$ npm install --save-dev react react-dom whatwg-fetch babel-polyfill

Next, to use these installed libraries, let’s them in all the client-side files where they
are needed, just like we imported our own files. Listings 7-14 to 7-17 show these changes.

Listing 7-14. App.jsx: Imports for Using React and ReactDOM

...
import React from 'react';
import ReactDOM from 'react-dom';
...

Listing 7-15. IssueAdd.jsx: Imports for Using React

...
import React from 'react';
...

Listing 7-16. IssueFilter.jsx: Imports for Using React

...
import React from 'react';
...

Listing 7-17. IssueList.jsx: Imports for Using React and Fetch

...
import React from 'react';
import 'whatwg-fetch';
...

125
Chapter 7 ■ Modularization and Webpack

You can now remove all the script references from index.html, as shown in Listing 7-18.

Listing 7-18. index.html: Remove Scripts Loaded from CDN

...
<script src="https://cdnjs.cloudflare.com/ajax/libs/react/15.2.1/react.
js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/react/15.2.1/react-dom.
js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/fetch/1.0.0/fetch.min.
js"></script>
...

Note that unlike React and React-DOM, fetch is imported into the global namespace,
because it is expected to be a substitute for window.fetch(), which should be available in
the browser in any case. If you have npm run watch already running, you will notice that
the number of hidden modules has shot up from four to more than a hundred, and also
that the size of app.bundle.js has increased from few kBs to more than 1Mb. The console
output of webpack before this step would have looked like this:

Hash: 8f5de02c8f9e672b4ccb
Version: webpack 1.13.2
Time: 10593ms
        Asset     Size  Chunks             Chunk Names
app.bundle.js  13.8 kB       0  [emitted]  main
    + 4 hidden modules

And after the change, it will look like this:

Hash: 983fd62bdc8309eb9fd5
Version: webpack 1.13.2
Time: 11886ms
        Asset     Size  Chunks             Chunk Names
app.bundle.js 1.02 MB       0  [emitted]  main
    + 472 hidden modules

The fact that the bundle includes all of the libraries is a minor problem. The libraries
will not change often, but the application code will, especially during the development
and testing. Even when the application code undergoes a small change, the entire bundle
is rebuilt, and therefore, a client will have to fetch (the now big) bundle from the server.
We’re not taking advantage of the fact that the browser can cache scripts when they are
not changed. This not only affects the development process, but even in production,
users will not have the optimum experience.

126
 Chapter 7 ■ Modularization and Webpack

A better alternative is to have two bundles, one for the application code and another
for all the libraries. It turns out that a plugin called CommonsChunkPlugin, a webpack
built-in plugin, can do this quite easily. The modified webpack.config.js is shown in
Listing 7-19.

Listing 7-19. webpack.config.js: Separate Vendor Bundle Configuration

const webpack = require('webpack');

module.exports = {
  entry: './src/App.jsx',
  entry: {
    app: './src/App.jsx',
    vendor: ['react','react-dom','whatwg-fetch'],
  },
  output: {
    path: './static',
    filename: 'app.bundle.js'
  },
  plugins: [
    new webpack.optimize.CommonsChunkPlugin('vendor','vendor.bundle.js')
  ],
  module: {
    loaders: [
      {
        test: /\.jsx$/,
        loader: 'babel-loader',
        query: {
          presets: ['react','es2015']
        }
      },
    ]
  }
};

First, we specified two entry points to webpack in the configuration file, one for
the app and the other for the libraries. Thus, the configuration variable entry is now
an object with two keys: one for the app and the other for the libraries, which we
called vendor.
Next, we added the plugin in a new configuration variable called plugins. The first
line, where we require webpack, is needed because the plugin is available as part of the
webpack module. We passed in two parameters to the plugin, the first being the name of
the entry identifier, and the second its output file name.

127
Chapter 7 ■ Modularization and Webpack

You must reload the configuration file by restarting npm run watch. You’ll now see
that there are two bundles being created. You may see something similar to the following
console output:

Hash: 82e5bb4d86c63757adb8
Version: webpack 1.13.2
Time: 13826ms
           Asset     Size  Chunks             Chunk Names
   app.bundle.js  13.8 kB       0  [emitted]  app
vendor.bundle.js     1 MB       1  [emitted]  vendor
   [0] multi vendor 52 bytes {1} [built]
    + 472 hidden modules

Further, if you modify any source file, you’ll see that only the application bundle is
being rebuilt, that too in much lesser time.

Hash: d8ef5154f4ffe1780984
Version: webpack 1.13.2
Time: 231ms
        Asset     Size  Chunks             Chunk Names
app.bundle.js  13.8 kB       0  [emitted]  app
    + 473 hidden modules

Now, let’s include the vendor bundle in index.html, so that the contents are
available to the rest of the code. This change is shown in Listing 7-20.

Listing 7-20. index.html: Including the Vendor Bundle Script

...
<script src="/vendor.bundle.js"></script>
<script src="/app.bundle.js"></script>
...

Refresh the browser and check whether vendor.bundle.js is getting reloaded in

the browser. Using the developer console, you should see a 304 response for the vendor
bundle, which means that it is not fetching the bundle if it is already there in the browser’s
cache. This should happen even when you change any source code and regenerate the
application bundle, and that’s the advantage of splitting the bundle in two.

128
 Chapter 7 ■ Modularization and Webpack

Hot Module Replacement

The watch mode of webpack works well for client-side code, but there is potential pitfall
with this approach. You must keep an eye on the console that is running the command
npm run watch to ensure that bundling is complete, before you refresh your browser to
see the effect of your changes. If you hit the refresh a little too soon, you will end up with
the previous version of your client-side code, and scratch your head wondering why your
changes didn’t work, and then spend time debugging.
The webpack-dev-server comes in handy here. Not only can it watch for changes,
but it can speed up delivery of the bundle by keeping the bundle completely in memory,
not writing it to disk. But the best part is that it makes the browser wait until the bundle is
ready, thus ensuring that if you have changed any client-side code, you are guaranteed to
not be served the previous version, saving some scratches on your head.
Let’s start by installing the webpack-dev-server:

$ npm install --save-dev webpack-dev-server

Since it is part of webpack, the webpack-dev-server understands webpack.config.

js, and produces app.bundle.js and vendor.bundle.js according to the specifications
in there. You just need to give it a couple parameters, the first of which is the port on
which it will listen as a server. The second parameter is to tell it where static files other
than the bundles reside. In our case, we need it to serve index.html from the static
directory. Thus, we can run webpack-dev-server as follows:

$ node_modules/.bin/webpack-dev-server --port=8000 --content-base=static

Open up a new window or tab in your browser and go to http://localhost:8000/.

Note that now you are using the port of the webpack dev server (8000), not the Express
web server (3000). If you make a change in any of the client-side code, you will find that
while the bundling is in process, the browser is made to wait as opposed to being given a
previous version of the bundles. But you will also find that although the assets are loaded,
the REST API calls fail, because webpack-dev-server has no clue how to handle them.
The original URL, http://localhost:3000/, continues to work in the other browser
window or tab. This is because the APIs and the static files are both still available via this
application, which is hosted by the Express server. What we want is the APIs to be served
from the Express server and the static content and the bundles to be served from the
webpack-dev-server. But we want it all from a single server.
The solution is to tell webpack-dev-server to act as a proxy for API requests, and
forward them to the Express server. The webpack-dev-server is capable of doing this,
but setting this up is a little involved, so it’s better done in a configuration file rather than
command line parameters. The webpack-dev-server looks into a special section called
devServer in webpack.config.js for its parameters. Let’s configure the proxy here, and
also move the command line parameters into this configuration. The partial changes are
shown in Listing 7-21.

129
Chapter 7 ■ Modularization and Webpack

Listing 7-21. webpack.config.js: Webpack-dev-server Configuration Added

...
  },
  devServer: {
    port: 8000,
    contentBase: 'static',
    proxy: {
      '/api/*': {
        target: 'http://localhost:3000'
      }
    }
  }
...

The first two parameters, port and contentBase, were replacements for the
command line. The proxy configuration tells webpack-dev-server to look for requests
matching '/api/*' and forwards them to your Express server, which runs at the original
URL at port 3000. Let’s restart webpack-dev-server, this time without any command line
parameters because the configuration is now available as part of webpack.config.js.

$ node_modules/.bin/webpack-dev-server

Now, when you refresh the browser page pointing to the 8000 port page, you will find
that the API calls are also working. If you make a change in any of the client-side code,
you will also see the webpack-dev-server rebuilding the bundles. This is great, but the
webpack-dev-server is capable of some more magic: hot module replacement (HMR).
HMR automatically causes a browser refresh whenever there is a change in the
client-side bundle. In fact, it incrementally replaces only the modules that have changed,
making it very efficient in terms of network data transfer. This is really useful for impatient
programmers. To enable HMR, all we have to do is add a couple of command line
parameters to the webpack-dev-server invocation:

$ node_modules/.bin/webpack-dev-server --hot --inline

Now, when you refresh the browser (for the last time, I may say), you will see the
following in the browser’s console log:

[HMR] Waiting for update signal from WDS...

[WDS] Hot Module Replacement enabled.

If you change any client-side code, you will see more log messages in the console,
and you will find that the bundle is rebuilt, and the browser automatically refreshes and
gets the new changes!
There’s still one small caveat, though: you will find that the browser is refreshing
rather than hot replacing the module. If you look at the console log (you may need to
preserve log across requests), you will also see a message saying it’s unable to replace.

130
 Chapter 7 ■ Modularization and Webpack

That is because your client-side code that is responsible for dealing with modules is yet to
accept HMR.
To enable this, we need a change in the topmost module, that is, App.jsx, that
accepts HMR. The new App.jsx is shown in in Listing 7-22.

Listing 7-22. App.jsx: Modified to Accept HMR

import React from 'react';
import ReactDOM from 'react-dom';

import IssueList from './IssueList.jsx';

const contentNode = document.getElementById('contents');

ReactDOM.render(<IssueList />, contentNode);   // Render the component inside 
the content Node

if (module.hot) {
  module.hot.accept();
}

Now, you should be able to see that the browser doesn’t do a full refresh. There is a
good chance that you’ll mistakenly type the familiar http://localhost:3000 instead of
your new 8000 port that webpack dev server is running on. To prevent this, it’s best if you
delete the files app.bundle.js and vendor.bundle.js so that they don’t get served by the
3000 server, and an error will be immediately apparent.
Finally, let’s replace your watch command in package.json to use webpack-dev-
server rather than just bundle using webpack. This change is listed in Listing 7-23.

Listing 7-23. package.json: Change Watch Script to use webpack-dev-server

...
    "watch": "webpack-dev-server  --hot --inline",
...

EXERCISE: HOT MODULE REPLACEMENT

1. How can you tell the difference between a browser refresh and
a hot module replacement? Can you compare the performance
of the two? Hint: Use the Network section of your browser’s
developer tools and watch what goes on.
2. Do you think webpack-dev-server is useful even without HMR?
Hint: What problems does the dev server HMR solve? Which of
them is damaging?
Answers are available at the end of the chapter.

131
Chapter 7 ■ Modularization and Webpack

HMR Using Middleware

Wouldn’t it be great if you didn’t have to run two servers and could thus avoid the
confusion with the ports? It turns out that there is an alternative to using webpack-dev-
server, by wiring in the HMR pieces within the Express code itself. This will make it a
single server that serves your APIs, and at the same time watches for changes to client-
side code, rebundles it, and sends incremental updates to the client.
The way to do this is to install HMR as middleware in the Express server. As
discussed earlier, middleware is something that can intercept requests and do some
special processing. The webpack-dev-middleware does the same work as it did as an
independent webpack-dev-server: it looks for changes to files, and blocks requests to the
bundle until the changes are incorporated into the bundle. The webpack-hot-middleware
does a similar thing for HMR. Let’s make these two middleware functions available by
installing the respective modules:

$ npm install --save-dev webpack-dev-middleware webpack-hot-middleware

We’ll use these modules within the Express server, but within an if condition that
does it only in a development environment. We’ll have to import these modules and
initialize them with a webpack configuration. Most of the configuration is already there
in webpack.config.js, but we need a few changes to the configuration for it to be used in
an HMR environment. They are the following:
1. We need additional entry points (other than App.jsx) so that
webpack can build the client-side code necessary for this extra
functionality into the bundle.
2. We need to add a plugin (which got enabled using the --hot
command line for the webpack-dev-server) that generates
incremental updates rather than entire bundles that can be
sent to the client.
We’ll make these changes by patching the configuration on the fly within the
Express server. But we need a few changes in webpack.config.js that will not affect the
production configuration, yet will allow us to patch in the changes. The changes required
are shown in Listing 7-24.

Listing 7-24. webpack.config.js: Compatible Changes to Allow Patching

...
    app: './src/App.jsx',
    app: ['./src/App.jsx'],
...
    path: './static',
    path: __dirname + './static',
...

132
 Chapter 7 ■ Modularization and Webpack

The first change is to use an array instead of a single entry point. This will let us add
more entry points on the fly. The next change is to use an absolute path name for the
assets output directory name. This is a requirement for the middleware, but doesn’t affect
the webpack command line. Now we can make the changes in the Express server that
initializes the HMR middleware. These changes are shown in in Listing 7-25, which can
be inserted before the first app.get() call.

Listing 7-25. server.js: Use HMR via Middleware for Development Convenience
...
if (process.env.NODE_ENV !== 'production') {
  const webpack = require('webpack');
  const webpackDevMiddleware = require('webpack-dev-middleware');
  const webpackHotMiddleware = require('webpack-hot-middleware');

  const config = require('../webpack.config');

  config.entry.app.push('webpack-hot-middleware/client',
'webpack/hot/only-dev-server');
  config.plugins.push(new webpack.HotModuleReplacementPlugin());

  const bundler = webpack(config);

  app.use(webpackDevMiddleware(bundler, { noInfo: true }));
  app.use(webpackHotMiddleware(bundler, { log: console.log }));
}
...

The first three lines in the block load the newly installed modules. Then, we
loaded up the default webpack configuration using another require statement, just as
webpack itself would have. Note that this can be done thanks to the fact that webpack
configurations are modules, not JSON files. Then we appended additional entry points
and plugins that are required for HMR. Finally, we created a bundler using the new
options, and passed it to the two middleware instantiations. To install the middleware,
we use app.use().
For HMR to work, just as in the previous section, you have to accept HMR in
App.jsx, as shown in Listing 7-22 in the previous section. With these changes, the server
should have automatically restarted if you were running npm start because nodemon
was watching for server file changes. Now, go back to the 3000 port server, without the
webpack-dev-server running. You need a manual browser refresh for the first time to kick
off HMR, and then, for every new client side change, you can see that modules are being
hot replaced automatically in the browser.

Comparison of HMR Alternatives

At first, I liked the middleware approach because it tied in nicely with the Express server,
and I did not have to run two consoles and monitor the restarts on each. Further, it did
not require another server and a listening port, and there was no question of making a
mistake by connecting to the wrong server and scratching heads.

133
Chapter 7 ■ Modularization and Webpack

But a problem with the middleware approach is that a change in the server-side code
causes the entire front-end bundle to be rebuilt, which takes approximately 13 seconds
on my environment. This is quite a long wait, and quite unnecessary too, since I did not
touch any client-side code. If there were a way to cache the bundles and not rebuild them
on every restart, the middleware option would be ideal.
Since there is no way to do that (at least as of writing this book), I have chosen to
revert back to the webpack-dev-server approach during my development. It does the
job: you don’t need to monitor the bundling and refresh the browser (because you’re
guaranteed to be not given stale bundles), so the extra console occupied for this can
remain hidden. As for mistakenly connecting to the wrong server, I have ensured that I
don’t ever run npm run compile, and I frequently check that the bundles don’t exist in
the file system.
I encourage you to make your own choice. Depending on how often you plan to
change the server-side code, you may find the middleware alternative more convenient,
unlike me.

Debugging
Until the previous chapter, if you needed to do any debugging on the client side, you
probably used console.log statements. Adding breakpoints using the developer tool
of your browser could have worked too, but you would have noticed that the generated
JavaScript code is quite different from the JSX we wrote. Creating a single bundle from
four different source files makes it even worse.
Fortunately, webpack solves this problem by its ability to give you source maps,
things that contain your original source code as you typed it in. The source maps also
connect the line numbers in the transformed code to your original code. Browsers’
development tools automatically understand source maps and correlate the two, letting
you put breakpoints in your code and converting them to breakpoints in the transformed
or transpiled code.
It’s quite simple to enable source maps. The small change to webpack.config.js is
listed in Listing 7-26.

Listing 7-26. webpack.config.js: Add Source Map Configuration

...
  devtool: 'source-map'
...

You need to restart npm run watch because of the configuration change. Note now
that two maps are generated by webpack apart from the two bundles. This can be seen in
the initial output of webpack-dev-server (you may need to scroll up in your console), or
when making changes to client-side code. Now, if you open up your Resources section in
your browser’s development tool, you will find the JSX files also listed, within which you
can set breakpoints. A sample development tool screen is shown in in Figure 7-1.

134
 Chapter 7 ■ Modularization and Webpack

Figure 7-1. Developer console showing source files

Server-Side ES2015
Even though Node.js supports many ES2015 features, it is not complete. For example,
you still need to use require instead of import to include other modules. Further, Node
4.5 does not support some useful things like destructuring assignments and spread
operators. On the client side, we assumed full ES2015 support and used these features. It
would be good to have the server side also support the same set, so that we don’t have to
be conscious of this fact when switching between client-side and server-side code. If we
do that, our coding style and features used can be uniform across the board.
Let’s start using the import and export style of modularization. Listing 7-27 shows
the changes to server.js, which includes removal of ‘use strict’ as well.

Listing 7-27. server.js: Replace Require with Imports

...
'use strict'

const express = require('express');

const bodyParser = require('body-parser');
const MongoClient = require('mongodb').MongoClient;
const Issue = require('./issue.js');
...
import express from 'express';
import bodyParser from 'body-parser';
import { MongoClient } from 'mongodb';
import Issue from './issue.js';
...

135
Chapter 7 ■ Modularization and Webpack

Listing 7-28 shows changes to issue.js, where we replace the module.exports with
a simple export default, just like we did for the client-side code, as well as the removal
of use strict, which is no longer required due to ES2015.

Listing 7-28. issue.js: Changes for Using ES2015 Export

...
'use strict'
...
export default {
  validateIssue: validateIssue
};
...

Let’s also create a directory where the compiled files will be saved. Let’s call this
directory dist (short for distribution), and try out a transform using babel, as we did in
the Hello World chapter, to transform all the JSX:

$ node_modules/.bin/babel server --presets es2015 --out-dir dist

Now, instead of running server.js from the original directory, you can run it
from the dist directory. Try it out manually to ensure that the server starts up just fine.
If you inspect the compiled files, you will notice that the files are fully ES5 compliant,
that is, you will find vars instead of consts, and you will see that arrow functions have
been converted to regular functions. That’s great, but this means that you are not taking
advantage of the fact that Node.js natively supports constructs such as consts and arrow
functions.
Fortunately, there are other babel presets available that let you use what is available
in Node.js and change only unsupported ES2015 features. Depending on the version of
Node.js, the presets are named differently, corresponding to the version. For Node.js 4.5,
it’s called babel-preset-es2015-node4. Let’s now install this and use the preset instead
of the default es2015 preset:

$ npm install --save-dev babel-preset-es2015-node4

For Node.js version 6, it is babel-preset-es2015-node6. Further, the compilation

only takes care of the new syntax of ES2015, but does not provide new objects and
methods that are part of the ES2015 standard library. For example, you will find that the
method values() is missing in any array. To enable these objects and methods as you did
in the client-side code, we need to include babel-polyfill once in the app. Let’s include
that in the server. The changes are listed in Listing 7-29.

Listing 7-29. server.js: Include ES2015 Polyfill

...
import 'babel-polyfill';
...

136
 Chapter 7 ■ Modularization and Webpack

Because of the compilation, all errors will now be reported as if they occurred at the
line number in the compiled file, not the source file. For a better reporting, we need to
generate source maps and tell Node.js to use the source maps. Generating source maps is
as easy as including a command line switch in the compilation:

$ node_modules/.bin/babel server --presets es2015 --out-dir dist --source-maps

To let Node.js report line numbers by using source maps, we need to install the
source-map-support module, and also call the library in the application once. Let’s first
install the package.

$ npm install --save source-map-support

To use this, we need to change server.js as shown in Listing 7-30.

Listing 7-30. server.js: Add Source Map Support

...
import SourceMapSupport from 'source-map-support';
SourceMapSupport.install();
...

Now, if you introduce an error anywhere in the server, the error message printed on
the console will have the line number from the source file instead of the compiled file.
You can try this out by temporarily introducing the following line anywhere in server.js:

...
throw new Error('Test!');
...

Now that the server depends on babel-polyfill during runtime, we need to move
the dependency from devDependencies to dependencies in package.json. While we’re
at it, let’s also change package.json to add new commands for compiling the server and
watching for changed files, and also change the start command to use the compiled file
from the dist directory. The changes are shown in Listing 7-31.

Listing 7-31. package.json: Changes to Scripts and Dependencies

...
  "scripts": {
    "start": "nodemon -w dist dist/server.js",
    "compile-server": "babel server --presets es2015-node4 
--out-dir dist --source-maps",
    "watch-server": "babel server --presets es2015-node4 
--out-dir dist  --source-maps --watch",
...

137
Chapter 7 ■ Modularization and Webpack

  "dependencies": {
    "babel-polyfill": "^6.13.0",
...
  "devDependencies": {
    "babel-polyfill": "^6.13.0",
...

This also means that we need one more console to run npm run watch-server,
taking the total number of consoles occupied to three. There is an alternative to static
compilation that can do the transformation on the fly. This method is called the require
hook. That’s because it binds itself to Node.js’ require and dynamically compiles any
new module that is loaded. Let’s install it to try it out:

$ npm install --save-dev babel-register

To use it, let’s create a separate entry point, load the babel-register module, and
then load your server.js to start the server. Let’s call this new file start_hook.js; the
contents are shown in Listing 7-32.

Listing 7-32. File start_hook.js

require('babel-register')({
  presets: ['es2015-node4']
});

require('./server.js');

We initialized the compiler with a preset (the same as for the static compilation using
babel-cli) because this module does not load any presets by default. Also, this module
automatically ignores libraries loaded from node_modules, so we did not have to explicitly
mention them to be ignored. To start the server using the require hook, let’s add a new
command in package.json for use by npm run. The command is as follows:

"start-hook": "nodemon -w server server/start_hook.js",

Now, use the command line npm run start-hook to start the server. Ensure that
functionally is all well, as before.
If you are not comfortable running different start and watch scripts on different
consoles, it is best to combine the different scripts into one single script that runs all
three in the background. You can do this by creating a new script that calls other npm
run <script> commands, separated by an &. The & has the effect of placing the script
preceding the & in background. Let’s add two combination scripts for each of the methods
that we used. The entire set of changes made to package.json in this section is listed in
Listing 7-33.

138
 Chapter 7 ■ Modularization and Webpack

Listing 7-33. package.json: Changes for New Commands and Moving Polyfill
...
  "scripts": {
    "start": "nodemon -w server server/server.js",
    "start": "nodemon -w dist dist/server.js",
    "compile-server": "babel server --presets es2015-node4 
--out-dir dist --source-maps",
    "watch-server": "babel server --presets es2015-node4 
--out-dir dist --source-maps --watch",
    "start-hook": "nodemon -w server server/start_hook.js",
    "compile": "webpack",
    "watch": "webpack-dev-server --hot --inline",
    "dev-all": "npm run watch & npm run watch-server & npm start",
    "dev-all-hook": "npm run watch & npm run start-hook",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
...
  "dependencies": {
    "babel-polyfill": "^6.13.0",
    "body-parser": "^1.15.2",
    ...
  },
...
  "devDependencies": {
    "babel-polyfill": "^6.13.0",
    ...
  }
...

Now, to run all development tasks using the statically compiled method, just use npm
run dev-all; for the start-hook method, use npm run dev-all-hook.

EXERCISE: SERVER-SIDE ES2015

1. We used the es2015-node4 preset rather than the es2015
preset, which would have worked for all versions of node, like
on the client side. Why did we do this? Hint: Think about the
advantages of native code.
2. Compare the pros and cons of the two ways of compiling:
statically using babel-cli or dynamically using the require hook.
Answers are available at the end of the chapter.

139
Chapter 7 ■ Modularization and Webpack

ESLint
A linter (something that lints) checks for suspicious code that may be a bug. It can also
check whether your code adheres to conventions and standards that you want to follow
across your team to make the code predictably readable.
While there are multiple opinions and debates on what is a good standard (tabs
vs. spaces, for example), there has been no debate on whether there needs to be a
standard in the first place. For one team or one project, adopting one standard is far more
important than adopting the right standard.
ESLint is a very flexible linter that lets you define the rules that you want to follow.
Yet, we need something to start off with and the rule set that has appealed to me the most
has been that of Airbnb. Part of the reason for its appeal has been its popularity: if more
people adopt it, the more standardized it gets, so more people end up following it, and
the cycle continues.
To get started, let’s install ESLint. Along with it, we need a plugin that understands
JSX, and then the Airbnb rule set. Installing the configuration automatically installs all
dependencies, which you will be able to see in the console output as well as package.json.

$ npm install --save-dev eslint-config-airbnb

ESLint requires that the configuration and the rules to apply be specified in a starter
file, .eslintrc. Let’s create this file in the project directory, and initialize it by extending
the Airbnb rule set. The initial file contents are shown in Listing 7-34.

Listing 7-34. .eslintrc: Default .eslintrc File Contents

{
  "extends": "airbnb"
}

To lint your code, you need to run the eslint command line against a file like this:

$ node_modules/.bin/eslint src/IssueAdd.jsx

Or, you can run it against an entire directory, say the server-side code, like this:

$ node_modules/.bin/eslint server

Since the default extension that ESLint looks for is only js, we need to tell it to look
for jsx files also. Further, we need to tell it to handle the client source directory as well. To
run it against all the code including the webpack configuration file, we need to do this:

$ node_modules/.bin/eslint --ext jsx,js src server webpack.config.js

140
 Chapter 7 ■ Modularization and Webpack

If you do this, you’ll see a lot of error reports. This is because we never gave full
attention to writing good code or sticking to a standard until now. Let’s set out to correct
the errors and clean them up. At this stage, it’s easier to deal with it one file at a time.
Alternatively, if your editor supports it, you can install a linter plugin that automatically
displays lint errors for you in the editor, so that you can correct them as you type. The
popular code editors Atom and Sublime do have plugins to handle this; please follow the
instructions on their respective websites to install the plugins.
As for the changes we need to make to fix the lint errors, there are too many to
discuss or even show in the listings individually. Other than the major rewrites, I’ll only
discuss the types of changes needed in order to get a clean ESlint report. For individual
changes, please refer to the GitHub repository accompanying this book and the
differences between this step and the previous. This is one section where I’ve skipped the
listings to show code changes.
For most of the errors, we are just going to change the code to adhere to the
suggested standard. But in a few cases, we will make exceptions to the Airbnb rule. The
first kind of exception is to override the rule itself, by setting different options for the
rule in the .eslintrc configuration file. A rule is identified by its name (as printed in
the console, for example, no-use-before-define). The value is the settings, which is an
array. The first element indicates whether the rule is disabled and causes a warning or an
error. Further elements are more options to the rule. Here is an example rule set:

...
"rules" {
    "no-console": ["off"]
  }
...

This rule set has only one rule specification, which switches off the rule, thus
allowing console.log statements throughout the project. It needs no further parameters.
But some rules can also take parameters, specifying conditions on which to apply them.
For example, to specify that parameter reassignment should be considered an error,
except for the parameters properties, this is what you write:

"no-param-reassign": ["error", { "props": false }]

The second kind of exception that you can make is for temporary code and for
things that are really localized, that is, not applicable for the project but for one particular
instance of that error. ESLint allows you to make such exceptions on a per-line basis by
adding an in-line comment starting with eslint-disable-line and optionally specifying
which rule to disable a check for (otherwise, it disables checking completely), like this:

console.log('Must see message');    // eslint-disable-line no-console-log

141
Chapter 7 ■ Modularization and Webpack

■■Note The errors can be quite different from what I have discussed, based on the
version of the Airbnb ESlint configuration (the module eslint-config-airbnb) because the
configuration is under continuous change and improvement. I have used the configuration
version 9.0.1.

Environment
ESLint needs to be told that a certain file is being used in a certain environment. This
lets ESLint ignore otherwise invalid global variables such as document, which is available
in the browser environment. Since you need to specify different environments for the
server-side and client-side code, you’ll have to create override configuration files (again
called .eslintrc) in each of the directories.
The environment is specified not as a rule, but as a configuration variable at the
same level as rules. The configuration takes in an environment name and whether the
source is expected to run in that environment (true/false). For example, to set the browser
environment, you add this configuration:

...
  "env": {
    "browser": true
  },
...

Now, let’s discuss the types of errors and changes we need to do to fix them.

Syntax Consistency
JavaScript is quite flexible in syntax, so there are many ways to write the same code. The
linter rules report some errors so that you use a consistent syntax throughout the project.
1. Missing semicolon: There’s a lot of debate on whether
semicolons everywhere or semicolons nowhere is better. Both
work, except for a few cases where the absence of semicolons
causes a behavior change. If you follow the no-semicolons
standard, it requires you to remember those special cases.
Let’s go with the Airbnb default, which is to require a
semicolon everywhere.
2. Strings must use single quotes: JavaScript allows both single
and double-quotes. In order to standardize, it’s better to
consistently use one and only one style. Let’s use the Airbnb
default, single quotes.

142
 Chapter 7 ■ Modularization and Webpack

3. Chained calls require a newline per new function: When

chaining calls, especially promises, it improves readability to
have each call on a new line. Thus, every .then and the final
.catch need to start on their own new lines.
4. Prefer shorthand: When the key and value in an object have
the same name, using the shorthand makes the code more
concise, and thus easier to read.
5. Prefer string templates: String templates improve readability
because you don’t have a + character in the middle.

Editing Convenience
The following rules reduce the chance of an error when you edit code.
1. Missing trailing comma: Requiring a comma for the last item
in a multi-line array or object is really handy when inserting
new items. In addition, when looking at the difference
between two versions in say, GitHub, the fact that a comma
is added to the last line highlights that the line has been
changed, whereas in reality, it hasn’t.
2. Require a curly after an if: This is useful when you add more
lines in the block. It’s possible that you may forget to add the
curly and you could end up with a dangling line that looks
inside the block but is really outside. Forcing a curly avoids
this problem.
If you need a very short if block, you could write the block in
the same line without a curly. The curly is required only if the
block is in a new line of its own.

Structural Issues
When the linter points out structural issues, there are very likely bugs lurking underneath.
1. Variables should be defined before using: This is a good
practice so that all global variables are at the top, and the
reader is not confused as to what the variable is all about
when reading code top to bottom.
2. Use const/let instead of var: Using var to declare variables
causes it to be at the function scope rather than block scope.
This can lead to undesirable behavior, especially if the name
is used again. Using let and const forces you to think and
declare variables in the scope that they belong.

143
Chapter 7 ■ Modularization and Webpack

3. For-in statements are disallowed: The linter suggests that you

guard this with a check for hasOwnProperty, since you may
unexpectedly get properties if you don’t. Note that for-in
statements are therefore best avoided; you can replace them
with a forEach iteration on the keys using Object.Keys().
4. Already declared (shadow) variables: It’s best to avoid a
variable in an inner scope having the same name as one in
the outer scope. It’s confusing, and also it hides access to the
outer scope variable in case it is required to be accessed.
5. Function parameter reassignment is disallowed: This really
exposed a bug in the code, and also points out that we should
in fact have had two separate functions to validate and clean
up the object. In the latter function, we should return a copy
of the issue passed in as a function parameter. The two new
functions are shown in Listing 7-35 and the changes to the
caller (server.js) are shown in Listing 7-36.

Listing 7-35. Rewrite of Function validateIssue as Two Functions

function cleanupIssue(issue) {
  const cleanedUpIssue = {};
  Object.keys(issue).forEach(field => {
    if (issueFieldType[field]) cleanedUpIssue[field] = issue[field];
  });
  return cleanedUpIssue;
}

function validateIssue(issue) {
  const errors = [];
  Object.keys(issueFieldType).forEach(field => {
    if (issueFieldType[field] === 'required' && !issue[field]) {
      errors.push(`Missing mandatory field: ${field}`);
    }
  });

  if (!validIssueStatus[issue.status]) {
    errors.push(`${issue.status} is not a valid status.`);
  }

  return (errors.length ? errors.join('; ') : null);

}

export default {
  validateIssue,
  cleanupIssue,
};

144
 Chapter 7 ■ Modularization and Webpack

Listing 7-36. Changes to server.js for cleanupIssue

...
  db.collection('issues').insertOne(Issue.cleanupIssue(newIssue)). 
then(result =>
...
But in IssueList.jsx, we do need to modify the properties of
objects that are passed in during replacement of date strings
with Date objects in Issue records. So, we chose to override
the default no-param-reassign rule by allowing modification
of properties alone. The rule for this in the root .eslintrc file
is as follows:

    "no-param-reassign": ["error", { "props": false }]

React Specifics
These are React-specific issues that are good practices to adopt.
1. Enforcing propTypes: Just like declaring function parameter
types, it’s a good practice to specify the properties and their
types that you pass to components from parent to child.
I discussed how this can be done in the chapter on React
components in the “Passing Data Using Properties” section;
now We’ll do it for all components.
2. Stateless functions: The linter detects the fact that some
components don’t have any state, and they are better
written as stateless functions. We have such a component in
IssueFilter, but this is only a temporary placeholder. We’ll
add an in-line exception for this at the end of the line:

// eslint-disable-line

Rule Overrides
Rather than correct the code, we’ve chosen to override the default Airbnb rule to not
report certain issues as errors.
1. Dangling _: These are meant to be internal variables that
you’re not supposed to use. But MongoDB’s _id is so
convenient for various things that you’ll allow the dangling
underscore only for this variable. The rule for this is

"no-underscore-dangle": ["error", { "allow": ["_id"] }],

145
Chapter 7 ■ Modularization and Webpack

2. No console.log: Typically, this is considered bad practice. In

order to insert debugging statements, it is recommended
that you use a module such as debug. We’ll adhere to this for
the client side but override this rule for the server-side code,
because we do want to see console messages (and maybe save
them in a log file) on the server. The rule for this, only in the
server configuration, is

"no-console": ["off"]

3. No alerts: The original intention of this rule was to weed out

debugging messages that were left unpruned. At least for the
foreseeable future of the application, we’ll be showing error
messages to the user as alerts. Until we find a better way to
do this, we’ll allow alerts. The rule is part of the client-side
.eslintrc, and is as follows:

"no-alert": ["off"]

The final set of .eslintrc files after including all the environment specifications and
the override rules is shown in the following listings. Listings 7-37 is for the configuration
at the root level, that is, common to all code. Listing 7-38 is for the server-side code and
Listing 7-39 is for the client-side code.

Listing 7-37. .eslintrc: Final root .eslintrc

{
  "extends": "airbnb",
  "rules": {
    "no-underscore-dangle": ["error", { "allow": ["_id"] }],
    "no-param-reassign": ["error", { "props": false }]
  }
}

Listing 7-38. server/.eslintrc: Final server .eslintrc

{
  "env": {
    "node": true
  },
  "rules": {
    "no-console": ["off"]
  }
}

146
 Chapter 7 ■ Modularization and Webpack

Listing 7-39. src/.eslintrc: Final client .eslintrc

{
  "env": {
    "browser": true
  },
  "rules": {
    "no-alert": ["off"]
  }
}

Finally, for convenience of running ESLint using npm without having to remember
all the options that it takes, let’s create a command under scripts in package.json that
can be used to run the linter. This change is shown in Listing 7-40.

Listing 7-40. package.json: New Command for Running Lint

...
  "scripts": {
    ...
    "lint": "eslint --ext js,jsx src server webpack.config.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
...

EXERCISE: ESLINT
1. We disabled ESLint checking completely, instead of disabling
only the check for stateless functions. Ideally, we should have
written // eslint-disable-line react/prefer-stateless-
function. We didn’t. Why not? What would happen if we did?

Answers are available at the end of the chapter.

Summary
Although we did not add any features to the application in this chapter, we made lots
of changes to get organized and improve our productivity. Going forward, we’ll closely
watch lint errors, which help keep the code clean and uniform, but also detect errors
that otherwise would only be seen after you run the application and test it. Setting up
modularization on both the client and server also lets us keep code for different classes
separated.
In the next chapter, we’ll go back to adding more features. We’ll explore an important
concept of client-side routing that will allow us to show different components or pages
and navigate between them.

147
Chapter 7 ■ Modularization and Webpack

Answers to Exercises
Exercise: Transform and Bundle
1. No, webpack does not rebuild if you save a file with just an
extra space. This is because the preprocessing or loader stage
produces a normalized JavaScript, which is no different from
the original. A rebundling is triggered only if the normalized
script is different.
2. As of now, we have only one page to display, the Issue List.
Going forward, we’ll have other pages to render, for example,
a page to edit the issue, maybe another to list all users, yet
another to show one’s profile information, and so on. The
file App.jsx was created keeping in mind different page
components that could come up in the future.
3. Not using the default keyword has the effect of exporting
the class as a property (rather than itself ) of the object that’s
exported. In the import statement, you would have to do this:

import { IssueList } from ‘./IssueList.jsx’;

Note the curly braces around the LHS. This allows multiple
objects to be exported from a single file, each object you want
from the import being separated by a comma. If there’s only
one object being exported, you just default it, so that the curly
braces are not required.

Exercise: Hot Module Replacement

1. There are many logs in the browser’s console that tell you that
HMR is being invoked. Further, even when the console log
window setting “preserve logs across browser requests” is off,
you will find that the logs still persist, which means there was
no new browser refresh.
If you look at the network requests, you will find that for a
browser refresh, requests are made to all of the assets. Take a
look at the sizes of these assets. Typically, vendor.bundle.js
is not fetched again when client-side code changes (it would
return a 304 response), but app.bundle.js will be reloaded.
But when HMR succeeds, you will see that the entire assets
are not fetched; instead, incremental files, much smaller than
app.bundle.js, are being transferred.

148
 Chapter 7 ■ Modularization and Webpack

2. The main objective of webpack-dev-server is to prevent

inadvertent errors, such as those caused by refreshing the
browser even before a bundle operation has completed. This
can be damaging because you may think you have changed
the code, but the new code is not what is running on the
browser. HMR is just icing on the cake. I would use webpack-
dev-server even if it did not support HMR.

Exercise: Server-Side ES2015

1. Native support for any JavaScript feature is expected to be
faster than the transpiled code, which uses ES5 constructs.
On the client side, you cannot predict which browsers will
be used by the end user, so you must use the least common
denominator. Whereas, on the server side, the node version
that you are going to use in production is predictable and
under your control, and you can indeed take advantage of this.
2. The static compilation alternative requires you to start up
two consoles just for the server and ensure that they are
functioning OK, whereas the dynamic alternative needs only
one console. Although we combined them into one npm
script, the output can be intermingled and it may not be easy
to spot any errors.
When there are syntax errors in your server code, in the
static compilation alternative, the compile step fails, but
the server continues to run the old code. This can lead to
confusion during development, if you are not monitoring the
server compilation console. In the require hook method, the
server crashes, and you will be forced to see what happened,
because the application will stop functioning.
The static compilation method is faster than the require hook,
especially when there are incremental changes to the server
code.
Using the static compilation, you are confident that the
production code is identical to the development code (require
hook is not intended for production, so you have to run the
statically compiled code). This removes at least that one
variable while chasing bugs in production.
My recommendation is to use static compilation regularly and
monitor the console. To me, the fact that the development and
production environments are identical is worth this trade-off.

149
Chapter 7 ■ Modularization and Webpack

Exercise: ESLint
1. If you disabled that specific rule, the length of the line
would have exceeded 100 characters, triggering an error on
another rule that restricts the maximum line length. We took
a shortcut and disabled all linting for the line, knowing that
we’ll revisit this class soon.
An alternative is to disable both rules like this:

// eslint-disable-line max-len,react/prefer-stateless-function

150