EECS E6893 Big Data Analytics

HW4: Data visualization with web dashboard

Hritik Jain, [email protected]

11/24/2020 1
Agenda
● Introduction to Web Development
○ HTML, CSS and Javascript
○ 2 important things to know: SVG and DOM
● D3.js demo for data visualization
● HW4 Part 1(35% of HW4)
○ Short-answer Questions
○ Draw a simple barchart
● Introduction to Django
● D3.js demo for data visualization
● HW4 Part 2 (65% of HW4)
○ Dashboard of HW3
○ Required results from HW2

I should preface the tutorial material by saying that if you are primarily interested in
web development, this tutorial is hardly enough. What we really focus on is visualizing
the results we obtained in our previous two HWs on a web application. So to have the
required background, we will review the basic front end technologies - HTML, CSS,
Javascript and then two key ideas of SVG and DOM. Then we will talk about d3 that
you will need spend time on a little bit beyond this tutorial to learn the core concepts.
Web Application

Reference: https://hackr.io/blog/web-application-architecture-definition-models-types-and-more 3

So any website you want to visit on the web, you enter a url in the address bar, the
browser requests for that particular web address. The server that is identified through
the address then sends files to the browser as a response to the request. The
browser then executes those files to show the requested page.

The code that is returned to the browser and responds to some user input is the client
side code, the end-user can see it as well as change it.

Now, the developers of the web application decide as to what the code on the server
will do with respect to the code in the browser. The server-side code is responsible for
creating the page that the user requested as well as storing different types of data,
including user profiles and user input, business logic etc. The end-user can’t see it.
Client-side of Web
● HTML, CSS and JS are the parts of all websites that users directly interact
with.
● HTML provides the basic structure of sites, which is enhanced and modified
by other technologies like CSS and JavaScript.
● CSS is used to control presentation, formatting, and layout.
● JavaScript is used to control the behavior of different elements.

Reference: https://tutorial.techaltum.com/webdesigning.html 4

Html css are used on the client side for how the webpage looks. Javascript is a
scripting language, it’s a very powerful tool that can do many things for a website. Like
it enables you to add interactivity. You can build rich UI components such as image
sliders, pop-ups, site navigation mega menus, form validations, tabs, accordions, and
much more.
HTML

Reference: https://www.w3schools.com/html/default.asp 5

HTML describes the structure of a Web page. It consists of a series of elements -

these elements tell the browser how to display the content. All that you see here -
heading 1, body, paragraph these are elements. An HTML element is defined by a
start tag, some content, and an end tag, some don’t always require that. They can be
nested as well. Then there are elements - all HTML elements can have attributes.
attributes provide additional information about elements and they are added in the
tags.
HTML DOM
Document Object Model (DOM) is important: With the HTML DOM,
JavaScript can access and change all the elements of an HTML document

Reference: https://developer.mozilla.org/en-US/docs/Web/API/HTML_DOM_API 6

We can think of the HTML representing a web page as a document that has a
hierarchical structure, you know with nested elements. So Document Object Model or
DOM represents such a document with a logical tree. Each branch of the tree ends in
a node, and each node contains objects. DOM methods allow programmatic access
to the tree. With them, you can change the document's structure, style, or content.
That’s where javascript comes in.
Javascript with HTML DOM
● With the HTML DOM, JavaScript can access and change all the elements of an
HTML document

● HTML DOM methods are actions you can perform (on HTML Elements).

● HTML DOM properties are values (of HTML Elements) that you can set or change.

7
Javascript with HTML DOM

getElementById is the method and innerHTML is the property

SVG in HTML
● SVG stands for Scalable Vector Graphics. It is used to define
vector-based graphics for the Web
● Every element and every attribute in SVG files can be animated
● SVG integrates with other W3C standards such as the DOM and XSL

Reference: https://developer.mozilla.org/en-US/docs/Web/SVG 9

Now, Scalable Vector Graphics (SVG) are an XML-based markup language for
describing two-dimensional based vector graphics it's a text-based standard for
describing images that can be rendered at any size and are designed specifically to
work well with other web standards including CSS, DOM, JavaScript. We’ll talk about
DOM in a bit. SVG is, essentially, to graphics what HTML is to text. The HTML <svg>
element is a container for SVG graphics.
What if we create a SVG elements and use DOM in Javascript to access its
attributes?

If we draw a series of SVG and texts based on data, and use DOM to
control their attributes, then we get a simple charts!
- D3.js, a library to do this in a simple way
10

This is an essence of how you add interactivity to a webpage right.

D3.js

D3.js is a JavaScript library for manipulating documents based on data.

It helps you bring data to life using HTML, SVG, and CSS. It provides a
data-driven approach to DOM manipulation.

Visit https://d3js.org for more tutorials!

11

So d3 provides a data-driven approach to DOM manipulation. Again, you would want

to look into the tutorials. Here’s a quick illustration on the important ideas.
 Declaration of data and variables

D3 provides operating on arbitrary sets of

nodes called selections. You can
manipulate individual nodes and set the
attributes

Data joins.
Once you bound data with
selection, each element in the
data array is paired with the
corresponding node in the
selection.
Tricky part of D3: If there are
fewer nodes than data, you can
use enter() to append nodes.
Again, please visit https://d3js.org for more tutorials! 12

On the top you see declaration of variables. Next, the idea of selections, with
selections, it lets us to grab a set of one or more elements of the webpage, ie the
nodes of a DOM. You will use selections to modify, append, or remove elements in a
relation to the predefined dataset.

That leads to the concept of data binding. Ultimately you are visualizing data right.
The power of d3 really lies in binding data to elements, svg elements in particular.
You see this data() operator here in the third code snippet, you are passing our
dataset that we defined above in a list to the operator (in the hw you will read this data
from bigquery). And you are using that on a selection of DOM elements, all the svg
elements of the type rectangle. Doing this you join both sets of items - the svgs and
the numbers in the list with each other item by item, it’s called a data join and then
you do something with them.

But the interesting this is you can do this data join even when you have more data
items than the selected dom items. That’s where it gets a little tricky. When the
dataset contains more items than there are available DOM elements, like through the
interactivity allowed through javascript you can remove svg elements from the DOM.
So the surplus data items are stored in a subset of this selection called the enter
selection. Then you call append to create new dom elements.

Similarly, there is an exit selection for the case when you more dom elements in the
selection than the joined dataset. You can call remove() to clean up the DOM
elements.
Example: Simple Bar Chart

13

The power of d3 really lies in binding data to elements, svg elements in particular.
Django

● Django is a high-level Python

Web framework that encourages
rapid development and clean,
pragmatic design.
● Takes care of much of the hassle
of Web development
● Free and open source.

https://www.djangoproject.com

https://docs.djangoproject.com/en/3.1/intro/tutorial01/

14

Now when you develop web applications, there is the server side, and there are
overhead associated with common activities performed, for example, a database,
database access, session management, etc

Web frameworks aim to automate these activities for you. Django is one such popular
framework.

We will django in this HW for part 2 to build the dashboard application. I will be
uploading another tutorial soon which is more like exact step by step instructions for
setting up a django app, that you can refer to when you begin. Most of this code for
setting up a django app will be given to you. For completing the exercise, your
primary task here is to write or modify the d3 javascript code that will display the
visualizations of previous results on a webpage.

That said, you may want to consider frameworks like django and flask for your project
if you build a web application, so it’s encouraged that you read up the documentations
in detail by yourself, for these exercises you will only need a basic familiarity.
HW4

● Part 1 (35%)
○ 4 short-answer questions (5% each)
○ Draw a simple bar chart as per instructions (15%)

● Part 2 (65%)
○ Create a Django project
○ Query the HW3 data from bigquery
○ Draw a Dashboard based on the data, showing a bar chart and a pie chart (20% each)
○ Process the results from HW2 and store the required data to BigQuery
○ Create a simple visualization to show connections between nodes of a connected component

15
References
● https://d3js.org
● https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial
● https://www.w3.org/Graphics/SVG/IG/resources/svgprimer.html
● https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute
● https://jtr13.github.io/d3book/index.html

16
Thanks!

17