Parekh Janvi (210130116029) DAV

A Laboratory Manual for

Data Analysis and

Visualization
(3161613)

B.E. Semester 6 (IT)

Directorate of Technical Education, Gandhinagar,

Gujarat
Parekh Janvi (210130116029) DAV

Certificate
This is to certify that Mr./Ms.
Enrollment No. _ of B.E. Semester _
Information Technology of this Institute (GTU Code: 028 ) has satisfactorily
completed the Practical / Tutorial work for the subject Data Analysis and
Visualization (3161613) for the academic year 2023-24.

Place:
Date:

Name and Sign of Faculty member

Head of the Department

Parekh Janvi (210130116029) DAV

Data Analysis and Visualization (3161613)

Preface

Main motto of any laboratory/practical/field work is for enhancing required skills as well as
creating ability amongst students to solve real time problem by developing relevant
competencies in psychomotor domain. By keeping in view, GTU has designed competency
focused outcome-based curriculum for engineering degree programs where sufficient weightage
is given to practical work. It shows importance of enhancement of skills amongst the students
and it pays attention to utilize every second of time allotted for practical amongst students,
instructors and faculty members to achieve relevant outcomes by performing the experiments
rather than having merely study type experiments. It is must for effective implementation of
competency focused outcome-based curriculum that every practical is keenly designed to serve
as a tool to develop and enhance relevant competency required by the various industry among
every student. These psychomotor skills are very difficult to develop through traditional chalk
and board content delivery method in the classroom. Accordingly, this lab manual is designed
to focus on the industry defined relevant outcomes, rather than old practice of conducting
practical to prove concept and theory.

By using this lab manual students can go through the relevant theory and procedure in advance
before the actual performance which creates an interest and students can have basic idea prior to
performance. This in turn enhances pre-determined outcomes amongst students. Each
experiment in this manual begins with competency, industry relevant skills, course outcomes as
well as practical outcomes (objectives). The students will also achieve safety and necessary
precautions to be taken while performing practical.

This manual also provides guidelines to faculty members to facilitate student centric lab
activities through each experiment by arranging and managing necessary resources in order that
the students follow the procedures with required safety and necessary precautions to achieve the
outcomes. It also gives an idea that how students will be assessed by providing rubrics.

Data Analytics involves data discovery that helps in making smart decisions, creating
suggestions for options based on previous choices. Data visualization sees the pattern in data
and also sees the pattern when data is not part of pattern.

Utmost care has been taken while preparing this lab manual however always there is chances of
improvement. Therefore, we welcome constructive suggestions for improvement and removal
of errors if any.
Parekh Janvi (210130116029) DAV

Data Analysis and Visualization (3161613)

Practical – Course Outcome matrix

Course Outcomes (COs):

CO1: Perform descriptive statistics and dimensionality reduction.
CO2: Perform clustering and detect outliers
CO3: Perform data visualization
CO4: Apply the analytics and visualization to real world problems.
Sr.
Objective(s) of Experiment CO1 CO2 CO3 CO4
No.

1. DATA STATISTICAL OPERATIONS √

DATA PRE-PROCESSING RELATED

2. √
OPERATIONS

3. K-MEANS DATA CLUSTERING √

4. KNN CLASSIFICATION √

OUTLIER DETECTION AND REMOVAL FOR

5. √
NORMALLY DISTRIBUTED FEATURES
OUTLIER DETECTION AND REMOVAL FOR
6. √
SKEWED DISTRIBUTED FEATURES
SHOWING THE DATA AS A BAR CHART USING
7. √
D3.JS
SHOWING THE DATA AS A LINE CHART USING
8. √
D3.JS

9. SHOWING THE DATA AS A PIE CHART USING D3.JS √

10. TABLEAU MAPS FROM SPATIAL FILES √

Parekh Janvi (210130116029) DAV

Data Analysis and Visualization (3161613)

Industry Relevant Skills

The following industry relevant competencies are expected to be developed in the student by
undertaking the practical work of this laboratory.
1. Data Visualization and Representation
2. Summarization of Data for Interpretations

Guidelines for Faculty members

1. Teacher should provide the guideline with demonstration of practical to the students
with all features.
2. Teacher shall explain basic concepts/theory related to the experiment to the students
before starting of each practical
3. Involve all the students in performance of each experiment.
4. Teacher is expected to share the skills and competencies to be developed in the
students and ensure that the respective skills and competencies are developed in the
students after the completion of the experimentation.
5. Teachers should give opportunity to students for hands-on experience after the
demonstration.
6. Teacher may provide additional knowledge and skills to the students even though not
covered in the manual but are expected from the students by concerned industry.
7. Give practical assignment and assess the performance of students based on task
assigned to check whether it is as per the instructions or not.
8. Teacher is expected to refer complete curriculum of the course and follow the
guidelines for implementation.

Instructions for Students

1. Students are expected to carefully listen to all the theory classes delivered by the faculty
members and understand the COs, content of the course, teaching and examination
scheme, skill set to be developed etc.
2. Students shall organize the work in the group and make record of all observations.
3. Students shall develop maintenance skill as expected by industries.
4. Student shall attempt to develop related hand-on skills and build confidence.
5. Student shall develop the habits of evolving more ideas, innovations, skills etc. apart from
those included in scope of manual.
6. Student shall refer technical magazines and data books.
7. Student should develop a habit of submitting the experimentation work as per the schedule
and s/he should be well prepared for the same.

Common Safety Instructions

1. Students are expected to carefully
Parekh Janvi (210130116029) DAV

Data Analysis and Visualization (3161613)

Sample Rubrics for Practical Assessment :

Understanding of Implementation of Presentation and report Total

Problem Problem writing (10
(3 marks) (4 marks) (3 marks) marks)

Excellent Moderate level Problem not

understanding of understanding of understood and
problem and 03 problem and 02 can't establish the 01
Understanding
relevance with the relevance with the relation with the
of Problem
theory clearly theory clearly theory.
understood. understood.
Efficient Moderate level of Partial
implementation implementation. implementation 01
Implementation 04 03
with proper naming Poor naming with poor to
of Problem
convention and convention. understanding. 02
understanding
Unique Ordinary Weak
documentation documentation of documentation of 01
Presentation (not copied from 03 given problem with 02 given problem to
and report other sources) of proper formatting without proper 02
writing given problem with and language formatting and
proper formatting language
and language.
Parekh Janvi (210130116029) DAV

Data Analysis and Visualization (3161613)

Index
(Progressive Assessment Sheet)

Sr. Objective(s) of Experiment Page Date of Date of Assessme Sign. of Remar

No. No. perform submiss nt Teacher ks
ance ion Marks with date

1 DATA STATISTICAL OPERATIONS

DATA PRE-PROCESSING RELATED
2
OPERATIONS
3 K-MEANS DATA CLUSTERING

4 KNN CLASSIFICATION
OUTLIER DETECTION AND REMOVAL
5 FOR NORMALLY DISTRIBUTED
FEATURES
OUTLIER DETECTION AND REMOVAL
6
FOR SKEWED DISTRIBUTED FEATURES
SHOWING THE DATA AS A BAR CHART
7
USING D3.JS
SHOWING THE DATA AS A LINE CHART
8
USING D3.JS
SHOWING THE DATA AS A PIE CHART
9
USING D3.JS
10 TABLEAU MAPS FROM SPATIAL FILES
Total
Parekh Janvi (210130116029) DAV

Experiment No: 1
DATA STATISTICAL OPERATIONS

Date:
Competency and Practical Skills:
Relevant CO: CO1
Objectives: Perform various data statistical operations on sample dataset.
Equipment/Instruments: Basic Statistics, Numpy, Statistic library , PANDAS Library
Theory:
Descriptive statistics is about describing and summarizing data. It uses two main approaches:
1. The quantitative approach describes and summarizes data numerically.
2. The visual approach illustrates data with charts, plots, histograms, and other graphs.
Learnings:
A. What numerical quantities you can use to describe and summarize your datasets
B. How to calculate descriptive statistics in pure Python
C. How to get descriptive statistics with available Python libraries
D. How to visualize your datasets
E. Central tendency tells you about the centers of the data. Useful measures include the mean,
median, and mode.
F. Variability tells you about the spread of the data. Useful measures include variance and standard
deviation.
G. Correlation or joint variability tells you about the relation between a pair of variables in a
dataset. Useful measures include covariance and the correlation coefficient.

https://realpython.com/python-statistics/
Set up diagram:
Procedure:
Observations:
Calculation:
Result:
Conclusion:
Quiz:
Suggested Reference:
References used by the students:
Rubric wise marks obtained:

Rubrics 1 2 3 4 5 Total
Marks
Parekh Janvi (210130116029) DAV

Experiment No: 2
AIM: Perform various data pre-processing related operationsover dataset using
machine learning libraries.

Date:
Competency and Practical Skills:
Relevant CO: CO1
Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV

Experiment No: 3
Aim: Demonstrate K-MEANS data clustering methods
Date:
Competency and Practical Skills:
Relevant CO: CO2
Objectives: Demonstrates K-means data clustering technique using sample dataset.
1. Get introduced to K-Means Clustering.
2. Understand the properties of clusters and the various evaluation metrics for
clustering.
3. Get acquainted with some of the many real-world applications of K-Means
Clustering.
4. Implement K-Means Clustering in Python on a real-world dataset.

Source code with output:

Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV

Conclusion: K-means clustering effectively segmented the dataset into distinct groups,
revealing underlying patterns and structures. Evaluation metrics confirmed the quality of
clustering, indicating meaningful separation between clusters. These clusters provide
valuable insights for targeted strategies or further analysis, such as in marketing where they
can represent different customer segments with unique preferences. Overall, k-means
clustering yielded actionable insights, guiding decision-making and strategy development
based on the inherent characteristics of the data.
Parekh Janvi (210130116029) DAV

Experiment No.: 4
Aim: Demonstrate the implementation of KNN classification
Date:

Competency and Practical Skills: Python and Supervised learning

Relevant CO: 2

Objectives: Develop a program to classify data using KNN algorithm.

 Understand the working of KNN and how it operates in python.
 Get to know how to choose the right value of k for KNN
 Understand the difference between training error rate and validation error rate

Source code with output:

Parekh Janvi (210130116029) DAV

Conclusion: The application of K-Nearest Neighbors (KNN) classification on the dataset

enabled accurate prediction of class labels based on the attributes of the data points.
Through the determination of the nearest neighbors, KNN effectively categorized new
instances into classes, demonstrating its capability to handle both binary and multi-class
classification tasks. Evaluation metrics such as accuracy, precision, recall, and F1-score
validated the performance of the classifier, indicating its effectiveness in accurately
predicting class labels. The simplicity and flexibility of KNN make it a valuable tool for
classification tasks, offering a straightforward approach to pattern recognition and decision-
making based on the characteristics of the data.
Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV

Experiment No: 5
Aim: Demonstrate outlier detection and removal for normallydistributed
features.
Date:
Competency and Practical Skills:
Relevant CO: CO2
Objectives: Demonstrate outlier detection and removal for normally distributed
features.
1. An Overview of outliers and why it’s important for a data scientist to
identify and remove them from data.
2. Understand different techniques for outlier treatment: trimming, capping,
treating as a missing value, and discretization.
3. Understanding different plots and libraries for visualizing and treating outliers in
a dataset.

Source code with output:

Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV

Conclusion: Utilizing outlier detection methods such as the Z-score and 3 standard
deviation approach on a normally distributed dataset facilitated the identification and
treatment of anomalous data points. By quantifying the deviation of each data point from
the mean in terms of standard deviations, these methods effectively pinpointed outliers
beyond a certain threshold. This process allowed for the detection of data points that
significantly deviate from the expected distribution, potentially indicating errors or
anomalies in the dataset. Subsequently, appropriate actions could be taken, such as data
cleansing or further investigation into the nature of the outliers. Overall, the application of
these outlier detection techniques contributed to the robustness and reliability of the data
analysis process, ensuring the integrity of the insights derived from the dataset.
Parekh Janvi (210130116029) DAV

Experiment No: 6
AIM: Demonstrate outlier detection and removal for skeweddistributed
features.
Date:
Competency and Practical Skills:
Relevant CO: CO2
Objectives: Demonstrate outlier detection and removal for skewed distributed
features.
1. An Overview of outliers and why it’s important for a data scientist to identify
and remove them from data.
2. Understand different techniques for outlier treatment: trimming, capping,
treating as a missing value, and discretization.
3. Understanding different plots and libraries for visualizing and treating outliers in a
dataset.

Source Code with Output:

Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV
Parekh Janvi (210130116029) DAV

Conclusion: Implementing an outlier detection method like the Interquartile Range

(IQR) on a skewed distributed dataset proved instrumental in identifying and
managing extreme values that could distort analysis. By computing the range between
the first and third quartiles, IQR effectively captured the central tendency of the data,
accommodating the dataset's skewed nature. This enabled the identification of data
points lying beyond a certain threshold, highlighting potential anomalies or influential
observations. Addressing these outliers appropriately, whether through removal,
transformation, or separate analysis, enhanced the reliability and interpretability of
subsequent analyses.
In summary, employing IQR to handle outliers in a skewed dataset contributed to the
robustness of data analysis, facilitating more accurate and meaningful insights.
Parekh Janvi (210130116029) DAV

Experiment No: 7
AIM: Develop a program showing the data as a bar chart usingd3.js.

Date:
Relevant CO: CO3
Objectives: Develop a program showing the data as a bar chart using d3.js.
Source code:
<!DOCTYPE html>
<html>
<head>
<title>Bar Chart Example</title>
<script src="https://d3js.org/d3.v7.min.js"></script>
<style>
/* Add your CSS styles here, if needed */
.bar {
fill: steelblue;
}
</style>
</head>
<body>
<h1>Bar Chart Example</h1>
<div id="bar-chart"></div>

<script>
// Sample data for the bar chart
var data = [
{ category: "A", value: 10 },
{ category: "B", value: 20 },
{ category: "C", value: 15 },
{ category: "D", value: 30 },
{ category: "E", value: 25 } ];

// Set the dimensions of the chart

var margin = { top: 30, right: 30, bottom: 40, left: 40 }, width = 600 - margin.left -
margin.right,
height = 400 - margin.top - margin.bottom;

// Create an SVG element

var svg = d3.select("#bar-chart")
.append("svg")
.attr("width", width + margin.left + margin.right)
.attr("height", height + margin.top + margin.bottom)
.append("g")
.attr("transform", "translate(" + margin.left + "," + margin.top + ")");
Parekh Janvi (210130116029) DAV

// Define scales for x and y axes

var xScale = d3.scaleBand()
.domain(data.map(function(d) { return d.category; }))
.range([0, width])
.padding(0.1);

var yScale = d3.scaleLinear()

.domain([0, d3.max(data, function(d) { return d.value;})])

.nice()
.range([height, 0]);
// Create the bars
svg.selectAll(".bar")
.data(data)
.enter().append("rect")
.attr("class", "bar")
.attr("x", function(d) { return xScale(d.category); })
.attr("y", function(d) { return yScale(d.value); })
.attr("width", xScale.bandwidth())
.attr("height", function(d) { return height - yScale(d.value); });
// Create x-axis
svg.append("g")
.attr("transform", "translate(0," + height + ")")
.call(d3.axisBottom(xScale));

// Create y-axis
svg.append("g")
.call(d3.axisLeft(yScale));

</script>
</body>
</html
Parekh Janvi (210130116029) DAV

Conclusion: In conclusion, creating a bar chart using D3.js is a powerful way

to visualize and communicate data effectively. D3.js provides the flexibility and
control needed to design custom and interactive bar charts for various data-
driven applications. Bar charts are valuable tools for displaying categorical data,
comparing values, and identifying trends. D3.js empowers developers and data
analysts to craft highly tailored and interactive visualizations that enhance data
understanding. By harnessing the capabilities of D3.js, one can create engaging
and informative bar charts to convey insights and make data-driven decisions.
Parekh Janvi (210130116029) DAV

Experiment No: 8
AIM: Develop a program showing the data as a Line chart usingd3.js.

Date:
Competency and Practical Skills:
Relevant CO: CO3
Objectives: Showing the data as a Line chart using d3.js.

Source Code with Output:

<!DOCTYPE html>
<html>
<head>
<title>Line Chart Example</title>
<script src="https://d3js.org/d3.v7.min.js"></script>
<style>
/* Add your CSS styles here, if needed */
.line {
fill: none;
stroke: steelblue; stroke-width: 2;
}
</style>
</head>
<body>
<h1>Line Chart Example</h1>
<div id="line-chart"></div>
<script>
// Sample data for the line chart
var data = [
{year:2000, value: 10},
{year:2001, value: 20},
{year:2002, value: 15},
{year:2003, value: 30},
{year:2004, value: 25}
];

// Set the dimensions of the chart

var margin = { top: 30, right: 30, bottom: 40, left: 40 }, width = 600 -
margin.left - margin.right,
height = 400 - margin.top - margin.bottom;

// Create an SVG element

var svg = d3.select("#line-chart")
.append("svg")
.attr("width", width + margin.left + margin.right)
.attr("height", height + margin.top + margin.bottom)
.append("g")
.attr("transform", "translate(" + margin.left + "," + margin.top + ")");
Parekh Janvi (210130116029) DAV

// Define scales for x and y axes

var xScale = d3.scaleLinear()
.domain([d3.min(data, function(d) { return d.year; }), d3.max(data, function(d) {
return d.year; })])
.nice()
.range([0, width]);
var yScale = d3.scaleLinear()
.domain([0, d3.max(data, function(d) { return d.value;})])
.nice()
.range([height, 0]);

// Define the line generator

var line = d3.line()
.x(function(d) { return xScale(d.year); })
.y(function(d) { return yScale(d.value); });

// Create the line path

svg.append("path")
.datum(data)
.attr("class", "line")
.attr("d", line);

// Create x-axis
svg.append("g")
.attr("transform", "translate(0," + height + ")")
.call(d3.axisBottom(xScale));

// Create y-axis
svg.append("g")
.call(d3.axisLeft(yScale));

</script>
</body>
</html
Parekh Janvi (210130116029) DAV

Conclusion: In conclusion, utilizing D3.js to create a line chart is a versatile and

dynamic approach to visualizing data. Line charts are particularly effective for
showing trends, patterns, and changes in data over time. D3.js offers a high level
of customization and interactivity, enabling the creation of engaging and
informative line charts for diverse applications, from financial analysis to
scientific research. The power of D3.js lies in its ability to transform data into
compelling visual narratives, making it a valuable tool for data visualization and
storytelling. Whether it's for business reports, research presentations, or
interactive web applications, D3.js facilitates the construction of impactful line
charts that convey data-driven insights effectively.
Parekh Janvi (210130116029) DAV

Experiment No: 9
Aim: Develop a program showing the data as a pie chart usingd3.js.

Date:

Competency and Practical Skills: HTML, JavaScript and D3

Relevant CO: 3

Objectives: Plotting a pie chart in D3.js

Source Code with Output:

<!DOCTYPE html>
<html>
<head>
<title>Pie Chart Example</title>
<script src="https://d3js.org/d3.v7.min.js"></script>
</head>
<body>
<h1>Pie Chart Example</h1>
<div id="pie-chart"></div>

<script>
// Sample data for the pie chart
var data = [
{category: "AUS", value: 10 },
{category: "ENG", value: 20 },
{category: "NED", value: 15 },
{category: "IND", value: 30 },
{category: "NZL", value: 25 }
];

// Set the dimensions of the chart

var width = 400; var height = 400;
var radius = Math.min(width, height) / 2;

// Create an SVG element for the pie chart

var svg = d3.select("#pie-chart")
.append("svg")
.attr("width", width)
.attr("height", height)
.append("g")
.attr("transform", "translate(" + width / 2 + "," + height / 2 + ")");

// Create a color scale

var color = d3.scaleOrdinal(d3.schemeCategory10);

// Define the pie layout

var pie = d3.pie()
Parekh Janvi (210130116029) DAV

.value(function(d) { return d.value; });

// Generate the pie chart data

var pieData = pie(data);

// Create arc generator

var arc = d3.arc()
.innerRadius(0)
.outerRadius(radius);

// Create arcs for the pie slices

var arcs = svg.selectAll("arc")
.data(pieData)
.enter()
.append("g");

// Create path elements for the pie slices

arcs.append("path")
.attr("d", arc)
.attr("fill", function(d, i) { return color(i); });

// Add labels to the pie slices

arcs.append("text")
.attr("transform", function(d) { return "translate(" + arc.centroid(d) + ")"; })
.text(function(d) { return d.data.category; });
</script>
</body>
</html>

Conclusion: In conclusion, leveraging D3.js to construct a pie chart is an

effective means of displaying data distributions and proportions. Pie charts excel
at illustrating the composition of a whole and the relative sizes of its parts. D3.js
enhances this by allowing for interactive and customizable pie charts that engage
and inform the audience. Whether used in business reports, educational
materials, or web applications, D3.js empowers data analysts and developers to
craft engaging and informative pie charts. Through the combination of data and
Parekh Janvi (210130116029) DAV

visualization, D3.js facilitates the clear communication of complex information,

making pie charts a valuable tool for data-driven insights and decision-making.
Parekh Janvi (210130116029) DAV

Experiment No: 10
AIM: Create Tableau Maps from Spatial Files

Date:
Competency and Practical Skills:
Relevant CO: CO4

Objectives: Create Tableau Maps from Spatial Files.

Source Code with Output:

Step 1: Install Tableau

We need to install Tableau Desktop to generate spatial maps. You can get a free
trial version or a paid licence from the Tableau website.

Step 2: Prepare Your Data

Open Tableau Desktop.
Click on "Connect to Data" and choose "Text File."
Select the CSV file that contains your data (including the location names and
populations). Click "Open."

Step 3: Connect to Data

In the "Data Source" tab, ensure that your data is displayed correctly.
Check if Tableau has correctly identified the data types for each column.
Make sure the location data is recognized as a geographic role. You may
need to specify the geographical role if it's not recognized automatically.
Parekh Janvi (210130116029) DAV

Step 4: Create the Map Visualization

Tableau will generate a map visualization with default settings. You'll see a
map with circles representing locations and colors indicating population.

Conclusion: In conclusion, creating Tableau maps from spatial files offers a

powerful means of geospatial analysis and visualization. Tableau's intuitive
interface and mapping capabilities enable users to transform complex location
data into meaningful insights. Whether it's for business intelligence, urban
planning, or any field that relies on spatial understanding, Tableau maps provide
a dynamic and interactive way to communicate and explore geographic data. By
Parekh Janvi (210130116029) DAV

harnessing the features of Tableau and integrating spatial files, users can
uncover patterns, trends, and relationships within their data, facilitating data-
driven decision-making and enhancing the understanding of geospatial
information.