Unit 3: Introduction to Data Visualization

Overview

Data visualization is a crucial process in data analysis, enabling us to present data in a way that is
accessible and interpretable. It allows analysts to identify trends, patterns, and outliers,
facilitating decision-making. This chapter covers the fundamental concepts of data visualization,
including its stages, methods for processing and mapping data, and various visualization
techniques.

1. The Seven Stages of Visualizing Data

Visualizing data involves a structured approach that can be broken down into seven essential
stages:

1.1 Define Objectives

 Description: This initial stage is crucial for determining what you aim to achieve with
the visualization. Clear objectives help to focus the analysis.
 Key Questions:
o What question am I trying to answer?
o Who is the audience for the visualization?
o What decisions will this visualization support?
 Example: A marketing team wants to visualize customer demographics to tailor their
advertising strategies. Their objective is to identify which age groups are purchasing
specific products.

1.2 Data Collection

 Description: Collecting relevant data is the foundation of effective visualization. This

may involve gathering data from multiple sources, including databases, APIs, surveys,
and existing datasets.
 Methods:
 o Surveys: Directly collect data from users.
o Databases: Query existing databases using SQL.
o APIs: Use APIs (Application Programming Interfaces) to pull data from web
services.
 Example: A researcher collecting data from a public health database, customer feedback
forms, and sales reports.

1.3 Data Cleaning

 Description: Raw data often contains inaccuracies, missing values, or irrelevant

information. Data cleaning is the process of preparing this data for analysis.
 Techniques:
o Handling Missing Values: Options include deletion, imputation (replacing
missing values with statistical measures), or using algorithms that handle missing
data.
o Removing Duplicates: Ensuring no repeated entries distort the analysis.
o Standardization: Ensuring consistency in formats (e.g., date formats).
 Example: A dataset with customer entries might have missing phone numbers. The
analyst can either remove these entries or replace missing numbers with a placeholder.

1.4 Data Transformation

 Description: Data transformation involves converting data into a suitable format or

structure for analysis. This may include normalizing values, aggregating data, or creating
new calculated fields.
 Techniques:
o Aggregation: Summarizing data at a higher level (e.g., daily to monthly totals).
o Normalization: Adjusting values to a common scale.
o Pivoting: Restructuring data for easier analysis.
 Example: Converting daily sales figures into monthly sales by summing daily values.

1.5 Data Analysis

  Description: Analyzing the data involves applying statistical techniques to extract
meaningful insights and patterns. This step is often iterative, requiring adjustments based
on findings.
 Methods:
o Descriptive Statistics: Summarizing data using mean, median, mode, and
standard deviation.
o Inferential Statistics: Making predictions or generalizations about a population
based on sample data.
o Regression Analysis: Exploring relationships between variables.
 Example: A company might perform a regression analysis to understand how changes in
advertising spend affect sales revenue.

1.6 Data Visualization

 Description: This stage is where you create visual representations of your data. The
choice of visualization depends on the data type and the insights you want to convey.
 Common Visualizations:
o Bar Charts: Useful for comparing categories.
o Line Charts: Ideal for showing trends over time.
o Heatmaps: Displaying data density across geographical locations or matrices.
 Example: Creating a bar chart to compare sales performance across different product
lines.

1.7 Interpretation and Presentation

 Description: After visualizing the data, the next step is to interpret the results and
prepare to communicate them effectively. This often involves creating reports or
presentations.
 Key Aspects:
o Highlighting Key Findings: Focus on the most significant insights.
o Storytelling: Use narrative techniques to guide the audience through the data.
o Visual Design: Ensure that visualizations are clear, accessible, and appealing.
 Example: Presenting a dashboard that includes key metrics and visualizations, explaining
trends in sales and marketing effectiveness.
2. Getting Started with Processing

Data processing is a critical preliminary step that involves organizing, cleaning, and preparing
data for visualization.

2.1 Tools for Data Processing

 Python:
o Libraries: Pandas for data manipulation, NumPy for numerical data operations.
o Example:

import pandas as pd

df = pd.read_csv('sales_data.csv')

df.dropna(inplace=True) # Removes missing values

 R:
o Libraries: dplyr for data manipulation, tidyr for tidying data.
o Example:

library(dplyr)

cleaned_data<- sales_data %>%

filter(!is.na(sales)) # Filters out rows with NA in sales

 SQL: For querying databases and extracting data directly.

o Example:
SELECT * FROM sales WHERE sales > 1000;

2.2 Basic Data Operations

 Aggregation: Summarizing data to derive insights. Common functions include:

o Sum: Total of a numerical column.
 o Mean: Average of a numerical column.
o Count: Number of entries in a column.
 Filtering: Selecting subsets based on certain criteria.
o Example: Filtering sales records where revenue exceeds a threshold.

filtered_df = df[df['revenue'] > 1000]

 Transformation: Modifying data, such as scaling numerical values or creating new

calculated fields.
o Example: Creating a profit margin column.

df['profit_margin'] = df['profit'] / df['revenue']

3. Mapping

Mapping techniques help visualize spatial relationships and distributions within the data.

3.1 Types of Maps

 Choropleth Maps:
o Description: These maps use colors to represent data values in specific
geographical regions.
o Example: A choropleth map showing unemployment rates across different states,
where darker shades indicate higher unemployment.
 Heatmaps:
o Description: Heatmaps indicate density or intensity of data points over a
geographical area.
o Example: A heatmap showing areas of high customer engagement in a city based
on social media activity.
 Dot Maps:
o Description: Represent individual data points as dots on a map, providing a
visual indication of data distribution.
o Example: A dot map displaying the location of all customers within a region.
3.2 Creating Maps

 Libraries and Tools:

o Tableau: User-friendly software for creating interactive maps.
o Python: Libraries like Folium for web-based maps and Geopandas for
geographic data analysis.
o Example using Folium:

import folium

m = folium.Map(location=[45.5236, -122.6750], zoom_start=13)

folium.Marker([45.5236, -122.6750], popup='Portland').add_to(m)

m.save('map.html')
4. Data Exploration and Visualization - Detailed Notes

1. Time Series Analysis

Time series data consists of observations made sequentially over time. Examples include stock
prices, temperature readings, and daily sales data.

Key Concepts:

 Trend: The long-term direction of data (upward, downward, or flat).

 Seasonality: Repeating patterns at regular intervals (e.g., monthly sales spikes).
 Noise: Random variations or irregularities in the data.

Formulas & Calculations:

 Moving Average: A method to smooth out fluctuations.

Where 𝑀𝐴𝑡 the moving is average at time 𝑡, and 𝑥𝑡−𝑖 is the observation at lag 𝑖.

 Exponential Smoothing: Gives more weight to recent observations.

Where 𝑆𝑡 is the smoothed value, α is the smoothing factor, and 𝑥𝑡 is the current
observation.
Example: Suppose we have monthly sales data: January: 100, February: 120, March:
130. Using a 3-month moving average:
2. Connections and Correlations

Covariance:

Covariance measures how two variables move together. If the covariance is positive, both
variables tend to increase together; if it's negative, one increases while the other decreases.

Key Concepts:

 Covariance: Measures how two variables change together.

Where:

 X and Y = variables
 𝑋̅ and 𝑌̅ = means of X and Y
 n = number of data points

If the covariance is positive, the variables tend to increase together.

Example:

Consider height and weight data for 5 people:

 First, find the means 𝑋̅ =180 and 𝑌̅ =70. Now compute the covariance:

A positive covariance suggests that as height increases, weight also increases.

 Correlation: Standardized measure of the strength and direction of a relationship.

Correlation is a standardized version of covariance, measuring the strength and

direction of the linear relationship between two variables. It ranges from -1 to 1,
where:

o 1 = perfect positive correlation,

o -1 = perfect negative correlation,
o 0 = no correlation.

Where:

 σX and σY are the standard deviations of X and Y.

Example:

From the earlier covariance example, if σX=14.1 and σY=12.5, the correlation
would be:

This indicates a moderately strong positive correlation between height and

weight.
3. Scatterplot Maps
Scatterplots help visualize the relationship between two continuous variables. Each point on the
plot represents a pair of values for those two variables.

Trendline (Line of Best Fit):

A trendline is added to scatterplots to summarize the direction of the relationship between

variables. If the points are tightly clustered around the line, it suggests a strong relationship.

Equation of a Straight Line:

y=mx+c

Where:

 m = slope of the line (rate of change of y with respect to x)

 c = intercept (the value of y when x=0)

Calculation of Slope:

The slope is calculated as:

Example:

Let’s use the height and weight data again. After plotting the points, we can calculate the
trendline:
The trendline has a slope of 0.36, indicating that for each unit increase in height, weight
increases by 0.36.

4. Trees, Hierarchies, and Recursion

Trees:

A tree is a hierarchical structure where each node has a parent (except the root) and may have
children. It’s used in many areas of computer science, including data structures (binary trees),
decision making (decision trees), and database indexing.

 Nodes: Represent individual entities.

 Edges: Connect nodes and define relationships between them.

Hierarchies:

Hierarchies represent a parent-child relationship. Organizational charts, file systems, and

taxonomies are examples of hierarchies.

Recursion:

Recursion is a technique where a function calls itself to break down a problem into smaller
subproblems.

Example:

Factorial calculation:

This problem can be broken down using recursion, with 4!=4×3!

5. Networks and Graphs

A graph is a collection of nodes (vertices) and edges (connections) used to model pairwise
relations. Graphs can represent various structures like social networks, roads, and the internet.

Types of Graphs:

 Directed Graph: Edges have a direction (e.g., Twitter followers).

 Undirected Graph: Edges have no direction (e.g., Facebook friends).
 Weighted Graph: Edges have weights to indicate the strength of relationships (e.g., road
distances).

Degree Centrality:

Degree centrality measures the importance of a node based on how many connections it has.

Formula: For an undirected graph:

𝐶𝐷 (v)=deg(v)

Where deg(v) is the number of edges connected to node v.

6. Acquiring Data

Data acquisition is the process of gathering data from various sources like sensors, databases,
APIs, or web scraping. Proper data acquisition is essential for ensuring quality and relevance.

Example:

To acquire stock price data, you can use APIs like Alpha Vantage or Yahoo Finance, which
allow you to retrieve real-time stock prices programmatically.
7. Parsing Data

Parsing involves processing raw data and converting it into a usable format. This can include
reading text, cleaning data, or extracting relevant parts of a dataset.

Example:

Web scraping using Python’s BeautifulSoup library can parse HTML pages and extract data
from specific tags.