Data Analytics Fundamentals

Prepared by: Fatimetou Sidina

 Course Overview

This course introduces fundamental concepts and techniques in data analytics,

focusing on understanding, processing, and extracting insights from data. Through
a combination of theoretical lectures, hands-on exercises, and case studies,
students will develop practical skills in data analysis and interpretation applicable
to various domains.

2
 Course Objectives

● Understand the basics of data analytics and its importance in

decision-making.
● Learn essential data analysis techniques and tools.
● Gain practical experience in working with real-world datasets.
● Develop critical thinking skills for interpreting and communicating
data insights.
● Explore applications of data analytics in different industries.

3
Introduction to Data Analytics

4
 Overview of Data Analytics

● Definition: Data analytics is the process of analyzing, interpreting, and deriving

actionable insights from data to support decision-making.
● Significance: Data analytics plays a crucial role in various industries, including
healthcare, finance, marketing, and more.
● Evolution: Data analytics has evolved significantly over the years, driven by
advancements in technology and increasing availability of data.

5
 Types of Data

● Structured Data: Data organized in a predefined format, such as databases and

spreadsheets.
● Unstructured Data: Data without a predefined structure, including text documents,
images, and videos.
● Semi-Structured Data: Data that does not conform to a strict structure but contains
some organizational properties, such as XML or JSON files.

6
 Data Sources and Collection

● Data Sources: Data can be sourced from various sources such as databases, sensors,
social media, and IoT devices.
● Data Collection Methods: Importance of collecting data through appropriate methods
to ensure data quality and reliability.
● Ethical Considerations: Ethical implications of data collection, including privacy,
consent, and data security.

7
 Hands-on Exercise: Data Exploration
Objective: Explore a sample dataset using Python and Jupyter Notebook to
understand its structure, characteristics, and basic statistics.
Dataset: “sales_data.csv", sample dataset containing information about sales
transactions.
Steps:
1. Import Libraries: Start by importing necessary libraries for data analysis,
such as pandas and NumPy.
import pandas as pd
import numpy as np

2. Load the Dataset: Read the sample dataset into a pandas DataFrame.
sales_df = pd.read_csv("sales_data.csv") 8
 Hands-on Exercise: Data Exploration
3. Exploratory Data Analysis (EDA):
a. Display the first few rows of the dataset to get an overview of the data structure.
b. Check the dimensions of the dataset (number of rows and columns).
c. Explore the data types of each column.
d. Check for missing values and handle them appropriately.

# Display the first few rows of the dataset

print(sales_df.head())

# Check the dimensions of the dataset

print("Dimensions of the dataset:", sales_df.shape)

# Check data types of each column

print("Data types of each column:")
print(sales_df.dtypes)

# Check for missing values

print("Missing values:")
print(sales_df.isnull().sum())
9
 Hands-on Exercise: Data Exploration

4. Summary Statistics:
a. Calculate summary statistics such as mean, median, standard deviation, etc., for numerical
columns.
b. Generate summary statistics for categorical columns (e.g., value counts).

# Summary statistics for numerical columns

print("Summary statistics for numerical columns:")
print(sales_df.describe())

# Summary statistics for categorical columns

print("Summary statistics for categorical columns:")
print(sales_df['category'].value_counts())

10
 Hands-on Exercise: Data Exploration
5. Data Visualization:
a. Visualize distributions of numerical features using histograms or box plots.
b. Create bar plots or pie charts to visualize categorical data.
import matplotlib.pyplot as plt
import seaborn as sns

# Histogram of sales amounts

plt.figure(figsize=(8, 6))
sns.histplot(sales_df['sales_amount'])
plt.title('Distribution of Sales Amounts')
plt.xlabel('Sales Amount')
plt.ylabel('Frequency')
plt.show()

# Bar plot of sales by category

plt.figure(figsize=(10, 6))
sns.countplot(x='category', data=sales_df)
plt.title('Sales by Category')
plt.xlabel('Category')
plt.ylabel('Number of Sales')
plt.xticks(rotation=45)
plt.show() 11
 Hands-on Exercise: Data Exploration

Conclusion: By completing this hands-on exercise, you've gained valuable insights

into the structure and characteristics of the dataset. You've learned how to explore
data using Python and basic data analysis techniques, setting the stage for further
analysis and exploration in subsequent tasks.

12
 Assignment

1. Case Study Analysis:

a. Choose a real-world case study where data analytics has been applied to
solve a problem or optimize a process. You can find case studies in various
domains such as healthcare, finance, marketing, or social media.
b. Analyze the case study and identify:
i. The problem or challenge addressed using data analytics.
ii. The data sources used in the analysis.
iii. The data analytics techniques or methods employed.
iv. The outcomes or insights gained from the analysis.
v. Any limitations or challenges encountered during the process.
2. Write a reflection paper summarizing your analysis of the case study in 800-
1000 words in length.
13
 Additional Resources

❖ How To Use Jupyter NoteBook For Data Analysis (Beginner Tutorial)

❖ Data Analysis and Visualization with Jupyter Notebook

14
Exploratory Data Analysis (EDA) and Data
Wrangling

15
 Introduction to Exploratory Data Analysis
(EDA)

● Definition: EDA is the process of analyzing data sets to summarize their main
characteristics, often with visual methods.
● Importance: EDA helps in understanding the underlying patterns, distributions, and
relationships within the data before building models.
● Key techniques: Summary statistics, data visualization, and handling missing values.

16
 Key Steps in EDA

1. Data Cleaning: Identifying and handling missing values, outliers, and inconsistencies in
the data.
2. Univariate Analysis: Examining the distribution and summary statistics of individual
variables.
3. Bivariate Analysis: Exploring relationships between pairs of variables, often using
scatter plots or correlation matrices.
4. Multivariate Analysis: Analyzing interactions between multiple variables using
techniques like dimensionality reduction or clustering.

17
 Introduction to Data Wrangling

● Definition: Data wrangling, also known as data preprocessing, involves cleaning,

transforming, and enriching raw data into a suitable format for analysis.
● Importance: Data wrangling ensures data quality and prepares the data for further
analysis and modeling.
● Key techniques: Handling missing values, data transformation, and feature
engineering.

18
 Key Steps in Data Wrangling

1. Data Cleaning: Identifying and handling missing or erroneous data points, including
imputation or removal.
2. Data Transformation: Converting data into a format suitable for analysis, such as
normalization or standardization.
3. Feature Engineering: Creating new features or transforming existing ones to improve
model performance or interpretability.

19
 Hands-on Exercise: EDA and Data Wrangling

Objective: Perform exploratory data analysis (EDA) and data wrangling on a sample dataset using Python and
pandas.

Dataset: "customer_transactions.csv", a sample dataset containing information about customer transactions.

Steps:

1. Import Libraries: Start by importing necessary libraries for data analysis, such as pandas and
matplotlib.

import pandas as pd
import matplotlib.pyplot as plt

2. Load the Dataset: Read the sample dataset into a pandas DataFrame.

data = pd.read_csv("customer_transactions.csv")
20
 Hands-on Exercise: EDA and Data Wrangling
3. Exploratory Data Analysis (EDA):
● Display the first few rows of the dataset to understand its structure.
● Check for missing values and handle them appropriately.
● Explore summary statistics and distributions of numerical features.
● Visualize relationships between variables using scatter plots or correlation matrices.

# Display the first few rows of the dataset

print(data.head())

# Check for missing values

print("Missing values:")
print(data.isnull().sum())

# Summary statistics for numerical features

print("Summary statistics:")
print(data.describe())

# Scatter plot of sales amount vs. number of transactions

plt.figure(figsize=(8, 6))
plt.scatter(data['sales_amount'], data['num_transactions'])
plt.title('Sales Amount vs. Number of Transactions')
plt.xlabel('Sales Amount')
plt.ylabel('Number of Transactions')
21
plt.show()
 Hands-on Exercise: EDA and Data Wrangling

4. Data Wrangling and Preprocessing:

● Handle missing values by imputation or removal.
● Perform data transformation such as normalization or standardization.
● Engineer new features or encode categorical variables as necessary.

# Handle missing values (e.g., imputation)

data['sales_amount'].fillna(data['sales_amount'].mean(), inplace=True)

# Data transformation (e.g., normalization)

data['normalized_sales'] = (data['sales_amount'] - data['sales_amount'].mean()) /
data['sales_amount'].std()

# Feature engineering (e.g., creating a new feature)

data['total_revenue'] = data['sales_amount'] * data['num_transactions']

22
 Hands-on Exercise: EDA and Data Wrangling

Conclusion: By completing this hands-on exercise, you've gained practical experience in

performing exploratory data analysis (EDA) and data wrangling on a sample dataset using
Python and pandas. You've learned how to understand the structure of the dataset, explore
its characteristics, handle missing values, and preprocess the data for further analysis.

23
 Assignment
Your task is to apply exploratory data analysis (EDA) and data wrangling techniques to the provided dataset. Here's a breakdown of
the assignment:

Exploratory Data Analysis (EDA):

● Conduct a thorough exploratory data analysis (EDA) to comprehend the structure and characteristics of the dataset.
● Explore various statistical measures, distributions, and patterns within the data.
● Utilize visualizations to uncover insights and trends that may exist in the dataset.
Data Wrangling and Preprocessing:
● Perform data wrangling and preprocessing steps to handle missing values, outliers, and inconsistencies.
● Transform the data as necessary to ensure its quality and suitability for analysis.
● Consider feature engineering techniques to create new features that may enhance the predictive power of the dataset.
Report Writing:
● Compile a detailed report summarizing your findings from the exploratory analysis and data preprocessing.
● Include insights gained from the EDA process, highlighting any significant observations or patterns discovered.
● Document the steps taken during data preprocessing, explaining the rationale behind each transformation.
● Provide recommendations or suggestions based on your analysis, if applicable.
24
 Additional Resources

❖ Exploratory Data Analysis (EDA) using Python and Jupyter Notebooks

❖ Exploratory Data Analysis with Python Jupyter Notebook

25
Applications of Data Analytics

26
 Introduction

● Definition of Data Analytics: Data analytics is the process of analyzing raw data to
derive insights and make informed decisions.

● Importance of Data Analytics: Data analytics empowers organizations to unlock

the value of their data, leading to improved efficiency, decision-making, and
innovation.

27
 Data Analytics in Business

● Business Intelligence: Leveraging data analytics to gain insights into market trends,
customer behavior, and competitor analysis.
● Predictive Analytics: Forecasting future trends and outcomes based on historical data,
enabling proactive decision-making and risk management.
● Customer Relationship Management (CRM): Using data analytics to enhance
customer experiences, personalize marketing strategies, and optimize sales processes.

28
 Data Analytics in Healthcare

● Predictive Modeling: Predicting disease outbreaks, patient readmissions, and

treatment outcomes to improve healthcare delivery and patient care.
● Clinical Decision Support Systems (CDSS): Assisting healthcare professionals in
making evidence-based decisions by analyzing patient data and medical literature.
● Personalized Medicine: Utilizing genetic and clinical data to tailor treatments and
interventions to individual patients, improving treatment efficacy and outcomes.

29
 Data Analytics in Finance

● Fraud Detection: Identifying fraudulent activities and transactions through anomaly

detection and pattern recognition techniques.
● Risk Management: Assessing and managing financial risks using predictive analytics
models to optimize investment strategies and mitigate losses.
● Algorithmic Trading: Using data analytics and machine learning algorithms to analyze
market trends and execute trades automatically, optimizing investment returns.

30
 Data Analytics in Marketing

● Market Segmentation: Dividing customers into distinct groups based on

demographics, behaviors, and preferences to tailor marketing campaigns and
messaging.
● Sentiment Analysis: Analyzing social media and customer feedback data to
understand public sentiment towards products, brands, and campaigns.
● Recommendation Systems: Providing personalized product recommendations to
customers based on their past behaviors and preferences, enhancing customer
engagement and satisfaction.

31
 Data Analytics in Government

● Smart Cities: Leveraging data analytics to optimize city infrastructure, transportation

systems, and public services to improve efficiency and quality of life.
● Public Safety and Security: Analyzing crime data and surveillance footage to identify
patterns, allocate resources effectively, and prevent crime.
● Policy Making: Using data analytics to inform policy decisions and measure the impact
of government initiatives on society and the economy.

32
 Challenges and Opportunities

● Data Privacy and Security: Addressing concerns around data privacy, security
breaches, and ethical use of data in analytics applications.
● Talent Shortage: Overcoming the shortage of skilled data analysts and data scientists
by investing in training and education programs.
● Integration of Technologies: Integrating data analytics with emerging technologies
such as artificial intelligence, machine learning, and IoT to unlock new opportunities
and insights.

33
Final Project: Exploratory Data Analysis and Data Wrangling
Dataset Selection:
● Each student chooses a dataset containing information relevant to a specific domain or topic.
● The dataset will include a variety of variables, including numerical, categorical, and possibly time-series data.
Exploratory Data Analysis (EDA):
● Conduct a comprehensive exploratory data analysis to understand the structure and characteristics of the dataset.
● Explore key statistical measures, distributions, and relationships within the data.
● Utilize visualization techniques to uncover patterns, trends, and anomalies in the data.
Data Wrangling and Preprocessing:
● Perform data wrangling and preprocessing steps to prepare the dataset for analysis.
● Handle missing values, outliers, and inconsistencies using appropriate techniques such as imputation, removal, or transformation.
● Normalize, standardize, or scale numerical features as necessary.
● Engineer new features that may enhance the predictive power or interpretability of the dataset.
Analysis and Interpretation:
● Analyze the cleaned and preprocessed dataset to derive meaningful insights and actionable recommendations.
● Identify trends, correlations, and patterns that may inform decision-making in the relevant domain.
● Use descriptive and inferential statistics to support your analysis and interpretations.
Presentation of Findings:
● Prepare a visually appealing and informative presentation summarizing your findings from the exploratory analysis and data
preprocessing.
● Clearly communicate key insights, trends, and observations derived from the dataset.
● Discuss any challenges encountered during the analysis and the strategies employed to address them.
34