Data Science Essentials: From Beginner to Expert

Your Name
June 7, 2025

Contents
1 Statistics & Probability 2
1.1 Descriptive Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.1 Measures of Central Tendency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.2 Measures of Dispersion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Probability Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2.1 Discrete Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2.2 Continuous Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Hypothesis Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3.1 Common Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Practice Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Python for Data Science 4

2.1 Python Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 Key Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2.1 NumPy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2.2 Pandas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2.3 Matplotlib . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.3 Practice Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 Data Wrangling 5
3.1 Handling Missing Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.1.1 Example in Pandas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.2 Data Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.3 Feature Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.4 Practice Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

4 Data Visualization 6
4.1 Types of Visualizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.2 Seaborn Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.3 Best Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.4 Practice Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

5 Machine Learning Basics 6

5.1 Supervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
5.2 Unsupervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
5.3 Model Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
5.4 Practice Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1
6 SQL for Data Science 7
6.1 Basic Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
6.2 Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
6.2.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
6.3 Practice Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

7 Model Evaluation 8
7.1 Classification Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
7.2 Confusion Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
7.3 ROC & AUC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
7.4 Practice Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

8 End-to-End Projects 9
8.1 Project 1: Customer Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
8.2 Project 2: House Price Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
8.3 Project 3: Sentiment Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1 Statistics & Probability

1.1 Descriptive Statistics
Descriptive statistics summarize and organize characteristics of a data set.

1.1.1 Measures of Central Tendency

• Mean: The average value
n
1X
x̄ = xi
n i=1

• Median: The middle value when data is ordered

• Mode: The most frequent value

1.1.2 Measures of Dispersion

• Variance:
n
1X
2
σ = (xi − x̄)2
n i=1

• Standard Deviation: √
σ= σ2

• Range: Max - Min

1.1.3 Example
Given dataset: [12, 15, 18, 22, 25, 25, 30]

• Mean = (12+15+18+22+25+25+30)/7 21
• Median = 22
• Mode = 25

• Variance 38.57
• Standard Deviation 6.21

2
1.2 Probability Distributions
1.2.1 Discrete Distributions
• Binomial: Fixed trials, two outcomes

P (X = k) = C(n, k)pk (1 − p)n−k

• Poisson: Events in fixed interval

λk e−λ
P (X = k) =
k!

1.2.2 Continuous Distributions

• Normal/Gaussian:
1 (x−µ)2
f (x) = √ e− 2σ2
σ 2π
• Exponential:
f (x) = λe−λx for x ≥ 0

1.3 Hypothesis Testing

Steps:
1. State null (H0 ) and alternative (H1 ) hypotheses
2. Choose significance level (, typically 0.05)

3. Calculate test statistic

4. Determine p-value
5. Make decision (reject or fail to reject H0 )

1.3.1 Common Tests

• t-test: Compare means

• Chi-square: Test independence

• ANOVA: Compare multiple means

1.4 Practice Questions

1. Calculate mean, median, and mode for [8, 10, 12, 14, 14, 16, 20]
2. If X N(50, 10), find P(X ¿ 60)
3. Perform t-test to compare sample [22,25,30] with population mean=20

3
 2 Python for Data Science
2.1 Python Basics
1 # Variables and data types
2 x = 5 # integer
3 y = 3.14 # float
4 name = " Alice " # string
5 is_true = True # boolean
6
7 # Lists
8 numbers = [1 , 2 , 3 , 4 , 5]
9 numbers . append (6)
10
11 # Loops
12 for i in range (5) :
13 print ( i )

2.2 Key Libraries

2.2.1 NumPy

1 import numpy as np
2
3 # Create array
4 arr = np . array ([1 , 2 , 3 , 4 , 5])
5
6 # Array operations
7 mean = np . mean ( arr )
8 std_dev = np . std ( arr )
9
10 # Matrix operations
11 matrix = np . array ([[1 , 2] , [3 , 4]])
12 inverse = np . linalg . inv ( matrix )

2.2.2 Pandas

1 import pandas as pd
2
3 # Create DataFrame
4 data = { ’ Name ’: [ ’ Alice ’ , ’ Bob ’] , ’ Age ’: [25 , 30]}
5 df = pd . DataFrame ( data )
6
7 # Data operations
8 mean_age = df [ ’ Age ’ ]. mean ()
9 filtered = df [ df [ ’ Age ’] > 25]

2.2.3 Matplotlib

1 import matplotlib . pyplot as plt

2
3 # Line plot
4 x = [1 , 2 , 3 , 4]
5 y = [1 , 4 , 9 , 16]
6 plt . plot (x , y )
7 plt . xlabel ( ’X - axis ’)
8 plt . ylabel ( ’Y - axis ’)
9 plt . title ( ’ Simple Plot ’)
10 plt . show ()

4
 2.3 Practice Project
Analyze the Titanic dataset:
• Load data using pandas
• Clean missing values
• Calculate survival rates by passenger class
• Visualize age distribution

3 Data Wrangling
3.1 Handling Missing Data
Strategies:
• Deletion (remove rows/columns)
• Imputation (mean, median, mode)
• Prediction (model-based imputation)

3.1.1 Example in Pandas

1 # Check missing values

2 df . isnull () . sum ()
3
4 # Drop rows with missing values
5 df . dropna ()
6
7 # Fill missing values
8 df . fillna ( df . mean () )

3.2 Data Transformation

• Normalization: Scale to [0,1]
x − min(X)
x′ =
max(X) − min(X)
• Standardization: z-score
x−µ
x′ =
σ

3.3 Feature Engineering

Creating new features:
• Binning continuous variables
• Creating interaction terms
• Encoding categorical variables

3.4 Practice Questions

1. Clean a dataset with 30% missing values
2. Normalize a column with values ranging 0-1000
3. Create dummy variables for a categorical column

5
 4 Data Visualization
4.1 Types of Visualizations
• Histograms: Distribution of data
• Scatter plots: Relationship between variables
• Bar charts: Compare categories
• Box plots: Show spread and outliers

4.2 Seaborn Examples

1 import seaborn as sns
2
3 # Scatter plot with regression line
4 sns . regplot ( x = ’ age ’ , y = ’ income ’ , data = df )
5
6 # Box plot
7 sns . boxplot ( x = ’ class ’ , y = ’ age ’ , data = titanic )
8
9 # Heatmap
10 corr = df . corr ()
11 sns . heatmap ( corr , annot = True )

4.3 Best Practices

• Label axes clearly
• Use appropriate scales
• Choose colors carefully
• Avoid chart junk

4.4 Practice Project

Visualize housing price trends:
• Plot price distribution
• Show relationship between square footage and price
• Compare prices by neighborhood

5 Machine Learning Basics

5.1 Supervised Learning
• Regression: Predict continuous values
– Linear Regression
– Polynomial Regression
• Classification: Predict categories
– Logistic Regression
– Decision Trees
– SVM

6
 5.2 Unsupervised Learning
• Clustering: Group similar data points
– K-means
– Hierarchical
• Dimensionality Reduction:
– PCA
– t-SNE

5.3 Model Implementation

1 from sklearn . linear_model import L i n e a r R e g r e s s i o n
2 from sklearn . mo de l _s el ec t io n import t r a i n _t e s t _ s p l i t
3
4 # Split data
5 X_train , X_test , y_train , y_test = t r a i n _ t e s t _ s p l i t (X , y , test_size =0.2)
6
7 # Train model
8 model = L i n e a r Re g r e s s i o n ()
9 model . fit ( X_train , y_train )
10
11 # Predict
12 predictions = model . predict ( X_test )

5.4 Practice Project

Predict diabetes risk:
• Load Pima Indians Diabetes dataset
• Split into train/test sets
• Train logistic regression model
• Evaluate accuracy

6 SQL for Data Science

6.1 Basic Queries
1 -- Select data
2 SELECT column1 , column2 FROM table WHERE condition ;
3
4 -- Aggregate functions
5 SELECT AVG ( price ) , MAX ( price ) , COUNT (*) FROM products ;
6
7 -- Group by
8 SELECT department , AVG ( salary ) FROM employees GROUP BY department ;

6.2 Joins
• INNER JOIN: Matching rows only
• LEFT JOIN: All rows from left table
• RIGHT JOIN: All rows from right table
• FULL JOIN: All rows from both tables

7
 6.2.1 Example

1 SELECT orders . order_id , customers . name

2 FROM orders
3 INNER JOIN customers ON orders . customer_id = customers . id ;

6.3 Practice Questions

1. Find top 5 customers by total purchases

2. Calculate monthly sales growth

3. Identify products never ordered

7 Model Evaluation
7.1 Classification Metrics
• Accuracy:
TP + TN
TP + TN + FP + FN
• Precision:
TP
TP + FP
• Recall/Sensitivity:
TP
TP + FN
• F1 Score:
P recision × Recall
2×
P recision + Recall

7.2 Confusion Matrix

Predicted Positive Predicted Negative
Actual Positive True Positive (TP) False Negative (FN)
Actual Negative False Positive (FP) True Negative (TN)

7.3 ROC & AUC

• ROC curve plots TPR vs FPR at different thresholds
• AUC measures overall performance (1 = perfect, 0.5 = random)

7.4 Practice Project

Evaluate a spam classifier:

• Calculate precision, recall, F1

• Plot ROC curve
• Analyze false positives

8
8 End-to-End Projects
8.1 Project 1: Customer Segmentation
1. Load customer transaction data

2. Clean and preprocess data

3. Perform exploratory analysis
4. Apply K-means clustering

5. Visualize segments
6. Recommend marketing strategies

8.2 Project 2: House Price Prediction

1. Collect housing data (features + prices)
2. Handle missing values and outliers
3. Engineer new features
4. Train regression models

5. Evaluate using RMSE

6. Deploy as web app

8.3 Project 3: Sentiment Analysis

1. Scrape product reviews
2. Clean text data
3. Create word embeddings

4. Train classification model

5. Analyze sentiment trends

Conclusion
This guide covers the essential data science skills from statistics to machine learning. Master these concepts
through practice and real-world projects to become proficient in data science.

References
[1] James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An Introduction to Statistical Learning.
Springer.
[2] McKinney, W. (2017). Python for Data Analysis. O’Reilly Media.
[3] Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.