Machine Learning

Mathematics in Python
Jamie Flux
https://www.linkedin.com/company/golden-dawn-
engineering/
Contents

1 Introduction to Machine Learning and Mathemat-

ics 14
Overview of Machine Learning . . . . . . . . . . . . 14
Importance of Mathematics in Machine Learning . . 14
Basic Notations and Terminology . . . . . . . . . . . 15
1 Datasets . . . . . . . . . . . . . . . . . . . . . 15
2 Parameters and Variables . . . . . . . . . . . 15
3 Loss Function . . . . . . . . . . . . . . . . . . 15
4 Optimization . . . . . . . . . . . . . . . . . . 16
5 Inference . . . . . . . . . . . . . . . . . . . . 16
6 Python Code . . . . . . . . . . . . . . . . . . 16

2 Linear Algebra Review 18

Vectors and Matrices . . . . . . . . . . . . . . . . . . 18
1 Vectors . . . . . . . . . . . . . . . . . . . . . 18
2 Matrices . . . . . . . . . . . . . . . . . . . . . 18
3 Matrix Operations . . . . . . . . . . . . . . . 19
4 Eigenvalues and Eigenvectors . . . . . . . . . 20
Summary . . . . . . . . . . . . . . . . . . . . . . . . 21

3 Calculus for Machine Learning 22

Derivatives and Integrals . . . . . . . . . . . . . . . . 22
1 Derivatives . . . . . . . . . . . . . . . . . . . 22
2 Integrals . . . . . . . . . . . . . . . . . . . . . 23
3 Partial Derivatives . . . . . . . . . . . . . . . 23
Gradient and Hessian . . . . . . . . . . . . . . . . . 24
1 Gradient . . . . . . . . . . . . . . . . . . . . . 24
2 Hessian . . . . . . . . . . . . . . . . . . . . . 25
Conclusion . . . . . . . . . . . . . . . . . . . . . . . 26

1
4 Probability Theory 27
Basic Probability Rules . . . . . . . . . . . . . . . . 27
1 Sample Space and Events . . . . . . . . . . . 27
2 Probability Axioms . . . . . . . . . . . . . . . 27
3 Conditional Probability . . . . . . . . . . . . 28
4 Independence . . . . . . . . . . . . . . . . . . 28
5 Law of Total Probability . . . . . . . . . . . . 28
Random Variables and Distributions . . . . . . . . . 29
1 Random Variables . . . . . . . . . . . . . . . 29
2 Probability Mass Function (PMF) . . . . . . 29
3 Probability Density Function (PDF) . . . . . 29
4 Cumulative Distribution Function (CDF) . . 30
5 Expected Value and Variance . . . . . . . . . 30
6 Common Probability Distributions . . . . . . 31
Bayes’ Theorem . . . . . . . . . . . . . . . . . . . . . 32
1 Statement of Bayes’ Theorem . . . . . . . . . 32
2 Applications of Bayes’ Theorem . . . . . . . . 32
Conclusion . . . . . . . . . . . . . . . . . . . . . . . 33

5 Statistics Fundamentals 34
Descriptive Statistics . . . . . . . . . . . . . . . . . . 34
1 Measures of Central Tendency . . . . . . . . 34
2 Measures of Variability . . . . . . . . . . . . 35
3 Percentiles and Quartiles . . . . . . . . . . . 35
4 Box Plots . . . . . . . . . . . . . . . . . . . . 35
5 Python Implementation . . . . . . . . . . . . 36
Hypothesis Testing . . . . . . . . . . . . . . . . . . . 36
1 Formulating Hypotheses . . . . . . . . . . . . 37
2 Test Statistic and P-value . . . . . . . . . . . 37
3 Types of Errors . . . . . . . . . . . . . . . . . 37
4 Python Implementation . . . . . . . . . . . . 37
Confidence Intervals . . . . . . . . . . . . . . . . . . 38
1 Interpreting Confidence Intervals . . . . . . . 38
2 Constructing Confidence Intervals . . . . . . 38
3 Python Implementation . . . . . . . . . . . . 39

6 Simple Linear Regression 41

Introduction . . . . . . . . . . . . . . . . . . . . . . . 41
Model Formulation . . . . . . . . . . . . . . . . . . . 41
Least Squares Estimation . . . . . . . . . . . . . . . 41
Model Evaluation . . . . . . . . . . . . . . . . . . . . 42
Python Implementation . . . . . . . . . . . . . . . . 43

2
7 Multiple Linear Regression 45
Introduction . . . . . . . . . . . . . . . . . . . . . . . 45
Model Formulation . . . . . . . . . . . . . . . . . . . 45
Ordinary Least Squares Estimation . . . . . . . . . . 45
Model Evaluation . . . . . . . . . . . . . . . . . . . . 46
Python Implementation . . . . . . . . . . . . . . . . 47

8 Logistic Regression 48
Introduction . . . . . . . . . . . . . . . . . . . . . . . 48
Model Formulation . . . . . . . . . . . . . . . . . . . 48
Maximum Likelihood Estimation . . . . . . . . . . . 49
Model Interpretation . . . . . . . . . . . . . . . . . . 49
Python Implementation . . . . . . . . . . . . . . . . 50

9 Gradient Descent 51
Introduction . . . . . . . . . . . . . . . . . . . . . . . 51
Basic Gradient Descent Algorithm . . . . . . . . . . 51
1 Algorithm . . . . . . . . . . . . . . . . . . . . 51
Stochastic Gradient Descent . . . . . . . . . . . . . . 52
1 Python Implementation . . . . . . . . . . . . 52
Convergence Issues and Optimization . . . . . . . . . 53
Conclusion . . . . . . . . . . . . . . . . . . . . . . . 54

10 Gradient Descent Variants 55

Introduction . . . . . . . . . . . . . . . . . . . . . . . 55
Mini-Batch Gradient Descent . . . . . . . . . . . . . 55
1 Python Implementation . . . . . . . . . . . . 56
Adaptive Learning Rates . . . . . . . . . . . . . . . . 57
1 Python Implementation . . . . . . . . . . . . 57
Momentum-Based Methods . . . . . . . . . . . . . . 58
1 Python Implementation . . . . . . . . . . . . 59

11 Ordinary Least Squares (OLS) 61

Introduction . . . . . . . . . . . . . . . . . . . . . . . 61
Derivation and Explanation . . . . . . . . . . . . . . 61
Gauss-Markov Theorem . . . . . . . . . . . . . . . . 62
Applications in Regression Analysis . . . . . . . . . . 62
1 Python Implementation . . . . . . . . . . . . 63

3
12 Ordinary Least Squares (OLS) 64
Introduction . . . . . . . . . . . . . . . . . . . . . . . 64
Derivation and Explanation . . . . . . . . . . . . . . 64
Applications in Regression . . . . . . . . . . . . . . . 66
1 Python Implementation . . . . . . . . . . . . 66

13 Bayesian Inference 68
Introduction . . . . . . . . . . . . . . . . . . . . . . . 68
Prior and Posterior Distributions . . . . . . . . . . . 68
1 Conjugate Priors . . . . . . . . . . . . . . . . 69
Practical Implementation in Python . . . . . . . . . 69
1 MCMC Sampling with PyMC3 . . . . . . . . 69
2 Variational Inference with PyMC3 . . . . . . 70
Conclusion . . . . . . . . . . . . . . . . . . . . . . . 71
1 Code Snippet: MCMC Sampling with PyMC3 71
2 Code Snippet: Variational Inference with PyMC3 72

14 Naive Bayes Classifier 73

Introduction . . . . . . . . . . . . . . . . . . . . . . . 73
Derivation of the Naive Bayes Classifier . . . . . . . 73
Gaussian Naive Bayes Classifier . . . . . . . . . . . . 74
Application in Text Classification . . . . . . . . . . . 75
Python Code . . . . . . . . . . . . . . . . . . . . . . 75
1 Code Explanation . . . . . . . . . . . . . . . 75
Conclusion . . . . . . . . . . . . . . . . . . . . . . . 76

15 K-Nearest Neighbors (K-NN) 77

Introduction . . . . . . . . . . . . . . . . . . . . . . . 77
Distance Metrics . . . . . . . . . . . . . . . . . . . . 77
1 Euclidean Distance . . . . . . . . . . . . . . . 78
2 Manhattan Distance . . . . . . . . . . . . . . 78
3 Minkowski Distance . . . . . . . . . . . . . . 78
K-NN Algorithm . . . . . . . . . . . . . . . . . . . . 78
1 Step 1: Select K . . . . . . . . . . . . . . . . 78
2 Step 2: Calculate Distances . . . . . . . . . . 79
3 Step 3: Find K Nearest Neighbors . . . . . . 79
4 Step 4: Assign Class Label . . . . . . . . . . 79
Computational Complexity . . . . . . . . . . . . . . 79
Python Code . . . . . . . . . . . . . . . . . . . . . . 80
1 Code Explanation . . . . . . . . . . . . . . . 80
Conclusion . . . . . . . . . . . . . . . . . . . . . . . 81

4
16 Decision Trees 82
Introduction . . . . . . . . . . . . . . . . . . . . . . . 82
Entropy and Information Gain . . . . . . . . . . . . 82
Decision Tree Algorithm . . . . . . . . . . . . . . . . 83
1 Step 1: Select Best Split . . . . . . . . . . . . 83
2 Step 2: Assign Leaf Node or Recurse . . . . . 83
Computational Complexity . . . . . . . . . . . . . . 83
Python Code . . . . . . . . . . . . . . . . . . . . . . 84
1 Code Explanation . . . . . . . . . . . . . . . 84

17 Random Forests 85
Introduction . . . . . . . . . . . . . . . . . . . . . . . 85
Bagging and Decision Trees . . . . . . . . . . . . . . 85
1 Bagging . . . . . . . . . . . . . . . . . . . . . 85
2 Decision Trees . . . . . . . . . . . . . . . . . 85
Random Forest Algorithm . . . . . . . . . . . . . . . 86
1 Random Subset of Features . . . . . . . . . . 86
2 Building the Ensemble . . . . . . . . . . . . . 86
Python Code . . . . . . . . . . . . . . . . . . . . . . 86
1 Code Explanation . . . . . . . . . . . . . . . 87
Conclusion . . . . . . . . . . . . . . . . . . . . . . . 87

18 Support Vector Machines (SVM) 89

Margin and Hyperplanes . . . . . . . . . . . . . . . . 89
1 Margin Maximization . . . . . . . . . . . . . 89
2 Soft Margin SVM . . . . . . . . . . . . . . . 90
Kernel Trick . . . . . . . . . . . . . . . . . . . . . . . 90
1 Commonly Used Kernels . . . . . . . . . . . . 90
2 Kernel Trick in Dual Formulation . . . . . . . 91
Python Code . . . . . . . . . . . . . . . . . . . . . . 91
1 Code Explanation . . . . . . . . . . . . . . . 91

19 Principal Component Analysis (PCA) 93

Covariance Matrix . . . . . . . . . . . . . . . . . . . 93
1 Python Code . . . . . . . . . . . . . . . . . . 93
Eigen Decomposition . . . . . . . . . . . . . . . . . . 93
1 Python Code . . . . . . . . . . . . . . . . . . 94
Dimensionality Reduction . . . . . . . . . . . . . . . 94
1 Python Code . . . . . . . . . . . . . . . . . . 94

5
20 K-Means Clustering 95
Introduction . . . . . . . . . . . . . . . . . . . . . . . 95
Distance Metrics . . . . . . . . . . . . . . . . . . . . 95
1 Python Code . . . . . . . . . . . . . . . . . . 95
Algorithm Steps . . . . . . . . . . . . . . . . . . . . 96
1 Initialization . . . . . . . . . . . . . . . . . . 96
2 Assignment . . . . . . . . . . . . . . . . . . . 96
3 Update . . . . . . . . . . . . . . . . . . . . . 96
4 Iteration . . . . . . . . . . . . . . . . . . . . . 96
5 Python Code . . . . . . . . . . . . . . . . . . 96
Computational Complexity . . . . . . . . . . . . . . 97
Choosing the Number of Clusters . . . . . . . . . . . 97
1 Elbow Method . . . . . . . . . . . . . . . . . 97
2 Python Code . . . . . . . . . . . . . . . . . . 97
Conclusion . . . . . . . . . . . . . . . . . . . . . . . 98

21 Expectation-Maximization (EM) 99
Gaussian Mixture Models . . . . . . . . . . . . . . . 99
1 Python Code . . . . . . . . . . . . . . . . . . 100
The Expectation-Maximization Algorithm . . . . . . 100
1 The E-step . . . . . . . . . . . . . . . . . . . 100
2 The M-step . . . . . . . . . . . . . . . . . . . 100
3 Python Code . . . . . . . . . . . . . . . . . . 101
Convergence Criteria . . . . . . . . . . . . . . . . . . 101
1 Python Code . . . . . . . . . . . . . . . . . . 102
Conclusion . . . . . . . . . . . . . . . . . . . . . . . 102

22 Hierarchical Clustering 103

Agglomerative vs. Divisive Hierarchical Clustering . 103
Linkage Criteria . . . . . . . . . . . . . . . . . . . . 104
Dendrograms Interpretation . . . . . . . . . . . . . . 104
1 Python Code . . . . . . . . . . . . . . . . . . 105

23 Reinforcement Learning Basics 106

Markov Decision Processes (MDPs) . . . . . . . . . . 106
Bellman Equations . . . . . . . . . . . . . . . . . . . 107
1 Value function . . . . . . . . . . . . . . . . . 107
2 Action-value function . . . . . . . . . . . . . 107
Policy and Value Iterations . . . . . . . . . . . . . . 107
1 Policy Iteration . . . . . . . . . . . . . . . . . 107
2 Value Iteration . . . . . . . . . . . . . . . . . 108
3 Python Code . . . . . . . . . . . . . . . . . . 108

6
24 Q-Learning 110
Introduction . . . . . . . . . . . . . . . . . . . . . . . 110
Q-Function . . . . . . . . . . . . . . . . . . . . . . . 110
Bellman Update . . . . . . . . . . . . . . . . . . . . 110
1 Python Code . . . . . . . . . . . . . . . . . . 111
Convergence . . . . . . . . . . . . . . . . . . . . . . . 111
Conclusion . . . . . . . . . . . . . . . . . . . . . . . 111

25 Deep Q-Learning 112

Introduction . . . . . . . . . . . . . . . . . . . . . . . 112
Q-Network . . . . . . . . . . . . . . . . . . . . . . . 112
Experience Replay . . . . . . . . . . . . . . . . . . . 112
Target Network . . . . . . . . . . . . . . . . . . . . . 113
Loss Function . . . . . . . . . . . . . . . . . . . . . . 113
1 Python Code . . . . . . . . . . . . . . . . . . 113
Epsilon-Greedy Exploration . . . . . . . . . . . . . . 114
Conclusion . . . . . . . . . . . . . . . . . . . . . . . 114

26 Policy Gradient Methods 115

Introduction . . . . . . . . . . . . . . . . . . . . . . . 115
Policy Function . . . . . . . . . . . . . . . . . . . . . 115
REINFORCE Algorithm . . . . . . . . . . . . . . . . 115
1 Python Code . . . . . . . . . . . . . . . . . . 116
Advantage Actor-Critic (A2C) . . . . . . . . . . . . 116
1 Python Code . . . . . . . . . . . . . . . . . . 116
Proximal Policy Optimization (PPO) . . . . . . . . . 117
1 Python Code . . . . . . . . . . . . . . . . . . 117
Conclusion . . . . . . . . . . . . . . . . . . . . . . . 118

27 Convolutional Neural Networks (CNNs) 119

Introduction . . . . . . . . . . . . . . . . . . . . . . . 119
Convolution Operation . . . . . . . . . . . . . . . . . 119
1 Python Code . . . . . . . . . . . . . . . . . . 120
Pooling Layers . . . . . . . . . . . . . . . . . . . . . 120
1 Python Code . . . . . . . . . . . . . . . . . . 120
Activation Functions . . . . . . . . . . . . . . . . . . 121
1 Python Code . . . . . . . . . . . . . . . . . . 121
Conclusion . . . . . . . . . . . . . . . . . . . . . . . 122
Introduction . . . . . . . . . . . . . . . . . . . . . . . 123
Convolution Operation . . . . . . . . . . . . . . . . . 123
Pooling Layers . . . . . . . . . . . . . . . . . . . . . 124
Activation Functions . . . . . . . . . . . . . . . . . . 124

7
 1 ReLU Activation Function Code . . . . . . . 125

28 Recurrent Neural Networks (RNNs) 126

Introduction . . . . . . . . . . . . . . . . . . . . . . . 126
The Basic RNN Structure . . . . . . . . . . . . . . . 126
Training RNNs using Backpropagation Through Time
(BPTT) . . . . . . . . . . . . . . . . . . . . . . . . . 127
Python Code: RNN Forward Pass . . . . . . . . . . 127

29 Generative Adversarial Networks (GAN) 129

Introduction . . . . . . . . . . . . . . . . . . . . . . . 129
The GAN Framework . . . . . . . . . . . . . . . . . 129
1 Generator . . . . . . . . . . . . . . . . . . . . 129
2 Discriminator . . . . . . . . . . . . . . . . . . 130
3 Objective Function . . . . . . . . . . . . . . . 130
GAN Training Procedure . . . . . . . . . . . . . . . 130
1 Discriminator Updates . . . . . . . . . . . . . 130
2 Generator Updates . . . . . . . . . . . . . . . 131
Python Code: GAN Training Procedure . . . . . . . 131

30 Transfer Learning 133

Introduction . . . . . . . . . . . . . . . . . . . . . . . 133
Problem Formulation . . . . . . . . . . . . . . . . . . 133
Transfer Learning Strategies . . . . . . . . . . . . . . 133
1 Feature-Based Transfer Learning . . . . . . . 134
2 Model-Based Transfer Learning . . . . . . . . 134
3 Instance-Based Transfer Learning . . . . . . . 135
Python Code: Fine-tuning . . . . . . . . . . . . . . . 135

31 Hyperparameter Tuning 137

Introduction . . . . . . . . . . . . . . . . . . . . . . . 137
Problem Formulation . . . . . . . . . . . . . . . . . . 137
Grid Search . . . . . . . . . . . . . . . . . . . . . . . 138
Random Search . . . . . . . . . . . . . . . . . . . . . 138
Bayesian Optimization . . . . . . . . . . . . . . . . . 138
Python Code: Grid Search . . . . . . . . . . . . . . . 139

32 Cross-Validation Techniques 140

Introduction . . . . . . . . . . . . . . . . . . . . . . . 140
k-Fold Cross-Validation . . . . . . . . . . . . . . . . 140
Leave-One-Out Cross-Validation . . . . . . . . . . . 141
Stratified Cross-Validation . . . . . . . . . . . . . . . 141
Python Code: k-fold Cross-Validation . . . . . . . . 142

8
33 Regularization Techniques 143
Introduction . . . . . . . . . . . . . . . . . . . . . . . 143
L1 (Lasso) and L2 (Ridge) Regularization . . . . . . 143
1 L1 Regularization (Lasso) . . . . . . . . . . . 144
2 L2 Regularization (Ridge) . . . . . . . . . . . 144
Dropout . . . . . . . . . . . . . . . . . . . . . . . . . 144
1 Python Code: Dropout . . . . . . . . . . . . 145
Batch Normalization . . . . . . . . . . . . . . . . . . 145
1 Python Code: Batch Normalization . . . . . 146

34 Dimensionality Reduction Techniques 148

Introduction . . . . . . . . . . . . . . . . . . . . . . . 148
t-SNE . . . . . . . . . . . . . . . . . . . . . . . . . . 148
1 Python Code: t-SNE . . . . . . . . . . . . . . 149
UMAP . . . . . . . . . . . . . . . . . . . . . . . . . . 149
1 Python Code: UMAP . . . . . . . . . . . . . 150
ICA . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
1 Python Code: ICA . . . . . . . . . . . . . . . 151

35 Markov Chain Monte Carlo (MCMC) 152

Introduction . . . . . . . . . . . . . . . . . . . . . . . 152
Metropolis-Hastings Algorithm . . . . . . . . . . . . 152
1 Python Code: Metropolis-Hastings Algorithm 153
Gibbs Sampling . . . . . . . . . . . . . . . . . . . . . 154
1 Python Code: Gibbs Sampling . . . . . . . . 154
Applications in Bayesian Inference . . . . . . . . . . 155

36 Hidden Markov Models (HMM) 156

Introduction . . . . . . . . . . . . . . . . . . . . . . . 156
Hidden Markov Model . . . . . . . . . . . . . . . . . 156
1 HMM Notation . . . . . . . . . . . . . . . . . 156
2 HMM Probabilities . . . . . . . . . . . . . . . 157
Forward-Backward Algorithm . . . . . . . . . . . . . 157
1 Python Code: Forward-Backward Algorithm 158
Viterbi Algorithm . . . . . . . . . . . . . . . . . . . 161
1 Python Code: Viterbi Algorithm . . . . . . . 161
Conclusion . . . . . . . . . . . . . . . . . . . . . . . 162

37 Time Series Analysis 163

ARIMA Models . . . . . . . . . . . . . . . . . . . . . 163
1 Autoregressive Model . . . . . . . . . . . . . 163
2 Moving Average Model . . . . . . . . . . . . . 164

9
 3 ARIMA Model . . . . . . . . . . . . . . . . . 164

38 Text Mining and NLP 166

Introduction . . . . . . . . . . . . . . . . . . . . . . . 166
1 Text Representation . . . . . . . . . . . . . . 166
2 Text Preprocessing . . . . . . . . . . . . . . . 167
3 Text Mining Techniques . . . . . . . . . . . . 168

39 Sequence Modeling 170

Introduction . . . . . . . . . . . . . . . . . . . . . . . 170
1 Hidden State Representation . . . . . . . . . 170
2 Beam Search . . . . . . . . . . . . . . . . . . 171
3 Sequence-to-Sequence Models . . . . . . . . . 171
4 Beam Search with Hidden State Prediction . 173

40 Entropy and Information Theory 177

Introduction . . . . . . . . . . . . . . . . . . . . . . . 177
Shannon Entropy . . . . . . . . . . . . . . . . . . . . 177
KL Divergence . . . . . . . . . . . . . . . . . . . . . 178
Mutual Information . . . . . . . . . . . . . . . . . . 179

41 Computational Complexity 182

Introduction . . . . . . . . . . . . . . . . . . . . . . . 182
Time Complexity . . . . . . . . . . . . . . . . . . . . 182
1 Big O Notation . . . . . . . . . . . . . . . . . 182
2 Common Time Complexities . . . . . . . . . 183
Space Complexity . . . . . . . . . . . . . . . . . . . . 183
1 Common Space Complexities . . . . . . . . . 183
Python Implementation . . . . . . . . . . . . . . . . 184
Conclusion . . . . . . . . . . . . . . . . . . . . . . . 185

42 Game Theory 186

1 Nash Equilibrium . . . . . . . . . . . . . . . . 186

43 Optimization Techniques 189

1 Convex Optimization . . . . . . . . . . . . . 189

44 Sparse Coding 192

Basis Functions . . . . . . . . . . . . . . . . . . . . . 192
1 Mathematical Representation . . . . . . . . . 192
2 Sparsity Constraint . . . . . . . . . . . . . . . 192
3 Optimization Problem . . . . . . . . . . . . . 193
Dictionary Learning . . . . . . . . . . . . . . . . . . 193

10
 1 Sparse Coding . . . . . . . . . . . . . . . . . 194
2 Dictionary Update . . . . . . . . . . . . . . . 194
3 Dictionary Learning Algorithm . . . . . . . . 195

45 Multi-Task Learning 196

Introduction . . . . . . . . . . . . . . . . . . . . . . . 196
Problem Formulation . . . . . . . . . . . . . . . . . . 196
1 Single-Task Learning . . . . . . . . . . . . . . 196
2 Multi-Task Learning . . . . . . . . . . . . . . 197
Benefits of Multi-Task Learning . . . . . . . . . . . . 197
1 Improved Generalization . . . . . . . . . . . . 197
2 Data Efficiency . . . . . . . . . . . . . . . . . 197
3 Reduced Overfitting . . . . . . . . . . . . . . 197
4 Transfer Learning . . . . . . . . . . . . . . . 198
Multi-Task Learning Algorithms . . . . . . . . . . . 198
1 Parameter Sharing . . . . . . . . . . . . . . . 198
2 Regularization . . . . . . . . . . . . . . . . . 198
3 Task Relationship Modeling . . . . . . . . . . 199
Summary . . . . . . . . . . . . . . . . . . . . . . . . 199
Python Code . . . . . . . . . . . . . . . . . . . . . . 200

46 Meta-Learning 201
Introduction . . . . . . . . . . . . . . . . . . . . . . . 201
Problem Formulation . . . . . . . . . . . . . . . . . . 201
1 Single-Learning Task . . . . . . . . . . . . . . 201
2 Meta-Learning . . . . . . . . . . . . . . . . . 201
Meta-Learning Algorithms . . . . . . . . . . . . . . . 202
1 Model-Agnostic Meta-Learning (MAML) . . 202
2 Meta-Learning with Recurrent Neural Net-
works (meta-RNN) . . . . . . . . . . . . . . . 202
Mathematical Representation . . . . . . . . . . . . . 203
1 Model-Agnostic Meta-Learning (MAML) . . 203
2 Meta-Learning with Recurrent Neural Net-
works (meta-RNN) . . . . . . . . . . . . . . . 204
Python Code . . . . . . . . . . . . . . . . . . . . . . 204

47 Bayesian Networks 206

Introduction . . . . . . . . . . . . . . . . . . . . . . . 206
Formal Definition . . . . . . . . . . . . . . . . . . . . 206
Conditional Probability Distribution . . . . . . . . . 206
Bayesian Network Inference . . . . . . . . . . . . . . 207
1 Exact Inference: Variable Elimination . . . . 207

11
 2 Approximate Inference: Markov Chain Monte
Carlo . . . . . . . . . . . . . . . . . . . . . . 207
Learning Bayesian Networks . . . . . . . . . . . . . . 207
1 Constraint-Based Methods: PC Algorithm . . 207
2 Score-Based Methods: Maximum Likelihood
Estimation . . . . . . . . . . . . . . . . . . . 208
Python Implementation . . . . . . . . . . . . . . . . 208

48 Optimization Techniques 209

Introduction . . . . . . . . . . . . . . . . . . . . . . . 209
Convex Optimization . . . . . . . . . . . . . . . . . . 209
Quadratic Programming . . . . . . . . . . . . . . . . 210
Lagrange Multipliers . . . . . . . . . . . . . . . . . . 210
Python Implementation . . . . . . . . . . . . . . . . 211

49 Bifurcation Theory 213

Stability Analysis . . . . . . . . . . . . . . . . . . . . 213
Fixed Points and Periodic Orbits . . . . . . . . . . . 214
Applications in Dynamical Systems . . . . . . . . . . 214
Python Implementation . . . . . . . . . . . . . . . . 215

50 Topological Data Analysis (TDA) 217

Persistent Homology . . . . . . . . . . . . . . . . . . 217
1 Definition of Persistence Diagrams . . . . . . 217
2 Computation of Persistent Homology . . . . . 218
3 Applications of Persistent Homology . . . . . 218
Betti Numbers . . . . . . . . . . . . . . . . . . . . . 219
1 Definition of Betti Numbers . . . . . . . . . . 219
2 Computing Betti Numbers . . . . . . . . . . 220
3 Interpretation of Betti Numbers . . . . . . . 220
Python Implementation . . . . . . . . . . . . . . . . 220

51 Spiking Neural Networks (SNN) 222

Neuron Models . . . . . . . . . . . . . . . . . . . . . 222
1 Introduce Leaky Integrate-and-Fire (LIF) model222
2 Describe Spike Generation and Resetting . . 222
3 Describe Spike-Timing-Dependent Plasticity
(STDP) . . . . . . . . . . . . . . . . . . . . . 223
SNN Architecture . . . . . . . . . . . . . . . . . . . . 223
1 Describe Feedforward Architecture . . . . . . 223
2 Describe Spiking Activation Functions . . . . 224
3 Describe Spike-Time Encoding for Inputs . . 224

12
 Python Implementation . . . . . . . . . . . . . . . . 225

52 Federated Learning 227

Introduction . . . . . . . . . . . . . . . . . . . . . . . 227
Data Privacy Concerns . . . . . . . . . . . . . . . . . 227
1 Secure Aggregation . . . . . . . . . . . . . . . 227
2 Differential Privacy . . . . . . . . . . . . . . . 228
Distributed Training . . . . . . . . . . . . . . . . . . 228
1 Server-Client Communication . . . . . . . . . 228
2 Aggregation and Model Update . . . . . . . . 229
3 Model Synchronization . . . . . . . . . . . . . 229
Applications in Collaborative AI . . . . . . . . . . . 230
1 Healthcare . . . . . . . . . . . . . . . . . . . 230
2 Smart Grids . . . . . . . . . . . . . . . . . . . 230
3 Internet of Things (IoT) . . . . . . . . . . . . 230
Conclusion . . . . . . . . . . . . . . . . . . . . . . . 230

53 Quantum Machine Learning 231

Quantum Computing Basics . . . . . . . . . . . . . . 231
1 Quantum Bits (Qubits) . . . . . . . . . . . . 231
2 Quantum Gates . . . . . . . . . . . . . . . . . 232
3 Quantum Entanglement . . . . . . . . . . . . 232
4 Quantum Algorithms . . . . . . . . . . . . . 233
Quantum Algorithms for Machine Learning . . . . . 233
1 Quantum Kernels . . . . . . . . . . . . . . . . 234
2 Quantum Support Vector Machines (QSVM) 234
Potential and Challenges . . . . . . . . . . . . . . . . 235

13
Chapter 1

Introduction to
Machine Learning and
Mathematics

Overview of Machine Learning

Machine Learning (ML) is a subfield of Artificial Intelligence that
focuses on the development of algorithms and statistical models
that enable computers to learn from and make predictions or de-
cisions based on data. The goal of ML is to extract meaningful
patterns or representations from data, without being explicitly pro-
grammed. It is highly interdisciplinary, drawing from various fields
such as statistics, optimization, linear algebra, and probability the-
ory.

Importance of Mathematics in Machine

Learning
Mathematics plays a crucial role in the theory and practice of ma-
chine learning. ML algorithms are based on mathematical princi-
ples and mathematical formalisms provide a solid foundation for
understanding and analyzing these algorithms. Mathematics helps
us quantify uncertainty, optimize model parameters, and make in-
ference based on data. Furthermore, mathematical techniques are

14
used to design and train neural networks, tune hyperparameters,
and evaluate model performance.

Basic Notations and Terminology

In order to understand machine learning concepts and algorithms,
it is important to be familiar with some basic notations and termi-
nology used in this field.

1 Datasets
A dataset is a collection of instances or examples that are used to
train, validate, and test machine learning models. In mathematical
notation, a dataset can be represented as follows:

D = {(x1 , y1 ), (x2 , y2 ), . . . , (xn , yn )}

where xi denotes the i-th input instance and yi denotes the
corresponding output or target value. The dataset D typically
consists of a training set, a validation set, and a test set.

2 Parameters and Variables

In machine learning models, parameters are variables that can be
learned or adjusted based on the given data. These parameters de-
fine the behavior or characteristics of the model. In mathematical
notation, parameters are denoted by θ or w. For example, in linear
regression, we have the following equation:

y = w0 + w1 x1 + w2 x2 + . . . + wn xn
where w0 , w1 , . . . , wn are the parameters to be learned.
Variables, on the other hand, are quantities that can take dif-
ferent values. In the context of ML, variables represent the features
or attributes of the data. They are denoted by xi .

3 Loss Function
A loss function measures the discrepancy between the predicted
output of a ML model and the true output. It quantifies the error or
the cost associated with the model’s predictions. In mathematical
notation, a loss function can be denoted by L(θ) or L(w), where θ
or w represents the parameters of the model. Commonly used loss

15
functions include mean squared error (MSE), cross-entropy loss,
and hinge loss.

4 Optimization
In machine learning, optimization is the process of finding the best
set of parameters that minimize the loss function. This is typically
done through an iterative algorithm known as optimization algo-
rithm or solver. The most commonly used optimization algorithm
in ML is gradient descent, which updates the parameters in the di-
rection of steepest descent of the loss function. Other optimization
algorithms include stochastic gradient descent, Adam, and LBFGS.

5 Inference
Inference refers to the process of making predictions or decisions
based on the trained machine learning model. Given a new in-
put instance, the model uses the learned parameters and performs
calculations to generate an output or prediction. The process of
inference can be mathematically represented as:

ŷ = f (x; θ)
where ŷ is the predicted output, f (·) represents the ML model,
x is the input instance, and θ denotes the learned parameters.

6 Python Code
Here is an example of a Python code snippet that demonstrates
the calculation of mean squared error (MSE) loss:

import numpy as np

def mean_squared_error(y_true, y_pred):

return np.mean((y_true - y_pred)**2)

# Example usage
y_true = [1, 2, 3, 4, 5]
y_pred = [1.1, 2.2, 2.8, 3.9, 5.1]

mse = mean_squared_error(y_true, y_pred)

print("Mean Squared Error:", mse)

16
 In this code, the mean_squared_error function takes two input
arrays, y_true and y_pred, and calculates the mean squared error
between them using the NumPy library. The resulting MSE value
is then printed.

17
Chapter 2

Linear Algebra Review

Vectors and Matrices

Linear algebra serves as a fundamental mathematical framework for
understanding many concepts in machine learning. In this chapter,
we will review key concepts in linear algebra, starting with vectors
and matrices.

1 Vectors
A vector is a mathematical object that represents both magnitude
and direction. In machine learning, vectors are often used to repre-
sent features or inputs. A vector of dimension n can be denoted as
x = [x1 , x2 , . . . , xn ]T , where the superscript T denotes the trans-
pose operation.
In Python, vectors can be represented using NumPy arrays.
The following code snippet demonstrates the creation of a vector:

import numpy as np

x = np.array([1, 2, 3]) # Create a vector

print("Vector x:", x)

2 Matrices
A matrix is a 2-dimensional array of numbers, where each element
is called a scalar. Matrices play a crucial role in linear transforma-

18
tions and computations. A matrix with m rows and n columns can
be denoted as A = [aij ]m×n .
In Python, matrices can be represented using NumPy arrays.
The following code snippet demonstrates the creation of a matrix:
import numpy as np

A = np.array([[1, 2, 3], [4, 5, 6]]) # Create a

,→ matrix
print("Matrix A:")
print(A)

3 Matrix Operations
Various operations can be performed on matrices, including addi-
tion, subtraction, and multiplication.

Addition and Subtraction

Addition and subtraction of matrices are performed element-wise.
Given two matrices A and B of the same dimensions, the sum of
A and B is denoted as C = A + B, and the difference of A and B
is denoted as D = A − B.
In Python, matrix addition and subtraction can be done using
NumPy:
import numpy as np

A = np.array([[1, 2], [3, 4]])

B = np.array([[5, 6], [7, 8]])

C = A + B # Matrix addition
D = A - B # Matrix subtraction

print("Matrix C (Addition):")
print(C)
print("Matrix D (Subtraction):")
print(D)

Multiplication
Matrix multiplication is a more involved operation. The product
of two matrices A and B is denoted as C = AB. For two matrices

19
to be compatible for multiplication, the number of columns in the
first matrix must be equal to the number of rows in the second
matrix.
In Python, matrix multiplication can be done using the numpy.matmul
function or using the @ operator:

import numpy as np

A = np.array([[1, 2], [3, 4]])

B = np.array([[5, 6], [7, 8]])

C = np.matmul(A, B) # Matrix multiplication

D = A @ B # Matrix multiplication using @ operator

print("Matrix C (Multiplication):")
print(C)
print("Matrix D (Multiplication with @):")
print(D)

4 Eigenvalues and Eigenvectors

Eigenvalues and eigenvectors are important concepts in linear al-
gebra, with applications in various machine learning algorithms.
Given a square matrix A, an eigenvector v and its corresponding
eigenvalue λ satisfy the equation:

Av = λv
Eigenvectors represent the direction of linear transformations,
while eigenvalues represent the scalar factor by which the eigenvec-
tor is scaled.
In Python, eigendecomposition can be done using numpy.linalg.eig:

import numpy as np

A = np.array([[1, 2], [3, 4]])

eigenvalues, eigenvectors = np.linalg.eig(A)

print("Eigenvalues:")
print(eigenvalues)

20
print("Eigenvectors:")
print(eigenvectors)

In this code, numpy.linalg.eig returns an array of eigenvalues

and a 2-dimensional array of eigenvectors.

Summary
In this chapter, we reviewed key concepts in linear algebra, includ-
ing vectors, matrices, and their operations. We also introduced
eigenvectors and eigenvalues. Understanding these concepts is cru-
cial for the understanding of many machine learning algorithms
and their implementations.

21
Chapter 3

Calculus for Machine

Learning

Derivatives and Integrals

In this section, we will review the concepts of derivatives and inte-
grals, which form the basis of calculus. These concepts are essential
in machine learning for optimizing models and understanding the
behavior of functions.

1 Derivatives
The derivative of a function measures how the function changes
as its input changes. Geometrically, it represents the slope of the
tangent line to the curve at a given point. Mathematically, the
derivative of a function f (x) at a point x is defined as:

f (x + h) − f (x)
f ′ (x) = lim
h→0 h
To compute derivatives in Python, we can use the SymPy li-
brary. The following code snippet demonstrates how to calculate
the derivative of a function:

import sympy as sp

x = sp.symbols('x') # Define the variable

f = x**2 + 2*x + 1 # Define the function

22
derivative = sp.diff(f, x) # Calculate the
,→ derivative

print("Derivative of f(x):")
print(derivative)

The code above defines the function f (x) = x2 + 2x + 1 and

calculates its derivative with respect to x. The result is f ′ (x) =
2x + 2.

2 Integrals
The integral of a function measures the area under the curve de-
fined by the function. It is the reverse process of differentiation.
Mathematically, the integral of a function f (x) over an interval
[a, b] is denoted as:
Z b
f (x) dx
a
To compute integrals in Python, we can again use the SymPy
library. The following code snippet demonstrates how to calculate
the integral of a function:

import sympy as sp

x = sp.symbols('x') # Define the variable

f = x**2 + 2*x + 1 # Define the function

integral = sp.integrate(f, (x, 0, 1)) # Calculate

,→ the integral from 0 to 1

print("Integral of f(x) from 0 to 1:")

print(integral)

The code above defines the function f (x) = x2 + 2x + 1 and

R1
calculates its integral from 0 to 1. The result is 0 (x2 +2x+1) dx =
3.
7

3 Partial Derivatives
In machine learning, it is common to deal with functions that de-
pend on multiple variables. In such cases, we can compute partial

23
derivatives to measure the rate of change of the function with re-
spect to each variable individually.
The partial derivative of a function f (x, y) with respect to x
∂x , and it measures how f changes as x changes
is denoted as ∂f
while keeping y constant. Similarly, the partial derivative of f
with respect to y is denoted as ∂f
∂y .
To compute partial derivatives using SymPy, we can modify our
previous code snippet as follows:

import sympy as sp

x, y = sp.symbols('x y') # Define the variables

f = x**2 + 2*x*y + y**2 # Define the function

partial_x = sp.diff(f, x) # Calculate the partial

,→ derivative with respect to x
partial_y = sp.diff(f, y) # Calculate the partial
,→ derivative with respect to y

print("Partial derivative of f with respect to x:")

print(partial_x)

print("Partial derivative of f with respect to y:")

print(partial_y)

In this code, we define the function f (x, y) = x2 + 2xy + y 2 and

calculate its partial derivatives with respect to x and y.

Gradient and Hessian

The gradient and Hessian matrix are important concepts in calcu-
lus, particularly in optimization. They provide information about
the rate of change and curvature of a function, respectively.

1 Gradient
The gradient of a function f (x) is a vector that consists of the par-
tial derivatives of f with respect to each variable. Mathematically,
the gradient is defined as:
 T
∂f ∂f ∂f
∇f (x) = , ,...,
∂x1 ∂x2 ∂xn

24
 The gradient provides the direction of the steepest ascent of the
function. By taking steps in the opposite direction of the gradient,
we can iteratively approach the minimum of a function.
In Python, we can compute the gradient using SymPy:

import sympy as sp

x, y = sp.symbols('x y') # Define the variables

f = x**2 + 2*y**2 + 3*x*y # Define the function

gradient = [sp.diff(f, var) for var in (x, y)] #

,→ Calculate the gradient

print("Gradient of f(x, y):")

print(gradient)

The code above defines the function f (x, y) = x2 + 2y 2 + 3xy

and computes its gradient with respect to x and y.

2 Hessian
The Hessian matrix of a function f (x) is a square matrix that
consists of the second-order partial derivatives of f . The (i, j)-th
element of the Hessian matrix is defined as:

∂2f
∂xi ∂xj
The Hessian matrix provides information about the curvature
and the rate of change of the gradient. By analyzing the eigenvalues
of the Hessian matrix, we can determine whether a given point is
a maximum, minimum, or saddle point.
To compute the Hessian matrix using SymPy, we can modify
our previous code snippet as follows:

import sympy as sp

x, y = sp.symbols('x y') # Define the variables

f = x**2 + 2*y**2 + 3*x*y # Define the function

hessian = sp.hessian(f, (x, y)) # Calculate the

,→ Hessian matrix

25
print("Hessian Matrix of f(x, y):")
print(hessian)

In this code, we define the function f (x, y) = x2 + 2y 2 + 3xy

and compute its Hessian matrix.

Conclusion
In this chapter, we reviewed the concepts of derivatives and inte-
grals, which are fundamental in calculus. We also discussed partial
derivatives and their applications in multivariable functions. Fi-
nally, we introduced the gradient and Hessian matrix, which pro-
vide valuable information about the behavior of functions. Under-
standing these concepts is crucial for optimizing machine learning
models and comprehending the underlying mathematics.

26
Chapter 4

Probability Theory

Basic Probability Rules

In this chapter, we will explore the fundamental principles of prob-
ability theory. Probability theory is a branch of mathematics that
deals with uncertainty and randomness. It provides a framework
for quantifying and analyzing the likelihood of events.

1 Sample Space and Events

We begin by defining the concepts of the sample space and events.
The sample space, denoted by Ω, is the set of all possible outcomes
of an experiment. An event is a subset of the sample space.
For example, consider the rolling of a fair six-sided die. The
sample space is Ω = {1, 2, 3, 4, 5, 6}, and an event can be, for in-
stance, "rolling an even number", which corresponds to the subset
{2, 4, 6}.

2 Probability Axioms
Probability is defined as a function that assigns a numerical value
to each event. It satisfies the following axioms:

1. Non-negativity: For any event A, the probability P (A) is

greater than or equal to zero: P (A) ≥ 0.
2. Normalization: The probability of the entire sample space
is equal to one: P (Ω) = 1.

27
 3. Additivity: For any collection of mutually exclusive events
A1 , A2 , . . ., the probability of their union is equal to the sum
of their individual probabilities: P (A1 ∪ A2 ∪ . . .) = P (A1 ) +
P (A2 ) + . . ..

These axioms form the foundation of probability theory, allow-

ing us to assign probabilities and reason about uncertain events.

3 Conditional Probability
Conditional probability considers the probability of an event oc-
curring given that another event has already occurred. The condi-
tional probability of event A given event B, denoted by P (A|B), is
defined as:

P (A ∩ B)
P (A|B) =
P (B)
where P (A ∩ B) represents the probability of both A and B
occurring.

4 Independence
Two events A and B are said to be independent if the occurrence
of one event does not affect the probability of the other event.
Mathematically, two events are independent if and only if:

P (A ∩ B) = P (A) · P (B)
This notion of independence is crucial in probability theory and
has applications in various fields.

5 Law of Total Probability

The law of total probability allows us to calculate the probability
of an event by considering all possible ways it can occur. Let
B1 , B2 , . . . , Bn be mutually exclusive and exhaustive events. Then,
for any event A:
n
X
P (A) = P (A|Bi ) · P (Bi )
i=1

This formula provides a method to compute the probability of

A by conditioning on each possible scenario Bi .

28
Random Variables and Distributions
In this section, we introduce the concept of random variables, which
are used to represent uncertain quantities in probability theory. We
also discuss probability distributions, which describe the likelihood
of different values that a random variable can take.

1 Random Variables
A random variable is a variable that takes on different values de-
pending on the outcome of a random event. It can be discrete or
continuous, depending on whether its possible values are countable
or uncountable, respectively.
Formally, a random variable X is a function that maps each
outcome in the sample space to a real number. The probability of
a random variable taking on a particular value or falling within a
certain range is quantified by its probability distribution.

2 Probability Mass Function (PMF)

For a discrete random variable X, the probability mass function
(PMF) gives the probability of each possible value. The PMF is
defined as pX (x) = P (X = x), where x belongs to the range of the
random variable.
The PMF provides a complete description of the underlying
probability distribution of a discrete random variable. It allows us
to calculate the probability of specific events and compute various
statistical measures.

3 Probability Density Function (PDF)

For a continuous random variable X, the probability density func-
tion (PDF) describes the probability of the random variable taking
on different values. Unlike the PMF, the PDF does not directly
give the probability at a specific value, but rather the likelihood of
observing a value in a certain range.
Mathematically, the PDF of a continuous random variable X is
denoted by fX (x) and satisfies the following properties:

1. fX (x) ≥ 0 for all x in the range of X.

R∞
2. −∞ fX (x) dx = 1.

29
 3. The probability of X falling within a specific interval [a, b] is
Rb
given by a fX (x) dx.

The PDF characterizes the shape of the probability distribution

of a continuous random variable. It is often used to analyze and
calculate probabilities of events.

4 Cumulative Distribution Function (CDF)

The cumulative distribution function (CDF) of a random variable
X is defined as FX (x) = P (X ≤ x). It gives the probability that
X takes on a value less than or equal to x.
The CDF provides a complete description of the probability
distribution for both discrete and continuous random variables. It
can be used to compute probabilities of specific events and calculate
various statistical measures, such as percentiles and moments.

5 Expected Value and Variance

The expected value and variance are important measures of central
tendency and dispersion, respectively, for a random variable.
The expected value, denoted by E[X], is a weighted average of
the possible values of the random variable, where the weights are
determined by the probabilities. For a discrete random variable X
with PMF pX (x), the expected value is computed as:
X
E[X] = x · pX (x)
x

For a continuous random variable X with PDF fX (x), the ex-

pected value is given by:
Z ∞
E[X] = x · fX (x) dx
−∞

The variance measures the dispersion or spread of the random

variable around its expected value. It is denoted by Var[X] and is
computed as:

Var[X] = E (X − E[X])2
 

The standard deviation, SD[X], is the square root of the vari-

ance.

30
6 Common Probability Distributions
In probability theory, several common probability distributions are
widely used to model random variables in various applications.
Some important distributions include:

• Bernoulli distribution: Models the probability of a binary

outcome (success or failure) with a single parameter.
• Binomial distribution: Generalizes the Bernoulli distribu-
tion to multiple independent trials.
• Normal distribution: A continuous distribution that is
widely used due to the central limit theorem. It is char-
acterized by its mean and variance.
• Poisson distribution: Models the number of events occur-
ring in a fixed interval of time or space.
• Exponential distribution: Describes the time between events
occurring in a Poisson process.
• Uniform distribution: Represents a constant probability
over a fixed interval.
• Gamma distribution: Generalizes the exponential distri-
bution to accommodate a shape parameter.
• Beta distribution: Models the probabilities of events with
unknown proportions.

Understanding these common probability distributions is cru-

cial for analyzing and modeling a wide range of random phenomena.
In Python, probability distributions and related functions are
available in the ‘scipy.stats‘ module. The following code snip-
pet demonstrates how to compute the probability density function
(PDF) and draw random samples from the normal distribution:

import numpy as np
from scipy.stats import norm

x = np.linspace(-5, 5, 100)
pdf = norm.pdf(x, loc=0, scale=1)

samples = norm.rvs(loc=0, scale=1, size=1000)

31
 In this code, we use the ‘norm‘ class from ‘scipy.stats‘ to com-
pute the PDF of a normal distribution with mean 0 and standard
deviation 1. We also generate a sample of 1000 random numbers
from the same distribution using the ‘rvs‘ method.

Bayes’ Theorem
Bayes’ theorem is a fundamental concept in probability theory that
provides a way to update our beliefs or knowledge about an event
based on new evidence. It relates conditional probabilities and
is widely used in various fields, including statistics and machine
learning.

1 Statement of Bayes’ Theorem

Bayes’ theorem states that for two events A and B:

P (B|A) · P (A)
P (A|B) =
P (B)
where P (A|B) represents the probability of event A occurring
given that event B has occurred. P (B|A) is the probability of event
B occurring given that event A has occurred. P (A) and P (B) are
the probabilities of events A and B, respectively.
Bayes’ theorem allows us to update our prior belief or knowl-
edge (represented by P (A)) about an event based on new evidence
(represented by P (B|A) and P (B)).

2 Applications of Bayes’ Theorem

Bayes’ theorem has numerous applications in different domains.
Some notable applications include:

• Medical Diagnosis: Bayes’ theorem is used to update the

probability of a disease given a positive or negative test result.
• Spam Filtering: Bayes’ theorem is employed to classify
emails as spam or non-spam based on the occurrence of cer-
tain words or phrases.
• Document Classification: Bayes’ theorem is used in text
mining and natural language processing to classify documents
into different categories.

32
 • Machine Learning: Bayes’ theorem is a fundamental com-
ponent of Bayesian machine learning methods, such as Naive
Bayes classifiers and Bayesian networks.

Bayes’ theorem provides a principled way to update our beliefs

or make predictions with the incorporation of new information. It
is a powerful tool for reasoning under uncertainty.

Conclusion
In this chapter, we explored the foundational concepts of proba-
bility theory. We discussed the basic rules of probability, includ-
ing the sample space, events, and probability axioms. We also
introduced random variables and their probability distributions,
including the probability mass function (PMF), probability den-
sity function (PDF), and cumulative distribution function (CDF).
Finally, we learned about Bayes’ theorem and its applications in
various fields. Probability theory is a key mathematical framework
for reasoning about uncertainty and randomness, and its principles
form the basis of many statistical and machine learning techniques.

33
Chapter 5

Statistics
Fundamentals

Descriptive Statistics
In this chapter, we will delve into the fundamentals of statistics.
Statistics is a branch of mathematics that deals with the collection,
analysis, interpretation, presentation, and organization of data.
Descriptive statistics is the branch that focuses on summarizing
and describing the main features of a dataset.

1 Measures of Central Tendency

Measures of central tendency provide a summary of the typical or
central value in a dataset. The most commonly used measures of
central tendency are the mean, median, and mode.
The mean, denoted by µ, is the sum of all values in a dataset
divided by the number of observations. It is calculated as:
n
1X
µ= xi
n i=1
where xi represents each observation and n is the total number
of observations.
The median is the middle value in an ordered dataset. It is
often used for datasets with outliers or skewed distributions. If the
number of observations, n, is odd, the median is the middle value.
If n is even, the median is the average of the two middle values.

34
 The mode is the most frequently occurring value in a dataset.
It is used for categorical or discrete datasets to identify the most
common category or value.

2 Measures of Variability
Measures of variability describe the spread or dispersion of the val-
ues in a dataset. The most commonly used measures of variability
are the range, variance, and standard deviation.
The range is the difference between the maximum and minimum
values in a dataset:

range = max(x) − min(x)

where x represents the values in the dataset.
The variance, denoted by σ 2 , measures the average squared
deviation from the mean. It is calculated as:
n
1X
σ2 = (xi − µ)2
n i=1
where xi represents each observation, µ is the mean, and n is
the total number of observations.
The standard deviation, denoted by σ, is the square root of the
variance. It provides a measure of the average distance between
each data point and the mean.

3 Percentiles and Quartiles

Percentiles and quartiles divide a dataset into equal parts, provid-
ing insights into how data are distributed. The pth percentile is
the value below which p percent of the data falls. For example, the
median represents the 50th percentile.
Quartiles, which divide the dataset into quarters, are commonly
used measures of variability. The first quartile (Q1) represents the
25th percentile, the second quartile (Q2) represents the median,
and the third quartile (Q3) represents the 75th percentile.

4 Box Plots
Box plots, also known as box-and-whisker plots, visualize the dis-
tribution of a dataset using quartiles. They provide a graphical

35
representation of the minimum and maximum values, quartiles,
and outliers.
A box plot consists of a rectangle (the box) that spans the
interquartile range (from Q1 to Q3), a line (the median), and two
lines (the whiskers) that extend from the box to the minimum
and maximum values (excluding outliers). Outliers, depicted as
individual points, are values that lie outside the whiskers.
Box plots are especially useful for comparing distributions of
different groups or variables.

5 Python Implementation
In Python, the ‘numpy‘ and ‘matplotlib‘ libraries provide functions
to calculate descriptive statistics and create box plots. The follow-
ing code snippet demonstrates how to calculate the mean, median,
and standard deviation, as well as create a box plot:

import numpy as np
import matplotlib.pyplot as plt

data = np.array([3, 5, 7, 8, 9, 10, 11, 14, 15, 17])

mean_value = np.mean(data)
median_value = np.median(data)
std_dev = np.std(data)

plt.boxplot(data)
plt.show()

In this code, we use the ‘numpy‘ library to calculate the mean,

median, and standard deviation of a dataset represented by the
array ‘data‘. We then use the ‘matplotlib‘ library to create a box
plot of the same dataset using the ‘boxplot()‘ function.

Hypothesis Testing
In this section, we will explore the concept of hypothesis testing,
which is an essential component of statistical inference. Hypothesis
testing allows us to make inferences about population parameters
based on sample data.

36
1 Formulating Hypotheses
Hypothesis testing involves formulating a null hypothesis (H0 ) and
an alternative hypothesis (H1 ). The null hypothesis represents
the status quo or the absence of an effect, while the alternative
hypothesis represents the claim or the presence of an effect.
For example, suppose we want to test whether a new drug is
effective in reducing blood pressure. The null hypothesis would be
that the drug has no effect (H0 : µ = µ0 ), where µ0 represents the
population mean blood pressure. The alternative hypothesis would
be that the drug has an effect (H1 : µ ̸= µ0 ).

2 Test Statistic and P-value

The test statistic is a numerical summary calculated from the sam-
ple data that is used to assess the evidence against the null hy-
pothesis. The choice of test statistic depends on the nature of the
hypothesis being tested.
The p-value is a probability that measures the evidence against
the null hypothesis. It represents the probability of obtaining a
test statistic as extreme as the one observed, assuming the null hy-
pothesis is true. The smaller the p-value, the stronger the evidence
against the null hypothesis.

3 Types of Errors
Hypothesis testing involves the possibility of making two types of
errors: Type I and Type II errors.
A Type I error occurs when the null hypothesis is rejected,
even though it is true. It represents a false positive conclusion.
The probability of a Type I error is denoted by α and is set prior
to conducting the test (e.g., at 0.05 or 0.01).
A Type II error occurs when the null hypothesis is not rejected,
even though it is false. It represents a false negative conclusion.
The probability of a Type II error is denoted by β and depends
on various factors, such as sample size, effect size, and significance
level.

4 Python Implementation
In Python, the ‘scipy.stats‘ module provides functions to conduct
hypothesis tests. The following code snippet demonstrates how to
perform a one-sample t-test:

37
import numpy as np
from scipy.stats import ttest_1samp

data = np.array([3, 5, 7, 8, 9, 10, 11, 14, 15, 17])

t_statistic, p_value = ttest_1samp(data, 10)

print("T-Statistic:", t_statistic)
print("P-Value:", p_value)

In this code, we use the ‘ttest_1samp()‘ function from the

‘scipy.stats‘ module to perform a one-sample t-test. The test statis-
tic and p-value are obtained as the output of the function.

Confidence Intervals
In this section, we will explore confidence intervals, which are a
way to estimate the range of plausible values for a population pa-
rameter. Confidence intervals provide a measure of the precision
and uncertainty associated with an estimate.

1 Interpreting Confidence Intervals

A confidence interval is a range of values computed from the sample
data that is likely to include the population parameter of interest.
The level of confidence, denoted by (1 − α), represents the propor-
tion of confidence intervals that will contain the true population
parameter if the same sampling procedure is repeated many times.
For example, if we construct a 95

2 Constructing Confidence Intervals

The construction of a confidence interval depends on the sample
data and the desired level of confidence. The most commonly used
method for constructing confidence intervals is the use of critical
values from the standard normal or t-distribution.
For example, if we are constructing a confidence interval for the
mean with a known population standard deviation, we can use the
standard normal distribution and the formula:

38
 σ
Confidence interval = x ± z · √
n
where x represents the sample mean, z is the critical value from
the standard normal distribution corresponding to the desired level
of confidence, σ is the population standard deviation, and n is the
sample size.
If the population standard deviation is unknown, we can use
the t-distribution and the formula:
s
Confidence interval = x ± t · √
n
where s is the sample standard deviation and t is the critical
value from the t-distribution corresponding to the desired level of
confidence and the sample size minus one degree of freedom.

3 Python Implementation
In Python, the ‘scipy.stats‘ module provides functions to calcu-
late confidence intervals. The following code snippet demonstrates
how to calculate a confidence interval for the mean using the t-
distribution:

import numpy as np
from scipy.stats import t

data = np.array([3, 5, 7, 8, 9, 10, 11, 14, 15, 17])

sample_mean = np.mean(data)
sample_std = np.std(data)
sample_size = len(data)

confidence_level = 0.95
degree_of_freedom = sample_size - 1

critical_value = t.ppf((1 + confidence_level) / 2,

,→ df=degree_of_freedom)
margin_of_error = critical_value * (sample_std /
,→ np.sqrt(sample_size))

confidence_interval = (sample_mean - margin_of_error,

,→ sample_mean + margin_of_error)

39
print("Confidence Interval:", confidence_interval)

In this code, we use the ‘np.mean()‘ and ‘np.std()‘ functions

from the ‘numpy‘ library to calculate the sample mean and sample
standard deviation of a dataset represented by the array ‘data‘.
We then calculate the critical value from the t-distribution using
the ‘t.ppf()‘ function and compute the margin of error. Finally,
we construct the confidence interval using the sample mean and
margin of error.
This code snippet demonstrates how to construct a confidence
interval for the mean using the t-distribution. The confidence level,
sample size, and sample data are defined, and the critical value
and margin of error are calculated. The confidence interval is then
computed based on the sample mean and margin of error.

40
Chapter 6

Simple Linear
Regression

Introduction
Linear regression is a fundamental statistical technique used to
model the relationship between a dependent variable and one in-
dependent variable. In this chapter, we will focus on simple linear
regression, which involves predicting a continuous dependent vari-
able based on a single independent variable.

Model Formulation
The simple linear regression model can be represented mathemat-
ically as:

y = β0 + β1 x + ϵ
where y is the dependent variable, x is the independent variable,
β0 is the intercept, β1 is the slope, and ϵ is the error term (residual).
The error term captures the variability in the dependent variable
that is not explained by the independent variable.

Least Squares Estimation

The goal of simple linear regression is to estimate the intercept
(β0 ) and slope (β1 ) that minimize the sum of squared differences

41
between the observed and predicted values.
The least squares estimates of the intercept and slope can be
calculated as:
Pn
(x − x̄)(yi − ȳ)
β̂1 = i=1Pn i
i=1 (xi − x̄)
2

β̂0 = ȳ − β̂1 x̄
where n is the number of observations, xi and yi are the in-
dividual observations of the independent and dependent variables,
and x̄ and ȳ are the sample means of x and y.

Model Evaluation
To evaluate the fit of the simple linear regression model, we consider
the residual sum of squares (RSS), total sum of squares (TSS), and
R-squared (R2 ) statistic.
The RSS represents the sum of squared residuals and can be
calculated as:
n
X
RSS = (yi − ŷi )2
i=1

where yi is the observed value of the dependent variable and ŷi

is the predicted value based on the model.
The TSS represents the total variability in the dependent vari-
able and can be calculated as:
n
X
TSS = (yi − ȳ)2
i=1

The R statistic measures the proportion of the total variability

2

in the dependent variable that is explained by the independent

variable and can be calculated as:
RSS
R2 = 1 −
TSS
A higher value of R2 indicates a better fit of the model to the
data.

42
Python Implementation
In Python, the ‘scikit-learn‘ library provides functions for fitting a
simple linear regression model and evaluating its performance. The
following code snippet demonstrates how to perform simple linear
regression and calculate the RSS, TSS, and R2 using ‘scikit-learn‘:

from sklearn.linear_model import LinearRegression

from sklearn.metrics import mean_squared_error,
,→ r2_score

# Independent variable
X = np.array([3, 5, 7, 8, 9, 10, 11, 14, 15,
,→ 17]).reshape(-1, 1)

# Dependent variable
y = np.array([4, 7, 5, 12, 11, 13, 19, 20, 21, 25])

# Create linear regression object

model = LinearRegression()

# Fit the model

model.fit(X, y)

# Predict the dependent variable

y_pred = model.predict(X)

# Calculate RSS, TSS, and R^2

RSS = mean_squared_error(y, y_pred) * len(y)
TSS = mean_squared_error(y, np.mean(y)) * len(y)
R_squared = r2_score(y, y_pred)

In this code, we first define the independent variable (X) and

the dependent variable (y) as numpy arrays. We then create an
instance of the ‘LinearRegression‘ class and fit the model using
the ‘fit()‘ method. The ‘predict()‘ method is used to generate the
predicted values based on the independent variable.
Finally, we calculate the RSS using the ‘mean_squared_error()‘
function from ‘scikit-learn‘, which computes the mean squared er-
ror between the observed (y) and predicted (ŷ) values. Similarly,
the TSS is computed by comparing the observed values to their

43
mean. The R2 statistic is calculated using the ‘r2_score()‘ func-
tion.
By evaluating the RSS, TSS, and R2 , we can assess the perfor-
mance and goodness of fit of the simple linear regression model.

44
Chapter 7

Multiple Linear
Regression

Introduction
Multiple linear regression is an extension of simple linear regression
that allows for the prediction of a continuous dependent variable
based on multiple independent variables. In this chapter, we will
delve into the mathematics behind multiple linear regression and
explore its various nuances.

Model Formulation
The multiple linear regression model can be represented mathe-
matically as:

y = β 0 + β 1 x 1 + β2 x 2 + . . . + βp x p + ϵ

where y is the dependent variable, x1 , x2 , . . . , xp are the indepen-

dent variables, β0 is the intercept, β1 , β2 , . . . , βp are the coefficients,
p is the number of independent variables, and ϵ is the error term.

Ordinary Least Squares Estimation

The ordinary least squares (OLS) method is commonly used to
estimate the coefficients in multiple linear regression. The goal is

45
to find the estimators β̂ = (β̂0 , β̂1 , . . . , β̂p ) that minimize the sum
of squared differences between the observed and predicted values.
This can be achieved by minimizing the residual sum of squares
(RSS):
X n
RSS = (yi − ŷi )2
i=1
where n is the number of observations, yi is the observed value of
the dependent variable, and ŷi is the predicted value based on the
model.
The OLS estimators β̂ can be obtained by solving the normal
equations:
XT Xβ̂ = XT y
where X is the design matrix of independent variables with dimen-
sions n × (p + 1), and y is the vector of observed values of the
dependent variable with dimensions n × 1. The design matrix X
is augmented with a column of ones to account for the intercept
term. The least squares estimators are given by:

β̂ = (XT X)−1 XT y

Model Evaluation
To evaluate the fit of the multiple linear regression model, various
metrics can be considered. One common measure is the coefficient
of determination (R2 ) which quantifies the proportion of the total
variability in the dependent variable explained by the independent
variables. R2 can be calculated as:
TSS − RSS
R2 =
TSS
where TSS represents the total sum of squares:
n
X
TSS = (yi − ȳ)2
i=1

and ȳ is the mean of the observed values of the dependent variable.

A higher value of R2 indicates a better fit of the model to the data.
Another metric to consider is the adjusted R2 , which takes into
account the number of independent variables and sample size:
n−1
Adjusted R2 = 1 − 1 − R2


n−p−1

46
where n is the sample size and p is the number of independent
variables.

Python Implementation
In Python, multiple linear regression can be performed using the
‘statsmodels‘ library. The following code snippet demonstrates how
to fit a multiple linear regression model and obtain the coefficients,
residuals, and the R2 statistic:
import statsmodels.api as sm

# Independent variables
X = np.array([[1, 3], [1, 5], [1, 7], [1, 8], [1, 9],
,→ [1, 10], [1, 11], [1, 14], [1, 15], [1, 17]])

# Dependent variable
y = np.array([4, 7, 5, 12, 11, 13, 19, 20, 21, 25])

# Fit the model

model = sm.OLS(y, X)
results = model.fit()

# Obtain the coefficients

coefficients = results.params

# Obtain the residuals

residuals = results.resid

# Obtain the R-squared statistic

R_squared = results.rsquared

In this code, we create a matrix X that consists of the design

matrix augmented with a column of ones. The dependent variable
y is defined as a numpy array. We then create an ‘OLS‘ model ob-
ject using the ‘sm.OLS()‘ function and fit the model using the ‘fit()‘
method. The ‘params‘ attribute of the ‘results‘ object provides the
estimated coefficients, the ‘resid‘ attribute gives the residuals, and
the ‘rsquared‘ attribute provides the R2 statistic.
By evaluating the coefficients, residuals, and R2 , we can gain
insights into the relationships between the independent variables
and the dependent variable in multiple linear regression.

47
Chapter 8

Logistic Regression

Introduction
In this chapter, we will delve into the mathematical foundations of
logistic regression. Logistic regression is a powerful classification
algorithm that is widely used in machine learning and statistics.
We will explore its underlying mathematics and discuss how it can
be used to model the relationship between a set of independent
variables and a binary dependent variable.

Model Formulation
Logistic regression aims to model the probability of a binary out-
come, typically denoted as y ∈ {0, 1}, based on a set of independent
variables denoted as x = (x1 , x2 , . . . , xp ). The logistic regression
model can be formulated as follows:
1
P (y = 1|x) = (8.1)
1+ e−(β0 +β1 x1 +β2 x2 +...+βp xp )
where P (y = 1|x) represents the conditional probability of y
being 1 given the values of the independent variables x. The term
e denotes the base of the natural logarithm, and β0 , β1 , . . . , βp rep-
resent the model’s coefficients or parameters.
The odds ratio can be defined as the ratio of the probability of
an event occurring to the probability of it not occurring. In the
case of logistic regression, the odds ratio can be written as:

48
 P (y = 1|x)
Odds = = eβ0 +β1 x1 +β2 x2 +...+βp xp (8.2)
P (y = 0|x)

Maximum Likelihood Estimation

To estimate the coefficients in logistic regression, the maximum
likelihood estimation (MLE) method is commonly used. The goal
is to find the values of the coefficients that maximize the likelihood
function. The likelihood function can be written as:
n
Y
L(β0 , β1 , . . . , βp ) = P (yi |xi ) (8.3)
i=1

To simplify calculations, it is common to maximize the log-

likelihood function instead:
n
X
ℓ(β0 , β1 , . . . , βp ) = log(P (yi |xi )) (8.4)
i=1

The coefficients that maximize the log-likelihood function can

be obtained using numerical optimization techniques such as gra-
dient descent or Newton’s method.

Model Interpretation
In logistic regression, the coefficients can be interpreted as the
change in the logarithm of the odds ratio for a one-unit change in
the corresponding independent variable, holding all other variables
constant. Mathematically, the interpretation of the coefficients can
be written as follows:

P (y = 1|x)
 
d
log = βi (8.5)
dxi P (y = 0|x)
This implies that a positive coefficient βi indicates that an in-
crease in the corresponding independent variable xi leads to an
increase in the odds ratio, and vice versa.

49
Python Implementation
In Python, logistic regression can be performed using various li-
braries such as ‘scikit-learn‘, ‘statsmodels‘, or ‘tensorflow‘. Here is
an example using ‘scikit-learn‘:

from sklearn.linear_model import LogisticRegression

# Independent variables
X = [[1, 3], [1, 5], [1, 7], [1, 8], [1, 9], [1, 10],
,→ [1, 11], [1, 14], [1, 15], [1, 17]]

# Dependent variable
y = [0, 0, 0, 0, 1, 0, 1, 1, 1, 1]

# Fit the model

model = LogisticRegression()
model.fit(X, y)

# Obtain the coefficients

coefficients = model.coef_
intercept = model.intercept_

In this code, we define the independent variables as a nested

list ‘X‘ where each sublist represents the values of the independent
variables for a particular observation. The dependent variable ‘y‘
is defined as a list indicating the binary outcome for each observa-
tion. We create a ‘LogisticRegression‘ model object, fit the model
using the ‘fit()‘ method, and then obtain the coefficients using the
‘coef‘ attribute, andtheinterceptusingthe‘intercept‘ attribute.
By using logistic regression, we can analyze the relationship
between the independent variables and the log-odds of the binary
outcome, providing valuable insights for classification tasks.

50
Chapter 9

Gradient Descent

Introduction
In this chapter, we will delve into the mathematical foundations
of gradient descent, a fundamental optimization algorithm used
in machine learning and numerical optimization. We will explore
the mathematical intuition behind gradient descent and discuss its
variants.

Basic Gradient Descent Algorithm

At its core, gradient descent is an iterative optimization algorithm
used to find the minimum of a function. Given a function f : Rn →
R, the goal of gradient descent is to find the values of the input
variables x = (x1 , x2 , . . . , xn ) that minimize f (x).
The basic gradient descent algorithm can be summarized as
follows:

1 Algorithm
Step 1: Initialization
(0) (0) (0)
Pick an initial guess for the input variables x(0) = (x1 , x2 , . . . , xn ).
This initial guess can be arbitrary or, in some cases, chosen based
on domain knowledge.

51
Step 2: Iteration
Iteratively update the values of x until convergence is reached. The
update rule is given by:

x(t+1) = x(t) − α∇f (x(t) ) (9.1)

where α is the learning rate, t represents the iteration number,
and ∇f (x(t) ) denotes the gradient of f evaluated at x(t) .

Step 3: Convergence Check

Repeat Step 2 until a stopping criterion is met. This criterion can
be the maximum number of iterations or reaching a desired level
of precision.

Stochastic Gradient Descent

Stochastic gradient descent (SGD) is a variant of gradient descent
that is commonly used in large-scale machine learning problems.
Rather than computing the gradient of the entire training dataset
at each iteration, SGD approximates the gradient using a single or
a small batch of randomly sampled training examples.
The update rule for SGD can be written as follows:

x(t+1) = x(t) − α∇fi (x(t) ) (9.2)

where fi (x(t) ) represents the loss function computed on a ran-
domly selected training example.

1 Python Implementation
Here is an example implementation of stochastic gradient descent
in Python using NumPy:

import numpy as np

# Initialize variables
learning_rate = 0.01
max_iterations = 1000
epsilon = 1e-6

# Initialize x

52
x = np.array([1.0, 1.0])

# Define the gradient function

def gradient(x):
return np.array([2 * x[0], 4 * x[1]])

# Perform stochastic gradient descent

for iteration in range(max_iterations):
# Randomly select training example
t = np.random.randint(0, len(X))
xi = X[t]

# Update x
x_new = x - learning_rate * gradient(xi)

# Check for convergence

if np.linalg.norm(x_new - x) < epsilon:
break

x = x_new

In this code, we initialize the learning rate, maximum number

of iterations, and a small value for convergence check. We also
initialize the input variables x.
The gradient() function represents the gradient of the func-
tion f. In each iteration, we randomly select a training example
xi and update the input variables x using the learning rate and
the gradient of that specific training example. We then check for
convergence by computing the norm of the difference between the
updated and previous x, and if it falls below the threshold, the
algorithm terminates.

Convergence Issues and Optimization

While gradient descent is a powerful optimization algorithm, it is
not without its challenges. Several issues can affect its convergence,
such as choosing an appropriate learning rate, dealing with ill-
conditioned or non-convex objective functions, and avoiding local
minima.
To mitigate these issues, various optimization techniques have
been proposed. These techniques include adaptive learning rates,

53
momentum-based methods, and second-order optimization meth-
ods such as the Newton’s method or the Broyden-Fletcher-Goldfarb-
Shanno (BFGS) algorithm.
In addition, the performance of gradient descent can be further
improved by incorporating regularization techniques such as L1 or
L2 regularization to prevent overfitting and to encourage sparsity.

Conclusion
Gradient descent is an essential optimization algorithm used in var-
ious machine learning and numerical optimization problems. By
iteratively updating the input variables based on the negative gra-
dient of the objective function, it aims to find the minimum of the
function. Additionally, stochastic gradient descent provides a more
scalable approach, approximating the gradient using a small num-
ber of randomly selected training examples. While convergence
issues exist, optimizations such as adaptive learning rates and reg-
ularization techniques can improve the algorithm’s performance.

54
Chapter 10

Gradient Descent
Variants

Introduction
In this chapter, we explore various variants of the gradient descent
algorithm. We begin by discussing the limitations of the basic
gradient descent algorithm and then delve into different variations
that address these limitations. These variants include mini-batch
gradient descent, adaptive learning rates, and momentum-based
methods.

Mini-Batch Gradient Descent

The mini-batch gradient descent (MBGD) algorithm, an extension
of stochastic gradient descent (SGD), addresses the issue of noisy
and unstable updates associated with stochastic gradient descent.
Instead of using a single training example (as done in SGD) or the
entire training dataset (as done in batch gradient descent), MBGD
computes the gradient based on a small batch of training examples.
The update rule for MBGD can be described as follows:

x(t+1) = x(t) − α · ∇fbatch (x(t) ) (10.1)

where ∇fbatch (x ) represents the gradient of the objective func-
(t)

tion f computed over a mini-batch of training examples.

55
1 Python Implementation
Here is an example implementation of mini-batch gradient descent
in Python:

import numpy as np

# Initialize variables
learning_rate = 0.01
batch_size = 32
max_iterations = 1000
epsilon = 1e-6

# Initialize x
x = np.array([1.0, 1.0])

# Define the gradient function

def gradient(x, batch):
return np.sum(2 * batch, axis=0)

# Perform mini-batch gradient descent

for iteration in range(max_iterations):
# Randomly select mini-batch
indices = np.random.choice(len(X),
,→ size=batch_size, replace=False)
batch = X[indices]

# Update x
x_new = x - learning_rate * gradient(x, batch)

# Check for convergence

if np.linalg.norm(x_new - x) < epsilon:
break

x = x_new

In this code, we initialize the learning rate, batch size, maxi-

mum number of iterations, and a small value for convergence check.
We also initialize the input variables x.
The gradient() function represents the gradient of the objec-
tive function f computed over a mini-batch batch. In each itera-
tion, we randomly select a mini-batch of training examples batch

56
and update the input variables x using the learning rate and the
computed gradient. We then check for convergence by computing
the norm of the difference between the updated and previous x,
and if it falls below the threshold, the algorithm terminates.

Adaptive Learning Rates

One of the challenges with gradient descent is finding an appro-
priate learning rate that allows for efficient convergence. Setting a
fixed learning rate can lead to slow convergence or instability.
Adaptive learning rate algorithms aim to address this challenge
by automatically adjusting the learning rate during the optimiza-
tion process. One such algorithm is AdaGrad.
The update rule for AdaGrad can be expressed as follows:
α
x(t+1) = x(t) − √ ⊙ ∇f (x(t) ) (10.2)
G(t) +ϵ
where G(t) represents the diagonal matrix of the sum of the element-
wise squares of the gradients up to iteration t, α denotes the initial
learning rate, ϵ is a small constant to prevent division by zero, and
⊙ represents element-wise multiplication.

1 Python Implementation
Here is an example implementation of AdaGrad in Python:

import numpy as np

# Initialize variables
learning_rate = 0.1
max_iterations = 1000
epsilon = 1e-6

# Initialize x
x = np.array([1.0, 1.0])
G = np.zeros_like(x)

# Define the gradient function

def gradient(x):
return np.array([2 * x[0], 4 * x[1]])

57
# Perform AdaGrad
for iteration in range(max_iterations):
# Compute gradient
grad = gradient(x)

# Update G and x
G += grad**2
x -= (learning_rate / np.sqrt(G + epsilon)) *
,→ grad

# Check for convergence

if np.linalg.norm(learning_rate * grad) <
,→ epsilon:
break

In this code, we initialize the learning rate, maximum number

of iterations, and a small value for convergence check. We also
initialize the input variables x and the matrix G to store and update
the sum of squared gradients.
The gradient() function represents the gradient of the objec-
tive function f. In each iteration, we compute the gradient grad
and update G by adding the squared gradient. We then update x
using the computed gradient, adjusted by the learning rate divided
by the square root of G. We check for convergence by computing
the norm of the learning rate multiplied by the gradient, and if it
falls below the threshold, the algorithm terminates.

Momentum-Based Methods
Momentum-based methods introduce a notion of inertia to the gra-
dient descent process, helping to accelerate convergence, especially
in the presence of sparse gradients or noisy updates.
The basic idea is to maintain a momentum term that accumu-
lates a fraction of the previous updates. This momentum term
guides the direction of the update, making it less susceptible to
oscillations or getting stuck in local minima.
The update rule for momentum-based methods can be described
as follows:

58
 v(t) = β · v(t−1) + α · ∇f (x(t) ) (10.3)
x(t+1) = x(t) − v(t) (10.4)

where β represents the momentum hyperparameter, α denotes

the learning rate, ∇f (x(t) ) denotes the gradient of the objective
function, v(t) denotes the momentum term, and x(t) represents the
current iterate.

1 Python Implementation
Here is an example implementation of momentum-based gradient
descent in Python:

import numpy as np

# Initialize variables
learning_rate = 0.1
momentum = 0.9
max_iterations = 1000
epsilon = 1e-6

# Initialize x and v
x = np.array([1.0, 1.0])
v = np.zeros_like(x)

# Define the gradient function

def gradient(x):
return np.array([2 * x[0], 4 * x[1]])

# Perform momentum-based gradient descent

for iteration in range(max_iterations):
# Compute gradient
grad = gradient(x)

# Update v and x
v = momentum * v + learning_rate * grad
x -= v

# Check for convergence

59
 if np.linalg.norm(learning_rate * grad) <
,→ epsilon:
break

In this code, we initialize the learning rate, momentum hyper-

parameter, maximum number of iterations, and a small value for
convergence check. We also initialize the input variables x and the
momentum term v.
The gradient() function represents the gradient of the objec-
tive function f. In each iteration, we compute the gradient grad
and update v by combining a fraction of the previous momentum
v with the current learning rate multiplied by the gradient. We
then update x using the momentum term v. We check for con-
vergence by computing the norm of the learning rate multiplied
by the gradient, and if it falls below the threshold, the algorithm
terminates.

60
Chapter 11

Ordinary Least Squares

(OLS)

Introduction
In this chapter, we focus on ordinary least squares (OLS), a widely
used method for estimating the parameters in linear regression
models. We begin by providing a theoretical background for OLS,
followed by the derivation of the estimator and its properties.

Derivation and Explanation

Consider a linear regression model with n observations and p pre-
dictors, represented by the following equations:

y = Xβ + ϵ (11.1)
1
       
y1 x11 ··· x1p β0 ϵ1
 y2  1 x21 ··· x2p   β1   ϵ2 
y =  .  , X = . .. .. ..  , β =  ..  , ϵ =  .. 
       
 ..   .. . . .  . .
yn 1 xn1 ··· xnp βp ϵn
(11.2)

Here, y represents the response variable, X denotes the matrix

of predictor variables (including an intercept term), β represents

61
the vector of unknown parameters, and ϵ represents the vector of
error terms.
The ordinary least squares (OLS) estimator β̂ is obtained by
minimizing the sum of squared residuals:
2
β̂ = arg min ∥y − Xβ∥ (11.3)
β

To find the minimum, we take the derivative with respect to β:

∂ T
(y − Xβ) (y − Xβ) = 0 (11.4)
∂β
Expanding and simplifying, we obtain the normal equations:

XT Xβ = XT y (11.5)
Solving this equation for β, we find the OLS estimator:

β̂ = (XT X)−1 XT y (11.6)

This estimator minimizes the sum of squared residuals and pro-
vides the best linear unbiased estimates of the parameters in the
model.

Gauss-Markov Theorem
The Gauss-Markov theorem states that under the assumptions
of the linear regression model (linearity, independence, and ho-
moscedasticity of residuals, and absence of multicollinearity), the
OLS estimator β̂ is the Best Linear Unbiased Estimator (BLUE)
of the true parameter vector β.
The BLUE property means that among all linear unbiased es-
timators, the OLS estimator has the smallest variance. This result
holds even if the error terms are not normally distributed.

Applications in Regression Analysis

OLS regression is extensively used in various fields, including eco-
nomics, social sciences, finance, and engineering. It allows us to
estimate the relationships between predictor variables and a re-
sponse variable, make predictions, and infer the significance of the
predictors.

62
 Python provides several libraries, such as NumPy, SciPy, and
scikit-learn, that offer efficient built-in functions for performing
OLS regression. These libraries handle the matrix computations
efficiently and provide tools for hypothesis testing, confidence in-
terval estimation, and diagnostics.

1 Python Implementation
Here is an example implementation of OLS regression in Python
using the scikit-learn library:

from sklearn.linear_model import LinearRegression

# Define the predictor variables X and the response

,→ variable y

# Create an instance of the LinearRegression class

model = LinearRegression()

# Fit the model to the data

model.fit(X, y)

# Get the estimated coefficients

coefficients = model.coef_

# Get the intercept

intercept = model.intercept_

In this code, we import the LinearRegression class from the

scikit-learn library. We then create an instance of the LinearRegression
class and fit the model to the predictor variables X and the response
variable y. The fit() method estimates the coefficients and the
intercept using the OLS method. The estimated coefficients can
be accessed using the coef_ attribute, and the intercept can be
accessed using the intercept_ attribute.

63
Chapter 12

Ordinary Least Squares

(OLS)

Introduction
Linear regression is a fundamental tool in statistics and machine
learning for modeling the relationship between a response variable
and one or more predictor variables. The ordinary least squares
(OLS) method is a widely used approach for estimating the pa-
rameters in a linear regression model. In this chapter, we present
the derivation of the OLS estimator and discuss its properties.

Derivation and Explanation

Consider a linear regression model with n observations and p pre-
dictor variables. We represent the model as follows:

yi = β0 + β1 xi1 + β2 xi2 + · · · + βp xip + ϵi (12.1)

where yi represents the response variable for the ith observa-
tion, β0 is the intercept, βj (j = 1, 2, . . . , p) are the coefficients
for the predictor variables xij , and ϵi is the error term for the ith
observation.
To estimate the unknown coefficients βj , the OLS method min-
imizes the sum of squared residuals (SSR), which is the sum of the
squared differences between the observed response values and the
values predicted by the model:

64
 n
X
SSR(β) = (yi − (β0 + β1 xi1 + β2 xi2 + · · · + βp xip ))2 (12.2)
i=1

Mathematically, we can express the OLS estimator β̂ as the

solution to the following optimization problem:

n
( )
X
β̂ = arg min (yi − (β0 + β1 xi1 + β2 xi2 + · · · + βp xip ))2
β
i=1
(12.3)
To find the minimum, we take the partial derivatives of the SSR
with respect to each coefficient βj and set them equal to zero:

n
∂SSR X
= −2 (yi − (β0 + β1 xi1 + β2 xi2 + · · · + βp xip )) = 0
∂β0 i=1
(12.4)
n
∂SSR X
= −2 xi1 (yi − (β0 + β1 xi1 + β2 xi2 + · · · + βp xip )) = 0
∂β1 i=1
(12.5)
..
. (12.6)
n
∂SSR X
= −2 xip (yi − (β0 + β1 xi1 + β2 xi2 + · · · + βp xip )) = 0
∂βp i=1
(12.7)

Expanding and simplifying these equations, we obtain the nor-

mal equations:

 Pn Pn Pn     Pn 
P n Pi=1 xi1 Pn i=1 xi2 ··· Pn i=1 xip β0 Pn i=1 yi
 ni=1 xi1 n
x2i1 i=1 xi1 xi2 ··· Pni=1 xi1 xip  β1  Pi=1 xi1 yi 
i=1
  
Pn Pn P n 2 n 
 i=1 xi2 i=1 xi1 xi2 i=1 xi2 ··· i=1 xi2 xip  β2  = 
   
i=1 xi2 yi 

.. .. .. .. ..   ..   ..

.
Pn . Pn . Pn . Pn . 2
 .   .
 
 
Pn
i=1 xip i=1 xi1 xip i=1 xi2 xip ··· x
i=1 ip β p x
i=1 ip i y
(12.8)

Solving these equations, we obtain the OLS estimator:

65
 βˆ0
 
ˆ
 β1 
ˆ
−1 T
 .2  = X X
β̂ =  β  T
X y (12.9)
.
.
βˆp
where X is the design matrix consisting of the predictor vari-
ables, XT is its transpose, and y is the vector of observed responses.

Applications in Regression
OLS regression is commonly used in various fields for modeling the
relationship between variables and making predictions. The OLS
estimator provides the best linear unbiased estimates of the regres-
sion coefficients under certain assumptions. It allows researchers
to investigate the impact of different predictor variables on the re-
sponse variable, perform hypothesis tests on the coefficients, and
assess the overall fit of the model.
Python provides several libraries that offer efficient implemen-
tations of OLS regression, such as NumPy and scikit-learn. These
libraries handle the matrix computations efficiently and provide
tools for model fitting, coefficient estimation, prediction, and diag-
nostics.

1 Python Implementation
Here is an example implementation of OLS regression in Python
using the NumPy library to estimate the coefficients:

import numpy as np

# Define the predictor variables X and the response

,→ variable y

# Add a column of ones to X for the intercept term

X = np.column_stack((np.ones(len(X)), X))

# Compute the OLS estimator

beta_hat = np.linalg.inv(X.T @ X) @ X.T @ y

66
 In this code, we first add a column of ones to the design matrix
X to account for the intercept term. Then, we compute the OLS
estimator β̂ using the formula β̂ = (XT X)−1 XT y. The @ oper-
ator is used for matrix multiplication, and the np.linalg.inv()
function computes the inverse of a matrix.

67
Chapter 13

Bayesian Inference

Introduction
In this chapter, we delve into the mathematical foundations of
Bayesian inference, a powerful framework for statistical modeling
and reasoning. We begin by introducing the fundamental concepts
of prior and posterior distributions, and discuss the role of Bayes’
theorem in updating our beliefs. We then explore various aspects
of Bayesian estimation, including the computation of posterior dis-
tributions and the choice of prior distribution. Additionally, we
touch upon the practical implementation of Bayesian inference in
Python.

Prior and Posterior Distributions

At the core of Bayesian inference lies the notion of prior and poste-
rior distributions. Let us consider a parameter of interest, denoted
as θ, and denote our prior beliefs about θ as P (θ). This prior dis-
tribution represents our initial knowledge or uncertainty about the
value of θ before observing any data. Once we observe data D, we
can use Bayes’ theorem to update our beliefs and obtain the poste-
rior distribution P (θ|D), which represents our updated knowledge
about θ conditioned on the observed data. Bayes’ theorem can be
expressed as follows:

P (D|θ)P (θ)
P (θ|D) = (13.1)
P (D)

68
 Here, P (D|θ) represents the likelihood function, which describes
the probability of observing the data D given a specific value of θ.
The normalization term P (D), often called the marginal likelihood
or evidence, ensures that the posterior distribution integrates to 1.

1 Conjugate Priors
In many cases, the prior distribution and likelihood function be-
long to the same family of probability distributions, resulting in a
posterior distribution that also belongs to this family. Such prior
distributions are known as conjugate priors. Conjugate priors offer
computational convenience as they allow for closed-form solutions
and simplify the estimation process. Common examples include
the normal distribution as the conjugate prior for the mean of a
Gaussian likelihood and the gamma distribution as the conjugate
prior for the rate parameter of a Poisson likelihood.
Conjugate priors enable efficient Bayesian estimation by sim-
plifying the computation of posterior distributions. By choosing
appropriate conjugate priors based on the likelihood function, we
can obtain closed-form expressions for the parameters of the pos-
terior distribution, avoiding the need for costly numerical methods
or simulations.

Practical Implementation in Python

To perform Bayesian inference in practice, we often resort to nu-
merical methods due to the complexity of analytical calculations for
many models. Fortunately, there exist powerful Python libraries
that facilitate Bayesian modeling and estimation.

1 MCMC Sampling with PyMC3

PyMC3 is a popular Python library that provides a high-level
interface for Bayesian modeling and Markov chain Monte Carlo
(MCMC) sampling. MCMC methods, such as the Metropolis-
Hastings algorithm and Gibbs sampling, allow us to draw samples
from the posterior distribution without requiring explicit analytical
forms.
Here is an example code snippet demonstrating the use of PyMC3
for Bayesian inference:

69
import pymc3 as pm

# Define the model using a context manager

with pm.Model() as model:
# Define the prior distribution
theta = pm.Normal('theta', mu=0, sigma=1)

# Define the likelihood function

likelihood = pm.Normal('likelihood', mu=theta,
,→ sigma=1, observed=data)

# Perform MCMC sampling

trace = pm.sample(1000, tune=1000)

In this example, we define a simple Bayesian model with a

normal prior distribution and a normal likelihood function. The
observed data is incorporated using the observed argument. We
then use the sample() function to draw samples from the posterior
distribution. The tune parameter controls the number of burn-in
samples to discard before convergence.

2 Variational Inference with PyMC3

PyMC3 also supports variational inference, an alternative to MCMC
that approximates the posterior distribution using optimization-
based techniques. Variational inference transforms the problem of
inference into an optimization problem by minimizing the diver-
gence between a parametric approximation and the true posterior
distribution.
Here is an example code snippet illustrating the use of varia-
tional inference in PyMC3:

import pymc3 as pm

# Define the model using a context manager

with pm.Model() as model:
# Define the prior distribution
theta = pm.Normal('theta', mu=0, sigma=1)

# Define the likelihood function

70
 likelihood = pm.Normal('likelihood', mu=theta,
,→ sigma=1, observed=data)

# Perform variational inference

approx = pm.fit(method='advi')
trace = approx.sample(1000)

In this code snippet, we specify the model in a similar manner

as before. Afterward, we use the fit() function to perform varia-
tional inference with automatic differentiation variational inference
(ADVI). The resulting approximation is then used to draw samples
from the approximate posterior distribution using the sample()
function.
Both MCMC sampling and variational inference offer flexible
and computationally efficient ways to estimate the posterior distri-
bution in Bayesian inference. The choice between these methods
depends on the specific modeling problem and the available com-
putational resources.

Conclusion
In this chapter, we explored the foundational concepts of Bayesian
inference and discussed the importance of prior and posterior distri-
butions in updating our beliefs. We also highlighted the advantages
of using conjugate priors for efficient estimation and provided code
snippets demonstrating the practical implementation of Bayesian
inference using the PyMC3 library. Bayesian inference offers a
flexible framework for incorporating prior knowledge, propagating
uncertainty, and making probabilistic predictions, making it a pow-
erful tool in statistical modeling and machine learning.

1 Code Snippet: MCMC Sampling with PyMC3

import pymc3 as pm

# Define the model using a context manager

with pm.Model() as model:
# Define the prior distribution
theta = pm.Normal('theta', mu=0, sigma=1)

# Define the likelihood function

71
 likelihood = pm.Normal('likelihood', mu=theta,
,→ sigma=1, observed=data)

# Perform MCMC sampling

trace = pm.sample(1000, tune=1000)

2 Code Snippet: Variational Inference with PyMC3

import pymc3 as pm

# Define the model using a context manager

with pm.Model() as model:
# Define the prior distribution
theta = pm.Normal('theta', mu=0, sigma=1)

# Define the likelihood function

likelihood = pm.Normal('likelihood', mu=theta,
,→ sigma=1, observed=data)

# Perform variational inference

approx = pm.fit(method='advi')
trace = approx.sample(1000)

72
Chapter 14

Naive Bayes Classifier

Introduction
The Naive Bayes classifier is a simple yet powerful algorithm used
for classification tasks. It is based on the Bayes’ theorem, which
provides a principled way to update our beliefs about class labels
given observed features. In this chapter, we will explore the math-
ematical foundations of the Naive Bayes classifier and discuss its
assumptions and applications.

Derivation of the Naive Bayes Classifier

The Naive Bayes classifier is derived from Bayes’ theorem, which
relates the posterior probability of a class label given some observed
features to the prior probability and likelihood of the class label
and features, respectively. Let C represent a class label, and let
x = (x1 , x2 , . . . , xn ) represent a set of n independent features.
Bayes’ theorem can be expressed as follows:

P (x|C)P (C)
P (C|x) = (14.1)
P (x)
Here, P (x|C) represents the likelihood of observing the features
x given the class label C, P (C) represents the prior probability of
class C, and P (x) represents the marginal likelihood of the features
x.
To make classification decisions, we need to compare the poste-
rior probabilities of different class labels. The Naive Bayes classifier

73
makes a simplifying assumption that the features are conditionally
independent given the class label. This allows us to rewrite the
likelihood term as follows:
n
Y
P (x|C) = P (xi |C) (14.2)
i=1

This assumption is called "naive" because it is often unrealistic

to assume that features are truly independent. However, despite
this simplification, the Naive Bayes classifier often performs well in
practice and is widely used in various applications.

Gaussian Naive Bayes Classifier

A common variation of the Naive Bayes classifier is the Gaussian
Naive Bayes classifier, which assumes that the likelihood term can
be modeled using a Gaussian (normal) distribution. This assump-
tion is appropriate when the features are continuous and can take
on any real value.
In the Gaussian Naive Bayes classifier, we model the likelihood
term P (xi |C) for each feature as a Gaussian distribution with class-
specific mean µij and variance σij 2
. The mean and variance pa-
rameters can be estimated from the training data using maximum
likelihood estimation or another appropriate estimation technique.
The probability density function (PDF) of the Gaussian distri-
bution is given by:
−(xi −µij )2
1 2σ 2
P (xi |C) = q e ij (14.3)
2πσij
2

where xi is the value of feature i. The class probabilities P (C)

can also be estimated from the training data.
To classify a new instance with feature values x, we compute
the posterior probabilities of each class label and choose the class
with the highest probability:

Ĉ = arg max P (C|x) (14.4)

C

where Ĉ represents the predicted class label.

74
Application in Text Classification
The Naive Bayes classifier is particularly well-suited for text clas-
sification tasks, where the features are often discrete and represent
the presence or absence of words in a document.
In text classification, the likelihood term P (xi |C) is typically
estimated using the multinomial distribution, which models the
counts of words. The class probabilities P (C) can be estimated
from the relative frequencies of the class labels in the training data.
The Naive Bayes classifier has been successfully applied to tasks
such as spam detection, sentiment analysis, and document catego-
rization, demonstrating its effectiveness and versatility in dealing
with text data.

Python Code
Here is an example Python code snippet demonstrating the use of
the Gaussian Naive Bayes classifier from the scikit-learn library:

from sklearn.naive_bayes import GaussianNB

# Create a Gaussian Naive Bayes classifier

clf = GaussianNB()

# Fit the classifier to the training data

clf.fit(X_train, y_train)

# Predict class labels for the test data

y_pred = clf.predict(X_test)

In this example, X_train and y_train represent the training

features and labels, respectively, while X_test represents the test
features. The fit() method is used to train the classifier, and the
predict() method is used to predict the class labels for the test
data.

1 Code Explanation
In this section, we break down the code snippet and provide a brief
explanation of each step.

75
 • Line 3: Create a Gaussian Naive Bayes classifier using the
GaussianNB() class from scikit-learn.
• Line 6: Fit the classifier to the training data using the fit()
method. This step estimates the mean and variance param-
eters for each feature.

• Line 9: Predict class labels for the test data using the predict()
method. The classifier assigns the class label with the highest
posterior probability to each instance.

This code snippet demonstrates the simplicity and ease of use of

the scikit-learn library for implementing the Gaussian Naive Bayes
classifier in Python.

Conclusion
In this chapter, we derived the Naive Bayes classifier from Bayes’
theorem and discussed its assumptions and applications. We specif-
ically explored the Gaussian Naive Bayes classifier, which is suit-
able for continuous features, and demonstrated its application in
text classification tasks. We also provided a Python code snippet
showcasing the use of the Gaussian Naive Bayes classifier in scikit-
learn. The Naive Bayes classifier is a valuable tool in machine
learning due to its simplicity, effectiveness, and interpretability.

76
Chapter 15

K-Nearest Neighbors
(K-NN)

Introduction
The K-Nearest Neighbors (K-NN) algorithm is a non-parametric
classification algorithm that is widely used in machine learning.
At its core, K-NN classifies an input sample by finding the K near-
est neighbors from the training data and assigns a label based on
the majority vote or weighted vote of the neighbors. In this chap-
ter, we will delve into the mathematical foundations of the K-NN
algorithm, discuss the distance metrics used to measure similarity,
and explore the algorithm’s computational complexity.

Distance Metrics
The choice of distance metric plays a crucial role in the K-NN
algorithm as it determines how similarity is measured between
samples. Common distance metrics include Euclidean distance,
Manhattan distance, and Minkowski distance. For a given feature
vector x = (x1 , x2 , ..., xn ) and y = (y1 , y2 , ..., yn ), these distance
metrics can be formally defined as follows:

77
1 Euclidean Distance
The Euclidean distance between x and y is given by the following
formula: v
u n
uX
Euclidean(x, y) = t (xi − yi )2 (15.1)
i=1

2 Manhattan Distance
The Manhattan distance between x and y is given by the following
formula:
Xn
Manhattan(x, y) = |xi − yi | (15.2)
i=1

3 Minkowski Distance
The Minkowski distance between x and y is a generalized form that
includes both Euclidean and Manhattan distances. It is defined as:
n
! p1
X
Minkowski(x, y) = |xi − yi |p (15.3)
i=1

where p is a parameter that determines the form of the distance

metric. When p = 1, the Minkowski distance is equivalent to the
Manhattan distance, and when p = 2, it reduces to the Euclidean
distance.

K-NN Algorithm
The K-NN algorithm assigns a class label to an input sample by
considering the labels of its K nearest neighbors from the training
data. The algorithm consists of the following steps:

1 Step 1: Select K
Choose a positive integer K, which corresponds to the number of
nearest neighbors to consider.

78
2 Step 2: Calculate Distances
Calculate the distance between the input sample x and each train-
ing sample xi using a distance metric, such as Euclidean distance
or Manhattan distance.

3 Step 3: Find K Nearest Neighbors

Select the K training samples with the smallest distances to the
input sample.

4 Step 4: Assign Class Label

Assign the class label to the input sample based on the majority
vote or weighted vote of the K nearest neighbors. If majority vot-
ing is used, the class label with the highest count among the K
neighbors is assigned. If weighted voting is used, the class label is
assigned based on a weighted sum of the votes, where the weight
for each neighbor is determined by its inverse distance to the input
sample.
The choice of K influences the bias-variance trade-off of the K-
NN algorithm. Smaller values of K tend to have low bias but high
variance, resulting in a more flexible decision boundary. On the
other hand, larger values of K have higher bias but lower variance,
leading to a smoother decision boundary.

Computational Complexity
The computational complexity of the K-NN algorithm is primar-
ily determined by the calculation of distances between the input
sample and all training samples. Assuming there are m training
samples and n features, calculating the distance between two sam-
ples has a time complexity of O(n). Therefore, the overall time
complexity of the K-NN algorithm for a single prediction is O(mn).
During prediction, the algorithm needs to compute distances
for all m training samples. As a result, the overall time complexity
of predicting the class labels for all test samples is O(kmn), where
k is the number of test samples.
It is worth noting that the K-NN algorithm can be computa-
tionally demanding for large datasets or high-dimensional feature
spaces since it requires computing distances between each pair of
samples.

79
Python Code
Here is a Python code snippet demonstrating the use of the K-NN
algorithm in scikit-learn:

from sklearn.neighbors import KNeighborsClassifier

# Create a K-NN classifier with K=5

knn = KNeighborsClassifier(n_neighbors=5)

# Fit the classifier to the training data

knn.fit(X_train, y_train)

# Predict class labels for the test data

y_pred = knn.predict(X_test)

In this example, X_train and y_train represent the training

features and labels respectively, while X_test represents the test
features. The parameter n_neighbors is set to 5, indicating that
the algorithm will consider the 5 nearest neighbors to make predic-
tions.

1 Code Explanation
In this section, we break down the code snippet and provide a brief
explanation of each step.

• Line 3: Create a K-NN classifier using the KNeighborsClassifier()

class from scikit-learn. The parameter n_neighbors is set to
control the value of K, which determines the number of near-
est neighbors to consider.
• Line 6: Fit the classifier to the training data using the fit()
method. This step involves calculating the distances between
the training samples and building the internal data structure
of the classifier.
• Line 9: Predict class labels for the test data using the predict()
method. The classifier assigns the class label based on the
majority vote or weighted vote of the nearest neighbors.

This code snippet demonstrates the simplicity and ease of use

of the K-NN algorithm in scikit-learn for classification tasks.

80
Conclusion
The K-Nearest Neighbors algorithm is a non-parametric classifi-
cation algorithm that leverages the concept of similarity to assign
class labels to input samples. Through the use of distance metrics,
the K-NN algorithm identifies the K nearest neighbors and predicts
the class label based on majority or weighted voting. The K-NN
algorithm’s computational complexity depends on the number of
training samples and features, making it important to consider the
efficiency of distance calculations when working with large datasets
or high-dimensional feature spaces.

81
Chapter 16

Decision Trees

Introduction
Decision trees are widely used in machine learning for both classi-
fication and regression tasks. They are intuitive and provide trans-
parent decision-making processes. In this chapter, we will discuss
the mathematical foundations of decision trees, explore the infor-
mation gain criterion, and examine the algorithm’s computational
complexity.

Entropy and Information Gain

Entropy is a measure of impurity or unpredictability in a set of
samples. In the context of decision trees, entropy is used to evaluate
the homogeneity of a target variable within a node. Given a set of
samples S, with p(i) denoting the probability of class i within S,
the entropy H(S) is calculated as:
X
H(S) = − p(i) log2 p(i)
i

The information gain criterion is used to select the best feature

for splitting the data. It measures the reduction in entropy achieved
by splitting the data based on a specific feature. Given a feature A
with v distinct values and a set of subsets S1 , S2 , ..., Sv , obtained
by splitting the data based on A, the information gain IG(A) is
computed using the following equation:

82
 v
X |Si |
IG(A) = H(S) − H(Si )
i=1
|S|

where |Si | and |S| denote the number of samples in Si and S,

respectively.

Decision Tree Algorithm

The decision tree algorithm constructs a tree-like structure in which
each internal node represents a test on a feature, each branch rep-
resents an outcome of the test, and each leaf node represents a class
label. The algorithm follows these steps:

1 Step 1: Select Best Split

Choose the feature that maximizes the information gain or mini-
mizes the Gini impurity to determine the best split.

2 Step 2: Assign Leaf Node or Recurse

If the stopping criteria are met (e.g., maximum depth or minimum
number of samples), assign a leaf node with the majority class of
the samples. Otherwise, split the data based on the selected feature
and repeat the process recursively for each child node.

Computational Complexity
The computational complexity of the decision tree algorithm de-
pends on the number of samples m and the number of features
n in the dataset. Let f (n) denote the complexity of finding the
best split at each node. The overall time complexity for building a
decision tree is given by:

T (m, n) = depth × f (n)

During prediction, the decision tree takes O(log m) time to tra-
verse the tree and assign a class label to a new sample.

83
Python Code
Below is a Python code snippet demonstrating the use of the deci-
sion tree algorithm using the scikit-learn library:

from sklearn.tree import DecisionTreeClassifier

# Create a decision tree classifier

dt_classifier = DecisionTreeClassifier()

# Fit the classifier to the training data

dt_classifier.fit(X_train, y_train)

# Predict class labels for the test data

y_pred = dt_classifier.predict(X_test)

In this example, X_train and y_train represent the training

features and labels, respectively, while X_test represents the test
features.

1 Code Explanation
Here is a brief explanation of the code snippet:

• Line 3: Create a decision tree classifier using the DecisionTreeClassifier()

class from scikit-learn.
• Line 6: Fit the classifier to the training data using the fit()
method. The algorithm will construct the decision tree based
on the training samples and labels.

• Line 9: Predict class labels for the test data using the predict()
method. The decision tree assigns a class label to each test
sample based on the tree structure and the selected features.

This code snippet demonstrates the simplicity and ease of use

of the decision tree algorithm in scikit-learn for classification tasks.

84
Chapter 17

Random Forests

Introduction
In this chapter, we will delve into the mathematical foundations
of random forests, a popular ensemble learning method. Random
forests combine multiple decision trees to make predictions. They
utilize the principles of bagging and random feature selection to
create a diverse set of decision trees, improving their predictive
power and generalization performance.

Bagging and Decision Trees

1 Bagging
Bagging (short for bootstrap aggregating) is a technique used to
reduce variance and improve the stability of machine learning mod-
els. It involves creating multiple bootstrap samples from the origi-
nal dataset and training separate models on each of these samples.
The predictions of the models are then combined, often by aver-
aging or majority voting, to obtain the final prediction. Bagging
helps to reduce the impact of outliers and noisy data, leading to
more robust models.

2 Decision Trees
Decision trees are versatile and easy to interpret models that re-
cursively partition the feature space based on the selected features.

85
Each internal node of the tree represents a decision based on a fea-
ture, and each leaf node corresponds to a predicted label or value.
Despite their simplicity, decision trees tend to overfit the training
data. This is where bagging comes into play, as it helps to alleviate
overfitting by constructing an ensemble of decision trees.

Random Forest Algorithm

The random forest algorithm builds upon the principles of bagging
and decision trees. It creates an ensemble of decision trees grown
on different subsets of the training data and features.

1 Random Subset of Features

At each split of a decision tree, only a random subset of features is
considered for splitting. This random feature selection injects di-
versity among the trees, and reduces the correlation between them.
It also allows individual trees to focus on different aspects of the
data, leading to more robust predictions.

2 Building the Ensemble

The random forest algorithm follows these steps:

1. Create multiple bootstrap samples from the original training

data.
2. For each bootstrap sample, build a decision tree using a ran-
dom subset of features.

3. Grow the trees to their maximum depth, without pruning.

4. To make predictions, aggregate the predictions of all the de-
cision trees. In classification tasks, majority voting is com-
monly used, while in regression tasks, the predictions are av-
eraged across trees.

Python Code
Let’s demonstrate the use of the random forest algorithm in scikit-
learn with a Python code snippet:

86
from sklearn.ensemble import RandomForestClassifier

# Create a random forest classifier

rf_classifier =
,→ RandomForestClassifier(n_estimators=100)

# Fit the classifier to the training data

rf_classifier.fit(X_train, y_train)

# Predict class labels for the test data

y_pred = rf_classifier.predict(X_test)

In this example, X_train and y_train represent the training

features and labels, respectively, while X_test represents the test
features.

1 Code Explanation
Let’s explain the code snippet step by step:

• Line 3: Create a random forest classifier using the RandomForestClassifier()

class from scikit-learn. The n_estimators parameter deter-
mines the number of decision trees in the random forest.
• Line 6: Fit the random forest classifier to the training data
using the fit() method. The algorithm builds an ensemble
of decision trees by applying bagging and feature randomiza-
tion.
• Line 9: Predict class labels for the test data using the predict()
method. The random forest combines the predictions of all
the decision trees to obtain the final prediction for each test
sample.

This code snippet demonstrates the simplicity and ease of use

of the random forest algorithm in scikit-learn for both classification
tasks.

Conclusion
In this chapter, we explored random forests, an ensemble learning
method that combines bagging and decision trees. We discussed

87
the benefits of bagging and how it reduces variance and improves
model stability. Additionally, we elaborated on the random forest
algorithm, which incorporates random feature selection to enhance
diversity among decision trees. Finally, we provided a Python code
snippet demonstrating the implementation of random forests using
the scikit-learn library.

88
Chapter 18

Support Vector
Machines (SVM)

Support Vector Machines (SVM) are powerful machine learning

models that are widely used for classification and regression tasks.
In this chapter, we will explore the mathematical foundations of
SVM and understand how they work.

Margin and Hyperplanes

In SVM, the goal is to find a hyperplane that separates the data
points of different classes such that the margin, which is the dis-
tance between the hyperplane and the closest data points, is max-
imized. The hyperplane is defined as:

f (x) = w · x + b = 0
where w is the normal vector to the hyperplane and b is the
bias or intercept term.

1 Margin Maximization
To maximize the margin, SVM aims to solve the following opti-
mization problem:

minimize ∥w∥
w,b

subject to yi (w · xi + b) ≥ 1, ∀i

89
 where xi is a data point and yi is its corresponding class label.
The inequality constraint ensures that all data points are correctly
classified and lie on the correct side of the margin.

2 Soft Margin SVM

In many cases, the data may not be linearly separable. To handle
such cases, a slack variable ξi is introduced, allowing some data
points to be misclassified or fall within the margin. This leads to
the formulation of the soft margin SVM:
n
1 X
minimize ∥w∥2 + C ξi
w,b,ξ 2 i=1
subject to yi (w · xi + b) ≥ 1 − ξi , ∀i
ξi ≥ 0, ∀i
where C is the regularization parameter that controls the trade-
off between maximizing the margin and minimizing the misclassi-
fication.

Kernel Trick
The kernel trick is a fundamental concept in SVM that allows us
to handle nonlinearly separable data by implicitly mapping the
original input space into a higher-dimensional feature space. This
is achieved by defining a kernel function K(x, x′ ) that computes
the inner product between the feature vectors of two data points
without explicitly computing the transformation.

1 Commonly Used Kernels

Some commonly used kernels in SVM include:

• Linear Kernel: K(x, x′ ) = x · x′

• Polynomial Kernel: K(x, x′ ) = (γx · x′ + r)d
• Gaussian RBF Kernel: K(x, x′ ) = exp(−γ∥x − x′ ∥2 )

Here, γ, r, and d are kernel parameters.

90
2 Kernel Trick in Dual Formulation
Using the kernel trick, the soft margin SVM problem can be rewrit-
ten in the dual form:
n n n
X 1 XX
maximize αi − αi αj yi yj K(xi , xj )
α
i=1
2 i=1 j=1
n
X
subject to αi yi = 0
i=1
0 ≤ αi ≤ C, ∀i
where α = [α1 , α2 , . . . , αn ] are the Lagrange multipliers associ-
ated with the constraints.

Python Code
Let’s illustrate the usage of the SVM classifier in scikit-learn with
a Python code snippet:

from sklearn.svm import SVC

# Create an SVM classifier with a radial basis

,→ function (RBF) kernel
svm_classifier = SVC(kernel='rbf')

# Fit the classifier to the training data

svm_classifier.fit(X_train, y_train)

# Predict class labels for the test data

y_pred = svm_classifier.predict(X_test)

In this example, X_train and y_train represent the training

features and labels, respectively, while X_test represents the test
features.

1 Code Explanation
• Line 3: Create an SVM classifier using the SVC() class from
scikit-learn. The kernel=’rbf’ parameter specifies the ra-
dial basis function (RBF) kernel, which is commonly used for
SVM classification tasks.

91
 • Line 6: Fit the SVM classifier to the training data using the
fit() method. The algorithm finds the optimal separating
hyperplane that maximizes the margin.
• Line 9: Predict class labels for the test data using the predict()
method. The SVM classifier uses the learned model to make
predictions for unseen data points.

This code snippet demonstrates the usage of the SVM classifier

in scikit-learn for classification tasks.

92
Chapter 19

Principal Component
Analysis (PCA)

Covariance Matrix
The covariance matrix provides important information about the
relationships between different features in a dataset. Given a data
matrix X ∈ Rn×p , where each row represents a data point and each
column represents a feature, the covariance matrix Σ is defined as:
1
Σ= (X − µ)T (X − µ)
n
where µ is the mean vector of X, obtained by subtracting the
mean of each column from the respective column.

1 Python Code
import numpy as np

# Calculate the covariance matrix

cov_matrix = np.cov(X.T)

Eigen Decomposition
Eigen decomposition is a key step in Principal Component Analysis
(PCA) and is used to obtain the principal components of a dataset.

93
The eigen decomposition of the covariance matrix Σ is given by:

Σ = VΛVT
where V is a matrix whose columns are the eigenvectors of Σ,
and Λ is a diagonal matrix containing the corresponding eigenval-
ues.

1 Python Code
eigenvals, eigenvects = np.linalg.eig(cov_matrix)

Dimensionality Reduction
PCA allows for dimensionality reduction by selecting a subset of
the principal components. The principal components of a dataset
are the eigenvectors of the covariance matrix, ordered by their cor-
responding eigenvalues in descending order. By selecting the first
k principal components, we obtain a lower-dimensional representa-
tion of the data.

1 Python Code
from sklearn.decomposition import PCA

# Create a PCA instance and specify the desired

,→ number of components
pca = PCA(n_components=k)

# Fit the PCA model to the data

pca.fit(X)

# Transform the data to the lower-dimensional space

X_reduced = pca.transform(X)

94
Chapter 20

K-Means Clustering

Introduction
K-means clustering is a widely used algorithm for partitioning data
into clusters. It is an unsupervised learning method that aims
to find a set of cluster centroids that minimize the within-cluster
variance. In this chapter, we will discuss the key concepts and steps
involved in the K-means algorithm.

Distance Metrics
To measure the similarity between data points, a distance metric
is needed. The most common distance metric used in K-means
clustering is the Euclidean distance. For two data points xi and
xj in a d-dimensional space, the Euclidean distance is calculated
as follows:
v
u d
uX
Euclidean Distance(xi , xj ) = t (xik − xjk )2
k=1

where xik and xjk represent the k-th feature values of xi and
xj respectively.

1 Python Code
from scipy.spatial.distance import euclidean

95
# Calculate the Euclidean distance between two data
,→ points
distance = euclidean(x_i, x_j)

Algorithm Steps
The K-means clustering algorithm consists of the following steps:

1 Initialization
Randomly select K data points as initial cluster centroids.

2 Assignment
Assign each data point to the nearest cluster centroid based on the
chosen distance metric.

3 Update
Update the cluster centroids by computing the mean of the data
points assigned to each cluster.

4 Iteration
Repeat the assignment and update steps until convergence, i.e.,
there is no change in the assignment of data points to clusters or
a specified number of iterations is reached.

5 Python Code
from sklearn.cluster import KMeans

# Create a KMeans instance and specify the desired

,→ number of clusters
kmeans = KMeans(n_clusters=K, random_state=0)

# Fit the KMeans model to the data

kmeans.fit(X)

96
# Retrieve the cluster assignments for each data
,→ point
cluster_labels = kmeans.labels_

Computational Complexity
The computational complexity of the K-means algorithm depends
on the number of data points n, the number of features d, and the
number of clusters K. The time complexity is typically denoted
as O(n · d · K · I · t), where I is the number of iterations required
to converge and t is the average time complexity of computing the
distance metric.

Choosing the Number of Clusters

Determining the optimal number of clusters, K, is an important
task in K-means clustering. There are various methods available,
such as the elbow method and silhouette analysis, to guide this
selection process.

1 Elbow Method
The elbow method involves plotting the within-cluster sum of squared
distances against different values of K. The optimal number of
clusters is chosen at the "elbow" point, where the rate of decrease
in the sum of squared distances significantly diminishes.

2 Python Code
import matplotlib.pyplot as plt

# Calculate the sum of squared distances for

,→ different values of K
ssd = []
for k in range(1, max_k+1):
kmeans = KMeans(n_clusters=k, random_state=0)
kmeans.fit(X)
ssd.append(kmeans.inertia_)

97
# Plot the sum of squared distances against different
,→ values of K
plt.plot(range(1, max_k+1), ssd)
plt.xlabel('Number of Clusters (K)')
plt.ylabel('Sum of Squared Distances')
plt.show()

Conclusion
In this chapter, we discussed the K-means clustering algorithm, in-
cluding the distance metric, algorithm steps, computational com-
plexity, and methods for choosing the number of clusters. K-means
clustering is a powerful tool for discovering patterns and structures
within data, making it widely applicable in various domains.

98
Chapter 21

Expectation-
Maximization (EM)

Gaussian Mixture Models

A Gaussian Mixture Model (GMM) is a probabilistic model that
represents a set of data points as a mixture of Gaussian distribu-
tions. It is widely used for clustering and density estimation tasks.
Let X = {x1 , x2 , ..., xn } be the observed data, where xi ∈ Rd rep-
resents a d-dimensional data point.
The GMM assumes that each data point xi is generated from
one of K Gaussian components, with each component characterized
by a mean vector µk and a covariance matrix Σk . The GMM
also introduces a latent variable zi to indicate the component from
which xi is generated. zi is a one-hot vector, with the k-th element
being 1 if xi is generated from the k-th Gaussian component, and
0 otherwise.
n Y
Y K
p(X, Z, θ) = [πk N (xi |µk , Σk )]zik
i=1 k=1

where Z = {z1 , z2 , ..., zn } is the set of latent variables, and θ =

{µ1 , ..., µK , Σ1 , ..., ΣK , π1 , ..., πK } is the set of model parameters.
Here, πk represents the PK mixing coefficient for the k-th Gaussian
component, satisfying k=1 πk = 1.

99
1 Python Code
from sklearn.mixture import GaussianMixture

# Create a GaussianMixture instance and specify the

,→ desired number of components
gmm = GaussianMixture(n_components=K, random_state=0)

# Fit the GaussianMixture model to the data

gmm.fit(X)

# Retrieve the cluster assignments for each data

,→ point
cluster_labels = gmm.predict(X)

The Expectation-Maximization Algorithm

The Expectation-Maximization (EM) algorithm is an iterative al-
gorithm used to estimate the parameters of models with latent
variables. In the case of GMMs, EM is used to estimate the un-
known model parameters θ given the observed data X.
The EM algorithm consists of two main steps: the E-step and
the M-step.

1 The E-step
In the E-step, the posterior probabilities of the latent variables Z
are calculated given the current estimate of the model parameters
θ (t) .
(t) (t) (t)
(t) π N (xi |µk , Σk )
γik = PK k (t) (t) (t)
j=1 πj N (xi |µj , Σj )
(t)
where γik is the responsibility of the k-th component for the
i-th data point under the model parameters θ (t) .

2 The M-step
In the M-step, the model parameters θ are updated by maximizing
the expected complete log-likelihood with respect to θ, using the
responsibilities calculated in the E-step.

100
 N
(t+1) 1 X (t)
µk = γ xi
Nk i=1 ik
N
(t+1) 1 X (t) (t+1) (t+1) T
Σk = γ (xi − µk )(xi − µk )
Nk i=1 ik

(t+1) Nk
πk =
N
PN (t)
where Nk = i=1 γik is the effective number of data points
assigned to the k-th component.

3 Python Code
# Perform the E-step: calculate the responsibilities
responsibilities = gmm.predict_proba(X)

# Perform the M-step: update the model parameters

gmm.means_ = np.dot(responsibilities.T, X) /
,→ np.sum(responsibilities, axis=0)[:, np.newaxis]
gmm.covariances_ = np.einsum('ijk,ikl->ijl',
,→ responsibilities * (X[:, :, np.newaxis] -
,→ gmm.means_),
X[:, :, np.newaxis] -
,→ gmm.means_) /
,→ np.sum(responsibilities,
,→ axis=0)[:,
,→ np.newaxis,
,→ np.newaxis]
gmm.weights_ = np.mean(responsibilities, axis=0)

Convergence Criteria
The EM algorithm iterates between the E-step and the M-step un-
til convergence. One common convergence criterion is the change
in the value of the log-likelihood between iterations. The algo-
rithm terminates when the change in the log-likelihood falls below
a predefined threshold.

101
1 Python Code
# Specify the convergence threshold
tolerance = 1e-3

# Initialize the previous log-likelihood value

prev_log_likelihood = -np.inf

while True:
# Perform the E-step and M-step

# Calculate the current log-likelihood value

current_log_likelihood = gmm.score(X)

# Check for convergence

if current_log_likelihood - prev_log_likelihood <
,→ tolerance:
break

# Update the previous log-likelihood value

prev_log_likelihood = current_log_likelihood

Conclusion
In this chapter, we discussed the Expectation-Maximization (EM)
algorithm for Gaussian Mixture Models (GMMs). The EM al-
gorithm provides a framework for estimating the parameters of
GMMs, leveraging the latent variables to iteratively improve the
model. The EM algorithm is a powerful tool for density estima-
tion and clustering tasks, enabling the modeling of complex data
distributions.

102
Chapter 22

Hierarchical Clustering

Hierarchical clustering is a popular method for clustering analysis

that aims to group similar data points into nested clusters. This
chapter provides an overview of hierarchical clustering, including
the agglomerative and divisive approaches, different linkage crite-
ria, and the interpretation of dendrograms.

Agglomerative vs. Divisive Hierarchical

Clustering
Hierarchical clustering can be performed using two main strategies:
agglomerative and divisive.
In agglomerative clustering, all data points start as individ-
ual clusters and are successively merged together based on their
similarity. This bottom-up approach begins with each data point
as a separate cluster and iteratively merges the two closest clusters
until a stopping criterion is met. The algorithm builds a hierarchy
of clusters, known as a dendrogram, which can be further analyzed
to determine the number of clusters.
In divisive clustering, all data points initially belong to a
single cluster, and the algorithm proceeds to divide the clusters
into smaller subclusters until a stopping criterion is met. This top-
down approach starts with one cluster and recursively splits it into
two clusters using a specified algorithm until the desired number
of clusters or another stopping criterion is reached.
While agglomerative clustering is generally more common, divi-
sive clustering may be useful in certain scenarios where the division

103
process is easier than the agglomeration process.

Linkage Criteria
One crucial aspect of hierarchical clustering is the choice of link-
age criteria, which measure the dissimilarity or similarity between
clusters. There are several commonly used linkage criteria:

• Single Linkage: The distance between two clusters is de-

fined as the minimum distance between any two points in the
two clusters. It tends to create long, straggly clusters.
• Complete Linkage: The distance between two clusters is
defined as the maximum distance between any two points
in the two clusters. It tends to create compact, spherical
clusters.
• Average Linkage: The distance between two clusters is de-
fined as the average distance between every pair of points
from different clusters. It provides a balance between single
and complete linkage, suppressing the impact of outliers.
• Ward’s Linkage: The distance between two clusters is de-
fined as the increase in the error sum of squares (ESS) that
would result from merging the two clusters. Ward’s method
aims to minimize within-cluster variance.

The choice of linkage criteria depends on the nature of the data

and the intended interpretation of the clusters.

Dendrograms Interpretation
A dendrogram is a hierarchical tree-like structure that represents
the relationships between clusters during agglomerative clustering.
It illustrates the merging process and helps determine the number
of clusters in the dataset.
A dendrogram is typically plotted with the dissimilarity or sim-
ilarity measure on the vertical axis and the data points on the
horizontal axis. The height of the linkage between two clusters
represents the dissimilarity or distance between them. The longer
the linkage, the more dissimilar the clusters.

104
 To determine the number of clusters from a dendrogram, a hor-
izontal cut can be made at a particular height. The vertical lines
intersected by this cut represent the clusters. The number of clus-
ters can be determined by the number of vertical lines intersected.

1 Python Code
from scipy.cluster.hierarchy import dendrogram,
,→ linkage
import matplotlib.pyplot as plt

# Perform hierarchical clustering using a specific

,→ linkage criterion
Z = linkage(X, method='average')

# Generate a dendrogram
dendrogram(Z)

# Set the distance threshold for the cut

distance_threshold = 50

# Draw a horizontal cut at the desired threshold

plt.axhline(y=distance_threshold, color='r',
,→ linestyle='--')
plt.show()

Hierarchical clustering, with its agglomerative and divisive ap-

proaches, offers a versatile and interpretable method for cluster-
ing analysis. By selecting the appropriate linkage criterion and
interpreting the resulting dendrogram, hierarchical clustering can
provide valuable insights into the natural groupings of a dataset.

105
Chapter 23

Reinforcement
Learning Basics

Markov Decision Processes (MDPs)

A Markov decision process (MDP) is a mathematical framework
for modeling sequential decision-making problems. It is defined by
a tuple (S, A, P, R, γ), where:
• S is the set of states in the environment.
• A is the set of actions available to the agent.

• P is the state transition function; P (s, a, s′ ) denotes the prob-

ability of transitioning from state s to state s′ under action
a.
• R is the reward function; R(s, a, s′ ) represents the immediate
reward received after transitioning from state s to state s′
under action a.
• γ is the discount factor, which determines the importance
of future rewards relative to immediate rewards. It should
satisfy 0 ≤ γ < 1.
The goal of an agent in an MDP is to learn a policy π : S → A,
which maps states to actions, in order to maximize its cumulative
discounted reward over time.

106
Bellman Equations
The Bellman equations are a set of mathematical equations that
decompose the value function into more manageable sub-problems.
These equations are central to reinforcement learning algorithms.

1 Value function
The value function V π (s) for a policy π represents the expected
cumulative discounted reward starting from a state s and follow-
ing policy π thereafter. It can be expressed using the Bellman
expectation equation:
X X
V π (s) = π(a|s) P (s, a, s′ ) [R(s, a, s′ ) + γV π (s′ )]
a∈A s′ ∈S

2 Action-value function
The action-value function Qπ (s, a) for a policy π represents the
expected cumulative discounted reward starting from a state s,
taking action a, and then following policy π thereafter. It can be
expressed using the Bellman expectation equation:
" #
X X
Q (s, a) =
π
P (s, a, s ) R(s, a, s ) + γ
′ ′
π(a |s )Q (s , a )
′ ′ π ′ ′

s′ ∈S a′ ∈A

Policy and Value Iterations

Policy iteration and value iteration are two fundamental algorithms
used to solve the reinforcement learning problem.

1 Policy Iteration
Policy iteration is an iterative algorithm that alternates between
policy evaluation and policy improvement until convergence. In
each iteration, the value function is first updated using the Bell-
man expectation equation. Then, the policy is improved by acting
greedily with respect to the current value function.

107
2 Value Iteration
Value iteration is a simplified version of policy iteration that di-
rectly combines policy evaluation and policy improvement in a
single update step. It repeatedly applies the Bellman optimality
equation to update the value function until convergence.

3 Python Code
def policy_evaluation(pi, P, R, gamma, tol=1e-6,
,→ max_iterations=1000):
V = np.zeros(len(P))
for _ in range(max_iterations):
V_prime = np.copy(V)
for s in range(len(P)):
a = pi[s]
V[s] = sum(P[s, a, s_prime] * (R[s, a,
,→ s_prime] + gamma * V_prime[s_prime])
for s_prime in range(len(P)))
if np.max(np.abs(V - V_prime)) < tol:
break
return V

def policy_iteration(P, R, gamma, tol=1e-6,

,→ max_iterations=1000):
n_states, n_actions, _ = P.shape
pi = np.zeros(n_states, dtype=int)
for _ in range(max_iterations):
V = policy_evaluation(pi, P, R, gamma, tol,
,→ max_iterations)
policy_stable = True
for s in range(n_states):
a = pi[s]
max_q = np.max(sum(P[s, a, s_prime] *
,→ (R[s, a, s_prime] + gamma *
,→ V[s_prime])
for s_prime in
,→ range(n_states)))
for a_prime in range(n_actions):

108
 q = sum(P[s, a_prime, s_prime] *
,→ (R[s, a_prime, s_prime] + gamma *
,→ V[s_prime])
for s_prime in
,→ range(n_states))
if q > max_q:
pi[s] = a_prime
policy_stable = False
break
if policy_stable:
break
return pi, V

def value_iteration(P, R, gamma, tol=1e-6,

,→ max_iterations=1000):
V = np.zeros(len(P))
for _ in range(max_iterations):
V_prime = np.copy(V)
for s in range(len(P)):
max_q = np.max(sum(P[s, a, s_prime] *
,→ (R[s, a, s_prime] + gamma *
,→ V_prime[s_prime])
for s_prime in
,→ range(len(P))))
V[s] = max_q
if np.max(np.abs(V - V_prime)) < tol:
break
pi = np.argmax([sum(P[s, a, s_prime] * (R[s, a,
,→ s_prime] + gamma * V[s_prime])
for s_prime in
,→ range(len(P))) for a in
,→ range(len(P[0]))],
,→ axis=1)
return pi, V

109
Chapter 24

Q-Learning

Introduction
In this chapter, we delve into the mathematical foundations of Q-
Learning, a popular reinforcement learning algorithm. Q-Learning
is a model-free, off-policy method that enables an agent to learn
an optimal policy by iteratively updating its action-value function,
known as the Q-function. We begin by defining the Q-function and
its update rule, and then explore its convergence properties.

Q-Function
The Q-function, denoted as Q(s, a), represents the expected cu-
mulative discounted reward if the agent takes action a in state s
and then follows a certain policy π. This function is defined for all
state-action pairs (s, a).

Bellman Update
The key idea behind Q-Learning is to iteratively update the Q-
function based on observed rewards. The update rule, known as
the Bellman update, is given by the following equation:

h i
Q(s, a) ← Q(s, a) + α R(s, a) + γ max
′
Q(s′ ′
, a ) − Q(s, a)
a

where:

110
 • α is the learning rate, which determines the extent to which
newly observed information overrides existing knowledge.
• R(s, a) is the immediate reward received when taking action
a in state s.

• γ is the discount factor, which controls the importance of

future rewards.
• maxa′ Q(s′ , a′ ) represents the maximum Q-value among all
possible actions in the next state s′ .

1 Python Code
import numpy as np

def q_learning(Q, s, a, r, s_prime, alpha, gamma):

max_q_prime = np.max(Q[s_prime])
Q[s, a] += alpha * (r + gamma * max_q_prime -
,→ Q[s, a])

Convergence
Under certain conditions, Q-Learning has been shown to converge
to the optimal Q-function as the number of iterations approaches
infinity. Convergence is guaranteed if the agent explores all state-
action pairs infinitely often, a property known as exploration with
probability one.

Conclusion
Q-Learning is a powerful algorithm that can enable an agent to
learn an optimal policy without requiring a model of the environ-
ment. By iteratively updating the Q-function based on observed
rewards, the agent can make informed decisions and achieve better
performance over time. In the next chapter, we will explore deep
Q-learning, an extension of Q-learning that leverages deep neural
networks to handle high-dimensional state spaces.

111
Chapter 25

Deep Q-Learning

Introduction
In this chapter, we explore the mathematical foundations of Deep
Q-Learning, an extension of Q-Learning that employs deep neu-
ral networks to handle high-dimensional state spaces. Deep Q-
Learning has proven to be an effective approach in solving complex
reinforcement learning problems by utilizing function approxima-
tion with neural networks.

Q-Network
The core idea behind Deep Q-Learning is to approximate the Q-
function, denoted as Q(s, a), using a neural network. The Q-
network takes in a state s as input and outputs the predicted
Q-values for each possible action a. By training the network to
minimize the difference between the predicted Q-values and the
target Q-values, the Q-network learns to estimate the optimal Q-
function.

Experience Replay
To improve the training of the Q-network, Deep Q-Learning utilizes
experience replay. Experience replay involves storing the agent’s
experiences, typically in the form of a tuple (s, a, r, s′ ) representing
the state, action, immediate reward, and next state. During train-

112
ing, a batch of experiences is sampled uniformly at random from
the memory buffer to break the correlation between consecutive
experiences. This allows the network to learn from a more diverse
set of experiences.

Target Network
To stabilize the training process, Deep Q-Learning incorporates a
separate target network in addition to the Q-network. The target
network is a copy of the Q-network that is periodically updated
to match the current Q-network. The target network is used to
compute the target Q-values for training, providing a more stable
and reliable target for the Q-network to learn from.

Loss Function
The loss function used in Deep Q-Learning is the mean squared
error (MSE) loss between the predicted Q-values and the target
Q-values. The loss for a single training sample is given by:
 2
L(θ) = Q(s, a; θ) − (r + γ · max
′
Q(s′ ′ −
, a ; θ ))
a

where:

θ is the parameters of the Q-network

θ −
is the parameters of the target network
s, a, r, s are the state, action, reward, and next state of an experience
′

γ is the discount factor

1 Python Code
import tensorflow as tf

def mse_loss(Q, Q_target):

return tf.reduce_mean(tf.square(Q - Q_target))

113
Epsilon-Greedy Exploration
To balance between exploration and exploitation, Deep Q-Learning
employs epsilon-greedy exploration. With probability ϵ, the agent
selects a random action to explore the environment, while with
probability 1 − ϵ, it selects the action with the highest Q-value
according to the Q-network.

Conclusion
Deep Q-Learning leverages the power of deep neural networks to
handle complex state spaces, enabling agents to learn optimal poli-
cies in challenging reinforcement learning environments. By utiliz-
ing experience replay, target networks, and epsilon-greedy explo-
ration, Deep Q-Learning improves the stability and convergence
of the learning process. In the next chapter, we will delve into
Policy Gradient Methods, another class of reinforcement learning
algorithms that directly optimize the policy without using a value
function.

114
Chapter 26

Policy Gradient
Methods

Introduction
Reinforcement learning (RL) focuses on designing intelligent agents
that learn through interaction with an environment. One class of
RL algorithms, known as policy gradient methods, aims to directly
optimize the policy without using a value function. This chapter
explores the mathematical foundations of policy gradient methods
in the context of reinforcement learning.

Policy Function
In policy gradient methods, the policy is represented by a param-
eterized function πθ (a|s) that outputs the probability of taking
action a given state s and parameter θ. The goal is to find the op-
timal policy parameters θ∗ that maximize the expected cumulative
reward over a trajectory τ .

REINFORCE Algorithm
The REINFORCE algorithm is a fundamental policy gradient method
that uses the likelihood ratio gradient estimator to update the pol-
icy parameters. The update rule for the parameter θ is given by:

115
 θ ← θ + α∇θ J(θ)
where α is the learning rate and ∇θ J(θ) is the gradient of the
expected cumulative reward J(θ) with respect to the policy param-
eters θ.

1 Python Code
The following Python code snippet implements the REINFORCE
algorithm:

import numpy as np

def reinforce_update(policy, alpha, states, actions,

,→ rewards):
gradient = np.zeros_like(policy.weights)
for t in range(len(states)):
state = states[t]
action = actions[t]
reward = rewards[t]
log_prob = policy.log_prob(state, action)
gradient += log_prob * reward
gradient /= len(states)
policy.weights += alpha * gradient

Advantage Actor-Critic (A2C)

The Advantage Actor-Critic (A2C) algorithm combines the pol-
icy gradient approach with a learned value function estimation.
The A2C algorithm maintains both a policy network, denoted as
πθ (a|s), and a value function network, denoted as Vϕ (s). The policy
is updated using the policy gradient update rule, while the value
function is updated to minimize the mean squared error between
the predicted value and the observed returns.

1 Python Code
The following Python code snippet outlines the update steps in the
A2C algorithm:

116
import numpy as np

def a2c_update(policy, value_function, alpha_p,

,→ alpha_v, states, actions, rewards, returns):
# Update policy
gradient_p = np.zeros_like(policy.weights)
for t in range(len(states)):
state = states[t]
action = actions[t]
log_prob = policy.log_prob(state, action)
gradient_p += log_prob * (rewards[t] -
,→ value_function.predict(state))
gradient_p /= len(states)
policy.weights += alpha_p * gradient_p

# Update value function

gradient_v =
,→ np.zeros_like(value_function.weights)
for t in range(len(states)):
state = states[t]
target = rewards[t] + returns[t]
gradient_v += (value_function.predict(state)
,→ - target) * state
gradient_v /= len(states)
value_function.weights += alpha_v * gradient_v

Proximal Policy Optimization (PPO)

Proximal Policy Optimization (PPO) is an advanced policy gradi-
ent algorithm that addresses the issue of policy divergence during
update steps. PPO introduces a surrogate objective that constrains
the policy update to a certain "trust region" around the current
policy. By optimizing this surrogate objective, PPO ensures that
the policy update remains within a reasonable distance from the
previous policy.

1 Python Code
The following Python code snippet illustrates the surrogate objec-
tive used in the PPO algorithm:

117
import tensorflow as tf

def ppo_objective(old_probs, new_probs, actions,

,→ advantages, epsilon):
ratio = new_probs / old_probs
clipped_ratio = tf.clip_by_value(ratio, 1 -
,→ epsilon, 1 + epsilon)
surrogate_loss = tf.reduce_mean(tf.minimum(ratio
,→ * advantages, clipped_ratio * advantages))
return -surrogate_loss

Conclusion
Policy gradient methods provide a powerful framework for train-
ing reinforcement learning agents by directly optimizing the policy.
The REINFORCE algorithm, A2C, and PPO are popular policy
gradient algorithms that have achieved excellent results in a wide
range of RL tasks. In the next chapter, we will delve into the
mathematical foundations of artificial neural networks (ANNs), a
key component of many modern machine learning models.

118
Chapter 27

Convolutional Neural
Networks (CNNs)

Introduction
In this chapter, we explore Convolutional Neural Networks (CNNs),
a class of artificial neural networks specifically designed for process-
ing grid-like data such as images. CNNs have achieved remarkable
success in image recognition tasks, demonstrating their ability to
learn hierarchical representations directly from raw pixel data. We
will discuss the mathematical foundations and key components of
CNNs, including convolutional layers, pooling layers, and activa-
tion functions.

Convolution Operation
The convolution operation is a fundamental building block of CNNs.
It involves sliding a filter (or kernel) over the input image, comput-
ing element-wise multiplications between the filter weights and the
corresponding image patch, and summing the results to produce a
single output. Mathematically, the convolution operation can be
defined as follows:

XX
Output[i, j] = Input[i + m, j + n] · Filter[m, n]
m n

119
 where Input is the input image, Filter is the filter or kernel, and
Output is the resulting feature map.

1 Python Code
The following Python code snippet demonstrates the convolution
operation using the NumPy library:

import numpy as np

def convolution(image, filter):

height, width = image.shape
f_height, f_width = filter.shape
output = np.zeros((height - f_height + 1, width -
,→ f_width + 1))

for i in range(output.shape[0]):
for j in range(output.shape[1]):
output[i, j] = np.sum(image[i:i+f_height,
,→ j:j+f_width] * filter)

return output

Pooling Layers
Pooling layers are another vital component of CNNs, used to reduce
the spatial dimensions of the input and extract the most salient fea-
tures. The most common pooling operation is max pooling, which
selects the maximum value within a specified window. Mathemat-
ically, max pooling can be defined as:

Output[i, j] = max(Input[i × stride, j × stride])

where Input is the input feature map, Output is the resulting
downsampled feature map, and stride is the stride length.

1 Python Code
The following Python code snippet illustrates the max pooling op-
eration:

120
import numpy as np

def max_pooling(feature_map, stride):

height, width = feature_map.shape
pool_height = height // stride
pool_width = width // stride
output = np.zeros((pool_height, pool_width))

for i in range(pool_height):
for j in range(pool_width):
output[i, j] =
,→ np.max(feature_map[i*stride:(i+1)*stride,
,→ j*stride:(j+1)*stride])

return output

Activation Functions
Activation functions introduce non-linearities to CNNs, enabling
them to model complex relationships in the data. Common acti-
vation functions used in CNNs include the sigmoid function, the
hyperbolic tangent function, and the rectified linear unit (ReLU)
function. The ReLU function, defined as f (x) = max(0, x), is par-
ticularly popular due to its simplicity and ability to mitigate the
vanishing gradient problem.

1 Python Code
The following Python code snippet illustrates the ReLU activation
function:

import numpy as np

def relu(x):
return np.maximum(0, x)

121
Conclusion
In this chapter, we have explored the mathematical foundations
of Convolutional Neural Networks (CNNs) and their key compo-
nents, namely the convolution operation, pooling layers, and acti-
vation functions. CNNs have revolutionized the field of computer
vision and have become a cornerstone of modern image recognition
systems. In the next chapter, we will delve into Recurrent Neu-
ral Networks (RNNs), another class of neural networks specifically
designed for processing sequential data.

122
 **Chapter 29: Convolutional Neural Networks (CNNs)**

Introduction
In this chapter, we delve into the realm of Convolutional Neural
Networks (CNNs), a class of deep learning models that have rev-
olutionized the field of computer vision. CNNs are specifically de-
signed to process grid-like data such as images, leveraging the prin-
ciples of convolution, pooling, and non-linear activation functions.
In this chapter, we will explore the mathematical foundations of
CNNs and discuss their key components.

Convolution Operation
The convolution operation lies at the heart of Convolutional Neural
Networks. It involves sliding a filter, also known as a kernel, over
the input image and computing the element-wise multiplication
between the filter weights and the corresponding image patch. The
results of these multiplications are then summed to produce a single
output. Mathematically, the convolution operation can be defined
as follows:

XX
Output[i, j] = Input[i + m, j + n] · Filter[m, n]
m n

where Input represents the input image, Filter denotes the filter
or kernel, and Output is the resulting feature map. To demonstrate
this operation, we provide the following Python code snippet uti-
lizing the NumPy library:

import numpy as np

def convolution(image, filter):

height, width = image.shape
f_height, f_width = filter.shape
output = np.zeros((height - f_height + 1, width -
,→ f_width + 1))

for i in range(output.shape[0]):
for j in range(output.shape[1]):

123
 output[i, j] = np.sum(image[i:i+f_height,
,→ j:j+f_width] * filter)

return output

Pooling Layers
Pooling layers play a prominent role in Convolutional Neural Net-
works. Their primary purpose is to reduce the spatial dimensions
of the input data, while simultaneously extracting salient features.
Max pooling, a common pooling operation, selects the maximum
value within a defined window. Mathematically, max pooling can
be expressed as:

Output[i, j] = max(Input[i × stride, j × stride])

where Input represents the input feature map, Output denotes
the resulting downsampled feature map, and stride represents the
stride length. To illustrate this operation, we provide the following
Python code snippet:
import numpy as np

def max_pooling(feature_map, stride):

height, width = feature_map.shape
pool_height = height // stride
pool_width = width // stride
output = np.zeros((pool_height, pool_width))

for i in range(pool_height):
for j in range(pool_width):
output[i, j] =
,→ np.max(feature_map[i*stride:(i+1)*stride,
,→ j*stride:(j+1)*stride])

return output

Activation Functions
Activation functions introduce non-linearities to Convolutional Neu-
ral Networks, enabling them to model complex relationships in the

124
data. Common activation functions employed in CNNs include the
sigmoid function, the hyperbolic tangent function, and the recti-
fied linear unit (ReLU) function. The ReLU function, defined as
f (x) = max(0, x), is particularly popular due to its simplicity and
its ability to alleviate the vanishing gradient problem.

1 ReLU Activation Function Code

The following Python code snippet illustrates the ReLU activation
function:

import numpy as np

def relu(x):
return np.maximum(0, x)

In this chapter, we have explored the fundamental principles of

Convolutional Neural Networks, including the convolution opera-
tion, pooling layers, and activation functions. These mathematical
foundations are essential for understanding the inner workings of
CNNs and their remarkable capabilities in computer vision tasks.

125
Chapter 28

Recurrent Neural
Networks (RNNs)

Introduction
Recurrent Neural Networks (RNNs) are a powerful class of neural
networks that excel at processing sequential data, such as time
series or natural language. Unlike feedforward neural networks,
RNNs have internal memory that allows them to retain information
about past inputs. This memory enables the network to capture
dependencies and patterns in sequential data. In this chapter, we
will explore the mathematical foundations of RNNs and discuss
their architecture and training algorithms.

The Basic RNN Structure

The basic structure of an RNN consists of recurrent layers where
the hidden state at each time step is computed based on the current
input and the hidden state from the previous time step. Let xt ∈
Rd denote the input at time step t, ht ∈ Rh denote the hidden state
at time step t, and yt ∈ Rc denote the output at time step t with
d, h, c representing the input dimension, hidden state dimension,
and output dimension, respectively. The hidden state is calculated
as:

ht = f (Whx xt + Whh ht−1 + bh )

126
 where Whx ∈ Rh×d represents the weight matrix between the
input and hidden state, Whh ∈ Rh×h represents the weight matrix
between the hidden states, bh ∈ Rh represents the bias term, and
f (·) represents the activation function applied element-wise. The
output at each time step is obtained by:

yt = g(Wyh ht + by )
where Wyh ∈ Rc×h represents the weight matrix between the
hidden state and output, and by ∈ Rc represents the output bias
term. The function g(·) is typically the softmax function for mul-
ticlass classification problems.

Training RNNs using Backpropagation Through

Time (BPTT)
To train RNNs, we adapt the Backpropagation Through Time
(BPTT) algorithm, which is an extension of the backpropagation
algorithm for feedforward neural networks. BPTT unrolls the RNN
through time, creating a computational graph that allows us to
compute gradients by propagating errors backward in time.
The loss at each time step can be quantified using a suitable loss
function, such as the cross-entropy loss for classification problems.
The total loss is the sum of losses across all time steps:
T
X
L= Lt
t=1

where Lt represents the loss at time step t.

To update the model parameters, we compute the gradient of
the loss with respect to the parameters using backpropagation and
then apply an optimization algorithm, such as Stochastic Gradi-
ent Descent (SGD), to perform parameter updates. The gradients
are calculated by backpropagating through time, considering the
dependencies introduced by the recurrent connections.

Python Code: RNN Forward Pass

The following Python code snippet illustrates the forward pass of
an RNN:

127
import numpy as np

def rnn_forward(x, W_hx, W_hh, W_yh, b_h, b_y,

,→ activation):
T, d = x.shape
h = W_hx.shape[0]
c = W_yh.shape[0]
h_t = np.zeros((T, h))
y_hat = np.zeros((T, c))

for t in range(T):
if t == 0:
h_t[t] = activation(np.dot(W_hx, x[t]) +
,→ b_h)
else:
h_t[t] = activation(np.dot(W_hx, x[t]) +
,→ np.dot(W_hh, h_t[t-1]) + b_h)
y_hat[t] = np.dot(W_yh, h_t[t]) + b_y

return h_t, y_hat

In this chapter, we have explored the mathematical founda-

tions of Recurrent Neural Networks (RNNs) and discussed their
basic structure and training algorithm. RNNs are powerful mod-
els for sequential data processing, allowing us to model complex
dependencies over time.

128
Chapter 29

Generative Adversarial
Networks (GAN)

Introduction
Generative Adversarial Networks (GANs) have emerged as a pow-
erful framework for generative modeling, capable of learning to
generate synthetic data that closely resembles real data distribu-
tions. In this chapter, we will explore the mathematical founda-
tions of GANs and their training procedure, which involves a game-
theoretic approach between two neural networks: the generator and
the discriminator.

The GAN Framework

At the core of the GAN framework lies a minimax game between
the generator and the discriminator. The generator, denoted as G,
aims to generate realistic samples that resemble real data, while the
discriminator, denoted as D, tries to distinguish between real and
fake samples. The ultimate goal is to train G to generate samples
that are indistinguishable from real data by D.

1 Generator
The generator takes as input a random noise vector z sampled
from a known prior distribution, typically a Gaussian distribution,

129
and maps it to a high-dimensional data space to generate synthetic
samples. Mathematically, the generator can be represented by a
neural network with parameters θ G and is denoted as G(z; θ G ).

2 Discriminator
The discriminator is responsible for classifying samples as real or
fake. It takes either a real sample x or a generated sample G(z; θ G )
as input and outputs a probability D(x; θ D ), where θ D represents
the discriminator’s parameters.

3 Objective Function
The objective of the GAN framework is to find a Nash equilibrium
between the generator and the discriminator. This can be achieved
by solving the following minimax game:

min max V (D, G) = Ex∼preal (x) [log D(x; θ D )]+Ez∼pnoise (z) [log(1−D(G(z; θ G )); θ D )]
θG θD
(29.1)
where preal (x) denotes the true data distribution, and pnoise (z)
represents the prior noise distribution.

GAN Training Procedure

GANs are typically trained using an alternating optimization pro-
cedure. In each training iteration, the discriminator and generator
are updated in sequence.

1 Discriminator Updates
To update the discriminator, we sample a batch of real data {x1 , . . . , xm }
from the true data distribution and a batch of noise samples {z 1 , . . . , z m }
from the prior noise distribution. The discriminator’s parameters
θ D are updated by performing gradient ascent on the objective
function V (D, G):
m
1 X
∇θ D [log D(xi ; θ D ) + log(1 − D(G(z i ; θ G )); θ D )].
m i=1

130
2 Generator Updates
Once the discriminator is updated, we fix its parameters and up-
date the generator by performing gradient descent on the objective
function V (D, G):
m
1 X
∇θ G log(1 − D(G(z i ; θ G )); θ D ).
m i=1

Python Code: GAN Training Procedure

The following Python code snippet showcases the training proce-
dure for GANs:

import numpy as np

def train_gan(generator, discriminator, real_data,

,→ noise_data):
discriminator.train()
generator.train()

optimizer_D =
,→ torch.optim.Adam(discriminator.parameters(),
,→ lr=0.001)
optimizer_G =
,→ torch.optim.Adam(generator.parameters(),
,→ lr=0.001)

# Discriminator update
optimizer_D.zero_grad()
real_loss =
,→ torch.mean(torch.log(discriminator(real_data)))
fake_loss = torch.mean(torch.log(1 -
,→ discriminator(generator(noise_data))))
D_loss = -(real_loss + fake_loss)
D_loss.backward()
optimizer_D.step()

# Generator update
optimizer_G.zero_grad()
G_loss = torch.mean(torch.log(1 -
,→ discriminator(generator(noise_data))))

131
 G_loss.backward()
optimizer_G.step()

In this chapter, we have explored the mathematical foundations

of Generative Adversarial Networks (GANs). The minimax game
between the generator and discriminator forms the basis of GAN
training. Through iterative updates, GANs can learn to generate
synthetic data samples that resemble real data.

132
Chapter 30

Transfer Learning

Introduction
Transfer learning is a widely used technique in machine learning
that allows us to leverage knowledge gained from one task and
apply it to a different but related task. In this chapter, we will delve
into the mathematical foundations of transfer learning and explore
different strategies for transferring knowledge between tasks.

Problem Formulation
Let us consider two tasks: a source task and a target task. The
source task has a labeled dataset Dsource = {(xsource
i , yisource )}ni=1
source
,
where xi source
represents the input features and yi source
represents
the corresponding labels. Similarly, the target task has a labeled
ntarget
dataset Dtarget = {(xtarget
i , yitarget )}i=1 .
The goal of transfer learning is to improve the performance on
the target task by utilizing the knowledge gained from the source
task. This can be achieved by transferring a learned model or
representations from the source task to the target task.

Transfer Learning Strategies

There are several strategies for transfer learning, which can be
broadly categorized into three main approaches:

133
1 Feature-Based Transfer Learning
In feature-based transfer learning, we transfer knowledge by adapt-
ing the features learned from the source task to the target task.
This involves extracting relevant features from the dataset of the
source task and using them as input features for the target task.
One popular technique used in feature-based transfer learning
is fine-tuning. Fine-tuning involves taking a pre-trained model on
the source task and then updating its parameters using the target
task data. Mathematically, this can be represented as:

θ ∗source = arg min Lsource (θ source ),

θ source

θ ∗target = arg min Ltarget (θ target ),

θ target

θ ∗fine-tuned = arg min Ltarget (θ fine-tuned ),

θ fine-tuned

where Lsource and Ltarget are the loss functions for the source and
target tasks, respectively.

2 Model-Based Transfer Learning

In model-based transfer learning, we transfer the entire model
trained on the source task to the target task. This involves us-
ing the pre-trained model as a starting point for the target task
and fine-tuning the model using the target task data.
One common approach in model-based transfer learning is called
pre-training and fine-tuning. First, a model is pre-trained on a
large dataset from the source task. Then, the pre-trained model is
fine-tuned on the smaller dataset from the target task. Mathemat-
ically, this can be represented as:

θ ∗pre-trained = arg min Lsource (θ pre-trained ),

θ pre-trained

θ ∗fine-tuned = arg min Ltarget (θ pre-trained , θ fine-tuned ),

θ fine-tuned

where θ pre-trained represents the parameters of the pre-trained model,

and θ fine-tuned represents the parameters that are fine-tuned on the
target task data.

134
3 Instance-Based Transfer Learning
In instance-based transfer learning, we transfer knowledge by reusing
labeled instances from the source task to aid the learning process on
the target task. This involves using source task data as additional
training data for the target task.
One approach in instance-based transfer learning is called do-
main adaptation, which aims to align the source and target do-
mains to reduce the distribution discrepancy between them. This
can be achieved by minimizing a discrepancy metric, such as the
Maximum Mean Discrepancy (MMD), between the feature distri-
butions of the source and target domains.

Python Code: Fine-tuning

The following Python code snippet demonstrates the fine-tuning
process for transfer learning:

import torch
import torch.nn as nn
import torch.optim as optim

# Define the pre-trained model

pretrained_model =
,→ torchvision.models.resnet18(pretrained=True)

# Freeze the pre-trained layers

for param in pretrained_model.parameters():
param.requires_grad = False

# Replace the last fully connected layer

fc =
,→ nn.Linear(in_features=pretrained_model.fc.in_features,
out_features=num_classes)
pretrained_model.fc = fc

# Define the loss function and optimizer

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(pretrained_model.parameters(),
,→ lr=0.001, momentum=0.9)

# Training loop

135
for epoch in range(num_epochs):
running_loss = 0.0
for inputs, labels in dataloader:
# Zero the parameter gradients
optimizer.zero_grad()

# Forward pass
outputs = pretrained_model(inputs)

# Compute the loss

loss = criterion(outputs, labels)

# Backward pass
loss.backward()

# Update the parameters

optimizer.step()

# Accumulate the loss

running_loss += loss.item()

# Print the average loss for this epoch

print(f"Epoch {epoch+1}/{num_epochs}, Loss:
,→ {running_loss/len(dataloader):.4f}")

In this chapter, we have explored the mathematical foundations

of transfer learning and discussed different strategies for trans-
ferring knowledge between tasks. Feature-based transfer learning
adapts the features learned from the source task to the target task,
while model-based transfer learning transfers the entire pre-trained
model to the target task. Instance-based transfer learning leverages
labeled instances from the source task to aid the learning process
on the target task. Transfer learning is a valuable tool that can
significantly improve the performance of machine learning models
in various real-world scenarios.

136
Chapter 31

Hyperparameter
Tuning

Introduction
In machine learning, hyperparameters play a crucial role in de-
termining the performance of a model. The process of selecting
the optimal values for these hyperparameters is known as hyper-
parameter tuning. In this chapter, we will explore different tech-
niques for hyperparameter tuning and discuss their mathematical
foundations.

Problem Formulation
Let X denote the input features matrix with dimensions m × n,
where m is the number of samples and n is the number of features.
Let y denote the corresponding target vector with dimensions m ×
1.
A machine learning model M with hyperparameters θ takes X
as input and outputs a prediction vector ŷ. The goal of hyperpa-
rameter tuning is to find the optimal values for θ that minimize a
predefined loss function L(y, ŷ).

137
Grid Search
Grid search is a commonly used technique for hyperparameter tun-
ing. It involves defining a grid of hyperparameter values and ex-
haustively evaluating the model’s performance for each combina-
tion of these values. The optimal set of hyperparameters is selected
based on the performance metric, such as accuracy or mean squared
error.
Mathematically, let Θ denote the grid of hyperparameter values
with dimensions p × q, where p is the number of hyperparameters
and q is the number of candidate values for each hyperparame-
ter. The optimal hyperparameters θ ∗ are selected by solving the
following optimization problem:

θ ∗ = arg min L(y, ŷ).

θ∈Θ

Random Search
Random search is an alternative approach to hyperparameter tun-
ing that addresses some of the limitations of grid search. Instead of
evaluating all possible combinations of hyperparameter values, ran-
dom search randomly samples from the predefined hyperparameter
space. The number of samples is determined in advance.
Mathematically, let N denote the number of random samples.
The optimal hyperparameters θ ∗ are selected by solving the fol-
lowing optimization problem:
N
1 X
θ ∗ = arg min L(y, ŷ).
θ N i=1

Bayesian Optimization
Bayesian optimization is a sequential model-based optimization
technique that leverages previous observations to select the next
set of hyperparameters to evaluate. It builds a probabilistic model,
such as a Gaussian Process (GP), to model the performance of the
model as a function of its hyperparameters. The GP model is up-
dated as new observations are made.
The acquisition function guides the selection of the next hyper-
parameters to evaluate based on the current GP model. Commonly

138
used acquisition functions include Upper Confidence Bound (UCB)
and Expected Improvement (EI).
Mathematically, let D denote the set of observed hyperparam-
eters and their corresponding performance values. The optimal
hyperparameters θ ∗ are selected by solving the following optimiza-
tion problem:

θ ∗ = arg max acquisition_function(θ|D).

θ

Python Code: Grid Search

The following Python code snippet demonstrates the implementa-
tion of grid search for hyperparameter tuning:

from sklearn.model_selection import GridSearchCV

from sklearn.svm import SVC

# Define the hyperparameters and their candidate

,→ values
param_grid = {'C': [0.1, 1, 10], 'gamma': [0.001,
,→ 0.01, 0.1]}

# Perform grid search with cross-validation

grid_search = GridSearchCV(SVC(), param_grid, cv=5)

# Fit the model with the training data

grid_search.fit(X_train, y_train)

# Print the best hyperparameters

print("Best Hyperparameters: ",
,→ grid_search.best_params_)

In this chapter, we have explored different techniques for hy-

perparameter tuning, including grid search, random search, and
Bayesian optimization. These techniques play a vital role in fine-
tuning machine learning models to achieve optimal performance.

139
Chapter 32

Cross-Validation
Techniques

Introduction
In machine learning, it is crucial to evaluate the performance of a
model on unseen data to assess its generalization ability. However,
simply training and testing a model on a single dataset may lead
to overfitting or biased performance estimates. Cross-validation
techniques provide a solution to this problem by partitioning the
available data into multiple subsets and repeatedly evaluating the
model on different combinations of these subsets. This chapter
focuses on discussing various cross-validation techniques and their
mathematical foundations.

k-Fold Cross-Validation
k-fold cross-validation is one of the most widely used cross-validation
techniques. It involves splitting the dataset into k equally sized
folds or subsets. The model is then trained k times, where each
time it uses k − 1 folds for training and the remaining fold for test-
ing. The performance metric of interest is computed as the average
across the k test folds.
Mathematically, given a dataset D with m samples, k-fold cross-
validation partitions D into k folds D1 , D2 , . . . , Dk . For each fold
Di , the model M is trained on the remaining k − 1 folds and eval-
uated on Di . The performance metric P is then computed as:

140
 k
1X
P= L(MDi ),
k i=1
where L is a predefined loss or scoring function.

Leave-One-Out Cross-Validation
Leave-One-Out (LOO) cross-validation is a special case of k-fold
cross-validation where k = m, i.e., each fold contains only one
sample. In this technique, the model is trained m times, leaving
out one sample for testing at each iteration. The performance
metric is then computed as the average across all iterations.
Mathematically, given a dataset D with m samples, LOO cross-
validation trains the model M on all but one sample, and evaluates
its performance on the left-out sample for each iteration i. The
performance metric P is computed as:
m
1 X
P= L(MD\i ),
m i=1
where D\i denotes the dataset with the ith sample removed.

Stratified Cross-Validation
Stratified cross-validation is particularly useful when dealing with
imbalanced datasets, where the distribution of classes is uneven. It
ensures that the class distribution in each fold remains consistent
with the original dataset, reducing the risk of biased performance
estimation.
Mathematically, given a dataset D with m samples and c classes,
stratified cross-validation partitions D into k folds D1 , D2 , . . . , Dk .
The class proportions in each fold are maintained approximately
equal to the original dataset D. The performance metric P is then
computed as:
k
1X
P= L(MDi ),
k i=1
where L is a predefined loss or scoring function.

141
Python Code: k-fold Cross-Validation
The following Python code snippet demonstrates the implementa-
tion of k-fold cross-validation using scikit-learn:

from sklearn.model_selection import cross_val_score

from sklearn.linear_model import LogisticRegression

# Define the model

model = LogisticRegression()

# Perform 5-fold cross-validation

scores = cross_val_score(model, X, y, cv=5)

# Print the average accuracy

print("Average Accuracy: ", scores.mean())

In this chapter, we have discussed different cross-validation

techniques, including k-fold cross-validation, LOO cross-validation,
and stratified cross-validation. These techniques are fundamental
in properly assessing the performance of machine learning models
and mitigating the risk of overfitting or biased estimates.

142
Chapter 33

Regularization
Techniques

Introduction
In the field of machine learning, regularization techniques play a
vital role in preventing overfitting and improving the generalization
performance of models. Overfitting occurs when a model becomes
too complex and starts to fit the noise in the training data, re-
sulting in poor performance on new, unseen data. Regularization
methods aim to address this issue by adding a penalty term to the
model’s objective function, discouraging overly complex solutions
and promoting simpler ones.
This chapter focuses on discussing various regularization tech-
niques employed in machine learning, including L1 and L2 regular-
ization, dropout, and batch normalization. These techniques aid in
optimizing model performance and mitigating the risk of overfitting
on training data.

L1 (Lasso) and L2 (Ridge) Regularization

L1 and L2 regularization are two widely used techniques that mod-
ify the cost or loss function of a machine learning model by adding
a penalty term. Both these techniques are commonly applied to
linear and logistic regression models.

143
1 L1 Regularization (Lasso)
L1 regularization, also known as Lasso regularization, adds the
sum of the absolute values of the model’s coefficients multiplied
by a regularization parameter, λ, to the objective function. The
objective function with L1 regularization is given as:
n
X
Objective function = Loss function + λ |θi |,
i=1

where θi represents the ith coefficient of the model and λ con-

trols the strength of the regularization.
L1 regularization acts as a feature selector by encouraging spar-
sity in the model, i.e., driving some coefficients to zero and elimi-
nating the corresponding features from the model.

2 L2 Regularization (Ridge)
L2 regularization, also known as Ridge regularization, adds the
sum of the squared values of the model’s coefficients multiplied
by a regularization parameter, λ, to the objective function. The
objective function with L2 regularization is given as:
n
X
Objective function = Loss function + λ θi2 .
i=1

Similar to L1 regularization, θi represents the ith coefficient of

the model, and λ controls the strength of the regularization.
L2 regularization shrinks the coefficients towards zero, but does
not drive them to exactly zero. This encourages co-adaptation of
features within the model, leading to better generalization perfor-
mance.

Dropout
Dropout is a regularization technique that combats overfitting by
randomly dropping out (setting to zero) a fraction of the input units
or nodes during training. This prevents the model from relying too
heavily on individual nodes and encourages the network to learn
more robust and generalized features.
Mathematically, dropout can be represented as follows:

Output = Input × Mask,

144
 where the mask is a binary vector with the same dimension as
the input, and each element is set to 0 or 1 with a specified dropout
probability.

1 Python Code: Dropout

The following Python code snippet demonstrates the implementa-
tion of dropout in a neural network using the Keras library:

from keras.models import Sequential

from keras.layers import Dense, Dropout

# Create the model

model = Sequential()
model.add(Dense(64, activation='relu',
,→ input_dim=100))
model.add(Dropout(0.5))
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(10, activation='softmax'))

# Compile and fit the model

model.compile(optimizer='adam',
,→ loss='categorical_crossentropy',
,→ metrics=['accuracy'])
model.fit(X_train, y_train, epochs=10, batch_size=32)

In this code snippet, dropout layers are added after the dense
layers in a neural network model. The dropout rate is set to 0.5,
indicating that half of the input units will be dropped out during
training.

Batch Normalization
Batch normalization is a technique used to normalize the activa-
tions of a neural network layer, making the optimization process
more stable. It involves normalizing the inputs of each layer by
subtracting the mean and dividing by the standard deviation of
the mini-batch.
Mathematically, batch normalization can be expressed as fol-
lows:

145
 Input − µ
 
Output = γ √ + β,
σ2 + ϵ
where γ and β are learnable parameters, µ is the mean of the
mini-batch, σ is the standard deviation of the mini-batch, and ϵ is
a small constant added to the denominator for numerical stability.
Batch normalization helps in addressing vanishing and explod-
ing gradient problems and improves the overall training speed and
performance of the neural network.

1 Python Code: Batch Normalization

The following Python code snippet demonstrates the use of batch
normalization in a neural network using the Keras library:

from keras.models import Sequential

from keras.layers import Dense, Activation,
,→ BatchNormalization

# Create the model

model = Sequential()
model.add(Dense(64, input_shape=(10,)))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(Dense(1))
model.add(Activation('sigmoid'))

# Compile and fit the model

model.compile(optimizer='adam',
,→ loss='binary_crossentropy')
model.fit(X_train, y_train, epochs=10, batch_size=32)

In this code snippet, batch normalization layers are added after

the dense layers in a neural network model. Batch normalization
helps in improving the stability and convergence of the model dur-
ing training.
This chapter has covered several regularization techniques, such
as L1 and L2 regularization, dropout, and batch normalization.
These techniques play a crucial role in mitigating overfitting and
improving the generalization performance of machine learning mod-
els. By incorporating these techniques into the model’s objective

146
function or architecture, researchers and practitioners can enhance
the robustness and reliability of their models.

147
Chapter 34

Dimensionality
Reduction Techniques

Introduction
Dimensionality reduction is a fundamental technique used in ma-
chine learning to reduce the number of features or variables while
preserving the essential information and structure of the data.
In this chapter, we explore three popular dimensionality reduc-
tion techniques: t-SNE (t-Distributed Stochastic Neighbor Embed-
ding), UMAP (Uniform Manifold Approximation and Projection),
and ICA (Independent Component Analysis).

t-SNE
t-SNE is a nonlinear dimensionality reduction technique commonly
used for visualizing high-dimensional data. It aims to map the
original data points to a lower-dimensional space while preserving
the pairwise similarities between data points. The t-SNE algorithm
constructs a probability distribution over pairs of high-dimensional
objects such that similar objects have a higher probability of being
chosen. It also constructs a probability distribution over pairs of
low-dimensional points, attempting to match the pairwise similari-
ties from the high-dimensional space. The objective is to minimize
the Kullback-Leibler divergence between these two distributions.
The t-SNE objective function can be expressed as follows:

148
 N
X
C= KL(Pi ||Qi ),
i=1
where N is the number of data points, Pi is the probability dis-
tribution over pairwise similarities in the high-dimensional space,
and Qi is the probability distribution over pairwise similarities in
the low-dimensional space.

1 Python Code: t-SNE

The following Python code snippet demonstrates the usage of t-
SNE for dimensionality reduction and visualization using the scikit-
learn library:

from sklearn.manifold import TSNE

# t-SNE
tsne = TSNE(n_components=2, random_state=42)
X_tsne = tsne.fit_transform(X)

In this code snippet, the scikit-learn library’s TSNE class is uti-

lized to perform t-SNE dimensionality reduction. The parame-
ter n_components specifies the number of dimensions in the low-
dimensional space.

UMAP
UMAP is a dimensionality reduction technique that aims to pre-
serve the local and global structure of the data. Unlike t-SNE,
UMAP uses a different optimization objective based on fuzzy sim-
plicial sets and is known for its scalability to large datasets. UMAP
constructs a high-dimensional graph representation of the data,
capturing complex relationships between data points. It then op-
timizes low-dimensional embeddings to match the graph structure,
utilizing a cross-entropy loss function.
The UMAP optimization objective function can be expressed
as follows:

 
N X 1

X w ij − w ij
X
C= α wij log + (1 − wij ) log + (1 − α) d2ij  ,
i=1
q ij 1 − qij
j∈Ni j∈Ni

149
 where N is the number of data points, Ni represents the neigh-
borhood of data point i, wij is the weight of the directed edge
from i to j, qij denotes the membership probability of j being in
the neighborhood of i, dij represents the distance between i and
j, and α is a trade-off parameter that balances the importance of
preserving the graph structure and the distribution of distances.

1 Python Code: UMAP

The following Python code snippet demonstrates the usage of UMAP
for dimensionality reduction and visualization using the umap-
learn library:

import umap

# UMAP
umap_model = umap.UMAP(n_components=2,
,→ random_state=42)
umap_embedding = umap_model.fit_transform(X)

In this code snippet, the umap-learn library’s UMAP class is uti-

lized to perform UMAP dimensionality reduction. The parame-
ter n_components specifies the number of dimensions in the low-
dimensional space.

ICA
ICA is a dimensionality reduction technique based on the statistical
method of blind source separation. It aims to recover the original
independent sources of the observed data and is particularly useful
when the sources are statistically independent and non-Gaussian.
ICA assumes that the observed data is a linear mixture of the
sources, and it aims to estimate a linear transformation matrix to
recover the sources.
The ICA model can be expressed as follows:

X = AS,
where X represents the observed data, A is the mixing matrix,
and S denotes the independent sources.
ICA aims to find an unmixing matrix W such that the esti-
mated sources Ŝ can be obtained as:

150
 Ŝ = WX.

1 Python Code: ICA

The following Python code snippet demonstrates the usage of ICA
for dimensionality reduction using the scikit-learn library:

from sklearn.decomposition import FastICA

# ICA
ica = FastICA(n_components=2, random_state=42)
X_ica = ica.fit_transform(X)

In this code snippet, the scikit-learn library’s FastICA class is

utilized to perform ICA dimensionality reduction. The parame-
ter n_components specifies the number of independent sources to
estimate.
In this chapter, we explored three dimensionality reduction
techniques: t-SNE, UMAP, and ICA. These techniques enable us to
represent high-dimensional data in a lower-dimensional space with-
out losing critical information. By reducing the dimensionality of
the data, we can simplify the learning process, visualize complex
datasets, and enhance the performance of machine learning models.

151
Chapter 35

Markov Chain Monte

Carlo (MCMC)

Introduction
Markov Chain Monte Carlo (MCMC) methods are widely used in
statistical inference and Bayesian analysis. These methods allow us
to efficiently sample from a target distribution, even when the dis-
tribution is complex and its exact form is unknown. In this chapter,
we explore MCMC methods, specifically the Metropolis-Hastings
algorithm and Gibbs sampling. We also discuss the applications of
MCMC in Bayesian inference.

Metropolis-Hastings Algorithm
The Metropolis-Hastings algorithm is a general-purpose MCMC
algorithm used to sample from a target probability distribution
p(x). Given an initial state x0 , the algorithm iteratively generates
a sequence of states x1 , x2 , . . . according to a Markov chain.
At each iteration, a proposal state x∗ is sampled from a proposal
distribution q(x∗ |xcurrent ), where xcurrent is the current state of the
Markov chain. The proposal distribution defines the probability of
moving from the current state to a new proposed state.
The acceptance probability α is then calculated as follows:

p(x∗ ) · q(xcurrent |x∗ )

 
α = min 1, .
p(xcurrent ) · q(x∗ |xcurrent )

152
 The proposed state x∗ is accepted with probability α. If the
proposed state is accepted, xcurrent is updated to x∗ . Otherwise,
xcurrent remains unchanged. This ensures that the Markov chain
converges to the target distribution p(x) in the long run.

1 Python Code: Metropolis-Hastings Algorithm

The following Python code snippet demonstrates the implemen-
tation of the Metropolis-Hastings algorithm for sampling from a
target probability distribution:

import numpy as np

def metropolis_hastings(target_distribution,
,→ proposal_distribution, num_iterations):
samples = []
current_state = initial_state

for i in range(num_iterations):
proposed_state =
,→ proposal_distribution(current_state)
acceptance_prob = min(1,

,→ (target_distribution(proposed_state
,→ *

,→ proposal_distribution(current_stat
,→ proposed_state)) /

,→ (target_distribution(current_state)
,→ *

proposal_distribution(proposed_sta
,→

current_state)))
,→

if np.random.rand() < acceptance_prob:

samples.append(proposed_state)
current_state = proposed_state
else:
samples.append(current_state)

return samples

153
 In this code snippet, the metropolis_hastings function im-
plements the Metropolis-Hastings algorithm. The target distribu-
tion is specified by the target_distribution function, and the
proposal distribution is specified by the proposal_distribution
function.

Gibbs Sampling
Gibbs sampling is another MCMC algorithm that is particularly
useful for sampling from high-dimensional distributions. Gibbs
sampling samples from the conditional distributions of each vari-
able given the current values of the other variables.
Given a joint distribution p(x) = p(x1 , x2 , . . . , xn ), the Gibbs
sampling algorithm iteratively updates the variables as follows:
(t+1) (t)
x1 n ),
∼ p(x1 |x2 , . . . , x(t)
(t+1) (t+1) (t)
x2 ∼ p(x2 |x1 n ),
, x3 , . . . , x(t)
...
(t+1) (t+1) (t+1)
x(t+1)
n ∼ p(xn |x1 , x2 , . . . , xn−1 ).
In each iteration, a single variable is updated while holding the
remaining variables fixed. The process is repeated until conver-
gence to the joint distribution is achieved.

1 Python Code: Gibbs Sampling

The following Python code snippet demonstrates the implemen-
tation of the Gibbs sampling algorithm for sampling from a joint
distribution:

def gibbs_sampling(joint_distribution,
,→ num_iterations):
samples = []
current_state = initial_state

for i in range(num_iterations):
for j, variable in enumerate(current_state):
current_state[j] =
,→ joint_distribution(variable,
,→ current_state[:j],
,→ current_state[j+1:])

154
 samples.append(current_state)

return samples

In this code snippet, the gibbs_sampling function implements

the Gibbs sampling algorithm. The joint distribution is specified
by the joint_distribution function.

Applications in Bayesian Inference

MCMC methods, such as the Metropolis-Hastings algorithm and
Gibbs sampling, are commonly used in Bayesian inference. Bayesian
inference allows us to estimate the posterior distribution of model
parameters given observed data. However, in many cases, the pos-
terior distribution cannot be analytically calculated and must be
approximated using MCMC methods.
By sampling from the posterior distribution, MCMC methods
enable us to make inferences about the values of model parameters.
These samples can be used to estimate posterior means, variances,
quantiles, and other summary statistics.
MCMC methods also allow us to perform model comparison
by calculating the marginal likelihood or the Bayes factor. By
comparing the posterior probabilities of different models, we can
select the most suitable model for the observed data.
Furthermore, MCMC methods are crucial for conducting Bayesian
hierarchical modeling, where model parameters are assumed to fol-
low certain prior distributions. The joint distribution of the param-
eters and hyperparameters can be explored using MCMC methods.
In summary, MCMC methods provide a powerful tool for Bayesian
inference, enabling us to estimate posterior distributions, make in-
ferences about model parameters, conduct model comparison, and
perform hierarchical modeling.

155
Chapter 36

Hidden Markov Models

(HMM)

Introduction
Hidden Markov Models (HMMs) are probabilistic models that are
widely used for modeling sequential data. HMMs are particularly
useful when the underlying process generating the data is assumed
to be a Markov process, where the future state depends only on
the current state. In this chapter, we discuss the fundamentals
of HMMs, including the forward-backward algorithm, the Viterbi
algorithm, and their applications in time-series data.

Hidden Markov Model

A Hidden Markov Model consists of a set of hidden states S =
{s1 , s2 , . . . , sN }, a set of observed symbols V = {v1 , v2 , . . . , vM },
and a set of transition probabilities A = {aij } representing the
probabilities of transitioning between hidden states. Additionally,
HMMs include a set of emission probabilities B = {bi (v)}, where
bi (v) represents the probability of emitting symbol v when in hid-
den state si .

1 HMM Notation
Let N be the number of hidden states, and M be the number of
observed symbols. The transition probability matrix A is defined

156
as:
 
a11 a12 ... a1N
 a21 a22 ... a2N 
A= . .. .. ..  .
 
 .. . . . 
aN 1 aN 2 ... aN N
The emission probability matrix B is defined as:

b1 (v1 ) b1 (v2 ) . . . b1 (vM )

 
 b2 (v1 ) b2 (v2 ) . . . b2 (vM ) 
B= . .. .. .. .
 
 .. . . . 
bN (v1 ) bN (v2 ) . . . bN (vM )
The initial state distribution π is a vector of length N repre-
senting the probabilities of starting in each hidden state:

π = [π1 , π2 , . . . , πN ].

2 HMM Probabilities
Given an HMM model and a sequence of observed symbols O =
(O1 , O2 , . . . , OT ), where T is the length of the sequence, there are
three fundamental probabilities of interest:
1. The probability of the observed sequence O given the model
λ = (A, B, π), denoted as P (O|λ).
2. The probability of being in a particular hidden state si at
time t, denoted as P (Xt = si |O, λ).
3. The probability of being in state si at time t, and state sj
at time t + 1, denoted as P (Xt = si , Xt+1 = sj |O, λ).
These probabilities can be computed efficiently using the forward-
backward algorithm and the Viterbi algorithm.

Forward-Backward Algorithm
The forward-backward algorithm, also called the Baum-Welch algo-
rithm, is used to compute the probability of the observed sequence
P (O|λ) and to estimate the model parameters A, B, and π given
the observed sequence.
The forward algorithm calculates the forward variable αt (i),
which represents the probability of being in state si at time t and

157
generating the observed sequence up to time t. It is computed as
follows:

α1 (i) = πi · bi (O1 ),

N
!
X
αt (j) = αt−1 (i) · aij · bj (Ot ), 1 < t ≤ T.
i=1

The backward algorithm calculates the backward variable βt (i),

which represents the probability of generating the observed se-
quence from time t + 1 to the end of the sequence, given being
in state si at time t. It is computed as follows:

βT (i) = 1,

N
X
βt (i) = aij · bj (Ot+1 ) · βt+1 (j), 1 ≤ t < T.
j=1

Using the forward and backward variables, we can estimate the

model parameters as follows:
PT −1
αt (i) · aij · bj (Ot+1 ) · βt+1 (j)
âij = PT −1t=1 PN ,
t=1 i=1 αt (i) · aij · bj (Ot+1 ) · βt+1 (j)
PT
αt (j) · βt (j)
b̂j (vk ) = PT t=1 PN ,
t=1 j=1 αt (j) · βt (j)

α1 (i) · β1 (i)
π̂i = PN .
i=1 α1 (i) · β1 (i)

1 Python Code: Forward-Backward Algorithm

The following Python code snippet demonstrates the implementa-
tion of the forward-backward algorithm for estimating the model
parameters of an HMM:

def forward_backward(observed_sequence, states,

,→ transition_probs, emission_probs,
,→ initial_state_probs):
T = len(observed_sequence)
N = len(states)

158
# Calculate forward variables
forward_variables = np.zeros((T, N))
forward_variables[0] = initial_state_probs *
,→ emission_probs[:, observed_sequence[0]]

for t in range(1, T):

for j in range(N):
forward_variables[t, j] =
,→ np.sum(forward_variables[t-1] *
,→ transition_probs[:, j]) *
,→ emission_probs[j,
,→ observed_sequence[t]]

# Calculate backward variables

backward_variables = np.zeros((T, N))
backward_variables[T-1] = 1.0

for t in range(T-2, -1, -1):

for i in range(N):
backward_variables[t, i] =
,→ np.sum(transition_probs[i] *
,→ emission_probs[:,
,→ observed_sequence[t+1]] *
,→ backward_variables[t+1])

# Estimate model parameters

estimated_transition_probs = np.zeros((N, N))
estimated_emission_probs = np.zeros((N,
,→ len(set(observed_sequence))))
estimated_initial_state_probs =
,→ forward_variables[0] * backward_variables[0]

for i in range(N):
for j in range(N):
estimated_transition_probs[i, j] =
,→ np.sum(forward_variables[:-1, i] *

,→ transition_p
,→ j]
,→ *

159
 ,→ emission_pro
,→ observed_seq
,→ *

,→ backward_var
,→ j])
,→ /
,→ np.sum(forwa
,→ *
,→ backward_var

for j in range(N):
for k in set(observed_sequence):
estimated_emission_probs[j, k] =
,→ np.sum(forward_variables[:, j] *

,→ backward_vari
,→ j]
,→ *

,→ (observed_seq
,→ ==
,→ k))
,→ /
,→ np.sum(forwar
,→ *
,→ backward_vari

return estimated_transition_probs,
,→ estimated_emission_probs,
,→ estimated_initial_state_probs

In this code snippet, the forward_backward function imple-

ments the forward-backward algorithm. The observed sequence is
provided as observed_sequence, and the model parameters are
specified by states, transition_probs, emission_probs, and
initial_state_probs.

160
Viterbi Algorithm
The Viterbi algorithm is used to find the most likely sequence of
hidden states given the observed sequence in an HMM. This se-
quence is known as the Viterbi path.
The Viterbi algorithm calculates the Viterbi variable δt (i), which
represents the probability of the most likely path ending in state si
and generating the observed sequence up to time t. It is computed
as follows:

δ1 (i) = πi · bi (O1 ),

N
δt (j) = max(δt−1 (i) · aij ) · bj (Ot ), 1 < t ≤ T.
i=1

The most likely path can be backtracked from the final state as
follows:

x∗T = arg max δT (i),

i

x∗t = ψt+1 (x∗t+1 ) (for t = T − 1, T − 2, . . . , 1),

where ψt (i) is the backtracking variable that keeps track of the
most likely previous state.

1 Python Code: Viterbi Algorithm

The following Python code snippet demonstrates the implementa-
tion of the Viterbi algorithm for finding the most likely sequence
of hidden states in an HMM:

def viterbi(observed_sequence, states,

,→ transition_probs, emission_probs,
,→ initial_state_probs):
T = len(observed_sequence)
N = len(states)

# Calculate Viterbi variables and backtracking

,→ variables
viterbi_variables = np.zeros((T, N))
backtracking_variables = np.zeros((T, N),
,→ dtype=int)

161
 viterbi_variables[0] = initial_state_probs *
,→ emission_probs[:, observed_sequence[0]]

for t in range(1, T):

for j in range(N):
viterbi_variables[t, j] =
,→ np.max(viterbi_variables[t-1] *
,→ transition_probs[:, j]) *
,→ emission_probs[j,
,→ observed_sequence[t]]
backtracking_variables[t, j] =
,→ np.argmax(viterbi_variables[t-1] *
,→ transition_probs[:, j])

# Backtrack to find the most likely path

viterbi_path = [np.argmax(viterbi_variables[-1])]
for t in range(T-2, -1, -1):
viterbi_path.insert(0,
,→ backtracking_variables[t+1,
,→ viterbi_path[0]])

return viterbi_path

In this code snippet, the viterbi function implements the Viterbi

algorithm. The observed sequence is provided as observed_sequence,
and the model parameters are specified by states, transition_probs,
emission_probs, and initial_state_probs.

Conclusion
In this chapter, we discussed the fundamentals of Hidden Markov
Models (HMMs), including the model notation, essential probabil-
ities, and the application of the forward-backward algorithm and
the Viterbi algorithm. HMMs are powerful tools for modeling se-
quential data and have various applications in time-series analysis,
speech recognition, natural language processing, and bioinformat-
ics.

162
Chapter 37

Time Series Analysis

ARIMA Models
In this chapter, we focus on one of the widely used models for time
series analysis, namely Autoregressive Integrated Moving Average
(ARIMA) models. ARIMA models are capable of capturing the
temporal dependencies and trends present in time series data. We
will discuss the components of ARIMA models and the process of
model identification, estimation, and forecasting.

1 Autoregressive Model
Let us start by considering the autoregressive (AR) model of order
p, denoted as AR(p). In an AR(p) model, each observation in a
time series is expressed as a linear combination of its p previous
observations, weighted by certain coefficients. The general form of
an AR(p) model is given by the equation:

Xt = c + ϕ1 Xt−1 + ϕ2 Xt−2 + . . . + ϕp Xt−p + εt ,

where Xt is the value of the time series at time t, c is a constant
term, ϕ1 , ϕ2 , . . . , ϕp are the autoregressive coefficients, and εt is the
white noise error term at time t.
Python code snippet for estimating AR(p) model parameters
using ordinary least squares (OLS):

import numpy as np
from statsmodels.regression.linear_model import OLS

163
def estimate_ar_parameters(X, p):
X_lagged = np.column_stack([np.roll(X, i) for i
,→ in range(p)])
X_lagged = X_lagged[p:]
X = X[p:]

model = OLS(X, X_lagged)

results = model.fit()

ar_parameters = results.params
return ar_parameters

2 Moving Average Model

Next, we consider the moving average (MA) model of order q, de-
noted as MA(q). In an MA(q) model, each observation in a time
series is expressed as a linear combination of q previous error terms,
weighted by certain coefficients. The general form of an MA(q)
model is given by the equation:

Xt = µ + εt + θ1 εt−1 + θ2 εt−2 + . . . + θq εt−q ,

where Xt is the value of the time series at time t, µ is the
mean of the series, εt is the white noise error term at time t, and
θ1 , θ2 , . . . , θq are the moving average coefficients.
Python code snippet for estimating MA(q) model parameters
using ordinary least squares (OLS):

from statsmodels.tsa.arima.model import ARIMA

def estimate_ma_parameters(X, q):

model = ARIMA(X, order=(0, 0, q))
results = model.fit()

ma_parameters = results.params
return ma_parameters

3 ARIMA Model
Finally, we introduce the integrated component of ARIMA models.
The integrated component takes into account the differencing of the

164
time series to achieve stationarity. Differencing refers to the com-
putation of differences between consecutive observations in order
to eliminate trends or seasonal patterns. The differenced series can
be modeled using an ARMA model, combining autoregressive and
moving average components. The general form of an ARIMA(p, d,
q) model is given by the equation:

∆d Xt = c+ϕ1 ∆d Xt−1 +. . .+ϕp ∆d Xt−p +εt +θ1 εt−1 +. . .+θq εt−q ,

where ∆d represents the differencing operator applied d times.

Python code snippet for estimating ARIMA(p, d, q) model pa-
rameters using maximum likelihood estimation:

def estimate_arima_parameters(X, p, d, q):

model = ARIMA(X, order=(p, d, q))
results = model.fit(method_kwargs={'solver':
,→ 'newton'})

arima_parameters = results.params
return arima_parameters

In the above code snippets, X represents the time series data,

and p, d, and q are the order parameters of the ARIMA model.
ARIMA models provide a flexible framework for time series
analysis, allowing us to capture the temporal dependencies and
trends inherent in the data. By properly identifying and estimating
the model parameters, we can make accurate forecasts and gain
insights into the underlying dynamics of the time series.

165
Chapter 38

Text Mining and NLP

Introduction
Text mining and Natural Language Processing (NLP) are inter-
disciplinary fields that focus on extracting meaningful information
and insights from text data. With the exponential growth of tex-
tual information available on the internet and in various domains,
the need for automated text analysis techniques has become cru-
cial. In this chapter, we will explore the fundamental concepts and
techniques used in text mining and NLP, along with their applica-
tions.

1 Text Representation
Before delving into text mining techniques, it is essential to under-
stand how text data is represented to make it suitable for analy-
sis. In NLP, text is typically represented as a sequence of discrete
symbols, such as words, characters, or subword units. The most
common representation is the Bag-of-Words (BoW) model.
The Bag-of-Words model represents a text document as a col-
lection or "bag" of words, disregarding their order and grammar.
Each document is transformed into a fixed-length vector, where the
dimensionality is equal to the vocabulary size. The value in each
dimension represents the frequency or occurrence of a particular
word in the document. The BoW model is simplistic but effective
in capturing the overall content and context of a text document.
Another popular representation is the Term Frequency-Inverse
Document Frequency (TF-IDF). It takes into account not only the

166
occurrence of words in a document but also their importance in
the entire corpus. The TF-IDF score is calculated by multiplying
the term frequency (TF), which represents the frequency of a word
in a document, by the inverse document frequency (IDF), which
measures the importance of a word across the entire corpus.
Python code snippet for calculating TF-IDF scores:

from sklearn.feature_extraction.text import

,→ TfidfVectorizer

corpus = ['This is the first document.',

'This document is the second document.',
'And this is the third one.',
'Is this the first document?']

vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(corpus)

2 Text Preprocessing
Text preprocessing is a crucial step in text mining and NLP. It
involves transforming raw text data into a clean and standardized
format suitable for analysis. Common preprocessing steps include:

• Tokenization: Splitting a text document into individual words

or tokens.
• Stopword Removal: Removing common words that do not
carry significant meaning, such as "the," "is," and "and."
• Stemming and Lemmatization: Reducing words to their base
or root form, such as "running" to "run."
• Removing Special Characters: Eliminating punctuation, sym-
bols, and special characters.
• Lowercasing: Converting all text to lowercase to ensure con-
sistency.

These preprocessing steps help in reducing noise, standardizing

text representations, and improving the quality of analysis.
Python code snippet for text preprocessing using the NLTK
library:

167
import nltk
from nltk.corpus import stopwords
from nltk.stem import PorterStemmer
from nltk.tokenize import word_tokenize

nltk.download('stopwords')
nltk.download('punkt')

stop_words = set(stopwords.words('english'))
ps = PorterStemmer()

def preprocess_text(text):
tokens = word_tokenize(text)
tokens = [token.lower() for token in tokens if
,→ token.isalpha()]
tokens = [token for token in tokens if token not
,→ in stop_words]
tokens = [ps.stem(token) for token in tokens]
return tokens

text = "This is an example sentence for text

,→ preprocessing."
preprocessed_text = preprocess_text(text)

3 Text Mining Techniques

Text mining techniques aim to extract meaningful information and
patterns from text data. Some common techniques used in text
mining and NLP include:

• Sentiment Analysis: Analyzing the emotional tone of a text,

often used in social media monitoring and customer feedback
analysis.
• Topic Modeling: Identifying latent topics or themes present
in a collection of documents, commonly used for document
clustering and recommendation systems.
• Named Entity Recognition: Identifying and classifying named
entities, such as person names, locations, and organizations
mentioned in a text.

168
 • Text Classification: Categorizing text documents into prede-
fined classes or categories, frequently used in spam detection
and sentiment analysis.
• Text Summarization: Automatically generating concise sum-
maries of longer text documents.

These techniques provide powerful tools for extracting valuable

insights from text data and enable various applications in fields like
marketing, finance, healthcare, and social sciences.
In the upcoming sections, we will explore these techniques in
more detail, along with their mathematical formulations and prac-
tical examples.

169
Chapter 39

Sequence Modeling

Introduction
Sequence modeling is a fundamental concept in machine learning
that focuses on modeling and predicting sequences of data. Se-
quences can be found in various domains such as natural language
processing, speech recognition, and genomics, where the order of
elements plays a crucial role. In this chapter, we will delve into the
mathematics behind sequence modeling and explore various models
used in this field.

1 Hidden State Representation

One of the key components in sequence modeling is the hidden state
representation. A hidden state serves as an internal memory of the
model that captures the information from the previous elements
in the sequence. It summarizes the relevant information needed to
make predictions for the current element in the sequence.
Mathematically, for a sequence x = {x1 , x2 , . . . , xT }, the hidden
state representation ht at time step t is computed based on the
previous hidden state ht−1 and the current input xt as follows:

ht = f (ht−1 , xt )
Here, f (·) denotes the mapping function that captures the de-
pendency between the hidden state and the input.

170
2 Beam Search
Beam search is a decoding technique commonly used in sequence
modeling tasks like machine translation and speech recognition. It
is used to generate the most likely sequence of outputs given a
trained sequence model.
The basic idea behind beam search is to maintain a fixed-size
set of candidate sequences, known as the beam. At each time step,
the beam is expanded by considering all possible extensions of the
current candidate sequences, up to a certain predefined size. The
candidate sequences are then scored based on a scoring function,
and the top-k sequences with the highest scores are retained in the
beam.
Mathematically, the beam search algorithm can be defined as
follows:

Input: Sequence model P (yt |ht , yt−1 )

Output: Optimal sequence y ∗
Procedure:
Initialize beam B with initial sequence y0
for each time step t from 1 to T do:
Create an empty set B ′
for each sequence y in beam B do:
Compute the hidden state ht using f (ht−1 , yt−1 )
Generate the distribution P (yt |ht , yt−1 )
for each possible next output o do:
Compute the score score(y · o) using a scoring function
Select the top-k sequences with the highest scores and add them to
Set beam B to B ′
Select the sequence with the highest score from beam B as the optimal

In practice, beam search greatly improves the quality of output

sequences by considering multiple hypotheses during the decoding
process.

3 Sequence-to-Sequence Models
Sequence-to-sequence (seq2seq) models, also known as encoder-
decoder models, are widely used in various sequence modeling tasks

171
like machine translation, text summarization, and speech recogni-
tion. These models consist of two main components: an encoder
and a decoder.
The encoder takes an input sequence x = {x1 , x2 , . . . , xT } and
maps it to a fixed-dimensional vector representation called the con-
text or thought vector c. The context vector captures the infor-
mation from the input sequence that is relevant for the decoding
process.
The decoder, on the other hand, takes the context vector c and
generates the output sequence y = {y1 , y2 , . . . , yT ′ }, where T ′ may
differ from T . At each time step t, the decoder generates an output
yt based on the context vector c and the hidden state ht .
Mathematically, for an input sequence x and an output se-
quence y, the sequence-to-sequence model can be formulated as
follows:

c = Encoder(x)
ht = f (ht−1 , yt−1 , c)
P (yt |ht , yt−1 , c) = Decoder(ht , yt−1 , c)
where Encoder(·) and Decoder(·) represent the encoder and de-
coder functions, respectively, and f (·) is the hidden state mapping
function as defined in Section 1.

172
 Python code for computing hidden states in a sequence model:

import torch.nn as nn

class SequenceModel(nn.Module):
def __init__(self, input_size, hidden_size):
super(SequenceModel, self).__init__()
self.rnn = nn.RNN(input_size, hidden_size,
,→ batch_first=True)

def forward(self, x):

# Compute hidden states
_, h = self.rnn(x)
return h

4 Beam Search with Hidden State Prediction

In some sequence modeling tasks, such as speech recognition or
handwriting generation, predicting the hidden state representation
can also be important. In this scenario, an additional prediction
step is introduced in the beam search algorithm to predict the
hidden state given the input sequence and the current candidate
sequence.
Mathematically, the modified beam search algorithm can be
defined as follows:

173
 Input: Sequence model P (yt |ht , yt−1 )
Hidden state model g(ht−1 , yt−1 , x)
Output: Optimal sequence y ∗
Procedure:
Initialize beam B with initial sequence y0
for each time step t from 1 to T do:
Create an empty set B ′
for each sequence y in beam B do:
Compute the hidden state ht−1
Compute the predicted hidden state ĥt = g(ht−1 , yt−1 , x)
Compute the distribution P (yt |ĥt , yt−1 )
for each possible next output o do:
Compute the score score(y · o) using a scoring function
Select the top-k sequences with the highest scores and add them to
Set beam B to B ′
Select the sequence with the highest score from beam B as the optimal

In this modified version of beam search, the hidden state model

g(·) is used to predict the hidden state ĥt based on the previous
hidden state ht−1 , the previous output yt−1 , and the input sequence
x.

174
 Python code for beam search with hidden state prediction in a
sequence model:

def beam_search_hidden_state(model, beam_size,

,→ input_sequence):
# Initialize beam
beam = [(torch.zeros(1, model.hidden_size),
,→ [START_TOKEN], 0)]

for t in range(MAX_LENGTH):
new_beam = []

for hidden_state, output_seq, score in beam:

# Get hidden state prediction
predicted_hidden_state =
,→ model.get_hidden_state(hidden_state,
,→ output_seq, input_sequence)

# Generate next possible outputs

output_probs =
,→ model.generate_outputs(predicted_hidden_state,
,→ output_seq)
top_probs, top_indices =
,→ torch.topk(output_probs, beam_size)

for prob, index in zip(top_probs[0],

,→ top_indices[0]):
new_score = score +
,→ torch.log(prob).item()
new_output_seq = output_seq +
,→ [index.item()]

,→ new_beam.append((predicted_hidden_state,
,→ new_output_seq, new_score))

# Sort the new beam based on scores

new_beam.sort(key=lambda x: x[2],
,→ reverse=True)
beam = new_beam[:beam_size]

# Select the sequence with the highest score from

,→ the beam

175
return beam[0][1]

176
Chapter 40

Entropy and
Information Theory

Introduction
In this chapter, we will explore the concept of entropy and its appli-
cations in information theory. Entropy is a fundamental measure
of uncertainty or randomness, and it plays a crucial role in various
areas, including communication theory, data compression, and sta-
tistical inference. We will delve into the mathematical formulation
of entropy, discuss its properties, and examine its applications in
the context of information theory.

Shannon Entropy
Shannon entropy, named after Claude Shannon, is a measure of
the average amount of information contained in a random variable
or a probability distribution. It provides a quantitative measure
of uncertainty or randomness associated with the outcomes of the
random variable.
Given a discrete random variable X with a probability mass
function P (X), the Shannon entropy H(X) is defined as:
X
H(X) = − P (x) log2 P (x)
x
where x represents the possible outcomes of the random variable
X.

177
 The term P (x) log2 P (x) is known as the self-information of an
outcome x, which quantifies the amount of surprise associated with
that outcome. The negative sign in front of the summation ensures
that the entropy is always non-negative.
The Shannon entropy satisfies several important properties:

• Non-negativity: The entropy is always non-negative, i.e.,

H(X) ≥ 0.

• Maximum entropy for uniform distribution: The max-

imum value of entropy is achieved when all outcomes are
equally likely, i.e., H(X) ≤ log2 n, where n is the number of
possible outcomes.
• Additivity: The entropy of the joint distribution of two
random variables is equal to the sum of the entropies of the
individual random variables if they are independent.

Shannon entropy has various applications, such as quantifying

the information content of a message, measuring the average num-
ber of bits required to encode symbols from a given distribution,
and characterizing the amount of uncertainty in a system.

KL Divergence
Kullback-Leibler (KL) divergence, also known as relative entropy,
is a measure of the difference between two probability distributions.
It quantifies how one distribution differs from another in terms of
information content.
Given two discrete probability distributions P (X) and Q(X)
defined over the same set of outcomes, the KL divergence DKL (P ∥Q)
from Q to P is defined as:

P (x)
X  
DKL (P ∥Q) = P (x) log2
x
Q(x)

KL divergence is not a symmetric measure, i.e., DKL (P ∥Q) ̸=

DKL (Q∥P ) in general. It is always non-negative and is equal to
zero if and only if P and Q are identical.
KL divergence has several important properties:

• Non-negativity: KL divergence is always non-negative, i.e.,

DKL (P ∥Q) ≥ 0.

178
 • Zero divergence for identical distributions: KL diver-
gence is equal to zero only if P and Q are identical.
• Lack of symmetry: KL divergence is not symmetric, i.e.,
DKL (P ∥Q) ̸= DKL (Q∥P ).
• Additivity: The KL divergence between two distributions P
and Q is additive, i.e., DKL (P ∥Q)+DKL (Q∥R) = DKL (P ∥R).
KL divergence is widely used in various applications, including
information theory, statistics, machine learning, and data science.
It serves as a measure of dissimilarity between probability distribu-
tions and is frequently utilized in tasks such as model comparison,
hypothesis testing, and model selection.

Mutual Information
Mutual information is a measure of the dependence between two
random variables. It quantifies how much knowing the value of one
variable reduces the uncertainty about the other variable.
Given two discrete random variables X and Y with joint prob-
ability mass function P (X, Y ) and marginal probability mass func-
tions P (X) and P (Y ), the mutual information I(X; Y ) between X
and Y is defined as:

P (x, y)
X  
I(X; Y ) = P (x, y) log2
x,y
P (x)P (y)
Mutual information is always non-negative and is equal to zero
if and only if X and Y are statistically independent.
Mutual information satisfies several important properties:
• Non-negativity: Mutual information is always non-negative,
i.e., I(X; Y ) ≥ 0.
• Zero for independent variables: Mutual information is
equal to zero if and only if X and Y are statistically inde-
pendent.
• Symmetry: Mutual information is symmetric, i.e., I(X; Y ) =
I(Y ; X).
• Chain rule: The mutual information between multiple ran-
dom variables can be decomposed using the chain rule of
mutual information.

179
 Mutual information is widely used in various applications, in-
cluding feature selection, dimensionality reduction, clustering, and
correlation analysis. It provides a measure of the statistical depen-
dence between variables and enables us to quantify the amount of
shared information.

180
 Python code for computing Shannon entropy:

import numpy as np

def shannon_entropy(probabilities):
entropy = -np.sum(probabilities *
,→ np.log2(probabilities))
return entropy

Python code for computing KL divergence:

def kl_divergence(p, q):

kl = np.sum(p * np.log2(p / q))
return kl

Python code for computing mutual information:

def mutual_information(p_xy, p_x, p_y):

mi = np.sum(p_xy * np.log2(p_xy / (p_x * p_y)))
return mi

These functions can be used to calculate entropy, KL diver-

gence, and mutual information for discrete probability distribu-
tions.

181
Chapter 41

Computational
Complexity

Introduction
In this chapter, we delve into the field of computational complexity,
which focuses on the study of the resources required to solve com-
putational problems. We examine the time and space complexities
of algorithms, explore different classes of computational problems
based on their complexity, and provide an overview of the notation
used to express these complexities.

Time Complexity
Time complexity is a measure of the amount of time required to
run an algorithm as a function of its input size. It provides an
estimation of the number of basic operations or steps performed
by the algorithm during its execution. We express time complexity
using the Big O notation, which captures the asymptotic behavior
of the algorithm in the worst-case scenario.

1 Big O Notation
The Big O notation, denoted as O(·), describes an upper bound on
the growth rate of a function. For instance, a time complexity of
O(n) indicates that the running time of the algorithm increases lin-
early with the input size n. The Big O notation provides a concise

182
representation of the order of magnitude of the time complexity
without delving into the exact constant factors.

2 Common Time Complexities

Several common time complexity classes are encountered in the
analysis of algorithms:

• O(1): Constant time complexity, where the running time of

the algorithm remains constant regardless of the input size.
• O(log n): Logarithmic time complexity, typically observed in
algorithms that divide the input size by a constant factor in
each step, such as binary search.
• O(n): Linear time complexity, where the running time of the
algorithm scales linearly with the input size.
• O(n2 ): Quadratic time complexity, common in algorithms
that involve nested loops.

• O(2n ): Exponential time complexity, often associated with

brute-force algorithms that iterate through all possible com-
binations of the input.
• O(n!): Factorial time complexity, encountered in algorithms
that need to examine all permutations of the input.

The choice of an appropriate algorithm often involves a trade-off

between time complexity and the problem’s inherent requirements.

Space Complexity
Space complexity is a measure of the amount of memory or storage
required by an algorithm as a function of its input size. Similar
to time complexity, we use the Big O notation to express space
complexity.

1 Common Space Complexities

Several common space complexity classes include:

• O(1): Constant space complexity, where the memory usage

remains constant regardless of the input size.

183
 • O(n): Linear space complexity, where the memory usage
scales linearly with the input size.
• O(n2 ): Quadratic space complexity, commonly observed in
algorithms that store all pairwise relationships between ele-
ments in the input.

• O(2n ): Exponential space complexity, often seen in algo-

rithms that require storing all subsets of the input.

Analyzing the space complexity of an algorithm is crucial to

ensure that the available memory resources are sufficient for exe-
cuting the algorithm with the given input size.

Python Implementation
Python code can be used to measure the time complexity of an
algorithm empirically. The ‘timeit‘ module is commonly employed
to capture the execution time of a specific piece of code.
Consider the following example, which measures the execution
time of a function that finds the maximum element in a list:

import timeit

def find_max(lst):
return max(lst)

input_size = 1000000
lst = list(range(input_size))

execution_time = timeit.timeit(lambda: find_max(lst),

,→ number=100)
average_execution_time = execution_time / 100

print(f"Average execution time:

,→ {average_execution_time} seconds")

This code snippet calculates the average execution time of the

‘find_max‘ function over 100 iterations, using a list of one million
elements. Such empirical measurements can provide insights into
the time complexity of an algorithm in practice.

184
Conclusion
In this chapter, we explored the concept of computational com-
plexity, focusing on time and space complexities. We introduced
the Big O notation to express the growth rate of algorithms and
discussed common time and space complexity classes. Addition-
ally, we provided a Python code snippet that illustrates how to
measure the time complexity of an algorithm empirically using the
‘timeit‘ module. Understanding the computational complexity of
algorithms is crucial for evaluating their efficiency and scalability.

185
Chapter 42

Game Theory

1 Nash Equilibrium
In the field of game theory, the concept of Nash equilibrium plays
a fundamental role. Named after the mathematician John Nash,
a Nash equilibrium represents a stable state in a game where no
player can improve their outcome by unilaterally changing their
strategy. In this section, we define Nash equilibrium mathemati-
cally and explore its significance.
Consider a strategic game with N players. Each player i has
a set of strategies Si , and their strategy profile is denoted by s =
(s1 , s2 , . . . , sN ), where si ∈ Si represents the strategy chosen by
player i. The payoff received by player i under strategy profile
s is denoted by ui (s). We assume that every player’s goal is to
maximize their payoff.
A strategy profile s∗ is a Nash equilibrium if and only if, for
every player i and every alternative strategy s′i ∈ Si , the following
inequality holds:

ui (s′i , s∗−i ) ≤ ui (s∗ ),

where s∗−i denotes the strategy profile of all players except
player i in the Nash equilibrium s∗ . In other words, no player has
an incentive to deviate from their strategy in a Nash equilibrium.
It represents a state of mutual consistency in strategies where each
player’s strategy is the best response to the strategies chosen by
the other players.
To illustrate the concept of Nash equilibrium, consider the fa-
mous Prisoner’s Dilemma game, represented by the following payoff

186
matrix:

Player 2
C D
Player 1 C (3, 3) (0, 5)
D (5, 0) (1, 1)
In this game, both players have two possible strategies, coop-
erate (C) or defect (D). The payoff for each player is given in the
form of (Player 1’s payoff, Player 2’s payoff).
To find the Nash equilibrium, we need to identify the strategy
profile where no player can unilaterally improve their payoff. In this
case, the strategy profile (D, D) is a Nash equilibrium. If Player 1
deviates from D to C while Player 2 continues to play D, Player 1’s
payoff decreases from 1 to 0. Similarly, if Player 2 deviates from D
to C while Player 1 continues to play D, Player 2’s payoff decreases
from 1 to 0. Thus, (D, D) is a Nash equilibrium.
Python code for finding the Nash equilibrium in a game can be
implemented using the ‘nashpy‘ library as follows:

import nashpy as nash

# Define the payoff matrix of the game

payoff_matrix = [[(3, 3), (0, 5)],
[(5, 0), (1, 1)]]

# Create the game object

game = nash.Game(payoff_matrix)

# Find the Nash equilibrium

nash_equilibria = game.support_enumeration()

# Print the computed Nash equilibrium(s)

for equilibrium in nash_equilibria:
print(f"Nash Equilibrium: {equilibrium}")

The code uses the ‘nashpy‘ library to define the payoff matrix
of the game and create a game object. The ‘support_enumeration‘
method is then used to find all Nash equilibria in the game. The
resulting Nash equilibria are printed to the console.
Understanding Nash equilibria enables us to predict the out-
comes of strategic interactions and analyze the rational behavior

187
of players in various scenarios. By identifying Nash equilibria, we
can gain insights into the stability and strategic dynamics of games.

188
Chapter 43

Optimization
Techniques

1 Convex Optimization
Convex optimization is a field of study that deals with the min-
imization of convex objective functions subject to constraints. It
finds applications in various domains such as machine learning,
signal processing, and operations research. In this section, we will
introduce the concept of convex optimization and its key proper-
ties.

Convex Functions
A convex function is a real-valued function f : Ω → R defined on
a convex set Ω ⊂ Rn that satisfies the following inequality for all
x, y ∈ Ω and 0 ≤ λ ≤ 1:

f (λx + (1 − λ)y) ≤ λf (x) + (1 − λ)f (y).

Intuitively, this inequality implies that the graph of the function
lies below the line connecting any two points on the graph. Con-
vex functions possess several important properties, such as having
global minima, no local minima, and being differentiable almost
everywhere.
One commonly encountered convex function is the quadratic
function f (x) = xT Ax + bT x + c, where A is a positive semidefinite
matrix. The minimization of quadratic functions is a fundamental
problem in convex optimization.

189
Convex Sets
A convex set is a set Ω ⊂ Rn that satisfies the following inequality
for all x, y ∈ Ω and 0 ≤ λ ≤ 1:

λx + (1 − λ)y ∈ Ω.
Geometrically, this inequality states that for any two points
in the set, the line segment connecting them is entirely contained
within the set. Convex sets play a crucial role in formulating con-
straints in optimization problems.

Convex Optimization Problem

A convex optimization problem can be formulated as follows:

Minimize f (x)
subject to gi (x) ≤ 0, i = 1, 2, . . . , m
hj (x) = 0, j = 1, 2, . . . , p
where f (x) is a convex objective function, gi (x) are convex in-
equality constraints, and hj (x) are affine equality constraints.
The goal of convex optimization is to find a feasible point x∗
that minimizes the objective function f (x). It is important to note
that any local minimum of a convex optimization problem is also
a global minimum.

Optimality Conditions
A point x∗ is said to be optimal if it satisfies the following condi-
tions:

• Feasibility: x∗ satisfies all the inequality and equality con-

straints.

• Lagrange Multiplier Condition: There exist Lagrange

multipliers λ∗ and ν ∗ such that the following conditions hold:

m
X p
X
∇f (x∗ ) + λ∗i ∇gi (x∗ ) + νj∗ ∇hj (x∗ ) = 0
i=1 j=1

gi (x∗ ) ≤ 0, λ∗i ≥ 0, λ∗i gi (x∗ ) = 0

hj (x∗ ) = 0

190
 • Dual Feasibility: λ∗i ≥ 0 for all i.

The Lagrange multipliers λ∗ and ν ∗ represent the marginal util-

ity or cost of violating the constraints.
Python code for solving a convex optimization problem using
the CVXPY library is as follows:

import cvxpy as cp

# Define the optimization variables

x = cp.Variable(n)

# Define the objective function

objective = cp.Minimize(f(x))

# Define the constraints

constraints = [g_i(x) <= 0, h_j(x) == 0]

# Define the optimization problem

problem = cp.Problem(objective, constraints)

# Solve the problem

problem.solve()

The code uses the CVXPY library to define the optimization

variables, objective function, and constraints. The optimization
problem is then defined using the Problem class, and the solve
method is used to obtain the optimal solution.
Convex optimization provides a powerful framework for solving
a wide range of optimization problems. Its mathematical founda-
tions and optimization techniques play a crucial role in many areas
of science and engineering.

191
Chapter 44

Sparse Coding

Basis Functions
Sparse coding involves representing signals or data points as linear
combinations of a small number of basis functions. These basis
functions, also known as atoms, form a dictionary that captures
the intrinsic structure of the data. In this section, we will explore
the concept of basis functions and their role in sparse coding.

1 Mathematical Representation
Let y ∈ Rm be a signal or data point that we wish to represent
using sparse coding. We can express y as a linear combination of n
basis functions ϕi ∈ Rm , each associated with a coefficient xi ∈ R:
n
X
y= xi ϕi ,
i=1
where xi represents the contribution of the i-th basis function
ϕi to the signal y.

2 Sparsity Constraint
In sparse coding, we aim to find a sparse representation of the signal
y, where only a few coefficients xi are non-zero. This sparsity
constraint allows us to capture the essential information of the
signal using a small number of basis functions.
To enforce sparsity, we typically use regularization techniques
such as the ℓ1 -norm or ℓ0 -norm of the coefficients. The ℓ1 -norm

192
regularization encourages sparse solutions by promoting coefficient
values close to zero, while the ℓ0 -norm regularization directly pe-
nalizes the number of non-zero coefficients.

3 Optimization Problem
The sparse coding problem can be formulated as an optimization
problem, where we seek to find the sparsest representation of a
signal y given a dictionary Φ = [ϕ1 , ϕ2 , . . . , ϕn ] ∈ Rm×n :

min ∥x∥p
x
s.t. y = Φx,
where ∥x∥p denotes either the ℓ1 -norm or the ℓ0 -norm, depend-
ing on the desired sparsity level. The constraint y = Φx ensures
that the linear combination of the basis functions reconstructs the
original signal.
Here is a Python code snippet using CVXPY to solve the sparse
coding problem for the ℓ1 -norm regularization:

import cvxpy as cp

# Define the optimization variable

x = cp.Variable(n)

# Define the objective function

objective = cp.Minimize(cp.norm(x, 1))

# Define the constraint

constraint = [y == Phi @ x]

# Define the optimization problem

problem = cp.Problem(objective, constraint)

# Solve the problem

problem.solve()

Dictionary Learning
In practice, the basis functions or atoms of the dictionary are not
given, and they need to be learned from the data. Dictionary

193
learning is an iterative process that alternates between finding the
sparse coding of the data and updating the dictionary.

1 Sparse Coding
Given a dictionary Φ, we can find the sparse coding x of a signal
y by solving the optimization problem:

min ∥x∥p
x
s.t. y = Φx.
This can be solved using various optimization algorithms, such
as the proximal gradient method or the interior-point method.
Here is a Python code snippet using CVXPY to solve the sparse
coding problem for the ℓ1 -norm regularization:

import cvxpy as cp

# Define the optimization variable

x = cp.Variable(n)

# Define the objective function

objective = cp.Minimize(cp.norm(x, 1))

# Define the constraint

constraint = [y == Phi @ x]

# Define the optimization problem

problem = cp.Problem(objective, constraint)

# Solve the problem

problem.solve()

2 Dictionary Update
After obtaining the sparse coding x, the dictionary Φ can be up-
dated to better capture the underlying structure of the data. Var-
ious algorithms, such as the K-SVD algorithm or the method of
optimal directions (MOD), can be employed for dictionary update.
The dictionary update step aims to minimize the reconstruction
error between the data and the learned sparse coding. It can be
formulated as an optimization problem:

194
 min ∥y − Φx∥22
Φ
s.t. ∥ϕi ∥2 = 1, ∀i,
where the constraint ensures that each basis function ϕi has
unit norm.

3 Dictionary Learning Algorithm

The dictionary learning process iterates between the sparse coding
step and the dictionary update step until convergence is reached.
Typically, a stopping criterion based on the change in the objective
function value is used to terminate the algorithm.
Below is an outline of the dictionary learning algorithm:

1. Initialize the dictionary Φ.

2. Repeat until convergence:

(a) Perform sparse coding to obtain x.

(b) Update the dictionary Φ.
3. Output the learned dictionary Φ.

Dictionary learning is a powerful technique for sparse represen-

tation and dimensionality reduction of data. It finds applications
in various fields such as image processing, signal analysis, and com-
puter vision.
By learning a dictionary from the data, we can effectively cap-
ture the most representative features of the signals or data points,
leading to improved performance in various tasks such as denoising,
compression, and classification.

195
Chapter 45

Multi-Task Learning

Introduction
Multi-task learning (MTL) is a machine learning paradigm that
aims to improve the performance of multiple related tasks by learn-
ing them jointly. In many real-world scenarios, there are multiple
tasks that share some common underlying knowledge or structure.
MTL leverages this shared information to learn better models for
each task, leading to improved generalization performance and en-
hanced efficiency.

Problem Formulation
1 Single-Task Learning
Before diving into the details of multi-task learning, let’s first re-
view the problem formulation for single-task learning. In single-
task learning, we have a training set composed of N samples, de-
noted as D = {(xi , yi )}N
i=1 , where xi represents the input features
for sample i and yi represents the corresponding target value.
The goal of single-task learning is to learn a function f : X → Y
that maps the input space X to the output space Y. This function
can be represented by a model with learnable parameters, such as
a neural network.

196
2 Multi-Task Learning
In multi-task learning, we consider K related tasks, each with its
own training set. Let Dk = {(xk,i , yk,i )}N i=1 denote the training
k

set for task k, where xk,i and yk,i represent the input features and
target value, respectively, for sample i of task k.
The goal of multi-task learning is to learn a set of K functions
{fk : X → Yk }K k=1 , where Yk is the output space for task k. In
other words, we aim to learn models that map the input space X
to the specific output spaces for each task.

Benefits of Multi-Task Learning

There are several advantages of using multi-task learning over single-
task learning:

1 Improved Generalization
By learning multiple tasks jointly, multi-task learning can leverage
the shared information across tasks. This allows the models to
learn a more robust and generalizable representation of the data.
The shared knowledge can help to regularize the learning process,
leading to improved generalization performance on each individual
task.

2 Data Efficiency
In many scenarios, the availability of labeled data for each individ-
ual task is limited. Multi-task learning provides a means to leverage
the data from related tasks to improve the learning performance.
By jointly learning multiple tasks, the models can effectively uti-
lize the information from each task, resulting in better performance
with fewer training examples per task.

3 Reduced Overfitting
Multi-task learning can also help to reduce overfitting in the pres-
ence of limited training data. By simultaneously learning multiple
tasks, the models are encouraged to focus on the common struc-
tures shared by the tasks and avoid overfitting to the idiosyncrasies
of individual tasks.

197
4 Transfer Learning
Another benefit of multi-task learning is its ability to facilitate
transfer learning. The knowledge learned from one task can be
transferred to another related task, even when the target domains
differ. This transfer of knowledge can provide a head start in learn-
ing new tasks and enable models to adapt more quickly to new
domains.

Multi-Task Learning Algorithms

There are various approaches to solving multi-task learning prob-
lems, including parameter sharing, regularization, and task rela-
tionship modeling. These approaches aim to effectively share in-
formation across tasks while still allowing for task-specific learning.

1 Parameter Sharing
Parameter sharing is a common approach in multi-task learning,
where the models for different tasks share some or all of their pa-
rameters. By sharing parameters, the models can effectively trans-
fer knowledge across tasks, capturing the shared information and
exploiting the similarities among tasks.
For example, in neural networks, parameter sharing can be
achieved by using shared layers that process the input features
for all tasks. This allows the network to learn a common feature
representation across tasks while maintaining task-specific output
layers.

2 Regularization
Regularization techniques can also be employed in multi-task learn-
ing to encourage the sharing of information among tasks. By in-
corporating regularization terms in the loss function, the models
are incentivized to learn shared structures and avoid overfitting to
task-specific noise.
One common regularization technique is the ℓ1 /ℓ2 norm regu-
larization, which promotes sparsity in the task-specific parameters.
This encourages the models to focus on a subset of features that
are shared across tasks while allowing for task-specific variations.

198
3 Task Relationship Modeling
Task relationship modeling is another approach in multi-task learn-
ing that captures the relationships among different tasks. This can
be achieved by learning task-specific weights that reflect the im-
portance or relevance of each task during training.
For instance, task relationship modeling can be performed using
graph-based methods, where each task corresponds to a node in the
graph, and the edges represent the relationships between tasks. By
incorporating the graph structure into the learning process, the
models can effectively leverage the task relationships to improve
performance.

Summary
Multi-task learning offers several benefits over single-task learning,
including improved generalization, data efficiency, reduced overfit-
ting, and transfer learning capabilities. By jointly learning multiple
tasks, the models can effectively leverage shared information and
improve performance on each individual task. Various algorithms,
such as parameter sharing, regularization, and task relationship
modeling, can be used to facilitate multi-task learning.

199
Python Code
Here is an example of applying multi-task learning using the scikit-
learn library:

from sklearn.linear_model import MultiTaskLasso

# Create a MultiTaskLasso object

model = MultiTaskLasso(alpha=0.1)

# Fit the model to the training data

model.fit(X_train, Y_train)

# Predict the outputs for the test data

Y_pred = model.predict(X_test)

In this example, we use the MultiTaskLasso class, which imple-

ments multi-task Lasso regression. We fit the model to the training
data, where X_train represents the input features and Y_train
represents the target values for each task. We then use the trained
model to predict the outputs for the test data (X_test), obtaining
the predictions in Y_pred.

200
Chapter 46

Meta-Learning

Introduction
Meta-learning is a field of study that focuses on algorithms and
techniques for learning to learn. This higher-level learning process
involves acquiring knowledge or skills that can be applied to a wide
range of learning tasks. In this chapter, we explore the foundational
concepts and methods in meta-learning with a mathematical per-
spective.

Problem Formulation
1 Single-Learning Task
We begin with the formulation of a single-learning task. Let D
denote the dataset containing N samples, represented as pairs of in-
put features and corresponding target values, i.e., D = {(xi , yi )}N
i=1 ,
where xi ∈ Rd and yi ∈ R for regression problems, or yi ∈ {0, 1}
for classification problems. The goal of single-learning task is to
learn a function f : X → Y that can map an input x to an output
y, where X is the input space and Y is the output space.

2 Meta-Learning
In meta-learning, we consider a distribution of learning tasks, de-
noted as T . Each task T ∈ T is characterized by a dataset DT and
a corresponding function fT : X → Y. The goal of meta-learning

201
is to learn a meta-learner algorithm that can efficiently adapt to
new tasks drawn from T .
Formally, the meta-learner takes as input a dataset DT for a new
task T , and outputs a function fT ′ : X → Y that can effectively
map inputs x to outputs y for the new task. The meta-learner
is trained on a distribution of tasks in order to generalize to new
tasks by learning patterns or regularities across the training tasks.

Meta-Learning Algorithms
Meta-learning algorithms can generally be classified into two cate-
gories: model-agnostic meta-learning (MAML) and meta-learning
with recurrent neural networks (meta-RNN). We provide a brief
overview of these algorithms below.

1 Model-Agnostic Meta-Learning (MAML)

MAML is a widely used approach in meta-learning that aims to
learn an initialization point for the model parameters, such that
fine-tuning on a new task is efficient and effective. MAML makes
minimal assumptions about the underlying model architecture and
task distribution, making it flexible and widely applicable.
The main idea behind MAML is to learn a set of initial model
parameters, denoted as θ, that can be quickly adapted to a new task
with only a few gradient steps. This is accomplished by optimizing
the meta-objective function, which measures the performance of
the model after task adaptation. The meta-learner aims to find
the optimal initialization θ∗ that yields good performance across
tasks.

2 Meta-Learning with Recurrent Neural Networks

(meta-RNN)
Meta-RNN is another approach to meta-learning that utilizes re-
current neural networks (RNNs) as the meta-learner model. RNNs
are particularly suitable for meta-learning scenarios due to their
ability to process sequential data and capture temporal dependen-
cies.
In meta-RNN, the meta-learner consists of an RNN that takes
the task-specific dataset DT as input and produces a sequence of

202
model parameters over time, denoted as {θ1 , θ2 , ..., θT }. By pro-
cessing the task data in a sequential manner, the RNN can capture
the patterns and regularities in the task-specific datasets, enabling
effective adaptation to new tasks.

Mathematical Representation
To provide a mathematical representation of meta-learning algo-
rithms, we introduce the following notation:

• T : The distribution of learning tasks.

• T : A specific task drawn from T .
• DT : The dataset associated with task T .

• fT : The function representing the task T .

• θ: The model parameters.
• L: The loss function used to measure the discrepancy be-
tween predicted and true outputs.

• ∇θ : The gradient operator with respect to the model param-

eters θ.

1 Model-Agnostic Meta-Learning (MAML)

The MAML algorithm aims to find the optimal initial parameters
θ∗ by minimizing the meta-objective function. Given a specific task
T and its associated dataset DT , the MAML optimization can be
mathematically represented as:

θ∗ = arg min L(DT , fT (DT , θ′ ))

θ

where θ = θ − α∇θ L(DT , fT (DT , θ)) represents the adaptation

′

of the model parameters to the task T , and α is the learning rate

for the adaptation process.

203
2 Meta-Learning with Recurrent Neural Networks
(meta-RNN)
In meta-RNN, the meta-learner consists of an RNN that processes
the task-specific dataset DT and obtains a sequence of model pa-
rameters {θ1 , θ2 , ..., θT }. The RNN is optimized by minimizing the
loss function L across all tasks drawn from T :
X
min L(DT , fT (DT , θT ))
θ
T ∼T
where θT represents the model parameters for task T .

Python Code
Here’s an example of applying MAML using the PyTorch library:

import torch
import torch.nn as nn
import torch.optim as optim

# Define the model architecture

model = nn.Linear(input_dim, output_dim)

# Define the loss function

criterion = nn.MSELoss()

# Define the optimizer for meta-training

meta_optimizer = optim.SGD(model.parameters(),
,→ lr=meta_lr)

# Meta-training loop
for task_batch in meta_train_loader:
for task in task_batch:
# Adapt the model to the task-specific data
adapted_model = MAMLAdaptation(model,
,→ task.data)

# Compute the loss with adapted model

,→ parameters
loss = criterion(adapted_model(task.data),
,→ task.target)

204
 # Update the model parameters using the
,→ meta-optimizer
meta_optimizer.zero_grad()
loss.backward()
meta_optimizer.step()

# Meta-testing loop
for task_batch in meta_test_loader:
for task in task_batch:
# Adapt the model to the task-specific data
adapted_model = MAMLAdaptation(model,
,→ task.data)

# Evaluate the adapted model on the task

predictions = adapted_model(task.data)

# Compute performance metrics

# ...

In this example, we define a model architecture using the Py-

Torch’s nn.Linear module. We then define the loss function (nn.MSELoss)
and the optimizer for meta-training (optim.SGD). The code demon-
strates the meta-training and meta-testing loops, where we adapt
the model to the task-specific data, compute the loss with the
adapted model parameters, and update the model parameters us-
ing the meta-optimizer.

205
Chapter 47

Bayesian Networks

Introduction
Bayesian Networks (BNs) are probabilistic graphical models that
represent dependencies among a set of random variables using a
directed acyclic graph (DAG). In this chapter, we will explore the
mathematical foundations and properties of Bayesian Networks.

Formal Definition
Let X = {X1 , X2 , ..., Xn } be a set of random variables. A Bayesian
Network for X is defined as a directed acyclic graph G = (X , E),
where E is a set of directed edges (Xi , Xj ) representing the depen-
dencies among random variables.

Conditional Probability Distribution

The strength of Bayesian Networks lies in their ability to model
joint probability distributions using conditional probability distri-
butions (CPDs). A CPD specifies the conditional probability of a
random variable given its parents in the graph.
For each random variable Xi , we define a CPD P(Xi |Pai ),
where Pai represents the parents of Xi . The CPD defines the
conditional probabilities for all possible assignments of values to
Xi and its parents.

206
Bayesian Network Inference
Given a Bayesian Network, we are often interested in making infer-
ences about the probability distribution of certain variables, given
observed evidence. This can be done using both exact and approx-
imate inference methods.

1 Exact Inference: Variable Elimination

One popular exact inference method for BNs is variable elimina-
tion. The idea behind variable elimination is to sequentially elim-
inate variables from the joint distribution by summing (or inte-
grating) over their possible values, until the desired variables are
obtained.

2 Approximate Inference: Markov Chain Monte

Carlo
When exact inference is intractable due to the size or complexity
of the BN, we can resort to approximate methods such as Markov
Chain Monte Carlo (MCMC). MCMC methods, like Gibbs Sam-
pling and Metropolis-Hastings, provide efficient ways to generate
samples from the posterior distribution of interest.

Learning Bayesian Networks

Learning Bayesian Networks involves estimating the structure (i.e.,
the graph) and the parameters (i.e., the CPDs) of the network
from observational or experimental data. This can be done using
various algorithms such as constraint-based, score-based, or hybrid
approaches.

1 Constraint-Based Methods: PC Algorithm

The PC algorithm is a widely used constraint-based method that
infers the structure of a BN using conditional independence tests.
Starting with an empty graph, the algorithm iteratively adds edges
based on statistical tests until the Markov equivalence class is iden-
tified.

207
2 Score-Based Methods: Maximum Likelihood
Estimation
Score-based methods aim to find the structure and parameters that
maximize a scoring criterion given the data. Maximum Likelihood
Estimation (MLE) is a common score-based approach that esti-
mates the parameters of the CPDs by maximizing the likelihood of
the observed data.

Python Implementation
Here’s a Python implementation of the PC algorithm using the
pgmpy library:

from pgmpy.estimators import PC

# Instantiate the PC algorithm object

pc = PC(data)

# Run the PC algorithm to learn the structure

estimated_model = pc.estimate()

In this code snippet, we first instantiate the PC algorithm ob-

ject using the observed data. We then run the estimate() method
to learn the structure of the Bayesian Network. The resulting
estimated_model object contains the learned structure and pa-
rameters of the network.

208
Chapter 48

Optimization
Techniques

Introduction
In this chapter, we explore various optimization techniques used in
the field of machine learning. Optimization plays a crucial role in
training models and finding optimal solutions to complex problems.
We will discuss convex optimization, quadratic programming, and
Lagrange multipliers, highlighting their mathematical foundations
and applications in machine learning.

Convex Optimization
Convex optimization is a subfield of mathematical optimization
that deals with finding the minimum of a convex objective function
subject to a set of linear equality and inequality constraints. It is
widely used in machine learning due to the nice properties of convex
functions and efficient algorithms for optimization.
The mathematical formulation of convex optimization can be
written as follows:

209
 Minimize f (x)
Subject to Ax ⪯ b
Cx = d
x⪰0

Here, x ∈ Rn represents the optimization variable, f (x) is the

convex objective function, A and C are the constraint matrices,
and b and d are the corresponding constraint vectors. The inequal-
ity constraints are expressed as Ax ⪯ b, the equality constraints
as Cx = d, and the non-negativity constraints as x ⪰ 0.

Quadratic Programming
Quadratic programming is a specific form of convex optimization
that deals with quadratic objective functions and linear constraints.
It is commonly used in machine learning for tasks such as support
vector machines (SVM) and portfolio optimization.
The general form of quadratic programming can be expressed
as:

1 T
Minimize x Qx + cT x
2
Subject to Ax ⪯ b
Cx = d
x⪰0

Here, x ∈ Rn is the optimization variable, Q is a positive

semidefinite matrix, c is the linear coefficient vector, and the con-
straint matrices and vectors have the same meaning as in convex
optimization.

Lagrange Multipliers
Lagrange multipliers provide a method for solving constrained op-
timization problems by introducing additional variables, the La-
grange multipliers, to convert the constrained optimization into an
unconstrained optimization problem.

210
 Consider the following constrained optimization problem with
equality constraints:

Minimize f (x)
Subject to hi (x) = 0, i = 1, 2, ..., m

We introduce Lagrange multipliers λ = (λ1 , λ2 , ..., λm )T to form

the Lagrangian:

L(x, λ) = f (x) + λT h(x)

where h(x) = (h1 (x), h2 (x), ..., hm (x))T .
To find the optimal solution, we take the partial derivatives of
the Lagrangian with respect to x and λ, and set them equal to
zero:

∇x,λ L(x, λ) = 0
Solving this system of equations provides the values of x and λ
that yield the optimal solution.

Python Implementation
Here’s a Python code snippet that demonstrates solving an op-
timization problem using quadratic programming with the cvxpy
library:

import cvxpy as cp

# Define the optimization variables

x = cp.Variable(n)

# Define the objective function and constraints

objective = cp.Minimize(0.5 * cp.quad_form(x, Q) +
,→ c.T @ x)
constraints = [A @ x <= b, C @ x == d, x >= 0]

# Define the problem and solve it

problem = cp.Problem(objective, constraints)
problem.solve()

211
 In this code snippet, we define the optimization variable x as a
cvxpy variable. We then specify the objective function, quadratic
form, and linear constraints using the cvxpy syntax. Finally, we
create a cvxpy problem object and solve it using the solve()
method. The optimal solution is stored in the x.value attribute.

212
Chapter 49

Bifurcation Theory

Stability Analysis
Stability analysis is a crucial tool in studying the behavior of dy-
namic systems. In the context of bifurcation theory, stability anal-
ysis helps determine the stability or instability of equilibria and
their associated solutions as system parameters vary.
Consider a dynamical system described by the ordinary differ-
ential equation:

ẋ = f (x, p), (49.1)

where x ∈ Rn represents the system state, p ∈ Rm denotes the
system parameters, and f (·) is a vector-valued function determining
the dynamics.
An equilibrium point x0 corresponds to a constant solution of
the system, defined by f (x0 , p) = 0. Stability analysis investigates
the behavior of the system near these equilibria by examining the
Jacobian matrix J(x0 , p) = ∂x ∂f
(x0 , p), which represents the lin-
earization of the system dynamics around x0 .
The stability of an equilibrium can be determined by studying
the eigenvalues of the Jacobian matrix. If all eigenvalues have
negative real parts, the equilibrium is asymptotically stable. If
at least one eigenvalue has a positive real part, the equilibrium is
unstable. Additionally, in the case of complex eigenvalues, Hopf
bifurcations may occur, leading to the emergence of stable limit
cycles.

213
Fixed Points and Periodic Orbits
Fixed points and periodic orbits are important solutions that arise
in dynamical systems. Fixed points correspond to equilibria, where
the system remains unchanged over time. Periodic orbits, on the
other hand, correspond to states that the system visits repeatedly
after a certain period.
A fixed point x0 is defined as a solution to f (x0 , p) = 0. It
represents a stable equilibrium if all nearby trajectories converge
to x0 . Conversely, an unstable equilibrium is one where nearby
trajectories diverge from x0 .
Periodic orbits, also known as limit cycles, occur when the sys-
tem follows a closed trajectory in the state space. They represent
stable solutions that the system repeatedly visits. The period of a
limit cycle represents the time taken to complete one cycle.
To determine fixed points and periodic orbits, we can perform
numerical simulations or algebraic analysis. Numerical methods,
such as Euler’s method or Runge-Kutta methods, iterate the sys-
tem dynamics until equilibrium or periodic behavior is observed.
Algebraic analysis involves solving the equations f (x, p) = 0 or ex-
amining the conditions for periodic solutions, such as the Poincaré-
Bendixson theorem.

Applications in Dynamical Systems

Bifurcation theory finds broad applications in understanding the
behavior of dynamical systems across various disciplines. Some
notable applications include:

• Physics: Bifurcation theory helps analyze complex physical

phenomena, such as phase transitions, chaos, and pattern
formation. Examples include the investigation of bifurcations
in systems described by the Lorenz equations or the Swift-
Hohenberg equation.

• Biology: Bifurcation analysis plays a significant role in study-

ing biological systems, such as population dynamics, neural
oscillations, and genetic regulatory networks. It aids in un-
derstanding phenomena like the emergence of synchronized
behavior or the bifurcations associated with the onset of epilep-
tic seizures.

214
 • Engineering: Bifurcation theory is relevant in engineering
disciplines dealing with dynamic systems, such as control the-
ory, electrical circuits, or chemical reactors. It helps identify
critical points, stability regions, and parameter ranges that
lead to desirable or undesirable system behavior.

• Economics: Bifurcation analysis contributes to economic

modeling by examining macroeconomic phenomena and fi-
nancial market dynamics. It elucidates phenomena like eco-
nomic crises, emerging patterns in business cycles, or the for-
mation of stable economic equilibria.

The application of bifurcation theory in these fields enables

deeper insights into the behavior and stability of complex systems,
leading to enhanced control, prediction, and decision-making capa-
bilities.

Python Implementation
Here is a Python code snippet demonstrating the numerical simula-
tion of a dynamical system using the solver from the SciPy library:

import numpy as np
from scipy.integrate import solve_ivp

def dynamics(t, x):

# Define the system dynamics
dx_dt = [...]
return dx_dt

# Define the initial conditions and time span

x0 = [...]
t_span = [t_start, t_end]

# Solve the dynamical system

sol = solve_ivp(dynamics, t_span, x0)

# Access the solution

t = sol.t # Time points
x = sol.y # State variables

215
 In this code snippet, we first import the necessary libraries, in-
cluding NumPy for numerical computations and SciPy’s solve_ivp()
function to solve the initial value problem.
Next, we define the function dynamics(t, x) that encapsulates
the system dynamics. The input t represents time, and x is the
vector of state variables.
We then assign the initial conditions and time span to x0 and
t_span, respectively.
Finally, we solve the dynamical system using solve_ivp(dynamics,
t_span, x0). The resulting solution is stored in the sol object,
which contains the time points sol.t and the corresponding state
variable values sol.y. These can be accessed and analyzed further
as needed.

216
Chapter 50

Topological Data
Analysis (TDA)

Persistent Homology
Persistent homology is a mathematical tool used in topological data
analysis to extract robust topological features from data. It pro-
vides a framework for identifying and quantifying topological struc-
tures, such as connected components, holes, and voids, that persist
across different spatial scales. The persistence of these features can
be represented using the concept of a persistence diagram.

1 Definition of Persistence Diagrams

Given a filtered simplicial complex K and a field F (often taken as
the field of real or integer numbers), the persistence of a topological
feature, such as a connected component or hole, is represented by
a pair of real numbers denoting the filtration values at which the
feature is born and dies. This pair is known as a persistence point.
A collection of persistence points is called a persistence dia-
gram, denoted as D. Each point (b, d) in the diagram corresponds
to a feature that appears at a filtration value b and disappears at a
filtration value d. Persistent homology aims to analyze the distribu-
tion of persistence points in the persistence diagram to characterize
the underlying topological structure of the data.

217
2 Computation of Persistent Homology
The computation of persistent homology involves constructing a
filtration of a simplicial complex and then computing the homology
groups at each stage of the filtration. This process relies on the
concept of boundary operators and the notion of persistent Betti
numbers.

Boundary Operators
Given a simplicial complex, the boundary operators, denoted by
∂k , map k-simplices to (k − 1)-simplices. For example, in a 2-
dimensional simplicial complex, the boundary operator ∂2 maps
triangles to edges, while ∂1 maps edges to vertices.

Persistent Betti Numbers

The persistent Betti numbers, denoted as βk , measure the number
of k-dimensional topological features (e.g., connected components,
holes) present in the simplicial complex at each stage of the filtra-
tion.
To compute the persistent Betti numbers, one constructs a
boundary matrix Bk that represents the boundary operators ∂k
as a matrix. By performing Gaussian elimination on Bk , one ob-
tains its reduced echelon form Rk . The persistent Betti numbers
βk can then be obtained by counting the number of non-zero rows
in Rk .

3 Applications of Persistent Homology

Persistent homology has found applications in various fields, includ-
ing computer vision, computational biology, and materials science.
Some specific applications include:

Shape Recognition and Classification

Persistent homology can be used to extract topological features
from shapes and provide a robust representation for shape recog-
nition and classification tasks. By analyzing the persistence dia-
grams, one can identify distinctive topological signatures and dis-
criminate between different shape categories.

218
Point Cloud Analysis
In point cloud data, persistent homology can identify and quantify
topological features, such as loops and voids, which may be crit-
ical for characterizing the geometric properties of the data. This
analysis allows for the development of algorithms for point cloud
segmentation, denoising, and anomaly detection.

Neuroimaging Analysis
In neuroimaging, persistent homology has been applied to study
the brain’s structural connectivity networks. By representing brain
regions as nodes and fiber tracts as edges, persistence diagrams can
capture topological features that reflect the brain’s organization,
such as clusters, bridges, and tunnels.

Betti Numbers
The Betti numbers, denoted as βk , are fundamental topological
invariants that provide information about the number and con-
nectivity of k-dimensional holes in a topological space. Persistent
homology utilizes the concept of Betti numbers to capture the topo-
logical features present in data.

1 Definition of Betti Numbers

Consider a topological space X and its associated simplicial com-
plex K. The k-th Betti number, denoted as βk , represents the rank
of the k-th homology group of K. Intuitively, it counts the number
of k-dimensional holes in X.
Formally, the k-th homology group Hk (X) is defined as the
quotient group of the k-cycles (k-dimensional closed loops) modulo
the k-boundaries (k-dimensional loops with no boundary). Math-
ematically, we can express this as:

ker(∂k )
Hk (X) =
im(∂k+1 )
where ∂k is the boundary operator mapping k-simplices to (k −
1)-simplices.

219
2 Computing Betti Numbers
Computing Betti numbers typically involves constructing the bound-
ary matrices Bk and applying linear algebra techniques to analyze
their structure.
Given a simplicial complex, the boundary matrix Bk represents
the boundary operator ∂k as a matrix. By performing Gaussian
elimination on Bk , one obtains its reduced row echelon form Rk .
The number of non-zero rows in Rk corresponds to the rank of the
boundary matrix and thereby provides the Betti number βk .

3 Interpretation of Betti Numbers

Betti numbers provide insights into the topological properties of a
space. Here are some interpretations of Betti numbers for different
values of k:

• β0 : The first Betti number β0 represents the number of con-

nected components in a space. For example, in a single con-
nected shape, β0 = 1.
• β1 : The second Betti number β1 corresponds to the number
of non-contractible loops in a space. In a simply-connected
shape without any holes, β1 = 0.
• β2 : The third Betti number β2 indicates the number of voids
or cavities present in a space. For instance, in a solid torus,
β2 = 1.

By examining the Betti numbers across different dimensions,

one can identify and characterize the topological features of a space,
which can be further analyzed using persistent homology.

Python Implementation
Here is a Python code snippet showcasing the computation of per-
sistent homology using the Gudhi library:

import gudhi

# Create the simplicial complex

simplicial_complex = gudhi.SimplexTree()

220
# Add simplices to the complex
simplicial_complex.insert([0]) # Vertex 0
simplicial_complex.insert([1]) # Vertex 1
simplicial_complex.insert([2]) # Vertex 2
simplicial_complex.insert([0, 1]) # Edge (0, 1)
simplicial_complex.insert([1, 2]) # Edge (1, 2)
simplicial_complex.insert([0, 2]) # Edge (0, 2)
simplicial_complex.insert([0, 1, 2]) # Triangle (0,
,→ 1, 2)

# Compute the persistence diagram

persistence = simplicial_complex.persistence()

# Print the persistence pairs

for pair in persistence:
print(f"Birth: {pair[0]}, Death: {pair[1]}")

In this code snippet, we first import the Gudhi library, which

provides tools for topological data analysis.
We create a simplicial complex using the SimplexTree() class.
We then add the desired simplices to the complex using the insert()
method.
To compute the persistence diagram, we use the persistence()
method of the simplicial complex. This method returns the persis-
tence pairs, which represent the birth and death values of topolog-
ical features.
Finally, we print the birth and death values of each persistence
pair to display the persistence diagram.

221
Chapter 51

Spiking Neural
Networks (SNN)

Neuron Models
1 Introduce Leaky Integrate-and-Fire (LIF) model
The Leaky Integrate-and-Fire (LIF) model is a widely used neuron
model in the field of spiking neural networks. It captures the basic
behavior of a neuron by simulating the integration and generation
of action potentials (spikes) in response to incoming stimuli.
The LIF model describes the membrane potential of a neuron
as a function of time. It incorporates the leakage of charge through
the membrane and the generation of spikes when the membrane po-
tential exceeds a certain threshold. Mathematically, the membrane
potential of a LIF neuron can be represented as:
dV
τm = −(V − Vrest ) + RI
dt
where τm is the membrane time constant, V is the membrane
potential, Vrest is the resting potential, R is the membrane resis-
tance, and I is the input current to the neuron.

2 Describe Spike Generation and Resetting

When the membrane potential V exceeds a predefined threshold
Vth , the neuron generates a spike and the membrane potential is

222
reset to a reset potential Vreset . This spike generation and resetting
behavior can be represented mathematically as:

if V ≥ Vth , then V ← Vreset

Additionally, a refractory period can be introduced after each
spike, during which the neuron is temporarily insensitive to incom-
ing stimuli. This can be represented as:

if V ≥ Vth , then V ← Vreset , and the neuron enters a refractory period

3 Describe Spike-Timing-Dependent Plasticity (STDP)

Spike-Timing-Dependent Plasticity (STDP) is a synaptic plasticity
rule that modulates the strengths of connections between neurons
based on the timing of their spikes. It is a Hebbian learning rule
that strengthens or weakens synapses depending on the relative
timing of pre-synaptic and post-synaptic spikes.
Mathematically, the weight update rule for STDP can be rep-
resented as:
  
 A + · exp − ∆t
τ if ∆t > 0
 +


∆w = −A- · exp ∆t if ∆t < 0
 τ-
0 if ∆t = 0



where ∆w is the weight change, ∆t is the time difference of the

pre- and post-synaptic spikes, A+ and A- control the magnitude
of weight changes for pre- and post-synaptic spikes, and τ+ and τ-
control the decay rates of weight changes.

SNN Architecture
1 Describe Feedforward Architecture
A feedforward spiking neural network (SNN) architecture consists
of layers of neurons connected in a feedforward manner, without
any recurrent connections. The information flows from the input
layer to the output layer, with each neuron in a layer receiving
inputs only from the previous layer.
Mathematically, the activation alj of a neuron j in layer l can
be computed as:

223
 nl−1
X
alj = l
wjk · sl−1
k
k=1

where wjk
l
is the weight connecting neuron k in layer l − 1 to
neuron j in layer l, and sl−1
k is the spike train of neuron k in layer
l − 1.

2 Describe Spiking Activation Functions

Spiking activation functions determine when a neuron in an SNN
generates a spike based on its membrane potential. These functions
are typically non-linear and model the spiking behavior of biological
neurons.
The most commonly used spiking activation function is the
thresholding function, which generates an output spike when the
input potential exceeds a predefined threshold. Mathematically,
this can be represented as:
(
1 if V ≥ Vth
y=
0 otherwise
where y is the output spike and V is the membrane potential.
Other spiking activation functions, such as the sigmoidal func-
tion or the rectified linear function, can also be used depending on
the specific requirements of the SNN.

3 Describe Spike-Time Encoding for Inputs

In spiking neural networks, inputs are typically encoded as spike
trains, representing the timing of spikes rather than the magni-
tudes of inputs. This spike-time encoding allows for the efficient
representation and processing of temporal information.
The encoding of an input x(t) as a spike train can be accom-
plished using various methods, such as rate-based encoding or tem-
poral coding. Rate-based encoding maps the input magnitude to
the firing rate of the neuron, while temporal coding encodes the
input value based on the timing of spikes.
For example, a spike-time encoding scheme based on the Integrate-
and-Fire model can be represented as:
N
X
x(t) → s(t) = δ(t − ti )
i=1

224
 where x(t) is the input signal, s(t) is the resulting spike train,
δ(t − ti ) is a Dirac delta function at time ti , and N is the total
number of spikes generated.
To decode the encoded information, spike-based decoding tech-
niques can be applied in combination with appropriate spiking ac-
tivation functions and synaptic weights.

Python Implementation
Here is a Python code snippet showcasing the computation of the
membrane potential dynamics of a LIF neuron:

import numpy as np

def simulate_lif_neuron(I, tau_m, R, V_rest, V_reset,

,→ V_th, dt, t_max):
num_steps = int(t_max / dt)
time = np.arange(0, t_max, dt)

V = np.zeros(num_steps)
spikes = np.zeros(num_steps)

for i in range(1, num_steps):

dV_dt = (-(V[i - 1] - V_rest) + R * I) /
,→ tau_m
V[i] = V[i - 1] + dt * dV_dt

if V[i] >= V_th:

V[i] = V_reset
spikes[i] = 1

return time, V, spikes

# Define LIF neuron parameters

I = 1.0 # Input current
tau_m = 10.0 # Membrane time constant
R = 1.0 # Membrane resistance
V_rest = 0.0 # Resting potential
V_reset = 0.0 # Reset potential
V_th = 1.0 # Threshold potential
dt = 0.1 # Time step

225
t_max = 100.0 # Maximum simulation time

# Simulate LIF neuron

time, membrane_potential, spike_train =
,→ simulate_lif_neuron(I, tau_m, R, V_rest, V_reset,
,→ V_th, dt, t_max)

# Plot membrane potential and spike train

import matplotlib.pyplot as plt

plt.figure()
plt.subplot(2, 1, 1)
plt.plot(time, membrane_potential)
plt.xlabel('Time (ms)')
plt.ylabel('Membrane Potential (V)')

plt.subplot(2, 1, 2)
plt.eventplot(time[spike_train.nonzero()],
,→ linelengths=0.6)
plt.xlabel('Time (ms)')
plt.yticks([], [])
plt.title('Spike Train')

plt.tight_layout()
plt.show()

In this code snippet, we define a function simulate_lif_neuron()

that numerically simulates the LIF neuron using the Euler method.
It takes as input the parameters of the LIF neuron and returns the
time, membrane potential, and spike train.
We then define the LIF neuron parameters and call the simulate_lif_neuron()
function to obtain the membrane potential dynamics and spike
train. Finally, we plot the membrane potential and spike train
using the Matplotlib library.

226
Chapter 52

Federated Learning

Introduction
Federated learning is a distributed machine learning approach that
enables training models across multiple decentralized devices with-
out requiring data to be centralized. This chapter explores the
concept of federated learning and discusses its application in col-
laborative AI.

Data Privacy Concerns

The decentralized nature of federated learning raises concerns about
data privacy and security. In traditional machine learning, data is
often collected and stored in a centralized server or cloud, which
poses risks such as unauthorized access or data breaches. With
federated learning, data remains on the local devices, preserving
user privacy. However, there are still potential privacy concerns to
be addressed.
One key privacy concern in federated learning is the exposure of
individual user data during the model training process. To mitigate
this, techniques such as secure aggregation and differential privacy
can be employed.

1 Secure Aggregation
Secure aggregation is a cryptographic technique used in federated
learning to protect the privacy of individual user data during the

227
aggregation process. It allows local devices to encrypt their model
updates before sending them to the server. The server can then
aggregate the encrypted updates without accessing the raw data,
preserving user privacy.
The aggregation process in federated learning can be mathe-
matically represented as:
N
1 X (i)
wglobal = w
N i=1 local
(i)
where wglobal is the aggregated global model, wlocal is the local
model update from device i, and N is the total number of devices.

2 Differential Privacy
Differential privacy is a privacy-preserving technique that adds
noise to the model updates to protect individual user data. By
introducing carefully calibrated noise, differential privacy ensures
that the impact of an individual’s data on the final model is limited,
making it difficult to infer sensitive information from the model.
Mathematically, the differential privacy mechanism can be de-
fined as:

Pr[M (D) ∈ S] ≤ eε · Pr[M (D′ ) ∈ S]

where Pr[M (D) ∈ S] is the probability that the output of the
mechanism M on dataset D lies in set S, ε is the privacy budget
controlling the amount of noise added, and D′ is a neighboring
dataset that differs by at most one data point.

Distributed Training
In federated learning, model training is performed in a distributed
manner across multiple local devices. Each local device trains a
model using its own data while keeping the data on the device.
The trained models are then combined to form a more accurate
global model.

1 Server-Client Communication
The training process in federated learning involves communication
between the server and the client devices. The server sends the

228
global model to the client devices, and the client devices train their
local models using their local data. The updated local models are
then sent back to the server for aggregation.
Mathematically, the client-side model update can be represented
as:

∆wlocal = ϵ∇L(wglobal , Dlocal )

where ∆wlocal is the local model update, wglobal is the global
model, L is the loss function, Dlocal is the local dataset, and ϵ is
the learning rate.

2 Aggregation and Model Update

After receiving the model updates from the client devices, the
server performs aggregation to combine the local model updates
into a new global model.
Mathematically, the aggregation process can be represented as:
N
(i)
X
wglobal ← wglobal − η ∆wlocal
i=1
(i)
where wglobal is the global model, ∆wlocal is the local model
update from device i, N is the total number of devices, and η is
the aggregation learning rate.

3 Model Synchronization
Synchronization of the global model across the client devices is im-
portant to ensure consistency and accuracy. The server distributes
the updated global model to the client devices, and the local models
are synchronized by updating them with the new global model.
Mathematically, the model synchronization process can be rep-
resented as:
(i)
wlocal ← wglobal
(i)
where wlocal is the local model on device i.

229
Applications in Collaborative AI
Federated learning has numerous applications in the domain of
collaborative AI, where multiple users collaborate and contribute
to the improvement of shared models while preserving data privacy.

1 Healthcare
In healthcare, federated learning allows medical institutions to col-
laborate on training models while keeping sensitive patient data
decentralized and secure. Models trained through federated learn-
ing can be used for applications such as disease prediction, drug
discovery, and personalized medicine.

2 Smart Grids
Federated learning can be applied to smart grids, enabling collab-
oration among energy providers to optimize energy consumption
patterns while ensuring privacy. By training models using local
data from different providers, the global model can be used to im-
prove energy efficiency and grid stability.

3 Internet of Things (IoT)

With the proliferation of IoT devices, federated learning can be
used to train models on the edge devices themselves, reducing
communication costs and preserving user privacy. This enables
applications such as activity recognition, anomaly detection, and
predictive maintenance using local sensor data.

Conclusion
In this chapter, we explored the concept of federated learning and
its application in collaborative AI. We discussed privacy concerns
in federated learning and techniques such as secure aggregation
and differential privacy to mitigate them. Furthermore, we exam-
ined the distributed training process, including server-client com-
munication, aggregation, and model synchronization. Lastly, we
highlighted several applications of federated learning in healthcare,
smart grids, and IoT. The next chapter will delve into the ethical
considerations in machine learning.

230
Chapter 53

Quantum Machine
Learning

Quantum Computing Basics

In this chapter, we delve into the field of quantum machine learn-
ing, which combines principles from quantum computing and ma-
chine learning to tackle complex computational problems. Before
diving into quantum machine learning algorithms, it is crucial to
understand the basics of quantum computing.

1 Quantum Bits (Qubits)

In classical computing, bits represent the fundamental units of in-
formation, taking on values of 0 or 1. In quantum computing,
qubits are the counterparts of classical bits. However, qubits can
exist in a superposition of both 0 and 1 states simultaneously,
thanks to the principles of quantum mechanics.
Mathematically, a qubit’s state can be represented as:

|ψ⟩ = α|0⟩ + β|1⟩

where α and β are complex numbers, and |0⟩ and |1⟩ denote
the computational basis states.

231
2 Quantum Gates
Quantum gates are the building blocks of quantum circuits, respon-
sible for performing operations on qubits. Similar to classical logic
gates, quantum gates manipulate the state of qubits to perform
specific computations.
One of the most fundamental quantum gates is the Pauli-X gate,
which operates on a single qubit and performs a bit-flip operation.
The Pauli-X gate transforms the state of a qubit as follows:

0 1 α
    
β
=
1 0 β α
 
α
where represents the state vector of the qubit.
β

# Pauli-X Gate
import numpy as np

X_gate = np.array([[0, 1], [1, 0]])

state_vector = np.array([alpha, beta])

result = np.dot(X_gate, state_vector)

3 Quantum Entanglement
Quantum entanglement is a property in which two or more qubits
become intrinsically connected, regardless of the distance between
them. When qubits are entangled, measuring the state of one qubit
instantaneously determines the state of the other qubit, even if they
are separated by vast distances.
Mathematically, an entangled state of two qubits can be repre-
sented as:
1
|ψ⟩ = √ (|00⟩ + |11⟩)
2
where |00⟩ represents qubit 1 being in the state 0 and qubit 2
being in the state 0, and |11⟩ signifies both qubit 1 and qubit 2
being in the state 1.

232
4 Quantum Algorithms
Quantum algorithms exploit the peculiarities of quantum mechan-
ics to solve computational problems more efficiently than classical
algorithms. One well-known quantum algorithm is Grover’s algo-
rithm, which can search an unsorted database with N elements in
√
N time, compared to the N time required by classical algorithms.
Grover’s algorithm leverages the principles of superposition and
quantum interference to amplify the amplitude of the desired solu-
tion, enabling efficient searching.

# Grover's Algorithm
import numpy as np

def grover_search(database, target):

N = len(database)
amplitude = np.array([1/np.sqrt(N)] * N)
marked_indices = [i for i, value in
,→ enumerate(database) if value == target]
marked_amplitude = [2/np.sqrt(N) if i in
,→ marked_indices else 0 for i in range(N)]
iteration_count = int(np.pi/4 * np.sqrt(N))

for _ in range(iteration_count):
amplitude = amplitude - 2 * np.dot(amplitude,
,→ marked_amplitude) * marked_amplitude

return amplitude

Quantum Algorithms for Machine Learn-

ing
In recent years, researchers have explored leveraging quantum com-
puting to enhance machine learning algorithms. The application
of quantum algorithms in machine learning holds the potential
to solve computationally intensive problems such as optimization,
pattern recognition, and data analysis more efficiently.
One prominent quantum machine learning algorithm is the quan-
tum support vector machine (QSVM). QSVM leverages the quan-
tum computing power to classify data by utilizing quantum kernels,

233
which can potentially outperform classical support vector machines
in specific scenarios.

1 Quantum Kernels
Kernels form a crucial component of support vector machines (SVMs)
and play a significant role in classification tasks. Quantum kernels
extend the concept of classical kernels to operate on quantum data.
A popular quantum kernel is the quantum Gaussian radial basis
function (RBF) kernel, which enables the classification of quantum
data. The quantum RBF kernel measures the similarity between
two quantum states and can be mathematically expressed as:
2
K(|ψ1 ⟩, |ψ2 ⟩) = e−γ|||ψ1 ⟩−|ψ2 ⟩||
where γ is a parameter that determines the width of the kernel.

# Quantum RBF Kernel

import numpy as np

def quantum_rbf_kernel(state_vector_1,
,→ state_vector_2, gamma):
squared_norm =
,→ np.linalg.norm(np.abs(state_vector_1 -
,→ state_vector_2))
kernel_value = np.exp(-gamma * squared_norm)

return kernel_value

2 Quantum Support Vector Machines (QSVM)

QSVM, an extension of classical support vector machines, utilizes
quantum kernels to classify data in a quantum computing frame-
work. QSVM combines the power of quantum computing with the
mathematical foundation of support vector machines to create a
novel approach to classification tasks.
The QSVM algorithm involves training a quantum model using
a quantum kernel, followed by the optimization of hyperparameters
using classical methods. The resulting quantum model can then be
used to classify new data points.

# Quantum Support Vector Machines (QSVM)

import numpy as np

234
def qsvm_training(dataset, labels, quantum_kernel):
# Quantum model training
quantum_data =
,→ convert_to_quantum_representation(dataset)
kernel_matrix =
,→ compute_kernel_matrix(quantum_data,
,→ quantum_data, quantum_kernel)
alpha_vector =
,→ solve_optimization_problem(kernel_matrix,
,→ labels)

return alpha_vector

def qsvm_classification(test_data, quantum_kernel,

,→ alpha_vector):
# Classification using quantum model
quantum_test_data =
,→ convert_to_quantum_representation(test_data)
kernel_vector =
,→ compute_kernel_vector(quantum_test_data,
,→ quantum_data, quantum_kernel)
predicted_labels = apply_threshold(kernel_vector,
,→ alpha_vector)

return predicted_labels

The QSVM algorithm involves training a quantum model using

a quantum kernel.

Potential and Challenges

The integration of quantum computing and machine learning holds
immense potential for solving complex computational problems
more efficiently. Quantum machine learning algorithms can poten-
tially outperform classical algorithms in several domains, including
optimization, data analysis, and pattern recognition.
However, several challenges exist in the practical implementa-
tion of quantum machine learning. One primary challenge is the
limited availability of quantum hardware with sufficient coherence
and computational power to perform complex calculations. Ad-

235
ditionally, the noisiness and susceptibility of quantum systems to
errors pose significant challenges in maintaining the integrity of
quantum computations.
Despite these challenges, ongoing research and advancements
in quantum computing technology continue to pave the way for
the application of quantum machine learning in solving real-world
problems. The field holds promise for revolutionizing the field of
machine learning and enabling the development of novel algorithms
that harness the power of quantum mechanics.

236