Machine Learning Algorithms 2018 Contents
Machine Learning Algorithms 2018 Contents
0. General 1
1. Introduction 6
1.1 About machine learning: Example 1 7
1.2 Tasks: about problems that can be solved with machine
learning 21
1.3 Variables in machine learning 36
2. On classification and related tasks 46
2.1 Binary classification 49
2.2 Beyond binary classification 69
3. Probabilistic models 78
3.1 Turning data into probabilities 78
3.2 Minimising risk 92
3.3 Naive Bayes classifier 100
3.4 Some useful statistical formulas 108
3.5 The Gaussian 117
4. Nearest neighbor searching 127
4.1 Foundation of nearest neighbors 127
4.2. KD-trees 139
4.3 Weighted nearest neighbor searching 149
5. Discriminant analysis 157
5.1 Quadratic discriminant analysis 163
5.2 Linear discriminant analysis 171
5.3 Logistic discrimination 188
6. Support vector machines 195
6.1 Linearly separable data (classes) 196
6.2 Nonseparable classes 208
6.3 Multiclass classification 213
7. Decision or classification trees 224
7.1 Construction of decision trees 228
7.2 ID3 232
7.3 Dealing with continuous variables 246
7.4 Computational complexity 251
7.5 Classification and regression trees (CART) 252
8. Ensemble learning 263
8.1 Boosting 268
8.2 Bootstrap 276
8.3 Bagging 281
8.4 Random forests 282
9. Clustering 286
9.1 Partition-based or objective function-based clustering 288
9.2 Hierarchical clustering 293
10. Evolutionary learning 309
10.1. The genetic algorithm (GA) 310
10.2 Generating offspring: genetic operators 318
10.3 The basic genetic algorithm 325