Week 11: Programming for Data Analysis
0 likes•198 views
The document discusses binary classification techniques including Naive Bayes classifiers and support vector machines (SVM). It explains that Naive Bayes classifiers use Bayes' theorem to calculate conditional probabilities for classification. Specifically, it describes how to calculate priors and fit probability distributions like binomial, multinomial, and Gaussian distributions for the Naive Bayes model. The document also outlines how SVMs find the optimal separating hyperplane between two classes by maximizing the margin between them. It provides code examples for implementing Naive Bayes and SVM classifiers and evaluating their performance.
Week 11: Programming for Data Analysis
- 1. Programming for Data Analysis Week 11 Dr. Ferdin Joe John Joseph Faculty of Information Technology Thai – Nichi Institute of Technology, Bangkok
- 2. Today’s lesson Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 2 • Binary Classification • Naïve Bayes Classifier • Support Vector Machine
- 3. Naïve Bayes Classifier • Conditional Probability Model of Classification Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 3
- 4. Conditional Probability Model of Classification • The conditional probability can be calculated using the joint probability, although it would be intractable. • Bayes Theorem provides a principled way for calculating the conditional probability. Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 4
- 5. Bayes Theorem • P(A|B) = P(B|A) * P(A) / P(B) • We can frame classification as a conditional classification problem with Bayes Theorem as follows: P(yi | x1, x2, …, xn) = P(x1, x2, …, xn | yi) * P(yi) / P(x1, x2, …, xn) Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 5
- 6. Naïve Bayes • For simplifying the calculation • The Bayes Theorem assumes that each input variable is dependent upon all other variables. • This is a cause of complexity in the calculation. Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 6
- 7. Calculation of Prior and Conditional Probabilities • P(yi) = examples with yi / total examples • In the case of categorical variables, such as counts or labels, a multinomial distribution can be used. • If the variables are binary, such as yes/no or true/false, a binomial distribution can be used. • If a variable is numerical, such as a measurement, often a Gaussian distribution is used. Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 7
- 8. Naïve Bayes Distribution • Binomial Naïve Bayes: Naïve Bayes that uses a binomial distribution. • Multinomial Naïve Bayes: Naïve Bayes that uses a multinomial distribution. • Gaussian Naïve Bayes: Naïve Bayes that uses a Gaussian distribution. Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 8
- 9. Preparation of dataset Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 9
- 10. Fitting Probability Distribution Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 10
- 11. Sorting Data Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 11
- 12. Calculate Priors Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 12
- 13. Fit Distribution Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 13
- 14. Summary Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 14
- 15. Independent Conditional Probability Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 15
- 16. Taking a sample Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 16
- 17. Probability Scores Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 17
- 18. Wrap Up Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 18
- 19. Gaussian Naïve Bayes Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 19
- 20. Lab Exercise • Use this source code and make a classification report which gives accuracy, precision, recall and F1-score Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 20
- 21. Support Vector Machine Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 21
- 22. SVM Objective • A training set, S, for an SVM is comprised of m samples. • The features, x, consist of real numbers and the classifications, y, must be -1 or 1. Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 22
- 23. SVM Hyperplane • The SVM hyperlane is defined by the weight vector, w, and the bias, b, and is defined as: Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 23
- 24. Example for 2 Feature • Hyperplane for two features can be written as: Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 24
- 25. Libraries Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 25
- 26. Data Clusters Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 26
- 27. Data Clusters Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 27
- 28. Prepare Datasets Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 28
- 29. Learning Rate Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 29
- 30. Learning Rate Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 30
- 31. Draw Decision Boundary Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 31
- 32. Plot Data Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 32
- 33. Plot Data Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 33
- 34. Test Classifier Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 34
- 35. Test Classifier Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 35
- 36. Output Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 36
- 37. Lab Exercise • Create Confusion Matrix and classification report for SVM Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 37
- 38. DSA 207 – Binary Classifier Faculty of Information Technology, Thai - Nichi Institute of Technology, Bangkok 38