CS446: Machine Learning

Lecture 01-02 (Course Introduction & Road Map)

Instructor:
Dr. Muhammad Kabir
Assistant Professor
[email protected]

School of Systems and Technology

Department of Computer Science
University of Management and Technology, Lahore
 Administrative Information
 Office
Level – 3, CB1-507 (Cohort Building 1) , Cubicle-3 (On the right side)
 Email
[email protected]
 Counselling Hours (BS-Students)
Every Tuesday (11:00 – 13:30)
 Prior appointment is mandatory before coming to the OFFICE
 Appointment via Email
 Moodle Website: https://lms.umt.edu.pk/course/view.php?id=14933
 Class Schedule: V2 – Wednesday and Thursday (14:00 – 15:15)
 Make Sessions – Discussion (Most Probably on Saturdays)
 About Instructor - Background
 PhD-Computer Science and Technology
Nanjing University of Science and Technology, Nanjing, China
(2016-2020)
 MS-Computer Science
Abdul Wali Khan University Mardan (2013-2016)
 BS-Computer Science
Islamia College Peshawar (2008-2012)
 Total Publications (26) – Journals (24) & Conferences (2)
 Research Interests
Bioinformatics
Machine Learning
Deep Learning
Bio-imaging
 For more details (https://mdkabir1.github.io/webpage/) – Not updated
 Google Scholar
(https://scholar.google.com/citations?user=YCjWxh4AAAAJ&hl=en)
 Classroom Etiquettes
 Bring your own Notebook/Register to make notes for yourself.
 Slides will be shared on LMS after LECTURE. You can download it from LMS.
 I encourage class participation
 Mobile phones – Silent or switch off
 80% Attendance (No Relaxation)
 Arrive on time in class (If you come late, missed topics will not be repeated for you).
 If you do not understand a point, raise your hand and ask me to explain or contact during
office hours.
 Prior appointment is mandatory for coming to my office during counselling hours.
 No messages or phone call on personal mobile number.
 Saturday & Sunday if OFF, so no one will be entertained on OFF days.
 No disturbance!!!! No Misconduct!!!!
 REMEMBER: Your first priority must be your studies
 Grading Policy – Tentative
 20% Programming Assignments and
Homeworks

- 4 Programming Assignments

- 4 Homeworks
 10% Quizzes
- Tentatively 5 Quizzes

 45% Exams
- One midterm (15%)
- Final (30%)

 10% Project
 10% participation + Attendance
 5% Viva (End of Semester)
 Grading Policy – Tentative
 Course
CS446 – Machine Learning
 Homework Late Policy
- 10% per day for 5 days. No submission after 5 days (120 hours)
 Missed Quiz Policy
- No make-up for quiz
 Plagiarism will be strictly dealt with as per university policies (take it seriously).
 Zero Tolerance for Plagiarism and Cheating.
 What is this course about?
Introductory course in Machine Learning (ML) – Fundamental topics in
- Supervised learning
- Unsupervised learning

Course Objectives:
• To provide a thorough introduction to ML methods
• To build mathematical foundations of ML and provide an appreciation
for its applications
• To provide experience in the implementation and evaluation of ML
algorithms
• To develop research interest in the theory and application of ML
 Is this course a right choice for you?

Undergraduate students
- Interested in pursuing AI, Deep Learning and/or Machine Learning in their
graduate school
- Interesting in pursuing a professional career focused on the development of
Machine Learning solutions

Graduate or FYP students

- Want to do fundamental research in the area of Machine Learning
- Wish to apply Machine Learning in their research work
 Course Prerequisites

► Undergraduate students
- Linear Algebra
- Probability
- Programming

► We expect all the students to have good programming skills (in

C/Python/MATLAB).
► We will use Python for class, assignments, projects etc.
 Modules
Course Overview, notation
Artificial Intelligence vs Machine Learning vs Deep Learning
Supervised Learning Setup
1- ML
Types of Machine Learning: Supervised, Unsupervised & Reinforcement
Overview
Learning

Weeks: 1,2

Components:
• Programming Assignment 1: Intro to Python, Setting up
Environment
 Modules

Scalars, Vectors, Matrices and Tensors

Multiplying Matrices and Vectors
2- Identity and Inverse Matrices
Linear
Eigen decomposition
Algebra
Singular value decomposition
Weeks: 3,4

Components:
• Programming Assignment 1: Intro to Python, Setting up
Environment
 Modules
Classification

KNN

3- Evaluation Metrics, Curse of Dimensionality

Classification Multi-class Classification

Weeks: 4,5

Components:
• Programming Assignment 2: KNN based (Using Images)
• Homework 1A
 Modules
Linear Regression
Gradient Descent
Multi-variate Regression
4-
Regression Polynomial Regression

Bias-Variance Trade-off, Regularization

Weeks: 6,7

Components:
• Programming Assignment 3: Regression
• Homework 1B
 Modules
Logistic Regression

5-
Logistic
Regression

Weeks: 7

Components:
• Programming Assignment 4: Logistic Regression
 Modules
Bayes Theorem
Naive Bayes Classification

6 – Bayesian
Framework

Weeks: 8,9 (Mid-Exam)

Components:
• Programming Assignment 5: Naïve Bayes
Classifier (may be merged with Assignment 4)
• Homework 2
 Modules
Perceptron Algorithm

SVM
7–
Perceptron, Neural Networks
SVM and
Neural
Network

Weeks: 10,11,12,13

Components:
• Programming Assignment 6: Neural Networks
• Homework 3
 Modules
Unsupervised Learning Overview

Clustering (k-means)
8–
Clustering

Weeks: 14,15

Components:
• Homework 3
 Suggested Reference Books

• Pattern Recognition and Machine Learning, Christopher M. Bishop

• Machine Learning: a Probabilistic Perspective, Kevin Murphy
• Machine Learning, Tom Mitchell
• The Elements of Statistical Learning: Data mining, Inference, and Prediction, by Hastie,
Tibshirani, Friedman
 Main Communication Channel
MOODLE
https://lms.umt.edu.pk/course/view.php?id=14933
“As to methods, there may be a million and then
some, but principles are few. The man who
grasps principles can successfully select his own
methods.”
Ralph Waldo Emerson
 Learning Agents
The agent adapts its action(s) based on feedback (not only sensors).
Machine Learning at a Glance
 Traditional Programming Vs ML

- Machine Learning: Automating the process of automation

- Getting computers to program themselves

Given examples (training data), make a machine learn system

behavior or discover patterns
 The difference between AI and ML
- Artificial intelligence and machine learning are very closely related and
connected.

- Artificial intelligence Artificial intelligence is the capability of a

computer system to mimic human cognitive functions such as learning
and problem-solving.

- Machine learning is an application of AI. It’s the process of using

mathematical models of data to help a computer learn without direct
instruction. This enables a computer system to continue learning and
improving on its own, based on experience.
Machine Learning – A definition
 Capabilities of AI and ML
- Companies in almost every industry are discovering new opportunities
through the connection between AI and machine learning. These are just a
few capabilities that have become valuable in helping companies transform
their processes and products:
- Predictive analytics
- Speech recognition and natural language understanding
- Sentiment analysis
- Recommendation engines
- Image and video processing
- Bioinformatics

https://azure.microsoft.com/en-us/solutions/ai/artificial-intelligence-vs-machine-
learning/#benefits
Machine Learning Overview - Example
Classical Example: Recognize hand-written 2!
 Machine Learning: Overview
Example Applications
► Medical Diagnosis
► Autonomous Driving
► Information extraction
► Computer/Machine Vision
► Finance
► Web Search
► Robotics
► Social networks
► Production Industry
► Logistics
► Waste Management

• Your research/favorite area

 Machine Learning: Overview
What is Machine Learning?
- Study, Design and Analysis of algorithms that improve their
performance at some task with experience

Data
Model

Algorithm f x
Final output which
enables us to make
Given to us We need to design it predictions
 Machine Learning: Overview
Nature of ML Problems
1. Supervised Learning
The learning algorithm would receive a set of inputs along with the
corresponding correct outputs to train a model
Training Data

All labeled Model Prediction

data

Classification: Discrete Prediction Regression: Quantitative Prediction on a continuous scale

Given a data sample, predict its class Given a data sample, predict a numerical value
 Machine Learning: Overview
Nature of ML Problems
2. Unsupervised Learning
The learning algorithm would receive unlabeled raw data to train a model and
to find patterns in the data
Training Data
All
unlabeled Model Clustering
data
 Machine Learning: Overview
Nature of ML Problems
3. Semi-supervised Learning
• The learning algorithm receives labeled and unlabeled raw data to train a model
• Main objective is to efficiently accommodate the unlabeled data

Training Data
Labeled
+ Data Modeling and
Model Augmentation
unlabeled
Data

Anomaly -> ignore it

Use labeled data to Place unlabeled data Use the model to label Fit the model again
build a model with model the unlabeled data with the combined data
 Machine Learning: Overview
Typical Flow

Problem Nature Training Data

Model Training
Identification Collection

f x

Model Model
Deployment Validation
Machine Learning: Overview
General Steps in Designing a ML Model
Machine Learning: Overview
Steps in Programming
 Examples - Machine Learning: Overview
Training Data Collection

x Process or System y
Input
Observed Output

PROCESS or SYSTEM : Underlying physical or logical phenomenon which maps our input data to our observed output

Collect the training data by observing our unknown PROCESS or SYSTEM

 Machine Learning: Overview
Example Systems

• Previous Sales
• Prices Future
Process or System
• Inflation sales
• Pandemic

Image Process or System Object detection

Or recognition
 Machine Learning: Overview
Example Systems

Market
Consumer segmentation based
Transaction Process or System on consumers’
Data spending patterns

• Income
• Credit History
Process or System Loan Approval
• Employment
• Marital Status
 Machine Learning: Overview
Data Types
Tabulated Data
• Each column is a feature and adds one dimension to the data
• Number of columns define total number of features and hence data dimensionality
• Ideally, these columns should contain different information -> uncorrelated
 Machine Learning: Overview
Data Types

Feature 2 Feature 2

Feature 1 Feature 1
Features are correlated Features are uncorrelated

Find all those dimensions (features) which carry very less information and discard them
 Machine Learning: Overview
Data Types

Images

• Each image (greyscale) is a 2D

data which can be represented as
a matrix
 Image Descriptors: Manually extract features Deep Learning: Let machine learning extract most important features
• Speeded-Up Robust Features (SURF)
• Binary Robust Independent Elementary Features (BRIEF) Neural Networks such as Convolutional Neural Networks
• Oriented FAST and Rotated BRIEF (ORB) (CNN) take raw images as input and first process it inside
• Scale-Invariant Feature Transform (SIFT) multiple layers to get best features called intrinsic
• Histogram of Oriented Gradients (HOG) dimension of the data
• HAAR features
• etc.
 Machine Learning: Overview
Data Types
Time Series

• Much difficult to deal with

o Enormous Size
o Cumbersome Labeling
o Not so standard feature extraction
strategies as in case of images.
(Direct modeling of time series is
preferred)
 Time series Models: Manually model it ! Machine Learning: Let machine learning handle the task !
• Exponential Smoothing (ETS)
• Autoregressive Integrated Moving Average
(ARIMA) models Neural Networks such as
• Linear regression Convolutional Neural
• Generalized Autoregressive Conditional Networks (CNN) take raw
Heteroskedasticity (GARCH) waveforms as input and
• Bayesian Models first process it inside
• Vector auto-regression (VAR) models multiple layers to get best
• etc. predictive model for it
 Machine Learning: Overview
Data Types

Heterogeneous data

Multimodal Machine Learning !

+

9/29/2022
 Chapter Reading

Chapter 1
Machine Learning - A Probabilistic Perspective
by
Kevin P. Murphy