Machine Learning

Introduction to Machine Learning

Slides adapted from material created by E. Alpaydin

Prof. Mordohai, Prof. Greenstadt, Pattern Classification (2nd Ed.),
Pattern Recognition and Machine Learning
12/26/2024 Matt Burlick, Drexel University 1
Objectives
• Understand common machine learning problems
• Understand basic ML terminology

12/26/2024 Matt Burlick, Drexel University 2

What is Machine Learning?
• Definition: “The study of computer algorithms that improve
automatically through experience”
• Formally:
• Improve at task 𝑇𝑇
• With respect to performance measure 𝑃𝑃
• Based on experience 𝐸𝐸
• Example: Recognize a Person
• 𝑇𝑇: recognize a person
• 𝑃𝑃: number of time we recognized a person correctly
• 𝐸𝐸: a database of labeled faces

12/26/2024 Matt Burlick, Drexel University 3

ML vs AI
• How is this different than AI?
• ML can be thought of as a sub-topic within AI
• AI deals with any “intelligent” task
performed by a non-human agent
• Often “path finding algorithms”
• ML specifically deals with making decisions
based on acquired data
• Both past and current

12/26/2024 Matt Burlick, Drexel University 4

Why do we care?
• It’s everywhere!!!

12/26/2024 Matt Burlick, Drexel University 5

Example: Photograph or Not?

12/26/2024 Matt Burlick, Drexel University 6

Example: Character Recognition

12/26/2024 Matt Burlick, Drexel University 7

Example: Speech Understanding

12/26/2024 Matt Burlick, Drexel University 8

How Do We Do It?
• We need to think about
1. What needs to be learned?
• What’s our task/goal?
2. What feedback can we get and in what form?
• Supervised learning (correct answers for each example)
• Unsupervised learning (correct answers not given. Not covered in this class)
3. What representation should we use (features)?

12/26/2024 Matt Burlick, Drexel University 9

Problems in ML
• There are two types of machine learning problems that we’ll tackle in the course
• Regression
• Given some data, can we predict an outcome value?
• Example: We have a car’s brand, year, mpgs and want to figure out its worth
• This is an example of supervised learning
• To build our prediction system, we have data with labels.

12/26/2024 Matt Burlick, Drexel University 10

Problems in ML
• Classification
• Given data, can we predict which category something belongs to.
• Typically involves learning some rules.
• This is also an example of supervised learning
• To build our prediction system, we have data with labels.

12/26/2024 Matt Burlick, Drexel University 11

Course Objectives
• Foundations of Machine Learning
• Regression
• Classification
• Applications of Machine Learning algorithms
• Implementation and use of Machine Learning algorithms
• Note: This course should probably be called something like
“Foundations of Traditional Machine Learning”

12/26/2024 Matt Burlick, Drexel University 12

CCI AI Course Offerings

Computer Science Information Science/Data Science

• CS 380/510 – Artificial Intelligence • DSCI 471 – Applied Deep Learning
• CS 383/613 – Machine Learning • DSCI 631 - Applied Machine Learning for Data
Science
• CS 387/611 – Game AI Development
• DSCI 691 – Natural Language Processing with Deep
• CS 481/610 – Advanced Artificial Intelligence
Learning
• CS 482/589 – Robust Machine Learning
• INFO 629 – Applied Artificial Intelligence
• CS 486/770 – Topics in Artificial Intelligence
• INFO 692 – Explainable Artificial Intelligence
• CS 614 – Applications of Machine Learning
• INFO 693 – Human-Artificial Intelligence
• CS 615 – Deep Learning Interaction
• CS 616 – Robust Deep Learning
• CS 617 – Reinforcement Learning

12/26/2024 Matt Burlick, Drexel University 13

Administrative Stuff…

12/26/2024 Matt Burlick, Drexel University 14

Contacts

Instructor Teacher Assistants

• Professor Matt Burlick: • Armando Parra
• Email: [email protected] • Email: ajp434drexel.edu
• Office Hours:
• Office: • Mondays 12:00pm-02:00pm (CLC)
• 3675 Market St., Room 925
• Office Hours: • Sing Zheng
• Wednesdays 02:00pm – 04:00pm (Hybrid) • Email: [email protected]
• Thursdays 03:00pm – 05:00pm (Hybrid) • Office Hours:
• Fridays 06:00pm – 08:00pm (CLC)

12/26/2024 Matt Burlick, Drexel University 15

Contacts
• Questions are to be asked during class or office hours.
• Time is allocated for questions at both the beginning and end of class time.
• Outside of logistical questions, questions will not be answered over
email or discord.
• Discord will be used for student-led discussions (although monitored by
faculty) and for announcements.

12/26/2024 Matt Burlick, Drexel University 16

Pre-Requisites
• CS 260 [Min Grade: C] – Data Structures and Algorithms
• MATH 201 [Min Grade: C] or ENGR 231 [Min Grade: D] – Linear Algebra
• MATH 221 [Min Grade: C] or MATH 222 [Min Grade: C] – Discrete Math
• MATH 311 [Min Grade: C] or MATH 410 [Min Grade: C] or ECE 361 [Min
Grade: D] – Prob and Stats
• The idea is that you should be a proficient programmer such that you
can pick up a new language “on the fly” and use it as a tool.
• You should also be comfortable with linear algebra, probability,
statistics, and calculus.

12/26/2024 Matt Burlick, Drexel University 18

Course Resources
• Official Textbook:
• None
• Recommended Textbooks:
• Basic: An Introduction to Statistical Learning (free PDF), Gareth James, et. al.
• Medium: Introduction to Machine Learning (Alpaydin)
• Advanced: Machine Learning (Murphy)
• Blackboard for lecture material, labs and assignments
• Discord Channel
CS838-Wi25
• Use this as your first place to pose questions
• Hopefully not just I can help
• But don’t post code.

12/26/2024 Matt Burlick, Drexel University 19

Course Software
• Programming Environment: Python 3.x w/
• NumPy
• MatplotLib
• Opencv-python
• Pandas
• Pillow

12/26/2024 Matt Burlick, Drexel University 20

Course Software
• Typesetting Environment:
• LaTeX, MS Word w/ Equation Editor (or similar)
• If you opt to do LaTeX, unless you already have a LaTeX environment set up, I
recommend using an online LaTeX typesetter, ala www.overleaf.com

12/26/2024 Matt Burlick, Drexel University 21

Grade Breakdown
• Labs 30%
• Homework Assignments 20%
• Exams 50%

Points Grade Points Grade Points Grade

[97-100] A+ [83-98) B [70-73) C-
[93-97) A [80-83) B- [67-70) D+
[90-93) A- [77-80) C+ [60-67) D
[87-90) B+ [73-77) C [0-60) F

12/26/2024 Matt Burlick Drexel University 22

Labs
• Most weeks there will be a lab conducted during the second lecture.
• Lab attendance is required
• But you can miss up to two without penalty.
• Subsequent will result in zeros.
• You may work with another student if you like.
• Labs are due the day after they are administered.
• Your lowest lab grade will be dropped.

12/26/2024 Matt Burlick Drexel University 23

Assignments
• Most weeks there will be a homework assignment.
• These focus on the theory and math.
• These are to be typeset via LaTeX or Microsoft Equation Editor (or similar) and
converted to PDF.
• These are due Monday evening.
• No late submissions will be accepted (even a minute late!)
• However, I will also drop your lowest HW grade.

12/26/2024 Matt Burlick Drexel University 24

Exams
• There will be a midterm and final exam.
• These will be similar in style to the questions in the assignments.

12/26/2024 Matt Burlick Drexel University 25

Additional Course Policies
• Assignments and Exams are to be done individually unless otherwise
noted
• While you are encouraged to use a versioning system like github or
bitbucket, please make your work for this course private.
• Any dispute about an assignment grade must be formally made (email)
and resolved within 5 days of receiving your grade. After this period
your grade cannot be adjusted.

12/26/2024 Matt Burlick, Drexel University 27

Notation/Mathematics/Matlab
• I have placed on Blackboard a number of resources:
• Course Notation – There will be a lot of symbols used in this course. This document
tries to give you an overview of them.
• Similarity and Distance Functions – Often we will need to compute the distance
and/or similarity between observations. This document includes several commonly
used ones.
• Python Functions – Here’s a list of most of the Python functions I used in developing
this course.
• Math Review – A quick review of the most critical math needed for this course.
Including..
• Calculus
• Linear Algebra
• Probability and Statistics
• Math Reference Sheet – This is a quick reference sheet that we’ll use often when
doing derivations and whatnot.

12/26/2024 Matt Burlick, Drexel University 28

Fundamental ML Concepts

12/26/2024 Matt Burlick, Drexel University 29

 ML Overview
ML

Supervised
Unsupervised
𝑋𝑋𝑖𝑖 , 𝑌𝑌𝑖𝑖 𝑁𝑁
𝑖𝑖=1 𝑋𝑋𝑖𝑖 𝑁𝑁
𝑖𝑖=1

• We can basically break machine learning tasks into two categories

1. Supervised Learning
2. Unsupervised Learning
• Supervised learning
• Data 𝑋𝑋𝑖𝑖 and correct answer (label) 𝑌𝑌𝑖𝑖 given for each example 𝑖𝑖 ∈ 1, … , 𝑁𝑁
• Unsupervised learning
• Only data given for each example, 𝑋𝑋𝑖𝑖
• Again, we’ll be just focusing on supervised learning.
12/26/2024 Matt Burlick, Drexel University 30
Types of Problems
ML

Supervised Unsupervised
𝑋𝑋𝑖𝑖 , 𝑌𝑌𝑖𝑖 𝑁𝑁
𝑖𝑖=1 𝑋𝑋𝑖𝑖 𝑁𝑁
𝑖𝑖=1

Classification Regression

12/26/2024 Matt Burlick, Drexel University 31

Types of Data
Data

Categorical-Nominal Categorical-Ordinal Continuous Valued

• Each piece of information pertaining to an observation can fall into one of three categories:
• Continuous Valued
• Examples: Blood Pressure, Height
• Categorical-Nominal (unordered)
• Examples: Car Model, School
• Categorical-Ordinal (can be ordered)
• Examples: Colors, small < medium < large

12/26/2024 Matt Burlick, Drexel University 32

No Free Lunch Theorem
• Unfortunately, there’s no single machine learning algorithm to rule
them all 
• Hopefully, the nature of the problem and data will guide us towards
some subset of the options.
• We then try them out and select the best.

12/26/2024 Matt Burlick, Drexel University 33

ML Algorithms
• Here’s a list of algorithms we’ll look at in the class and what types they are
1. Principal Component Analysis (PCA)
2. Linear Regression
3. Classification
a. Binary
a. Logistic Regression
b. Support Vector Machines (SVMs)
b. Multi-Class
a. Decision Trees (DTs)
b. Nearest Neighbors (KNN)
c. Statistical Classification
d. Markov Models

12/26/2024 Matt Burlick, Drexel University 34