Learning Based Vision

Fundamentals of Computer Vision – Week 2

Assist. Prof. Özge Öztimur Karadağ
ALKÜ – Department of Computer Engineering
Alanya
Learning
• Learning is the acquisition of knowledge about the world.
Kupfermann 1985

• Learning is an adaptive change in behavior caused by experience.

Shepherd 1988
Machine Learning
• [Arthur Samuel, 1959]
• Field of study that gives computers
the ability to learn without being explicitly programmed

• [Kevin Murphy] algorithms that

• automatically detect patterns in data
• use the uncovered patterns to predict future data or other outcomes of interest

• [Tom Mitchell] algorithms that

• improve their performance (P)
• at some task (T)
• with experience (E)
Learning
• A computer program is said to learn from experience E with respect to
some class of tasks T and performance measure P, if its performance
at tasks in T, as measured by P , improves with experience E.
• What is a task?
• Machine learning tasks are usually described in terms of how the machine
learning system should process an example.
• An example is a collection of quantitatively measured features, represented
as 𝑥 ∈ ℝ𝑛 , where each entry 𝑥𝑖 of the vector is a feature. Example, the
features of an image are usually the values of the pixels in the image.
Learning
• The most common machine learning tasks:
• Classification:
• In this type of task, the computer program is asked to specify which of k
categories some input belongs to. To solve this task, the learning algorithm is
usually asked to produce a function 𝑓: ℝ𝑛 → 1, … , 𝑘
• When y=f(x), the model assigns an input described by x to a category
indentified by numeric code y.
Learning
• The most common machine learning tasks:
• Regression:
• In this type of task, the computer program is asked to predict a numerical
value given some input. To solve this task, the learning algorithm is asked to
output a function 𝑓: ℝ𝑛 → ℝ.
• Similar to classification but the format of output is different.
 A computer program is said to learn from experience E with
Learning respect to some class of tasks T and performance measure P, if
its performance at tasks in T, as measured by P , improves with
experience E.

• Performance Measure
• To evaluate the abilities of a machine learning algorithm, we must design a
quantitative measure of its performance. Usually this performance measure P
is specific to the task T being carried out by the system.
• For classification we usually measure the accuracy of the model.
• Accuracy is the proportion of examples for which the model produces the correct output.
• We can also measure the error rate.
• Error rate is the proportion of examples for which the model produces an incorrect
output.
 A computer program is said to learn from experience E with
Learning respect to some class of tasks T and performance measure P, if
its performance at tasks in T, as measured by P , improves with
experience E.

• The Experience:
• Machine learning algorithms can be categorized as Supervised and
Unsupervised by what kind of experience they are allowed to have during the
learning process.
• Learning algorithms are allowed to experience a dataset.
• A dataset is a collection of many samples.
• Ex: Iris dataset.
• 150 iris plants with measurements of different parts.
• Features: sepal length, sepal width, petal length, petal width
• The specie of each plant is recorded (3 different species)
 A computer program is said to learn from experience E with
Learning respect to some class of tasks T and performance measure P, if
its performance at tasks in T, as measured by P , improves with
experience E.

• The Experience:
• Unsupervised Learning: experience a dataset containing many features, then learn useful
properties of the structure of this dataset. Ex: clustering, dividing the dataset into
clusters of similar examples.

• Supervised Learning: experience a dataset containing features, but each example is also
associated with a label or target. For example, the Iris dataset is annotated with the
species of each iris plant. A supervised learning algorithm can study the Iris dataset and
learn to classify iris plants into three different species based on their measurements.
 A computer program is said to learn from experience E with
Learning respect to some class of tasks T and performance measure P, if
its performance at tasks in T, as measured by P , improves with
experience E.

• The Experience:
• Unsupervised learning involves observing several examples of a random vector x and
attempting to implicitly or explicitly learn the probability distribution p(x), or some
interesting properties of that distribution;
• Supervised learning involves observing several examples of a random vector x and an
associated value or vector y, then learning to predict y from x, usually by estimating
p(y | x).
• The term supervised learning originates from the view of the target y being provided
by an instructor or teacher who shows the machine learning system what to do. In
unsupervised learning, there is no instructor or teacher, and the algorithm must
learn to make sense of the data without this guide.
 A computer program is said to learn from experience E with
Learning respect to some class of tasks T and performance measure P, if
its performance at tasks in T, as measured by P , improves with
experience E.

• The Experience – Representing Data

• Design Matrix: common way of describing a dataset. Each row of the matrix is
a different example. Each column of the matrix is a different feature.
• Ex: İris dataset
• 150 examples with four features of each example
• A design matrix of 𝑋 ∈ ℝ150𝑥4 where 𝑋𝑖,1 is the sepal length of plant i, 𝑋𝑖,2 is the sepal width
of plant i etc.

• In case of supervised learning, the example also contains a label or target.

• When working with a dataset with design matrix X, we often provide a vector of labels 𝑦,
with 𝑦𝑖 providing the label for example 𝑖.
Ex: Linear Regression
• Build a system that can take a vector 𝑥𝜖ℝ𝑛 as input and predict the the value of a
scalar y𝜖ℝ as its output.
• The output of the linear regression is a linear function of the input.
• Let 𝑦ො be the value that our model predicts 𝑦 should take on.
• We define the output to be
𝑦ො = 𝑤 𝑇 𝑥
Where y𝜖ℝ𝑛 is a vector of parameters.
• W is a set of weights that determine how each feature affects the prediction.
• positive weight  increasing the value of that feature increases the value of our prediction,
• negative weight increasing the value of that feature decreases the value of our prediction,
• If a feature’s weight is large in magnitude, then it has a large effect on the prediction.
• If a feature’s weight is zero, it has no effect on the prediction.
Ex: Linear Regression
Task: to predict 𝑦 from 𝑥 by outputting 𝑦ො = 𝑤 𝑇 𝑥

• Performance ?
• Test set: m example inputs that will not be used for training
• Design matrix 𝑋 𝑡𝑒𝑠𝑡
• Vector of regression targets: 𝑦 𝑡𝑒𝑠𝑡
• How to measure the performance?
• Mean Squared Error
1 2
• 𝑀𝑆𝐸𝑡𝑒𝑠𝑡 = 𝑚 σ𝑖(𝑦ො 𝑡𝑒𝑠𝑡 − 𝑦 𝑡𝑒𝑠𝑡 )𝑖
Ex: Regression
• How to improve the performance for the regression task?
• Instead of linear regression employ polynomial regression.
• Polynomial regression is a form of Linear regression known as a special case of Multiple
linear regression which estimates the relationship as an nth degree polynomial.

• y=ax+b  y=a0+a1x1+a2x2+a3x3+⋯  y=a0+a1x+a2x2+a3x3+⋯

(straight line fit; a:slope, b:intercept) (multidimensional linear model) (polynomial regression)
Regression example with python
• Color estimation by regression
References
• https://colab.research.google.com/github/jakevdp/PythonDataScienc
eHandbook/blob/master/notebooks/05.06-Linear-
Regression.ipynb#scrollTo=gb5CKXcD2i8a