Artificial Intelligence

Institute of Technology
University of Gondar
Biomedical Engineering Department

By Ewunate Assaye (MSc.)

 Chapter Three
Machine Learning

Outlines: -
» Machine learning

» Types of learning
o Supervised learning

✓ Decision tree, naïve bayes, SVM, K nearest neighbour

o Unsupervised learning
✓ K means clustering

o Reinforcement learning
✓ Markov decision process
 Machine learning

» It is the field of study that gives computers the ability to learn without being
explicitly programmed (Arthur Samuel, 1959).

» A computer program is said to learn from experience E with respect to some class
of tasks T and performance measure P, if its performance at tasks in T, as
measured by P, improves with experience E (Tom Mitchell, 1999).
o Task T: classifying handwritten digits from images

o Performance measure P : percentage of digits classified correctly

o Training experience E: dataset of digits given classifications

 Machine learning

A technique which enables a

Artificial machine to mimic human
Intelligence(AI) behaviors

Machine
Learning(ML)
A subset of AI technique which uses
statistical methods to enable machine
to improve with experiences
Deep Learning

A subset of ML which makes the

computation of multilayers neural
network feasible
 Machine learning

» Observe set of examples: Training data

» Infer something about process that generated that data

» Use inference to make predictions about previously unseen data: test data.
Machine learning
Machine learning
 Supervised Learning

» It is the machine learning task of learning a function that maps an input to an

output based on example input output pairs.

» It infers a function from labeled training data consisting of a set of training

examples 𝑦 = 𝑓 𝑥 where y is target and x input values

» supervised learning can be

o Classification or

o Regression
 Supervised Learning

Classification
o Predicts categorical class labels (discrete or nominal)
o Classifies data (constructs a model) based on the training set and the values (class
labels) in a classifying attribute and uses it in classifying new data
o Example: classify
✓e-mail as Spam or Non-Spam
✓Bank customers as Fraud or default
✓Patient as Diabetic or Non- diabetic

9
 Supervised Learning

» Example of binary Classification

» Illustration of a binary classification

problem (plus, minus) and two feature
variable (x1 and x2).
 Supervised Learning

Regression/Numeric Prediction
o models continuous-valued functions, i.e., predicts unknown or missing values

» Example:
o predict electric power consumption of a city in Kilo-watt

o Predict house price based on square meter

o Predict sale amount based on advertisement

 Supervised Learning
» How do we predict housing prices
o What do we start with?
✓ Training set (this is your data set)
✓ m = number of training examples(…)
Size Feet2 (x) Price($)in 1000’s(y)
✓ x's = input variables / features
2104 460
✓ y's = output variable "target" variables 1416 232
» With our training set defined - how do we used it? 1534 315
852 178
o Take training set
… …
o Pass into a learning algorithm
o Algorithm outputs a function i.e. y=f(x)
✓ This function takes an input (e.g. size of new house)
✓ Tries to output the estimated value of Y
 Supervised Learning

» Given this data, a friend has a house 750

square feet - how much can they be
expected to get?

» What approaches can we use to solve this?

o Straight line through data
✓ Maybe $150 000
 Supervised Learning

» Second order polynomial

o Maybe $200 000

» Each of these approaches

represent a way of doing
supervised learning
 Unsupervised Learning
» The class labels of training data is unknown

» Given a set of measurements, observations, etc. with the aim of establishing the
existence of classes or clusters in the data
» Used to draw pattern/form cluster x x x
x x x x x x x
x xx x x x x
✓ Given a set of points, with a notion of distance x x x x x x
x x x xx x
between points, group the points into some number of
clusters, so that members of a cluster are in some x x
sense as close to each other as possible. x x x x
x x x
✓ While data points in the same cluster are similar,
x
those in separate clusters are dissimilar to one another
15
 Unsupervised Learning

» The example below demonstrates the clustering of padlocks of same kind. There are
a total of 10 padlocks which various in color, size, shape, etc.

» How many possible clusters of padlocks can be identified?

✓ There are three different kind of padlocks; which can be grouped into three different clusters.

✓ The padlocks of same kind are clustered into a group as shown below

16
 Unsupervised Learning

» Example- customer Segmentation

o A company with a data of past customers
✓ the demographic information

✓ the past transactions with the company

o The company may want to see the distribution of the profile of its customers, to see
what type of customers frequently occur.

o Clustering allocates similar customer in the same group

o Once such groups are found, the company may decide strategies, for example,
services and products, specific to different groups.

17
 Reinforcement learning

» Reinforcement is the process of learning from rewards

while performing a series of actions.
» In reinforcement learning, we do not tell the learner or
agent, for example, a (ro)bot, which action to take but
merely assign a reward to each action and/or the overall
outcome
» Instead of having /correct/false" label for each step, the
learner must discover or learn a behavior that maximizes
the reward for a series of actions

18
 Reinforcement Learning

» The machine learning program should be able to assess the goodness of policies
and learn from past good action sequences to be able to generate a policy i.e. RL

» Example: Game playing

o A good example is game playing where a single move by itself is not that important;

o it is the sequence of right moves that is good.

o A move is good if it is part of a good game playing policy

19
 Application of Machine Learning

Learning Associations:
o Basket analysis- In the case of retail-finding associations between products bought by
customers: If people who buy X typically also buy Y
o going over our data, we calculate that P(chips/beer) = 0.7.
✓ we can define the rule: 70 percent of customers who buy beer also buy chips.

Classification:
o Credit scoring -the bank calculates the risk given the amount of credit and the information
about the customer.
✓ Customer Information:-income, savings, collaterals, profession, age, past financial history, and
so forth
✓ record of past loan- the loan was paid back or not
20
 Application of Machine Learning

» From this data of particular applications, the aim is to infer a general rule coding
the association between a customer’s attributes and his risk

» The machine learning system fits a model to the past data to be able to calculate
the risk for a new application and then decides to accept or refuse it accordingly.

» After training with the past data, a classification rule learned may be of the form

Discriminant rule: IF income > θ1 AND savings > θ2

THEN low-risk
ELSE high-risk
 Application of Machine Learning

22
 Application of Machine Learning

Pattern recognition- recognizing character codes from their images

✓ Handwritten—for example, to read zip codes on envelopes or amounts on checks.

✓ People have different handwriting styles; characters may be written small or large, slanted,
with a pen or pencil, and there are many possible images corresponding to the same
character.

Face recognition
✓ The input is an image, the classes are people to be recognized, and the learning program
should learn to associate the face images to identities.

✓ Difficulty in lighting, glasses may hide the eyes and eyebrows, and a beard may hide the
chin.
23
 Application of Machine Learning
In medical diagnosis
o The inputs are the relevant information we have about the patient and the classes are the
illnesses.
o The inputs contain the patient’s age, gender, past medical history, and current symptoms

In speech recognition-
o the input is acoustic and the classes are words that can be uttered
o This time the association to be learned is from an acoustic signal to a word of some
language
o Different people, because of differences in age, gender, or accent, pronounce the same
word differently, which makes this task rather difficult.
24
 Application of Machine Learning

Biometrics is recognition
o Recognition or authentication of people using their physiological and/or behavioral
characteristics

o Examples of physiological characteristics are images of the face, fingerprint, iris, and
palm; examples of behavioral characteristics are dynamics of signature, voice, gait,
and key stroke

Outlier detection
o Which is finding outlier detection the instances that do not obey the rule and are
exceptions

o For example – fraud

25
 Application of Machine Learning

Handwritten Digit Recognition: isolating a single digit

Email Spam:
✓ try to predict whether the email was junk email, or “spam.” . The objective was to design
an automatic spam detector that could filter out spam before clogging the users’ mailboxes

26
 Application of Machine Learning

Knowledge Extraction:
o Learning a rule from data also allows knowledge extraction.

o The rule is a simple model that explains the data, and looking at this model we
have an explanation about the process underlying the data.

o For example, once we learn the discriminant separating low-risk and high-risk
customers, we have the knowledge of the properties of low-risk customers.

o We can then use this information to target potential low-risk customers more
efficiently, for example, through advertising.

27
 Application of Machine Learning

Regression
» Predict the price of a used car
o Inputs are the car attributes—brand, year, engine capacity, mileage, and other
information—that we believe affect a car’s worth.
o The output is the price of the car. Such problems where the output is a number are
regression problems

» A machine that roasts coffee

o The machine has many inputs that affect the quality: various settings of temperatures,
times, coffee bean type, and so forth.
o We measure the quality of the coffee, for example, as consumer satisfaction.
28
 Application of Machine Learning

» To find the optimal setting, we fit a regression model linking these inputs to coffee
quality and choose new points to sample near the optimum of the current model to
look for a better configuration.

Document clustering
o the aim is to group similar documents

o For example, news reports can be subdivided as those related to politics, sports,
fashion, arts, and so on

o Documents are then grouped depending on the number of shared words.

 Steps to apply machine learning on your data…

» Evaluating model performance: Because each machine

learning model results in a biased solution to the learning
problem, it is important to evaluate how well the algorithm
learned from its experience

» Improving model performance

30
 5 Steps for Approaching a ML

1. Define the problem to be solved.

2. Collect data.

3. Choose an algorithm class.

4. Choose an optimization metric for learning the model.

5. Choose a metric for evaluating the model.

31
 Steps to apply machine learning on your data:

o Any machine learning task can be broken down into a series of more manageable steps.
✓ Collecting data: Whether the data is written on paper, recorded in text files and spreadsheets,
or stored in an SQL database, you will need to gather it in an electronic format suitable for
analysis.

✓ Exploring and preparing the data: The quality of any machine learning project is based
largely on the quality of data it uses. An often cited statistic suggests that 80 percent of the
effort in machine learning is devoted to data.

✓ Training a model on the data: The specific machine learning task will inform the selection of
an appropriate algorithm, and the algorithm will represent the data in the form of a model.

32
 Quiz 2

1. List the types of machine learning and give an example to each of them.