MSE491: Application of Machine Learning in

Mechatronic Systems

Introduction

Mohammad Narimani, Ph.D. P.Eng.

Lecturer
School of Mechatronic Systems Engineering
Simon Fraser University

MSE 491: Application of Machine Learning in Mechatronic Systems

Objectives & Aims
Objectives:
To learn:
▪ some of the basic mathematical concepts underpinning Machine Learning (ML)
algorithms
▪ different types of machine learning approached including supervised and
unsupervised learning algorithms
▪ ML algorithms in Python (maybe MATLAB as well)
▪ the implement of a trained ML algorithm on an embedded system

Aims:
You will learn to:
▪ explain the strengths and limitations of the various machine learning algorithms
▪ evaluate ML methods performance and select a proper one for your data
▪ perform and implement a trained ML model on an embedded platform

MSE 491: Application of Machine Learning in Mechatronic Systems

What is the meaning of “Machine Learning”?

▪ ML is a category of algorithms that allows users to teach

computers how to accurately perform tasks by providing
examples (data) of how they should be done

▪ The basic premise of ML is to build algorithms that can

receive input data/experience and use statistical analysis to
predict an output. Building such a model helps a precise
approximation of outputs using new data.

MSE 491: Application of Machine Learning in Mechatronic Systems

What is the meaning of “Machine Learning”?

▪ Examples in industries:
Netflix: Making suggestions to Viewers based on their previous
choice (input: Customer transactions => output: consumer
behavior/selection)

MSE 491: Application of Machine Learning in Mechatronic Systems

What is the meaning of “Machine Learning”?

▪ Examples in industries:
Pfizer: modification of immune
system's ability to help fight cancer.
input : patient’s immune system
structure
output: how well the immune system
can recognize and treat cancer cells

https://www.sciencedirect.com
by Hooiveld-Noeken et. al.

MSE 491: Application of Machine Learning in Mechatronic Systems

When to use Machine Learning?

▪ A system driven by ML algorithms can learn to

▪ make decisions
▪ adapt to its environment

▪ ML is used when:
▪ Humans are unable to explain a system’s behavior (retail
customer behavior)
▪ Solution needs to be adapted to particular cases
(personalized medicine)

▪ There is no need to “learn” to calculate payroll!!

MSE 491: Application of Machine Learning in Mechatronic Systems

What is inside a Machine Learning algorithm?

▪ Statistics: Inference from a sample

▪ Optimization: a performance criterion evaluation

using example data (experience)

▪ Computer science: efficient algorithms to

▪ Solve the optimization problem
▪ Representing and evaluating the model for inference

MSE 491: Application of Machine Learning in Mechatronic Systems

Applications

▪ Retail: Customer relationship management (CRM)

▪ Finance: fraud detection, market prediction

▪ Manufacturing: Control, robotics, troubleshooting

▪ Medicine: Medical diagnosis

▪ Web mining: Search engines

▪ ...

MSE 491: Application of Machine Learning in Mechatronic Systems

Main Area of Applications

▪ Learning Associations

▪ Supervised Learning
▪ Classification
▪ Regression

▪ Unsupervised Learning
▪ Clustering
▪ Non-clustering

▪ Reinforcement Learning

MSE 491: Application of Machine Learning in Mechatronic Systems

Learning Associations

▪ Association rules analysis is a technique to uncover

how items are associated to each other.
▪ To understand consider items X and Y frequently
bought together:
▪ Both X and Y can be placed on the same shelf, so that
buyers of one item would be prompted to buy the other.
▪ Promotional discounts could be applied to just one out of
the two items.
▪ Advertisements on X could be targeted at buyers who
purchase Y.
▪ X and Y could be combined into a new product, such as
having Y in flavors of X.
MSE 491: Application of Machine Learning in Mechatronic Systems
Learning Associations
▪ Basket analysis:
P (Y | X ): probability that somebody who buys X also
buys Y where X and Y are products/services.
Example: P ( camera | case ) = 0.8

Example: the rule {onions,potatoes} => {burger}

found in the sales data of a supermarket would
indicate that:
if a customer buys onions and potatoes together, they are
likely to also buy hamburger meat. Such information can be
used as the basis for decisions about marketing activities such
as, e.g., promotional pricing or product placements.
MSE 491: Application of Machine Learning in Mechatronic Systems
Supervised Learning

▪ In supervised learning, we are given a data set and already

know what our correct output should look like, having the idea
that there is a relationship between the input and the output.

▪ Supervised learning problems are categorized into "regression"

and "classification" problems.
• In a regression problem, we are trying to predict results within a
continuous output, meaning that we are trying to map input
variables to some continuous function.
• In a classification problem, we are instead trying to predict
results in a discrete output. In other words, we are trying to map
input variables into discrete categories.

MSE 491: Application of Machine Learning in Mechatronic Systems

Regression Applications
▪ Given data about the error signal 𝑒(𝑡), in a feedback control
system, try to predict control signal 𝑢(𝑡). The magnitude of
control signal 𝑢 𝑡 is a continuous function of error signal
𝑒(𝑡). This is a regression problem.

𝑢(𝑡)

𝑒(𝑡)

MSE 491: Application of Machine Learning in Mechatronic Systems

Regression Applications

▪ Price of a used car

▪ x : car attributes
y : price
y = g (x) y = wx+w0

g ( ): model

MSE 491: Application of Machine Learning in Mechatronic Systems

Regression Applications

▪ Navigating a car: Angle of the steering

▪ Kinematics of a robot arm

(x,y)
α1= g1(x,y)
α2
α2= g2(x,y)

α1

MSE 491: Application of Machine Learning in Mechatronic Systems

Classification Applications

▪ Given the information of a patient with a tumor, such as tumor

size, patient’s age, … predict if the tumor is malignant (1) or
benign (0).

75
Age

50

25

Tumor Size

MSE 491: Application of Machine Learning in Mechatronic Systems

Classification Applications

▪ Example: Credit
scoring
▪ Differentiating
between low-risk and
high-risk customers
from their income and
savings

Discriminant: IF income > θ1 AND savings > θ2

then low-risk
else high-risk
MSE 491: Application of Machine Learning in Mechatronic Systems
Classification: Some Applications
▪ Face recognition: Pose, lighting, occlusion (glasses,
beard), make-up, hair style

▪ Character recognition: Different handwriting styles

▪ Speech recognition: Temporal dependency

▪ Medical diagnosis: From symptoms to illnesses

▪ Biometrics: Recognition/authentication using

physical and/or behavioral characteristics: Face, iris,
signature, etc

MSE 491: Application of Machine Learning in Mechatronic Systems

Face Recognition
Training examples of a person

Test images

ORL dataset,
AT&T Laboratories, Cambridge UK

MSE 491: Application of Machine Learning in Mechatronic Systems

Unsupervised Learning

▪ Unsupervised learning allows us to approach problems with

little or no idea what our results should look like. We can derive
structure from data where we don't necessarily know the effect
of the variables.

▪ We can derive this structure by

clustering the data based on
relationships among the variables
in the data.

▪ With unsupervised learning there

is no feedback based on the
prediction results.

MSE 491: Application of Machine Learning in Mechatronic Systems

Unsupervised Learning-Examples
▪ Take a collection of 1,000,000 different genes, and find a way
to automatically group these genes into groups that are
somehow similar or related by different variables, such as
lifespan, location, roles, and so on.

▪ Customer segmentation in CRM

▪ Image compression: Color quantization

▪ Bioinformatics: Learning motifs

MSE 491: Application of Machine Learning in Mechatronic Systems

Unsupervised Learning-Examples
▪ Non-clustering: The "Cocktail Party Algorithm", allows you to
find structure in a chaotic environment. (i.e. identifying
individual voices and music from a mesh of sounds at a
cocktail party).

▪ https://www.youtube.com/watch/T0HP9cxri0A

MSE 491: Application of Machine Learning in Mechatronic Systems

Reinforcement Learning

▪ In RL, an AI faces a game-like situation.

▪ The computer employs trial and error to come up
with a solution to the problem.
▪ To get the machine to do what the programmer wants,
the AI gets either rewards or penalties for the actions
it performs.
▪ Its goal is to maximize the total reward.

▪ Examples:
▪ Game playing
▪ Robot in a maze
MSE 491: Application of Machine Learning in Mechatronic Systems
Reinforcement Learning

Taken from Berkeley Artificial Intelligence Research website

Robustness to
perturbations
MSE 491: Application of Machine Learning in Mechatronic Systems
Machine Learning Process

MSE 491: Application of Machine Learning in Mechatronic Systems

Resources: Journals
▪ Journal of Machine Learning Research www.jmlr.org

▪ IEEE Trans on Neural Networks and Learning Systems

▪ IEEE Trans on Pattern Analysis and Machine Intelligence

▪ Journals on Statistics/Data Mining/Signal Processing/Natural Language

Processing/Bioinformatics/...

MSE 491: Application of Machine Learning in Mechatronic Systems

Resources: Conferences
▪ International Conference on Machine Learning (ICML)

▪ European Conference on Machine Learning (ECML)

▪ Neural Information Processing Systems (NIPS)

▪ Uncertainty in Artificial Intelligence (UAI)

▪ Computational Learning Theory (COLT)

▪ International Conference on Artificial Neural Networks (ICANN)

▪ International Conference on AI & Statistics (AISTATS)

▪ International Conference on Pattern Recognition (ICPR)

▪ ...

MSE 491: Application of Machine Learning in Mechatronic Systems

Question?

▪ What are some examples of artificial Intelligence that are not Machine
Learning?

MSE 491: Application of Machine Learning in Mechatronic Systems