Machine Learning

Introduction
and
its Types

Prof.N.Nalini
SCOPE
VIT

Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
Relationship between AI, Machine
Learning, Deep Learning & Data Science?
• Artificial Intelligence (AI) Human Intelligence
Exhibited by Machines
• Intelligence exhibited by machines
• Broadly defined to include any simulation of human
intelligence
• Expanding and branching areas of research,
development, and investment
• Includes robotics, rule-based reasoning, natural
language processing (NLP), knowledge
representation techniques (knowledge graphs) …

CSE3047: Predictive Analysis - Prof.N.Nalini

Relationship between AI, Machine
Learning, Deep Learning & Data Science?
• Machine Learning (ML) An Approach to Achieve
Artificial Intelligence
• Subfield of AI that aims to teach computers the
ability to do tasks with data, without explicit
programming
• Uses numerical and statistical approaches,
including artificial neural networks to encode
learning in models
• Models built using “training” computation runs or
through usage
CSE3047: Predictive Analysis - Prof.N.Nalini
Relationship between AI, Machine
Learning, Deep Learning & Data Science?
• Deep Learning (DL) A Technique for Implementing
Machine Learning
• Subfield of ML that uses specialized
techniques involving multi-layer (2+) artificial
neural networks
• Layering allows cascaded learning and
abstraction levels (e.g. line -> shape -> object ->
scene)
• Computationally intensive enabled by clouds,
GPUs, and specialized HW such as FPGAs, TPUs,
etc.
CSE3047: Predictive Analysis - Prof.N.Nalini
 Data Science
• Data science is a wide-ranging discipline that unifies aspects of statistics, data
analysis and machine learning to harness data to solve business problems.
• Data Science - Scientific methods, algorithms and systems to extract knowledge or
insights from big data also known as Predictive or Advanced Analytics
• Algorithmic and computational techniques and tools for handing large data sets
• The field involves many different disciplines and tools, including statistical inference,
domain knowledge (expertise), data visualization, experiment design, and
communication.
• Increasingly focused on preparing and modeling data for ML & DL tasks
• Encompasses statistical methods, data manipulation and streaming technologies
(e.g. Spark, Hadoop) are key skill and tools behind building modern AI technologies

CSE3047: Predictive Analysis - Prof.N.Nalini

Machine Learning
• Machine learning is a subset of artificial intelligence (AI)
that focuses on learning new data and improving their
accuracy over time to build new applications without being
programmed to do so.

Prof.N.Nalini,SCOPE,VIT,Vellore
Learning
• Inductive learning is the process of using observations to draw
conclusions .
• Deductive learning is the process of using conclusions to form
observations.

• The classic machine learning procedure follows the scientific paradigm

of induction and deduction.
• In the inductive step we learn the model from raw data (so called training set), and in
the deductive step the model is applied to predict the behavior of new data.

Prof.N.Nalini,SCOPE,VIT,Vellore
 The machine learning
framework
• Apply a prediction function to a feature representation of
the image to get the desired output:

f( ) = “apple”
f( ) = “tomato”
f( ) = “cow”
Prof.N.Nalini,SCOPE,VIT,Vellore
Slide credit: L. Lazebnik
 The machine learning
framework
y = f(x)
output prediction Image
function feature

• Training: given a training set of labeled examples {(x1,y1), …, (xN,yN)},

estimate the prediction function f by minimizing the prediction error on
the training set
• Testing: apply f to a never before seen test example x and output the
predicted value y = f(x)

Prof.N.Nalini,SCOPE,VIT,Vellore
Slide credit: L. Lazebnik
Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
 Steps
Training Training
Labels
Training
Images
Image Learned
Training
Features model

Testing

Image Learned
Prediction
Features model
Prof.N.Nalini,SCOPE,VIT,Vellore
Test Image Slide credit: D. Hoiem and L. Lazebnik
Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
Semi-Supervised Learning

Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
 ML in a Nutshell
• Tens of thousands of machine learning
algorithms
• Hundreds new every year
• Every machine learning algorithm has
three components:
– Representation
– Evaluation
– Optimization

Prof.N.Nalini,SCOPE,VIT,Vellore
 Representation
• Decision trees
• Sets of rules / Logic programs
• Instances
• Graphical models (Bayes/Markov nets)
• Neural networks
• Support vector machines
• Model ensembles
• Etc.
Prof.N.Nalini,SCOPE,VIT,Vellore
 Evaluation
• Accuracy
• Precision and recall
• Squared error
• Likelihood
• Posterior probability
• Cost / Utility
• Margin
• Entropy
• K-L divergence
• Etc.

Prof.N.Nalini,SCOPE,VIT,Vellore
 Optimization
• Combinatorial optimization
– E.g.: Greedy search
• Convex optimization
– E.g.: Gradient descent
• Constrained optimization
– E.g.: Linear programming

Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
PERSPECTIVES IN MACHINE LEARNING

• It involves searching a very large space of possible

hypotheses to determine one that best fits the
observed data and any prior knowledge held by the
learner.
• The learner's task is thus to search through this vast
space to locate the hypothesis that is most consistent
with the available training examples.
• The LMS algorithm for fitting weights achieves this goal
by iteratively tuning the weights, adding a correction to
each weight each time the hypothesized evaluation
function predicts a value that differs from the training
value.

Prof.N.Nalini,SCOPE,VIT,Vellore
 PERSPECTIVES IN MACHINE LEARNING
• Many algorithms search a hypothesis space defined by
some underlying representation (e.g., linear functions,
logical descriptions, decision trees, artificial neural
networks).
• These hypothesis representations are appropriate for
learning different kinds of target functions.
• Learning is a search problem, in order
• to characterize learning methods by their search strategies and
• by the underlying structure of the search spaces they explore.
• Useful in formally analysing the relationship between the
size of the hypothesis space to be searched, the number
of training examples available.

Prof.N.Nalini,SCOPE,VIT,Vellore
Reasons for failure
• Asking the wrong question
• Trying to solve the wrong problem
• Not having enough data
• Not having the right data
• Having too much data
• Hiring the wrong people
• Using the wrong tools
• Not having the right model
• Not having the right yardstick

Prof.N.Nalini,SCOPE,VIT,Vellore
Issues in Machine Learning
• What algorithms exist for learning general target functions from specific training
examples?
• In what settings will particular algorithms converge to the desired function, given sufficient
training data?
• Which algorithms perform best for which types of problems and representations?
• How much training data is sufficient?
• What general bounds can be found to relate the confidence in learned hypotheses to the
amount of training experience and the character of the learner's hypothesis space?
• When and how can prior knowledge held by the learner guide the process of generalizing
from examples?
• Can prior knowledge be helpful even when it is only approximately correct?
• What is the best strategy for choosing a useful next training experience, and how does the
choice of this strategy alter the complexity of the learning problem?
• What is the best way to reduce the learning task to one or more function approximation
problems?
• what specific functions should the system attempt to learn? Can this process itself be
automated?
• How can the learner automatically alter its representation to improve its ability to
represent and learn the target function?

Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore
Prof.N.Nalini,SCOPE,VIT,Vellore