Lecture Slides for

ARTIFICIAL INTELLIGENCE
AND MACHİNE LEARNİNG

MD ABDUL ALIM SHEIKH

Aliah University
Dept. of Electronics and Communication Engineering
[email protected]
CHAPTER 1:

INTRODUCTİON
 Topics of this lecture
3

 A brief review of AI history

 What is artificial intelligence?
 Related research fields
 Scope of this course
 Early work (Around 1900 ）
4
 Early work(1930 ～）
5
 The 1stwave of AI (1950 ～ )
6
 The 2ndwave of AI (1980 ～ )
7
 The 3rdwave of AI (2000 ～ )
8

 Representative technologies
 Internet: Tim Berners-Lee, WWW inventor, 1989
 Internet of things: Kevin Ashton, MIT Auto-ID Center,
1999
 Cloud computing
 Main frame (1950s)
 Virtual machine (1970s)
 Cloud (1990s)
 Big data: John R. Masey, SGI, 1998
 Deep learning: Geoffrey Hinton, UoT, 2006
 A brief summary
9
 Current status of AI
10

 In March 2016, Alpha-Go of DeepMind defeated

Lee Sedol, who was the strongest human GO
player at that time.
 This is a big news that may have profound meaning
in the human history.
 Cycle for AI, 2020
11
 After all, what is intelligence?
12

 Intelligence is an umbrella term used to describe a property of

the mind that encompasses many related abilities, such as the
capacities
 to reason,
 to plan,

 to solve problems,

 to think abstractly,

 to comprehend ideas,

 to use language, and

 to learn.

Dictionary.com: capacity for learning, reasoning, understanding, and similar

(from Wikipedia)
forms of mental activity
 What is intelligence?

13

 Dictionary.com: capacity for learning, reasoning,

understanding, and similar forms of mental activity
 Ability to perceive and act in the world
 Reasoning: proving theorems, medical diagnosis
 Planning: take decisions
 Learning and Adaptation: recommend movies,
learn traffic patterns
 Understanding: text, speech, visual scene
 Artificial Intelligence
14

 Subject matters in AI
 Get machines to do what humans do but
machines can’t
 Unlike any classical subject, the frontier of
what is AI is not static.
 Intelligence can be defined as the ability for
solving problems
15

 Problem solving is to find the “best” solution in the

problem space.
 Reasoning is to interpret or justify solutions or sub-
solutions.
 Planning is to find ways for solving the problem.
 Thinking abstractly is to simulate the problem solving
process inside the system (brain).
 Idea/language comprehension is a way (or means) for
data/problem/knowledge representation;
 Learning is the process to find better ways for solving a
problem (or a class of problems).
 What is AI ?
16

 Textbooks often define artificial intelligence as “the

study and design of computing systems that
perceives its environment and takes actions like
human beings”.
 The term was introduced by John McCarthy in
1956 in the well-known Dartmouth Conference.
 In my study, an AI is defined as a system that
possesses at least one (not necessarily all) of the
abilities mentioned in the previous page.
As a research area, AI studies theories and technologies for
obtaining systems that are partially or fully intelligent.
 What is intelligence
17

 Acting humanly: Turing Test

 Turing (1950) "Computing machinery and
intelligence":
 "Can machines think?"
 Imitation Game
 Acting humanly: Turing Test
18
 What is AI?
19

 four views:

Think like a Think rationally

human

Act like a human Act rationally

 Search
20

 Brute-force search
 Depth-first search
 Breadth-first search
 Heuristic Search
 Hill climbing search
 Best-first search
 A* Algorithm
 Intelligent search
 Genetic algorithms
 Meta-heuristics
 Three MAPs for knowledge acquisition
21

 What is the input?

 Map from real world to the mind model
 What is the output?
 Map from the mind model to the real world
 What is the relation between the input and the output?
 Abstracted knowledge about the real world
 Representation methods
22

 Representation of the problem

 State space representation
 Vector representation
 Representation of knowledge
 Production (decision) rules
 Semantic network and ontology
 Predicate logic
 Fuzzy logic
 Neural network (for tacit knowledge)
 Learning models and algorithms
23

 Neural network learning

 Including MLP, SVM, deep learning, etc.
 Evolutionary learning
 GA or meta-heuristics in general
 Reinforcement learning
 Artificial immune system
 Fuzzy logic
 Decision tree
 Hybrid system
 Scope of this course
24

 Search
 Problem formulation and basic search algorithms
 Expert system-based reasoning
 Production system, semantic network, and frame
 Logic based-reasoning
 Propositional logic and predicate logic
 Soft computing based reasoning
 Fuzzy logic and multilayer neural network
 Scope of this course
25

 Machine learning
 Pattern recognition
 Self-organization
 Neural networks
 Decision trees
 Intelligent search (if we have time)
 Genetic algorithm
 Ant colony optimization
 Purpose of this course
26

 Learn how to use the basic search methods;

 Understand the basic methods for problem
formulation and knowledge representation;
 Understand the basic idea of automatic reasoning;
 Know some basic concepts related to pattern
recognition and machine learning.

Be able to learn more by yourself to

follow the newest trends!
 AI History
27

 Timeline : Prehistory / Early AI

 Pre-history: Pascal, Leibniz
hoaxes
Babbage
 1943 McCulloch & Pitts:
Boolean circuit model
of neuron
 1950 Turing's "Computing
Machinery and Intelligence”
 1956 Dartmouth meeting: von kempelen’s chess-playing turk, 1769 (hoax)

“Artificial Intelligence” name

 1955: coining the name “Artificial
28
Intelligence”
John McCarthy, Marvin
Minsky, N Rochester,
and Claude Shannon:
(1955 ) :

“the conjecture that every

aspect of learning or
any other feature of
intelligence can in
principle be so
precisely described that
a machine can be made
to simulate it.”
 Timeline : AI – Logical Models
29

 1943 McCulloch & Pitts: Boolean circuit

model of brain
 1950 Turing's "Computing Machinery and
Intelligence"
 1956 Dartmouth meeting: "Artificial
Intelligence" name
 1956 Newell & Simon's Logic Theorist,
 1959 Samuel's checkers program: learned by
playing itself
30
31
32
33
 Rosenblatt-Perceptrons
34
35
36
37
38
39
 AI-Learning
40
 Big Data
41

 Widespread use of personal computers and wireless

communication leads to “big data”
 We are both producers and consumers of data
 Data is not random, it has structure, e.g., customer
behavior
 We need “big theory” to extract that structure from
data for
(a) Understanding the process
(b) Making predictions for the future
 Why “Learn” ?
42

 Machine learning is programming computers to

optimize a performance criterion using example data
or past experience.
 There is no need to “learn” to calculate payroll
 Learning is used when:
 Human expertise does not exist (navigating on Mars),
 Humans are unable to explain their expertise (speech
recognition)
 Solution changes in time (routing on a computer network)
 Solution needs to be adapted to particular cases (user
biometrics)
 What We Talk About When We Talk About
“Learning”
43

 Learning general models from a data of particular

examples
 Data is cheap and abundant (data warehouses, data
marts); knowledge is expensive and scarce.
 Example in retail: Customer transactions to
consumer behavior:
People who bought “Blink” also bought “Outliers”
(www.amazon.com)
 Build a model that is a good and useful
approximation to the data.
 Data Mining
44

 Retail: Market basket analysis, Customer

relationship management (CRM)
 Finance: Credit scoring, fraud detection
 Manufacturing: Control, robotics, troubleshooting
 Medicine: Medical diagnosis
 Telecommunications: Spam filters, intrusion
detection
 Bioinformatics: Motifs, alignment
 Web mining: Search engines
 ...
 What is Machine Learning?
45

 Optimize a performance criterion using example

data or past experience.
 Role of Statistics: Inference from a sample
 Role of Computer science: Efficient algorithms to
 Solve the optimization problem
 Representing and evaluating the model for inference
 Applications
46

 Association
 Supervised Learning
 Classification
 Regression
 Unsupervised Learning
 Reinforcement Learning
 Learning Associations
47

 Basket analysis:
P (Y | X ) probability that somebody who buys X
also buys Y where X and Y are products/services.
Example: P ( chips | drink ) = 0.7
 Classification
48

 Example: Credit
scoring
 Differentiating
between low-risk and
high-risk customers
from their income and
savings
Discriminant: IF income > θ1 AND savings > θ2
THEN low-risk ELSE high-risk
 Classification: Applications
49

 Aka Pattern recognition

 Face recognition: Pose, lighting, occlusion (glasses,
beard), make-up, hair style
 Character recognition: Different handwriting styles.
 Speech recognition: Temporal dependency.
 Medical diagnosis: From symptoms to illnesses
 Biometrics: Recognition/authentication using
physical and/or behavioral characteristics: Face,
iris, signature, etc
 Outlier/novelty detection:
 Face Recognition
50

Training examples of a person

Test images

ORL dataset,
AT&T Laboratories, Cambridge UK
Regression

 Example: Price of a
used car
y = wx+w0
 x : car attributes
y : price
y = g (x | q )
g ( ) model,
q parameters

51
 Regression Applications
52

 Navigating a car: Angle of the steering

 Kinematics of a robot arm
(x,y) α1= g1(x,y)
α2= g2(x,y)
α2

α1

 Response surface design

 Supervised Learning: Uses
53

 Prediction of future cases: Use the rule to predict

the output for future inputs
 Knowledge extraction: The rule is easy to
understand
 Compression: The rule is simpler than the data it
explains
 Outlier detection: Exceptions that are not covered
by the rule, e.g., fraud
 Unsupervised Learning
54

 Learning “what normally happens”

 No output
 Clustering: Grouping similar instances
 Example applications
 Customer segmentation in Customer Relationship
Management (CRM)
 Image compression: Color quantization
 Bioinformatics: Learning motifs
 Reinforcement Learning
55

 Learning a policy: A sequence of outputs

 No supervised output but delayed reward
 Credit assignment problem
 Game playing
 Robot in a maze
 Multiple agents, partial observability, ...
 Resources: Datasets
56

 UCI Repository:
http://www.ics.uci.edu/~mlearn/MLRepository.html
 Statlib: http://lib.stat.cmu.edu/
 Resources: Journals
57

 Journal of Machine Learning Research www.jmlr.org

 Machine Learning
 Neural Computation
 Neural Networks
 IEEE Trans on Neural Networks and Learning Systems
 IEEE Trans on Pattern Analysis and Machine
Intelligence
 Journals on Statistics/Data Mining/Signal
Processing/Natural Language
Processing/Bioinformatics/...
 Resources: Conferences
58
 International Conference on Machine Learning (ICML)
 European Conference on Machine Learning (ECML)
 Neural Information Processing Systems (NIPS)
 Uncertainty in Artificial Intelligence (UAI)
 Computational Learning Theory (COLT)
 International Conference on Artificial Neural Networks
(ICANN)
 International Conference on AI & Statistics (AISTATS)
 International Conference on Pattern Recognition (ICPR)
 ...