Artificial Intelligence

Pramod Kumar Soni

1
 Course Objectives

• To p rov id e b as ic knowled g e of Artific ial

Intelligence

• To familiarize students with different search

techniques

• To acquaint students with the fields related to AI

and the applications of AI

2
Course Contents

Future

NLP Learning

Expert Genetic E Gxeptrnploe ertcigto

Systems Algorithms PLLSNInireseoa
L arcPrhn
o
S A
AIing
yl
g s o t
e m
r t
ihs ms
Search Prolog

Intro to AI Lisp

3
 Syllabus
1. Introduction to AI

2. Problem solving

3. Search techniques

4. Knowledge representation, inference and reasoning

5. Structured knowledge representation

6. Machine learning

7. Applications of AI

4
 References

E. Rich and Knight, Artificial Intelligence, McGraw Hill, 1991.

D. W. Patterson, Artificial Intelligence and Expert Systems, Prentice Hall, 2001.

P. H. Winston, Artificial Intelligence, Addison Wesley, 1984.

Stuart Russel and Peter Norvig, Artificial Intelligence A Modern Approach,

Pearson

Ivan Bratko, PROLOG Programming for Artificial Intelligence, Addison Wesley,

2001.

Leon Sterling, Ehud Shapiro, The Art of PROLOG: Advanced Programming

Techniques, Prentice Hall, 1996.

5
An Introduction to AI

7
 What is AI?

How does How do we

the human emulate the
brain work? human brain?

How do we
create
What is intelligence?
intelligence?

Who cares? L e t ! s
do some cool and
useful stuff! 8
 Meaning of the word: ``intelligence''

1 (a) The capacity to acquire and apply knowledge.

(b) The faculty of thought and reason.
(c)Superior powers of mind. See Synonyms at mind.

2 An intelligent, incorporeal being, especially an angel.

3 Information; news. See Synonyms at news.

4 (a) Secret information, especially about an actual or potential enemy.

(b) An agency, staff, or office employed in gathering such information.

9
 Meaning of the word: ``intelligence''

1: the ability to comprehend; to understand and

profit from experience [ant: stupidity]
2: a unit responsible for gathering and interpreting intelligence

3: secret information about an enemy (or potential enemy); "we sent out
planes to gather intelligence on their radar coverage"

4: new information about specific and timely events; "they awaited

news of the outcome" [syn: news, tidings, word]

5: the operation of gathering information about an enemy [syn:

intelligence activity, intelligence operation]

10
 What Behaviors are Intelligent?
Everyday tasks: recognize a friend, recognize who
is calling, translate from one language to another,
interpret a photograph, talk, cook a dinner

Formal tasks: prove a logic theorem, geometry,

calculus, play chess, checkers, or Go

Expert tasks: engineering design, medical

designers, financial analysis
11
 Artificial Intelligence

Based on the above, `artificial intelligence' is about

the science and engineering necessary to create
artifacts that can

• acquire knowledge, i.e., can learn and extract

knowledge; and

• reason with knowledge (leading to doing tasks

such as planning, explaining, diagnosing, acting
rationally, etc.),

12
 Formal Definitions
Barr and Feigenbaum

“Artificial Intelligence is the part of computer

science concerned with designing intelligence
computer systems, that is, systems that exhibit the
characteristics we associate with intelligence in
human behavior.”

Elaine Rich

“AI is the study of how to make computers do things

at which, at the moment, people are better”
13
Different Types of Artificial Intelligence
• Modeling exactly how humans actually think
– cognitive models of human reasoning

• Modeling exactly how humans actually act

– models of human behavior (what they do, not how they think)

• Modeling how ideal agents “should think”

– models of “rational” thought (formal logic)
– note: humans are often not rational!

• Modeling how ideal agents “should act”

– rational actions but not necessarily formal rational reasoning
– i.e., more of a black-box/engineering approach

• Modern AI focuses on the last definition

– we will also focus on this “engineering” approach
– success is judged by how well the agent perform
-- modern methods are inspired by cognitive & neuroscience (how people think).
14
 Acting humanly: Turing Test
• Turing (1950) "Computing machinery and intelligence":

• "Can machines think?" or "Can machines behave intelligently?"

• Operational test for intelligent behavior: the Imitation Game
• The interrogator can communicate two with sources: one is human and the
other is a machine
• He must decide which is which
• If he is wrong half the time, then the machine is intelligent

15
AI Today

Many more

Data
Predictions
Mining

Astronomy NLP

Biometric Robotics
Identification
Medical
Biology
Systems

16
 AI Today

• Diagnose lymph- node diseases [Heckerman, 91]

• Monitor space shuttle missions [Horvitz, 92]

• Automatic vehicle control [Jochem et al., 96]

• Large- scale scheduling [Smith et al., 96]

• Classify astronomical objects [Goebel et al., 89]

• Automatic design and configuration systems

• First commercial speech understanding systems

• Beat world’s best players in chess, checkers, and backgammon. 17

 Prolog

First Prolog program: France, 1970

Based on theorem proving research
Major development at University of Edinburgh, 1975-79
Adopted by the Japanese Fifth Generation Computing Project

Logic programming language:

Programs composed of facts and rules
Executes by applying first-order predicate calculus/unification to programs
Interactive interpreter, compiler
Tell the computer what is true and what needs to be done, rather than how to do

likes(deb, horses).
likes(deb, dogs).
likes(deb, Y) :- horse(Y). ?- likes(deb, horses).
horse(robin). yes
?- likes(deb, robin). ?- likes(deb, X).
yes X=horses
X=dogs 18
 LISP
• Proposed by McCarthy, late 1950s; contemporary of COBOL,
FORTRAN
• Functional programming language based on lambda
calculus/recursive function theory
• Intended as a language for symbolic rather than numeric computation
• Interactive interpreter, compiler
• Uses atoms, lists, functions.

(defun hypotenuse (x y)
(sqrt (+ (square x)
(square y))))

> (hypotenuse 4 3)
5
19
 History of AI

• 1943 McCulloch & Pitts: Boolean circuit model of brain

• 1950 Turing's "Computing Machinery and Intelligence"
• 1956 Dartmouth meeting: "Artificial Intelligence" adopted
• 1950s Early AI programs, including Samuel's checkers
program, Newell & Simon's Logic Theorist, Gelernter's
Geometry Engine
• 1965 Robinson's complete algorithm for logical reasoning
• 1966—73 AI discovers computational complexity
Neural network research almost disappears
• 1969—79 Early development of knowledge-based systems
• 1980-- AI becomes an industry
• 1986-- Neural networks return to popularity
• 1987-- AI becomes a science
• 1995-- The emergence of intelligent agents

20
 Artificial Intelligence
• Common sense reasoning
• Reasoning under uncertain conditions
• Learning from experience
• Planning and executing complex tasks
• Understanding and communicating in spoken/written language
• Visual comprehension

Note: "easy for human" ! = "easy for machine".

Chess is easy for computer, hard for human.
Visual recognition of chess pieces is easy for human, hard for computer.

"Tasks that require knowledge of some domain"

21
 Knowledge Definition

• “The fact or condition of knowing something with

familiarity gained through experience or
association.” (Webster’s Dictionary, 1988)(Knowing
something via seeing, hearing, touching, feeling, and
tasting.)

• “The fact or condition of being aware of something” .

(Ex. Sun is hot, balls are round, sky is blue,…)

22
 Knowledge Storing

Natural language for people

• Symbols for computer: a number or character
string that represents an object or idea (Internal
representation of the knowledge).
• The core concepts: mapping from facts to an
internal computer representation and also to a
form that people can understand.

23
 Knowledge Representation

• Simple facts or complex relationships

• Mathematical formulas or rules for natural
language syntax
• Associations between related concepts
• Inheritance hierarchies between classes of
objects
• Knowledge is not a “one-size-fits-all”
proposition.

24
 Knowledge
Example: automated language translation (English <----->
Japanese)

”My name is hari bahadur."

”Madan bahadur is my friend."

Must determine what this means using knowledge of human discourse in English

word-by-word
translation
--------------------->

Then generate japanese form of that meaning

25
 Knowledge
Note: knowledge != data.
Example: determining voltage from current and resistance

• Knowledge is more compact and faster to manipulate.

• Knowledge is more general (that is, may be applied to situations we have not
been programmed for).
• Important feature of intelligence: creating knowledge from data.

26
 Knowledge

"Heuristics for making good decisions when no algorithm exists for doing so"
That is, for many of the above problem, we must often make intelligent guesses due
to:
• Lack of complete knowledge about how to solve problem.
• Lack of complete data about current situation.
• Lack of time to completely explore situation.

Example: walking across room.

Do not have complete knowledge of physical laws associated with motion.
Do not have complete knowledge of room (such as what might be behind objects).
Do not have time to map out and compare all possible paths through room.

27
 What is Learning?
Learning is one of those everyday terms
which is broadly and vaguely used in the
English language
• Learning is making useful changes in our minds
• Learning is constructing or modifying representations of what
is being experienced
• Learning is the phenomenon of knowledge acquisition in the
absence of explicit programming
Herbert Simon, 1983

Learning denotes changes in the system that are

adaptive in the sense that they enable the system to do
the same task or tasks drawn from the same population
more efficiently and more effectively next time. 28
 What is Learning?
Learning involves 3 factors:

Learning changes the learner: for machine

changes learning the problem is determining the nature
of these changes and how to best represent them
generalization
Learning leads to generalization: performance
improvement must improve not only on the same task but on
similar tasks

Learning leads to improvements: machine

learning must address the possibility that
changes may degrade performance and find ways
to prevent it.

29
 Consider what might be involved in
building a “intelligent” computer….

• What are the “components” that might be useful?

• Fast hardware?
• Foolproof software?
• Chess-playing at grandmaster level?
• Speech interaction?
speech synthesis
speech recognition
speech understanding
• Image recognition and understanding ?
• Learning?
• Planning and decision-making?

30
 Can we build hardware as complex as the
brain?
• How complicated is our brain?
• a neuron, or nerve cell, is the basic information processing unit
• estimated to be on the order of 10 11 neurons in a human brain
• many more synapses (10 14) connecting these neurons
• cycle time: 10 -3 seconds (1 millisecond)

• How complex can we make computers?

• 106 or more transistors per CPU
• supercomputer: hundreds of CPUs, 10 9 bits of RAM
• cycle times: order of 10 - 8 seconds

• Conclusion
• YES: in the near future we can have computers with as many basic processing
elements as our brain, but with
far fewer interconnections (wires or synapses) than the brain
much faster updates than the brain
• but building hardware is very different from making a computer behave like a brain!

31
 Must an Intelligent System be Foolproof?

• A “foolproof” system is one that never makes an error:

• Types of possible computer errors
hardware errors, e.g., memory errors
software errors, e.g., coding bugs
“human-like” errors
• Clearly, hardware and software errors are possible in practice
• what about “human-like” errors?

• An intelligent system can make errors and still be intelligent

• humans are not right all of the time
• we learn and adapt from making
mistakes e.g., consider learning to
surf or ski
we improve by taking risks and falling
an intelligent system can learn in the same way
• Conclusion:
• NO: intelligent systems will not (and need not) be foolproof

32
 Can Computers Talk?
• This is known as “speech synthesis”
• translate text to phonetic
form e.g., “fictitious”-> fik-
tish-es
• use pronunciation rules to map phonemes to actual
sound e.g., “tish” -> sequence of basic audio
sounds
• sounds made by this “lookup” approach sound unnatural
• • sounds are not independent
Difficulties
e.g., “act” and “action”
modern systems (e.g., at AT&T) can handle this pretty well
• a harder problem is emphasis, emotion, etc
humans understand what they are saying
machines don’t: so they sound unnatural

• Conclusion: NO, for complete sentences, but YES for individual words

34
 Can Computers Recognize Speech?

• Speech Recognition:
• mapping sounds from a microphone into a list of words.
• Hard problem: noise, more than one person
talking, occlusion, speech variability,..
• Even if we recognize each word, we may not understand its meaning.

• Recognizing single words from a small vocabulary

systems can do this with high accuracy (order of 99%)
e.g., directory inquiries
limited vocabulary (area codes, city names)
computer tries to recognize you first, if unsuccessful hands you over to
a human operator
saves millions of dollars a year for the phone companies

35
Recognizing human speech (ctd.)
• Recognizing normal speech is much more difficult
• speech is continuous: where are the boundaries between
words? e.g., “John’s car has a flat tire”
• large vocabularies
can be many thousands of possible words
we can use context to help figure out what someone said
try telling a waiter in a restaurant:
“I would like some dream and sugar in my coffee”
• background noise, other speakers, accents, colds, etc
• on normal speech, modern systems are only about 60% accurate

• Conclusion: NO, normal speech is too complex to accurately recognize, but YES
for restricted problems
• (e.g., recent software for PC use by IBM, Dragon systems, etc)

36
Can Computers Understand speech?

• Understanding is different to recognition:

• “Time flies like an arrow”
assume the computer can recognize all the words
but how could it understand it?
1. time passes quickly like an arrow?
2. command: time the flies the way an arrow times the flies
3. command: only time those flies which are like an arrow
4. “time-flies” are fond of arrows
only 1. makes any sense, but how could a computer figure this out?
clearly humans use a lot of implicit commonsense knowledge in
communication

• Conclusion: NO, much of what we say is beyond the capabilities of a computer to

understand at present

37
Can Computers Learn and Adapt ?

• Learning and Adaptation

• consider a computer learning to drive on the freeway
• we could code lots of rules about what to do
• or we could let it drive and steer it back on course when it heads for the
embankment
systems like this are under development (e.g., Daimler Benz)
e.g., RALPH at CMU
in mid 90’s it drove 98% of the way from Pittsburgh to San Diego
without any human assistance

• machine learning allows computers to learn to do things without explicit

programming
• Conclusion: YES, computers can learn and adapt, when presented with
information in the appropriate way

38
 Can Computers “see”?
• Recognition v. Understanding (like Speech)
• Recognition and Understanding of Objects in a
scene look around this room
you can effortlessly recognize objects
human brain can map 2d visual image to 3d “map”

• Why is visual recognition a hard problem?

• Conclusion: mostly NO: computers can only “see” certain types of objects
under limited circumstances: but YES for certain constrained problems (e.g.,
face recognition)

39
 Can Computers plan and make decisions?

• Intelligence
• involves solving problems and making decisions and plans
• e.g., you want to visit your cousin in
Boston you need to decide on dates,
flights
you need to get to the airport, etc
involves a sequence of decisions, plans, and actions

• • the
What worldplanning
makes is not predictable:
hard?
your flight is canceled or there’s a backup on the 405
• there is a potentially huge number of details
do you consider all flights? all dates?
no: commonsense constrains your solutions
• AI systems are only successful in constrained planning problems

• Conclusion: NO, real-world planning and decision-making is still beyond the

capabilities of modern computers
• exception: very well-defined, constrained problems: mission planning for
satelites.
40
 Academic Disciplines important to AI.
• Philosophy Logic, methods of reasoning, mind as physical
system, foundations of learning, language,
rationality.

• Mathematics Formal representation and proof, algorithms,

computation, (un)decidability, (in)tractability,
probability.

• Economics utility, decision theory, rational economic agents

• Neuroscience neurons as information processing units.

• Psychology/ how do people behave, perceive, process Cognitive Science

information, represent knowledge.

• Computer building fast computers

engineering

• Control theory design systems that maximize an objective

function over time

• Linguistics knowledge representation, grammar

41
 Intelligent Systems in Your Everyday
Life
• Post Office
• automatic address recognition and sorting of mail

• Banks
• automatic check readers, signature verification systems
• automated loan application classification

• Telephone Companies
• automatic voice recognition for directory inquiries
• automatic fraud detection,

• Credit Card Companies

• automated fraud detection

• Computer Companies
• automated diagnosis for help-desk applications

• Recommendation systems
• Product recommendation based on history 42
 AI Applications: Consumer Marketing
• Have you ever used any kind of credit/ATM/store card while shopping?
• if so, you have very likely been “input” to an AI algorithm
• All of this information is recorded digitally
• Companies like Nielsen gather this information weekly and search for patterns
• general changes in consumer behavior
• tracking responses to new products
• identifying customer segments: targeted marketing, e.g., they find out that
consumers with sports cars who buy textbooks respond well to offers of new credit
cards.
• Currently a very hot area in marketing

• How do they do this?

• Algorithms (“data mining”) search data for patterns
• based on mathematical theories of learning
• completely impractical to do manually

43
AI Applications: Identification Technologies
• ID cards
• e.g., ATM cards
• can be a nuisance and security risk:
cards can be lost, stolen, passwords forgotten, etc
• Biometric Identification
• walk up to a locked
door camera
fingerprint device
microphone
• computer uses your biometric signature for
identification face, eyes, fingerprints, voice pattern

44
AI Applications: Predicting the Stock Market
Value of
the Stock
?

time in days
• The Prediction Problem
• given the past, predict the future
• very difficult problem!
• we can use learning algorithms to learn a predictive model from historical data
prob(increase at day t+1 | values at day t, t-1,t-2....,t-k)

• such models are routinely used by banks and financial traders to manage
portfolios worth millions of dollars
45
 AI-Applications: Machine Translation
• Language problems in international business
• e.g., at a meeting of Japanese, Korean, Vietnamese and Swedish investors, no
common language
• or: you are shipping your software manuals to 127 countries
• solution; hire translators to translate
• would be much cheaper if a machine could do this!

• How hard is automated translation

• very difficult!
• e.g., English to Russian
“The spirit is willing but the flesh is weak” (English)
“the vodka is good but the meat is rotten” (Russian)
• not only must the words be translated, but their meaning also!

• Nonetheless....
• commercial systems can do alot of the work very well (e.g.,restricted vocabularies
in software documentation)
• algorithms which combine dictionaries, grammar models, etc.
• see for example babelfish.altavista.com

46
AI-Applications: Face Detection

47
AI-Applications: Product Recommendation

48
AI-Applications: Image Guided Surgery

49
AI-Applications: Speech Recognition

50
AI-Applications: Exploration of the
Universe

51
Intelligent Agents

52
 Agents

An agent is anything that can be viewed as perceiving its

environment through sensors and acting upon that
environment through actuators

Human agent:
eyes, ears, and other organs for sensors;
hands, legs, mouth, and other body parts for
actuators

Robotic agent:
cameras and infrared range finders for sensors; various
motors for actuators
53
 Agents and environments

The agent function maps from percept histories to

actions:
[f: P* A]

The agent program runs on the physical architecture to

produce f
agent = architecture + program 54
 Vacuum-cleaner world

Percepts: location and state of the environment, e.g.,

[A,Dirty], [B,Clean]

Actions: Left, Right, Suck, NoOp

55
 Rational agents

Rational Agent: For each possible percept sequence, a

rational agent should select an action that is expected to
maximize its performance measure, based on the evidence
provided by the percept sequence and whatever built-in
knowledge the agent has.
Click to add text

Performance measure: An objective criterion for success of

an agent's behavior

E.g., performance measure of a vacuum-cleaner agent could

be amount of dirt cleaned up, amount of time taken, amount
of electricity consumed, amount of noise generated, etc.

56
 Rational agents

Rationality is distinct from omniscience (all-knowing

with infinite knowledge)

Agents can perform actions in order to modify future

percepts so as to obtain useful information
(information gathering, exploration)

An agent is autonomous if its behavior is determined

by its own percepts & experience (with ability to learn
and adapt)
without depending solely on build-in knowledge
57
 Task Environment

Before we design an intelligent agent, we must

specify its “task environment”:

PEAS:

Performance measure
Environment
Actuators
Sensors

58
 PEAS

Example: Agent = taxi driver

Performance measure: Safe, fast, legal, comfortable trip,

maximize profits

Environment: Roads, other traffic, pedestrians,customers

Actuators: Steering wheel, accelerator, brake, signal,

horn

Sensors: Cameras, sonar, speedometer, GPS, odometer,

engine sensors, keyboard
59
 PEAS

Example: Agent = Medical diagnosis system

Performance measure: Healthy patient, minimize costs, lawsuits

Environment: Patient, hospital, staff

Actuators: Screen display (questions, tests, diagnoses,

treatments, referrals)

Sensors: Keyboard (entry of symptoms, findings, patient's

answers)

60
 PEAS
Example: Agent = Part-picking robot

Performance measure: Percentage of parts in correct

bins

Environment: Conveyor belt with parts, bins

Actuators: Jointed arm and hand

Sensors: Camera, joint angle sensors

61
 Agent types

Five basic types in order of increasing generality:

Table Driven agent

Simple reflex agents

Model-based reflex agents

Goal-based agents

Utility-based agents
62
 Table Driven Agent.
current state of decision process

Impractical

table lookup
for entire history

63
 Simple reflex agents

Fast but too simple

NO MEMORY
Fails if environment
is partially observable

example: vacuum cleaner world

64
 Model-based reflex agents
description of Model the state of the world by:
current world state modeling how the world chances
how it’s actions change the world

• This can work even with partial information

• It’s is unclear what to do
without a clear goal

65
 Goal-based agents
Goals provide reason to prefer one action over the other.
We need to predict the future: we need to plan & search

66
 Learning agents
How does an agent improve over time?
By monitoring it’s performance and suggesting
better modeling, new action rules, etc.

Evaluates
current
world
state

changes
action
rules
“old agent”=
model world
and decide on
suggests actions
explorations to be taken

67