Scanned by CamScanner

Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
Scanned by CamScanner
 Problems, Problem Spaces and
Search

By
Prof. Mukesh Barapatre

SB JAIN INSTITUTE OF TECHNOLOGY MANAGMENT & RESEARCH ,

Department of Information Technology
 Contents
• History
• Block Architecture of AI
• Turing Test
• Typical AI Problems
• AI Technique
• Defining the problem as a State Space Search
• Production Systems
• Control Strategies
• Breadth First Search
• Depth First Search
• Heuristic Search
• Problem Characteristics
• Is the Problem Decomposable?
• Can Solution Steps be ignored or undone?
• Production system characteristics
• Issues in the design of search programs
Department of Information Technology By Mukesh Barapatre 2
 One Vision of an AI

Department of Information Technology By Mukesh Barapatre 3

 A Calmer Vision

Department of Information Technology By Mukesh Barapatre 4

 Could AI Stop This?

Department of Information Technology By Mukesh Barapatre 5

 One Vision of an AI

Department of Information Technology By Mukesh Barapatre 6

 A Calmer Vision

Department of Information Technology By Mukesh Barapatre 7

Could AI Stop This?

Department of Information Technology By Mukesh Barapatre 8

 What is Artificial Intelligence?

A.I. is the study of how to make computers do things at which, at the

moment, people are better.

Department of Information Technology By Mukesh Barapatre 9

 Or, Stepping Back Even Farther, Can We Build
Artificial People?

•Historical attempts
•The modern quest for robots and intelligent agents
•Us vs. Them

Department of Information Technology By Mukesh Barapatre 10

 Historical Attempts - Frankenstein

The original story, published

by Mary Shelley, in 1818,
describes the attempt of a true
scientist, Victor Frankenstein,
to create life.

Frankenstein creates the fiend - illustration by

http://members.aon.at/frankenstein/frankenstein-novel.htm Bernie Wrightson (© 1977)
Department of Information Technology By Mukesh Barapatre 11
 Historical Attempts – The Turk

http://www.theturkbook.com

Department of Information Technology By Mukesh Barapatre 12

 Historical Attempts - Euphonia
Joseph Faber's Amazing Talking Machine (1830-40's). The Euphonia and other early
talking devices are described in detail in a paper by David Lindsay called "Talking Head",
Invention & Technology, Summer 1997, 57-63.
About this device, Lindsay writes:
It is "... a speech synthesizer
variously known as the Euphonia and
the Amazing Talking Machine. By
pumping air with the bellows ... and
manipulating a series of plates,
chambers, and other apparatus
(including an artificial tongue ... ),
the operator could make it speak any
European language. A German
immigrant named Joseph Faber spent
From
seventeen years perfecting the http://www.haskins.yale.edu/haskins/HEADS/SIMU
Euphonia, only to find whenByheMukesh
Department of Information Technology
was Barapatre LACRA/euphonia.html 13
finished that few people cared."
 Historical Attempts - RUR

In 1921, the Czech author Karel Capek produced the play R.U.R.
(Rossum's Universal Robots).
"CHEAP LABOR. ROSSUM'S ROBOTS."
"ROBOTS FOR THE TROPICS. 150 DOLLARS EACH."
"EVERYONE SHOULD BUY HIS OWN ROBOT."
"DO YOU WANT TO CHEAPEN YOUR OUTPUT?
SomeORDER ROSSUM'S
references state that ROBOTS"
term "robot" was derived from the Czech word
robota, meaning "work", while others propose that robota actually means "forced
workers" or "slaves." This latter view would certainly fit the point that Capek was
trying to make, because his robots eventually rebelled against their creators, ran
amok, and tried to wipe out the human race. However, as is usually the case
with words, the truth of the matter is a little more convoluted. In the days when
Czechoslovakia was a feudal society, "robota" referred to the two or three days
of the week that peasants were obliged to leave their own fields to work without
remuneration on the lands of noblemen. For a long time after the feudal system
had passed away, robota continued to be used to describe work that one wasn't
exactly
Department doing voluntarily
of Information Technology or By
forMukesh
fun, while today's younger Czechs and Slovaks
Barapatre 14
tend to use robota to refer to work that’s boring or uninteresting.
http://www.maxmon.com/1921ad.htm
 The Roots of Modern Technology

5thc B.C. Aristotelian logic invented

1642 Pascal built an adding machine

1694 Leibnitz reckoning machine

Department of Information Technology By Mukesh Barapatre 15

 The Roots, continued

1834 Charles Babbage’s

Analytical Engine

Ada writes of the engine, “The Analytical

Engine has no pretensions whatever to
originate anything. It can do whatever we
know how to order it to perform.”

The picture is of a model built in the late 1800s by Babbage’s son

from Babbage’s drawings.
Department of Information Technology By Mukesh Barapatre 16
 The Roots: Logic

1848 George Boole The Calculus of Logic

chocolate and  nuts and mint

chocolate
nuts

mint

Department of Information Technology By Mukesh Barapatre 17

 Mathematics in the Early 20th Century –
(Looking Ahead: Will Logic be the Key to
Thinking?)
1900 Hilbert’s program and the effort to formalize mathematics

1931 Kurt Gödel’s paper, On Formally Undecidable Propositions

1936 Alan Turing’s paper, On Computable Numbers with an application to the

Entscheidungs problem

Department of Information Technology By Mukesh Barapatre 18

 The Advent of the Computer
1945 ENIAC The first electronic digital computer

Department of Information Technology By Mukesh Barapatre 19

 1949 EDVAC
The first stored program
computer

Department of Information Technology By Mukesh Barapatre 20

The Dartmouth Conference and the Name Artificial
Intelligence

J. McCarthy, M. L. Minsky, N. Rochester, and C.E. Shannon. August

31, 1955. "We propose that a 2 month, 10 man study of artificial
intelligence be carried out during the summer of 1956 at
Dartmouth College in Hanover, New Hampshire. The study is to
proceed on the basis of the conjecture that every aspect of
learning or any other feature of intelligence can in principle be so
precisely described that a machine can be made to simulate it."

Department of Information Technology By Mukesh Barapatre 21

 Time Line – The Big Picture

academic $ academic and routine

50 60 70 80 90 00 10

1956 Dartmouth conference.

1981 Japanese Fifth Generation project launched as the
Expert Systems age blossoms in the US.
1988 AI revenues peak at $1 billion. AI Winter begins.

Department of Information Technology By Mukesh Barapatre 22

 How Will We Recognize AI?

1950 Alan Turing’s paper, Computing Machinery and Intelligence,

described what is now called “The Turing Test”.
http://www.abelard.org/turpap/turpap.htm
1990 Loebner Prize established. Grand Prize of $100,000 and a Gold Medal
for the first computer whose responses are indistinguishable from a human.

Department of Information Technology By Mukesh Barapatre 23

 The Origins of AI Hype

1950 Turing predicted that in about fifty years "an average interrogator will not
have more than a 70 percent chance of making the right identification after five
minutes of questioning".

1957 Newell and Simon predicted that "Within ten years a computer will be the
world's chess champion, unless the rules bar it from competition."

Department of Information Technology By Mukesh Barapatre 24

 Was Turing Right?
A conversation with Jabberwacky, 2003

Jabberwacky: Have you been to the Moon?

User: No. Few have. Have you?
Jabberwacky: Many a time my dear chap!
User: What was the most interesting thing you saw?
Jabberwacky: Fred.
User: Why did you find him interesting?
Jabberwacky: It just came into my head.
User: What sorts of things do you generally find interesting?
Jabberwacky: Nothing but smelly socks.

Department of Information Technology By Mukesh Barapatre 25

 Another One

A Conversation with Alice, 2004

>What's the best place to eat Chinese food?

ALICE: My favorite places are California and Maine. Holland is cool too.
>What do you like about Maine?
ALICE: The special effects are the best.

http://www.alicebot.org/

Department of Information Technology By Mukesh Barapatre 26

 Alice, Again

>Who invented paper towels?

ALICE: The Chinese, long before Europeans.
>Who invented pet rocks?
ALICE: The Wright Brothers.
>Who invented Barbie dolls?
ALICE: Thomas Edison.

Department of Information Technology By Mukesh Barapatre 27

 Were Newell and Simon Right?

•1950 Claude Shannon published a paper describing how

a computer could play chess.

•1957 Newell and Simon predicted that a computer will

beat a human at chess within 10 years.

•1967 MacHack was good enough to achieve a class-C

rating in tournament chess.

Department of Information Technology By Mukesh Barapatre 28

 Chess Today

In 1997, Deep Blue beat Gary Kasparov.

Department of Information Technology By Mukesh Barapatre 29

 Why Did They Get it Wrong?

They failed to understand at least three key things:

•The need for knowledge (lots of it)
•Scalability and the problem of complexity and exponential growth
•The need to perceive the world

Department of Information Technology By Mukesh Barapatre 30

 Scalability

Solving hard problems requires

search in a large space.

To play master-level chess

requires searching about 8 ply
deep. So about 358 or 21012
nodes must be examined.

Department of Information Technology By Mukesh Barapatre 31

 Exponential Growth

Department of Information Technology By Mukesh Barapatre 32

 But Chess is Easy

•The rules are simple enough to fit on one page

•The branching factor is only 35.

Department of Information Technology By Mukesh Barapatre 33

 A Harder One

John saw a boy and a girl with a red wagon with one blue and one white wheel
dragging on the ground under a tree with huge branches.

Department of Information Technology By Mukesh Barapatre 34

 How Bad is the Ambiguity?
•Kim (1)
•Kim and Sue (1)
•Kim and Sue or Lee (2)
•Kim and Sue or Lee and Ann (5)
•Kim and Sue or Lee and Ann or Jon (14)
•Kim and Sue or Lee and Ann or Jon and Joe (42)
•Kim and Sue or Lee and Ann or Jon and Joe or Zak (132)
•Kim and Sue or Lee and Ann or Jon and Joe or Zak and Mel (469)
•Kim and Sue or Lee and Ann or Jon and Joe or Zak and Mel or Guy (1430)
•Kim and Sue or Lee and Ann or Jon and Joe or Zak and Mel or Guy and Jan (4862)

The number of parses for an expression with n terms is the n’th Catalan number:
 2n   2n 
Cat ( n) =   −  
 n   n − 1
Department of Information Technology By Mukesh Barapatre 35
 Can We Get Around the Search Problem ?

Department of Information Technology By Mukesh Barapatre 36

 How Much Compute Power Does it Take?

Department of Information
From Hans Technology
Moravec, Robot By Mukesh
Mere Machine BarapatreMind 1998.
to Transcendent 37
 How Much Compute Power is There?

Department of Information Technology By Mukesh Barapatre 38

From Hans Moravec, Robot Mere Machine to Transcendent Mind 1998.
 Evolution of the Main Ideas

•Wings or not?
•Games, mathematics, and other knowledge-poor tasks
•The silver bullet?
•Knowledge-based systems
•Hand-coded knowledge vs. machine learning
•Low-level (sensory and motor) processing and the resurgence
of subsymbolic systems
•Robotics
•Natural language processing
•Programming languages
•Cognitive modeling
Department of Information Technology By Mukesh Barapatre 39
 Symbolic vs. Subsymbolic AI

Subsymbolic AI: Model intelligence at a level

similar to the neuron. Let such things as
knowledge and planning emerge.

Symbolic AI: Model such things as

knowledge and planning in data (blueberry (isa fruit)
structures that make sense to the (shape round)
programmers that build them. (color purple)
(size .4 inch))

Department of Information Technology By Mukesh Barapatre 40

 The Origins of Subsymbolic AI

1943 McCulloch and Pitts A Logical Calculus of the Ideas Immanent in Nervous
Activity

“Because of the “all-or-none” character of nervous activity, neural events

and the relations among them can be treated by means of propositional
logic”
Department of Information Technology By Mukesh Barapatre 41
 Interest in Subsymbolic AI

40 50 60 70 80 90 00 10

Department of Information Technology By Mukesh Barapatre 42

 Low-level (Sensory and Motor) Processing
and the Resurgence of Sub symbolic Systems
•Computer vision
•Motor control
•Subsymbolic systems perform cognitive tasks
•Detect credit card fraud

•The backpropagation algorithm eliminated a formal weakness of earlier systems

•Neural networks learn.

Department of Information Technology By Mukesh Barapatre 43

 The Origins of Symbolic AI

•Games
•Theorem proving

Department of Information Technology By Mukesh Barapatre 44

 Games

•Chess
•Checkers:
•1952-1962 Art Samuel built the first checkers program
•Chinook became the world checkers champion in 1994
•Othello:
•Logistello beat the world champion in 1997

Department of Information Technology By Mukesh Barapatre 45

 Games

•Chess
•Checkers: Chinook became the world checkers champion in
1994
•Othello: Logistello beat the world champion in 1997

•Go:

•Role Playing Games: now weByneed

Department of Information Technology
knowledge
Mukesh Barapatre 46
 Mathematics

1956 Logic Theorist (the first running AI program?)

1961 SAINT solved calculus problems at the college

freshman level

1967 Macsyma

Gradually theorem proving has become well enough understood that it is usually no
longer considered AI

1996 J Moore and others verified the correctness of the

AMD5k86 Floating-Point Division algorithm

Department of Information Technology By Mukesh Barapatre 47

 The Silver Bullet?
Is there an “intelligence algorithm”?
1957 GPS (General Problem Solver)

Start Goal

Department of Information Technology By Mukesh Barapatre 48

 But What About Knowledge?
•Why do we need it?

Find me stuff about dogs who save people’s lives.

Around midnight, two beagles spotted a fire in the

house next door. Their barking alerted their owners,
who called 911.

•How can we represent it and use it?

•How can we acquire it?

Department of Information Technology By Mukesh Barapatre 49

 Representing Knowledge - Logic
1958 McCarthy’s paper, “Programs with Common Sense”

at(I, car)  can (go(home, airport, driving))

at(I, desk)  can(go(desk, car, walking))

1965 Resolution theorem proving invented

Department of Information Technology By Mukesh Barapatre 50

 Representing Knowledge- Semantic Nets

1961

Department of Information Technology By Mukesh Barapatre 51

 Representing Knowledge – Capturing
Experience
Representing Experience with Scripts, Frames, and Cases
1977 Scripts

Joe went to a restaurant. Joe ordered a hamburger. When the hamburger came, it
was burnt to a crisp. Joe stormed out without paying.

The restaurant script:

Did Joe eat anything?

Department of Information Technology By Mukesh Barapatre 52
 Representing Knowledge - Rules
Expert knowledge in many domains can be captured in rules.

From XCON (1982):

If: the most current active context is distributing
massbus devices, and
there is a single-port disk drive that has not been
assigned to a massbus, and
there are no unassigned dual-port disk drives, and
the number of devices that each massbus should support is known, and
there is a massbus that has been assigned at least one disk drive that
should support additional disk drives, and
the type of cable needed to connect the disk drive to the previous
device on the massbus is known
Then: assign the disk drive to the massbus.
Department of Information Technology By Mukesh Barapatre 53
 Representing Knowledge – Probabilistically
1975 Mycin attaches probability-like numbers to rules
If: (1) the stain of the ogranism is gram-positive, and
(2) the morphology of the organism is coccus, and
(3) the growth conformation of the organism is clumps
Then: there is suggestive evidence (0.7) that the identity of the organism is
stphylococcus.

1970s Probabilistic models of speech recognition

1980s Statistical Machine Translation systems
1990s large scale neural nets

Department of Information Technology By Mukesh Barapatre 54

 The Rise of Expert Systems
1967 Dendral – a rule-based system that infered molecular structure from
mass spectral and NMR data
1975 Mycin – a rule-based system to recommend antibiotic therapy
1975 Meta-Dendral learned new rules of mass spectrometry, the first
discoveries by a computer to appear in a refereed scientific journal
1979 EMycin – the first expert system shell
1980’s The Age of Expert Systems

Department of Information Technology By Mukesh Barapatre 55

 Expert Systems – The Heyday
1979 Inference
1980 IntelliCorp Selling expert system shells
1981 Teknowledge
1983 Carnegie Group
1980 XCON (R1) – first real commercial expert system at DEC,
configures VAX systems
1981 Japanese Fifth Generation project launched as the
Expert Systems age blossoms in the US.

1984 Gold Hill Common Lisp

1986 neural net start up companies appear
1988 AI revenues peak at $1 billion. AI Winter begins.

Department of Information Technology By Mukesh Barapatre 56

 Expert Systems – Today: Medicine

One example domain, medicine, has expert systems whose tasks include:
•arrhythmia recognition from electrocardiograms
•coronary heart disease risk group detection
•monitoring the prescription of restricted use antibiotics
•early melanoma diagnosis
•gene expression data analysis of human lymphoma
•breast cancer diagnosis

Department of Information Technology By Mukesh Barapatre 57

 Expert Systems – Today: Build Your Own

eXpertise2Go: http://www.expertise2go.com/
AcquiredIntelligence: http://www.aiinc.ca/demos/
(whales, graduate school)

Department of Information Technology By Mukesh Barapatre 58

 What About Things that People Do Easily?

•Common sense

•Moving Around

•Language

Department of Information Technology By Mukesh Barapatre 59

 What About Things that People Do Easily?

•Common sense
•CYC
•UT (http://www.cs.utexas.edu/users/mfkb/RKF/tree/ )
•WordNet (http://www.cogsci.princeton.edu/~wn/)
•Moving around

•Language

Department of Information Technology By Mukesh Barapatre 60

 Robotics – Hopkins Beast
1964 Two versions of the Hopkins beast, which used sonar to guide it in the
halls. Its goal was to find power outlets.

Department of Information Technology By Mukesh Barapatre 61

 Robotics - Shakey

1970 Shakey (SRI) was

driven by a remote-controlled
computer, which formulated
plans for moving and acting. It
took about half an hour to move
Shakey one meter.

Department of Information Technology By Mukesh Barapatre 62

 Robotics – Stanford Cart
1971-9 Stanford cart. Remote
controlled by person or
computer.
1971 follow the white line
1975 drive in a straight line by
tracking skyline
1979 get through obstacle
courses. Cross 30 meters in
five hours, getting lost one time
out of four

Department of Information Technology By Mukesh Barapatre 63

 Planning vs. Reacting
In the early days: substantial focus on planning (e.g., GPS)
1979 – in “Fast, Cheap and Out of Control”, Rodney Brooks argued for a very
different approach. (No, I’m not talking about the 1997 movie.)

The Ant, has 17 sensors. They are

designed to work in colonies.

http://www.ai.mit.edu/people/brooks/papers/fast-cheap.pdf
Department of Information Technology By Mukesh Barapatre 64
http://www.ai.mit.edu/projects/ants/
 Robotics - Dante

1994 Dante II (CMU) explored the Mt.

Spurr (Aleutian Range, Alaska) volcano.
High-temperature, fumarole gas samples are
prized by volcanic science, yet their
sampling poses significant challenge. In
1993, eight volcanologists were killed in
two separate events while sampling and
monitoring volcanoes.

Using its tether cable anchored at the crater rim, Dante II is able to descend down sheer
crater walls in a rappelling-like manner to gather and analyze high temperature gasses
from the crater floor.
Department of Information Technology By Mukesh Barapatre 65
 Robotics - Sojourner

Oct. 30, 1999 Sojourner on Mars. Powered by a 1.9 square foot solar array,
Sojourner can negotiate obstacles tilted at a 45 degree angle. It travels at less than
half an inch per second.
Department of Information Technology By Mukesh Barapatre
http://antwrp.gsfc.nasa.gov/apod/ap991030.html 66
 Robotics – Mars Rover

Tutorial on Rover:
http://marsrovers.jpl.nasa.gov/gallery/video/animation.html
Department of Information Technology By Mukesh Barapatre 67
 Sandstorm

March 13, 2004 - A DARPA Grand Challenge: an unmanned offroad race, 142 miles
from Barstow to Las Vegas.
Department of Information Technology By Mukesh Barapatre 68
 Moving Around and Picking Things Up

Phil, the drug robot, introduced in 2003

Department of Information Technology By Mukesh Barapatre 69
 Robotics - Aibo
1999 Sony’s Aibo pet dog

Department of Information Technology By Mukesh Barapatre 70

 What Can You Do with an Aibo?
1997 – First official Rob-Cup soccer match

Picture from 2003

competition

Department of Information Technology By Mukesh Barapatre 71

 Robotics - Cog

1998 – now Cog

Humanoid intelligence
requires humanoid
interactions with the world.

http://www.eecs.mit.edu/100th/images/Brooks-Cog-Kismet.html
Department of Information Technology By Mukesh Barapatre 72
 At the Other End of the Spectrum - Roomba

2001 A robot vacuum

cleaner

Department of Information Technology By Mukesh Barapatre 73

 Natural Language Processing
1964 STUDENT solves algebra word problems

The distance from New York to Los Angeles is 3000 miles. If the
average speed of a jet plane is 600 miles per hour, find the time it
takes to travel from New York to Los Angeles by jet.
1965 ELIZA models a Rogerian therapist
young woman: Men are all alike.
eliza: In what way?
young woman: They're always bugging us about something
specific or other.
eliza: Can you think of a specific example?
young woman: Well, my boyfriend made me come here.
Department of Information Technology By Mukesh Barapatre 74
eliza: Your boyfriend made you come here?
 NLP, continued
1966 Alpac report kills work on MT
1971 SHRDLU

Department of Information Technology By Mukesh Barapatre 75

 NLP, continued
1973 Schank – a richer limited domain: children’s stories
Suzie was invited to Mary’s birthday party. She knew she wanted a new doll
so she got it for her.
1977 Schank – scripts add a knowledge layer – restaurant
stories
1970’s and 80’s sophisticated grammars and parsers

But suppose we want generality? One approach is “shallow” systems that punt the
complexities of meaning.

Department of Information Technology By Mukesh Barapatre 76

 NLP Today
•Grammar and spelling checkers
•Spelling: http://www.spellcheck.net/

•Chatbots
•See the list at: http://www.aaai.org/AITopics/html/natlang.html#chat/

•Speech systems

•Synthesis: The IBM system:

•http://www.research.ibm.com/tts/coredemo.html

Department of Information Technology By Mukesh Barapatre 77

 Machine Translation: An Early NL Application
1949 Warren Weaver’s memo suggesting MT
1966 Alpac report kills government funding
Early 70s SYSTRAN develops direct Russian/English system
Early 80s knowledge based MT systems
Late 80s statistical MT systems

Department of Information Technology By Mukesh Barapatre 78

 Why Is It So Hard?

Olive oil

Department of Information Technology By Mukesh Barapatre 79

 Why Is It So Hard?

Olive oil

Department of Information Technology By Mukesh Barapatre 80

 Why Is It So Hard?

Peanut oil

Department of Information Technology By Mukesh Barapatre 81

 Why Is It So Hard?

Coconut oil

Department of Information Technology By Mukesh Barapatre 82

 Why Is It So Hard?

Baby oil

Department of Information Technology By Mukesh Barapatre 83

 Why Is It So Hard?

Cooking oil

Department of Information Technology By Mukesh Barapatre 84

 Text Retrieval and Extraction

•Try Ask Jeeves: http://www.askjeeves.com

•To do better requires:

•Linguistic knowledge
•World knowledge

•Newsblaster: http://www1.cs.columbia.edu/nlp/newsblaster/

Department of Information Technology By Mukesh Barapatre 85

 Programming Languages

1958 Lisp – a functional programming language with a

simple syntax.

(successor SitA ActionP)

1972 PROLOG - a logic programming language whose

primary control structure is depth-first search

ancestor(A,B) :- parent(A,B)
ancestor(A,B) :- parent(A,P), ancestor(P,B)

1988 CLOS (Common Lisp Object Standard) published.

Draws on ideas from Smalltalk and semantic nets

Department of Information Technology By Mukesh Barapatre 86

 Making Money – Software

•Expert systems to solve problems in particular domains

•Expert system shells to make it cheaper to build new systems in new domains
•Language applications
•Text retrieval
•Machine Translation
•Text to speech and speech recognition
•Data mining

Department of Information Technology By Mukesh Barapatre 87

 Making Money - Hardware
1980 Symbolics founded
1986 Thinking Machines introduces the Connection Machine
1993 Symbolics declared bankruptcy

Symbolics 3620 System c 1986:

Up to 4 Mwords (16 Mbytes)
optional physical memory, one
190 Mbyte fixed disk, integral
Ethernet interface, five backplane
expansion slots, options include an
additional 190 Mbyte disk or 1/4''
tape drive, floating point
accelerator, memory, RS232C
Department of Information Technology By Mukesh Barapatre ports and printers. 88
 Making Money - Robots
1962 Unimation, first industrial robot
company, founded. Sold a die casting robot to
GM.
1990 iRobot founded, a spinoff of MIT
2000 The UN estimated that there are
742,500 industrial robots in use worldwide.
More than half of these were being used in
Japan.
2001 iRobot markets Roomba for $200.

Department of Information Technology By Mukesh Barapatre 89

 The Differences Between Us and Them

Emotions

Understanding

Consciousness

Department of Information Technology By Mukesh Barapatre 90

 Emotions
The robot Kismet shows emotions

sad surprise

http://www.ai.mit.edu/projects/humanoid-robotics-group/kismet/
Department of Information Technology By Mukesh Barapatre 91
 Understanding

Searle’s Chinese Room

Department of Information Technology By Mukesh Barapatre 92

 Consciousness

Department of Information Technology By Mukesh Barapatre 93

 Today: The Difference Between Us and Them

Department of Information Technology By Mukesh Barapatre 94

 Today: Computer as Artist

Two paintings done by Harold Cohen’s Aaron program:

Department of Information Technology By Mukesh Barapatre 95

 Why AI?

"AI can have two purposes. One is to use the power of computers to
augment human thinking, just as we use motors to augment human or
horse power. Robotics and expert systems are major branches of that.
The other is to use a computer's artificial intelligence to understand
how humans think. In a humanoid way. If you test your programs not
merely by what they can accomplish, but how they accomplish it, they
you're really doing cognitive science; you're using AI to understand the
human mind."
- Herb Simon

Department of Information Technology By Mukesh Barapatre 96

 Block Architecture of AI System
• Collection of facts and rules.
• Place where the domain specific knowledge is stored
• Consist of body of facts axioms, rules and concepts about different entities.
• Means of utilizing facts Knowledge Inference
and rules. Base Engine
• Carries out the search
through the Knowledge base
• To prove the hypothesis & to arrive at
User
a conclusion
Interface
• Provides the needed facilities for the user to communicate
with the system.

Department of Information Technology By Mukesh Barapatre 97

 Turing Test Approach
Alan Turing (1912-1954)
“Computing Machinery and Intelligence” (1950)

To detect whether the m/c has become intelligent as human or not.

If Interrogator not able to distinguish between human & Comp then Comp posses
intelligence.

Department of Information Technology By Mukesh Barapatre 98

 Typical AI problems / Task Domains of AI
• Mundane Tasks
• Perception
• Vision
• Speech
• Natural language
• Understanding
• Generation
• Translation
• Commonsense
• Reasoning
• Robot Control

Department of Information Technology By Mukesh Barapatre 99

• Formal Tasks
• Games
• Chess
• Checkers (Strategy Board Game)
• Mathematics
• Geometry
• Logic
• Proving Properties of
• Programs

Department of Information Technology By Mukesh Barapatre 100

• Expert Tasks
• •Engineering
• • Design
• • Manufacturing and
• Planning
• •Scientific Analysis
• •Medical Analysis
• •Financial Analysis
• Advanced Task
• Brain Computer Interface

Department of Information Technology By Mukesh Barapatre 101

 AI Techniques
• There are various AI techniques which make use of following.
• Search:
• It provides a way of solving problems for which no more direct approach is
• available.
• Use of Knowledge:
• It provides a way of solving complex problems.
• Abstraction:
• It provides a way of separating important features from unimportant .
• Examples: Neural Network, Genetic Algorithms, Fuzzy logic.

Department of Information Technology By Mukesh Barapatre 102

Things required to build a system to solve an
AI problem?
• Define the problem precisely
• Analyze the problem.
• Isolate and represent the task knowledge that is necessary to solve the
problem.
• Choose the best problem solving technique and apply them to the particular
problem.

Department of Information Technology By Mukesh Barapatre 103

 Problem Space / State Space / Space of the
problem state
• Problem Space – The environment where the search is performed.

• State Space - set of all states reachable from the initial state.

• The set of all possible configuration is the space of the problem states.

Department of Information Technology By Mukesh Barapatre 104

 Introductory Problem: Tic-Tac-Toe

Comments:

1. Not efficient in time, as it has to check several

conditions before making each move.

2. Easier to understand the program’s strategy.

3. Hard to generalize.

Department of Information Technology

105 By Mukesh Barapatre
 Introductory Problem: Tic-Tac-Toe

8 3 4
1 5 9
6 7 2
15 − (8 + 5)
Department of Information Technology
106 By Mukesh Barapatre
 Introductory Problem: Tic-Tac-Toe
Comments:

1. Checking for a possible win is quicker.

2. Human finds the row-scan approach easier, while

computer finds the number-counting approach more
efficient.

Department of Information Technology

107 By Mukesh Barapatre
 Introductory Problem: Tic-Tac-Toe

Program 3:

1. If it is a win, give it the highest rating.

2. Otherwise, consider all the moves the opponent could make next.
Assume the opponent will make The move that is worst for us. Assign
the rating of that move to the current node.

3. The best node is then the one with the highest rating.

Department of Information Technology

108 By Mukesh Barapatre
 Introductory Problem: Tic-Tac-Toe

Comments:

1. Require much more time to consider all possible moves.

2. Could be extended to handle more complicated games.

Department of Information Technology

109 By Mukesh Barapatre
 To build a system to solve a problem

❖ Define the problem precisely.

❖ Analyze the problem.

❖ Isolate and represent the task knowledge that is necessary to solve the
problem.

❖ Choose the best problem-solving techniques and apply it to the particular

problem.

Department of Information Technology By Mukesh Barapatre 110

 Defining the problem as State Space Search

• The state space representation forms the basis of most of the AI methods.
• Its structure corresponds to the structure of problem solving in two
important ways:
– It allows for a formal definition of a problem as the need to convert
some given situation into some desired situation using a set of
permissible operations.
– It permits us to define the process of solving a particular problem as a
combination of known techniques (each represented as a rule defining a
single step in the space) and search, the general technique of exploring
the space to try to find some path from current state to a goal state.
– Search is a very important process in the solution of hard problems for
which no more direct techniques are available.

Department of Information Technology By Mukesh Barapatre 111

 Example: Playing Chess

• To build a program that could “play chess”, we could first have to specify
the starting position of the chess board, the rules that define the legal
moves, and the board positions that represent a win for one side or the
other.

• In addition, we must make explicit the previously implicit goal of not only
playing the legal game of chess but also winning the game, if possible,

Department of Information Technology By Mukesh Barapatre 112

 Playing chess

• The starting position can be described as an 8by 8 array where each position
contains a symbol for appropriate piece.
• We can define as our goal the check mate position.
• The legal moves provide the way of getting from initial state to a goal state.
• They can be described easily as a set of rules consisting of two parts:
– A left side that serves as a pattern to be matched against the current board
position.
– And a right side that describes the change to be made to reflect the move
• However, this approach leads to large number of rules 10120 board positions !!
• Using so many rules poses problems such as:
– No person could ever supply a complete set of such rules.
– No program could easily handle all those rules. Just storing so many rules
poses serious difficulties.
Department of Information Technology By Mukesh Barapatre 113
 Defining chess problem as State Space search

• We need to write the rules describing the legal moves in as general a way
as possible.

• For example:
– White pawn at Square( file e, rank 2) AND Square( File e, rank 3) is
empty AND Square (file e, rank 4) is empty, then move the pawn from
Square( file e, rank 2) to Square( file e, rank 4).

• In general, the more succinctly we can describe the rules we need, the less
work we will have to do to provide them and more efficient the program.

Department of Information Technology By Mukesh Barapatre 114

 Water Jug Problem

• The state space for this problem can be described as the set of ordered pairs
of integers (x,y) such that

x = 0, 1,2, 3 or 4 and
y = 0,1,2 or 3;

x - represents the number of gallons of water in the 4-gallon jug &

y - represents the quantity of water in 3-gallon jug

• The start state is (0,0)

• The goal state is (2,n)

Department of Information Technology By Mukesh Barapatre 115

 Production rules for Water Jug Problem

• The operators to be used to solve the problem can be described as follows:

Sr.No Current state Next State Description

1 (x,y) if x < 4 (4,y) Fill the 4 gallon jug

2 (x,y) if y <3 (x,3) Fill the 3 gallon jug

3 (x,y) if x > 0 (x-d, y) Pour some water out of the 4 gallon

jug

4 (x,y) if y > 0 (x, y-d) Pour some water out of the 3-gallon
jug

5 (x,y) if x>0 (0, y) Empty the 4 gallon jug

6 (x,y) if y >0 (x,0) Empty the 3 gallon jug on the ground

7 (x,y) if x+y >= 4 and y >0 (4, y-(4-x)) Pour water from the 3 –gallon jug
into the 4 –gallon jug until the 4-
gallon jug is full

Department of Information Technology By Mukesh Barapatre 116

 Production rules
8 (x, y) if x+y >= 3 and x>0 (x-(3-y), 3) Pour water from the 4-gallon jug into the
3-gallon jug until the 3-gallon jug is full

9 (x, y) if x+y <=4 and y>0 (x+y, 0) Pour all the water from the 3-gallon jug
into the 4-gallon jug

10 (x, y) if x+y <= 3 and x>0 (0, x+y) Pour all the water from the 4-gallon jug
into the 3-gallon jug

11 (0,2) (2,0) Pour the 2 gallons from 3-gallon jug into

the 4-gallon jug

12 (2,y) (0,y) Empty the 2 gallons in the 4-gallon jug on

the ground

Department of Information Technology By Mukesh Barapatre 117

 To solve the water jug problem

• Required a control structure that

loops through a simple cycle in Gallons in the 4- Gallons in the 3- Rule applied
which some rule whose left side gallon jug gallon jug
matches the current state is chosen, 0 0 2
the appropriate change to the state
is made as described in the 0 3 9
corresponding right side, and the
resulting state is checked to see if it 3 0 2
corresponds to goal state.
3 3 7
• One solution to the water jug
problem 4 2 5 or 12

• Shortest such sequence will have a 0 2 9 0r 11

impact on the choice of appropriate
mechanism to guide the search for 2 0
solution.

Department of Information Technology By Mukesh Barapatre 118

 Formal Description of the problem

1. Define a state space that contains all the possible configurations of the
relevant objects.

2. Specify one or more states within that space that describe possible
situations from which the problem solving process may start ( initial
state)

3. Specify one or more states that would be acceptable as solutions to the

problem. ( goal states)

4. Specify a set of rules that describe the actions ( operations) available.

Department of Information Technology By Mukesh Barapatre 119

 Production Systems

A production system consists of:

• A set of rules, each consisting of a left side that determines the applicability of
the rule and a right side that describes the operation to be performed if that rule is
applied.

• One or more knowledge/databases that contain whatever information is

appropriate for the particular task. Some parts of the database may be permanent,
while other parts of it may pertain only to the solution of the current problem.

• A control strategy that specifies the order in which the rules will be compared to
the database and a way of resolving the conflicts that arise when several rules
match at once.

• A rule applier

Department of Information Technology By Mukesh Barapatre 120

 Production system
In order to solve a problem:

• We must first reduce it to one for which a precise statement can be given.
This can be done by defining the problem’s state space ( start and goal
states) and a set of operators for moving that space.

• The problem can then be solved by searching for a path through the space
from an initial state to a goal state.

• The process of solving the problem can usefully be modeled as a

production system.

Department of Information Technology By Mukesh Barapatre 121

 Control Strategies

• How to decide which rule to apply next during the process of searching for
a solution to a problem?

• The two requirements of good control strategy are that

– it should cause motion.

– It should be systematic

Department of Information Technology By Mukesh Barapatre 122

 Breadth First Search
• Algorithm:

1. Create a variable called NODE-LIST and set it to initial state.

2. Until a goal state is found or NODE-LIST is empty do

a. Remove the first element from NODE-LIST and call it E. If NODE-

LIST was empty, quit
b. For each way that each rule can match the state described in E do:

i. Apply the rule to generate a new state

ii. If the new state is a goal state, quit and return this state
iii. Otherwise, add the new state to the end of NODE-LIST

Department of Information Technology By Mukesh Barapatre 123

 BFS Tree for Water Jug problem

(0,0)

(4,0) (0,3)

(0,0) (1,3) (4,3) (0,0) (3,0)

(4,3)

Department of Information Technology By Mukesh Barapatre 124

 Algorithm: Depth First Search

1. If the initial state is a goal state, quit and return success

2. Otherwise, do the following until success or failure is signaled:

a. Generate a successor, E, of initial state. If there are no more

successors, signal failure.

b. Call Depth-First Search, with E as the initial state

c. If success is returned, signal success. Otherwise continue in this

loop.

Department of Information Technology By Mukesh Barapatre 125

 Backtracking

• In this search, we pursue a single branch of the tree until it yields a solution
or until a decision to terminate the path is made.

• It makes sense to terminate a path if it reaches dead-end, produces a

previous state. In such a state backtracking occurs.

• Chronological Backtracking: Order in which steps are undone depends only

on the temporal sequence in which steps were initially made.

• Specifically most recent step is always the first to be undone.

• This is also simple backtracking.

Department of Information Technology By Mukesh Barapatre 126

 Advantages of Depth-First Search

• DFS requires less memory since only the nodes on the current path are
stored.

• By chance, DFS may find a solution without examining much of the search
space at all.

Department of Information Technology By Mukesh Barapatre 127

 Advantages of BFS

• BFS will not get trapped exploring a blind alley.

• If there is a solution, BFS is guaranteed to find it.

• If there are multiple solutions, then a minimal solution will be found.

Department of Information Technology By Mukesh Barapatre 128

 TSP
• A simple motion causing and systematic control structure could solve this
problem.

• Simply explore all possible paths in the tree and return the shortest path.

• If there are N cities, then number of different paths among them is 1.2….(N-1)
or (N-1)!

• The time to examine single path is proportional to N

• So the total time required to perform this search is proportional to N!

• For 10 cities, 10! = 3,628,800

• This phenomenon is called Combinatorial explosion.

Department of Information Technology By Mukesh Barapatre 129
 Branch and Bound
• Begin generating complete paths, keeping track of the
shortest path found so far.
• Give up exploring any path as soon as its partial length
becomes greater than the shortest path found so far.
• Using this algorithm, we are guaranteed to find the
shortest path.
• It still requires exponential time.
• The time it saves depends on the order in which paths are
explored.
Department of Information Technology By Mukesh Barapatre 130
 Heuristic Search

• A Heuristic is a technique that improves the efficiency of a search process,

possibly by sacrificing claims of completeness.
• Heuristics are like tour guides
• They are good to the extent that they point in generally interesting directions;
• They are bad to the extent that they may miss points of interest to particular
individuals.
• On the average they improve the quality of the paths that are explored.
• Using Heuristics, we can hope to get good ( though possibly nonoptimal )
solutions to hard problems such asa TSP in non exponential time.
• There are good general purpose heuristics that are useful in a wide variety of
problem domains.
• Special purpose heuristics exploit domain specific knowledge

Department of Information Technology By Mukesh Barapatre 131

 Nearest Neighbour Heuristic
• It works by selecting locally superior alternative at each step.
• Applying to TSP:
1. Arbitrarily select a starting city
2. To select the next city, look at all cities not yet visited and select the
one closest to the current city. Go to next step.
3. Repeat step 2 until all cities have been visited.
– This procedure executes in time proportional to N2
– It is possible to prove an upper bound on the error it incurs. This

provides reassurance that one is not paying too high a price in

accuracy for speed.

Department of Information Technology By Mukesh Barapatre 132

 Heuristic Function
• This is a function that maps from problem state descriptions to
measures of desirsability, usually represented as numbers.
– Which aspects of the problem state are considered,
– how those aspects are evaluated, and
– the weights given to individual aspects are chosen in such a way that
• the value of the heuristic function at a given node in the search process
gives as good an estimate as possible of whether that node is on the
desired path to a solution.
• Well designed heuristic functions can play an important part in
efficiently guiding a search process toward a solution.

Department of Information Technology By Mukesh Barapatre 133

 Example Simple Heuristic functions
• Chess : The material advantage of our side over opponent.
• TSP: the sum of distances so far
• Tic-Tac-Toe: 1 for each row in which we could win and in we already have
one piece plus 2 for each such row in we have two pieces

Department of Information Technology By Mukesh Barapatre 134

 Problem Characteristics
• In order to choose the most appropriate method for a particular
problem, it is necessary to analyze the problem along several key
dimensions:
– Is the problem decomposable into a set of independent smaller or easier
sub problems?
– Can solution steps be ignored or at least undone if they prove unwise?
– Is the problem’s universe predictable?
– Is a good solution to the problem obvious without comparison to all
other possible solutions?
– Is the desired solution a state of the world or a path to a state?
– Is a large amount of knowledge absolutely required to solve the
problem or is knowledge important only to constrain the search?
– Can a computer that is simply given the problem return the solution or
will the solution of the problem require interaction between the
computer and a person?
Department of Information Technology By Mukesh Barapatre 135
 Is the problem Decomposable?
• Whether the problem can be decomposed into smaller problems?
• Using the technique of problem decomposition, we can often solve very
large problems easily.

Department of Information Technology By Mukesh Barapatre 136

 Blocks World Problem
Start: ON(C,A)
• Following operators are A
C B
available: A B C
• CLEAR(x) [ block x Goal:
ON(B,C) and
has nothing on it]-> ON(A,B)

ON(x, Table) ON(B,C) and ON(A,B)

• CLEAR(x) and ON(B,C) ON(A,B)

CLEAR(y) -> ON(x,y)
[ put x on y] ON(B,C)
CLEAR(A)
ON(A,B)
137
CLEAR(A)
Department of Information Technology By Mukesh Barapatre ON(A,B)
 Can Solution Steps be ignored or undone?
• Suppose we are trying to prove a math theorem.We can prove a lemma. If we
find the lemma is not of any help, we can still continue.
• 8-puzzle problem
• Chess: A move cannot be taken back.
• Important classes of problems:
– Ignorable ( theorem proving)
– Recoverable ( 8-puzzle)
– Irrecoverable ( Chess)
• The recoverability of a problem plays an important role in determining the
complexity of the control structure necessary for the problem’s solution.
– Ignorable problems can be solved using a simple control structure that never
backtracks
– Recoverable problems can be solved by a slightly more complicated control strategy
that does sometimes make mistakes
– Irrecoverable problems will need to be solved by systems that expends a great deal
of effort making each decision since decision must be final.

Department of Information Technology By Mukesh Barapatre 138

 Is the universe Predictable?
• Certain Outcome ( ex: 8-puzzle)
• Uncertain Outcome ( ex: Bridge, controlling a robot arm)
• For solving certain outcome problems, open loop
approach ( without feedback) will work fine.
• For uncertain-outcome problems, planning can at best
generate a sequence of operators that has a good
probability of leading to a solution. We need to allow for
a process of plan revision to take place.

Department of Information Technology By Mukesh Barapatre 139

 Is a good solution absolute or relative?
• Any path problem
• Best path problem

• Any path problems can often be solved in a reasonable amount of time by

using heuristics that suggest good paths to explore.
• Best path problems are computationally harder.

Department of Information Technology By Mukesh Barapatre 140

 Is the solution a state or a path?
• Examples:
– Finding a consistent interpretation for the sentence “The bank
president ate a dish of pasta salad with the fork”. We need to
find the interpretation but not the record of the processing.
– Water jug : Here it is not sufficient to report that we have solved
, but the path that we found to the state (2,0). Thus the a
statement of a solution to this problem must be a sequence of
operations ( Plan) that produces the final state.

Department of Information Technology By Mukesh Barapatre 141

 What is the role of knowledge?
• Two examples:
– Chess: Knowledge is required to constrain the search for a solution
– Newspaper story understanding: Lot of knowledge is required even to be able to
recognize a solution.
• Consider a problem of scanning daily newspapers to decide which are
supporting the democrats and which are supporting the republicans in some
election. We need lots of knowledge to answer such questions as:
– The names of the candidates in each party
– The facts that if the major thing you want to see done is have taxes lowered, you are
probably supporting the republicans
– The fact that if the major thing you want to see done is improved education for
minority students, you are probably supporting the democrats.
– etc

Department of Information Technology By Mukesh Barapatre 142

Does the task require Interaction with a
person?
• The programs require intermediate interaction with people for additional
inputs and to provided reassurance to the user.
• There are two types of programs:
– Solitary
– Conversational:
• Decision on using one of these approaches will be important in the choice
of problem solving method.

Department of Information Technology By Mukesh Barapatre 143

 Problem Classification
• There are several broad classes into which the problems fall. These classes
can each be associated with generic control strategy that is appropriate for
solving the problems:
– Classification : ex: medical diagnostics, diagnosis of faults in
mechanical devices
– Propose and Refine: ex: design and planning

Department of Information Technology By Mukesh Barapatre 144

 Production System Characteristics
1. Can production systems, like problems, be described by a set of
characteristics that shed some light on how they can easily be implemented?
2. If so, what relationships are there between problem types and the types of
production systems best suited to solving the problems?
• Classes of Production systems:
– Monotonic Production System: the application of a rule never prevents the later
application of another rule that could also have been applied at the time the first
rule was selected.
– Non-Monotonic Production system
– Partially commutative Production system: property that if application of a
particular sequence of rules transforms state x to state y, then permutation of
those rules allowable, also transforms state x into state y.
– Commutative Production system

Department of Information Technology By Mukesh Barapatre 145

 Monotonic Production Systems
• Production system in which the application of a rule never prevents the
later application of another rule that could also have been applied at the
time the first rule was applied.
• i.e., rules are independent.

Department of Information Technology By Mukesh Barapatre 146

 Commutative Production system
• A partially Commutative production system has a property that if the
application of a particular sequence of rules transform state x into state y,
then any permutation of those rules that is allowable, also transforms state
x into state y.
• A Commutative production system is a production system that is both
monotonic and partially commutative.

Department of Information Technology By Mukesh Barapatre 147

 Partially Commutative, Monotonic
• These production systems are useful for solving ignorable problems.
• Example: Theorem Proving
• They can be implemented without the ability to backtrack to previous
states when it is discovered that an incorrect path has been followed.
• This often results in a considerable increase in efficiency, particularly
because since the database will never have to be restored, It is not
necessary to keep track of where in the search process every change
was made.
• They are good for problems where things do not change; new things get
created.

Department of Information Technology By Mukesh Barapatre 148

 Non Monotonic, Partially Commutative
• Useful for problems in which changes occur but can be
reversed and in which order of operations is not critical.
• Example: Robot Navigation, 8-puzzle, blocks world
• Suppose the robot has the following ops: go North (N), go
East (E), go South (S), go West (W). To reach its goal, it
does not matter whether the robot executes the N-N-E or
N-E-N.

Department of Information Technology By Mukesh Barapatre 149

 Not partially Commutative
• Problems in which irreversible change occurs
• Example: chemical synthesis
• The ops can be :Add chemical x to the pot, Change the temperature to t
degrees.
• These ops may cause irreversible changes to the potion being brewed.
• The order in which they are performed can be very important in determining
the final output.
• (X+y) +z is not the same as (z+y) +x
• Non partially commutative production systems are less likely to produce the
same node many times in search process.
• When dealing with ones that describe irreversible processes, it is partially
important to make correct decisions the first time, although if the universe is
predictable, planning can be used to make that less important.

Department of Information Technology By Mukesh Barapatre 150

 Four Categories of Production System

Monotonic NonMonotonic

Partially Theorem proving Robot Navigation

Commutative

Not Partially Chemical Synthesis Bridge

Commutative

Department of Information Technology By Mukesh Barapatre 151

 Issues in the design of search programs
• The direction in which to conduct the search ( forward versus backward
reasoning).
• How to select applicable rules ( Matching)
• How to represent each node of the search process ( knowledge
representation problem)

Department of Information Technology By Mukesh Barapatre 152

 Summary
• Four steps for designing a program to solve a problem:
1. Define the problem precisely
2. Analyse the problem
3. Identify and represent the knowledge required by the task
4. Choose one or more techniques for problem solving and apply those
techniques to the problem.

Department of Information Technology By Mukesh Barapatre 153

 Presentation on
Un informed / Informed search strategies

By
Prof. Mukesh Barapatre

SB JAIN INSTITUTE OF TECHNOLOGY MANAGMENT & RESEARCH ,

Department of Information Technology
 Heuristic Search Methods
• BFS
• DFS
• Generate And Test
• Hill Climbing
• Best First
• Problem Reduction
• Constraint Satisfaction
• Mean Ends Analysis
 Search Strategies: Blind Search

• Breadth-first search
Expand all the nodes of
one level first.

• Depth-first search
Expand one of the nodes at
the deepest level.
 BFS
• Root node is expanded first.
• Then all the successors of the root are expanded next.
• And then their successors and so on.
 Search Strategies

(0, 0)

(4, 0) (0, 3)

(4, 3) (0, 0) (1, 3) (4, 3) (0, 0) (3, 0)

 Breadth-first search

Algorithm BFS:
1. Create a variable called NODE-LIST and set it to the initial state.
2. Until a goal state is found or NODE-LIST is empty do:
 Remove the first element from NODE-LIST and call it E. If
NODE-LIST was empty, quit.
 For each way that each rule can match the state described in E do:
 Apply the rule to generate a new state.
 If the new state is a goal state, quit and return this state.
 Otherwise, add the new state to the end of NODE-LIST.
 Depth-first search

 Algorithm DFS:
1. If the initial state is a goal state, quit and return success.
2. Otherwise, do the following until success or failure is signaled:
 Generate a successor, E, of the initial state. If there are no more
successors, signal failure.
 Call Depth-First Search with E as the initial state.
 If success is returned, signal success. Otherwise continue in this
loop.
Continue this process until some rule
produces a goal state.
 Advantage BFS

1. Will not get trapped exploring blindly.

2. If there is a solution, then BFS is guaranteed to
find it.
3. If there are multiple solutions then a minimal
solution will be found.
 Advantage DFS

1. Requires less memory.

2. May find a solution without examining
much of the search space.
 Search Strategies: Blind Search

Criterion Breadth- Depth-First

First
Time bd bm
Space bd bm
Optimal? Yes No

Complete? Yes No

b: branching factor d: solution depth m: maximum depth

 Generate And Test
1. Generate A possible Solution
For some problems, this means generating a particular point in the problem space
i.e Generating a path from a start state.

2. Test to see if this Actually a Solution by compairing the chosen point or the endpoint
of the chosen path to the set of acceptable goal states.

3. If a solution has been found QUIT

Otherwise return to step-1

Exa. Throwing a die to get particular score(6) 1

6 3
 Informed Search Techniques/ Heuristic
Search
It exploit domain specific knowledge.
• Generate And Test
• Hill Climbing
– Simple hill climbing
– Steepest Ascent hill climbing
• Best First Search
– OR Graph
– A* Algorithm
• Problem Reduction
– AND OR Graph
– AO* Algorithm
• Constraint satisfaction
 Generate And Test
1. It is Depth first Search procedure since complete solution must be generated
before they can be generated

2. Exhaustive Search

3. It is also known as British Museum algorithm

(Finding an object in the museum randomly.)

4. ==DFS with Backtracking.

 Hill Climbing

 Similar to generate-and-test
 Feedback from the test procedure is used to help
the generator, decides which direction to move
in the search space.
 In pure generate & test procedure the test
function responds only Yes or No.
 It is augmented with heuristic functionshow
close a given state is to goal state.
 Simple Hill Climbing

1. Evaluate the initial state. If it is a goal state ,return & Quit else continue
with initial state as the current state.
2. Loop until a solution is found or until there are no operators left to be applied
in the current state.
a) Select an operator that has not yet been applied to the
current state and apply it to produce a new state
b) Evaluate a New State
i) If it is a goal state then return it & quit
ii) If it is not a goal state, but it is better than the
current state then make it the current state.
iii) If it is not better than the current state,…then continue in the loop.
EXA. Water jug problem by path
 Simple Hill Climbing
• At each point in the search path, a successor node that appears to lead most
quick to the goal is selected for exploration.
• Hill climbing is like Depth first Search where the most promising child is
selected for expansion to get goal state.
(or near to goal state.)
Steepest-Ascent Hill Climbing

 Consider all moves from the current state

 Select the best one as the next state
 Algorithm page 67....
 Steepest-Ascent Hill Climbing
1.Evaluate the initial state.If it is also a goal state, then return it & quit.
Otherwise continue with the initial state as the current state.
2. Loop until a solution is found or until a complete iteration produce no change to the
current state.
a) Let SUCC be a state such that any possible successor of the current state will be better
than succ.
b) For each operator that applies to the current state do:
i)Apply the operator & Generate new state.

ii)Evaluate new state..If it is a goal state then return &

quit.If not ,compare it to SUCC.If it is better,then set
SUCC to this state.If not Then leave SUCC alone.

iii)If SUCC is better than current state,then set current

state to SUCC.
 A water jug solution
4-Gallon Jug 3-Gallon Jug Rule Applied
0 0
0 3 2
3 0 9
3 3 2
4 2 7
0 2 5 or 12
2 0 9 or 11

Solution : path / plan

 Solution-2
Rules Applied--??
(0, 0)
(4, 0)
(1, 3)
(1, 0)
(0, 1)
(4, 1)
(2, 3)
 Best First Search
• It combines the advantages of DFS & BFS
• DFS:-It allows solution to be found w/o all competing
branches being expanded.
• BFS:-It does not get trapped on dead end paths.
• Follow a single path at a time, but switch path s whenever
some competing paths looks more promising than the current
one.
• We have to apply appropriate Heuristic Function.
• The old branch is not forgotten.
 Best First Search

• At each step of the BFS process

1)Select the most promising of the nodes we have generated so far.
2)Then expand the chosen node by using the rules to generate its successors
If one of them is a solution then quit.
3)If not all those new nodes are added to the set of nodes generated so far.
4)Again the most promising node is selected and the process continues.
 OR Graph

• If each of the branches represent an alternative problem solving path it is

called as OR Graph.
• In place of using search tree,it uses the search graph called as OR graph.
• Each node of the graph contains a description of the problem state.
 Step-2
Step-1
A
A

A A A

(3) (5) (1)

 Step-3
Step-4
A
A

B C D
B C D
(3) (5)
(5)
E F G H E F
(4) (6) (6) (5)
(4) (6)
Step-5

B C D

(5)
G H E F
(6) (5)
(4) (6)

I J

(2) (1)
 Lists Required?
• OPEN:-Nodes that have been generated and have had the heuristic function
applied to them but which have not been examined.
(Visited, but not expanded)
• CLOSED:-nodes that have already been examined.
(Visited, expanded)
 Algorithm :-Best First Search
1.Start with OPEN containing just the initial state.
2.Until a goal is found or there are no nodes left on OPEN do:
a) Pick them best node on OPEN
b) Generate its successor.
For each successor do:
(i) If it has not been generated before. evaluate it. add it to OPEN, and record its
parent.

(ii) If it has been generated before, change the parent if this new path is better
than the previous one.
in that case update the cost of getting to this node and to any successors that
this node may already have.

EXA. Travelling Salesman Problem.

 Heuristics: Example

Oradea Neamt
71
Zerind Iasi
87
75 151 92
Arad Vaslui
140 Sibiu99 Fagaras
118 142
Timisoara 220 Rimnicu Vilcea
211 98
Urziceni Hirsova
Pitesti 85
111 Lugoj 97
101 Bucharest
70 146
Mehadia 86
138 90
75
Dobreta Giurgiu
120 Craiova
 Notations.
• F’-Heuristic function to be used

• g-measure of the cost of getting from the initial state to the current node.

• h’ –Estimate of the additional cost of getting from the current node to the
goal state.
• F’(x)=g(x)+h(x)
 A* Algorithm
1.Place the starting node S on open.
2.If open is empty, Stop & return failure.
3.Remove from open the node n that has the smallest value of f*(n).
If the node is a goal node, return success & Stop. Otherwise.
4. Expand n, generating all of its successor n’ and place n on closed.
For every successor n’, if n’ is not already on open or closed
attach a back-pointer to n, Compute f*(n’) and place it on open.
5.Each n’ that is already on open or closed should be attached to
back-pointers which reflect the lowest g*(n’) path. If n’ was on
closed and its pointer was changed, remove it and place it on
open.
6.Return to step 2
 Heuristics: Example
• Travel: h(n) = distance(n, goal)
Oradea Neamt (234)
71 (380)
Zerind (374) Iasi (226)
87
75 151 92
Arad (366) Vaslui (199)
140 Sibiu99
(253) Fagaras
118 (176)
142
Timisoara 220 Rimnicu Vilcea
211 98
(329) (193) Urziceni Hirsova
Pitesti (98) 85 (80) (151)
111 Lugoj (244) 97
101 Bucharest (0)
70 146
Mehadia 86
138 90
75
Dobreta Giurgiu (77) Eforie (161)
(242) 120 Craiova
(160)
 Problem Characteristics
To choose an appropriate method for a
particular problem:
1) Is the problem decomposable?
2) Can solution steps be ignored or undone?
3) Is the universe predictable?
4) Is a good solution absolute or relative?
5) Is the solution a state or a path?
6) What is the role of knowledge?
7) Does the task require human-interaction?
 1) Is the problem decomposable?
• Can the problem be broken down to smaller problems to be solved
independently?

• Decomposable problem can be solved easily.

Symbolic Integration

∫ ½(1+cos2X)dx
Is the problem decomposable?

Start Goal
A ON(B,C)
C B ON(A,B)

A B C
Blocks World

CLEAR(x)  ON(x, Table)

CLEAR(x) and CLEAR(y)  ON(x, y)

Is the problem decomposable?
ON(B, C) and ON(A, B)

ON(B, C) ON(A, B)

CLEAR(A) ON(A, B)

A
C B
A B C
 2)Can solution steps be ignored or undone?

Theorem Proving
A lemma that has been proved can be ignored for next steps.

Ignorable!
Can solution steps be ignored or undone?

The 8-Puzzle

2 8 3 1 2 3
1 6 4 8 4
7 5 7 6 5

Moves can be undone and backtracked.

Recoverable!
Can solution steps be ignored or undone?

Playing Chess
Moves cannot be retracted.

Irrecoverable!
Can solution steps be ignored or undone?

• Ignorable problems can be solved using a simple control structure that

never backtracks.
• Recoverable problems can be solved using backtracking.
• Irrecoverable problems can be solved by recoverable style methods via
planning.
 3) Is the universe predictable?
The 8-Puzzle
Every time we make a move, we
know exactly what will happen.

Certain outcome!
 Is the universe predictable?

Playing Bridge
We cannot know exactly where all the cards are or what
the other players will do on their turns.

Uncertain outcome!
 Is the universe predictable?

• For certain-outcome problems, planning can used to generate a

sequence of operators that is guaranteed to lead to a solution.
• For uncertain-outcome problems, a sequence of generated
operators can only have a good probability of leading to a
solution.
Plan revision is made as the plan is carried out and the necessary
feedback is provided.
4)Is a good solution absolute or relative?

1. Marcus was a man.

2. Marcus was a Pompeian.
3. Marcus was born in 40 A.D.
4. All men are mortal.
5. All Pompeians died when the volcano
erupted in 79 A.D.
6. No mortal lives longer than 150 years.
7. It is now 2004 A.D.
Is a good solution absolute or relative?

1. Marcus was a man.

2. Marcus was a Pompeian.
3. Marcus was born in 40 A.D.
4. All men are mortal.
5. All Pompeians died when the volcano
erupted in 79 A.D.
6. No mortal lives longer than 150 years.
7. It is now 2004 A.D.

Is Marcus alive?
Is a good solution absolute or relative?

1. Marcus was a man.

2. Marcus was a Pompeian.
3. Marcus was born in 40 A.D.
4. All men are mortal.
5. All Pompeians died when the volcano
erupted in 79 A.D.
6. No mortal lives longer than 150 years.
7. It is now 2009 A.D.
Is Marcus alive?
Different reasoning paths lead to the answer. It does not
matter which path we follow.
 Solution 1
1. Marcus was man. axiom 1
4. All men are mortal. axiom 4
8. Marcus is mortal. 1,4
3. Marcus was born in 40 A.D. axiom 3
7. It is now 2009 A.D. axiom 7
9. Marcus’ age is 1961 years. 3,7
6. No mortal lives longer than 150 years axiom 6
10. Marcus is dead. 8,6,9
 Solution 2

7. It is now 2009 A.D. axiom 7

5. All Pompeians died in 79 A.D. axiom 5
11. All Pompeians are dead now. 7,5
2. Marcus was a Pompeian. axiom 2
12. Marcus is dead. 11,2
Is a good solution absolute or relative?
The Travelling Salesman Problem
We have to try all paths to find the shortest one.
Boston New York Miami Dallas Sanfransisc
o

Boston - 250 1450 1700 1300

New York 250 - 1200 1500 2900
Miami 1450 1200 - 1600 3300
Dallas 1700 1500 1600 - 1700
Sanfransisco 3000 2900 3300 1700 -
 Is a good solution absolute or relative?

• Any-path problems can be solved using heuristics that suggest good paths
to explore.

• For best-path problems, much more exhaustive search will be performed.

 5)Is the solution a state or a path?
Finding a consistent interpretation
“The bank president ate a dish of pasta salad with
the fork”.
– “bank” refers to a financial situation or to a side of a river?
– “dish” or “pasta salad” was eaten?
– Does “pasta salad” contain pasta, as “dog food” does not contain “dog”?
– Which part of the sentence does “with the fork” modify?
What if “with vegetables” is there?

No record of the processing is necessary.

 Is the solution a state or a path?

The Water Jug Problem

The path that leads to the goal must
be reported.
 Is the solution a state or a path?

• A path-solution problem can be reformulated as a state-solution problem by

describing a state as a partial path to a solution.

• The question is whether that is natural or not.

 6) What is the role of knowledge

Playing Chess
Knowledge is important only to
constrain the search for a solution.

Reading Newspaper
Knowledge is required even to be able
to recognize a solution.

Exa-Cricket news(players)
 7) Does the task require human-interaction
• Solitary problem, in which there is no intermediate communication and no
demand for an explanation of the reasoning process.

• Conversational problem, in which intermediate communication is to

provide either additional assistance to the computer or additional
information to the user.
 Knowledge Representation

By
Prof. Mukesh Barapatre

S.B. JAIN INSTITUTE OF TECHNOLOGY MANAGMENT & RESEARCH ,

Department of Information Technology

AI by Mukesh Barapatre
Knowledge Representation

Once acquired, knowledge

must be organized for use

PREPARED BY M Barapatre 2
 Introduction
• Knowledge representation is a study of ways of how knowledge is actually
represented or picturized .

• A good knowledge representation naturally represents the problem

domain

• An unintelligible knowledge representation is wrong

• Most artificial intelligence systems consist of:

– Knowledge Base
– Inference Mechanism (Engine)

PREPARED BY M Barapatre 3
• Knowledge Base
 Forms the system's intelligence source
 Inference mechanism uses to reason and draw
conclusions

• Inference mechanism: Examines the knowledge base to

answer questions, solve problems or make decisions
within the domain

PREPARED BY M Barapatre 4
• Many knowledge representation schemes
– Can be programmed and stored in memory
– Are designed for use in reasoning

• Major knowledge representation schemas:

– Production rules
– Frames

PREPARED BY M Barapatre 5
 Representation and Mapping
• Facts: things we want to represent.
• Representations of facts: things we can manipulate.
• Eg Sky is blue Sky(BLUE)

• Structuring of this entities can be done in two level

• Knowledge level
• Symbol Level

6
 Representation and Mapping
• Representation Mapping
• Forward
• Backward
reasoning
programs
Internal
Facts
Representations

English English
understanding generation
English
Representations
7
Representation and Mapping

Initial desired real reasoning Final

facts facts

forward backward
representation representation
mapping mapping
Internal Internal
representations representations
of initial facts operation of final facts
of program

8
 Representation
• Sets of Syntax and Semantic
– Convention which make it possible to describe things

– Syntax: Specified Symbols and rules.

– Semantic: How meaning is associated with symbol arrangement

allowed by syntax

PREPARED BY M Barapatre 9
 Representation and Mapping

• Spot is a dog

• Every dog has a tail

Spot has a tail

10
 Representation and Mapping

• Spot is a dog
dog(Spot)

• Every dog has a tail

x: dog(x) → hastail(x)

hastail(Spot)
Spot has a tail
11
 Representation and Mapping

• Fact-representation mapping is not one-to-one.

• Good representation can make a reasoning program trivial.

12
 Properties of Representation

Good representation:
Representational adequacy

– It’s a ability to represent required knowledge that are

needed in that domain
Inferential adequacy
– It’s a ability to manipulate the knowledge represented to
produce new knowledge

13
 Properties of Representation
• Inferential efficiency
– The ability to direct the inferential mechanisms into the
most productive directions by storing appropriate guides.
• Acquisitional efficiency
– the ability to acquire new knowledge using automatic
methods wherever possible rather than reliance on
human intervention

PREPARED BY M Barapatre 14
 Approaches to KR

• It’s a simplest way to represent facts.

• It uses set of relations to store fact as that of database.
• Each fact about a set of objects is set out systematically in columns.
• It stores set of attributes & its associated values.

Palyer Height Weight

Sachin 5.2 57
Gambhir 5.10 62

15
 Approaches to KR
Inheritable knowledge:
• This approach uses the concept of inheritance ,in which elements
of specific classes inherits attributes and values from more
general classes.
• Objects are organized into classes and classes are organized in a
generalization hierarchy.
• Inheritance is a powerful form of inference, but not adequate.

16
PREPARED BY M Barapatre 17
 Approaches to KR

Inferential knowledge:
▪ Facts represented in a logical form, which facilitates reasoning.
▪ Inheritance is not the only form to represent the knowledge.
▪ This approach make use of power of traditional logic
▪ It make use of predicate logic to represent the knowledge.
▪ An inference engine is required.

18
 Approaches to KR
Procedural knowledge:
• Representation of “how to make it” rather than “what it is”.
• May have inferential efficiency, but no inferential adequacy and
acquisitional efficiency.
• In this approach Knowledge is encoded by using some procedures.
• The most common way is simply a code in some programming
language Such as LISP.
• The machine uses the knowledge when it executes the code to perform
the task.

19
 Issues in Knowledge Representation
• Important Attributes.
• Relationship Among Attributes.
• Choosing the granularity of representation.
• Representing Set of objects.
• Finding right structure as needed.

PREPARED BY M Barapatre 20
 Important Attributes
• Are there any attributes so basic, that occur in many different types of
problem?
• If there are we need to make sure that they are handled appropriately
in each of problem.
• There are two such attributes as instance & isa, which are important
because each supports property of inheritance.

PREPARED BY M Barapatre 21
 Relationship Among Attributes
• The attributes we use to describe object are themselves act as entity.
• The relationship between attributes of an object, is independent of
specific knowledge they encode, & may hold properties like
– Inverse
– Existence in a isa hierarchy
– Technique for reasoning about values.
– Single value attributes.

PREPARED BY M Barapatre 22
 Relationship Among Attributes

• Inverse
– Entities in the world are related to each other in many different ways.
– Eg Student- Read -Book, Student-Purchase-Book.
– Eg team ( Pee-wee-reese, Brooklyn-Dodgers)
– 1st assumption team=pee wee reese team= Brooklyn Dodgers
– 2nd assumption team member =pee wee reese team= Brooklyn
Dodgers
Existence in a isa hierarchy
– Generalization & Specialization relationships for attributes are
important because they support inheritance

PREPARED BY M Barapatre 23
 Relationship Among Attributes
• Technique for reasoning about values.
– Some constraints has to be followed by the value of an attributes.
– Information about type of value.
– Eg Value of height must be a number measured in unit of length.
– Constraints on the value
– Eg Age of person can not be greater than his parents age.
• Single value attributes
– This is about specific attribute that is guaranteed to take unique
value.
– Eg Baseball player can at a time have single height.

PREPARED BY M Barapatre 24
Choosing Granularity of Representation
• High-level facts may not be adequate for inference.
• Low-level primitives may require a lot of storage.
• At what level of detail the knowledge should be represented & what
should be our primitive.
• Eg Depending on the level of audience we should represent
knowledge.

25
 Representing Set of objects.
• There are certain properties of an object which are true as member of
• set but not as individual.
• Eg A person is very strict at a company but he is very calm at his
• home.
• Eg Page number written on page of book.

PREPARED BY M Barapatre 26
 Finding right structure as needed.
• Selection of appropriate structure is necessary in order to represent
knowledge effectively.
Following problems were raised while selecting appropriate structure.
• How to perform initial selection of most appropriate structure.
• How to fill appropriate details from current situation.
• How to find better structure if one chosen is not appropriate.
• What to do if none of the available structure is appropriate

PREPARED BY M Barapatre 27
 Ontology
• Ontology: the study of "being" or existence
• An ontology = "A catalog of types of things that are assumed to exist in
a domain of interest" (Sowa, 2000)
• An ontology = "The arrangement of kinds of things into types and
categories with a well-defined structure" (Passin 2004)

28
Ontology
top-level
categories

domain-specific

29
 Ontology

Aristotle's categories Being

Substance Accident

Property Relation

Inherence Directedness Containment

Quality Quantity Movement Intermediacy Spatial Temporal

Activity Passivity Having Situated

30
 Ontology

Geographical categories Geographical-Feature

Area Point Line

Block Dam On-Land On-Water

Terrain Town Road
Country Bridge Border River

Wetland Airstrip Railroad

Mountain Heliport Power-Line

31
 Ontology

Relation

32
 Ontology

Relation

33
 Ontology

ANIMAL
Eat
FOOD

PERSON: john Eat CAKE: *

34
 CG Projection

1 2
PERSON: john Has-Relative PERSON: *

1 2
PERSON: john Has-Wife WOMAN: mary

35
Representation in Logic and Other Schemas

• General form of any logical process

• Inputs (Premises)

• Premises used by the logical process to create the output,

consisting of conclusions (inferences)

• Facts known true can be used to derive new facts that are true

PREPARED BY M Barapatre 36
• Symbolic logic: System of rules and procedures that
permits the drawing of inferences from various
premises

• Basic Forms of Computational Logic

 Propositional logic (or propositional calculus)
 Predicate logic (or predicate calculus)

Department of Information Technology By Mukesh Barapatre

 Propositional Logic
• Is the branch of logic that studies way of joining or modifying
statements or sentence into more complicated statements

• A proposition is a statement that is either true or false

• Once known, it becomes a premise that can be used to derive new

propositions or inferences

• Rules are used to determine the truth (T) or falsity (F) of the new
proposition

PREPARED BY M Barapatre 38
• A: Ram is boy
• B: He is well mannered
• A and B
• Ram is boy and well mannered

PREPARED BY M Barapatre 39
 Properties of PL

• Prepositional Logic is declarative (piece of syntax corresponds to fact)

• Prepositional logic is compositional
• Meaning in Prepositional logic is content independent

PREPARED BY M Barapatre 40
 Syntax for Preposition Logic
• Symbols
• Truth Symbols
• Connectives

PREPARED BY M Barapatre 41
 Using Propositional Logic
Representing simple facts
It is raining
RAINING
It is sunny
SUNNY
It is windy
WINDY
If it is raining, then it is not sunny
RAINING → SUNNY

42
 Examples
Weather examples
1. It is hot.
2. It is humid.
3. It is raining.

If it is humid, then it is hot.

If it is hot and humid then it is not raining.

PREPARED BY M Barapatre 43
Weather examples
1. It is hot. -A
2. It is humid. - B
3. It is raining. - c

If it is humid, then it is hot. B→A

If it is hot and humid then it is not raining.

(A  B)→ ¬ C

PREPARED BY M Barapatre 44
 Using Propositional Logic

• Theorem proving is decidable

• Cannot represent objects and quantification

45
 Using Predicate Logic

• Can represent objects and quantification

• Theorem proving is semi-decidable

46
 Using Predicate Logic

1. Marcus was a man.

2. Marcus was a Pompeian.
3. All Pompeians were Romans.
4. Caesar was a ruler.
5. All Pompeians were either loyal to Caesar or hated him.
6. Every one is loyal to someone.
7. People only try to assassinate rulers they are not loyal to.
8. Marcus tried to assassinate Caesar.

47
 Using Predicate Logic

1. Marcus was a man.

man(Marcus)

48
 Using Predicate Logic

2. Marcus was a Pompeian.

Pompeian(Marcus)

49
 Using Predicate Logic

3. All Pompeians were Romans.

x: Pompeian(x) → Roman(x)

50
 Using Predicate Logic

4. Caesar was a ruler.

ruler(Caesar)

51
 Using Predicate Logic

5. All Pompeians were either loyal to Caesar or hated him.

inclusive-or
x: Roman(x) → loyalto(x, Caesar)  hate(x, Caesar)

exclusive-or
x: Roman(x) → (loyalto(x, Caesar)  hate(x, Caesar)) 
(loyalto(x, Caesar)  hate(x, Caesar))

52
 Using Predicate Logic

6. Every one is loyal to someone.

x: y: loyalto(x, y) y: x: loyalto(x, y)

53
 Using Predicate Logic

7. People only try to assassinate rulers they are not loyal to.

x: y: person(x)  ruler(y)  tryassassinate(x, y)

→ loyalto(x, y)

54
 Using Predicate Logic

8. Marcus tried to assassinate Caesar.

tryassassinate(Marcus, Caesar)

55
 Using Predicate Logic

Was Marcus loyal to Caesar?

man(Marcus)
ruler(Caesar)
tryassassinate(Marcus, Caesar)
 x: man(x) → person(x)
loyalto(Marcus, Caesar)

56
 Using Predicate Logic
• Many English sentences are ambiguous.
• There is often a choice of how to represent knowledge.
• Obvious information may be necessary for reasoning
• We may not know in advance which statements to deduce (P or P).

57
 Reasoning

1. Marcus was a Pompeian.

2. All Pompeians died when the volcano erupted in 79 A.D.
3. It is now 2008 A.D.

Is Marcus alive?

58
 Reasoning

1. Marcus was a Pompeian.

Pompeian(Marcus)

2. All Pompeians died when the volcano erupted in 79 A.D.

erupted(volcano, 79)  x: Pompeian(x) → died(x, 79)

3. It is now 2008 A.D.

now = 2008

59
 Reasoning

1. Marcus was a Pompeian.

Pompeian(Marcus)

2. All Pompeians died when the volcano erupted in 79 A.D.

erupted(volcano, 79)  x: Pompeian(x) → died(x, 79)

3. It is now 2008 A.D.

now = 2008

x: t1: t2: died(x, t1)  greater-than(t2, t1) → dead(x, t2)

60
 Resolution

Robinson, J.A. 1965. A machine-oriented logic based on the resolution

principle. Journal of ACM 12 (1): 23-41.

61
 Resolution

The basic ideas

KB |=   KB   |= false

62
 Resolution

The basic ideas

KB |=   KB   |= false

(  )  (   )  (  )

63
 Resolution

The basic ideas

KB |=   KB   |= false

(  )  (   )  (  )

sound and complete

64
Resolution in Propositional/Predicate Logic

1. Convert all the propositions of KB to clause form (S).

2. Negate  and convert it to clause form. Add it to S.
3. Repeat until either a contradiction is found or no progress can be
made.
a. Select two clauses (  P) and (   P).
b. Add the resolvent (  ) to S.

65
 Resolution in Propositional Logic

Example:

KB = {P, (P  Q) → R, (S  T) → Q, T}

=R

66
 Resolution in Predicate Logic

Example:

KB = {P(a), x: (P(x)  Q(x)) → R(x), y: (S(y)  T(y)) → Q(y), T(a)}

 = R(a)

67
 Resolution in Predicate Logic

Unification:
UNIFY(p, q) = unifier  where SUBST(, p) = SUBST(, q)

68
 Resolution in Predicate Logic

Unification:
x: knows(John, x) → hates(John, x)
knows(John, Jane)
y: knows(y, Leonid)
y: knows(y, mother(y))
x: knows(x, Elizabeth)

UNIFY(knows(John, x), knows(John, Jane)) = {Jane/x}

UNIFY(knows(John, x), knows(y, Leonid)) = {Leonid/x, John/y}
UNIFY(knows(John, x), knows(y, mother(y))) = {John/y, mother(John)/x}
UNIFY(knows(John, x), knows(x, Elizabeth)) = FAIL
69
 Resolution in Predicate Logic

Unification: Standardization
UNIFY(knows(John, x), knows(y, Elizabeth)) = {John/y, Elizabeth/x}

70
 Resolution in Predicate Logic

Unification: Most general unifier

UNIFY(knows(John, x), knows(y, z)) = {John/y, John/x, John/z}
= {John/y, Jane/x, Jane/z}
= {John/y, v/x, v/z}
= {John/y, z/x, Jane/v}
= {John/y, z/x}

71
 Resolution in Predicate Logic

Unification: Occur check

UNIFY(knows(x, x), knows(y, mother(y))) = FAIL

72
 Conversion to Clause Form

1. Eliminate →.
P → Q  P  Q
2. Reduce the scope of each  to a single term.
(P  Q)  P  Q
(P  Q)  P  Q
x: P  x: P
x: p  x: P
 P  P

3. Standardize variables so that each quantifier binds a unique variable.

(x: P(x))  (x: Q(x))  (x: P(x))  (y: Q(y))

73
 Conversion to Clause Form
4. Move all quantifiers to the left without changing their relative order.
(x: P(x))  (y: Q(y))  x: y: (P(x)  (Q(y))
5. Eliminate  (Skolemization).
x: P(x)  P(c) Skolem constant
x: y P(x, y)  x: P(x, f(x))Skolem function
6. Drop .
x: P(x)  P(x)
7. Convert the formula into a conjunction of disjuncts.
(P  Q)  R  (P  R)  (Q  R)
8. Create a separate clause corresponding to each conjunct.
9. Standardize apart the variables in the set of obtained clauses.
74
 Conversion to Clause Form

1. Eliminate →.
2. Reduce the scope of each  to a single term.
3. Standardize variables so that each quantifier binds a unique variable.
4. Move all quantifiers to the left without changing their relative order.
5. Eliminate  (Skolemization).
6. Drop .
7. Convert the formula into a conjunction of disjuncts.
8. Create a separate clause corresponding to each conjunct.
9. Standardize apart the variables in the set of obtained clauses.
75
 FOL To CNF
“Every one who love all animal is loved by someone”
Fol➔
x:[y:animal(y)→ love(x,y)]→ [y:love(y,x)]

Remove implification
x:[y:  animal(y)  love(x,y)]→ [y:love(y,x)]
x:[y:  animal(y)  love(x,y)]→ [y:love(y,x)]

PREPARED BY M Barapatre 76
 Example
1. Marcus was a man.
2. Marcus was a Pompeian.
3. All Pompeians were Romans.
4. Caesar was a ruler.
5. All Pompeians were either loyal to Caesar or hated him.
6. Every one is loyal to someone.
7. People only try to assassinate rulers they are not loyal to.
8. Marcus tried to assassinate Caesar.

77
 Example

1. Man(Marcus).
2. Pompeian(Marcus).
3. x: Pompeian(x) → Roman(x).
4. ruler(Caesar).
5. x: Roman(x) → loyalto(x, Caesar)  hate(x, Caesar).
6. x: y: loyalto(x, y).
7. x: y: person(x)  ruler(y)  tryassassinate(x, y)
→ loyalto(x, y).
8. tryassassinate(Marcus, Caesar).

78
 Example

Prove:
hate(Marcus, Caesar)

79
 Question Answering

1. When did Marcus die?

2. Whom did Marcus hate?
3. Who tried to assassinate a ruler?
4. What happen in 79 A.D.?.
5. Did Marcus hate everyone?

80
 Question Answering
PROLOG:
• Only Horn sentences are acceptable

81
 Question Answering

PROLOG:
• Only Horn sentences are acceptable
• The occur-check is omitted from the unification: unsound
test  P(x, x)
P(x, f(x))

82
 Question Answering

PROLOG:
• Only Horn sentences are acceptable
• The occur-check is omitted from the unification: unsound
test  P(x, x)
P(x, f(x))
• Backward chaining with depth-first search: incomplete
P(x, y)  Q(x, y)
P(x, x)
Q(x, y)  Q(y, x)
83
 Question Answering

PROLOG:
• Unsafe cut: incomplete
A  B, C A
B  D, !, E
D  B, C

 D, !, E, C

 !, E, C
84
 Question Answering

PROLOG:
• Unsafe cut: incomplete
A  B, C A
B  D, !, E
D  B, C

 D, !, E, C

 !, E, C

• Negation as failure: P if fails to prove P 85

• Symbols represent propositions, premises or
conclusions
Statement: A = The mail carrier comes Monday through Friday.
Statement: B = Today is Sunday.
Conclusion: C = The mail carrier will not come today.

• Propositional logic: limited in representing real-

world knowledge

PREPARED BY M Barapatre 86
 Predicate Calculus

 Predicate logic breaks a statement down into component parts, an object,

object characteristic or some object assertion
 Predicate calculus uses variables and functions of variables in a symbolic
logic statement
 Predicate calculus is the basis for Prolog (PROgramming in LOGic)
 www.swi-prolog.org
 Prolog Statement Examples
 comes_on(mail_carrier, monday).
 likes(jay, chocolate).

(Note - the period “.” is part of the statement)

PREPARED BY M Barapatre 87
 Scripts

Knowledge Representation Scheme

Describing a Sequence of Events

• Elements include
– Entry Conditions
– Props
– Roles
– Tracks
– Scenes

PREPARED BY M Barapatre 88
 Semantic Networks

• Graphic Depiction of Knowledge

• Nodes and Links Showing Hierarchical Relationships Between Objects

• Nodes: Objects

• Arcs: Relationships
– is-a
– has-a

PREPARED BY M Barapatre 89
• Semantic networks can show inheritance

• Semantic Nets - visual representation of relationships

• Can be combined with other representation methods

PREPARED BY M Barapatre 90
 Semantic Network Example
Human
Boy Being

Goes to Needs
Joe Woman
School

Has Food
a child
Kay

PREPARED BY M Barapatre 91
 Frames
Definitions and Overview

• Frame: Data structure that includes all the knowledge about a

particular object
• Knowledge organized in a hierarchy for diagnosis of knowledge
independence
• Form of object-oriented programming for AI and ES.

• Each Frame Describes One Object

• Special Terminology

PREPARED BY M Barapatre 92
 Frame Terminology

Default Instantiation

Demon Master frame

Facet Object

Hierarchy of Range
frames

If added Slot

If needed Value (entry)

Instance of

PREPARED BY M Barapatre 93
• Concise, natural, structural representation of knowledge
• Encompasses complex objects, entire situations or a management
problem as a single entity
• Frame knowledge is partitioned into slots
• Slot can describe declarative knowledge or procedural knowledge

• Major Capabilities of Frames

• Typical frame describing an automobile

• Hierarchy of Frames: Inheritance

PREPARED BY M Barapatre 94
 Frame Capabilities

Ability to clearly document information about a domain model; for example,

a plant's machines and their associated attributes

Related ability to constrain the allowable values that an attribute can take on

Modularity of information, permitting ease of system expansion and

maintenance

More readable and consistent syntax for referencing domain objects in the
rules

Platform for building a graphic interface with object graphics

Mechanism that will allow us to restrict the scope of facts considered during
forward or backward chaining

Access to a mechanism that supports the inheritance of information down a

class hierarchy

PREPARED BY M Barapatre 95
 Multiple Knowledge
Representations
• Rules + Frames
• Others

Knowledge Representation Must Support

• Acquiring knowledge
• Retrieving knowledge
• Reasoning

PREPARED BY M Barapatre 96
 Structured Knowledge

By
Prof. Mukesh Barapatre

S.B. JAIN INSTITUTE OF TECHNOLOGY MANAGMENT & RESEARCH ,

Department of Information Technology
Knowledge Representation as a medium for human
expression
• An intelligent system must have KRs that can be
interpreted by humans.

– We need to be able to encode information in the knowledge base

without significant effort.

– We need to be able to understand what the system knows and how

it draws its conclusions.

Department of Information Technology by Mukesh Barapatre 2

 Knowledge Representation

• Logic (prepositional, predicate)

• Network representation
– Semantic nets
• Structured representation
– Frames
• Issues in KR
– Hierarchies, inheritance, exceptions
• Advantages and disadvantages

Department of Information Technology by Mukesh Barapatre 3

 Semantic Networks

• First introduced by Quillian back in the late-60s

M. Ross Quillian. "Semantic Memories", In M. M. Minsky, editor, Semantic
Information Processing, pages 216-270. Cambridge, MA: MIT Press, 1968

• Semantic network is simple representation scheme which

uses a graph of labeled nodes and labeled directed arcs to
encode knowledge
– Nodes – objects, concepts, events
– Arcs – relationships between nodes
• Graphical depiction associated with semantic networks is
a big reason for their popularity
Department of Information Technology by Mukesh Barapatre 4
 Nodes and Arcs
• Arcs define binary relations which hold between objects
denoted by the nodes.

mother age
Sue John 5 mother (john, sue)
age (john, 5)
age father wife (sue, max)
age (max, 34)
…
34 Max

Department of Information Technology by Mukesh Barapatre 5

 Non-binary relations

• We can represent the generic give event as a relation

involving three things:
– A giver
– A recipient
– An object

recipient giver
Mary GIVE John

object

book
Department of Information Technology by Mukesh Barapatre 6
 Inheritance

• Inheritance is one of the main kind of Animal

reasoning done in semantic nets isa
• The ISA (is a) relation is often used to Bird
hasPart
Wings
link a class and its superclass.
isa
• Some links (e.g. haspart) are
inherited along ISA paths Robin
• The semantics of a semantic net can isa isa
be relatively informal or very formal
– Often defined at the implementation level Rusty Red

Department of Information Technology by Mukesh Barapatre 7

 Multiple Inheritance

• A node can have any number of superclasses that contain it,

enabling a node to inherit properties from multiple parent nodes
and their ancestors in the network. It can cause conflicting
inheritance.
Nixon Diamond
(two contradictory inferences from the same data)

subclass Person subclass P ? !P

pacifist non-pacifist
Quaker Republican Q R

instance N
instance
Nixon
Department of Information Technology by Mukesh Barapatre 8
 Advantages of Semantic nets

• Easy to visualize
• Formal definitions of semantic networks have been
developed.
• Related knowledge is easily clustered.
• Efficient in space requirements
– Objects represented only once
– Relationships handled by pointers

Department of Information Technology by Mukesh Barapatre 9

 Disadvantages of Semantic nets

• Inheritance (particularly from multiple sources and when

exceptions in inheritance are wanted) can cause problems.
• Facts placed inappropriately cause problems.
• No standards about node and arc values

Department of Information Technology by Mukesh Barapatre 10

 Conceptual Graphs

• Conceptual graphs are semantic nets representing the meaning

of (simple) sentences in natural language
• Two types of nodes:
– Concept nodes; there are two types of concepts, individual concepts and
generic concepts
– Relation nodes(binary relations between concepts)

GO
NEW YORK
JOHN
Who Where
How

Department of Information Technology BUS

by Mukesh Barapatre 11
 Frames
• Frames – semantic net with properties
• A frame represents an entity as a set of slots (attributes) and
associated values
• A frame can represent a specific entry, or a general concept
• Frames are implicitly associated with one another because the
value of a slot can be another frame

3 components of a frame Book Frame

•frame name Slot → Filler
•attributes (slots) •Title → AI. A modern Approach
•values (fillers: list of values,
•Author → Russell & Norvig
range, string, etc.)
•Year
Department of Information → 2003
Technology
12
by Mukesh Barapatre
 Features of Frame Representation

• More natural support of values then semantic nets (each

slots has constraints describing legal values that a slot can
take)
• Can be easily implemented using object-oriented
programming techniques
• Inheritance is easily controlled

Department of Information Technology by Mukesh Barapatre 13

 Inheritance
• Similar to Object-Oriented programming paradigm
Hotel Chair
•what → chair
Hotel Room •height →20-40cm Hotel Phone
•what → room •legs →4
•what → phone
•where →hotel
•billing → guest
•contains→
–hotel chair
Hotel Bed
–hotel phone
–hotel bed •what → bed
•size →king
Mattress
•part → mattress
•price → 100$
Department of Information Technology
14
by Mukesh Barapatre
 Frames
Animals
Alive T
Flies F

isa
Birds Mammals
Legs 2 Legs 4
Flies T

isa
Penguins Cats Bats
Flies F Legs 2
Flies T

instance
Opus Bill Pat
Name Opus Name Bill Name Pat
Friend Friend
Department of Information Technology by Mukesh Barapatre 15
Modern Data-Bases combine three approaches:
conceptual graphs, frames, predicate logic
(relational algebra)

Department of Information Technology by Mukesh Barapatre 16

 Benefits of Frames

• Makes programming easier by grouping related knowledge

• Easily understood by non-developers
• Expressive power
• Easy to set up slots for new properties and relations
• Easy to include default information and detect missing
values

Department of Information Technology by Mukesh Barapatre 17

 Drawbacks of Frames

• No standards (slot-filler values)

• More of a general methodology than a specific
representation:
– Frame for a class-room will be different for a professor and for a
maintenance worker
• No associated reasoning/inference mechanisms

Department of Information Technology by Mukesh Barapatre 18

 Conceptual Dependency
• Schank’s theory
• Offers a set of four equal and independent
primitive conceptualizations
• From the primitives the word of meaning is built

Department of Information Technology

19
by Mukesh Barapatre
Conceptual dependency theory: An Example

Department of Information Technology by Mukesh Barapatre 20

• The primitives are used to define conceptual dependency relationships
• Conceptual syntax rules

Department of Information Technology by Mukesh Barapatre 21

Some basic conceptual dependencies and their use in representing
more complex English sentences

Department of Information Technology by Mukesh Barapatre 22

Conceptual dependency representing “John ate the egg”

 the direction of dependency

➔ The agent-verb relationship
P past tense
INGEST a primitive act of the theory
O object relation
D the direction of the object in the action
Department of Information Technology by Mukesh Barapatre 23
Conceptual dependency representation of the sentence “John
prevented Mary from giving a book to Bill”

More tenses and modes

p past
f future
t transition
k continuing
c conditional
/ negative
? Interrogative
Department of Information Technology by Mukesh Barapatre pil present 24
 Scripts

• Designed by Schank in 1974

• A structured representation describing a stereotyped sequence of
events in a particular context
• A means of organizing conceptual dependency structures
• Used in natural language understanding for knowledge base

Department of Information Technology by Mukesh Barapatre 25

 Scripts
• Why represent knowledge in this way?
– Because real-world events do follow stereotyped
patterns. Human beings use previous experiences to
understand verbal accounts; computers can use scripts
instead.
– Because people, when relating events, do leave large
amounts of assumed detail out of their accounts. People
don't find it easy to converse with a system that can't
fill in missing conversational detail.
Department of Information Technology by Mukesh Barapatre 26
 Script Components

• Entry conditions or descriptors of the world that must be

true for the script to be called.
• Results or facts that are true once the script has
terminated.
• Props or the “things” that support the content of the script.
• Roles are actions that the individual participants perform
• Scenes are a sequence of what represents a temporal
aspect of the script.

Department of Information Technology by Mukesh Barapatre 27

 A
Restaurant
Script

Department of Information Technology by Mukesh Barapatre 28

What is Natural Language Processing (NLP)
• The process of computer analysis of input provided in a human
language (natural language), and conversion of this input into a useful
form of representation.
• The field of NLP is primarily concerned with getting computers to
perform useful and interesting tasks with human languages.
• The field of NLP is secondarily concerned with helping us come to a
better understanding of human language.

Department of Information Technology by Mukesh Barapatre 29

 Forms of Natural Language
• The input/output of a NLP system can be:
– written text
– speech
• We will mostly concerned with written text (not speech).
• To process written text, we need:
– lexical, syntactic, semantic knowledge about the
language
– discourse information, real world knowledge
• To process spoken language, we need everything required to process
written text, plus the challenges of speech recognition and speech
synthesis.

Department of Information Technology by Mukesh Barapatre 30

 Components of NLP
• Natural Language Understanding
– Mapping the given input in the natural language into a useful representation.
– Different level of analysis required:
morphological analysis,
syntactic analysis,
semantic analysis,
discourse analysis, …
• Natural Language Generation
– Producing output in the natural language from some internal representation.
– Different level of synthesis required:
deep planning (what to say),
syntactic generation
• NL Understanding is much harder than NL Generation. But, still both of them are
hard.

Department of Information Technology by Mukesh Barapatre 31

 Why NL Understanding is hard?
• Natural language is extremely rich in form and structure, and
very ambiguous.
– How to represent meaning,
– Which structures map to which meaning structures.
• One input can mean many different things. Ambiguity can be at different
levels.
– Lexical (word level) ambiguity -- different meanings of words
– Syntactic ambiguity -- different ways to parse the sentence
– Interpreting partial information -- how to interpret pronouns
– Contextual information -- context of the sentence may affect the
meaning of that sentence.
• Many input can mean the same thing.
• Interaction among components of the input is not clear.

Department of Information Technology by Mukesh Barapatre 32

 Knowledge of Language
• Phonology – concerns how words are related to the sounds that realize
them.

• Morphology – concerns how words are constructed from more basic

meaning units called morphemes. A morpheme is the primitive unit of
meaning in a language.

• Syntax – concerns how can be put together to form correct sentences and
determines what structural role each word plays in the sentence and what
phrases are subparts of other phrases.

• Semantics – concerns what words mean and how these meaning combine
in sentences to form sentence meaning. The study of context-independent
meaning.
Department of Information Technology by Mukesh Barapatre 33
 Knowledge of Language (cont.)
• Pragmatics – concerns how sentences are used in different situations
and how use affects the interpretation of the sentence.

• Discourse – concerns how the immediately preceding sentences

affect the interpretation of the next sentence. For example, interpreting
pronouns and interpreting the temporal aspects of the information.

• World Knowledge – includes general knowledge about the world.

What each language user must know about the other’s beliefs and
goals.

Department of Information Technology by Mukesh Barapatre 34

 Ambiguity
I made her duck.

• How many different interpretations does this sentence have?

• What are the reasons for the ambiguity?
• The categories of knowledge of language can be thought of as
ambiguity resolving components.
• How can each ambiguous piece be resolved?
• Does speech input make the sentence even more ambiguous?
– Yes – deciding word boundaries

Department of Information Technology by Mukesh Barapatre 35

 Ambiguity (cont.)
• Some interpretations of : I made her duck.
1. I cooked duck for her.
2. I cooked duck belonging to her.
3. I created a toy duck which she owns.
4. I caused her to quickly lower her head or body.
5. I used magic and turned her into a duck.
• duck – morphologically and syntactically ambiguous:
noun or verb.
• her – syntactically ambiguous: dative or possessive.
• make – semantically ambiguous: cook or create.
• make – syntactically ambiguous:
– Transitive – takes a direct object. => 2
– Di-transitive – takes two objects. => 5
– Takes a direct object and a verb. => 4

Department of Information Technology by Mukesh Barapatre 36

 Ambiguity in a Turkish Sentence
• Some interpretations of: Adamı gördüm.
1. I saw the man.
2. I saw my island.
3. I visited my island.
4. I bribed the man.
• Morphological Ambiguity:
– ada-m-ı ada+P1SG+ACC
– adam-ı adam+ACC
• Semantic Ambiguity:
– gör to see
– gör to visit
– gör to bribe

Department of Information Technology by Mukesh Barapatre 37

 Resolve Ambiguities
• We will introduce models and algorithms to resolve ambiguities at
different levels.
• part-of-speech tagging -- Deciding whether duck is verb or noun.
• word-sense disambiguation -- Deciding whether make is create or
cook.
• lexical disambiguation -- Resolution of part-of-speech and
word-sense ambiguities are two important kinds of lexical
disambiguation.
• syntactic ambiguity -- her duck is an example of syntactic
ambiguity, and can be addressed by probabilistic parsing.

Department of Information Technology by Mukesh Barapatre 38

 Resolve Ambiguities (cont.)
I made her duck

S S

NP VP NP VP

I V NP NP I V NP

made her duck made DET N

her duck

Department of Information Technology by Mukesh Barapatre 39

Models to Represent Linguistic Knowledge
• We will use certain formalisms (models) to represent the required
linguistic knowledge.
• State Machines -- FSAs, FSTs, HMMs, ATNs, RTNs
• Formal Rule Systems -- Context Free Grammars, Unification
Grammars, Probabilistic CFGs.
• Logic-based Formalisms -- first order predicate logic, some higher
order logic.
• Models of Uncertainty -- Bayesian probability theory.

Department of Information Technology by Mukesh Barapatre 40

 Algorithms to Manipulate Linguistic Knowledge
• We will use algorithms to manipulate the models of linguistic
knowledge to produce the desired behavior.
• Most of the algorithms we will study are transducers and parsers.
– These algorithms construct some structure based on their input.
• Since the language is ambiguous at all levels,
these algorithms are never simple processes.
• Categories of most algorithms that will be used can fall into following
categories.
– state space search
– dynamic programming

Department of Information Technology by Mukesh Barapatre 41

 Language and Intelligence
Turing Test

Computer Human

Human Judge

• Human Judge asks tele-typed questions to Computer and Human.

• Computer’s job is to act like a human.
• Human’s job is to convince Judge that he is not machine.
• Computer is judged “intelligent” if it can fool the judge
• Judgment of intelligence is linked to appropriate answers to questions
from the system.

Department of Information Technology by Mukesh Barapatre 42

 NLP - an inter-disciplinary Field
• NLP borrows techniques and insights from several disciplines.
• Linguistics: How do words form phrases and sentences? What
constraints the possible meaning for a sentence?
• Computational Linguistics: How is the structure of sentences are
identified? How can knowledge and reasoning be modeled?
• Computer Science: Algorithms for automatons, parsers.
• Engineering: Stochastic techniques for ambiguity resolution.
• Psychology: What linguistic constructions are easy or difficult for
people to learn to use?
• Philosophy: What is the meaning, and how do words and sentences
acquire it?

Department of Information Technology by Mukesh Barapatre 43

 Some Buzz-Words
• NLP – Natural Language Processing
• CL – Computational Linguistics
• SP – Speech Processing
• HLT – Human Language Technology
• NLE – Natural Language Engineering
• SNLP – Statistical Natural Language Processing
• Other Areas:
– Speech Generation, Text Generation, Speech Understanding,
Information Retrieval,
– Dialogue Processing, Inference, Spelling Correction, Grammar
Correction,
– Text Summarization, Text Categorization,
Department of Information Technology by Mukesh Barapatre 44
 Some NLP Applications
• Machine Translation – Translation between two natural languages.
– See the Babel Fish translations system on Alta Vista.
• Information Retrieval – Web search (uni-lingual or multi-lingual).
• Query Answering/Dialogue – Natural language interface with a
database system, or a dialogue system.
• Report Generation – Generation of reports such as weather reports.
• Some Small Applications –
– Grammar Checking, Spell Checking, Spell Corrector

Department of Information Technology by Mukesh Barapatre 45

 Brief History of NLP
• 1940s –1950s: Foundations
– Development of formal language theory (Chomsky, Backus, Naur, Kleene)
– Probabilities and information theory (Shannon)
• 1957 – 1970s:
– Use of formal grammars as basis for natural language processing (Chomsky,
Kaplan)
– Use of logic and logic based programming (Minsky, Winograd, Colmerauer, Kay)
• 1970s – 1983:
– Probabilistic methods for early speech recognition (Jelinek, Mercer)
– Discourse modeling (Grosz, Sidner, Hobbs)
• 1983 – 1993:
– Finite state models (morphology) (Kaplan, Kay)
• 1993 – present:
– Strong integration of different techniques, different areas.

Department of Information Technology by Mukesh Barapatre 46

 Natural Language Understanding

Words
Morphological Analysis
Morphologically analyzed words (another step: POS tagging)
Syntactic Analysis
Syntactic Structure
Semantic Analysis
Context-independent meaning representation
Discourse Processing
Final meaning representation

Department of Information Technology by Mukesh Barapatre 47

 Natural Language Generation
Meaning representation
Utterance Planning
Meaning representations for sentences
Sentence Planning and Lexical Choice
Syntactic structures of sentences with lexical choices
Sentence Generation
Morphologically analyzed words
Morphological Generation
Words

Department of Information Technology by Mukesh Barapatre 48

 Morphological Analysis
• Analyzing words into their linguistic components (morphemes).
• Morphemes are the smallest meaningful units of language.
cars car+PLU
giving give+PROG
geliyordum gel+PROG+PAST+1SG - I was coming
• Ambiguity: More than one alternatives
flies flyVERB+PROG
flyNOUN+PLU

adamı adam+ACC - the man

(accusative)
adam+P1SG - my man
ada+P1SG+ACC - my island
(accusative)

Department of Information Technology by Mukesh Barapatre 49

 Morphological Analysis (cont.)
• Relatively simple for English. But for some languages such as Turkish, it is
more difficult.
uygarlaştıramadıklarımızdanmışsınızcasına
uygar-laş-tır-ama-dık-lar-ımız-dan-mış-sınız-casına
uygar +BEC +CAUS +NEGABLE +PPART +PL +P1PL +ABL +PAST +2PL +AsIf
“(behaving) as if you are among those whom we could not civilize/cause to become civilized”
+BEC is “become” in English
+CAUS is the causative voice marker on a verb
+PPART marks a past participle form
+P1PL is 1st person plural possessive marker
+2PL is 2nd person plural
+ABL is the ablative (from/among) case marker
+AsIf is a derivational marker that forms an adverb from a finite verb form
+NEGABLE is “not able” in English
• Inflectional and Derivational Morphology.
• Common tools: Finite-state transducers

Department of Information Technology by Mukesh Barapatre 50

 Part-of-Speech (POS) Tagging
• Each word has a part-of-speech tag to describe its category.
• Part-of-speech tag of a word is one of major word groups
(or its subgroups).
– open classes -- noun, verb, adjective, adverb
– closed classes -- prepositions, determiners, conjuctions, pronouns,
particples
• POS Taggers try to find POS tags for the words.
• duck is a verb or noun? (morphological analyzer cannot make decision).
• A POS tagger may make that decision by looking the surrounding words.
– Duck! (verb)
– Duck is delicious for dinner. (noun)

Department of Information Technology by Mukesh Barapatre 51

 Lexical Processing
• The purpose of lexical processing is to determine meanings of
individual words.
• Basic methods is to lookup in a database of meanings -- lexicon
• We should also identify non-words such as punctuation marks.
• Word-level ambiguity -- words may have several meanings, and the
correct one cannot be chosen based solely on the word itself.
– bank in English
– yüz in Turkish
• Solution -- resolve the ambiguity on the spot by POS tagging (if
possible) or pass-on the ambiguity to the other levels.

Department of Information Technology by Mukesh Barapatre 52

 Syntactic Processing
• Parsing -- converting a flat input sentence into a hierarchical structure
that corresponds to the units of meaning in the sentence.
• There are different parsing formalisms and algorithms.
• Most formalisms have two main components:
– grammar -- a declarative representation describing the syntactic
structure of sentences in the language.
– parser -- an algorithm that analyzes the input and outputs its
structural representation (its parse) consistent with the grammar
specification.
• CFGs are in the center of many of the parsing mechanisms. But they are
complemented by some additional features that make the formalism
more suitable to handle natural languages.
Department of Information Technology by Mukesh Barapatre 53
 Semantic Analysis
• Assigning meanings to the structures created by syntactic analysis.
• Mapping words and structures to particular domain objects in way
consistent with our knowledge of the world.
• Semantic can play an import role in selecting among competing
syntactic analyses and discarding illogical analyses.
– I robbed the bank -- bank is a river bank or a financial
institution
• We have to decide the formalisms which will be used in the meaning
representation.

Department of Information Technology by Mukesh Barapatre 54

 Knowledge Representation for NLP
• Which knowledge representation will be used depends on the application -
- Machine Translation, Database Query System.
• Requires the choice of representational framework, as well as the specific
meaning vocabulary (what are concepts and relationship between these
concepts -- ontology)
• Must be computationally effective.
• Common representational formalisms:
– first order predicate logic
– conceptual dependency graphs
– semantic networks
– Frame-based representations

Department of Information Technology by Mukesh Barapatre 55

 Discourse
• Discourses are collection of coherent sentences (not arbitrary set of
sentences)
• Discourses have also hierarchical structures (similar to sentences)
• anaphora resolution -- to resolve referring expression
– Mary bought a book for Kelly. She didn’t like it.
• She refers to Mary or Kelly. -- possibly Kelly
• It refers to what -- book.
– Mary had to lie for Kelly. She didn’t like it.
• Discourse structure may depend on application.
– Monologue
– Dialogue
– Human-Computer Interaction

Department of Information Technology by Mukesh Barapatre 56

 Natural Language Generation
• NLG is the process of constructing natural language outputs from
non-linguistic inputs.
• NLG can be viewed as the reverse process of NL understanding.
• A NLG system may have two main parts:
– Discourse Planner -- what will be generated. which
sentences.
– Surface Realizer -- realizes a sentence from its internal
representation.
• Lexical Selection -- selecting the correct words describing the
concepts.
Department of Information Technology by Mukesh Barapatre 57
 Machine Translation
• Machine Translation -- converting a text in language A into the
corresponding text in language B (or speech).
• Different Machine Translation architectures:
– interlingua based systems
– transfer based systems
• How to acquire the required knowledge resources such as mapping
rules and bi-lingual dictionary? By hand or acquire them automatically
from corpora.
• Example Based Machine Translation acquires the required knowledge
(some of it or all of it) from corpora.

Department of Information Technology by Mukesh Barapatre 58

 Machine Learning
What is learning ? Some Quotations
• Herbert Simon, 1983
Learning denotes changes in a system that enable a system to do the same
task more efficiently the next time.
• Marvin Minsky, 1986
Learning is making useful changes in the workings of our minds.
• Ryszard Michalski, 1986
Learning is constructing or modifying representations of what is being
experienced.
• Mitchell, 1997
A computer program is said to learn from experience E with respect to
some class of tasks T and performance measure P, if its performance at
tasks in T, as measured by P, improves with experience E.

Department of Information Technology by Mukesh Barapatre 59

 Definition
A computer program is said to learn from experience E with respect to
some class of tasks T and performance measure P, if its performance
at tasks in T, as measured by P, improves with experience E. (Mitchell
1997) This means :
Given : A task T
A performance measure P
Some experience E with the task
Goal : Generalize the experience in a way that
allows to improve your performance on the task.

Department of Information Technology by Mukesh Barapatre 60

 Why do you require Machine Learning ?

• Understand and improve efficiency of human learning.

• Discover new things or structure that is unknown to humans.

• Fill in skeletal or incomplete specifications about a domain.

Department of Information Technology by Mukesh Barapatre 61

 Learning Agents

• An agent is an entity that is capable of perceiving and do action.

An agent can be viewed as perceiving its environment through
sensors and acting upon that environment through actuators.

Environment Sensors Actuators

Agent

Human agent Eyes, ears, etc Leg, hands, mouth

Robotic agent Cameras, IR range finders motors

Software agent Key stroke, File contents Displays to screen,

write files

In computer science an agent is a software agent that assists users and

acts in performing computer-related tasks.
Department of Information Technology by Mukesh Barapatre 62
 Intelligent Agent (Learning Agent)

• Agent is an entity that is capable of perceiving and do action. In

computer science an agent is a software agent.
• In artificial intelligence, the term used for agent is an intelligent agent.
Learning is an important feature of “Intelligence”.

Percept : agent's perceptual inputs

Percept sequence : history of everything the agent has perceived
Agent function : describes agent’s behavior
Agent program : Implements agent's function

Department of Information Technology by Mukesh Barapatre 63

Components of a Learning System

Department of Information Technology by Mukesh Barapatre 64

 Components of a Learning System
• Performance Element: The Performance Element is the agent itself
that acts in the world. It takes in percepts and decides on external
actions.
• Learning Element: It responsible for making improvements, takes
knowledge about performance element and some feedback,
determines how to modify performance element.
• Critic: Tells the Learning Element how agent is doing (success or
failure) by comparing with a fixed standard of performance.
• Problem Generator: Suggests problems or actions that will generate
new examples or experiences that will aid in training the system
further.
Department of Information Technology by Mukesh Barapatre 65
 Example : Automated Taxi on city roads
• Performance Element: Consists of knowledge and procedures for
driving actions.
e.g., turning, accelerating, braking are performance element on roads.
• Learning Element: Formulates goals.
e. g., learn rules for braking, accelerating, learn geography of the city.
• Critic: Observes world and passes information to learning element.
e. g. , quick right turn across three lanes of traffic, observe reaction of
other drivers.
• Problem Generator: Try south city road .

Department of Information Technology by Mukesh Barapatre 66

 Paradigms of Machine Learning
• Rote learning – One-to-one mapping from inputs to stored
representation. “Learning by memorization.” Association-based storage
and retrieval.
• Induction – Use specific examples to reach general conclusions
• Clustering – Unsupervised identification of natural groups in data
• Analogy – Determine correspondence between two different
representations
• Discovery – Unsupervised, specific goal not given
• Genetic algorithms – “Evolutionary” search techniques, based on an
analogy to “survival of the fittest”
• Reinforcement – Feedback (positive or negative reward) given at the
end of a sequence of steps
Department of Information Technology by Mukesh Barapatre 67
 Expert System

By
Prof. Mukesh Barapatre

S.B. JAIN INSTITUTE OF TECHNOLOGY MANAGMENT & RESEARCH ,

Department of Information Technology
 Syllabus
• Knowledge acquisition methods
• knowledge engineering process.
• goals in knowledge system development.
• basic architecture of expert system.
• problem domain versus knowledge domain,
• Development of ES and life cycle of ES.
• Advantages of expert system.
• structure of Rule based expert system.
• characteristics of conventional system and expert system.

Department of Information Technology AI by Mukesh Barapatre 2

 What is an expert system?

“An expert system is a computer system that

emulates, or acts in all respects, with the
decision-making capabilities of a human expert.”

Professor Edward Feigenbaum

Stanford University
Department of Information Technology AI by Mukesh Barapatre
3
 Definition
• An expert system, is an interactive computer-based decision tool that uses both facts and
heuristics to solve difficult decision making problems, based on knowledge acquired
from an expert.

• An expert system is a model and associated procedure that exhibits, within a specific
domain, a degree of expertise in problem solving that is comparable to that of a human
expert.

• An expert system compared with traditional computer :

Inference engine + Knowledge = Expert system

( Algorithm + Data structures = Program in traditional computer )

• First expert system, called DENDRAL, was developed in the early 70's at Stanford
University.

Department of Information Technology AI by Mukesh Barapatre 4

 Definition
• A piece of software which uses databases of expert
knowledge to offer advice or make decision in such
areas as medical diagnosis, account, coding, games, etc.

Department of Information Technology AI by Mukesh Barapatre 5

 Expert System Main Components

• Knowledge base – obtainable from books, magazines, knowledgeable

persons, etc.

• Inference engine – draws conclusions from the knowledge base

Department of Information Technology AI by Mukesh Barapatre 6

 Basic Functions of Expert Systems

Department of Information Technology AI by Mukesh Barapatre

7
 4 Components of an ES
1. Knowledge Base

2. Reasoning or Inference Engine

3. User Interface

4. Explanation Facility
 Expert System Structure

Knowledge
Based Rules Expert
Interpreter Natural
Inference Language
Data base
Engine Context Set Interface
of facts
User
 Knowledge Base

• Represents all the data and information imputed by experts in the field.
• Stores the data as a set of rules that the system must follow to make
decisions.

Department of Information Technology AI by Mukesh Barapatre 10

 Inference Engine
• Asks the user questions about what they are looking for.

• Applies the knowledge and the rules held in the knowledge base.

• Appropriately uses this information to arrive at a decision.

Department of Information Technology AI by Mukesh Barapatre 11

 User Interface
• Allows the expert system and the user to communicate.

• Finds out what it is that the system needs to answer.

• Sends the user questions or answers and receives their response.

Department of Information Technology AI by Mukesh Barapatre 12

 Explanation Facility
• Explains the systems reasoning and justifies its
conclusions.

Department of Information Technology AI by Mukesh Barapatre 13

 Knowledge Acquisition

Expert Knowledge Human

System Engineer Expert
 Knowledge Engineering
• An engineering discipline that involves integrating knowledge into
computer systems in order to solve complex problems normally requiring
a high level of human expertise (Feigenbaum and Pamela, 1983)
• It normally involves five distinct steps in transferring human knowledge
into some form of knowledge based systems (KBS)

Department of Information Technology AI by Mukesh Barapatre 15

 Five Steps of Knowledge Engineering
• Knowledge acquisition
• Knowledge validation
• Knowledge representation
• Inferencing
• Explanation and justification

Department of Information Technology AI by Mukesh Barapatre 16

Two Main Views of Knowledge Engineering
• Transfer view – This is the traditional view. In this view, the
key idea is to apply conventional knowledge engineering
techniques to transfer human knowledge into the
computerized system.
• Modeling view – In this view, the knowledge engineer
attempts to model the knowledge and problem solving
techniques of the domain expert into the computerized
system.

Department of Information Technology AI by Mukesh Barapatre 17

 Knowledge Engineering (KE) vs. Knowledge
Management (KM)
• KE is primarily concerned with constructing a knowledge-
bases system while KM is primarily concerned with identifying
and leveraging knowledge to the organization’s benefit.
• KE and KM activities are inherently interrelated.
• Knowledge engineers are interested in what technologies are
needed to meet the enterprise’s KM needs.

Department of Information Technology AI by Mukesh Barapatre 18

 Knowledge Engineers
• A knowledge engineer is responsible for obtaining knowledge from human
experts and then entering this knowledge into some form of KBS.
• In developing KBS, the knowledge engineer must apply methods, use tools,
apply quality control and standards, plan and manage projects, and take into
account human, financial, and environmental constraints.
• Required skills of a knowledge engineer
• Knowledge representation
– Fact finding (knowledge elicitation)
– Human skills
– Visualization skills
– Analysis
– Creativity
– Managerial

Department of Information Technology AI by Mukesh Barapatre 19

 Knowledge acquisition methods
• Transferring knowledge from the human expert to a computer is often the
most difficult part of building an expert system.

• The knowledge acquired from the human expert must be encoded in such
a way that it remains a faithful representation of what the expert knows,
and it can be manipulated by a computer.

Department of Information Technology AI by Mukesh Barapatre 20

 Knowledge acquisition methods
Issues in Knowledge Acquisition
The important issues in knowledge acquisition are
• knowledge is in the head of experts
• Experts have vast amounts of knowledge
• Experts have a lot of tacit knowledge
‡ They do not know all that they know and use
‡ Tacit knowledge is hard (impossible) to describe
• Experts are very busy and valuable people
• One expert does not know everything
• Knowledge has a "shelf life"
Department of Information Technology AI by Mukesh Barapatre 21
 Techniques for Knowledge Acquisition
The techniques for acquiring, analyzing and modeling knowledge are :
• Protocol-generation techniques,
• Protocol analysis techniques,
• Hierarchy-generation techniques,
• Matrix-based techniques,
• Sorting techniques,
• Limited-information and constrained-processing tasks,
• Diagram-based techniques.
• Each of these are briefly stated in next few slides.

Department of Information Technology AI by Mukesh Barapatre 22

 Protocol-generation techniques
• Include many types of interviews (unstructured, semi-structured and
structured), reporting and observational techniques.

Department of Information Technology AI by Mukesh Barapatre 23

 Protocol analysis techniques
• Used with transcripts of interviews or text-based information to identify
basic knowledge objects within a protocol, such as goals, decisions,
relationships and attributes. These act as a bridge between the use of
protocol-based techniques and knowledge modeling techniques.

Department of Information Technology AI by Mukesh Barapatre 24

 Hierarchy-generation techniques
Involve creation, reviewing and modification of hierarchical knowledge.
Hierarchy-generation techniques, such as laddering, are used to
build taxonomies or other hierarchical structures such as goal trees
and decision networks. The Ladders are of various forms like concept
ladder, attribute ladder, composition ladders.

Department of Information Technology AI by Mukesh Barapatre 25

 Matrix-based techniques

• Involve the construction and filling-in a 2-D matrix (grid, table), indicating
such things, as may be, for example, between concepts and properties
(attributes and values) or between problems and solutions or between
tasks and resources, etc. The elements within the matrix can contain:
symbols (ticks, crosses, question marks ) , colors , numbers , text.

Department of Information Technology AI by Mukesh Barapatre 26

 Sorting techniques

Used for capturing the way people compare and order concepts;
it may reveal knowledge about classes, properties and priorities.

Department of Information Technology AI by Mukesh Barapatre 27

 Limited-information and constrained-
processing tasks
• Techniques that either limit the time and/or information available to the
expert when performing tasks. For example, a twenty-questions technique
provides an efficient way of accessing the key information in a domain in
a prioritized order.

Department of Information Technology AI by Mukesh Barapatre 28

 Diagram-based techniques
• Include generation and use of concept maps, state transition networks,
event diagrams and process maps. These are particularly important
in capturing the "what, how, when, who and why" of tasks and events.

Department of Information Technology AI by Mukesh Barapatre 29

 Problem Domain vs. Knowledge Domain

• An expert’s knowledge is specific to one problem

domain – medicine, finance, science,
engineering, etc.
• The expert’s knowledge about solving specific
problems is called the knowledge domain.
• The problem domain is always a superset of the
knowledge domain.

Department of Information Technology AI by Mukesh Barapatre 30

Problem and Knowledge Domain Relationship

Department
of
Information
31 Technology
AI by
 Representing the Knowledge

The knowledge of an expert system can be

represented in a number of ways, including IF-
THEN rules:

IF you are hungry THEN eat

Department of Information Technology AI by Mukesh Barapatre 32

 Development of an Expert System

Department of Information Technology AI by Mukesh Barapatre 33

Department of Information Technology AI by Mukesh Barapatre 34
Cyclical development
 Cyclical development
• In the 1980s (and before), the favourite approach, was based on rapid
prototyping (also known as cyclical development or evolutionary
prototyping).
 Cyclical development
• The essential principles:
• get a prototype - a small, preliminary version of the final system - up &
running at an early stage;
• present this to the domain expert, for criticism;
• proceed to refine this prototype with repeated debugging &
knowledge accretion stages;
• continue with this cycle until the knowledgebase is finished.
 Cyclical development

Knowledge
acquisition

Prototype
Prototype
critiquing
development
 Cyclical development

Knowledge Preliminary
acquisition exploration of field -
initial k.e. interviews

Prototype
Prototype
critiquing
development
 Cyclical development

Knowledge Build a small system

acquisition containing a few of
the features

Prototype
Prototype
critiquing
development
 Cyclical development
Present the
prototype to the Knowledge
domain expert for acquisition
him/her to criticise
and improve

Prototype
Prototype
critiquing
development
 Cyclical development

Knowledge Correct & expand the

acquisition knowledge on the
basis of the D.E.’s
comments

Prototype
Prototype
critiquing
development
 Cyclical development

Knowledge Add more features,

acquisition and debug existing
features

Prototype
Prototype
critiquing
development
 Cyclical development
Present the revised
prototype to the domain Knowledge
expert for him/her to acquisition
criticise and improve

Prototype
Prototype
critiquing
development
 Cyclical development

Knowledge Correct & expand the

acquisition knowledge on the
basis of the D.E.’s
comments

Prototype
Prototype
critiquing
development
 Cyclical development

Knowledge Add more features,

acquisition and debug exisiting
features

Prototype
Prototype
critiquing
development
 Cyclical development
Present the revised
prototype to the domain Knowledge
expert for him/her to acquisition
criticise and improve

Prototype
Prototype
critiquing
development
 Cyclical development
• This has the advantage that the KE is able to show early progress in the
Knowledge elicitation task.
 Cyclical development
• It also generates enthusiasm in the domain expert.
 Turban’s account of the
expert system development
lifecycle
 The expert system development lifecycle

• The expert system development lifecycle, as described by Turban

• Turban (1993) identifies the following phases and sub-phases in the

development of an expert system. This is probably a good account of the
way knowledge engineering projects are currently organised in America.
 The expert system development lifecycle

• Phase 1: Project initialisation

– Problem definition
– Needs assessment
– Evaluation of alternative solutions
– Verification that an ES approach is appropriate
– Consideration of management issues
 The expert system development lifecycle

• Comment on Phase 1:
– it's important to discover what problem/problems the client expects
the system to solve for them, and what their real needs are. The
problem may very well be that more knowledge is needed in the
organisation, but there may be other, better ways to provide it.
– 'Management issues' include availability of finance, legal constraints,
and finding a 'champion' in top management.
 The expert system development lifecycle

• Phase 2: System analysis & design

– Produce conceptual design
– Decide development strategy
– Decide sources of knowledge, and ensure co-operation
– Select computer resources
– Perform a feasibility study
– Perform a cost-benefit analysis
 The expert system development lifecycle

• Comment on Phase 2:
– the 'conceptual design' will describe the general capabilities of the
intended system, and the required resources.
– The problems of selecting software, and finding a domain expert (and
persuading him/her to co-operate) have been discussed in the last two
lectures.
 The expert system development lifecycle

• Comments on Phase 3:
– Comments: See below on the question of what exactly this
prototype is used for.
– It's important to establish the feasibility (economic, technical and
operational) of the system before too much work has been done, and
it's easier to do this if a prototype has been built.
 The expert system development lifecycle

• Phase 4: System development

– Build the knowledge base
– Test, evaluate and improve the knowledge base
– Plan for integration
 The expert system development
lifecycle
• Comments on Phase 4:
– See below on the question of how the knowledge base, as
constructed, relates to the prototype.
– The evaluation of an expert system (in terms of validation and
verification) is a particularly difficult problem.
 The expert system development
lifecycle
• Phase 5: Implementation
– Ensure acceptance by users
– Install, demonstrate and deploy the system
– Arrange orientation and training for the users
– Ensure security
– Provide documentation
– Arrange for integration and field testing
 The expert system development
lifecycle
• Comments on Phase 5:
– If the system is not accepted by the users, the project has largely
been a waste of time.
– Field testing (leading to refinement of the system) is essential, but may
be quite lengthy.
 The expert system development
lifecycle
• Phase 6: Post-implementation
– Operation
– Maintenance
– Upgrading
– Periodic evaluation
 The expert system development
lifecycle
• Comments on Phase 6:
– A person or group of people must be put in charge of maintenance
(and, perhaps, expansion). They are responsible for correcting bugs,
and updating the knowledgebase. They must therefore have some
knowledge engineering skills.
– The system should be evaluated, once or twice a year, in terms of its
costs & benefits, its accuracy, its accessibility, and its acceptance.
 The expert system development
lifecycle
• Turban leaves open the options that the prototype which features in
phase 3 might be an evolutionary prototype or a throwaway prototype.
 The Role of AI

• An algorithm is an ideal solution guaranteed to

yield a solution in a finite amount of time.
• When an algorithm is not available or is
insufficient, we rely on artificial intelligence
(AI).
• Expert system relies on inference – we accept a
“reasonable solution.”

Department of Information Technology AI by Mukesh Barapatre 64

 Shallow and Deep Knowledge

• It is easier to program expert systems with

shallow knowledge than with deep knowledge.
• Shallow knowledge – based on empirical and
heuristic knowledge.
• Deep knowledge – based on basic structure,
function, and behavior of objects.

Department of Information Technology AI by Mukesh Barapatre 65

 Early Expert Systems
• DENDRAL – used in chemical mass
spectroscopy to identify chemical constituents
• MYCIN – medical diagnosis of illness
• DIPMETER – geological data analysis for oil
• PROSPECTOR – geological data analysis for
minerals
• XCON/R1 – configuring computer systems

Department of Information Technology AI by Mukesh Barapatre 66

 Broad Classes of Expert Systems

Department of Information Technology AI by Mukesh Barapatre 67

 Problems with Algorithmic Solutions
• Conventional computer programs generally solve
problems having algorithmic solutions.

• Algorithmic languages include C, Java, and C#.

• Classic AI languages include LISP and

PROLOG.

Department of Information Technology AI by Mukesh Barapatre 68

Considerations for Building Expert Systems

• Can the problem be solved effectively by

conventional programming?
• Is there a need and a desire for an expert system?
• Is there at least one human expert who is willing to
cooperate?
• Can the expert explain the knowledge to the
knowledge engineer can understand it.
• Is the problem-solving knowledge mainly heuristic
and uncertain?

Department of Information Technology AI by Mukesh Barapatre 69

 Languages, Shells, and Tools

• Expert system languages are post-third

generation.
• Procedural languages (e.g., C) focus on
techniques to represent data.
• More modern languages (e.g., Java) focus on data
abstraction.
• Expert system languages (e.g. CLIPS) focus on
ways to represent knowledge. Department
of
Information
70 Technology
AI by
Expert systems Vs conventional programs I

Department
of
Information
71 Technology
AI by
Expert systems Vs conventional programs II

Department
of
Information
72 Technology
AI by
Expert systems Vs conventional programs III

Department
of
Information
73 Technology
AI by
 Elements of an Expert System

• User interface – mechanism by which user and

system communicate.
• Exploration facility – explains reasoning of
expert system to user.
• Working memory – global database of facts used
by rules.
• Inference engine – makes inferences deciding
which rules are satisfied and prioritizing.
Department of Information Technology AI by Mukesh Barapatre 74
 Elements Continued

• Agenda – a prioritized list of rules created by the

inference engine, whose patterns are satisfied by
facts or objects in working memory.
• Knowledge acquisition facility – automatic way for
the user to enter knowledge in the system bypassing
the explicit coding by knowledge engineer.
• Knowledge Base – includes the rules of the expert
system
Department of Information Technology AI by Mukesh Barapatre 75
 Production Rules

• Knowledge base is also called production memory.

• Production rules can be expressed in IF-THEN

pseudocode format.

• In rule-based systems, the inference engine

determines which rule antecedents are satisfied by
the facts.
Department of Information Technology AI by Mukesh Barapatre 76
 Structure of a Rule-Based Expert System

Department of Information Technology AI by Mukesh Barapatre 77

 Rule-Based ES

Department of Information Technology AI by Mukesh Barapatre 78

 Example Rules

Department of Information Technology AI by Mukesh Barapatre 79

 Inference Engine Cycle

80
Department of Information Technology AI by Mukesh Barapatre
 Foundation of Expert Systems

Department of Information Technology AI by Mukesh Barapatre 81

 General Methods of Inference

• Forward chaining (data-driven)– reasoning from

facts to the conclusions resulting from those facts –
best for prognosis, monitoring, and control.
– Examples: CLIPS, OPS5
• Backward chaining (query driven)– reasoning in
reverse from a hypothesis, a potential conclusion to
be proved to the facts that support the hypothesis –
best for diagnosis problems.
– Examples: MYCIN

Department of Information Technology AI by Mukesh Barapatre 82

 Production Systems

• Rule-based expert systems – most popular type

today.
• Knowledge is represented as multiple rules that
specify what should/not be concluded from different
situations.
• Forward chaining – start w/facts and use rules do
draw conclusions/take actions.
• Backward chaining – start w/hypothesis and look for
rules that allow hypothesis to be proven true.
Department of Information Technology AI by Mukesh Barapatre 83
 Forward/Backward Chaining

• Forward chaining – primarily data-driven.

• Backward chaining – primarily goal driven.

Department of Information Technology AI by Mukesh Barapatre 84

 Post Production System

• Basic idea – any mathematical / logical system is

simply a set of rules specifying how to change one
string of symbols into another string of symbols.
• these rules are also known as rewrite rules
• simple syntactic string manipulation
• no understanding or interpretation is required\also used to define
grammars of languages
– e.g BNF grammars of programming languages.
• Basic limitation – lack of control mechanism to
guide the application of the rules.
Department of Information Technology AI by Mukesh Barapatre 85
 Markov Algorithm

• An ordered group of productions applied in order or

priority to an input string.
• If the highest priority rule is not applicable, we apply
the next, and so on.
• An efficient algorithm for systems with many rules.
• Termination on (1) last production not applicable to a
string, or (2) production ending with period applied
• Can be applied to substrings, beginning at left
Department of Information Technology AI by Mukesh Barapatre 86
 Markov Algorithm

Department of Information Technology AI by Mukesh Barapatre 87

 Procedural Paradigms

• Algorithm – method of solving a problem in a

finite number of steps.

• Procedural programs are also called sequential

programs.

• The programmer specifies exactly how a problem

solution must be coded.
Department of Information Technology AI by Mukesh Barapatre 88
 Procedural Languages

Department of Information Technology AI by Mukesh Barapatre 89

 Imperative Programming

• Also known as statement-oriented

• During execution, program makes transition from the initial state to
the final state by passing through series of intermediate states.
• Provide rigid control and top-down-design.
• Not efficient for directly implementing expert systems.

Department of Information Technology AI by Mukesh Barapatre 90

 Functional Programming

• Function-based (association, domain, co-domain); f:

S→ T
• Not much control
• Bottom-up→ combine simple functions to yield more
powerful functions.
• Mathematically a function is an association or rule
that maps members of one set, the domain, into
another set, the codomain.
• e.g. LISP and Prolog

Department of Information Technology AI by Mukesh Barapatre 91

 Nonprocedural Paradigms

• Do not depend on the programmer giving exact

details how the program is to be solved.
• Declarative programming – goal is separated from
the method to achieve it.
• Object-oriented programming – partly imperative
and partly declarative – uses objects and methods
that act on those objects.
• Inheritance – (OOP) subclasses derived from parent
classes.
Department of Information Technology AI by Mukesh Barapatre 92
 Figure 1.9 Nonprocedural Languages

Department of Information Technology AI by Mukesh Barapatre 93

 What are Expert Systems?

Can be considered declarative languages:

• Programmer does not specify how to achieve a

goal at the algorithm level.

• Induction-based programming – the program

learns by generalizing from a sample.
Department of Information Technology AI by Mukesh Barapatre 94
 Uncertain KR&R
1

BY
PROF. MUKESH BARAPATRE

S.B. JAIN INSTITUTE OF TECHNOLOGY MANAGMENT & RESEARCH ,

Department of Information Technology

By Mukesh Barapatre
 Outline
2

 Probability
 Bayesian networks
 Fuzzy logic
 Crisp set

By Mukesh Barapatre
 Probability
3

FOL fails for a domain due to:

Laziness: too much to list the complete set of rules, too

hard to use the enormous rules that result

Theoretical ignorance: there is no complete theory for the

domain

Practical ignorance: have not or cannot run all necessary

tests

By Mukesh Barapatre
 Probability
4

 Probability = a degree of belief

 Probability comes from:

Frequentist: experiments and statistical assessment
Objectivist: real aspects of the universe
Subjectivist: a way of characterizing an agent’s beliefs

 Decision theory = probability theory + utility

theory

By Mukesh Barapatre
 Probability
5

Prior probability: probability in the absence of any

other information
P(Dice = 2) = 1/6

random variable: Dice

domain = <1, 2, 3, 4, 5, 6>
probability distribution: P(Dice) = <1/6, 1/6, 1/6, 1/6, 1/6,
1/6>

By Mukesh Barapatre
 Probability
6

Conditional probability: probability in the presence

of some evidence
P(Dice = 2 | Dice is even) = 1/3

P(Dice = 2 | Dice is odd) = 0

P(A | B) = P(A  B)/P(B)

P(A  B) = P(A | B).P(B)

By Mukesh Barapatre
 Probability
7

Example:
S = stiff neck
M = meningitis
P(S | M) = 0.5
P(M) = 1/50000
P(S) = 1/20

P(M | S) = P(S | M).P(M)/P(S) = 1/5000

By Mukesh Barapatre
 Probability
8

Joint probability distributions:

X: <x1, …, xm> Y: <y1, …, yn>

P(X = xi, Y = yj)

By Mukesh Barapatre
 Probability
9

Axioms:
• 0  P(A)  1
• P(true) = 1 and P(false) = 0
• P(A  B) = P(A) + P(B) - P(A  B)

By Mukesh Barapatre
 Probability
10

Derived properties:
• P(A) = 1 - P(A)
• P(U) = P(A1) + P(A2) + ... + P(An)
U = A1  A2  ...  An collectively exhaustive
Ai  Aj = false mutually exclusive

By Mukesh Barapatre
 Probability
11

Bayes’ theorem:

P(Hi | E) = P(E | Hi).P(Hi)/iP(E | Hi).P(Hi)

Hi’s are collectively exhaustive & mutually exclusive

By Mukesh Barapatre
 Probability
12

Problem: a full joint probability distribution P(X1,

X2, ..., Xn) is sufficient for computing any
(conditional) probability on Xi's, but the number of
joint probabilities is exponential.

By Mukesh Barapatre
 Probability
13

 Independence:
P(A  B) = P(A).P(B)
P(A) = P(A | B)
 Conditional independence:
P(A  B | E) = P(A | E).P(B | E)
P(A | E) = P(A | E  B)
Example:
P(Toothache | Cavity  Catch) = P(Toothache | Cavity)
P(Catch | Cavity  Toothache) = P(Catch | Cavity)

By Mukesh Barapatre
 Probability
14

"In John's and Mary's house, an alarm is installed

to sound in case of burglary or earthquake. When
the alarm sounds, John and Mary may make a call
for help or rescue."

By Mukesh Barapatre
 Probability
15

"In John's and Mary's house, an alarm is installed

to sound in case of burglary or earthquake. When
the alarm sounds, John and Mary may make a call
for help or rescue."
Q1: If earthquake happens, how likely will John
make a call?

By Mukesh Barapatre
 Probability
16

"In John's and Mary's house, an alarm is installed

to sound in case of burglary or earthquake. When
the alarm sounds, John and Mary may make a call
for help or rescue."
Q1: If earthquake happens, how likely will John
make a call?

Q2: If the alarm sounds, how likely is the house

burglarized?

By Mukesh Barapatre
 Bayesian Networks
17

• Pearl, J. (1982). Reverend Bayes on Inference Engines: A Distributed

Hierarchical Approach,
presented at the Second National Conference on Artificial
Intelligence (AAAI-82), Pittsburgh, Pennsylvania

• 2000 AAAI Classic Paper Award

By Mukesh Barapatre
 Bayesian Networks
18

P(B) P(E)
Burglary .001 Earthquake .002

B E P(A)

Alarm T T .95
T F .94
F T .29
F F .001

A P(J) A P(M)
JohnCalls T .90 MaryCalls T .70
F .05 F .01

By Mukesh Barapatre
 Bayesian Networks
19

Syntax:
 A set of random variables makes up the nodes

 A set of directed links connects pairs of nodes

 Each node has a conditional probability table that quantifies the

effects of its parent nodes
 The graph has no directed cycles

A A

yes B B no
C C

D D
By Mukesh Barapatre
 Bayesian Networks
20

Semantics:
 An ordering on the nodes: Xi is a predecessor of Xj  i < j

 P(X1, X2, …, Xn)

= P(Xn | Xn-1, …, X1).P(Xn-1 | Xn-2, …, X1). … .P(X2 | X1).P(X1)

= iP(Xi | Xi-1, …, X1) = iP(Xi | Parents(Xi))

P (Xi | Xi-1, …, X1) = P(Xi | Parents(Xi)) Parents(Xi)  {Xi-1, …,

X1}

Each node is conditionally independent of its predecessors given its

parents
By Mukesh Barapatre
 Bayesian Networks
21

Example:

P(J  M  A  B  E)

= P(J | A).P(M | A).P(A | B  E).P(B).P(E)

= 0.00062
B E

J M

By Mukesh Barapatre
 Bayesian Networks
22

 Why Bayesian Networks?

By Mukesh Barapatre
 Uncertain Question Answering
23

P(Query | Evidence) = ?

Diagnostic (from effects to causes): P(B | J)

Causal (from causes to effects): P(J | B)
Intercausal (between causes of a common effect): P(B | A, E)
Mixed: P(A | J, E), P(B | J, E)
B E

J M

By Mukesh Barapatre
 Uncertain Question Answering
24

 The independence assumptions in a Bayesian

Network simplify computation of conditional
probabilities on its variables

By Mukesh Barapatre
 Uncertain Question Answering
25

Q1: If earthquake happens, how likely will John

make a call?

Q2: If the alarm sounds, how likely is the house

burglarized?

Q3: If the alarm sounds, how likely both John and

Mary make calls?

By Mukesh Barapatre
 Uncertain Question Answering
26

P(B | A)

= P(B  A)/P(A)

= aP(B  A)

P(B | A)

= aP(B  A)

 a = 1/(P(B  A) + P(B  A))

By Mukesh Barapatre
 General Conditional Independence
27

 A Bayesian Network implies all conditional

independence among its variables

By Mukesh Barapatre
 General Conditional Independence
28

U1 Um

Z1j Znj

Y1 Yn

A node (X) is conditionally independent of its non-descendents

(Zij's), given its parents (Ui's)
By Mukesh Barapatre
 General Conditional Independence
29

U1 Um

Z1j Znj

Y1 Yn

A node (X) is conditionally independent of all other nodes, given

its parents (Ui's), children (Yi's), and children's parents (Zij's)
By Mukesh Barapatre
 General Conditional Independence
30

X and Y are conditionally independent given E

X E Y
Z

By Mukesh Barapatre
 General Conditional Independence
31

Example:
Battery

Radio Ignition Gas

Starts

Moves

By Mukesh Barapatre
 General Conditional Independence
32

Example:
Battery

Radio Ignition Gas

Gas - (Ignition) - Radio Starts

Gas - (Battery) - Radio
Gas - (no evidence at all) - Radio
Moves
Gas  Radio | Starts
Gas  Radio | Moves

By Mukesh Barapatre
 Vagueness
33

• The Oxford Companion to Philosophy (1995):

“Words like smart, tall, and fat are vague since in most contexts of
use there is no bright line separating them from not smart, not tall,
and not fat respectively …”

By Mukesh Barapatre
 Vagueness
34

• Imprecision vs. Uncertainty:

The bottle is about half-full.

vs.

It is likely to a degree of 0.5 that the bottle is full.

By Mukesh Barapatre
 Fuzzy Sets
35

• Zadeh, L.A. (1965). Fuzzy Sets

Journal of Information and Control

By Mukesh Barapatre
 Fuzzy Sets
36

By Mukesh Barapatre
 Fuzzy Sets
37

By Mukesh Barapatre
 Fuzzy Set Definition
38

A fuzzy set is defined by a membership function

that maps elements of a given domain (a crisp set)
into values in [0, 1].

mA: U → [0, 1] 1
young
mA  A 0.5

0
25 30 40 Age

By Mukesh Barapatre
 Fuzzy Set Representation
39

 Discrete domain:
high-dice score: {1:0, 2:0, 3:0.2, 4:0.5, 5:0.9, 6:1}

 Continuous domain:

A(u) = 1 for u[0, 25]

1
A(u) = (40 - u)/15 for u[25, 40]
A(u) = 0 for u[40, 150]
0.5

0
25 30 40 Age

By Mukesh Barapatre
 Fuzzy Set Representation
40

 a-cuts:
Aa = {u | A(u)  a}
Aa+ = {u | A(u) > a} strong a-cut

A0.5 = [0, 30]

0.5

0
25 30 40 Age
By Mukesh Barapatre
 Fuzzy Set Representation
41

 a-cuts:
Aa = {u | A(u)  a}
Aa+ = {u | A(u) > a} strong a-cut
A(u) = sup {a | u  Aa}

A0.5 = [0, 30]

0.5

0
25 30 40 Age
By Mukesh Barapatre
 Fuzzy Set Representation
42

 Support:
supp(A) = {u | A(u) > 0} = A0+

 Core:
core(A) = {u | A(u) = 1} = A1

 Height:
h(A) = supUA(u)

By Mukesh Barapatre
 Fuzzy Set Representation
43

 Normal fuzzy set: h(A) = 1

 Sub-normal fuzzy set: h(A)  1

By Mukesh Barapatre
 Membership Degrees
44

 Subjective definition

By Mukesh Barapatre
 Membership Degrees
45

 Subjective definition

 Voting model:
Each voter has a subset of U as his/her own crisp
definition of the concept that A represents.

A(u) is the proportion of voters whose crisp definitions

include u.

By Mukesh Barapatre
 Membership Degrees
46

 Voting model:

P1 P2 P3 P4 P5 P6 P7 P8 P9 P10
1

3 x x
4 x x x x x
5 x x x x x x x x x
6 x x x x x x x x x x

By Mukesh Barapatre
 Fuzzy Subset Relations
47

A  B iff A(u)  B(u) for every uU

A is more “specific” than B

“X is A” entails “X is B”

By Mukesh Barapatre
 Fuzzy Set Operations
48

 Standard definitions:

Complement: A(u) = 1 - A(u)

Intersection: (AB)(u) = min[A(u), B(u)]

Union: (AB)(u) = max[A(u), B(u)]

By Mukesh Barapatre
 Fuzzy Set Operations
49

 Example:

not young = young

not old = old

middle-age = not youngnot old

old = young

By Mukesh Barapatre
 Fuzzy Relations
50

 Crisp relation:
R(U1, ..., Un)  U1 ... Un

R(u1, ..., un) = 1 iff (u1, ..., un)  R or = 0 otherwise

By Mukesh Barapatre
 Fuzzy Relations
51

 Crisp relation:
R(U1, ..., Un)  U1 ... Un

R(u1, ..., un) = 1 iff (u1, ..., un)  R or = 0 otherwise

 Fuzzy relation: a fuzzy set on U1 ... Un

By Mukesh Barapatre
 Fuzzy Relations
52

 Fuzzy relation:
U1 = {New York, Paris}, U2 = {Beijing, New York, London}
R = “very far”

NY Paris
Beijing 1 .9
NY 0 .7
London .6 .3

R = {(NY, Beijing): 1, ...}

By Mukesh Barapatre
 Fuzzy Numbers
53

 A fuzzy number A is a fuzzy set on R:

A must be a normal fuzzy set
Aa must be a closed interval for every a(0, 1]
supp(A) = A0+ must be bounded

By Mukesh Barapatre
 Basic Types of Fuzzy Numbers
54

1 1

0 0

1 1

0 0

By Mukesh Barapatre
 Basic Types of Fuzzy Numbers
55

1 1

0 0

By Mukesh Barapatre
 Operations of Fuzzy Numbers
56

 Interval-based operations:

(A  B)a = Aa  Ba

By Mukesh Barapatre
 Operations of Fuzzy Numbers
57

 Arithmetic operations on intervals:

[a, b][d, e] = {fg | a  f  b, d  g  e}

By Mukesh Barapatre
 Operations of Fuzzy Numbers
58

 Arithmetic operations on intervals:

[a, b][d, e] = {fg | a  f  b, d  g  e}

[a, b] + [d, e] = [a + d, b + e]

[a, b] - [d, e] = [a - e, b - d]

[a, b]*[d, e] = [min(ad, ae, bd, be), max(ad, ae, bd, be)]

[a, b]/[d, e] = [a, b]*[1/e, 1/d] 0[d, e]

By Mukesh Barapatre
 Operations of Fuzzy Numbers
59

about 2 about 3
1
about 2 + about 3 = ?

about 2  about 3 = ?

0
2 3 +

By Mukesh Barapatre
 Operations of Fuzzy Numbers
60

 Discrete domains:

A = {xi: A(xi)} B = {yi: B(yi)}

AB=?

By Mukesh Barapatre
 Operations of Fuzzy Numbers
61

 Extension principle:
f: U1 U2 → V

induces
~ ~ ~
g: U1 U2 → V

[g(A, B)](v) = sup{(u1, u2) | v = f(u1, u2)}min{A(u1),

B(u2)}

By Mukesh Barapatre
 Operations of Fuzzy Numbers
62

 Discrete domains:

A = {xi: A(xi)} B = {yi: B(yi)}

(A  B)(v) = sup{(xi, yj) | v = xi°yj)}min{A(xi),

B(yj)}

By Mukesh Barapatre
 Fuzzy Logic
63

if x is A then y is B
x is A*
------------------------

y is B*

By Mukesh Barapatre
 Fuzzy Logic
64

 View a fuzzy rule as a fuzzy relation

 Measure similarity of A and A*

By Mukesh Barapatre
 Fuzzy Controller
65

 As special expert systems

 When difficult to construct mathematical models

 When acquired models are expensive to use

By Mukesh Barapatre
 Fuzzy Controller
66

IF the temperature is very high

AND the pressure is slightly low
THEN the heat change should be slightly
negative

By Mukesh Barapatre
 Fuzzy Controller
67

FUZZY
actions Defuzzification CONTROLLER
model

Controlled Fuzzy inference Fuzzy rule

process engine base

Fuzzification
conditions model

By Mukesh Barapatre
 Fuzzification
68

0
x0

By Mukesh Barapatre
 Defuzzification
69

 Center of Area:

x = (A(z).z)/A(z)

By Mukesh Barapatre
 Defuzzification
70

 Center of Maxima:

M = {z | A(z) = h(A)}

x = (min M + max M)/2

By Mukesh Barapatre
 Defuzzification
71

 Mean of Maxima:

M = {z | A(z) = h(A)}

x = z/|M|

By Mukesh Barapatre
 SUBJECT:- Artificial Intelligence
Semester:-VII

UNIT - 1 Problems, Problem Spaces and Search [ 14M ]

Syllabus: History and Application of AI, The Turing Test approach, AI Problems and AI
Techinques, Defining problem as state space representation. Production system, Problem
characteristics, monotonic and non-monotonic production systems. Solving problems by
searching-Toyproblems,Real-Worldproblems.

Content : 1. History and Application of AI.

2. Turing Test approach.
3. AI Problems and AI Techniques.
4. Defining problem as state space representation.
5. Production system, Problem characteristics, monotonic and
non- monotonic production systems.
6. Problems bysearching-Toy problems, Real-World problems.

Que . Define AI ? Write the applications of AI. { S-18, W-16 } [ 5m ]

Ans :-
Intelligence:
“Ability to learn, Understand and think”(oxford Dictionary) ie also known as Intelligencee.
A.I. is the study of how to make computers do things at which, at the moment, people are
better.
Example: Speech Recognization, smell ,face ,object, learning New skills.

Thinking Humanly Thinking Rationally

Acting Humanly Acting Rationally

Artificial Intelligence :

1
 Artificial Intelligence (AI) is the intelligence of machines and the
branch of computer science that aims to create it A.I. is the study of how
to make computers do things at which, at the moment, people are better.

Introduction - AI
 Artificial Intelligence (AI) is the intelligence of machines and the branch
of computer science that aims to create it.
 AI is the study of : How to make computers do things which, at the
moment, people do better.
 AI is the study and design of intelligent agents, where an intelligent
agent is a system that perceives its environment and takes actions
that maximize its chances of success. Relate to tasks involving higher
mental processes. Examples: creativity, solving problems, pattern
recognition, learning.

Block Architecture of AI System

• Collection of facts and rules.

• Place where the domain specific knowledge is stored
• Consist of body of facts axioms, rules and concepts about different entities.
• Means of utilizing facts
• To prove the hypothesis & to arrive at a conclusion
• Provides the needed facilities for the user to communicate with the system.

General application of AI

2
 1. Finance
2. Medical
3. Industries
4. Telephone maintains
5. Telecom
6. Transport
7. Entertainment
8. Pattern reorganization
9. Robotics
10. Neural Networks
11. Genetic Algorithm
12. Data mining

Specific Application of AI

1) Game Playing

Much of the early research in state space search was done using common board games such
as checkers, chess, and the 15-puzzle. Games can generate extremely large search spaces.
Theses are large and complex enough to require powerful techniques for determining what
alternative to explore

2) Automated reasoning and Theorem Proving

Theorem-proving is one of the most fruitful branches of the field. Theorem-proving research
was responsible in formalizing search algorithms and developing formal representation
languages such as predicate calculus and the logic programming language.

3) Expert System

One major insight gained from early work in problem solving was the importance of
domain-specific knowledge. Expert knowledge is a combination of a theoretical
understanding of the problem and a collection of heuristic problem-solving rules

4) Natural Language Understanding and Semantics

One of the long-standing goals of AI is the creation of programming that are capable of
understanding and generating human language

5) Modeling Human Performance

Capture the human mind (knowledge representation)

6) Robotics

3
A robot that blindly performs a sequence of actions without responding to changes or being
able to detect and correct errors could hardly considered intelligent. It should have some
degree of sensors and algorithms to guild it

7) Machine Learning

Learning has remained a challenging area in AI. An expert system may perform extensive
and costly computation to solve a problem; unlike human, it usually don’t remember the
solution.

Intelligent Behavior -

 Perceiving one’s environment,

 Acting in complex environments,

 Learning and understanding from experience,

 Reasoning to solve problems and discover hidden knowledge,

 Knowledge applying successfully in new situations,

 Thinking abstractly, using analogies,

 Communicating with others, and more like

Understanding AI / Hard or Strong AI

 How knowledge is acquired, represented, and stored;

 How intelligent behavior is generated and learned;
 How motives, emotions, and priorities are developed and used;
 How sensory signals are transformed into symbols;
 How symbols are manipulated to perform logic, to reason about past,
and plan for future.
 Strong AI aims to build machines whose overall intellectual ability is
indistinguishable from that of a human being.
 Generally, artificial intelligence research aims to create AI that can
replicate human intelligence completely.

 Strong AI refers to a machine that approaches or supersedes human

intelligence,
◊ If it can do typically human tasks,
◊ If it can apply a wide range of background knowledge and
◊ If it has some degree of self-consciousness.

GOALS OF AI

4
Q.2) Write a short note on AI techniques.

5
Ans: -
1) It is voluminous.
2) 2. It is hard to characterize and identify correctly
3) It is endless changing
4) Knowledge is different from data, because it is organized in a way that agrees to
the ways it will be used.

In what ways should knowledge be represented in order to be used by AI techniques?AI

technique is a method that achieve knowledge that should be represented in such way that:

1) The knowledge captures generalization. What this mean is grouping situation that
share important properties rather than representing each situation separately.
2) The advantage that knowledge has would be that unreasonable amounts of memory
and updating will no longer be required. Anything without this property is called
‘data’ rather than knowledge.
3) It should be represented in such a way that it can be understood by people who must
prepare it. For many program the size of the data be achieved automatically by taking
reading from a number of instruments, but in many AI areas, most of the knowledge a
program has must basically be provides by the people in terms that they understand
it.
4) it could easily be adjusted to correct errors and to demonstrate changes in the world.
5) It could be used to help overcome it own through volume by helping to restrict the
range of responsibilities that must usually be considered or discussed.

Define AI technique and give few of them.

(i) Artificial Intelligence is“ability to learn, understand and think ”.

AI is the study of how to make computers make things which at the moment people
do better.
Examples: Speech recognition, Smell, Face, Object, Intuition, Inferencing, Learning
new skills, Decision making, Abstract thinking
(ii)Acting Humanly: The Turing Test :Predicted that by 2000, a machine might have a
30% chance of fooling a lay person for 5 minutes
Anticipated all major arguments against AI in following 50 years. Suggested major
components of AI: knowledge,
reasoning, language, understanding, learning.
(iii)Thinking Humanly: Cognitive Modelling:- Not content to have a program correctly
solving a problem. More concerned with comparing its reasoning steps to traces of
human solving the same problem.

(iv)Requires testable theories of the workings of the human mind: cognitive science.
Thinking Rationally: Laws of Thought:- Aristotle was one of the first to attempt to
codify “right thinking, i.e., irrefutable reasoning processes. Formal logic provides a

6
 precise notation and rules for representing and reasoning with all kinds of things in
the world.

(v)There are few Obstacles:- (a)Informal knowledge representation.(b)Computational

complexity and resources.

*.Turing Test Approach.* [5/6 m ]

What is Turing Test? Explain the purpose of turing test? [w-17, s-16]

Or

Explain the Turing test approach carried out by Alan Turing. [w-16]

Or

Explain the following : i) Turing test

ii) AI Techniques [s-18, s-17]

{s-16, w-16, s-17, w-17, s-18}

Turing Test Approach

Statement :
The art of creating machines that perform functions requiring
intelligence when performed by people; that it is the study of, how to make
computers do things which at the moment people do better.

To detect whether the m/c has become intelligent as human or not.

If Interrogator not able to distinguish between human & Comp then Comp posses
intelligence.
Alan Turing (1912-1954)
“Computing Machinery and Intelligence” (1950)

7
 .

Turing Test :
If Interrogator not able to distinguish between human & Comp then Comp posses
intelligence.The machine tries to fool the interrogator to believe that it is the
human, and the person also tries to convince the interrogator that it is the human.

A . HUMAN INTERROGATOR

B. HUMAN

C. AI SYSTEM.

The Turing Test, proposed by Alan Turing (Turing, 1950), was designed to provide a
satisfactory operational definition of intelligence. Turing defined intelligent behavior as the
ability to achieve human-level performance in all cognitive tasks, sufficient to fool an
interrogator. Roughly speaking, the test he proposed is that the computer should be
interrogated by a human via a teletype, and passes the test if the interrogator cannot tell if
there is a computer or a human at the other end.
Programming a computer to pass the test provides plenty to work on. The computer would
need to possess the following capabilities:

 natural language processing: to enable it to communicate successfully in English

(or some other human language);
 knowledge representation: to store information provided before or during the
interrogation;
 automated reasoning: to use the stored information to answer questions and to
draw new conclusions;
 Machine learning: to adapt to new circumstances and to detect and extrapolate
patterns.
Turing's test deliberately avoided direct physical interaction between the
interrogator and the computer, because physical simulation of a person is unnecessary for
intelligence. However, the so called total Turing Test includes a video signal so that the
interrogator can test the subject's perceptual abilities, as well as the opportunity for the

8
interrogator to pass physical objects ``through the hatch.'' To pass the total Turing Test, the
computer will need computer vision to perceive objects, and robotics to move them about.

Within AI, there has not been a big effort to try to pass the Turing test. The issue of
acting like a human comes up primarily when AI programs have to interact with people, as
when an expert system explains how it came to its diagnosis, or a natural language
processing system has a dialogue with a user. These programs must behave according to
certain normal conventions of human interaction in order to make themselves understood.
The underlying representation and reasoning in such a system may or may not be based on
a human model.

9
 AI Techniques
Various techniques that have evolved, can be applied to a variety of AI tasks.
The techniques are concerned with how we represent, manipulate and reason
with knowledge in order to solve problems.

There are various AI techniques which make use of following.

1. Search
2. Use of knowledge
3. Abstraction
4. Transition Network
5. Machine learning
Explaination :
• Search: It provides a way of solving problems for which no more direct
approach is available.
• Use of Knowledge: It provides a way of solving complex problems.
• Abstraction: It provides a way of separating important features from
unimportant .

10
11
Task Domains of AI / Typical AI problems

Que . Give Different Task Domain of AI { S-15 }

Q. 1) Define AI? Outline nature of problem that led to the development of AI? Explain
typical task domains of AI?

Ans :
• Define the problem precisely
• Analyze the problem.
• Isolate and represent the task knowledge that is
necessary to solve the problem.
• Choose the best problem solving technique and apply
them to the particular problem.

12
The Foundations of AI:- Philosophy (423 BC - present):
(a) Logic, methods of reasoning.
(b)Mind as a physical system.
(c) Foundations of learning, language, and rationality.
1. Mathematics (c.800 - present):
- Formal representation and proof.
- Algorithms, computation, decidability, tractability.
- Probability.
2. Psychology (1879 - present):
- Adaptation.
- Phenomena of perception and motor control.
- Experimental techniques.
3. Linguistics (1957 - present):
- Knowledge representation.
- Grammar.

Task Domains of AI:-

1. Mundane Tasks:
– Perception
• Vision
• Speech
– Natural Languages
• Understanding
• Generation
• Translation
– Common sense reasoning
– Robot Control

2. Formal Tasks
– Games : chess, checkers etc
– Mathematics: Geometry, logic,Proving properties of programs
–
3. Expert Tasks:
– Engineering ( Design, Fault finding, Manufacturing planning)
– Scientific Analysis
– Medical Diagnosis
– Financial Analysis

13
AI Technique:- Intelligence requires Knowledge, knowledge posesses less desirable
properties such as:
– Voluminous
– Hard to characterize accurately
– Constantly changing
– Differs from data that can be used

AI technique is a method that exploits knowledge that should be represented in such a

way that:
– Knowledge captures generalization
– It can be understood by people who must provide it
– It can be easily modified to correct errors.
– It can be used in variety of situations.

14
 Problem characteristics (W-15)
(S-16)
Explain the problem characteristics for solving following AI-problems :- (W-16)
i) Water – jug problem. (S-17)
ii) Chess Playing. (W-17)
iii) 8-puzzle problem.
iv) Travelling sales man problem
OR
Define AI. Explain the 7 problem characteristics with example. .
OR
Analyze the different problem characteristics for the following problem :
(i) Travelling Salesman problem
(ii) 8-puzzle problem.
OR
what are characteristics of Artificial Intelligence problems with suitable example

Ans : AI Problems

Statement : A important idea is to have description of the kinds of problem is a situation

which does not have its answer readily available. The answer or situation needs to be found
out with the help of some mechanism or methodology using entities available for an efficient
problems solving process. To build a system to solve the particular problem we need to do
Define the problems precisely.

1) Analysis the problems.

2) Isolate & represent the task knowledge i.e. what kind of knowledge is necessary to
solve the problems.
3) Choose the best possible problem solving technique & apply it to the particular
problems.

Steps involved in designing a program to solve an AI problem?

To build a system to solve particular problem we need four steps:-

(i) Define the problem precisely:- This definition must include precise
specification of what the initial situations will be as well as what final situations
constitute acceptable to solution of problem.
(ii) Analyze the problem:- A few important features can have a immense impact on
the appropriateness of various possible technique for solving the problem.
(iii) Isolate and represent the task knowledge that is necessary to solve the
problem.
Choose the best problem solving technique and apply it to the particular problem

15
 Problem characteristics

Problem Characteristics of Artificial Intelligence: Heuristic search is a very general method

applicable to a large class of problem . It includes a variety of techniques. In order to choose
an appropriate method, it is necessary to analyze the problem with respect to the following
considerations.

1) Is the problem decomposable ?

A very large and composite problem can be easily solved if it can be broken into smaller
problems and recursion could be used. Suppose we want to solve.
Ex:- ∫ x2 + 3x+sin2x cos 2x dx
This can be done by breaking it into three smaller problems and solving each by applying
specific rules. Adding the results the complete solution is obtained.

2) Can solution steps be ignored or undone?

Problem fall under three classes ignorable , recoverable and irrecoverable. This
classification is with reference to the steps of the solution to a problem. Consider thermo
proving. We may later find that it is of no help. We can still proceed further, since nothing is
lost by this redundant step. This is an example of ignorable solutions steps.
Now consider the 8 puzzle problem tray and arranged in specified order. While
moving from the start state towards goal state, we may make some stupid move and
consider theorem proving. We may proceed by first proving lemma. But we may backtrack
and undo the unwanted move. This only involves additional steps and the solution steps are
recoverable.
Lastly consider the game of chess. If a wrong move is made, it can neither be ignored nor be
recovered. The thing to do is to make the best use of current situation and proceed. This is
an example of an irrecoverable solution steps.
1. Ignorable problems Ex:- theorem proving In which solution steps can be ignored.
2. Recoverable problems Ex:- 8 puzzle In which solution steps can be undone
3. Irrecoverable problems Ex:- Chess In which solution steps can’t be undone
A knowledge of these will help in determining the control structure.

3) Is the Universal Predictable?

Problems can be classified into those with certain outcome (eight puzzle and water jug
problems) and those with uncertain outcome ( playing cards) . in certain – outcome
problems, planning could be done to generate a sequence of operators that guarantees to a
lead to a solution. Planning helps to avoid unwanted solution steps. For uncertain out come
problems, planning can at best generate a sequence of operators that has a good probability
of leading to a solution. The uncertain outcome problems do not guarantee a solution and it
is often very expensive since the number of solution and it is often very expensive since the
number of solution paths to be explored increases exponentially with the number of points
at which the outcome can not be predicted. Thus one of the hardest types of problems to
solve is the irrecoverable, uncertain – outcome problems ( Ex:- Playing cards).

16
4) Is good solution absolute or relative ? (Is the solution a state or a path ?)

There are two categories of problems. In one, like the water jug and 8 puzzle problems, we
are satisfied with the solution, unmindful of the solution path taken, whereas in the other
category not just any solution is acceptable. We want the best, like that of traveling sales
man problem, where it is the shortest path. In any – path problems, by heuristic methods we
obtain a solution and we do not explore alternatives. For the best-path problems all possible
paths are explored using an exhaustive search until the best path is obtained.

5) The knowledge base consistent ?

In some problems the knowledge base is consistent and in some it is not. For example
consider the case when a Boolean expression is evaluated. The knowledge base now
contains theorems and laws of Boolean Algebra which are always true. On the contrary
consider a knowledge base that contains facts about production and cost. These keep
varying with time. Hence many reasoning schemes that work well in consistent domains are
not appropriate in inconsistent domains.
Ex.Boolean expression evaluation.

6) What is the role of Knowledge ?

Though one could have unlimited computing power, the size of the knowledge base
available for solving the problem does matter in arriving at a good solution. Take for
example the game of playing chess, just the rues for determining legal moves and some
simple control mechanism is sufficient to arrive at a solution. But additional knowledge
about good strategy and tactics could help to constrain the search and speed up the
execution of the program. The solution would then be realistic.
Consider the case of predicting the political trend. This would require an enormous
amount of knowledge even to be able to recognize a solution , leave alone the best.
Ex:- 1. Playing chess 2. News paper understanding

7) Does the task requires interaction with the person.

The problems can again be categorized under two heads.
1. Solitary in which the computer will be given a problem description and will produce an
answer, with no intermediate communication and with he demand for an explanation of the
reasoning process. Simple theorem proving falls under this category . given the basic rules
and laws, the theorem could be proved, if one exists.
Ex:- theorem proving (give basic rules & laws to computer)
8) Problem Classification
Actual problems are examined from the point of view , the task here is examine an input and
decide which of a set of known classes.
Ex:- Problems such as medical diagnosis , engineering design.

17
Problem Solution :
i) Water – jug problem.
ii) Chess Playing.
iii) 8-puzzle problem.
iv) Travelling sales man problem

18
 Production System
Explain different categories of Production system.
OR
Give the difference between monotonic and non-monotonic Reasoning System.
OR
Explain the working of production system. Differentiate between monotonic and
nonmonotonic production system. Give one example of monotonic and non-
monotonic production system.
OR
What are the characteristics of a production system? Write one example of
monotonic and nonmonotonic production system.
OR
What do you mean by production systems? State its different types and explain each
of the types in brief.

Ans:-

Statement : Production System

A Knowledge representation formalism consists of collections of condition-action

rules(Production Rules or Operators), a database which is modified in accordance with the
rules, and a Production System Interpreter which controls the operation of the rules i.e The
'control mechanism' of a Production System, determining the order in which Production
Rules are fired.
1. A system that uses this form of knowledge representation is called a production system.
2. The goal database is the central data structure used by an AI production system.
3. The production system. The production rules operate on the global database.
4. Each rule has a precondition that is either satisfied or not by the database. If the
precondition is satisfied, the rule can be applied.
5. Application of the rule changes the database. The control system chooses which
applicable rule should be applied and ceases computation when a termination
condition on the database is satisfied.
6. If several rules are to fire at the same time, the control system resolves the conflicts.

Four classes of production systems:-

1. A monotonic production system

2. A non monotonic production system
3. A partially commutative production system
4. A commutative production system.

19
Production System Characteristics

1. Can production systems, like problems, be described by a set of

characteristics that shed some light on how they can easily be
implemented?

2. If so, what relationships are there between problem types and the types
of production systems best suited to solving the problems?
3. Classes of Production systems:
1. Monotonic Production System: the application of a rule never
prevents the later application of another rule that could also have
been applied at the time the first rule was selected.
2. Non-Monotonic Production system
3. Partially commutative Production system: property that if
application of a particular sequence of rules transforms state x to
state y, then permutation of those rules allowable, also transforms
state x into state y.
4. Commutative Production system

Monotonic Production Systems

• Production system in which the application of a rule never prevents the

later application of another rule that could also have been applied at the
time the first rule was applied. i.e., rules are independent.

• These production systems are useful for solving ignorable problems.

• Example: Theorem Proving

20
 • They can be implemented without the ability to backtrack to previous
states when it is discovered that an incorrect path has been followed.
• This often results in a considerable increase in efficiency, particularly
because since the database will never have to be restored, It is not
necessary to keep track of where in the search process every change was
made.
• They are good for problems where things do not change; new things get
created.

Non Monotonic

• Useful for problems in which changes occur but can be reversed and in
which order of operations is not critical.
• Example: Robot Navigation, 8-puzzle, blocks world
• Suppose the robot has the following ops: go North (N), go East (E), go
South (S), go West (W). To reach its goal, it does not matter whether the
robot executes the N-N-E or N-E-N.

Partially commutative production system:-

1. A production system in which the application of a particular sequence of rules

transforms state X into state Y, then any permutation of those rules that is allowable
also transforms state x into state Y.
2. Theorem proving falls under monotonic partially communicative system. Blocks world
and 8 puzzle problems like chemical analysis and synthesis come under monotonic, not
partially commutative systems. Playing the game of bridge comes under non monotonic
, not partially commutative system.
3. For any problem, several production systems exist. Some will be efficient than others.
Though it may seem that there is no relationship between kinds of problems and kinds
of production systems, in practice there is a definite relationship.
4. Partially commutative , monotonic production systems are useful for solving ignorable
problems. These systems are important for man implementation standpoint because
they can be implemented without the ability to backtrack to previous states, when it is
discovered that an incorrect path was followed. Such systems increase the efficiency
since it is not necessary to keep track of the changes made in the search process.
5. Monotonic partially commutative systems are useful for problems in which changes
occur but can be reversed and in which the order of operation is not critical (ex: 8
puzzle problem).
6. Production systems that are not partially commutative are useful for many problems in
which irreversible changes occur, such as chemical analysis. When dealing with such

21
 systems, the order in which operations are performed is very important and hence
correct decisions have to be made at the first time itself..

Advantages of production systems:-

1. Production systems provide an excellent tool for structuring AI programs.

2. Production Systems are highly modular because the individual rules can be added,
removed or modified independently.
3. The production rules are expressed in a natural form, so the statements contained in the
knowledge base should the a recording of an expert thinking out loud.

Disadvantages of Production Systems:-

1. One important disadvantage is the fact that it may be very difficult analyse the flow of
control within a production system because the individual rules don’t call each other.
2. Production systems describe the operations that can be performed in a search for a
solution to the problem. They can be classified as follows.
3. Monotonic production system :- A system in which the application of a rule never
prevents the later application of another rule, that could have also been applied at the
time the first rule was selected.

State Space Representation / Problem Space

Explain the state-space representation used for water jug problem.

OR
Give the state space representation of the water-jug problem and give one solution to it

Ans :
Problem Space : The environment where the search is performed.

22
23
State Space - set of all states reachable from the initial state.
• The set of all possible configuration is the space of the problem states.

24
25
Topic 3: AI Example

Explain Water Jug Problem along with the problem solution.

OR

Suppose, you are given two jugs 4 gallon and 3 gallon capacity. Neither has any
measuring mark on it. There is a pump that can be used to fill jug with water. How can
you get exactly 2 gallon of water in 3 gallon jug & you cannot throw water on ground.
Determine final state and give the state space representation for this problem.
Assume, initial state of both the jugs are empty.

Ans:-
Water Jug Problem

The water jug problem: You are given two jugs, a 4-litre one and a 3-litre one. Neither have
any measuring markers on it. There is a pump that can be used to fill the jugs with water.
How can you get exactly 2 liters of water into 4-litre jug.
Let x and y be the amounts of water in 4-Lt and 3-Lt Jugs respectively Then (x,y)
refers to water available at any time in 4-Lt and 3-Lt jugs. Also (x, y) (x-d, y + dd) means
drop some unknown amount d of water from 4-Lt jug and add onto 3-Lt jug.
All possible production rules can be written as follows

• The state space for this problem can be described as the set of ordered pairs of
integers (x,y) such that

x = 0, 1,2, 3 or 4 and

y = 0,1,2 or 3;

x - represents the number of gallons of water in the 4-gallon jug &

y - represents the quantity of water in 3-gallon jug

• The start state is (0,0)

• The goal state is (2,n)

The operators to be used to solve the problem can be described as follows:

Remember: There is NO hard and fast rules to follow this sequence. In any state space
search problem, there can be numerous ways to solve, your approach can be different to
solve a problem and sequence of actions too.

26
Production rules for Water Jug Problem :

Solution of Water Jug Problem

Obviously to solve water jug problem, we can perform following sequence of actions,
(0,0) (0,3) (3,0) (3,3) (4,2) (0,2) (2,0)
By applying rules 2,9,2,7,5 and 9 with initial empty jugs

Gallon in 4-gallon jug Gallon in 3-gallon jug Rule applied

0 0 2
0 3 9
3 0 2
3 3 7
4 2 5 or 11
0 2 9 or 11
2 0

27
Q.6) b) Explain the term “combinatorial explosion” W-14,(3) Summer-13 (3)

Ans:-
The search tree for a travelling salesman problem becomes unmanageably large as the
number of cities increase, many problems have the property that as the number of
individual items being considered increase, the number of possible paths in the search tree
increases exponentially, meaning that as the problems gets larger, it becomes more and
more unreasonable to expect a computer program to be able to solve it. This problem is
known as combinatorial explosion because the amount of work that a program needs to do
to solve the problems seems to grow at an explosive rate, due to the possible combinations it
must consider.

Q.4) Explain Human Intelligence w.r.t. communicational skills, learning and

perceptions. What essential abilities of intelligence? Winter-14, 4M
Ans:-
Introductory Problem: Question Answering

“Mary went shopping for a new coat. She found a red one she really liked. When she got it
home, she discovered that it went perfectly with her favourite dress”.

Q1: What did Mary go shopping for?

Q2: What did Mary find that she liked?
Q3: Did Mary buy anything?

Program 1:
1. Match predefined templates to questions to generate text patterns.
2.Match text patterns to input texts to get answers.

“What did X Y”
“What did Mary go shopping for?”
“Mary go shopping for Z”
Z = a new coat

Program 2:
Structured representation of sentences:
Event2: Thing1:
instance: Finding instance: Coat
tense: Past colour: Red
agent: Mary
object: Thing 1

28
Program 3:
Background world knowledge:
C finds M

C leaves L C buys M

C leaves L

C takes M

29