CES 510 Intelligent System

Design

B. Ravikumar
Department of Engg Science
116 I Darwin Hall

664 3335
[email protected]
 Textbook
 Chris Manning and Hinrich Shutze, Foundations of
Statistical Natural Language Processing, MIT
Press, 1999.
 Various supplementary readings.

Other Useful Books:

Jurafsky & Martin, SPEECH and LANGUAGE

PROCESSING: An Introduction to Natural Language
Processing, Computational Linguistics, and Speech
Recognition.
 Overview of Artificial Intelligence

• major applications
• image processing and vision
• robotics
• game playing
• speech recognition
• natural language understanding
• etc.
 What is Artificial Intelligence
(John McCarthy , Basic Questions)

 What is artificial intelligence?

 It is the science and engineering of making intelligent machines,
especially intelligent computer programs. It is related to the similar
task of using computers to understand human intelligence, but AI
does not have to confine itself to methods that are biologically
observable.

 Yes, but what is intelligence?

 Intelligence is the computational part of the ability to achieve goals
in the world. Varying kinds and degrees of intelligence occur in
people, many animals and some machines.

 Isn't there a solid definition of intelligence that doesn't depend on relating it

to human intelligence?
 Not yet. The problem is that we cannot yet characterize in general
what kinds of computational procedures we want to call intelligent.
We understand some of the mechanisms of intelligence and not
others.

 More in: http://www-formal.stanford.edu/jmc/whatisai/node1.html

 What is Artificial Intelligence?

 Human-like (“How to simulate humans intellect and

behavior on by a machine.)
• Mathematical problems (puzzles, games, theorems)
• Common-sense reasoning (if there is parking-space, probably
illegal to park)
• Expert knowledge: lawyers, medicine, diagnosis
• Social behavior
 Rational-like:
• achieve goals, have performance measure
 What is Artificial Intelligence

 Thought processes
• “The exciting new effort to make computers think ..
Machines with minds, in the full and literal sense”
(Haugeland, 1985)
 Behavior
• “The study of how to make computers do things at which,
at the moment, people are better.” (Rich, and Knight,
1991)
 The Turing Test
(Can Machine think? A. M. Turing, 1950)

 Requires
• Natural language
• Knowledge representation
• Automated reasoning
• Machine learning
• (vision, robotics) for full test
 What is AI?
 Turing test (1950)
 Requires:
• Natural language
• Knowledge representation
• automated reasoning
• machine learning
• (vision, robotics.) for full test
 Thinking humanly:
• Introspection, the general problem solver (Newell and Simon 1961)
• Cognitive sciences
 Thinking rationally:
• Logic
• Problems: how to represent and reason in a domain
 Acting rationally:
• Agents: Perceive and act
 History of AI
 McCulloch and Pitts (1943)
• Neural networks that learn
 Minsky (1951)
• Built a neural net computer
 Darmouth conference (1956):
• McCarthy, Minsky, Newell, Simon met,
• Logic theorist (LT)- proves a theorem in Principia Mathematica-Russel.
• The name “Artficial Intelligence” was coined.
 1952-1969
• GPS- Newell and Simon
• Geometry theorem prover - Gelernter (1959)
• Samuel Checkers that learns (1952)
• McCarthy - Lisp (1958), Advice Taker, Robinson’s resolution
• Microworlds: Integration, block-worlds.
• 1962- the perceptron convergence (Rosenblatt)
 The Birthplace of
“Artificial Intelligence”, 1956
 Darmouth workshop, 1956: historical meeting of the
perceived founders of AI met: John McCarthy, Marvin Minsky,
Alan Newell, and Herbert Simon.

 A Proposal for the Dartmouth Summer Research Project on

Artificial Intelligence. J. McCarthy, M. L. Minsky, N. Rochester,
and C.E. Shannon. August 31, 1955. "We propose that a 2
month, 10 man study of artificial intelligence be carried out
during the summer of 1956 at Dartmouth College in Hanover,
New Hampshire. The study is to proceed on the basis of the
conjecture that every aspect of learning or any other feature
of intelligence can in principle be so precisely described that
a machine can be made to simulate it." And this marks the
debut of the term "artificial intelligence.“
 History, continued
 1966-1974 a dose of reality
• Problems with computation
 1969-1979 Knowledge-based system
• Expert systems:
— Dendral:Inferring molecular structures
— Mycin: diagnosing blood infections
— Prospector: recomending exploratory drilling (Duda).
 1986-present: return to neural networks
 Machine learning theory

 Genetic algorithms, genetic programming

 Statistical approaches and data mining
 State of the art

 Deep Blue defeated the reigning world chess champion

Garry Kasparov in 1997
 Proved a mathematical conjecture (Robbins conjecture)
unsolved for decades
 No hands across America (driving autonomously 98% of the
time from Pittsburgh to San Diego)
 During the 1991 Gulf War, US forces deployed an AI logistics
planning and scheduling program that involved up to 50,000
vehicles, cargo, and people
 NASA's on-board autonomous planning program controlled
the scheduling of operations for a spacecraft
 Proverb solves crossword puzzles better than most humans
 DARPA grand challenge 2003-2005, Robocup
 What’s involved in Intelligence?
Intelligent agents

 Ability to interact with the real world

• to perceive, understand, and act
• e.g., speech recognition and understanding and synthesis
• e.g., image understanding
• e.g., ability to take actions, have an effect

 Knowledge Representation, Reasoning and Planning

• modeling the external world, given input
• solving new problems, planning and making decisions
• ability to deal with unexpected problems, uncertainties

 Learning and Adaptation

• we are continuously learning and adapting
• our internal models are always being “updated”
— e.g. a baby learning to categorize and recognize animals
 Course overview

 Intelligent systems are autonomous systems (hardware /

software or a combination) that behaves as if it exhibits some
form of intelligence.

 Concept goes back to Alan Turing who thought about

machine intelligence and devised Turing test to distinguish a
machine from a human through interaction.

 Some major areas:


Symbolic information processing – deductive systems

Game playing – chess, backgammon etc.

natural language understanding – answering queries,
translation, text classification etc.

Machine learning - adaptive behavior through stimulus

Neural networks

Statistical modeling

Fuzzy logic, genetic programming etc.
Course overview

 In this course we will introduce statistical

techniques for inferring structure from text.
The aim of the course is to introduce
existing techniques in statistical NLP and to
stimulate thought into bettering these.

 Applications of NLP

Information Retrieval

Information Extraction

Natural language interface to database

Statistical Machine Translation
 Tools
 Probability Theory
 Information Theory
 Algorithms
 Data Structures
 Probabilistic AI
 Grammars and automata
 The Steps in NLP
Discourse

Pragmatics

Semantics

Syntax

Morphology
 The steps in NLP (Cont.)
 Morphology: Concerns the way words are built up from
smaller meaning bearing units. (come(s),co(mes))
 Syntax: concerns how words are put together to form
correct sentences and what structural role each word
has.
 Semantics: concerns what words mean and how these
meanings combine in sentences to form sentence
meanings.
 Pragmatics: concerns how sentences are used in
different situations and how use affects the
interpretation of the sentence.
 Discourse: concerns how the immediately preceding
sentences affect the interpretation of the next
sentence.
 Parsing (Syntactic Analysis)
 Assigning a syntactic and logical form to an input
sentence
 uses knowledge about word and word meanings (lexicon)
 uses a set of rules defining legal structures (grammar)

(S (NP (NAME Sam))

(VP (V ate)
(NP (ART the)
(N apple))))

I made her duck.

 Word Sense Resolution

 Many words have many meanings or senses.

 We need to resolve which of the senses of an
ambiguous word is invoked in a particular use of the
word.
 I made her duck. (made her a bird for lunch or made
her move her head quickly downwards?)
 Reference Resolution
 Domain Knowledge (banking transaction)
 Discourse Knowledge
 World Knowledge
U: I would like to open a fixed deposit account.
S: For what amount?
U: Make it for 800 dollars.
S: For what duration?
U: What is the interest rate for 3 months?
S: Six percent.
U: Oh good then make it for that duration.
 Why NLP is difficult?
 Different ways of Parsing a sentence
 Word category ambiguity
 Word sense ambiguity
 Words can mean more than their sum of parts (The Times of
India)
 Imparting world knowledge is difficult ("the blue pen ate the
ice-cream")
 Fictitious worlds ("people on mars can fly")
 Defining scope ("people like ice-cream," does this mean all
people like ice cream?)
 Language is changing and evolving
 Complex ways of interaction between the kinds of
knowledge
 exponential complexity at each point in using the
knowledge
Inferring Knowledge from text
 Words
 word frequencies

 collocations

 word sense

 n-grams (words appear in certain order)

 Grammar
 word categories

 syntactic structure

 Discourse
 Sentence meanings

 Applications
 Information Retrieval

 Information Extraction

 Natural language interface

 Statistical Machine Translation

 Simple Applications
 Word counters (wc in UNIX)
 Spell Checkers, grammar checkers
 Predictive Text on mobile handsets
 More significant Applications
 Intelligent computer systems
 NLU interfaces to databases
 Computer aided instruction, automatic graders
 Information retrieval
 Intelligent Web searching
 Data mining
 Machine translation
 Speech recognition
 Natural language generation
 Question answering
 Spoken Dialogue System

Speech Semantic Discourse

Recognition Interpretation Interpretation
U
s
e
r
Speech Response Dialogue
Synthesis Generation Management
Parts of the Spoken Dialogue System
 Signal Processing:
 Convert the audio wave into a sequence of feature

vectors.
 Speech Recognition:
 Decode the sequence of feature vectors into a sequence

of words.
 Semantic Interpretation:
 Determine the meaning of the words.

 Discourse Interpretation:
 Understand what the user intends by interpreting

utterances in context.
 Dialogue Management:
 Determine system goals in response to user utterances

based on user intention.

 Speech Synthesis:
 Generate synthetic speech as a response.
 Levels of Sophistication in a
Dialogue System
 Touch-tone replacement:
System Prompt: "For checking information, press or say
one."
Caller Response: "One."
 Directed dialogue:
System Prompt: "Would you like checking account
information or rate information?"
Caller Response:
"Checking", or "checking account," or "rates."
 Natural language:
System Prompt: "What transaction would you like to
perform?"
Caller Response: "Transfer Rs. 500 from checking to
savings."