IST 511 Information Management: Information and

Technology
Artificial Intelligence and the Information Sciences

Dr. C. Lee Giles

David Reese Professor, College of
Information Sciences and Technology
Professor of Computer Science and
Engineering
Professor of Supply Chain and Information
Systems
The Pennsylvania State University,
University Park, PA, USA
[email protected]

Special thanks http://clgiles.ist.psu.edu

to Y. Peng at UMBC and P. Parjanian of USC
 Last time
• What is complexity
– Complex systems
– Measuring complexity
• Computational complexity – Big O
– Scaling
• Why do we care
– Scaling is often what determines if information
technology works
– Scaling basically means systems can handle a
great deal of
• Inputs
• Users
• growth
• Methodology – scientific method
 The Scientific Method
• Observe an event(s).
• Develop a model (or hypothesis) which
makes a prediction to explain the event
• Test the prediction with data
• Observe the result. model

• Revise the hypothesis.

• Repeat as needed.
• A successful hypothesis becomes atest
Scientific Theory.
 Today
• What is AI
– Definitions
– Theories/hypotheses
• Why do we care
• Impact on information science
• Great resource
– AI Topics
 Tomorrow
Topics used in IST
• Machine learning
• Information retrieval and search
• Text
• Encryption
• Social networks
• Probabilistic reasoning
• Digital libraries
• Others?
Theories in Information Sciences
• Enumerate some of these theories in
this course.
• Issues:
– Unified theory?
– Domain of applicability
– Conflicts
• Theories here are mostly algorithmic
• Quality of theories
– Occam’s razor
– Subsumption of other theories
• If AI is really true, unified theory of
most (all?) of information science
Artificial Intelligence in the
Movies
 Artificial Intelligence in Real
Life
A young science (≈ 50 years old)
– Exciting and dynamic field, lots of uncharted territory
left
– Impressive success stories
– “Intelligent” in specialized domains
– Many application areas

Face detection Formal verification

 Why the interest in AI?

Search engines
Science

Medicine/
Diagnosis
Labor
Appliances What else?
 What is artificial intelligence?
• There is no clear consensus on the definition of AI
• John McCarthy coined the phrase AI in 1956
http://www.formal.stanford.edu/jmc/whatisai/whatisai.html
Q. What is artificial intelligence?
A. It is the science and engineering of making
intelligent machines, especially intelligent computer
programs. It is related to the similar task of using
computers to understand human or other
intelligence, but AI does not have to confine itself to
methods that are biologically observable.
Q. Yes, but what is intelligence?
A. Intelligence is the computational part of the ability
to achieve goals in the world. Varying kinds and
degrees of intelligence occur in people, many
What is AI? (Cont’d)
 Other possible AI definitions
• AI is a collection of hard problems which can be solved by
humans and other living things, but for which we don’t
have good algorithms for solving.
– e. g., understanding spoken natural language, medical
diagnosis, circuit design, learning, self-adaptation,
reasoning, chess playing, proving math theories, etc.
• Russsell & Norvig: a program that
– Acts like human (Turing test)
– Thinks like human (human-like patterns of thinking
steps)
– Acts or thinks rationally (logically, correctly)
• Some problems used to be thought of as AI but are now
considered not
– e. g., compiling Fortran in 1955, symbolic mathematics
in 1965, pattern recognition in 1970, what for the
future?
What is the scientific method hypothesis behind AI?
 One Working Definition of AI

Artificial intelligence is the study of how to make

computers do things that people are better at or would be
better at if:
• they could extend what they do to a World Wide
Web-sized amount of data and
• not make mistakes.
 AI Purposes

"AI can have two purposes. One is to use the

power of computers to augment human thinking,
just as we use motors to augment human or horse
power. Robotics and expert systems are major
branches of that. The other is to use a computer's
artificial intelligence to understand how humans
think. In a humanoid way. If you test your
programs not merely by what they can
accomplish, but how they accomplish it, they
you're really doing cognitive science; you're using
AI to understand the human mind."
- Herb Simon
 What’s easy and what’s hard?
• It’s been easier to mechanize many of the high level
cognitive tasks we usually associate with “intelligence” in
people
– e. g., symbolic integration, proving theorems, playing
chess, some aspect of medical diagnosis, etc.
• It’s been very hard to mechanize tasks that animals can
do easily
– walking around without running into things
– catching prey and avoiding predators
– interpreting complex sensory information (visual,
aural, …)
– modeling the internal states of other animals from
their behavior
– working as a team (ants, bees)
• Is there a fundamental difference between the two
categories?
• Why are some complex problems (e.g., solving
differential equations, database operations) are not
 History of AI
• AI has roots in a number of scientific disciplines
– computer science and engineering (hardware and
software)
– philosophy (rules of reasoning)
– mathematics (logic, algorithms, optimization)
– cognitive science and psychology (modeling high level
human/animal thinking)
– neural science (model low level human/animal brain
activity)
– linguistics
• The birth of AI (1943 – 1956)
– McCulloch and Pitts (1943): simplified mathematical
model of neurons (resting/firing states) can realize all
propositional logic primitives (can compute all Turing
computable functions)
– Alan Turing: Turing machine and Turing test (1950)
– Claude Shannon: information theory; possibility of chess
playing computers
• Early enthusiasm (1952 – 1969)
– 1956 Dartmouth conference
John McCarthy (Lisp);
Marvin Minsky (first neural network machine);
Alan Newell and Herbert Simon (GPS);
– Emphasis on intelligent general problem solving
GSP (means-ends analysis);
Lisp (AI programming language);
Resolution by John Robinson (basis for
automatic theorem proving);
heuristic search (A*, AO*, game tree search)
• Emphasis on knowledge (1966 – 1974)
– domain specific knowledge is the key to overcome
existing difficulties
– knowledge representation (KR) paradigms
– declarative vs. procedural representation
• Knowledge-based systems (1969 – 1999)
– DENDRAL: the first knowledge intensive system
(determining 3D structures of complex chemical
compounds)
– MYCIN: first rule-based expert system (containing 450
rules for diagnosing blood infectious diseases)
EMYCIN: an ES shell
– PROSPECTOR: first knowledge-based system that made
significant profit (geological ES for mineral deposits)
• AI became an industry (1980 – 1989)
– wide applications in various domains
– commercially available tools
– AI winter
• Current trends (1990 – present)
– more realistic goals
– more practical (application oriented)
– distributed AI and intelligent software agents
– resurgence of natural computation - neural networks and
emergence of genetic algorithms – many applications
–
 AI is Controversial
• AI Winter – too much promised
• 1966: the failure of machine translation,
• 1970: the abandonment of connectionism,
• 1971−75: DARPA's frustration with the Speech Understanding Research
program at Carnegie Mellon University
• 1973: the large decrease in AI research in the United Kingdom in response to
the Lighthill report,
• 1973−74: DARPA's cutbacks to academic AI research in general,
• 1987: the collapse of the Lisp machine market,
• 1988: the cancellation of new spending on AI by the Strategic Computing
Initiative
• 1993: expert systems slowly reaching the bottom
• 1990s: the quiet disappearance of the fifth-generation computer project's
original goals,

• AI will cause
– social ills, unemployment
– End of humanity
Thinking Humanly: Cognitive
Science
• 1960 “Cognitive Revolution”: information-
processing psychology replaced behaviorism

• Cognitive science brings together theories and

experimental evidence to model internal
activities of the brain
– What level of abstraction? “Knowledge” or “Circuits”?
– How to validate models?
• Predicting and testing behavior of human subjects (top-
down)
• Direct identification from neurological data (bottom-up)
• Building computer/machine simulated models and
reproduce results (simulation)
 Thinking Rationally: Laws of
Thought
• Aristotle (~ 450 B.C.) attempted to codify “right thinking”
What are correct arguments/thought processes?
• E.g., “Socrates is a man, all men are mortal; therefore
Socrates is mortal”

• Several Greek schools developed various forms of logic:

notation plus rules of derivation for thoughts.

• Problems:
1) Uncertainty: Not all facts are certain (e.g., the flight might be
delayed).
2) Resource limitations: There is a difference between solving a
problem in principle and solving it in practice under various
resource limitations such as time, computation, accuracy etc.
(e.g., purchasing a car)
 Strong AI
"I find it useful to distinguish what I will call
"strong" AI from "weak" or "cautious" AI. According
to weak AI, the principle value of the computer in
the study of the mind is that it gives us a very
powerful tool. For example, it enables us to
formulate and test hypothesis in a more rigourous
and precise fashion. But according to strong AI, the
computer is not merely a tool in the study of the
mind; rather, the appropriately programmed
computer really is a mind, in the sense that
computers given the right programs can be literally
said to understand and have other cognitive
states."
[Searle, 1980, Minds, Brains and Programs]
 Weak and Strong AI Claims
• Weak AI:
– Machines can be made to act as if
they were intelligent.
• Strong AI:
– Machines that act intelligently have
real, conscious minds.
 Eliza
• Called after Eliza Doolittle of Pygmalion
fame.
• Developed in 1964-1966 by Joseph
Weizenbaum in MIT
• Models (parodies) the rôle of a Rogerian
psychotherapist engaged in an initial
interview with a patient. Much or the
technique of the Rogerian
psychotherapist involves drawing the
patient out by reflecting the patient’s
statements back at him.

http://www-ai.ijs.si/eliza/eliza.html
 Eliza

Men are all alike

IN WHAT WAY
They’re always bugging us about something or other.
CAN YOU THINK OF A SPECIFIC EXAMPLE
Well, my boyfriend made me come here.
YOUR BOYFRIEND MADE YOU COME HERE
He says I am depressed much of the time.
I AM SORRY TO HEAR YOU ARE DEPRESSED
It’s true. I am unhappy.
DO YOU THINK COMING HERE WILL HELP YOU NOT TO BE UNHAPPY
I need some help, that much seems certain.
WHAT WOULD IT MEAN TO YOU IF YOU GOT SOME HELP
Perhaps I could learn to get along with my mother.
 Eliza
TELL ME MORE ABOUT YOUR FAMILY
My mother takes care of me.
WHO ELSE IN YOUR FAMILY TAKES CARE OF YOU
My father.
YOUR FATHER
You are like my father in some ways
WHAT RESEMBLANCE DO YOU SEE
You are not very aggressive but I think you don’t want me to notice that.
WHAT MAKES YOU THINK I AM NOT VERY AGGRESSIVE
You don’t argue with me
WHY DO YOU THINK I DON’T ARGUE WITH YOU
 What is/was Eliza?
• A chatterbot
• Turing test competitor
• A friend
• A model for other bots
 What is a chatterbot?
• A chatter robot, chatterbot, chatbot, or chat bot is a
computer program designed to simulate an intelligent
conversation with one or more human users via auditory
or textual methods, primarily for engaging in small talk.
– The primary aim of such simulation has been to fool the user into
thinking that the program's output has been produced by a human
(the Turing test).
– Programs playing this role are sometimes referred to as Artificial
Conversational Entities, talk bots or chatterboxes.
– Uses:
• chatterbots are often integrated into dialog systems for various practical purposes
such as online help, personalised service, or information acquisition.
• Spam in chatrooms
– Some chatterbots use sophisticated natural language processing
systems, but many simply scan for keywords within the input and
pull a reply with the most matching keywords, or the most similar
wording pattern, from a textual database.
– Collections:

http://www.simonlaven.com/
 Types of Chatterbots
• Classic Chatterbots
• Complex Chatterbots
• Friendly Chatterbots
• Teachable Bots
• AIML Bots
• JFred Bots
• NativeMinds Bots Non-English Bots
• Alternative Bots

http://www.simonlaven.com/
A.L.I.C.E
 Philosophical criticisms of AI
• Two categories of criticism:
– It cannot be done because ...
– It cannot be done the way you are
trying to do it.
The danger of can’t be done arguments…
"Philosophers are forever telling scientists what they can't do,
what they can't say, what they can't know, and so on and so
forth. In 1844 the philosopher August Compte said that if there
was one thing man would never know it would be the
composition of the distant stars and planets. But three years
after Compte died physicists discovered that an object's
composition can be determined by its spectrum no matter how
far off the object happens to be."
What is Intelligence?
The Turing Test

A machine can be described

as a
thinking machine if it passes
the
Turing Test. i.e. If a human
agent is engaged in two
isolated dialogues (connected
by teletype say); one with
a computer, and the other
with
another human and the
human agent cannot reliably
identify which dialogue is
 Intelligence
• Turing Test: A human communicates
with a computer via a teletype. If the
human can’t tell he is talking to a
computer or another human, it passes.
– Natural language processing
– knowledge representation
– automated reasoning
– machine learning
• Add vision and robotics to get the total
Turing test.
 Objections to the TT
• The Theological Objection
– "Thinking is a function of man’s immortal
soul. God has given an immortal soul to
every man and woman, but not to any
other animal or to machine. Hence no
animal or machine can think."
• The “Head in the Sand” Objection
– "The consequences of machines thinking
are to dreadful to think about."
 Objections to the TT
• Mathematical Objections
– "There are a number of results of
mathematical logic that can be used to
show that there are limitations to the
power of discrete state machines.“
• (eg. Gödel’s incompleteness theorem)
• The Argument for Consciousness
– “A machine cannot write a sonnet or
compose a concerto because of
thoughts or emotions felt.”
Types of Intelligence Tests
 Connectionist (Subsymbolic)
Hypothesis
“The intuitive processor is a
subconceptual connectionst
dynamical system that does not
admit a complete, formal and
precise conceptual-level
description.” [Smolensky 1988]
The inner workings of an ANN are difficult to
interpret – but are they substantially different to
a symbolic system?
 Physical Symbol System
Hypothesis
• A physical symbol system has the
necessary and sufficient means for
intelligent action Newell & Simon 1976

– a system, embodied physically, that is engaged

in the manipulation of symbols
– an entity is potentially intelligent if and
only if it instantiates a physical symbol system
– symbols must designate
– symbols must be atomic
– symbols may combine to form expressions
What does the PSSH mean?
• Intelligent action
can be modelled by
a system
manipulating
symbols.
• Nothing special
about our wetware.
• Intelligence can be
implemented on
other platforms, e.g.
silicon.
 Symbolic AI: Rule-Based
Systems
• Whale Watcher Demo
– http://www.aiinc.ca/demos/
whale.shtml
 Rule-Based System: Car
Maintenance
BadElecSys:
IF car:SparkPlusCondition #= Bad Or
car:Timing #= OutOfSynch Or
car:Battery #= Low;
THEN car:ElectricalSystem = Bad;

GoodElecSys:
IF car:SparkPlugCondition #= Ok And
car:Timing #= InSynch And
car:Battery #= Charged;
THEN car:ElectricalSystem = Ok;
 Consider the following rules
If A and B then F J
If C and D A
and E then K Goal
B F
If F and K then G
If J and G then Goal C
G
D
K
E

We can Forward Chain from Premises to Goals

or Backward Chain from Goals and try to prove them
 A model of knowledge-based
systems development

Real
Reasoning
World Problem
Analysis System ?
Proble Solution
m

Representation
• Logical AI
Branches of AI
• Search
• Natural language processing
• Computer vision
• Pattern recognition
• Knowledge representation
• Inference From some facts, others can be inferred.
• Reasoning
• Learning
• Planning To generate a strategy for achieving some goal
• Epistemology This is a study of the kinds of knowledge that
are required for solving problems in the world.
• Ontology Ontology is the study of the kinds of things that
exist.
• Agents
• Games
• Artificial life / worlds?
• Emotions?
• Knowledge Management?
• Socialization/communication?
• …
 Approaches to AI
• Searching
• Learning
• From Natural to Artificial Systems
• Knowledge Representation and
Reasoning
• Expert Systems and Planning
• Communication, Perception, Action
 Search
• “All AI is search”
– Game theory
– Problem spaces
• Every problem is a feature space
of all possible (successful or
unsuccessful) solutions.
• The trick is to find an efficient
search strategy.
Search: Game Theory

9!+1 = 362,880
 Approaches to AI
• Searching
• Learning
• From Natural to Artificial Systems
• Knowledge Representation and
Reasoning
• Expert Systems and Planning
• Communication, Perception, Action
 Learning
• Explanation
– Discovery
– Data Mining
• No Explanation
– Neural Nets
– Case Based Reasoning
 Learning: Explanation
• Cases to rules
 Learning: No Explanation
• Neural nets
 Approaches to AI
• Searching
• Learning
• From Natural to Artificial Systems
• Knowledge Representation and
Reasoning
• Expert Systems and Planning
• Communication, Perception, Action
Neural Networks


 Approaches to AI
• Searching
• Learning
• From Natural to Artificial Systems
• Knowledge Representation and
Reasoning
• Expert Systems and Planning
• Communication, Perception, Action
 Rule-Based Systems
• Logic Languages
– Prolog, Lisp
• Knowledge bases
• Inference engines
Rule-Based Languages:
Prolog

Father(abraham, isaac). Male(isaac).

Father(haran, lot). Male(lot).
Father(haran, milcah). Female(milcah).
Father(haran, yiscah). Female(yiscah).
Son(X,Y)  Father(Y,X), Male(X).
Daughter(X,Y)  Father(Y,X), Female(X).

Son(lot, haran)?
 Rule
Based
System
s
• KRS
 Approaches to AI
• Searching
• Learning
• From Natural to Artificial Systems
• Knowledge Representation and
Reasoning
• Expert Systems and Planning
• Communication, Perception, Action
 Approaches to AI
• Searching
• Learning
• From Natural to Artificial Systems
• Knowledge Representation and
Reasoning
• Expert Systems and Planning
• Communication, Perception, Action
 Ability-Based Areas

• Computer vision
• Natural language recognition
• Natural language generation
• Speech recognition
• Speech generation
• Robotics
• Games/entertainment
MIT’s NLP online
 Natural Language:
Translation

“The flesh is weak, but the spirit is

strong”
 Translate to Russian
 Translate back to English

“The food was lousy, but the vodka was

great!”
 Natural Language
Recognition
OBJ GOLD: X

PERSON: PERSON:
Semantics REPT TRANSACTION AGNT Fred
Joe

Context

sentence
w

VP
VP
NP

Syntax VP NP NP

pronoun verb pronoun article noun

n d

Words You give me the gold

Audio
 Natural Language
Recognition
PERSON: BELIEF
“Tom Tom
EXPR

believes
PTNT
Mary
wants to
marry a PROPOSITION
:
sailor.” PERSON:
EXPR WANT
Mary

PTNT

SITUATION:

T AGNT MARRY PTNT SAILOR

The Jetsons - 1962
 Honda Humanoid Robot

Walk

Turn

Stairs
Domestic Robots
Military robots
 Robocup

www.robocup.org
 How far have we got?
• General intelligence of a frog?
 But then ask Garry K.

But don’t try to ask Deep Blue

 Watson

• “The goal is to have computers start to

interact in natural human terms across a
range of applications and processes,
understanding the questions that humans ask
and providing answers that humans can
understand and justify” - IBM
 Watson

• IBM’s Artificial
Intelligence
computer system
• Capable of
answering
questions in
natural language
• Competed against
champions on
Jeopardy and won
 Watson
• IBM describes this AI as:
"an application of advanced Natural
Language Processing, Information
Retrieval, Knowledge
Representation and Reasoning,
and Machine Learning technologies
to the field of open domain
question answering“
• What this means…
High-Level Architecture used in Watson
 Watson
• Specifics
– 16 Terabytes of RAM
– Can process 500 gigabytes (1 million
books) per second
– Content was stored in Watson’s RAM rather
than memory to be more easily accessed
– Cost about $3 Million
 Watson’s sources of
information
• Encyclopedias
• Dictionaries
• Thesauri
• Newswire articles
• Literary works
• Databases, taxonomies, and
ontologies.
• Wikipedia articles
• And more
 How Watson Works
• Receives the clues (questions) as
electronic texts
• It then divides these texts into different
keywords and sentence fragments and
searches for statistically related phrases
• Quickly executes thousands of language
analysis algorithms
• The more algorithms that find the same
answer increase Watson’s confidence of
his answer and it calculates whether or
not to make a guess
 How to achieve AI?
• How is AI research and engineering done?
• AI research has both theoretical and experimental sides.
The experimental side has both basic and applied
aspects.
• Competitions!
• There are two main lines of research:
– One is biological, based on the idea that since humans are
intelligent, AI should study humans and imitate their
psychology or physiology.
– The other is phenomenal, based on studying and
formalizing common sense facts about the world and the
problems that the world presents to the achievement of
goals.
• The two approaches interact to some extent, and both
should eventually succeed. It is a race, but both racers
seem to be walking. [John McCarthy]
 AI competitions
• Robotics - Robocup
• Chess /other games
• Turing Test (Loebner prize)
• Theorem proving
• Planning (agent)
• Data mining
• DOD autonomous cross country driving
• Finance
• Recently:
– Mario AI competition
– Google AI Challenge
 AI as an Agent

sensors
?
?
environment
agent ?
actuators

model
 What is an (Intelligent)
Agent?
• An over-used, over-loaded, and miss-used term.

• Anything that can be viewed as perceiving its environment

through sensors and acting upon that environment through
its effectors to maximize progress towards its goals.
– Crawlers?
– Daemons?

• PAGE (Percepts, Actions, Goals, Environment)

• Task-specific & specialized: well-defined goals and

environment

Many AI systems can be recast as Agents Systems

Agents can be quite
sophisticated

Utility agent
 Intelligent Agents in the
Knowledge Representation World
Machine Learning abilities
Reasoning +
Decision Theory

Natural Language
Generation
Natural Language +
Understanding Robotics
+ +
Computer Vision Human Computer
Speech Recognition /Robot
+ Interaction
Physiological Sensing
Mining of Interaction Logs 92
 Strong vs Weak AI
• Strong AI is artificial intelligence that matches or exceeds
human intelligence — the intelligence of a machine that can
successfully perform any intellectual task that a human being
can.[1]
– It is a primary goal of artificial intelligence research and an important topic
for science fiction writers and futurists.
– Strong AI is also referred to as "artificial general intelligence"[2] or as the
ability to perform "general intelligent action".[3]
– Science fiction associates strong AI with such human traits as consciousness,
sentience, sapience and self-awareness.

• Weak AI is an artificial intelligence system which is not

intended to match or exceed the capabilities of human beings,
as opposed to strong AI, which is. Also known as applied AI or
narrow AI.
– The weak AI hypothesis: the philosophical position that machines can
demonstrate intelligence, but do not necessarily have a mind, mental states
or consciousness. (See philosophy of artificial intelligence or John Searle's
definition of Strong AI in Chinese Room)
AI State of the art - applications
• AI achievements:
– Facilitate and replace human decision
making World-class chess and game playing
– Robots
– Automatic process control
– Understand limited spoken language
– Smarter search engines
– Engage in a meaningful conversation
– Observe and understand human emotions
– Solving mathematical problems
– Discover and prove mathematical theories
– …
world robot population
world robot population
 What we know
• Applications of AI everywhere
• With Moore’s law, more will appear
– Why?
 Future of AI
• Based on the continued progress of Moore’s law

• Measure progress

• Brute force vs cleverness

• New apps

“By 2010 computers will disappear. They’ll be so small,

they’ll be embedded in our clothing, in our environment.
Images will be written directly to our retina, providing full-
immersion virtual reality, augmented real reality. We’ll be
interacting with virtual personalities.” (Ray Kurzweil in
2005)
The Singularity
 AI questions
• What is the sicentific method hypothesis
behind AI?
• Future of AI, friend or foe
• What is the impact and role of AI on/in
information sciences
• How can AI be used in information
sciences research
• Will AI ever exceed NI?
• Will we work together?
• Human-computing collaboration (Shyam Sankar –
Ted)
• Human-based computation