CC306/ Intelligent Systems

Lecture 1
Introduction to Intelligent
Systems and AI
 Goals of this Course
• This class is a broad introduction to intelligent
systems or artificial intelligence (AI)

– AI is a very broad field with many subareas

• We will cover many of the primary concepts/ideas
• But in 14 weeks we can’t cover everything
 Why this course could change
your life…..
• As we begin the new millennium “2023”
– science and technology are changing rapidly
– “old” sciences such as physics are relatively well-understood
– computers are universal

• Grand Challenges in Science and Technology

– understanding the brain
• reasoning, cognition, creativity
– creating intelligent machines
• is this possible?
• what are the technical and philosophical challenges?
– arguably AI poses the most interesting challenges and questions in computer science
today
 What will You Get..?
• Knowledge
– General Concept of AI
– Basics of AI
– Application of AI
– Programming of AI

• How close your life with AI

• An A (depends)
• Machine Learning
• Expert System
• Knowledge Base

4
 What I Want….
• Always be prepared
• Minimize absenteeism
• Always submit your works!
• Punctuality
• Think ‘artificially’
• Read, read ….and read
5
“What is Artificial
Intelligence?
 BIG QUESTIONS?
• how does a human mind work? Eg: React with
problems, solutions, emotions
• can non-humans have minds? Eg: Robot, Smart
Phone, computer

7
 What is Intelligence?
• Intelligence:
– “the capacity to learn and solve problems” (Websters dictionary)
– in particular,
• the ability to solve novel problems
• the ability to act rationally
• the ability to act like humans

• Artificial Intelligence
– build and understand intelligent entities or agents
– 2 main approaches: “engineering” versus “cognitive modeling”
 INTELLIGENCE DEFINED
1. Someone’s intelligence is their ability to understand and learn things.
2. Intelligence is the ability to think and understand instead of doing things by instinct
or automatically. (Essential English Dictionary, Collins, London, 1990)
3. Thinking is the activity of using your brain to consider a problem or to create an
idea. (Essential English Dictionary, Collins, London, 1990)

9
 Goal of Artificial Intelligence (AI)

• The goal of artificial intelligence (AI) as a science is to

make machines do things that would require
intelligence if done by humans.
 What’s involved in Intelligence?

• Ability to interact with the real world

– to perceive, understand, and act
– e.g., speech recognition and understanding and synthesis
– e.g., image understanding
– e.g., ability to take actions, have an effect
• Reasoning and Planning
– modeling the external world, given input
– solving new problems, planning, and making decisions
– ability to deal with unexpected problems, uncertainties
• Learning and Adaptation
– we are continuously learning and adapting
– our internal models are always being “updated”
• e.g., a baby learning to categorize and recognize animals
 Definitions, Overview, History
1. What Is Artificial Intelligence
–Definition of AI
–The Turing Test
2. The History of Artificial Intelligence
3. Overview of AI application Areas

13
 Human Being: Mental
Capacities

• Perceive - distinguish
• Understand – recognize, identify
• Predict – foresee, guess
• Manipulate – control, influence

Our capacities are not equal and lie in different

areas.
14
 Artificial Intelligence?

Artificial ? “made or produced imitation of

something natural”

• Artificial flower
• Artificial light

Intelligence? “power of learning, understanding

and reasoning, mental ability ”

So what is AI?
15
 What is Artificial Intelligence?

• “A mechanical system capable of performing actions for human considered to be

intelligent”
[John McCarthy, Father of AI,1956]

• “The science of how to make computer/machine do things at this moment, people do

better”
[Elaine Rich,1991]

• “In which we consider what it means to be intelligent and whether machines could be
such a thing”
[Michael Negnevitsky, 2002]

• “Inspired by nature, Engineered by us”

[Azizi Ab Aziz, 2002]
16
 The History of
Artificial Intelligence
AI - State of the Art Timeline

18
 AI: The Past

• Greek philosophers - Plato (428-248 BC), Aristotle (384-322

BC), Diogenes (400-325 BC)  logics, philosophy of mind.
• Muslim mathematician – Al-Khawarizmi (800-840) work in
algorithm and algebra.
• Descartes (1596-1650), Leibniz (1646-1716), Kant (1724-
1804) works in conceptual and philosophy in intelligence.
• Babbage (1792-1871) works in Difference Engine, and Ada
Lovelace (1815-1852) ~ first programmer!
19
 AI: The Birth

• McCulloch and Pitts (1943)  neural computation “a logical

calculus for the ideas of immanent in nervous activity”.
• Turing (1950)  The Turing Test : “Computing Machinery and
Intelligence”
• Shannon (1950)  “The Chess Playing Computer”
• Dartmouth Conference (1956)  coined the term of “Artificial
Intelligence” : Father of AI  John McCarthy.
• Newell and Simon (1956)  General Problem Solver.
20
 AI: The Downfall

• Cancellation of machine translation project by US

government (1969)
• The Lighthill Report (1973) – claims AI techniques
are useless. (research budget has been cut-
down!)

21
 AI: The Renaissance
(The rebirth of AI)

• The emergence of knowledge based system @ expert systems

(1969-1980)  DENDRAL, MYCIN & … etc!
• The birth of artificial neural networks (1986- present)
• The birth of genetic algorithm (Holland, 1972; Schewfel, 1995)
• The emergence of intelligent agents, swarm intelligence
(1995 – present)
• Chess playing computer – Deep Blue II beats Kasparov (1997)
• Sociable and Affective Computing (1997 –present)

22
 AI in Chronology
• 1943: early beginnings
– McCulloch & Pitts: Boolean circuit model of brain
• 1950: Turing
– Turing's "Computing Machinery and Intelligence“
• 1956: birth of AI
– Dartmouth meeting: "Artificial Intelligence“ name adopted
• 1950s: initial promise
– Early AI programs, including
– Samuel's checkers program
– Newell & Simon's Logic Theorist
• 1955-65: “great enthusiasm”
– Newell and Simon: GPS, general problem solver
– Gelertner: Geometry Theorem Prover
– McCarthy: invention of LISP
 AI in Chronology
• 1966—73: Reality dawns
– Realization that many AI problems are intractable
– Limitations of existing neural network methods identified
• Neural network research almost disappears

• 1969—85: Adding domain knowledge

– Development of knowledge-based systems
– Success of rule-based expert systems,
• E.g., DENDRAL, MYCIN
• But were brittle and did not scale well in practice

• 1986-- Rise of machine learning

– Neural networks return to popularity
– Major advances in machine learning algorithms and applications

• 1990-- Role of uncertainty

– Bayesian networks as a knowledge representation framework

• 1995-- AI as Science
 Success Stories in AI
• Deep Blue defeated the reigning world chess champion Garry Kasparov in 1997

• AI program proved a mathematical conjecture (Robbins conjecture) unsolved for decades

• During the 1991 Gulf War, US forces deployed an AI logistics planning and scheduling program
that involved up to 50,000 vehicles, cargo, and people

• NASA's on-board autonomous planning program controlled the scheduling of operations for a
spacecraft

• Proverb solves crossword puzzles better than most humans

• Robot driving: DARPA grand challenge 2003-2007

• 2006: face recognition software available in consumer cameras

DENDRAL AND MYCIN
 ... history of AI (2)
• The first, very ambitious, tasks that computing science
set itself included Machine Translation and Chess
Playing (Shannon 1950). Artificial Intelligence was not
in the cards yet...
• These have not been too successful: machine
translation is still more craft than science, and
computer chess has only recently become truly
competitive, thanks to specialized or superfast
hardware.

27
 ... history of AI (3)
• The term "Artificial Intelligence" has been
coined in mid-1950s by John McCarthy (later
the inventor of Lisp).
• The first period of growth -- and funding -- came
in the 1960s. General Problem Solver (Newell &
Simon 1972): Aristotelian (!) means-ends
analysis.
• Other early applications: analogy discovery;
simple question-answering systems in toy
domains. 28
 ... history of AI (4)
• There followed a letdown and the withdrawal of
funds.
• Renewed interest in the late 1970s brought large
funding (particularly from the military). In this
period: more and more subtle knowledge
representation methods, first of all standard logic
and various advanced logics.
• AI is sometimes seen as "applied logic" (Nilsson,
early 1970s).
29
 ... history of AI (5)
• Programming languages best suited to AI tasks
are Lisp (1960) and Prolog (1972). There also
have been specialized knowledge
representation systems and languages, used to
develop knowledge bases and knowledge-
based systems. This includes expert systems, in
which probability and beliefs play an important
role. Commercialization of some expert systems
is one the signs of the growing maturity of AI.

30
 ... history of AI (6)
• First textbooks appeared late (1971, then
1984). No theory of AI exists in spite of the
massive publication rate and the bandwagon
effect (Genesereth & Nilsson 1987 is a rare
textbook devoted to the foundations of AI).
• fads and trends: expert systems, genetic
algorithms, neural networks, data mining.
Successes have been rare and sometimes
bizarre: are intelligent warheads a success?
31
 AI:
Over
the
Years
32
 Some highly visible recent AI successes in
games

Watson defeats DeepMind AlphaGo defeats Go CMU’s Libratus defeats

Jeopardy champions achieves human- champion (2016) top human poker
(2011) level players (2017)
performance on
many Atari
games (2015)
 Typical picture in news articles

BusinessInsider reporting
on the poker match…
 AI: The Fundamental

Artificial
Intelligence
“Understanding of
Human
Mind”

•Cognitive science,
“Increase
of Human Abilities” developmental psychology,
linguistic, philosophy.
•Engineering, mathematics,
economics, computer science,
operational research.

35
 Augmenting Human Abilities

• To develop intelligent system that capable to

diagnose, to recognize, to predict, to reason,
…..etc (as human counterparts)
• Example: biometrics, speech recognition,
decision support system, image recognition,
emotion recognition etc.
Crowd Counting Car Plate Recognition System

36
Speech Recognition System
Augmenting Human Abilities
 Meet HAL
http://www.youtube.com/watch?v=LE1F7d6f1Qk

• 2001: A Space Odyssey

– classic science fiction movie from 1969

• HAL
– part of the story centers around an intelligent computer called HAL
– HAL is the “brains” of an intelligent spaceship
– in the movie, HAL can
• speak easily with the crew
• see and understand the emotions of the crew
• navigate the ship automatically
• diagnose on-board problems
• make life-and-death decisions
• display emotions

• In 1969 this was science fiction: is it still science fiction?

 Hal and AI

• HAL’s Legacy: 2001’s Computer as Dream and

Reality
– MIT Press, 1997, David Stork (ed.)
– discusses
• HAL as an intelligent computer
• are the predictions for HAL realizable with AI today?
 Understanding Human Mind
• To model how our mind works. – for
better human-machine interaction,
understand mental related disorder,
coordination.
– http://www.acm.org/crossroads/xrds10-2
/robotcog.html COG: to understand human
social interaction
• E.g: humanoid COG from AI Lab MIT,
Infanoid from Japan.
– http://univ.nict.go.jp/people/xkozima/infa
noid/index-eng.html

INFANOID: to understand
infant-adult interaction Courtesy: CSAIL, MIT

40
 AI vs Our Intelligence
Artificial Intelligence Natural Intelligence
Consistent Keep on changing … chaotic

Easy to replicate No way! .. (can people download their memory?)

Cheap Too… expensive

Documental Hard to be documented
There are steps needed Creativity is the key

Symbolical input Five / six senses

A bit narrow down Broad mind! (experience tells)

41
 AI vs Conventional
Computing
AI Computing Paradigm Conventional Computing Paradigm

Based on well defined problem and representation ~ Steps by steps solution and not that robust
dynamic! enough

Symbolical manipulation More to numerical manipulation

Qualitative aspect Quantitative aspect

Capable to infer decision No!

42
 Can we build hardware as
complex as the brain?
• How complicated is our brain?
– a neuron, or nerve cell, is the basic information processing unit
– estimated to be on the order of 10 12 neurons in a human brain
– many more synapses (10 14) connecting these neurons
– cycle time: 10 -3 seconds (1 millisecond)

• How complex can we make computers?

– 108 or more transistors per CPU
– supercomputer: hundreds of CPUs, 10 12 bits of RAM
– cycle times: order of 10 - 9 seconds

• Conclusion
– YES: in the near future we can have computers with as many basic processing elements as our brain, but with
• far fewer interconnections (wires or synapses) than the brain
• much faster updates than the brain
– but building hardware is very different from making a computer behave like a brain!
 Daily Applications

• GOOGLE : self-organizing searching engine (web mining).

• Microsoft Word: office assistant (intelligent agent + decision
theoretic reasoning), spelling checker (natural language
processing).
• Washing Machine: automated setting (neural networks and
fuzzy logic).
• Digital camera (Sony) with DVD burner: image stabilizer (neural
networks and fuzzy logic).
• Hotmail, Yahoo e-mail spam filtering: data mining.
44
 Daily Applications

www.google.com

Samsung fuzzy logic

MS Word + office
washing machine
assistant

Hotmail junk mail

filtering

45
Daily AI Computing
 Can Computers Understand speech?

• Understanding is different to recognition:

– “Time flies like an arrow”
• assume the computer can recognize all the words
• how many different interpretations are there?
 Can Computers Understand speech?

• Understanding is different to recognition:

– “Time flies like an arrow”
• assume the computer can recognize all the words
• how many different interpretations are there?
– 1. time passes quickly like an arrow?
– 2. command: time the flies the way an arrow times the flies
– 3. command: only time those flies which are like an arrow
– 4. “time-flies” are fond of arrows
 Can Computers Understand speech?

• Understanding is different to recognition:

– “Time flies like an arrow”
• assume the computer can recognize all the words
• how many different interpretations are there?
– 1. time passes quickly like an arrow?
– 2. command: time the flies the way an arrow times the flies
– 3. command: only time those flies which are like an arrow
– 4. “time-flies” are fond of arrows
• only 1. makes any sense,
– but how could a computer figure this out?
– clearly humans use a lot of implicit commonsense knowledge in communication

• Conclusion: NO, much of what we say is beyond the capabilities of a computer

to understand at present
 Can Computers Learn and Adapt ?

• Learning and Adaptation

– consider a computer learning to drive on the freeway
– we could teach it lots of rules about what to do
– or we could let it drive and steer it back on course when it heads for the embankment
• systems like this are under development (e.g., Daimler Benz)
• e.g., RALPH at CMU
– in mid 90’s it drove 98% of the way from Pittsburgh to San Diego without any human assistance
– machine learning allows computers to learn to do things without explicit programming
– many successful applications:
• requires some “set-up”: does not mean your PC can learn to forecast the stock market or become a brain
surgeon

• Conclusion: Computers can learn and adapt to a limited extent, when presented with information in
the appropriate way
 Can Computers “see”?
• Recognition v. Understanding (like Speech)
– Recognition and Understanding of Objects in a scene
• look around this room
• you can effortlessly recognize objects
• human brain can map 2d visual image to 3d “map”

• Why is visual recognition a hard problem?

• Conclusion:
– mostly NO: computers can only “see” certain types of objects under very limited circumstances
– YES for certain constrained problems (e.g., face recognition)
 Can computers plan and
make optimal decisions?
• Intelligence
– involves solving problems and making decisions and plans
– e.g., you want to take a holiday in Brazil
• you need to decide on dates, flights
• you need to get to the airport, etc
• involves a sequence of decisions, plans, and actions

• What makes planning hard?

– the world is not predictable:
• your flight is canceled or there’s a backup on the 405
– there are a potentially huge number of details
• do you consider all flights? all dates?
– no: commonsense constrains your solutions
– AI systems are only successful in constrained planning problems

• Conclusion: NO, real-world planning and decision-making is still beyond the capabilities of modern
computers
– exception: very well-defined, constrained problems
 Summary of State of AI Systems in
Practice

• Speech synthesis, recognition and understanding

– very useful for limited vocabulary applications
– unconstrained speech understanding is still too hard

• Computer vision
– works for very constrained problems (hand-written zip-codes)
– understanding real-world, natural scenes is still too hard

• Learning
– adaptive systems are used in many applications: have their limits

• Planning and Reasoning

– only works for constrained problems: e.g., chess
– real-world is too complex for general systems

• Overall:
– many components of intelligent systems are “doable”
– there are many interesting research problems remaining
– AI applications nowadays are very far from being “intelligent”
 AI in the News – July 2005

• 21st – Artificial Brains to be used in robots for Mars

• 19th – Computer bots play statistically perfect poker
• 18th – Electronic Brain Helps Cut Credit Card Fraud
• 13th – Soccer robots given even odds to beat human team by 2050
• 9th – Webcrawler can complete crossword puzzles
• 8th – Robots paint and do oragami in Venice
• 2nd – Robot cleans pool without poles or hoses

54
 Worries about AI - superintelligence

writes influences donates to

Nick Bostrom Elon Musk is co-

founded
by

Yuval Noah Harari (Oct writes

2018): “for every dollar
and every minute we
invest in improving AI, we Max Tegmark
would be wise to invest a
dollar and a minute in
exploring and developing
human consciousness.”
 Worries about AI - near term

technological unemployment

autonomous
weapon systems
autonomous vehicles – legal and other issues …
 What is Artificial Intelligence?

A general classification of AI systems, due to Russell and Norvig (1995, 2003):

Thought Systems that think like Systems that think rationally

process
/ reasoning
humans
(Turing Test)
Systems that act like humans Systems that act rationally
Behavior/ Action

Human performance Ideal concept –’rationality’

rationality: it does the

Right thing given what it knows

57
“THE TURING TEST”

58
The Turing Test

59
 Turing Test
• When does a system behave intelligently?
– Alan Turing (1950) Computing Machinery and Intelligence

Turing Test
• Designed to provide a satisfactory operational test of intelligence: imitation
game
• Test still relevant now, yet might be the wrong question.
• Requires the collaboration of major components of AI: natural language
processing, knowledge representation, automated reasoning, machine
learning……

60
 Systems that act like
humans
• The Turing test involves a
computer, a human interrogator
and a human.
• The interrogator attempts to
determine, by asking questions of
the other two participants.
• All communication is via
keyboard and screen.
• The human must help the
interrogator to make a correct
identification. A number of
different people play the roles of
interrogator and human, 61
 Acting humanly:
Turing test
• Turing (1950) "Computing machinery and intelligence“

• "Can machines think?"  "Can machines behave intelligently?“

• Operational test for intelligent behavior: the Imitation Game

• Suggests major components required for AI:

- knowledge representation
- reasoning,
- language/image understanding,
- learning

* Question: is it important that an intelligent system act like a human?

 Turing Imitation Game – Phase 1

In the first phase, the interrogator, a man

and a woman are each placed in separate
rooms. The interrogator’s objective is to
work out who is the man and who is the
woman by questioning them. The man
should attempt to deceive the
interrogator that he is the woman, while
the woman has to convince the
interrogator that she is the woman.
 Turing Imitation Game – Phase 2

In the second phase of the game, the

man is replaced by a computer
programmed to deceive the interrogator
as the man did. It would even be
programmed to make mistakes and
provide fuzzy answers in the way a
human would. If the computer can fool
the interrogator as often as the man did,
we may say this computer has passed
the intelligent behaviour test.
 The Turing Test

“If the computer can fool the interrogator

as often as the man did, we may say this
computer has passed the intelligent
behaviors test.”

Alan Turing

65
Well it Worked! – Sort Of
 The sources of
Artificial Intelligence
• Philosophy (ontology, epistemology, ...)
• Mathematics (logic, geometry, probability,
decision theory, ...)
• Psychology
• Linguistics, psycholinguistics
• Computing (theory; engineering practice)
67
 The Areas of Artificial
Intelligence
• Search (blind, informed, adversarial)
• Knowledge representation (logic, semantic networks, frames, rules, neural
networks)
• Planning
• Machine Learning (symbolic, statistical; data mining)
• Natural Language Processing (symbolic, statistical; text mining)
• Perception (vision, speech)
• Robotics

68
 Realistic Perspective on AI
• Very important to get an objective and honest view of current state of AI and the limitations of AI.
• Forget about what you see in “futuristic movies” or various hypes that surround this field.
• The term “artificial intelligence” or “intelligent system” is a misnomer.
– There is no real intelligence to them at all.
• Most impressive state-of-the-art achievements actually come from machine learning which is a
sub-area under AI.
• Machine learning is simply statistical learning (i.e. modelling and optimization) applied to
problems that are being researched under the field of AI.
• In this course and in your life, whenever you see the term “intelligent” as associated with
computers, realize that it is only a metaphor.
– For computers, it’s “garbage in and garbage out”.
– The intelligence that we see is the result of the collective intelligence of human beings that
code these intelligent systems.
 Realistic Perspective on AI
• The more we learn about this field, the more we find how much difficult it is to
achieve human-like intelligence.
• We are even not sure whether computers can actually gain human-like general
intelligence in the first place.
– Most likely it is not possible at all.
• In fact, even human beings do not understand about their own brains or minds.
• Goal of strong AI: to develop human-like general intelligence
• Goal of weak AI: just to create useful applications that appear “intelligent” in some
narrow tasks  this should be the goal of everybody.
• Weak AI can also considered as the “practical AI” or “engineering approach to AI”.
– AI is just a tool to build useful applications and systems.
Common Techniques in AI
 Discussion
• Can machines think?
• Can machines see?
• What is consciousness?
• How does a human mind work?
• Can minds come physical materials?
• Can non-humans have minds?
• Can machines replace a human worker?
• Are intelligent machines good or bad for humans?
• Would you trust one?
Questions ?
END OF LECTURE 1