Advanced Artificial Intelligence

Lecture 1-2: Introduction

Instructor: Dr. Sakeena Javaid

 Basic information about course

 Textbook: Artificial Intelligence: A Modern Approach, Third

Edition
 Authors: Stuart J. Russel and Peter Norving
 Instructor: Dr. Sakeena Javaid
– Ph.D. CUI 2020
– Research on energy management in smart homes and smart buildings,
heuristic algorithms,

2
 Course Objectives
 Provide a concrete grasp of the fundamentals of various
techniques and branches that currently constitute the
field of Artificial Intelligence, e.g.,
i. Agents
ii. Search
iii. Knowledge Representation
iv. Autonomous planning
v. Reasoning under uncertainty
vi. Other advanced techniques for optimization

3
 Prerequisites
 Comfortable programming in language such as C (or C++) or Java
 Some knowledge of algorithmic concepts such as running times of algorithms;
having some rough idea of what NP-hard means
 Some familiarity with probability (we will go over this from the beginning but we
will cover the basics only briefly)
 Not scared of mathematics, some background in discrete mathematics, able to do
simple mathematical proofs
 If you do not have a standard undergraduate computer science background, talk to me
first.
 Well-prepared undergraduates are certainly welcome
 You do not need to have taken an undergraduate AI course (though of course it will
help if you have)

4
 Grading
 Assignments and quizzes: 10%
– Midterm exams (OHT-1 & OHT-II): 30%
 Final exam: 50%
 Research Activities: 10%
 Research should be on individual basis
 May discuss with another person
 Idea, write-up and code must be your own

5
 Course Overview
 Introduction and Agents (Chapters 1,2)

 Search (Chapters 3,4,5,6)

 Logic (Chapters 7,8,9)

 Planning (Chapters 11,12)

 Reasoning under uncertainty …

6
 Agenda of the Lecture

What is AI?
Foundations and history of AI
The AI becomes an industry
Recent trends

7
What is artificial intelligence?

The ability of a digital computer or computer-

controlled robot to perform tasks commonly
associated with intelligent beings.

The capability of a machine to imitate

the intelligent human behavior

8
 What is artificial intelligence?
 Definitions of artificial intelligence according to eight recent textbooks are shown in
Figure1.1.
 These definitions vary along two main dimensions. The ones on top are concerned
with thought processes and reasoning, whereas the ones on the bottom address
behavior. They are organized into four categories:
Systems that think like humans. Systems that think rationally.

"The exciting new effort to make computers "The study of mental faculties through the use of
think . . . machines with minds, in the full and computational models" (Charniak and
literal sense" (Haugeland, 1985) McDermott, 1985)
"The automation of activities that we associate "The study of the computations that make it
with human thinking, activities such as decision- possible to perceive, reason, and act" (Winston,
making, problem solving, learning ..."(Bellman, 1992)
1978)
"The art of creating machines that perform "A field of study that seeks to explain and
functions that require intelligence when emulate intelligent behavior in terms of
performed by people" (Kurzweil, 1990) computational processes" (Schalkoff, 1990)
"The study of how to make computers do things "The branch of computer science that is
at which, at the moment, people are better" concerned with the automation of intelligent
(Rich and Knight, 1 99 1 ) behavior" (Luger and Stubblefield, 1993)

9
Systems that act like humans. Table 1: Some definitions of AI. Systems that act rationally
 What is artificial intelligence?
 Thinking humanly: The cognitive modelling approach
 Expressing the Theory of Mind as a Computer Program
 There are two ways to do this: through introspection—trying to catch
our own thoughts as they go by—or through psychological
experiments
– GPS (Newell & Simon 1961) does not only need to solve the
problems but should also follow human thought process
 Requires scientific theories of internal activities of the brain.
– Cognitive Science: Predicting and testing behavior of human
subjects
– Cognitive Neuroscience: Direct identification from neurological
data
10
Acting humanly: The Turing Test Approach
 Turing (1950) “Computing machinery and intelligence”.
 (Human) judge communicates with a human and a machine over text-
only channel
 Both human and machine try to act like a human
 A computer passes the test if a human interrogator, after posing some
written questions, cannot tell whether the written responses come from a
person or from a computer.
– http://www.jabberwacky.com/
– http://turingtrade.org/

11
 Acting humanly: The Turing Test Approach

• Major Components of Turing Test:

– Natural Language Processing: To enable it to communicate
successfully in English.
– Knowledge Representation: To store what it knows or hears.
– Automated Reasoning: To use the stored information to
answer questions and to draw conclusions.
– Machine Learning: To adapt to new circumstances and to
detect and extrapolate patterns.
• Total Turing Test also includes:
– Computer Vision: To perceive objects
– Robotics: To manipulate objects and move about

12
Thinking rationally: The laws of thought
approach
 Thinking rationally: The laws of thought approach
 Initiated the field of logic: Greek philosopher Aristotle was one of
the first to attempt to codify "right thinking,"
 For example, "Socrates is a man; all men are mortal; therefore
Socrates is mortal."
 Several Greek schools developed various forms of logic:
– Notation and rules of derivation for thoughts
 By 1965, programs existed that could, in principle, solve any
solvable problem described in logical notation.

13
Thinking rationally: The laws of thought
approach
 Problems:
– Not easy to state informal knowledge in logical notation
– Big difference between solving a problem "in principle" and solving it
“in practice”
• Problems with just a few hundred facts can exhaust the
computational resources of any computer

14
Acting rationally: The rational agent
approach
 Rational behavior: doing the right thing
 The right thing: the optimal (best) thing that is expected to maximize
the chances of achieving a set of goals, in a given situation
 Acting rationally means acting so as to achieve one's goals, given
one's beliefs. An agent is just something that perceives and acts.
 For example, pulling one's hand off of a hot stove is a reflex action
that is more successful than a slower action taken after careful
deliberation.
 Making correct inferences is sometimes part of being a rational agent

15
Acting rationally: The rational agent
approach
 Advantages over other approaches
– More general than the "laws of thought" approach
– More amenable to scientific development than are approaches
based on human behavior or human thought
– Standard of rationality is mathematically well defined and
completely general

16
 Rational Agents

• An agent is an entity that perceives and acts

• This course is about designing rational/intelligent agents
• Abstractly, an agent is a function which percepts histories to
actions:
– f : P* -> A
• For any given class of environments and tasks, we seek the
agent (or class of agents) with the optimal (best) performance
• Caveat: computational limitations make perfect rationality
unachievable
– So we attempt to design the best (most intelligent) program, under
the given resources.
17
 Definitions of AI
if our system can be more rational
than humans in some cases, why not?

Systems that think like Systems that think

Focus on action avoids
humans rationally
philosophical issues such as “is
the system conscious” etc. Systems that act like humans Systems that act
rationally
 We will follow “act rationally” approach, i.e., the creation, design and

implementation of rational agents

– Distinction may not be that important
 acting rationally/like a human presumably requires (some sort of)
thinking rationally/like a human,
 humans much more rational anyway in complex domains 18
The Foundations of Artificial Intelligence

 Although AI itself is a young field, it has inherited many

ideas, viewpoints, and techniques from other disciplines
 Philosophy: Logic, methods of reasoning, mind as physical
system, foundations of learning, language, rationality
 Mathematics: Formal representation and proof, Algorithms,
Computation, (un)decidability, (in)tractability, probability
 Psychology: Adaptation, phenomena of perception and
motor control, experimental techniques (with animals, etc.)
 Economics: Formal theory of rational decisions
19
The Foundations of Artificial Intelligence

 Linguistics: Knowledge representation, grammar

 Neuroscience: Plastic physical substrate for mental activity
 Control theory: Homeostatic systems, Stability, Simple optimal
agent designs.

20
 The History of Artificial Intelligence
 By identifying loosely defined and overlapping phases in its
development
 By chronicling the various different and intertwined conceptual
threads that make up the field

21
 The History of Artificial Intelligence

 The gestation of artificial intelligence (1943-1956)

 The first work that is now generally recognized as AI was done by
Warren McCulloch and Walter Pitts (1943)
- also introduced theory of neurons as logical computing circuits
- They drew on few sources: knowledge of the basic physiology and
function of neurons in the brain;
• Proposed a model of artificial neurons
• Each neuron is characterized as being "on" or “off,"
• Switch to "on" occurring in response to stimulation by a sufficient
number of neighboring neurons.
22
 The History of Artificial Intelligence
• The state of a neuron was conceived of as "factually equivalent to a
proposition
• Any computable function could be computed by some network of
connected neurons
• All the logical connectives (and, or, not, etc.) could be implemented
by simple net structures.
• McCulloch and Pitts also suggested that suitably defined networks
could learn.
• First Neural Network Computer (1950)

23
 The History of Artificial Intelligence

 Dartmouth conference (1956)

– Organized by John McCarthy attended by Marvin Minsky, Allen
Newell, Herb Simon, and a few others.
– Coined term “artificial intelligence.”
– Presentation of game playing programs and Logic Theorist
(computer program that could prove theorems in symbolic logic)

24
 The History of Artificial Intelligence
 Early enthusiasm, great expectations (1952-1969)
 Development of General Problem Solver (thinking humanly
approach) by Newell and Simon in early sixties
 Arthur Samuel’s (1956) work on learning to play checkers
 Frank Rosenblatt’s Perceptron (1962) for training simple neural
networks
 Herbert Gelemter (1959) constructed the Geometry Theorem Prover
(a package implemented in Maple for manipulating and proving
geometric theorems.)

25
 The History of Artificial Intelligence

• Work in the sixties at MIT lead by Marvin Minsky and John

McCarthy
- Development of LISP (1958) symbolic programming language
- SAINT (1963a): solved freshman calculus problems
- ANALOGY: Tom Evans's ANALOGY program (1968) solved IQ
test analogy problems
- SIR: Bertram Raphael's (1968) answered simple questions in English
- STUDENT: Daniel Bobrow's STUDENT program (1967) solved
algebra story problems
- SHRDLU: Obeyed simple English commands in the blocks world
26
 The History of Artificial Intelligence

• Early Limitations
– Hard to scale solutions to toy problems to more realistic ones due to
difficulty of formalizing knowledge and combinatorial explosion of
search space of potential solutions.
– Limitations of Perceptron demonstrated by Minsky and Papert
(1969).

27
 Knowledge is Power: Expert Systems

 Knowledge is Power: Expert Systems

 Discovery that detailed knowledge of the specific domain
can help control search and lead to expert level
performance for restricted tasks.
 First expert system DENDRAL for interpreting mass
spectrogram data to determine molecular structure by
Buchanan, Feigenbaum, and Lederberg (1969).

28
 Knowledge is Power: Expert Systems

• Early expert systems developed for other tasks:

– MYCIN: diagnosis of bacterial infection (1975)
– PROSPECTOR: Found molybendum deposit based on geological
data (1979)
– R1: Configure computers for DEC (1982)

29
 AI becomes an Industry

 Development of numerous expert systems in early eighties

 Japanese started “Fifth Generation” project in 1981 to
build intelligent computers based on Prolog logic
programming.
 MCC established in Austin in 1984 to counter Japanese
project.

30
 AI becomes an Industry
 Limitations become apparent, prediction of AI Winter
– Brittleness and domain specificity
– Knowledge acquisition bottleneck

31
 Rebirth of Neural Networks

• New algorithms discovered for training more complex

neural networks (1986)
• Cognitive modelling of many psychological processes
using neural networks, e.g. learning language

32
 Rebirth of Neural Networks

 Industrial applications:
– Character and hand-writing recognition
– Speech recognition
– Processing credit card applications
– Financial prediction
– Chemical process control

33
 Recent events (1987-present)

 General focus on learning and training methods to address

knowledge-acquisition bottleneck
 Shift of focus from rule-based and logical methods to
probabilistic and statistical methods (e.g. Bayes nets,
Hidden Markov Models)

34
 Recent events (1987-present)

 Increased interest in particular tasks and applications

– Data mining
– Intelligent agents and Internet applications (softbots, believable
agents, intelligent information access)
– Scheduling/configuration applications (Successful companies: I2,
Red Pepper, Trilogy)

35
Cutting Edge Applications of AI

36
Applications of AI in different
business areas

37
 Construction

 AI-based applications have been

widely being used in the construction
sectors.
 The AI-based application will make
the engineers more productive and
capable of delivering high-quality
work in the stipulated time frame.

38
 Agriculture
 Agriculture is one of the core sectors
and we have been modifying the
cultivation process to yield more from
it.
 There are incredible opportunities for
AI or machine learning in agriculture.
 The technologies like AI & IoT will be
very useful in understanding a timely
planting, getting predictions, using
fertilizers, harvesting and the climate.
39
 Sports

 The implementation of artificial intelligence

in the sports industry is a game changer.
 There has been a huge demand for AI-based
applications in the sports industry as it
possesses significant capabilities.
 The implementation of the technology is
certainly going to solve many major
changes in the sports world and it could
bring the true competition among the
players and athletes.

40
 Entertainment

 AI or machine learning has brought a

big change in the entertainment
industry.
 AI is everywhere and it is making a
big difference in our lives.
 When it comes to the entertainment,
the algorithms being used by various
application make our life much
simpler.
 AI has really changed the
entertainment industry and it will
make it more lively in the days to
come.

41
 Life On Other Planets
 NASA is already using AI to
look for life on other planets,
which will be the key for “
Mars 2020,” the mission where
the Red Planet will be explored
more thoroughly.

 The devices they’ll send, better

known as rovers, will be able to
explore Mars’ terrain in more
detail and reveal the properties
of the planet’s elements to
determine the possibility of life
with more certainty.

42
 Ways Global Defense Forces Use AI
 Military drones for surveillance
 Robot soldiers for combat
 Intelligent systems for
awareness
 Secure web-portals for
cybersecurity

43
 State of the art
• Deep Blue defeated the reigning world chess champion
Garry Kasparov in 1997
• During the 1991 Gulf War, US forces deployed an AI
logistics planning and scheduling program that involved up
to 50,000 vehicles, cargo, and people
• NASA's on-board autonomous planning program
controlled the scheduling of operations for a spacecraft
• Proverb solves crossword puzzles better than most humans

44
 Natural Language Processing

 Speech technologies
– Automatic speech recognition (ASR)
– Text-to-speech synthesis (TTS)
– Dialog systems
 Language Processing Technologies
– Machine Translation
– Information Extraction
– Information Retrieval
– Text classification, Spam filtering.
45
Robotics

46
 Others..
 Computer Vision:
– Object and Character Recognition
– Image Classification
– Scenario Reconstruction etc.
 Game-Playing
– Strategy/FPS games, Deep Blue etc.
 Logic-based programs
– Proving theorems
– Reasoning etc.

47
 Homework from first lecture
 Write a report using about 500 words in English to describe one of
the key persons who made a great contribution to the AI world.
 You may choose one from those introduced in this lecture, or find
someone in the internet.
 When you refer to any information taken from a paper, a report, a
web-site, or any published material, please add a reference and cite it
in the correct places in your report.
 Add your name, student ID, and date below the title of your report,
create a pdf-file, and submit within one week.

48
 Research Paper Discussion
• Title of paper
• Authors (first author, other contributing
authors),
• Affiliations
• Email addresses

49
 Research Paper Discussion
• Abstract
• First sentence (s) should be from the existing domain.
• Second sentence (s) should discuss the applications in which you are
working
• Third sentence (s) should discuss the challenges you are addressing
• Fourth sentence (s) should be discussing the proposed model and its
techniques.
• Fifth sentence (s) should mention the list of achievements while
highlighting the trade-offs.

50
 Research Paper Discussion
• Introduction
• First paragraph should elaborate the domain knowledge,
• Second paragraph should discuss about its application details
• Third paragraph should have the details of existing challenges and
which challenge you want to address and how?
• Fourth paragraph should describe the list of contributions of your
work

51
 Research Paper Discussion
• Related work\ literature review
• Recent plus previous papers (recent means from current and
previous two years, 50% papers should be from recent years)
• Critical analysis should be included in a table

52
 Research Paper Discussion
• Problem Statement
• Existing challenges from the base paper while citing them in the
problem statement.
• It should be clearly written with proper justification.
• You can also include figures for highlighting the existing challenges

53
 Research Paper Discussion
• Proposed methodology
• Proposed method should be clearly elaborated by elaborating its all
components and techniques
• Add a diagram for its visual representation.
• Include mathematical formulation

54
 Research Paper Discussion
• Results and discussions
• Mention the tool used for simulation or the real testbeds
description
• Mention list of parameters
• Add results details with figures and tables.
• Add proper captions for figures and table.
• Highlight the tradeoffs of the results and discussion.

55
 Research Paper Discussion
• Conclusion
• Add the sentence for proposed work implementation in
your domain.
• Mention the achievement for the aforementioned list of
parameters in figures or percentage.
• Add the trade-off in figures or percentage.

56
Questions

57