Introduction to

Artificial Intelligence
Course Overview

Sumaira Saeed Spring' 2024 1

Class Details
• Meetings
• Monday/Wednesday 2:30 PM – 3:45 PM
• MCS-3 Aman CED Building
• Instructor
• Sumaira Saeed
• [email protected]
• Counseling Hours
• Tues/Thurs 4:00pm to 5:00 pm (Appointment via Email before coming)
• Course Wiki
• Available on LMS

Sumaira Saeed Spring' 2024 2

Pre - requisites
• Data Structures and Algorithms
• Logic and Discrete Structures

Sumaira Saeed Spring' 2024 3

Grading Scheme (tentative)
• Assignments – 15%
• Class Participation – 5%
• Exams - 60%
• Quizzes (n-1) – 10%
• Term Exam – 25%
• Final Exam – 30%
• Term Report / Project – 15%

Sumaira Saeed Spring' 2024 4

Assignments
• Written Assignment
• Programming Assignment

• Submission
• Electronic Submission on LMS
• Timely submission is important

• Collaborations
• No collaborations on homework assignments, unless group projects

Sumaira Saeed Spring' 2024 5

Quizzes
• Minimum four quizzes
• N-1 Policy

Sumaira Saeed Spring' 2024 6

Skills
• Programming
• Python (OR any other programming language)
• Prolog (http://www.swi-prolog.org/)
• Tools
• KNIME (http://www.knime.org/)
• GeNIe (http://genie.sis.pitt.edu/)

Sumaira Saeed Spring' 2024 7

Code of Conduct
• Strict on Attendance (Email one day before class for Emergencies)
• No Cell phones (attend Urgent calls outside class)
• Class Participation
• Carry Calculators every day
• CR

Sumaira Saeed Spring' 2024 8

Reading
• Tim Jones. Artificial Intelligence: A Systems Approach
• Russell and Norvig. Artificial Intelligence: A Modern Approach
• R.B. Mishra. Artificial Intelligence
• Prateek Joshi. Artificial Intelligence with Python
• Wolfgang Ertel. Introduction to Artificial Intelligence

Sumaira Saeed Spring' 2024 9

Outline
• Applications of AI
• What is Artificial Intelligence
• Strong AI Vs Weak AI

Sumaira Saeed Spring' 2024 10

AI – In Perspective
Application areas

Sumaira Saeed Spring' 2024 11

Applications of AI: AI is Everywhere!

Sumaira Saeed Spring' 2024 12

Natural language processing
• Text generation: ChatGPT

• Spam filtering

• Information retrieval:
• rank web pages based on relevance to query

• Machine translation:
• Google Translate handles more than 100 languages

Sumaira Saeed Spring' 2024 13

Face Recognition
• Unlock phones
• Counting in Crowd
• MetaVerse

Sumaira Saeed Spring' 2024 14

Recommendation Systems AND Customer Service
Representatives
Live Chats

Users who
Automated
bought ‘A’ ,
Email
also Recommendation Responses
baught ‘B’
Systems
AND
Customer
Service
“You May Representatives Suggested
Also Like” Friends

Similar
Products

Sumaira Saeed Spring' 2024 15

Virtual Assistants

Sumaira Saeed Spring' 2024 16

Transportation/ Aviation
• Transportation:
• Fleet management
• Logistic Planning
• Route Planning

• Technological Improvement
• Self driving cars
• Automated Traffic Signals
• Break assist/ collision avoidance, ABS

Sumaira Saeed Spring' 2024 17

Google Duplex
• Google Duplex, an upgrade of Google Assistant that can understand
context.

• On November 21, 2019, Google announced that Google Duplex can

also purchase a movie ticket for you on the web!

• in almost all states across the USA, and you can use the updated
Google Assistant
• to make reservations at a restaurant,
• an appointment at a salon,
• and also buy movie tickets.

Sumaira Saeed Spring' 2024 18

Industry/ Robotics
• Assembly line automation
• Warehouse automation
• Amazon using 45,000 Kiva robots in their warehouses.

• Resource optimization
• Robotics
• Firefighting
• Search and rescue
• Disaster management
• Robotic surgery

Sumaira Saeed Spring' 2024 19

 Medical and Health
• BioMind:
• Chinese AI beats 15 doctors in
tumor diagnosis competition
(July 2018)

Sumaira Saeed Spring' 2024 20

Fraud Detection/Prevention
• Money Laundering
• Loan Defaulter
• Identity Protection
• Fingerprint Detection
• Face Recognition

Sumaira Saeed Spring' 2024 21

 Spring' 2024

Toys and Games

• Board Games
• Chess,
• Go,
• Checkers,
• Poker

• Modern Video Games:

• Path finding,
• Strategy design,
• Navigation

Sumaira Saeed 22
 Spring' 2024

Image / Video Generation

• Midjourney
• generates images from natural language descriptions

• Synthesia:
• a web-based platform for creating videos with AI avatars

Sumaira Saeed 23
What is artificial intelligence
Strong Vs Weak AI

Sumaira Saeed Spring' 2024 24

What is Intelligence?
• Intelligence is a set of properties of the mind – ability:
• To plan
• Solve problems
• Reason (in general)

• Intelligence is the ability to make right decision given a set of inputs

and a variety of possible actions

• Can it be applied to animals?

• Human embody many aspects of intelligence; animals have few characteristics
and at much lower level

Sumaira Saeed Spring' 2024 25

Sumaira Saeed Spring' 2024 26
Strong AI
• Create a computer that can literally think and is conscious in the
same way that a human is conscious
• E.g: Understand expressions and emotions, make new discoveries,
decision-making in critical scenarios etc.

• To many philosophers and AI researchers this view is false, and

even absurd

Sumaira Saeed Spring' 2024 27

Weak AI
• Represents the wider domain of AI technologies

• Weak AI features can be added to systems to give them intelligent

qualities

• A view that intelligent behavior can be modeled and used by

computers to solve complex problems

• The focus of the course would be Weak AI

Sumaira Saeed Spring' 2024 28
Summary
• Intelligence is difficult to define, and as a result Artificial
Intelligence is also hard to define.

• One definition of Artificial Intelligence is:

Artificial intelligence is the study of systems that act in a way that to any
observer would appear to be intelligent.

• We are looking into the practical aspects of AI that focuses on

getting inspirations from natural processes to solve real-life
complex problems.
Sumaira Saeed Spring' 2024 29
 Introduction to
Artificial Intelligence
Unit 2

Sumaira Saeed Spring' 2023 30

1950s – The Birth of AI
• Idea of intelligent machine was based on the belief that the machine would
be intelligent if it could do something that people do (and perhaps find
difficult).

• Proving Equations
• Playing complex games such as checkers and chess

Sumaira Saeed Spring' 2023 31

1950s – The Birth of AI
• Attempt to define a standard for a machine to be called “intelligent”
• 1950: Alan Turing publishes "Computing Machinery and Intelligence.“
• The Turing test
Computer would need capability of:
1. natural language processing to enable it to communicate successfully in English
2. knowledge representation to store what it knows or hears
3. automated reasoning to use the stored information to answer questions and to draw new
conclusions
4. machine learning to adapt to new circumstances and to detect and extrapolate patterns
5. computer vision to perceive objects, and
6. robotics to manipulate objects and move about
Most of AI is composed of the above six disciplines

Sumaira Saeed Spring' 2023 32

Overwhelming Optimism

“I believe that in about fifty years’ time it will be possible to

program computers…to make them play the imitation game so
well that an average interrogator will not have more than 70
percent chance of making the right identification after five
minutes of questioning.”

Alan Turing, Computing machinery and intelligence, 1950.

Sumaira Saeed Spring' 2023 33

1950s – The Birth of AI
• 1956: John McCarthy coins the term, "Artificial Intelligence" at a Dartmouth computer conference.
• 1958: John McCarthy invents the Lisp language, an AI programming language (MIT).
• 1962: First compiler implemented in 1962 at MIT for IBM 704
• FOCUS ON STRONG AI

Sumaira Saeed Spring' 2023 34

Early Enthusiasm (1952 – 1969)
• ELIZA
• Written at MIT by Joseph
Weizenbaum in 1966
• Program that parodied a
psychologist
• Used simple pattern matching
techniques, considered as first
chatterbots
• https://www.eclecticenergies.com/
ego/eliza

Sumaira Saeed Spring' 2023 35

Perceptron and AI’s Winter (1966 – 1973)
• Perceptron Algorithm
• Invented by Frank Rosenblatt at Cornell Aeronautical Laboratory in 1957
• Intended to be a machine rather than a program
• Implemented in custom-built hardware “Mark 1 Perceptron” designed for
image recognition
• An IBM 704 –
• a 5-ton computer the size of a room – was fed a series of punch cards.
• computer taught itself to distinguish cards marked on the left from cards
marked on the right.

Sumaira Saeed Spring' 2023 36

Rosenblatt wrote in 1958.

“Yet we are about to witness the birth of such a machine – a machine capable of
perceiving, recognizing and identifying its surroundings without any human training
or control.”

He was right – but it took half a century to prove it.

Sumaira Saeed Spring' 2023 37

1970s – The Fall of AI – AI Winter
• 1969: Excitement of neural networks came to an end in 1969 with the
publication of the mongraph titled “Perceptrons.” by Marvin Minsky and Seymour
Paper
• Demonstrated that single layer perceptron are limited
• Could not be trained to recognized many classes of patterns

• The 1970s represented the fall of AI after an inability to meet irrational

expectations.

• End of fundings
• Hardware built for AI also suffered a lost of interest

• The First AI winter 1974-1980

Sumaira Saeed Spring' 2023 38

1980 - 1987 – AI Rises Again
• Started mid to late 1980s
• Focus on specific goals in the weak aspect of AI
• Incorporates many new approaches, for example, biologically
inspired approaches
• Transition from AI algorithms and methods to standard algorithms
that are practically useful
• Use of genetic algorithms in: optimization, scheduling, modeling
• Artificial Immune System
• Inspiration from human immune system
• Used in domains of optimization, pattern recognition and data mining

Sumaira Saeed Spring' 2023 39

1980 - 1987 – AI Rises Again (Con’t)
• Advancement in probabilistic reasoning, machine learning, intelligence
agents, and computational intelligence systems.
• Expert Systems
• Knowledge Representation
• Return of Funding

Sumaira Saeed Spring' 2023 40

1988 - 1993 – Second AI Winters
• The first indication of a change in weather was the sudden collapse of the market for
specialized AI hardware in 1987.

• Desktop computers from Apple and IBM had been steadily gaining speed and power

• In 1987 they became more powerful than the more expensive Lisp machines made
by Symbolics and others.

• An entire industry worth half a billion dollars was demolished overnight.

• In 1984, John McCarthy criticized expert systems because they lacked common
sense and knowledge about their own limitations
• Unreal Expectations

Sumaira Saeed Spring' 2023 41

Intelligence in machines vs. humans
What has happened over the last 40-50 years - to the
disappointment of all those who made breathless predictions
about where AI would go –
• playing chess have turned out to be incredibly easy for computers,
• whereas learning to walk and learning to get around in the world
without falling over has proved to be unbelievably difficult.

Sumaira Saeed Spring' 2023 42

 MILESTONES

Sumaira Saeed Spring' 2023 43

IBM computer Deep Blue beats world champion Garry Kasparov in
chess match.

Sumaira Saeed Spring' 2023 44

IBM WATSON wins Jeopard

Sumaira Saeed Spring' 2023 45

Sumaira Saeed Spring' 2023 46
AI beats humans in Poker! (Jan 2017)

Sumaira Saeed Spring' 2023 47

Google uses deep learning to beat top
human player at GO (May 2017)

Sumaira Saeed Spring' 2023 48

Sept 2016
• AI takes first step towards creating Art!

Sumaira Saeed Spring' 2023 49

Crashes!
• March 2018 Uber self driving vehicle
• The death of Elaine Herzberg (August 2, 1968 – March 18, 2018) was the first recorded case
of a pedestrian fatality involving a self-driving car

• April 2018 Tesla auto pilot incident

• Nobody was hurt in the January 2018 crash.

Sumaira Saeed Spring' 2023 50

History of AI

Sumaira Saeed Spring' 2023 51

 AI Competitions

Sumaira Saeed Spring' 2023 52

 The Dream
“By mid-21st century, a team of fully
autonomous humanoid robot soccer players
shall win the soccer game, comply with the
official rule of the FIFA, against the winner of
the most recent World Cup.”
• Karachi Koalas
(https://cs.iba.edu.pk/FCS_Pakistan_Robo_
Cup_2013_PR.html)

• NUST SMME RISE Lab

Sumaira Saeed Spring' 2023 53
More AI competations
• DARPA Challenges
• The DARPA Robotics Challenge (started in
2012)
• The Robotics Challenge will focus on disaster
or emergency-response scenarios

Sumaira Saeed Spring' 2023 54

More AI competitions
• Angry Birds competition
• The long term goal is to build AI agents that can play new levels better
than the best human players
• http://aibirds.org/ (IJCAI 2018)

Sumaira Saeed Spring' 2023 55

More AI competitions
• Pacific Asia Conference on Knowledge Discovery and Data Mining
(PAKDD)
• 2016 – April 19 – 22, Auckland, New Zealand
• “Sarcasm detection on Reddit comments”
• Task of designing an effective algorithm for sarcasm detection in the
domain of opinion mining
• Google AI Challenge
• https://ai.google/social-good
• ….
• And many more
Sumaira Saeed Spring' 2023 56
Suggested Readings
• Chapter 1: Tim Jones. Artificial Intelligence: A Systems Approach

Sumaira Saeed Spring' 2023 57

 Introduction to
Artificial Intelligence
Unit 3

Sumaira Saeed Spring' 2020 58

Outline
• Problem Solving as Search
• Trees, Graphs, and Representation
• General State Space Search
• Create General Search Space for Games

Sumaira Saeed Spring' 2020 59

 Problem Solving as Search

Sumaira Saeed Spring' 2020 60

Problem Solving as Search
• Search is the process of looking for a sequence of actions that reaches the
goal.
• A search algorithm takes a problem as input and returns a solution in the
form of an action sequence.

Sumaira Saeed Spring' 2020 61

 Trees, Graphs, and Representation

Sumaira Saeed Spring' 2020 62

Trees, Graphs, and Representation
• Graph
• Finite set of vertices (nodes) that
are connected by edges (arcs)
• Cycle
• An arc leading back to the
original node
• Undirected – arcs do not imply
a direction (fig 1)
• Directed – direction is implicit
in arc (fig 2)

Sumaira Saeed Spring' 2020 63

Trees, Graphs, and Representation
• Weighted Graph
• Arc may carry a weight implying
cost is associated with a path
• Connected Graph
• Every pair of nodes is connected
by an arc
• Complete Graph
• Every node is connected to every
another node
• Tree
• Connected graph with no cycles

Sumaira Saeed Spring' 2020 64

Trees, Graphs, and Representation

Sumaira Saeed Spring' 2020 65

Trees, Graphs, and Representation
• Adjacency Matrix
• One of the simplest ways of representing a graph
• N by N matrix,
• N = no of nodes
• Element of matrix defines connectivity between node referenced as the
row and node referenced as the column
• Undirected graph – symmetric matrix
• Directed graph – non symmetric matrix
• Rows identify source nodes whereas columns identify destinations

Sumaira Saeed Spring' 2020 66

 General State Space Search

Sumaira Saeed Spring' 2020 67

A Well – defined Problem
• Problem can be defined formally by five
components
1. The initial state
2. A description of the possible actions
available
3. Transition Model – description of what each
action does
4. Goal test – whether a given state is a goal
state
5. Path cost function that assigns a numeric
cost to each path

Sumaira Saeed Spring' 2020 68

A Well – defined Problem
• Initial state, actions, and transition model implicitly define the
state space of the problem
• State Space
• Set of all states reachable from the initial state by any sequence of
actions
• Solution to the problem is a sequence of actions that lead from
the initial state to a goal state
• Quality of Solution – measured by cost function
• Optimal solution has the lowest path cost among all solutions

Sumaira Saeed Spring' 2020 69

General State Space Search
• As in general problem solving
• Some paths lead to dead – ends and some lead to solution
• Possibility of multiple solutions
• One solution may be better than the other

• Avoiding the dead ends and selecting the best solution is the result
of the chosen search strategy

Sumaira Saeed Spring' 2020 70

General State Space Search
• Search in a Physical Space
• The figure represent a search
problem in a physical space
• State/Place – letters
• Action/Operator – a legal move
between one place and another
• Initial position – A
• Possible actions
• A→B
• A→C
• A→D

Sumaira Saeed Spring' 2020 71

General State Space Search
• Search space reduced to a tree
structure
• Node – physical location
• Arc – legal move between places
• Depth of tree is distance from
the initial position

Sumaira Saeed Spring' 2020 72

General State Space Search
Maze Traversing

Sumaira Saeed Spring' 2020 73

Sumaira Saeed Spring' 2020 74
 Lets play some games!

Sumaira Saeed Spring' 2020 75

The Towers of Hanoi
• We have three pegs and a number of disks of different sizes.
• The aim is to move from the starting state where all the disks are
on the first peg, in size order (smallest at the top) to the goal state
where all the pegs are on the third peg, also in size order.
• We are allowed to move one disk at a time, as long as there are no
disks on top of it, and as long as we do not move it on top of a peg
that is smaller than it.

Sumaira Saeed Spring' 2020 76

General State Space Search

• http://www.coolmath-games.com/0-tower-of-hanoi
Sumaira Saeed Spring' 2020 77
The Towers of Hanoi (Partial Search Tree)

Sumaira Saeed Spring' 2020 78

General State Space Search
• Missionaries and Cannibals
• Three missionaries and three
cannibals are on one side of a river,
with a canoe.
• They all want to get to the other side
of the river.
• The canoe can only hold one or two
people at a time.
• At no time should there be more
cannibals than missionaries on either
side of the river, as this would
probably result in the missionaries
being eaten

Sumaira Saeed Spring' 2020 79

General State Space Search
• Missionaries and Cannibals
• Legal Moves / Operators
• Move one cannibal to the other side
• Move two cannibals to the other side
• Move one missionary to the other side
• Move two missionaries to the other side
• Move one cannibal and one missionary to the other side

Sumaira Saeed Spring' 2020 80

 Create General State Space Search

Sumaira Saeed Spring' 2020 81

General State Space Search

Sumaira Saeed Spring' 2020 82

Eight Queens

Sumaira Saeed Spring' 2020 83

• https://www.coolmath-games.com/0-eight-queens
 Create General State Space Search for 4 Queens

Sumaira Saeed Spring' 2020 84

Four Queens Example

Sumaira Saeed Spring' 2020 85

Measuring problem-solving performance
• Criteria that might be used to choose among algorithms.
• We can evaluate an algorithm’s performance in four ways:
• COMPLETENESS • Completeness: Is the algorithm guaranteed to find a
solution when there is one?
• OPTIMALITY • Optimality: Does the strategy find the optimal solution?
• TIME COMPLEXITY • Time complexity: How long does it take to find a
solution?
• SPACE COMPLEXITY • Space complexity: How much memory is needed
to perform the search?

Sumaira Saeed Spring' 2020 86

 Try Practice Questions

Sumaira Saeed Spring' 2020 87

The Frog Problem

Sumaira Saeed Spring' 2020 88

Sumaira Saeed Spring' 2020 89
What about Rubik’s Cube?
• Can it be represented as a graph?
• If yes then:
• What should the vertices be representing?
• What should be the arcs representing?

Sumaira Saeed Spring' 2020 90

 Introduction to
Artificial Intelligence
Unit 4
Acknowledgment
• Some of the slides of this lecture have been taken from:
• The lecture slides of CS188 – “Introduction to Artificial Intelligence” UC
Berkeley

3/7/2024 Spring' 2024 92

Outline
• Uninformed Search Strategies
• Breadth First Search
• Depth First Search
• Depth Limited Search
• Iterative Deepening Search

3/7/2024 Spring' 2024 93

BFS – Explained
• In BFS, graph is searched from the root node in order of the
distance from the root

• BFS checks each node nearest the root before descending to the
next level

• All the nodes are expanded at a given depth in the search tree
before any nodes at the next level are expanded

3/7/2024 Spring' 2024 94

 BFS - Explained
• BFS implementation uses a FIFO (first-in first-out)
queue

• New nodes go back of the queue, old ones expanded

first

• Goal test is applied to each node when it is generated

rather than when it is selected for expansion

• The algorithm discards any new path to a state already

in the frontier or explored set
• It is easy to see that any such path must be at least as deep
as the one already found. Thus, breadth-first search always
has the shallowest path to every node on the frontier.

3/7/2024 Spring' 2024 95

Breadth-First Search
Strategy: expand a G
a shallowest b c
node first e
d f
Implementation: S h
Fringe is a FIFO p r
queue q

d e p
Search
b c e h r q
Tiers
a a h r p q f

p q f q c G

q c G a
3/7/2024 Spring' 2024 96
a
BFS - Comments
d is the
• Complete? depth

• Yes
• Optimal?
• Yes, in a non-weighted graph
• Time Complexity
• O(bd)
• Space Complexity
• O(bd), d is depth of the solution, not the depth of the tree

3/7/2024 Spring' 2024 97

BFS – Comments
• Useful when
• Space is not a problem
• You want to find the solution containing the fewest arcs (Does not
consider cost of the solution)
• Few solutions may exist, and at least one has a short path length
• Infinite paths may exist – explores all of the search space, even with
infinite paths
• Poor method when
• All solutions have a long path length
• There is some heuristic knowledge available

3/7/2024 Spring' 2024 98

DFS – Explained
• Searching begins at the root node

• Exhaustively searches each branch to its greatest depth before

backtracking to previously unexplored branches

• The implementation of DFS uses a LIFO (last-in first-out) queue /

Stack

3/7/2024 Spring' 2024 99

Depth-First Search
Strategy: expand a a G
deepest node first b c

Implementation: Fringe e
d f
is a LIFO stack S h
p q r

d e p

b c e h r q

a a h r p q f

p q f q c G

q c G a
3/7/2024 Spring' 2024 100
a
DFS – Comments
• Complete?
• No, in case of loops the algorithm cycles indefinitely
• Optimal?
• No
• Space Complexity
• O(bd)
• Time Complexity d is the
• O(bd) depth

3/7/2024 Spring' 2024 101

DFS – Comments
• Appropriate when
• Space is restricted
• Many solutions exist, perhaps with long path lengths, particularly for the
case where nearly all paths lead to a solution
• The order of the neighbors of a node are added to the stack can be tuned
so that solutions are found on the first try (A smart or Lucky Tie breaker)
• Not suitable when
• It is possible to get caught in infinite paths – when the graph is infinite or
when there are cycles in the graph
• Solutions exist at shallow depth – the search may look at many long paths
before finding the short solutions

3/7/2024 Spring' 2024 102

 BFS and DFS Algorithms
BFS DFS
• Let L be a list containing the • Let L be a list containing the
initial state initial state
• Loop
• If L is empty return failure • Loop
• Node  remove-first(L) – If L is empty return failure
• If Node is a goal – Node  remove-first(L)
• Return the path from initial – If Node is a goal
state of Node
• Else • Return the path from initial state of
• Generate all successors of Node
Node – Else
• Add generated nodes to the
back of L • Generate all successors of Node
• End Loop • Add generated nodes to the front
of L
• End Loop
3/7/2024 Spring' 2024 103
Activity
• Start Node – A
• Goal – G
• Path taken for DFS, BFS and
nodes explored
• Status of queue/stack on every
move

3/7/2024 Spring' 2024 104

Depth First vs. Breadth First

3/7/2024 Spring' 2024 105

Variants of DFS
• Depth – Limited Search (DLS)
• Iterative Deepening Search (IDS)

3/7/2024 Spring' 2024 106

Depth – Limited Search
• Modification of DFS

• Depth is provided below which the algorithm would not go

• Nodes below the depth are omitted from the search

• Keeps algorithm from indefinite cycling

• Reduces search scope

3/7/2024 Spring' 2024 107

Depth – Limited Search
• Complete?
• No, if goal node is below the depth
• Yes, depth of search is same as the depth of the tree (which is finite)
• Optimal?
• No, first path to the goal is found instead of the shortest
• Space Complexity
• Same as DFS, O(bd)
• Time Complexity
• Same as DFS, O(bd)
• d is the imposed depth

3/7/2024 Spring' 2024 108

Iterative Deepening Search (IDS)
• Derivate of DLS that combines features of depth-first search with
breadth-first search

• Performs DLS search with increased depth till the goal is found

• Depth begins at one, and keep increasing till either goal is found or no
further nodes can be expanded

• Not susceptible to cycles (feature from DLS)

• Finds goal nearest to the root node (feature of BFS)

3/7/2024 Spring' 2024 109

Iterative Deepening Search
• Complete?
• Yes
• Optimal?
• Yes
• Space Complexity
• O(bd)
• Time Complexity
• O(bd)

3/7/2024 Spring' 2024 110

Exercise
• Find path from S to G
• For each DFS, DLS (d=3), IDS,
BFS
• Search tree constructed during
the search
• List of nodes in the order in
which they were expanded
• State of the Queue when the
node was expanded

3/7/2024 Spring' 2024 111

Solution to DFS

3/7/2024 Spring' 2024 112

Depth – Limited Search

3/7/2024 Spring' 2024 113

Homework
• Solutions for BFS and IDS are left as an exercise.

3/7/2024 Spring' 2024 114

 Introduction to
Artificial Intelligence
Unit 5
Outline
• Uninformed Search
• Uniform Cost Search
• Informed Search
• Introduction
• Heuristics
• Best First Greedy Search
• A* Search

3/7/2024 Spring' 2024 116

Acknowledgment
• Some of the slides of this lecture have been taken from:
• The lecture slides of CS188 – “Introduction to Artificial Intelligence” UC
Berkeley

3/7/2024 Spring' 2024 117

Uniform Cost Search UCS
• Uniform-cost search is a searching algorithm used for traversing a
weighted tree or graph. This algorithm comes into play when a different cost
is available for each edge. The primary goal of the uniform-cost search is to
find a path to the goal node which has the lowest cumulative cost.

• UCS can be applied to find the least-cost path through a graph by maintaining
an ordered list / priority list of nodes in order of descending cost.

• Uniform-cost search is an uninformed search method because no heuristic is

actually used.

• The algorithm measures the actual cost of the path without attempting to
estimate it.

3/7/2024 Spring' 2024 118

Uniform Cost Search

3/7/2024 Spring' 2024 119

3/7/2024 Spring' 2024 120
3/7/2024 Spring' 2024 121
3/7/2024 Spring' 2024 122
3/7/2024 Spring' 2024 123
3/7/2024 Spring' 2024 124
3/7/2024 Spring' 2024 125
3/7/2024 Spring' 2024 126
3/7/2024 Spring' 2024 127
3/7/2024 Spring' 2024 128
3/7/2024 Spring' 2024 129
3/7/2024 Spring' 2024 130
Example Graph

3/7/2024 Spring' 2024 131

 UCS Algorithm

3/7/2024 Spring' 2024 132

UCS Example

• It is important to note in this example that the USC does not

stop until the goal node is the top most node of the list.
Which means that we have found the optimal path.

3/7/2024 Spring' 2024 133

UCS Properties
• What nodes does UCS expand?
• Processes all nodes with cost less than cheapest solution!
• If that solution costs C* and arcs cost at least  , then the
“effective depth” is roughly C*/
• Takes time O(bC*/) (exponential in effective depth)

• How much space does the fringe take?

• Has roughly the last tier, so O(bC*/)

• Is it complete?
• Assuming best solution has a finite cost and minimum arc
cost is positive, yes!

• Is it optimal?
3/7/2024
• Yes! Spring' 2024 134
Romania, use UCS

3/7/2024 Spring' 2024 135

 Informed Searches

3/7/2024 Spring' 2024 136

Informed vs. Uniformed Search
• A search method or heuristic is informed if it uses additional
information about nodes that have not yet been explored to
decide which nodes to examine next.

• If a method is not informed, it is uninformed or blind.

• The more informed a search method is, the more efficiently it will
search.

3/7/2024 Spring' 2024 137

The 8-puzzle
• The puzzle consists of a 3 x 3 grid, with the
numbers 1 through 8 on tiles within the grid and
one blank square.
• Tiles can be slid about within the grid, but a tile can
only be moved into the empty square if it is
adjacent to the empty square.
• The start state of the puzzle is a random
configuration, and the goal state is as shown in the
picture below.

Goal State

3/7/2024 Spring' 2024 138

Heuristic Function
• Heuristic is an “estimate of the proximity of the goal”.

• A heuristic is a rule of thumb that may help solve a given problem. Heuristics
take problem knowledge into consideration to help guide the search within
the domain.

• A heuristic evaluation function is a function that when applied to a node

gives a value that represents a good estimate of the distance of the node
from the goal.

• For two nodes m and n, and a heuristic function f, if f(m) < f(n), then it should
be the case that m is more likely to be on an optimal path to the goal node
than n.

3/7/2024 Spring' 2024 139

Heuristics
( Source: Wikipedia)
• Heuristic refers to experience-based techniques for problem
solving, learning, and discovery.

• Heuristic methods are used to speed up the process of finding

a good enough solution, where an exhaustive search is
impractical. Examples of this method include using a "rule of
thumb", an educated guess, an intuitive judgment, or common
sense.

3/7/2024 Spring' 2024 140

Basic Idea Behind a Heuristic Search
• We assume that we have a heuristic (evaluation) function, f, to
help decide which node is the best one to expand next. This
function is based on information specific to the problem domain.

• Expand next node, n, having the smallest value of f(n). Resolve ties
arbitrarily.

• Terminate when the node to be expanded next is a goal node.

3/7/2024 Spring' 2024 141

Heuristic # 1
• The first heuristic we consider is to count
how many tiles are in the wrong place.
We will call this heuristic, h1(node).

• In the case of the first state shown in

Figure, h1 (node) = 8 because all the tiles
are in the wrong place.

• However, this is misleading because we

could imagine a state with a heuristic
value of 8 but where each tile could be
moved to its correct place in one move.

3/7/2024 Spring' 2024 142

Heuristic # 2
• An improved heuristic, h2, takes into account how
far each tile had to move to get to its correct state.
• This is achieved by summing the Manhattan
distances of each tile from its correct position.
• Manhattan distance is the sum of the horizontal
and vertical moves that need to be made to get
from one position to another, named after the grid
system of roads used in Manhattan.
• h2 (node) = 2 + 2 + 2 + 2 + 3 + 3 + 1 + 3 = 18
• It is worth noting that h2 (node) ≥ h1 (node) for any
node. This means that h2 dominates h1, implying
that a search method using heuristic h2 will
always perform more efficiently than the same
search method using h1.

3/7/2024 Spring' 2024 143

Best-first search
• Best-first search expands the node that appears to be closest to
goal
• Evaluation function f(n) = h(n) (heuristic)
= estimate of cost from n to goal

3/7/2024 Spring' 2024 144

Best-first Search
(Using Heuristic # 1)

Goal State

3/7/2024 Spring' 2024 145

Best-first Search
(Using Heuristic # 2)
2 8 3
1 6 4
7 5
Goal State

• Expand the tree

3/7/2024 Spring' 2024 146

 Romania with step costs in km (Russell & Norvig)

3/7/2024 Spring' 2024 147

Best-first search example

3/7/2024 Spring' 2024 148

Best-first search example

3/7/2024 Spring' 2024 149

Best-first search example

3/7/2024 Spring' 2024 150

Best-first search example

3/7/2024 Spring' 2024 151

A search
*

• Idea: avoid expanding paths that are already expensive

• Evaluation function f(n) = g(n) + h(n)
• g(n) = cost so far to reach n
• h(n) = estimated cost from n to goal
• f(n) = estimated total cost of path through n to goal

3/7/2024 Spring' 2024 152

Romania with step costs in km
(Russell & Norvig)

3/7/2024 Spring' 2024 153

 A* search example

3/7/2024 Spring' 2024 154

 A* search example

3/7/2024 Spring' 2024 155

A search example
*

3/7/2024 Spring' 2024 156

 A* search example

3/7/2024 Spring' 2024 157

 A* search example

3/7/2024 Spring' 2024 158

 A* search example

3/7/2024 Spring' 2024 159

Combining UCS and Greedy
• Uniform-cost orders by path cost, or backward cost
g(n)
• Greedy orders by goal proximity, or forward cost
h(n)
8

e h=1
1
1 3 2
S a d G
h=6 1 h=5 h=2 h=0
1
c b
h=7 h=6

• A* Search orders by the sum: f(n) = g(n) + h(n)

3/7/2024 Spring' 2024 160
Admissibility of a heuristic
• A heuristic h is admissible (optimistic) if:

where is the true cost to a nearest goal

• Example:

• Coming up with admissible heuristics is most of what’s

involved in using A* in practice.

3/7/2024 Spring' 2024 161

 Playing with A* Heuristics
• The heuristic can be used to control A*’s behavior.
• If h(n) is always ≤ the cost of moving from n to the goal, then
A* is guaranteed to find a shortest path. The lower h(n) is,
the more node A* expands, making it slower.

• If h(n) is exactly equal to the cost of moving from n to the

goal, then A* will only follow the best path and never expand
anything else, making it very fast. It’s nice to know that given
perfect information, A* will behave perfectly.

• If h(n) is sometimes ˃ the cost of moving from n to the goal,

then A* is not guaranteed to find a shortest path, but it can
run faster.

• At the other extreme, if h(n) is very high relative to g(n), then

only h(n) plays a role, and A* turns into Best-First-Search.
3/7/2024 Spring' 2024 162
Introduction to Artificial
Intelligence
Unit 6 – adversarial search
Outline
• Introduction
• Game Tree
• The Minimax Algorithm
• Alpha-Beta Pruning

Spring 2020 164

Introduction

Spring 2020 165

Introduction
• Search algorithms tend to utilize a cause-and-effect concept
• The search considers each possible action available to it at a
given moment
• Considers its subsequent moves from each of those states, and
so on, in an attempt to find terminal states which satisfy the goal
conditions it was given
• Upon finding a goal state, it then follows the steps it knows are
necessary to achieve that state

Spring 2020 166

Introduction
• However, in a competitive multi-player game, when other agents
are involved and have different goals in mind things get more
complicated
• Even if a search algorithm can find a goal state, it usually cannot
simply take a set of actions which will reach that state
• For every action our algorithm takes towards its goal, the opposing
player can take an action which will alter the current state
(presumably in a way unfavorable to our algorithm.)

Spring 2020 167

Game Tree

Spring 2020 168

Game Tree
• Many two-player games can be efficiently represented using trees
called game trees
• A game tree is an instance of a tree in which
oThe root node represents the state before any moves have been
made
oThe nodes in the tree represent possible states of the game (or
positions)
oArcs in the tree represent moves

Spring 2020 169

Game Tree
• It is usual to represent the two players’ moves on alternate levels
of the game tree
• All edges leading from the root node to the first level represent
possible moves for the first player
• Edges from the first level to the second represent moves for the
second player, and so on

Spring 2020 170

Game Tree – Tic-Tac-Toe

Spring 2020 171

Game Tree – Ply
• Ply – refers to the depth of the tree
• When a computer evaluates a game tree to ply 5, it is examining
the tree to a depth of 5
• In the games involving two players, sequential plies in the tree will
alternately represent the two players
• A game tree with a ply of 8 will represent a total of eight choices in
the game, which corresponds to four moves for each player

Spring 2020 172

Game Tree – Move
• The word ply represents a single level of choice in the game tree
• The word move represent two such choices—one for each player

Spring 2020 173

Game Tree – Searching
• Possible Approach
• Use a tree search algorithm such as depth-first or breadth-first search,
looking for a goal state (i.e., a final state of the game where the computer
has won)
• It can be inefficient for a computer to exhaustively search the tree
• For Tic-Tac-Toe, with a maximum depth of 9 and maximum branching
factor of 9
• Approximately 9 x 8 x 7 . . . 2 x 1 nodes in the tree
• More than 350,000 nodes to examine

Spring 2020 174

Game Tree – Searching
• Chess has an average branching factor of about 35 and games
often go to 50 moves by each player, so the search tree has about
35100 or 10154 nodes
• The Deep Blue chess computer which defeated Kasparov in 1997
would typically search to a depth of between six and sixteen plies
to a maximum of forty plies in some situations

Spring 2020 175

Minimax Algorithm

Spring 2020 176

Minimax Algorithm
• A useful method for simple two-player games. It is a method for
selecting the best move given an alternating game where each
player opposes the other working toward a mutually exclusive goal
• A tactic, which uses the fact that the two players are working
towards opposite goals to make predictions about which future
states will be reached as the game progresses
• It then proceeds accordingly to optimize its chance of victory

Spring 2020 177

Minimax Algorithm
• Assumption in a two-person game
• Opponent has the same knowledge that you do, and
• Applies it as well as you do
• So at each stage of the game we assume our opponent makes the
best available move. This is the basis of the minimax procedure

Spring 2020 178

Minimax Algorithm
• The theory behind minimax is that the algorithm's opponent will
be trying to minimize whatever value the algorithm is trying to
maximize (hence, "minimax")
• Thus, the computer should make the move which leaves its
opponent capable of doing the least damage

Spring 2020 179

Minimax Algorithm
• It is assumed that a suitable static evaluation function is
available, which is able to give an overall score to a given position
• In applying Minimax, the static evaluator will only be used on leaf
nodes, and the values of the leaf nodes will be filtered up through
the tree, to pick out the best path that the computer can achieve

Spring 2020 180

Minimax Algorithm
• The principle behind Minimax is that a path through the tree is
chosen by assuming that
• At its turn (a max node), the computer will choose the move that
will give the highest eventual static evaluation, and that
• At the human opponent’s turn (a min node), he or she will choose
the move that will give the lowest static evaluation
• In other words, we assume that each player makes the next move
that benefits them the most

Spring 2020 181

Minimax Algorithm – Example

Spring 2020 182

Minimax Algorithm – Activity I

Spring 2020 183

Minimax Algorithm – Activity II

Spring 2020 184

Minimax Algorithm – Activity III

Spring 2020 185

Minimax Algorithm
• In the ideal case computer investigate every game outcome
possible from the game's current state as well as the paths taken
to reach those states
• It assigns those outcomes values
• For a win-or-lose game like chess or tic-tac-toe, there are only
three possible values-win, lose, or draw
• They are assigned numeric values of 1, -1, and 0 respectively
• For other games like backgammon or poker with a "score" to
maximize, the scores themselves are used

Spring 2020 186

Minimax Algorithm – Tic-Tac-Toe

Spring 2020 187

Minimax Algorithm – Tic-Tac-Toe

Spring 2020 188

Lookahead
• Minimax is a very simple algorithm and is unsuitable for use in
many games, such as chess, where the game tree is extremely
large
• In such cases, bounded lookahead is very commonly used and
can be combined with Minimax
• The idea of bounded lookahead is that the search tree is only
examined to a particular depth
• All nodes at this depth are considered to be leaf nodes and are
evaluated using a static evaluation function

Spring 2020 189

Horizon Effect
• When we employ lookahead strategy, we suffer from what is
called the horizon effect
• When we can’t see beyond the horizon, it becomes easier to make
a move that looks good now, but leads to problems later as we
move further into this subtree

Spring 2020 190

ALPHA-BETA PRUNING

Spring 2020 191

Alpha-Beta Pruning
• The minimax algorithm performs a depth-first search
• For a tree with maximum depth m and b possible actions at each
state
• Time complexity is O(bm)
• Space complexity is O(bm) if all actions are generated at once,
O(m) if actions are generated one at a time
• For most real games, this time complexity is totally impractical

Spring 2020 192

Alpha-Beta Pruning
• Minimax good for small depth and branching factors
• Exponential time complexity of the minimax algorithm is a
problem
• Need a way to avoid searching branches of the game that are
obviously bad
• Approach – Prune away those parts of the tree that don’t need to
be examined because they would not change the optimal decision

Spring 2020 193

Alpha-Beta Pruning – Example

Spring 2020 194

Alpha-Beta Pruning – Example

X X

Spring 2020 195

Alpha-Beta Pruning – Example

X X

Spring 2020 196

Alpha-Beta Pruning – Example

X X

Spring 2020 197

Alpha-Beta Pruning – Example
• 7 out of 12 nodes are explored

X X

Spring 2020 198

Alpha-Beta Pruning
• Alpha-Beta – a pruning algorithm used in conjunction with the
minimax algorithm
• Using alpha–beta pruning, it is possible to remove sections of the
game tree that are not worth examining
• This speed up depth-first minimax search to make searching for a
good move more efficient

Spring 2020 199

Alpha-Beta Pruning
• Principle behind alpha–beta pruning
• If a move is determined to be worse than another move that has
already been examined, then further examining the possible
consequences of that worse move is pointless
• Identify moves that are not beneficial, and remove them from the
game tree
• Higher in the game tree that branches are pruned the greater
effect in minimizing the search of the tree

Spring 2020 200

Alpha-Beta Pruning
• During the depth-first search of the game tree, we calculate and
maintain two variables alpha and beta
• The alpha variable defines the best move that can be made to
maximize (our best move)
• The beta variable defines the best move that can be made to
minimize (the opposing best move)

Spring 2020 201

Alpha-Beta Pruning
• Initially alpha is negative infinity and beta is positive infinity
• While we traverse the tree, if alpha is ever greater than or equal to
beta, then the opponent’s move forces us into a worse position
• In this case, we avoid evaluating this branch any further

Spring 2020 202

Alpha-Beta Pruning – Example II

Spring 2020 203

Alpha-Beta Pruning – Example II

Spring 2020 204

Alpha-Beta Pruning – Example II

Spring 2020 205

Alpha-Beta Pruning – Example II

Spring 2020 206

Alpha-Beta Pruning – Example II

Spring 2020 207

Alpha-Beta Pruning – Example II

Spring 2020 208

Alpha-Beta Pruning – Example II

Spring 2020 209

Alpha-Beta Pruning – Example II

Spring 2020 210

Alpha-Beta Pruning – Example II

Spring 2020 211

Alpha-Beta Pruning – Example II

Beta should be -51

Spring 2020 212

Spring 2020 213
Spring 2020 214
Spring 2020 215
Spring 2020 216
Spring 2020 217
Alpha-Beta Pruning – Example II

Spring 2020 218

 Lecture
Introduction to AI
Muhammad Adil Saleem
Spring 2024
Today's Agenda
• Unsupervised Learning
• Clustering
• K-Means
• Fuzzy C-Means
• Hierarchical Clustering

CSE307: Introduction to AI (Spring 2023) 220

Unsupervised Learning
• Unsupervised learning is a type of machine learning
where the algorithm learns to identify patterns and
relationships in data without explicit guidance or labeled
examples.

CSE307: Introduction to AI (Spring 2023) 221

 Question
Does unsupervised learning qualify as AI?
Common Unsupervised Learning Tasks
• Clustering: Grouping similar data points together into clusters
based on their similarity or proximity to each other.
• Dimensionality reduction: Reducing the number of features or
dimensions in a dataset while preserving important information
and structure.
• Anomaly detection: Identifying unusual or rare observations in a
dataset that do not fit the typical patterns or distributions.
• Association rule mining: Identifying frequent co-occurrences or
correlations between items in a dataset, such as products
frequently purchased together in a store.
CSE307: Introduction to AI (Spring 2023) 223
 Question
What are the challenges with unsupervised learning?
Challenges with Unsupervised Learning
• Lack of labeled data
• Subjectivity in interpretation
• Lack of transparency
• Sensitivity to noise and outliers
• Difficulty in handling high-dimensional data

CSE307: Introduction to AI (Spring 2023) 225

Clustering
• Organizing data into clusters such that there is
• high intra-cluster similarity
• low inter-cluster similarity
• Informally, finding natural groupings among objects.

CSE307: Introduction to AI (Spring 2023) 226

 Question
Why is clustering difficult?
Natural Grouping

CSE307: Introduction to AI (Spring 2023) 228

Similarity is Hard To Define

CSE307: Introduction to AI (Spring 2023) 229

Types of Clustering
There are several types of clustering:
• Partitioning-based clustering: Divides the data into non-
overlapping partitions.
• Fuzzy clustering: Allows a data point to belong to multiple clusters
with varying degrees of membership, such as fuzzy c-means.
• Hierarchical clustering: Builds a tree-like hierarchy of clusters,
either by merging smaller clusters into larger ones or by dividing
larger clusters into smaller ones.
• Density-based clustering: Groups data points based on regions of
higher density.

CSE307: Introduction to AI (Spring 2023) 230

How Many Clusters?

CSE307: Introduction to AI (Spring 2023) 231

K-Means Clustering
• K-means is a popular partitioning-based clustering algorithm that
aims to divide the data into K non-overlapping clusters.
• The algorithm starts by randomly selecting K centroids, which
represent the centers of the initial clusters.
• Each data point is then assigned to the nearest centroid, based on the
Euclidean distance between the data point and the centroid.
• After all data points have been assigned to a cluster, the centroids are
updated as the mean of all the data points assigned to that cluster.
• The process of assigning data points and updating centroids is repeated
iteratively until convergence, i.e., when the assignments no longer change
or a maximum number of iterations is reached.

CSE307: Introduction to AI (Spring 2023) 232

K-Means Clustering

CSE307: Introduction to AI (Spring 2023) 233

Activity: K-Means Clustering
A B C
2 3 4
1 2 3
2 2 4
5 6 5
6 5 6
7 5 7
8 9 7
9 10 8
10 8 9
9 9 8

CSE307: Introduction to AI (Spring 2023) 234

How many clusters?

CSE307: Introduction to AI (Spring 2023) 235

Limitations of K-Means
K-Means has several limitations:
• Selection of K: One of the main challenges in k-means clustering
is choosing the appropriate number of clusters K.
• Spherical clusters: K-means assumes that the clusters are
spherical and have equal variance in all dimensions. This may not
be appropriate for all datasets, where the clusters may have
different shapes or orientations.
• Non-deterministic: K-means is a non-deterministic algorithm,
which means that different initializations of the centroids can lead
to different solutions.
CSE307: Introduction to AI (Spring 2023) 236
 Question
How to evaluate clustering performance?
Evaluation Of K-Means Clustering
• Most common measure is Sum of Squared Error (SSE)
• For each point, the error is the distance to the nearest cluster
• To get SSE we square these errors and sum them

• x is a data point in cluster Ci and mi is the representative point for cluster

Ci
• One easy way to reduce SSE is to increase K, the number of clusters
• A good clustering with smaller K can have a lower SSE than a poor clustering with
higher K

CSE307: Introduction to AI (Spring 2023) 238

 Question
Can SSE be help in determining ideal number of clusters?
Scree Plot (Elbow Curve)

CSE307: Introduction to AI (Spring 2023) 240

 Question
Can we use different distance metrics?
Hierarchical Clustering
• Hierarchical clustering is a type of clustering algorithm that builds
nested clusters by iteratively merging or splitting clusters based
on the distance between data points.
• It starts by treating each data point as its own cluster, and then
combines the closest pairs of clusters into a larger cluster.
• This process continues iteratively, with each iteration resulting in a new
level of clustering hierarchy.

CSE307: Introduction to AI (Spring 2023) 242

Hierarchical Clustering
There are two types of hierarchical clustering:
• Agglomerative clustering starts with each data point as a separate
cluster and successively merges the two closest clusters until a
single cluster containing all data points is formed.
• Divisive clustering starts with a single cluster containing all data
points and successively splits it into smaller clusters until each
data point is in its own cluster.

CSE307: Introduction to AI (Spring 2023) 243

Dendogram

CSE307: Introduction to AI (Spring 2023) 244

Different Types of Linkages

CSE307: Introduction to AI (Spring 2023) 245

Activity: Hierarchical Clustering
x1 x2
1 1.0 2.0
2 1.5 3.0
3 3.0 4.0
4 5.0 7.0
5 3.5 5.0

CSE307: Introduction to AI (Spring 2023) 246