AI Notes For The Exam Midterm
AI Notes For The Exam Midterm
What is computation?
Important observations about Turing machines
1. Extending the design of the basic Turing Machine doesn’t make it any Example Problems and Their Complexity
more powerful.
2. Turing Machines are notional. Although it’s in principle possible, 1. Problem: Sorting two lists of numbers of length n using Bubblesort.
nobody builds physical Turing Machines. Complexity: O(n2). Remember, we ignore constants.
3. The Turing Machine is a universal model of computation: The class of 2. Problem: Determine if a given number n odd or even.
problems they can solve is identical to those solvable by a standard von Complexity: O(1). Requires just one operation.
Neumann architecture computer, a neural network, or any other model of 3. Problem: Find the max value of a list of n (unsorted) numbers.
universal computation. Complexity: O(n).
What are the limits of computation? 4. Problem: Determine if two-character strings of length n are equal.
Complexity: O(n).
Two issues need to be considered when asking if a problem to be 5. Problem: Determine if a string of length n a palindrome.
solvable by a computer: tractability and computability. Complexity: O(n).
1. Intractable problems are those that demand infeasible resources to
solve. Key resources are time (the solution takes too long to compute)
and space (finding the solution requires too much memory).
What kind of problems are computationally
2. Uncomputable problems are those that cannot be solved by any intractable?
computer. Some problems are probably uncomputable.
Example: The Traveling Salesperson Problem
Many problems we consider in artificial intelligence are computationally Given:
intractable, and some are uncomputable. 1. A set of n locations.
2. Connections between locations.
3. Distances between pairs of connected of locations.
Find:
4. Find the shortest tour, where a tour starts and ends at the same location, and
visits all n locations.
The traveling salesperson problem is computationally intractable.
A naive approach is to consider all possibilities, but this is O(n!)
See how fast it grows...
n n!
1 1
2 2
9 362,880
13 6,227,020,800
20 2,432,902,008,176,640,000
Practical 1 - Problem Representation On one side of a river, there are three gamblers and three robbers. They both want
to travel to the other side. However, there is only one boat that can hold up to two
people. If the robbers outnumber the gamblers, they will rob them. What sequence of
Example from Russel & Norvig (ch. 3): Knuth’s Conjecture actions would result in all the gamblers and robbers crossing to the other side of the
”[S]tarting with the number 4, a sequence of square root, floor, and river?
factorial can reach any desired positive integer.”
Some integers are easy to reach: √4 = 2 ➢ Give the problem definition
Others are hard to reach:
√√√√(4!)!c = 5 ➢ How can the initial state, actions, and goal state be represented to the agent?
What is the problem definition for this example? ➢ Can you make the representation as short as
possible?
States: Positive real numbers
➢ Which actions are available in the initial state?
• Initial state: 4
• Actions: Apply square root, floor, or factorial operation (factorial for integers
only)
• Transition model: As given by the mathematical definitions of the operations. ➢ States: 0-3 gamblers and robbers on the initial side, the rest on the goal side,
subject to the robbers not outnumbering the gamblers on either side and the
• Goal state: The desired positive integer boat on either side.
• Action cost: Each action costs 1 ➢ Initial state: Three gamblers, three robbers, and a boat on the initial side
➢ Actions: All actions that move 1 or 2 people and the boat to another side
Jugs of Water - Problem ➢ Transition model: The people moved, and the boat was subtracted from one
side and moved to the other.
You have three jugs. One currently holds 8L of water. The other two are empty but
can hold 5L and 3L respectively. How can you use the three jugs to divide the water ➢ Goal state: Three gamblers, three robbers, and a boat on the goal side
such that you have two jugs that each hold exactly 4L? ➢ Action cost: Each action costs 1
➢ Give the problem definition. ➢ Initial state: Only need to keep track of numbers on the initial side: (3, 3, 1)
➢ How can the initial state, actions, and goal state be represented to the agent? ➢ Actions: (gamblers, robbers)
➢ Which actions are available in the initial state? ➢ Why not boat and direction?
▪ States: Any division of the water over the jugs, subject to the jugs’ capacity ➢ Goal state: (0, 0, 0)
▪ Initial state: 8L in the first jug and 0L in the other two ➢ Actions available in initial state: (0, 1) (0, 2) (1, 1)
▪ Actions: Pour water from one jug into the other until one is full, or one is empty ➢ Actions not available in initial state: (1, 0) (2, 0)
▪ Transition model: The amount of water poured is subtracted from one and
added to the other jug.
▪ Goal state: 4L in the first two jugs, 0L in the last
▪ Action cost: Each action costs 1
➢ Early in Boden’s career, AI was largely unheard of, and cognitive science
didn’t exist. There was very little appreciation for the role that computational
thinking could play in the psychological sciences.
Discussion 1: Margaret Boden ➢ Boden was one of the first philosophers to recognize how AI could impact
fundamental questions in philosophy, such as the nature of mind,
consciousness, learning, etc.
➢ Singularity - a hypothetical future point at which AI systems attain
superhuman intelligence and the ability to rapidly self-improve, potentially
surpassing human capabilities and leading to uncertain societal ➢ For cognitive science to exist, an integration of multiple disciplines/concerns
consequences. was required. Boden, as a philosopher, was well-placed to guide such an
integration. She set up the School of Cognitive Science at Sussex University,
which was one of the first.
➢ Generative systems - Generative systems, as defined by Margaret Boden,
are computational systems or algorithms that have the capability to generate
➢ In addition, she established creativity as a concern for AI, the “computational
creative and novel outputs, such as artistic works, music, literature, or other basis for new ideas”. She focused on creativity because she considered it one
forms of creative expression. These systems are designed to mimic some of the hardest problems.
aspects of human creativity and can produce content that is not simply a
recombination of existing data but rather exhibits an element of originality and
innovation.
What, according to Maggie Boden, are the limits of
➢ Machine translation/thesaurus - Machine translation refers to the automated AI and how do these limits relate to the concept of
process of translating text or speech from one language to another using computation?
computer algorithms and software.
➢ In theory, there are no limits to understanding the mind/brain as a
computational
➢ Machine thinking - refers to the capacity of computational systems, device, “to say otherwise is to say that the human mind works by magic, and it
particularly artificial intelligence (AI) systems, to perform cognitive tasks and doesn’t”. [What view on AI is being assumed here?]
problem-solving activities that exhibit characteristics of human thought ➢ In practice, however, it might not be possible to replicate human cognition.
processes. Machine thinking encompasses the ability of AI systems to
process information, reason, learn, and make decisions, often relying on ➢ Mind is the most complex system out there, and language is a particularly
algorithms, heuristics, and computational techniques to mimic aspects of problematic capacity to understand.
human cognition. ➢ We know little about the limits. We have very little idea about what
consciousness is.
➢ Information retrieval - Information retrieval generally refers to the process of ➢ Computers speaking with humans in a natural way is a “fantasy” and a long
accessing and extracting relevant information from large collections of data or way off.
documents, typically using computer systems and algorithms. ➢ Most importantly, how we think about computation will change, as it has
done already. The concepts we have now will not be enough to understand
the mind/brain.
➢ Combinatorial vs. transformational vs. exploratory creativity –
➢ Singularity: More than a few decades away, at the very least.
1. Combinatorial Creativity: involves recombining existing ideas, concepts, or
elements in novel and meaningful ways. It focuses on generating new creations by
assembling or rearranging existing components. This form of creativity is often
associated with lateral thinking and finding fresh combinations among known
elements.
The game consists of numbered tiles. Tiles can be moved by shifting them into the
empty position. From the starting configuration, the task is to reach the goal state
where the tiles are in order shown here.
Lecture 3: Classical Search How many board configurations, or states, are there?
16! = 16 × 15 × . . . × 1 = 20, 922, 789, 888, 000
9! = 9 × 8 × . . . × 1 = 362, 880
Categories of problem: Certainty vs. Uncertainty
➢ Preferable when solution paths are long. ➢ The number of states out of place differs.
➢ Shouldn’t we prefer states “closer” to the solution? In short, use knowledge of
the task to improve search.
➢ In short, use knowledge of the task to improve search.
➢ Only requires linear space O(b · n).
Summary
Fine-tuning the heuristic
If two states have the same number of tiles out of place, we should prefer the state
➢ How can we solve problems like the 8-puzzle? with the
shortest path from the start state.
1. Devise some way of representing the problem. 1. Let g (n) be the distance from state n to the start state.
This allows us to represent different states of problems we need to 2. Let h(n) be the number of tiles out of place in state n.
consider. The final heuristic,
2. Devise operators which move us from one state to another. f (n) = g (n) + h(n)
3. Devise a search strategy for applying the operators, which hopefully
takes us from the
initial state to the goal state.
Which search strategy we use will depend on the problem.
9! = 362, 880
➢ To apply the Breadth-First Search algorithm, we use the open and closed
lists:
Step Open list Closed list
0 (0,0)
➢ If we expand a state, the generated states are added to the end of the open
list.
➢ The rest of the algorithm is the same.
➢ Our search can be visualised in a search tree, as well as in the maze.
➢ Expanding the first (and only) node finds Zerind, Sibiu, and Timisoara
➢ We can look up their cost in the table and add them to the open list:
➢ Apply the Best-First Search Algorithm until you reach the goal. That is, until
you expand the Bucharest node.
➢ The cost of the initial node does not change, as the actual cost so far is 0:
2. Clairvoyance refers to the alleged ability to gain information about a
person, object, event, or location through extrasensory means. It is often
associated with the ability to "see" or perceive things that are distant in
space or time without using the physical senses. Clairvoyance implies an
intuitive or psychic awareness.
àExpanding Arad again finds Zerind, Sibiu, and Timisoara 3. Psychokinesis, often abbreviated as PK, is a purported psychic ability
àThe costs of these nodes are (S,140+253=393), (T,118+329=447), that involves influencing or manipulating physical objects or events using
(Z,75+374=449) only the power of the mind, without any physical interaction. It suggests
the capacity to move or affect objects through mental concentration or
willpower.
not: “not of theoretical importance”. Computing is all that counts.
4. Telepathy is the claimed ability to transmit thoughts, feelings, or ➢ One can’t distinguish between different kinds of computers using
information directly from one person's mind to another's, without the use this test.
of verbal or written communication or any known sensory channels. It
implies a form of direct mind-to-mind communication.
Universal machine - refers to a computational model that can Objection 1: The theological objection
simulate the behavior of any other Turing machine. It's a concept
related to theoretical computer science and the idea that there can Thinking requires a soul, and only god can provide one. Animals and
be a single, versatile machine capable of emulating the machines don’t have a
functionality of any other machine. soul, so machines can’t think in this sense.
➢ This issue is a question of faith, which we can’t resolve
Discrete state machine - a mathematical model that consists of a scientifically.
finite number of states, transitions between those states, and an ➢ Not all people of faith will agree with this argument anyway.
initial state. Discrete state machines are used to represent and
analyze systems or processes with distinct, well-defined states and
➢ Issues like these are why Turing has replaced the question!
transitions. ➢ !!! He doesn’t accept this fact, he thinks that God decides which ones
have a soul and for God animals have souls. He can also improve the
thinking of a certain animal
➢ !!! Humans don’t have such rules that they regulate their whole life, that’s why they
can’t be compared to machines. (Rules of conduct)
Objection 5: The argument from various disabilities
Machines will never be able to do “X”. E.g., have a sense of humour.
➢ Why not? What we refer to as “machines” are changing. Objection 9: Argument from extra-sensory
➢ If we can describe “X”, then why can’t a machine do it? perception (!)
➢ !!! Humans draw conclusions based on the experience they have with Phenomena like telepathy and clairvoyance are real, and tell us that the
machines: they are ugly, they are designed for limited purposes and can’t human mind doesn’t work according to the laws of physics. Could the
deal with other things that are outside of their requirements. interrogator in the Turing test use it to distinguish human from machine?
➢ Phenomena like telepathy and clairvoyance are not real. There is
no evidence.
Objection 6: Lady Lovelace’s objection
Learning machines
Machines can only ever do what we program them to do. They cannot
What is Turing telling us in this section of the article?
think for themselves.
➢ Turing’s argument from complexity: Machines often do the 1. He’s drawing us away from viewing “machines” as simple sets of
unexpected. rules, and toward
➢ Machines are unpredictable, and can beat their designers at viewing them as devices that can learn by themselves like humans.
various games. 2. Could we program a “child” machine, and let it learn for itself, like a
human?
3. After all, humans are not created in one shot, we are the product of
years of experience
and learning.
Objection 7: Argument from the continuity of the 4. If we equip a machine with the ability to think logically, perhaps it
could work things out
nervous system for itself.