******************************************

- PROBLEM-SOLVING AGENTS -
******************************************
[~] Intelligent agents are supposed to maximize their performance measure.
[~] Goal formulation - first step in problem solving.
[-] organize behavior by limiting objectives & actions it needs to consider.
[~] Problem formulation - process of deciding what actions & states to consider.
[-] Agent with many immediate opts of unknown val can decide what to do by
first examining future opts that lead to states of known val.
[~] Process of looking for a seq of actions that reaches the goal is called search.
[-] Takes problem as inp & returns a sol in form of an action seq.
[~] Once sol is found, actions can be carried out -> execution phase. Simple design
for agent is "formulate, search, execute".

*******************************************************
- WELL-DEFINED PROBLEMS AND SOLUTIONS -
*******************************************************
[~] A search problem can be defined formally by 5 components:
[-] Initial state agent starts in. For ex: state s.
[-] A description of possible actions available. For ex: ACTION(s) returns set
of actions can be executed in s.
[-] A description of what each action does - transition model. RESULT(s, a)
returns the state results from doing action a in state s.
[+] Successor refers to any state reachable from given state by a single
action.
[+] Initial state, actions, and transition model define the state space of
the problem - set of all states reachable from initial state.
[*] Can be represented as a graph, vertices - state, directed edges -
actions.
[-] Goal test - determines whether a given state is a goal state.
[-] A path cost function that assigns a numeric cost to each path.
[+] Step cost (action cost) of doing action a in state s to reach state s'
- denoted by c(s, a, s').
[+] Total cost - sum of individual action costs.

****************************************
- FORMULATING PROBLEMS -
****************************************
[~] Abstraction involves removing detail from a representation.
[-] Create an approximate and simplified model of the world
[~] Abstraction is valid if can expand any abstract sol into a sol in a more
detailed world.
[~] Abstraction is useful if carrying out each of the actions in sol is easier than
orginal problem.
[~] Good abstraction removes as much detail as possible while retaining validity
and ensuring abstract actions are easy to carry out.

********************************************
- VACUUM WORLD (page 70) -
********************************************
[~] Problem can be formulated as follow:
[-] States: determined by agent location and dirt locations. Agent can be in 2
locations, each location may or may not contain dirt.
=> 2 * 2^2 = 8 possible world states.
[+] Larger environment would be n * 2^n states.
[-] Initial state: Any state can be the initial state.
[-] Actions: In simple environment, each state has 3 actions: go left, go
right, and suck dirt. Larger environment may involves up and down.
[-] Transition model
 [-] Goal test: checks whether all squares are clean.
[-] Path cost: Each step costs 1, so path cost is number of steps in the path.

**********************************************
- 8-PUZZLE WORLD (page 71) -
**********************************************
[~] States: a state description specifies the location of each of 8 tiles and blank
tile in one of 9 squares.
[~] Initial state: Any state can be initial state.
[~] Actions: simplest formulation defines actions as movements of blank space:
left, right, up, down.
[~] Transition model: Given a state and action, this returns the resulting state.
[~] Goal state: checks whether the state matches the goal configuration.
[~] Path cost: each step costs 1, path cost is number of steps in path.

************************************
- TOWER OF HANOI -
************************************
[~] States: a state description specifies 3 pegs and 3 disks on pegs such that
large disk is at lower location than small disk.
[-] Continously place disk from large to small onto 3 pegs, we have n^3 states
(n disks).
[~] Initial state: One of three peg has enough 3 disks in order, two other pegs are
empty.
[~] Actions: Put a top disk from a peg and place it on other peg so that no disk is
above the lower one.
[~] Transition model: Given a state and action, this returns the resulting state.
[~] Goal state: checks whether the destination rod has enough 3 disks such that no
disk is above the lower one.
[~] Path cost: each action costs 1, so path cost is number of actions in path.

******************************************************
- ALGORITHM'S PERFORMANCE CRITERIA -
******************************************************
[~] Completeness: is the algo guaranteed to find a sol when there's one ?
[~] Optimality: does the strategy find the optimal sol ?
[~] Time complexity: how long does it take to find a sol ?
[~] Space complexity: how much mem needed to perform the search ?
[~] Complexity is expressed in terms of 3 quantities:
[-] b: the branching factor (max # of successors of any node).
[-] d: depth of shallowest goal node.
[-] m: max length of any path in state space.

**********************************************************************
- Breadth-first search (Uninformed search strategy) -
**********************************************************************
[~] Use FIFO queue.
[~] All nodes are expanded at a given depth before any nodes at next level are
expanded.
[~] BFS search always has the shallowest path to every node on the frontier ->
reduce time complexity.
[~] Slight changes: goal test is applied to each node when generated rather than
selected for expansion.
[~] Memory requirements are a bigger problem for BFS search than is the execution
time.
[~] Time is still a major factor, so BFS (or any uninformed search) is unoptimized.
=> exponential-complexity search problems can't be solved by uninformed methods
for any but the smallest instances (unless small constraints).
**********************************************************************
- Uniform-cost search (Uninformed search strategy) -
**********************************************************************
[~] BFS is only optimal when all step costs are equal. We can expand for any step-
cost function by expand the node with lowest path cost.
[~] This is done by storing the frontier as a priority queue.
[~] 2 main diff from BFS:
[-] Goal test is applied to a node when it is selected for expansion (because
the generated node may be in suboptimal path).
[-] A test is added in case a better path is found to a node which is currently
on the frontier.
[~] Whenever a node is selected for expansion, the optimal path to that node has
been found.

*********************************************************************
- Depth-first search (Uninformed search strategy) -
*********************************************************************
[~] Always expands the deepest node in the current frontier. Use LIFO queue. Apply
early goal test.
[~] Space complexity is small.
[~] Depth-limited search: to alleviate when DFS is put in infinite state spaces.
[-] Supply DFS with a predetermined depth limit L. Nodes at depth L are
considered as "no successors".
[-] Incompleteness: choose l < d
[-] Inoptimal: choose l > d
[-] Using the diameter of the state space (mostly not known) would give a
better depth limit.
[~] Iterative deepening search: gradually inc the limit, combine with dfs, until a
goal is found. Occur when depth limit reaches d.
[-] Like DFS, memory requirement is modest: O(bd)
[-] Like BFS, algo complete when branching factor is finite and optimal when
path cost is a non-dec function of depth of node.
[~] Bidirectional search: run 2 simultaneous searches - 1 forward from initial
state and other backward from goal.
[-] Motivation: b^(d/2) + b^(d/2) much less than b^d.
[-] Replacing goal test with check to see if frontiers of two searches
intersect.

************************************************************
- Informed (Heuristic) Search Strategies -
************************************************************
[~] A node is selected for expansion based on an evaluation function, f(n). f(n) is
construed (được hiểu là) as a cost estimate, node with lowest evaluation is
expanded first.
[~] Most best-first algo include a heuristic function as a component of f, denoted
h(n).
[-] h(n) = estimated cost of cheapest path from the state at node n to the goal
state. h(n) only depends on state at the node.
[-] If n is a goal node, then h(n) = 0.

*********************************************************
- Greedy best-first search (informed) -
*********************************************************
[~] Expand the node closest to the goal (likely to lead to sol quickly).
=> Evaluate nodes using just the heuristic function: f(n) = h(n).
[~] Apply early goal test.
[~] Implementation is identical to uniform-cost search, use f instead of g.
[~] Only complete in finite state space. Not optimal.
[-] Time: reduced substantially with good heuristic.
 [-] Space: all nodes kept in memory.

*****************************************************************************
- A* search: minimize total estimated sol cost (informed) -
*****************************************************************************
[~] Evaluate nodes by combining g(n) - optimzed cost to reach the node, and h(n) -
estimated cost to get from the node to the goal.
=> f(n) = g(n) + h(n).
[~] f(n) = estimated cost of the cheapest solution through n.
[~] Identical to UCS except uses g + h instead of g => late goal test.
[~] Conditions for optimality: admissibility (tính khả thi) and consistency (tính
nhất quán)
[-] h(n) must be an admissible heuristic => never overestimates the cost to
reach the goal.
[+] f(n) = g(n) + h(n) => f(n) never overestimates the true cost of a sol
along current path through n.
[-] Admissible heuristic is nature optimistic because it thinks the problem-
solving cost is less than it actually is.
[-] Slightly stronger condition: consistency is required only for applications
of A* to graph search.
[+] h(n) is consistent if, for every node n and every successor n' of n gen
by action a, the estimated cost of reaching the goal from n is no greater than the
step cost of getting to n' plus the estimated cost of reaching goal from n'.
h(n) <= c(n, a, n') + h(n') (general triangle inequality)
[+] Prove based on admissible: if there were a route from n to goal via n'
that was cheaper than h(n), that would violate the property that h(n) is the lower
bound on the cost to reach the goal from n.
[~] Optimality of A*:
[-] Tree-search ver of A* is optimal if h(n) is admissible, graph-search ver of
A* is optimal if h(n) is consistent.
[-] If h(n) is consistent, then f(n) along any path are nondecreasing.
[+] Prove (n' is successor of n): f(n') = g(n') + h(n') = g(n) + c(n, a,
n') + h(n') >= g(n) + h(n) = f(n).
[-] Whenever A* selects a node n for expansion, the optimal path to that node
has been found.
[~] Search contours:
[-] Are concentric bands in which contour i has all nodes with f = f(i) where
f(i) < f(i + 1).
[-] A* contours vs UCS contours:
[+] UCS bands are "circular" around the start state.
[+] A* bands, with more accurate heuristics, stretch toward the goal state
and become more narrowly focused to the optimal path.
[-] If C* is the cost of optimal sol path:
[+] A* expands all nodes with f(n) < C*
[+] A* might expand some nodes right on "goal contour" (f(n) = C*) before
selecting a goal node.
[~] A* search is complete, optimal, and optimally efficient.
[~] Weighted A* search:
[-] Detour index: E.g., value of 1.3 means if 2 cities are 10 miles apart in
straight-line, best path between them is 13 miles.
[-] For most localities, detour index ranges between [1.2, 1.6].
[-] It weighs the heuristic value more heavily, which yields the evaluation
function:
f(n) = g(n) + W * h(n) (W > 1)
[~] Memory-bounded heuristic search (A* usually out of space before out of time):
[-] Idea: use iterative deepening with cutoff = f-cost. Cutoff value of each
iteration = min(f-cost of node that exceeded cutoff of previous iteration).
[-] Recursive best-first search (RBFS).
[-] SMA* (simplifided memory-bounded A*).