NPTEL

Video Course on Machine Learning

Professor Carl Gustaf Jansson, KTH

Week 3 Forms of Representation

Video 3.1 Forms of Representation

General System Components

Representation Problemsolving
Reasoning
Decisionmaking

Learning
Computer Science is an Engineering discipline.

Artificial Intelligence as well.

The choices of representation and problemsolving schemes are

based on many pragmatical engineering decisions.

Typically the same domain and problemarea can be modelled in any

of the alternative representation and problemsolving schemes.

For specific domains, mappings from one set of schemes to another is

not atypical.

Problem solving and Machine Learning for a domain may be more

feasible to implement in some of the schemes.
Forms of Represention and Problem solving
chosen for separate lectures

Decision Trees
Bayesian Networks
Neural Networks
Genetic Algoritms
Logic programming
Structure of Lecture for each of the themes

General Characteristics of this Representation

Interdisciplinary sources of inspiration for this Representation
Core components of this Representation
Structural aspects of this Representation
Problemsolving for this Representation
Learning for this Representation
Additional themes relevant for Representation and Problem Solving

Representations as Liststructures and Listprocessing

Arrays, Vectors, Matrices, Tensors
Graph Theory
Graph Search
Semantic Networks
Production Rule Systems
Representations as Liststructures and Listprocessing
One of the most established forms of representation in Artificial Intelligence is Liststructures.
The corner stone are expressions in the form of lists, the elements of which are atoms or lists
recursively. The role and interpretations of atoms and lists in various positions in the complex
recusive symbolic structures are defined for each domain or application. This creates great
flexibility but also a lack of share semantics across domains and applications.

Functional programming is the classical programming paradigm or computational model used

for computing on List structures. Computation is here viewed as a recursive application of a
system of mathematical functions. It is a declarative programming paradigm, which means
programming is done with expressions instead of statements. Functional programming has its
origins in lambda calculus, a formal system developed in the 1930s to investigate computability.

LISP is the second oldest programming still in use and an archetypical functional programming
language. LISP has been used extensively for all kinds of Artificial Intelligence applications.
Many times alternative representation schemes have been mapped onto a pure Listsstructure and
LISP based form.

Example: representation of a Semantic Network in LISP

(setq *database* ‘
((canary (is-a bird) (color yellow) (size small))
(penguin (is-a bird) (movement swim))
(bird (is-a vertebrate) (has-part wings) (reproduction egg-laying))))
Arrays, Tables, Vectors, Tuples, Matrices and Tensors
An array is any ordered n-dimensional collection, where elements each identified
by at least one array index or key.

Tables are often implemented in the form of arrays. the word table is sometimes
used as a synonym of array.

In some cases the term Vector is used to refer to a one dimensional array,
although tuple rather than vector is the more mathematically correct equivalent.

Because the mathematical concept of a Matrix can be represented as a two-

dimensional grid, two-dimensional arrays are sometimes called Matrices even if a
matrix is a separate kind of mathematical entity related to linear mappings.

For similar reasons multidimensional arrays may be referred to as Tensors even if

a tensor is a special kind of mathematical entity as well.

Many times alternative representation schemes have been mapped onto a pure
array structure with the pupose to base computations on Array processing.
Arrays, Vectors, Matrices and Tensors
Graph Theory
In mathematics, graph theory is the study of graphs, which are mathematical structures
used to model pairwise relations between objects. A graph in this context is made up
of vertices, nodes, or points which are connected by edges, arcs, or lines.

Edges of a graph has an associated numerical value, called a weight. Usually, the edge
weights are nonnegative integers. Weighted graphs may be either directed or undirected.

A graph may be undirected, meaning that there is no distinction between the two vertices
associated with each edge, or its edges may be directed from one vertex to another.

A graph is connected if there is a path from any point to any other point in the graph. A
graph that is not connected is said to be disconnected. An acyclic graph is a graph having
no graph cycles. A connected acyclic graph is known as a tree. A disconnected acyclic
graph is known as a forest. A cycle is a path of edges and vertices wherein a vertex is
reachable from itself.

A bipartite graph, also called a bi-graph, is a set of graph vertices decomposed into two
disjoint sets such that no two graph vertices within the same set are adjacent. All acyclic
graphs are bipartite. A cyclic graph is bipartite iff all its cycles are of even length.

The order of a graph is the number of vertices. The size of a graph is the number of edges.
The degree of a vertex is the number of edges that connect to it.
Graph theoretical problems
• The Seven Bridges of Köningsberg is a historically notable problem in
mathematics. The problem was to devise a walk through the city that would
cross each of the seven bridges once and only once.

• A Hamiltonian path is a path in an undirected or directed graph that visits each

vertex exactly once.

• An Euler path is a path that uses every edge of a graph exactly once.

• The travelling salesman problem (TSP) is finding the shortest possible route
that visits each city and returns to the origin city given a list of cities and the
distances between each pair of cities.

• The shortest path problem is the problem of finding a path between

two vertices such that the sum of the weights of its constituent edges is
minimized.

• A minimum spanning tree (MST) or minimum weight spanning tree is a subset

of the edges of a connected, edge-weighted undirected graph that connects all
the vertices together, without any cycles and with the minimum possible total
edge weight.
Graph search – terminology and techniques
Problem Space Graph − Represents the problem state. States are shown by nodes and operators are shown by edges.
Problem Instance − An Initial state + a Goal state.
Branching Factor − The average number of child nodes in the problem space graph.
Depth of a problem − Length of a shortest path or shortest sequence of operators from Initial State to goal state.
Complexity − The maximum number of nodes expanded
Admissibility − A property of an algorithm to always find an optimal solution.

Brute-Force Search Strategies

Breadth-First Search
Depth-First Search
Bidirectional Search
Uniform Cost Search
Iterative Deepening Depth-First Search
Informed (Heuristic) Search Strategies
Pure Heuristic Search
A * Search
Greedy Best First Search
Local Search Algorithms
Hill-Climbing Search
Local Beam Search
Simulated Annealing
Semantic Network (Conceptual Graph)
A semantic network is a graphical formalism that
represents semantic relations between concepts. It is
a directed or undirected graph consisting of vertices which
represent concepts, and edges which represent semantic relations
between the concepts.

Edges can be labelled in arbitrary ways, but there is a small set of

Standard edge types that occurs frequently:
- IS-A which corresponds to a generalization relation in a taxonomy
- HAS which is a standard attribute (feature) relation
- PART OF which is acomponent relation
- Causal relations.

A drawback with semantic network representations is that there is no

agreed-upon notion of what a given representational structure means
or some formal semantics as there is in logic.

The expressive power of semantic power can be enhanced by

Introduction of partitions, in the sense that a node can be a
subnetwork.
Examples of Semantic Networks
Rule-based systems
Synonyms: IF-THEN rule system, Production system,
Rule-based Expert Systems

Rules-based systems separates declarative and procedural knowledge. Rules specify inference steps

Production rules are formulated in the following manner:

IF Condition 1 ∧ . . . ∧ Condition N THEN Perform some actions.

Production rules may be interpreted in several different ways:

premise → conclusion antecedence → consequence,
evidence → hypothesis situation → action

A rule-based system consists of the following core parts

• A collection of facts
• A collection of rules – ´knowledge base´
• An inference engine that can match rules against
facts in both a forward and backward chaining manner.
• Stop criteria for terminating the computation.
1. IF (fever (high)) THEN Malaria
2. IF (Coldness) THEN Malaria
3. IF (Throb) THEN Malaria
4. IF (Sweat) THEN Malaria
5. IF (Sometimes colour of urine is black water fever) THEN Malaria
6. IF (Headache) THEN Malaria
7. IF (Vomiting) THEN Malaria
8. IF (Muscle pain) THEN Malaria
9. IF (High temperature) THEN Malaria
10. IF (Diarrhoea) THEN Malaria
11. IF (Coma (Seizure)) THEN Tuberculosis
12. IF ( Stiff Neck) THEN Tuberculosis
Example of a Rule-based Diagnosis system
13. IF (Headache) THEN Tuberculosis
14. IF (AbdoIfminal Pain) THEN Tuberculosis
15. IF ( Weight Pain) THEN Tuberculosis
16. IF ( Fever) THEN Tuberculosis
17. IF (Masses along the neck) THEN Tuberculosis
18. IF (Draining Sinus) THEN Tuberculosis
19. IF (Small Reddish brown lesions ( face, eyelid, nose, cheek and ear)) THEN Tuberculosis
20. IF (Reddish brown wart-like growth on the body) THEN Tuberculosis
21. IF (Skin lesions on hand, feet, elbow and knees) THEN tuberculosis.
22. IF ( Ulcer or abscesses on the Skin) THEN Tuberculosis
23. IF ( Necrosis of infected Skin) THEN Tuberculosis
24. IF (Stiffness of affected area) THEN Tuberculosis
25. IF (Blood present in Urine) THEN Tuberculosis
26. IF (Painful or uncomfortable Urination) THEN Tuberculosis
27. IF ( Hemopysis (coughing up blood)) THEN Tuberculosis
28. IF (Fatigue) then Tuberculosis
29. IF ( Chest pain) THEN Tuberculosis
30. IF ( Night Sweat) THEN Tuberculosis
31. IF (A lump or pain in the breast) THEN Breast cancer.
32. IF (Thickening or swelling of part of the breast) THEN Breast cancer
33. IF (Irritation or dimpling of breast skin) THEN Breast cancer
34. IF (Redness or flaky skin on the breast) THEN Breast cancer
35. IF (Pulling in of the nipple or pain in the nipple area) THEN Breast cancer
36. IF (Fluid other than breast milk from the nipple, especially blood) THEN Breast cancer
37. IF (A change in the size or the shape of the breast) THEN Breast cancer
38. IF (Vomiting) THEN Cholera
39. IF (Watery diarrhoea) THEN Cholera
40. IF (Leg cramps) THEN Cholera
41. IF (Poor appetite) THEN Typhoid Fever
42. IF ( Headache) Then Typhoid Fever
43. IF ( Generalised aches and pains) THEN Typhoid Fever
44. IF (Fever as high as 104 degrees Fahrenheit) THEN Typhoid Fever
45. IF (Lethargy) THEN Typhoid Fever
46. IF (diarrhoea) THEN Typhoid Fever
NPTEL

Video Course on Machine Learning

Professor Carl Gustaf Jansson, KTH

Thanks for your attention!

The next lecture 3.2 will be on the topic:

Decision Trees