UNIT 4

CONTEXT AND WORLD KNOWLEDGE

Knowledge representation and reasoning - Using World Knowledge, Discourse Structure, Local discourse
context and reference.

KNOWLEDGE REPRESENTATION AND REASONING

Knowledge representation and reasoning (KR, KRR) is the part of Artificial intelligence

concerned with AI agents thinking and how thinking contributes to intelligent behavior of agents.

o It is responsible for representing information about the real world so that a computer can understand and
can utilize this knowledge to solve the complex real world problems such as diagnosis a medical condition
or communicating with humans in natural language.
o It is also a way which describes how we can represent knowledge in artificial intelligence. Knowledge
representation is not just storing data into some database, but it also enables an intelligent machine to learn
from that knowledge and experiences so that it can behave intelligently like a human.

Following are the kind of knowledge which needs to be represented in AI systems:

o Object: All the facts about objects in our world domain. E.g., Guitars contains strings, trumpets are brass
instruments.
o Events: Events are the actions which occur in our world.
o Performance: It describe behavior which involves knowledge about how to do things.
o Meta-knowledge: It is knowledge about what we know.
o Facts: Facts are the truths about the real world and what we represent.
o Knowledge-Base: The central component of the knowledge-based agents is the knowledge base. It is
represented as KB. The Knowledgebase is a group of the Sentences (Here, sentences are used as a technical
term and not identical with the English language).
Types of knowledge

1. Declarative Knowledge:

o Declarative knowledge is to know about something.

o It includes concepts, facts, and objects.
o It is also called descriptive knowledge and expressed in declarativesentences.
o It is simpler than procedural language.

2. Procedural Knowledge

o It is also known as imperative knowledge.

o Procedural knowledge is a type of knowledge which is responsible for knowing how to do something.
o It can be directly applied to any task.
o It includes rules, strategies, procedures, agendas, etc.
o Procedural knowledge depends on the task on which it can be applied.

3. Meta-knowledge:

o Knowledge about the other types of knowledge is called Meta-knowledge.

4. Heuristic knowledge:

o Heuristic knowledge is representing knowledge of some experts in a filed or subject.

o Heuristic knowledge is rules of thumb based on previous experiences, awareness of approaches, and which
are good to work but not guaranteed.
5. Structural knowledge:

o Structural knowledge is basic knowledge to problem-solving.

o It describes relationships between various concepts such as kind of, part of, and grouping of something.
o It describes the relationship that exists between concepts or objects.

Here are key aspects related to knowledge representation and reasoning using world knowledge:

1. World Knowledge Integration:

 External Knowledge Sources: Incorporating information from external databases, ontologies, or
knowledge graphs that capture a broad range of facts about the world.
 Semantic Web Technologies: Utilizing standards like RDF (Resource Description Framework)
and OWL (Web Ontology Language) to represent and link diverse knowledge on the web.
2. Ontologies for World Knowledge:
 Definition: Ontologies provide a formal and shared representation of concepts and their
relationships within a specific domain.
 Semantic Enrichment: Extending ontologies to include external world knowledge, creating a
more comprehensive understanding of the subject matter.
3. Commonsense Reasoning:
 Incorporating Everyday Knowledge: Enhancing systems with the ability to reason about
common knowledge that may not be explicitly stated but is assumed to be shared by individuals.
 Conceptual Blending: Integrating disparate pieces of information to form new understandings
and insights.
4. Natural Language Processing (NLP):
 Utilizing Linguistic Knowledge: Incorporating linguistic resources, such as WordNet, to
understand the meanings of words and their relationships.
 Contextual Understanding: Considering the context of language use and incorporating
contextual knowledge for more accurate interpretation.
5. Machine Learning and Knowledge Embeddings:
 Embedding World Knowledge in Vectors: Using knowledge graph embeddings or pretrained
language models (such as BERT or GPT) to capture and represent world knowledge in a
computationally efficient manner.
 Transfer Learning: Leveraging pretrained models trained on vast amounts of diverse data to
improve performance on specific tasks by transferring learned knowledge.
 6. Event and Situation Awareness:
 Temporal Knowledge: Representing information about the temporal aspects of events and
situations to understand how knowledge evolves over time.
 Causal Relationships: Modeling cause-and-effect relationships between events and entities in the
world.
7. Dynamic Knowledge Updating:
 Real-time Information Integration: Updating knowledge representations dynamically as new
information becomes available.
 Adaptive Systems: Systems that can adapt and learn from new world knowledge, adjusting their
reasoning processes accordingly.
8. Ethical Considerations:
 Bias Mitigation: Addressing biases present in world knowledge sources to ensure fair and
unbiased decision-making.
 Transparency and Accountability: Ensuring that the knowledge used by AI systems is
transparent and accountable, especially in critical applications.

Incorporating world knowledge into knowledge representation and reasoning systems enhances their capabilities to
understand, reason, and make decisions in a manner that aligns more closely with human-like cognitive processes.
This integration is particularly important for applications where a broad and nuanced understanding of the world is
necessary, such as natural language understanding, intelligent assistants, and decision support systems.

Techniques of knowledge representation

There are mainly four ways of knowledge representation which are given as follows:

1. Logical Representation
2. Semantic Network Representation
3. Frame Representation
4. Production Rules
1. Logical Representation

Logical representation is a language with some concrete rules which deals with propositions and has no ambiguity in
representation. Logical representation means drawing a conclusion based on various conditions. This representation
lays down some important communication rules. It consists of precisely defined syntax and semantics which
supports the sound inference. Each sentence can be translated into logics using syntax and semantics.

Syntax:

o Syntaxes are the rules which decide how we can construct legal sentences in the logic.
o It determines which symbol we can use in knowledge representation.
o How to write those symbols.

Semantics:

o Semantics are the rules by which we can interpret the sentence in the logic.
o Semantic also involves assigning a meaning to each sentence.

Logical representation can be categorised into mainly two logics:

a. Propositional Logics
b. Predicate logics

Advantages of logical representation:

1. Logical representation enables us to do logical reasoning.

 2. Logical representation is the basis for the programming languages.

Disadvantages of logical Representation:

1. Logical representations have some restrictions and are challenging to work with.
2. Logical representation technique may not be very natural, and inference may not be so efficient.

2. Semantic Network Representation

Semantic networks are alternative of predicate logic for knowledge representation. In Semantic networks, we can
represent our knowledge in the form of graphical networks. This network consists of nodes representing objects and
arcs which describe the relationship between those objects. Semantic networks can categorize the object in different
forms and can also link those objects. Semantic networks are easy to understand and can be easily extended.

This representation consist of mainly two types of relations:

a. IS-A relation (Inheritance)

b. Kind-of-relation

Example: Following are some statements which we need to represent in the form of nodes and arcs.

Statements:

a. Jerry is a cat.
b. Jerry is a mammal
c. Jerry is owned by Priya.
d. Jerry is brown colored.
e. All Mammals are animal.
In the above diagram, we have represented the different type of knowledge in the form of nodes and arcs. Each
object is connected with another object by some relation.

Drawbacks in Semantic representation:

1. Semantic networks take more computational time at runtime as we need to traverse the complete network
tree to answer some questions. It might be possible in the worst case scenario that after traversing the entire
tree, we find that the solution does not exist in this network.
2. Semantic networks try to model human-like memory (Which has 1015 neurons and links) to store the
information, but in practice, it is not possible to build such a vast semantic network.
3. These types of representations are inadequate as they do not have any equivalent quantifier, e.g., for all, for
some, none, etc.
4. Semantic networks do not have any standard definition for the link names.
5. These networks are not intelligent and depend on the creator of the system.

Advantages of Semantic network:

1. Semantic networks are a natural representation of knowledge.

2. Semantic networks convey meaning in a transparent manner.
3. These networks are simple and easily understandable.

3. Frame Representation

A frame is a record like structure which consists of a collection of attributes and its values to describe an entity in
the world. Frames are the AI data structure which divides knowledge into substructures by representing stereotypes
situations. It consists of a collection of slots and slot values. These slots may be of any type and sizes. Slots have
names and values which are called facets.

Facets: The various aspects of a slot is known as Facets. Facets are features of frames which enable us to put
constraints on the frames. Example: IF-NEEDED facts are called when data of any particular slot is needed. A frame
may consist of any number of slots, and a slot may include any number of facets and facets may have any number of
values. A frame is also known as slot-filter knowledge representation in artificial intelligence.

Frames are derived from semantic networks and later evolved into our modern-day classes and objects. A single
frame is not much useful. Frames system consist of a collection of frames which are connected. In the frame,
knowledge about an object or event can be stored together in the knowledge base. The frame is a type of technology
which is widely used in various applications including Natural language processing and machine visions.

Example: 1

Let's take an example of a frame for a book

Slots Filters
Title Artificial Intelligence

Genre Computer Science

Author Peter Norvig

Edition Third Edition

Year 1996

Page 1152

Example 2:

Let's suppose we are taking an entity, Peter. Peter is an engineer as a profession, and his age is 25, he lives in city
London, and the country is England. So following is the frame representation for this:

Slots Filter

Name Peter

Profession Doctor

Age 25

Marital status Single

Weight 78

Advantages of frame representation:

1. The frame knowledge representation makes the programming easier by grouping the related data.
2. The frame representation is comparably flexible and used by many applications in AI.
3. It is very easy to add slots for new attribute and relations.
4. It is easy to include default data and to search for missing values.
5. Frame representation is easy to understand and visualize.

Disadvantages of frame representation:

1. In frame system inference mechanism is not be easily processed.

 2. Inference mechanism cannot be smoothly proceeded by frame representation.
3. Frame representation has a much generalized approach.

4. Production Rules

Production rules system consist of (condition, action) pairs which mean, "If condition then action". It has mainly
three parts:

o The set of production rules

o Working Memory
o The recognize-act-cycle

In production rules agent checks for the condition and if the condition exists then production rule fires and
corresponding action is carried out. The condition part of the rule determines which rule may be applied to a
problem. And the action part carries out the associated problem-solving steps. This complete process is called a
recognize-act cycle.

The working memory contains the description of the current state of problems-solving and rule can write knowledge
to the working memory. This knowledge match and may fire other rules.

If there is a new situation (state) generates, then multiple production rules will be fired together, this is called
conflict set. In this situation, the agent needs to select a rule from these sets, and it is called a conflict resolution.

Example:

o IF (at bus stop AND bus arrives) THEN action (get into the bus)
o IF (on the bus AND paid AND empty seat) THEN action (sit down).
o IF (on bus AND unpaid) THEN action (pay charges).
o IF (bus arrives at destination) THEN action (get down from the bus).

Advantages of Production rule:

1. The production rules are expressed in natural language.

2. The production rules are highly modular, so we can easily remove, add or modify an individual rule.

Disadvantages of Production rule:

1. Production rule system does not exhibit any learning capabilities, as it does not store the result of the
problem for the future uses.
2. During the execution of the program, many rules may be active hence rule-based production systems are
inefficient.
Discourse:

One of the primary challenges that we face in the world of Artificial Intelligence is processing Natural
Language data by computers. We can even say that Natural Language Processing is quite a difficult issue in the field
of AI. Now if we are talking about the major problem in Natural Language Processing, then we are talking about the
processing of Discourse in NLP.

So, we can see that the real problem is the processing of the Discourse in NLP, and hence we need to work
on it so that our model can be trained well, which will help in better processing of Natural Language data by the
computers and hence the Artificial Intelligence can predict the desired result.

Now a question that comes to our mind is what is Discourse in NLP? Well, in simple terms, we can say that
discourse in NLP is nothing but coherent groups of sentences. When we are dealing with Natural Language
Processing, the provided language consists of structured, collective, and consistent groups of sentences, which are
termed discourse in NLP. The relationship between words makes the training of the NLP model quite easy and more
predictable than the actual results. Discourse Analysis is extracting the meaning out of the corpus or text. Discourse
Analysis is very important in Natural language Processing and helps train the NLP model better.

Any linguistic unit that consists of multiple sentences Speakers describe “some situation or state of the real
or some hypothetical world” (Webber, 1983) Speakers attempt to get the listener to construct a similar model of the
situation. For natural language understanding: Most information is not contained in a single sentence. The system
has to aggregate information across paragraphs or entire documents. For natural language generation: When systems
generate text, that text needs to be easy to understand and it has to be coherent.

For example, On Monday, John went to Einstein’s. He wanted to buy lunch. But the cafe was closed. That
made him angry, so the next day he went to Green Street instead. Understanding discourse requires (among other
things): 1) doing coreference resolution: ‘the cafe’ and ‘Einstein’s’ refer to the same entity He and John refer to the
same person. That refers to ‘the cafe was closed’. 2) identifying discourse (‘coherence’) relations: ‘He wanted to buy
lunch’ is the reason for ‘John went to Bevande.’

Discourse models an explicit representation of the events and entities that a discourse talks about - the
relations between them (and to the real world). This representation is often written in some form of logic. Discourse
models should capture... Physical entities: John, Einstein’s, lunch Events: On Monday, John went to Einstein’s
involve entities, take place at a point in time States: It was closed. involve entities and hold for a period of time
Temporal relations: afterwards between events and states Rhetorical (‘discourse’) relations: ... so ... instead between
events and states

The most difficult problem of AI is to process the natural language by computers or in other words natural
language processing is the most difficult problem of artificial intelligence. If we talk about the major problems in
NLP, then one of the major problems in NLP is discourse processing − building theories and models of how
utterances stick together to form coherent discourse. Actually, the language always consists of collocated,
structured and coherent groups of sentences rather than isolated and unrelated sentences like movies. These coherent
groups of sentences are referred to as discourse.

Concept of Coherence:

Coherence and discourse structure are interconnected in many ways. Coherence, along with property of
good text, is used to evaluate the output quality of natural language generation system. The question that arises here
is what does it mean for a text to be coherent? Suppose we collected one sentence from every page of the newspaper,
then will it be a discourse? Of-course, not. It is because these sentences do not exhibit coherence. The coherent
discourse must possess the following properties −

Coherence relation between utterances

The discourse would be coherent if it has meaningful connections between its utterances. This property is called
coherence relation. For example, some sort of explanation must be there to justify the connection between utterances.

Relationship between entities

Another property that makes a discourse coherent is that there must be a certain kind of relationship with the entities.
Such kind of coherence is called entity-based coherence.

Discourse structure

An important question regarding discourse is what kind of structure the discourse must have. The answer to this
question depends upon the segmentation we applied on discourse. Discourse segmentations may be defined as
determining the types of structures for large discourse. The segmentation is a difficult thing to implement, but it is
very necessary as discourse segmentation is used in fields like:

 Information Retrieval,
 Text summarization,
 Information Extraction, etc.
Algorithms for Discourse Segmentation:

In this section, we will learn about the algorithms for discourse segmentation. The algorithms are described below:

1. Unsupervised Discourse Segmentation

The class of unsupervised discourse segmentation is often represented as linear segmentation. We can understand
the task of linear segmentation with the help of an example. In the example, there is a task of segmenting the text
into multi-paragraph units; the units represent the passage of the original text. These algorithms are dependent on
cohesion that may be defined as the use of certain linguistic devices to tie the textual units together. On the other
hand, lexicon cohesion is the cohesion that is indicated by the relationship between two or more words in two units
like the use of synonyms.

2. Supervised Discourse Segmentation

The earlier method does not have any hand-labeled segment boundaries. On the other hand, supervised discourse
segmentation needs to have boundary-labeled training data. It is very easy to acquire the same. In supervised
discourse segmentation, discourse marker or cue words play an important role. Discourse marker or cue word is a
word or phrase that functions to signal discourse structure. These discourse markers are domain-specific.

Text Coherence

Lexical repetition is a way to find the structure in a discourse, but it does not satisfy the requirement of being
coherent discourse. To achieve the coherent discourse, we must focus on coherence relations in specific. As we
know that coherence relation defines the possible connection between utterances in a discourse. Hebb has proposed
such kind of relations as follows: We are taking two terms S0 and S1 to represent the meaning of the two related
sentences

Result

It infers that the state asserted by term S0 could cause the state asserted by S1. For example, two statements show the
relationship result: Ram was caught in the fire. His skin burned.

Explanation

It infers that the state asserted by S1 could cause the state asserted by S0. For example, two statements show the
relationship − Ram fought with Shyam’s friend. He was drunk.
Parallel

It infers p(a1,a2,…) from assertion of S0 and p(b1,b2,…) from assertion S1. Here ai and bi are similar for all i. For
example, two statements are parallel − Ram wanted car. Shyam wanted money.

Elaboration

It infers the same proposition P from both the assertions − S0 and S1 For example, two statements show the relation
elaboration: Ram was from Chandigarh. Shyam was from Kerala.

Occasion

It happens when a change of state can be inferred from the assertion of S0, final state of which can be inferred
from S1 and vice-versa. For example, the two statements show the relation occasion: Ram picked up the book. He
gave it to Shyam.

Building Hierarchical Discourse Structure

The coherence of entire discourse can also be considered by hierarchical structure between coherence relations. For
example, the following passage can be represented as hierarchical structure:

 S1 − Ram went to the bank to deposit money.

 S2 − He then took a train to Shyam’s cloth shop.
 S3 − He wanted to buy some clothes.
 S4 − He do not have new clothes for party.
 S5 − He also wanted to talk to Shyam regarding his health
Reference Resolution

Interpretation of the sentences from any discourse is another important task and to achieve this we need to know
who or what entity is being talked about. Here, interpretation reference is the key element. Reference may be
defined as the linguistic expression to denote an entity or individual. For example, in the passage, Ram, the manager
of ABC bank, saw his friend Shyam at a shop. He went to meet him, the linguistic expressions like Ram, His, He are
reference. On the same note, reference resolution may be defined as the task of determining what entities are
referred to by which linguistic expression.

Terminology Used in Reference Resolution:

We use the following terminologies in reference resolution:

 Referring expression − The natural language expression that is used to perform reference is called a
referring expression. For example, the passage used above is a referring expression.
 Referent − It is the entity that is referred. For example, in the last given example Ram is a referent.
 Corefer − When two expressions are used to refer to the same entity, they are called corefers. For
example, Ram and he are corefers.
  Antecedent − The term has the license to use another term. For example, Ram is the antecedent of the
reference he.
 Anaphora & Anaphoric − It may be defined as the reference to an entity that has been previously
introduced into the sentence. And, the referring expression is called anaphoric.
 Discourse model − The model that contains the representations of the entities that have been referred to in
the discourse and the relationship they are engaged in.

Types of Referring Expressions:

Let us now see the different types of referring expressions. The five types of referring expressions are described
below:

1. Indefinite Noun Phrases

Such kind of reference represents the entities that are new to the hearer into the discourse context. For example, in
the sentence Ram had gone around one day to bring him some food − some is an indefinite reference.

2. Definite Noun Phrases

Opposite to above, such kind of reference represents the entities that are not new or identifiable to the hearer into the
discourse context. For example, in the sentence - I used to read The Times of India – The Times of India is a definite
reference.

3. Pronouns

It is a form of definite reference. For example, Ram laughed as loud as he could. The word he represents pronoun
referring expression.

4. Demonstratives

These demonstrate and behave differently than simple definite pronouns. For example, this and that are
demonstrative pronouns.

5. Names

It is the simplest type of referring expression. It can be the name of a person, organization and location also. For
example, in the above examples, Ram is the name-refereeing expression.
 Reference Resolution Tasks:
1. Coreference Resolution

In the Co-reference Resolution, the main aim is to find the referring expression from the provided text that
refers to the same entity. In a discourse in NLP, Co-refer is a term used for an entity if two or more expressions are
referring to the same entity.

For example, Rahul and He is used for the same entity i.e., Rahul.

The Co-reference Resolution can be simply termed as finding the relevant co-refer expressions among the provided
discourse text. Let us take an example for more clarity.

For example, Rahul went to the farm. He cooked food. In this example, Rahul and He is the referring expressions.

We have some sort of constraints present on the Co-reference Resolution. Let us learn about the constraint.

2. Constraint on Coreference Resolution

In English, the main problem for coreference resolution is the pronoun it. The reason behind this is that the pronoun
it has many uses. For example, it can refer much like he and she. The pronoun it also refers to the things that do not
refer to specific things. For example, It’s raining. It is really good.

3. Pronominal Anaphora Resolution

Unlike the coreference resolution, pronominal anaphora resolution may be defined as the task of finding the
antecedent for a single pronoun. For example, the pronoun is his and the task of pronominal anaphora resolution is
to find the word Ram because Ram is the antecedent.

By the terms Pronominal Anaphora Resolution, we are aiming to find the antecedent for the current single
pronoun.

For example, in the passage - Rahul went to the farm. He cooked food., Rahul is the antecedent of the reference He.

Conclusion:

 Discourse in NLP is nothing but coherent groups of sentences. When we are dealing with Natural
Language Processing, the provided language consists of structured, collective, and consistent groups of
sentences, which are termed discourse in NLP.

 Discourse Analysis is very important in Natural language Processing and helps train the NLP model better.
 Coherence in terms of Discourse in NLP means making sense between the utterances or making
meaningful connections and correlations. We use the property of good text, coherence, etc. to evaluate the
quality of the output generated by the natural language processing generation system.

 The extraction of the meaning or interpretation of the sentences of discourse is one of the most important
tasks in natural language processing, and to do so, we first need to know what or who is the entity that we
are talking about.

 Indefinite noun reference is a kind of reference that represents the entity that is new to the discourse
context's hearer.

 Definite noun reference is a kind of reference that represents the entity that is not new to the discourse
context's hearer. The discourse context's hearer can easily identify the definite noun reference.

 In the Co-reference Resolution, the main aim is to find the referring expression from the provided text that
refers to the same entity. By the terms Pronominal Anaphora Resolution, we are aiming to find the
antecedent for the current single pronoun.