MIZAN TEPI UNIVERSITY

SCHOOL OF COMPUTING AND INFORMATICS

DEPARTMENT OF SOFTWARE ENGINEERING
Artificial Intelligence

CHAPTER - 6
Natural Language Processing (NLP)
Basics

1 By: Melkamu D.
 Introduction to Natural Language Processing(NLP)
2

 Natural Language Processing (NLP): refers to the AI method of

communicating with an intelligent system using a natural language (Human
Language) such as Amharic, or English.
 The process of computer analysis of input provided in a human language
(natural language), and conversion of this input into a useful form of
representation.
 NLP is primarily concerned with getting computers to perform useful and
interesting tasks with human languages.
 NLP is secondarily concerned with helping us come to a better
understanding of human language.
3
Cont.…
 Processing of Natural Language is required when you want an intelligent system
like a robot to perform as per your instructions, when you want to hear decisions
from a dialogue-based clinical expert system, etc.
 The field of NLP involves making computers perform useful tasks with the
natural languages humans use. The input and output of an NLP system can
be :
 Speech
 Written Text
 Components of NLP
4

 There are two components of NLP as given:

 Natural Language Understanding (NLU)
 Understanding involves the following tasks:
 Mapping the given input in natural language into useful representations.
 Analyzing different aspects of the language.
 Natural Language Generation (NLG)
 It is the process of producing meaningful phrases and sentences in the form
of natural language from some internal representation. It involves:
 Text planning: It includes retrieving the relevant content from the knowledge
base.
 Cont.….
5

 Sentence planning: It includes choosing required words, forming meaningful

phrases, and setting the tone of the sentence.
 Text Realization: It is mapping sentence plan into sentence structure.
 The NLU is harder than NLG.
 Difficulties in NLU
 NL has an extremely rich form and structure.
 It is very ambiguous. There can be different levels of ambiguity:
 Lexical ambiguity: It is at a very primitive level such as word level.
 For example, treating the word “board” as a noun or verb?
 Cont.…
6

 Syntax Level ambiguity: A sentence can be parsed in different ways.

 For example, “He lifted the beetle with red cap.” − Did he use cap to lift the
beetle or he lifted a beetle that had red cap?
 Referential ambiguity: Referring to something using pronouns.
 For example, Rima went to Gauri. She said, “I am tired.” − Exactly who is
tired?
 One input can mean different meanings.
 Many inputs can mean the same thing.
 NLP Terminology
7

 Phonology: It is study of organizing sound systematically.

 Morphology: It is a study of construction of words from primitive meaningful units.

 Morpheme: It is primitive unit of meaning in a language.

 Syntax: It refers to arranging words to make a sentence. It also involves determining the structural role of words
in the sentence and in phrases.

 Semantics: It is concerned with the meaning of words and how to combine words into meaningful phrases and
sentences.

 Pragmatics: It deals with using and understanding sentences in different situations and how the interpretation of
the sentence is affected.

 Discourse: It deals with how the immediately preceding sentence can affect the interpretation of the next
sentence.

 World Knowledge: It includes the general knowledge about the world.

 Speech Recognition
8

 Speech is the dominant modality for communication between

humans and promises to be important for communication between
humans and machines.
 Speech recognition: is the task of mapping from a digitally
encoded acoustic signal to a string of words.
 Speech understanding: is the task of mapping from the acoustic signal all
the way to an interpretation of the meaning of the word.
 Steps in NLP
9

 There are general five steps:

 Lexical Analysis: It involves identifying and analyzing the structure of words.
 The lexicon of a language means the collection of words and phrases in a
language. Lexical analysis is dividing the whole chunk of text into
paragraphs, sentences, and words.
 Syntactic Analysis (Parsing): It involves the analysis of words in the sentence
for grammar and arranging words in a manner that shows the relationship
among the words.
 A sentence such as “The school goes to a boy” is rejected by an English
syntactic analyzer.
 Cont.…
10

 Semantic Analysis: It draws the exact meaning or the dictionary meaning from
the text. The text is checked for meaningfulness. It is done by mapping syntactic
structures and objects in the task domain. The semantic analyzer disregards
sentences such as “hot ice cream”.
 Discourse Integration: The meaning of any sentence depends upon the
meaning of the sentence just before it. In addition, it also brings about the
meaning of the immediately succeeding sentence.
 Pragmatic Analysis: During this, what was said is re-interpreted on what it
actually meant. It involves deriving those aspects of language which require
real-world knowledge.
Cont.….
 Machine Learning Application in NLP
12

 Machine Learning is an application of AI that provides systems the ability to

automatically learn and improve from experience without being explicitly
programmed.
 Machine Learning can be used to help solve AI problems and to improve NLP
by automating processes and delivering accurate responses.
 Machine Learning Applications of NLP
13

 Machine Translation
 Database Access

 Information Retrieval

 Selecting from a set of documents the ones that are relevant to a query

 Text Categorization
 Sorting text into fixed topic categories
 Extracting data from text
 Converting unstructured text into structured data
 Spoken language control systems
 Spelling and grammar checkers
14
Natural Language Interaction (NLI)
 Natural Language Interaction (NLI): enables people to interact with any
connected device using normal, everyday language.
 It understands the meaning of conversational input, and reacts accordingly,
creating value and enhancing the user experience.
 Responds in an Intelligent, Humanlike Way to Every Conversation
 With NLI, users can converse with technology using complex sentences,
containing multiple pieces of information and more than one request.
 There is no need for the user to repeat details during a conversation because
memory, personal preferences, and contextual understanding comes into play,
just as they would in a conversation with a real person.
 Cont.….
15

 Speak Your Customer’s Language

 George Bernard Shaw once said that England and America were two nations separated by a
common language.
 You only have to consider the differences between the words boot and trunk, to see how
confusing this might be to a machine.
 Natural Language Interaction removes the need for your customers to know and understand
your terminology.
 It’s clever enough to figure out in over 20 languages what someone means when they use
their everyday words and phrases, not yours.
 Create a Richer Experience
 Using NLI, Artificial Solutions takes natural language to the next level.
 NLI development and analytics platform Teneo allows you to deliver sophisticated NLI
solutions over mobiles, PCs, wearables, consumer devices and appliances, applications and
so much more to deliver a compelling customer experience that surpasses any other
technology available today.
 Cont.…..
16

 Deeper Understanding Brings its own Rewards

 Natural Language Interaction allows you to discover more about your customer and your business.
By taking unstructured, natural language data from all of an enterprise’s sources such as web,
mobile, social, and live chat, conversations can be analyzed in the context in near real-time.

 This provides deep insight into the mind of your customer.

 It can help with back office tasks such as improving answers given or discovering why a customer
didn’t complete their purchase online to transforming individual interactions by cross-selling or
offering personalized deals at a critical moment.

 Natural Language Interaction gives you the power to use the information your customer provides to
deliver a superior customer experience and help increase your bottom line.
 17
Computer vision and Image processing
 What is Image Processing?
 As the phrase "image processing" suggests, this technique involves "processing" an image.

 It denotes that an input file has been subjected to at least one alteration. Additionally, a person with the
help of some specialized software can accomplish this task (to name just Photoshop, InDesign, GIMP,
Gravit, CorelDRAW, and many more).

 Some of the modifications are carried out by themselves automatically. Such modifications include
contrast enhancement, edge detection, sharpening, and filtering.

 There is no human involvement at any point in any of these processes. It is sufficient for a graphic to
initiate a specific activity.
 Cont.…

 Resizing, stretching, improving, and adding new layers or texts are examples of the
types of transformations that fall under the category of manual work.

 These procedures necessitate a great deal more focus and activity on the part of the
graphic.

 In image processing, you begin with an image X, process it, and then you end up
with an image Y as a result of your work.

 Setting the goal, and the problem that needs to be solved all play a role in
determining what kinds of changes are required.
 Cont.…
 What is Computer Vision?
 The primary objective of computer vision is to derive information from the
pictures or videos that are used as input in order to have an accurate grasp of the
data and to anticipate the visual data in the same way that the human brain does.

 The objective is to make sense of the picture and what it depicts.

 Although some of the image processing algorithms might be utilized in the

process of computer vision problem solving, the processing of images is never the
major focus.
 Cont.……
 In reality, the procedures of image processing are utilized in order to accomplish the
responsibilities of computer vision.
 The automotive industry is home to one of the most significant applications of
computer vision today.
 In this scenario, computer vision serves as an assistant for the driver, which is
especially helpful when the weather is terrible.
 It examines the area around the vehicle and looks for potential dangers, impediments,
and other important circumstances that require a reaction from the driver while they are
driving. One example of such a situation is a person crossing the street.
 Applications of Computer Vision and Image Processing
 Computer Vision in the Automotive Sector
 Computer Vision can assist you in remaining awake and in determining whether or not you
are too exhausted to safely operate a vehicle.
 The program that uses Computer Vision can continuously check your condition by analyzing
the status of your eyes or the movements of your head.
 Computer Vision in the Production Industry
 Computer vision is utilized by Pharma Packaging Systems in order to carry out automated
counting of tablets and capsules on production lines.
 Computer Vision in Athletics and Fitness
 Image Processing in Medical Imaging
 Image Processing in the Search for the Missing
 Comparison between Image Processing and Computer Vision
Basis of Comparison Image Processing Computer Vision

Processing the raw images that are entered into the system The primary objective of computer vision is
in order to improve them or get them ready for usage in to derive information from the pictures or
other applications is the primary emphasis of image videos that are used as input in order to have
Definition
processing. an accurate grasp of the data and to
anticipate the visual data in the same way
that the human brain does.
Methods such as anisotropic diffusion, hidden Markov Image processing is just one of the many
models, independent component analysis, different techniques that are employed in computer
Applicable methods
filtering, and many more are utilised throughout the image vision; other approaches, such as machine
processing process. learning, CNN, and so on are also utilised.
The field of Computer Vision includes Image Processing as Image processing is the subfield that
Function
one of its subfields. Computer Vision falls under.
Some applications of image processing include rescaling Object detection, face detection, handwriting
the image (also known as digital zoom), correcting the recognition, and other similar tasks are all
Applications
illumination, and changing the tones, among other things. examples of applications that computer
vision can perform.
 Summery
 The methods that are used in Image Processing can alter images in a variety of ways,
including sharpening, smoothing, filtering, enhancing, restoring, and blurring amongst
others.
Computer vision, on the other hand, is concerned with deciphering the meaning of
what may be seen by computers.
 A computer vision system receives an image as an input and generates images as an
output based on a particular job, such as the labeling of objects and their coordinates.
 Both of them collaborate in many different situations; in fact, many computer vision
systems depend on image processing techniques to function properly.
 Image processing entails the processing of raw input images, as well as the
enhancement of those images or the preparation of those images to carry out certain
activities.
24