NATURAL LANGUAGE PROCESSING

INTRODUCTION
NLP is used to analyze text, allowing machines to understand how
humans speak. This human-computer interaction enables real-
world applications like automatic text summarization, sentiment
analysis, topic extraction, named entity recognition, parts-of-
speech tagging, relationship extraction, stemming, and more.

Natural language processing (NLP) refers to the branch of

computer science—and more specifically, the branch of artificial
intelligence or AI—concerned with giving computers the ability to
understand text and spoken words in much the same way human
beings can.

WHAT IS NATURAL LANGUAGE PROCESSING?

Natural language processing studies interactions between humans
and computers to find ways for computers to process written and
spoken words similar to how humans do. The field blends
computer science, linguistics and machine learning
Natural language processing has heavily benefited from recent
advances in machine learning, especially from deep learning
techniques. The field is divided into the three parts:
 Speech recognition — the translation of spoken language into text.
 Natural language understanding — a computer’s ability to
understand language.
 Natural language generation — the generation of natural
language by a computer.
NATURAL LANGUAGE GENERATION
Natural language generation (NLG) is the use of artificial intelligence (AI)
programming to produce written or spoken narratives from a data set.
NLG is related to human-to-machine and machine-to-human interaction,
including computational linguistics, natural language processing (NLP)
and natural language understanding (NLU).
How NLG works
NLG is a multi-stage process, with each step further refining the data
being used to produce content with natural-sounding language. The six
stages of NLG are as follows:

1. Content analysis. Data is filtered to determine what should be included

in the content produced at the end of the process. This stage includes
identifying the main topics in the source document and the relationships
between them.
2. Data understanding. The data is interpreted, patterns are identified and
it's put into context. Machine learning is often used at this stage.
3. Document structuring. A document plan is created and a narrative
structure chosen based on the type of data being interpreted.
4. Sentence aggregation. Relevant sentences or parts of sentences are
combined in ways that accurately summarize the topic.
5. Grammatical structuring. Grammatical rules are applied to generate
natural-sounding text. The program deduces the syntactical structure of
the sentence. It then uses this information to rewrite the sentence in a
grammatically correct manner.
6. Language presentation. The final output is generated based on a
template or format the user or programmer has selected.

HISTORY OF NLP
The history of natural language processing describes the advances
of natural language processing (Outline of natural language processing).
There is some overlap with the history of machine translation, the history
of speech recognition, and the history of artificial intelligence.
Research and development
The history of machine translation dates back to the seventeenth century,
when philosophers such as Leibniz and Descartes put forward proposals
for codes which would relate words between languages. All of these
proposals remained theoretical, and none resulted in the development of
an actual machine.
The first patents for "translating machines" were applied for in the mid-
1930s. One proposal, by Georges Artsrouni was simply an automatic
bilingual dictionary using paper tape. The other proposal, by Peter
Troyanskii, a Russian, was more detailed. It included both the bilingual
dictionary, and a method for dealing with grammatical roles between
languages, based on Esperanto.
In 1950, Alan Turing published his famous article "Computing Machinery
and Intelligence" which proposed what is now called the Turing test as a
criterion of intelligence. This criterion depends on the ability of a
computer program to impersonate a human in a real-time written
conversation with a human judge, sufficiently well that the judge is unable
to distinguish reliably — on the basis of the conversational content
alone — between the program and a real human.
In 1957, Noam Chomsky’s Syntactic Structures revolutionized Linguistics
with 'universal grammar', a rule based system of syntactic structures.[1]
The Georgetown experiment in 1954 involved fully automatic translation
of more than sixty Russian sentences into English. The authors claimed
that within three or five years, machine translation would be a solved
problem.[2] However, real progress was much slower, and after
the ALPAC report in 1966, which found that ten years long research had
failed to fulfill the expectations, funding for machine translation was
dramatically reduced. Little further research in machine translation was
conducted until the late 1980s, when the first statistical machine
translation systems were developed.
Some notably successful NLP systems developed in the 1960s
were SHRDLU, a natural language system working in restricted "blocks
worlds" with restricted vocabularies.
In 1969 Roger Schank introduced the conceptual dependency theory for
natural language understanding.[3] This model, partially influenced by the
work of Sydney Lamb, was extensively used by Schank's students at Yale
University, such as Robert Wilensky, Wendy Lehnert, and Janet Kolodner.
In 1970, William A. Woods introduced the augmented transition
network (ATN) to represent natural language input.[4] Instead of phrase
structure rules ATNs used an equivalent set of finite state automata that
were called recursively. ATNs and their more general format called
"generalized ATNs" continued to be used for a number of years. During
the 1970s many programmers began to write 'conceptual ontologies',
which structured real-world information into computer-understandable
data. Examples are MARGIE (Schank, 1975), SAM (Cullingford, 1978),
PAM (Wilensky, 1978), TaleSpin (Meehan, 1976), QUALM (Lehnert,
1977), Politics (Carbonell, 1979), and Plot Units (Lehnert 1981). During
this time, many chatterbots were written including PARRY, Racter,
and Jabberwacky.
Up to the 1980s, most NLP systems were based on complex sets of hand-
written rules. Starting in the late 1980s, however, there was a revolution in
NLP with the introduction of machine learning algorithms for language
processing. This was due both to the steady increase in computational
power resulting from Moore's Law and the gradual lessening of the
dominance of Chomskyan theories of linguistics (e.g. transformational
grammar), whose theoretical underpinnings discouraged the sort of corpus
linguistics that underlies the machine-learning approach to language
processing.[5] Some of the earliest-used machine learning algorithms, such
as decision trees, produced systems of hard if-then rules similar to existing
hand-written rules. Increasingly, however, research has focused
on statistical models, which make soft, probabilistic decisions based on
attaching real-valued weights to the features making up the input data.
The cache language models upon which many speech recognition systems
now rely are examples of such statistical models. Such models are
generally more robust when given unfamiliar input, especially input that
contains errors (as is very common for real-world data), and produce more
reliable results when integrated into a larger system comprising multiple
subtasks.
Many of the notable early successes occurred in the field of machine
translation, due especially to work at IBM Research, where successively
more complicated statistical models were developed. These systems were
able to take advantage of existing multilingual textual corpora that had
been produced by the Parliament of Canada and the European Union as a
result of laws calling for the translation of all governmental proceedings
into all official languages of the corresponding systems of government.
However, most other systems depended on corpora specifically developed
for the tasks implemented by these systems, which was (and often
continues to be) a major limitation in the success of these systems. As a
result, a great deal of research has gone into methods of more effectively
learning from limited amounts of data.
Recent research has increasingly focused on unsupervised and semi-
supervised learning algorithms. Such algorithms are able to learn from
data that has not been hand-annotated with the desired answers, or using a
combination of annotated and non-annotated data. Generally, this task is
much more difficult than supervised learning, and typically produces less
accurate results for a given amount of input data. However, there is an
enormous amount of non-annotated data available (including, among other
things, the entire content of the World Wide Web), which can often make
up for the inferior results.
History of Natural Language Processing?

History of natural language processing are described:

The Beginning 
As stated above the idea had emerged from the need for Machine
Translation in the 1940s. Then the original language was English and
Russian. But the use of other words such as Chinese also came into
existence in the initial period of the 1960s. Then a lousy era came for
MT/NLP during 1966, this fact was supported by a report of ALPAC,
according to which almost died because the research in this area did not
have the pace at that time. This condition became better again in the 1980s
when the product related to it started providing some results to customers.
After reaching in dying state in the 1960s, the got a new life when the idea
and need of Artificial Intelligence emerged
LUNAR is developed in 1978 by W.A woods; it could analyze, compare
and evaluate the chemical data on a lunar rock and soil composition that
was accumulating as a result of Apollo moon missions and can answer the
related question. In the 1980s the area of computational grammar became
a very active field of research which was linked with the science of
reasoning for meaning and considering the user ‘s beliefs and intentions.
In the period of 1990s, the pace of growth of it increased. Grammars, tools
and Practical resources related to it became available with the parsers.
The research on the core and futuristic topics such as word sense
disambiguation and statistically colored NLP, the work on the lexicon got
a direction of research. This quest of the emergence of it was joined by
other essential topics such as statistical language processing, Information
Extraction and automatic summarising.

The First Chatbot - ELIZA

The discussion on the history cannot be considered complete without the
mention of the ELIZA, a chatbot program which was developed from
1964 to 1966 at the Artificial Intelligence Laboratory of MIT. It was
created by Joseph Weizenbaum. It was a program which was based on
script named as DOCTOR which was arranged to Rogerian
Psychotherapist and used rules, to response the questions of the users
which were psychometric-based. It was one of the chatbots which were
capable of taking the Turing test at that time.

What are the applications of Natural Language Processing?

Nowadays everybody wants the machine to talk, and the only way by
which a computer can speak is through it. Take the example of Alexa, a
 conversational product by Amazon. A query is passed to it by the medium
of voice, and it can reply by the same medium, i.e., voice. It can be used to
ask anything, search for anything, for playing songs or even for cab
booking. It seems to be magic, but it is not because of any magic spell, see
the below diagram. This simple diagram is the demonstration of the
procedure of Natural Language Processing in Alexa.
Alexa is not a single example, and these talking machines which are
popularly known as Chatbot can even manage complicated interactions
and the processes related to the streamlined business using it only. In the
past chatbot were utilized for only customer interaction with limited
capabilities of conversation because they were generally rule-based but
after the emergence of Natural Language processing and its integration
with Machine Learning and Deep Learning, now chatbot can handle many
different areas such Human Resources and Health. This is not the only use
case of it where it emerges as a game changer; there are other examples
also. Let’s have a brief look at them. Below is the description of some use
cases.
 Health Care
 Sentiment Analysis 
 Cognitive Analytics 
 Spam Detection 
 Recruitment
 Conversational Framework

Linguistic Knowledge in Natural Language Processing

Ever since diving into Natural Language Processing (NLP), I’ve always
wanted to write something rather introductory about it at a high level, to
provide some structure in my understanding, and to give another
perspective of the area — in contrast to the popularity of doing NLP using
Deep Learning.

Stages of analysis in NLP

NLP Pyramid — Coursera Natural Language Processing

Given a sentence, traditionally the following are the different stages on

how a sentence would be analyzed to gain deeper insights.
1. Morphology

At this stage we care about the words that make up the sentence, how they
are formed, and how do they change depending on their context. Some
examples of these include:

 Prefixes/suffixes

 Singularization/pluralization

 Gender detection

 Word inflection (modification of word to express different grammatical

categories such tenses, case, voice etc..). Other forms of inflection
includes conjugation (inflection of verbs) and declension (inflection of
nouns, adjectives, adverbs etc…).

 Lemmatization (the base form of the word, or the reverse of inflection)

 Spell checking

2. Syntax (Parsing)

In this stage, we focus more on the relationship of the words within a

sentence — how a sentence is constructed.

In a way, syntax is what we usually refer to as grammar — NLP For

Hackers

To derive this understanding, syntactical analysis is usually done at a

sentence-level, where as for morphology the analysis is done at word level.
When we’re building dependency trees or processing parts-of-speech —
we’re basically analyzing the syntax of the sentence.

Dependency tree of a sentence — Spacy.io

3. Semantics

Once we’ve understood the syntactic structures, we are more prepared to

get into the “meaning” of the sentence (for a fun read on what meaning can
actually mean in NLP — head over here to dive into a Twitter discussion
on the subject ).

In the above example from Spacy for example; Google, Apple and
Amazon has been tagged as an organization based on it’s NER annotator.
While Google and Amazon is a straightforward case, Apple is rather
different since it could either be a fruit or a the company — Apple. In this
case, the result given is based off of a prediction based on statistical
analysis from trained dataset.

4. Pragmatics

At this level, we try to understand the text as a whole. Popular problems

that we’re trying to solve at this stage are:
1. Topic modelling

2. Coreference/Anaphora

3. Summarization

4. Question & Answering

Summary

NLP as an area of study isn’t something that’s actually new. And if one is
already well versed in Deep Learning, one could also be forgiven to think
that traditional NLP methods (ie. breaking texts into smaller structures,
parsing, annotate meanings) are no longer needed.

However in cases where labelled data are hard to come by, and high
precision is needed (for example when one is hard pressed for time with
a tight deadline and the results really need to be accurate else the
product wouldn’t be launched) — is where traditional NLP really shines.

Understanding the above topics aren’t really something that is needed

for one to be able to solve NLP-related tasks. For me, it’s more of a way
to put into perspective where does the different problems that I’ve
solved previously belong to and what are the other gaps or areas where I
could improve on.

Disclaimer: In writing this I did a lot of read up on Coursera’s NLP

course, Wikipedia, and my latest favourite website for NLP materials
— NLP for Hackers. Most likely some of the stuff you see here are a
mishmash between those sources, so be forewarned that I might forget
to properly reference their content where needed

REFRENCES

1. https://www.ibm.com/in-en/topics/natural-
language-processing
2. https://www.datarobot.com/blog/what-is-natural-
language-processing-introduction-to-nlp/
3. https://builtin.com/data-science/introduction-nlp
4. https://www.techtarget.com/searchenterpriseai/
definition/natural-language-generation-NLG
5. https://www.xenonstack.com/blog/evolution-of-nlp
6. [1] Natural Language Processing | Coursera

[2] Wikipedia, the free encyclopedia

[3] Natural Language Processing — Introduction —

NLP-FOR-HACKERS

7. [4] Linguistic Features, Spacy.io