INDIAN INSTITUTE OF MANAGEMENT AMRITSAR

Master of Business Administration

Course Outline

Course Code and Natural Language Processing (NLP)

Course Title
Course type Elective
(Regular/Elective)
Pre-requisites (if any) None
Course credit/Units
Total no. of sessions 20
Session Duration 75mins
Term 4
Year and Batch 2023-2025
Sections (if any) Two sections

Instructor(s) Shankar Venkatagiri

Contact Details [email protected]
Office
Consultation Hours
The Bible describes how people of the world, who once spoke a common language, resolved
to build a tower that would reach the heavens. Their plan was foiled by the Lord, who
declared,

“If as one people speaking the same language, they have

begun to do this, then nothing they plan to do will be
impossible for them. Come, let us go down and confuse their
language so they will not understand each other.”
Genesis 11:6

Since then, humans have spread all across the planet, speak a multitude of languages, and
struggle to communicate with one another. Computers and NLP can bridge this gap.

Introduction

This course on Natural Language Processing (NLP) builds an application-oriented

appreciation for the fast-emerging field. NLP has its foundations in linguistics, statistics,
machine learning, and deep learning. At first, we orient the learner towards each aspect.
Then, we illustrate classic NLP tasks such as sentiment analysis, information extraction,
topic modelling, building dialog systems, and social media analytics.

Next, we examine the topic of Gen AI, which has revolutionized the way we have questions
answered, after over two decades of “Googling” for information, and assembling the
responses into something workable. Large language models (LLM) that power ChatGPT and
Bard are the current rage. While their outputs may not be optimal. In the Figure below, there
is a basic fault in the Python code that is generated (Can you spot it?). Regardless, there is
scope for progress with work that has been increasingly published since 2020. The
remainder of the course is divided between business case studies and guest speaker sessions.

There shall be no emphasis on coding. The Instructor will demonstrate concepts with
code in class where needed. All examples shall be circulated among participants, so they can
execute them on their own if they wish to.
Background

Natural language processing (NLP) is a field that harnesses “computational techniques to

learn, understand and produce human language content”. 1 Many business activities
traditionally rely on the spoken word; for instance, customer service calls have been recorded
“for quality purposes”. Automated speech recognition gathered momentum during the
1980s, with the emergence of the Hidden Markov Model. It became feasible to properly
transcribe conversations and analyse them. By the late 1990s, companies like Dragon
Systems had productised continuous dictation; their platform provided welcome relief to
people suffering from disorders like carpal tunnel syndrome.

The core of NLP is linguistics; besides tackling syntax and semantics, NLP models must
address the nuances of ambiguity, common sense, context, sarcasm, and so on. Modern day
LLM models combine machine learning, big data, and AI approaches to realise human-level
accuracy with complex tasks such as speech recognition and answer generation. Voice-based
assistants such as Alexa, Cortana, Ok Google, and Siri have capably exploited all three
aspects of NLP, namely, learning, understanding and producing human language content.
While these advancements are speeding up response times for businesses, they are also
transforming the way we live.

NLP powers ground-breaking innovations. During Google I/O 2022, CEO Sundar Pichai
showcased several NLP-driven platforms.2 A novel, mono-lingual approach has allowed the
company to translate a large set of low resource (LR) Indian languages such as Assamese,
Bhojpuri, Konkani, Maithili, Manipuri, and Sanskrit.

Wearing a Glass (see Fig 1), a sighted person can see what is being said in real-time; the
words can also be translated into the user’s native tongue. This opens up a world of
opportunities for businesses that are focused on inclusion. In the near future, education
may be fully transacted in one’s native language.

Voice-powered apps on the phone use synthesised speech to establish a critical connect with
the user for handy tasks like route navigation. Google Maps have helped drivers tangibly
reduce carbon emissions by nudging them to choose a slightly slower but more energy-
efficient route. Automated summarisation of lengthy text documents is possible with Google
Docs. YouTube has begun to incorporate auto-generated chapters.

1
Hirschberg, J., & Manning, C. D. (2015). Advances in natural language processing. Science, 349(6245), 261-266
2
Watch the entire event at https://www.youtube.com/watch?v=nP-nMZpLM1A
We live in a “renaissance era” of automation that is fuelled by easy access to training data,
elastic computation platforms, and huge investments from governments and businesses.
There is a growing emphasis on inclusivity; speech-to-text technologies from Google are
bringing different sections of society together by enabling communication between them.
Hugging Face has open sourced pre-trained models like BERT to automate NLP tasks such
as MT and sentiment analysis. DALL.E from OpenAI and Midjourney takes creativity to the
next level, by generating imagery from a supplied description.

Learning Outcomes/Course Objectives

The course is designed with the following specific objectives and learning outcomes:

a. Explore the field of NLP with a systematic knowledge of the foundations

b. Develop a hands-on feel for key NLP tasks: sentiment analysis, topic modelling,
building a chatbot, analysing customer reviews, and so on.
c. Examine applications of NLP to healthcare, retail, and social media analytics
d. Build a strategic understanding of NLP-powered businesses via case studies

Text Book(s)

Practical Natural Language Processing: A Comprehensive Guide to Building Real-World NLP

Systems (Greyscale Indian Edition)

Authors: Sowmya Vajjala , Bodhisattwa Majumder , et al.

Published: 17 July 2020

Reference Book(s)

 Aggarwal, C. C. (2015). Mining text data. In Data mining (pp. 429-455). Springer,
Cham.
 Bengfort, B., Bilbro, R., & Ojeda, T. (2018). Applied text analysis with python:
Enabling language-aware data products with machine learning. O'Reilly Media,
Inc.
 Hagiwara, M. (2021). Real-World Natural Language Processing: Practical
applications with deep learning. Simon and Schuster.
 Hapke, H. M., Lane, H., & Howard, C. (2019). Natural language processing in action.
Manning
 Rao, D., & McMahan, B. (2019). Natural language processing with PyTorch: build
intelligent language applications using deep learning. O'Reilly Media, Inc.
 Russell, M. A. (2013). Mining the social web: data mining Facebook, Twitter,
LinkedIn, Google+, GitHub, and more. O'Reilly Media, Inc.
 Thomas, A. (2020). Natural Language Processing with Spark NLP: Learning to
Understand Text at Scale. O'Reilly Media, Inc.
 Tunstall, L., von Werra, L., & Wolf, T. (2022). Natural Language Processing with
Transformers. O'Reilly Media, Inc.
 Vasiliev, Y. (2020). Natural Language Processing with Python and spaCy: A
Practical Introduction. No Starch Press.

Additional Reading(s)
Specified against the topics.

Pedagogy Used/Learning Process

The course employs a mix of lecture and hands-on exploration in class.
Evaluation Components/Assessment of Student Learning

Assessment Tool Percentage Description

Quizzes (2) 30 These are required to check and
evaluate the students’ understanding
of concepts and application of tools
and techniques.
Endterm Exam 50 Exam
Project 20 Students are expected to work in
teams as it would provide them
opportunities to explore diverse
perspectives for a business problem.
Session Plan

Session Topic
1-2 The Landscape of Natural Language Processing

We discuss the history and evolution of NLP. The initial decades saw
the rise of heuristic approaches such as regular expression matches,
word frequencies, etc. Forensic “fingerprints” helped capture the
Unabomber. Starting with the 1990s, NLP began to benefit from
developments in machine learning and AI. We explore the landscape of
commercial offerings powered by NLP.

NLP today is changing the way we perform routine tasks. Dictating

emails and SMS messages rather than typing in text has become
routine. Large language models (LLMs) such as ChatGPT, introduced
in late 2022, are revolutionising the way we compose essays, debug
programs, and accomplish several other tasks.

Textbook: Chapter 1 – NLP: A Primer

Readings:
1. Dale, R. NLP commercialisation in the last 25 years. Medium
(2019)
2. Dave Davies. (2017). FBI Profiler says Linguistic Work was
Pivotal in capture of Unabomber. npr.org

3 Python primer

Python shall be the lingua franca for all of our demonstrations

because there are free open-source libraries that support NLP at
industry scale. The emphasis will be on running (rather than writing)
Python code, so anyone can get a hands-on feel for concepts that are
discussed during the course. During this course, we will execute code
inside a browser with Google Colab, which needs no installation. We
end the session by examining a chatbot.

4 Linguistics orientation

We begin the session by developing a foundational understanding of

language, context, and common knowledge. NLP is also called
computational linguistics. Using a text-processing pipeline, we
discuss the initial stages of data acquisition, text extraction, cleaning,
and augmentation. We use NLTK and spaCy libraries to perform
pre-processing tasks such as tokenisation, stemming, lemmatising,
part-of-speech (POS) tagging, and dependency labelling.

A naïve way to analyse a text document is to regard it as a Bag of

Words (BoW), free of context. We conclude by learning how to
represent text as vectors so we can apply ML algorithms.

Textbook: Chapter 2 – NLP Pipeline, Chapter 3 – Text Representation

5-6 Machine Learning orientation

The first session goes over the basics of machine learning (ML). We
contrast the programmatic approach to derive regression coefficients,
with one that uses AI. We conclude by illustrating some supervised
and unsupervised models.

Classification plays an important role in NLP tasks. The second

session goes over classification models such as Naïve Bayes, SVM, and
logistic regression. We apply them to a problem of sentiment
analysis.

7 Case Discussion: Tamarin App

Srikant Datar & Caitlin Bowler (2018). HBS Press.

The case highlights the application of a Naïve Bayes model in text

classification, which supports tasks such as spam detection and multi-
class topic identification. In addition, we examine text classifiers based
on SVM and Logistic Regression.

Textbook: Chapter 4 – Text Classification

8-9 Deep Learning orientation

Deep learning (DL) methods facilitate complex tasks like machine

translation. Neural networks form the foundations of DL. In the
first session, we describe feedforward nets, CNNs, and RNNs. Using
the TensorFlow library, we perform sentiment analysis of IMDb
reviews and categorise Reuters newswires.

The distributional hypothesis posits that vectors representing words

that occur in the same context should be close to each other.
Embeddings help reduce the high dimensionality of the vector space.
The Word2vec model “learns” to represent words in a corpus using a
two-layer neural network. We conclude with a discussion of classifiers
governed by neural embeddings.

10 Guest Session: NLP in Healthcare

We listen to an industry expert on the possibilities opened up by NLP-

powered analytics of electronic health records (EHR).

11 Information Extraction

Humans are driven by basic questions such as who, what, where, when
and why. Named entities form the answers to such questions, and
NE recognition (NER) plays a key role in extracting relevant
information from unstructured text, which forms the bulk of business
transactions. NER is the starting step for extracting keywords,
temporal information, event and relation. After going over the basics,
we apply transformers to perform NER and build a Question-
Answering system.

Textbook: Chapter 5 – Information Extraction

12 Case Discussion: Taiger AI: Unbundling the business value of NLP
Seema Chokshi, Lipika Bhattacharya & Wee-Kiat
Lym (2021)
Singapore Management University

While NLP helps build nifty applications, monetising them comes with
its set of challenges. The case explains how an entrepreneur identified
a gap in the market and created a suite of NLP offerings under a
unified platform called Omnitive. The case is focused on strategic
aspects of the business model.

13 Spoken Dialog Systems

The quality of customer service is a key differentiator in a competitive

market. For decades, customer complaints were registered using
phone-based interactive voice response (IVR) systems, which were
rudimentary and incapable of acting on nuanced inputs. Recent
advances in signal processing and deep learning have vastly improved
speech recognition, natural language understanding (NLU), dialog
management (DM), response generation, and speech synthesis; these
steps are at the heart of chatbots.

While chatbots have ready application to business settings, the greater

context of dialog management (DM) merits further attention. In this
session, we go over dialog systems, and build a conversational chatbot
using Google Dialogflow.

Textbook: Chapter 6 – Chatbots

14 Topic Modelling

Retrieving documents that match a query is critical to many

businesses. Instead of assessing a term by its raw frequency of
occurrence, the TF-IDF score makes better sense by factoring in its
relevance to the corpus.
We show how principal component analysis (PCA) can assist with
semantic analysis, and apply it to classify SMS messages as spam.
Latent semantic analysis (LSA) produces a minimal representation of
text, and facilitates topic modelling.

Textbook: Chapter 7 – Topics in Brief

15 State of the art with Transformers

The transfer learning mechanism expedited the construction of

vision applications by harnessing large, pre-trained CNN models like
ResNet. In 2017, research groups made this idea work for NLP with the
Transformer architecture. BERT and GPT rely on this mechanism.

In this session, we go over the encoder-decoder architecture, and then

show how to use the transformers in the Hugging Face ecosystem to
accomplish NLP tasks such as machine translation, text
summarisation and text generation.

Article: Robert A.Gonzalves. AI-Memer: Using Machine Learning to

 Create Funny Memes. TowardsDataScience (2021)

16 Guest Session: Large Language Models

17 Case Discussion: Amazon Alexa and Patient Engagement.

Kevin Schulman & Stacy Wood (2019) Stanford

As healthcare moves to the centre stage of government agendas, it is

clear that preventive measures are beneficial to the patients and saves
money for the providers. The case goes over the uses of Amazon
Voice Service to engage better with patient populations. We also
discuss use cases in pharmacovigilance, drug discovery, outcome
prediction, and information extraction with Amazon Comprehend.

Textbook: Chapter 10 – Healthcare, Finance and Law

18 Case Discussion: Twiggle: e-Commerce with Semantic Search.

Shane Greenstein & Danielle Golan (2019). HBSP.

Searching on e-Commerce sites relies on entering the right keywords,

and tends to generate a generic set of results. Thanks to NLP, queries
can be made precise and meaningful. The case describes the creation of
a search tool based on query semantics, and explores opportunities
and challenges to market this platform. We also discuss the prospects
for analysing product reviews, and connect ratings with aspects about
the product.

Textbook: Chapter 9 – E-Commerce and Retail

19 Social Media Analytics

NLP finds a plethora of applications in social media: trending topic

detection, opinion mining, fake news detection, adult content filtering,
customer support. There are unique challenges given that the content
is human generated, such as misspelling, emojis and mixed language.
After going over a few examples, we end with a primer on processing
tweets.

Textbook: Chapter 8 – Social Media

20 What the future holds…

Products we did not expect to materialise are being engineered at a

steady clip. There are ethical issues to consider. On the creative front,
Open AI’s Sora enables users to generate a video from starter text,
with applications to movies. However, the same mechanism could be
used to falsify information and cause harm. Summarising a long text
document is useful when the output is meaningful. On the flip side,
writing a convincing essay on any topic can now be automated. With
ChatGPT

The final session is dedicated to the prospects that the future holds.

Reading: Hirschberg, J., & Manning, C. D. (2015). Advances in

natural language processing. Science, 349 (6245), 261-266
21 If time permits: Scaling and deploying NLP applications

Businesses solicit and process large volumes of text data, such as

reviews and complaints. A contemporary approach to scale an
operation is to parallelise it in a distributed manner; the Spark NLP
library executes models in memory. In this session, we describe
features of this library, and apply it to a problem in NER.

We conclude by discussing best practices for developing NLP

applications, as well as deploying them.

Reading: Kocaman, V. Introduction to Spark NLP.

TowardsDataScience (2019)

Website: John Snow Labs