Natural Language Processing In The Real World

Text Processing Analytics And Classification 1st

Edition Jyotika Singh download

https://ebookbell.com/product/natural-language-processing-in-the-
real-world-text-processing-analytics-and-classification-1st-
edition-jyotika-singh-50165494

Explore and download more ebooks at ebookbell.com

Here are some recommended products that we believe you will be
interested in. You can click the link to download.

Natural Language Processing In The Real World Text Processing

Analytics And Classification Jyotika Singh

https://ebookbell.com/product/natural-language-processing-in-the-real-
world-text-processing-analytics-and-classification-jyotika-
singh-58954076

Applied Natural Language Processing In The Enterprise Teaching

Machines To Read Write And Understand 1st Edition Ankur A Patel

https://ebookbell.com/product/applied-natural-language-processing-in-
the-enterprise-teaching-machines-to-read-write-and-understand-1st-
edition-ankur-a-patel-33946944

The Application Of Natural Language Processing To Open Source

Intelligence For Ontology Development In The Advanced Persistent
Threat Domain Corey T Holzer

https://ebookbell.com/product/the-application-of-natural-language-
processing-to-open-source-intelligence-for-ontology-development-in-
the-advanced-persistent-threat-domain-corey-t-holzer-10517910

A Free Press If You Can Keep It What Natural Language Processing

Reveals About Freedom Of The Press In Hong Kong Giovanna Maria Dora
Dore

https://ebookbell.com/product/a-free-press-if-you-can-keep-it-what-
natural-language-processing-reveals-about-freedom-of-the-press-in-
hong-kong-giovanna-maria-dora-dore-50166064
Natural Language Processing As A Foundation Of The Semantic Web
Foundations And Trends In Web Science Yorick Wilks

https://ebookbell.com/product/natural-language-processing-as-a-
foundation-of-the-semantic-web-foundations-and-trends-in-web-science-
yorick-wilks-1908784

Advances In Natural Language Processing Intelligent Informatics And

Smart Technology Selected Revised Papers From The Eleventh
International Symposium On Natural Language Processing Snlp2016
Thanaruk Theeramunkong
https://ebookbell.com/product/advances-in-natural-language-processing-
intelligent-informatics-and-smart-technology-selected-revised-papers-
from-the-eleventh-international-symposium-on-natural-language-
processing-snlp2016-thanaruk-theeramunkong-7024526

Advances In Intelligent Informatics Smart Technology And Natural

Language Processing Selected Revised Papers From The Joint
International Symposium On Artificial Intelligence And Natural
Language Processing Isainlp 2017 1st Ed Thanaruk Theeramunkong
https://ebookbell.com/product/advances-in-intelligent-informatics-
smart-technology-and-natural-language-processing-selected-revised-
papers-from-the-joint-international-symposium-on-artificial-
intelligence-and-natural-language-processing-isainlp-2017-1st-ed-
thanaruk-theeramunkong-7323448

Supertagging Using Complex Lexical Descriptions In Natural Language

Processing Srinivas Bangalore

https://ebookbell.com/product/supertagging-using-complex-lexical-
descriptions-in-natural-language-processing-srinivas-
bangalore-48955428

Natural Language Processing In Action Second Edition Meap V08 2

Chapters 1 To 8 Of 12 Hobson Lane

https://ebookbell.com/product/natural-language-processing-in-action-
second-edition-meap-v08-2-chapters-1-to-8-of-12-hobson-lane-49479024
 Natural Language Processing
in the Real World
Natural Language Processing in the Real World is a practical guide for applying data science
and machine learning to build Natural Language Processing (NLP) solutions. Where traditional,
academic-taught NLP is often accompanied by a data source or dataset to aid solution building,
this book is situated in the real world where there may not be an existing rich dataset.

This book covers the basic concepts behind NLP and text processing and discusses the appli-
cations across 15 industry verticals. From data sources and extraction to transformation and
modeling, and classic Machine Learning to Deep Learning and Transformers, several popular
applications of NLP are discussed and implemented.

This book provides a hands-on and holistic guide for anyone looking to build NLP solutions,
from students of Computer/Data Science to those working as Data Science professionals.
CHAPMAN & HALL/CRC DATA SCIENCE SERIES

Reflecting the interdisciplinary nature of the field, this book series brings together researchers,
practitioners, and instructors from statistics, computer science, machine learning, and analyt-
ics. The series will publish cutting-edge research, industry applications, and textbooks in data
science.

The inclusion of concrete examples, applications, and methods is highly encouraged. The scope
of the series includes titles in the areas of machine learning, pattern recognition, predictive ana-
lytics, business analytics, Big Data, visualization, programming, software, learning analytics,
data wrangling, interactive graphics, and reproducible research.

Published Titles
Urban Informatics
Using Big Data to Understand and Serve Communities
Daniel T. O’Brien
Introduction to Environmental Data Science
Jerry Douglas Davis
Hands-On Data Science for Librarians
Sarah Lin and Dorris Scott
Geographic Data Science with R
Visualizing and Analyzing Environmental Change
Michael C. Wimberly
Practitioner’s Guide to Data Science
Hui Lin and Ming Li
Data Science and Analytics Strategy
An Emergent Design Approach
Kailash Awati and Alexander Scriven
Telling Stories with Data
With Applications in R
Rohan Alexander
Data Science for Sensory and Consumer Scientists
Thierry Worch, Julien Delarue, Vanessa Rios De Souza and John Ennis
Big Data Analytics
A Guide to Data Science Practitioners Making the Transition to Big Data
Ulrich Matter
Data Science in Practice
Tom Alby
Natural Language Processing in the Real World
Text Processing, Analytics, and Classification
Jyotika Singh

For more information about this series, please visit: https://www.routledge.com/

Chapman--HallCRC-Data-Science-Series/book-series/CHDSS
Natural Language Processing
in the Real World
Text Processing, Analytics, and Classification

Jyotika Singh
First edition published 2023
by CRC Press
6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742
and by CRC Press
4 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN
CRC Press is an imprint of Taylor & Francis Group, LLC
© 2023 Jyotika Singh
Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright
holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowl-
edged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or
utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including pho-
tocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission
from the publishers.
For permission to photocopy or use material electronically from this work, access www.copyright.com or contact the
Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. For works that are
not available on CCC please contact [email protected]
Trademark notice: Product or corporate names may be trademarks or registered trademarks and are used only for
identification and explanation without intent to infringe.

Library of Congress Cataloging-in-Publication Data

Names: Singh, Jyotika, author.

Title: Natural language processing in the real-world : text processing,
analytics, and classification / Jyotika Singh.
Description: First edition. | Boca Raton, FL : CRC Press, 2023. | Includes
bibliographical references and index. | Summary: “This book introduces
the basic concepts of Natural Language Processing (NLP) along with a
wide variety of applications of NLP across 15 industry verticals.
Practical examples containing tools, techniques and Python code are
included alongside the basic concepts for a hands-on experience. The
book includes applications from recruiting, social media and
entertainment, finance, marketing and advertising, research and
education, medical and healthcare, travel and hospitality, gaming, oil
and gas, supply chain, writing, retail, real estate, insurance and
telecommunications. It also includes implementation and code examples
around advanced NLP utilizations, as well as popular industrial
products”-- Provided by publisher.
Identifiers: LCCN 2022060257 (print) | LCCN 2022060258 (ebook) | ISBN
9781032195339 (hbk) | ISBN 9781032207032 (pbk) | ISBN 9781003264774
(ebk)
Subjects: LCSH: Natural language processing (Computer science) | Data
mining.
Classification: LCC QA76.9.N38 S526 2023 (print) | LCC QA76.9.N38 (ebook)
| DDC 006.3/5--dc23/eng/20230123
LC record available at https://lccn.loc.gov/2022060257
LC ebook record available at https://lccn.loc.gov/2022060258

ISBN: 978-1-032-19533-9 (hbk)

ISBN: 978-1-032-20703-2 (pbk)
ISBN: 978-1-003-26477-4 (ebk)
DOI: 10.1201/9781003264774
Typeset in LM Roman
by KnowledgeWorks Global Ltd.
Publisher’s note: This book has been prepared from camera-ready copy provided by the authors.
 To my late grandfather, Sardar Sardul Singh. He was a
passionate reader and would have been very happy seeing a book
published by his youngest granddaughter. Thank you for your
wisdom and love.
Contents

List of Figures xv

List of Tables xxi

Preface xxiii

Author Bio xxvii

Acknowledgments xxix

Section I NLP Concepts

Chapter 1 ■ NLP Basics 5

1.1 NATURAL LANGUAGE PROCESSING 5

1.2 LANGUAGE CONCEPTS 11
1.2.1 Understanding language 11
1.2.2 Components of language 13
1.3 USING LANGUAGE AS DATA 14
1.3.1 Look-up 15
1.3.2 Linguistics 15
1.3.3 Data quantity and relevance 16
1.3.4 Preprocessing 17
1.3.5 Numerical representation 18
1.4 NLP CHALLENGES 19
1.4.1 Language diversity 19
1.4.1.1 Writing styles 19
1.4.1.2 Sentence ambiguities 20
1.4.1.3 Different languages 21
1.4.2 Language evolution 22
1.4.3 Context awareness 22
1.4.4 Not always a one-size-fits-all 23

vii
viii ■ Contents

1.5 SETUP 26
1.6 TOOLS 27

Section II Data Curation

Chapter 2 ■ Data Sources and Extraction 35

2.1 SOURCES OF DATA 35

2.1.1 Generated by businesses 35
2.1.2 Openly accessible 36
2.1.3 Conditionally available 38
2.2 DATA EXTRACTION 40
2.2.1 Reading from a PDF 40
2.2.2 Reading from a scanned document 41
2.2.3 Reading from a JSON 43
2.2.4 Reading from a CSV 44
2.2.5 Reading from HTML page (web scraping) 45
2.2.6 Reading from a Word document 46
2.2.7 Reading from APIs 46
2.2.8 Closing thoughts 52
2.3 DATA STORAGE 52
2.3.1 Flat-file database 54
2.3.2 Elasticsearch 54
2.3.2.1 Query examples 57
2.3.3 MongoDB 58
2.3.3.1 Query samples 59
2.3.4 Google BigQuery 61
2.3.4.1 Query examples 61

Section III Data Processing and Modeling

Chapter 3 ■ Data Preprocessing and Transformation 75

3.1 DATA CLEANING 75

3.1.1 Segmentation 76
3.1.2 Cleaning 78
3.1.3 Standardization 82
3.1.4 Example scenario 84
 Contents ■ ix

3.2 VISUALIZATION 85
3.3 DATA AUGMENTATION 87
3.4 DATA TRANSFORMATION 89
3.4.1 Encoding 90
3.4.2 Frequency-based vectorizers 92
3.4.3 Co-occurrence matrix 94
3.4.4 Word embeddings 95

Chapter 4 ■ Data Modeling 105

4.1 DISTANCE METRICS 105

4.1.1 Character-based similarity 105
4.1.2 Phonetic matching 106
4.1.3 Semantic similarity metrics 107
4.2 MODELING 108
4.2.1 Classic ML models 111
4.2.1.1 Clustering 111
4.2.1.2 Classification 113
4.2.2 Deep learning 117
4.2.2.1 Convolutional neural network (CNN) 117
4.2.2.2 Recurrent neural network (RNN) 120
4.2.2.3 Long short term memory (LSTM) 121
4.2.2.4 Bi-directional LSTMs (BiLSTMs) 122
4.2.3 Transformers 123
4.2.3.1 Main innovations behind transformers 124
4.2.3.2 Types of transformer models 125
4.2.3.3 Using transformer models 128
4.2.4 Model hyperparameters 131
4.3 MODEL EVALUATION 132
4.3.1 Metrics 136
4.3.2 Hyperparameter tuning 138

Section IV NLP Applications across Industry Verticals

Chapter 5 ■ NLP Applications - Active Usage 149

5.1 SOCIAL MEDIA 149

5.1.1 What is social media? 149
x ■ Contents

5.1.2 Language data generated 149

5.1.3 NLP in social media 149
5.2 FINANCE 153
5.2.1 What is finance? 153
5.2.2 Language data generated 154
5.2.3 NLP in finance 154
5.3 E-COMMERCE 155
5.3.1 What is e-commerce? 155
5.3.2 Language data generated 156
5.3.3 NLP in e-commerce 156
5.4 TRAVEL AND HOSPITALITY 160
5.4.1 What is travel and hospitality? 160
5.4.2 Language data generated 161
5.4.3 NLP in travel and hospitality 161
5.5 MARKETING 163
5.5.1 What is marketing? 163
5.5.2 Language data generated 163
5.5.3 NLP in marketing 163
5.6 INSURANCE 166
5.6.1 What is insurance? 166
5.6.2 Language data generated 166
5.6.3 NLP in insurance 166
5.7 OTHER COMMON USE CASES 170
5.7.1 Writing and email 170
5.7.2 Home assistants 170
5.7.3 Recruiting 171

Chapter 6 ■ NLP Applications - Developing Usage 173

6.1 HEALTHCARE 173

6.1.1 What is healthcare? 173
6.1.2 Language data generated 173
6.1.3 NLP in healthcare 173
6.2 LAW 176
6.2.1 What is law? 176
6.2.2 Language data generated 176
6.2.3 NLP in law 176
 Contents ■ xi

6.3 REAL ESTATE 178

6.3.1 What is real estate? 178
6.3.2 Language data generated 179
6.3.3 NLP in real estate 179
6.4 OIL AND GAS 181
6.4.1 What is oil and gas? 181
6.4.2 Language data generated 181
6.4.3 NLP in oil and gas 182
6.5 SUPPLY CHAIN 184
6.5.1 What is supply chain? 184
6.5.2 Language data generated 184
6.5.3 NLP in supply chain 184
6.6 TELECOMMUNICATION 186
6.6.1 What is telecom? 186
6.6.2 Language data generated 186
6.6.3 NLP in telecom 186
6.7 AUTOMOTIVE 188
6.7.1 What is automotive? 188
6.7.2 Language data generated 188
6.7.3 NLP in automotive 189
6.8 SERIOUS GAMES 191
6.8.1 What is a serious game? 191
6.8.2 Language data generated 191
6.8.3 NLP in serious games 192
6.9 EDUCATION AND RESEARCH 194
6.9.1 What is education and research? 194
6.9.2 Language data generated 194
6.9.3 NLP in education and research 194

Section V Implementing Advanced NLP Applications

Chapter 7 ■ Information Extraction and Text Transforming Models 203

7.1 INFORMATION EXTRACTION 203

7.1.1 Named entity recognition (NER) 204
7.1.1.1 Rule-based approaches 204
7.1.1.2 Open-source pre-trained models 205
xii ■ Contents

7.1.1.3 Training your own model 210

7.1.1.4 Fine-tuning on custom datasets using transformers 213
7.1.2 Keyphrase extraction (KPE) 217
7.1.2.1 textacy 218
7.1.2.2 rake-nltk 219
7.1.2.3 KeyBERT 219
7.2 TEXT SUMMARIZATION 221
7.2.1 Extractive summarization 221
7.2.1.1 Classic open-source models 221
7.2.1.2 Transformers 223
7.2.2 Abstractive summarization 224
7.2.2.1 Transformers 225
7.3 LANGUAGE DETECTION AND TRANSLATION 227
7.3.1 Language detection 227
7.3.2 Machine translation 227
7.3.2.1 Paid services 227
7.3.2.2 Labeled open-source 228
7.3.2.3 Transformers 229

Chapter 8 ■ Text Categorization and Affinities 231

8.1 TOPIC MODELING 231

8.1.1 Latent dirichlet allocation (LDA) 231
8.2 TEXT SIMILARITY 235
8.2.1 Elasticsearch 235
8.2.2 Classic TF-IDF approach 236
8.2.3 Pre-trained word embedding models 237
8.3 TEXT CLASSIFICATION 239
8.3.1 Off-the-shelf content classifiers 239
8.3.1.1 Zero-shot classification 239
8.3.2 Classifying with available labeled data 241
8.3.2.1 Classic ML 241
8.3.2.2 Deep learning 248
8.3.3 Classifying unlabeled data 253
8.3.3.1 Solution 1: Labeling 253
8.3.3.2 Solution 2: Clustering 253
8.3.3.3 Solution 3: Hybrid approach 253
 Contents ■ xiii

8.4 SENTIMENT ANALYSIS 254

8.4.1 Classic open-source models 254
8.4.2 Transformers 256
8.4.3 Paid services 257

Section VI Implementing NLP Projects in the Real-World

Chapter 9 ■ Chatbots 263

9.1 TYPES OF CHATBOTS 263

9.2 COMPONENTS OF A CHATBOT 265
9.3 BUILDING A RULE-BASED CHATBOT 266
9.4 BUILDING A GOAL-ORIENTED CHATBOT 271
9.4.1 Chatbots using service providers 273
9.4.2 Create your own chatbot 274
9.4.3 Using RASA 275
9.5 CLOSING THOUGHTS 285

Chapter 10 ■ Customer Review Analysis 287

10.1 HOTEL REVIEW ANALYSIS 287

10.1.1 Sentiment analysis 289
10.1.2 Extracting comment topic themes 291
10.1.3 Unlabeled comment classification into categories 296

Chapter 11 ■ Recommendations and Predictions 311

11.1 CONTENT RECOMMENDATION SYSTEM 311

11.1.1 Approaches 311
11.1.2 Building a social media post recommendation system 311
11.1.2.1 Evaluating a classic TF-IDF method, spaCy model,
and BERT model 313
11.1.3 Conclusion and closing thoughts 318
11.2 NEXT-WORD PREDICTION 318
11.2.1 Building a next-word prediction for the data science topic 318
11.2.1.1 Training a BiLSTM model 320
xiv ■ Contents

Chapter 12 ■ More Real-World Scenarios and Tips 325

12.1 DATA SCENARIOS 325

12.2 MODELING SCENARIOS 327
12.3 DEPLOYING YOUR MODEL 331
12.4 MODEL AND OUTCOME EXPLAINABILITY 334

Bibliography 337

Index 357
List of Figures

1.1 An example of named-entity recognition. 7

1.2 An example of keyphrase extraction. 7
1.3 Topic modeling. 8
1.4 Understand language - humans versus machines. 12
1.5 Some popular applications of NLP that leverage different language
components. 14
1.6 Word cloud of top 100 most spoken languages across the world. 21
1.7 Happiness expressions of individuals representing diversity in styles
of communication. 23
1.8 Transfer learning versus traditional machine learning. 25

2.1 Image of a page in a book [132] scanned from a smart phone. 42

2.2 Results of OCR on Figure 2.1. On the left, results are produced with-
out any image filtering. On the right, results are produced with the
thresholding filter applied to the image. The errors are highlighted in
grey. 43
2.3 Data Science project phases. 72

3.1 An example of a word cloud visual. 86

3.2 ScatterText sample output. 86
3.3 Translation from English to Hindi, and back to English. 89
3.4 Relationships between words using distances between word
embeddings. 96

4.1 An example of a look-up based model. 109

4.2 Popular ML models for text-based applications. 110
4.3 Different types of clustering. 111
4.4 Matrix factorization in LDA. 112
4.5 SVM hyperplane separating data samples. 115
4.6 An example of a decision tree. 116
4.7 CNN architecture. 118

xv
xvi ■ List of Figures

4.8 RNN architecture. 120

4.9 Examples of the pre-training and fine-tuning flow in transformer
models. 130
4.10 An example of a confusion matrix for a binary classification case and a
multi-class classification case. TP stands for true positives. TN stands
for true negatives. FP stands for false positives. FN stands for false
negatives. 135

5.1 Global count of social media users by year. 150

5.2 Social media content recommendations on the right based on the
currently viewed content. 150
5.3 Search typo correction on social media. 151
5.4 Brand chatbots on social media messengers. 152
5.5 Sentiment analysis gone wrong. 153
5.6 Impact of a social media post on an unrelated stock price. 154
5.7 A bank’s chatbot. 155
5.8 Results on an e-commerce website from search with a spelling error. 156
5.9 Tumbler with a straw - product page. 157
5.10 Tumbler with a straw – more to consider section. 157
5.11 Customer review section showcasing comment classification into types. 159
5.12 E-commerce chatbot. 160
5.13 Automatic detection of potential sensitive information in live chat. 160
5.14 Hospitality bookings chatbot. 161
5.15 Searching the web for king bed frames. 163
5.16 Advertisement on social media for king mattress. 164
5.17 AI-based slogan recommendation. Source [175]. 166
5.18 Insurance chatbot example. 168
5.19 Conversion of invoices to text using OCR. 169
5.20 Auto sentence completion suggestions in Gmail. 170
5.21 Email filtering leading to division of incoming mail between inbox and
spam. 171
5.22 Email classification in Gmail. 171

6.1 Classification of text into protected health information (PHI) cate-

gories. Source [202]. 174
6.2 Example of clinical record with annotated PHI categories. Source [202]. 174
6.3 Document summarization in medical records. Source [63]. 175
6.4 A GDPR compliance guidance chatbot. 178
 List of Figures ■ xvii

6.5 Real estate listing description with information extraction results on

the right to identify key pieces of information. 180
6.6 Convoboss: Real estate chatbot for 24/7 lead generation. Source [54]. 180
6.7 Petroleum company chatbot example. 183
6.8 Uses of chatbots in different supply chain operations. 185
6.9 An example of a supply chain procurement chatbot. 185
6.10 Telecom company’s chatbot. 187
6.11 Infotainment systems in vehicles. 190
6.12 Chatbot for car dealerships. 191
6.13 iSTART self-explanation assessment. Source [6]. 192
6.14 Facade game. Source [121]. 193
6.15 Google Translate for quick language translation between English and
Punjabi. 195
6.16 NLP applications and projects by industry. 197

7.1 Named entity recognition (NER) on a sentence. 204

7.2 spaCy NER output with displacy. 207
7.3 spaCy transformer (RoBERTa) NER output with displacy. 208

8.1 Books whose descriptions were used to build our LDA model. Source
doc1 [23], doc2 [82], doc3 [90], doc4 [76]. 232
8.2 The book used to test our LDA model. Source [188]. 234
8.3 The book description to test our LDA model. 234
8.4 Confusion matrix for spam vs ham classification model using Multi-
nomial Naive Bayes classifier. 245
8.5 Training and validation accuracy and loss for ham/spam CNN model. 252
8.6 Curating labeled data using clustering experiments. 254
8.7 Where data science modeling fits within a business’s goal and its
driving factors. 262

9.1 E-commerce chatbot conversation (left to right). 264

9.2 Chatbot system overview. 266
9.3 Building a chatbot from a company KPI perspective. 267
9.4 Building a pizza-ordering chatbot from a company KPI perspective. 272
9.5 Training data for building a custom NER model with spaCy. 276
9.6 Test results for our custom NER model built using spaCy for entities
related to pizza attributes. 277
9.7 RASA folder. 278
9.8 RASA components for a chatbot system. 278
xviii ■ List of Figures

9.9 nlu.yml intents related to greeting, user agreement, and user disagree-
ment. 278
9.10 nlu.yml intents related to pizza ordering. 279
9.11 RASA pizza-ordering chatbot - sample conversations. 282
9.12 RASA chatbot conversation with typos. 282
9.13 RASA chatbot bad conversation samples. 282

10.1 Performing comment review analysis from a company KPI perspective. 288
10.2 Data science tasks breakdown for customer review analysis project. 289
10.3 Data science tasks breakdown for customer review analysis project
(sentiment analysis). 292
10.4 Word cloud for positive comments. 293
10.5 Word cloud for negative comments. 293
10.6 Word cloud for positive comments (nouns only). 295
10.7 Word cloud for negative comments (nouns only). 295
10.8 Data science tasks breakdown for customer review analysis project
(identification of topics and themes). 296
10.9 Room-word cloud. 299
10.10 Location-word cloud. 299
10.11 Service and staff-word cloud. 300
10.12 Data science tasks breakdown for customer review analysis project
(curating training data). 301
10.13 Confusion matrix for hotel review classification model. 302
10.14 Data science tasks breakdown for customer review analysis project
(training a classification model). 302
10.15 Data science tasks breakdown for customer review analysis project
(model evaluation). 304
10.16 Data science tasks breakdown for customer review analysis project
(pipeline). 305
10.17 Data science tasks breakdown for customer review analysis project
(curating training data). 307

11.1 Building a recommendation system from a company KPI perspective. 312

11.2 TF-IDF method: top 8 content recommendations. 315
11.3 spaCy word embeddings method: top 8 content recommendations. 316
11.4 BERT method: top 8 content recommendations. 317
11.5 Building next word prediction models from a company KPI perspective. 319
11.6 Next word prediction BiLSTM model accuracy and loss at 10 epochs. 322
 List of Figures ■ xix

11.7 Next word prediction BiLSTM model accuracy and loss at 20 and 40
epochs. 323
11.8 Next word prediction output from the BiLSTM model with the pre-
dicted words in bold. 324

12.1 Phases of creating NLP projects. 325

12.2 Phases of creating NLP projects - data. 326
12.3 Phases of creating NLP projects - modeling. 327
12.4 Phases of creating NLP projects - outcome. 334
List of Tables

2.1 Publicly available text datasets. 37

2.2 BigQuery string functions. 62

3.1 Word vectors based on features. 96

4.1 Transformer models and applications. 129

4.2 ML models and applications. 131
4.3 Common hyperparameters of classic ML classification models. At-
tached URLs contain further details for each hyperparameter. 132
4.4 Common hyperparameters of deep learning-based classification mod-
els. Attached URLs contain further details for each hyperparameter. 133
4.5 ML model evaluation metrics. 138

7.1 State-of-the-art translation services. 228

9.1 Chatbot service providers. 275

xxi
Preface

In the modern day, data digitization has scaled and there are means to store every
interaction happening across the world. Text data is heavily generated across the
globe. Some common sources of text data include social media data, consumer inter-
action, reviews, articles, documents, emails, and others. More and more businesses
have started leveraging machine learning, and a large majority have some type of text
data available to them. Over the last decade, several businesses have explored and
been successful in getting intelligence out of text data generated by them or publicly
available from the web. While many are on that path, many want to get on that path
and exploit the potential of building data-driven offerings. Thus, knowing about NLP
and how you can use it is prime in today’s time.
Natural language processing (NLP) is a hot topic with a lot of applications and
an increasing amount of research across the globe. NLP refers to a machine’s process
to understand language. With the immense amount of text data generated today,
there is an increase in the scope for leveraging NLP to build intelligent solutions.
Google Trends suggests a 112% increase in searches on the topic of natural language
processing in the past seven years. Many businesses today offer products and ser-
vices powered by NLP. Common examples include Amazon Alexa, Gmail sentence
auto-completion, and Google Translate for language translation. With the increasing
demand for NLP-based products and services, there is a strong need for a workforce
that is able to understand and implement NLP solutions.
I started working in the industry as a Data Scientist after finishing grad school.
At the time, I didn’t have any guidance in my field at the company I was working at.
I was faced with tasks that seemed impossible to solve given my grad school back-
ground. In an educational setting, you are working on defined problems. In the real
world, you need to define these problems yourself given the knowledge of the business
objective. In an educational setting, you have data available. You’re either working on
publicly available datasets or one available at your educational institution. In the real
world, you may not have labeled data, you may not have enough data, and you may
not even have any data at all. Having faced these obstacles, I learned several lessons
that over time helped me to excel at my work. I would often share my learnings
and findings with the Python and Data Science community in the form of talks and
presentations at conferences across the globe. After accumulating close to a decade
of experience in working with language data and building NLP solutions in the real
world, I wrote this book.

xxiii
xxiv  Preface

What does this book contain?

This book starts by introducing NLP, underlying concepts, and popular tools.
Then, the book dives into everything around data – data curation, data extraction,
and data storage. The data needs to be cleaned and converted to a language that
a machine can understand. The book implements several data preprocessing meth-
ods, data transformation methods, distance metrics, machine learning, deep learning,
and transformers. In a practical sense, businesses make use of the technique that best
solves their use case, including classic/traditional models and state-of-the-art models.
This book covers them all through a practical lens. With the knowledge about data
and models, you are ready to put it together to build NLP applications. But what
are these NLP applications, who uses them, and for what? This book dives into NLP
applications across 15 industry verticals. Then, we pick the most commonly used
applications and implement them in many different ways using Python and various
open-source tools. Then, this book describes NLP projects in the real world, in an
actual business setting. Why do you decide to build an NLP-based project? How do
you measure success? Where does it fit into your company’s goals? How is the model
then consumed by other users and applications? All these aspects are discussed, and
these NLP projects are implemented using Python and the knowledge gained from the
previous sections of the book. https://github.com/jsingh811/NLP-in-the-real-world
contains all the code used in this book. This book is structured as shown below.

Who this book is for?

This book is an ideal resource for those seeking to expand their knowledge of
NLP and develop practical NLP solutions. Whether you are new to NLP, seeking
to deepen your understanding, or exploring NLP for a specific use case, this book
caters to all levels of expertise. By emphasizing practical applications of NLP and
providing insights into how more than 15 industry verticals leverage NLP, this book
offers valuable guidance for those looking to develop their own solutions using text
data.
But how would you go about it? What sets this book apart is its focus on
implementation. With numerous real-world NLP applications and projects using
 Preface ■ xxv

open-source tools and the Python programming language, readers will gain hands-on
experience and be able to apply the solutions in your work. Readers will be able to
learn the concepts and refer back to the book any time they need to brush up on
their understanding of NLP usage and applications across industry verticals.
Assuming the reader has a basic understanding of machine learning and program-
ming in Python, this book focuses on practical aspects of NLP, covering the basic
concepts from a practical perspective, rather than diving into detailed architectures.
As such, this book is set to be a valuable resource for anyone looking to develop
practical NLP solutions.
The solutions we build involve using classic machine learning approaches, deep
learning models, and transformers, covering everything from the basics to the state-
of-the-art solutions that are used by companies for building real-world applications.
The reader will:

• Gain knowledge about necessary concepts and methods to build NLP solutions.

• Curate, extract, process, transform, and model text data for various use cases.

• Learn about how several industries solve NLP problems and apply the learnings
to new and unseen NLP tasks.

• Experience hands-on practical examples to build NLP applications and projects

of the real world using classic as well as cutting-edge algorithms.

• Implement real-world NLP projects to get real-world experience.

• Learn to use open-source Python tools for quick NLP implementations.

• Get practical tips throughout the book around different scenarios with data,
processing, and modeling.
Author Bio

For nearly a decade, Jyotika has focused her career on Machine Learning (ML) and
Natural Language Processing (NLP) across various industry verticals, using practical
real-world datasets to develop innovative solutions. Her work has resulted in multiple
patents that have been utilized by well-known tech companies for their advancements
in NLP and ML. Jyotika’s expertise in the subject has made her a highly sought-after
public speaker, having presented at more than 20 conferences and events around the
world.
Her work on building proprietary NLP solutions for ICX Media, a previous em-
ployer, resulted in unique business propositions that played a pivotal role in secur-
ing multi-million dollar business and the successful acquisition by Salient Global.
Jyotika currently holds the position of Director of Data Science at Placemakr, a
leading technology-enabled hospitality company in the USA. Moreover, Jyotika is
the creator and maintainer of open-source Python libraries, such as pyAudioProcess-
ing, that have been downloaded over 24,000 times.
Jyotika’s commitment to promoting diversity in STEM is evident through her ac-
tive support of women and underrepresented communities. She provides early-career
mentorship to build a diverse talent pool and volunteers as a mentor at Data Science
Nigeria, where she engages in mentorship sessions with young Nigerians aspiring for
a career in data and technology. Furthermore, Jyotika serves as a mentor at Women
Impact Tech, US, supporting women in technology, product, and engineering.
Jyotika has received numerous awards for her contributions to the field, including
being recognized as one of the top 50 Women of Impact in 2023 and being named
one of the top 100 most Influential people in Data 2022 by DataIQ. Additionally,
Jyotika has been honored with the Data Science Leadership award in 2022, Leadership
Excellence in Technology award in 2021, and other accolades.

xxvii
Acknowledgments

Writing this book would not have been possible without the plethora of excellent
resources, such as papers, articles, open-source code, conferences, and online tools.
I am thankful to the Python, machine learning, and natural language processing
community for their efforts and contributions toward knowledge sharing. Along my
journey, I have asked a lot of individuals I do not personally know a lot of questions
about this topic and the book publishing process. Thank you all for selflessly taking
the time to answer my questions. Thank you to all the companies and publishers that
have permitted me to use their figures to aid the material of my book. I am grateful
for your contributions to this field and your prompt responses.
I am grateful to everyone who has reviewed sections and chapters of this book.
Thank you Shubham Khandelwal, Manvir Singh Walia, Neeru, Jed Divina, Rebecca
Bilbro, Steven McCord, Neha Tiwari, Sumanik Singh, Joey McCord, Daniel Jolicoeur,
Rekha, and Devesh for taking the time and sharing all your helpful suggestions along
my writing journey. Your feedback helped shape this book into what it is today, and
I could not have completed it without your input and support. It has been a pleasure
knowing each one of you and being able to count on your support.
The team at Taylor and Francis has been incredibly helpful throughout this pro-
cess. Your prompt responses and incredible input into this book are huge contributors.
Thank you, Randi (Cohen) Slack, for being a part of this journey.
I am grateful to my employer, Placemakr, for always encouraging and supporting
my book-writing journey. Thank you for sharing my excitement and supporting me
with everything I needed to be able to write this book.
On a personal note, I want to thank my family, the Walias and the Khandelwals,
for motivating me throughout this process. I wrote this book alongside my full-time
job responsibilities, volunteer mentorship work, and other life struggles. It has in-
volved a lot of late nights and weekends to get this book completed. My husband
and my parents have been tremendously helpful in taking care of everything else so
I got to focus on this book. Thank you Shubham, Mumma, and Papa. Your support
means the world to me. I want to especially acknowledge my late grandparents, Sar-
dar Sardul Singh and Raminder Kaur, and my husband’s grandmother, Radhadevi
Khandelwal. I have received nothing but love, support, and blessings from you all.
Thank you for being a part of my life.

xxix
 I
NLP Concepts
 In this section, we will go over some basic concepts that lead up to natural lan-
guage processing (NLP). Believe it or not, each one of us has at some point interacted
with a technology that uses NLP. Yes, it is that common! We will describe NLP and
share some examples of where you may have seen a product or technology powered
by NLP.
We will dive into where it all starts from and is centered around, i.e., language.
We will follow it with a brief introduction to concepts of linguistics that form the
basis for many NLP tasks. Often when thinking of how to implement a method for a
machine to do a task that humans perform well, it is useful to consider the perspective
– how would I (human) solve this? The answer often inspires mathematical modeling
and computer implementation for the task. Thus, we will spend some time in this
section on how the human-based understanding of language influences NLP tasks.
Language data needs preparation before a machine can find meaning from it. Have
you ever received a text message from a friend with a term you didn’t understand that
you had to look up on the Internet? Have you ever needed to translate a sentence from
one language to another to understand its meaning? Machines can require similar and
various additional types of preprocessing before they can make sense of the language
input. In general, language is not numeric (not represented as numbers), whereas a
machine understands data in only binary numbers – 1’s and 0’s. We’ll introduce the
basic concepts of converting language into numeric features before diving into further
details in the later chapters.
To build successful NLP solutions, it is important to note challenges in NLP and
why they arise. There are many challenges, some that remain challenging, and some
that can be fully or partially solved by using certain techniques. We will introduce
NLP challenges and potential solution options.
 Finally, we will list setup requirements and introduce popular tools that we will
use in the rest of the book.
This section entails the following topics:

• Natural language processing

• Language concepts

• Using language as data

• NLP challenges

• Setup

• Tools
 CHAPTER 1

NLP Basics

1.1 NATURAL LANGUAGE PROCESSING

Language is a way that humans have been using for communicating with one another
since the beginning of time. The term ‘natural language’ refers to language that
has naturally evolved over time due to repeated use by humans. In essence, natural
language is referred to as the language humans use to communicate with one another.
Natural language processing, often abbreviated as NLP, refers to the field of
programming computers to allow the processing and analysis of natural language.
From something as basic as a computer program to count the number of words
in a piece of text, to something more complex such as a program that can serve
replies to questions asked by humans or translate between languages, all qualify as
NLP. Essentially, regardless of the difficulty level, any task that involves a computer
dealing with language through a program qualifies as natural language processing.
Knowing about the range of applications helps us understand the impact of NLP.
Consider the following example. You are cooking in the kitchen and want your voice
assistants, such as Alexa or Google Home, to turn on your TV.

DOI: 10.1201/9781003264774-1 5
6 ■ Natural Language Processing in the Real-World

You: Turn on the Living Room TV

TV turns on

You: Play the soccer match on NBC Sports 11.

Match starts playing on your TV

You: Pause TV

TV pauses your video

You: At what temperature should I bake vegetables?

‘400 degrees Fahrenheit is the perfect temperature for most vegetables

for a crispy exterior and a tender interior.’

You: Play TV

TV resumes your paused video

Conversation as the above in these voice assistants is powered by NLP. Further-

more, you may have noticed the auto-correct and word recommendation features on
your cell phone. Have you noticed how most spam email successfully makes it to the
Spam or Junk folder? What about the times when you are purchasing a product
online and need to contact customer service regarding an issue? Have you noticed
how in many online retailers the chat service starts with an automatic reply service
that tries to get you what you need without, or before, having to connect to a cus-
tomer service representative? Examples include assistance with returns, order status,
and product information. All these are instances of how humans interact with NLP
systems regularly where the machine can understand what you type or what you
speak.
There exist popular applications that are built using NLP across several different
industry verticals. Some of these remain common across the board, while some ap-
plications remain specific to particular industries. We’ll be looking at how and where
several industries utilize or explore NLP in Section IV of this book. These indus-
try verticals include Social Media, Real Estate, Finance, Medical and Healthcare,
E-commerce, Travel and Hospitality, Marketing, Oil and Gas, Supply chain, Insur-
ance, Gaming, Law, Telecommunication, Automotive, Education and Research, and
others.
 NLP Basics  7

Often, implementations of NLP form a part of a larger product. For instance, is

Alexa all NLP? No, but NLP is a part of making Alexa a successful product. We’ll be
diving into the popular NLP applications in Section V that often help in contributing
to larger products across different industry verticals. We’ll also dive into industrial
projects that make use of the different NLP applications in Section VI.
NLP applications
The advanced applications of NLP that are discussed and implemented in Section
V include the following.

1. Named-entity recognition: Named entity recognition (NER) is a form of natural

language processing and is also known as entity extraction, entity identification,
or entity chunking. This technique identifies segments of key information within
a piece of text and categorizes the segments into predefined categories such as
person name, location, date, timestamp, organization name, percentages, codes,
numbers, and more. See Figure 1.1 for an example.

FIGURE 1.1 An example of named-entity recognition.

2. Keyphrase extraction: Key-phrase extraction is a textual information processing

task concerned with the automatic extraction of representative and character-
istic phrases from a document that express all the key aspects of its content.
Keyphrases aim to represent a succinct conceptual summary of a text docu-
ment. They find use in various applications such as digital information man-
agement systems for semantic indexing, faceted search, document clustering,
and classification [129]. See Figure 1.2 for an example.

FIGURE 1.2 An example of keyphrase extraction.

3. Topic modeling: Topic modeling is the process of identifying different topics

from a set of documents by detecting patterns of words and phrases within
them as seen in Figure 1.3. Topic modeling finds applications in document
clustering, text organization, information retrieval from unstructured text, and
feature selection [24].
8  Natural Language Processing in the Real-World

FIGURE 1.3 Topic modeling.

4. Text similarity: Text similarity is a popular NLP application that finds use in
systems that depend on finding documents with close affinities. A popular ex-
ample is content recommendations seen on social media platforms. Ever noticed
that when you search for a particular topic, your next-to-watch recommended
list gets flooded with very similar content? Credit goes to text similarity al-
gorithms, among some other data points that help inform user interest and
ranking.

5. Text classification: Text classification refers to classifying text into user-defined

categories. This can be something as basic as binary labels to hundreds and
thousands of categories. Examples include categorizing social media content
into topics and consumer complaint categorization in customer service.

6. Text summarization: Long blobs of text such as articles, papers, or documents

are condensed into a summary that aims to retain vital information using text
summarization techniques. Google News1 , the Inshorts app2 , and various other
news aggregator apps take advantage of text summarization algorithms.

7. Language detection and translation: Detection of language from text refers to

language detection. The process of translating text from one language to an-
other is language translation. There exist many pre-trained models for numer-
ous language tasks that can be used right out of the box by practitioners. Most
models are trained on a particular text language. Such models don’t perform
as well if used on text of a different language. In such cases, practitioners often
resort to language detection and translation techniques. Such techniques also
find use in language translation tools to help people communicate in non-native
languages.
1
https://news.google.com/
2
https://www.inshorts.com/
 NLP Basics ■ 9

8. Sentiment analysis: Sentiment analysis is the work of a model that is built

to gauge human sentiment in a sentence. This application is a part of many
analytics-powered organizations that rely on understanding consumer sentiment
toward a product or content.

NLP industry projects

Some popular NLP projects used across various industries that are discussed and
implemented in Section VI include the following.

1. Chatbots: Chatbots, also called chatterbots, are bots or artificial intelligence

systems that are able to chat with humans, often customers of a business.
Chatbots can handle tasks from understanding the customer’s question to giv-
ing replies and answers. This tool often adds a convenient way for apps and
businesses to provide their customers with a human-like interaction experience
while keeping the costs involved low.
2. Customer review analysis: Customer reviews are important to understand the
feedback for a product or business. However, customer comments are often
not categorized and thus do not enable quick analysis. Analyzing customer
feedback and sentiment, and creating relevant classification models finds use in
many industry domains.
3. Recommendation systems: Many industries deploy products based on recom-
mendation algorithms. Examples include ads that are recommended to you by
advertisers and marketers, product recommendations on e-commerce websites,
and social media post recommendations.
4. Faster documentation services: Next word prediction models aid in implement-
ing an industry-specific or topic-specific auto-complete service to enable faster
documentation processes.

While natural language is not only text but also other forms of com-
munication, such as speech or gestures, the methods and implementation
in this book are focused primarily on text data. Here are a few reasons for that.

- A lot of popular products using speech as input often first transcribe speech
to text and then process the text data for further analysis. The resultant
text is converted to speech after analysis for applications using a speech output.

- Speech processing is a large field of its own. On the other hand, gesture
detections fall under the realm of image processing and computer vision, which
is also a large field of its own. These fields are different, rich, diverse, and call for
a massive write-up like an entire book pertaining to these individual topics to
do them justice. For reference, a brief introduction, some resources, and open-
source Python tools are listed below that you might find useful if interested in
diving further into language processing for speech or gestures.
10 ■ Natural Language Processing in the Real-World

Speech
Introduction
Speech is a form of audio that humans use to communicate with one another.
Speaking is the exercise where forced air is passed through the vocal cords, and
depending on the pressure areas and amount, certain sounds are produced. Reading
speech using a Python program, speech signals are seen as time-series events where
the amplitude of one’s speech varies at different points. Often in speech processing,
frequency is of massive interest. Any sound contains underlying frequencies of its
component sounds. Frequency can be defined as the number of waves that pass a fixed
place in a given amount of time. These frequencies convey a great deal of information
about speech and the frequency domain representation is called the spectrum. Derived
from the spectrum is another domain of speech, called cepstrum. Common features
used from speech signals for machine learning applications include spectral features,
cepstral features, and temporal (time-domain) features.
Challenges
Common challenges in this field include the quality and diversity of data. Speech
in the presence of different background noises forms challenges for a machine to
interpret the signals and distinguish between the main speech versus the background
sounds. Basic techniques such as spectral subtraction [186], and more sophisticated
and actively researched noise removal models are used. There is scope for speech
recognition to be made available for more languages and cover wider topics [164].
Tools
Some popular tools help extract features from speech and audio and build
machine learning models [154]. Examples of such open-source tools include
pyAudioProcessing3 [156], pyAudioAnalysis,4 pydub,5 and librosa.6
Gestures
Introduction
Gestures form an important type of language. Many individuals rely on gestures
as their primary source of communication. Building systems that understand gestures
and smart machines that can interact with gestures is a prime application vertical.
Other applications include programming a system to understand specific gestures and
programming smart devices to optionally take an action based on the gesture, e.g.,
turn off a room light, play music, etc. For gesture analysis, there has been ongoing
research in improving and creating gesture detection and recognition systems [79].
Challenges
Some of the main issues have been around image quality and dataset sizes. Train-
ing a model to recognize images from a clean/fixed dataset may seem simpler. But
in a more realistic setting, the image quality is not always homogeneous or clean,
and training a model to recognize images that it hasn’t seen before in real-time can
be challenging. Data augmentation techniques to artificially add noise to clean sam-
3
https://github.com/jsingh811/pyAudioProcessing
4
https://pypi.org/project/pyAudioAnalysis/
5
https://pypi.org/project/pydub/
6
https://librosa.org/doc/latest/index.html
 NLP Basics ■ 11

ples have been popularly implemented in this area to build a model that is able to
circumvent the noise.
Tools
Popular libraries include OpenCV7 , scikit-image8 , SciPy9 and PIL10 . Artificial
neural networks have been popular in image processing. [79] walks through a simple
model to understand gestures. Here’s another guide to developing a gesture recogni-
tion model using convolutional neural networks (CNN) [36].
We have visited several applications and products powered by NLP. How does a
machine make sense of language? A lot of the inspiration comes from how humans
understand language. Before diving further into machine processes, let’s discuss how
humans understand language and some basic linguistic concepts.

1.2 LANGUAGE CONCEPTS

1.2.1 Understanding language
Humans can understand and decipher language with the help of their brains. Let’s
learn a bit more about how this happens.
The brain has certain areas responsible for forming, expressing, and processing
language. To summarize, the brain receives signals from the inner ear at different
frequencies and deciphers the words being spoken based on its understanding of
language. Similarly, the image an eye sees, reflecting different pixel values, excites
certain neurons and the text is interpreted based on the ability of the human being
to associate meaning with the structure of the way the words are written. Signs are
interpreted in the same way and are interpreted by the brain based on the under-
standing of the meaning of gestures. Further details on the works of biology can be
found below.

Ears
Per Sound Relief Healing Center [43], the human ear is fully developed at birth
and responds to sounds that are very faint as well as very loud sounds. Even
before birth, infants respond to sound. Three parts in the human ear help relay
signals to the brain; the outer ear, middle ear, and inner ear. The outer ear canal
collects sounds and causes the eardrum to vibrate. The eardrum is connected to
three bones called ossicles. These tiny bones are connected to the inner ear at the
other end. Vibrations from the eardrum cause the ossicles to vibrate which, in
turn, creates movement of the fluid in the inner ear. The movement of the fluid
in the inner ear, or cochlea, causes changes in tiny structures called hair cells
that sends electric signals from the inner ear up the auditory nerve to the brain.
The brain then interprets these electrical signals as sound.
7
https://opencv.org/
8
https://scikit-image.org/
9
https://scipy.org/
10
https://pillow.readthedocs.io/en/stable/
12  Natural Language Processing in the Real-World

Eyes
An article in Scientific Journal on ‘The Reading Brain in the Digital Age: The
Science of Paper versus Screens’ [88] yields insights on how the eyes help in
reading. Regarding reading text or understanding gestures, the part of the brain
that processes visual information comes into play, the visual cortex. Reading
is essentially object detection done by the brain. Just as we learn that certain
features—roundness, a twiggy stem, smooth skin—characterize an apple, we learn
to recognize each letter by its particular arrangement of lines, curves, and hollow
spaces. Some of the earliest forms of writing, such as Sumerian cuneiform, began
as characters shaped like the objects they represented—a person’s head, an ear of
barley, or a fish. Some researchers see traces of these origins in modern alphabets:
C as a crescent moon, S as a snake. Especially intricate characters—such as
Chinese hanzi and Japanese kanji—activate motor regions in the brain involved
in forming those characters on paper: The brain goes through the motions of
writing when reading, even if the hands are empty.

How we make sense of these signals as a language that conveys meaning comes
from our existing knowledge about language rules and different components of lan-
guage including form, semantics, and pragmatics. Even though some language rules
apply, because of the different ways people can communicate, often there are no
regular patterns or syntax that natural language follows. The brain relies on an in-
dividual’s understanding of language and context that lies outside of linguistic rules.
Whether we are consciously aware of it or not, any external sound, gesture, or
written text is converted to signals that the brain can operate with. To perform
the same tasks using a machine, language needs to be converted to signals that a
computer can interpret and understand. The processing required to do so is referred
to as Natural Language Processing (See Figure 1.4).

FIGURE 1.4 Understand language - humans versus machines.

 NLP Basics ■ 13

1.2.2 Components of language

Let’s consider the language of the form of speech and text. Humans can understand
each other’s language when it is of the form that a person has been exposed to in
the past. For instance, an English speaker born in the US will be able to understand
the language of an English speaker born in the UK. A non-native individual with a
different first language, who still knows the English language will also be able to un-
derstand other English speakers. How does the exposure help in the understanding of
language? Language is governed by certain rules that apply to characters, the usage
of characters to form words, and the usage of words to form phrases and sentences.
People who are aware of these rules are also able to understand the language.

In this book, the primary focus will be on the English

language.
There are three major components of language – form, semantics, and pragmatics.
There are three types of forms, namely phonology, morphology, and syntax [22]. Let’s
see what each of these means.

1. Form

(a) Phonology: Phonology is the study of individual sound units within a

language and the combination of these units to create larger language
units. Each unit of sound is called a phoneme. Examples include \s\, \f\.
The use of different phonemes can alter the meaning of words such as
– ‘sit’ and ‘fit’. There are a total of about 40 phonemes in the English
language which can be vowels or consonants.
(b) Morphology: Morphemes are the smallest unit of language that conveys
a meaning. Examples include ‘car’ and ‘teach’. Prefixes and suffixes when
attached to these words may change the meaning – ‘teach’ -> ‘teacher’.
(c) Syntax: Syntax is the study of rules by which words are organized into
phrases or sentences. When combining two or more words to form a sen-
tence, following the language’s grammar rules, or syntax, makes it under-
standable. For instance, ‘I tied my shoes’ conveys meaning, whereas ‘my
tied I shoes’ does not.

2. Semantics: Semantics relates to the meaning of the language which is formed by

the use of words together to convey a meaning. This includes objects, actions,
events, or relationships between different objects, actions, and/or events. It is
not just the syntax that conveys all the meaning, but also our understanding
of figurative language. The understanding of semantics is what helps us un-
derstand that the popular phrase ‘getting cold feet’ does not literally convey
information about the temperature of one’s feet.

3. Pragmatics: Pragmatics is the use of language for communication in a social or

interactive environment. The language used to convey a variety of intentions
14 ■ Natural Language Processing in the Real-World

FIGURE 1.5 Some popular applications of NLP that leverage different language com-
ponents.

such as requesting, asserting, or questioning falls under this category. Further-

more, the way a person speaks with their toddler is completely different from
the way the same individual may speak with a co-worker or a friend. The un-
derstanding of the difference between such communication styles, and when to
use which style is the essence of pragmatics.

Each component described above forms a basis for how we, as humans, interpret
the meaning of speech or text. It also forms the basis for many language features that
are used popularly in NLP to understand language. Figure 1.5 shows popular NLP
applications that make use of the different language components discussed above.
Tokenization refers to breaking down sentences into words, and words into the base
form. Part-of-speech tagging marks words in the text as parts of speech such as nouns,
verbs, etc.

1.3 USING LANGUAGE AS DATA

Using language as data needs specific types of processing that a machine can under-
stand and make sense of. Parsing language is one thing for a human, but language
data for a machine is unstructured. Unstructured data is data that does not conform
to a predefined data model, such as rows and columns with defined relationships.
Thus, it is not a ready-to-be-analyzed type of data. While language may be usually
unstructured, it is not completely random. Language is governed by linguistic rules
that make it interpretable by humans. For a machine to be able to learn the context,
 NLP Basics  15

it requires to have seen such data and its usage patterns before and learn the language
rules that humans are likely to follow. This section lists some methods and factors
that are important when thinking of using language as data.

1.3.1 Look-up

It is a popular thought that when an NLP task needs

to be performed, it must entail some sort of complex model-
ing. That is not always true. It can be a very fruitful exercise
to start by thinking of the most basic solutions first. See the
example below.

Consider a task where you need to find all the entries in a dataset of sentences
where the entry contains content about the movie – Ghostbusters. What solutions
come to mind? Curate training data, manually label some samples, and train a model
that predicts – Ghostbusters versus not-Ghostbusters?
Let’s look at a much easier and much faster solution. Why not look up the pres-
ence of the string ‘ghostbusters’ in each data sample? If it is present, mark it as
Ghostbusters, else not-Ghostbusters.
Limitations?
Some samples may mention ‘ecto-1’ which is the vehicle name in the movie and
not the term ‘ghostbusters’. Such a sample would be missed by our approach. Solution
– how about using multiple relevant keywords to search the samples with, including
popular actor names, character names, director names, and popular movie elements
such as the vehicle name? The results may not be all-encompassing but would cer-
tainly return an easy and fast solution and could serve as a great first approach before
a complex solution needs to be scoped out. Furthermore, this method can form a first
step for curating data labels for your dataset that can come in handy for future model
building.
Look-ups and similarly other basic approaches to NLP tasks such as using word
counts work as a great starting point for simpler tasks and result in simple yet effective
solutions.

1.3.2 Linguistics
Let’s look at the following sentence where the task is to identify location names from
text:

Arizona Smith uses a car in New York.

How could we solve this problem? One simple solution might be to have a list
of all location names and search for their presence in the sentence. While it is not
an incorrect solution and would work perfectly for many use cases, there are certain
limitations.
16 ■ Natural Language Processing in the Real-World

Arizona (location) Smith uses a car in New York (location).

The look-up approach would detect ‘Arizona’ and ‘New York’ as location names.
We, as humans, know that Arizona is a location name, but based on the sentence
above, it refers to a person and not the location.
There are advanced techniques that can distinguish between Arizona and New
York in the above example. The process of being able to recognize such entities
is called named-entity recognition, information extraction, or information retrieval
and leverages the syntax rules of language. How does it work? The process includes
tagging the text, detecting the boundaries of the sentence, and capitalization rules.
You can use a collection of data sets containing terms, and their relationships or use
a deep learning approach using word embeddings to understand the semantic and
syntactic relationship between various words. Don’t worry if this sounds unfamiliar.
We’ll dive further into it in Section III and Section V. The best part is that there
are existing tools that offer models that do a reasonable job for such tasks. Using the
spaCy library en_core_web_sm trained model, the below results can be accomplished:

Arizona Smith (name) uses a car in New York (location).

With the knowledge of linguistics and the relationship between terms, the machine
can accomplish the detection of location names from a challenging sentence.
Many other NLP tasks can be solved using the knowledge of linguistics as seen
previously in Figure 1.5.

1.3.3 Data quantity and relevance

Linguistic rules seem to have importance. Look-up methods are also a good option
for some simple NLP tasks. What else do we need to know about using language as
data? A good quantity of data and relevant data are some of the most important
requirements. For instance, consider the following fill-in-the-blank scenario:

David is going out for a _ with his spouse.

Humans may guess ‘meal’, ‘game’, ‘date’, ‘movie’, ‘vacation’, or ‘holiday’. Given
enough text samples, a machine could guess the same answers. The guesses can only
be as good as the data it has seen before. If all our training dataset contains is a
few samples of someone going out for a ‘movie’ with their spouse, then that’s the
best prediction we can get. But if the dataset is more representative, we could have
the machine capture many other possible answers, such as ‘date’, ‘game’, ‘meal’,
‘vacation’, ‘grocery run’, and even the less common events that every human may
not be able to guess. Why may that be? We as humans meet several people in our
lives, watch TV and movies, text our friends, read, and perform many such activities
that open our imagination to different degrees. Let’s consider a person named Emma.
Emma is unmarried and has very few married friends. She may not able to guess where
one may go out with their spouse. This is because Emma hasn’t seen many examples
of such an event. However, a machine has the capacity to learn from a lot more data
 NLP Basics  17

than what a human brain can process and remember. Having large enough datasets
can not only represent Emma’s imagination of what David may be going out with his
spouse for, but also represent the imagination of several such individuals, and thus
make guesses that a single individual may not think of.

Does it mean the bigger the dataset, the better? Not

necessarily. The dataset needs to be big enough for the task
at hand. One way to measure it would be to evaluate your
model’s performance while incrementally increasing the size
of the data and observe at what data size the model stops
improving considerably. At that point, you probably have a
good enough amount of data!

Now we know that data quantity matters, let’s consider something a bit more
ambiguous now. Let’s say we want to infer which sentence is related to Art:

Fry onions in Safflower oil.

I use Safflower oil to slow the drying of my oil paints.

While ‘safflower oil’ is used in both examples, the topic of the first is completely
different from the second. This is known to humans because when we see the word
‘fry’ or ‘onions’ used with ‘oil’, it becomes apparent that it is likely not about art.
Similarly, ‘oil paints’ and ‘safflower oil’ used together seem likely to be about art. We
are able to make that inference because we know what food is and what paints are.
To make a machine understand the same, it is important to feed in relevant
training data so it can make similar inferences based on prior knowledge. If the
machine has never seen food items used in a sentence or has not seen it enough,
it would be an easy mistake to mark the first sentence as art if it has seen enough
samples of ‘safflower oil’ usage in art.
To successfully build an art/not-art classifier, we not only need a representative,
relevant, and good quantity of training dataset, but also preprocessing and cleaning
of data, a machine learning model, and numerical features constructed from the text
that can help the model learn.

1.3.4 Preprocessing
Data preprocessing refers to the process of passing the data through certain cleaning
and modification methods before analyzing or converting it into numerical represen-
tation for modeling. Depending on the source of data, the text can contain certain
noises that may make it hard for the machine to interpret.
For instance, consider a task where you have a list of text documents that were
written by people regarding a private review of a product. The product owners have
permission to display these reviews selectively on their websites. Now let’s talk about
18  Natural Language Processing in the Real-World

constraints. The program that needs these reviews as input to display on the website
cannot parse language other than English. So as a preprocessing step, you’ll remove
any non-English language content. Furthermore, the product managers desire to not
display any reviews having less than 10 characters of text on the website. Thus, you’ll
further apply a filtering step where you only pass the documents that have a length
of more than 10. But when you further look at the data samples resulting after the
filters are applied, you find some reviews contain meaningless information in the form
of random URLs and non-alphabets. Thus, for a cleaner output, you may pass the
data through further steps, such as removing URLs, checking for the presence of
alphabets, stripping leading and trailing spaces to get the relevant text lengths, etc.
All these steps count as preprocessing and are very tailored towards the goal.

For any text processing, it is valuable to first assess

your data for the types of noise it contains before analyzing
it for the final goal, whether that’s something like simply
displaying the text, or passing the data ahead for numerical
feature extraction. Your final NLP application is only ever
as good as your data!

Popular data preprocessing techniques are discussed in further detail in Chapter

3 (Section 3.1) and include data segmentation, cleaning, and standardization tech-
niques.

1.3.5 Numerical representation

There are several numerical representations of text that are known to convey meaning-
ful information. These are also referred to as features in machine learning. Whichever
way we obtain and prepare the text, to build a machine learning model we need to
have numerical vectors. This can be achieved by passing the text through numerical
transformations for the machine to make sense of it. This is also called feature for-
mation. Often, the data needs to go through preprocessing before the features can be
formed to remove noise and pass a cleaner input into the feature formation techniques.
Some popular numerical transformation methods include encoding, term frequency-
based vectorizers, co-occurrence matrix, and word embedding models. These will be
discussed in further detail in Chapter 3 (Section 3.4).
Once features are formed, they can be used as input to build machine learn-
ing models. Machine learning models aim to make sense of the data that is fed in.
Examples include sentiment analysis model, text classification model, and language
translation model. We’ll dive further into modeling including classic machine learn-
ing, deep learning, and transformers in Section III (Chapter 4) along with model
evaluation. We’ll also visit text visualization using Python and data augmentation
techniques in Section III (Chapter 3, Section 3.2 and 3.3).
 NLP Basics ■ 19

1.4 NLP CHALLENGES

All the applications of NLP that we have looked at thus far seem very useful. Since
practitioners have implemented some NLP applications before, are there existing
solutions available that one can leverage? Will an existing and already solved solution
fit any new or different NLP needs? Humans interpret language based on context
and understanding that the brain has picked up over the years of existence and
experiences. What does that mean for a machine?
Now that you know what NLP is, it is important to learn about the challenges
of building good NLP solutions. For instance, in machine learning, concept drift and
data drift are principles that relate to changes in data or data distribution requiring
model re-training and other cautionary considerations. The same applies to language
data as well. Below, we describe the challenges that occur due to language diversity,
language evolution, context awareness, and utilizing existing solutions.

1.4.1 Language diversity

The nature of natural language is diverse. People use completely different words while
describing different topics. People express themselves differently. While some basic
grammar rules exist in different languages, each individual writes and speaks in a
style that is unique to them. This variety in writing styles and language further adds
to the complexity of building NLP solutions.

1.4.1.1 Writing styles

Some sources of text data such as formal language articles and documents are less
likely to have a large variety in grammar usage, whereas platforms such as social media
allow users to use any language, abbreviations, emojis, emoticons, punctuation, or a
combination thereof. Thus, a perfectly built text category classifier trained on journal
articles may not work as well when applied to social media data, given the differences
in language style.
The difference in abbreviation usage, writing styles, and industry-specific jargon
usage can further bring massive differences in how training data is interpreted. Here
is an example of the same sentence that is written in different styles.

I am honored to be awarded the president’s award at tomorrow’s graduation event.

Im honored 2 b awarded d presidents award @ 2mrws grad event.

It is often found useful to communicate to the machine that ‘grad’ and ‘gradua-
tion’ mean the same, and words with or without apostrophes can be considered the
same for a dataset as such. There are techniques to achieve normalization as such,
such as stemming, lemmatizing, ensuring diverse dataset representation, and creat-
ing custom maps for word normalizations. This will be further discussed in detail in
Chapter 3 (Section 3.1).
Additionally, different geographical locations can represent different accents and
usage of words. Not only is the way of saying the same word different for the same
language across the world, but what they mean at times changes with geography. For
Random documents with unrelated
content Scribd suggests to you:
expedition, nor could it be of use or value, but rather a great
vexation and care to all; that he would consider the matter well as
the case demanded, and if it were advisable he would be the first to
assist in its execution, as he desired it."
 CHAPTER VII

P hilip II was much annoyed at the result of his interview with the
Nuncio Ormanetto, because he saw that the Pope was seriously
thinking of raising D. John from his dependent position by giving him
a crown, and that D. John, on his part, would go as far as his
chivalrous loyalty permitted. Antonio Pérez confirmed the King's
fears, making him see that the preservation of the forts of Tunis
contrary to D. Philip's opinion, and Juan de Soto's secret journey to
Rome, were acts of real independence; and, although he did not
dare to accuse D. John openly, he threw the blame on the secretary
Soto, attributing everything to his influence and intrigues, and again
insisted on the necessity of separating D. John from such a
dangerous adviser, and of substituting for him an energetic,
temperate man, who would know how to moderate these fiery
ambitions. This temperate, energetic man whom Pérez now ventured
to propose was Juan de Escovedo, a former retainer of the Prince of
Évoli and completely his creature, who was then secretary of the
King at the Treasury.
All this irritated and perplexed D. Philip. He did not wish to vex the
Pope, whose disinterestedness and holy aims he well knew; nor did
he wish either to deprive his brother of all hope, because, although
he did not doubt D. John's loyalty, he was afraid, having become
suspicious, of subjecting it to undue proof.
In this difficulty he judged very wisely that the danger would
cease with the opportunity, and he resolved to get rid, as far as
possible, of "this care and worry of Tunis," and in this sense wrote to
D. John the letter which we have already quoted. He also decided to
follow the advice of Antonio Pérez, appointing Juan de Escovedo as
secretary to D. John; and as his natural prudence and judgment saw
no reason for injuring Juan de Soto, or motive for depriving himself
of Soto's useful services elsewhere, he confirmed him in the
appointment of Commissary of the Fleet in Italy, which was both an
honourable and lucrative post.
So Juan de Escovedo set out to join D. John at Naples, bearing
express orders from the King and strong recommendations from
Antonio Pérez, to moderate D. John's ambitious aspirations, and
reduce him to a mere instrument of the policy of his brother, without
any views of his own. This man, celebrated afterwards for the
gloomy drama of which he was the victim, was then between forty-
five and fifty years old. He appeared rather a clownish peasant from
anywhere than a noble of the Asturias. He was of middle height,
thick-set, with heavy shoulders, and so swarthy and bilious-looking,
that in the secret correspondence of Philip II and Antonio Pérez he is
often designated by the name of "verdinegro" (the dark green one).
However, he compensated with interest for his rough ways and
absolute want of manners by his generous, unselfish nature, his
sterling honesty, clear understanding, and energetic activity, which
rendered him capable of facing all obstacles. Ruy Gómez and Luis
Quijada esteemed him much and the latter honoured him not a little
in his lifetime, and Doña Magdalena de Ulloa had retained, in her
retirement, such a happy recollection of his honesty and
uprightness, that no sooner did she hear of his new appointment
than she hastened to write him the following letter:
"Illustrious Sir; I desire to write to you to tell you the
pleasure that it gives me to see you in the company of the
Lord D. John, because I desire nothing in the world so much
as to see such persons about him, for I know the necessity
he has of this, and how he will profit by it, and as H.H. does
not neglect to keep me informed, I have begged him to
make the duty over to you, whom I ask to write by every
post whatever you think that I might care to hear about D.
John and what he does, and I also beg of you that no post
should leave without bringing me information, because if this
is not carried out, and the post comes with nothing, it gives
 me a great shock, and you can send the letters to the house
of D. Pedro Manuel, and I will reply by the same means or as
you may direct; and because I think that, although it will be
a trouble to you, you will do it as a favour to me, I end by
praying Our Lord to give you as good a journey and as much
success as I wish you. May Our Lord keep and prosper your
illustrious person as I wish. At your service,
Doña Magdalena de Ulloa."

Gregory XIII, on his part, never wavered in his ideas, and losing
all hope that Philip II would help him in his plans for the kingdom of
Tunis, he turned to another scheme which had been a failure in the
time of Pius V, but which he desired to resuscitate with fresh life and
vigour by confiding its execution to D. John of Austria, "because of
his valour and good fortune," quoth the Pontiff, who matured in
silence his secret project, which was to bring so much good to
Christendom and so much glory to D. John, and until he could
divulge it he contented himself with lavishing proofs of esteem and
affection on him, such as were then only shown to kings and
reigning princes.
In March of that year, 1574, he sent to Naples by his Chamberlain
the Golden Rose, blessed on Palm Sunday, which, according to
ancient custom, the Pope was in the habit of sending to the king or
queen who had deserved the most gratitude from the Holy See
during the year. This unusual honour frightened the Viceroy of
Naples, Cardinal de Granvelle, who was no friend of D. John, and he
hastened to apprise Philip II of the fact. The Chamberlain arrived at
Naples with the Golden Rose on the 24th of March, and on the 25th
the solemn presentation took place in the church of St. Clara. The
friars of St. Clara, great admirers of D. John, put a crimson velvet
carpet on the Gospel side, to receive him, with a chair and curtain,
as they would have done for an Infante of Spain. Granvelle knew of
this, but held his tongue and let it pass, in order to have something
against D. John, if he had accepted the honour; but he, warned in
time, ordered the canopy to be removed and another chair to be
placed at the left of his own for Granvelle.
Great was the enthusiasm in Naples over the new honour
bestowed on D. John, and all wished to take part in it. It was a point
of honour with the ladies to go to the festival with symbolical roses
in their head-dresses and at their breasts, and they bombarded
everyone, from the Cardinal to the smallest acolyte, with demands
for seats. It was, however, impossible to satisfy them all, and on this
day noble ladies were seen in the gutter, crowded on the stairs, in
the doorways, and even in the recesses of the chapel, anxious to see
everything and to be seen.
There were faintings from lack of air, cries of protest, bad-
tempered pushing, crumpled ruffs, crooked caps, creased petticoats,
unfastened shawls, lost jewels, and heaps of petals from the roses
that had occupied such honourable positions. The necklace of the
wife of an important Councillor was broken; it was a string of pearls,
and only half a dozen could be recovered.
D. John came between the Cardinal Granvelle and the Archbishop
of Monreale, and was followed by all the numerous princes,
marquises and counts who were in Naples, and by a crowd of
gentlemen. A Bishop celebrated the Mass, and the Bishop of
Castellamare, who was Chaplain to the King, gave D. John the Pax
and presented him with the Gospels to kiss. The Pope's Chamberlain
was on the Epistle side, on a seat without a back covered with
crimson velvet. He wore a black velvet cassock, and over it a
crimson garment. The Golden Rose was displayed on the high altar
in a big silver jar. It was of massive gold, with its foliage a foot high;
it had diamonds sprinkled over it like drops of dew, and the green
leaves were made of emeralds, some of enormous size. Mass over,
the Chamberlain took a brief of the Pope and gave it to D. John to
kiss, and then to a secretary to read aloud. The reading finished, D.
John knelt down on a cushion of crimson velvet before the Bishop
who had celebrated the Mass, and who, taking the Golden Rose from
the hands of an ecclesiastic, gave it to D. John, saying, "Our Holy
Father, Gregory XIII, Very Serene Prince, sends you this consecrated
rose, in token of his benevolence and paternal love. By his orders I
give it to your Highness."
To which D. John replied, "I kiss the feet of His Holiness for so
great a favour, and I receive the rose with the veneration due to
something so sacred, sent by the Vicar of Christ, universal Shepherd
and head of the Church."
At this time there broke out in Genoa the famous disturbances
between the old and the new nobility, called respectively "the Porch
of St. Luke" and "the Porch of St. Peter," and Philip II, who held the
protectorate of this republic, hastened to send D. John of Austria
with a few galleys to pacify the insurgents with skill and cleverness;
and if it were not possible to quiet them by any other means, to do
so by force of arms. The Pope heard of his passing Gaeta, which was
only twenty leagues from Rome, and on the pretext of greeting him,
sent his son Jacobo Boncompagni, who carried secret instructions to
apprise D. John of those mysterious plans over which the Pope had
long been meditating. Marco Antonio Colonna accompanied Jacobo
on his own account, as also did the Spanish ambassador in Rome, D.
Juan de Zúñiga.
The three illustrious personages came to visit D. John on the 18th
of April on board his galley with a numerous and brilliant suite, and
the next day D. John landed to give them a royally sumptuous
banquet in the house of the Governor of Gaeta. The long, wide table
was set in the principal saloon, with two places side by side laid with
services of rich plate, D. John giving the place of honour to Jacobo.
On the right, but at a respectful distance, was a similar place for
Marco Antonio Colonna, and at an equal distance on the left another
for D. Juan de Zúñiga. One hundred and twenty-three dishes were
served with all the viands and exquisite sauces for which Italian
cooking was then so famous, without counting dessert, which
covered the table three times, with different conceits of towers,
tournaments, castles, and wild beasts, with pastry and delicious
sweetmeats; more than forty kinds of wine were passed round. The
merriment and good temper of the illustrious guests never flagged
for a moment, and the crowd of noble gentlemen, who stood
respectfully watching the banquet, snatched a mouthful at the
sideboard, and were satisfied with abundant cups of wine.
At the end of dinner Boncompagni asked D. John's permission to
present him with the gift that Gregory XIII had sent: some very rich
tournament armour, a great black velvet pouch full of gold medals
that had been blessed, which D. John divided among those present,
and a little chest of red velvet containing a beautiful group of the
Crucifixion, of great artistic merit. The Pope kept this chest in his
rooms, and it was enriched with numerous indulgences.
In return for these presents D. John gave Boncompagni a horse
worth 500 ducats, and its trappings which cost 2500, and a sword
ornamented with gold worth 800 ducats.
The next day, on board the galley "Real," under the awning of
striped red and white damask which stretched in front of D. John's
cabin, Boncompagni confided to him the mysterious enterprise which
Gregory XIII proposed to undertake with D. John's help. D. John
listened attentively in silence, from time to time his blue eyes flashed
with enthusiasm. It was a question of setting at liberty a beautiful
captive queen and snatching a kingdom from the heretics.
 CHAPTER VIII

M eanwhile the capture of Tunis had made patent to all Europe how
deep was the wound that the credit and power of the Ottoman
Empire had received at Lepanto. This great defeat was no doubt a
disaster for the Turks; but a glorious disaster, both on account of the
deeds of valour they had performed, and the titanic efforts it cost
the victors to gain the triumph. The flight from Tunis without firing a
shot, at the mere presence of D. John two years after this rude
warning, showed how deeply rooted was the terror in the souls of
the infidels, and how the renown of the Christian bravery had been
enhanced, especially that of the Spaniards. All this cruelly wounded
the overweening pride of Selim, and he was consumed with the
desire of avenging himself by reconquering Tunis and Goletta. He
was urged, with malicious eagerness, to undertake this enterprise by
Aluch Ali and the renegade Mustafa, one of the engineers who built
Goletta in the time of Charles V. The name of this traitor was Jacobo
Zitolomini. Resenting the treatment he had received from Philip II,
he fled to Algiers to Aluch Ali, who took him to Constantinople,
where he revealed to Selim a sure and secret way of taking Goletta.
At the beginning of May, 1574, D. John received an urgent
message from Gabrio Cervelloni, to say that the Turks were
preparing a very powerful fleet; that it was feared that they would
fall suddenly upon Tunis, and that, in consequence of this, funds
should be sent as quickly as possible to finish the new fort, which
was not yet completed. D. John was at Genoa, quieting the
disturbances there, and he hastened to send the Commissary of the
Fleet Juan de Soto to Madrid, to warn Philip II of the danger which
threatened. The King was not much alarmed about this, and,
perhaps, saw a prompt and certain opportunity of ridding himself of
this care and worry of Tunis. At any rate, his answer makes it clear
that this new conquest was the least of his cares, and while writing
to Cardinal Granvelle, Viceroy of Naples, and to the Duque de
Terranova, Regent of Sicily, that they were to guard the ports and
reinforce the garrisons, especially in Messina, Augusta, Syracuse,
Trapani and Palermo, he contented himself with adding that they
were not to forget to help his brother, and to look after the coast of
Barbary. He also ordered D. García de Toledo and the Marqués de
Santa Cruz to watch how D. John was garrisoning Goletta, and to
the latter the King wrote that he was to do what he judged best in
the matter, but that he was to remember that he had said that 2000
foot soldiers were enough to defend Goletta. D. John then sent D.
Juan de Cardona with all the galleys under his command to Tunis
without loss of time, taking the help for which Gabrio Cervelloni
craved. This was not sufficient, and those in Tunis reiterated their
request. D. John then exhausted all his resources, and sent D.
Bernardino de Velasco with twenty Neapolitan galleys and four
companies of Italian infantry. With these comings and goings
summer was getting old, and on the 13th of August, at the Cape of
Carthage, appeared the dreaded Turkish fleet of 300 ships and
60,000 soldiers, the fleet being commanded by Aluch Ali, and the
troops by Selim's son-in-law Sinan Pasha, the renegade.
A great outcry arose at the extreme peril of the Barbary
Christians, and by every means in their power they sent to ask help
from Granvelle, Terranova, and, above all, from D. John of Austria, in
virtue of his office and Christian piety. He wished to fly to their aid,
abandoning everything. He wrote first, however, to the Duque de
Sesa, "To urge the Cardinal to send people to help Goletta, as that
province was in his charge." But the imperturbable Granvelle replied
coldly, "That he had much to guard in the Kingdom, and that it did
not suit him to divide his forces." "This was," says Vander Hammen,
commenting on the fact, "to give colour to the excuse. The real
reason was Granvelle's dislike to support D. John, jealous of his
favours with Mars and Venus, and because he was a foreigner, and
because his brothers conspired in the Flemish rebellion;" and Luis
Cabrera de Córdoba expresses himself in similar words, equally
severe, not forgetting Mars and Venus. And D. John himself wrote to
his sister Donna Margarita: "In short, Lady, everything goes badly;
and in truth it is not entirely the fault of His Majesty, except for
permitting those who govern his States to forget that those in their
vicinity, or those that are not, are as much His Majesty's as those
which each minister has charge of."
Meanwhile D. John, tired of waiting for orders, troops and money
which did not come, and making it a point of honour to go to Tunis,
moved with desperate activity from Genoa to Naples, Messina and
Palermo, recruiting soldiers everywhere, chartering ships, and
pledging for all this his plate, his jewels, and even his word, until he
had collected at Messina a moderate fleet with no lack of fighting
men. He was all ready to sail for Africa, when he met with another
obstacle, more powerful than the calculated coldness of Philip II, or
the jealous hatred of the Cardinal. The sea! The terrible sea which
rose in a furious storm which threw him to Trapani, much against his
will, and kept him there days and days, giving time for Christians to
perish and for the Turks to become victorious. Three times he tried
to leave the port, defying the storm, and as often had to retire
before the surging waves. Then he sent four galleys without quarter-
decks and platforms on the forecastles to take the mere hope of
help to Goletta, but the implacable tempest prevented this by
destroying two of them. At last the weather improved; but before D.
John could put to sea, a French galley, without masts and knocked
about by the storm, was driven into the port of Trapani. On board of
her was D. Juan Zagonera, with fifty soldiers, all that remained at
liberty of the garrison that D. John had left in Barbary. From them he
heard of the terrible disaster. The Turks were in possession of Tunis;
three thousand soldiers were dead, and the rest wounded or
captives; Pagano Doria had been beheaded; Gabrio Cervelloni, D.
Pedro Portocarrero and D. Francisco de Avila were the slaves of
Sinan; the new fort razed before it was finished; and Goletta, the
glorious legacy of Charles V, blown up by mines, and erased from
the African soil by Aluch Ali, as the wind of the desert obliterates
footprints.
Those who were jealous of D. John blamed him for this
catastrophe, with which he had had nothing to do; the sensible
public opinion, at times so right and sharp, blamed Granvelle, and
songs, which have come down to us, were sung on the subject in
the streets. A few, but very few, said in a whisper, as in those days it
was necessary to do, that the Cardinal was not responsible, since in
refusing aid to Goletta he had obeyed secret orders from Madrid. Of
this, however, absolutely no proof exists.
The energetic nature of D. John was not depressed by this bad
news; but it awoke a thousand different sentiments in his mind, and
under the impression of disgust, sorrow and wounded dignity, and,
above all, his loyal frankness, which always urged him to treat
questions openly and in a straightforward manner, he resolved to go
to Spain to confer with his brother Philip II face to face about three
different questions which were connected with each other—as to his
remaining permanently in Italy as Lieutenant-Governor of those
States, his recognition as Infante of Castille, and the mysterious
scheme that Gregory XIII had proposed to him.
So it fell out, and by January, 1575, D. John was already in
Madrid. On the 15th of February he wrote to his sister Donna
Margarita:
"Lady, I, praised be God, arrived a few days ago at this
Court, where I have received such kindness from His Majesty
that only to have gained this I consider that I have spent my
life well. Since my arrival I think that he understands Italian
affairs very differently from what he did before. I had
thought, as I had prayed His Majesty, to stay some time in
Madrid; but in the end he is resolved to order me to return
to those parts, and is in a great hurry to send me off. I think
that I shall start in the middle of the coming month, and I
also think that I go to begin a new sort of service according
to what suits His Majesty. Meanwhile one has to overcome
 difficulties and hasten on the things required for this
summer's campaign.
"To all this I pay so much attention that each day, in
councils and out of them, I do nothing else. It is already
drawing so near summer, that I am satisfied with nothing
that I do not see. Here, Lady, everything is councils; every
day I hold two, besides a thousand other occupations, which
leave me no time that I can call my own."
D. Philip was under the spell of the fascination that D. John always
exercised, and, notwithstanding the groundless suspicions of Antonio
Pérez, he received his brother with loving affection and the gratitude
and graciousness due to a leader who had added such lustre and
glory to the arms and name of Spain. He listened long, and with
great interest, to D. John's information about Italian affairs,
changing his opinion much about them. He agreed with D. John in
blaming the Ministers and Viceroys of those States, especially
Granvelle and the Duque de Terranova. He talked over and fixed the
loans which should be made to the various councils to enable them
to guard themselves that summer against the Turk, whose pride had
to be humbled after the recent capture of Tunis; and finally
appointed him, with the approval of the whole council, and to the
secret horror of Antonio Pérez, his Lieutenant-General of all Italy,
with authority over all the Viceroys and Ministers who governed
those States. This, however, was to remain a secret, to spare the
reputations and prestige of these functionaries, and was only to be
manifested in case of abuse of authority or boast of independence.
"This for Y. Highness only, I beg for many reasons," wrote D. John
from Naples to Donna Margarita. "I also bring an order that
everyone has to act with obedience; but this is only to be used when
some Minister persuades himself to the contrary, which I do not
think will happen, as by letters they have learnt what concerns
them."
D. John, encouraged by this, dared to present the second part of
his programme, which was that, in order to wound no one and to
give an outward sign of this supremacy over the Italian Ministers,
the King should concede to him the rank and title of Infante, which
was spontaneously given him by all, great and small. D. Philip did
not like to refuse this well-deserved favour, but with excuses made
D. John understand that the time was not ripe for this. He did not do
this out of ill-will, or from miserly stinginess, or still less from
jealousy of his fame and renown, as some say, but because it was
one of the maxims of this prudent King, inherited from his father
Charles V, to stimulate the services of the Grandees with a reward in
proportion to their rank; and without giving D. John a crown, which
Philip did not wish to do, there remained no other reward worthy of
him but the title of Infante, and it seemed premature to give him
this now, considering the many and important services Philip hoped
to obtain from him in the future.
As to the project of Gregory XIII, D. John did not have to broach
the subject to his brother. D. Philip himself began it, having already
talked over and settled it with the Nuncio Ormanetto.
 CHAPTER IX

F our years before these events, in June, 1571, a little old, nervous
and active Italian arrived in Madrid. He called himself Giulio
Benasai, a merchant from Genoa; he stopped at an inn, near the
gate of the Viper, now the Puerta Cerrada, and very early the next
day began his visits, which were anything but commercial ones. He
visited Monsignor Ormanetto, the Pope's Nuncio; Dr. Milio, governor,
in the Duke's absence, of the Alba estates; the secretaries Zayas and
Mateo Vázguez, and lastly, five days after his arrival, on the 28th, he
visited the King, Philip II, at the Castle. This visit, however, was very
different from the others, it was paid secretly at night, and once
inside the Castle he no longer called himself Giulio Benasai, or a
native of Genoa, or a merchant. His name was Roberto Ridolfi, a
banker in London, and secret agent of His Holiness Pius V in that
heretic country.
Ridolfi gave three letters, substantially alike, into Philip's own
hands. These begged him to give Ridolfi his entire confidence, and
to undertake what he would explain, granting all the resources he
deemed prudent in order to further the enterprise. They were from
no less personages than Pius V, the Queen of Scots, Mary Stuart,
then a prisoner in England, and the third one from the Duke of
Norfolk.
The project was this; to capture the heretic Queen of England,
Elizabeth, and the lords of her Council, and shut them up in the
Tower of London; to marry the lawful Queen, Mary Stuart, to the
Duke of Norfolk, and in this way to re-establish Catholicism in
England and Scotland. Philip's aid was sought for the plan, and they
had already obtained the support of the most influential English
lords and of Mary's partisans in Scotland, who were then numerous
and powerful. The Pope had prepared the way by hurling his terrible
Bull against Elizabeth, declaring her to be an obstinate heretic and
an abettor of heresy, deposing her from the English throne and
absolving her subjects from their oaths of fealty and obedience. He
promised, moreover, all the funds that the Holy See had at its
disposal.
For this enterprise the Duke of Norfolk asked the King of Spain for
6000 arquebusiers, 4000 arquebuses, 2000 cuirasses and 25 pieces
of artillery, with the necessary money and ammunition. He promised,
for his part, to raise in England 3000 horsemen and 2000 foot
soldiers and to undertake the dangerous task of capturing the Queen
and her Councillors and of setting Mary Stuart free. He also
promised to remain on his estates in Norfolk, facing the coast of
Holland, to protect the landing of the troops that the Duque de Alba
was to send from Flanders. The Duque had talked to Ridolfi in
Brussels and approved of the plan, with certain reservations, and
even thought it an easy one, once Elizabeth was either captured or
dead; he waited, however, for the orders and consent of his
Sovereign.
Philip II listened to Ridolfi with his usual reserve and caution, and
sent him to the Escorial, where the Duque de Feria examined him at
length, and where an important council was held on the 7th of July,
the minutes of which are preserved in the archives at Simancas.
They all approved of the plan and agreed to order its prompt
execution by the Duque de Alba. But such was the slowness of Philip
in settling the details and such was his indecision about dictating the
last orders, that time was given for Norfolk to be denounced, tried,
and publicly beheaded in London.
It was this scheme, ruined by the death of Norfolk, which Gregory
XIII wished to resuscitate. He sent another Bull, similar to the one of
Pius V, giving the sovereignty of England to her legitimate Queen,
Mary Stuart, and marrying her to D. John of Austria, who was to
command the Spanish hosts which were to invade England. The
Pope had already consulted the English and Scotch lords and other
magnates who were willing to support Norfolk's movement, and they
undertook to perform all that they had previously promised to the
unhappy Duke. To reinstate the plan in the same advantageous
position it had held in the days of Pius V only the consent and help
of Philip and D. John were lacking. At his interview at Gaeta with
Jacobo Boncompagni D. John enthusiastically gave his consent,
subject to his brother's will, which was for him an unbreakable law.
But Philip, on his part, received the proposal coldly when it was
unfolded to him in the name of Gregory XIII by the Nuncio
Ormanetto; he very courteously thanked the Pope for the favour
shown to his brother, but excused himself from helping the
enterprise because of the necessity there was of concentrating large
armies in Italy for fear of the Turk, who had been heartened by the
triumph of Tunis, and in Flanders where the rebels were also
encouraged by the departure of the Duque de Alba. And as the
Nuncio argued, pointing out the truth so well known to the
politicians of the day, that the focus of the rebellion had to be
stamped out, not in Flanders, but in England, where the Queen was
always stirring it up and helping the rebels in every way, D. Philip
answered that this was true and that he knew it full well; but that all
the same he could not remove a single pike from Flanders until the
new policy of gentleness and reconciliation, which he had entrusted
to the Knight Commander Requesens, had taken effect. Then he
would consider whether or no the expedition to England would suit
him.
Philip gave his brother the same answer when they treated of the
circumstance, adding other reasons, all tending to bind D. John
tighter to his service, without disappointing him or at once
dissipating the dreams he might have woven round such a romantic
plan as conquering a kingdom by setting a beautiful captive queen at
liberty, which must have appealed so strongly to D. John's chivalrous
fancy. So D. Philip promised, without any intention of fulfilling it,
according to Antonio Pérez, or as we think, meaning to do so if it
suited the plans of his policy to favour Gregory's scheme when the
danger of a fresh war with the Turk, which then threatened, was
over.
 And as if to bring D. John down from the sphere of heroic ideas,
where genius usually dwells, to the petty weaknesses among which
most mortals struggle, in the next line the King spoke of what in
certain ways was the only thing which could humiliate and shame D.
John, and which embittered his life—the conduct of his mother—
which had reached such a pitch that no one frequented her house
but low persons, among whom was an Englishman, supposed to be
on too intimate terms with her. The Duque de Alba, who, though
severe, was not straitlaced, had upbraided her without success
several times, and, tired out, had decided to write the following
letter to the secretary Zayas:
"Very magt. Sir. An affair is taking place here which much
troubles me, because I have tried by every means to remedy
it, without success, and it has reached such lengths, that it
would be well if H.M. should quickly cure it. You will be doing
me a favour to tell H.M. that the mother of D. John lives with
so much liberty, in a manner so unlike that in which the
mother of such a son should live, that it is necessary to put a
stop to it, as the affair is so public and so free and open that
they tell me that no honourable woman will enter her doors.
Things have come to such a pass that they are changing the
servants every week, and in my absence she has gone so far,
that most days there are dances and banquets. She has
turned out the two honourable old spinsters I placed near
her and has filled their places with low women. She is
dreadful and very obstinate. His Majesty will order what he
wills, I had resolved to take her by night and put her in a
convent, but I did not like to do so without first consulting
him."
D. Philip answered the Duque de Alba by the following letter
written in cipher.
"The King.
"Duke and Cousin. Cayas has shown me the letter you
wrote about my brother D. John's mother, which, for reasons
you mention and enter into, grieves me much, because she
 does not live with seemly modesty and respectability; and it
appears to me, as it does to you, that the only thing to do is
to bring her here, and her son is also of the same opinion, to
whom I have sent Juan de Soto, to say I have done it for her
welfare, these States being in the condition they are, without
saying more, as there was no need to do so, and, as I
understand the journey is to be by sea, if they were to tell
her beforehand, it is very likely that she would do something
foolish; it would be well to keep her in the dark until a safe
ship is found, and then, everything being ready and the
weather fine, to put her on board, whether she likes it or
not, with a suitable retinue, giving orders that everything
necessary for the journey should be provided, and that
during it she should be well treated. Let me know in time,
that she may be met at the port, and from there taken to the
nearest and most suitable convent, which I have not yet
decided on."
It was not the first time that the brothers had talked about this
painful subject; but now D. John knew all, without palliation or
reserve. D. Philip told him in wise and tactful words, like a kind
surgeon, who, without wishing to pain, probes a wound, and
suggested the remedy like a father who discusses a sad family
matter. As there was no other way, they determined to remove
Barbara Blombergh from Flanders by deceiving her, and to bring her
to Spain, where, by D. John's proposal, she should be given into the
charge of Doña Magdalena de Ulloa, that this noble lady should
settle her in her own proximity, wherever, in her prudence, discretion
and charity, she deemed best. This idea appeared an excellent one
to D. Philip, and a few days later D. John set out for Abrojo, where
Doña Magdalena was expecting him.
Never had the mourning figure seemed so dignified to him, or had
he found such sweet and deep repose in her company, or thought
that he saw in her still beautiful eyes such intense love, such
maternal solicitude, or such tender grace as when she showed him
the big coffers of linen which she had ready for him, neckties of
Flemish point which she herself tried on, and the full starched ruffs,
very full as she knew that he liked them.
And it was his yearning for a mother, exasperated by the
disillusion about his own parent, that was comforted by the pure
love and great virtues of that other whom a merciful Heaven had
sent to him. D. John stayed four days at Abrojo, confiding everything
that was on his mind to Doña Magdalena, joys and sorrows, hopes
and fears, triumphs and disappointments, errors and repentances;
and when she said good-bye at the convent door, she thought, as
she did the first time she saw him on the staircase of Villagarcia, "It
is a pity he is not really my son!" And he said to himself with infinite
bitterness, as he kissed her hand for the last time, "It is a pity that
she is not really my mother."
D. John left Abrojo with the profound regret and vague mistrust
that a wanderer feels who sets out on the desert sand after a day's
rest in an oasis. A friendly voice, however, encouraged him all that
day, telling him that a glorious future was his, if he fought with
firmness and waited with patience, which is the advice constancy
gives that fiery activity should achieve its purpose: that Gregory
XIII's idea was certain to be realised because it was great and just,
and easy and feasible, and in the end he would share the throne
with the hitherto unfortunate Queen of Scots; the England of D.
John and the Spain of Philip being the two strong pillars of the Holy
Catholic Church.
He who spoke thus to D. John was Escovedo, whom Philip had
commissioned to moderate D. John's ambitious thoughts. And the
most strange thing is that Escovedo was clever and honest and was
talking seriously.
 CHAPTER X

A ntonio Pérez relates in his famous "Memorial" that at first the

secretary Escovedo served the King very well in moderating the
ambitious ideas of D. John, but that "as time went on it was evident
that he not only did not fulfil the purpose for which he had been
sent (to Italy), but that his boldness, like that of Juan de Soto,
increased, and that in particular it was known that he began to have
communication with Rome for some benefit or grandeur for D. John,
without informing his Majesty."
There is truth in all this, mingled with much falsehood, as is the
case with all the contents of this venomous book. Escovedo never
thought that D. John was acting through vulgar ambition; although
vulgarity is found in all spheres, it was distasteful to his heroic
nature; but he really thought, as Antonio Pérez assured him, that D.
John was blinded by his ambitious ideas, and was craving the
protection of Rome to carry into effect visionary plans, which, to say
the least, would hamper his brother's policy, and that, in short, he
was a brave youth, flushed by his victories, whom it would be
necessary to lead by the hand along the beaten path of common
sense, that he should not engulf his own great qualities in the abyss
of daring and fantasy. This is what Escovedo believed about D. John
when he went for the first time to Italy in the capacity of secretary:
but, learning to appreciate the frank charm of his ways and the
simple cheerfulness of his loyal character, he retracted this opinion,
and, little by little, the more he knew of D. John and his affairs, the
more he became convinced that what Antonio Pérez called his
foolhardy notions were really vigorous flights of genius; and that
what he named fantastic plans were the well-considered schemes of
two Pontiffs like Pius V and Gregory XIII, who were those that
Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.

More than just a book-buying platform, we strive to be a bridge

connecting you with timeless cultural and intellectual values. With an
elegant, user-friendly interface and a smart search system, you can
quickly find the books that best suit your interests. Additionally,
our special promotions and home delivery services help you save time
and fully enjoy the joy of reading.

Join us on a journey of knowledge exploration, passion nurturing, and

personal growth every day!

ebookbell.com