NLP Lab Internal Question Bank
NLP Lab Internal Question Bank
import spacy
import pytextrank
nlp = spacy.load("en_core_web_lg")
nlp.add_pipe("textrank")
example_text = """Deep learning (also known as deep structured learning) is part of a
broader family of machine learning methods based on artificial neural networks with
representation learning. Learning can be supervised, semi-supervised or unsupervised.
Deep-learning architectures such as deep neural networks, deep belief networks, deep
reinforcement learning,recurrent neural networks and convolutional neural networks have been
applied to fields including computer vision, speech recognition, natural language
processing,machine translation, bioinformatics, drug design, medical image analysis, material
inspection and board game programs, where they have produced results comparable to and in some
cases surpassing human expert performance. Artificial neural networks (ANNs) were inspired by
information processing and distributed communication nodes in biological systems. ANNs have
various differences from biological brains. Specifically, neural networks tend to be static and
symbolic, while the biological brain of most living organisms is dynamic (plastic) and analogue. The
adjective "deep" in deep learning refers to the use of multiple layers in the network. Early work
showed that a linear perceptron cannot be a universal classifier,
but that a network with a nonpolynomial activation function with one hidden layer of unbounded
width can.Deep learning is a modern variation which is concerned with an unbounded number of
layers of bounded size,which permits practical application and optimized implementation, while
retaining theoretical universalityunder mild conditions. In deep learning the layers are also
permitted to be heterogeneous and to deviate widelyfrom biologically informed connectionist
models, for the sake of efficiency, trainability and understandability,whence the structured part."""
print('Original Document Size:',len(example_text))
doc = nlp(example_text)
for sent in doc._.textrank.summary(limit_phrases=2, limit_sentences=2):
print(sent)
print('Summary Length:',len(sent))
3. How you can perform transliteration using the Indic NLP Library
Code:
from indicnlp.transliterate.unicode_transliterate import
UnicodeIndicTransliterator
# Input text "Today the weather is good. Sun is bright and there are no
signs of rain. Hence we can play today."
input_text='आज मौसम अच्छा है। सूरज उज्ज्वल है और बाररश के कोई संकेत नह ं हैं।
इसललए हम आज खेल सकते हैं!'
# Transliterate from Hindi to Telugu
print(UnicodeIndicTransliterator.transliterate(input_text,"hi","te"))
Code:
from indicnlp.tokenize import sentence_tokenize
indic_string="""तो क्या लिश्व कप 2019 में मैच का बॉस टॉस है? यान मैच में हार-ज त में \ टॉस क भूलमका
अहम है? आप ऐसा सोच सकते हैं। लिश्वकप के अपने-अपने पहले मैच में बुर तरह हारने िाल एलशया क दो ट मों \पालकस्तान
और श्र लंका के कप्तान ने हालांलक अपने हार के प छे टॉस क दल ल तो नह ं द , लेलकन यह जरूर कहा था लक िह एक अहम
टॉस हार गए थे।"""
# Split the sentence, language code "hi" is passed for
hingisentences=sentence_tokenize.sentence_split(indic_string, lang='hi')
# print the sentences
for t in sentences:
print(t)
Code:
import nltk
from nltk.chat.util import Chat, reflections
nltk.download('punkt')
# Sample patterns and responses
patterns = [
(r'hi|hello|hey', ['Hello!', 'Hi there!', 'Hey!']),
(r'how are you', ['I am good, thank you!', 'I\'m doing well, how about
you?']),
(r'(.*) your name', ['I am a chatbot.', 'You can call me ChatGPT.']),
(r'(.*) (age|old)', ['I am just a computer program, so I don\'t have an
age.']),
(r'quit', ['Bye!', 'Goodbye.', 'Take care!']),
]
# Create a chat instance
chatbot = Chat(patterns, reflections)
def main():
print("Hello! I'm a chatbot. Type 'quit' to exit.")
while True:
user_input = input('You: ')
if user_input.lower() == 'quit':
print("Chatbot: Bye!")
break
response = chatbot.respond(user_input)
print(f'Chatbot: {response}')
if __name__ == "__main__":
main()
Code:
from sklearn.feature_extraction.text import CountVectorizer
Code:
from indicnlp.transliterate.unicode_transliterate import
ItransTransliterator
input_text='आज मौसम अच्छा है। इसललए हम आज खेल सकते हैं !'
# Transliterate Hindi to Roman
print(ItransTransliterator.to_itrans(input_text, 'hi'))