Module-5

Recurrent and Recursive Neural Networks: Unfolding Computational Graphs, Recurrent Neural
Network, Bidirectional RNNs, Deep Recurrent Networks, Recursive Neural Networks, The Long
Short-Term Memory and Other Gated RNNs.
Applications: Large-Scale Deep Learning, Computer, Speech Recognition, Natural Language
Processing and Other Applications.
Textbook 1: Chapter: 10.1-10.3, 10.5, 10.6, 10.10, 12.

###I. Define Recurrent Neural Networks. Explain the Unfolding Computational

Graphs and its advantages.

Recurrent Neural Networks (RNNs) are specialized neural networks for handling sequential data, like time
series or sentences, just as convolutional neural networks (CNNs) are specialized for image data. Key points
include:
1. Sequence Processing: RNNs process sequences of variable length, making them suitable for tasks like
language translation or time-series prediction.
2. Parameter Sharing: Unlike traditional networks, RNNs share parameters across time steps. This
allows generalization across sequences of different lengths and positions.
o Example: If a model is asked to find a specific year in sentences, RNNs can identify the year
regardless of its position.
3. Comparison with Convolutional Approaches:
o CNNs (in time-delay neural networks) use convolutions to process temporal sequences, sharing
parameters over time but in a shallow manner.
o RNNs, however, use deep computational graphs, where each output depends on previous
outputs, applying the same update rule iteratively.
4. Flexibility in Data: RNNs can work with both temporal data (e.g., time-series) and spatial data (e.g.,
images). They can also process sequences bidirectionally if the entire sequence is available.
5. Cycles in Computational Graphs: RNNs include cycles in their computational structure, enabling
them to model dependencies over time.
This structure makes RNNs powerful tools for tasks like speech recognition, text analysis, and time-series
forecasting.

RNNs are neural networks designed to process sequential data, like time series, language, or speech, where
each part depends on the previous ones. They have a "hidden state" (memory) that updates with each input,
allowing them to remember past information and make better predictions.

Key Features:
1. Memory: RNNs use a hidden state to store information across time steps.
2. Efficiency: The same weights are used for all steps, making them effective for sequences of any length.
3. Learning: They are trained using a method called Backpropagation Through Time (BPTT) to
improve performance.

10.1 Unfolding Computational Graphs

Unfolding a computational graph is a method used in Recurrent Neural Networks (RNNs) to simplify
repetitive calculations and make training easier. Here's what it means and why it’s useful:

What is a Computational Graph?

• A computational graph visually shows how data flows and calculations are performed in a neural network.
• In RNNs, the graph forms a loop because each output depends on the previous state (hidden state).
How It Helps:
1. Understand Data Flow:
o Unfolding shows how information moves step-by-step, instead of being hidden in a loop.
2. Simplifies Training:
o Backpropagation (adjusting the model to learn) becomes easier when you can see every step.

Advantages of Unfolding:
1. Parameter Sharing: The same function ff and settings (θ) are used at every step, saving resources and
ensuring consistency.
2. Handles Different Lengths: The same model can process sequences of different lengths using this setup.
3. Fixed Input Size: After unfolding, each step processes inputs of the same size, simplifying calculations.
4. Easier Gradient Calculation: Unfolding the graph helps calculate the gradients (used to improve the
model) more easily.
5. Generalization: The same function is applied at each time step, which helps the model work with
sequences of different lengths, even if it hasn't seen those exact lengths during training.

Analogy:
• Think of a train journey with multiple stations:
o Each station depends on the previous one (you can’t skip).
o Unfolding lists all the stations in order, showing the train’s exact path.
o This helps understand the journey and fix problems if needed.
Unfolding is crucial for RNNs to process sequential data (e.g., text, time-series) effectively. It transforms
complex loops into clear, linear steps, making models easier to train and understand.

#MQP.With an example, show how the Recurrent Neural Networks (RNN’s) process data sequence
using computational graphs. ( 3 different design patterns of RNN).

Recurrent Neural Networks (RNNs) are a powerful class of neural networks designed to process sequential
data, such as time series or natural language. These networks are equipped with recurrent connections,
allowing them to maintain and process information from previous time steps in a sequence. The design patterns
for RNNs can be illustrated through different network structures, as described in the computational graphs.
•Recurrent networks that produce an output at each time step and have recurrent connections between hidden
units, illustrated in figure . 10.3
•Recurrent networks that produce an output at each time step and have recurrent connections only from the
output at one time step to the hidden units at the next time step, illustrated in figure 10.4
•Recurrent networks with recurrent connections between hidden units, that read an entire sequence and then
produce a single output, illustrated in figure . 10.5

1. RNN with Recurrent Hidden-to-Hidden Connections (Figure 10.3):

1. RNN with Hidden-to-Hidden Connections:

● Structure: The network looks at each input in a sequence and passes information from one
step to the next.
● How it works: The hidden states (memory) from each time step are passed to the next time
step, which helps the model remember things from earlier inputs. This is useful when the model
needs to understand long sequences (e.g., predicting the next word in a sentence).
● Training: The model adjusts its parameters by comparing the output to the expected result
and updating based on that difference.

RNN Overview: Takes a sequence of inputs x(t), processes them step by step, and produces outputs o(t).
Weights:
● U: Input to hidden.
● W: Hidden to hidden (recurrence).
● V: Hidden to output.
Loss: Softmax applied to o(t) gives predictions ypred(t), compared to actual yt to compute loss L.
Unfolded Graph: Visualizes RNN processing over time, showing how each step depends on the previous
one.

2. RNN with Output-to-Hidden Recurrence (Figure 10.4):

● Structure: Here, only the output at each time step is used to influence the hidden state at the
next time step.
● How it works: This setup is simpler but less powerful because it doesn't allow the hidden state
to retain much of the sequence’s context.
● Training: It can be easier to train but might struggle with tasks that need remembering long-
term information.

● Example: Sentiment analysis on text sequences.

3. RNN with Sequence-to-One Output (Figure 10.5):

Structure:
• The network processes the entire sequence and produces a single output at the end (o(τ)o(τ)).
• This output summarizes the entire sequence into a fixed-size representation.

How It Works:
• Useful for tasks where the decision depends on the whole sequence, like:
o Sentiment analysis (e.g., determining if a sentence is positive or negative).
o Sequence classification.

Training:
• The network learns by comparing the final output to the correct target and adjusting weights
using Backpropagation Through Time (BPTT).
• Errors are propagated from the final output (o(τ)o(τ)) back through the entire sequence.

• Applications:
• Tasks like summarizing text, classifying sequences, or even generating music/text rely on this
architecture.

Figure 10.5 illustrates a time-

unfolded RNN that processes a sequence x(1),x(2),…,x(τ) and produces a single output o(τ) at the final time
step. This output summarizes the entire sequence into a fixed-size representation, suitable for tasks like
sequence classification or sentiment analysis. Training involves backpropagation through time (BPTT),
propagating the error from o(τ) backward through all time steps. This architecture is ideal for tasks requiring
the entire sequence to inform the final output.

#10.2.1 Teacher Forcing and Networks with Output Recurrence

Teacher forcing is a training method for Recurrent Neural Networks (RNNs) that have connections
from their outputs to the hidden states in the next time step. It helps stabilize and improve training,
especially for sequence generation tasks. Here's a simple breakdown:
Advantages:
1. Faster Training: Ensures stable learning and quick convergence by feeding correct inputs.
2. Error Prevention: Reduces the impact of compounding errors during early learning.

Challenges:
1. Training-Testing Gap: The model may struggle when transitioning to testing, as it hasn’t learned to
rely on its own outputs.
2. Open-Loop Issues: Predicted inputs during testing might drift, leading to poor results.

Solutions:
1. Mixed Training: Combine teacher-forced and free-running inputs to prepare the model for real-
world usage.
2. Scheduled Sampling: Gradually transition from ground truth inputs to model-generated outputs
during training.

#10.2.2 Computing the Gradient in a Recurrent Neural Network

When training an RNN, we need to calculate how much each parameter (weights, biases) contributes to the
error (difference between predicted and actual values). This is done using gradients—essentially instructions
on how to adjust these parameters to make the model better.

How Do We Compute Gradients in an RNN?

1. Unrolling the RNN:
o An RNN has a loop where outputs depend on previous steps.
o To compute gradients, we "unroll" this loop, turning it into a step-by-step sequence of operations for
all time steps.
2. Backward Pass (Backpropagation Through Time - BPTT):
o Once the graph is unrolled, we calculate gradients by moving backward in time, starting from the
last step and working to the first step.
o This involves:
§ Gradients for the output at each step.
§ Gradients for the hidden state (the "memory" of the RNN).
§ Gradients for the parameters (weights).
3. Using the Chain Rule:
o Gradients are computed for each step by applying the chain rule, which links the effect of each
parameter through time.

Example:
• Output Gradient:
o How wrong was the model’s prediction? (Difference between predicted and actual values.)
• Hidden State Gradient:
o How much does the memory of the RNN (hidden state) affect the output?
• Parameter Gradient:
o How much do the weights (W,U,V) contribute to the error?

Challenges in Gradient Computation:

1. Vanishing Gradients:
o When gradients become very small as you go backward in time, it’s hard for the model to learn
long-term dependencies.
2. Exploding Gradients:
o When gradients grow too large, they can destabilize training.
Without proper gradient computation, the model won’t know how to adjust its parameters, making training
ineffective.
Techniques like gradient clipping (limiting large gradients) are often used to prevent instability.

In simpler terms, computing gradients in an RNN involves breaking the loop into steps, figuring out how each
step affects the error, and using this to improve the model's performance. This process, though powerful, can
be tricky for long sequences due to vanishing or exploding gradients.

#10.2.3 Recurrent Networks as Directed Graphical Models

Recurrent Neural Networks (RNNs) can be interpreted as directed graphical models that represent sequential
data. Here's a simplified explanation:
4. Hidden States for Efficiency
• RNNs introduce hidden states (h(t)) to summarize past information efficiently.
o Hidden states act as intermediate variables linking past and future.
o Instead of storing all previous values explicitly, h(t) condenses them, reducing computation.

5. Advantages of Using RNNs as Graphical Models

• Parameter Sharing: The same parameters (θ) are reused across all time steps, making the model
efficient.
• Handles Long-Term Dependencies: Hidden states help capture relationships between outputs, even if
they are far apart in the sequence.
• Compact Representation: Unlike a traditional graphical model that grows in complexity with sequence
length, an RNN remains efficient.

6. Challenges
• Optimization Complexity:
o While efficient in parameter usage, optimizing RNN parameters can be difficult due to vanishing or
exploding gradients.
• Sequence Length:
o RNNs assume that relationships between time steps are stationary (unchanging over time), which may
not always hold.
RNNs act as efficient graphical models by using hidden states to represent dependencies across time. They
simplify parameter usage and computation while effectively modeling sequences.

#MQP. Discuss about Bidirectional RNNs.

SIMP:Explain the concept of Bidirectional RNNs. How do they differ from standard
RNNs, and what are their advantages?
Differences from Standard RNNs:
1. Processing Direction:
o Standard RNNs process the sequence in a single direction (forward).
o BRNNs process it in both directions, leveraging context from both past and future inputs.
2. Contextual Awareness:
o Standard RNNs only use past information to make predictions.
o BRNNs use both past and future information, providing a richer understanding of the data.
3. Architecture:
o BRNNs require two RNNs (one for each direction), effectively doubling the computational
requirements compared to standard RNNs.

Advantages of Bidirectional RNNs:

1. Better Context Understanding:
o They consider the full sequence context, which is particularly useful when the meaning of a time step
depends on both earlier and later inputs.
o Example: In speech recognition, understanding a sound might depend on sounds both before and after it.
2. Improved Accuracy:
o By combining past and future information, BRNNs often achieve better performance on tasks involving
sequential data.
3. Versatility:
o Useful for tasks where full sequence information is available, such as:
§ Speech recognition.
§ Language translation.
§ Text classification.

#Q. Describe the architecture of Deep Recurrent Networks. How do they improve upon
standard RNNs?

Deep RNNs introduce additional layers into the traditional RNN architecture, specifically within the three
core operations:
1. Input to Hidden State
2. Previous Hidden State to Next Hidden State
3. Hidden State to Output

These extra layers enhance the representational capacity of the network by enabling more complex
transformations at each step.
Working of Deep RNNs:
1. Input to Hidden State:
The raw input is passed through a multi-layer perceptron (MLP), which maps it into a higher-level feature
representation before feeding it to the hidden state.
2. Hidden State Transitions:
Instead of a simple linear transformation, the transition from the previous hidden state to the next
one can involve multiple hidden layers, enabling richer transformations and better learning of
complex temporal dependencies.
3. Hidden State to Output:
The hidden state is processed by another MLP to produce the final output. This MLP can be deep,
allowing for more sophisticated outputs.

Advantages of Deep RNNs:

● Increased Capacity: They capture complex temporal patterns and hierarchical
representations, improving tasks like speech recognition and machine translation.
● Improved Representations: Layers transform raw inputs into abstract representations,
enhancing learning at higher levels.

How Do DRNs Improve Upon Standard RNNs?

1. Hierarchical Feature Learning:
o Each layer captures progressively higher-level features.
o Lower layers may capture short-term dependencies, while higher layers capture long-term
relationships.
2. Better Representation:
o DRNs can model complex patterns in the data that single-layer RNNs struggle with.
o This is particularly useful for tasks requiring understanding of both local and global sequence
structures.
3. Improved Performance:
o By stacking layers, DRNs achieve better accuracy and generalization across tasks like speech
recognition, text translation, and time-series prediction.
4. Flexibility:
o DRNs can incorporate advanced RNN cells like LSTMs or GRUs to handle vanishing gradients better
and maintain stability in deep architectures.
##Q. Explain the concept of Recursive Neural Networks.
Recursive NN

Recursive Neural Networks (Recursive NNs) are a generalization of Recurrent Neural Networks (RNNs),
but instead of processing sequential data in a chain-like structure, they operate on hierarchical data using
a tree-like computational graph.

How Recursive NNs Work:

● Input Structure:
Recursive NNs work on data structured as a tree. Examples include:
○ Parse trees of sentences in natural language processing (NLP).
○ Hierarchical structures in images or graphs.
● Computation:
○ Each node in the tree represents a combination of inputs (e.g., words or features) and
applies a neural transformation.
○ The model computes representations for subtrees and propagates them upward in the tree.
○ The final root node represents the entire input.
● Parameters:
○ Recursive NNs use shared weights across all nodes, reducing the number of parameters
while maintaining consistency

Advantages:
1. Depth Reduction: Efficiently reduces network depth to O(logτ) for sequences of length τ.
2. Flexibility: Handles hierarchical data like trees and graphs.
3. Custom Trees: Works with predefined (e.g., parse trees) or learned structures.

##10.10 The Long Short-Term Memory and Other Gated RNNs

What Are Gated RNNs?
• Gated RNNs, including Long Short-Term Memory (LSTM) and networks based on Gated Recurrent
Units (GRU), are advanced sequence models.
• They are designed to prevent gradients from vanishing or exploding by creating stable paths through
time.
How Do Gated RNNs Work?
• Like leaky units, they accumulate information over time, but instead of using fixed or parameterized
connection weights, they use gates that dynamically control the information flow.
• These gates allow the network to:
o Accumulate useful information over long durations.
o Forget old, irrelevant information when needed.

How LSTM Works:

• LSTM uses memory cells connected recurrently, replacing ordinary hidden units.
• Components of an LSTM memory cell:
1. Input Gate:
§ Decides if new information should be added to the state.
2. Forget Gate:
§ Controls whether the state should be reset or retained.
3. Output Gate:
§ Controls what information from the state should be sent as output.
• Key Features:
o The state unit has a linear self-loop with weight controlled by the forget gate.
o Gating units use a sigmoid nonlinearity.
o The state unit can also serve as input to the gates.

Why Use Gated RNNs?

• They allow the model to:
o Learn when to forget old information and when to accumulate new information.
o Handle sequences composed of sub-sequences by learning to clear and reuse the state dynamically.

Figure - Block diagram of LSTM

###MQP.Explain the working principles of LSTM with necessary formulas.(8M

LongShort-TermMemory(LSTM):

LSTM is a type of Recurrent Neural Network (RNN) that is designed to handle long-term
dependencies in sequence data by overcoming the vanishing gradient problem during
backpropagation. It achieves this through a system of gates that regulate the flow of information into,
out of, and within its memory cell.

LSTM (Long Short-Term Memory) is a special type of RNN (Recurrent Neural Network) designed to
handle long-term patterns in data, such as time-series data or text sequences. It avoids the problem of
"forgetting" older information using memory cells and gates.

Core Components of LSTM:

###Discuss the applications of deep learning in Computer Vision
Computer vision is one of the most prominent fields where deep learning excels. Its applications range from
mimicking human vision capabilities, like recognizing faces, to entirely new tasks like recognizing sound
waves through object vibrations in videos. Here are some notable applications:

1. Object Recognition and Detection

• Purpose: Identify and classify objects within an image or video.
• Tasks:
1. Reporting which objects are present in an image.
2. Annotating images with bounding boxes around objects.
3. Labeling each pixel with the identity of the object it belongs to (semantic segmentation).
• Applications:
o Face detection.
o Traffic sign recognition.
o Autonomous vehicles.

2. Optical Character Recognition (OCR)

• Purpose: Transcribe text from images.
• Examples:
o Reading license plates.
o Digitizing handwritten documents.
o Extracting text from scanned books.

3. Image Synthesis and Restoration

• Image Synthesis:
o Generative models create realistic images, often from scratch.
o Useful in entertainment, gaming, and virtual reality.
• Image Restoration:
o Fixing defects in images or removing unwanted objects.
o Examples:
§ Denoising images.
§ Removing scratches or blemishes.

4. Dataset Augmentation
• Purpose: Improve model generalization by increasing the effective size of the training set through
transformations.
• Techniques:
o Random translations, rotations, flips.
o Perturbing colors or applying nonlinear distortions.
• Outcome: Enhanced robustness of models to variations in real-world scenarios.
5. Advanced Vision Tasks
• Recognizing sound waves through vibrations visible in videos.
• Enhancing edge and corner detection in complex images using contrast normalization.

Preprocessing for Computer Vision

• Global Contrast Normalization (GCN):
o Adjusts image contrast by subtracting the mean and rescaling pixels.
o Helps models handle images with varying contrast levels.
• Local Contrast Normalization:
o Normalizes contrast within small regions, making edges and details stand out.
o Often used as both a preprocessing step and a nonlinearity in network layers.

Deep learning revolutionizes computer vision by enabling complex tasks like object detection, text
recognition, and image synthesis. Through techniques like dataset augmentation and contrast normalization,
these models become more accurate and robust, making them pivotal in fields ranging from autonomous
vehicles to content creation.

##MQP.Write a note on Speech Recognition and NLP.

q. Define Natural Language Processing. Describe different steps involved in NLP.(8M)

Definition of NLP:
Natural Language Processing (NLP) is a subfield of Artificial Intelligence (AI) that focuses on
enabling computers to understand, interpret, and respond to human language in a way that is both
meaningful and useful. NLP combines computational linguistics with machine learning and deep
learning to analyze and process large amounts of natural language data.

Applications of NLP:
● Machine Translation (e.g., Google Translate).
● Virtual Assistants (e.g., Siri, Alexa).
● Sentiment Analysis (e.g., analyzing customer feedback).
● Text Summarization.
● Spam Email Detection.

Steps Involved in NLP:

1. Text Preprocessing:
Prepares raw text data for further analysis.

○ Tokenization: Splitting text into smaller units (words, sentences).

○ Stop Words Removal: Eliminating common words like "and," "is," "the" that don't
contribute much meaning.
○ Lowercasing: Converting all text to lowercase for consistency.
○ Stemming: Reducing words to their root form (e.g., "running" → "run").
○ Lemmatization: Mapping words to their dictionary base form (e.g., "better" → "good").

2. Syntactic Analysis:
Focuses on the grammatical structure of sentences.

○ Part-of-Speech (POS) Tagging: Assigning grammatical roles (noun, verb, etc.) to words.
○ Parsing: Analyzing sentence structure to establish relationships between words.
 3. Semantic Analysis:
Focuses on the meaning of words and sentences.

○ Named Entity Recognition (NER): Identifies entities like names, dates, locations.
○ Word Sense Disambiguation: Resolves ambiguity by determining the context-specific
meaning of words.

4. Feature Extraction:
Converts text data into numerical representations.

○ Techniques include Bag-of-Words, TF-IDF, and Word Embeddings (e.g., Word2Vec, GloVe).

5. Model Building:
Uses machine learning or deep learning models for tasks like text classification, sentiment
analysis, or language translation.

6. Evaluation:
Measures model performance using metrics like accuracy, F1-score, and BLEU (for
translation tasks).

Significance: NLP bridges the gap between human communication and machine understanding,
driving innovations in AI-powered communication systems.

#q.What is speech recognition? Explain the different types of speech recognition systems.

Definition of Speech Recognition:

Speech recognition is the process of converting spoken language into text using algorithms, signal
processing, and machine learning. It forms the foundation of many modern technologies, such as
virtual assistants, transcription services, and voice-controlled devices.

How Speech Recognition Works:

1. Audio Signal Input: A microphone captures audio.
2. Feature Extraction: Processes the audio to extract meaningful features (e.g., Mel-Frequency
Cepstral Coefficients (MFCCs)).
3. Acoustic Modeling: Maps audio features to phonemes (basic units of sound).
4. Language Modeling: Predicts word sequences based on grammar and context.
5. Decoding: Combines the acoustic and language models to generate text output.

Types of Speech Recognition Systems:

1. Speaker-Dependent Systems:

○ Customized for a specific user's voice.

○ Requires training to adapt to the speaker's pronunciation and accent.
○ Advantages: High accuracy for the trained speaker.
○ Disadvantages: Limited usability for others.
○ Applications: Personal virtual assistants.

2. Speaker-Independent Systems:
 ○ General-purpose systems that work for any user without prior training.
○ Advantages: Broad applicability.
○ Disadvantages: Lower accuracy compared to speaker-dependent systems.
○ Applications: Public voice-controlled systems, call center automation.

3. Continuous Speech Recognition:

○ Recognizes natural, uninterrupted speech.

○ Handles variations in speaking speed and style.
○ Applications: Dictation software, transcription tools.

4. Discrete Speech Recognition:

○ Requires users to speak with pauses between words.

○ Applications: Command-based systems like Interactive Voice Response (IVR).

5. Isolated Word Recognition:

○ Recognizes individual words spoken in isolation.
○ Applications: Simple systems like answering machines.

6. Connected Speech Recognition:

○ Processes sequences of connected words with slight pauses.

○ Applications: Command-and-control systems, simple assistants.

How Deep Learning Enhances Speech Recognition:

• End-to-End Models:
o Deep learning enables models to learn directly from raw audio inputs, reducing reliance on
feature engineering.
• Improved Accuracy:
o Advanced architectures like RNNs, LSTMs, GRUs, and Transformers achieve higher
recognition rates by capturing context and long-term dependencies.
• Adaptability:
o Models can adapt to diverse accents, languages, and noisy environments using robust training
datasets.

Speech recognition systems have revolutionized human-computer interaction, making technology

more accessible and intuitive in various domains such as healthcare, automotive, and personal
computing.

###Discuss applications of RNNs in Natural Language Processing, such as machine

translation and text generation.

Recurrent Neural Networks (RNNs) are highly effective in Natural Language Processing (NLP) due to their
ability to process sequential data. Below are two key applications:

1. Machine Translation
• Objective:
o Convert a sentence from one language into another while maintaining the same meaning.
• How It Works:
 o Machine translation systems involve multiple components:
1. Candidate Generation:
§ Proposes several possible translations for a sentence.
2. Language Model:
§ Evaluates these translations to ensure they are grammatical and contextually
appropriate.
o Example: Transforming "apple red" into "red apple" for better grammatical accuracy in English.
• Neural Approaches:
o Early machine translation used n-gram models, which relied on word frequencies and co-
occurrence statistics.
o Neural Machine Translation (NMT) enhances this by incorporating RNNs, which:
§ Handle variable-length inputs and outputs.
§ Summarize the input sentence into a context vector that represents its meaning.
§ Use an encoder-decoder framework for translating sentences.
• Advancements:
o Attention mechanisms allow models to focus on specific parts of a sentence during translation,
improving accuracy and handling longer sequences.

2. Text Generation
• Objective:
o Generate coherent and contextually relevant text, word by word or character by character.
• How It Works:
o The RNN predicts the probability of the next word in a sequence given the previous words,
using:
§ A softmax layer to output probabilities for possible next words.
§ Sequentially generating text by sampling from these probabilities.
• Flexible Output Length:
o RNNs allow for dynamic sequence lengths in generated text, making them ideal for
applications such as:
§ Dialogue generation.
§ Writing assistance.
§ Creative tasks like poetry or story writing.
• Improvements with Modern Techniques:
o Advanced strategies like adding attention or incorporating transformers can further enhance
the quality of generated text, particularly for long and complex sequences.

RNNs play a vital role in NLP tasks like machine translation and text generation. They leverage their
sequential processing ability to maintain context and generate outputs tailored to the input data, making them
indispensable for modern language-based applications.