Advanced Techniques in Training and Applying

Large Language Models

1st LIPSA DAS 2nd NEELANJAN MUKHERJI 3rd POOJA ANAND
Amity University Amity University Amity University
Greater Noida, India Greater Noida, India Greater Noida, India
[email protected] [email protected] [email protected]

4th BHANU PRAKASH LOHANI 5th DURGESH KUMAR JHA

Amity University Amity University
Greater Noida, India Greater Noida, India
[email protected] [email protected]

Abstract—This study, entitled ”Advanced Techniques in Train- • Speech Recognition converts spoken words into text.
ing and Applying Large Language Models,” investigates the • Natural Language Understanding is where computers
advancements and applications of large language models (LLMs). comprehend human language.
The paper examines the process of developing LLMs, with a
primary focus on the many models now available. We will start • Natural Language Generation is when computers pro-
by covering basic principles, such as the mathematical models duce human language.
utilized in the process. LLMs provide a comprehensive foundation Key NLP Techniques:
for the development of future applications in artificial intelligence
and natural language processing. • Syntactic Analysis: Examines sentence structure using
Index Terms—Large Language Models (LLMs), Natural Lan- formal grammar, identifying parts like noun and verb
guage Processing(NLP), Artificial Intelligence(AI), Informational phrases.
Retrival, Data Preprocessing, Model Training. • Semantic Analysis: Understands meaning and context,
through computers struggle with deep comprehension.
I. I NTRODUCTION • Parsing: Breaks down sentences into grammatical com-
The incorporation of artificial intelligence (AI) and natural ponents for further analysis.
language processing (NLP) into applications throughout the • Parsing: Breaks down sentences into grammatical com-
digital transformation period has significantly transformed user ponents for further analysis.
interactions, facilitating more intuitive and effective informa- • Stemming: Reduces words to their root form for efficient
tion retrieval processes. Large language models (LLMs) have processing.
become a fundamental aspect of this development, with the • Text Segmentation: Divides text into meaningful units
ability to comprehend and produce text that resembles human like words or phrases.
language. As a result, they improve the speed and appropri- • Named Entity Recognition (NER): Identifies and cate-
ateness of digital systems. This study explores the structure gorizes entities like names, organizations, or locations.
and execution of LLMs, providing a detailed explanation of • Relationship Extraction: Discerns relationships between
the systematic approach for constructing these models from entities identified by NER.
the beginning. The study attempts to create a comprehensive • Sentiment Analysis: Determines sentiments in text, use-
framework for using LLMs to alter interactions and develop ful for analyzing reviews and opinions.
AI and NLP applications. It focuses on crucial issues such as
data collecting, preprocessing, and model training. [1] B. Introduction to Large Language Models(LLMs)

II. BACKGROUND A large language model refers to a sophisticated form of

language model that undergoes training utilizing deep learning
A. Natural Language Processing(NLP) Overview techniques on vast volumes of textual data. These models
Natural Language Processing (NLP) integrates computer possess the ability to produce text that resembles human
science, linguistics, and machine learning to facilitate com- language and carry out a range of tasks related to natural
puters in comprehending and producing human language. language processing.
Applications encompass voice assistants such as Alexa and On the other hand, a language model is defined as the pro-
Siri, machine translation, and text filtering. [2] cess of assigning probability to word sequences by analyzing
Recent advances in deep learning have significantly en- text corpora. A language model can range in complexity, from
hanced NLP, which comprises: basic n-gram models to advanced neural network models.
 Typically, the term ”large language model” is used to de-
scribe models that employ deep learning methods and possess
a substantial number of parameters, ranging from millions to
billions. These AI models have the ability to comprehend intri-
cate linguistic patterns and generate material that is frequently
indistinguishable from human-written content. [3]

III. T HE T RANSFORMER A RCHITECTURE

Fig. 1. Transformer Architecture

Fig. 2. Encoder
The Transformer architecture, introduced in 2017 revo-
lutionized the field of natural language processing (NLP).
The Transformer, in contrast to traditional sequential models
like RNNs (Recurrent Neural Networks), is not reliant on of dimensions. This vector depicts the word inside a
the sequence’s order and may analyze whole sequences in continuous vector space, capturing the semantic nuances.
parallel. This attribute renders it very efficient and efficacious, • Positional Encoding: In order to address the lack of
especially for tasks of immense magnitude. inherent word order comprehension in the Transformer
Transformers are essential elements in several modern large model, which is not present in RNNs, positional encoding
language models (LLMs) including GPT, BERT, and T5. is incorporated into the embeddings. Positional encodings
These models are built using a self-attention mechanism, are vector representations that encode the precise location
allowing the model to assess the importance of each word of each word inside a given sequence. These are incor-
in a sentence relative to the others. [4] porated into the word embeddings to provide the model
with knowledge about the arrangement of words. [6]
A. The Core Components of the Transformer • Multi-Head Attention: is the essential element of the
The Transformer architecture has two primary components: encoder. Multi-head attention enables the model to con-
currently concentrate on distinct segments of the input
• Encoder: Executes the supplied sequence.
stream. The system calculates attention ratings for each
• Decoder: Produces the resulting sequence.
word pair, allowing the model to capture the links,
1) The Encoder: is a vital element in the Transformer dependencies, and context between words. [7]
design, with its main role being to parse input sequences • Feed-Forward Network: is a neural network that follows
and produce significant representations that may be employed the attention mechanism. It consists of a fully connected
by the decoder for tasks like translation, summarization, and layer and a non-linear activation function, through which
others. The primary function of the encoder is to receive an the output is transmitted. This introduces further intricacy
input sequence, such as a phrase in a particular language, and and facilitates the conversion of the input representation.
transform it into a comprehensive and contextually meaningful [8]
representation. The decoder utilizes this representation to • Layer Normalization and Residual Connections: After
produce the output sequence, such as a phrase in a different each sub-layer (such as multi-head attention and feed-
language. [5] forward network), there is a further phase of layer normal-
a) Components of the Encoder: ization, which normalizes the output. In addition, residual
• Embedding Layer: The input tokens (words) undergo connections, often referred to as skip connections, are
a transformation in an embedding layer, where each included around each sub-layer to enhance the model’s
word is converted into a vector with a large number learning efficiency and mitigate the issue of disappearing
 gradients. [9] a) Components of the Decoder:
b) Stacking Encoder Layers: The encoder is comprised • Embedding Layer: functions analogously to the encoder.
of many identical layers that are vertically stacked. Each The process involves taking the input tokens, which are
succeeding layer enhances the representation generated by the the output tokens generated earlier, and converting each
prior layer. Generally, the majority of Transformer models token into a vector with a large number of dimensions.
utilize 6 layers and they are usually equal to the number of [12]
the decoders, however this number may be adjusted. [10] • Positional Encoding: Similar to the encoder, positional

The final outcome of the encoder is a sequence of vectors, encoding is utilized in the embeddings to convey informa-
where each vector corresponds to an input token. These vectors tion on the sequence of the tokens. This assists the model
not only capture the semantic meaning of the token, but also in understanding the organization of the series. [13]
• Masked Multi-Head Attention: is the term for the
their contextual importance throughout the entire sequence.
[11] decoder’s initial covered attention mechanism. Conse-
quently, while creating a forecast for a certain output
2) The Decoder: The decoder in the Transformer design
token, the model may just take into account the preceding
is tasked with producing the output sequence, such as a
tokens in the sequence and disregard the upcoming ones.
translated phrase, by considering both the previously created
It is imperative to prevent the model from ”cheating” by
output tokens and the output of the encoder. It has a crucial
seeing the tokens it is supposed to predict.
function in activities like as translation, summarization, and
• Encoder-Decoder Attention: denotes the secondary at-
text production. The decoder produces the output sequence by
tention mechanism employed in the decoder, specifically
generating one token at a time. For every output token, it takes
targeting the output generated by the encoder. This allows
into account both the preceding output tokens and the encoded
the decoder to choose focus on different regions of the
representation of the input sequence from the encoder.
input sequence when generating each token in the output
sequence. It helps the decoder to incorporate context from
the whole input sequence. [14]
• Feed-Forward Network: After the attention processes,
the output is sent via a feed-forward neural network,
which adds complexity and helps to further modify the
representation.
• Layer Normalization and Residual Connections: In a
manner akin to the encoder, a layer normalization process
is employed after each sub-layer, such as attention or
feed-forward network. Residual connections are included
around each sub-layer to improve learning efficiency and
simplify the flow of gradients. [15]
b) Stacking Decoder Layers: The decoder, like to the
encoder, consists of many identical layers that are stacked
vertically. Each succeeding layer enhances the representation
generated by the previous layer. Typically, most Transformer
models include 6 layers. [16]
c) Final Linear Layer and Softmax: Once the input has
gone through the decoder layers, it is then processed by a
linear layer and subsequently fed via a softmax function to
produce the final output. This layer maps the output to the
size of the vocabulary, creating a probability distribution that
represents the likelihood of each potential next token.
d) Final Output: The decoder generates the output se-
quence by producing one token at a time. Each token considers
the previously produced tokens, the whole input sequence
(using encoder-decoder attention), and the collected context
up to the current point.
IV. M ATHEMATICAL M ODELS I NVOLVED IN THE
A RCHITECTURE
Fig. 3. Decoder A. Math involved in The Encoder
1) Input Embeddings Let the input sequence be X =
[4] x1 , x2 , ..., xn where xi is a token(word). The first step is
 to convert these tokens into embeddings. The embedding network, which consists of two linear transforma-
layer maps each token xi to a continuous vector ei in tion with Rectified Linear Unit(ReLU) activation in
Rdmodel where dmodel is the dimension of the embed- between:
dings:
F F N (x) = ReLu(xW1 + b1 )W2 + b2
E = Embedding(X)
where W1 , W2 , b1 and b2 are learned parameters.
Here, E is the matrix of embeddings with shape
(n, dmodel ) [17]
2) Positional Encoding To incorporate positional informa-
tion, we add positional encodings to the embeddings.
The positional encoding for the position pos and dimen-
sion 2i is defined as:
pos
P E (pos,2i) = sin( )
100002i/dmodel
And for the odd dimensions:
pos
P E (pos,2i+1) = cos( )
100002i/dmodel
The positional encoding matrix P E is added to the
embedding matrix E:
E′ = E + P E
3) Multi-Head Attention allows the model to jointly attend
to information from different representation subspaces.
It involves several steps:

Fig. 5. Feed Forward

Fig. 4. Multi Head Attention • Layer Normalization and Residual Connections:

The Output of each sub-layer (e.g, attention or
• Linear Projections: For each attention head, the feed-forward) is added to the original input(residual
E ′ is linearly projected to three different matrices: connection) and then normalized:
queries Q, Keys K, Values V :
Output = LayerN orm(x + SubLayer(x))
Q = E ′ WQ , K = E ′ WK , V = E ′ WV
this helps in stabilizing the training and improving
where WQ , WK and WV are learned weight matri- the gradient flow.
ces. • Stacking Layers: The encoder consists of N iden-
• Scaled Dot Product Attention: The Attention tical layers, where each layer performs the above
scores are calculated by taking the steps. If x1 is the output of the l layer, then:
√ dot product of
the queries and keys, scaling by dk (where dk is xl+1 = EncoderLayer(xl )
the dimension of the keys), and applying a softmax
function: Finally, xN (the output of the last layer) is the final
QK T encoder output.
Attention(Q, K, V ) = sof tmax( √ )V
dk B. Math involved in The Decoder
• Concatenation of Heads: The outputs of the at- 1) Input Embeddings Let the input sequence be Y =
tention heads are concatenated and linearly trans- y1 , y2 , ..., ym is a token(word). The first step is to convert
formed: these tokens into embeddings. The embedding layer
maps each token yi to a continuous vector ei in Rdm odel :
M ultiHead(Q, K, V ) = Concat(head1 , ..., headh )W0
E = Embedding(Y )
where W0 is the output projection matrix.
• Feed-Forward Network: The output of the multi- Here, E is the matrix of embeddings with shape
head attention is passed through a feed-forward (m, dmodel )
2) Positional Encoding As in the encoder, positional encod- encoder’s keys, and used to weigh the encoder’s
ings are added to the embeddings to incorprate positional values:
information. The positional encoding for position pos QK T
and dimension 2i is defined as: Attention(Q, K, V ) = sof tmax( √ )V
dk
pos
P E (pos,2i) = sin( ) Concatenation of Heads: The outputs are concate-
•
100002i/dmodel
nated and linearly transformed as before. [22]
And for the odd dimensions: 5) Feed-Forward Network
pos • The output of the encoder-decoder attention is
P E (pos,2i+1) = cos( )
100002i/dmodel passed through a feed-forward network:
The positional encoding matrix P E is added to the F F N (x) = ReLu(xW1 + b1 )W2 + b2
embedding matrix E:
where W1 , W2 , b1 and b2 are learned parameters.
E′ = E + P E [22]
3) Masked Multi-Head Attention The first attention mech- 6) Stacking Decoder Layers
l
anism in the decoder is masked. This ensures that when • The decoder consists of N identical layers. If y is
th
predicting the i−th token, the model can only attend to the output of the l layer, then:
tokens y1 to yi−1 , not future tokens. [18] The process
y l+1 = DecoderLayer(y l , H)
involves several steps:
• Linear Projections: For each attention head, the where H is the encoder’s output. The decoder uses
E ′ is linearly projected to three different matrices: the encoded input sequence representation H in
queries Q, Keys K, Values V : each layer to refine its output representation. [23]
7) Final Linear Layer and Softmax
Q = E ′ WQ , K = E ′ WK , V = E ′ WV
• The final output of the stacked decoder layers is

where WQ , WK and WV are learned weight matri- passed through a linear transformation to project it
ces. [19] onto the vocabulary space:
• Masked Scaled Dot Product Attention: The atten- z = y N Wout + bout
tion scores are calculated by taking√the dot product
of the queries and keys, scaling by dk (where dk is where Wout and bout are learned parameters. The
the dimension of the keys), and applying a softmax softmax function is then applied to generate a prob-
function, but only over the allowed tokens(previous ability distribution over the vocabulary, which can
tokens). The mask M ensures that future tokens are be used to predict the next token in the sequence
not attended to: [24]
QK T P robabilities = sof max(z)
Attention(Q, K, V ) = sof tmax( √ + M )V
dk V. C ONCLUSION
• Concatenation of Heads: The outputs of the atten- • This paper discussed various methods and strategies to
tion heads are concatenated and linearly transformed enhance the performance and application of large lan-
[20]: guage models.
• We explore advanced techniques used during the training
M ultiHead(Q, K, V ) = Concat(head1 , ..., headh )W0 phase of LLMs, with the Transformer Architecture.
• The paper addresses the Transfomer Architecture in detail
where W0 is the output projection matrix. • The paper outlines future research direction and potential
4) Encoder-Decoder Attention The second attention advancements in the field of LLMs.
mechanism in the decoder attends to encoder’s output
H, which is the context representation of the input R EFERENCES
sequence. The process involves [21]: [1] S. Hochreiter and J. Schmidhuber., “Long short-term memory.,” Neural
′
• Linear Projections The decoder input E is again
computation, vol. 9, no. 8, p. 1735–1780, 1997.
[2] F. Chollet., “Xception: Deep learning with depthwise separable convo-
projected to Q, while the encoder’s output H is lutions.,” arXiv preprint, no. arXiv:1610.02357, 2016.
projected K and V : [3] K. He, X. Zhang, S. Ren, and J. Sun., “Deep residual learning for image
recognition.,” in Proceedings of the IEEE Conference on Computer
Q = E ′ WQ , K = HWK , V = HWV Vision and Pattern Recognition, p. 770–778, 2016.
[4] A. Vaswani and et al., “Attention is all you need.,” Advances in neural
information processing systems, vol. 30, 2017.
• Scaled Dot-Product Attention Attention scores are [5] Łukasz Kaiser and I. Sutskever., “Neural gpus learn algorithms.,” in
computed between the decoder’s queries and the International Conference on Learning Representations (ICLR), 2016.
 [6] J. Gehring, M. Auli, D. Grangier, D. Yarats, and Y. N. Dauphin.,
“Convolutional sequence to sequence learning.,” arXiv preprint,
no. arXiv:1705.03122v2, 2017.
[7] R. Jozefowicz, O. Vinyals, M. Schuster, N. Shazeer, and Y. Wu.,
“Exploring the limits of language modeling.,” arXiv preprint,
no. arXiv:1602.02410, 2016.
[8] N. Kalchbrenner, L. Espeholt, K. Simonyan, A. van den Oord, A. Graves,
and K. Kavukcuoglu., “Neural machine translation in linear time.,” arXiv
preprint, no. arXiv:1610.10099v2, 2017.
[9] S. Hochreiter, Y. Bengio, P. Frasconi, and J. Schmidhuber., “Gradient
flow in recurrent nets: the difficulty of learning long-term dependen-
cies.,” 2001.
[10] P. M. Nadkarni, L. Ohno-Machado, and W. W. Chapman, “Natural
language processing: an introduction.,” Journal of the American Medical
Informatics Association, vol. 18, no. 5, pp. 544–551, 2011.
[11] J. L. Ba, J. R. Kiros, and G. E. Hinton., “Layer normalization.,” arXiv
preprint, no. arXiv:1607.06450, 2016.
[12] D. Bahdanau, K. Cho, and Y. Bengio., “Neural machine translation by
jointly learning to align and translate.,” CoRR, no. abs/1409.0473, 2014.
[13] D. Britz, A. Goldie, M.-T. Luong, and Q. V. Le., “Massive exploration
of neural machine translation architectures.,” CoRR, no. abs/1703.03906,
2017.
[14] J. Cheng, L. Dong, and M. Lapata., “Long short-term memory-networks
for machine reading.,” arXiv preprint, no. arXiv:1601.06733, 2016.
[15] K. Cho, B. van Merrienboer, C. Gulcehre, F. Bougares, H. Schwenk, and
Y. Bengio., “Learning phrase representations using rnn encoder-decoder
for statistical machine translation.,” CoRR, no. abs/1406.1078, 2014.
[16] J. Chung, Çaglar Gülçehre, K. Cho, and Y. Bengio., “Empirical evalua-
tion of gated recurrent neural networks on sequence modeling.,” CoRR,
no. abs/1412.3555, 2014.
[17] A. Graves., “Generating sequences with recurrent neural networks.,”
arXiv preprint, no. arXiv:1308.0850, 2013.
[18] M.-T. Luong, H. Pham, and C. D. Manning., “Effective ap-
proaches to attentionbased neural machine translation.,” arXiv preprint,
no. arXiv:1508.04025, 2015.
[19] Z. Lin, M. Feng, C. N. dos Santos, M. Yu, B. Xiang, B. Zhou, and
Y. Bengio., “A structured self-attentive sentence embedding.,” arXiv
preprint, no. arXiv:1703.03130, 2017.
[20] O. Kuchaiev and B. Ginsburg., “Factorization tricks for lstm networks.,”
arXiv preprint, no. arXiv:1703.10722, 2017.
[21] S. Bengio and Łukasz Kaiser., “Can active memory replace attention?,”
in Advances in Neural Information Processing Systems, 2016.
[22] A. Parikh, O. Täckström, D. Das, and J. Uszkoreit., “A decomposable at-
tention model.,” in Empirical Methods in Natural Language Processing,
2016.
[23] R. Paulus, C. Xiong, and R. Socher., “A deep reinforced model for ab-
stractive summarization.,” arXiv preprint, no. arXiv:1705.04304, 2017.
[24] O. Press and L. Wolf., “Using the output embedding to improve language
models.,” arXiv preprint, no. arXiv:1701.06538, 2017.