Machine Translation

Tushar B. Kute,
http://tusharkute.com
Machine Translation

• Machine translation (MT) is the field of computer

science that uses artificial intelligence (AI) to
automatically translate text from one natural
language (source language) to another (target
language).
– Goal: To bridge the communication gap between
people speaking different languages by
automatically converting text.
Machine Translation

(Target: Marathi)
Language
Machine Translation: Techniques

• Rule-based MT: Uses grammatical rules and dictionaries to

translate text. This approach is less common nowadays due to
limitations in capturing language nuances.
• Statistical MT (SMT) and Neural MT (NMT): These are the
dominant techniques today. They rely on statistical models or
neural networks trained on large amounts of bilingual text
data.
– SMT: Learns statistical patterns between words and phrases
in source and target languages.
– NMT: Uses deep learning architectures like recurrent neural
networks (RNNs) to capture complex relationships between
languages and produce more natural-sounding translations.
Machine Translation: Need

• Improved Communication: MT allows people who speak

different languages to understand each other more easily. This
can be beneficial for:
– Travel and Tourism: Tourists can access information and
interact with locals more effectively.
– International Business: Businesses can communicate with
partners, customers, and employees worldwide.
– Global News and Information: People can access news and
information from around the world in their native language.
– Education and Research: MT can help overcome language
barriers in research collaboration and educational resources.
Machine Translation: Need

• Increased Accessibility: MT makes information and

content more accessible to a wider audience. This can be
particularly useful for:
– E-commerce: Businesses can translate their online
stores and product descriptions to reach a global
market.
– Software and Technology: Software interfaces and
documentation can be translated to accommodate
users who don't speak the original language.
– Entertainment: Movies, TV shows, and music can be
enjoyed by a wider audience with the help of subtitles
or dubbing generated through MT.
Machine Translation: Need

• Enhanced Efficiency and Productivity: MT helps save

time and resources by automating translation tasks.
This can be beneficial for:
– Customer Support: Companies can provide
multilingual customer support at a lower cost.
– Real-time Communication: MT can be used for live
chat or video conferencing, enabling real-time
conversations across languages.
– Content Localization: Websites, documents, and
marketing materials can be translated quickly and
efficiently for international audiences.
Machine Translation: Need

• Breaking Down Cultural Barriers:

– MT can foster cultural exchange and
understanding by enabling people to access
information and communicate with each other
more easily.
– This can lead to greater collaboration and
cooperation across cultures.
Machine Translation: Problems

• Accuracy and Fluency:

• Limited Context: Machine translation systems often struggle to
capture the nuances of language, including context, sarcasm, and
humor. This can lead to misunderstandings and awkward phrasing in
the translated text.
• Word Choice and Idioms: MT systems might struggle with translating
words that have multiple meanings or idiomatic expressions that
don't have direct equivalents in the target language. The translated
text might be grammatically correct but lack natural flow or convey
the intended meaning inaccurately.
• Cultural Nuances: Cultural references and social norms can be
challenging to translate accurately. A literal translation might not
convey the intended meaning or even be offensive in the target
culture.
Machine Translation: Problems

• Domain Specificity:
• Technical Language:
– Machine translation systems trained on general
data might struggle with translating technical
documents, legal jargon, or other domain-
specific languages.
– The lack of specialized vocabulary and
knowledge can lead to inaccurate or misleading
translations.
Machine Translation: Problems

• Grammar and Syntax:

• Sentence Structure:
– Different languages have different grammatical
structures and sentence orders.
– MT systems might not always correctly translate
complex sentence structures or rearrange them
appropriately for the target language.
Machine Translation: Problems

• Ambiguity:
– Homographs: Words with multiple spellings or
pronunciations can be misinterpreted by MT systems,
leading to incorrect translations.
– Part-of-Speech Disambiguation: MT systems might
struggle to determine the correct part of speech for
a word, especially in cases of homonyms (words with
the same spelling but different meanings). This can
lead to grammatical errors in the translation.
Machine Translation: Problems

• Data Bias:
• Training Data Bias:
– The quality and bias present in the training data
used for MT systems can be reflected in the
translations.
– For example, a system trained on data biased
towards a particular gender or culture might
produce translations that perpetuate those
biases.
Machine Translation: Problems

• Evolving Language:
– Language constantly evolves with new words,
slang, and expressions.
– MT systems need to be continuously updated
with new data to keep pace with these changes
and maintain translation accuracy.
Machine Translation: Approaches

• 1. Rule-based Machine Translation (RBMT)

• 2. Statistical Machine Translation (SMT)
• 3. Direct Machine Translation
• 4. Knowledge Based Translation
Machine Translation: Approaches
Rule-based Machine Translation

• This traditional approach relies on explicitly defined

linguistic rules and bilingual dictionaries.
• It involves:
– Linguistic Analysis: The source language text is analyzed
grammatically to understand its structure and meaning.
– Transfer: The extracted meaning is then transferred to a
set of rules that map it to the target language's syntax
and semantics.
– Generation: The target language sentence is generated
based on the transferred meaning and target language
grammar rules.
Rule-based Machine Translation
RBMT : Sub-approaches

• Direct Translation: Words from the source language are

directly translated to their equivalents in the target
language, with minimal syntactic analysis.
• Transfer-Based Translation: The syntactic structure of the
source language is analyzed and then mapped to the
corresponding structure in the target language.
• Interlingua-Based Translation: Source language text is
converted into an intermediate representation
(interlingua) that captures the meaning independent of
any specific language. The translated sentence is then
generated from the interlingua representation in the
target language.
RBMT : Advantages

• Can be highly accurate for specific domains

where the rules are well-defined.
• Offers greater control over the translation
process.
RBMT : Disadvantages

• Requires significant manual effort to create and

maintain the linguistic rules and dictionaries.
• May not handle unseen words or complex
sentence structures well.
• Limited scalability to new languages or
domains.
Statistical Machine Translation

• This data-driven approach leverages large amounts of parallel text

data (sentences in both source and target languages) to train
statistical models for translation.
• SMT models typically involve:
– Sentence Alignment: Matching sentences in the source and target
language corpus that correspond to the same meaning.
– Word Alignment: Identifying corresponding words between
aligned sentences.
– Model Training: Training statistical models (e.g., phrase-based,
neural network-based) on the aligned text data to learn the
probability of translating a word or phrase from the source
language to the target language.
– Decoding: Using the trained model to generate the most likely
translation for a new source language sentence.
Statistical Machine Translation
SMT: Process

• Data Preparation:
– Parallel Text Corpus: A large collection of text data
containing sentences in both the source and target
languages that correspond to the same meaning is
essential.
– Sentence Alignment: Matching sentences in the source and
target language corpus that convey the same meaning.
Tools and techniques are used to identify corresponding
sentence pairs.
– Word Alignment: Identifying word-to-word
correspondences within aligned sentences. This helps the
model understand how words in one language translate to
another.
SMT: Process

• Model Training:
– Statistical models are trained on the aligned and
word-aligned text data. Common models include:
• Phrase-based SMT:
– This model learns the probability of translating
entire phrases from the source language to the
target language. It breaks sentences down into
smaller phrases and translates them individually,
considering their statistical co-occurrence patterns in
the training data.
SMT: Process

• Neural Network-based SMT:

– This approach utilizes neural network
architectures (e.g., recurrent neural networks) to
learn complex relationships between source and
target language sentences.
– These models can capture longer-range
dependencies and context compared to phrase-
based models.
SMT : Advantages

• Less reliant on manual rule creation compared

to RBMT.
• Can handle unseen words and complex
structures to some extent based on statistical
patterns.
• More scalable to new languages with sufficient
training data.
SMT : Disadvantages

• May not always capture the nuances of

language or context.
• Relies heavily on the quality and size of the
training data.
• Can generate grammatically correct but
semantically inaccurate translations.
Neural Machine Translation

• This is a recent advancement that utilizes deep

learning architectures, particularly recurrent
neural networks (RNNs) or transformers, to
translate languages.
• NMT models learn complex relationships
between source and target languages by
directly processing entire sentences at once.
Neural Machine Translation
Neural Machine Translation
Neural Machine Translation
Neural Machine Translation

• NMT models typically involve:

• Encoder-Decoder Architecture:
– An encoder network processes the source
language sentence, and a decoder network
generates the target language translation.
– Attention Mechanism: The decoder can focus on
specific parts of the source sentence while
generating the translation, improving accuracy.
NMT: Advantages

• Often achieves state-of-the-art performance on

many language pairs.
• Can capture long-range dependencies and
context within sentences.
• Can be effective even with limited training data
compared to traditional SMT.
NMT: Disadvantages

• Requires significant computational resources

for training and inference.
• Can be prone to factual errors or hallucinations
if not trained properly.
• Limited interpretability compared to rule-based
approaches.
How to choose right approach?

• Available resources:
– RBMT might be suitable for specific domains with
limited resources, while NMT often requires
significant computational power.
• Language pair:
– The complexity of the language pair can influence
the effectiveness of each approach.
• Desired accuracy and fluency:
– While NMT often achieves higher accuracy, RBMT can
offer more control over grammatical correctness.
Direct Machine Translations

• Direct Machine Translation (DMT) is a sub-approach

within Rule-Based Machine Translation (RBMT) that
focuses on a simpler and more efficient way of
translating text between languages.
Direct Machine Translations
Direct Machine Translations

• DMT aims for a word-by-word translation approach,

relying heavily on bilingual dictionaries to find the
closest equivalents in the target language.
• It performs some basic grammatical adjustments
based on pre-defined rules to ensure the translated
sentence has a proper structure.
Direct Machine Translations

• Sentence Segmentation: The source language sentence is

broken down into individual words or short phrases.
• Dictionary Lookup: Each word or phrase is looked up in a
bilingual dictionary containing source language words
mapped to their corresponding target language translations.
• Word Order Adjustment: Basic rules might be applied to
adjust the word order if necessary to conform to the target
language's grammar (e.g., verb conjugation, noun phrase
structure).
• Output Generation: The translated sentence is formed by
combining the translated words or phrases with any
necessary grammatical adjustments.
DMT: Advantages

• Simplicity:
– Easy to implement and requires less computational
resources compared to other MT approaches.
• Efficiency:
– Can be quite fast for translating large volumes of
text.
• Control:
– Offers some control over the translation process
through pre-defined rules.
DMT: Use Cases

• DMT might be suitable for simple tasks where quick

and basic translation is sufficient, such as
translating technical terms or short phrases in
specific domains.
• It can be a starting point for building more
sophisticated RBMT systems that combine DMT
with additional rule-based components for handling
grammar and syntax.
Knowledge Based MT System

• In Knowledge-Based Machine Translation (KBMT),

the system leverages external knowledge
resources to improve the accuracy and fluency of
translations compared to purely statistical or rule-
based approaches.
• KBMT goes beyond simple word-to-word or phrase-
to-phrase translation.
• It aims to understand the deeper meaning and
context of the source language sentence using
external knowledge sources.
Knowledge Based MT System
Knowledge Based MT System

• Knowledge can include:

– World knowledge: General facts and information
about the world.
– Domain-specific knowledge: Specialized knowledge
relevant to a particular domain (e.g., medicine,
finance).
– Ontologies: Formal representations of concepts and
relationships within a domain.
– Semantic networks: Graphs that connect concepts
and their relationships.
Knowledge Based MT System

• Source Language Analysis: Similar to other MT approaches, the

system analyzes the source language sentence to understand
its grammatical structure and identify key elements.
• Meaning Representation: Using the extracted information and
external knowledge resources, the system attempts to
represent the meaning of the sentence in a way that is
independent of any specific language. This representation
might involve concepts, relationships, and logical propositions.
• Target Language Generation: The system leverages its
knowledge of the target language and the meaning
representation to generate a fluent and grammatically correct
sentence that conveys the same meaning as the source
language sentence.
IBM Models

• A series of five (and sometimes six) statistical models

used in SMT for word alignment and translation
probability estimation.
– Each model builds upon the previous one, increasing
complexity and handling more intricate aspects of
word alignments.
– Examples: Model 1 (Lexical Translation), Model 2
(Alignment Model), etc.
EM Algorithm

• An iterative optimization technique used for

finding maximum likelihood estimates of
parameters in statistical models with latent
variables (unobserved data).
Application in SMT (IBM models)

• Model Definition: Each IBM model defines a set of

parameters.
– For example, Model 1 estimates the lexical
translation probability t(e|f), which represents the
probability of a target word e translating to a source
word f.
• Latent Variables:
– In word alignment, the correspondence between
words in the source and target sentences is often not
explicitly provided. These word alignments act as
latent variables.
Application in SMT (IBM models)

• E-Step (Expectation):
– In this step, the algorithm estimates the expected value of the
missing data (word alignments) for each sentence pair in the
training corpus.
– This involves calculating the probability of each possible word
alignment given the source and target sentences, and the current
model parameters.
• M-Step (Maximization):
– Based on the expected word alignments from the E-step, the
algorithm updates the model parameters to maximize the
expected log-likelihood of the training data.
– This involves using the expected alignment counts to re-estimate
the model parameters (e.g., lexical translation probabilities) for
the next iteration.
Benefits using EM

• Iterative Refinement:
– The EM algorithm allows for iterative improvement of the
model parameters by leveraging the estimated word
alignments in each step.
• Handling Missing Data:
– By estimating the missing word alignments, EM enables
learning from incomplete data, which is common in word
alignment tasks.
• Guaranteed Convergence:
– Under certain conditions, the EM algorithm is guaranteed
to converge to a local maximum of the log-likelihood
function.
Encoder-decoder architecture

• The encoder-decoder architecture is a

fundamental building block for many natural
language processing (NLP) tasks, particularly
those involving sequence-to-sequence learning.
Encoder-decoder architecture

• Encoder:
– Processes the input sequence (e.g., a sentence in a
source language).
– Typically a recurrent neural network (RNN) like Long
Short-Term Memory (LSTM) or Gated Recurrent Unit
(GRU) that can handle long-term dependencies
within the sequence.
– The encoder aims to capture the meaning and
context of the input sequence and create a
compressed representation (often a vector) that
summarizes the essential information.
Encoder-decoder architecture

• Decoder:
– Generates the output sequence (e.g., a sentence in a
target language).
– Also typically an RNN, but it might have a different
structure or initialization compared to the encoder.
– The decoder takes the encoded representation from
the encoder as input, along with a starting token (e.g.,
"START" symbol).
– At each step, the decoder predicts the next element
(word) in the output sequence based on the encoded
representation and the previously generated outputs.
ED Architecture Interaction

• Encoding:
– The input sequence is fed into the encoder one element
(word) at a time. The encoder's RNN processes each element
and updates its internal state, capturing the context of the
sequence so far.
• Context Vector:
– After processing the entire input sequence, the encoder
outputs a context vector. This vector is a condensed
representation that encapsulates the meaning of the input
sequence.
• Decoding:
– The decoder receives the context vector as input. It also uses a
special start token to initiate the generation process.
ED Architecture Interaction

• Output Generation: At each step, the decoder performs the

following:
– Uses its internal state, the previously generated outputs,
and the context vector to predict the next element (word)
in the output sequence.
– Updates its internal state based on the prediction and the
context vector.
– The predicted element is added to the output sequence.
• End of Sequence: The decoding process continues until an
end-of-sequence token (e.g., "END" symbol) is predicted, or a
maximum length for the output sequence is reached.
Summary

• Machine translation bridges the language gap! It

automatically converts text from one language (source)
to another (target).
• Statistical models or neural networks power this magic,
analyzing massive amounts of translated text to learn
the best way to convert words and sentences.
• The result: computers can understand and translate
languages more effectively, fostering communication
across borders.
 Thank you
This presentation is created using LibreOffice Impress 7.4.1.2, can be used freely as per GNU General Public License

@mITuSkillologies @mitu_group @mitu-skillologies @MITUSkillologies

Web Resources
https://mitu.co.in
@mituskillologies http://tusharkute.com @mituskillologies

[email protected]
[email protected]