Machine translation using SMT and Hybrid model

using natural language processing

Tanmaya Darisi
Lovely Professional University, Punjab
[email protected]

Abstract:
Machine Translation is the translation of text by a 2.3 Word Embeddings
computer with no human involvement. It is research with Word embeddings like Word2Vec and GloVe represent
different types methods being created, like rule-based, words in a continuous vector space, capturing semantic
statistical and example- based machine translation. similarity between words. These embeddings have proven
Neural networks have made a leap forward to machine valuable in numerous NLP tasks and can enhance
translation. This paper discusses the building of a deep traditional models by providing contextual word
neural network that functions as a part of end-to- end meanings.
translation pipeline. The completed pipeline would
accept English text as input and return the Hindi
Translation. The project has three main parts which are 3. Proposed Methodology
preprocessing, creation of models and Running the This paper proposes a three-step translation pipeline:
model on English Text.
1. Word Embedding Training: Word2Vec is
Keyword: Machine Translation, Neural Networks, Word used to train word embeddings on a large
Embeddings, Statistical Machine Translation, Neural corpus of English text. The embeddings
Machine Translation, Word2Vec, MarianMT. capture semantic similarities between words,
which can be leveraged to refine the output of
SMT.
Introduction 2. SMT Translation: An SMT model generates
Machine Translation (MT) has seen significant initial translations, treating the problem as a
advancements over the last few decades. Traditional phrase-based probabilistic task.
SMT systems and more recent NMT models each have 3. Embedding-Enhanced Refinement: Word
their strengths and weaknesses. This paper proposes a embeddings are used to enhance the SMT
hybrid system that combines SMT and word output by suggesting semantically related
embeddings, with the goal of improving the robustness words.
of MT systems. We show that integrating word 4. Final Translation with NMT: The enhanced
embeddings trained via neural networks into an SMT translation is passed to an NMT model
framework enhances translation quality, particularly for (MarianMT) to produce the final translation
phrase-based translations output.
Loss function’s equation in detailed:
1.1 Problem Statement
SMT systems often suffer from fluency issues and lack
the ability to effectively capture semantic context. On the
other hand, NMT systems, while strong in general, can
struggle with domain-specific language or low-resource
settings. This research explores how a hybrid SMT-NMT
system with word embeddings can mitigate these issues.
Where |S|= Sentence length, |V|= Vocabulary
2. Related Work length, y^w,e= Probability estimated with the entry 1 of
2.1 Statistical Machine Translation (SMT) vocabulary, yw, e= 1 for vocab entry is the correct, yw, e= 0
SMT has been a most used approach in machine for vocab entry is an incorrect word.
translation until the rise of neural methods. In SMT,
translation is treated as a probabilistic process, relying on 3.1 Word2Vec Training
large bilingual corpora to generate translation rules. We train a Word2Vec model on the Brown corpus using a
Models such as Moses are widely used for SMT, skip-gram approach, with a vector size of 100 and a
focusing on phrase-level translations. However, they context window of 5. This produces embeddings that can
often lack fluency and semantic accuracy. identify semantically related words.

2.2 Neural Machine Translation (NMT) 3.2 SMT Output Generation

NMT models, based on deep learning architectures like A standard phrase-based SMT system is used as a
RNNs, LSTMs, and more recently transformers, have baseline to produce initial translations. While SMT is
made breakthroughs in translation accuracy. However, effective for phrase translation, it often lacks fluency and
NMT systems are often data-hungry and can struggle in coherence when handling complex sentences.
domains where parallel corpora are limited. MarianMT is
one of the state-of-the-art NMT models that provides
highly accurate translations for various language pairs.

1
3.3 Embedding-Based Enhancement 5.1.2 METEOR Scores
The SMT output is refined by replacing words with their METEOR scores consider synonyms, stemming, and
semantically closest alternatives, as determined by the semantic similarity, making it an effective metric for
Word2Vec embeddings. This step improves the fluency assessing the impact of word embeddings.
and meaning of the translated sentence.

3.4 NMT Post-Processing

Finally, the enhanced SMT output is passed through
MarianMT, an NMT system, for further refinement. The
NMT system benefits from the higher quality input The METEOR scores also highlight the hybrid system’s
generated through the previous steps. effectiveness in capturing semantic similarity. The
hybrid system’s incorporation of word embeddings
3.5 Algorithms enables it to select synonyms and contextually
This is the symbolic representation of the process to appropriate words, leading to higher METEOR scores
translate the word into one language to another language than the baseline SMT system.
in the Natural Language Processing:
5.2 Qualitative Results
A qualitative assessment was conducted by analyzing the
output translations of sample sentences from both
models. The following examples demonstrate the
differences:

Example 1 (English to Hindi)

Source: "The government is implementing new policies
4. Experimental Setup to address the issue."
4.1 Dataset SMT Output: "सरकार समस्या को हल करने के ललए
For the training of word embeddings, we use the Brown नई नीलियाां लागू कर रही है।"
corpus, a collection of over one million words of Hybrid Output: "सरकार समस्या को सुलझाने के ललए
American English text. For translation tasks, we utilize
नई नीलियाां पेश कर रही है।"
the English-to-Hindi and English-to-Spanish parallel
Analysis: While both translations are generally accurate,
corpora from the OPUS dataset.
the hybrid model’s output ("सुलझाने" for "address")
more closely captures the intended meaning of "address
4.2 Evaluation Metrics
the issue" in this context.
To evaluate translation quality, we use BLEU (Bilingual
Evaluation Understudy) and METEOR (Metric for
Example 2 (English to Spanish)
Evaluation of Translation with Explicit Ordering) scores,
Source: "They introduced innovative measures to
comparing translations from the hybrid system against
improve the situation."
those from SMT and NMT models alone.
SMT Output: "Ellos introdujeron medidas innovadoras
para mejorar la situación."
5. Results and Comparative Analysis
This section evaluates the effectiveness of the hybrid
Hybrid Output: "Han introducido medidas innovadoras
translation system (SMT + Word Embeddings + NMT)
para mejorar la situación."
against a baseline SMT system. We use BLEU
Analysis: The hybrid model’s translation uses "Han
(Bilingual Evaluation Understudy) and METEOR
introducido" (present perfect) instead of "Ellos
(Metric for Evaluation of Translation with Explicit
introdujeron" (simple past), which is more contextually
Ordering) scores as the primary metrics, along with
appropriate and sounds more fluent in Spanish.
qualitative assessments of translation fluency and
semantic accuracy.
5.3 Results
This section provides a detailed comparative evaluation
5.1 Quantitative Results
of traditional and advanced NLP methods using metrics
including loss, BLEU, METEOR, and accuracy for
5.1.1 BLEU Scores
machine translation tasks. Key algorithms compared
BLEU scores are widely used in machine translation to
include LSTM (Long Short-Term Memory), CNN
measure the closeness of machine-translated output to
(Convolutional Neural Network) each playing a critical
human translations. A higher BLEU score says similarity
role in capturing sequence dependencies and contextual
in the given inputs
information.

The results show a substantial improvement in BLEU

scores for both language pairs, with the hybrid model
outperforming the baseline SMT by 18.8% and 15.6%
for English to Hindi and English to Spanish,
respectively.

2
 Handling Ambiguity and Polysemy:
Embeddings may not sufficiently resolve word ambiguity
(polysemy) or multiple meanings of the same word based on
context.
Evaluation Metrics and Benchmarking:
The standard metrics used to evaluate translation quality, such
as BLEU, may not fully capture the improvements brought by
embeddings.
7. Conclusion
This paper demonstrates the effectiveness of combining
traditional SMT methods with neural network-based
word embeddings to enhance machine translation
systems. The proposed hybrid system provides a
practical solution for improving translation quality,
especially in low-resource language settings or domain-
specific applications. Future work will explore
integrating more sophisticated embeddings and fine-
tuning NMT models for specific domains.
1. Evaluation Metrics However, there are still many challenges to be addressed
1. Loss Function: Measures the model's in this field, such as developing techniques to handle
performance during training, quantifying the low- resource languages, handling out-of-domain data,
discrepancy between predicted and actual and improving the interpretability of NMT models.
values. Lower values indicate better model Overall, improved NMT using NLP is a highly active
accuracy in the prediction phase. area of research with significant potential for practical
2. BLEU Score: Evaluates translation quality by applications in fields such as international business,
comparing machine-translated text to one or diplomacy, and education.
more reference translations. Higher scores
indicate closer matches to human-level 8. References
translation.
3. METEOR Score: This metric considers
 J. (2013). Efficient Estimation of Word
Representations in Vector Space. arXiv
synonyms, stemming, and word order,
preprint arXiv:1301.3781.
making it a good measure for semantic
relevance in translations. Higher METEOR
scores reflect improved semantic accuracy.  Vaswani, A., Shazeer, N., Parmar, N.,
4. Accuracy: Measures the overall correctness Uszkoreit, J., Jones, L., Gomez, A. N., &
of predictions made by the model on a Polosukhin, I. (2017). Attention is All You
Need. In Advances in Neural Information
sentence level.
Processing Systems, 5998-6008.
5.4 Comparative Results Analysis
 Junczys-Dowmunt, M., Grundkiewicz, R.,
Enhancing dataset size and refining hyperparameters, Dwojak, T., & Heafield, K. (2018). Marian:
especially in the Hybrid model, could further improve Fast Neural Machine Translation in C++.
translation quality, particularly for idiomatic and arXiv preprint arXiv:1804.00344
domain- specific phrases. Additionally, exploring
advanced transformer-based embeddings in place of  Koehn P, Och FJ, Marcu D (2003) Statistical
Word2Vec could further enhance context capture in the phrase-based translation. University of
Hybrid approach. Southern California Marina Del Rey
Information Sciences Inst
6. Research Gaps
Limited Coverage of Contextual Understanding:  Sutskever I, Vinyals O, Le QV (2014)
While embedding models can capture semantic meaning to Sequence to sequence learning with neural
some extent, they may not fully understand nuanced or networks. Advances in neural information
culturally embedded expressions, which can affect translation processing systems 27
accuracy.
Human-Like Evaluation of Translations:  Liu B, Lane I (2016) Attention-based
recurrent neural network models for joint
The paper may not fully address how human evaluators
perceive the quality of translations produced with intent detection and slot filling. arXiv preprint
embedding-enhanced SMT. arXiv:1609.01454
Domain-Specific Translation Challenges:
 Sinha R, Mahesh K (2004) An engineering
The current model may struggle with translations specific to perspective of machine translation:
certain domains (e.g., medical, legal, or technical texts) anglabharti-II and anubharti-II architectures.
where terminology and phrase usage are highly specialized.
Proceedings of international symposium on
machine translation, NLP and translation

3
 support system (iSTRANS-2004)

 Chatterji S et al. (2009) A hybrid approach

for bengali to hindi machine translation.
Proceedings of ICON-2009: 7th
International Conference on Natural
Language Processing