THE LIONBRIDGE 2023

MACHINE TRANSLATION REPORT

Lionbridge experts analyze trends in Machine Translation,
provide insight into likely developments, and project how
advancements will help businesses flourish.
 EXECUTIVE SUMMARY More and more companies have turned to
Machine Translation technology to enhance
customer experience — for good reasons. MT
During the past few years, businesses have offers a fast and effective alternative to human
increasingly embraced Machine Translation (MT) translation and enables companies to have
technology. Even so, MT’s quality stagnated real-time conversations with their global customers.
in 2022. The Lionbridge Machine Translation Tracker
— the industry’s longest-standing measurement of Machine Translation has also helped companies
the major MT engines — bears this out. It registered improve their employees’ productivity across
no significant quality improvements among the top languages and geographies, assisting brands in
five engines in 2022. delivering better workforce experiences during the
COVID-19 pandemic.
These results were a harbinger of a new Machine
Translation technology paradigm emerging in 2023. Business use cases for Machine Translation have
As it takes hold, MT quality will improve significantly. grown over the last few years.

Here’s how companies from a variety Legal and Financial Services

of sectors use Machine Translation for These service providers have been among the
business gain: early adopters of MT, using it for eDiscovery
processes and market research activities.
E-commerce and Retail MT has enabled them to process substantial
Retailers use MT to quickly and efficiently translate amounts of multilingual content.
product descriptions and other marketing collateral
to bring their offerings to multiple markets. The Public Sector
The technology has enabled these businesses to Governmental agencies use MT to provide
grow cross-border trading at exponential rates. local municipal services in the languages of
the local communities, thereby cost-effectively
Travel and Hospitality overcoming language barriers.
Travel-related businesses use MT to translate
descriptions of destination sites, such as hotels, Global Companies of Any Sector
rental properties, restaurants, or other points Companies with global workforces use MT
of interest, and to translate customer reviews. internally to facilitate internal communications,
The technology has enabled these businesses to whether the communications involve a chat or a
enhance customer experience and connect with simple document.
prospective customer bases faster.
They also use it to permit support teams to
Healthcare extend their coverage and reduce incident
Medical-related organizations use MT to resolution times. The technology has enabled
translate medical research and clinical trial data. global companies to bolster office productivity.
The technology has enabled these users to make
information more readily available to the public
and improve treatments and patient outcomes.

Lionbridge | 2023 Machine Translation Report Page 2 | LIONBRIDGE.COM

 Every global company wishing to thrive in our interconnected
economy must embrace and fully leverage Machine Translation.

Despite past advancements and growing use cases, What Does the Future Hold
MT still has limitations. Some of its longstanding quality for Machine Translation?
issues include its inability to attain and consistently Having tracked the major MT engines for many
achieve the right formality level, tone, or handling of years and masterfully leveraging new technologies,
negation. These limitations deter growth. Research Lionbridge is well-suited to analyze developments in
into and the use of Large Language Models (LLMs) 2023 and beyond. We anticipate MT’s existing Neural
hold promise to resolve these issues and unlock a new Machine Translation (NMT) paradigm will end. A
technological leap for Machine Translation. new paradigm will replace it, likely based on Large
Language Models (LLMs) like ChatGPT. The release
Big Tech’s investments in LLM technology — such as of GPT-4 and growth in LLMs are having significant
Microsoft’s $10B investment in OpenAI, the company business implications.
behind the ChatGPT, GPT-3, and GPT-4 models — is
accelerating the development of this technology and You can expect the following:
advancing the Natural Language Processing (NLP)
field. These advancements will inevitably disrupt the A significant leap in MT quality, including
translation and localization industry and change how workflow automations
companies create and translate content. Increased content output
A reduced supply of top-notch human translators
The exponential advancements of NLP, specifically Increased adoption of Machine Translation
LLMs, will transform how content is created and Machine Translation as a means for Customer
localized. The upshot will be exponential gains in Experience (CX) enhancement
productivity and speed as human translators process
much larger volumes of content. Every global company wishing to thrive in our
interconnected economy must embrace and fully
Companies that master and leverage AI in their content leverage Machine Translation. Read on as we examine
engines will gain a significant competitive advantage in the technology’s developments — or lack thereof — in
our increasingly digital economies. 2022, what that has meant in 2023, and what it will
mean for the years ahead.

Lionbridge | 2023 Machine Translation Report Page 3 | LIONBRIDGE.COM

What Does the Machine Translation Quality
of 2022 Tell Us About How It’s Evolving?
Machine Translation (MT) is having a moment.
Companies increasingly realize the benefits of using
MT technology. This realization is evident even in
sectors like life sciences, which traditionally have
been reluctant to embrace MT technology.

But as far as output quality goes, improvement

was flat in 2022, which paves the way for new and
exciting changes for MT.

Except for minor improvements by Microsoft at

the end of 2022 and by Amazon, the major MT
engines are performing similarly in quality.
Further, these minor improvements are less
significant than quality jumps in previous years.

At the end of the Statistical MT paradigm, we saw

similar indicators: flat lines for quality improvement
and a uniform MT output quality among all the top
engines. It leads us to one conclusion — we are
approaching the end of the current Neural Machine
Translation (NMT) paradigm.

Large Language Models (LLMs) — with their massive

amounts of content, including multimodality and
multilingualism — will be a big part of what’s next.

Companies that master and leverage

AI in their content engines will gain a
significant competitive advantage in
our increasingly digital economies.

Lionbridge | 2023 Machine Translation Report Page 4 | LIONBRIDGE.COM

 A MACHINE TRANSLATION PRIMER

To fully capitalize on Machine Translation and task at hand and more advanced Machine Learning,
enjoy its profound benefits — for the first time, which is how computers gain the knowledge required
enabling companies to Locailize everything™ — it’s for AI applications.
necessary to have a fundamental understanding of
the evolution of the technology. Machine Learning
Machine Learning (ML) is a branch of computer
What is MT? What has triggered widespread global science that uses massive amounts of data to teach
adoption? What are its major strengths and pitfalls to computers how to perform tasks. Machine Learning
avoid when using it? And what is the backdrop against examines data related to a particular task, finds
which it has been evolving? patterns in those data, makes associations among
those patterns, and then uses those new learnings to
Artificial Intelligence shape how the computer performs the task.
In the most basic sense, Machine Translation uses
Artificial Intelligence (AI), or the “intelligence” If, after this analysis, the computer gets better at
machines demonstrate, to perform tasks that usually performing the task, then Machine Learning has
require inherently human thinking, such as learning occurred. Because we have vast language and
and problem-solving. In this case, AI is used to localization data, people are using Machine Learning
perform translations. In recent years, AI has benefited to improve computer performance in everything from
from increasing computer power. More powerful weather forecasting to automatic stock selection to
computers yield more intensive processing during a Machine Translation.

Machine Learning examines data related to a particular task,

finds patterns in those data, makes associations among
those patterns, and then uses those new learnings to shape
how the computer performs the task.

Lionbridge | 2023 Machine Translation Report Page 5 | LIONBRIDGE.COM

 What Is Machine Translation Technology?
Machine Translation is automated translation. When you
present source material to a computer in one language, it
gives it back to you in another language. MT technology
is imperfect, but it’s one of the most powerful tools for
producing usable translations more efficiently.

Over the last several decades, MT has improved the

quality of its output and the breadth of languages it
supports. From simple word replacement systems in the
very early days of MT to the explicitly coded grammar
and lexicons of rules-based MT, to the number-crunching
paradigm of Statistical MT, to the Deep Learning and
neural networks of Neural MT, the development of
Machine Translation has mirrored our increasingly
sophisticated use of computers.

How Has Machine Translation Evolved?

Rule-based Machine Translation
Early Machine Translation engines used rule-based
methods. As the name suggests, Rule-based Machine
Translation (RBMT) relies on language rules developed
by humans and from dictionaries to execute translations.
These systems are word-based.

Although this method can produce accurate translations,

Rule-based Machine Translation systems work best with
simplified languages that use simplified grammar and a
smaller dictionary.

The reduced complexity allows the Machine Translation

engine to perform better. Rule-based methods are less
successful when translations are needed for specific
domains, when idioms are used, or when the text has
ambiguity. Since the introduction of RBMT, language
technology has evolved considerably.

MT technology is imperfect, but it’s one

of the most powerful tools for producing
usable translations more efficiently.

Lionbridge | 2023 Machine Translation Report Page 6 | LIONBRIDGE.COM

 The learning algorithm then builds a language model
that calculates the likelihood that given words and phrases
appear next to one another in the target language.

Statistical Models language. When it’s time to translate new material, the
Statistical Machine Translation (SMT) relies on a large SMT system breaks the new source sentence down into
number of translation candidates for a given source n-grams, finds the highly associated target language
sentence, then selects the best one based on the n-grams, and generates candidate sentences.
likelihood of words and phrases appearing together in
the target language. The final translation is that sentence whose target
language n-grams correlate most highly with the
SMT learns about translation through the lens of source sentence’s n-grams and whose target language
“n-grams” — small groupings of words that appear words are most likely to appear together in the target
together in the source and target language. language. SMT works surprisingly well, especially
The SMT system is given training material — that is, since there is nothing linguistic about an SMT system;
many examples of sentences in the source language indeed, the system only considers n-grams, never a
and their translations into the target language. complete sentence.

The learning algorithm divides source sentences and Hybrid Machine Translation
target sentences into n-grams. It determines which Companies then began experimenting with
target language n-grams are likely to appear in a Hybrid Machine Translation (HMT), which combined
translation when a certain source language n-gram the output of Statistical Machine Translation and
appears in a sentence. Rule-based Machine Translation systems.
These advancements popularized Machine Translation
The learning algorithm then builds a language model technology and helped adoption on a global scale.
that calculates the likelihood that given words and Another technological leap would come from a newer
phrases appear next to one another in the target approach to MT: Neural Machine Translation.

Lionbridge | 2023 Machine Translation Report Page 7 | LIONBRIDGE.COM

Neural Machine Translation Visualizing How Neural
Neural Machine Translation (NMT) overcomes the Machine Translation Works
most significant shortcoming of SMT: its reliance While NMT trains on Translation Memories (TMs) as
on n-gram analysis. NMT empowers the machine Statistical Machine Translation does, it uses deep
— the system receives the training material, just learning — and possibly a higher volume of training
as it would with SMT, but there’s a fundamental data — to build an artificial neural network.
difference. Once the system receives the material,
it decides on its own how to learn everything You can describe Neural Machine Translation as
possible about that data. “raising” a neural system. Think of it like playing the
piano: When you make a mistake, you back up, try
NMT systems build information vectors for each again, and repeat the process until you succeed.
source sentence, associating information about Neural MT systems try to find their way through
each word with surrounding words. Some systems neural networks similarly.
develop hundreds of pieces of information per
word, creating a deep sense of accuracy. When you present training material to the deep
learning algorithms, you don’t necessarily tell them
Through deep learning, NMT systems capture a how to proceed. You let the system find patterns,
massive amount of information about each word such as contextual clues around the source
and source sentence. Then, it uses an attention sentence. The specifics of the process, however,
model to hone the critical features it has learned remain mysterious in many ways.
by analyzing these massive data streams, which are
important for the translation process. Much of the processing is done in “hidden layers”
of complex data, meaning it’s hard to see how the
The result is translations that show marked neural network makes its decisions, which makes
improvements in fluency, which means that it difficult to build a concrete mental picture of the
computer-generated translations are starting to NMT process. We can only present the training
sound more and more natural. Neural Machine material, let the algorithms do their thing, and
Translation has addressed some of Machine tweak the training material if the translations
Translation’s longstanding shortcomings, aren’t accurate.
including the poor readability of automated
translations and its incompatibility with
certain languages.

This development resulted in Machine Translation

being good enough for comprehending or gisting Once the system receives the material,
large volumes of documents and for regular, it decides itself how to learn everything
non-mission-critical business documents. Of the possible about that data.
three approaches, NMT is the clear winner. It has
been a game-changer as it has increased the use
of MT to accelerate production processes.

Lionbridge | 2023 Machine Translation Report Page 8 | LIONBRIDGE.COM

What Is the History of Machine Translation?
Machine Translation has come a long way since its inception in the 1950s.
Here are some of its major milestones.

1954 Georgetown researchers perform the first-ever public demonstration of an early MT system.

1962 The Association for Machine Translation and Computational Linguistics is formed in the U.S.

1964 National Academy of Sciences forms a committee (ALPAC) to study MT.

1970 The French Textile Institute begins translating abstracts using an MT system.

1978 Systran begins to translate technical manuals.

1989 Trados is the first to develop and market Translation Memory technology.

1991 The first commercial MT system between Russian, English, and German-Ukrainian is
developed at Kharkov State University.

1996 Systran and Babelfish offer free translations of small texts on the web.

2002 Lionbridge executes its first commercial MT project using its rule-based MT engine.

Mid-2000s Statistical MT systems launch to the public. Google Translate launched in 2006, and Microsoft
Live Translator launched in 2007.

2012 Google announces that Google Translate translates enough text to fill 1 million books daily.

2016 Both Google and Microsoft enable Neural Machine Translation (NMT), slashing word order
mistakes and significantly improving lexicon and grammar.

2020 As of October, Google Neural Machine Translation (GNMT) supports 109 languages.

2022 ChatGPT, a Large Language Model (LLM) that can generate human-like text based on context,
goes mainstream in November with significant implications for Machine Translation.

2023 A major MT paradigm shift is anticipated as a type of LLM evolves and disrupts MT.

Lionbridge | 2023 Machine Translation Report Page 9 | LIONBRIDGE.COM

What Are Machine Translation’s Benefits? On average, Lionbridge customers achieve
Machine Translation is highly appealing for two the following results when using Machine
primary reasons: Translation as part of their workflow.

It’s incredibly fast, providing reams of translations

within seconds
Greater Translation Efficiency
It’s highly cost-effective, enabling companies to Up to 40% cost savings.
translate more than they ever thought possible

Speed + cost-savings = more localized content and

enhanced customer experience Faster Turnaround Times
Up to 60% shorter delivery times.
Automated translations enable companies to Locailize
Everything and meet the ever-growing demand for
more content to be pushed out to the market faster, Enhanced Customer Experience
all within the same or even smaller budget. The ability to speak to customers in
their native language everywhere.
Why Else Should Companies
Implement Machine Translation?
In addition to saving time and reducing
localization costs, using MT for non-critical
content enables companies to:

Localize more material, thereby extending their

reach based on the number of languages they
translate and the number of markets they enter
Improve their terminology and brand
voice consistency
Localize real-time customer interactions and
thereby enhance customer satisfaction
Increase the productivity of global teams by
removing communication barriers

Machine Translation also helps companies

allocate more of their resources and budget to
content creation since they require less of their
budget for translation.

This budget reallocation can help companies

increase their content velocity and attract more
readers to fuel their growth. This gain is especially
true for digital-first businesses.

Lionbridge | 2023 Machine Translation Report Page 10 | LIONBRIDGE.COM

What Is Machine Translation’s
Primary Shortcoming?
Despite MT’s ever-growing sophistication and
improvements to its quality output, engines still have
a long way to go before they can match humans’
understanding of nuance, tone, sarcasm, humor,
meaning — the list goes on.

If you rely solely on Machine Translations, its robotic

tone could end up compromising your message. That,
in turn, could suggest to customers that your service is
low-quality or unreliable.

Translations of public-facing content must read

naturally and fit seamlessly into the cultural context of
your target market to be effective. To produce high-
quality, localized content, it’s often worth paying extra What Other Risks Are Associated
for native speakers to review MT output. That’s where With Machine Translation?
Machine Translation Post-Editing (MTPE) comes in. Though MT has made significant advances,
the technology has yet to be perfected.
How Does Machine Translation As such, you are taking some risks when
Post-Editing Provide the Best of MT deploying the technology beyond getting a
and Human Translation? robotic output — especially if you take no
Machine Translation Post-Editing (MTPE) is a hybrid action to mitigate MT’s shortcomings.
of MT and traditional human translation. As its name
suggests, a human provides post-editing following What Exactly Can Go Wrong?
Machine Translation. First, the software produces Here’s what can happen when using
an initial translation of the material. Then, a human Machine Translation:
translator edits the text, checking for accuracy, clarity,
flow, and local resonance. This approach offers the Your copy may not be inclusive
unparalleled speed of MT with the attention and or reflect your intended level of
sensitivity of human translators. language formality due to the
engine’s inability to know when to
While machines make short work of bulk translations, use neutral words or match the
human translators infuse the copy with the translation appropriate formality level to your
quality only a human can provide. This hybrid approach target audience.
is a good option when you translate customer-facing Your copy may have errors — to
content and want a tailored method that maximizes varying degrees — in the
quality and minimizes turnaround times. This option translation output.
is one of the quickest and most affordable ways to get
high-quality translation results. While these issues can cause negative
consequences, errors have the potential to
be especially harmful to your company.

Lionbridge | 2023 Machine Translation Report Page 11 | LIONBRIDGE.COM

 Machines cannot exercise
judgment the way people can.

MT Risk: Errors Catastrophic errors typically show up in three

The Type of Errors Resulting From MT primary contexts:
MT engines produce two major types of errors:
Mistranslation of Key Entities
Standard Errors This error is the mistranslation of proper
These are target language mistakes that names (individuals or organizations), important
pertain to the content’s linguistic features. numbers, or units of measurement.
Standard errors include grammar, spelling, or Negation and Opposite Meaning
punctuation mistakes. While native speakers are This error involves mistakes in the target
likely to notice these slip-ups, they rarely cause language that result in the opposite meaning
disastrous consequences for a company. of the original text’s intended meaning.
Catastrophic Errors Hallucinations
MT catastrophic errors transcend linguistic This error occurs when MT introduces content
features and inaccurately convey the intent that is not present in the source. Engines may
of the source text. Misinformation or generate offensive, profane, aggressive,
misunderstandings that result from these errors or highly sensitive words under certain
can potentially cause companies reputational, circumstances. When this type of catastrophic
financial, or legal repercussions and lead to error happens, there is a problem with the
adverse public safety or health consequences. MT engine software itself.

Why MT Engines Make Catastrophic Errors The public witnessed a real-world example of a
Think of a catastrophic error as an MT engine catastrophic error involving a proper name on a
malfunction. It can occur if the engine doesn’t Spanish governmental agency website. In that
understand the context of the text, such as when one instance, the department head’s name, Dolores del
word has two meanings or if there is a typo in the Campo, was omitted from the ministry’s official site.
source text. These errors can happen if the engine is Instead, the literal translation — It is pain of field —
not trained well or a flawed glossary is used, which appeared in place of the name.
then causes the same mistakes to appear repeatedly.
Catastrophic errors occur because engines are
imperfect despite their sophistication. Machines
cannot exercise judgment the way people can.

Lionbridge | 2023 Machine Translation Report Page 12 | LIONBRIDGE.COM

How to Minimize The Risk of Catastrophic Errors
Only when computer scientists improve existing MT
technology to prevent these errors from occurring in
the first place can we use automated technology to
identify potential issues, revise problematic sentences,
and promote accuracy during the translation process. Catastrophic errors occur
For instance, Lionbridge administers specific automated because engines are imperfect
quality checks in translated texts via its Smairt MT™ despite their sophistication.
offering — and in conjunction with its cutting-edge Machines cannot exercise
Smairt Content™ language AI — to detect errors while judgment the way people can.
maintaining MT speed and minimizing the need for
post-editing by human translators.

These automated methods detect:

Incorrect translations of proper names,

including the names of individuals or organizations,
by identifying entities in the source text that can be
both a named entity and a common word
Offensive, profane, or highly sensitive words
by combining a supervised Machine Learning (ML)
algorithm and a list of offensive terms
Opposite meanings between original and
translated texts by identifying negative particles
(sentences that contain the word not or its
shortened form n’t) in the original text or the
translated text, but not in both
Hallucinations in the translation — words that
appear in the translation but not the source — via
dictionaries or a list of offensive words to address
insulting hallucinations

Automated quality checks do not guarantee the

elimination of catastrophic errors. These checks can miss
errors, causing a false negative result. Nonetheless, they
are highly effective at helping us find problems.

Using this approach, we can focus professional

translators on the flagged sentences and avoid
reworking the whole document. When we can alert
professional translators to where problems are most
likely to be found, we enhance the efficiency of the
localization process.

Lionbridge | 2023 Machine Translation Report Page 13 | LIONBRIDGE.COM

 MT Risk: Mismatch in Language Formality
What is Language Formality,
And Why Does it Matter?
Language formality is the degree of formality with
which we express ourselves. We typically wouldn’t
speak to our supervisor like we would talk to a close
friend, even if we related the same story to each
person. We may choose to use different vocabulary
and grammar.

We’re more likely to use formal language in a

business context or a more formal situation.
Formal language connotes politeness or respect.
We typically use informal language during relaxed
situations and for people we know well.

Speakers from certain cultures may perceive a wrong

formality level as rude. For instance, you wouldn’t
speak to an elder like you would to a child in Korean.
There are different verb endings for each of the seven
speech levels in Korean, five of which are categorized
as formal and the remaining two informal. Misusing
formality levels can be embarrassing.

Additionally, there are many cases in which formal

language may be wrong, not because it is offensive
but because it is totally out of place. Errors like these
can alienate your audience. For example, translations
for computer games and student programs require a
casual tone.

When companies take the time and try to adapt the

appropriate style to the target during translations,
they send a strong message that they care about their
customers. Such actions may be even more appealing
to potential prospects. Effectively connecting with
your audience helps you to succeed.

Speakers from certain cultures may

perceive a wrong formality level as rude.

Lionbridge | 2023 Machine Translation Report Page 14 | LIONBRIDGE.COM

 Addressing Language Formality When Using MT
Companies can control language formality through
rules-based techniques that use rules to replace the
Why it’s Difficult For Engines to
undesired style with a correct translation and
Get Language Formality Right
non-rules-based techniques that involve the use of
Machine Translation and formal vs. informal
custom MT models.
language can be problematic as engines
can produce incorrect and inconsistent
While most commercial MT systems do not support
formality. Why? MT models typically return
language formality or gender parameters, companies
a single translation for each input segment.
are making progress. DeepL (API) and Amazon (console
When the input segment is ambiguous, the
and SDK) currently offer features that control formality.
model must choose a translation among
several valid options, regardless of the
There are three options to choose from: default, formal
target audience.
and informal. The default option does not change the
formality of the Neural Machine Translation (NMT)
Letting the model choose between different
output. The formal/informal function allows the user
valid options may result in inconsistent
to choose between a formal or informal tone of voice.
translations or translations with an incorrect
Specifically, the function sets the pronouns and related
formality level.
words used in the translation.

It is especially challenging to get the correct

Lionbridge’s enterprise-grade Machine Translation
output when the source language has fewer
solution, Smairt MT, allows linguistic rules to be
formality levels than the target language.
applied to the target text to produce Machine
For instance, languages like French have
Translations with the desired style or formality. It can
well-defined formal modes — tu vs. vous —
even promote neutral language and prevent bias in
while English does not.
localization — the difference between referring to
someone as a salesperson versus a salesman.

Our specialists maintain an updated database of rules

that are fed back into the analysis of MT outputs to
control the outcome. Provided that there is sufficient
material, we combine a rule-based strategy with
custom MT models to achieve an optimal result.

While it’s challenging to get language formality right

Lionbridge’s enterprise-grade when using MT, it’s possible. Later in the whitepaper,
Machine Translation solution, we’ll explore exciting technological developments that
Smairt MT, allows linguistic rules may solve language formality-related issues.
to be applied to the target text
to produce Machine Translations
with the desired style or formality.

Lionbridge | 2023 Machine Translation Report Page 15 | LIONBRIDGE.COM

 2022 MACHINE TRANSLATION YEAR-IN-REVIEW

What Are the 2022 MT Key Trends? We calculated how well three major engines handled
Lionbridge MT experts found 2022 to be notable automated translations from English into numerous
for both what did — and did not — transpire. languages. We determined the quality by calculating
Having observed so many MT-related technological the average edit distance — the number of edits a
advancements during the past few years, our team human must make to the MT output for the resulting
anticipated more of the same. But MT did not make translation to be as good as a human translation.
significant strides, as our Machine Translation
Tracker revealed. The lower the number, the more effective the
automated translation is. As shown in Figure 1, paying
With rare exceptions, the major engines made attention to these results is worth a company’s while.
little-to-no improvements during the year. This trend
has implications for the future. But first, let’s take a According to our analysis, in certain situations:
closer look at the 2022 results.
DeepL translated Spanish better than
Google and Microsoft
How Did the Top MT Engines Perform in 2022?
When a company wants to start using MT or improve Google translated Japanese better than DeepL
the way it currently uses MT, it is critical to identify Microsoft translated Polish better than DeepL
which MT engines will work best based on their The three engines performed similarly for
specific needs. As we delve deeper into how the major Italian, Turkish, and Hebrew
MT engines performed in 2022, one thing becomes
very clear: One engine can’t do it all. These results demonstrate the complexity and
challenges inherent in Machine Translation, which
Comparison of MT Engine Performance involves navigating the nuances and complexities of
Based on Language different languages, cultures, and domains.
A company working with Spanish content benefited
from selecting DeepL for its automated translations; It is not surprising to see variations in performance
it had better alternative options when translating across various MT engines, as no single algorithm or
Japanese. That’s because each engine’s performance approach can work perfectly for all languages and
varies based on the language it handles. content types.

Lionbridge | 2023 Machine Translation Report Page 16 | LIONBRIDGE.COM

Why Does MT Perform Differently the level of optimization and customization of the MT
Depending on the Language? engine may also play a role. Some MT engines may be
Possible reasons for these performance differences optimized and customized specifically for particular
include the quality and quantity of training data languages or domains, resulting in more accurate and
available for each language. Machine Translation effective translations for those use cases.
models rely on a large corpus of high-quality bilingual
data to learn how to translate a language accurately. Overall, these factors contribute to the complexity
The MT engine may struggle to produce accurate and variability of Machine Translation, highlighting
translations if there is a shortage of such data for a the importance of understanding the strengths
particular language. and limitations of different MT engines for various
languages and use cases.
Additionally, each MT engine’s technical and linguistic
features vary, with each engine employing its own set of Choosing the best MT engine for a specific use
algorithms, architectures, and approaches to Machine case requires expertise in Machine Translation.
Translation. Specific engines may perform better on It is not a simple task. Careful evaluation and
certain language pairs or content types, depending on ongoing optimization are critical to ensure the
the features and capabilities of their models. Moreover, best possible results.

The Average Performance of MT Engines Per Language

Target Languages

Turkish MT Provider
Thai
DeepL
Google
Swedish
MicrosoftV3
Spanish

Slovenian

Romanian

Polish

Norwegian

Latvian

Japanese

Italian

Hebrew

Greek

French

Dutch

Danish

Chinese

Brazilian

0.3 0.4 0.5 Edit Distance

Figure 1. The average edit distance per language and MT provider

Lionbridge | 2023 Machine Translation Report Page 17 | LIONBRIDGE.COM

Automated Translation Performance Engines found Media and Marketing content —
Based on Domain known for its originality — the most challenging
The more creative your content, the more difficult it type of content to translate
will be for engines to translate it effectively. Content related to Textile and Fashion — which
often has imaginative and nuanced descriptions
Translation quality based on domain makes that — was the next most challenging type of content
point. We tracked the average edit distance — the for machines to process
number of edits a human must make to the MT output Machines handled clear-cut content in the
for the resulting translation to be as good as a human Automotive and Machinery sector best
translation — for the main domains to determine how
well the engines handle various types of content.
We expected results like these. Generic MT engines
The lower the number, the better the translation shine when dealing with straightforward content that
quality is. is relatively simple and easy to understand and has
a clear structure and vocabulary that is not highly
According to our analysis, as shown in Figure 2, specialized or technical.

The Quality of MT Output Per Domain

Domains

Media and Marketing

Textile and Fashion

Travel and Leisure

Computing Hardware

Life Sciences

Financial

Electronics

Computing Software

Business and Commerce

Automotive and Machinery

0.35 0.40 Edit Distance

Figure 2. The average edit distance per domain.

Lionbridge | 2023 Machine Translation Report Page 18 | LIONBRIDGE.COM

Comparison of MT Engine Performance Financial: Microsoft and DeepL performed
Based on Domain and MT Engine better than Google
A company translating Media and Marketing content Automotive and Machinery: Google performed
benefited from selecting DeepL as its MT provider; slightly better than Microsoft and DeepL
a company translating Textile and Fashion-related
content had better alternative options. Based on However, note that for us to recommend the right
available data, we considered how well four major system, we need to analyze your specific content. That
engines handled content for various sectors. We way, we know what you need and which engine will
determined the quality by calculating the average edit address your requirements best.
distance — the number of edits a human must make
to the MT output for the resulting translation to be as The above results suggest the performance of MT
good as a human translation. The lower the number, engines can depend not only on language pairs as
the more effective the automated translation is. As we commented on, but also on the specific domain or
shown in Figure 3, engines don’t perform similarly industry terminology and phrasing used.
across sectors and content types in certain situations.
Therefore, when selecting an MT engine, it is essential
According to our analysis, in these specific examples: to consider the language pairs being translated
and the specific domain and industry context. This
Media and Marketing: DeepL performed better approach requires expertise and knowledge of the
than Google and Microsoft language and the particular domain and can result in
Life Sciences: DeepL performed better than better translation quality and accuracy for the given
Microsoft and Google content type and industry.

The Average Performance of MT Engines Per Domain

Domains

MT Provider
Travel and Leisure
Amazon
DeepL
Textile and Fashion
Google
Media and Marketing MicrosoftV3

Life Sciences

Financial

Electronics

Computing Software

Computing Hardware

Business and Commerce

Automotive and Machinery

Edit Distance
0.2 0.2 0.2 0.2 0.2

Figure 3. The average edit distance per domain and MT provider

Lionbridge | 2023 Machine Translation Report Page 19 | LIONBRIDGE.COM

Comparison of Machine Translation According to our analysis:
Engine Quality
How do the main engines perform against one Microsoft Bing aggressively catches up
another overall? In figure 4, we compared the output with the leaders
quality between the five major engines from May Amazon and Google trend lines are almost
2018 through December 2022 for German, Spanish, perfectly parallel
Russian, and Chinese using the inverse edit distance. As a technology, Machine Translation did not
significantly improve during 2022
The edit distance measures the number of edits a
human must make to the MT output for the resulting
translation to be as good as a human translation. The By the beginning of 2023, the difference between
inverse edit distance means the higher the resulting the major engines is minimal. Suppose the Neural
number, the better the quality. Machine Translation (NMT) paradigm continues to be
dominant and MT providers continue to invest at a
For figure 5, we used trend lines for each engine, similar rate. In that case, we anticipate that MT engine
which provides interesting insights. Figure 4 and performance will ultimately converge within the year,
figure 5, taken together, demonstrate engine quality though there may be some differences between
among the top MT providers converging. language pairs and domains.

By the beginning of 2023, the difference

between the major engines is minimal.

Lionbridge | 2023 Machine Translation Report Page 20 | LIONBRIDGE.COM

 0.4
0.41
0.42
0.43
0.44
0.45
0.46
0.47
0.48
0.49
0.5
0.4
0.41
0.42
0.43
0.44
0.45
0.46
0.47
0.48
0.49
0.5
May | 2018 May | 2018

June | 2018 June | 2018

July | 2018 July | 2018

Aug | 2018 Aug | 2018

Sep | 2018 Sep | 2018

Oct | 2018 Oct | 2018

Nov | 2018 Nov | 2018

Dec | 2018 Dec | 2018

Inverse Edit Distance

Inverse Edit Distance

Jan | 2019 Jan | 2019

Feb | 2019 Feb | 2019

Mar | 2019 Mar | 2019

Apr | 2019 Apr | 2019

May | 2019 May | 2019

June | 2019 June | 2019

July | 2019 July | 2019
Performance of MT Engines

Aug | 2019 Aug | 2019

Sept | 2019 Sept | 2019

Oct | 2019 Oct | 2019

Lionbridge | 2023 Machine Translation Report

Nov | 2019 Nov | 2019
Dec | 2019 Dec | 2019

Jan | 2020 Jan | 2020

Figure 5. Trendlines of the five major MT engines’ performances

Feb | 2020 Feb | 2020

Performance of MT Engines: Trendlines

Mar | 2020 Mar | 2020

Apr | 2020 Apr | 2020

May | 2020 May | 2020

June | 2020 June | 2020

July | 2020 July | 2020

Figure 4. A comparison of overall MT quality based on the inverse edit distance

Aug | 2020 Aug | 2020

Sept | 2020 Sept | 2020

Oct | 2020 Oct | 2020

Nov | 2020 Nov | 2020

Dec | 2020 Dec | 2020

Jan | 2021 Jan | 2021

Feb | 2021 Feb | 2021

Mar | 2021 Mar | 2021

Apr | 2021 Apr | 2021

May | 2021 May | 2021

June | 2021 June | 2021

July | 2021 July | 2021

Aug | 2021 Aug | 2021

Sept | 2021 Sept | 2021

Oct | 2021 Oct | 2021

Nov | 2021 Nov | 2021

Dec | 2021 Dec | 2021

Jan | 2022 Jan | 2022

Feb | 2022 Feb | 2022

Mar | 2022 Mar | 2022

Apr | 2022 Apr | 2022

May | 2022 May | 2022

June | 2022 June | 2022

July | 2022 July | 2022

Aug | 2022 Aug | 2022

Sept | 2022 Sept | 2022

Oct | 2022 Oct | 2022

Nov | 2022 Nov | 2022

Dec | 2022 Dec | 2022

Timeline
Timeline

Bing
Bing

DeepL
DeepL

Google
Google

Yandex
Yandex

Amazon
Amazon

Page 21 | LIONBRIDGE.COM
Comparison of Machine Translation As shown in Figure 6:
Engine Quality per Language
How did the main engines perform against one There were minimal MT improvements overall,
another in 2022, specifically for German, Spanish, as reflected by the scale used to measure the
Russian, and Chinese? We measured quality based inverse edit distance
on the inverse edit distance. Microsoft Bing made minor improvements
in German, Spanish, and Chinese during
The edit distance measures the number of edits a October/November
human must make to the MT output for the resulting 2022 proved to be a flat year
translation to be as good as a human translation.
We can conclude that Neural Machine Translation has
The inverse edit distance means the higher the hit a plateau. A new iteration will be necessary for MT
resulting number, the better the quality. to make significant quality gains.

Performance of Machine Translation Engines per Select Languages via Inverse Edit Distance

German Spanish
Inverse Edit Distance Inverse Edit Distance

Timeline Timeline

Russian Chinese
Inverse Edit Distance Inverse Edit Distance

Timeline Timeline

Google NMT Bing NMT Amazon DeepL Yandex

Figure 6. A comparison of MT quality per language based on the inverse edit distance

We can conclude that Neural Machine Translation has hit a plateau.

Lionbridge | 2023 Machine Translation Report Page 22 | LIONBRIDGE.COM

Comparison of Machine Translation Engine Bing clearly led two domains intrinsically related
Quality per Domain to its parent company, Microsoft: Computing
How did the main engines perform against one Software and Financials. Over the year, this MT
another in 2022 for specific domains? We measured engine demonstrated significant improvement,
quality based on the inverse edit distance, as shown rising from the third position to claim the top
in Figure 7. spot in the ranking. No other MT engine has
shown a comparable level of advancement in the
The edit distance measures the number of edits a ranking within any domain.
human must make to the MT output for the resulting Yandex is the only MT provider analyzed that
translation to be as good as a human translation. The did not lead any domain. It ultimately surpassed
inverse edit distance means the higher the resulting and performed better than DeepL by the end
number, the better the quality. of 2022 in only two domains: Automotive and
Computing Software.
According to our analysis:

Machine Translation performs better

for procedural content than more creative
content because procedural content is
typically more straightforward and easier
for machines to process. As such, it was no
surprise that MT quality was better for the
Automotive and Computing Software sectors
than for the Retail, Marketing, and Travel
& Tourism industries since they all have
more complex content. These results are
similar to and amplify the results we found
when measuring the automated translation
performance based on domain and the average
edit distance. (See Figure 2.)
Google MT and DeepL had stable performances
over the year compared to Amazon and Bing.
Amazon did not improve significantly
during the period we analyzed. Several peaks
and valleys brought it back to its starting point
by the end of the time we measured, except
for two domains: Legal & Law and Media,
Advertising & Marketing. For these two sectors,
we observed a positive evolution that, in the
case of Legal & Law, led Amazon to outperform
its competitors slightly and for Media, Adverting,
& Marketing, enabled them to co-lead the
ranking with Google.

Lionbridge | 2023 Machine Translation Report Page 23 | LIONBRIDGE.COM

Performance of Machine Translation Engines per Select Domains
Automotive Business, Retail, and Commerce Services
Inverse Edit Distance Inverse Edit Distance

Timeline Timeline

Computing Software Financials

Inverse Edit Distance Inverse Edit Distance

Timeline Timeline

Legal and Law Health Care and Life Sciences

Inverse Edit Distance Inverse Edit Distance

Timeline Timeline

Media, Advertising, and Marketing Travel, Tourism, Recreation, Leisure, and Arts
Inverse Edit Distance Inverse Edit Distance

Timeline Timeline

Google NMT Bing NMT Amazon DeepL Yandex

Figure 7. A comparison of MT quality per domain based on the inverse edit distance

Lionbridge | 2023 Machine Translation Report Page 24 | LIONBRIDGE.COM

How Can I Get the Most Out of Ranking languages by translatability is not a
Machine Translation? straightforward process; however, we can use
To get the most out of MT technology, consider taking different metrics for evaluation. Edit Distance, the
the following steps: number of changes a post-editor makes to ensure the
final text has a human quality, can provide a sense of
Identify the ease with which MT engines MT complexity and translatability for each language
translate specific languages — the machine pair. To help you compare languages, we’ve ranked
translatability or m-translatability of languages the machine translatability or m-translatability of the
— to help inform your MT strategy and decide top 28 target languages from English.
which markets to pursue. We’ve done the
legwork for you. As shown in Table 1, most Romance languages, such as
Use terminology effectively to improve output Portuguese, Spanish, French, and Italian, require fewer
quality for any domain, even the ones that MT changes to reach high-quality levels when translated
often bungles. from English. We identified these target languages as
the easiest for machines to handle, and they took the
Consider when to conduct MT customization
first four spots in our m-translatability ranking.
vs. MT training. Each method can improve MT
output, but they are not interchangeable.
Hungarian and Finnish — two Uralic languages —
are more complex languages; they placed last in
Machine Translatability
our ranking, taking the 27th and 28th spots.
Identifying how challenging it is for engines to
Estonian, another language in the same family,
handle specific language pairs will help you allocate
is also among the more complex languages.
your budget when planning translation costs across
Based on millions of sentences processed by
languages. You will better understand which language
Lionbridge, these results underscore the importance
pairs will require more effort to translate. Having
of language families in MT results.
insight into language complexity can help support
your business decisions and help you answer the
While language comparison has limitations, the
following questions:
ranking can provide interesting insights to manage
multilingual projects better.
Should a larger share of the budget be used to
post-edit more complex language pairs?
Will light post-editing or focused post-editing,
which targets only critical areas of the content
for post-editing, be sufficient for some
languages when dealing with a tight budget?
For which languages should I use these
post-editing methods?
Should my company add language ranking to
business and cultural factors when considering
how to best allocate its budget, particularly for
low-budget projects? If a culture accepts a lower
quality level, should my company translate into a
language with a low m-translatability ranking?

Lionbridge | 2023 Machine Translation Report Page 25 | LIONBRIDGE.COM

Table 1. Language M-translatability Ranking

Rank Language (From English) Rank Language (From English)

1 Portuguese 15 Turkish

2 Spanish 16 Slovak

3 French 17 Hebrew

4 Italian 18 Latvian

5 Chinese (Simplified) 19 Polish

6 Dutch 20 Chinese (Traditional)

7 Danish 21 Lithuanian

8 Japanese 22 Czech

9 Greek 23 Arabic

10 Romanian 24 Estonian

11 Thai 25 Korean

12 Norwegian 26 Russian

13 German 27 Hungarian

14 Swedish 28 Finnish

Lionbridge | 2023 Machine Translation Report Page 26 | LIONBRIDGE.COM

 The combination of trained MT engines, glossary customization, and the
identification of preprocessing and post-processing rules ensure MT output contains
proper terminology and is similar in style to the customer’s documentation.

Terminology To Improve Domain Performance Not Translate (DNT) and glossary lists added to a
As noted, generic MT engines can put out erroneous specific profile to address Machine Translation
translations; they can especially cause undesired terminology. We help our customers create and
results for specific domains from a terminological maintain glossaries, regularly refined to include new,
point of view. The impact can be particularly harmful relevant terms and retire obsolete terminology. When
to the medical and legal fields. The effective use of glossaries are created once in Smairt MT, they can be
terminology can enable you to improve the quality of used for all the MT engines, saving time and money.
MT and achieve accurate, consistent translations no
matter what your subject matter is. Using glossaries for MT projects is more complex
than it may seem. Glossaries, if used inappropriately,
It’s imperative to train customized MT systems with can negatively affect the overall quality of Machine
domain-specific bilingual texts that include specialized Translation. The best way to follow terminology in MT
terminology. Still, when engines are trained with is through MT training. The combination of trained MT
specialized texts, accurate translations cannot be engines, glossary customization, and the identification
guaranteed if the terminology is not used consistently. of preprocessing and post-processing rules ensure MT
Research in this area proposes to inject linguistic output contains proper terminology and is similar in
information into Neural Machine Translation (NMT) style to the customer's documentation.
systems. Implementing manual or semi-automatic
annotation depends on available resources, such as MT Customization vs. MT Training
glossaries, and constraints, such as time, cost, and MT customization and MT training can help you get
availability of human annotators. more out of your MT output, but you must be intentional
about when to apply these methods. Table 2 provides
Lionbridge’s Smairt MT allows the application of an overview of Machine Translation customization
linguistic rules to the source and target text and vs. Machine Translation training and offers some
the enforcement of terminology based on Do considerations when evaluating each method.

Lionbridge | 2023 Machine Translation Report Page 27 | LIONBRIDGE.COM

 Table 2. Machine Translation Customization vs. Machine Translation Training

MT Customization MT Training

An adaptation of a pre-existing Machine The building and training of an MT engine by

What it is and Translation engine with a glossary and Do Not using extensive bilingual data from corpora
how it works Translate (DNT) list to improve the accuracy of and Translation Memories (TMs) to improve the
machine-generated translations accuracy of machine-generated translations

Improves MT’s suggestions for more accurate Improves MT’s suggestions for more accurate
What it does output and reduces the need for post-editing output and reduces the need for post-editing

Enables companies to adhere to their brand name Enables companies to attain a specific brand voice,
Specific benefits and terminology and achieve regional variations tone, and style and achieve regional variations

MT training may fail to impact output if there

is not enough quality data to train the engine;
The risks The MT could make poor suggestions and negatively
the MT could generate poor suggestions and
of using it impact overall quality when executed improperly
negatively impact overall quality if inexperienced
authors overuse terminology

Ideal for technological and detail-oriented content Ideal for highly specialized content, marketing and
and any content that requires: creative content, and any content that requires:
When to use it
• Accurate translations of terminology • A specific brand voice, tone, or style

• Regional variation, but you lack • Regional variation, and you have
sufficient data for MT training enough data for MT training

An experienced MT expert who can successfully

A minimum of 15K unique segments to
Success factors manage input and output normalization rules,
adequately train the engine
glossaries, and DNT

There are costs associated with the first training

There is a one-time cost to update the profile that and potential costs for additional training,
goes into the MT engine and some ongoing costs which may be considered over time if the MT
Cost to maintain a glossary over time; costs are relatively performance monitoring indicates room for
considerations inexpensive when factoring in the potential benefits improvement; MT training can be worth the
and are typically lower than MT training costs investment in certain cases when factoring in the
potential benefits

Lionbridge | 2023 Machine Translation Report Page 28 | LIONBRIDGE.COM

 CONCLUSIONS AND THE OUTLOOK FOR MACHINE TRANSLATION

What can we conclude about the state of Machine We’re betting that Large Language Models (LLMs)
Translation from the 2022 data and surprising results — with their massive amounts of content, including
that mainly showed stagnant quality performances for multimodality and multilingualism — will have
the year? The technology is mature and will continue something to do with a future paradigm.
to attain widespread adoption as it has unequivocally
proven its value as a business-grade technology. Why do we think this? Because of the results of our
ground-breaking analysis that compared ChatGPT's
People recognize the technology’s usefulness for translation performance with the performance
almost any translation case — with or without human of MT engines.
intervention and hybrid approaches. Indeed, according
to Global Market Insights, the translation market size OpenAI’s ChatGPT produced inferior results than
is projected to grow at a Compound Annual Growth designated MT engines — but not by much. Its
Rate (CAGR) of 30% from 2022 to 2030. Companies will performance was nothing short of remarkable.
increasingly embrace MT — including those businesses GPT-4 even surpassed one major Neural Machine
in traditionally MT-resistant domains, such as games Translation engine in one instance and one
and life sciences. The ability to fully capitalize on the language pair. These results undoubtedly have
technology — in conjunction with the use of AI-driven implications for the future of Machine Translation.
technology that automates workflows and translator
selection — will position companies to increase their Why Is a New Machine Translation Paradigm
content velocity, produce captivating multilingual Likely Underway?
content that is always on brand, grow their markets, Current MT engine trends give us a sense of déjà vu.
and thrive in what has become a brutally competitive During the end of the Statistical Machine Translation
digital market. era, which NMT replaced, there was virtually no change
in MT quality output. In addition, the quality output
What is the Future of Machine Translation? of different MT engines converged. These things are
2022 Machine Translation results made us question the happening now.
current Neural Machine Translation paradigm.
While NMT may not be replaced imminently if we believe
Is the NMT paradigm reaching a plateau? in exponential growth and accelerating returns theories,
Is a new paradigm shift needed, given the consider Rule-based MT’s 30-year run and Statistical MT’s
engines’ inability to make significant strides? decade-long prominence, and note that NMT is now in
What could be next? its sixth year, a new paradigm shift is near.

Lionbridge | 2023 Machine Translation Report Page 29 | LIONBRIDGE.COM

What Could Be the Next Machine Would Large Language Models Be a
Translation Paradigm? Good Alternative to a Neural Machine
Important advances to LLMs during 2022 have primed Translation Paradigm?
the technology for 2023. LLMs are generic models that To assess the promise of LLMs to replace the NMT
are trained to do many things. However, we saw some paradigm, we compared the translation quality of
dedicated — or fine-tuned — LLMs make essential ChatGPT and GPT-4 to the performance of the five
advancements in some specific areas by the end of major MT engines we use in our MT Quality Tracking.
2022. These developments position the technology to As shown in Figure 8, ChatGPT performed almost as
perform translations with some additional training. well as the specialized engines. And as shown in Figure
9 below, GPT-4 outperformed Yandex in the English-to-
For instance, take ChatGPT. OpenAI fine-tuned this Chinese language pair.
model to conduct question-and-answer dialogues
while still being able to do anything generic LLMs do. How Did We Assess the Quality of
The company made even more improvements with its ChatGPT vs. Generic MT Engines?
GPT-4 model. Expect to see more LLMs fine-tune for We calculated the quality level of the engines based
translation as time goes on. on the inverse edit distance using multiple references.
The edit distance measures the number of edits a
In What Way Would Large Language Models human must make to the MT output for the resulting
Need to Be Fine-Tuned To Handle Translation? translation to be as good as a human translation.
It would be more probable to use LLMs to execute For our calculation, we compared the raw MT output
translation if the machines are trained with a more against 10 different human translations — multiple
balanced language corpus. GPT-3’s training corpus references — instead of just one human translation.
was 93% English; only 7% constituted corpora from all The inverse edit distance means the higher the
the other languages. We guess that GPT-4 has more resulting number, the better the quality.
training data for more languages, which could explain
its improved quality output. A more language-balanced
corpus may be the basis for building a fine-tuned
model on top of LLMs specialized in translation.

A more language-balanced corpus

Another interesting aspect of this hypothetical new
may be the basis for building a
MT paradigm based on LLMs is the multimodality
fine-tuned model on top of LLMs
trend. We may train LLMs using linguistic and other
specialized in translation.
training data, such as images and video. This type
of training may provide additional world knowledge
for a better translation.

Lionbridge | 2023 Machine Translation Report Page 30 | LIONBRIDGE.COM

Multiple References Evaluation via Inverse Edit Distance
0.740

0.722
0.720
0.714
0.708 0.706
0.700 0.697

0.680

0.660 0.658

0.640

0.620

0.600
Bing NMT Yandex Google NMT DeepL Amazon ChatGPT

Figure 8. Comparison of automated translation quality between ChatGPT and the major Machine Translation engines based on the inverse edit distance using multiple references
for the English-to-Spanish language pair.

English-to-Chinese Translation Quality

0.600
0.553 0.552 0.550
0.525 0.521
0.505 0.488
0.500
0.459

0.400

0.300

0.200

0.100

0
Bing NMT Amazon Google NMT DeepL GPT-4 Yandex ChatGPT GPT-3

Figure 9. Comparison of automated translation quality between GPT models and the five major Neural MT engines based on the inverse edit distance using multiple references for
the English-to-Chinese language pair.

Lionbridge | 2023 Machine Translation Report Page 31 | LIONBRIDGE.COM

 The great thing about Large Language
“Generic” Models is that they can do many
different things and offer outstanding
quality in most of their tasks.

Why Are the LLM Translation communicate with humans in a conversational manner.
Results Noteworthy? Specialization adds accuracy to the performed tasks.
The results of our comparative analysis are remarkable
because the generic model has been trained to do many What Does the Future Hold for Large Language
different Natural Language Processing (NLP) tasks as Models in General?
opposed to the single NLP task of translation that MT The great thing about Large Language “Generic” Models
engines have been trained to do. And even though GPT is that they can do many different things and offer
has not been specifically trained to execute translations, outstanding quality in most of their tasks. For example,
its quality is exceptional. DeepMind’s GATO, another general intelligence model,
has been tested in more than 600 tasks, with State-of-
How Might Machine Translation Evolve as a the-Art (SOTA) results in 400 of them.
Result of Large Language Models?
Given the growth of LLMs — based on the Two development lines will continue to exist — generic
public’s attention and the significant investments models, such as GPT and GATO, and specialized models
tech companies are making in this technology — we for specific purposes based on those generic models.
may soon see whether MT will start adopting a The generic models are important for advancing
new LLM paradigm. Artificial Generic Intelligence (AGI) and possibly
advancing even more impressive developments
MT may use LLMs as a base but then fine-tune the in the longer term. Specialized models will have
technology specifically for Machine Translation. It would practical uses in the short run for specific areas.
be like what OpenAI and other LLM companies are
doing to improve their generic models for specific use One of the remarkable things about LLMs is that both
cases, such as making it possible for the machines to lines can progress and work in parallel.

Lionbridge | 2023 Machine Translation Report Page 32 | LIONBRIDGE.COM

What Are the Implications of a Paradigm Shift The Use of Machine Translation to Enhance
in Machine Translation? Customer Experiences
As the current Neural Machine Translation technology With improved MT quality and the need for more
paradigm reaches its limit and a new dominant personalized and tailored customer experiences,
Machine Translation technology paradigm emerges — companies will use MT more frequently to improve
likely based on LLMs — we anticipate some changes their global customers’ digital experiences and create
to the MT space. Most of the effects will benefit stronger relationships.
companies, though we expect additional challenges
for companies seeking to execute human translations.

Here’s what to expect:

Enhanced Quality
There will be a leap in Machine Translation quality as
technological advancements resolve longstanding
issues, such as language formality and other quality
issues pertaining to tone. LLMs may even solve
MT engines’ biggest problem: their lack of world
knowledge. This achievement may be made possible
through their multimodality training.

Technologists not only train modern LLMs with vast

amounts of text, but they also use images and video.
This type of training enables the LLMs to have a more
linked knowledge that helps the machines to interpret
the texts’ meaning.

Increased Content Output and a Reduced Supply

of Top-Notch Translators
Companies will be able to create more content faster,
and content creation will outpace the growth of the
pool of translators capable of translating this content.
Even with improved MT and increased translator
productivity, the translation community will continue
to struggle to satisfy translation demands.
LLMs may even solve MT
Increased Adoption of Machine Translation
engines’ biggest problem:
As the new technology paradigm becomes available
their lack of world knowledge.
and the quality of Machine Translation improves, the
demand for translation services will continue to grow,
increasing adoption in more situations and use cases.

Lionbridge | 2023 Machine Translation Report Page 33 | LIONBRIDGE.COM

What’s the Bottom Line?
Technology companies are demonstrating immense
interest in LLM technology. Microsoft invested $10B
in OpenAI. Nvidia, Google, and other companies are
also investing heavily in LLM and AI technology.
To fully understand GPT and localization and the real
potential of Large Language Models to generate value
for the language industry, we need to:

Conduct real-world tests at scale to evaluate

the error rate for each type of localization and
editing task
Analyze the detailed macro and micro user
journeys occurring within the localization value
chains and identify where they are likely to be
disrupted with this type of text automation
Understand how to prompt and provide relevant
context to GPT and other LLMs at scale, and
document pitfalls and best practices
Develop new automation and human-in-the-loop
editing workflows, inventing what post-editing
and QA will mean tomorrow with such an AI in
the loop
Design new automation and User Experience
(UX) interaction contexts for both localization
agents and customers for each possible
improvement opportunity
Ensure that the economics of licenses,
deployment costs, and maintenance makes
sense for our business

We are intrigued by what the future holds. We will

continue to evaluate LLMs, help you stay current on
this exciting evolution, and enable you to gain from it.

Technology companies are demonstrating

immense interest in LLM technology.

Lionbridge | 2023 Machine Translation Report Page 34 | LIONBRIDGE.COM

 LIONBRIDGE MACHINE TRANSLATION EXPERTS

Rafa Moral
Vice President, Innovation
Rafa oversees R&D activities related to language and translation, including Machine
Translation initiatives, Content Profiling and Analysis, Terminology Mining, and
Linguistic Quality Assurance and Control.

Yolanda Martin
MT Specialist
Yolanda is responsible for the creation of customized translation models, as well as
quality analysis and the development of strategies to fine-tune them. In parallel, she
collaborates with the R&D department to develop new linguistic tools and resources.

Thomas McCarthy
MT Business Analyst
Thomas ensures Lionbridge customers and stakeholders obtain maximum benefits
from MT-related technologies, services, and consultancy.

Lionbridge | 2023 Machine Translation Report Page 35 | LIONBRIDGE.COM

 Lionbridge experts make it easy for customers to implement Machine
Translation effectively and achieve business gains. Lionbridge is closely
evaluating emerging technologies to enable customers to benefit further as
the Machine Translation space undergoes rapid change.

To learn more about how Lionbridge can help you fully capitalize on
automated translations, contact our team today.

ABOUT LIONBRIDGE
Lionbridge partners with brands to break barriers and build bridges all over the
world. For over 25 years, we have helped companies connect with their global
customers and employees by delivering translation and localization solutions in
350+ languages. Through our world-class platform, we orchestrate a network
of passionate experts across the globe who partner with brands to create
culturally rich experiences. Relentless in our love of linguistics, we use the best
of human and machine intelligence to forge understanding that resonates with
our customers’ clients. Based in Waltham, Massachusetts, Lionbridge maintains
solution centers in 24 countries.

LEARN MORE AT

LIONBRIDGE.COM

Lionbridge | 2023 Machine Translation Report © 2023 Lionbridge. All Rights Reserved. Page 36 | LIONBRIDGE.COM