Annotation Guidelines for Enablement of Reels

Translation and Auto Dubbing

Table of Contents
Introduction ............................................................................................................................ 1
Project Overview .................................................................................................................... 1
Annotation & Transcription Correction Guidelines ...................................................................... 2
A. Terminology ................................................................................................................ 2
B. Requirements ............................................................................................................. 2
C. General Guidelines ..................................................................................................... 2
D. Rejection Reasons ..................................................................................................... 2
E. Detailed Guidelines .................................................................................................... 3
User Interface: ........................................................................................................................ 1

Introduction
These project guidelines contain comprehensive information about transcription and annotation
for project Anagram. Annotators are requested to read all the topics in detail before working on
this project.

Project Overview
In this project, annotators will be required to:
- Correct transcribed source text
- An audio file will contain an excerpt of speech. A transcription will
accompany this excerpt. Annotators will be required to review and correct
the transcription so that it matches the excerpt of speech, with exceptions
for technical issues detailed later in the guidelines.
- Add Audio labels
- Answer a Yes/No question regarding the quality and nature of the audio
clip
- Check a box if there are background noises or if speech is cut off in the
audio clip
Annotation & Transcription Correction Guidelines

A. Terminology
a. Source audio file we are interested in annotating.
b. Labels the characteristics of the source.
c. Transcribe the act of assigning words to speech audio producing text.
d. Post refers to the unit consisting of source audio files and source transcribed
text.
e. List of languages refers to the languages we are interested in annotating, this
list is comprised of: English, Spanish, French, German, Italian, Mandarin.
B. Requirements
a. Annotators need to be at a minimum native in the assigned non-English
languages, aware of the cultures of such languages, and understand the
nuances of social media language.
b. Annotators should be able to correct incorrect transcriptions and assign labels to
the audio files provided.
C. General Guidelines
a. Listen to the audio carefully: note the words that you hear and any background
noise if present.
i. If background noise was present, select the relevant labels.
b. Fix a source transcription if it deviates from the audio, see the detailed
guidelines for transcription considerations
i. If no corrections are necessary, ensure that the exact provided text is
submitted. Do NOT leave a blank text box, a substitution character (such
as -), or a message stating that no correction is needed.

D. Rejection Reasons
Reject a job for the following reasons:

1. Source has more than one intelligible speaker: More than 50% of the audio contains
two or more speakers speaking at the same time. This includes musical lyrics being
sung over someone speaking.
2. Source contains more than 10% in a different language: Audio is in a language other
than that of the queue you are transcribing or the user’s accent is such that you are
unable to understand the utterance (this includes “child speak”)
3. Source has more than 75% unintelligible speech
4. Source has no speech to transcribe: Ensure the entire audio clip has no speech
before rejecting.
E. Detailed Guidelines
1. Instructions
a. Listen to the audio carefully and note the words that you hear.

b. Edit transcript. If the speech and provided transcript do not match, please
edit the transcript so that they do match. For unintelligible words on jobs that
don’t fit the rejection criteria, please type [x] in its place. If audio is in a
different language, but does not meet the rejection criteria, type [c] in place of
the different language. See guidelines for further details.

c. Background noise. If you noticed any background noises while listening,

check the appropriate boxes. No need to re-listen for this step.

d. [Rejection Reasons] In the case there are multiple intelligible speakers,

more than 10% speech that does not match the identified language, the
speech is unintelligible, or the audio contains no speech, please REJECT the
job and select the appropriate rejection reason. You can skip step #2 if
rejecting the job.
Rejection Reasons:

i. [multiple_speakers] Source has more than one intelligible speaker.

ii. [foreign] Source contains more than 10% in a different language.

iii. [unintelligible] Source has more than 75% unintelligible speech.

iv. [no_speech] Source has no speech to translate.

v. [other] Other - Provide a reason for rejecting.

2. Label Assignment Considerations

a. Original audio is only in the expected source language, and it is NOT a mix of
multiple languages.
Original audio has high quality clean speech, i.e. NO background music/
strong noise/ NO distorted or broken speech, etc.
Original audio has NO extremely offensive profanity.
Original audio does NOT have a strong accent and is easy to understand.
 Original audio has low gibberish/ incoherent/ content, and has less than 25%
unintelligible content.
● Only mark yes if meets all the criteria above
b. Background Noise Labels:
● Only mark labels that were present in the audio
c. Audio speech cut off:
● Only mark if the provided audio contains speech that has been cut off
1. Case 1: Audio has speech cut off AND it is not reflected in the
provided transcript. For example, an audio clip starts midway
through the speaker saying something and it is not captured in the
provided transcript. Another example, audio clip ends midway
through speaker saying something and it is not in the provided
transcript
2. Case 2: Audio has speech cut off AND has a reflected word in the
provided transcript. For example, audio clip starts midway through
speaker saying “alphabetical” such that only “abetical” is heard
and the provided transcript has “alphabetical”.

3. Transcription Considerations

1.Speech and sounds

Transcribe all speech you hear in the audio:

● Include transcription of:

a. hesitations ("um", "er", "hmm", etc)
b. informal words ("gonna", "wanna", "they’ll", "I’ve", etc)
c. repeated words ("they they was gonna be there")
● Transcribe unintelligible speech by writing [x] for the following reasons,
in general if an audio has a portion you cannot transcribe you should
mark as [x]:
a. The audio is unintelligible for a portion of the audio (muttering,
too quiet, you don’t know what the speaker is saying, too much
background noise etc), if the majority/entire audio is
unintelligible/lacks speech follow guidelines in h.
b. For multiple sounds in sequence that are unintelligible only
write [x] once
c. If audio is in a different language, but does not meet the
rejection criteria in h, mark the speech in a different language
with [c]. If there multiple different language words in sequence,
write [c] only once
 d. The audio temporarily cuts off and you cannot discern what is
said. For example, the audio clip starts midway through what
the speaker is saying something and you cannot
understand/discern it, you would transcribe as [x], if you can
tell what is being said please provide the transcription instead.
This is encouraged but not required so no need to spend extra
time trying to understand what is being said. When in doubt just
mark as [x])
e. Be sure to double check that the square brackets are properly
closed off [x].
● Exclude transcription of:
a. laughing, crying, sighs, music, dogs barking, or other noises
(thuds, bangs, closing doors, footsteps, crackles in the audio,
etc) heard in the clip.
b. Stutters: For example if speech audio for the word “call” is
stuttered (ex: “ca-call”) transcribe as the word only and do not
transcribe the stutter (“ca-call” → “call”).
c. Do not expand or replace information, do not add any
explanatory/parenthetical information, definitions, etc.
d. Do not transcribe any echo heard in the audio.

● Do not ignore any meaningful information that was present in the source.
● If unsure of how to spell a word, check Merriam-Webster dictionary.

Note: some words are spelled/spaced differently according to their

usage, some examples are but not limited to:

f. pickup (noun) and pick up (verb)

g. setup (noun) and set up (verb)
h. shutdown (noun) and shut down (verb)
i. standby (noun) and stand by (verb)

2. Numbers and Symbols

● Spell out all words and numbers exactly as they sound in the audio file
a. Do NOT use any symbols (+ - : $ & @ §# etc.) to represent a
spoken word.
b. Time: should be transcribed to the equivalent form in the
source language.
● URLs and email addresses
a. In the case the audio contains a URL or email address, do not
separate elements as they are spoken and provide their symbol
equivalent in the language spoken. For example, the
 transcription for the speech “www dot facebook dot com” would be
transcribed as “www.facebook.com”; another example, for the
speech “<email username> at meta dot com” would be transcribed
as “<email username>@meta.com”

3.Punctuation and Capitalization

● Punctuation and capitalization rules are not standard across languages,
apply best practices for punctuation and capitalization for your given
source language.
● In the case the source transcript provided for the audio has punctuation
that does not match the source audio, correct the punctuation to match
source audio.
User Interface:
a. Instructions (in blue) alongside audio and reference completed transcript of the audio (to the right).
b. Transcription Correction Input Box (first section after blue instruction box):
c. Label Assignment Questions (Section that follows the transcription correction input box):