Introduction 3

Task Description 3

Judgment Criteria 3

Golden Thumb Rule 4

Policies 5
Judgments 5
Defects 6

Query Research & Examples 6

1. SQ: intuos s 6
2. SQ: tpost driver 6
3. SQ: b000jqm1de 7

Product Research 7
SQ: bulb dimmable 7
SQ: 2.5 inch hard drive 8

Using Categories 9
How to Approach a Query Based on Category 10
Examples 11
1. SQ: acorn 11
2. SQ: reddit 11

Judgment Examples 12
Exact 12
Tips 12
Examples 12
Substitute 13
Tips 13
Examples 14
Complement 15
Examples 15
Accessories 16
Irrelevant 17
Tips 17
Examples 17
Unjudgeable 19
Examples 19

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

1
Category Specific Examples 20
Electronics 20
Media and Games 21
Fashion 23
Luggage & Bags 25
DIY & Tools 26
Beauty 27
Medicines 28
Grocery 29
Fresh 32
Pets 34
Toys 34
Home & Garden 35
Kitchen 35
Automobiles 36
Shoes 36
Theme 36
Adult 37

Books Queries 38

Special Mentions 41
Products not sold by Amazon 41
Queries with Multiple Meanings 42
Query-Alias Conflict 43
Truncated Queries 43
Misspelled Queries 43
Franchise/ Brand Queries 43
Quantity/ Quality/ Price/ Shipping/ Condition 44
Widget 44
Name 44
Deals Related Queries 45

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

2
Query & ESCI Annotation Guidelines Updated May 2022

Introduction
Our core goal is to represent customers' interests by interpreting the meanings of the search queries and
judging the results from the perspective of a reasonable customer. When a customer in a particular
marketplace sends a query to a shopping search engine, he or she usually has a set of targeted products
in mind. Our primary task is to determine what products are targeted by the queries they encounter. The
overall aim is to improve the quality of search results.

Task Description
Each judging page contains a search query (SQ), search alias (sometimes referred to as category),
marketplace, a list of research links in case you are not familiar with the query, and one or more products
(ASIN) to be judged for relevance to that query.

1. amazon.com.au: aps 🡪 This link represents the marketplace in which the query has been
searched. In this case it is the Australian Market (AU).
2. aps 🡪 Represents the store under which the query has been searched.
3. Google site: Amazon 🡪 This link represents the marketplace wise amazon search result page. In
the above example it is specific to the Australian marketplace (AU).
4. Google 🡪 This link simply googles the query that the customer is looking for.

Judgment Criteria
Before making judgments, reviewers need to understand the query to anticipate what kind of product
the customer is looking for, and what kinds of products could be a Complement or Substitute. In many

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

3
cases this will be very clear, but if the wording is unfamiliar, then the links to the right of the query in the
live working environment can help in researching the intent of the query. If some part of the query is
unclear even after research then the products may still be judgeable based on the parts of the query that
are clear. The title of each product is clickable and the Product Description Page (PDP) should be checked
to find out if the product has the features requested in the query.

N.B. The relevance of the product cannot be determined just on the basis of impression. The PDP must
be checked. The PDP has various factors influencing the reviewer’s decision and henceforth the ranking
of the products like size, color, design or model variations.

The Judgments are as follows:

1. Exact
2. Substitute
3. Complement
4. Irrelevant
5. Unjudgable

Golden Thumb Rule

Reviewers should follow the Golden thumb rule shown below to make accurate judgements.

1. Firstly, research the SQ using Amazon or Google.

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

4
 2. Consider the category. If a category/alias/store appears in the link immediately to the right of the
query, this means the customer selected that category before entering the query. This
information can impact the user intent and even clarify a broad query.
3. Understand the ASIN. This is achieved by reviewing the PDP (titles are clickable in the live
working environment) or using Amazon or Google for further research.
4. Lastly, the judgment is submitted by the reviewer on the basis of the SQ-ASIN relevancy by
selecting the appropriate judgment option.

Policies

Judgments
Exact – This product fully satisfies the query. A broad or ambiguous query could be satisfied by any of a
diverse set of products, so long as it is reasonable to expect that a customer who entered that query
might buy that product. A very specific query will only be satisfied if the product matches all of the
characteristics included in the query.

Substitute – This product differs from the one intended, but it is functionally equivalent. This will be the
same type of product, but may differ by brand, model, color, title, or other detail. If the requested
product is included in a bundle with other associated products, it is judged as Substitute, so long as it is
reasonable that the customer may want to buy both together.

Complement – This product is not the one intended, but it could be useful to the customer if they did
buy the intended product. This could be an accessory, a component, replacement parts, or even another
item designed to be used with the product.

Irrelevant – This product does not satisfy a reasonable understanding of the customer’s intent. It could
be entirely unrelated or it could differ by one crucial feature that would make it unsuitable for the
customer.

Unjudgeable – This judgment is only to be used if the product cannot be judged, either because the
query cannot be understood, or because the product is lacking enough information to determine if it
matches the query. An Unjudgeable query is one that is complete gibberish, is seeking help, is
attempting to navigate to a website, is in a non-market language, or is a string of numbers that does not
correspond to an ISBN or model number in Google searches.

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

5
Defects
Offered products can differ from the search query or the intended product for various reasons, which we
refer to as defects. ASINs with defects are not marked as an Exact match. Depending upon the defect
present, the judgment could range from Substitute to Irrelevant. Please consult the defect type guide
for a list of defects, their associated judgments, and examples.

Query Research & Examples

1. SQ: intuos s

The query here is “intuos s”. It is important that you pay attention to all elements of the query,
including words like ‘the’ and any stray letters. For this query, the customer entered the letter “s” to
indicate the size of the product, as you would see among the product titles if you were to click the
Googlesite:Amazon.com link in the task. Please note the category is listed as aps meaning
all-product-search. The following table shows the corresponding judgment for the offered products
(OPs):

Remember, s = Small size

Offered Product (OP) Judgment

Small size Wacom Intuos Art tablet Exact

Large size and/or competing brand Substitute

Accessories: case, stylus, etc. Complement

2. SQ: tpost driver

If the results themselves do not reflect the terminology used in the query, some research may be
required. In this case, research reveals “T post driver” and “post driver” are among many
interchangeable names for the tools that pound metal posts into the ground. The following table shows
the corresponding judgment to offered products:

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

6
 OP Judgment

T post driver Exact

Post driver Exact

Fence post driver Exact

As the search was broad, Exact is the judgment for any design of post driver, whether powered or
manual.

3. SQ: b000jqm1de

A Google search of Amazon shows that this string of characters is a model of a Brother sewing
Machine. The following table shows the corresponding judgment to offered products:

OP Judgment

Correct model of sewing machine Exact

Sewing machine accessory Complement

Different brand or model of sewing machine Substitute

Product Research
Insufficient information should not be used as an excuse for insufficient research. Rather thorough
research of the Product Description Page (PDP) and external sources should be done, as the answers can
usually be found for an ASIN by looking at reviews and questions or using Google Search. See the
examples below for better understanding.

SQ: bulb dimmable

OP Judgment

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

7
 TORCHSTAR 5W A15 LED Light Bulbs Irrelevant

PDP mentions ‘Dimmable: No’ under Specifications and it won’t work in dimming fixtures. Information
also present in Q&A section, see details below.

Q: Are these lights dimmable?

A: According to the product information, this A15 LED bulb is not dimmable. You can't use any dimmer
with it or install it on any dimmable light fixture.

By TORCHSTAR customer support on April 23, 2018 - If the ASIN does not mention a
feature/style/characteristic, then a research has to be according to the Golden thumb rule. The given
ASIN, for instance, is a A15 bulb, which when searched on Google for the feature shows that it is not
dimmable (By TORCHSTAR customer support on April 23, 2018). A one-time research also shows that
dimmable and non-dimmable bulb holders/fixtures have different compatibility. Hence, the decision is
irrelevant. Had the holders been the same, then the ASIN will be substitute as then, it would only have a
feature defect.

SQ: 2.5 inch hard drive

When you are uncertain if a product matches the query or not, you can use Google Search to understand
the features listed on the detail page. This may help you determine the most suitable judgment.

OP Judgment

2.5-Inch Internal Bare Drive Substitute

This is a Solid State Hybrid Drive, which is like a hard drive (HDD) but much faster, according to the detail
page:

Google can help you understand the features on the detail page:

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

8
In this case, the customer seeks a hard disk drive (HDD) with a 2.5 inch spinning magnetic platter, so a
SSHD would be a Substitute, and a SSD would be Irrelevant, since the size requested in the query does
not apply to a drive without a spinning magnetic platter.

Finally, please note that Amazon products can have lengthy titles/descriptions. We recommend reading
carefully to make sure you judge accurately.

Using Categories

If a category/alias appears in the link immediately to the right of the query, this means the customer
selected that category before entering the query. This information can help clarify a broad query.

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

9
How to Approach a Query Based on Category
Product Type mentioned in SQ

Category: aps Category: Specified (other than aps)

When the SQ clearly mentions the product type When the SQ mentions a product type and a
and the selected category is aps, then we only specific category is selected, they should be
consider the searched product as Exact. For considered together. This can give you a better
example, if the SQ is basketball (aps) and Amazon understanding of the specific needs of the
sells basketballs, then only basketballs should be customer. For example, if the SQ is basketball
judged Exact. (electronics) the likely intent of the customer is
different. One can find phone covers, pen drives,
etc. in the shape of a basketball. In this case, such
products would be marked Exact and not
basketballs.

Product Type not mentioned in SQ

Category: aps Category: Specified (other than aps)

When the SQ has not specified a product type As stated, when the SQ has not specified a
and the category is aps, we can consider the SQ product type it can be considered a theme. When
as a theme. For example, if the SQ is marvel (aps), a category other than aps is selected, it can give
and we know that Amazon sells a lot of products us a greater understanding of the query intent. In
related to Marvel, we will mark all products as these cases, we can consider the category as a
Exact such as T-shirts, books, DVDs, figurines, etc. product type. For example, if the SQ is marvel
(toys) we will only mark marvel toys, figurines,
etc. as Exact. Other product types like books and
movies will be Irrelevant.

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

10
*aps = all-product-search

Examples

1. SQ: acorn

The category for the query listed above is fashion-womens-shoes. Clicking on the first link amazon.com:
fashion-womens-shoes will show the products within that category. Given its inclusion, the user is most
likely looking for shoes by the women’s clothing brand Acorn. The following table shows example items
and their corresponding judgment:

OP Judgment Explanation

Footwear from the correct Exact This product satisfies the query.
brand

Footwear from another brand Substitute This product differs from the one intended, but it is
functionally equivalent.

2. SQ: reddit

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

11
The only Exact judgment would be a Reddit app by Reddit. Other apps such as “Reddit Is Fun” or
“ReddTV” that are related but not from “Reddit, Inc” would be judged Substitute. Since the category is
mobile-apps, any product which is not an app or is unrelated to Reddit is to be judged Irrelevant.

Judgment Examples

Exact
This product fully satisfies the query. A broad or ambiguous query could be satisfied by a diverse set of
products, so long as it is reasonable to expect that a customer who entered that query might buy that
product. A very specific query will only be satisfied if the product matches all of the characteristics
included in the query.

Tips
● Availability in the widget/drop-down makes the product exact, despite the change in ASIN.
● Universal products are compatible with all model types. For example, SQ: iphone 10 case; OP:
universal phone case; Rating will be Exact.
● Multiple interpretation SQs: a query can have multiple meanings and an ASIN that is Exact for
any of the meanings should be judged Exact. For many broad queries, the customer’s intent may
be unclear. Multiple interpretations of the query could result in different judgments for an ASIN.
For example, SQ: battery scooter; OP1: scooter with battery, OP2: battery for scooter

Examples
SQ Category OP Judgment Reason

adidas t-shirt aps Adidas Short Exact In this query we have all the basic
green short Sleeve Tee criteria such as brand (Adidas), type of
sleeve clothing (T-shirt), color (green) and type

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

12
 of sleeve (short sleeves) corresponding
100% with the criteria in the query. This
is an Exact product.

asus laptop aps ASUS Laptop Exact This is a single example of an Exact
product, however, multiple products will
be Exact for this query because it is
general and does not have any specific
features included like screen size,
processor, RAM, etc.

Fujifilm electronics Instant Films Exact This is designed to be used with the
instax mini 9 Sticker for camera and the query specifies the
accessories Fujifilm customer is looking for accessories. If
Instax Mini 9 the query had just been for the camera,
this would have been Complement.

Fujifilm electronics Camera Case Exact This is designed to be used with the
instax mini 9 Cover for camera and the query specifies the
accessories Fujifilm customer is looking for accessories. If
Instax Mini 9 the query had just been for the camera,
this would have been Complement.

Cricket Stripbooks Cricket Exact Due to the selected category, the OP is

Explained an Exact match. Under the aps category,
the ASIN would be complementary to
the SQ.

Substitute
This product differs from the one intended, but it is functionally equivalent. This will be the same type of
product, but may differ by brand, model, color, title, or other detail. If the requested product is included
in a bundle with other associated products, it is judged as Substitute, so long as it is reasonable that the
customer may want to buy both together. When a brand is specified in the query, then products that are
a different brand will be Substitute, but unrelated products by the same brand will be irrelevant.

Tips
● +/- 1 size and age rule: +/-1 difference in size and age from asked SQ is
replaceable/substitutable, otherwise Irrelevant.
● If the query asks for universal compatibility, but the ASIN offered is for a specific model then, we
will judge it Substitute. E.g., universal phone case; and ASIN: iphone10 cases; Rating will be
Substitute and Defect will be Feature Wrong.

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

13
 ● Retailer definition is prioritized over customer comments. In cases where the retailer
information is incorrect, we could rely on customer reviews. For example, when the title might
have too many keywords [pullover, sweat-shirt, sweater- all in one] or when there is a mis-match
between the title and description/image [sweat-shirt being described as pullover], we could seek
clarity from customer reviews at the bottom of the PDP. In other words, a holistic review of the
product and detail page is necessary and recommended. While product title and description
provided via retailers usually hold priority/preference over customer comments (chiefly when
description provides specific details to sizes, dimensions, compatibility, etc.), it is also imperative
to gather a comprehensive knowledge of the OP (especially when a retailer may have
inaccurately described a product).

Examples
SQ Category OP Judgment Reason

Panasonic aps Braun Shaver Substitute The OP is the same product type, but
Shaver (B07Z6HWZX4) from a different brand.

Disney Home & Powerpuff Substitute The OP is a different theme.

Princess Kitchen Girls Bedsheet
Bedsheet (B08XQ7K7YF)

iPhone X aps iPhone XR Substitute The OP is a different model.

(B07P978C2R)

Scanner and aps Scanner Substitute As the OP is the main product from the
Scanner SQ it can't be complementary to the
cover accessory.

Shirt XL size aps Shirt XXL size Substitute Different Size (+1/-1)

Pen aps Pen and pencil Substitute The OP is a bundle.

set
(B07KCB8TCG)

Universal AC aps LG Ks-q18enza Substitute Feature Wrong

cover cover
(B0859Y75DQ)

When the query includes details or features, then products that lack those features can be judged
Substitute only if they will still serve the same purpose. If they cannot be used in the same way, they are
Irrelevant.

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

14
 SQ Category OP Judgment Reason

baby tub aps Disney Minnie Substitute This is a different design of baby tub that
seat with Mouse does not use suction cups, but it still is
suction cups Inflatable non-slip, because it uses an air-filled tube
Safety Bathtub to fit tightly into a bathtub.

Complement
This product is not the one intended, but it could be useful to the customer if they did buy the intended
product. This could be an accessory, a component, replacement parts, or even another item designed to
be used with the product.

Examples
SQ Category OP Judgment Reason

adidas shoe aps shoelaces Complement Shoelaces are an accessory to the

customer’s original query. When a
product is commonly used with
another product, then they would be
judged Complement to each other.

samsung aps headphones Complement Headphones are not a perfect match,

galaxy note 9 but they are an accessory.
phone

small trash Home & 1.2 Gallon Extra Complement This is designed to be used with a small
can bedroom Garden Strong Clear trash can.
Small Trash Bag

decorative Home & Set of 2 Feather Complement This product can be used to make a
throw Garden Throw Pillow throw pillow (i.e, the
pillows Insert insert goes inside the cover).

decorative Home & Linen Burlap Complement The PDP states the listing is for pillow
throw Garden Square Throw cases only and the inserts are not
pillows Pillow Covers included. This product can be used to
make a throw pillow.

Samsung S20 Electroni Samsung S20 Complement Accessory

cs Case
(B082ZYLC63)

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

15
 Mazda CX-5 Automob Mazda CX-5 Complement Accessory
iles Remote control
transmitter
(B00KWEOCXU)

Pencil aps Eraser Complement Items used together

(B00AZEK8TY)

Circus aps Face Jewel Complement It’s a component of a costume.

costume Rhinestone
Stickers
(B07N1HD3FL)

Magma aps Autoglym Polar Complement This is a car washing product, but the
autoglym Blast Magma product is specifically for
(B079QNQL1R) treating rust or iron spots, so this is
complementary.

Printer and aps Scanner Complement As both products hold the same
scanner weightage (i.e., each can be a main
product), it should be judged
complement.

Scanner and aps Scanner cover Complement The cover is an accessory to the scanner
scanner and is a component of the SQ.
cover

Accessories
The product name of most accessories usually includes the name of the product they’re designed for,
and the image may include the accessory ON the product, even if this is only being sold as an accessory.

SQ Category OP Judgment Reason

happy planner aps Me & my BIG Complement Happy Planner is a brand of notebooks
ideas Punch and calendars. This is a branded
accessory for adding paper to the
notebooks, so it is judged
Complement.

fitbit charge 2 aps POY Complement This is a band that is compatible with
Replacement the Fitbit, as it says on the PDP.
Bands

yellow buffalo aps HomeCrate Complement This would match a runner, and the

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

16
 check runner Classic Buffalo customer may want to buy both.
Check
Napkins

When the query is an accessory, a different accessory is Substitute if it could be used for the same
purpose instead of the queried accessory, Complement if the customer may buy them both to meet
their need, and Irrelevant if the accessory is unrelated to the purpose of the queried one.

SQ Category OP Judgment Reason

2010 aps Mini DisplayPort to HDMI Irrelevant There is no reason to expect

macbook VGA DVI, CableCreation 3 in the customer needs an adapter
pro graphic 1 Gold Plated Mini DP when they’re improving their
card (Thunderbolt) Adapter graphics processing speed.

Irrelevant
This product does not satisfy a reasonable understanding of the customer’s intent. It could be entirely
unrelated or it could differ by one crucial feature that would make it unsuitable for the customer.

Tips
● Any absence of a clear mention of model/sub-model or year is Irrelevant.
● Complement of a substitute product is Irrelevant.
● Substitute of a complementary product is Irrelevant.
Example - SQ: Neem
OP1: Neem Seeds – Exact
OP2: Books about Neem – Compliment
OP3: Books about banyan tree (Substitute of a Compliment) – Irrelevant

Examples
SQ Category OP Judgment Reason

samsung s8 aps S10 case Irrelevant This product is meant for a different
case (B07M9ZYYB6) phone model. This makes the product
unusable for the customer and they
would not buy it as a Substitute or a
Complement to the original query.

ipod nano 7th aps Evach Active Irrelevant This product is not compatible with an

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

17
 generation Stylus Digital Pen iPod nano 7th generation and the
customer would not buy it as a
Substitute or a Complement to the SQ.

micrometer industrial Digital Caliper Irrelevant Calipers are a different kind of

measuring tool, and operate at a
different scale: micrometers measure
distances one hundred times smaller.
When the query is specific,
substitutability has to be at the same
scale of operation.

usbc 3 aps Thunderbolt 3 to Irrelevant Adaptors by their nature are converting

thunderbolt 2 Mini DisplayPort from one thing to another, so the
Converter query will usually include the two
different formats being switched
between. Query research on Google
will show that for adaptors,
Thunderbolt 3 is the same as USB-C,
and Thunderbolt 2 is the same shape
of plug as mini DP, but will not work for
Thunderbolt 2 devices. Another
product is only substitutable if it will
connect to both of the formats
requested in the query, such as this
Multi-port hub. Adaptors that use mini
DP, HDMI and DVI formats are all
Irrelevant for this query.

Dolittle Movie & The Past Irrelevant Irrelevant genre

(Adventure/ TV (Horror)
Comedy) (B07W1XDVCM)

Women Shoes aps Men Shoes Irrelevant Gender wrong

(B08CZ7FLKT)

Xiaomi Watch aps Apple watch Irrelevant Compatibility issue

Band
(B07TC5T515)

Cryptocurrency aps T-shirt with Irrelevant Product type defect

cryptocurrency
symbol

Pastel Paper aps Art Supply Set Irrelevant A very big bundle of products (It
Pad (B08MWSWKTN) includes 4 Pastel Pencils in Different

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

18
 Shades, 6 Pastel Sticks in Different
Colors, 3 Paper Blending Stumps, 1
Plastic Sharpener, 1 Two-Hole
Sharpener, 1 Sandpaper Block, 1 Roll of
Masking Tape, 1 Craft Snap-Off Blade
Knife, 2 Binder Clips, Plus four different
styles of full-size premium sketch and
drawing pads)

Unjudgeable
This judgment is only to be used if the product cannot be judged, either because the query cannot be
understood, or because the product is lacking enough information to determine if it matches the query.
An Unjudgeable query is one that is
● complete gibberish,
● seeking help,
● attempting to navigate to a website,
● in a non-market language (*English is not considered a foreign language),
● a string of numbers/symbols that does not correspond to an ISBN or model number in Google
searches.

Examples
SQ Category OP Judgment Reason

823^%0* aps Socks Unjudgeable Gibberish

(B0895LL44V)

Flipkart.com aps Pen Unjudgeable Navigational query

(B082PN4X5J)

博朗 aps Chocolate Unjudgeable Foreign Language

(B08J37H3QK) (non-market)

How to cancel my aps Women’s black Unjudgeable Service-related SQ

prime Top
membership? (B07SN3KMS6)

Volume of aps The Ghost and Unjudgeable Non-product type SQ

hemisphere with the Hemispheres
radius 15cm? Kindle edition
(B08KNSWQ5G)

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

19
 Gift for mother aps Notebook Unjudgeable ASIN detail page in another
(searched in (B08TYJYD71) non-market language
English locale)

A page like the below is also considered unjudgeable:

Category Specific Examples

In this section we have categorized search queries by type and provided some example products to
illustrate how different queries need to be judged. Category in this case should not be confused with
store/category/alias. In this case, the store/category/alias is aps unless otherwise mentioned.

1. Electronics
Subcategory SQ Exact Substitute Complement Irrelevant

Cellphone Samsung note Samsung note iPhone XR Note 10+ cover & Samsung note 10
10+ 10+ phone protective glass lite Cover

Computers & Motherboard Motherboard - Processor PC screen

Peripherals
Asus Zen book Asus Zen book HP Elite book laptop cover 13-15 laptop charger for
14 inches Lenovo

Headphones Gaming HyperX gaming Apple earpods headphone keyboard

headphones headphones & case charging support

TV TV support 55 TV support Console table Screws TV

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

20
 inches Samsung 45-75 with universal
inches TV stand

Amazon Echo dot Echo dot Echo dot & Echo dot cover JBL speaker
Devices smart bulb

Camera Canon EOS Canon EOS Nikon cool pix SDXC memory card Canon printing
2000D 2000D & lens & 16 GB cartridge
tripod

Watches Men's intelligent Fitbit versa 3 Men's analog Screen protector Boys’ Fitbit
watch (unisex) watch for Fitbit versa 3

Headphones: Earbuds/Headphones/Earphones all are Exact (i.e., headphones can be considered an

umbrella term that encompasses these terms).

2. Media and Games

Media primarily includes films and music. Under films, we have formats like DVD, Blu-Ray and Video
cassette. Under songs we have CD, Vinyl, cassette, and streaming. Genres in movies are the primary
judgability criterion. If genres don't match, then movies are irrelevant

Games here mainly refer to the video games including all PS models, Nintendo, all X boxes and Windows
games. The category also include game accessories like controllers, consoles, game codes etc. Model
compatibility is paramount for video games.

Substitutability in Media 🡪 When the query is a topic or media title such as music, or movie, then other
media that are in the same general category, the judgment is Substitute. If a media product can be
considered to be in multiple categories, then either could be Substitute. Also if any of the same
contributors to one media product are also in another media product, then they are Substitute.

Complements in Media 🡪 Media products that are about other media products are considered to be
secondary material and are judged Complement so long as they match the category included in the
query. This includes reviews, analyses, study guides, anthologies, or commentary, where the video is
meant as a companion to the central product they are about.

Subcategory SQ Exact Substitute Complement Irrelevant

Infinity war Infinity war 1. Spiderman - Titanic (Romance,

Movies/
(Action) 2. Infinity war Genre different)
Prime Video
themed products

Video Games COD ps4 COD PS4 game PS4 GTA game 1. Games CD COD XBOX ONE

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

21
 and all COD stand,
themed product 2. Guide on COD
for PS4 ps4

ps5 charging Charging station - Cables for Charging station for

station for PS5 console charging station PS5 Controllers
for console

Xbox XBOX consoles PS4 Console Cases, Cases, Accessories

(only consoles are Accessories, for Xbox Controllers
the main product Games for Xbox (Accessories to
type) Console Accessories)

Endgame Endgame Blu-ray Endgame prime Blu-ray holder Marvel kindle

Blu-ray video/Endgame version/vice-versa
DVD format DVDs, CDs, SACDs
and DVD audio
discs

One Direction One Direction 1. Linkin park Spiderman movie

music and brand music
products 2. One Direction
themed products

Justin Bieber Justin Bieber - Harry Styles - Fine Vinyl cleaning 1. Purpose CD
Vinyl Purpose Vinyl Line Vinyl kit

2. Vinyl player
Music, Vinyl,
Audio

Walls CD Walls CD Reputation CD - 1. Walls Vinyl

2. CD player

Snapchat Snapchat Instagram 1. Guide to use 1. Temple Run app

snapchat 2. General Gift cards
Apps, 2. Gift card to
Software buy Snapchat

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

22
 Windows 10 Windows 10 Windows Server Windows 10 Laptop with inbuilt
Home 2019 guide Windows 10
*DVDs can be substitutable for Blu-ray but not vice versa due to compatibility issues.

3. Fashion
Subcategory SQ Exact Substitute Complement Irrelevant

Pink Cushion Pink Cushion Blue Cushion Any decorative Cushion cover of
cover 50 x 501 covers, size 50 x covers, size 50 x to decorate the different size 30 x
50 50 cushion covers, 50
Bedding
straps, zips, etc.
for cushion
covers

Batman T-shirt Batman T-shirt Superman T-shirt Batman pants2 Different genre
themed T-shirt
Titanic themed
T-shirt

Cotton T-shirt T-shirt made with 1. Polyester Cotton Woolen pants

>=50% cotton T-shirt shorts/pants
2. T-shirt made of
other material or
with less than
50% cotton

Clothing
Cotton (fabric) Cotton fabric Leather, - Cotton T-shirt
(pure or >=50%) Polyester, Silk
(Fabric)

Shirt Any shirt3 1. Tunic 1. Pants Sweatshirt/

2. Blouse 2. Jeans Pullover
3. Tank top

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

23
 Top (anything 1. Tunic - Pants Any inner wears
from neck to 2. Jacket
waistline) 3. Coat

Dress A one-piece 1. Jumpsuit Buttons/ Saree, Salwar

garment from top 2. 2 piece dress Accessories for a
till ankles. Top till dress
knees

Salwar suit/ Ready made Semi-stitched Unstitched Dress

Lehenga choli Lehenga Lehenga/ salwar materials for
suit lehenga/ salwar
suit

Ready to wear
Salwar suits

Pajama/ PJs/ Any pant/set of - Pajama Top Any normal

Pyjama clothes, which can pants/shirts not
be worn while used as sleep wear
sleeping or including
lounging sportswear like
sweat pants and
tracksuits.

Bikini 2-piece bikini One-piece bikini Bikini Cover ups -

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

24
 Necklace Any necklace Necklace set with Pendant Anything other than
bracelet and ring necklace like rings,
bracelets, anklets,
etc.

Ruby Ruby Stone 1. Different - -

Jewelry
gemstone

2. Any jewelry
made out of ruby
stone

Wrist watch Any wrist watch/ Watch with 1. Watch Strap -

Smart watch bracelet 2. Screen
protector
3. Watch tool kit

Wearables
Shoes Any sneakers, - 1. Shoe lace -
sandals, pumps, 2. Socks
sliders, training 3. Insoles
shoes4

Exceptions:
1
If the SQ was “cushion cover 30 x 40” then even if we get a cushion cover of 40 x 30 size, it will be
marked Exact.
2
For Complement: If the pants are not related to the Batman theme, then Irrelevant.
3
Only in the IN market place, a T-shirt is considered a Substitute for the query shirt.
4
Only in IN market place sandals/sliders will be Irrelevant.

4. Luggage & Bags

Luggage: By definition, luggage refers to empty bags or suitcases, used for carrying a person’s clothes
and things on a journey. Anything which is used to carry personal stuff for the purpose of travelling can
be considered as a luggage.

Bags: a strong container made from cloth, plastic, leather, etc., usually with one or two handles, used to
carry things in when travelling, shopping, etc.
Subcategory SQ Exact Substitute Complement Irrelevant

Luggage Rolling Suitcases, - - -

Luggage
Backpacks, Duffel

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

25
 Bags, Softside
Luggage, Hardside
Luggage,
Two-Wheeled
Luggage, Spinner
Suitcases

Skybags Skybags suitcase Aristocrat suitcase Suitcase cover, Backpack

Suitcase
suitcase Wheels, handle

Bags A bag used for any - - Bags for carrying

purpose. pets
Handbag, Backpacks,
Bags
Tote, Slings, Lunch
bag, Carry bags, Gift
bags, etc.

Caprese Caprese Tote bag/sling Lavie Tote bag/sling Bag Strap, Chain, Suitcase/ Backpack
Handbags
Handbag* bag/purse/wallet bag/purse/wallet bag Charms

MK purse MK purse Gucci Purse straps Suitcase/

Purses
Purse/Handbags Backpack

Allen Solly Allen Solly Tommy Hilfiger Zips/chains, Suitcase/

Wallet Wallet Wallet/clutch wallet/clutch/ Wallet charms Backpack
handbags
Exceptions:
*A bundled item of 3 bags can be given as exact, as all are handbags

5. DIY & Tools

Subcategory SQ Exact Substitute Complement Irrelevant

Patchwork Patches and scale Beads or other Needle/ thread for If the alias is DIY,
for patchwork decorating sewing the chair/bed sheet
products for textile patchwork on the having patchwork
(if alias aps, textile is Irrelevant.
chair/bed sheet However, if the
Art/Craft/ having patchwork) category is "aps"
DIY the SQ can be
considered as
"style" and the
chair with
patchwork judged
as Exact.

DIY/Tools and Cloud decoration Cloud deco DIY kit 1. Star shaped DIY One accessory/ Non-deco
Garden Alias: DIY decoration part of cloud deco products, utility

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

26
 2. cloud deco DIY kit objects like chairs,
tables etc.

Nail (Tools) Bundle of different Nail & hammer - Nail paint or

Alias: DIY sizes of nails (bundle) artificial nail (as
alias is DIY/
industrial)

solar light DIY kit Solar Light DIY 1.Normal light DIY Battery or any -
Alias: DIY kit (not solar) other single part
2. Solar light set for solar light
(FINISHED
PRODUCT)

wood working 1.tools/ Accessory of Wooden products

machines for wood-working like a chair, table.
working with wood tools etc.
Industrial/ 2. book on the
Research and theme
development
products

6. Beauty
Perfume, Cologne and Eau de Toilette are all exact.
Subcategory SQ Exact Substitute Complement Irrelevant

Perfume 1. Spray (eau de - Empty perfume Deodorant

toilette) spray bottle (perfume and
2. Mist perfume deodorant have
Spray different purposes,
Perfumes deodorant is used
to stop odor while
perfume is used to
bring new aromas
to be enjoyed.

Hair care & Shampoo Shampoo Shampoo & Conditioner -

products Conditioner

Body care & Wigs Hair wigs - Clips/caps to attach Hair Extensions
products a wig (Wigs and hair

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

27
 extensions have
markedly different
functionalities. A
wig can be used
over hair or over
scalp, but hair
extensions cannot
be used on scalp,
and must attach to
hair)

Oil Hair oil Oil + Shampoo Empty bottles for oil 1. Shampoo with
castor oil (as main
Alias: Hair product type is
Product shampoo here and
not oil)
2. Cooking oil

Moisturizer Moisturizers Body cream Skin tightening

creams

Foundation Foundation in liquid 1. BB cream 1. Brushes/pad to Lip gloss

form and powder 2. CC cream apply foundation to
form 3. DD cream face
2. directly related
products like
primers, concealers,
setting powders

Eye liner Pencil, kajal Eyeliner brush 1. Contact lenses

Face care & Liquid, Gel liner 2. lip color
products
Lipstick Liquid lip color 1. Lip gloss Lip primer Eyeliner
2. lip balm

Face mask Sheet mask Mask (e.g., tea tree 1. Scrub Mascara
sheet clay mask) 2. toner
3. moisturizer

Blush Blush Cheek tint 1. Highlighter 2. Eye shadow

blush brush

7. Medicines
*For any formula foods we need to ensure that the age of the baby/kids matches.
Subcategory SQ Exact Substitute Complement Irrelevant

Supplements Weight Loss Fat Burner Dietary Supplement Book on Weight Weight gain powder

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

28
 (metabolism Loss
booster/digestive
supplement)

Fever Fever reducing pill Cold and runny Nose Thermometer Fertility Support
Suspension.
(Here, it also can be
used for fever as per
the product
description).

Drug Store

8. Grocery
Subcat. SQ Exact Substitute Complementary Irrelevant
Tomato 1. Tomato 1. Tomato ketchup Vegetable wash Tomato shaped
2. Tomato puree 2. All veggies spray earrings

Food

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

29
 Orange juice Tropicana orange Juice Apple juice Juicer

Beverages

Whiskey Bourbon Whiskey Rum Whisky - The Islay Orgain organic

Edition nutritional shake
(documentary)

Alcohol

Potato 1. Potato (raw) Potato chips Book on Potato wine

2. Frozen or dried potato consumption of
potatoes

Amazon
Fresh

Wheat flour Wheat flour 5Kg X 3 Wheat flour 2.5Kg X Wheat bran Wheat drinks
5Kg 2

Amazon
Pantry

Almonds Almonds 1. Other dry fruits Almond chocolate

[salted/unsalted] 2. Almond flour
3. Almond butter
4. Almond milk
5. Almond oil

Nuts

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

30
 Mayo Mayo 1. Ketchup Fritters with mayo
2. Mustard filling

Peanut Peanut Butter 1. Chocolate spread 1. Ketchup

Condiment Butter 2. Almond butter 2. Mustard
3. Nutella spread

Cake Cake Pancake cake mix Chips

Gourmet

Oats Oats 1. Oat milk 1. Honey Any Spices

2. oats biscuit/oats 2. Dry fruit
cake (primary
ingredient is oats)
3. Any breakfast
cereal

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

31
 Pringles Pringles Chips of other brand salsa dip/any Biscuits
other dip

Vegan Anything vegan Anything non

vegan

Non Meat 1. Vegetarian food

vegetarian 2. Vegan food
Comment: 3. Plant based meat
Non
vegetarians
may
consume
vegetarian/v
egan food,
not vice
versa

9. Fresh
SQ Exact Subsitute Complement Irrelevant

Apple Apple 1. Apple juice 1. Fruit basket 1. Apple chocolate

2. Apple jam 2. Apple peeler 2. Apple yogurt
3. Dried fruit 3. Mixed fruit juice
4. Apple puree

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

32
Lemon Lemon 1. Any fruit 1. lemon sqeezer 1. Mint
2. Lemon pickle 2. lemon zest 2. Any vegetable
3. Lime 3. Pie

Coconut Fresh & dry coconut 1. Coconut milk - Coconut ice cream
2. Coconut oil (for
consumption)
3. Grated coconut flakes

Ginger Ginger 1. Garlic - Tea with ginger and lemon*

2. Ginger paste*
3. Ginger powder* Note - Flavors will be
Irrelevant but when added as
an ingredient then Substitute

Meat Any animal flesh 1. Eggs - Vegetable

2. Processed meat (e.g.,
Salami, Sausages, etc.)
3. Uncooked flavored
meat, Meat patte or
cutlets

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

33
Fish Raw fish 1. Cooked fish Fish shaped snack/chocolate
2. Frozen fish
3. Processed

10. Pets
Subcat. SQ Exact Substitute Complement Irrelevant

Animal (domestic Pet toy, Stuff toy, Pet toy, Stuff toy, Pet food, Pet Product with their Image
e.g., Labrador) Figurines, miniature Figurines , miniature accessory (Print), themed
Alias: pets/aps (of some other
breed/pet)

Pet
Wild Animals Product with their - - -
Alias Image (Print),
themed

11. Toys
Subcategory SQ Exact Substitute Complement Irrelevant

Board game* 1. Ludo Card games like Dices, pawns, chips Soft toys/figurines
2. Catan Uno

Harry Potter 1. Harry potter Fantastic beasts Harry Potter wand Batman mask
costume costume costume
2. Any other
character costume
Toys
from Harry Potter
like Dumbledore or
Hagrid

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

34
 Games for 5 Games for 0-5 year Game for 0-4 years Guide to the game Game for adults
year olds olds or 6 years (we
allow -1 and +1
age only)

Plush toys Any Plush toys - Accessory for plush Figurine, Plush toy for
toys pets, Adult toy

*Be mindful of the product’s genre if the product is media themed.

12. Home & Garden

Subcategory SQ Exact Substitute Complement Irrelevant

Bed 1. Single bed Sofa cum bed 1. Mattress Chair

Furniture 2. double bed 2. bed sheet

Wall paper Wall paper Poster Glue stick Photo frame

Wall Art

Bath towel Towel Hand towel 1. Hook Bath mat

Bathroom 2. Rack

Storage & Wardrobe Almirah Clothes stand Wardrobe light Bed

Organization

Plant/ flower Real plant/ flower Artificial plant/ 1. Planter Planter stand
Garden (including seeds) flower 2. Fertilizer

13. Kitchen
Subcategory SQ Exact Substitute Complement Irrelevant

Dining Plate Plate Thali, Bowl Spoon Cooker

Cookware Induction Induction Gas Stove Induction cleaner Cylinder

Chimney Chimney Exhaust fan Chimney pipe Ceiling fan

Appliance
Microwave Microwave OTG (Oven toaster Microwave plate Sandwich toaster
grill)

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

35
 14. Automobiles
Subcategory SQ Exact Substitute Complement Irrelevant

Car/ Car (same model) - Accessory (of Plastic model,

Vehicle (model) the same model) miniature, figure
Alias: aps (of the same
model)

Tire/ for same model - Bulbs -

Automobiles
Headlights/ (compatible)
Battery

Car/ Plastic model, - - -

Vehicle (model) miniature, figure
Alias: toys

15. Shoes
Subcategory SQ Exact Substitute Complement Irrelevant

Clogs Clogs for any Bathroom slippers Sole for sandals Sport shoes for
gender any gender

Sports shoes Any - Laces for sports 1. High heels

for women sports/running shoes 2. formal shoes
Shoes shoes 3. sports shoes
for kids or men

Tommy Shoes, wallet, Shoes, wallet and - T-shirt from TF or

Hilfiger shoe laces, etc. accessories from any other brand
Alias: shoes from Tommy another brand
& bags Hilfiger

16. Theme
Subcategory SQ Exact Substitute Complement Irrelevant

Covid Book about Covid - Equipment Covid themed

(Disease or (masks, wipes mug
global event) etc)
Theme
Cricket All cricket - Book on Normal shoes
equipment/gears. cricket

*Note: unlike SQ - Football

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

36
 which is a product type in
itself (the ball is called
football), cricket is treated
as a theme.
SQ: Football
OP: Football shoes-
complement

Profession Anything for that - - Toy sets for

(e.g., Doctor) profession children
like stethoscope associated with
or on that profession that profession
or merchandise (figurine) like Doctor set

17. Adult
Subcategory SQ Exact Substitute Complement Irrelevant

Electric Electric body Wooden Manual for using Adult massager

massager massager hand-held electric (sexual context),
body massagers massager Vibrators

Toy Normal toys - Accessories of Sex doll/toys

(e.g., Doll) (Doll) dolls

Condoms XL Condoms XL Condoms L Lubricant 1. Condoms S

2. Sex toy

Sex toy for Sex toy for men - Lubricant 1. Sex toy for women
men 2. Condom
Adult
Costume Costume (e.g., - Clown costume 1. Sexual role play
Harry Potter, etc.) accessories costume
2. Normal shirt being
called Cat costume
(B07HJSG2JS)

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

37
Books Queries
Important points to consider in specificity to ‘books’ queries, divergent to the above/usual guidelines:
Key Points:

● Price, adjectives, services – to be considered

● Overlapping ages to be considered Exact, adjacent ages to be considered Substitute
● Sub-genres of the books to be deep-dived
● Target audience to be explored and rated irrelevant, if too distant from the search query
● Merchandise to be rated as Substitute

Further Elaboration:

(1) Price/Free: any book (in any format) matching the price constraints chosen by the customer are to be
taken into consideration. If a customer is looking for “free books”, the price must be 0 in the local
currency. Any books which are non-free would thus be rated Substitute.

SQ OP Judgment

HP 20 dollars HP 15 dollars Substitute

Free books HP 15 dollars Substitute

(2) Bestsellers/Validated Adjectives: query constraints must be respected. For example, products
offered for the query “romance bestseller” will need to satisfy both bestseller and genre to be judged
Exact.

Adjectives such as ‘cute’, ‘best book’ may be subjective and can continue to be disregarded at the time of
relevance judgment, however, inclusions like ‘top seller’, ‘high rated’, ‘bestsellers’ must be considered for.
The criteria for researching and dive deep here may include pointers and hints such as:

* any Book with the Amazon Charts OR Editor’s Pick OR Bestseller badges
* any Book with “New York Times bestseller” reported on the cover
* any book ranked within Books/Kindle Store Zeitgeist top 2,000
* any book with more than 1,000 reviews

Below books would be rated Substitutes:

* any book ranked between 2,000 and 10,000 in Books/Kindle Store
* any book having between 500 and 1000 reviews

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

38
(3) Service queries: while service or membership seeking queries such as “cancel kindle unlimited”
continue to be disregarded, keywords (and functional equivalents) such as “kindle unlimited”, “prime
reading”, “audible plus”, “kids plus”, “comixology unlimited” all contextually hint towards a customer
searching for books and must be considered for and judged accordingly.

E.g., while the SQ: ‘kindle unlimited’ could entail all books to be marked as Exact, ‘audible plus’ hints at a
particular format of book and thus must be provided the relevant judgment, distinguishing format.

(4) Age range/reading level: a children's book for an age range adjacent to the one the customer was
looking for (e.g., a book suitable for 3-4 yrs old when customer is looking for “children books 5-6” would
be rated Substitute.
Overlapping age range, e.g., books for 3-4-year-old and resulting product with 4-5-year-old book may be
rated Exact.

Grade Age

Pre-kindergarten 3-5

Kindergarten 5-6

1st grade 6-7

2nd grade 7-8

3rd grade 8-9

4th grade 9 - 10

5th grade 10 - 11

6th grade 11 - 12

7th grade 12 - 13

8th grade 13 - 14

9th grade (High School) 14 - 15

10th grade (High School) 15 - 16

11th grade (High School) 16 - 17

12th grade (High School) 17 - 18

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

39
(5) Genre, sub-genres, and the subject of a book should be deep dived to determine OP’s relevancy if it
is not an exact match: genre refers to categories of works that are characterized by similar plots, themes,
settings, situations, and characters (reference). In other words, the genre describes what a book is.
Fiction and Non-fiction are macro genres and they each can be subdivided into sub-genres. By contrast,
the Subject describes what a book is about or its topic. Example subjects include architecture,
photography, sports, etc.

*Fiction books offered for a non-fiction book query are Irrelevant and vice versa.

In many cases, it will be necessary to determine both genre and subject to assign an ESCI label to an
ASIN. As a rule of thumb, books in the same genre/macro-subject are substitutes, unless the customer
query is for a specific sub-genre. If the customer specifies a sub-genre/macro-subject in the query, ASINs
not belonging to that subgenre are Irrelevant.
* Please utilize the BISAC Category Labeling for researching and taking the correct judgement calls.

(6) Target Audience: in association to genre and subject, the intentional target audience of the book
must be considered. For example:
SQ OP Judgment

living with coronary heart Living with Coronary Heart Exact

disease Disease: A Guide for Patients
and Families

living with coronary heart Coronary Heart Disease in Irrelevant

disease Clinical Practice

The desired book is directed towards a non-professional audience on the subject of cardiology. The
second product is aimed towards an academic audience. Since an exact match for the customer query
exists and is a non-professional book, the second ASIN is Irrelevant.

(8) Media vs Non-Media/Merchandise Products: all the media products that exactly match the
keywords and are either the main work(s) in the original medium or other media derived directly from
the original medium are Exacts. Conversely, non-media products about the original work(s) are
Substitutes. For example, all Harry Potter movies and books are Exacts as they are either the primary
work (the books) or direct media derivations of the primary work (the film transpositions).

However, all non-media results, like merchandising (https://www.amazon.com/s?k=harry+potter), are

Substitutes.

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

40
 Similarly for people's names (historical figures, celebrities, authors) and fictional characters – any media
product authored by or about the person is an Exact. Whereas, merchandising and related products are
Substitutes.
SQ OP Judgment Explanation

Harry Potter Harry Potter coffee Substitute For this query, all Harry Potter movies and books
mug (merchandise) are Exacts as they are either the primary work
(the books) or direct media derivations of the
primary work and all non-media results (like
merchandise) are Substitutes.

Harry Potter Harry Potter coffee Irrelevant This would be the same for SQ "harry potter
book mug mug", the mug is exact, the book irrelevant. The
PTs are significantly different. We will not be
looking at Complementary here as a rating.

For further detailed insight on approaching book queries, please refer to the books SOP that is shared
separately.

Special Mentions
In each marketplace there are classes of products that are not carried by Amazon for legal and
Products not logistical reasons. Customers are not necessarily aware of these constraints, and still issue
sold by queries for many of these products. The ESCI judgment framework enables reviewers to
Amazon accurately label items that are in stock relative to the explicit query intent. With the ESCI
framework, the judgments should reflect the product’s relationship to the query without
consideration of whether the product is sold by Amazon. So for some product types, there will
be no Exact judgments.

Exact will be the exact product, for ex- the actual vintage car or refile. Functional
equivalency has to be checked.

Query: Cat animal

OP Judgment Explanation

1. Cat food 1. Complement Accessories for domesticated animals

2. Cat Figurine/Soft toy 2. Exact would be complementary. The only
3. Products with image 3. Irrelevant exception in this case is Soft
toy/Figurines.
of cats

Query: AK 47

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

41
 OP Judgment Explanation

1. Airsoft gun 1. Irrelevant 1, 2 - Are not functionally equivalent

2. Toy gun 2. Irrelevant 4 - AK 47 is not treated as a theme
3. Book to clean AK 47 3. Complement when it is an object in it’s own right
4. Irrelevant (product type is clearly defined)
4. Image of AK 47 on
T-shirt

Exceptions:

● We mark accessories to cars or pets as complementary as Amazon might be willing to

sell those products in future.
● Certain queries like dandruff, cancer, sexy woman or dirty laundry are not the products
in themselves and cannot be sold by Amazon ever.
● In such queries, all the products like anti dandruff shampoo, serum to treat dandruff or
books on itchy dandruff scalp, all shall be marked exact. For dirty laundry products like
detergent, fabric softener, etc. can be marked as exact.

For many broad queries, the customer’s intent may be unclear. Multiple interpretations of the
Queries with query could result in different judgments for an ASIN. In these cases, the judgments should
Multiple follow the sequence Exact, Substitute, Complement, Irrelevant, Unjudgeable: if one of the
Meanings interpretations would result in an Exact judgment for the query, then that should be the
judgment, even if other interpretations would result in Complement or Irrelevant judgments. If
the query is issued within a specific category, then all the interpretations used for judgment
should respect that marketplace category.

Query: glasses
OP Judgment

Eyeglasses Exact

Drinking glasses Exact

Sunglasses Irrelevant

Lens cleaning cloth for glasses Complement

6 drinking glasses plus a pitcher Substitute

Query: ona
OP Judgment

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

42
 Ona (brand of air fresheners) Exact

ONA (brand of leather bags and straps) Exact

Ona bar Irrelevant

We do not know if the customer entered ONA or Ona - the queries are all converted to
lower-case. Since the word stands alone it could refer to either, so product lines from either
would be Exact. Ona Bar is judged Irrelevant, since a customer looking for that product would
have entered “ona bar”.
 
When the query is in conflict with the marketplace category (sometimes referred to as Alias),
Query-Alias and there is no reasonable interpretation for what the customer intended for that query, then it
Conflict can be judged as if the category was aps. For example, if the search query is ‘Jacket’ and the
search alias is ‘Shoes & bags’, we will ignore the category and consider only Jackets as the Exact
product.

Some of the most difficult queries are partial words or partial titles. If they are issued within an
Truncated alias such as books or instant-video, then it’s easier to judge based upon products that begin
Queries with that string. Use Amazon or Google search to see what the auto-complete query options
are, and consider each of them as valid interpretations for the query. Each of those
  interpretations may be in different categories, so ASINs can be judged Substitutes or
Complements for any of the interpretations. Do not consider truncations if a word is a product
by itself, only if the query is an incomplete product title.

Queries which are misspelled may have multiple interpretations depending upon which letters
Misspelled are substituted. Any spell correction offered may or may not be accurate to the customer’s
Queries intent. If the query is unclear, consider all common misspellings as different interpretations for
the query, you can use Google or Amazon search to identify common spell corrections. If a
 
word is spelled correctly and clearly a product query, then do not consider misspellings.

Queries for a brand or merchant which sell different product types could be referring to any of
Franchise/ them, so ASINs are to be judged based upon any of the different products which the customer
Brand may have intended within those product lines. Some brands have their logos displayed on a
Queries broad range of merchandise: these are judged Exact unless the query has an alias or other
  terms specifying which type of product the customer seeks.

For a query which is a sports team, videos of the games and merchandise are Exact.

SQ: name of a Sports team

OP: Videos of their games -  Exact
OP: Merchandise of the team - Exact

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

43
 If any of these terms appear in a query, they are not used when determining ESCI relevance, and
Quantity/ the query is judged as if those terms were removed. Subjective terms such as ‘cute’, ‘cool’, ‘nice’,
Quality/ Price/ ‘unlocked’, ‘refurbished’, ‘best seller’, ‘best’, ‘renewed’, ‘cheap’, ‘free’, ‘sexy’,’ awesome’, ‘new’
Shipping/ can be ignored.
Condition
Please note, quantity (measurable in units) is distinguished from quantity (in numbers). For
example, if the SQ is 5kg of rice and the OP is 2kg of rice the judgment would be Substitute. In
this case, the weightage or unit of product is considered whereas multipacks (e.g., 2- or 3- pack
of rice or any other product) should be ignored. For example, if the SQ is a T-shirt and the OP is a
2-pack of T-shirts it is Exact.

Widget is to be seen. If the SQ is diaper for 5 months old, and the offered ASIN is a diaper of 3
Widget months old, with an availability of 5 months old, we will mark the ASIN as Exact.  

When a query consists of a person’s name, it is judged differently depending upon

Name what relationship that person has to the ASIN:

● If they are a primary contributor towards a book, video, piece of music or art, then that
ASIN is Exact, and any ASINs in the same category are Substitute, in case of author name
we check for genres.
● If the ASIN is about them or depicts them, then it is Complement.

Actor (generally stars Movie of the asked actor in main lead role - Exact
as main lead)
Movie of the asked actor in supporting role - Exact
Merchandise of the Actor - Substitute

Actors which have Some actors are known for making one type of film than others,
known for certain for ex Jackie Chan is known for making martial arts film, so other
genres martial arts films will be Substitute.
Name of a music band Merchandise of that band - Substitute

Name of a singer ( For Music album/songs of the singer- Exact

ex Beyonce)
Multiple artists in the same song- (Beyonce + Shakira "Beautiful
liar)- Exact, in case of multiple contributors, we consider all as
primary contributors

Megan Thee Stallion - Savage Remix (feat. Beyoncé)- Exact

Merchandise of the singer- Substitute

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

44
 William Morris (Name William Morris was a designer, who designed for many types of
of a designer) products- furniture, fabrics, stained glass, wallpaper etc, so any
product bearing his designs can be exact. Similarly, if SQ is an
apparel designer and OP is an apparel from that designer (ex), but
if OP is an apparel from a different designer then (subsi). Product
type designed by the asked designer will be narrowed down, for ex
jewelry designer, interior designer etc.

For a query for a fictional character-

● the primary book/movie/game that character came from is to be judged Exact

● costume & merchandise- such as posters, toys, clothing- Substitute.
● other characters from the same book/movie would be Substitute,
● and books of analysis/commentary on the subject of the fictional character would
be Complement

John Snow OP- GOT season 1 Exact- If the query is character’s name, the
OP- character’s primary book/movie/game that character
figurines or came from is to be judged Exact, unless are
merchandise also represented in many types of
products, such as posters, toys, clothing,
any of which can be Substitute.
Other characters from the same
book/movie would be Substitute, and
books of analysis/commentary on the
subject of the fictional character would be
Complement.
Kit Harington Game Game of throne Exact
of Throne series

Deals Related SQ Category OP Judgment Explanation

Queries Deals/ aps T-shirt Unjudgable This is an ambiguous query as the
Offers of product type is not mentioned and
the day the customer is looking for
offers/deals. We would mark this as
Unjudgeable.
Deals/ apparel Ladies skirt Exact Due to the selected category, we
Offers of would not mark this search query as
the day unjudgeable. Instead, we would
mark all the products belonging to
the selected category (apparel in this
case) as Exact.

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

45
 Iphone aps Iphone 13 Exact As the product type is clearly
deals of mentioned in the search query, we
the day would not mark it as unjudgeable.
Judgment would be given according
to the product type mentioned in
the query.

This material cannot be shared with third parties without TELUS International written consent. DO NOT
COPY. DO NOT DISTRIBUTE.

46