DECLARATION
DECLARATION
DECLARATION
I, Rishi Kumar, student of B.tech (IT) hereby declare that the project titled “AI/ML role in
Locamart Private Limited” which is submitted by me to Department of Information Technology,
Amity School of Engineering and Technology, Amity University Uttar Pradesh, Noida, in partial
fulfilment of requirement for the award of the degree of Bachelor of Technology in Information
Technology, has not been previously formed the basis for the award of any degree, diploma or other
similar title or recognition.
Noida
Rishi Kumar
Date:
2
CERTIFICATE
On the basis of declaration submitted by, Mr. Rishi Kumar student of B.Tech. (Information
Technology), Batch: 2021-2025, Amity School of Engineering and Technology, Amity University
Uttar Pradesh, Noida. I hereby certify that the internship project titled “AI/ML Role in Locamart
Private Limited.” which is submitted to Amity School of Engineering & Technology, Amity
University Uttar Pradesh, Noida in partial fulfillment of the requirement for the award of the degree
of B.Tech. (Information Technology), is an original contribution with existing knowledge and faithful
record of work carried out by him under my guidance and supervision.
To the best of my knowledge this work has not been submitted in part or full for any degree or
diploma to this university or elsewhere.
Place: Noida
Date:
3
ACKNOWLEDGEMENT
I take this opportunity to express my profound sense of gratitude and respect to all those who helped
me throughout our project.
This report acknowledges the intense driving and technical competence of the entire individual that has
contributed to it. It would have been almost impossible to complete this project without the support of
these people. I extend thanks and gratitude to Prof. (Dr.) Nitasha Hasteer, HOD, Department of
Information Technology and Dr. Ritu Punhani, Assistant Professor, Department of Information
Technology, who have imparted me the guidance in all aspects. They shared their valuable time from
their busy schedule to guide me and provide their active and sincere support for my activities.
This report is an authentic record of my own work which was accomplished by the sincere and active
support by all the teachers of my college. I have tried my best to summarize this report.
Rishi Kumar
B.tech (IT)
ASET
Noida
4
TABLE OF CONTENTS
1. Abstract 6
2. Introduction 7
3. Literature Review 8
4. MODEL DEVELOPMENT 10
7. Conclusion 29
8. References 30
5
1. ABSTRACT
This report represents the role carried out during my AI internship at Locamart, a shopping
application based on the idea of connecting local vendors to customers which will increase the
connections and sales of local vendors. The primary goal was to develop a search model capable
of processing both text and image queries.
For this, three distinct models were developed: a text query searching using the Bert model, an
image query searching using the Mobilenet model, and a combined model that handles both
query types. These models had been trained in various sets to ensure high performance in real-
time scenarios. Additionally, these models were converted to TensorFlow Lite format to optimize
them for the Firebase platform. Also, there is the role of integrating background removal API to
improve image search accuracy.
The results show a noticeable enhancement in searching capabilities, giving customers a more
user-friendly and effective approach to locating products. This research demonstrated how
artificial intelligence (AI) may revolutionize e-commerce apps while also improving the user
experience.
6
2. INTRODUCTION
2.2 Objectives
The objectives of my role as an AI intern in the Locamart are
Development of the searching model based on text to test the accuracy of the model
Development of the model having capabilities of both text and image searching.
Conversion of these models into t-flite version to integrate it into the app which is being
developed in the firebase platform.
Integration of a background removal API which increase the accuracy of the image
findingand analyzing.
7
3. LITERATURE REVIEW
3.3 MobileNet
MobileNet was introduced by Howard et al. in 2017 in the class of convolutional neural
networks designed for vision based mobile applications. It utilizes the depth separable
convolution which had reduced the number of parameters and computational costs as compared
to standard convolutional networks without compromising the quality.
In the field of e-commerce, it offers a great image recognition ai models with less searching time
and creates a great searching environment as it converts the image into vectors and thereafter by
using cosine similarity, it provides the results.
3.4 Multimodal
The concept of multimodal was introduced first in the Kaggle competition by the Shopify. In this
model, their will be the combined power of both BERT and MobileNet models with less
searching times. The given diagram will show its working.
8
Figure1. Working Diagram of Multimodal
9
4. MODEL DEVELOPMENT
1
Then joining all the tokens back to a single string
The above operations require the punkt, stopwords, and wordnet modules
After the preprocessing that single stored string has been stored into the new variable named
clean_name.
After that there has been the integration of the AutoModel and AutoTokenizer for the using of
the Bert model which will help for the bidirectional searching.
1
After this the clean_name has been embedded by the help of this function:
After this, there has been importing of Pytorch library, which is a open source Machine
Learningused for a computer vision and NLP. It provides two high level features:
Tensors computation with strong GPU acceleration
And, deep neural networks built on a tape based autograd system
After this the lambda x: get_embedding([x]}. numpy () [0] is used over each element
present inthe clean_name variable.
get_embedding([x]}:it takes as a list of string and returns their embedding
numpy (): It converts the Pytorch tensor returned by get_embedding to a numpy array.
[0]: This indexing extracts the first element of the Numpy Array and also retrieves
theembedding vector itself.
1
Assigning Embedding results the creation of new column name embedding from the
columnname clean_name in the product_data Data frame.
This is the final output which shows that the required goal has been achieved from this
modelfor the text based searching model with target variable Description.
1
4.2 Image and text-based query-based searching model
The image and text based search combinedly called the multimodal where the text
is basedon the Bert and the image is based on the Mobilenet is being used
For this the required libraries are needed to be import
For the GPU installation, and verification the platform, following functions as
well asmethods are used
For the importing of databases, the pandas libraries has been used and the
differentmanipulations has been used
1
For the plotting of image and processing of the images matplot libraries and PIL has
been used.
This method is being used for the listing of all the images along the row.
1
From the Sklearn library testing and training data has ben split into two parts as is the
ratio2:3.
From the sentence transformers library all the text is being transformed into tokens
and afterthat MobileNet model has been loaded for taking the input and conversion of
all images in the columns with some specific columns and is specified in the next
image.
1
After that the time library has been imported to improve the load time of the image and
text after the conversion of them into its embedded form. Here the load time of the
model is 1204.0009
After that the pickle model has been imported for the storing of text and image storing
into thefile named textEmb and imgEmb.
Then the whole processing list of key, combined embedding, image list and title list
has beenloaded by the below method.
1
In our work, we utilize the Sklearn Neighbors library to import the nearest neighbors
module. Here are some concise details:
Nearest Neighbors play a vital role in various scenarios such as recommendationsystems,
anomaly detection, clustering, and more.
When setting n_neighbors to 5 for recommendation systems, the model will return the5
most similar items for any input query.
A leaf size of 5000 is considered large and should be selected based on dataset size
tooptimize search speed.
The choice of algorithm ('kd_tree', 'ball_tree', 'brute', or 'auto') impacts performancebased
on data characteristics and speed requirements.
In conclusion, the selection of algorithm depends on data structure and processingspeed
needs.
1
In Python, the code begins by setting up the necessary components. Two essential lists,
postingidList and matchesList, start off empty. These lists play a crucial role in storing the
outcomes of the nearest neighbor search. As we begin, index is set to 0 - a counter that
helps navigate through keyList and cembList effectively.
The code indeed generates multiple images, yet it does not produce any new content. Let's
break down its functionality:
matplotlib. pyplot as plt: This line brings in the matplotlib plotting library, giving it the
alias plt for easier accessibility.
from PIL import Image: This imports the Python Imaging Library (PIL) for image
processing and assigns an alias to the Image class for convenience.
Sample Data (Substitute with your real data):
1
It tries to display the image with plt. imshow including a title derived from the matching
product's title.
In case of an error (such as an out-of-range index or missing image), a general error
message is printed.
Limiting Loop Iterations (Optional):
The loop increments an index variable (index) after each iteration.
2
This piece of code forms a crucial component of a system for matching products, likely
designed for an e-commerce platform such as Shopee. The primary objective of this system
is to identify and exhibit products that are alike or match each other, relying on various
attributeslike titles, keys, and images. This feature can be highly beneficial for:
Product Recommendations: Offering suggestions for similar products to users basedon their
current browsing activity past views.
Duplicate Identification: Spotting duplicate listings within the database to ensure the
consistency of data.
2
Visual Search: Enabling users to conduct product searches by uploading
images anddiscovering visually akin items.
2
From this output its being clear that the objectives of the model has been achieved.
2
5. Conversion of Models into t-flite version
Output:
2
Then the Mobilenet model has been converted into t-flite version.
Input:
Output:
After that these models combinedly converted into multimodel tflite version
2
After that the file is downloaded into the file sections and the output is given in the next page.
2
2
6. Using of Background remover API
For the removal of background from the image I use backgroung.removal Api which
has beenintegrated with python code:
Input Image:
Figure 2: Camera
Output Image:
2
7. Conclusion
Locamart is a promising self-sufficient platform which vision is to connect customer
with localsellers. It seeks to streamline the shopping experience by offering:
It seeks to streamline the shopping experience by offering:
Easy product discovery: Consumers are able to easily locate the products they are
lookingfor with text or image-based searches.
Improved user experience: Being connected with a huge network of vendors and
being easy to use make for a pleasant shopping experience.
Increased vendor revenue: Effective navigation improves the chances of finding a
productthus increasing the number of customers buying the products sold by the
vendors.
Time savings: Super buyers gain convenience in terms of time in searching for
products they want to buy or actually buying the products while super sellers target
to enjoy convenience in a way that they will not spend too much time in marketing
their products.
The following outlines the activities aiming at the accomplishment of these objectives.
The internwill focus on, After that, my focus will be focus on:
Developing accurate search models: These models will cater for text based and
image based search thus making it easier for the users to locate the products.
Optimizing and integrating models: Finally, the models will be converted to
firebase mobile app compatible models for ease of integration.
Background removal API integration: This will also improve image search
relevance sincethe removal of the background discards unwanted data that clouds
the search.
Hence, the following AI parts that have been applied at Locamart can enhance the
purchaseexperience for consumers and seller in a way. This will mean higher sales
from vendors, more customer satisfaction, and a prosperous market for the local
vendors.
2
8. References