A Real-time Hybrid Clothing Recommender System:

Integrating Content-Based Learning with User Interaction

Prof. Revathy B D, A P Bharatesh Aradhya, Mohan K, Kunal R, Sitamshu S D

Abstract:
A real-time hybrid recommender system for clothes is shown in this work. It uses a swipe-
based interface in which users indicate their preferences by making simple gestures: left swipes
for products they dislike and right swipes for items they like. This user-friendly interaction
model improves usability and engagement by collecting user feedback with simplicity. In order
to generate comprehensive item profiles, the system analyzes important garment qualities like
color, pattern, material, design, and style using a combination of content-based and
collaborative filtering algorithms. These profiles aid in the recommendation model's efficient
comprehension of user preferences. Concurrently, the model incorporates user interactions,
creating an adaptive loop that continuously improves suggestions. The system is guaranteed to
remain sensitive to changing preferences because to its dynamic feedback mechanism, which
provides tailored suggestions that closely match each user's preferred style.
This hybrid strategy provides a scalable, responsive answer to issues like the cold start
problem and low user engagement. Initial recommendations for new users are made possible
by content features, and accuracy is gradually increased through ongoing learning from user
interactions. By tailoring recommendations to each user's preferences, this strategy not only
increases user satisfaction and engagement but also increases conversion rates. The system
also gives retailers useful information about consumer preferences and fashion trends,
enabling them to make data-driven choices about marketing and inventories. All things
considered, this recommender system is a potent instrument for the fashion sector, combining
accuracy, versatility, and easy usability to provide incredibly customized experiences.

Keywords:
Collaborative Filtering, Content-Based Filtering, Hybrid Recommender System, Deep
Learning, Machine Learning, Hybrid Clothing Recommender System, Similarity Measures,
Item-User Matrix, Neural Networks, Dimensionality Reduction, Cold Start Problem, User-
Item Interaction.

Introduction:
These days, online clothing sales have grown in popularity and appeal due to their affordability
and high quality. Yintai.com, Vancl.com, and Shop.vipshop.com are a few instances of these
prosperous websites that offer hundreds of clothing items to online buyers. Online buyers face
the difficulty of selecting a quality product from a wide range of possibilities. To make buying
easier, we present a collaborative clothing recommender in this post. This system's capacity to
suggest apparel based on both user ratings and clothing qualities is one of its distinctive
features. Our simulation environment's experiments demonstrate that the suggested
recommender can more effectively meet user needs. 【2:1】
Introduced in the mid-1990s, recommender systems are crucial e-commerce platforms that
make product recommendations based on user behavior. They play a big part in increasing
revenue, improving cross-selling prospects, and cultivating customer loyalty in addition to
helping consumers find suitable goods. These systems can be broadly classified into three
types: collaborative, which leverages the preferences of comparable users; content-based,
which depends on product qualities; and hybrid, which combines the two methods. This article
introduces a hybrid recommender system made especially for clothes purchasing, which makes
use of both product qualities and customer ratings to provide more individualized and fulfilling
suggestions. 【2:2】

These days, recommender systems are a necessary component of our lives. Recommender
systems are used by the majority of social websites to improve user experience. A
recommender is very important because clothing is one of the most popular domains on online
buying platforms like Amazon. Retrieving goods based on user preference—that is,
resemblance to the user's prior purchases—was a common focus of earlier recommender
systems. But in the clothing industry, recommendations also depend on how well the
prospective items fit the ones that have already been bought. For instance, if a customer has
recently bought a shirt, he or she might wish to buy a new pair of jeans instead of suit pants.
Such matched partnerships are equally common in other areas of life. Our goal in this study is
to suggest new clothing that will better fit the user's existing wardrobe. This novel
recommendation approach would enhance the existing recommendation literature and perform
better in the clothing sector. Experiment results show that our method can lead to better
recommendation performance in the Clothing domain.【4:1】

Recommendation systems are crucial in today's digital world, especially when it comes to
online shopping, where they improve user experience by making product recommendations
based on past purchases. In addition to customisation, they increase company value by drawing
in new clients. Considering how prevalent fashion is in e-commerce, clothing
recommendations are particularly crucial. Conventional systems mostly consider user
preferences and frequently promote related products, such as jeans if a user has already bought
them. Nevertheless, these systems frequently fail to consider whether newly purchased
products correspond with previous purchases, which may result in underutilized items. This
essay discusses the significance of matching new clothing with previously purchased items
while taking color, texture, and style into account. Users want clothes that go well with their
outfits, therefore compatibility is important. The suggested approach blends matching strength
and user preferences to make better suggestions.【4:2】

By effectively managing complicated data, deep learning plays a critical role in improving
recommender systems, which seek to provide suggestions based on user preferences.
Recommendation algorithms are being used more and more by online shopping platforms,
particularly in the apparel industry, to increase sales. This tendency has been driven by the
COVID-19 pandemic-related shift to online activities. This study introduces a deep neural
network-based content-based apparel recommender system that automatically generates
product attributes, such as category and gender, obviating the requirement for manual feature
extraction. The method provides fresh, pertinent, and surprising suggestions while resolving
the cold start issue for new goods. Even without taking demographic information like gender
into account, experimental results show that it performs better than comparable models by
attaining a lower loss. 【5:1】
In a variety of industries, including e-commerce, music, and film, recommender systems are
crucial tools that let customers locate pertinent products fast and effectively. There are four
primary groups into which these systems fall. When new users or things lack data,
Collaborative Filtering (CFRS) faces the cold start issue. CFRS makes suggestions for items
based on similar user behavior. Although they too have the cold start problem, content-based
systems (CBRS) suggest products that are comparable to those a consumer has already dealt
with. Knowledge-Based Recommenders avoid the cold start issue by employing user-defined
preferences to cater to specific commodities, like cars or apartments. Multiple strategies are
combined in hybrid systems to improve accuracy and dependability.【5:2】

Fast fashion and the growth of e-commerce are the main drivers of the fashion and textile
industries' explosive growth, according to the study. By providing personalized
recommendations amidst the plethora of online options, a successful fashion recommendation
system is crucial to enhancing the user experience. FRSs analyze user preferences and forecast
fashion trends by leveraging advances in image processing and computer learning.【7:2】
Despite FRSs' potential, there isn't a thorough analysis of the academic literature that
incorporates cutting-edge models and methodologies. By thoroughly examining the main
models, filtering strategies, and assessment criteria employed in the creation of FRS, this
research seeks to close that gap. Researchers and practitioners can learn more about the
advantages and disadvantages of current systems from the review.【7:2-3】
The writers stress how the popularity of internet buying and growing living standards have
made fashion more significant in daily life. Conventional recommender systems sometimes
overlook consumers' visual preferences because they are based on past purchases.【7:1】

This disparity drives the creation of image-based fashion recommendation systems that provide
visually comparable products to increase customer satisfaction. This study suggests a
customized fashion suggestion system utilizing image-based neural networks in light of the
difficulties customers encounter when choosing clothing online. In contrast to traditional
approaches, the suggested system makes recommendations based on product photos, satisfying
customers' desire for aesthetically pleasing fashion goods.【9:2】

Literature Survey:
The idea of hybrid recommendation systems designed especially for the online apparel retail
sector is presented in the work by Hu et al. The dual requirement of making product
recommendations based on item qualities and user preferences is met by this system.
Conventional recommender systems, which usually utilize content-based approaches or
collaborative filtering, sometimes fail when item features are not fully captured or when
consumers do not have long interaction histories. Hu et al.'s hybrid model offers a more
thorough recommendation framework by combining various strategies to address drawbacks
like the cold-start issue. This is especially important in online buying settings because
customers require individualized direction due to the abundance of possibilities.【2:1-2】
In order to identify garment traits like prominent colors, the HCRS system architecture is
organized around a multi-stage process that starts with human detection and image
preprocessing. Traditional collaborative filtering techniques are enhanced by an enlarged item-
rating matrix that incorporates these qualities. HCRS is a modified K-means clustering
technique that incorporates fuzzy set theory, which allows things to belong to many clusters at
the same time, in contrast to traditional matrix-based recommenders. A more accurate and
customized suggestion output is made possible by this, which allows for a nuanced
representation of user preferences and item features.【2:3-4】
Hu et al. focus on measures like Mean Absolute Error (MAE) in their extended trials utilizing
a dataset created from 163 individuals and 50 apparel items. The findings show that, in
comparison to conventional collaborative filtering methods, HCRS dramatically lowers
prediction error. Furthermore, a richer user experience is offered by the system's capacity to
incorporate both textual and visual elements into its recommendation engine. The results
indicate that by providing more context-aware and targeted product recommendations, this
hybrid strategy not only improves user satisfaction but may also have implications for raising
online sales conversion rates.【2:5-6】

The study by Gharaei et al. is set in the backdrop of quickly expanding online retail platforms
where the volume of data makes human feature extraction for product recommendations
unfeasible. The study highlights that the cold-start problem, which occurs when there is
inadequate user-item interaction history, frequently causes traditional collaborative filtering
techniques to fail. To get around this, the authors suggest a content-based approach that uses
deep neural networks (DNNs) to automatically extract features from product photos. This
improves scalability and does away with the requirement for human input.【4:1-2】
A deep convolutional neural network (CNN) that can handle feature extraction and
classification tasks concurrently forms the basis of the suggested system. For binary
classification problems like gender detection, this design consists of 13 layers with ReLU
activation functions, followed by a final sigmoid layer. To reduce errors, the network is trained
by combining the Nadam optimization method with category focal loss. Furthermore, the
system makes use of transfer learning by optimizing previously trained models to improve
performance on particular tasks, such as gender recognition and apparel category classification.
【4:4-5】
Results from experiments show a significant increase in prediction accuracy and suggestion
precision. With a precision rate of 73.7%, the system outperforms baseline models like ResNet-
50 in terms of training efficiency and loss reduction. Furthermore, the system efficiently
suggests new and surprising items by utilizing cosine similarity to assess the proximity of
feature vectors, which raises user happiness and engagement. This illustrates the system's
usefulness in actual e-commerce platforms where customer retention depends on prompt and
precise recommendations.【4:6-7】
Lei et al. investigate a new method for making clothing suggestions by emphasizing how well
freshly suggested goods match ones that have already been bought. This approach takes into
account visual and contextual characteristics including color, texture, and shape, in contrast to
conventional systems that rank user preferences only on the basis of past purchases. The study
addresses a critical gap in current recommender systems, which often overlook the importance
of item compatibility in fashion, where matching aesthetics play a crucial role in user
satisfaction【5:2-3】
The approach integrates visual features through hierarchical clustering and uses a time-
enhanced version of SVD++ to record changing user preferences. This hybrid methodology
evaluates the visual compatibility of new suggestions with already-owned wardrobe items in
addition to tracking changes in user preferences over time. By grouping previously bought
products according to their visual and contextual characteristics, the method lowers computing
complexity and improves item matching accuracy.【5:4-5】

This study offers a thorough analysis of fashion recommendation systems (FRS), outlining
their importance in improving customers' individualized purchasing experiences. By selecting
pertinent products according to user preferences, the authors highlight their function in
mitigating information overload as they examine the development of FRS, namely in the e-
commerce industry. To illustrate the complexity of consumer behavior, a number of elements
that impact fashion choices are analyzed, including demography, location, and social
environment.【7:2】
The review divides FRS into several categories, including hybrid approaches, content-based
filtering, and collaborative filtering. The authors describe cutting-edge methods such as
recurrent neural networks (RNNs) and convolutional neural networks (CNNs), demonstrating
how they are used to extract textual and visual characteristics from fashion items. By examining
both explicit and implicit user feedback, these methods help to improve theaccuracy.【7:8】
The authors list a number of issues with the present FRS, such as the requirement for real-time
processing, cold-start issues, and data sparsity. To increase system resilience, they suggest
incorporating social media data and investigating hybrid methods. To improve personalization,
future studies could concentrate on improving deep learning algorithms and adding user-
generated information. The article ends by providing recommendations for creating FRS
models that are more successful based on the knowledge gained from their review.【7:32】

This article uses image-based neural networks to tackle the problem of tailored fashion
recommendations. This method concentrates on visual characteristics taken from input
photographs, as opposed to conventional systems that leverage user purchasing history. By
creating recommendations that closely correspond with each user's personal aesthetic tastes,
the authors hope to improve the user experience.【9:1】
In their study of current recommender systems, the authors draw attention to their limited visual
analysis and reliance on textual data. They talk about earlier attempts to use convolutional
neural networks (CNNs) to integrate visual features, but they point out that there aren't any
complete solutions that combine visual similarities with tailored recommendations【9:2】
.Their suggested system, which employs a nearest-neighbor strategy supported by deep
learning models, was developed in response to this gap.【9:3】

Methodologies:
E-commerce sites provided the dataset for HCRS, which was gathered with an emphasis on
product attributes and user reviews. The item-rating matrix and the group-rating matrix are the
two sorts of matrices that are constructed in order to build the system. While the group-rating
matrix depicts item groups based on common product attributes like color, texture, and style,
the item-rating matrix records user interactions in the form of specific ratings. The authors used
a fuzzy K-means algorithm to improve their clustering mechanism in order to solve the sparsity
problem that frequently arises in collaborative filtering systems. This approach improves the
representation of overlapping item qualities and user preferences by enabling soft clustering,
in contrast to normal K-means, where an item can belong to numerous groups.【2:3-4】
HCRS incorporates a complex visual analysis pipeline that uses color extraction and histogram-
based approaches to increase recommendation accuracy. Support Vector Machines (SVMs) for
object categorization in conjunction with Histogram of Oriented Gradients (HOG) are used for
human detection on garment photos. Dominant colors are retrieved and quantified into vector
representations when the garment region has been identified. By including these vectors as
pseudo-ratings into the group-rating matrix, low-level visual input is efficiently converted into
high-level semantic information. By taking into account both objective product qualities and
subjective user ratings, this hybrid data representation improves the system's capacity to
forecast consumer preferences.【2:4-5】
Using both item-group and user-item interactions, the recommendation engine uses a cluster-
based collaborative filtering technique. The method finds the top-N nearest neighbors in the
expanded matrix and applies a weighted average deviation model to forecast ratings.. The
prediction for user iii on item kkk is computed using the following equation:

∑ ∈ 𝑤 , 𝑅 , −𝑅
𝑃, = 𝑅 +
∑ ∈ 𝑤 ,

where 𝑃 , is the predicted rating, 𝑅 is the average rating of item 𝑘, 𝜔 , is the weight
assigned to neighbor n, and N is the set of top-N neighbors. By combining collaborative
filtering and visual features, this approach guarantees more accurate predictions, which
improves MAE results and user satisfaction.【2:6】
44,000 photos of fashion products with labels for gender and article category make up the
dataset that Gharaei et al. used, which was obtained via Kaggle. To guarantee uniformity,
preprocessing entails scaling every image to a standard 224x224 pixel size and normalizing
pixel values to the [0,1] range. Additionally, to improve model resilience and avoid overfitting,
augmentation techniques including rotation, flipping, and cropping are used. Using
stratification, the dataset is divided into 80% for training and 20% for testing, guaranteeing
equal representation in each category.【4:4】

A 13-layer convolutional neural network (CNN) with convolutional, pooling, and fully
connected layers serves as the foundation for the suggested model. While max-pooling layers
minimize feature map dimensions while preserving important information, each convolutional
layer uses a 4x4 kernel with ReLU activation to extract spatial features. For binary
classification applications like gender detection, a final sigmoid layer is employed.
Training utilizes a categorical focal loss function defined by:

𝐹𝐿(𝑃𝑡) = −𝛼 (1 − 𝑃𝑡) 𝑙𝑜𝑔 𝑙𝑜𝑔 (𝑃𝑡)

where γ modifies the emphasis on challenging cases and 𝑎 serves as a balance element. For
adaptive gradient descent, the Nadam optimizer is used to improve stability and convergence
speed. This training regime streamlines the recommendation process by allowing the network
to do feature extraction and classification at the same time.【4:5-6】
The recommendation system matches user-uploaded photos with database entries by using
cosine similarity. Feature vectors extracted from the CNN are compared using the following
formula:
𝐴⋅𝐵
𝑐𝑜𝑠 𝑐𝑜𝑠 (𝜃) =
‖𝐴‖‖𝐵‖

where A and B represent the feature vectors of the product image and user input, respectively.
The algorithm creates top-N recommendations that are both visually and contextually
appropriate for the user by rating items according to similarity scores. This approach solves the
cold-start issue for new products by guaranteeing quick, scalable, and highly accurate
recommendations.【4:6-7】

In order to fully capture user preferences, Lei et al. suggest a model that incorporates both
visual and non-visual contextual variables, such as brand, color, and texture. Items and users
are represented as vectors, denoted as 𝐼 = 𝐼 , 𝐼 ,⋯ , 𝐼 and 𝑈 = {𝑈 , 𝑈 , ⋯ ⋯ 𝑈 },
respectively. Visual attributes are obtained using a CNN model that has already been trained,
whereas non-visual features are taken from metadata.. This combination makes it possible for
the system to forecast matching scores according to visual and contextual significance, which
is crucial for suggestions that are based on aesthetics.【5:4-5】
To arrange items into clusters with comparable visual and contextual characteristics, the system
uses hierarchical clustering. Cosine similarity is used to determine the degree of matching
between a candidate item and previously purchased items; temporal relevance is taken into
account by adjusting for decay:

𝑀(𝑖, 𝑗) = 𝛿 ⋅ 𝑠𝑖𝑚 𝐼 , 𝐼 + (1 − 𝛿)𝑠𝑖𝑚 𝐶 , 𝐶

Where M(i,j) is the matching score, δ controls the weighting between visual and non-visual
features, and sim represents similarity functions. By lowering computational cost, this
clustering-based method allows for effective large-scale matching without sacrificing
precision.【5:5-6】
Using the following predictive model, the recommendation score is calculated by combining
user preferences and matching strength:

𝛾∗ = 𝑀(𝐶, 𝐼 ) ⋅ 𝛾
∈ ( )

where 𝛾 ∗ is the final recommendation score, 𝑀(𝐶, 𝐼 ) is the matching strength between cluster
C and item 𝐼 , and 𝑁(𝐼 ) denotes the neighborhood set of 𝐼 . This hybrid model provides
context-aware, tailored recommendations that greatly increase user satisfaction and purchase
likelihood by striking a balance between preference and matching.【5:6-7】

In order to capture a variety of datasets from user interactions, including purchase histories,
social media activity, and product reviews, the article describes a multi-phase data collection
approach. To extract valuable information like item attributes and user preferences, these
datasets undergo preprocessing. To create strong user profiles, explicit feedback—such as
product ratings—and implicit feedback—such as browsing habits—are combined.【7:4】
Resizing, normalization, and feature extraction using deep learning models are examples of
preprocessing for image-based data. Techniques like Term Frequency-Inverse Document
Frequency (TF-IDF) are frequently used to tokenize and vectorize text data.【7:5】
A variety of recommendation algorithms are trained using these preprocessed datasets.
A number of algorithmic models that are suited to various input data types are examined.
Convolutional Neural Networks (CNNs) are employed for feature extraction and image
classification.. A typical CNN model comprises multiple convolutional layers followed by
pooling layers, where the activation function f(x) is often ReLU (Rectified Linear Unit):

𝑓(𝑥) = (0, 𝑥)

To create item embeddings, the output is processed through completely connected layers.
Temporal dependencies in user interactions are captured by applying Recurrent Neural
Networks (RNNs) to sequential data. The network can maintain long-term dependencies thanks
to the Long Short-Term Memory (LSTM) variation, which solves the vanishing gradient issue:

ℎ =𝑡𝑎𝑛 𝑡𝑎𝑛 ℎ(𝑤 ℎ +𝑤 𝑥 +𝑏 )

where ht represents the hidden state at time t, and 𝑤 , 𝑤 , and 𝑏 are learnable parameters.
【7:11】
The study places a strong emphasis on using a variety of evaluation measures to gauge model
effectiveness.
F1 Score, Precision, Recall, and Root Mean Square Error (RMSE) are examples of common
metrics. RMSE is calculated as:

1
𝑅𝑀𝑆𝐸 = (𝑃 − 𝛾 )
𝑁

where N is the number of predictions, 𝑃 is the predicted rating, and 𝛾 is the actual rating
【7:7】
The authors extract features from fashion photos using a pre-trained ResNet50 model. To
modify this model for the particular objective of fashion recommendation, transfer learning is
used. During the training phase, the network is adjusted by swapping out the last layers to better
fit the classification objective. Convolutional layers are used in the neural network's design to
extract features, while fully linked layers are used for classification:

𝑂𝑢𝑡𝑝𝑢𝑡 = 𝑆𝑜𝑓𝑡(𝑤 × 𝑥 + 𝑏)

where x is the feature vector, and W and b are the weight matrix and bias, respectively【9:3】

Conclusion:
To sum up, our hybrid recommender system delivers notable improvements in how customers
purchase apparel and other goods from a variety of businesses. The system offers highly
customized recommendations that adjust in real time to user preferences by fusing
collaborative and content-based filtering methods with an easy-to-use swipe-based interface.
This method presents products that closely match changing tastes and fashions, which not only
makes shopping easier but also increases consumer pleasure.
Through the analysis of interaction patterns and preferences, the system provides vendors with
profound insights into buyer behavior. Retailers can better anticipate trends, customize
marketing campaigns, and optimize inventory thanks to this data-driven insight. Businesses
may increase client retention and boost revenue by utilizing these insights to develop more
tailored and interesting shopping experiences. In the end, this technology creates a more
responsive and dynamic marketplace that helps both consumers and sellers by bridging the gap
between retail offerings and consumer expectations.

References:

【1】Anjan M, Abhishek V, C. Balamanikantan, Dr. Venugeetha Y. Fashion recommendation

system using CNN. Don Bosco Institute of Technology, Bengaluru, Karnataka.
【2】Xiaosong Hu, Wen Zhu, Qing Li. HCRS: A hybrid clothes recommender system based
on user ratings and product features. Sch of Economic Inf. Eng, Southwestern Univ. of Finance
& Econ, Chengdu, China.

【3】Pasquale Lops, Marco de Gemmis, Giovanni Semeraro. Content-based Recommender

Systems: State of the Art and Trends. Department of Computer Science, University of Bari
“Aldo Moro”, Via E. Orabona, 4, Bari (Italy).

【4】Narges Yarahmadi Gharaei, Chitra Dadkha, Lorence Daryoush. Content-based Clothing

Recommender System using Deep Neural Network, Faculty of Computer Engineering, K. N.
Toosi university of Technology Tehran, Iran.

【5】Yan Lei , Long Chen and Ziyu Guan. Cloth Recommender System Based on Item
Matching. Northwest university, Xi’an, China.

【6】Ying Huang, Tao Huang. Outfit Recommendation System Based on Deep Learning.
School of Information Engineering, Wuhan University of Technology, Wuhan 430070, China.

【7】Samit Chakraborty, Edgar Lobaton. Fashion Recommendation Systems, Models and

Methods: A Review. Department of Electrical and Computer Engineering, North Carolina State
University, Raleigh, NC 27695, USA.

【8】Yu Liu, Jingwen Nie, Lexi Xu, Yue Chen. Clothing Recommendation System Based
on Advanced User-Based Collaborative Filtering Algorithm. Beijing University of Posts and
Telecommunications and China Unicom Network Technology Research Institute, Beijing
100048, China.

【9】M Sridevi, N ManikyaArun. Personalized fashion recommender system with image

based neural networks. Department of Computer Science Engineering,Anurag Group of
Institutions, Hyderabad, Telangana, India.

【10】Muhammad Khiyarus Syiam, Agung Toto Wibowo, Erwin Budi Setiawan. Fashion
Recommendation System Using Collaborative Filtering. School of Computing, Informatics,
Telkom University, Bandung, Indonesia.