© 2021 JETIR December 2021, Volume 8, Issue 12 www.jetir.

org (ISSN-2349-5162)

A Machine Learning Based Chatbot Song

Recommender System
Guide Name: Prof. N.S.Sharma

1] First Author Name:-Amaan Shaikh Address:-Jalgaon, Maharashtra, India

College Name:-Sinhgad Academy of Engineering, Email: - [email protected]

Pune
3] Third Author Name:-Tejas Sonawane
Address:-Solapur, Maharashtra, India
College Name: - Sinhgad Academy of Engineering,
Email:[email protected] Pune

2] Second Author Name:-Bhushan Patil Address:-Jalgaon, Maharashtra, India

College Name: - Sinhgad Academy of Engineering, Email: - [email protected]

Pune

Abstract: - Music is an integral part of our lives. However, Playlist generation, IBM Tone Analyzer API, Last.fm API,
since the social media platforms like TikTok and Instagram CakeChat server.
have a huge influence on the music charts worldwide, users
are exposed solely to mainstream music, therefore the Introduction: - Everyday, each and every person undergoes
recommendations on music streaming platforms are not very lot of troubles and the reliever of all the stress that are
personalized. An song and emotion-based recommendation encountered is Music. If it is so, the vital part of hearing the
system permits the users to listen to music based on their song has to be in a facilitated way, that is player able to play
emotions. Existing systems use audio signals using the CNN the song in accordance to the person’s mood. The paper
approach and collaborative filtering to recommend songs proposes such a player and hence named Emotion based
based on the user's history. The proposed research work music player. Recognizing the human emotions is
develops a personalized system, where the user's current considered to be a global consistency but depict variability
emotion is analyzed with the help of the chatbot. The among the humans on the basis of their abilities. The
chatbot identifies the user's sentiment by asking some different approaches being the classification of emotions that
general questions. Based on the input provided by the user, are existing are depicted as knowledge-based, statistical and
current emotion or mood is analyzed by the chatbot and it hybrid techniques. However there relies several difficulties
will generate the playlist. The proposed recommendation while retrieving the music information such as querying by
system utilizes the APIs for the playlist generation and singing, genre classification, etc. The most possible
recommendation. implementation is by producing music suggestion that is
based on the content. The multiple efforts to detect and
describe the feeling, to classify based on features are all
wanted to obtain outstanding music recommendation
Keyword:- Interactive Chatbot, Application Program system. For describing a music taxonomy, a feeling
Interface, Interactive Systems, Recommender Systems, descriptor is found to be useful.

JETIR2112328 Journal of Emerging Technologies and Innovative Research (JETIR) www.jetir.org d222
© 2021 JETIR December 2021, Volume 8, Issue 12 www.jetir.org (ISSN-2349-5162)

In this project, we would be building an extensive recommendation, also other similar user opinions are
Chatbot service, to which you can talk to. And talking to a considered. The design of the proposed system, its
chatbot wouldn't be business-driven. It would just be casual implementation along with its evaluation procedure is also
conversations. Further, on top of it, the chatbot would also discussed. In order to relate emotions to movies, the users
be recommending songs to the user based on the tone of the have to answer a questionnaire about what movies or which
user. This song recommendation feature employs the use of categories of movies they liked to watch according to each
Last.fm API, very much similar to the popular Spotify API. emotion. Furthermore, the system captures user emotions by
Also for tone/emotion analysis of the conversation we will asking them to use 3 colours to decorate their avatar
be using the IBM Tone Analyzer API. Collaborating with
these types of APIs is very much critical as in today's world
the popular chatbots do much more than simply having a
3. Jae Sik Lee et al. [3] have used the concept of context
data-driven conversation; to supplement additional user-
reasoning wherein the context data is utilized to understand
oriented features. Also the reason to choose python to build
the user’s situation. They propose a music recommendation
the chatbot is because python boasts a wide array of open-
system that comprises the ability of context reasoning in this
source libraries for chatbots, including scikit-learn and
paper. Their proposed system contains modules such as
TensorFlow. It is great for small data sets and more simple
Intention Module, Mood Module and Recommendation
analyses; also Python's libraries are much more practical.
Module each of which provide a unique functionality to the
system and play a vital role for the system’s performance as
a whole. Context reasoning is done by the Intention Module
Literature Survey: - Few of the key features emphasized with the help of environmental context data and concludes
by the papers that have been surveyed are: whether the user is interested in listening to music or not.
Next, the type of music that is deemed to be most
1. Nikhil et al. [1] use algorithms and technologies which appropriate to the user’s context is determined by the Mood
include Haar cascade, Canny edge, Blob detection for the Module. Lastly, the music is recommended to the user by
process of emotion detection. The system captures pictures the Recommendation Module
of the user and according to that mood gets detected. Inputs
like face and emotions are taken from the picture, and the 4. Renuka R. Londhe et al. in [4]have studied the concept of
system also provides a chat box to give responses. The recognizing facial expressions by taking into account the
proposed system in the paper presents a new approach for various properties that are associated with a person’s face.
building desktop application for chat bot using text and Whenever there is a change in the facial expression, changes
gestures. The system is able to make a conversation through can be noticed in the curvatures on the face as well as
the chatting application. The system will send some links, features of the face such as nose, lips, eyebrows and mouth
web pages or information depending on the response from area. And accordingly, there will be changes in the intensity
the user. The system detects smile and stress. When a smile of the corresponding pixels of the images. These features are
is detected by the system, jokes pop-ups will be shown on then classified into six expressions which include anger,
the screen, and when stress is detected, inspirational quotes disgust, fear, happy, sad and surprise with the help of
pop-ups will be shown on the screen. Also, happy songs are artificial neural network. The Scaled Conjugate Gradient
played when a smile is detected. And similarly, inspirational back-propagation algorithm is used to train and test the two-
songs are played when stress is detected. layered feed forward neural network. They acquired a 92.2
% recognition rate. Here, they have made use of the JAFFE
database which consists of seven expressions for analysis
through the computer.
2. Ai Thanh Ho et al. in their paper [2], introduce an
Emotion-based Movie Recommender System (E-MRS)
which is intended to solve the problem that the conventional
system of user profile does not take into consideration how 5. Dolly Reney et al. in their paper [5] address the
important user’s emotions are and how they affect user’s importance of face and emotion identification in the field of
choices, which the recommender systems are unable to security and how it helps give solutions to the different
understand and capture the constantly changing preferences challenges faced. Database plays a major role when
of user. According to the paper, the objective of EMRS is to comparing the facial attributes and sound Mel frequency
give the users a list of suggestions that are customized using components, when it comes to whichever face and emotion
a combination of collaborative filtering and content-based identification system. The database is created for which
techniques. Here the user’s emotions as well as his facial characteristics are computed and these are then stored
preferences are taken into account when providing a in the database. Various algorithms are used in order to
JETIR2112328 Journal of Emerging Technologies and Innovative Research (JETIR) www.jetir.org d223
© 2021 JETIR December 2021, Volume 8, Issue 12 www.jetir.org (ISSN-2349-5162)

analyze the face and emotion with the help of the  Nearest Neighborhood Model:
aforementioned database. The implementation of the process
of recognizing the person’s face and the emotion being Nearest Neighborhood model involves collecting
expressed by him uses an effective method for the creation data from numerous clients at that point making
of a database comprising the facial expressions and emotion. forecasts according to the similarity measures
They have used the Viola-Jones algorithm for the face between users and between items.
identification process and the face and emotion This might be grouped into client-based and item
identification is evaluated by the KNN classifier. based models. In item-based model, it's expected
that songs that are frequently listened together by
certain clients will in general be indistinguishable
and are bound to be listened together in future
Architecture:-
additionally by another client.
According to client based similarity model, clients
who have similar listening histories, i.e., have
listened in to similar songs inside the past will in
general have comparative interests and may most
likely hear similar songs in future as well.

Advantages:-

 Chat-bots have 24/7 Availability: Chat-bots are

available to solve customer problems 24/7 whether
Algorithms:- it is day or night! They don’t need to sleep after all!
This is much more difficult to achieve using human
 SVD (Singular Value Decomposition) customer service as it would require rotating teams
Algorithm: that would be more complicated to manage as well.
This means that chat-bots can answer customer
SVD could also be a matrix factorization technique
queries whenever customers have queries which
that is typically wont to diminish the quantity of
help in increasing customer loyalty. If companies
feature of a dataset by reducing the matrix from N
don’t address customer problems as soon as
space to K space where K < N.
possible, then the customers may switch to another
company that provides better customer service.
For the point of the guidance framework be that as
it may, we are just interested about by the matrix
 Chat-bots cause an Increase in Sales: Customers
factorization part keeping same dimensional.
can get instant support and solutions for their
The matrix factorization is completed on the user- problems using chat-bots while using agents takes a
item ratings matrix built. longer time. When customers get such good
service, chances are high that they will be more
Each item is often represented with a q vector. loyal to the company which will drive up sales.
Similarly, each client are often represented by a p Companies can also use chat-bots to market their
vector such the inner product of these 2 vectors is latest products directly to their customers. When
that the expected rating. Find p and q such it customers get recommendations for a product while
minimizes the following: chatting to a chat-bot, it is much more personal
than a mail or a random ad and they are more likely
Figure:-
to buy the product. So chat-bots can increase sales
in this manner as well.

 Chat-bots provide Long-term Financial Savings:

Companies that have human customer support
teams have to pay a large amount of money as
salaries to their employees. And if the company is

JETIR2112328 Journal of Emerging Technologies and Innovative Research (JETIR) www.jetir.org d224
© 2021 JETIR December 2021, Volume 8, Issue 12 www.jetir.org (ISSN-2349-5162)

large then their expenditure will be equally large as Conclusion:-

well! Here, a chat-bot can reduce this expenditure
so that the company has long-term financial We have presented a survey and methodology for
savings. While it is true that initially, a company building the chatbot song recommender system. To
has to invest a lot of time and money in creating a perform this, we first identified various approaches
for building a chatbot known to date. We then
chat-bot, after some time the chat-bot will be able
evaluated the considered algorithms which are
to answer all the basic customer queries that would
useful in building of our system in terms of their
have taken a lot of time for humans. In addition to
ability to work on the recommendation process of
that, the chat-bot can give instantaneous replies
the system. We also gathered all the requirements
which saves time. And time is money in business!
needed for building our system and studied the
overall process involved in chatbot's working.
Lastly we summarized the deployment
Disadvantages:-
requirements of our system. On the conclusion
 Chat-bots can only handle basic questions: Chat- note our ‘Chatbot Song Recommender System’ is
bots are still a basic Artificial Intelligence used to facilitate the use by physically challenged
technology and so they can only answer the basic people to automate and give them better music
questions of customers and provide general player experience. The application solves the basic
needs of music listeners without troubling them as
information that is already available to them. They
existing applications do.
cannot solve complicated queries or answer out of
script questions and companies need to have human
customer service employees that can manage these
for them. However, this is changing with time and
currently, more and more advanced chat-bots are
entering the market.

 Chat-bots are difficult to create: Chat-bots are

created using Natural Language Processing which
is extremely popular for customer support
applications. Natural Language Processing is a part
of Machine Learning which can be used to interact
with the users in textual form and solve their
queries. However, this requires complex
programming and is not easy for companies. This Average accuracy of various emotions
becomes especially difficult if companies have to
Reference:-
create chat-bots from scratch and that is why many
online platforms help companies to build and [1] Anagha S.Dhavalikar, and Dr. R. K. Kulkarni, “Face
manage chat-bots easily.
Detection and Facial Expression Recognition System,”
 Chat-bots require constant maintenance: Interntional Conference on Electronics and Communication
Companies cannot just create a chat-bot and then System (ICECS - 2014), 2014.
leave it hoping that it will correctly answer
customer inquiries forever! The company products [2] Yong-Hwan Lee, Woori Han and Youngseop Kim,
change with time and more advanced Natural “Emotional Recognition from Facial Expression Analysis
Language Processing capabilities are also
using Bezier Curve Fitting,” 16th International Conference
developed with time. All of these changes need to
be programmed into the chat-bot so that it has the on Network-Based Information Systems, 2013.
most up to date information. In addition to that,
chat-bots also need to be periodically analyzed so [3] Arto Lehtiniemi and Jukka Holm, “Using Animated
that the most common questions that customers Mood Pictures in Music Recommendation,” 16th
have can be identified and then their answers International Conference on Information Visualisation,
updated for future customers.
2012.

JETIR2112328 Journal of Emerging Technologies and Innovative Research (JETIR) www.jetir.org d225
© 2021 JETIR December 2021, Volume 8, Issue 12 www.jetir.org (ISSN-2349-5162)

[4] F. Abdat, C. Maaoui and A. Pruski, “Human-computer [9] K. Lu and Y. Jia, “Audio-Visual Emotion Recognition
interaction using emotion recognition from facial With Boosted Coupled HMM,” no. Icpr, pp. 1148–1151,
expression,” UKSim 5th European Symposium on Computer 2012.
Modelling and Simulation, 2011.
[10]G.T. Shrivakshan, and C. Chandrasekar. “A comparison
[5] T.-H. Wang and J.-J.J. Lien, “Facial Expression of various edge detection techniques used in image
Recognition System Based on Rigid and Non-Rigid Motion processing,” International Journal of Computer Science
Separation and 3D PoseEstimation,” J. Pattern Recognition, Issues (IJCSI), vol. 9(5), pp. 272-276, 2012.
vol. 42, no. 5, pp. 962-977, 2009.
[11] S. Rani, D. Bansal, and B. Kaur, “Detection of edges
using mathematical morphological operators”, Open
Transactions on Information Processing, vol. 1(1), pp. 17-
[6] Renuka R. Londhe, Dr. Vrushshen P. Pawar, “Analysis
26, 2014.
of Facial Expression and Recognition Based On Statistical
Approach,” International Journal of Soft Computing and [12] R. C. Gonzalez and R. E. Woods, “Digital Image
Engineering (IJSCE) vol. 2, May 2012. Processing”, Pearson Prentice Hall, 2008.

[7] S. Yang and B. Bhanu, “Facial Expression Recognition [13] Z. Qiyuan, H. Xianxiang, T. Lilong, and Z. Bing, “A
Using method of deleting noise in a binary image based on the
mathematical morphology,” 8th International Conference on
Emotion Avatar Image,” pp. 866–871, 2011.
Electronic Measurement and Instruments, pp. 2-787 – 2-790,

[8] M. Bejani, D. Gharavian, and N. M. Charkari, 2007.

“Audiovisual emotion recognition using ANOVA feature
selection method and multiclassifier neural networks,”
Neural Comput. Appl., vol. 24, no. 2, pp. 399–412, 2014.

JETIR2112328 Journal of Emerging Technologies and Innovative Research (JETIR) www.jetir.org d226