ISSN (Online) 2581-9429

IJARSCT
International Journal of Advanced Research in Science, Communication and Technology (IJARSCT)

Volume 3, Issue 1, April 2023

Impact Factor: 7.301

Facial Expression Based Music Recommendation

System
Abutalib K1, Amandeep Gautam2, Aryan Khan3, Manish Tiwari4, Aditya Dayal Tyagi5
Students, Department of Computer Science and Engineering1,2,3,4
Assistant Professor, Department of Computer Science and Engineering5
ITS Engineering College, Greater Noida, India

Abstract: The human face is a crucial organ for conveying an individual's emotional state and behavior.
However, manually creating a playlist based on an individual's emotional features can be a labor-intensive
and time-consuming task. To automate this process, several algorithms have been proposed, but they are
often slow and inaccurate. To address this, a new system is proposed that utilizes facial expression
extraction to generate an appropriate playlist automatically. This system can significantly reduce the
computational time and overall cost of playlist generation while increasing accuracy. The system captures
facial expressions using an inbuilt camera, and the emotion detection algorithm used has an accuracy of
approximately 85-90% for real-time images and 98-100% for static images. By leveraging this high level of
accuracy and performance, the proposed system outperforms existing algorithms used in the literature
survey. Based on the detected emotion, the system creates a playlist that matches the individual's emotional
state. This novel approach offers a more efficient and accurate way to generate personalized playlists,
ultimately saving time and effort for users.

Keywords: Music suggestion, Facial Recognition, SVM, OpenCV, Python

I. INTRODUCTION
Humans have a natural ability to gauge someone's mood by looking at their face. If electronic devices could learn this
skill, it could have valuable real-world applications. Music, as a tool for evoking emotions and feelings, is more
powerful than language. It has the ability to tap deeply into our emotional core, and listening to good music can help
elevate our mood.
To simplify this problem, a proposed solution is to create an application that captures the user's emotions through facial
expression recognition algorithms. Once the emotion is captured, a list of songs is suggested based on that emotion.
This approach offers a convenient way for individuals to select music that aligns with their emotional state, potentially
leading to improved emotional wellbeing.
Existing System: The current recommendation systems available in the market include Spotify, Musixmatch, Shazam,
and others.

1.1 Existing System

Existing recommendation systems such as Spotify, Musix match, Shazam,etc.

Copyright to IJARSCT DOI: 10.48175/IJARSCT-9046 316

www.ijarsct.co.in
 ISSN (Online) 2581-9429
IJARSCT
International Journal of Advanced Research in Science, Communication and Technology (IJARSCT)

Volume 3, Issue 1, April 2023

Impact Factor: 7.301

1.2 Problems of the Existing System

 Current recommendation systems offer a static user experience, as they provide recommendations based solely
on historical data, without taking into account other parameters that may impact predictions, such as emotions
or feelings.
 These systems may also fail to provide accurate recommendations if their suggestions are based on outdated
input.
 Consequently, users may be dissatisfied with the system's output, as it fails to satisfy their emotional state.

III. PROPOSED SYSTEM

2.1 Benefits of the Proposed System

 These systems utilize facial recognition to determine the user's mood and select songs for creating a playlist that
reflects the user's current emotional state.
 The aim of this approach is to improve the user's mood by suggesting music that aligns with their emotional
state.
 Compared to traditional static recommendation systems, this approach is superior as it takes into account the
user's emotions and helps to enhance their mood by suggesting appropriate music.

2.2 Methods
The software requirements for the proposed system include React JS for the front-end, Node JS and Firebase for the
back-end. The system is compatible with Windows 10 or Windows 8 operating systems.
The hardware requirements for the system include a computer or laptop with a 32- or 64-bit operating system, an x-64-
bit processor, and 4 or 8 GB RAM. The recommended processor for optimal performance is either an Intel i3, i5, or i7.

Copyright to IJARSCT DOI: 10.48175/IJARSCT-9046 317

www.ijarsct.co.in
 ISSN (Online) 2581-9429
IJARSCT
International Journal of Advanced Research in Science, Communication and Technology (IJARSCT)

Volume 3, Issue 1, April 2023

Impact Factor: 7.301

III. LITERATURE SURVEY

The power of music to evoke emotions and influence mood has been well-established. Unlike other forms of media,
music can quickly affect our emotional state, both positively and negatively. However, manually classifying songs
based on mood to create playlists can be time-consuming and laborious. To address this challenge, our research
proposes an Android application called EmoPlayer, which utilizes facial expression recognition to suggest a playlist of
songs that align with the user's current emotion. By capturing the user's facial image through the camera, the system
detects emotions using the Viola Jones algorithm for face detection and Fisher faces classifier for emotion
classification. The proposed system not only plays music based on the recognized emotion but also suggests music that
enhances the user's mood.
Our research highlights the importance of automated systems that can accurately recognize emotions and provide
mood-based music recommendations. EmoPlayer integrates an emotion module, a music classification module, and a
recommendation module to suggest songs that match the user's mood. The emotion module utilizes deep learning
algorithms to identify the user's emotion with an accuracy of 90.23%. The music classification module uses audio
features to classify songs into four different mood classes with a remarkable accuracy of 97.69%. The recommendation
module then maps the user's emotion to the mood type of the song and suggests appropriate songs to enhance their
mood.
Our research aims to improve the accuracy of emotion recognition and expand the range of recognized emotions. We
also plan to develop an automatic music genre classification system and an automatic music emotion classification
system to automate the existing system of emotion-based mood-enhancing music recommendations further. In
summary, our research demonstrates the potential of automated emotion-based music recommendation systems to
enhance users' mood and provide a personalized music experience. The first paper discusses a music recommendation
system based on facial expressions. The system uses a convolutional neural network to recognize facial expressions and
recommend music based on the corresponding emotion. The study shows promising results in recognizing four moods -
happy, calm, sad, and angry - with an accuracy of 97.69%. The authors suggest expanding the study to include all seven
basic emotions and adding songs from different languages and regions to make the system more robust. They also
propose using collaborative filtering to collect user preferences and improve the overall system.
The second paper presents a music recommendation system based on a sentiment intensity metric, enhanced Sentiment
Metric (eSM). The system uses a lexicon-based sentiment metric with a correction factor based on the user's profile.
The authors conducted subjective tests in a laboratory environment to obtain the correction factor, which depends on
age, educational level, and gender. The study shows that the eSM outperforms a randomly assigned song suggestion,
reaching 91% user satisfaction in remote subjective tests. The authors also found that 78% of users preferred to listen to
a musical genre similar to their current emotional state, while only 22% preferred a different genre. The paper
highlights the importance of considering the user's profile in a sentiment metric and suggests applying the weight factor
used in the eSM to other sentiment metrics.

3.1 Module Specification

We have divided our project in to 2 modules. They are:
1. Facial Expression Recognition Module.
2. Music Suggestion module

Copyright to IJARSCT DOI: 10.48175/IJARSCT-9046 318

www.ijarsct.co.in
 ISSN (Online) 2581-9429
IJARSCT
International Journal of Advanced Research in Science, Communication and Technology (IJARSCT)

Volume 3, Issue 1, April 2023

Impact Factor: 7.301
A. Facial Expression Recognition Module
This module is designed to detect the emotions of a user. We have developed a model using the SVM supervised
machine learning algorithm. When an image is uploaded, we use Python's OpenCV library to extract its features. These
features are then applied to the SVM training model to predict the emotion of the user.

B. Music Suggestion Module

This module utilizes the output obtained from the Facial Expression Recognition module as its input. The emotion
detected from the user's facial expression is used to suggest music. A list of music that is related to the user's emotion is
displayed and the user is able to select the music and play it.

IV. ARCHITECTURAL DESIGN

Requirements of the software should be transformed into an architecture that describes the software's top-level structure
and identifies its components This is accomplished through architectural design(also called system design),which acts
as a preliminary 'blueprint' from which software can be developed IEEE defines architectural design as 'the process of
defining a collection of hardware and software components and their interfaces .
The software requirements document and designing a model for providing implementation details. These details are
used to specify the components of the system along with their inputs, outputs, functions, and the interaction
between them. An architectural design performs the following functions:
1. It defines an abstraction level at which the designers can specify the functional and performance behaviour of
the system.
2. It acts as a guideline for enhancing the system(whenever required) by describing those features of the system
that can be modified easily without affecting the system integrity.
3. It evaluates all top-level designs.
4. It develops and documents top-level design for the external and internal interfaces.
5. It develops preliminary versions of user documentation.
6. It defines and documents preliminary test requirements and the schedule for software integration.
7. The sources of architectural design are listed below.
8. Information regarding the application domain for the software to be developed.
9. Using data-flow diagrams.
10. Availability of architectural patterns and architectural styles.

4.1 Architectural
The sources of architectural design include information regarding the application domain, data-flow diagrams,
availability of architectural patterns, and architectural styles. This step is crucial in software engineering as it deals with
the essential requirements like reliability, cost, and performance. However, the paradigm shift from monolithic, stand-
alone, built-from-scratch systems to componentized, evolvable, standards-based, and product-line-oriented systems
makes this task cumbersome. To avoid these problems, designers adopt strategies such as reusability,
componentization, platform-based, standards-based, and so on.
In this project, the application's architectural view is shown in the above figure, which uses the Tkinter module,

Copyright to IJARSCT DOI: 10.48175/IJARSCT-9046 319

www.ijarsct.co.in
 ISSN (Online) 2581-9429
IJARSCT
International Journal of Advanced Research in Science, Communication and Technology (IJARSCT)

Volume 3, Issue 1, April 2023

Impact Factor: 7.301
OpenCV, SVM algorithm, and storage as a database.
The application is developed using Python 3.7, and all the screens are displayed according to the specified modules.
The software architecture is designed to ensure that all stakeholders are involved in the review process to minimize
risks and errors.

4.2 Design Concepts for Our Project

The software exhibits several design principles such as abstraction, modularity, and information hiding.
User data is abstracted from the interface through password protection, ensuring that only authorized users can access
the information.
The project also adheres to a modular design approach by breaking down the system into distinct modules, making it
easier to develop and maintain. Additionally, information hiding is employed to ensure that only the user with the
password can view the protected file.
The software also demonstrates a well-defined architecture by following program modules and maintaining conceptual
integrity throughout the system.

4.3 Algorithm Design

Step 1: Begin.
Step 2: Launch the Music Suggestion application.
Step 3: Prompt the user to upload an image containing a face.
Step 4: Pre-process the image and apply emotion detection to identify the emotion displayed on the face.
Step 5: Based on the identified emotion, display a list of songs that are suitable for that particular emotion.
Step 6: Allow the user to select a song from the list of suggestions.
Step 7: Stop the Music Suggestion application

4.4 Data Flow Process

Copyright to IJARSCT DOI: 10.48175/IJARSCT-9046 320

www.ijarsct.co.in
 ISSN (Online) 2581-9429
IJARSCT
International Journal of Advanced Research in Science, Communication and Technology (IJARSCT)

Volume 3, Issue 1, April 2023

Impact Factor: 7.301
V. RESULTS
User Interface

Upload an Image

Image is uploaded

Copyright to IJARSCT DOI: 10.48175/IJARSCT-9046 321

www.ijarsct.co.in
 ISSN (Online) 2581-9429
IJARSCT
International Journal of Advanced Research in Science, Communication and Technology (IJARSCT)

Volume 3, Issue 1, April 2023

Impact Factor: 7.301
Pre-processing Image & Detecting number of faces

Detecting Emotion

Select a song from the list based on the emotion

Copyright to IJARSCT DOI: 10.48175/IJARSCT-9046 322

www.ijarsct.co.in
 ISSN (Online) 2581-9429
IJARSCT
International Journal of Advanced Research in Science, Communication and Technology (IJARSCT)

Volume 3, Issue 1, April 2023

Impact Factor: 7.301
Play the Song

VI. USER MANUAL

To use this application, follow the steps below:
1. Download and install the application.
2. Use the webcam to capture an image containing your face before opening the application.
3. Open the application and upload the image you captured.
4. Perform the "Pre-process and detect faces" function, which will detect the number of faces in the image and
display a message.
5. Perform the "Detect Emotion" function to detect your emotion. A message box will display the detected
emotion.
6. A list of songs based on the detected emotion will appear in the song list field. Choose a song from the list and
click the play button to play the song.
7. That's it! You can repeat the process with a different image to detect and play songs based on the emotions
detected.

VII. CONCLUSION
This project aimed to create a model that recommends music based on the emotion detected from facial expressions.
An emotion-based music recommendation system was designed and developed using a face recognition system.
Music has the power to heal any stress or emotion, and this project capitalizes on recent developments in emotion-based
music recommendation systems.
The proposed system utilizes a face-based emotion recognition system to detect emotions and plays music accordingly.

ACKNOWLEDGEMENT
Our Sincere thanks to Department of Computer Science and Engineering, ITS engineering college for the guidance
provided.

BIBLIOGRAPHY
[1]. https://towardsdatascience.com/face-detection-recognition-and-emotion-detection-in-8-lines-of-code-
b2ce32d4d5de
[2]. https://medium.com/@hinasharma19se/facial-expressions-recognition-b022318d842a
[3]. https://www.geeksforgeeks.org/introduction-to-support-vector-machines-svm/
[4]. https://www.javatpoint.com/machine-learning-support-vector-machine-algorithm
[5]. https://www.python.org/downloads/release/python-370/
[6]. https://www.kdnuggets.com/2018/10/top-python-machine-learning-libraries.html
Copyright to IJARSCT DOI: 10.48175/IJARSCT-9046 323
www.ijarsct.co.in
 ISSN (Online) 2581-9429
IJARSCT
International Journal of Advanced Research in Science, Communication and Technology (IJARSCT)

Volume 3, Issue 1, April 2023

Impact Factor: 7.301
[7]. https://medium.com/analytics-vidhya/emotion-based-music-recommendation-system-using-a-deep-
reinforcement-learning-approach-6d23a24d3044
[8]. https://www.jetir.org/papers/JETIR2004499.pdf
[9]. https://ieeexplore.ieee.org/document/8374807
[10]. https://www.geeksforgeeks.org/best-python-libraries-for-machine-learning/
[11]. https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/
[12]. Panwar, S., Roopaei, M.,Rad,P., Choo, K. (2019). Are you emotional or depressed?
Learningaboutyouremotionalstatefromyour musicusingmachinelearning.JournalOfComputing.Vol.75Issue6,
p2986-3009.24p.
[13]. Ashu A., Chen, J., Hua-Yuan, L., Shun-Hao C. (2018). An Emotion-Aware Personalized
MusicRecommendationSystemUsingaConvolutionalNeuralNetworksApproach.AppliedSciences;BaselVol.
8,Issue.7
[14]. Oramas, S., Nieto, O., Sordo, M.,Serra,X. (2017). A Deep Multimodal Approach for Cold-
startMusicRecommendation.ACMProceedingsofthe2ndWorkshopondeeplearningforrecommender
systems,2017-08-27, p.32-37
[15]. Yepes, Fabio A ; López, Vivian F ; Pérez-Marcos, Javier ; Gil, Ana B ; Villarrubia, Gabriel(2018). Listen to
This: Music Recommendation Based on One-Class Support Vector Machine.Cham: Springer International
Publishing Hybrid Artificial Intelligent Systems, 2018-06-08,p.467-478
[16]. Vall, Andreu ; Widmer, Gerhard (2019). MachineLearningApproaches to Hybrid
MusicRecommenderSystems.Cham:SpringerInternationalPublishingMachineLearningandKnowledgeDiscov
eryinDatabases,2019-01-18,p.639-642
[17]. Antal, D., Fletcher, A., & Ormosi, P. L. (2021, September 20). Music streaming: Is it A levelplaying field?
Retrieved November 20, 2021, fromhttps://www.competitionpolicyinternational.com/music-streaming-is-it-
a-level-playing-field/
[18]. Prosvetov, A. V. (2019). Gan for recommendation system. Journal of Physics: ConferenceSeries, 1405(1),
012005.doi:10.1088/1742-6596/1405/1/012005
[19]. XinxiWangandYeWang.2014.ImprovingContentbasedandHybridMusicRecommendationusingDeepLearning.
In Proceedings of the 22nd ACM international conference on Multimedia(MM '14). Association for
ComputingMachinery,New York,NY,USA,627–636.DOI:https://doi.org/10.1145/2647868.2654940
[20]. H.Zhang,H.Yang,T.HuangandG.Zhan,"DBNCF:PersonalizedCoursesRecommendationSystemBasedonDBNi
nMOOCEnvironment,"2017InternationalSymposiumonEducationalTechnology(ISET),2017,pp.106-
108,Doi:10.1109/ISET.2017.33
[21]. VandenOord,A.,Dieleman,S.,&Schrauwen,B.(2013).Deepcontent-basedmusicrecommendation.InC.Burges,
[22]. L.Bottou,M.Welling,Z.Ghahramani,&K.Weinberger(Eds.),AdvancesinNeuralInformation Processing
Systems26 (2013) (Vol. 26). Presented at the NeuralInformation Processing Systems Conference (NIPS
2013), Lake Tahoe,NV,USA:NeuralInformationProcessingSystems Foundation(NIPS).
[23]. Min Gaoa, Junwei Zhanga, Junliang Yuc , Jundong Lid, Junhao Wena, and Qingyu Xiong (2020).
RecommenderSystemsBasedonGenerativeAdversarialNetworks:AProblem-
DrivenPerspective,KeyLaboratoryofDependableService Computing in Cyber Physical Society (Chongqing
University), Ministry of Education, Chongqing, 401331,China
[24]. Shan, M.-K., Kuo, F.-F., Chiang, M.-F., & Lee, S.-Y. (2009). Emotion-based music recommendation by
affinitydiscoveryfromfilmmusic.ExpertSystemswithApplications,36(4),7666–7674.https://doi.org/10.1016/
j.eswa.2008.09.042
[25]. H.ImmanuelJames1,J.JamesAntoArnold2,J.MariaMasillaRuban3,M.Tamilarasan4,R.Saranya5(2013)Emotio
nbasedmusicrecommendationsysteme-ISSN:2395-0056, p-ISSN:2395-0072
[26]. MikhailRumiantcev,OleksiyKhriyenko,EmotionBasedMusicRecommendationSystem,ISSN2305-7254
[27]. Heshmat,S.,Ph.D.(2019,August25).Music,emotion,andwell-being.RetrievedNovember21,2021,fromhttps://
www.psychologytoday.com/us/blog/science-choice/201908/music-emotion-and-well-being

Copyright to IJARSCT DOI: 10.48175/IJARSCT-9046 324

www.ijarsct.co.in
 ISSN (Online) 2581-9429
IJARSCT
International Journal of Advanced Research in Science, Communication and Technology (IJARSCT)

Volume 3, Issue 1, April 2023

Impact Factor: 7.301
[28]. Hill,R.(2020,April23).Theinfluenceofrapmusicinsociety.RetrievedNovember21,2021,fromhttps://spokeonline
.com/2020/04/the-influence-of-rap-music-in-society/
[29]. Clinicaldepression.(n.d.).RetrievedNovember21,2021,fromhttps://uhs.berkeley.edu/health-topics/mental-
health/clinical-depression
[30]. TrivediMH.Thelinkbetweendepressionandphysicalsymptoms.PrimCareCompanionJClinPsychiatry.2004;6(S
uppl1):12-16.
[31]. Monroe,J.(2021,February22).Spotifytopsongslisthighlightstheeffectsofmusiconemotion.RetrievedNovember
21, 2021, fromhttps://www.newportacademy.com/resources/treatment/effects-of-music/
[32]. Jang, S. (2017, December 13). MUSIC-BASED EMOTION REGULATION (MBER) INTERVENTION
MANUALFORPREVENTIONOFDEPRESSIONINOLDERPERSONS[Pdf].Kansas:GraduateFacultyoftheU
niversityofKansas.
[33]. Stefan, H. (2018, February 9). Guide to active learning in machine learning (ML). Retrieved November 21,
2021,fromhttps://www.datacamp.com/community/tutorials/active-learning
[34]. Settles, B. (2009, January 9). Active Learning Literature Survey [Pdf]. Wisconsin: Computer Sciences
TechnicalReport1648UniversityofWisconsin–Madison.

Copyright to IJARSCT DOI: 10.48175/IJARSCT-9046 325

www.ijarsct.co.in