e-ISSN: 2582-5208

International Research Journal of Modernization in Engineering Technology and Science

( Peer-Reviewed, Open Access, Fully Refereed International Journal )
Volume:06/Issue:10/October-2024 Impact Factor- 8.187 www.irjmets.com

AI BASED MOCK INTERVIEW SYSTEM

Ninad Chavan*1, Prathamesh Shivpuje*2, Sarthak Mali*3, Ayesha Sayyad*4
*1,2,3,4Pune University, India.
DOI: https://www.doi.org/10.56726/IRJMETS63116
ABSTRACT
The "AI-Based Mock Interview" project aims to transform the traditional interview process by utilizing artificial
intelligence to automate and enhance candidate evaluations. This AI-powered system assesses candidates
across multiple dimensions, such as verbal responses, body language, eye contact, and emotional expressions.
By incorporating cutting-edge technologies like machine learning, natural language processing, and facial
recognition, it offers a more efficient and objective approach to candidate assessment. The project seeks to
overcome the inherent biases and subjectivity often present in traditional interviews by adopting a data-driven,
impartial evaluation process.
The methodology centers around real-time video and audio analysis, leveraging emotion recognition and
sentiment analysis through convolutional neural networks (CNN) and long short-term memory (LSTM) models.
The system ultimately provides a comprehensive score that reflects both the emotional and intellectual aspects
of a candidate's performance. Designed to streamline recruitment, this AI-driven system aims to optimize
resource usage, reduce time, and enhance hiring precision.
I. INTRODUCTION
Artificial Intelligence (AI) has transformed numerous fields, including healthcare, finance, and education. In
recruitment, AI introduces innovative ways to assess candidates, enhancing the traditional hiring process.
While traditional interviews have their merits, they often lack fairness and efficiency, prompting a shift towards
skills-based assessments aimed at improving accuracy and impartiality in candidate evaluations. By utilizing
advanced technologies like facial recognition and machine learning algorithms, these AI systems evaluate
candidates' skills, personalities, and emotional responses. They analyze key aspects such as speech patterns,
facial expressions, and body language to deliver a more thorough assessment of a candidate's fit for a job. This
approach not only reduces the time and resources needed for recruitment but also minimizes human bias,
ensuring a more holistic evaluation. Additionally, AI-powered systems streamline the hiring process by
simultaneously managing multiple candidates.
The goal of this concept is to leverage AI in identifying suitable candidates by analyzing facial and vocal cues.
This strategy addresses the limitations of current interview practices by offering a more intelligent and efficient
way to evaluate candidates. For instance, research by Su Yusheng et al. (2021) introduced a real-time AI agent
that combines support vector machines (SVM) and convolutional neural networks (CNN) to predict job
applicants' behavioral potential through facial analysis. Similarly, Hung-Yue Suen and colleagues (2020)
proposed an asynchronous video interview system with intelligent decision-making capabilities to predict
communication and behavior from video recordings. Despite these advancements, current systems still face
challenges, including time consumption, the need for manual data input, and difficulties in measuring behavior
across different individuals.
To overcome these issues, the plan is to develop a smart conversational system that integrates facial and voice
analysis to better understand and evaluate candidates' personalities. The integration of advanced data
communication, deep learning models, and data processing techniques will further enhance the accuracy and
efficiency of candidate evaluations. The overarching aim is to revolutionize the interview process by using AI to
create a more reliable, unbiased, and effective system.
II. REVIEW OF LITERATURE
1) Paper 1: AI-Based Mock Interview Evaluator: An Emotion and Confidence Classifier Model
Publication Year: 2023
Authors: Rubi Mandal, Pranav Lohar, Dhiraj Patil, Apurva Patil, Suvarna Wagh

www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science

[5165]
 e-ISSN: 2582-5208
International Research Journal of Modernization in Engineering Technology and Science
( Peer-Reviewed, Open Access, Fully Refereed International Journal )
Volume:06/Issue:10/October-2024 Impact Factor- 8.187 www.irjmets.com
The paper "AI-Based Mock Interview Evaluator" reviews various studies on virtual interview systems,
emphasizing emotion recognition, sentiment analysis, and confidence evaluation. It highlights the use of
machine learning models to analyze non-verbal cues such as facial expressions, eye movements, and head
posture. Additionally, EEG-based techniques for tracking emotions are explored. The study also delves into
speech recognition and natural language processing, utilizing databases like Ryerson and Sentiment Speech to
classify emotions in speech. Personality traits and behavioral characteristics are assessed using human-
computer interaction (HCI) systems, enabling candidates to improve their interview performance in realistic
scenarios. This research supports the development of an AI system that evaluates both emotional and
confidence aspects, delivering a comprehensive mock interview experience.
2) Paper 2: AI-Based Interview Critique System
Publication Year: 2024
Authors: Nirgide Shubhangi Vishal, Sayyed Arsh Aktharali, Patil Paresh Narendra, Raktate Shriraj Vikas, Pathan
Md Fazal Mushtaque
The paper "AI-Based Interview Critique System" examines advancements in AI-driven mock interview
platforms, focusing on speech emotion recognition, behavioral analysis, and real-time evaluation techniques.
Studies discussed include the integration of deep learning models such as EmoConfidNet for speech emotion
recognition, enhancing confidence estimation through data preprocessing. The research also emphasizes the
importance of mock interview systems in improving candidate performance by assessing non-verbal cues,
including facial expressions and posture. Collectively, these studies contribute to the development of a
comprehensive AI system that offers personalized feedback, helping candidates refine both technical and
behavioral skills.
3) Paper 3: Smart Interviews Using AI
Publication Year: 2022
Authors: Aditi S. More, Samiksha S. Mobarkar, Siddhita S. Salunke, Reshma R. Chaudhari
The paper titled "Smart Interviews Using AI" proposes an AI-based system that evaluates candidates'
personality traits through video and speech emotion recognition. The system aims to address the inefficiencies
of traditional interview methods by offering an objective, efficient, and unbiased solution. By analyzing facial
expressions and speech emotion, the system gauges candidates' emotional states and behavioral traits during
interviews. This AI-based approach enhances the recruitment process by saving time and improving accuracy,
particularly in online interviews. The paper reviews various AI models, including Convolutional Neural
Networks (CNN) and TensorFlow-based systems, for real-time emotion and personality analysis. Future
improvements are suggested, such as incorporating additional parameters to further refine the system’s
accuracy and effectiveness for AI-driven interviews.
4) Paper 4: AI-Based Behavioral Analyser for Interviews/Viva
Publication Year: 2021
Authors: Dulmini Yashodha Dissanayake, Venuri Amalya, Raveen Dissanayaka, Lahiru Lakshan, Pradeepa
Samarasinghe, Madhuka Nadeeshani, Prasad Samarasinghe
The paper "AI-Based Behavioral Analyser for Interviews/Viva" presents a system that enhances the assessment
of behavioral cues in online interviews. Using deep learning techniques, the system analyzes non-verbal cues,
including emotions, eye movements, head movements, and smile patterns, to evaluate personality traits based
on the Big Five model. The proposed system demonstrates high accuracy (over 85%) in detecting behavioral
changes, offering a comprehensive analysis that allows for fair comparisons among candidates. This solution
addresses the limitations of previous systems that focused primarily on verbal communication by providing a
more detailed analysis of non-verbal behavior. The paper highlights the system's applicability in both
professional and academic settings, demonstrating significant improvements in accuracy and
comprehensiveness.
5) Paper 5: Development of an AI-Based Interview System for Remote Hiring
Publication Year: 2021
Authors: B C Lee, B Y Kim
www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science
[5166]
 e-ISSN: 2582-5208
International Research Journal of Modernization in Engineering Technology and Science
( Peer-Reviewed, Open Access, Fully Refereed International Journal )
Volume:06/Issue:10/October-2024 Impact Factor- 8.187 www.irjmets.com
The paper "Development of an AI-Based Interview System for Remote Hiring" introduces a deep-learning-
powered solution designed to improve recruitment processes, particularly for remote settings. The AI system,
developed using over 400,000 interview image datasets, achieved a high Pearson reliability score of 0.88 and
was successfully applied in multiple public enterprises in South Korea, with a satisfaction rate of 85%. The
paper highlights the growing global trend of AI adoption in hiring processes, focusing on job fitness evaluations,
personality assessments, and applicant screening. By incorporating advanced facial, vocal, and linguistic
analysis, the AI system aims to provide objective and unbiased evaluations, streamlining recruitment processes
and increasing overall hiring efficiency. The research underscores the potential of deep learning models in
enhancing both candidate evaluation and recruitment practices.
III. PROPOSED MODEL

The proposed model for emotion analysis from video and audio data outlines a comprehensive system that
involves several key components and their interactions. Users interact with the system through a user
interface, providing video or audio input. The system then captures and processes the input data, including
video preprocessing tasks like resizing, frame extraction, and noise reduction, as well as audio preprocessing
tasks like noise reduction and feature extraction.
Deep learning models are utilized to analyze the processed data and extract relevant features, such as facial
landmarks, audio spectrograms, or other characteristics indicative of emotions. These features are then used by
a classification module to predict the emotions present in the data. The system assesses the quality and
accuracy of the emotion classification results using metrics like precision, recall, and F1-score, and generates
feedback to improve the system's performance over time. Finally, the system presents the final emotion

www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science

[5167]
 e-ISSN: 2582-5208
International Research Journal of Modernization in Engineering Technology and Science
( Peer-Reviewed, Open Access, Fully Refereed International Journal )
Volume:06/Issue:10/October-2024 Impact Factor- 8.187 www.irjmets.com
classification results to the user in a visual format, such as a dashboard or report, and provides feedback to the
user or system based on the classification results. The system also stores trained deep learning models,
processed data, extracted features, and classification results in a database.
IV. ALGORITHMS
Convolutional Neural Networks (CNNs) for Facial Recognition
CNNs are a type of deep learning architecture specifically designed for processing and analyzing image data.
They have been particularly successful in tasks like image classification, object detection, and facial recognition.
CNNs work by applying filters to the input image, extracting features like edges, corners, and textures. These
features are then combined to form a representation of the image, which is used to classify it into different
categories or classes. CNNs are well-suited for facial recognition due to their ability to learn hierarchical
representations of facial features and their invariance to small variations in the input image. They have found
applications in various fields, including access control, biometric authentication, law enforcement, and social
media.
Natural Language Processing (NLP) for Speech Recognition
Natural Language Processing (NLP) is a field of artificial intelligence that focuses on the interaction between
computers and human language. In the context of speech recognition, NLP techniques are used to transform
spoken language into text. NLP plays a crucial role in speech recognition by providing the language context and
understanding necessary for accurate transcription. By leveraging the power of NLP, speech recognition
systems can improve their accuracy, robustness, and adaptability to different speaking styles and dialects.
LSTM (Long Short Term Memory)
LSTM (Long Short-Term Memory) models plays a crucial role in enhancing the system’s ability to accurately
process and recognize speech. Speech data is inherently sequential, where each word or sound depends on
previous words to convey meaning. LSTM, being a specialized type of recurrent neural network (RNN), excels in
handling such sequential data by learning long-term dependencies. The key feature of LSTM is its memory cell
structure, which allows it to retain important information over time while discarding irrelevant data through
its input, forget, and output gates. This mechanism enables LSTM to handle the temporal nature of speech
effectively, ensuring that past information is taken into account when processing current input.
Additionally, speech input often varies in length, and LSTM is well-suited for such variable-length sequences, as
it can manage input that does not have a fixed size. By leveraging its ability to remember context over long
sequences, LSTM helps to improve the recognition of complex speech patterns and longer sentences. This leads
to higher accuracy in the overall system compared to traditional models, which may struggle with maintaining
context in longer sequences. Therefore, incorporating LSTM into your speech recognition model significantly
enhances its performance, making it better equipped to interpret and recognize speech in real-world scenarios.
V. RESULTS
The AI-based mock interview system presented in this paper demonstrates a highly efficient and unbiased
approach to evaluating candidates through the use of advanced artificial intelligence techniques. The system
integrates convolutional neural networks (CNNs) for facial recognition and long short-term memory (LSTM)
models for speech recognition to evaluate candidates' performance across a variety of parameters, including
emotional and behavioral cues. Through real-time analysis of video and audio inputs, the system effectively
captures non-verbal cues such as facial expressions, eye contact, and speech patterns. The project’s results
indicate that AI models can outperform traditional interview processes by providing objective, data-driven
feedback, which reduces human bias and subjectivity.
Moreover, the system has been trained using large datasets of candidate behavior, improving its accuracy in
evaluating both technical and emotional aspects of a candidate's performance. By incorporating machine
learning, natural language processing (NLP), and emotion recognition technologies, the AI-based interview
system can handle both video and audio inputs seamlessly, providing a comprehensive evaluation based on key
factors such as confidence, emotional stability, and technical knowledge. The system’s precision, recall, and F1-
score metrics indicate high performance in accurately identifying emotions and providing detailed feedback for

www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science

[5168]
 e-ISSN: 2582-5208
International Research Journal of Modernization in Engineering Technology and Science
( Peer-Reviewed, Open Access, Fully Refereed International Journal )
Volume:06/Issue:10/October-2024 Impact Factor- 8.187 www.irjmets.com
both recruiters and candidates. This suggests that AI-driven interview systems could significantly improve the
efficiency and fairness of the recruitment process, leading to better hiring decisions.
VI. CONCLUSION
The AI-based mock interview system is a significant step forward in the automation of candidate evaluations.
By leveraging machine learning and AI technologies such as CNNs for facial emotion recognition and LSTM for
speech recognition, the system provides a robust platform for objective and comprehensive assessment. The
findings from the project suggest that AI systems can offer a more accurate and reliable way to assess
candidates' emotional and intellectual capabilities compared to traditional methods, which are prone to bias
and subjectivity. Additionally, the system’s ability to process both video and audio data in real-time makes it a
versatile tool for modern recruitment, especially in remote or large-scale hiring scenarios. As AI technology
continues to advance, integrating such systems into recruitment processes will likely become a standard
practice, enhancing both the fairness and efficiency of hiring across industries.
VII. REFERENCE
[1] Yang Li, Constantinos Papayiannis, Viktor Rozgic, Elizabeth Shriberg, Chao Wang Dept. of Electrical and
Computer Engineering, University of Rochester. “CONFIDENCE ESTI- MATION FOR SPEECH EMOTION
RECOGNITION BASED ON THE RELATIONSHIP BETWEEN EMOTION CATEGORIES AND PRIMITIVES.”,
IEEE Publication, 2023.
[2] Rubi Mandal, Pranav Lohar, Dhiraj Patil, Apurva Patil, Suvarna Wagh, ”AI-Based mock interview
evaluator: An emotion and confidence classifier model”, IEEE Publication, 2023
[3] Yi-Chi Chou, Felicia R. Wongso, Chun-Yen Chao, Han-Yen Yu , ”An AI Mockinterview Platform for
Interview Performance Analysis”, IEEE Publication, 2022.
[4] Dulmini Yashodha Dissanayake, Venuri Amalya, Raveen Dissanayaka, Lahiru Lakshan, Pradeepa
Samarasinghe, Madhuka Nadeeshani, Prasad Samarasinghe. ”AIbased Be- havioural Analyser for
Interviews/Viva”, Springer, 2021.
[5] B C Lee, B Y Kim, ”Development of an AI-based interview system for remote hiring”, IJARET
Publication, 2021.
[6] A. Sivasangari, V. Shivani, Y. Bindhu, D. Deepa and R. Vignesh, "Prediction Probability of Getting an
Admission into a University using Machine Learning", 2021 5th International Conference on
Computing Methodologies and Communication (ICCMC), pp. 1706-1709, 2021.
[7] Surendar B, Chakravarthy SS, Thangavel PR Facial feature analysis using deep convolutional neural
networks and HAAR classifier for real-time emotion detection. Int J Eng Tech Res, 2021
[8] S. Sridhar, S. Mootha and S. Kolagati, "A University Admission Prediction System using Stacked
Ensemble Learning", 2020 Advanced Computing and Communication Technologies for High
Performance Applications (ACCTHPA), pp. 162-167, 2020.
[9] Valstar M., Schuller B., Smith K., Eyben F., Jiang B., Bilakhia S., Sneddon I., Cowie R. "Speech Analysis for
Depression Assessment Using Deep Convolutional Neural Networks" 2014.
[10] Khan, R. A., Meyer A., Konik H., Bouakaz S. "Framework for reliable, real-time facial expression
recognition for low resolution images" 2013.

www.irjmets.com @International Research Journal of Modernization in Engineering, Technology and Science

[5169]