Volume 5, Issue 4, April – 2020 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165

Asynchronous Interview Analysis

1 2
Pratik Satpute Rohan Nair
Computer Engineering Computer Engineering
Rajiv Gandhi Institute of Technology Mumbai, India Rajiv Gandhi Institute of Technology Mumbai, India
3 4
Vaibhav Pathak Priyanka Bhilare
Computer Engineering Computer Engineering
Rajiv Gandhi Institute of Technology Mumbai, India Rajiv Gandhi Institute of Technology Mumbai, India

Abstract:- Conventional interviewing procedures often The final step involves an asynchronous face-to-face
lack to grasp a deep understanding of their inter- view. This interview will be recorded to understand
interviewees. Interviewers could fail to observe how the interviewee reacts to specific questions and how
important facial features, textual tone, and body he/she feels throughout the interview process.
language during the interview process. The proposed
system provides a thorough approach to understand
how the interviewee behaves and feels during the
interviewing process. A significant feature of this system
is the ability to measure and quantify the big 5 traits of
personality with the help of machine learning and
natural language processing. Additionally, this paper
presents how the amalgamation of various algorithms
and approaches can help in determining the overall
personality of the interviewee.

Keywords:- Natural Language Processing, Machine

Learn- ing, Artificial Intelligence, Sentiment Analysis,
FastAI.

I. INTRODUCTION

Sentiment Analysis is the process of determining

whether a piece of writing is positive, negative or neutral.
It is an essential component for understanding a person’s
emotions. Using such analysis as an automated system in
an interview environment can reveal critical information
about the person’s behavior. There’s also the assurance that
the system will not show bias towards a person.
Additionally, it can also detect many features that can be
overlooked by a human. However, judging a person only
based on their intellect is not sufficient. Only a complete
thorough analysis, can give us what the person is thinking
vs. what the person wants us to believe.

For this complete analysis, our system will be

integrating 3 essential measurement traits. The first one
determining the aptitude of a person. This aptitude will be
measured using a basic knowledge test where the person
will be asked questions on his applied position. This will be
followed by a psychometric test which involves OCEAN
analysis. Fig 1:- System Flowchart
The second step involves, determining their thought
process using a writing test. This can help employers
determine whether the student’s thought process is positive,
negative or neutral.

IJISRT20APR381 www.ijisrt.com 272

Volume 5, Issue 4, April – 2020 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
A. Openness
Openness to experience (inventive/curious vs. consis-
tent/cautious).

Openness is one of the factors which is used to

describe a person’s personality in the (FFM) Five-Factor
Model. Open- ness includes six dimensions that are
sensitivity, imagination, inner feelings, intellectual
curiosity, and variety. In-depth psychometric research has
indicated that these dimensions or qualities are significantly
correlated. Thus, openness can be seen as a global
personality trait comprises of a set of specific traits, habits,
and proclivity that cluster together.
Fig 2:- Big Five Traits
The type of questions ask are
II. LITERATURE REVIEW 1.Do you believe there is no absolute right and wrong?
2.Do you prefer to stick with things you know?
A. Personality Prediction
 predicts personality using OCEAN analysis for hand- B. Conscientiousness
writing. However, this approach has 2 major drawbacks. Conscientiousness (efficient/organized vs. easy-
Firstly, a major aspect of handwriting analysis is that it going/careless)
predicts accurate personality only if handwriting sample
is written in a relaxed mood. [1] does not consider that. Conscientiousness is the persona trait of being
Secondly The online tools used for automatic careful, or diligent. Conscientiousness implies a preference
personality assessment using handwriting analysis to do a challenge well and to take responsibilities to others
requires basic knowledge of various writing features. seriously. Conscientious people tend to be efficient and
The person with this knowledge can use these tools organized in preference to easy-going and disorderly. They
efficiently and effectively. However, a layman cannot showcase a tendency to reveal self-discipline, act dutifully,
use these tools for his/her benefit. There are possibilities and aim for achievement; they show planned rather than
of human errors incorrectly matching writing to the spontaneous be- havior, and they’re typically dependable. It
given sample. Hence, the accuracy of the results will is manifested in feature behaviors consisting of being neat,
be compromised. and systematic; also which include such elements as
carefulness, thoroughness, and deliberation.
B. Facial Expression Detection
 uses Mini Xception which is inspired by Xception The type of questions ask are
 model, This architecture combines the use of residual 1. Do you complete tasks successfully every time? 2.Are
modules and depth-wise separable convolutions you always prepared?
whereas we use deep residual network [4] resnet50. The
system proposed in [2] achieved an accuracy of 66% on C. Extraversion
7 class “angry”, “disgust”, “fear”, “happy”, “sad”, Extraversion (outgoing/energetic vs.
“surprise”, “neutral”. The major drawback is that the solitary/reserved).
data-set used i.e. Fer2013 has many mislabelled images
and the distribution of images is not equal in each class. The traits of extraversion (or extroversion) and
Class with less image is often predicted wrong. To introversion are a central measurement in a few human
overcome this we cleaned the Fer2013 data-set and personality theories. The terms introversion and
eliminated the class which was not required. extraversion have been popularized through Carl Jung,
although both the famous understanding and psychological
III. METHODOLOGY utilization vary from his authentic intent. Extraversion
tends to be manifested in outgoing, talkative, lively
 Aptitude And Personality Prediction behavior, while introversion is manifested in greater
For the first step, our system begins with general reserved and solitary behavior. Rather than that specialize
aptitude questions based on Quants, Verbal, Logical in interpersonal behavior, however, Jung described
reasoning each consisting of 5 questions, continuing to introversion as an ”attitude-type characterized by
psychometric analysis. In the psychometric analysis, our orientation in life via subjective psychic contents”, and
system determines the per- sonality by using OCEAN extraversion as ”an attitude- type characterized through
analysis. The Big Five personality traits, also known as the awareness of interest on the exter- nal object.”
five-factor model (FFM) and the OCEAN model, is a
taxonomy, or grouping, for personality traits. The big five The type of questions ask are 1.Do you make friends
traits are Openness, Conscientiousness, Extraversion, easily?
Agreeableness, Neuroticism. 2. Do you feel comfortable around people?

IJISRT20APR381 www.ijisrt.com 273

Volume 5, Issue 4, April – 2020 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
D. Agreeableness emotion or judgment whereas objective often refers to
Agreeableness (friendly/compassionate vs. challeng- factual information. Subjectivity also lies in the range of
ing/detached). [0,1].
Agreeableness is a character attribute indicating
itself in a person’s behavioral traits which might be let’s see an example:
perceived as sympathetic, kind, cooperative, understanding,
and tender. In modern-day character psychology,
agreeableness is one of the five essential dimensions of
personality structure, reflecting man or woman variations in
cooperation and social harmony. People who are rating
excessive on this are empathetic and altruistic, even as a
low agreeableness rating pertains to selfish behavior and a
lack of empathy. The type of questions ask are 1.Do you This tells us that the English phrase “not great at
English” has a polarity of about -0.2, meaning it is slightly
like to help the needy?
2.Do you like to boast about your virtues? negative, and subjectivity of about 0.375, meaning it is
fairly objective.
E. Neuroticism
Neuroticism is one of the Big Five higher-order
personality traits in the field of psychology. Individuals
who score high on neuroticism are much more likely to be
moody and to experience not so positive emotions such as
anxiety, worry, fear, frustration, envy, jealousy, guilt, and The wordbook it refers to is in en-sentiment.xml it is
loneliness. Neurotic people respond worse to stressors and an XML document that includes four entries for the word
are more likely to inter- pret everyday conditions as “great”.
threatening and minor frustrations as hopelessly difficult.
They are regularly self-conscious and shy, and they’ll have In addition to the polarity, subjectivity, and intensity
hassle controlling urges and delaying gratification. of the phrase there’s also “confidence”. In the case of
“great” it’s all the same part of speech (JJ, adjective), and
The type of questions ask are the senses are themselves natural language and not used.
1.Are you easily bothered by things? 2.Do you lose your
composure easily? To simplify for readability:
All of the above traits were given a score on a scale of
1-9. The person attempting the test is given a set of
questions. By answering the most appropriate answer score
for the particular personality trait is increased by 1. If not
answered with the most suitable option then scores are
given accordingly i.e. 0.75, 0.50 and 0. 25. After answering
all the questions of each trait we classify the person
personality using linear regression into one of the five
classes Extraverted, Serious, Responsible, Lively, When calculating sentiment for a single word,
Dependable. The data-set consists of 710 training samples TextBlob uses a sophisticated approach recognized to
and 316 test samples with attributes like gender, age, mathematicians as “averaging”
Openness, conscientiousness, extraversion, agreeableness,
neuroticism.

 Text Polarity
In the second step of our system, we give the test taker
a general topic like ”Describe the best day of your life” or
something on a similar line. The essay written by the test The polarity is multiplied by -0.5 when negation is
taker is analyzed for the sentiment. The result of the used and it does not affect subjectivity.
sentiment analysis would suggest the mindset of the person
whether he/she has a positive, negative or neutral opinion
towards life. To achieve this we use TextBlob. TextBlob is a
built-in python library and offers simple access to its
methods and perform basic Natural language processing
functions. The sentiment function of TextBlob returns two
attributes, subjectivity, and polarity. Polarity lies in the Recognizing “very” as a modifier word, TextBlob will
range of [-1,1] where 1 means the given statements are ignore polarity and subjectivity and simply use intensity to
positive and -1 means the given statement is negative. modify the following word:
Subjective sentences often refer to personal opinion,

IJISRT20APR381 www.ijisrt.com 274

Volume 5, Issue 4, April – 2020 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
are accountable for lowering and then restoring the
dimensions. The 3×3 layer is used as a bottleneck with
smaller input/output dimensions. Finally, the network has
an Average Pooling layer followed by a completely linked
layer having a thousand neurons.

The polarity gets maxed out at 1.0, but you may see
that subjectivity is also changed by “very” to turn out to be
0.6*1.3
= 0.78

 Facial Expression Detection During Interview

Finally, in the third step, we have used Resnet50 to
detect the certain facial expression of the
person/interviewee without any biases. Fer2013 was
utilized to train this CNN model with the following classes
[ 0: angry, 1: happy, 2: sad, 3: surprise, 4: neutral ]

Fig 3:- Learning Rate

Fig 4:- ResNet50 Architecture
ResNet-50 is a deep residual network. The “50” refers
to the range of layers it has. It’s a subclass of convolutional
neural networks. The most important innovation of ResNet
is the skip connection. As you know, without adjustments,
deep networks regularly are afflicted by vanishing
gradients, i.e. as the model backpropagates, the gradient
gets smaller and smaller.

The architecture of ResNet50 has four levels as

shown in the diagram. Every ResNet structure performs
the initial convolution and max-pooling by using 7×7
and 3×3 kernel sizes respectively. Afterward, level 1 of the
network begins and it has 3 Residual blocks containing
three layers each. The size of kernels used to carry out the
convolution operation in all 3 layers of the block of level
1 is 64, 64 and 128 respectively. As we progress from
one level to another, the channel width is doubled and the
dimensions of the input are reduced to half. For deeper
networks like ResNet50, ResNet152, etc, bottle- neck style
is employed. For every residual function F, three layers Fig 5:- Distribution of training data
are stacked one over the other. The three layers are
1×1, 3×3, 1×1 convolutions. The 1×1 convolution layers

IJISRT20APR381 www.ijisrt.com 275

Volume 5, Issue 4, April – 2020 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
The model achieved an accuracy of 75% with the help
of 24176 (Training) + 3037 (Public Test) + 3006 (Private
Test)
= 30219 images.

The loss obtained was 0.649742. This model was then

fed with real-time images of the interviewee. The
interviewee had to respond to the questions displayed. The
candidate’s reaction was then captured. These real-time
images were cropped using HAARCASCADE [5] to extract
the face of the interviewee and then resized to 48X48
pixels. These were fed to the neural network. Based on how
the interviewee is reacting, the model uses facial features to
identify the individual’s sentiment. The parameters are
happy, sad, neutral, surprised and angry. The model logs
these expressions in each frame in a text file. Then this
text file is analyzed and detects the amount of time the
individual was showing those expressions. Based on the
percentage of the parameters, the overall facial expression Fig 8:- Confusion Matrix
of the individual is calculated.
IV. RESULTS

A. Personality Prediction
The accuracy of the model for predicting personality
achieved 80% using logistic regression.

B. Facial Expression Detection During Interview

The facial Expression detection model achieved an
accuracy of 75% After analyzing the top losses shown
below, we found out the forced facial features were
classified with low confidence. Other facial features like in
image number 7 of the top losses were difficult to classify
as the capture features of 2 or more classes. To conclude,
the model performs reliably in an interviewing
environment where a single feature class will exist.

Fig 6:- Distribution of validation data

Fig 9:- Toploss

C. Text Polarity
The Accuracy of Text polarity i.e. finding sentiment
using TextBlob is 60.5%
Fig 7:- Distribution of testing data

IJISRT20APR381 www.ijisrt.com 276

Volume 5, Issue 4, April – 2020 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
V. CONCLUSION

The comprehensive system presented above helps

interview- ers to understand their interviewees better. This
system doesn’t aim to replace the interviewer but to aid
them in their process. Further, it helps to gain important
insights and details that the interviewers could have
missed. This thorough procedure can be used in a practical
environment to improve the overall evaluation process of a
candidate. Hence, this research aims at helping companies
with recruitment policies and it makes their recruitment
process efficient by filtering out candidates according to
their requirements.

REFERENCES

[1]. D. S. B. S. V. Kedar, “Automatic personality

assessment: A systematic review,” International
Conference on Information Processing (ICIP), 2015.
[2]. O. Arriaga, M. Valdenegro-Toro, and P.
Plöger, “Real- time convolutional neural networks
for emotion and gender classification,” CoRR, vol.
abs/1710.07557, 2017. [Online]. Available:
http://arxiv.org/abs/1710.07557
[3]. F. Chollet, “Xception: Deep learning with depthwise
separable convolutions,” CoRR, vol. abs/1610.02357,
2016. [Online]. Available:
http://arxiv.org/abs/1610.02357
[4]. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual
learning for image recognition,” pp. 770–778, 2016.
[5]. A. Sharifara, M. S. Mohd Rahim, and Y. Anisi, “A
general review of human face detection including a
study of neural networks and haar feature-based
cascade classifier in face detection,” pp. 73–78, 2014.

IJISRT20APR381 www.ijisrt.com 277