ABSTRACT

• Since the development of artificial intelligence, the automated classification of interviews conducted to ascertain individual
personality traits has grown in importance as a study area with applications in personality computing, human-computer
interaction, and psychological assessment (AI).

• Deep learning (DL)-based advances in pattern recognition and computer vision have led to the development of models that can
recognize human nonverbal cues and assign personality traits to them by utilizing only a camera. In the proposed paradigm, an
end-to-end AI interviewing system is built using asynchronous video interview (AVI) processing.

• Automated personality identification is carried out by the Tensor Flow AI engine using the attributes gathered from the AVIs
(APR). Facial expressions and self-reported surveys are used to get the genuine personality scores for real job applicants.

• For the purpose of predicting the personality trait scores, the Big Five personality traits—neuroticism, extroversion, openness,
agreeableness, and conscientiousness—are observed and assessed. The self-reported Big Five personality trait scores are
proposed to be predicted using Convolutional Neural Networks (CNN).
Introduction(KYRIE
• The advent of artificial intelligence has revolutionized various fields, one of the most notable
being the automation of psychological assessments. Traditionally, personality traits were
evaluated through self-reported surveys or in-person interviews, which often relied on
subjective interpretations.
• However, with the advancement of deep learning and computer vision, it has become possible
to automate the analysis of human personality traits by interpreting nonverbal cues and facial
expressions through video data.
• This has paved the way for the development of automated personality classification systems,
which offer a more objective and scalable solution for personality evaluation, especially in
recruitment and human-computer interaction scenarios.
• In this project, we aim to create an end-to-end AI-driven interviewing system that processes
asynchronous video interviews (AVIs) to identify personality traits in real job applicants.
• The system leverages the power of Convolutional Neural Networks (CNN) and TensorFlow to
analyze the visual and emotional cues extracted from facial expressions during interviews.
Introduction
• By employing the Big Five personality traits—neuroticism, extroversion, openness,
agreeableness, and conscientiousness—as the framework for personality assessment,
this model is designed to accurately predict an individual's personality scores without
relying on self-reports alone.
• The proposed AI interviewing system will utilize deep learning algorithms to extract
meaningful features from video interviews, which will then be used to predict
personality traits with a high degree of accuracy.
• By combining facial expression recognition and self-reported surveys, the system aims to
provide a comprehensive and unbiased evaluation of an individual's personality.
• This approach not only enhances the recruitment process by automating personality
assessment but also contributes to the broader field of psychological research by
providing a more efficient and scalable method for studying personality traits in various
contexts.
PROBLEM SCENARIO
• In traditional recruitment and psychological assessment processes, evaluating personality traits
often relies heavily on self-reported surveys or face-to-face interviews, both of which can
introduce biases or inaccuracies.
• Self-reports are prone to social desirability bias, where candidates may present themselves in an
overly favorable light, while in-person interviews are subject to the subjective judgments of
interviewers, which can vary based on personal perceptions or unconscious biases.
• Additionally, these methods are time-consuming and not scalable, particularly in large
recruitment scenarios.
• As organizations increasingly turn to AI to streamline hiring and improve decision-making, there is
a growing need for an automated system that can objectively and efficiently assess personality
traits from video data, providing a consistent and unbiased evaluation of candidates' personalities.
• This project seeks to address these challenges by developing an AI-powered system that can
accurately predict personality traits from asynchronous video interviews, using deep learning and
facial expression analysis to overcome the limitations of traditional methods.
PROPOSED SOLUTION
• The proposed solution is an AI-powered system designed to automate the classification of
personality traits through asynchronous video interviews (AVIs).
• The system utilizes deep learning techniques, specifically Convolutional Neural Networks
(CNN), to analyze facial expressions and nonverbal cues captured in the video, alongside
self-reported survey data. By focusing on the Big Five personality traits—neuroticism,
extroversion, openness, agreeableness, and conscientiousness—the model predicts
personality scores with high accuracy.
• The TensorFlow AI engine processes the video data to extract meaningful features, which
are then used to generate predictions about a candidate's personality, offering an objective
and scalable method for personality assessment.
• This solution aims to enhance recruitment processes, reduce biases, and provide more
accurate and efficient personality evaluations compared to traditional methods, ultimately
improving human-computer interaction and psychological assessments in various domains.
PURPOSE
• The purpose of this project is to develop an automated, AI-based system that
enhances the process of personality assessment by leveraging asynchronous video
interviews (AVIs).
• The system aims to provide a scalable, objective, and accurate method for evaluating
an individual’s personality traits using deep learning models, particularly Convolutional
Neural Networks (CNN).
• By analyzing facial expressions and self-reported survey data, the system seeks to
predict the Big Five personality traits—neuroticism, extroversion, openness,
agreeableness, and conscientiousness—without relying on subjective interpretations
or biases.
• This project’s goal is to improve the efficiency and accuracy of personality
assessments, making them more accessible and applicable in recruitment, human-
computer interaction, and psychological research.
SCOPE
• The scope of this project encompasses the development of an AI-driven system for
automated personality classification based on asynchronous video interviews (AVIs).
• The system focuses on predicting the Big Five personality traits—neuroticism,
extroversion, openness, agreeableness, and conscientiousness—by analyzing facial
expressions and nonverbal cues using deep learning techniques, specifically
Convolutional Neural Networks (CNN).
• The project will also integrate self-reported survey data to further refine personality
predictions. It is designed to work in real-world recruitment and psychological
assessment settings, providing a scalable and unbiased method of personality evaluation.
• The system will be tested and validated on a diverse dataset of video interviews, with an
emphasis on improving the accuracy, speed, and practicality of the solution for various
applications, including job recruitment, human-computer interaction, and psychological
studies.
Tools & Technologies
1. Hardware Requirements:
1. Computer/Server: For running deep learning models and processing video data.
2. Webcam or Camera: For capturing video data during asynchronous video interviews (AVIs).
3. GPU (Graphics Processing Unit): For accelerating deep learning model training and inference, especially for
CNN-based tasks.
4. Storage Devices: To store large video datasets, models, and survey data.
2. Software Requirements:
1. TensorFlow: An open-source machine learning framework used for implementing and training Convolutional
Neural Networks (CNN) to analyze facial expressions and predict personality traits.
2. Python: Programming language used for developing the AI model and integrating components of the system.
3. OpenCV: A computer vision library for processing and extracting features from video data.
4. Keras: High-level neural networks API, running on top of TensorFlow, used for developing and training deep
learning models.
5. Jupyter Notebook/IDE (Integrated Development Environment): For coding, testing, and running
experiments.
6. Database (e.g., MySQL, MongoDB): For storing self-reported survey data and results.
Functionl Requirements

• Users:
1. Job Applicants: Individuals who participate in asynchronous video interviews, which are recorded and processed for personality
trait analysis.
2. Recruiters/HR Managers: Users who view the analyzed personality trait scores of candidates, assist in interpreting the results,
and make decisions based on personality predictions.
3. System Admin: Manages the system's functionality, including user account management, data security, and system maintenance.
1. Functionalities:
1. Video Interview Processing: The system must allow job applicants to record asynchronous video interviews. These videos will be
processed to extract relevant facial expressions, gestures, and emotions for personality analysis.
2. Self-Reported Survey Integration: Applicants fill out a survey based on the Big Five personality traits. This data will be used
alongside facial expression data for more accurate predictions.
3. Facial Expression Analysis: The system analyzes video data to detect facial expressions, which will be used to predict personality
traits such as extroversion, openness, and neuroticism.
4. Personality Trait Prediction: Based on the extracted features from both the video and survey data, the system predicts and
assigns scores for the Big Five personality traits: neuroticism, extroversion, openness, agreeableness, and conscientiousness.
5. Results Display: HR managers or recruiters can view the predicted personality trait scores on a user-friendly dashboard or
interface. The system provides an explanation of the traits predicted.
6. Data Storage: Store applicant video interviews, self-reported survey data, and predicted personality scores in a secure database.
7. User Management: Admin users can manage user accounts, control access to data, and monitor system performance.
Non-Functional Requirements
1. Performance:
1. The system must be capable of processing video interviews and generating personality predictions in real-time or near-real-time.
2. The deep learning models should be optimized for efficient inference to minimize the time required for processing large volumes of interview data.
2. Scalability:
1. The system should be able to scale to handle multiple users and large datasets, especially during peak recruitment periods.
2. The architecture must be designed to allow easy integration of additional functionalities or support for additional personality traits in the future.
3. Accuracy:
1. The system must achieve a high level of accuracy in predicting personality traits based on facial expressions and self-reported surveys, aiming for a minimum
benchmark accuracy as determined through extensive testing.
2. Regular updates to the model should be performed to improve prediction accuracy over time.
4. Security:
1. All video data, survey responses, and predicted results must be stored securely, ensuring that sensitive user information is protected.
2. The system must comply with data protection regulations, such as GDPR or CCPA, to ensure user privacy and data security.
5. Usability:
1. The user interface must be intuitive and user-friendly, allowing both applicants and recruiters to interact with the system seamlessly.
2. The system should support both desktop and mobile devices for user accessibility.
6. Reliability:
1. The system should be highly reliable, with minimal downtime and robust error handling. It must ensure smooth operation even under heavy load or during
system maintenance.
7. Maintainability:
1. The system should be easy to maintain and update, allowing for the addition of new features, bug fixes, and performance optimizations without major
disruptions.
Initial Design
Architecture Diagram
 EXPLANATION

•Job Applicant: Contains methods to submit video interviews and surveys.

•AI Engine: Contains methods to process video data and predict personality traits using deep learning models.

•Recruiter/HR Manager: Contains methods to view personality trait results and review predictions.

•Database: Stores and retrieves all data, including video data, survey data, and predicted personality scores.

•Admin: Manages user accounts and system monitoring.

Process
Flow
Steps in the Diagram:
1.The Job Applicant submits both video interviews and survey data.
2.The system validates the video data. If invalid, the process stops with a
notification.
3.If valid, the AI engine processes the video to extract features and
combines them with survey data.
4.The combined data is used to predict personality traits using CNN.
5.If predictions are successful, the data and results are stored in the
database, and predictions are shared with the recruiter or HR manager.
6.The Recruiter/HR Manager views the results for evaluation.
USE CASE
DIAGRAM
1. Actors:
1. Job Applicant: Submits video interviews and survey data.
2. Recruiter/HR Manager: Views personality predictions.
3. Admin: Manages user accounts and monitors system performance.
2. Use Cases:
1. Submit Video Interview and Submit Survey Data: Actions performed by the job applicant.
2. Process Video and Survey Data: AI engine processes inputs.
3. Predict Personality Traits: Core functionality for prediction using CNN.
4. View Personality Predictions: Recruiter or HR manager evaluates personality data.
5. Manage User Accounts and Monitor System Performance: Admin-specific tasks.
6. Store and Retrieve Data: Backend interaction for secure data storage.
• Relationships:
• Applicant interacts with submission use cases.
• AI Engine processes submissions, predicts traits, and stores/retrieves data.
• Recruiter/HR Manager views predictions.
• Admin manages the system and ensures performance.
Application in the Real World

 Recruitment and Hiring: Streamlines candidate evaluation by predicting personality traits for better

job role alignment.

 Educational Assessment: Enhances understanding of student behavior for tailored learning

approaches.

 Psychological Research: Assists in studying non-verbal cues for deeper personality insights.

 Customer Service Training: Identifies employee personality traits to improve customer

interactions.
References
1. S. Song, S. Jaiswal, E. Sanchez, G. Tzimiropoulos, L. Shen and M. Valstar, "Self- supervised Learning of Person-specific
Facial Dynamics for Automatic Personality Recognition," in IEEE Transactions on Affective Computing, doi:
10.1109/TAFFC.2021.3064601.

2. F. Zhang and F. Wang, "Exercise Fatigue Detection Algorithm Based on Video Image Information Extraction," in IEEE
Access, vol. 8, pp. 199696-199709, 2020, doi: 10.1109/ACCESS.2020.3023648.

3. M. A. Moreno-Armendáriz, C. A. Duchanoy Martínez, H. Calvo and M. Moreno- Sotelo, "Estimation of Personality Traits
From Portrait Pictures Using the Five- Factor Model," in IEEE Access, vol. 8, pp. 201649-201665, 2020, doi:
10.1109/ACCESS.2020.3034639.

4. G. Singh, M. Chowdhary, A. Kumar and R. Bahl, "A Personalized Classifier for Human Motion Activities With Semi-
Supervised Learning," in IEEE Transactions on Consumer Electronics, vol. 66, no. 4, pp. 346-355, Nov. 2020, doi:
10.1109/TCE.2020.3036277.

5. R. D. P. Principi, C. Palmero, J. C. S. J. Junior and S. Escalera, "On the Effect of Observed Subject Biases in Apparent
Personality Analysis From Audio-Visual Signals," in IEEE Transactions on Affective Computing, vol. 12, no. 3, pp. 607- 621,
1 July-Sept. 2021, doi: 10.1109/TAFFC.2019.2956030.
1. L. Zhang, S. Peng and S. Winkler, "PersEmoN: A Deep Network for Joint Analysis of Apparent Personality,
Emotion and Their Relationship," in IEEE Transactions on Affective Computing, vol. 13, no. 1, pp. 298-305, 1
Jan.-March 2022, doi: 10.1109/TAFFC.2019.2951656.
2. J. C. S. Jacques Junior et al., "First Impressions: A Survey on Vision-Based Apparent Personality Trait Analysis,"
in IEEE Transactions on Affective Computing, vol. 13, no. 1, pp. 75-95, 1 Jan.-March 2022, doi:
10.1109/TAFFC.2019.2930058.
3. H. J. Escalante et al., "Modeling, Recognizing, and Explaining Apparent Personality From Videos," in IEEE
Transactions on Affective Computing, vol. 13, no. 2, pp. 894-911, 1 April-June 2022, doi:
10.1109/TAFFC.2020.2973984.
4. M. Ben Ayed, S. Elkosantini, S. A. Alshaya and M. Abid, "Suspicious Behavior Recognition Based on Face
Features," in IEEE Access, vol. 7, pp. 149952-149958, 2019, doi: 10.1109/ACCESS.2019.2947338.
5. F. Mokhayeri and E. Granger, "Video Face Recognition Using Siamese Networks With Block-Sparsity Matching,"
in IEEE Transactions on Biometrics, Behavior, and Identity Science, vol. 2, no. 2, pp. 133-144, April 2020, doi:
10.1109/TBIOM.2019.2949364.