Online Exam Proctoring System

Submitted in partial fulfillment of the requirements

of the degree of

Bachelor of Engineering
By

Mr. Bhumit Malvi (Roll No:62)

Mr. Sanket Suhagiya (Roll No:76)
Mr. Rithwik Vedpathak (Roll No:82)

Under the guidance of

Prof. Smita Jawale

DEPARTMENT OF COMPUTER ENGINEERING

VIDYAVARDHINI’S COLLEGE OF ENGINEERING
AND TECHNOLOGY
K. T. MARG, VASAI ROAD (W.) DIST-THANE, PIN: 401202

(AFFILIATED TO MUMBAI UNIVERSITY)

(A.Y. 2021-22)
 CERTIFICATE

This is to certify that the project entitled “Online Exam Proctoring System” is a bonafide

work of “Bhumit Malvi (Roll No. 62), Sanket Suhagiya (Roll No. 76) and Rithwik

Vedpathak (Roll No. 82) submitted to the University of Mumbai in partial fulfillment of the

requirement for the award of the degree of “Bachelor of Engineering” in “Computer

Engineering”.

______________________
Prof. Smita Jawale
Guide

Dr. Megha Trivedi Dr. Harish Vankudre

Head of Department Principal

2
 Project Report Approval for B.E

This project report entitled ‘Online Exam Proctoring System’ by ‘Bhumit

Malvi, Sanket Suhagiya and Rithwik Vedpathak’ is approved for the degree

of ‘Bachelor of Engineering’ in ‘Computer Engineering’.

Examiners

1.

2.

Date:

Place:

iii
 Declaration

We declare that this written submission represents our ideas in our own
words and where others’ ideas or words have been included, we have adequately
cited and referenced the original sources. We also declare that we have adhered
to all principles of academic honesty and integrity and have not misrepresented
or fabricated or falsified any idea/data/fact/source in my submission. We
understand that any violation of the above will be cause for disciplinary action
by the Institute and can also evoke penal action from the sources which have
thus not been properly cited or from whom proper permission has not been taken
when needed.

Bhumit Malvi (62)

Sanket Suhagiya (76)

Rithwik Vedpathak (82)

Date:

iv
 Acknowledgements

It is said that “learning is a never-ending process.” While working on the project we

have undergone the same experience of learning new things as we proceeded in our goal of
building a Proctoring tool using Machine Learning.
Working on the project was a new experience for us. As it opened a new gateway
wherein, we had an opportunity to work on a totally new concept as far as the engineering
syllabus is concerned where most of the concepts are to be learned by rote.
The joy of working in a new domain and learning new things was a welcome experience
for the three of us and all we have to say is that we have cherished all the moments as they
came by, right from working on the project to making this report.
We would like to thank our Principal Dr. Harish Vankudre for constant motivation
and support to excel and having faith in our ability. We would also like to thank our professor
Dr. Megha Trivedi (Head - Department of Computer Engineering) for providing her views
on the subject.
We would like to thank Prof. Smita Jawale, who guided us and shared their knowledge
& invaluable experience about the topic and gave their precious time towards solving our
difficulties. We would also like to thank our college management for providing us with the
facilities and infrastructure for working on the project.

Bhumit Malvi (62)

Sanket Suhaigya (76)

Rithwik Vedpathak (82)

Date:

v
 Abstract

Due to the lockdown environment, the world has shifted its normal
practices to online modes. Due to which the exams for various schools, colleges
face the problem of using a large number of personnel to proctor the students
manually failing to ensure continuous integrity and security aspects of the
examination. These results encouraging examinees to use unethical methods to
score more in exams and falsely grading the knowledge and ability of a student.
Also, in general, in online examinations, a human cannot proctor at large scales,
effectively.
Thus, we are trying to create a proctoring system which can monitor at scale and
ensure that examinees don’t practice cheating or other types of unethical
behaviours.

vi
 Table of Contents

Chapter No. Title Pg. No.

Abstract vi

1 Introduction 10

1.1 Aim and Objectives 11

2 Literature Review 12

2.1 Critical Review 12

2.2 Analysis Table 15

2.3 Problem Definition 16

3 Project Description 17

3.1 Modules 17

3.2 Process Diagram 19

3.3 Analysis 20

3.3.1 Functional Requirements 20

3.3.2 Non-functional Requirements 20

3.4 Implementation Methodology 21

3.5 Code 23

4 Result and Discussions 26

4.1 Result 26

4.2 Future Scope 28

5 Conclusion 29

References 30

vii
 List of figures

Figure Number Description Page No.

3.1 Process Diagram. 19

3.2 Analysis of cheating methods used (a survey). 20

3.3 Face recognition algorithm processing. 21

3.4 Weightages for conditional algorithm. 22

4.1 Result of head pose estimation. 27

4.2 Result of suspicious behaviour detection. 28

viii
 List of Tables

Table No. Description Page No.

2.1 Analysis Table 15

ix
 Chapter 1

Introduction
The current pandemic situation has forced colleges and schools to advance their
ongoing curriculum. Technology played a pivotal role in leveraging the online mode of
learning when the lockdown restrictions took place. As a result, many educational
institutions transitioned to online test-taking as physical examinations were stalled,
resulting in higher demand for online proctoring tools. Also, the exams for various schools,
colleges face the problem of using a large number of personnel to proctor the students
manually failing to ensure continuous integrity and security aspects of the examination and
a human cannot proctor at large scales, effectively. This poses several issues like abnormal
behaviour by the student, failing to ensure continuous integrity and security aspects of the
examination like unauthorized access to different system components (e.g. cheating and
malpractices). Although it allows students to take their tests online in a remote location
while managing the integrity of the examination, this may also result in falsely grading the
knowledge and ability of a student. Proposed system automates the proctoring process at
scale with the aid of computers, and reduces the load on human proctors.

10
1.1 Aim and Objectives
Aim
To develop a computer vision enabled and AI powered system to aid a human
proctor in various types of online examination. We use multimedia streams like video
and audio from the user as our input data, which is further processed along with few
system variables. These inputs are prepared and useful information is extracted, and
finally fed into an algorithm which provides a probability of the user indulging into
malpractices.

Objectives
Develop a proctoring system which will identify cheating behaviour:

• Efficiently

• Accurately

• Consume less resources and

• User Friendly

11
 Chapter 2

Literature Review

An extensive Critical Review was conducted on 5 papers related to the field. The critical
review was noted and an analysis Table was made.

2.1 Critical Review

[0] A Systematic Review of Online Exams Solutions in E-Learning: Techniques, Tools, and
Global Adoption
1. In this paper a systematic review of existing literature was performed.
2. In this paper, 9 relevant questions were answered in the area of Online Proctoring:
• leading studies,
• online exam features,
• development approaches,
• techniques/algorithms,
• existing tools,
• datasets,
• country participation in research,

12
 • key factors towards global adoption and
• challenges in research

[1] FaceNet: A Unified Embedding for Face Recognition and Clustering

This paper discusses efficient CNN based approach for

• face verification (is this the same person),
• recognition (who is this person) and
• clustering (find common people among these faces).

[2] Toward constructing a secure online examination system

• A basic framework of secure Online Examination System which can solve security
problems with proper design of Online Examination webpage and network.
• The combination of firewall in the system server and proxy and MMC on the client system
can be a security guarantee for the online examination system.

[3] Automated Online Exam Proctoring

• In this paper various techniques and algorithms are discussed related to online proctoring,
including hardware requirements.
• Techniques include
o user verification,
o text detection,
o speech detection,
o active window detection,
13
 o gaze estimation,
o phone detection and
o cheating behaviour detection.

[4] Implementation of e-proctoring in online teaching: A Study about Motivational Factors

• This study has sought to locate the motivational factors determining the implementation of
the evaluation system.

• The list is made up of the following motivational factors:

o Quality management (QM),
o available information (AI),
o external conditioning (EC),
o trust (T),
o perceived compatibility (PC),
o perceived usefulness (PU),
o attitude (A) and intention (I)

14
 2.2 Analysis Table

In Analysis Table No. 2.1, a detailed analysis of the research papers has been conducted.

Table 2.1 Analysis Table

Title Technique(s) Used Conclusion

A Systematic Review of Literature survey. Found various existing online

Online Exams Solutions in E- proctoring tools and their
Learning: Techniques, Tools, features.
and Global Adoption. [0]

FaceNet: A unified embedding Convolutional Neural Face verification, recognition

for face recognition and Network. and identification of common
clustering. [1] people among these faces
was done.

Toward constructing a secure Network security. Security problem was

online examination system. [2] discussed and solutions were
proposed. Network security
was also explored.

Automated Online Exam Discussion on various Multimedia was used as basis

Proctoring. [3] techniques and algorithms for analytics for proctoring
for various features required system.
for proctoring.

Implementation of e- Locates the motivational Limited to study of

proctoring in online teaching: factors determining the motivational factors, which
A Study about Motivational implementation of the could be eliminated by future
Factors. [4] evaluation system. studies.

15
 2.3 Problem Definition

In online examinations, a human cannot proctor at large scales, effectively.

This poses several issues like abnormal behaviour by the student, failing to ensure continuous
integrity and security aspects of the examination like unauthorized access to different system
components (e.g., cheating and malpractices). This results in falsely grading the knowledge
and ability of a student.
Proposed system automates the proctoring process at scale with the aid of computers, and
reduces the load on human proctors.

16
 Chapter 3

Project Description
The chapter has a complete description of the working and Implementation of the Project.
A process diagram along with the algorithm used are shown here.

3.1 Modules

1) Hardware Components:
We have assumed that the user will be attending the exam on a relatively modern system
with Camera and microphone available. The major challenge with the hardware is that the
complete detection and validation process is going to be processed on the user’s client
device which has limited processing resources. Thus, we have created light weight
algorithms which can run on low spec devices. The computer will provide few important
system parameters, the camera will provide raw video stream and the microphone will
provide raw audio stream.

17
2) Speech Detection:
Speech is one of the communication mediums through which one can gain assistance for
exams. We assume that the examinee is sitting in a quiet room with no one around. The
major challenge in speech detection is false identification, since there might inheritably be
some kind of sound in the environment, identifying it as assistance or not-assistance is a
major challenge. We have tackled this issue with an audio processing algorithm developed
in house. It detects the changes in amplitude of the background noise. It detects frequency
of changes of amplitude relative to idle noise value to infer whether the examinee is or his
accomplice is talking.

3) Head-pose Detection: The best way the supervisors use to detect abnormal behaviour is
the check where the examinee is looking, we use the same concept in this project to detect
the direction/angle of the head of the examinee. The major challenge is the computer vision
algorithm used to detect the face using the images data as well as the direction of the head,
such algorithms usually require large amounts of training data as well as discrete graphic
hardware to work properly. We tackle this problem by using third party open-source
algorithms from MediaPipe library by google which is designed to work on low end
devices with accuracy at par. The MediaPipe library provides accurate face landmarks,
these face landmarks are then processed by the Perspective-n-Points algorithm
implemented with help of an open computer vision library to provide the 3D orientation of
the user's head.

4) Cheating Behaviour Detection: This is the ultimate prediction algorithm which provides
the final output of the analysis done. The major challenge is to contextually use the data
points collected from individual modules of speech and video detections to ignore false
positives. We process these extracted features on a light weight algorithm developed in
house which provides the probability whether the used is cheating. The audio is analysed
to detect the speech of the user using the amplitude of sound. Then this data is stored as a
flag which shows whether the user is speaking or not. Head pose detection is used to detect
whether the user is looking up, down, left or right. This is detected by applying a threshold
over the head angle. If the user exceeds the angle then the two flags are used to store the
head pose which are named x-axis and y-axis. Then the flags of x-axis, y-axis, sound and
the previous output is fed into a probabilistic conditional algorithm to predict if the user is
showing suspicious behaviour in terms of percentage of surety.

18
 3.2 Process Diagram

Figure 3.1 Process Diagram.

The process 1 does the head pose estimation of the module in which computer vision is used
to get the angle of the head. Process 2 is used for speech detection; the audio is analysed using a
sound processing algorithm which detects speech based on the amplitude of the sound recorded.
process 3 is used to make sure that the user is currently on the exam window. Data is then fed in
a conditional algorithm which gives the suspicious behaviour detection percentage based on set
weightage.

19
 3.3 Analysis

3.3.1 Functional Requirements

• Sensors to monitor the user.

• Processing the output of sensors.

• Giving feedback to the proctor based on sensor data.

3.3.2 Non-Functional Requirements

• Portability.

• Reliability and Security.

• Performance and Flexibility.

• Capacity and Scalability.

• Usability

Figure 3.2 Analysis of cheating methods used (a survey).

20
 3.4 Implementation Methodology

Three different methods are used in this proposed system which explain below:
• Head pose estimation
• Speech Detection
• Cheating Behaviour detection

For head pose estimation, we use OpenCV library for python for images capture and
prepossessing. The MediaPipe library by Google is used for face recognition. It is an open-source
ML library for various computer vision applications. We use the face recognition module to detect
the face in the captured image. Then we use the PnP library to calculate the angle of the head and
use that data to get the head angle data. We apply a threshold to the angle of the head pose on the
X and Y axis such as if the user looks beyond the threshold, then it is detected as looking beyond
the screen.

Figure 3.3 Face recognition algorithm processing.

21
 If the user is looking right or left then the value of x-axis changes. If the user is looking left
then it goes to the negative side of zero and if the user looks right then the value goes to the positive
side of zero. And similarly, if the user looks up or down then the y-axis value changes. The values
of x-axis and y-axis are then passed through an algorithm which checks whether it goes above the
threshold then it changes the flag.

We detect audio in student’s environment using sounddevice library in python. We use frames as
input, and get its amplitude. Number of frames is monitored continuously and its amplitude is
averaged out. If this average is greater than the threshold set by us over a period of time we consider
it as suspicious behaviour and feed appropriate weights to our algorithm for further processing.

For suspicious behaviour detection we have used different flags pertaining to different inputs and
these flags are then fed into a conditional algorithm which has set biases for different inputs. When
a condition is satisfied, the resulting percentage output is added with a factor to the previous
percentage to make the resulting percentage by time graph more continuous and not be in a step
graph. When the suspicious percentage is above the set threshold then it is detected as cheating.

Figure 3.4 Weightages for conditional algorithm.

22
3.5 Code
MAIN CODE
import audio
import head_pose
import detection
import threading as th

if __name__ == "__main__":
# main()
head_pose_thread = th.Thread(target=head_pose.pose)
audio_thread = th.Thread(target=audio.sound)
detection_thread = th.Thread(target=detection.run_detection)

head_pose_thread.start()
audio_thread.start()
detection_thread.start()

head_pose_thread.join()
audio_thread.join()
detection_thread.join()

23
AUDIO PROCESSING CODE
import sounddevice as sd
import numpy as np

# place holders and global variables

SOUND_AMPLITUDE = 0
AUDIO_CHEAT = 0

# sound variables
# SUS means next sound packet is worth analyzing
CALLBACKS_PER_SECOND = 38 # callbacks per sec(system dependent)
SUS_FINDING_FREQUENCY = 2 # calculates SUS *n* times every sec
SOUND_AMPLITUDE_THRESHOLD = 20 # amplitude considered for SUS calc

# packing *n* frames to calculate SUS

FRAMES_COUNT = int(CALLBACKS_PER_SECOND/SUS_FINDING_FREQUENCY)
AMPLITUDE_LIST = list([0]*FRAMES_COUNT)
SUS_COUNT = 0
count = 0

def print_sound(indata, outdata, frames, time, status):

avg_amp = 0
global SOUND_AMPLITUDE, SUS_COUNT, count,
SOUND_AMPLITUDE_THRESHOLD, AUDIO_CHEAT
vnorm = int(np.linalg.norm(indata)*10)
AMPLITUDE_LIST.append(vnorm)
count += 1
AMPLITUDE_LIST.pop(0)
if count == FRAMES_COUNT:
avg_amp = sum(AMPLITUDE_LIST)/FRAMES_COUNT
SOUND_AMPLITUDE = avg_amp
if SUS_COUNT >= 2:
#print("!!!!!!!!!!!! FBI OPEN UP !!!!!!!!!!!!")
AUDIO_CHEAT = 1
SUS_COUNT = 0
if avg_amp > SOUND_AMPLITUDE_THRESHOLD:
SUS_COUNT += 1
#print("Sus...", SUS_COUNT)
else:
SUS_COUNT = 0
AUDIO_CHEAT = 0
count = 0

def sound():
with sd.Stream(callback=print_sound):
sd.sleep(-1)

24
def sound_analysis():
global AMPLITUDE_LIST, FRAMES_COUNT, SOUND_AMPLITUDE
while True:
AMPLITUDE_LIST.append(SOUND_AMPLITUDE)
AMPLITUDE_LIST.pop(0)

avg_amp = sum(AMPLITUDE_LIST)/FRAMES_COUNT

if avg_amp > 10:

print("Sus...")

if __name__ == "__main__":
sound()

25
 Chapter 4

Result and Discussions

4.1 Result

After implementation of the project the project is evaluated and tested on various
parameters. The results are listed and outputs are plotted accordingly.

For speech recognition we use sounddevice library in python to measure the intensity
of the sound in the audio. We use OpenCV library for python for images capture and
prepossessing. The MediaPipe library by Google is used for face recognition. It is an open-
source ML library for various computer vision applications. We use the face recognition
module to detect the face in the captured image. Then we use PnP library to calculate the angle
of the head and use that data to get the head angle data. We apply a threshold to the angle of
the head pose on the X and Y axis such as if the user looks beyond the threshold, then it is
detected as looking beyond the screen.

26
 Figure 4.1 Result of head pose estimation.

For suspicious behaviour detection we used different flags pertaining to different inputs and
these flags are then fed into a conditional algorithm which has set biases for different inputs. When
a condition is satisfied, the resulting percentage output is added with a factor to the previous
percentage to make the resulting percentage by time graph more continuous and not be in a step
graph. When the suspicious percentage is above the set threshold then it is detected as cheating.
The output is then plotted in a graph with time on X-axis and the percentage on Y-axis. The graph
is continuous, which means it will update progressively as time goes by.

27
 Figure 4.2 Result of suspicious behaviour detection.

4.2 Future Scope

In time, technology, dataset, and confidence of user will rise so that AI will be able to make
autonomous judgments on the severity of a situation and then take appropriate action, such as
ending or pausing an assessment.

28
 Chapter 5

Conclusion

This system is one of the popular revisited topics due to pandemic and the need for people to
conduct online tests. This system aimed to detect whether the user is showing suspicious paper using
Video and audio output. During the making of the system, we used various machine learning
algorithms for head pose detections and successfully implemented head pose estimation using
computer vision as well as speech detection using microphone. We successfully developed a system
which can detect suspicious behaviour and it is a light weight, low resource consuming system.

29
 References

[0] A. W. Muzaffar, M. Tahir, M. W. Anwar, Q. Chaudry, S. R. Mir and Y. Rasheed, "A Systematic
Review of Online Exams Solutions in E-Learning: Techniques, Tools, and Global Adoption," in
IEEE Access, vol. 9, pp. 32689-32712, 2021, doi: 10.1109/ACCESS.2021.3060192.

[1] F. Schroff, D. Kalenichenko and J. Philbin, "FaceNet: A unified embedding for face recognition
and clustering," 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
2015, pp. 815-823, doi: 10.1109/CVPR.2015.7298682.

[2] Abdul Wahid, Yasushi Sengoku, and Masahiro Mambo. 2015. Toward constructing a secure online
examination system. In Proceedings of the 9th International Conference on Ubiquitous
Information Management and Communication (IMCOM '15). Association for Computing
Machinery, New York, NY, USA, Article 95, 1–8. https://doi.org/10.1145/2701126.2701203

[3] Y. Atoum, L. Chen, A. X. Liu, S. D. H. Hsu and X. Liu, "Automated Online Exam Proctoring," in
IEEE Transactions on Multimedia, vol. 19, no. 7, pp. 1609-1624, July 2017, doi:
10.1109/TMM.2017.2656064.

[4] González-González, C.S.; Infante-Moro, A.; Infante-Moro, J.C. Implementation of E-Proctoring

in Online Teaching: A Study about Motivational Factors. Sustainability 2020, 12, 3488.
https://doi.org/10.3390/su12083488

30
 Publications

1. Published on ResearchGate.
2. Pending approval at 7th International Conference on Communication and
Electronics Systems ICCES 2022

Competitions
1. “Online Exam Proctoring System”, VCET National Project Showcase 2022.

31