1

TRAITOR AND MISSING PERSON ANALYSIS

WITH VIDEO SURVEILLANCE USING FACE
RECOGNITION AND MATCHING

2
ABSTRACT

There is an abnormal increase in the traitor’s rate and also the number of missing person
rate is increasing, this leads towards a great concern about the security issues. Crime
preventions and criminal identification are the primary issues before the police
personnel, since property and lives protection are the basic concerns of the police but to
combat the crime, the availability of police personnel is limited. To avoid this situation
an automated facial recognition system for criminal identification is proposed using
Haar feature-based cascade classifier. This paper presents a real-time face recognition
with matching system using an automated surveillance camera. This system will be able
to detect and recognize face automatically in real-time. With the advent of security
technology, cameras especially CCTV have been installed in many public and private
areas to provide surveillance activities. The footage of the CCTV can be used to identify
suspects on scene. In this paper, an automated facial recognition system for criminal
database was proposed using known Haar feature-based cascade classifier. This system
will be able to detect face and recognize face automatically in real time. An accurate
location of the face is still a challenging task. Viola-Jones framework has been widely
used by researchers in order to detect the faces and objects with matching technology in
a given image. Face detection classifiers are shared by public communities, such as
OpenCV. Thus the criminal or the missing person will be known to the policemen where
they can catch them easily.

3
 TABLE OF CONTENTS

CHAPTER
TITLE PAGE
NO.
ABSTRACT 4

LIST OF FIGURES 7

LIST OF ABBREVIATIONS 8

1 INTRODUCTION 8

2 LITERATURE SURVEY 13
3 SYSTEM ANALYSIS 16

4 SYSTEM REQUIREMENTS 23

5 SOFTWARE DESCRIPTION 30

6 MODULES DESCRIPTION 39

7 RESULT ANALYSIS 47

8 CONCLUSION AND FUTURE ENHANCEMENT 50

9 APPENDIXES 51

10 REFERENCES 73

5
 LIST OF FIGURES

3.1 Face recognition and matching block diagram 18

3.2 System Architecture of the proposed face 23

detection
6.1 Face detection using HAAR cascades 40

6.2 Classifier system 41

6.3 Level 0 DFD 43

6.4 Level 1 DFD 44

6.5 Use Case Diagram 45

6.6 Tested face matching system faces 46

6.7 Face matching system using RNN 47

6.8 Face Matching Response Time 48

6
 LIST OF ABBREVIATIONS

MF Matrix Factorization

NN Nearest Neighbour

PAD Presentation Attack Detection

RS Recommender System

UML Unified Modeling Language

RNN Recurrent Neural Network

PCA Principal Component Analysis

DoG Difference of Guassian

7
 CHAPTER 1

1 INTRODUCTION

1.1 CRIMINAL RECORD STRUCTURED SYSTEM

Criminal record contains personal information about a particular person along with
photograph. To identify any criminal we need identification regarding that person, which
are given by the eyewitness. Identification can be done by finger print, eyes, DNA etc.
One of the applications is face identification. The face is our primary focus of attention in
social intercourse playing a major role in conveying identity and emotion. Although it is
difficult to infer intelligence or character from facial appearance, the human ability to
remember and recognize faces is remarkable.
1.2 FACE RECOGNITION
A face recognition system uses a database of images and compares another image
against those to find a match, if one exists. For each facial image, identification can be
done using the RGB values for the eye color, the width and height of the face and also
using various ratios Key goal of computer vision researchers is to create automated face
recognition systems that can equal, and eventually surpass, human performance. To this
end, it is imperative that computational researchers know of the keyfindings from
experimental studies of face recognition [1]. These findings provide insights into the
nature of cues that the human visual system relies upon for achieving its impressive
performance and serve as the building blocks for efforts to artificially emulate these
abilities. The face recognition problem has been studied for more than two decades. The
approaches proposed in the literature so far can mainly be classified into two categories:
model based and appearance. The model based method tries to extract geometrical
parameters measuring the facial parts while the appearance based approach use the
intensity or intensity-derived parameters such as eigen faces coefficients to recognize
faces. Due to the changes of lighting condition, expression, occlusion, rotation, etc., the
8
human face appearance could change considerably. There are existing approaches
proposed to recognize faces under varying pose. One is the Active Appearance Model
which deforms a generic face model to fit with the input image and uses the control
parameters as the feature vector to be fed to the classifier. The second approach is based
on transforming an input image to the same pose as the stored prototypical faces and then
using direct template matching to recognize faces, and later extended.
1.3 PROJECT DESCRIPTION
This system is aimed to identify the criminals in any investigation department. In this
system, we are storing the images of criminals in our database along with his details and
then these images are segmented into four slices- forehead, eyes, nose and lips. These
images are again stored in another database record so as to make the identification
process easier. Eyewitnesses will select the slices that appear on the screen and by using
it we retrieve the image of the face from the database. Thus this system provides a very
friendly environment for both the operator and the eyewitness to easily identify the
criminal, if the criminals record exists in the database. This project is intended to identify
a person using the images previously taken. The developed system is also a first
milestone for video based face detection and recognition for surveillance
1.4 FACE RECOGNITION APPROACH
Face recognition systems have been conducted now for almost 50 years. Face
recognition is one of the researches in area pattern recognition & computer vision due to
its numerous practical applications in the area of biometrics, Information security, access
control, law enforcement, smart cards and surveillance system. The first large scale
application of face recognition was carried out in Florida. Biometric-based techniques
have emerged as the most promising option for recognizing individuals in recent years
since, instead of certifying people and allowing them access to physical and virtual
domains.

9
 1.5 FACE DETECTION AND RECOGNITION

Face detection is more challenging because of some unstable characteristics, for

example, glasses and beard will impact the detecting effectiveness. Moreover, different
kinds and angles of lighting will make detecting face generate uneven brightness on the
face, which will have an influence on the detection process. An intensive study of
OpenCV platform and its inbuilt libraries has been conducted to generate a code, which
does correct and reliable facial recognition with new and efficient use of hardware. This
proposed system also acts as a homesecurity system for both Person detection and
provide security for door access control by using facial recognition for the home
environment.
The human body is identified as an intruder within a home environment achieved by
capturing live video from web camera and processing will be done on captured video
frames. The web camera to capture the series of images as soon as the person press
switch. The advantage of this system is for accessing the door is that face detection and
recognition are performed by using face detection technique and the entire face
recognition is completed by pressing single and tiny push button switch. Face recognition
includes feature extraction from the facial image, recognition or classification and feature
reduction. As the world is progressing people are scared about the safety of their
possessions, information and themselves. With the model of Smart Door, a profound
impact is expected in the security industry and it is somewhat anticipated as the time has
come to make all daily life objects interconnected and interactive. This model will be a
major contribution to the field of Home Security.
1.6 FACIAL RECOGNITION ALERT SYSTEM

In present world, security is the high rated concern of everybody. People can go to
every extent to safeguard themselves and their family. They hire security guards and
install cameras to protect themselves from theft. But sometimes installing cameras is not
sufficient. Hence, to ensure complete security, an integrated security system has been
10
proposed known as FRAS- Facial Recognition Alert System for home, bank, office etc.
1.7 METHODS OF FACE RECOGNITION

Face recognition technique can be implemented via various approaches like

Template Matching Methods: Here various different pattern of human face is saved
to identify facial expressions. The input image and saved image are matched for
detection.
Appearance Based Methods: In this, the model or templates are generated from a set
of training images. These generated images are then used for detection. Knowledge
Based Methods: This method includes the knowledge acquired to determine the facial
features. The rules used for this method are used to capture the relationship between
facial features.
Feature Invariant Approaches: In this approach detection of face starts with
extraction of features and finding the face of person and then verifying it with the person.
Different features exist during different pose or lighting conditions and this approach
focuses on extracting facial features during different conditions.
In Gray Scale Image, the value of each pixel is single sample. This sample contains
black and white intensity where black has weakest intensity and white has strongest
intensity. After that gray scale image is converted into binary image. This method is
called Binarization. This provides high Security Solution to people. The proposed system
will provide the ease to keep a check on every visitor and maintain a database.

11
 CHAPTER 2

LITERATURE SURVEY
2.1 Collaborative Filtering-Based Recommendation Of Online Social Voting
Xiwang Yang ; Chao Liang ; Miao Zhao ; Hongwei Wang ; Hao Ding ; Yong
Liu ; Yang Li ; Junlin Zhang
Social voting is an emerging new feature in online social networks. It poses unique
challenges and opportunities for recommendation. In this paper, we develop a set of matrix-
factorization (MF) and nearest-neighbor (NN)-based recommender systems (RSs) that
explore user social network and group affiliation information for social voting
recommendation. Through experiments with real social voting traces, we demonstrate that
social network and group affiliation information can significantly improve the accuracy of
popularity-based voting recommendation, and social network information dominates group
affiliation information in NN-based approaches. We also observe that social and group
information is much more valuable to cold users than to heavy users. In our experiments,
simple metapath-based NN models outperform computation-intensive MF models in hot-
voting recommendation, while users' interests for nonhot votings can be better mined by MF
models. We further propose a hybrid RS, bagging different single approaches to achieve the
best top-k hit rate.

12
 CHAPTER 3
SYSTEM ANALYSIS

3.1 EXISTING SYSTEM:

In the existing system, the identification of the spoofing user with matching the
database is analyzed with the generated system. Face presentation attack detection (PAD)
is essential for securing the widely used face recognition systems. UDA-Net performs
unsupervised adversarial domain adaptation in order to optimize the source domain and
target domain encoders jointly, and obtain a common feature space shared by both
domains.
Criminal record generally contains personal information about particular person
along with photograph. To identify any criminal we need some identification regarding
person, which are given by eyewitnesses [6]. Based on the details given by eyewitnesses,
the criminal who did the crime will be identified manually.
3.1.1 ALGORITHM
Principle Component Analysis (PCA)
Principal Component Analysis commonly uses the eigenfaces in which the probe
and gallery images must be the same size as well as normalized to line up the eyes and
mouth of the subjects whining the images. Approach is then used to reduce the dimension of
data by the means of image compression basics and provides most effective low dimensional
structure of facial pattern. This reduction drops the unuseful information and decomposes the
face structure into orthogonal (uncorrelated) components known as eigenfaces. Each face
image is represented as weighted sum feature vector of eigenfaces which are stored in 1-D
array. A probe image is compared against the gallery image by measuring the distance
between their respective feature vectors then matching result has been disclosed. The main
advantage of this technique is that it can reduce the data needed to identify the individual to
13
 1/1000th of the data presented.

DRAWBACK

• In most cases the quality and resolution of the recorded image segments are poor and hard
to identify a face.
• If a eyewitnesses observe a criminal only from single direction, it may not be possible to
recognize him.
• The photograph, which is a hard copy, cannot be able to divide or split into different
modules. So it is very difficult to find, unless we get full-fledged details.
• Sometimes the eyewitness may not be able to draw, the face of criminal
• Sometimes, if we maintain the criminal details manually and physically. After a time
span, the photographs and other details may tend to tear out.

14
3.2 PROPOSED WORK

In our proposed system, a wide presentation of the face domain adaptation and
detection has been added. The implementation will be given with two order:
identification of the criminals and identification of the missing person with the face
recognition. Here the ordered frame based conversion is adapted with the HAAR
cascades feature point extraction system. Feature extraction domain has plenty of
collection of generalized face features from several images of the same subject. Then,
each face image is processed, features are extracted and the collection of features are
analyzed and combined into a single generalized features collection, which is written to
the database. The face is our primary focus of attention in social inter course playing a
major role in conveying identification and emotion. Although the ability to infer
intelligence or character from facial appearance is a guess but still the human ability to
recognize faces is remarkable. This analogy would give us enough scope to envisage a
new algorithm. There are mainly three important ways in construction of the face i.e., by
using the eyewitness function, adding details and clipping image. This offers us a face as
finally identification parameter to know who has committed the crime.
Advantages
• The image of the unauthorized person will be captured and intimate it to the user.
• The image can be captured in from the video thus user are not in need of watching
it continuously.
• Less amount of storage needed because only the captured image is stored in the
database.
• User can immediately alert the nearby friends because they can get intimation with
that picture.

15
3.3 BLOCK DIAGRAM
A block diagram of the proposed face recognition system is shown in fig. 3.1. In this
figure the three key steps of face recognition, namely, face detection, facial feature
extraction and face recognition are identified. Detailed descriptions of these steps are
addressed in the following sub-sections.

Fig 3.1 Face recognition and matching block diagram

16
3.4 ARCHITECTURE DIAGRAM
System architecture can be defined as a structure composed of components, and rules
characterizing the interaction of these components. System architecture diagram is a
starting point of product description visualization – a product structure overview.
Three types of system architectures are identified, integrated, distributed and mixed,
(partly integrated and partly distributed). It is shown that the type of interfaces defines
the type of architecture. Integrated systems have more interfaces, which furthermore are
vaguely defined. An application architecture diagram provides a high-level graphical
view of the application architecture, and helps you identify applications, sub-applications,
components, databases, services, etc, and their interactions. See Application and Business
Services (EAM). System - A packaged application.
If you restricted yourself to four layers, they may be defined as:
1) Algorithm,
2) Programming language/compiler,
3) processor/memory,
4) I/O. Other abstraction definitions may contain three layers:
1) Application,
2) System software,
3) Hardware
Various organizations define systems architecture in different ways, including:
• An allocated arrangement of physical elements which provides the design solution
for a consumer product or life-cycle process intended to satisfy the requirements of the
functional architecture and the requirements baseline.
• Architecture comprises the most important, pervasive, top-level, strategic
inventions, decisions, and their associated rationales about the overall structure (i.e.,
essential elements and their relationships) and associated characteristics and behavior.

17
• If documented, it may include information such as a detailed inventory of current
hardware, software and networking capabilities; a description of long-range plans and
priorities for future purchases, and a plan for upgrading and/or replacing dated equipment
and software.

An architecture diagram is a graphical representation of a set of concepts that are part of

architecture, including their principles, elements and components. Architecture diagram
can help system designers and developers visualize the high-level, overall structure of
their system or application, in order to ensure the system meets their users' needs. Using
architecture diagram, you can also describe patterns that are used throughout the design.
It's somewhat like a blueprint that you use as a guide, so that you and your colleagues can
discuss, improve and follow.

18
Fig 3.2 System Architecture of the proposed face detection

19
 CHAPTER 4
SYSTEM REQUIREMENTS
4.1 HARDWARE REQUIREMENTS
• Processor : Intel processor 3.0GHz

• RAM : 2GB

• Hard disk : 500 GB

• Compact Disk : 650 Mb

• Keyboard : Standard keyboard

• Mouse : Logitech mouse

• Monitor : 5 inch color monitor

4.2 SOFTWARE REQUIREMENTS

• Front End : PYTHON

• Back End : MYSQL

• Operating System : Windows OS

• Server : WAMP

• IDE : Jet Beans PyCharm 2018.3

20
 CHAPTER 5

SOFTWARE DESCRIPTION

5.1 FRONT END (Python):

Python is an interpreted, object-oriented, high-level programming language with dynamic

semantics. Its high-level built in data structures, combined with dynamic typing and dynamic
binding, make it very attractive for Rapid Application Development, as well as for use as a
scripting or glue language to connect existing components together. Python's simple, easy to
learn syntax emphasizes readability and therefore reduces the cost of program maintenance.
Python supports modules and packages, which encourages program modularity and code reuse.
The Python interpreter and the extensive standard library are available in source or binary form
without charge for all major platforms, and can be freely distributed.
Often, programmers fall in love with Python because of the increased productivity it provides.
Since there is no compilation step, the edit-test-debug cycle is incredibly fast. Debugging
Python programs is easy: a bug or bad input will never cause a segmentation fault. Instead,
when the interpreter discovers an error, it raises an exception. When the program doesn't catch
the exception, the interpreter prints a stack trace. A source level debugger allows inspection of
local and global variables, evaluation of arbitrary expressions, setting breakpoints, stepping
through the code a line at a time, and so on. The debugger is written in Python itself, testifying
to Python's introspective power. On the other hand, often the quickest way to debug a program
is to add a few print statements to the source: the fast edit-test-debug cycle makes this simple
approach very effective.
SOFTWARE SUPPORTS
INSTALLATION PROCEDURE
Introduction
Python is a widely used high-level programming language first launched in 1991. Since then,
Python has been gaining popularity and is considered as one of the most popular and flexible
21
server-side programming languages.

Unlike most Linux distributions, Windows does not come with the Python programming
language by default. However, you can install Python on your Windows server or local
machine in just a few easy steps.
PREREQUISITES
• A system running Windows 10 with admin privileges
• Command Prompt (comes with Windows by default)
• A Remote Desktop Connection app (use if you are installing Python on a remote
Windows server)

5.2 BACK END (Wamp Server)

MySQL is the world's most used open source relational database management system
(RDBMS) as of 2008 that run as a server providing multi-user access to a number of
databases. The MySQL development project has made its source code available under the
terms of the GNU General Public License, as well as under a variety of proprietary
agreements. MySQL was owned and sponsored by a single for-profit firm, the Swedish
company MySQL AB, now owned by Oracle Corporation.

MySQL is a popular choice of database for use in web applications, and is a central
component of the widely used LAMP open source web application software stack—LAMP
is an acronym for "Linux, Apache, MySQL, Perl/PHP/Python." Free-software-open source
projects that require a full-featured database management system often use MySQL.

For commercial use, several paid editions are available, and offer additional
functionality. Applications which use MySQL databases include: TYPO3, Joomla, Word
Press, phpBB, MyBB, Drupal and other software built on the LAMP software stack.
MySQL is also used in many high-profile, large-scale World Wide Web products, including
22
Wikipedia, Google (though not for searches), Imagebook, Twitter, Flickr, Nokia.com, and
YouTube.

Inter images
MySQL is primarily an RDBMS and ships with no GUI tools to administer MySQL
databases or manage data contained within the databases. Users may use the included
command line tools, or use MySQL "front-ends", desktop software and web applications
that create and manage MySQL databases, build database structures, back up data, inspect
status, and work with data records. The official set of MySQL front-end tools, MySQL
Workbench is actively developed by Oracle, and is freely available for use.

Graphical

The official MySQL Workbench is a free integrated environment developed by

MySQL AB, which enables users to graphically administer MySQL databases and visually
design database structures. MySQL Workbench replaces the previous package of software,
MySQL GUI Tools. Similar to other third-party packages, but still considered the
authoritative MySQL frontend, MySQL Workbench lets users manage database design &
modeling, SQL development (replacing MySQL Query Browser) and Database
administration (replacing MySQL Administrator).MySQL Workbench is available in two
editions, the regular free and open source Community Edition which may be downloaded
from the MySQL website, and the proprietary Standard Edition which extends and improves
the feature set of the Community Edition.

Command line

MySQL ships with some command line tools. Third-parties have also developed tools
to manage a MySQL server, some listed below. Maatkit - a cross-platform toolkit for
MySQL, PostgreSQL and Memcached, developed in Perl Maatkit can be used to prove
replication is working correctly, fix corrupted data, automate repetitive tasks, and speed up
23
servers. Maatkit is included with several GNU/Linux distributions such as CentOS and
Debian and packages are available for Programming. MySQL works on many different
system platforms, including AIX, BSDi, FreeBSD, HP-UX, eComStation, i5/OS, IRIX,
Linux, Mac OS X, Microsoft Windows, NetBSD, Novell NetWare, OpenBSD, OpenSolaris,
OS/2 Warp, QNX, Solaris, Symbian, SunOS, SCO Open Server, SCO UnixWare, Sanos and
Tru64. A port of MySQL to OpenVMS also exists.

MySQL is written in C and C++. Its SQL parser is written in yacc, and a home-
brewed lexical analyzer. Many programming languages with language-specific APIs include
libraries for accessing MySQL databases. These include MySQL Connector/Net for
integration with Microsoft's Visual Studio (languages such as C# and VB are most
commonly used) and the JDBC driver for Java. In addition, an ODBC interim age called
MyODBC allows additional programming languages that support the ODBC inter image to
communicate with a MySQL database, such as ASP or ColdFusion. The HTSQL - URL-
based query method also ships with a MySQL adapter, allowing direct interaction between a
MySQL database and any web client via structured URLs.

Features

As of April 2009, MySQL offered MySQL 5.1 in two different variants: the open source
MySQL Community Server and the commercial Enterprise Server. MySQL 5.5 is offered
under the same licenses. They have a common code base and include the following features:

• A broad subset of ANSI SQL 99, as well as extensions

• Cross-platform support
• Stored procedures
• Triggers
• Cursors

24
• Updatable Views
• Information schema
• Strict mode (ensures MySQL does not truncate or otherwise modify data to conform to an
underlying data type, when an incompatible value is inserted into that type)
• X/Open XAdistributed transaction processing (DTP) support; two phase commit as part
of this, using Oracle's InnoDB engine
• Transactions with the InnoDB, and Cluster storage engines
• SSL support
• Query caching
• Sub-SELECTs (i.e. nested SELECTs)
• Replication support (i.e. Master-Master Replication & Master-Slave Replication) with
one master per slave, many slaves per master, no automatic support for multiple masters per
slave.
• Full-text indexing and searching using MyISAM engine
• Embedded database library
• Partititoned tables with pruning of partitions in optimiser
• Shared-nothing clustering through MySQL Cluster
• Hot backup (via mysqlhotcopy) under certain conditions

Multiple storage engines, allowing one to choose the one that is most effective for each table
in the application (in MySQL 5.0, storage engines must be compiled in; in MySQL 5.1,
storage engines can be dynamically loaded at run time): Native storage engines (MyISAM,
Falcon, Merge, Memory (heap), Federated, Archive, CSV, Blackhole, Cluster, EXAMPLE,
Maria, and InnoDB, which was made the default as of 5.5). Partner-developed storage
engines (solidDB, NitroEDB, ScaleDB, TokuDB, Infobright (formerly Brighthouse),
Kickfire, XtraDB, IBM DB2). InnoDB used to be a partner-developed storage engine, but
with recent acquisitions, Oracle now owns both MySQL core and InnoDB

25
 CHAPTER 6

MODULES DESCRIPTION

• Video Acquisition

• Frame Conversion

• Face registration

• Face detection and recognition

• Face capture

• Criminal and missing person detection

• Alert

6.1 VIDEO ACQUISITION

Video capture is the process of converting an analog video signal such as that
produced by a video camera, DVD player, or television tuner to digital video. The
resulting digital data are referred to as a digital video stream, or more often, simply
video stream. Thus the input of the system is given with a video frame system

6.2 FRAME CONVERSION

Frame conversion is the process where the videos will be converted as frames.
Frames for an example the video with 60 sec where each will be converted as 60 frames.
Each frame will be shown as image with the image the face matching are made

6.3 FACE REGISTRATION

Here the user will register their face where the face will be getting with the
registration process. The user will register with the mail id and the faces which are to be
known with the user. The user will further preceded with the identification system
26
where the user face features will be captured. The face features are get known with the
user and the face features are further preceded for face detection.
Access control includes offices, computers, phones, ATMs, etc. Most of these
forms currently do not use face recognition as the standard form of granting entry, but
with advancing technologies in computers along with more refined algorithms, facial
recognition is gaining some traction in replacing passwords and fingerprint scanners.
Ever since the events of 9/11 there has been a more concerned emphasis on developing
security systems to ensure the safety of innocent citizens. Namely in places such as
airports and border crossings where identification verification is necessary, face
recognition systems potentially have the ability to mitigate the risk and ultimately
prevent future attacks from occurring. As for surveillance systems, the same point can.
6.3 FACE DETECTION AND RECOGNITION
Facial detection via the Viola-Jones algorithm is a common method used due to
its high detection rate and fast processing speed. The algorithm can be summed up in
four steps: feature selection, feature evaluation, feature learning to create a classifier,
and cascading classifiers. Simple features are used, inspired by Haar basis functions,
which are essentially rectangular features in various configurations. A two-rectangle
feature represents the difference between the sum of the pixels in two adjacent regions
of identical shape and size. This idea can be extended to the three-rectangle and four-
rectangle features. In order to quickly compute these rectangle features, an alternate
representation of the input image is required, called an integral image.
The learning portion of the face detection algorithm uses Adaboost which
basically uses a linear combination of weak classification functions to create a strong
classifier. Each classification function is determined by the perceptron which produces
the lowest error. However, this is characterized as a weak learner since the classification
function does not classify the data well. In order to improve results, a strong classifier is
created after multiple rounds of re-weighting a set weak classification functions. These

27
weights of the weak classification functions are inversely proportional to their errors.
The goal of this stage is to train the most relevant features of the face and to disregard
redundant features. The last step of the Viola-Jones algorithm is a cascade of classifiers.
The classifiers constructed in the previous step form a cascade. In this set up structure,
the goal is to minimize the computation time and achieve high detection rate. Sub-
windows of the input image will be determined a face or non-face with classifiers of
increasing complexity. If a there is a positive result from the first classifier, it then gets
evaluated by a second more complex classifier, and so on and so forth until the sub-
window is rejected. By doing this, the structure utilizes the early stages of the cascade in
order to reject as many negatives as possible
Here the face detection technique is enhanced using the extracted features of the
human. Face detection is further classified as face detection in images and real-time face
detection. In this project we will attempt to detect faces in video based detection system.
To do this it would be useful to study the Haar cascades face detection. In this research,
we implement Haar Cascade Classifier to detect human faces using an Open Source
Computer Vision Library. For human face detection, haar features are the main part of
haar cascade classifier

Fig 6.1 Face detection using HAAR cascades

28
6.4 FACE CAPTURE
As the face get detected at the registration process called training and recognized
at the face recognition process called testing. The recognition can be processed as
known and unknown person where the face with the registered one are get as known and
the remaining face as unknown. Here the known person is leave aside where the
unknown person face gets captured by the user. So that the unknown person cannot
enter the home
Haar Classifier is generated by training few hundred sample views of a particular
image, called positive examples, that are scaled to same size, and negative examples-
arbitrary image of the same size. After a classifier is trained, it is applied to region of
interest in an input image. The classifiers outputs as “1” if the region is likely to show
the object (i.e., image), and “0” otherwise. Here the word ‘cascade’ means that final
classifier consists of various simple classifier that are applied one after the other to the
area of interest till the detected face is rejected or passed completely.

Fig 6.2 Classifier system

When any of these features is found, the algorithm allows face candidate to move
on to the next stage of detection. A face candidate [2] is a rectangular part of the original
image called sub-window. This sub-window is arranged to attain a variety of different
size of faces. The algorithm uses an internal image [3] in order to process Haar features
of face candidate in constant time. It uses a cascade of stages which is used to remove

29
 non-face candidate immediately. Each stage contains different Haar features and each of
these features is classified by Haar feature classifier.
6.5 CRIMINALS AND MISSING PERSON DETECTION
After the database matching RNN classification system the matched features are
known. Thus the matched feature value will get the image of the missing person and
criminals details. The image analysis will give the complete details of the person.
6.5 ALERT
An, intimation will be sent to the user to their registered mail with that captured
image. Thus after the intimation the user can alert the nearby neighbor’s regarding these
suspicious activity occurring in the user home. The user intimation will be made only
when the user gets the person is unknown. Here the intimation is sent using the E-mail
method system where captured image will be sent. Here it makes an intimation system
so that the user can alert. The image of the unknown person given as a proof in their
email and they can alert their nearby people.

6.6 DATA FLOW DIAGRAM

A two-dimensional diagram explains how data is processed and transferred in a

system. The graphical depiction identifies each source of data and how it interacts with other
data sources to reach a common output. Individuals seeking to draft a data flow diagram must
identify external inputs and outputs, determine how the inputs and outputs relate to each other,
and explain with graphics how these connections relate and what they result in. This type of
diagram helps business development and design teams visualize how data is processed and
identify or improve certain aspects.
LEVEL 0:
DFD Level 0 is also called a Context Diagram. It’s a basic overview of the whole system or
process being analyzed or modeled. It’s designed to be an at-a-glance view, showing the
system as a single high-level process, with its relationship to external entities. It should be
30
easily understood by a wide audience, including stakeholders, business analysts, data analysts
and developers.

Face
Recognitionc

Traitor detection
riminal Police
and recognition

Database

Fig 6.3 LEVEL 0 DFD

LEVEL 1:
DFD Level 1 provides a more detailed breakout of pieces of the Context Level Diagram.
You will highlight the main functions carried out by the system, as you break down the high-
level process of the Context Diagram into its sub – processes. A level 1 data flow diagram
(DFD) is more detailed than a level 0 DFD but not as detailed as a level 2 DFD. It breaks down
the main processes into sub processes that can then be analyzed and improved on a more
intimate level.

31
 Face
Police Criminal recognition

Email Train CCTV Face pattern

Capture

Face train face Haar

detection cascades

face
CCTV view matching Matching

Mail

TRAITOR DETECTION
AND MATCHING

Fig.6.4 Level 1 DFD

32
6.7 UML DIAGRAM AND USE CASE DIAGRAM
The Unified Modeling Language (UML) is a general-purpose, developmental,
modeling language in the field of software engineering that is intended to provide a
standard way to visualize the design of a system.
Use case diagrams are usually referred to as behaviour diagrams used to describe
a set of actions (use cases) that some system or systems (subject) should or can perform
in collaboration with one or more external users of the system (actors).

Fig 6.5 :Use case diagram

33
 CHAPTER 7
RESULT ANALYSIS
A face image database was created for the purpose of benchmarking the face
recognition system. The image database is divided into two subsets, for separate training
and testing purposes. During SOM training, 30 images were used, containing six subjects
and each subject having 5 images with different facial expressions. fig 7.1 shows the
training and testing image database constructed.

Fig 7.1 Tested face matching system faces

RNN is a scale and in-plane rotation invariant detector and descriptor. [16] SURF

34
detectors are find the interest points in an image, and descriptors are used to extract the
feature vectors at each interest point just as in SIFT. RNN uses Hessian-matrix
approximation to locate the interest points instead of difference of Gaussians (DoG) filter
used in SIFT. RNN as a descriptor uses the first-order Haar wavelet responses in x and y,
whereas the gradient is used by SIFT. RNN usually uses 64 dimensions in Gaussian
extraction to reduce the time cost for both feature matching and computation. SURF has
three times better performance as compared to SIFT.

Fig 7.2 Face matching system using RNN

The cascade classifier contains a list of stages, where each stage consists of a list of
weak learners. The system detects the required object by moving a window over the image.
Each stage of the classifier labels the specific region defined by the current location of the
window as either positive or negative where positive means that an object was found and
negative means that the specified object was not found in the image . If the labelling yields
a negative result, then the classification of that particular region is over and the location of
the window is moved to the next location. If the labelling gives a positive result, then the
region moves to the next stage of classification. The classifier yields a final result as
positive, when all the stages, including the last one, yield a positive result, which implies
that the required object is found in the image.
35
 Fig 7.3 Face matching response time

The process of identifying and spotting a criminal is slow and difficult. Criminals,
these days are getting smarter by not leaving any form of biological evidence or fingerprint
impressions on the crime scene. A quick and easy solution is using state-of-the-art face
identification systems. With the advancement in security technology, CCTV cameras are
being installed at most of the buildings and traffic lights for surveillance purposes. The
video footage from the camera can be used to identify suspects, criminals, runaways,
missing persons etc. This paper explores a way to develop a criminal identification system
using ML and deep neural networks. The following method can be used as an elegant way
to make law enforcement hassle-free.

36
 CHAPTER 8

CONCLUSION AND FUTURE SCOPE

8.1 CONCLUSION
In this project, we implemented a facial recognition system using a global-approach to
feature extraction based on Histogram-Oriented Gradient. We then extracted the feature
vectors for various faces from the AT&T and Yale databases and used them to train a binary-
tree structure Haar Cascades learning model. Running the model on both databases resulted in
over 90% accuracy in matching the input face to the correct person from the gallery. We also
noted one of the shortcomings of using a global approach to feature extraction, which is that a
model trained using a feature vector of the entire face instead of its geometrical components
makes it less robust to angle and orientation changes. However, when the variation in facial
orientation is not large, the global-approach is still very accurate and simpler to implement
than component-based approaches
8.2 FUTURE ENHANCEMNT
In Future enhancement the system can be build the neural network model, there are many
parameters which can be tuned to increase the model performance. We can keep tuning our
models to increase its accuracy. Also, for a trained base model, we can re-train it using a
specific dataset. So another way to increase the whole system’s performance is to capture the
specific people’s images and re-train the model based on this small dataset. For example, if an
organization with 3000 people uses this system, the model can be trained to be very accurate
on these 3000 people. We can employ and automate this feature into the s

9.2 SCREENSHOT

37
38
39
40
41
 CHAPTER 10

REFERENCES

1. Kirti Dang, Shanu Sharma, "Review and Comparison of Face Detection Algorithms ",
International Conference on Cloud Computing, Data Science & Engineering - Confluence,
2017
2. S.Z. Li, Juwei Lu, "Generalizing capacity of face database for face recognition", IEEE
International Conference on Automatic Face and Gesture Recognition, 2002
3. Quanyou Zhao, Shujun Zhang, A Face Detection Method Based on Corner Verifying ,
International Conference on Computer Science and Service System (CSSS),2011
4. Dr. P.Shanmugavadivu , Ashish Kumar , Rapid Face Detection and Annotation with
Loosely Face Geometry, 2016
5. Chang Huang, Haizhou Ai, "High-Performance Rotation Invariant Multiview Face
Detection" , Ieee Transactions On Pattern Analysis And Machine Intelligence, 2007

42