IJCSN - International Journal of Computer Science and Network, Volume 7, Issue 6, December 2018

ISSN (Online): 2277-5420

www.IJCSN.org
Impact Factor: 1.5

Design and Implementation of the Smart Door

Lock System with Face Recognition Method
using the Linux Platform Raspberry Pi
1
Sourav Roy; 2 Md Nasir Uddin; 3 Md Zahirul Haque; 4 Md Jahidul Kabir

1, 2, 3, 4
School of Computer Science,
Nanjing University of Posts and Telecommunications,
Nanjing 210023, China

Abstract - Privacy and Security are two universal rights and, to ensure that in our daily life we are secure, a lot of research
is going on in the field of home security, and IoT is the turning point for the industry, where we connect everyday objects to
share data for our betterment. House security matters and people always try to make life easier at the same time. That’s why
we put up with this project, Face Recognition Door Lock System. Facial recognition is a well-established process in which
the face is detected and identified out of the image. We aim to create a smart door, which secures the gateway on the basis of
who we are. We want to develop this system based on Raspberry-pi 3, to make the house only accessible when your face is
recognized by the recognition algorithms from Open CV library and meanwhile you are allowed in by the house owner, who
could monitor entrance remotely. By doing so, the system is less likely to be deceived: since the owner can check each
visitor in the remote console, getting recognized by the camera using a photo won’t work. I want to add passcode function
for entrance in case that face recognition part corrupts.

Keyword - Raspberry Pi, Facial Recognition Door, Home Security, IoT.

1. Introduction
from the user or adding Infra-Red or Bluetooth
n today’s world of connectivity and smart modules to operate these devices.
For face recognition, an image will be captured by a

I devices there is an urgent need to modify our

existing day to day objects and make them smart,
also it is not the era when we can blindly trust the
old and conventional security measures,
specifically speaking is our door locks. To change
pi camera and pre-processed by Raspberry pi like
converting, re-sizing and cropping. Then face
detection and recognition are performed. Once the
face is recognized by the classifier based on a pre-
and modernize any object we need to eliminate its stored image library, the image will be sent to a
existing drawbacks and add extra functionality. remote console waiting for house owner’s decision.
For the passcode part, users could enter or reset
The major drawbacks in a common door lock are that passcode through a keypad.
anyone can open a conventional door lock by
duplicating or stealing the key and its simply 2. Related Literature and Comparison
impossible if we want our friends and family to enter with Existing Models
our house, without being actually present over there.
Thus why not just eliminate these problems. So, to
simply convert this normal door lock into a smart Since 2010 the industry has seen a dawn of work
lock, which can open the door whenever we turn up being done in fields of Artificial Intelligence,
in front of the gate or when we want it to open up for Machine Learning, Neural Networks, IoT, Big Data
someone else without being physically present, we Analytic all with a common goal to make things
need to modify the door. So an era has come where easier, self-supervising and to interconnect all kinds
devices can interact with its users and at the same of devices by making everyday objects
time ensure of their safety and keep improvising interconnected and interoperable.
themselves.
A need has been felt in the field of digitalizing
Users could operate on a touchscreen to select conventional security tools and thus a lot of work has
entering the house by recognizing the face or motors been modeled on making daily life locks smart by
and/or adding a digital number pad to take inputs introducing locks movable with the help of stepper

382

Copyright (c) 2018 International Journal of Computer Science and Network. All Rights Reserved.
IJCSN - International Journal of Computer Science and Network, Volume 7, Issue 6, December 2018
ISSN (Online): 2277-5420
www.IJCSN.org
Impact Factor: 1.5

An intensive study of literates implementing 4. Proposed Methodology

Smart Locks had been done and literature
implementing Door Locks with the help of GSM In order to implement the smart door model we
phones [1] and stepper motors have been studied. need a list of materials which is briefly
Also, literature regarding smart display have mentioned below:
been thoroughly reviewed [2]. The fault in
existing models is a complexity of a system and
unnecessarily relying on extra components. Our 4.1 Raspberry Pi
model is unique with its one of a kind
combination of functionalities offered and the Raspberry Pi (RP) is an ARM-based single
simplicity of the model. A major difference is in board computer. The Raspberry Pi 3 Model B is
the overhead reduction by the application as it the third generation Raspberry Pi [3].It has
detects the face out of the image and sends it Broadcom BCM2837 64bit ARM Cortex-A53
directly to application program interfaced with Quad Core Processor SoC running at 1.2GHz
our application, which has not been provided in and 1GB RAM. The operating system used for
any existing model and the efficient use of Raspberry Pi is Raspbian as it is open source
solenoids, which also eliminates the use of anyone can use. Raspbian is a Linux-based
stepper motors. So, we have avoided the use of computer operating system. It has 40 pins in
unnecessary components like stepper motors and which 24 are GPIO pins these pins are used for
drivers as done in existing models and also we general purpose, 8 ground pins, two of each 5V
have given newer and unprecedented features of and 3V power pin. It has four USB-2 ports and a
facial recognition as an access point control Micro USB power source. It runs on the 5V
system with a combination of relay module with power supply. Additionally, it adds wireless
a solenoid to open the gate and unique and LAN (BCM43143 WiFi on board (802.11
interactive User Interface. Also rather than using a/b/g/n)) and Bluetooth connectivity making it
a low-quality Raspberry Pi Interfaced Camera the ideal solution for powerfully connected
we have used USB attachable HD WebCam to designs.
do efficient and reliable facial recognition.
4.2 PI CAMERA
The objectives of the proposed work is to
implement a working model of a smart door and Pi Camera Module is a custom designed add-on
to give solutions to the problem faced by people for Raspberry Pi. This interface uses the
in day to day incidents of burglary or losing the dedicated CSI interface, which was designed
key and also to promote and ignite the work especially for interfacing to cameras [4] The CSI
being done on IOT systems and implementing it bus is capable of extremely high data rates, and
with the help of key research areas of Neural it exclusively carries pixel data. The sensor itself
Networks and IoT APIs and protocols. has a native resolution of 5 megapixels and has a
fixed focus lens on board. In terms of still
This model is allowing people to add more images, the camera is capable of 2592 x 1944-
functionality to it and thus induce more research pixel static images, and supports 1080p30,
work in the field of AI, Machine Learning, IoT 720p60 and 640x480p60/90 video.
and lot more.
4.3 KEYPAD
3. System Design and Framework
Attach matrix 7-pin interfaces to 7 free GPIO
pins, 3 column pins are set as output which are
directly connected with GPIO, while 4 row pins
are set as input with pull-up resistor.

4.4 SERVO

Attach the servo to a GPIO (we selected GPIO

17 here) of the Raspberry pi A and control its
rotation utilizing pulse-width modulation. The
servo is powered by a 6V-battery pack.

Figure1:Framework

383

Copyright (c) 2018 International Journal of Computer Science and Network. All Rights Reserved.
IJCSN - International Journal of Computer Science and Network, Volume 7, Issue 6, December 2018
ISSN (Online): 2277-5420
www.IJCSN.org
Impact Factor: 1.5

This proposed system uses Haar Cascades

classifier as a face detection algorithm [5].
Firstly, the algorithm needs a lot of positive
images and negative images to train the Haar
cascades classifier. Positive images are images
with clear faces where negative images are those
without any faces. Haar cascades are similar to
the convolutional kernel which are shown below
in figure 3.

Each feature is represented as a single value

obtained from the difference of the sums of
5. Software and Algorithm pixels in white rectangle from the sum of all
pixels in the black rectangle. All different
5.1 OpenCV possible sizes and locations of classifier is used
for calculating of plenty of features. As the
OpenCV is an open source computer vision number of classifiers increase the arithmetic
software library. The library has a lot of computations seems to take a long time. Instead
optimized algorithms, which can be used in of it, the concept of Integral Image has been
many IOT related sectors including face used. Image Processing Integral image is a data
detection and recognition. As the libraries of our structure which is a summed area table and
project we liked to use the Haar classifier, LBPH algorithm for quickly and efficiently generating
(Lower Binary Pattern histogram) face sum of values in a rectangular grid subset.
recognizer. Integral image is derived by using following
equation,
5.2 Image processing

Image processing is a mathematically intensive

operation & one of the biggest areas of research
for a big data field. It is a process on the image To avoid the complexity of calculation we use
to convert it into desired looking for which the Adaboost machine learning algorithm, which is
input is an image and the output may be an inbuilt in OpenCV library that is cascade
image or set of characters related to the classifier, to eliminate the redundancy of the
particular image. It refers to a variety of classifiers. Any classifier which has a probability
techniques that are used to maximize the of 50% of more in detection is treated as weak
information yield from a picture. classifier. The Sum of all weak classifier gives a
strong classifier which makes the decision about
5.3 Haar Cascade Classifier Algorithm detection. Although it is very vague to classify
with one strong classifier we use the cascade of
A Haar Cascade is basically a classifier which is classifiers. Classification takes place in stages, if
used to detect the object for which it has been the selected region fails in the first stage, we
trained for, from the source. discard it. We don’t use the classifiers on that
region which is discarded. The region which
passes all the stages i.e. all strong classifiers is
treated as the detected face. Detected Faces are
passed to the Face recognition phase. Local
Binary Patterns histogram algorithm (LBPH) has
been used for face recognition. Local binary
patterns are simple at the same time very
efficient texture operator which assigns the
pixels of the image by comparing with the
adjacent pixels as threshold and which results in
a binary result. The detected integral image is
subjected to this Local binary pattern. Face
recognition is extremely vulnerable to the
environment changes like brightness, facial
expressions and position. Face preprocessing is
the module which reduces the problems that
makes the picture unclear to recognize the face

384

Copyright (c) 2018 International Journal of Computer Science and Network. All Rights Reserved.
IJCSN - International Journal of Computer Science and Network, Volume 7, Issue 6, December 2018
ISSN (Online): 2277-5420
www.IJCSN.org
Impact Factor: 1.5

such as less brightness and contrast problems For better working, each stage of the cascade
and noise in the image and make sure the facial must have a low false negative rate, because if
features always be in a constant position. In this the actual object is classified as a non-object,
project we use histogram equalization for face then the classification of that branch stops, with
preprocessing. For efficiency we use separate no way to correct the mistake made. However,
preprocessing which is histogram equalization each stage can have a relatively high false
for left and right face. So histogram equalization positive rate, because even if the n-th stage
is done three times, firstly for the whole face and classifies the non-object as actually being the
the other two for side faces [6]. object, then this mistake can be fixed in n+1-the
and subsequent stages of the classifier [7].

6. TEST and Implementation

6.1 User Interfaces:

For user interfaces, we utilized python library
‘pygame’ to design the aesthetics of PiTFT
touchscreen and detect any user input.

6.1.1 Rasp Pi A: The way we linked

touchscreen buttons with system functions on Pi
A is displayed in Figure 5.

Figure 5: System Algorithm

the door is already locked, nothing will happen
Initially, on the PiTFT screen, there are 3 when the ‘Lock’ button is pressed.
buttons in the ‘main’ level: ‘Lock’, ‘Recognize
Face’ and ‘Enter Passcode’. The ‘Lock’ button is Once the ‘Face Recognize’ button is pressed,
designed for house owners to know whether the you could preview pi camera on the screen and
door is locked and lock the door manually. the image will be captured in 5 seconds. After a
When the door is unlocked, the house owner series of image processing steps, if the face
could press the ‘Lock’ button to lock the door. If cannot be recognized, a message ‘Could not

385

Copyright (c) 2018 International Journal of Computer Science and Network. All Rights Reserved.
IJCSN - International Journal of Computer Science and Network, Volume 7, Issue 6, December 2018
ISSN (Online): 2277-5420
www.IJCSN.org
Impact Factor: 1.5

recognize, please try again’ will appear on the show up when Pi B receives an image. The
screen for 3 seconds before screen return to image will be displayed on its PiTFT screen plus
‘main’ level. Otherwise, the recognized image 2 buttons ‘Yes’ or ‘No’, by pressing which the
will be displayed until remote confirmation is corresponding response will be sent to Pi A.
made. If Pi A receives a positive response from
Pi B, that is, the one is allowed in by the house 6.1.3 Test
owner, the door will be unlocked by the servo Since there are many levels of touch buttons and
and ‘The door is unlocked’ will show up on the we need to display various messages in different
screen. If Pi A receives a negative response from circumstances, we observed pygame window on
Pi B, you will be told that ‘Entrance not allowed the computer monitor and debugged our user
by the house owner’. At last, after 3 seconds, the interfaces layer by layer. We tested the
system will return to its ‘main’ level. sensitivity and reaction time of each touch
When a user presses the ‘Enter Passcode’ button button and experimented with different sleep
on the screen, the text ‘Please enter the time to make sure buttons all work well. Besides
passcode, end with #’ can be seen on the these, we also did some corner tests to see what
touchscreen and the user needs to enter the would happen.
passcode using the 3x4 matrix keypad. Each
pressed number on the keypad will be displayed 7. Face Detection and Haar Cascade
on the screen and then be covered by a star if the Classifier
next key has been pressed. If the passcode
entered is wrong, then we have the PiTFT
After preprocessing, like resizing and cropping,
display a message, ‘Password does not match,
the image will be used as input of Haar Cascade
please try again’, which will last for 3 seconds
Classifier to detect whether there is a single face
before going back to ‘main’ level.
detected in this image.
On the other hand, if the passcode entered is
Face detection is a process of finding out the
correct, the system will get into the second-level
face area in the image. In the project, we use
buttons: ‘Unlock’, ‘Reset Passcode’ and
Haar Cascade to detect faces. Haar-like features
‘Return’. Users could unlock the door by
such as edge features, line features, and center-
pressing ‘Unlock’ button and the servo will be
surround features are used and they are inputs of
driven to open the door. The password could be
classifiers. Cascade classifiers test the image by
reset by pressing the ‘Reset Passcode’ button.
cascade features. Since the amount of features is
The new passcode needs to be entered through
large, instead of applying all features on the
the keyboard too and will take effect next time.
window, the features are divided into different
The ‘Return’ button is designed to return to the
stages. The window will be tested stage by stage
previous level. For security reason, we set the
and initial stages usually have less haar-like
upper limit for times of entering the wrong
features. If the window fails in a stage, it will be
passcode. If the upper constraint is reached, the
discarded and the following stages won’t be
whole system will be locked for a certain
tested. The window which successfully passes
amount of time, such as 15 minutes and screen
all the stages is considered to be face image.
will tell the user that ‘Maximum failed passcode
Haar cascade classifiers has an advantage of its
attempts in restriction. Please try again after 5
fast detection speed compared to other
minutes.
classifiers.

6.1.2 Rasp Pi B
7.1 Test

We tried to resize captured images to different

sizes and different classifier tuning parameters
like scale factor, a minimum number of
neighbors, to find out the best configuration for
face detection. See details in ‘face.py’ in code
appendix.

8. Face Recognition and Eigenface

Classifier
If a single face is detected by Haar Cascade
classifier, the face will be cropped out of the
Pi B works as the server waiting for the scene. Then the Eigenface classifier that has
connection from Pi A and its user interfaces only already been trained by pre-stored image library,

386

Copyright (c) 2018 International Journal of Computer Science and Network. All Rights Reserved.
IJCSN - International Journal of Computer Science and Network, Volume 7, Issue 6, December 2018
ISSN (Online): 2277-5420
www.IJCSN.org
Impact Factor: 1.5

will try to recognize the cropped face and return special function. We changed the connection to
the confidence of its prediction at the same time. GPIO 21 and the problem was solved.
By setting a threshold for the predicting
confidence, we can determine when to 10. Communication over TCP Sockets
acknowledge that the face is truly recognized.
Face recognition is matching the input signal The two Raspberry is in the project are set to
with the pre-stored library. Though the input communicate using TCP sockets via wifi
signal is noisy due to the different angle, modular and a wifi router. The Raspberry Pi A,
position and intensity of light, the image could which displays main buttons, is the client and the
be recognized according to the position of eyes, other Raspberry Pi B, for remote confirmation, is
face, and mouth in the face, and their relative the server. With specific IP addresses, the client
distances between each other. These features are could connect to the server and send files like
called eigenfaces and they could be extracted images over TCP socket. Then the server will
from original image data by principal component send a response back to the client and once the
analysis. Each face is represented by a subset of response received, the client is going to close the
eigenfaces and the face could be reconstructed if connection. We developed a python script code
eigenface could be correctly calculated for each for the above communication routine by
proportion. The new image will be recognized leveraging python library ‘socket’. See details in
based upon eigenvectors and the Euclidean ‘server.py’ and ‘client.py’ & ‘client2.py’.
distance between eigenvectors.
10.1 Test
8.1 Test
When we started this communication part, there
We tested the eigenface classifier with different were 2 choices when deciding which Pi worked
confidence threshold to get the best recognition as a server or client. We tested on both 2
accuracy. After the system could recognize one methods and found that Pi B needs to wait for
face successfully, we also considered making the images sent by Pi A at any time, so it’d better be
system recognize multiple faces by setting up a server. By doing so, whenever Pi A, the client,
several classifiers and let them do recognition is trying to send an image to PiB, it could be
work in turns or in multi-thread way. As a result, connected to Pi B successfully since the server
due to the limited time, we continued on the keeps on all the time.
former choice which is simpler than multi-
thread. 11. Conclusion
9. Passcode Implementation We successfully finished a face recognition door
lock system as we planned. The passcode and
In the implementation of the keypad, the face recognition works well. There is high
approach for deciding the pressed button is accuracy in recognizing house owner faces and it
firstly setting outputs (column pins) and inputs could realize sending the matched face image to
(row pins) as high. When a button is pressed, it another Raspberry Pi in time and give a good
will produce a low signal. Then each column is output. And it takes a little bit time to recognize.
scanned in a ‘for’ loop. If the low signal is At all, we all are satisfied to build it.
detected, we could know which button is
pressed. See details in ‘passcode.py’ in code Acknowledgement
appendix. First of all, we are thankful to all our team
members for working hard around two months
9.1 Test on this project and gave their great approach on
it. At beginning of this project we were in
In the test of passcode function, the system trouble for starting it with the appropriate way.
worked well on the monitor screen at the We all were confused from where can start, at
beginning. But in the condition that all buttons that fortunately we got an opportunity to discuss
and layers were displayed on the PiTFT screen, this matter to our honorable lecturer Professor
the PiTFT screen would display sliding messy Xu he who was representing about a project
codes when we entered the password ending related to IoT. By getting that chance we told
with ‘#’. The TA helped us debugging and told him at once about our problem. Then he gave the
us the problem was caused by wrong circuit suggestion and guided us for the correct way.
connection. We checked our circuit and found Whenever we faced difficulty to work on it then
the reason is that we attached one of the keypad we never felt hesitate to go through him and
interface pins to GPIO 25 which is used for every time he made our work to overcome all the

387

Copyright (c) 2018 International Journal of Computer Science and Network. All Rights Reserved.
IJCSN - International Journal of Computer Science and Network, Volume 7, Issue 6, December 2018
ISSN (Online): 2277-5420
www.IJCSN.org
Impact Factor: 1.5

difficulty. So we all give him a great credit and [4] W. F. Abaya, J. Basa, M. Sy, A. C. Abad and E.
thanks as well. Finally, we are also delighted to P. Dadios, "Low cost smart security camera
complete this project as our demands eventually with night vision capability using Raspberry Pi
outweigh any negative challenges. and OpenCV," 2014 International Conference
on Humanoid, Nanotechnology, Information
gave the suggestion and guided us for the correct Technology, Communication and Control,
way. Whenever we faced difficulty to work on it Environment and Management (HNICEM),
then we never felt hesitate to go through him and Palawan, 2014, pp. 1-6
every time he made our work to overcome all the [5] Priya Pasumarti1, P. Purna Sekhar “Classroom
difficulty. So we all give him a great credit and Attendance Using Face Detection and
thanks as well. Finally, we are also delighted to Raspberry-Pi” International Research journal of
complete this project as our demands eventually Engineering and Technology (IRJET), Volume:
outweigh any negative challenges. 05 Issue: 03. p3-p5,Mar-2018
[6] S Rajkumar, J Prakash, “Automated attendance
using Raspberry pi”, International Journal Of
References Pharmacy & Technology (IJPT), Vol. 8, No. 3,
pp. 16214-16221, September 2016.
[1] Jie-Ci Yang et. all An Intelligent Automated [7] T. M. Inc., “Train a Cascade Object Detector,”
Door Control System Based on a Smart [Online]. Available:
Camera. http://www.mathworks.se/help/vision/ug/train-
[2] S. Nazeem Basha et all An Intelligent Door a-cascadeobject-d
system using Raspberry pi and Amazon Web
services IOT.
[3] Richard Grimmett, Raspberry Pi Robotic
Projects. Packt Publishing.

388

Copyright (c) 2018 International Journal of Computer Science and Network. All Rights Reserved.