319ITSC – INTERNET OF

THINGS
Face Recognition System

Abstract
This report will include all details about the Face Recognition System created for the Internet
of Things module of the Cybersecurity Degree in Coventry University Scarborough. This facial
Recognition System was created using a Raspberry Pi and its camera module alongside the
Microsoft Azure Cloud Framework.

8535045
 CONTENTS

Table Of Figures ........................................................................................................................................ 2

Introduction .............................................................................................................................................. 3

Problem Domain ....................................................................................................................................... 4

Design ....................................................................................................................................................... 5

Implementation ........................................................................................................................................ 7

Conclusion ................................................................................................................................................ 9

References .............................................................................................................................................. 10

Appendix................................................................................................................................................. 11

8535045 Page 1 12/06/2020

 TABLE OF FIGURES

Figure 1 - Facial Recognition System Diagram .......................................................................................... 5

Figure 2 - Raspberry Pi setup .................................................................................................................. 11
Figure 3 - Raspbian OS ............................................................................................................................ 12
Figure 4 - IoT Hub ................................................................................................................................... 13
Figure 5 - IoT Hub device ........................................................................................................................ 13
Figure 6 - IoT Hub device information .................................................................................................... 14
Figure 7 - Blob Storage ........................................................................................................................... 15
Figure 8 - Captured image of the criminal .............................................................................................. 16
Figure 9 - Email warning message .......................................................................................................... 16
Figure 10 - Face recognition script output ............................................................................................. 17

8535045 Page 2 12/06/2020

 INTRODUCTION

IoT devices englobe a diverse range of technologies that interact with the real world and affect the
end-users directly and indirectly, which also ends up affecting cybersecurity and privacy risks
surrounding the usage of its devices. In this assignment, a face detection system was created using a
Raspberry Pi 3 B+, Raspberry Pi’s camera module and Microsoft Azure Cloud Computing Framework,
demonstrating and implementing fundamental concepts of IoT.

8535045 Page 3 12/06/2020

 PROBLEM DOMAIN

The face detection system presented in this report operates using a Raspberry Pi and Microsoft Azure
to verify personal identity, using biometrics to map facial features from photographs or videos, with
the aid of the Python programming language and Microsoft Azure’s features such as the Face API
cognitive service. Once collected, the system stores the information in Azure’s blob storage and emits
an alert through email to the entities using the system. The system’s main purpose is to quickly
identify unauthorized access to restricted areas.

8535045 Page 4 12/06/2020

 DESIGN

Figure 1 - Facial Recognition System Diagram

8535045 Page 5 12/06/2020

Host (1) is connected to the Router (2) through Wi-Fi and sharing the connection with the Raspberry
Pi (3) using an Ethernet Cable. Using the Host (1), a Microsoft Azure (4) account is setup. Here, it is
proceeded with the creation of an IoT Hub, the setting up of the Face API cognitive service, the Blob
storage system and the automation of the warning message. The Raspberry Pi (3), using the camera
module, identifies a face through a photography, and makes use of Python scripts to interact with the
Face API and store the image in the Blob Storage, sending a warning message to the system user’s
email.

8535045 Page 6 12/06/2020

 IMPLEMENTATION

RASPBERRY PI:

For this Proof of Concept face recognition project, a Raspberry Pi 3 B+ was used with a simple camera
module incorporated. The Raspberry Pi is known as a low cost, single board computer with good
software support and processing power, making it perfect for prototyping using its Linux based
Operating System called Raspbian. Although great for prototyping, Raspberry’s default setup is not
suited for commercial use due to its lack of security features. The lack of hardware security cannot
provide enough confidence regarding the protection of data, and will most likely fail in compliance
with data privacy regulations such as GDPR (Guide to the General Data Protection Regulation (GDPR),
2020) and most consumer IoT product requirements such as the EU and UK Medical Device regulation
(Medical devices: EU regulations for MDR and IVDR, 2020). To circumvent this and enable the product
to be suited for commercial use, a number of procedures were taken into consideration, such as the
implementation of a secure password, secure boot and the use of key generation (Rigas, 2019), and
the use of cryptographic libraries such as OpenSSL (OpenSSL Foundation, 2020).

MICROSOFT AZURE:

To generate sensor data, the Microsoft Azure Cloud Computing Framework was used. Azure allows us
to create an IoT Hub to establish communications between the Raspberry Pi and its cognitive services,
while using its API’s and applications to manage the face detection system. Azure has built-in security
services that can be used to protect data, apps and infrastructures. These built-in services can
manage and enable advanced threat protection, safeguard cryptographic keys and other secrets used
by cloud apps, control and protect sensitive data such as emails and protect device security from
attacks such as Denial of Service (Cloud security | Microsoft Azure, 2020).

IOT HUB:

The IoT hub, as mention above, is a cloud device that acts as a central message hub for
communications between IoT applications and devices. The IoT hub also has management
capabilities, allowing the storage and query of information and management of devices state, also
granting secure authentication through both devices and IoT applications. A more detailed
explanation on how the IoT hub was integrated within the face detection system is demonstrated in
the appendix.

8535045 Page 7 12/06/2020

COGNITIVE SERVICES:

Azure’s Face API was used to perform the facial recognition capability of the system. The Face API
functionalities include face detection, through an image, and identification of the person in that
image, as well as its emotions through facial expressions, hair color and age. Although already
implemented by various organizations, facial recognition systems raise some privacy issues. One of
these issues is the fact that facial data is collected and stored, often without permission, and if the
right security measures are not implemented, hackers can access and steal it. More information on
how the Face API was implemented in the facial recognition system in the appendix.

STORAGE:

Azure Blob storage is Microsoft’s storage solution for the cloud that can store gigantic measures of
unstructured information, such as images, text, binary data, etc. More information about the
implementation of Blob storage with the facial detection system will be included in the appendix.

LOGIC APPS:

Microsoft Azure logic apps allow us to process, schedule and automate tasks integrated with
applications and devices. The facial recognition system uses logic apps to automate the warning
message sent to email once the system recognizes an unauthorized face.

8535045 Page 8 12/06/2020

 CONCLUSION

Combining a Raspberry Pi with Microsoft Azure to create a face detection system was a valuable
experience and revealed the intricacies of creating a real-world application using embedded systems.
Although extremely useful, a face detection system brings with it some ethical issues regarding the
privacy of those exposed to the application, causing a debate on how measurable it is the security of
private and governmental businesses against the loss of personal privacy, where everyone’s facial
details could be stored on a database in a not so distant future.

8535045 Page 9 12/06/2020

 REFERENCES

Ico.org.uk. 2020. Guide To The General Data Protection Regulation (GDPR). [online] Available at:
<https://ico.org.uk/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection-
regulation-gdpr/> [Accessed 10 June 2020].

GOV.UK. 2020. Medical Devices: EU Regulations For MDR And IVDR. [online] Available at:
<https://www.gov.uk/guidance/medical-devices-eu-regulations-for-mdr-and-ivdr> [Accessed 10 June
2020].

OpenSSL Foundation, I., 2020. /Index.Html. [online] Openssl.org. Available at:

<https://www.openssl.org/> [Accessed 10 June 2020].

Rigas, T., 2019. Enabling Verified Boot On Raspberry Pi 3. [online] NVISO Labs. Available at:
<https://blog.nviso.eu/2019/04/01/enabling-verified-boot-on-raspberry-pi-3/> [Accessed 10 June
2020].

Azure.microsoft.com. 2020. Cloud Security | Microsoft Azure. [online] Available at:

<https://azure.microsoft.com/en-gb/product-categories/security/> [Accessed 10 June 2020].

8535045 Page 10 12/06/2020

 APPENDIX

RASPBERRY PI

The face detection system uses a Raspberry Pi 3 B+ with a camera module. To setup the Raspberry Pi
you only need one USB or Micro-SD card with the Raspbian Operating System installed, one Ethernet
cable to connect the Raspberry to the host in order to be able to access the Wi-Fi connection, one
power adapter and the respective camera module. Once all setup you can boot the Raspbian OS while
connected to a monitor, or with SSH on your local host. For this demonstration, it was established an
SSH connection using the Putty software on Windows 10 and the VNC Viewer for graphical display.
Figure 2 displays the Raspberry Pi setup and Figure 3 the Raspbian OS.

Figure 2 - Raspberry Pi setup

8535045 Page 11 12/06/2020

 Figure 3 - Raspbian OS

8535045 Page 12 12/06/2020

IOT HUB AND FACE API

Microsoft Azure has the perfect guide on how to setup the IoT Hub (https://docs.microsoft.com/en-
us/azure/iot-hub/iot-hub-raspberry-pi-web-simulator-get-started). For the face detection system, an
IoT Hub with the name “IotModule” was created (Figure 4), as well as a device with the same name
and a resource group named “IoTCoventry”.

Figure 4 - IoT Hub

By going to the “Explorers” section in the left corner of the IoT Hub page, you can see the device
created (Figure 5). Clicking the device will takes to its settings, where you will be able to copy the
“Primary Connection String” that will be used later on in the scripts that will interact with the IoT Hub
in order to use the face detection system (Figure 6).

Figure 5 - IoT Hub device

8535045 Page 13 12/06/2020

 Figure 6 - IoT Hub device information

Once created the IoT Hub is time to add the Face API following the quick setup guide after adding it to
the system (https://azure.microsoft.com/en-gb/try/cognitive-services/my-apis/?api=face-api).

8535045 Page 14 12/06/2020

BLOB STORAGE AND LOGIC APP

The next step is to configure a Blob storage container, where you can upload images using a Python
script. Once uploaded, the image is stored on the container and the logic app send a message to an
email account.

To setup the Blob Storage, you can follow the Azure official guide (https://docs.microsoft.com/pt-
pt/azure/storage/blobs/storage-quickstart-blobs-portal).

For the face detection system, the blob “facedetection1” was created, connected to the IoT Hub’s
resource group “IoTCoventry”.

Figure 7 - Blob Storage

To finalize, the system needs to send a message to an email account warning about illegal activity in
the premises. To do that, you can use the Azure’s logic apps feature and choose an email to receive
the automatic message.

8535045 Page 15 12/06/2020

LAST STEPS

Once everything is configured, the face detection system can be tested running a script that will
identify an image captured by the Raspberry Pi and stored in the Blob container (Figure 8), and then
send a message to the email account issuing a warning (Figure 9). The script will output a JSON format
string containing all the information about the face image, ranging from the hair colour, age, eye
colour, to the emotions displayed by facial expressions of the person in the image. The script outputs
the likelihood of these characteristics through percentages (Figure 10).

Figure 8 - Captured image of the criminal

Figure 9 - Email warning message

8535045 Page 16 12/06/2020

 Figure 10 - Face recognition script output

8535045 Page 17 12/06/2020