1.

INTRODUCTION
This section of the dissertation will give a general introduction and problem
description. This section gives the information about the project, also the aims and
the motivation to be accomplished. Then, initially it discuss about the
methodologies which is used in the system. Moving on from next section is a
literature review, which combined with the introduction outlined in the coming
chapter and gives a foundation on which to base the technical and functional
specifications.

Computers have undergone rapid miniaturization from being a 'space saver'

to 'as tiny as your palm'. Disks and components grew smaller in size, but one
component still remained the same for decades - it's the keyboard. Miniaturization
of keyboard had proved nightmare for users. Users of PDAs and smart phones are
annoyed by the tiny size of the keys. The new innovation Virtual Keyboard uses
advanced technologies to project a full-sized computing key-board to any surface.
This device has become the solution for mobile computer users who prefer to do
touch-typing than cramping over tiny keys. Typing information into mobile
devices usually feels about as natural as a linebacker riding a Big Wheel. Virtual
Keyboard is a way to eliminate finger cramping. All that's needed to use the
keyboard is a flat surface. Using laser technology, a bright red image of a keyboard
is projected from a device such as a handheld. Detection technology based on
optical recognition allows users to tap the images of the keys so the virtual
keyboard behaves like a real one. It's designed to support any typing speed.

The Virtual Laser Keyboard (VKB) leverages the power of laser and
infrared technology and projects a full-size keyboard onto any flat surface. As you
type on the laser projection, detection technology based on optical recognition
enables the user to tap the images of the keys, complete with realistic tapping
sounds, which feed into the compatible Bluetooth enabled PDA, Smart phone,
laptop or PC. Unlike many small snap-on keyboards for PDAs.

We are witnesses of new technology improvements that are surprising us

every day when we hear about them or even look at them. The computers and
computer’s technology revolutionize the world and the way of life. Since the
beginning in the 50’s, every new achievement in this section was accepted by the

1
vast masses of people which were expecting new improvements with huge hope
for making business and life easier and more successful. In this century, the
computers rule the world. Beginning from the first computer ENIAC, which
proportions were huge, the technology came up with microchips, tiny devices
which are hundred times smaller then the basic of the structure of ENIAC, the
electronic tube. The technology of creating microchips, allows us to create smaller
devices like CPU’s, Integrated Circuits, Microcontrollers, and so on…From other
side, the classic science in the same time was making very important researches
and discoveries, especially physics. The physicist discovered laser rays and sensors
of the same rays, which are getting more sophisticated and more powerful. If we
put these two parts of science in one, we will have more than hundred products
which are making our life easier, more successful and more secured. Products like
this are IR, Bluetooth transmitters and receivers, optical mouse, LCD Projectors,
Large Video Beams and the latest gadget of the technology called Virtual Laser
Keyboard.

1.1 Motivation and Background

A virtual keyboard is basically a keyboard that has no physical support and
operated by the user via typing i.e. by moving fingers on it rather than typing on
depressing physical keys. A VK is simply a projection keyboard projected and
touched on any surface.
A virtual keyboard is one of the examples of today’s fast trend. Computing
is not just limited to laptops and desktops; it found the ways to enter into many
devices for illustration mobile devices like cell phones. The only thing which
remains constant is old QWERTY keyboard and virtual keyboard (VK)
technology is the latest development[1].
In project, simple virtual keyboard has been created using IR Sensors with
the help of simple 2D camera and without using any external hardware.
The first biggest problem in normal QWERTY keyboards is its size, they
are large in size and it is really very uncomfortable for a person to carry it from
one place to the other. The chances of breaking up may also get increased.

Project allows artificial intelligence to those people who want to work on

the simple surface thinking as if it is a keyboard. Virtual keyboard lets people to
have multilingual writing content or text content on existing platform. The idea

2
behind creating this virtual keyboard is simple and understandable. A Virtual
keyboard that have been created is small, well designed, handy and really very
easy to operate also, which results in the perfect solution for cross multilingual
text input.

Some of the Main features in the project: Simple and user friendly design
(interface), size of file really small, test whether every part working properly.
Researchers described the Virtual Keyboard in different forms out of which,
mostly virtual keyboards depends on CCD cameras and 3D optical ranging and
significantly depends on primarily on image processing, some researchers did a
research which was focussed on various forms like rings, gloves, projection and
devices based on hand gestures. A special type of 3D camera or two 2D cameras is
used.

In this project novel technique has been used where it will use finger
shadow to detect the whether the finger is touching the virtual keyboard or not. To
implement devices which are highly specialized is not that practical idea. But the
idea bestowed for finger, hand and fingertip detection are convincing in this
changeable field or activity done under virtual human interface devices (i.e.
VHID).

1.3 Objective
The objectives of this project holding two themes both focus on technical
side, first one deal with creation of the project and other with reconfiguration of
the project.

The primary objective of this dissertation project is to work on the

technologies which are going to be use in project, the designing and development
of the application. Then the secondary aim of this project is to go through the
literature reviews and do the research and find the best way to satisfy the user.

Firstly, this project aims to create a Virtual keyboard using IR Sensors using
simple camera and no additional hardware. Novel technologies have adopted
during the development of the application. Finally, the project aims to reconfigure
the keyboard, through which user can design their own keyboard layout and use it
for different purposes like for gamming.

3
1.3 Time - Plan
Table 1.1: Time plan of entire project
From Date To Date Achievements

15th July 2017 25th July 2017 Selection of topic

26th July 2017 10th August 2017 Literature review

11th August201 20th August Buying Components

20th August 5th September Licensing MATLAB

software

6th September 20th September Code to detect Red colour

and generate co-ordinates

21st September 5th October Code to generate an

alphabet on display

4
2. Review of Literature
Initially, the literature review concentrates on technical literature. So that, it
is easy for the user to understand the design opportunities and the number of
possible pitfalls while implementing the virtual keyboard.
Secondly literature review concentrates on the System Architecture.

The technical and hardware survey that we did is explained in a brief below
In the project, Arduino unoand IR Sensors Model has been used.

ARDUINO UNO BOARD :

Arduino is a single board. This means that everything you need for basic
arduino usage comes soldered to one little circuit board. A single board
microcontroller is different than a single board computer, like the raspberry pi.
Arduino is open source. As open source hardware, the schematics for Arduino are
available to anyone for free, so if you wanted you could buy the electronics
components and a circuit board, and build your own Arduino. Many people build
and sell third party Arduino hardware

Finally, Arduino is cheap. You can get the standard Arduino uno board for
just about $30. The most common Arduino board is the uno, which features an
ATmega328 microcontroller, mounted on a board with 14 digital I/O pins and 6
5
analog input pins, along with a power connector and a USB port that allows you to
program it with your PC. The downside of the Uno is that the ATmega328 is
pretty puny, it’s an 8-bit processor, with only 32K of flash memory and 2K of
system memory.

IR Sensor :
An infrared sensor is an electronic device, that emits in order to sense some
aspects of the surroundings. An IR sensor can measure the heat of an object as
well as detects the motion. These types of sensors measures only infrared
radiation, rather than emitting it that is called as a passive IR sensor.

An IR sensor is a device which detects IR radiation falling on it. There are

numerous types of IR sensors that are built and can be built depending on the
application. Proximity sensors (Used in Touch Screen phones and Edge Avoiding
Robots), contrast sensors (Used in Line Following Robots) and obstruction
counters/sensors (Used for counting goods and in Burglar Alarms) are some
examples, which use IR sensors.
SOFTWARE’S USED:
Image Processing with MATLAB Images are everywhere, from everyday
devices like cameras and smart phones to specialized devices for medical imaging,
automotive safety, industrial automation, and more. Each of these uses for image
processing has unique challenges. MATLAB and Image Processing Toolbox
provide a flexible environment to explore 684 | P a g e design ideas and create
unique solutions for imaging systems. Matlab toolbox used in our project are as
follows
 Image acquisition Toolbox.
 Image processing toolbox
 GUI build

6
2.1 REVIEW OF RESEARCH PAPERS
This technical literature review will first look into the architecture of a
Virtual Keyboard, through the information provided on the surroundings. Next,
will discuss about how virtual keyboard reconfigured using the architecture. After
studying all this will move to the languages, methodologies that have been used.
At last, it discussed about the implementation of a application with some screen
shots.

2.2 COMPARISON
Table 2.1 Literature survey for techniques used for key detection

Author Techniques used University/Publication

Eckert, M. Lopez, M. et al. Middleware for mapping Tech. Univ. of Madrid,

gestures Madrid, Spain

Motion sensing camera

device

Su, Xiaolin, Zhang et al. Every keystroke can be Cyber Technology in

determine accurately by Automation, Control, and
image processing including Intelligent Systems (CYBER),
morphology principle and 2015 IEEE International
ellipse fitting Conference on

Jun Hu, Guolin Li, Xiang Xie, Bare-finger touch IEEE Transactions on
Zhong Lv, and Zhihua Wang interaction on regular Circuits and Systems for
planar Video Technology Year:
2014

7
2.3 PROBLEM STATEMENT
To design a vision based Virtual Keyboard which detects interrupt as key
recognition instead of mechanical transducer operations of key pressing.

Co-ordinates of hand posture for pressing the keys is analyzed. The analyzed
coordinates of the hand posture is taken into account under various transactions to
estimate the key pressed. Mechanical transducers does two operations for key
estimation (key press and key release concepts), while Virtual Keyboard requires
only key press operation to estimate the key and not key release operation.

8
3. DESIGN AND METHODOLOGY
This part of the report deals with the initial Proposed design through which
it is easy to understand the designs and user flow for the application

3.1 BLOCK DIAGRAM

Fig 3.1: Block Diagram of the design

9
3.2 DESCRIPTION OF COMPONENTS
1. The keyboard will be drawn on blank paper.

2. A camera will be there to capture live feed of fingers typing on blank paper with
keyboard drawn on it.

3. Hence, with the Image processing, in real time typed words on keyboard will
be detected.

4. Those words will be screened on desktop.

3.3 FLOW CHART

User giving Inputs

Keyboard drawn on
paper

Camera unit sensing

the co-ordinates

Output alphabet
touched is displayed

Fig 3.2: Flow chart of working

10
4. IMPLEMENTATION AND RESULTS

 In MATLAB, we have created a video object acquired by the webcam. This

video is used for frame extraction.

 Our MATLAB code constantly extracts 2 frames from video and with the
help of Frame Differencing algorithm it monitors movement of the finger. If
any movement is detected then next frame to that is extracted.

 The extracted frame is firstly threshold to get pixels of references and tag.
Various data redundancy algorithms like addition of coordinates and
elimination are run to extract 1 pixel per reference.

 With the help of references, gridding is done. As of now we have

successfully implemented keyboard with any combination up to 4X4. Our
code has flexibility to allow a keyboard of any size with a mere change of 2
numbers.

 After gridding, positioning of the tag is done. For positioning we use left
upper and right lower points in the grid to detect corresponding key.

 Next module is GUI. GUI is used to display the key which is detected. It is
created using the GUIDE toolbox of MATLAB.

11
TESTING
System testing is the stage of implementation. It aims at testing and ensuring
that the system works accurately and efficiently before live operation commences.
The logical design and physical design should be thoroughly and continually
examined on paper to ensure that they will work when implemented. Thus the
system in implementation should be a confirmation that all system works. The
testing phase includes entering the sample data to verify whether the system is
suitably working to the requirements mentioned. This phase is important in the
way that it actually deals with the real data.

The rules that serve for testing are,

 Testing is a process of executing a program of executing a program with the

intent of finding an error.
 A good testing is the one that has the high probability of finding
undiscovered errors.
 A successful test is the one that uncovers a discovered error.
 If testing is conducted successfully according to the above objectives, it will
uncover the errors in the software.
 The various types of testing are:
 Unit testing
 Integration testing
 Validation testing

RESULTS
We have made and fully working interface of keyboard which can detect key
stroke for letters from a flat surface.

12
ADVANTAGES
Virtual input devices are designed to offer the functionality of their
hardware counterparts without the actual use of hardware. The concept applies to
any device that requires tactile interaction for the input of data or other
information, such as a computer keyboard or piano. Virtual keyboards may be
difficult to become accustomed to for anyone used to conventional hardware, but
they do offer advantages unique to their design.

 It can be projected on any surface or you can type in the plain air.

 It can be useful in places like operation theaters where low noise is

essential.

 The typing does not require a lot of force. So easing the strain on
wrists and digits.

 The Virtual Keyboard is not restricted to the QWERTY touch-typing;

adjustments
 can be done to the software to fit other touch-typing paradigms as
well.

 No driver software necessary, It can be used as a plug and play

device.

 High battery life.

MAINTENANCE
Unlike conventional hardware keyboards, virtual keyboards have no
moving parts, and the bulk of their functionality requires no actual contact with the
device. Whereas conventional hardware is prone to damage by spills, drops and
other malfunctions, virtual laser keyboards are easy to maintain. Manufacturers
often market these devices as "damage free," and this is an important advantage in
any piece of portable hardware that undergoes extensive use.

13
SPECIFICATIONS
The primary aim of this concept was to produce a low-cost alternative to the
commonly used keyboard. To maintain the low cost and due to unavailability of
some components, the components and their specifications had to be compromised.

1. Camera: suitable camera available and is being used for development.

We used C170 webcam by Logitech.

2. Processor: we are using processor of the computer. We have used

HPpavilion 8GB RAM, 1TB graphics laptop for testing and implementation.

3. Software: MATLAB R2015a has been installed on the computer on

which we tested the implementation.

RESULTS
We have made and fully working interface of keyboard which can detect key
stroke for letters from a flat surface.

14
5.FUTURE SCOPE AND CONCLUSION

FUTURE SCOPE
The virtual keyboard that has been implemented is like a small application
coming from the larger idea where it detects the finger edge with the help of
simple 2D camera. User doesn’t need to require any external hardware or
additional space for this. The main idea would be developing the virtual keyboard
application for the mobile devices which helps in playing the game on mobile
phones, 3D modelling and lot more. Countries like India where there are different
languages user can easily change the language of the keyboard by internal
mapping of characters and use it anytime anywhere without the requirement of
any change in hardware. This results in creating multilingual keyboards feasible
economically. Last but not the least such keyboards will play a great role for blind
people without the need of extra expenditure.

CONCLUSION
We developed a system to get an input of keyboard drawn on a blank paper
using a real - time camera and IR sensors providing the interrupts. However, it is
difficult to get stable results because of the variety of lighting and skin colors of
human races. Most vision algorithms have illumination issues. From the results, we
can expect that if the vision algorithms can work in all environments then our
system will work more efficiently. This system could be useful in presentations
and to reduce work space.

15
REFERENCES
[1]Eckert, M. Lopez, M. ; Lazaro, C. ; Meneses, J. ; Martinez Ortega, J.F., 2015
Mokey - A motion based keyboard interpreter .Tech. Univ. of Madrid, Madrid,
Spain

[2]Su, Xiaolin, Zhang, Yunzhou ; Zhao, Qingyang ; Gao, Liang, 2015 Virtual
keyboard: A human-computer interaction device based on laser and image
processing, Virtual keyboard: A human-computer interaction device based on laser
and image processing, College of Information Science and Engineering, North-
eastern University, Shenyang, China

[3]Jun Hu, Guolin Li, Xiang Xie, Zhong Lv, and Zhihua Wang, Senior Member,
IEEE:Bare-fingers Touch Detection by the Button’s Distortion in a Projector–
Camera System

16