ABSTRACT

A virtual keyboard is where a full-size image of a QWERTY keyboard is projected onto any surface. Touching
the image of a key generates a unique electronic signal corresponding to a key’s image. Using a
virtual keyboard eliminates the chance of breakage and infection transfer. Additionally virtual keyboards
require no cleaning and they have no wires, buttons, or switches. Virtual keyboards are also compatible with
many Smartphones and PDAs. A virtual keyboard is also called a projection keyboard. VKEY is the
trademarked name of the virtual keyboard developed by Virtual Devices Inc.
* Virtual keyboard image courtesy of Virtual Devices Inc.
INTRODUCTION

A virtual keyboard is a computer keyboard that a user operates by typing on or

within a wireless- or optical-detectable surface or area rather than by depressing
physical keys. Such a system can enable the user of a small handheld device, such
as a cellular telephone or a PDA (personal digital assistant) to have full keyboard
capability.

In one technology, the keyboard is projected optically on a flat surface and, as the
user touches the image of a key, the optical device detects the stroke and sends it to
the computer. In another technology, the keyboard is projected on an area and
selected keys are transmitted as wireless signals using the short-
range Bluetooth technology. Theoretically, with either approach, the keyboard
could even be projected in space and the user could type by moving fingers
through the air.

The term virtual keyboard is sometimes used to mean a soft keyboard , which

appears on a display screen as an image map . In some cases, a software-based
keyboard can be customized. Depending on the host system and specifc software,
the user (who may be someone unable to use a regular keyboard) can use a touch
screen or a mouse to select the keys.

Virtual keyboard

A virtual keyboard is a software component that allows the input of

characters without the need for physical keys.[1] The interaction with the
virtual keyboard happens mostly via a touchscreen interface, but can also take
place in a different form in virtual or augmented reality.

Types[edit]

On a desktop computer, a virtual keyboard might provide an alternative input

mechanism for users with disabilities who cannot use a conventional keyboard, or
for bi- or multi-lingual users who switch frequently between different character
sets or alphabets, which may be confusing over time. Although hardware
keyboards are available with dual keyboard layouts (e.g. Cyrillic/Latin letters in
various national layouts), the on-screen keyboard provides a handy substitute while
working at different stations or on laptops, which seldom come with dual layouts.

Virtual keyboards can be categorized by the following aspects:

 Virtual keyboards with touchscreen keyboard layouts or sensing areas[2]

 Optically projected keyboard layouts or similar arrangements of "keys" or
sensing areas[3][4]
 Optically detected human hand and finger motions[5]
 Online virtual keyboards for multiple languages that don't require OS
settings change[6]
 Depending on which device the keyboard is used (desktop / mobile / virtual
reality / augmented reality)

On the Internet, various JavaScript virtual keyboards have been created, allowing

users to type their own languages on foreign keyboards, particularly in Internet
cafes. Multitouch screens allow the possibility to create virtual chorded
keyboards for tablet computers,[7] touchscreens, touchpads and wired gloves.[8][9]

Examples of virtual keyboards

  Gboard - A virtual keyboard for iOS and Android created by Google,
featuring translation capabilities, GIFs, and customizable themes.
 Swype - A popular virtual keyboard that allows you to type by swiping a
finger across multiple letters, spelling out a word in a single motion.
 OSK (on-screen keyboard) - A virtual keyboard available in Microsoft
Windows when a mobile device or 2-in-1 laptop, such as the Surface Pro, is
operated in tablet mode. The OSK is also available when Ease of
Access accessibility features are enabled.

Mobile devices[edit]

Virtual keyboards are commonly used as an on-screen input method in devices

with no physical keyboard, where there is no room for one, such as a pocket
computer, personal digital assistant (PDA), tablet computer or touchscreen-
equipped mobile phone. Text is commonly inputted either by tapping a virtual
keyboard or finger-tracing.[10] Virtual keyboards are also used as features
of emulation software for systems that have fewer buttons than a computer
keyboard would have.

Historical development[edit]
PDA[edit]

The four main approaches to enter text into a PDA were: virtual keyboards
operated by a stylus, external USB keyboards, handwritten keyboards, and stroke
recognition. Many early PDAs were not primarily focused on virtual keyboards.
Microsoft's mobile operating system approach was to simulate a complete
functional keyboard, which resulted in a slightly overloaded keyboard layout.
[11]
 The main problem that early PDAs faced was support for multi-touch
technology, and as a result, usability problems for the user.

First iPhone[edit]

When Apple presented the first iPhone in 2007, the decision not to include a

physical keyboard was seen as a detriment to the device.[12] But Apple brought
the multi-touch technology into their new device, which enabled them to overcome
the usability problems of PDAs. Apple's virtual keyboard design pattern has
become a standard on mobile devices today.

Implementation and use[edit]

Both most common mobile operating systems, Android and iOS, give the

developer community the possibility to individually develop custom virtual
keyboards.

Android[edit]

The Android SDK provides a so-called InputMethodService.[13] This service

provides a standard implementation of an input method, which final
implementations can derive from and customize, enabling the Android
development community to implement their own keyboard layouts. The
InputMethodService ships with it on Keyboard View.[14] While the InputMethod
Service can be used to customize key and gesture inputs, the Keyboard Class loads
an XML description of a keyboard and stores the attributes of the keys.[15]

As a result, it is possible to install different keyboard versions on

an Android device, and that the keyboard is only an application; a simple
activation over the Android settings menu is possible.[16]

iOS[edit]
Apple also provides the possibility for the community to develop custom
keyboards, but does not give any access to the dictionary or general keyboard
settings. Further iOS is automatically switching between system and custom
keyboards, if the user enters text into the text input field.[17][18]

The UIInputViewController is the primary view controller for a custom keyboard

app extension. This controller provides different methods for the implementation
of a custom keyboard, such as a user interface for a custom keyboard, obtaining a
supplementary lexicon or changing the primary language of a custom keyboard.[19]

Text entry performance[edit]

Next to the classic virtual keyboard implementation Android, iOS and custom

keyboards, such as SwiftKey for example, are providing different features to
improve the usability and the efficiency of their keyboards.

Autocorrection and spelling checker[edit]

The Android platform offers a spelling checker framework that offers the
possibility to implement and access spell checking in the application itself. The
framework is one of the Text Service APIs offered by the Android platform. Based
on provided text, the session object returns spelling suggestions generated by the
spelling checker.[20][21]

iOS is using the class UITextChecker, an object used to check a string (usually the
text of a document) for misspelled words, commonly known as Apple's
autocorrection. UITextChecker spell-checks are using a lexicon for a given
language. It can be told to ignore specific words when spell-checking a particular
document and it can learn new words, which adds those words to the lexicon.[22]

Word suggestions[edit]
Diverse scientific papers at the beginning of the 2000s showed even before the
invention of smart phones, that predicting words, based on what the user is typing,
is very helpful to increase the typing speed.[23][24] At the beginning of development
of this keyboard feature, prediction was mainly based on static
dictionaries. Google implemented the predicting method in 2013 in Android 4.4.
This development was mainly driven by third party keyboard providers, such
as SwiftKey and Swype.[25] Both provide powerful word search engine with
corresponding databases.[26] In 2014 Apple presented iOS 8[27] which includes a
new predictive typing feature called QuickType, which displays word predictions
above the keyboard as the user types.

Gesture typing[edit]

iOS and Android allow developers to replace its keyboard with their own keyboard

apps. This has led to experimentation and new features, like the gesture-typing
feature that's made its way into Android's official keyboard after proving itself in
third-party keyboards. Research by Google itself confirmed that gesture-typing is
increasing the typing rate by 22% and is decreasing the error rate near to 0%.
[28]
 Google further showed that the gesture-typing method is also useful on smart
watches. Their scientific research is primarily based on research made by I. Scott
MacKenzie and papers about modeling finger touch with fitts’ law.

Haptic feedback[edit]

Haptic feedback provides for tactile confirmation that a key has been successfully
triggered i.e. the user hears and feels a "click" as a key is pressed.
Utilising hysteresis, the feel of a physical key can be emulated to an even greater
degree. In this case, there is an initial "click" that is heard and felt as the virtual key
is pressed down, but then as finger pressure is reduced once the key is triggered,
there is a further "unclick" sound and sensation as if a physical key is respringing
back to its original unclicked state. This behaviour is explained in Aleks Oniszczak
& Scott Mackenzie's 2004 paper "A Comparison of Two Input Methods for
Keypads on Mobile Devices" which first introduced haptic feedback with
hysteresis on a virtual keyboard.[29]

Special keyboard types[edit]

Keyboards are needed in different digital areas. Not only smartphones need a

virtual keyboards, also devices which create virtual worlds, for example virtual
reality or augmented reality glasses, need to provide text input possibilities.

Optical virtual keyboard[edit]

An optical virtual keyboard was invented and patented by IBM engineers in 1992.

[30]
 It optically detects and analyses human hand and finger motions and interprets
them as operations on a physically non-existent input device like a surface having
painted keys. In that way it allows to emulate unlimited types of manually operated
input devices such as a mouse or keyboard. All mechanical input units can be
replaced by such virtual devices, optimized for the current application and for the
user's physiology maintaining speed, simplicity and unambiguity of manual data
input.

Augmented reality keyboards[edit]

The basic idea of a virtual keyboard in an augmented reality environment is to give

the user a text input possibility. A common approach is to render a flat keyboard
into the augmented reality, e.g. using the Unity TouchScreenKeyboard.
The Microsoft HoloLens enables the user to point at letters on the keyboard by
moving his head.[31]

Another approach was researched by the Korean KJIST U-VR Lab in 2003. Their
suggestion was to use wearables to track the finger motion to replace a physical
keyboards with virtual ones. They also tried to give an audiovisual feedback to the
user, when a key got hit. The basic idea was to give the user a more natural way to
enter text, based on what he is used to.[32]

Virtual reality keyboards[edit]

The challenges, as in augmented reality, is to give the user the possibility to enter
text in a completely virtual environment. One big issue is that most augmented
reality systems on the market are not tracking the hands of the user. So many
available system provide the possibility to point at letters.[33]

In September 2016, Google released a virtual keyboard app for

their Daydream[34] virtual reality headset. To enter text, the user can point at
specific letters with the Daydream controller.[35]

In February 2017, Logitech presented experimental approach to bring their

keyboards into the virtual environment. With the Vive Tracker and the Logitech G
gaming keyboard it is possible to exactly track every finger movement, without
wearing any type of glove. 50 of such packages were send to exclusive developers,
enabling them, in combination of Logitche's BRIDGE developers kit, to test and
experiment with the new technology.[36][37]

Security considerations[edit]

Virtual keyboards may be used in some cases to reduce the risk of keystroke
logging.[38] For example, Westpac’s online banking service uses a virtual keyboard
for the password entry, as does TreasuryDirect (see picture). It is more difficult
for malware to monitor the display and mouse to obtain the data entered via the
virtual keyboard, than it is to monitor real keystrokes. However it is possible, for
example by recording screenshots at regular intervals or upon each mouse click.[39]
[40]
The use of an on-screen keyboard on which the user "types" with mouse clicks can
increase the risk of password disclosure by shoulder surfing, because:

 An observer can typically watch the screen more easily (and less
suspiciously) than the keyboard, and see which characters the mouse moves to.
 Some implementations of the on-screen keyboard may give visual feedback
of the "key" clicked, e.g. by changing its colour briefly. This makes it much
easier for an observer to read the data from the screen. In the worst case, the
implementation may leave the focus on the most recently clicked "key" until
the next virtual key is clicked, thus allowing the observer time to read each
character even after the mouse starts moving to the next character.
 A user may not be able to "point and click" as fast as they could type on a
keyboard, thus making it easier for the observer.
 REFERENCES

1.  "virtual keyboard". PCMag.

2. ^ US application 4725694 Auer Carol M, Castagno Daniel L/AT&T: "Computer interface device"
priority date 13.05.1986
3. ^ DE application 19734511 B. Kämmerer, C, Maggioni, H. Röttger/SIEMENS AG:
"Kommunikationseinrichtung" filing date 08.08.1997
4. ^ WO 0003348 C. Maggioni, B. Kämmerer/SIEMENS AG: "Projection Device / Vorrichtung zur
Projektion" priority date 10.07.1998
5. ^ EP 0554492 Hans E. Korth: "Method and device for optical input of commands or data" filing date
07.02.1992
6. ^ "Virtual keyboard".
7. ^ Chien-Hsu Chen. "Design and Evaluation of Chord Keyboard for Tablet Computer Manipulation".
National Cheng Kung University.
8. ^ "ASETNIOP concept designed to make touch typing on a touchscreen a mite easier". Gizmag.com.
9. ^ "ASETNIOP invisible keyboard aims to get you touch-typing on the iPad". todaysiphone.com.
10. ^ "Swype finger-tracing text entry seems fast, hypnotic, and magical".  Engadget. Retrieved 2016-03-
24.
11. ^ Georgiev, Tsvetozar; Georgieva, Evgenia (2008-06-12).  Mobile application for determination of
users' text entry speed.
12. ^ Titcomb, James (2017).  "How the world reacted to the first iPhone 10 years ago". The
Telegraph.  ISSN  0307-1235. Retrieved 2018-03-16.
13. ^ "InputMethodService | Android Developers".  developer.android.com. Retrieved 2018-03-19.
14. ^ "Create a Custom Keyboard on Android".  Code Envato Tuts+. Retrieved  2018-03-19.
15. ^ "Keyboard | Android Developers". developer.android.com. Retrieved  2018-03-19.
16. ^ "keyboard - Google Play". Retrieved  2018-03-19.
17. ^ "Managing the Keyboard".  developer.apple.com. Retrieved  2018-03-19.
18. ^ "UIInputViewController - UIKit | Apple Developer Documentation". developer.apple.com.
Retrieved 2018-03-19.
19. ^ "App Extension Programming Guide: Custom Keyboard". developer.apple.com. Retrieved 2018-03-
19.
20. ^ "UITextChecker - UIKit | Apple Developer Documentation".  developer.apple.com. Retrieved  2018-
03-19.