Blackbook
Blackbook
Blackbook
ON
AI Virtual Keyboard
Submitted By
CERTIFICATE
THIS IS TO CERTIFY THAT
AI Virtual Keyboard
IN PARTIAL FULFILLMENT OF
ACADEMIC REQUIREMENT FOR THE YEAR
2022-2023
AS PRESCRIBED BY
MAHARASHTRA STATE BOARD OF TECHNICAL EDUCATION,
MUMBAI
DATE:
PLACE: PUNE
CERTIFICATE
This is to certify that Mr. Omkar Mahamuni from Sou. Venutai Chavan
Project Guide
Mrs. Tejal R Vallakathi
Annexure A
CERTIFICATE
This is to certify that Mr. Darshan Patil from Sou. Venutai Chavan
Project Guide
Mrs. Tejal R Vallakathi
Annexure A
CERTIFICATE
This is to certify that Mr. Niraj Salunkhe from Sou. Venutai Chavan
Polytechnic, Pune having Enrollment No. 2000400 has completed the Project
Project Guide
Mrs. Tejal R Vallakathi
Annexure A
CERTIFICATE
This is to certify that Mr. Aditya Nivangune from Sou. Venutai Chavan
First and the foremost, I wish to record my gratitude and thanks to Prof. Tejal R
Vallakathi for his enthusiastic guidance and help in successful completion of project. I
express my thanks to Prof. M. S. Jadhav (Principal, Sou. Venutai Chavan Polytechnic),
Prof. A. V. Kurkute (Head of Department, Computer Technology) and Prof. A. Patil
(Project Coordinator) for their valuable guidance. I am also thankful to other teachers and
non- teaching staff of Computer Technology Department and Library for their co-operation
and help. Lastly, I need to extend to extend my thanks to all those, who helped us directly or
indirectly in completing this esteemed project
Omkar Mahamuni
Aditya Nivangune
Darshan Patil
Niraj Salunkhe
Abstract
The AI Virtual Keyboard is a software application designed to provide a virtual keyboard interface
for users to input text on their devices without the need for a physical keyboard. The keyboard is
displayed on the screen, and users can input text by tapping on the keys with their fingers.
The AI in the Virtual Keyboard refers to the use of artificial intelligence and machine learning
algorithms to improve the accuracy of the keyboard. The keyboard can learn the typing patterns of
the user and suggest corrections for misspelled words. The AI also helps in predicting the next word
based on the context and the previous words typed.The AI Virtual Keyboard can support multiple
languages, layouts, and devices, and can be customized to meet the user's requirements. The
keyboard can have features such as auto-capitalization, auto-correction, typing, and gestures.
The AI Virtual Keyboard provides a convenient and efficient way for users to input text on their
devices, making it an essential tool for modern communication and productivity. The application is
designed to be user-friendly and intuitive, with a simple and elegant interface that enables users to
type quickly and accurately. Overall, the AI Virtual Keyboard is a powerful and innovative tool that
is set to revolutionize the way people interact with their devices.
DECLARATION
CERTIFICATE
ACKNOWLEDGEMENT
ABSTRACT
1.1 Introduction 1
1.2 Background 2
2 LITERATURE SURVEY
3.2 Methodology 6
7
3.3 Hardware and Software requirements
6.1 Conclusion 17
7.2 Books 18
7.3 Websites 18
CHAPTER 01
INTRODUCTION AND
BACKGROUND
AI Virtual Keyboard
1.1 Introduction
A virtual keyboard is a software component that allows the input of characters without
the need for physical keys.[1] The interaction with the virtual keyboard happens mostly
via a touchscreen interface, but can also take place in a different form in virtual or
augmented reality.Computer vision is a field of artificial intelligence (AI) that enables
computers and systems to derive meaningful information from digital images, videos and
other visual inputs — and take actions or make recommendations based on that
information. If AI enables computers to think, computer vision enables them to see,
observe and understand.
Computer vision works much the same as human vision, except humans have a head
start. Human sight has the advantage of lifetimes of context to train how to tell objects
apart, how far away they are, whether they are moving and whether there is something
wrong in an image.
Though computer vision is not new but it’s the strength is literally grows exponentially.
One may think, what we can achieve from this technology. Well, sky is the limit you can
assume.The idea of this project came across my mind when I thought to do something
unique with computer vision.
In this project, I have used 1280x720 pixel video footage using imutils. This is important
as some of the laptop’s webcam only provides fixed height and width which can be very
small and not suitable for this project.
Virtual keyboard is the new development in this field. The virtual keyboard innovation
makes utilization of camera and picture handling (image processing) procedures
empowering client to use it away at any level surface utilizing paper console. Virtual
keyboard gives us a chance to make a console in any of the favored dialect on pretty
much every current stage. The properties of virtual console being little and simple to
utilize application make it a decent answer for content contribution crosswise over
various stages.
1.1 Background
Here we can find the distance between the top point of our index finger and middle finger, if the
distance between the two is less than a certain threshold, then we can type the letter on which we are
indicating. Once we get the position then we loop through the entire position list. From that list, we
find button position and button size and then we plot it on the frame according to a well-defined
manner.
After that, we need to find the distance between the top point of our index finger and middle finger.
In the above image, you can see the top points which we require are point 8 and point 12. Hence we
need to pass 8, 12 inside a distance finding function in order to get the distance between them. In the
above code you can see detector.findDistance() and there we passed 8, 12, and image in order to find
the distance and we set the draw flag to false so that we do not need any line between the two points.
If the distance between the points is very less we will use press() function to press the keys. In the
above code keyboard.press() and we are passing button.text in order to display that pressed key. And
finally, we draw a small white rectangular box just below our keyboard layout in order to display the
pressed key.
Hand motion acknowledgment is critical for human-PC connection. Right now, present a novel
constant strategy for hand motion recognition. The proposed framework is vision based, which
uses AI methods and contributions from a PC webcam. Vision based signal acknowledgment
following and motion acknowledgment In our structure, the hand area is separated from the
foundation with the foundation subtraction technique. At that point, fingers are portioned in order
to identify and perceive the fingers. At long last, a standard classifier is applied to anticipate the
names of hand motions. The examinations on the informational index of 1300 pictures show that
our strategy performs well and is exceptionally productive. Besides, our technique shows
preferred execution over a condition of-workmanship strategy on another informational
collection of hand motions.
An AI virtual keyboard project has a broad scope that can include various features and
functionalities. The main objective of the project is to provide users with a more natural and efficient
way to input text on digital devices, such as smartphones, tablets, and computers. Here are some of
the possible features and functionalities that could be included in an AI virtual keyboard project:
Prediction and Correction: The AI virtual keyboard can use natural language processing algorithms
to predict the user's next word or phrase, which can save time and reduce errors. The keyboard can
also automatically correct spelling and grammar mistakes.
Gesture Typing: The AI virtual keyboard can support gesture typing, which enables users to input
text by swiping their fingers across the keyboard.
Multilingual Support: The AI virtual keyboard can support multiple languages, allowing users to
switch between languages easily.
Personalization: The AI virtual keyboard can learn the user's typing habits and adjust its prediction
and correction algorithms accordingly.
Accessibility: The AI virtual keyboard can include accessibility features, such as support for voice
input and haptic feedback.
Integration with Other Apps: The AI virtual keyboard can integrate with other apps, such as
messaging and social media apps, allowing users to input text directly from the keyboard.
Security: The AI virtual keyboard can include security features, such as encrypted data transmission
and authentication mechanisms, to protect the user's data and privacy.
Customization: The AI virtual keyboard can allow users to customize the appearance and layout of
the keyboard to their liking.
3.2 Methodology
We have used the libraries and various functions of Python from which our Palm tracking Module
and Typing on air with hand gestures was carried out easily and conveniently without any latency or
lagging. First, we installed the essential libraries through Python Interpreter which were required for
coding of the ON AIR VIRTUAL KEYBOARD. Next, we started building the code and with the
technique of trial and error method we were able to build a dummy working module of the given
project. Then, after lots of improvements and constantly minimizing the errors, we were finally able
to construct the actual code of our project. Finally, for the better look of our project we used various
colors insertion and attractive and eye-catching methods for the elegant look of our keyboard.
Palm Tracking Module (PTM) This method was first to be tested. So, we created a module which
would sense the palm, detect it and track the outlines on the fingers with some filled
A circles as continuity between the bending of the fingers. Also, we created a real time box around
the palm which would adjust its dimensions as per the closing and opening of the palm in real-time.
And we added a real time working FPS (Frames per second) Display.
Virtual Keyboard Working After successfully testing the Palm Tracking Module, we started to build
the Virtual Keyboard for Computer Machine by the help of Gestures by using Python Language.
Using the PTM as dummy code, we built the Virtual Keyboard using the same Python Language and
several packages like OPENCV ZONE, MEDIAPIPE, HAND TRACKING MODULE, PYNPUT,
NUMPY AND TIME and we introduced new parameters in it such as KEYBOARD KEYS, BOX
TO DISPLAY TEXTS AND REAL TIME FPS DISPLAY.
The architecture of an AI virtual keyboard project may vary depending on its specific requirements and
functionalities. However, here is a general architecture that could be followed:
1. User Interface: The user interface would consist of the virtual keyboard display and any other
necessary components such as a text box for displaying typed text.
2. Input Processing: The input processing module would be responsible for interpreting the user's
input, whether it be through clicks on the virtual keyboard or voice commands. This module could
use machine learning algorithms to recognize patterns in the user's input and predict the next
characters or words.
3. Natural Language Processing: The natural language processing module would analyze the text
input and try to identify the context and intent behind it. It would also use algorithms to correct
spelling mistakes and suggest possible word completions based on context.
4. Language Modeling: The language modeling module would be responsible for generating
predictions of the user's intended words based on the previous words typed or spoken. This module
could use techniques such as n-gram models, neural language models, or other machine learning
approaches.
5. Output Generation: The output generation module would take the predicted words and convert
them into the desired output format, such as text or speech.
6. Machine Learning: The machine learning module would continuously learn from the user's input
and improve its predictive capabilities over time.
7. Backend and APIs: The backend and APIs would handle the communication between the various
modules and any external services or databases required by the project.
4.4 Flowcharts
Software Testing is a critical element of software quality assurance and represents the ultimate review
of specification, design and coding. Testing presents an interactive anomaly for the software engineers. The
increasing visibility of software quality assurance and attendant costs associated with software failure
always were the motivating force for well-planned through testing. Well planned and through testing is
required to deliver a good system.
Detects errors
Unit testing is a level of software testing where individual units/ components of a software are tested. The
purpose is to validate that each unit of the software perform as designed. A unit is the smallest testable part
of any software. It usually has one or a few inputs and usually a single output. In procedural programming, a
unit may be an individual program, function, procedure, etc. In object-oriented programming, the smallest
unit is a method, which may belong to a base/ super class, abstract class or derived/ child class. (Some treat
a module of an application as a unit. This is to be discouraged as there will probably be many individual
units within that module). Unit testing frameworks, drivers, stubs and mock /fake objects are used to assist
in unit testing. Unit testing increases confidence in changing /maintaining code. If good unit tests are written
and if they are run every time any code is changed, we will be able to promptly catch any defects introduced
due to change. Also, if codes are already made less interdependent to make unit testing possible, the
unintended impact of changes to any code is less. Codes are more reusable. In order to make unit testing
possible, codes need to be modulator. This means that codes are easier to reuse. Development is faster.
How? If you do not have unit testing in place, you write your code and perform that fuzzy ‘developer test’
(You set some breakpoints, fire up the GUI, provide a few inputs that hopefully hit your code and hope that
you are all set). But, if you have unit testing in place, you write the test, write the code and run the test.
Writing tests takes time but the time is compensated by the less amount of time it takes to run the tests; You
need not fire up the GUI and provide all those inputs. And, of course, unit tests are more reliable than
‘developer tests’. Development is faster in the long run too. How? The effort required to find and fix defects
found during unit testing is very less in comparison to the effort required to fix defects found during system
testing or acceptance testing.
Big Bang is an approach to integration testing where all or most of the units are combined together and
tested at one go. This approach is taken when the testing team receives the entire software in a handle. So
what is the difference between Big Bang Integration Testing and System Testing? Well, the former tests
only the interactions between the units while the latter tests the entire system. Top Down is an approach to
Integration Testing where top-level units are tested first and lower-level units are tested step by step after
that. This approach is when up down development approach is followed. Test Stubs are needed to stimulate
lower-level units which may not be available during the initial phases. Bottom Up is an approach to
Integration Testing where bottom level units are tested first and upper-level units’ step by step after
that. This approach is taken when bottom-up development approach is followed.
07 Order Of The Key Check The Order Order Of The Key Properly Pass
Of The Key Is Proper Arranged Key
08 User Interface Check The User User Interface Is User Interface Is Pass
Interface Of The Performing Performing
Key Properly For Each Properly For
Key Each Key
09 Color Of The Verify The Color Keyboard Must Be Keyboard Is Pass
Keyboard Of The Virtual Transparent Transparent
Keyboard
10 Performance Of Check The Performance Of Performance Of Pass
Each Key Performance Of Each Key Each Key
Each Key
11 Space Between Check The Space Maintain Space Space Between Pass
Each Key Between Each Key Between Each Key Key Is Taken
12 Data Capture Check The Data Data Capture Data Capture Pass
Capture Performing Well Working Well
13 Multimedia Or Not Check The Keyboard Is Not Not Multimedia Pass
Keyboard Is Multimedia
Multimedia Or Not
1. Accessibility: AI virtual keyboards can be useful for people with disabilities, such as those who have
motor impairments that make typing on a physical keyboard difficult.
2. Multilingual Support: AI virtual keyboards can support multiple languages and can switch between
languages seamlessly. This feature can be useful for people who need to communicate in different
languages.
3. Mobile Devices: AI virtual keyboards can be used on mobile devices, such as smartphones and
tablets, which have limited physical keyboard space.
4. Text Prediction: AI virtual keyboards can predict the next word or phrase that the user is likely to
type, making text entry faster and more efficient.
5. Security: AI virtual keyboards can be more secure than traditional physical keyboards since there is
no physical keyboard to be compromised by keyloggers or other malware.
6. Gesture-Based Input: AI virtual keyboards can use sensors such as accelerometers and gyroscopes
6.1 Conclusion
1. We have successfully built and implemented ON AIR VIRTUAL KEYBOARD based on Artificial
Intelligence coded and executed purely on Python platform (PyCharm Community Edition 2021.1.1
x64).
2. Hand gesture recognition module worked smoothly with a minimal latency but we were able to
generate our desired output.
3. Artificial Intelligence and Digital World is a vast topic and there is a lot we can explore in it.
4. With the help of sensors, actuators and programming it was possible to bring our hardware keyboard
on air and experience something new in the area of computer science and technology with some cool
method.
5. The vision based cursor control utilizing hand motion framework was created in the C++ language, utilizing
the OpenCV library. The framework had the option to control the development of a Cursor by following the
client’s hand. Cursor capacities were performed by utilizing distinctive hand motions. The framework has the
capability of being a practical trade for the PC mouse, anyway because of the imperatives experienced; it can't
totally supplant the PC mouse.
6. The significant requirement of the framework is that it must be worked in a sufficiently bright room. This is
the principle motivation behind why the framework can't totally supplant the PC mouse, since it is normal for
PCs to be utilized in open air situations with poor lighting condition.
1. In terms of future scope, there are several directions this project could take. One possible avenue
would be to expand the virtual keyboard's capabilities beyond text input to include other forms of
input, such as voice commands or gestures.
2. Another potential area of development would be to integrate the virtual keyboard with other AI-
powered technologies, such as virtual assistants or chatbots, to create a more seamless and intuitive
user experience.
4.1 Papers
E. Brumancia
School of Computing, Sathyabama Institute of Science and Technology, Chennai, India
K. Indira
School of Computing, Sathyabama Institute of Science and Technology, Chennai, India
4.2 Books
1) Practical Explainable AI Using Python: Artificial Intelligence Model Explanations Using Python-
based Libraries, Extensions, and Frameworks
2) Learn AI with Python: Explore Machine Learning and Deep Learning techniques for Building
Smart AI Systems Using Scikit-Learn, NLTK, NeuroLab, and Keras
3) Artificial Intelligence with Python Cookbook: Proven recipes for applying AI algorithms and deep
learning techniques using TensorFlow 2.x and PyTorch 1.6
4.3 Websites
1) https://www.computervision.zone/courses/ai-virtual-keyboard/
2) https://www.analyticsvidhya.com/blog/2021/09/develop-a-virtual-keyboard-using-
opencv/