Technical Answers to Real

Time Problems
FACULTY: PROF. SASIKALA R.
AUTOMATION OF USER
INTERFACE
FOR
PHYSICALLY DISABLED
Team Members

 16BCE0281 BELAL AHMED

 16BCE0396 SUYASH GUPTA
 16BCE0623 ANUP MANDHANA
 16BCE0820 KRATI AGARWAL
 16BCE0976 SANKALP MITTAL
Objective

 Provide our customer better user experience.

 Make the better use of technology for the Handicapped people(specially
having disability with hands).
 Our system is more user friendly as compared to the current technology
used as it can be utilized by any age group .eg. Amazon echo
 Easy gesture control by using voice control module and Artificial
Intelligence Algorithms.
  MICROPHONE

 GOOGLE API

 BLUETOOTH MODULE

Components  ARDUINO

 USB

 PYTHON IDLE
  MICROPHONE: Here we are using the
mobile’s microphone to capture the voice
and through mobile’s internal Bluetooth to
Arduino connected to a Bluetooth module.
 GOOGLE API: To convert the voice in signals
that is transferrable through Bluetooth
wirelessly.

Component  BLUETOOTH MODULE: It is connected to

Arduino module. It receives the speech signal
Description 
of microphone and give it to Arduino.
ARDUINO: It processes and organise the
words received and also decode and then
again encode as per the requirement to
transfer it to system.
Algorithm/Flowchart
MODEL FLOW Chart
Experimental Setup

 Various exclusive features converting speech to docs ,finding errors

,shifting the slides in the presentation by voice commands etc.
 Automating the whole system using Natural Language Processing for
decoding the user messages and use of Artificial Intelligence algorithm for
better understanding of customer wants.
 Basically Arduino is the processor to decode the statement as per the
machine and transfer to the system.
 Literature Survey
Paper 1 : A Comparison of GUI
and VUI Testing

While some VUI testing techniques are similar to those in GUI testing, VUI testing
presents new challenges due to the temporary nature of the voice input modality. Many of the
existing techniques for VUI testing focus on Interactive Voice Response (IVR) systems, which
are the telephone systems companies use to manage customer calls. These vary greatly from
GUI where the input modality is voice rather than the mouse/keyboard. While the input
modality is temporary, the feedback is permanent. The primary difference between GUI
and VUI testing is that GUI testing appears to be more empirical (objective)
where VUI testing appears to be more user-focused (subjective) to VUI testing would be to take
advantage of existing GUI techniques.
Paper 2: Swar: The Voice Operated PC

Theprepared system if visualized as a block diagram will have the following

components: Sound Recording and word detection component, feature extraction
component, speech recognition component, acoustic and language model.
Multi-dimensionalHidden Markov Model based component used. It is the most
important component of the system and is responsible for finding the best match in the
knowledge base, for the incoming feature vectors
 Paper 3: A Computer Remote Control System Based on
Speech Recognition Technologies of Mobile Devices and
Wireless Communication Technologies

 Google Speech Recognition. Google uses

artificial intelligence algorithms to recognize
spoken sentences, stores voice data
anonymously for analysis purposes, and
cross matches spoken data with written
queries on the server.
Paper 4: Designing Driver-centric
Natural Voice User Interfaces

 This paper explored the use of Natural Voice User Interfaces (NVUI) in driving
scenarios, analyze its risks and benefits and aims to propose design guidelines for
driver-centric applications based on previous research experience.
 NVUI APPLICATION IN AUTOMOTIVE ENVIRONMENTS : A natural voice user
interface (NVUI) is one in which a person can speak to a computer system as they
would when conversing with another human being. NVUIs are agent based
systems that understand what the user says and respond to it ; therefore, allowing
users to interact in a natural way eliminating the need for remembering a specific
lists of commands.
 Human speech refers to speech segments that are prerecorded by a
human while synthesized speech is an artificial production of speech via a
computer, and is often produced via text-to speech (TTS) engines. Utilizing natural
speech, in comparison with synthetic speech, in applications that deliver high
amounts of prerecorded speech segments is found to be difficult and rather costly
Paper 5:The Design of User Interfaces for
Digital Speech Recognition Software

DSRS is a command and control application integrated with the UNIX desktop
environment. It accepts user commands spoken into a microphone and converts them
into keystrokes.
 DSRS comprises several major components. Of these components, three are licensed
from Dragon Systems, Inc.: the front-end processor, the recognizer engine, and the
speaker-independent speech models. Data acquisition consists of the microphone,
audio card, and the multimedia services application programming interface (API) that
provides support for the sound card. The front-end processor analyzes a stream of
digitized data and differentiates between silence, noise, and speech; it then extracts a
set of computed features from the speech signals.
Speech recognition is an evolutionary step in the overall computer–user interface. It is
not a replacement for the keyboard and mouse and should be used to complement
these devices
Our CONTRIBUTION

 REVOLUTION for Handicapped people to help them fully AUTOMATE their

PERSONAL computers.
 Our PROJECT is different from existing work, as we are using ARDUINO
which is extremely CHEAP.
 We are NOT using ASP.NET, hence our project is not limited to WINDOWS
platforms only.
 LOW POWER
 EXTREMELY USER Friendly, easy to use for people from all ages.
 LESS hardware set up.
 The user of the automation system designed is
expected to be:
 Familiar with basic commands that will be
used in the voice command application.
User For example OPEN for opening an
Characteristics application or file, NEXT for going one
page up in a text file or moving on to the
Needed to next slide in a slide show.

operate the  Able to provide voice commands in a

clear and legible manner to the system
product  Familiar with basic operations that can be
performed on a file or application
 Requiring special handling of power point
presentations
Project TIME LINE

Secondly we will
At First we will do the complete the
survey to get the software part of this
And at Last we will
input from the users project which
connect the
regarding the involves the coding
hardware
automation of the in python. We will
connection which
computer system. In 01 take the voice 02
will capture the
this we get the words command and
voice and will
which people will analyze the words
transmit it to the
commonly use to and based on that
computer system.
perform a certain conditionally it will
task perform the required
task.
 •The product is efficient enough to produce

Efficiency a specific outcome with a minimum

amount or quantity of waste, expense, or
unnecessary effort.

•The ability of a product to perform its

Reliability required functions under stated conditions

WHY our
for a specified time.

PRODUCT? •software can be run (with little or no

Portability modification) on different hardware and/or

software platforms, and works with different
versions of the same hardware or program.

•Our software can be used by specified

Usability consumers to achieve quantified objectives

with effectiveness, efficiency, and
satisfaction in a quantified context of use.
Thank You