A Project Report on

“ Voice Programming AI”

Submitted in partial fulfillment of the requirement for

Third Year Engineering (Information Technology)

By

Anuja Patil (501848)

Rohit Raina (501855)
Aditya Sawant (501860)

Guided by:

Prof. Anand Pardeshi

Department of Information Technology

Fr. Conceicao Rodrigues Institute of Technology
Sector 9A, Vashi, Navi Mumbai — 400703

University of Mumbai
2020-2021
 CERTIFICATE
This is to certify that the project entitled

“Voice Programming AI”

Submitted By

Anuja Patil (501848)

Rohit Raina(501855)
Aditya Sawant (501860)

In partial fulfillment of Third Year Engineering in Information Technology for term work of
the project is approved.

External Examiner Internal Examiner

Internal Guide

Head of the Department Principal

Date: - College Seal

1
 Declaration
I declare that this written submission represents my ideas in my own words and
where others' ideas or words have been included, I have adequately cited and
referenced the original sources. I also declare that I have adhered to all principles
of academic honesty and integrity and have not misrepresented or fabricated or
falsified any idea/data/fact/source in my submission. I understand that any
violation of the above will be cause for disciplinary action by the Institute and can
also evoke penal action from the sources which have thus not been properly cited
or from whom proper permission has not been taken when needed.

________________
(Anuja Patil 501848)

_______________
(Rohit Raina 501855)

_______________
(Aditya Sawant 501860)

Date:

2
 ABSTRACT

The project proposed by us is Voice Programming. In which the user can

speak out the requirements and the keywords which he want in the code
and the system will write the code accordingly and provide it to the user.
Any user with little language specific knowledge can code using this
application. The application will be working on modern day technologies
and will be helpful for the user in many ways. As the user need to speak
the normal English language to code, so user will be free from remember
the syntax, and typing the code by keyboard, and making syntactic error.
It is much quicker and easier for any coder to code. It allow user to focus
on the logical part of programming and the syntax of the code will be
taken care by the application. It saves user from learning various
programming languages. This paper adds new way of creating interest in
beginners for judging their coding paradigm understanding and explore
one of the area for user experience field for better programming Integrated
Development Environment Development (IDE).

3
 INDEX

Sr. No. Topic Page No.

1 Introduction 6-7
1.1 Background 6
1.2 Motivation 6
1.3 Problem Definition 6
1.4 Scope 7
1.5 Limitations 7

2 Literature Survey and Analysis 8 - 10

2.1 Related work 8 -9
2.2 Existing System 9 - 10

3 System Design 11 - 13
3.1 Architecture Diagram 11
3.2 System Design 12
3.3 Flow Chart 13

4 Implementation 14
4.1 System Requirements 14
4.1.1 Hardware Requirements 14
4.1.2 Software Requirements 14
4.2 Solution Approach 14

5 Conclusions and Future Scope 15

5.1 Conclusions 15
5.2 Future Scope 15

References 16

Appendix A : Code Sample 17 -18

Acknowledgements 19

4
 LIST OF FIGURES

Sr. No. Name of the Figure Page No.

3.1 Architecture Diagram 12

3.2 System Design 13

3.3 Flow Chart 14

 1. Introduction

1.1 Background
We consider the problem of parsing natural language descriptions into source code
written in a general-purpose programming language like Python. Existing data driven
methods treat this problem as a language generation task without considering the
underlying syntax of the target programming language. Informed by previous work in
semantic parsing. In this paper we propose a novel neural architecture powered by a
grammar model to explicitly capture the target syntax as prior knowledge.
Experiments find this an effective way to scale up to generation of complex programs
from natural language descriptions, achieving state-of-the-art results that well
outperform previous code generation and semantic parsing approaches.

1.2 Motivation

Repetitive Stress Injury (RSI), also known as Repetitive finger Injury or Cumulative
Trauma, has been attributed to work requiring long periods of typing. A state of
disarray as more administrators, professionals, and white-collar employees are affected,
the issue is gaining traction. Carpal Tunnel Syndrome (CTS), tendonitis, extreme neck
pain, and back pain are all examples of these injuries. Attempting to repair the damage
caused by CTS is more expensive than preventing it. CTS is thought to be triggered by a
combination of repeated, fast keystroke movements over a long period of time,
aggravated by downward wrist extension or constant wrist strain. To avoid this the
workers are suggested to take breaks at regular intervals. But this may not be
possible in every profession. Hence, the idea of working by voice recognition comes
into focus.
.

6
1.3 Problem Definition

At the present, in the market there are many different programming languages
available. And user have to invest large amount of time in learning those different
languages. The user can get confused between syntax of different languages. Though the
user is good at coding , but every coder makes some error and it take time to find and
resolve the error mainly syntactical error. And many a time, mainly coder faces an issue
of repetitive stress injury in fingers due to constant movement of fingers while typing
the code, So they have to go through a lot of pain while typing the code. So the
solution to above problem is code diy voice which will help user to code efficiently. The
user without concerning about syntax can implement the logic . This application is also
helpful for paralyzed or handicap person to explore coding with full potential as
normal coders,

1.4 Scope

1.4.1 In future we plan to implement a fully functional platform where the user or programmer
will be able to code using voice commands only..
1.4.2 The additional functionality that can be developed is switching between the languages in
which the user wants to code. So the need of knowing syntax of different languages is
eliminated.

1.5 Limitations

• The limitation to this project is that the user will have to know the basics of programming
and also will have to know some special characters which would help them in converting
voice to code
• May find difficult for user to become familiar with it at earlier stages.

7
 2. Literature Survey and Analysis
2.1 Related Work
The said system implemented using following hardware and software requirements for initial
results generation. Hardware Requirements are: Processor –Core i3, Bus speed - 2.5 GT/s DMI,
Hard disk - 160 GB, Memory size – 4 GB RAM, Microphone and main interface of system is
developed with Software Requirements including PHP, MySQL 5.7.11, JQuery, HTML, CSS and
JavaScript. The developed system tested with Window 7.0 and above as well as on Linux 14.4 and
above.
The developed system has given an interface where in the beginning user will select Create
Program option and will start creation of program through voice, as each field in developed user
interface is voice enabled.

Traditionally ‘Text Editor through voice is based on experiencing the praxis using ‘Hidden Markov
Model’ and application was designed in Visual Basic 6.0 which translates input before finding
specific words, phrases and sentences stored in database using Speech Recognition Engine. After
finding and matching recognized input from database it puts that in document area of text. This
paper presents analysis, designing, development and implementation of same ‘Text Editor through
voice’ and approach is based on experiencing the praxis using ‘Hidden Markov Model’ and
application is designed in Visual Basic.Net framework. We have added some new phrases and
special characters in to existing application and designed extended Language Models and Grammar
in Speech Recognition Engine. We illustrate you list of extended phrases, words in tables with
figures that are effectively implemented and executed in our developed application.

It uses houndify SDK which matches the spoken word with most accurate word and responses with
json format which is then processed and converted into suitable python code and displayed in
trinket IDE.
It uses pyautogui to control screen automations. The response which is obtained in JSON format is
then processed using selenium to get the spoken response of the user which is later converted into
equivalent python syntax.
8
Mancodev is a tool similar to a programming IDE, which aims to assist students and people with
motor disabilities in arms or hands, interested in writing source code of basic programs through
voice.
It currently works with Javascript, a light, interpreted, dynamic and untyped language that allows
versatile programming. This tool has the ability to edit multiple files by browsing between tabs, as
well as using neutral colors and a minimalist aesthetic, which allows the user not to get lost among
the options available to the platform.

The paper implemented a project using Python language which is the most suitable language for
using Speech Recognition and in addition to that, they used PyQt4 for creating an Editor that will
enhance their project working i.e. converting voice to the code editor. There is also an alternative
way to develop the editor and its functionalities by the use of Electron JS. Electron is an open
source library developed by GitHub for building cross-platform desktop applications with HTML,
CSS and JavaScript.

2.2 Existing Systems

Serenade is the most powerful way to program using natural speech. Boost your productivity by
adding voice to your workflow. Serenade's speech to code engine is specifically designed for
developers. So you can quickly write code using natural commands without needing to worry about
syntax. Serenade is built on top of an open-sourced protocol. Create powerful custom voice
commands, write your own plugin, or discover ones built by the Serenade team and community.

Functional Requirements

Visual Studio Code

Visual Studio Code is a streamlined code editor with support for development operations like
debugging, task running, and version control. It aims to provide just the tools a developer needs for a
quick code-build-debug cycle and leaves more complex workflows to fuller featured IDEs, such as
Visual Studio IDE.
Python libraries
9
1. Pyaudio
2. Nltk
3. Gtts
4. Keyword
5. Speech recognition

10
 System Design
3.1 Architecture Diagram

Figure 3.1 – Architecture Diagram

11
3.2 System Design

Figure 3.2 – System Design

12
3.3 Flow Chart
In Figure 3.3, the flow of data through the system and the various processes that are carried out
using it.

Figure 3.3 - Flow Chart

13
 IMPLEMENTATION

4.1 System requirements

4. 2. l Hardware requirements

— Pc which supports latest vscode.

4. 3 Software requirements

— VS code
- python complier
- speech recognition module

4.4 Solution approach

1. Segregation of text

2. Entity Identification

3. Building neural network

4. Creating VS Code extension

14
 CONCLUSION
5.1 Conclusion

We tended to the issue of client while building up a PC program. Building up a PC program isn't an
simple errand it needs equipment assets which client need to handle. While ceaseless composing
the code there might be probability of wounds to the fingers of the client. To stay away from the
issues we are planning a framework in which the PC program can created through the voice. The
voice will perceived by the framework and that perceived words or word will be contrasted and the
put away watchwords in the information base furthermore, assuming they are coordinated, that will
be imprinted on supervisor and after this again by perceiving the particular watchwords the
program will be arranged and executed. This framework will be simple to utilize, it decrease
human endeavors and the utilization of equipment assets. It would be without a doubt helpful for
visually impaired.

5.2 Future Scope

In future we plan to implement a fully functional platform where the user or programmer will be
able to code using voice commands only. The additional functionality that can be developed is
switching between the languages in which the user wants to code. So the need of knowing syntax
of different languages is eliminated.

15
 References

1. Voice to Code Editor Using Speech Recognition Anurag Singh, Ganesh Tambatkar, Shashank

Hanwante, Nitish Agrawal, Rahul Hajare, Ketki Khante

2. Smart Python Coding through Voice Recognition II. A. ,lawale, A. B. Pawar, D. N. Kyatanavar

3. Text Editormthrough voice and its Analysis, designing, development and implementation using

Speech Recognition Farhan Ali Surahio, Awais Khan Jumani, Sawan Talpur

4. An Approach to Accept Voice in Code Editor through Speech Recognition G.Parthasarathy, A

R. Rangeesh, V S. Sri Kishowr, R. Sriram, S. Vijay TRP Engineering College Tiruchirapalli, India.

5. Programming By Voice Assistance Tool For Physical Impairment,Patients Classified Into

Peripheral Neuropathy Centered On Arms Or Hands kfovement Difficulty Jonathan Giovanni Soto

hJuñoz, Arturo Ivan de Casso Verrlugo, Jesus Andrés Sandoval Bringas, kliguel Parra Garcia,

Eliseo Geraldo Gonzalez

6. https://serenade.ai/

16
 Appendix

import pyaudio
import speech_recognition as sr
import webbrowser as wb
import subprocess
from time import ctime
import time
import os
import playsound
from gtts import gTTS
import random

from nltk.tokenize import word_tokenize

from keyword import kwlist

r = sr.Recognizer()

def record_audio(ask = False):

with sr.Microphone() as source:
if ask:
speak(ask)
audio = r.listen(source)
voice_data = ''
try:
voice_data = r.recognize_google(audio)
print('Input:')
print(voice_data)
except sr.UnknownValueError:
speak('Sorry I did not get it')
except sr.RequestError:
speak('Sorry my service is down')
return voice_data

def speak(audio_string):
tts = gTTS(text=audio_string, lang='en')
r = random.randint(1, 10000000)
audio_file = 'audio-' + str(r) + '.mp3'
tts.save(audio_file)
#print(audio_string)
playsound.playsound(audio_file)
17
 os.remove(audio_file)

def respond(voice_data):
l = word_tokenize(voice_data)
sp_chr = ['insert', 'add', 'parameter', 'goto', 'cursor']
key_words = kwlist
kwlist.append('print')
found_spchr = []
found_kw = []
constants = []
if l[0] in sp_chr: found_spchr.append(l[0])
for wrd in l:
if wrd in key_words:
found_kw.append(wrd)
elif wrd in sp_chr:
if wrd not in found_spchr:
found_spchr.append(wrd)
else:
constants.append(wrd)
print('Output:')
print('Command =',found_spchr,
'\nKeywords =',found_kw,
'\nConstants =',constants)

time.sleep(1)
speak('Listning...')

voice_data = record_audio()
respond(voice_data)

18
 Acknowledgements

We would like to take this opportunity to express our deepest appreciation to

everyone who helped us complete this report. We would like to express our heartfelt
gratitude to our project manager, Mr. Anand Pardeshi, whose supportive suggestions
and encouragement aided us in coordinating our project, especially in writing this
report. We must express our gratitude for the advice given in our project
presentation.

19