PERSONAL ASSISTANT DEVICEFOR BLIND THROUGH VOICE

Guide : Prof . Dr Sunitha

Kondrathi Sravani 20TP1A0561
Muthyala Yamini 20TP1A0584
Putcha Laxmi Prasanna 20TP1A0597
Samula Manohar Reddy 20TP1A05A7
 ABSTRACT

• Billions of devices belonging to different manufacturers and domains are connected to the Internet of Things.

The amount of consumer equipment that a single person can manage, however, is limited.

• Additionally, personal information is spread across several sources, which prevents a seamless integration.
Intelligent personal assistants are a solution to facilitate the integration of online services and data sources.

• However, the management of IoT equipment is still challenging due to difficulty of interaction, security
information integration, among other problems.
 INTRODUCTION

• Advances in technology have helped in the automation of everyday tasks. Scheduling

meetings, reserving hotels, and paying bills are now easily performed by autonomous
software agents on behalf of humans.

• These software agents, called intelligent personal assistants (IPA), are capable of gathering
information and triggering actions from online services
 EXISTING SYSTEM

• A central element is called Swarm Broker, which provides semantic registry

and discovery of services, an attribute-based access control system, and a
COAPHTTP proxy.

• The network size and complexity of the Swarm, however, poses usability
challenges, as human users may not be able to easily interact with the
plethora of available services.

• In this context, an approach to abstract the Swarm complexity is needed.

 DISADVANTAGES

1.Complicity is high
2.Not understanding commands
3.Software Integration problems
 PROPOSED SYSTEM

• The Swarm offers new opportunities for building a

personal assistant in a consumer electronics network,
such as opportunistic gathering of devices for user
interaction, for example an agenda may notify about an
appointment and show it in the dynamically recruited TV.
• This work proposes the creation of an intelligent personal
assistant to provide rich interaction with a network,
taking advantage of the Swarm platform
 ADVANTAGES

• Time consumption is less

• It is easy Handle
• It help blind people to be independent
 ANALYSING

• There are various personal assistant devices designed to assist individuals with visual impairments
through voice analysis.
• Examples include Amazon Echo and Google Home, which offer voice-controlled features and can
provide information audibly.
• Additionally, specialized devices like OrCam My Eye are designed to help with tasks such as reading text
aloud.
• Always check for the latest advancements and specific features to find the most suitable option.
 COMMUNICATION AGENT
• The Communication Agent is responsible for the user interaction, using different
methods, such as natural language, gestures and touch screens. The current version
of the Swarm Assistant uses natural language processing (NLP) for voice and text.
• This agent provides a multiplatform mobile web interface, and a backend that maps
intents in natural language to actual commands in the Swarm network. Intents are
associated to phrases that have the same semantic meaning.
 PERSONAL DATABASE

• The Personal database stores information for the Swarm Assistant to

act proactively on behalf of a person. That includes location, interests,
preferences, and relationships.
• This module is a semantic database, which stores ontology of concepts
and relationships related to the person domain.
• It leverages interoperability through semantic inference.
 TASK MANAGER

• Conversion of Speech-to-Text and Text-to-Speech is performed by task

manager.

Speech-to-Text
• Far from a being a fad, the overwhelming success of speech-enabled
products like Amazon Alexa has proven that some degree of speech support
will be an essential aspect of household tech for the foreseeable future.
• If you think about it, the reasons why are pretty obvious. Incorporating
speech recognition into your Python application offers a level of interactivity
and accessibility that few technologies can match.
 SERVICE MANAGER

• Analysis of commands in datasets which are stored by user and

matching them with user requested voice.
• It arranges all the things which are ready to execute the commands
if they are available.
 EXECUTE COMMAND

• Step 1 : After finding the match for the given command, run the
respective command
• Step 2 : Send the resource allocation for particular application are
within the console only depending up on type application we
need to access.
 DESIGN ARCHITECTURE

• Design Engineering deals with the various UML [Unified Modelling language] diagrams for the
implementation of project. Design is a meaningful engineering representation of a thing that is to be
built. Software design is a process through which the requirements are translated into representation
of the software.
SYSTEM ARCHITECTURE
 UML DIAGRAMS
• 1.Use Case Diagram
• The main purpose of a use case diagram is to show what system functions are performed for
which actor. Roles of the actors in the system can be depicted. The above diagram consists of
user as actor. Each will play a certain role to achieve the concept.
• 2.Class Diagram
• In this class diagram represents how the classes with attributes and methods are linked together to
perform the verification with security. From the above diagram shown the various classes involved
in our project.
• 3.Sequence Diagram
• Activity diagrams are graphical representations of workflows of stepwise activities and actions with
support for choice, iteration and concurrency. In the Unified Modeling Language, activity diagrams can be
used to describe the business and operational step-by-step workflows of components in a system. An
activity diagram shows the overall flow of control

.
• 4.Activity Diagram
• Activity diagrams are graphical representations of workflows of stepwise activities and actions
with support for choice, iteration and concurrency. In the Unified Modeling Language, activity
diagrams can be used to describe the business and operational step-by-step workflows of
components in a system. An activity diagram shows the overall flow of control.
 SAMPLE CODE
import pyttsx3

import webbrowser

import smtplib

import random

import speech_recognition as sr

import Wikipedia

import datetime

import wolframalpha

import os

import sys

import glob

engine = pyttsx3.init('sapi5’)

client = wolframalpha.Client('Your_App_ID’)

voices = engine.getProperty('voices’)

engine.setProperty('voice', voices[len(voices)-1].id)

def speak(audio):

print('Computer: ' + audio)

engine.say(audio)

engine.runAndWait():
def greetMe():

currentH = int(datetime.datetime.now().hour)

if currentH >= 0 and currentH < 12:

speak('Good Morning!’)

if currentH >= 12 and currentH < 18:

speak('Good Afternoon!’)

if currentH >= 18 and currentH !=0:

:speak('Good Evening!')

greetMe()

speak('Hello Sir, I am your digital assistant LARVIS the King Sreekanth!’)

speak('How may I help you?’)

def myCommand():

r = sr.Recognizer() with sr.Microphone() as source:

print("Listening...")

r.pause_threshold = 1

audio = r.listen(source)try:

query = r.recognize_google(audio, language='en-in’)

print('User: ' + query + '\n’)

except sr.UnknownValueError:

speak('Sorry sir! I didn\'t get that! Try typing the command!’)

query = str(input('Command: ‘))

return query

if _name_ == '_main_’:

while True:

query = myCommand();

query = query.lower()

if 'open youtube' in query:

speak('okay’)

webbrowser.open('www.youtube.com’)

elif 'open google' in query:

speak('okay’)

webbrowser.open('www.google.co.in’)

elif 'open facebook' in query or 'open fb' in query:

speak('okay’)

webbrowser.open('www.facebook.com’)

elif 'open gmail' in query or 'open mail' in query:

speak('okay’)

webbrowser.open('www.gmail.com’)

elif 'open twitter' in query :

speak('okay’)

webbrowser.open('www.twitter.com’)

elif "what\'s up" in query or 'how are you' in query:

stMsgs = ['Just doing my thing!', 'I am fine!', 'Nice!', 'I am nice and full of energy’]
elif 'email' in query:

speak('Who is the recipient? ‘)

recipient = myCommand()

if 'me' in recipient:

try:

speak('What should I say? ‘)

content = myCommand()

server = smtplib.SMTP('smtp.gmail.com', 587)

server.ehlo()

server.starttls()

server.login("Your_Username", 'Your_Password’)

server.sendmail('Your_Username', "Recipient_Username", content)

server.close()

speak('Email sent!’)

except:

Speak('Sorry Sir! I am unable to send your message at this moment!’)

elif 'nothing' in query or 'abort' in query or 'stop' in query:

speak('okay’)

speak('Bye Sir, have a good day.’)

sys.exit()

elif 'hello' in query:

speak('Hello Sir’)
elif 'bye' in query:

speak('Bye Sir, have a good day.’)

sys.exit()

elif 'play music' in query:

music_folder = 'music’

music=glob.glob("music/*.mp3")

#music = ['Enni_Soni', 'Psycho_Saiyaan','Deewana_Tune','Enna_Sona','Sanam_Re’]

random_music = music_folder +'\\'+ random.choice(music) + '.mp3’

os.system('start '+random.choice(music))

speak('Okay, here is your music! Enjoy!’)

sys.exit()

else:

query = query

speak('Searching...’)

try:

try:

res = client.query(query)

results = next(res.results).text

speak('WOLFRAM-ALPHA says - ‘)

speak('Got it.’)

speak(results)

except:

results = wikipedia.summary(query, sentences=2)speak('Got it.')speak('WIKIPEDIA says - ')speak(results)except:webbrowser.open('www.google.com')speak('Next Command!

 SCREEN SHOTS
This project is implements like application using python and the Server process is maintained using the SOCKET & SERVERSOCKET and
the Design part is played by Cascading Style sheet.
 CONCLUSION

This paper proposed architecture for a personal assistant that fulfills the challenges of interacting
with the system to perform the operations that user wants to execute. It was built on top of the
Swarm platform, and added natural language processing and semantic integration of personal
data gathered from different sources.
 REFERENCES

[1] L. C. P. Costa, J. Rabaey, A. Wolisz, M. Rosan, and M. K. Zuffo, “Swarm os control plane: an architecture
proposal for heterogeneous and organic networks,” IEEE Transactions on Consumer Electronics, vol. 61, no. 4, pp.
454–462, November 2015.
[2] L. C. De Biase, P. C. Calcina-Ccori, F. S. Silva, and M. K. Zuffo, “The semantic mediation for the swarm An
adaptable and organic solution for the internet of things,” in Consumer Electronics (ICCE), 2017 IEEE International
Conference on. IEEE, 2017, pp. 78–79.
[3] J. Santos, J. J. P. C. Rodrigues, J. Casal, K. Saleem, and V. Denisov, “Intelligent personal assistants based on
internet of things approaches,” IEEE Systems Journal, vol. 12, no. 2, pp. 1793–1802, June 2018.
[4] M. Funk, L.-L. Chen, S.-W. Yang, and Y.-K. Chen, “Addressing the need to capture scenarios, intentions and
preferences: Interactive intentional programming in the smart home,” International Journal of Design, vol. 12, no. 1,
pp. 53–66, 2018.