DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
VOICE-ACTIVATED EMAIL SYSTEM FOR THE
VISUALLY IMPAIRED
Guide name: Student names:C1
Dr.S.Ramesh Babu Y.Varshitha 215U1A05I5
Associate Professor T.Vinay 215U1A05H7
V.Vaishnavi 215U1A05I3
U.Anjaneyulu 215U1A05I0
1
Introduction
This research shows an email system designed to use for visually
impaired people, going through difficulties in digital communication.
Such as screen readers, tasks like attaching files remain challenging,
often requiring another persons help. Our solution including speech
recognition and text-to-speech, allowing users to compose, send, and
read emails entirely through voice commands . A facial recognition
component facilitates secure access, enabling hands-free operation
while protecting user privacy. The system is built making it beneficial
not only for visually impaired users but also for sighted people and
those who are illiterate. By streamlining email management and
ensuring secure access, this system aims to encourage visually impaired
users to engage fully in both personal and professional
communications, ultimately bridging the digital divide and promoting
greater inclusivity in the digital landscape.
2
Objectives
• Improve voice recognition accuracy
• Avoid misunderstanding between user and system
• Reduce time and efforts of user
• Ensure accessibility and efficiency
• Decrease number of commands given to the system
3
Literature review
S.No. Year of Title of the paper Methods used Parameters analysed Limitations
publication
1 2023 Real-time Email Interactive System for Mixed-methods research System usability, Limited offline
Visually Impaired Users design with both reliability in real-time functionality process
(Dr. T.Y.J. Naga Malleswari) qualitative and processing, voice delay due to real-time
quantitative techniques, recognition data handling, security
iterative development effectiveness quality in voice- based
approach, IVR integration login process
2 2022 Accessible Voice Email System for the TTS and STT tools, User satisfaction, Dependent on proper
Blind voice-command system accuracy in internet connection,
(Tulasi Sathwika) processing for email command processing, less voice recognition
navigation, prototype speed of email sending accuracy, lack of
testing with visually and reading tasks offline capabilities
impaired users
3 2021 Speech Recognition-Based Email Speech recognition Speech recognition Problem with speech
System accuracy with Google accuracy, command recognition in noisy
(Harsh D. Shah) APIs, integration with processing speed, conditions, dependent
SMTP/POP3 for email system reliability on cloud-based
management, user- services.
focused command
prompts.
4
Problem Statement
• Even though the system follow the commands given by the user, it takes
much time to access the mails which consists of more threads. The system
does not have the clarity of which thread should be retrieved.
• It takes more commands to get it. By this their may be misunderstandings
between user and system and misplacing of mails which confuses the user.
• By arranging them with keywords and making the system to describe
clearly about the mails which have many threads in it, then the user can
give the commands accordingly. This reduces the efforts and time of the
user.
Significance:
• Supporting mental health and well being
• Boosting participation in the workforce
• Enhancing quality of life
• Empowerment through technology
5
Proposed Model
Block Diagram:
Voice Email
User input Command Email thread
recognition management
(voice) processor manager
(STT and NLP) module
Backend and Email service Display Text-to-
storage APIs interface(UI) speech(TTS)
6
Data Collection
• Speech Recognition (STT) : Public speech datasets (e.g., Common Voice) and datasets collected
through user interactions.
• Text-to-Speech (TTS): Publicly available voice recordings and proprietary datasets.
• Natural Language Processing (NLP): Text corpora such as Wikipedia, news articles, and other
large text datasets.
• Email Data: Email datasets and anonymized user email data.
• Accessibility: Guidelines from organizations like W3C (Web Content Accessibility Guidelines)
and user feedback from visually impaired individuals.
• Security: Industry standards and best practices for data encryption and authentication.
7
Methodology:
• The mail is retrieved by using speech recognition and interactive
voice response without using the keyboard. While fetching a mail
which has many threads, the system queries the email API to retrieve
the relevant email thread based on user’s commands.
• The Natural language processing component analyses the text to
understand the user’s intent. Here we use graph-based algorithms and
clustering techniques to group related emails into threads. The text-
to-speech service converts the email content or system response back
into speech by reading the content.
• By using encryption algorithms the security and privacy of email data
is maintained. Then the user responds with another voice command,
continuing the cycle.
8
Design and development
1. Goal: Develop a voice-activated email system that enables visually impaired users to manage complex
email threads efficiently.
2. Requirements Analysis:
• Functional requirements-voice command recognition, email thread navigation, summarization.
• Non-Functional requirements-accessibility, performance, security.
3. System Design:
• Design the system architecture
• Develop detailed designs for each component
• Create wireframes and prototypes for the user interface.
4. Technology Stack:
• Frontend: SpeechRecognition, pyttsx3, CLI, Tkinter (optional GUI)
• Backend:Python, smtplib, imaplib, email, OAuth 2.0
9
5. Prototype Development:
• Set up the project environment
• Implement basic voice command functionalities
• Integrate STT and TTS services
• Develop a simple user interface
6. Development and Testing:-
• Implement core features in sprints.
• Conduct unit and integration testing for each feature
• Collect feedback from initial testing and refine the system.
7. Integration and Deployment:
• Ensure seamless integration of all components (frontend, backend, STT/TTS services, email APIs)
• Deploy the system on a web server
• Conduct final system testing (functional, performance, and security testing).
10
Implementation
• Voice Input: -
The user gives a voice command, e.g., "Compose email," "Read
emails," etc.
• Speech-to-Text (STT): -
The system converts the voice command into text.
• Voice Command Interpretation: -
The system interprets the text command to determine the desired
action (compose, read, reply, delete).
11
• Email Processing: - The system processes the email action: -
Compose: Converts the spoken message into text and sends it as an email.
Read: Retrieves emails from the inbox and converts them to speech.
Reply: Converts the spoken reply into text and sends it.
Delete: Deletes specified emails using voice confirmation.
• Text-to-Speech (TTS): - Converts email content or system responses
into speech.
• Voice Output: - The system speaks the response or email content back
to the user.
12
Results
13
Conclusion
• The voice-based email system for blind users provides a seamless and accessible
way to manage email communication using voice commands.
• By leveraging Speech-to-Text (STT) and Text-to-Speech (TTS) technologies, the
system ensures that visually impaired individuals can independently compose,
send, read, reply, and delete emails.
• The implementation of an Interactive Voice Response (IVR) system further
enhances user interaction, making the email experience intuitive and user-friendly.
14
Future Scope
• Improving STT and TTS Accuracy: Continuously enhance the accuracy of speech
recognition and synthesis to support a wider range of accents and dialects.
• Multilingual Support: Expand the system to support multiple languages, catering
to a global user base.
• Machine Learning Enhancements: Integrate machine learning models to
personalize the system based on user preferences and habits, improving overall
user experience.
• Mobile App Development: Develop a mobile application version to provide users
with greater accessibility on-the-go.
• Integration with Other Services: Extend the system's functionality by integrating
with other communication and productivity tools, such as calendar and task
management applications.
15
References
• https://github.com/Ankur3107/Voice-Based-Email-System
• https://cloud.google.com/speech-to-text
• https://azure.microsoft.com/en-us/services/cognitive-services/speech
-services/
• https://www.ibm.com/cloud/watson-speech-to-text
• https://www.section508.gov/create/
16
Thank You
17
