Rayat Shikshan Sanstha’s

Karmaveer Bhaurao Patil College of Engineering, Satara

Department : Computer Science & Engineering
Academic Year: 2024-25 Semester-V

Under the Guidance of

Prof. Anuja Jadhav

by
1. Arundhati Avinash Gadekar (22)
2. Awantika Satish Karpe (20)
3. Karan Vishnu Gite (52)
4. Suraj Sanjay Patil (53)
 Motivation
The development of Jarvis is driven by the need for efficient, voice-controlled
systems that simplify tasks and boost productivity. By automating tasks like
opening apps, sending messages, and browsing the web using voice
commands, Jarvis saves time and effort. It also aims to be a cost-effective
alternative to commercial assistants like Siri or Alexa, using free APIs and
open-source tools. The inclusion of facial recognition adds an extra layer of
security, making it both user-friendly and secure.

Case Study

1. Personal Productivity: A software developer uses Jarvis to multitask by

opening apps, sending WhatsApp messages, and browsing the web through
voice commands, increasing efficiency.
2.Accessibility: For visually impaired users, Jarvis offers hands-free
interaction with devices using voice commands, making tasks like messaging
and browsing easier.
3.Customer Service Automation: A small business owner uses Jarvis to
handle customer inquiries and automate WhatsApp messages, improving
business efficiency without high costs.
 Introduction
In the rapidly evolving digital communication landscape, automation tools are
becoming essential for enhancing user efficiency. This mini project focuses on
developing an AI-based voice assistant specifically for WhatsApp automation,
allowing users to perform various tasks through simple voice commands

sss
A standout feature of this voice assistant is the integration of facial recognition
technology for secure authentication. Additionally, the assistant utilizes
natural language processing powered by ChatGPT, enabling it to understand
and respond to user queries conversationally. This feature enhances user
experience by providing intuitive support and facilitating seamless
communication.

Overall, this project represents a significant innovation in automating and

securing WhatsApp interactions, making digital communication more efficient
and user-friendly.
 Literature Survey
Voice assistants have evolved rapidly due to advancements in artificial
intelligence (AI) and natural language processing (NLP). Early developments
such as IBM’s Shoebox (1961) and Dragon Dictate (1990s) paved the way
for modern AI-based assistants like Siri (Apple, 2011), Google Assistant
(2016), and Amazon Alexa (2014). These systems rely on speech recognition
and natural language understanding to interpret and respond to user
commands.

Speech Recognition Technologies: Systems like Google’s Speech-to-Text

API and Microsoft’s Azure Speech Services have improved accuracy using
deep learning models like Deep Neural Networks (DNNs) and Recurrent
Neural Networks (RNNs). Research has shown that integrating such models
with language models like Transformer-based architectures (e.g., BERT,
GPT) enhances understanding and response quality.
 Literature Survey

Text-to-Speech (TTS) has evolved from early concatenative synthesis to

modern neural TTS models such as WaveNet, which delivers more natural-
sounding speech. These innovations help bridge human-computer interaction
by producing highly intelligible and natural voice output.

Face Recognition for authentication has gained traction in securing voice

assistants, using technologies like convolutional neural networks (CNNs)
and feature-matching techniques for robust user identification. This helps
personalize and secure interactions.

Recent studies focus on improving conversational AI through models like

GPT-3 and open-source alternatives such as DialoGPT, emphasizing their
role in making interactions more dynamic and human-like.
 Existing Systems
Amazon Alexa Google Assistant Apple Siri

• Description: Amazon • Description: Google • Description: Siri is

Alexa is a cloud-based Assistant is an AI- Apple’s voice-activated
voice service available powered virtual assistant available on
on Amazon Echo and assistant developed by iOS devices, including
other Alexa-enabled Google. It is available iPhones, iPads, and
devices. It can perform on smartphones, smart Macs.
tasks such as controlling speakers, and other • Capabilities: Voice
lights, adjusting connected devices command recognition,
thermostats, and • Capabilities: Advanced natural language
managing entertainment speech recognition, understanding, and
systems. natural language seamless integration
• Capabilities: Voice processing, and with Apple’s ecosystem.
recognition, natural integration with Google
language understanding, services and third-party
and integration with devices.
numerous third-party
smart devices.
 Objectives

Develop a functional AI-based

voice assistant that can execute
Implement WhatsApp
commands, including converting
automation to send messages
text to speech, recognizing
programmatically based on voice
speech, and performing actions
commands.
like opening applications and
websites.

Integrate a free ChatGPT API

alternative to provide
conversational AI capabilities
without incurring high costs.
s
 Proposed System

Auto Classification Speech-to-text

(detect wake words) (transcribe query)

Spoken Query

Text-to –Speech Language model

(synthesis speech) (generate response)
Spoken Answer

- On device

- On the cloud
Proposed System

Frontend View of Voice Assistant

 System Architecture
User Interface(UI)
(Voice & Text Inputs)

Speech Recognition Modules –

(Speech-to-Text)
User Interface (UI)
Speech Recognition (Speech-to-
Command Processing Module Text)
(Logic and Task Execution) Command Processing Module
Text-to-Speech (TTS)
Application and Website Control
Face WhatsApp Automation
App Whatsapp ChatGPT API Alternative
TTS Control Automation Authenti-
cation Face Authentication

chatGPT API Alternative

(conversational AI)
 Advantages

Cost-Effective &
Helps to Blind
easily handle by
people to send
non-technical
messages
person also

Multi-Functional Improve response

and 24/7 time
Availability
 Limitations and Future Scope

Limitations Future Scope

• Dependency on Devices • Integration with Additional

• Dependency on internet Services
• Language • Enhanced platform services
• User Personalization
• Multiple Language options
 References

1. https://www.researchgate.net/publication/372394842_Development_o
f_AI-based_voice_assistants_using_Large_Language_Models
2. Artificial Intelligence-based Voice Assistant | IEEE Conference
Publication | IEEE Xplore
3. http://www.ijert.org
4. Voice Assistants: The Present and Future
5. A Comprehensive Review on Speech Emotion Recognition
6. Speech Synthesis with Transformers: A Review
Project Implementation