Project Title – Voice Assistant System with Home Automation facility

(EC 881) Project Stage – II

A Project Report Submitted in Partial Fulfillment of the Requirements for the
Degree of Bachelor of Technology
In Electronics and Communication Engineering by:

Name: Sayan Sarkar

Registration No: of 201300100310045 OF 2020-21
Roll No: 13000320080

Name: Kashyap Ghosh

Registration No: of 201300100310057 OF 2020-21
Roll No: 13000320068

Name: Anshuman Das

Registration No: of 201300100310062 OF 2020-21
Roll No: 13000320063

Name: MD Injamamul Haque Ansari

Registration No: of 211300100320010 OF 2021-22
Roll No: 13000321135

Under the guidance of

…………………………… ….. Dr. Arpita Adhikari
(Assistant Professor)
Department of Electronics and Communication Engineering

TECHNO MAIN SALT LAKE

EM 4/1 Salt Lake City, Sector – V Kolkata-700091

i
ACKNOWLEDGEMENT

We express our special thanks of gratitude to Dr. Arpita Adhikari of the Department of
Electronics & Communication Engineering (ECE) for her able guidance and valuable
support in completing our project.

Heartfelt thanks are also conveyed to all the members of the teaching and non- teaching
staff of the Department of ECE for their cordial support and help, whenever needed.

Date: 24/05/2024

………………
(Sayan Sarkar)

………………
(Kashyap Ghosh)

………………
(Anshuman Das)

………………
(Md Injamamul
Haque Ansari)

ii
 CERTIFICATE (8th Semester)

This is to certify that Sayan Sarkar, Kashyap Ghosh, Anshuman Das & Md
Injamamul Haque Ansari have carried out their project work entitled “Voice Assistant
System with Home Automation facility” as a part of the curriculum for the B.Tech
Degree in Electronics & Communication Engineering (ECE) under Maulana Abul
Kalam Azad University of Technology for the year 2020-2024.
This project report is approved by the undersigned only for the purpose for which it is
submitted. The candidate is entirely responsible for the statements, opinions and
conclusions contained herein.

(Dr. Arpita Adhikari)

(Signature of the Mentor)

(Signature of HOD, ECE Dept,

Techno Main Salt Lake, with date)

iii
 CONTENTS:

S. N Subtopics Page No.

1. List of Figures v
2. List of tables v
3. Abstract 1
4. Introduction 2
5. Literature Review 3
6. Methodology 6
6.1 Problem Definition 6
6.2 Functional Block Diagram 7
6.3 Planning and Approach
8
6.4 Design Issues
13
7. Implementation 14
8. Result Analysis 19
9. Discussion 27
10. Resource Utilization 29
11. Conclusion and Future Scope 30
12. References 31
13. Code Snippets 33

iv
1. LIST OF FIGURES

Figure Name Page No.

Fig. 1 Voice Assistant Project Layout 7
Fig. 2 Raspberry Pi 4 Model B 9
Fig. 3 Raspberry Pi Pins descriptions 10
Fig. 4 ESP32 Microchip Pins layout 11
Fig. 5 Raspberry PI with Connections 14
Fig. 6 Voice Assistant Device 15
Fig. 7 Voice Assistant Inner Connections 15
Fig. 8 SMS notification from voice assistant 15
Fig. 9 Voice Assistant Code Block Diagram 16
Fig. 10 Home Automation Circuit Diagram 18
Fig. 11 Home Automation Device 18
Fig. 12 Home Automation Inner Circuit 18

2. LIST OF TABLES

Table Name Page No.

Table 1: Test Case 1 19
Table 2: Test Case 2 20
Table 3: Test Case 3 20
Table 4: Test Case 4 21
Table 5: Test Case 5 22
Table 6: Test Case 6 22
Table 7: Test Case 7 23
Table 8: Test Case 8 24
Table 9: Test Case 9 25
Table 10: Test Case 10 26
Table 11: Test Case 11 26
Table 12: List of Components and their estimated cost 29
Table 13: Similar voice Assistant products in the market 29

v
3. ABSTRACT:

The project investigates the dynamic landscape of voice assistant technology, exploring its
evolution, capabilities, and potential impact on user interactions. The project focuses on the
development and implementation of a Voice Assistant System (VAS) designed to enhance
user experiences across various domains. The project explores the integration of voice
assistant technology into the realm of home automation, aiming to enhance user
convenience and efficiency in managing various household tasks.

The primary objectives of this project encompass the design and implementation of a robust
and context-aware voice assistant capable of understanding natural language commands,
providing relevant information, and executing tasks with precision. Also, the project
understands diverse user instructions related to home automation, and to seamlessly
integrate this interface with a robust home automation system.

Furthermore, the project discusses the potential applications of voice assistant technology in
diverse fields, including personal assistance, information retrieval, and hands-free device
control. It evaluates the impact of voice assistants on user efficiency, accessibility, and the
overall user experience in both consumer and enterprise environments. Ultimately, this
project seeks to foster a deeper understanding of voice assistant technology and its
transformative role in shaping the way users interact with digital systems, paving the way
for more intuitive and seamless human-machine communication.

1
4. INTRODUCTION:

In today’s era almost all tasks are digitalized. We have Smartphone in hands, and it is nothing
less than having world at your fingertips. These days we aren’t even using fingers. We just
speak of the task, and it is done. There exist systems that can perform specialized task such as
booking a flight, or finding cheapest book online from various ecommerce sites and then
providing an interface to book an order are helping automate search, discovery, and online
order operations. In an era dominated by technological advancements, the fusion of artificial
intelligence and smart home automation has given birth to a cutting-edge project – the Voice
Assistant System with Home Automation. This innovative solution seamlessly integrates voice
recognition technology with the capability to control and manage various aspects of a home's
automation [1].

Voice Assistants are programs that help you ease your day-to-day tasks, such as showing
weather report, creating reminders, making shopping lists etc. They can take commands via
text (online chat bots) or by voice. Voice based intelligent assistants need an invoking word or
wake word to activate the listener, followed by the command. We have so many virtual
assistants, such as Apple’s Siri, Amazon’s Alexa and Microsoft’s Cortana.[2] Virtual assistants
are turning out to be smarter than ever. Allow your intelligent assistant to make email work for
you. Detect intent, pick out important information, automate processes, and deliver
personalized responses.

Voice searches have dominated over text search. Web searches conducted via mobile devices
have only just overtaken those carried out using a computer and the analysts are already
predicting that 50% of searches will be via voice by 2020[3]. This project was started on the
premise that there is enough openly available data and information on the web that can be
utilized to build a virtual assistant that has access to making intelligent decisions for routine
user activities like home automation system.

The system is designed to understand and respond to natural language commands, providing an
intuitive and hands-free way for users to interact with their smart home devices. Users can
control lighting, thermostats, security systems, and more, simply by speaking commands.

This project extends beyond basic voice recognition, incorporates a system of voice reminder
with SMS notification on user’s smartphone. It facilitates setting of reminder messages through
voice commands thus eliminating the need for typing and manually setting reminder time. The
SMS notification helps the user in case he/she is not near the voice assistant during reminder.

2
5. LITERATURE REVIEW

Voice Recognition Technologies: Voice recognition technologies have witnessed significant

advancements in recent years. This includes the development of advanced Natural Language
Processing (NLP) algorithms and machine learning models [4]. NLP is used by virtual
assistants to translate a person's message or voice input into actionable commands. The natural
language speech signals generated whenever a user request her personal assistant a question to
complete a task are translated into command line instructions or digital information that can be
analyzed by software. Voice assistants can now understand and interpret spoken language more
accurately, resulting in a more natural and user-friendly interaction [5]. This advancement has
broadened their user base, encompassing people of all age groups, including children and the
elderly. Voice assistants have begun to integrate into people's lives. It has a positive effect on
the society as well as on the individual. For office, artificial intelligence voice assistant can
help people work efficiently. For individuals, voice assistants can not only provide
convenience in life. To a certain extent, it can make up for the lack of people's emotions.
Some people are willing to chat with a voice assistant after they start using it, which also has a
lot to do with the faster pace of life. While voice assistants are not as emotionally rich as
people, voice assistants can be a qualified listener.

Application Programming Interfaces (APIs) serve as bridges between different software

applications, allowing them to communicate and share data seamlessly. APIs enable developers
to leverage functionalities from existing systems, fostering interoperability and accelerating
software development. The versatility of APIs extends from web development to mobile
applications, creating a foundation for innovation in the digital realm [6].

Convergence of Voice Assistants and APIs

Major voice assistant platforms provide developers with APIs to integrate voice capabilities
into their applications. These APIs allow developers to harness the power of voice recognition,
natural language understanding, and speech synthesis. This section explores how voice
assistant APIs empower developers to create voice-enabled applications, expanding the scope
of user interaction. The integration of voice assistants and APIs offers a high degree of
customization. Developers can extend the capabilities of voice assistants by creating custom
skills or actions using APIs. This customization enhances user experiences by tailoring voice
interactions to specific applications, industries, or user preferences.

3
Technology Companies work on Voice Assistant and Home Automation

In the past 60 years, the technology of voice assistants has developed rapidly. From the
combination of three robust acoustic models at the beginning, the accuracy rate of voice
assistants has reached the level of human stenographers, reaching 95% accuracy. Users
often use the speech-to-text feature, which allows people on the go to send messages and take
shorthand [7].

Amazon launched smart speakers, which connect apps on smart phones with furniture in daily
life, opening up a broader space for voice interaction and occupying a larger market. The
reason for the success of Amazon speaker is the fact that it can truly interact with dialogue,
which is the most requested feature of today's social users. Amazon holds the largest share of
the facetious voice market. In 2014, it developed Alexa, a voice recognition system, which
leads all smart voice assistant companies to become the market leader. Amazon has combined
the company's appliances with a voice assistant to make a feature-rich product[8].

Siri is mainly applied in Apple's built-in, which is conducive to Apple's promotion to users
with the help of Apple's series of products. This kind of intelligent voice assistant is more
suitable for users who buy Apple’s companion device. Microsoft's Cortana is more associated
with the PC [9].

Cortana lets users experience when their phone and computer work together. For people who
use the voice assistant for office work, Cortana will help them have a better experience at work
than Siri, which is more used in daily life [10].

Google, as the world’s largest search engine, has a strong technical hand in Google
assistant. Google Assistant is a virtual assistant software application designed by Google,
primarily for mobile and home automation devices. Based on artificial intelligence, Google
assistant can hold two-way conversations, unlike Google Now, the company’s previous virtual
assistant. Google assistant has the most powerful technical advantage over the previous two
voice assistants, making it easier to search for information. Google assistant will analyze the
user's preferences built on the user's search frequency and provide users with more
valuable information[11].

Home Automation Devices: The key focus of the project is on how voice assistants
seamlessly integrate with a wide array of smart home devices. This integration is pivotal as it
allows users to control and automate various aspects of their homes. The Internet can be
described as the communication network that connects individuals to information while The
Internet of Things (IOT) is an interconnected system of distinctively address able physical
items with various degrees of processing, sensing, and actuation capabilities that share the
capability to interoperate and communicate through the Internet as their joint platform [12].
Thus, the main objective of the Internet of Things is to make it possible for objects to be
connected with other objects, individuals, at any time or anywhere using any network, path or
service.

4
The concept of development of a smart home system is not an isolated case it has existed since
the term "smart house" was first coined by the American Association of House builders in
1984. Implementation of these systems will not just increase the comfort level of modern
generation but also help elderly and physically disabled people [13]. All researchers are trying
to put some handheld device (e.g. mobile or some battery operated device) in hand on people to
increase level. In the real world (outside of research labs and the homes of the rich and
famous), home automation most commonly connects simple binary devices. This includes "on
and off" devices such as lights, power outlets and electronic locks, but also devices such as
security sensors which have only two states, open and close.

Recent technological advancement in voice assistant and home automation

I. Artificial Intelligence: voice-based AI technology breaks with previously established

forms of human-computer interaction because they are primarily based on the modality
of spoken language.[14]

II. Data Analytics: Modern virtual assistant systems can perform complex tasks and
communicate with people using natural language processing, machine learning, speech
recognition, and data analytics thanks to recent technological advancements. Virtual
assistants can now comprehend user queries, analyze them, and provide pertinent
information in real-time thanks to these cutting-edge technologies.[15]

III. Deep learning: Deep learning has enabled voice assistants to recognize speech with an
accuracy of over 95%. This makes voice assistants much more reliable than they were
in the past.

IV. Cloud computing: Cloud computing has made it possible to deploy voice assistants that
can handle millions of requests per second. This has made voice assistants more
scalable and accessible to users around the world.

V. Big data: Voice assistants are trained on massive datasets of speech and text data. This
data allows voice assistants to learn the patterns and nuances of human language, which
makes them more accurate and reliable.[16]

VI. Android Ecosystem: Android developers have provided with large number of IOT
supporting apps which makes implementation of home automation easier [17].

5
6. METHODOLOGY
6.1 PROBLEM DEFINITION

Existing solution pros:

i. Advancements in Technology: Integration of Advanced Natural Language Processing
(NLP) and machine learning techniques to enhance the voice assistant's understanding
of user intent, context, and language nuances.
ii. Device Integration and Apps: Voice assistance integration leads to positive aspects like
device integration and the development of third-party apps, expanding capabilities.
iii. Enhanced User Experience: Home automation enhances daily interactions, providing a
more convenient user experience. IOT positively influences daily routines, making
tasks easier and more accessible.
Existing Solution Cons:
i. Cost Challenges: Affordability is a significant hurdle in home automation due to costs
in manufacturing, development, installation, and ongoing support.
ii. Limited Personalization: Some solutions struggle to offer highly personalized
experiences, treating users with similar commands uniformly.
iii. Dependency on Cloud Services: A reliance on cloud services for voice processing
raises concerns about data security and offline functionality.
iv. Compatibility Challenges: Ensuring compatibility with a wide range of devices can be a
challenge in implementing voice assistance systems in home automation.
v. Lack of Standards: Absence of required standards in IOT leads to user unfamiliarity and
hinders commercial implementation.

Our Proposed Solution

After carefully analyzing different ideas for the project proposal, we came across with one of
the interesting and indeed the most emerging idea that would really cause a huge impact on the
people interact with the objects. It took us a while to go through various literature reviews and
methodologies to bring up with something that could be much more efficient and of course
sustainable.
Interactive AI Voice Assistant:
I. Develop an AI voice assistant system with advanced natural language processing (NLP)
for accurate and context-aware understanding of user commands.
II. Build an interactive AI voice assistant model capable of reminding users of their tasks
and send SMS notifications to their smartphones.
Home Automation Integration:
I. Integrate the voice assistant with home automation devices, allowing users to control
smart devices such as bulbs, curtains, and switches.
II. Build a smart box that converts conventional sockets in our homes into smart sockets.
Such sockets can be turned off or on through voice commands.

6
6.2 FUNCTIONAL BLOCK DIAGRAM

Fig. 1: Voice Assistant Project Layout

7
6.3 PLANNING AND APPROACH

In this section we will dive into the details of the functional blocks of the project. The Project
requires an efficient integration of hardware and software; we have carefully analyzed their
compatibility and have made sure they come under the budget.

6.3.1 PYTHON LANGUAGE

Python has become the go-to language for many machine learning and artificial intelligence
projects. Many popular machine learning libraries, such as TensorFlow and PyTorch, have
Python interfaces, making it convenient for developers to implement machine learning models
for tasks like speech recognition and natural language understanding [18].

The following libraries helps in voice assistant development:

1. Natural Language Processing (NLP) libraries: There are several NLP libraries available
for Python such as Speech Recognition, Natural Language Toolkit (NLTK), spaCy, and
Stanford CoreNLP. These libraries help with tasks such as sentiment analysis, named
entity recognition, and part-of-speech tagging.

2. Text to Speech (TTS) engines: Popular TTS engines include Google Text-to-Speech,
Amazon Polly, and Microsoft Speech Services. These engines can be used to generate
human-like speech output from text input [19].

3. Machine learning frameworks: Popular machine learning frameworks include

TensorFlow, PyTorch, and Scikit-learn. These frameworks can be used to train machine
learning models for speech recognition and NLP tasks [20]. Support Vector Classifier
(SVC) is a supervised learning algorithm that belongs to the family of Support Vector
Machines (SVMs). It's primarily used for classification tasks, where the goal is to
categorize input data into one of two or more classes.

Functional Requirements:

i. Wake word detection: The system detects a wake word such as "Hey, candy" to activate
the assistant.

ii. Voice recognition: The system accurately recognizes and interprets voice commands
from the user.

iii. Natural language processing: The system understands the user’s intent and responds
accordingly.

8
6.3.2 EMBEDDED C PROGRAM

Embedded C is used to program a wide range of microcontrollers and microprocessors.

Embedded C requires a smaller number of resources to execute in comparison with high-level
languages such as assembly programming language. Such programs can be written in Arduino
IDE and can be used for data exchange to and from cloud through API [21].

6.3.3 MICROPHONE

USB Plug Microphone offers compatibility with any plug-and-play enabled Raspberry Pi
Model B+, 2 model B, Raspberry Pi 3 as wells as it is also compatible with PC and Mac, Ideal
for the recording of sound. Advanced digital USB provides superior clarity with the simplicity
of a single USB plug [22].

6.3.4 SPEAKER
Standard and compact speaker with 3.5mm aux cable that can connect to audio output of
Raspberry Pi.

6.3.5 RASPBERRY PI 4 MODEL B - COMPUTER MODEL

Fig. 2: Raspberry Pi 4 Model B

9
REASONS TO CHOOSE RASPBERRY PI

1. Raspberry Pi has built-in Wi-Fi and Bluetooth connectivity, allowing it to connect to

the internet and communicate with other devices wirelessly. It makes it suitable for IoT
applications.

2. Raspberry Pi comes with 4 GB RAM which facilitates faster processing of programs.

3. It can run multiple operating systems, including Raspbian, Ubuntu, and others, and
supports various programming languages like Python, C++ etc.

4. Raspberry Pi is designed to be energy-efficient, with a power consumption of only a

few watts. It suits it for running on batteries.

5. Raspberry Pi is compact and portable, with a credit-card size form design. It makes
carrying around and integrating into various projects and applications accessible.

Fig. 3: Raspberry Pi Pins descriptions

10
6.3.6 UBIDOTS CLOUD PLATFORM

Ubidots is an IoT Platform empowering innovators and industries to prototype and scale IoT
projects to production. We can use the Ubidots platform to send data to the cloud from any
Internet-enabled device. Ubidots offers a REST API that allows you to read and write data to
the resources available: data sources, variables, values, events and insights. The API supports
both HTTP and HTTPS and an API Key is required.

Our data will be protected with two more replication, encrypted storage and optional TLS/SSL
data support. We can also customize permission groups to each module of the platform,
making sure the right information is shown to the right user.

6.3.7 ESP32 MICROCHIP

Fig. 4: ESP32 Microchip Pins layout

11
REASONS TO CHOOSE ESP32

1. Dual-Core Processing: The ESP32 features a dual-core processor, allowing it to handle

multiple tasks simultaneously, improving performance and responsiveness in complex
applications [23].

2. Wireless Connectivity: It has built-in Wi-Fi and Bluetooth capabilities, including BLE
(Bluetooth Low Energy), which makes it ideal for wireless communication and control
in IoT devices [24].

3. Low Power Consumption: The ESP32 is designed with energy efficiency in mind,
featuring various power-saving modes that make it suitable for battery-operated
devices.

4. Development Support: There is robust support for the ESP32 from the developer
community, including extensive documentation, libraries, and development tools.
Platforms like Arduino provide comprehensive support, making development easier and
faster.

5. Cost-Effective: The ESP32 is affordable, making it accessible for hobbyists, startups,

and educational purposes, without compromising on features or performance.

6. High Performance: With clock speeds up to 240 MHz and up to 520 KB of SRAM, the
ESP32 can handle demanding tasks, including real-time operations and multimedia
processing.

6.3.8 RELAY MODULE

A power relay module is an electrical switch that is operated by an electromagnet. The
electromagnet is activated by a separate low-power signal from a micro controller. When
activated, the electromagnet pulls to either open or close an electrical circuit.

6.3.9 AC to DC ADAPTER and MB102 MODULE

AC/DC adapters are commonly used external power supply units for electrical equipment
which cannot directly draw power from the mains network. They convert alternating current
(AC) into the required direct current (DC). The MB102 Breadboard Power Supply Module is a
3.3V/5V breadboard component that can be used for breadboard projects that require 3.3V, 5V,
or both power requirements.

12
6.4 DESIGN ISSUES

Designing a voice assistant project with home automation involves addressing various
technical and user experience challenges. Here are some key design issues to consider:

Speech Recognition Accuracy:

Challenge: Achieving high accuracy in speech recognition, especially in noisy environments or

with accents.
Solution: Use advanced speech recognition algorithms and consider noise cancellation
techniques. Regularly update the model to improve accuracy over time.

Parallel Processing of Tasks:

Challenge: Certain times the workflow of the system requires parallel processing for faster
output.
Solution: Use of threading and OS level computing with more powerful microprocessors like
ARM etc.

Context Awareness:

Challenge: Maintaining context over multiple interactions and understanding user intents
within a conversation.
Solution: Implement context-aware algorithms that remember previous user inputs and
maintain the context of the conversation. This can involve the use of context management
systems.

Compact Home Automation Devices:

Challenge: Ensuring compatibility with a wide range of home automation devices in a limited
space.
Solution: Use of PCB to minimize wiring and component space and use of IOT clouds that are
compatible with maximum IOT components.

Response Time:
Challenge: Providing quick and responsive interactions to enhance user experience and
performance.
Solution: Use of more powerful microprocessors for processing purpose and implement
caching and parallel processing where possible to speed up response times.

13
7. IMPLEMENTATION

7.1 VOICE ASSISTANT

We used Raspberry pi 4 (4 GB ram) as the computer to write and process our code and produce
the desired results. Raspberry pi is powered with a 5v power adapter. The speaker is connected
to its output audio jack and a USB microphone is connected to one of its USB ports. Users give
voice input using the microphone and the desired voice results are given out through speaker.
Raspberry Pi is a microcontroller that's compact, powerful, and affordable. It's also easy to
replace and only needs a keyboard and display to run. It occupies a very space. These qualities
make it a good tool and a suitable device for project tasks that needs smaller spaces and greater
processing.

Fig. 5: Raspberry PI with Connections

14
Fig. 6: Voice Assistant Device Fig. 7: Voice Assistant Inner Connections

Fig. 8: SMS notification from voice assistant

15
We used Python Programming Language was used to write the entire code for voice assistant.
VSCODE- a code IDE from Microsoft was used as a platform to write the python code. Driver
program is the main program of our assistant which runs as soon as the device is switched on.
It contains the main functionalities - detecting and recognizing voice, translating voice into
text, searching, and bringing information from the internet, translating text to audio, and
controlling led signals. The driver code also controls the other programs of the assistant and
invokes them as and when required. Speech recognition module of python along with
“recognize_google” library is used to recognize and convert speech into text. Python “Beautiful
Soup” library is used to filter out data from websites and “Wikipedia” library is used to bring
data specifically from Wikipedia. Finally, “Pico text-to-speech" library is used to convert text
to audio.

Fig. 9: Voice Assistant Code Block Diagram

Training intent and response program is written to make our voice assistant more interactive.
Machine Learning model - “Support vector Classifier” is used to train a model that would
understand the context of the questions asked and generate response accordingly. During
operation, user’s interaction is feed into the trained model which then generates responses by
understanding the context of the questions.

Reminder Program is written to add voice reminder system to the assistant. User’s message and
time of reminder is converted into text and time format respectively and stored in database.
SQLLITE database is used to store message description and time of reminder.

16
The program compares the local system time with the times stored in database. In case of a
match, the message corresponding to the matched time is fetched. Standard SQL queries are
used to store and fetch data to and from the database. A SMS is also sent to the user during
the time of reminder.

SMS Program is added to notify the user using a message, during the time of reminder. An
API key is used to connect the program to the “SMS CHEF” platform and send all the devices
and message details. Message is then sent to user’s mobile from the SMS CHEF.

Connect to Ubidots program defines two functions, send_to_ubidots_r1 and

send_to_ubidots_r2, to send data to the Ubidots API. Each function takes a value, constructs a
payload, and sends a POST request to the Ubidots endpoint with the respective payload (r1 or
r2). When user commands to “turn on” a socket then 1 is send and in case of “turn off” 0 is
send to Ubidots cloud.

7.2 HOME AUTOMATION

The home automation system aimed to revolutionize traditional home management by

integrating smart technologies. It employed a combination of hardware components including
Ubidots, AC to DC adaptor, power module, ESP-32 microcontroller, and relay module to
create an intelligent ecosystem for controlling household devices and appliances remotely.

UBIDOTS: - We created an account on the Ubidots platform and set up necessary data sources
and variables. Obtain the API keys or credentials required for communication between ESP32
and Ubidots.

AC to DC Converter: - AC to DC adaptor is connected to the power module to provide stable

power 5v supply.

Power Module: - The power module is connected to the ESP-32 for powering it with 3.5 volts
supply.

ESP-32: - We programmed the ESP32 microcontroller using Embedded C in Arduino IDE.

Configured the ESP32 to connect to the Wi-Fi network for internet access. Integrated the
ESP32 with the Ubidots platform by providing necessary API keys or credentials. And on the
other side ESP-32 is connected to the relay module through port no D2 and D4.

Relay Module: - We connected the Relay Module to ESP-32 through port INT-1 and INT-2.
Based on the received data or commands, control the relay module to turn on/off devices or
appliances connected to it. We have used relay module with power rating – 2500W.

By using all of this we have made a smart socket, which facilitates home appliances through
voice commands for enhanced convenience and comfort.

17
 Fig. 10: Home Automation Circuit Diagram

Fig. 11: Home Automation Device Fig. 12: Home Automation Inner Circuit

18
8. RESULT ANALYSIS

Table 1: Test Case 1

Test Case 1
Test Type Response generation
Test Name Response Evaluation
Test Description This test evaluates how well the voice assistant system provides
different responses to similar user queries
Expected Output Voice assistant will provide diverse, contextually relevant responses
tailored to user queries, enhancing engagement and overall user
satisfaction.
Example 1 “What do you mean by physics ?”
Actual Output Physics is the branch of science that deals with the structure of
matter and how the fundamental constituents of the universe
interact. It studies objects ranging from the very small using
quantum mechanics to the entire universe using general relativity.

Example 2 “Tell me about the plot of movie interstellar ?”

Actual Output Set in a dystopian future where humanity is embroiled in a
catastrophic blight and famine, the film follows a group of
astronauts who travel through a wormhole near Saturn in search of a
new home for humankind.
Example 3 “Who wrote the book titled pather panchali ?”
Actual Output Pather Panchali (Bengali: পেথর পাঁচালী, Pôther Pãchali; transl.
Song of the Little Road) is a 1929 novel written by Bibhutibhushan
Bandyopadhyay and was later adapted into a 1955 film of the same
name by Satyajit Ray.
Result PASS
Comment Working properly.

19
Table 2: Test Case 2

Test Case 2
Test Type Response generation
Test Name Verifying time
Test Description This test evaluates the voice assistant's ability to correctly respond
to queries related to time, such as asking for the current time of
particular country or city

Example “What is the current time at Bangalore ?”

Expected Output The voice assistant should accurately provide the current time upon
request
Actual Output 7:57 pm
Result PASS
Comment Working properly.

Table 3: Test Case 3

Test Case 3
Test Type Response generation
Test Name Verifying Weather
Test Description This test assesses the voice assistant's capability to accurately
respond to weather-related queries, including providing current
weather conditions, forecasts for specific locations, and responses to
weather-related inquiries.

Example “what is the current weather at london ?”

Expected Output The voice assistant should deliver accurate and up-to-date weather
information, including current conditions and forecasts for specified
locations, ensuring reliability and user satisfaction.

Actual Output 21°C Precipitation: 0% Humidity: 47% Wind: 10 km/h

Result PASS
Comment Working properly.

20
Table 4: Test Case 4

Test Case 4
Test Type Response generation
Test Name Search in Wikipedia
Test Description This test evaluates the voice assistant's ability to effectively search
and retrieve information from Wikipedia in response to user
queries.
Expected Output The voice assistant should accurately retrieve relevant information
from Wikipedia based on user queries, providing concise and
informative responses that address the user's information needs.

Example 1 “Embedded System in Wikipedia”

Actual Output An embedded system is a computer system—a combination of a
computer processor, computer memory, and input/output peripheral
devices—that has a dedicated function within a larger mechanical
or electronic system. It is embedded as part of a complete device
often including electrical or electronic hardware and mechanical
parts.
Because an embedded system typically controls physical operations
of the machine that it is embedded within, it often has real-time
computing constraints. Embedded systems control many devices in
common use.

Example 2 “Who is Narendra Modi ?”

Actual Outut Narendra Damodardas Modi (Gujarati: [ˈnəɾendɾə dɑmodəɾˈdɑs
ˈmodiː]; born 17 September 1950) is an Indian politician who has
served as the 14th prime minister of India since May 2014. Modi
was the chief minister of Gujarat from 2001 to 2014 and is the
Member of Parliament (MP) for Varanasi.

Result PASS
Comment Working properly.

21
Table 5: Test Case 5

Test Case 5
Test Type Home Automation
Test Name Turn on the Sockets of smart plug
Test Description This functional test assesses the voice assistant's proficiency
in home automation by verifying its capability to activate
socket 1 or socket 2 upon receiving the command.
Expected Output Upon the command "Turn on socket 1," or “Turn on socket 2”
or “Turn on both socket” the voice assistant should execute
the home automation task by sending a signal to activate
socket 1 or socket 2 or both sockets, thereby powering the
connected device.
Example "Turn on socket 1," or “Turn on socket 2” or “Turn on both
socket”
Actual Output “Turning on the socket 1” or “Turning on the socket 2” or
“Turning on both socket”.
[voice assistant system will send the signal to Ubidots cloud
to activate the corresponding socket or device. ]
Result PASS
Comment Working properly.

Table 6: Test Case 6

Test Case 6
Test Type Home Automation
Test Name Turn off the Sockets of smart plug
Test Description This functional test assesses the voice assistant's proficiency
in home automation by verifying its capability to deactivate
socket 1 or socket 2 upon receiving the command.
Expected Output Upon the command "Turn off socket 1," or “Turn off socket
2” or “Turn off both socket” the voice assistant should
execute the home automation task by sending a signal to
deactivate socket 1 or socket 2 or both sockets.
Example "Turn off socket 1," or “Turn off socket 2” or “Turn off both
socket”
Actual Output “Turning off the socket 1” or “Turning off the socket 2” or
“Turning off both socket”.
[voice assistant system will send the signal to Ubidots cloud
to deactivate the corresponding socket or device. ]
Result PASS
Comment Working properly.

22
Table 7: Test Case 7

Test Case 7
Test Type Functionality Test
Test Name Reminder Functionality Validation [Setting a reminder]
Test Description This test evaluates the reminder functionality of the voice assistant
system by assessing its ability to successfully set a reminder as per
user instructions.
Expected Output Upon receiving the command to set a reminder, the voice assistant
should confirm the reminder details with the user and acknowledge
successful scheduling of the reminder, ensuring accurate recording
and timely notification at the specified time.

Example User: “add reminder”

Candy: “add description”
User: “to drink water”
Candy: “add time”
User: “10:30 pm”
Candy: “reminder stored successfully”

Actual Output Upon the user's request to add a reminder, the voice assistant
prompts for a description. Once the user provides the description
"To drink water" , the system prompts for the time. After the user
specifies "10:30 PM" , the voice assistant confirms the successful
storage of the reminder.

Result PASS
Comment Working properly.

23
Table 8: Test Case 8

Test Case 8
Test Type Functionality Test
Test Name Reminder Functionality Validation [Getting a reminder]
Test Description This test validates the reminder functionality of the voice assistant
system by ensuring that reminders are delivered audibly through
voice output without relying on visual displays ensuring timely and
accurate delivery of SMS.

Expected Output At the scheduled time, the voice assistant should audibly announce
the reminder description to the user without requiring any visual
interaction. Simultaneously, an SMS notification containing the
reminder details should be sent to the user's registered mobile phone
number.

Example User sets a reminder named “to drink water” at 10:30 PM.
Actual Output At 10:30 PM, the voice assistant audibly announces: "Reminder: To
drink water."
Additionally, an SMS notification containing the reminder details
"Reminder: To drink water" is sent to the user's registered mobile
phone number.

Result PASS
Comment Working properly.

24
Table 9: Test Case 9

Test Case 9
Test Type Functionality Test
Test Name Reminder Functionality Validation [Delete a reminder]
Test Description This test validates the voice assistant's ability to delete a reminder
based on the specified time provided by the user.
Expected Output Upon receiving the command to delete a reminder, the voice
assistant should confirm the deletion and remove the reminder
from the system.

Example User: "Delete a reminder."

Candy: “Give the time of the reminder to be deleted”
User: “10:30 PM”
Candy: “Successfully Deleted Reminder”

Actual Output The user requests to delete a reminder. The voice assistant prompts
for the time of the reminder to be deleted. Upon receiving the
specified time "10:30 PM" from the user, the voice assistant
confirms the successful deletion of the reminder named “to drink
water” set a 10:30 PM.

Result PASS
Comment Working properly.

25
Table 10: Test Case 10

Test Case 10
Test Type Functionality Test
Test Name System Reboot Functionality Test
Test Description This test evaluates the voice assistant's ability to execute a system
reboot command upon user request.
Expected Output Upon receiving the command to reboot the system, the voice
assistant should confirm the action and initiate the reboot process.

Example “Reboot the System”

Actual Output “Rebooting the system”
[voice assistant confirming the system reboot and then initiating
the reboot process, as described in the example.]
Result PASS
Comment Working properly.

Table 11: Test Case 11

Test Case 11
Test Type Functionality Test
Test Name System Shutdown Functionality Test
Test Description This test evaluates the voice assistant's ability to execute a
complete system shutdown command upon user request.
Expected Output Upon receiving the command to Shut down the system, the voice
assistant should confirm the action and initiate the process.

Example “Shutdown the System”

Actual Output “Shutting down the system now”
[voice assistant confirming the system shutdown and then initiating
the shutdown process, as described in the example.]

Result PASS
Comment Working properly.

26
9. DISCUSSION

Voice assistants have become an integral part of modern technology, seamlessly integrating
into various aspects of daily life. Our project focuses on developing a versatile voice assistant
that not only answers questions but also sets reminders, sends SMS, and interfaces with a home
automation system. This multi-functional approach aims to enhance user convenience and
control, offering a holistic solution for managing daily tasks and home environments through
simple voice commands.

Key Features
1. Answering Questions
One of the primary functions of our voice assistant is to provide accurate and timely responses
to user queries. By leveraging advanced natural language processing (NLP) and access to vast
information databases, the assistant can answer a wide range of questions. This feature is
designed to be as comprehensive as possible, covering topics from general knowledge and
weather updates to specific user inquiries.

2. Setting Reminders
The reminder functionality of the voice assistant is crucial for managing personal schedules.
Users can set reminders for various tasks, appointments, and events using voice commands.
The assistant then ensures that these reminders are delivered at the appropriate times, helping
users stay organized and on top of their responsibilities.

3. Sending SMS
Incorporating the ability to send SMS messages enhances the assistant's communication
capabilities. Users get their reminders through SMS messages incase they are not near the
voice assistant.

4. Home Automation Integration

A standout feature of our voice assistant is its integration with a home automation system.
Through voice commands, users can control various smart devices within their homes, such as
turning on or off smart sockets. This level of control extends to lights, thermostats, and other
connected devices, offering users a hands-free and convenient way to manage their home
environment.

27
Challenges and Solutions

1. Accuracy and Reliability

One of the significant challenges is ensuring the accuracy and reliability of the voice assistant's
responses and actions. This can be addressed through continuous learning and updating of the
NLP models, as well as incorporating user feedback to refine and improve the system.

2. Privacy and Security

Given the sensitive nature of voice commands and personal data, privacy and security are top
priorities. Implementing robust encryption methods, secure data storage practices, and strict
access controls can help protect user information and prevent unauthorized access.

3. Integration Complexity
Integrating with various home automation systems and communication networks can be
complex. Standardizing protocols and developing adaptable interfaces can streamline this
process, allowing for easier expansion and compatibility with a wider range of devices and
services.

Future Enhancements
Looking ahead, there are several potential enhancements for our voice assistant. These include
expanding language support, incorporating advanced AI features like predictive analytics, and
enhancing the personalization of responses based on user behavior and preferences.
Additionally, exploring integration with more third-party applications and services can further
broaden the assistant's utility and appeal.

28
10. RESOURCE UTILIZATION

Table 12: List of Components and their estimated cost

Name of The Component Numbers Needed Estimated Cost

(Rs)
Raspberry Pi 4 Model B 1 4500
Microphone 1 300
Speaker 1 180
ESP32 Microchip 1 280
AC to DC Adapter 1 150
MB102 Bread Board Module 1 58
Bulb 1 30
Relay Module 2 35
LED, Wires, Electrical Box 1 Set 175

TOTAL: 5708 INR

Table 13: Similar voice Assistant products in the market

Similar Products Cost (Rs) Description

Google Home 8999 A smart speaker with Google
Assistant for voice control.
Amazon Echo 7499 A smart speaker with Alexa voice
assistant integration
Xiaomi Mi Smart Home Hub 5000 A central hub for controlling
various Xiaomi smart devices.
TP-Link Kasa Smart Wi-Fi Plug 4,818 A smart plug that allows remote
Mini control of appliances.
Amazon Smart Plug (works with 2000 A smart plug designed to work
Alexa) seamlessly with Alexa.

29
11. CONCLUSION AND FUTURE SCOPE

Our project represents a fundamental yet valuable contribution to the realm of voice
assistant technology. While it may not boast advanced features found in high-end systems,
its simplicity and accessibility make it an ideal solution for individuals seeking basic
voice-controlled functionalities. By utilizing affordable hardware such as the Raspberry Pi
and implementing straightforward Python programming, we've created a user-friendly
voice assistant capable of performing essential tasks like setting reminders and providing
information sourced from the web. Moving forward, there is ample opportunity to enhance
our project's capabilities through gradual improvements and additions. By maintaining a
focus on usability and simplicity, we can ensure that our voice assistant remains accessible
to a wide range of users while gradually incorporating more advanced features as needed.

The future scope of the voice assistant project encompasses several avenues for
advancement. Firstly, voice recognition can be integrated for heightened security,
leveraging voice biometrics for user authentication, and ensuring secure access to sensitive
information. Natural Language Understanding (NLU) capabilities can be enhanced
through machine learning techniques which allows for more accurate interpretation of user
queries and personalized responses. Multi-language support and real-time translation
capabilities can be used to facilitate seamless communication across diverse language
barriers. Facial recognition and emotion recognition technologies can be introduced, which
enhances user identification and interaction, adapting responses based on facial
expressions.

30
12. REFERENCES

[1] Jain, S. R., & Jason, P. F. (2023). Personal Desktop Voice Assistant - Research.
International Journal of Advanced Research in Computer and Communication
Engineering , 1-10.

[2] Reddy, K. V., Lahari, S., Naveen, A., & Chandra, G. S. (2020). PERSONAL VOICE
ASSISTANT. journal of Emerging Technologies and Innovative Research , 1-6.

[3] Sikarwar, S. S. (2022). AI BASED VOICE ASSISTANT. International Research

Journal of Modernization in Engineering Technology and Science , 1-3.

[4] Subhash, S., Srivatsa, P. N., Siddesh, S., Ullas, A., & Santhosh, B. (2020, July).
Artificial intelligence-based voice assistant. In 2020 Fourth world conference on
smart trends in systems, security and sustainability (WorldS4) (pp. 593-596). IEEE.

[5] Nasirian, F., Ahmadian, M., & Lee, O. K. D. (2017). AI-based voice assistant
systems: Evaluating from the interaction and trust perspectives.

[6] Sahu, A., Jha, A., Bhargava, R., Priya, P., & Kumari, R. (2022). Voice Assistant
Using Artificial Intelligence. SSRN , 1-4.

[7] Huang, Y. (2023). Research on the Development of Voice Assistants in the Era of
Artificial Intelligence. ResearchGate , 1-3.

[8] Alexa developer documentation [Online] Available: https://developer.amazon.com/en-

US/docs/alexa/documentation-home.html [Accessed 15 April 2024].

[9] Siri [Online] Available : https://developer.apple.com/design/human-interface-

guidelines/siri/ [Accessed: 15 April 2024].

[10] Cortana help & learning [Online] Available: https://support.microsoft.com/en-

us/cortana [Accessed:15 April 2024].

[11] Google Assistant for Developers [Online] Available : https://developers.google .

com/assistant [Accessed: 15 April 2024]

[12] Rahman, A & Hussein, H. (2019). Internet of Things (IOT) : Research Challenges
andFuture Applications.(IJACSA) International Journal of Advanced Computer
Science and Applications. , 2-6.

31
[13] Bhardwaj, P., Manchanda, P., Chahal, P., Chaudhary, P., & Singh, R. (2020). A
REVIEW PAPER ON SMART HOME AUTOMATION. International Journal of
Scientific Research and Management Studies (IJSRMS) , 279-283.

[14] Carolus, A., Augustin, Y., Markus, A., & Wienrich, C. (2022). Digital
interactionliteracy model – Conceptualizing competencies for literate interactions with
voice- based AI systems. ELSEVIER , 1-3.

[15] Fahad, M., Akbar, A., Fathima, S., & Bari, D. M. (2023). Windows Based AI-Voice
Assistant System using GTTS. Mathematical Statistician and Engineering
Applications , 1572 - 1580.

[16] Kumar, S., Tiwari, P., & Zymbler, M. (2019). Internet of Things is a revolutionary
approach for future technology enhancement: a review. Journal of Big Data , 2-10.

[17] Kumari, J., & Neelam, P. (2021). IoT Based Smart Home Automation System.
JETIR,1-5.

[18] Manojkumar, M. P., Patil, A., Shinde, S., Patra, S., & Patil, S. (2023). AI-Based
Virtual Assistant Using Python: A Systematic Review. International Journal for
Research in Applied Science & Engineering Technology (IJRASET) , 2-6.

[19] N, A. K., & B, V. (2023). AI BASED VIRTUAL ASSISTANT. International

Research Journal of Modernization in Engineering Technology and Science , 2-3.

[20] Pavitra, D. R., Ganesha, M., Pavithran, P., & Rajamurugan, G. (2023). A Review on
Intelligent Voice Assistant with Multilingual Support using Artificial Intelligence.
Journal of Emerging Technologies and Innovative Research , 1-6.

[21] Yadlapally, D. K., Vasireddy, B., Marimganti, M., Karthikeyan, C.,chowdary, T., &
T.Vignesh. (2023). A Review on the Potential of AI Voice Assistants for Personalized
and Adaptive Learning in Education. ICCMC , 1-6.

[22] Raspberry Pi [Online]. Available :https://en.wikipedia.org/wiki/Raspberry_Pi

[Accessed: 25 November 2023].

[23] ESP32 Home Automation [Online] Available: https://www.hackster.io

/mejdchennoufi06/esp32-home-automation-c41511 [Accessed: 26 November 2023].

[24] ESP32 [Online]. Available: https://en.wikipedia.org/wiki/ESP32 [Accessed: 26

November 2023]

32
13. CODE SNIPPETS
We have provided the driver program of voice assistant and embedded c program of home
automation. Please visit our GitHub repository for the entire programs of our project.

Link: https://github.com/KashyapGhosh02/voice-assistant-system-candy.git

Voice Assistant Driver Program

#import necessary modules

import os
import subprocess
# from datetime import datetime
import speech_recognition as sr
import wikipedia
import bs4
from bs4 import BeautifulSoup
import requests
import datetime
import time
import random

import RPi.GPIO as GPIO

import time
import warnings
warnings.filterwarnings("ignore")
#------------------------------------------------

# Set up GPIO using BCM numbering

GPIO.setmode(GPIO.BCM)

# Set GPIO NUMBER as output

green_led_pin = 26
yellow_led_pin=20
red_led_pin=21
GPIO.setup(green_led_pin, GPIO.OUT)
GPIO.setup(yellow_led_pin, GPIO.OUT)
GPIO.setup(red_led_pin, GPIO.OUT)

GPIO.output(green_led_pin, GPIO.LOW)
GPIO.output(yellow_led_pin, GPIO.LOW)

33
GPIO.output(red_led_pin, GPIO.HIGH)
#------------------------------
#import custom-made modules
import voice_greetings
import reminder_property
from wolframalpha_rename import wolframalpha_result
import ubidots_separatecode
#import prediction_func
#------------------------------

def say(text, language='en-US'):

# Define the command to generate speech using pico2wave
GPIO.output(yellow_led_pin, GPIO.HIGH)
pico_command = f'pico2wave -l {language} -w speech.wav "{text}"'

# Execute the pico2wave command

os.system(pico_command)

# Play the generated speech using aplay

os.system('aplay speech.wav')

# Clean up the temporary speech file

os.remove('speech.wav')
GPIO.output(yellow_led_pin, GPIO.LOW)
#----------------speech-recognition--------------
def takeCommand():
r = sr.Recognizer()
with sr.Microphone() as source:
print("Listening...")
GPIO.output(green_led_pin, GPIO.HIGH)
r.pause_threshold = 1
r.energy_threshold=3000
try:
audio = r.listen(source, timeout=3)
GPIO.output(green_led_pin, GPIO.LOW)
print("Recognizing...")
query = r.recognize_google(audio, language='en-in')
print(f"User said: {query}\n")
return query.lower()
except sr.UnknownValueError:
print("Speech Recognition could not understand audio.")
except sr.RequestError as e:

34
 print(f"Could not request results from Google Speech Recognition
service; {e}")
except Exception as e:
print(f"An error occurred: {e}")
return "none" # End of takeCommand() function

def wishMe(query):
#list_int_resp=prediction_func.predict_intent_with_nltk(text=query)
list_int_resp=voice_greetings.predict_intent(text=query)
return list_int_resp

def google_search(search_text):
result = ''
search_data = search_text
if "who is" in search_data or "who are" in search_data:
search_data = search_data.split(" ")[2:]
search_data = " ".join(search_data)
try:
result = wikipedia.summary(search_data, sentences=2)
return result
except Exception as e:
pass
else:
url = "https://www.google.co.in/search?q=" + search_data
try:
search_result = requests.get(url).text
soup = BeautifulSoup(search_result, 'html.parser')
result_div = soup.find_all('div', class_='BNeawe s3v9rd AP7Wnd')
result = result_div[0].text
return result

except Exception as e:
pass
return("sorry could not find any information")

ask_response = [
"Is there anything else on your mind?",
"Do you need assistance with anything else?",
35
 "Would you like to discuss anything further?",
"Are there any other topics you'd like to explore?",
"Feel free to ask me anything else.",
"Is there something specific you'd like to know more about?",
]
random_response = random.choice(ask_response)

def start():
try:
while True:
curr_time = datetime.datetime.now().strftime("%H:%M:00")
print(curr_time)
reminder_property.GetReminder(curr_time)
query = takeCommand()
#if query is None:
# continue # Continue listening if no query is obtained
if "candy" in query.lower():
list_intent_response = wishMe(query)
say(list_intent_response[1])
while True:
query = takeCommand()
list_intent_response = wishMe(query)

if list_intent_response[0] == "goodbye" or
list_intent_response[0]=="gratitude" :
say(list_intent_response[1])
break
elif "no" in query or query.lower() =="none" and
list_intent_response[0]=="nothing" :
say(list_intent_response[1])
break
else:
get_result(query, list_intent_response)
say(random_response)
except Exception as e:
GPIO.cleanup()
print(e)

def get_result(query,list_intent_response):

if query.lower()=="none" and list_intent_response[0]=="nothing":

say("No instructions found")
36
 elif "add" in query or "set" in query and "reminder" in query:
say("Add description")
desc_1=takeCommand()
if desc_1=="none":
say("Did not get any Description")
return
say("Add time")
time_1=takeCommand()
reminder_property.SetReminder(desc_1,time_1)

elif "delete" in query and "reminder" in query:

say("Give the time of the reminder to be deleted")
time_1=takeCommand()
reminder_property.DeleteReminder(time_1)

elif "time now" in query or "time in " in query or "time at " in query
or "date in "in query or "today's date " in query or "weather in " in
query or "weather at " in query or "temperature in " in query or
"temperature at " in query: # for time date and weather related info
x = datetime.datetime.now()
if "date" in query: #Only local Date
say("Date is "+x.strftime("%A ")+x.strftime("%d
")+x.strftime("%B ")+x.strftime("%Y ")) # Date
if "time" in query: #time can be searched for local and
international zones
url = "https://google.com/search?q=" + query
request_result = requests.get( url )
soup = bs4.BeautifulSoup( request_result.text , "html.parser" )
time = soup.find( "div" , class_='BNeawe' ).text
say("time is "+time )

if "weather" or "temperature" in query:

url = "https://www.google.com/search?q="+query
html = requests.get(url).content
soup = BeautifulSoup(html, 'html.parser')
temp = soup.find('div', attrs={'class': 'BNeawe iBp4i
AP7Wnd'}).text
str1 = soup.find('div', attrs={'class': 'BNeawe tAd8D
AP7Wnd'}).text

37
 data = str1.split('\n')
sky = data[1]
say("Temperature is "+ temp)
say("Sky Description: "+ sky)

elif 'wikipedia' in query: # Wikipedia search

say('Searching Wikipedia...')
query = query.replace("wikipedia", "")
results = wikipedia.summary(query, sentences=4)
say("According to Wikipedia")
say(results)

elif list_intent_response[0]=="math" and "math" in query:

result=wolframalpha_result(question=query)
say(result)
return
elif "turn on" in query:
if "socket one" in query or "socket 1" in query:
say("turning on the socket 1")
ubidots_separatecode.send_to_ubidots_r1(1)
if "socket two" in query or "socket 2" in query:
say("turning on the socket 2")
ubidots_separatecode.send_to_ubidots_r2(1)
if "both" in query:
say("turning on the both the socket")
ubidots_separatecode.send_to_ubidots_r1(1)
ubidots_separatecode.send_to_ubidots_r2(1)
elif "turn off " in query:
if "socket one" in query or "socket 1" in query:
say("turning off the socket one")
ubidots_separatecode.send_to_ubidots_r1(0)
if "socket two" in query or "socket 2" in query:
say("turning off the socket two")
ubidots_separatecode.send_to_ubidots_r2(0)
if "both" in query:
say("turning off the both the socket")
ubidots_separatecode.send_to_ubidots_r1(0)
ubidots_separatecode.send_to_ubidots_r2(0)
38
 elif "reboot" in query:
say("rebooting the system")
os.system("sudo reboot")
elif "shutdown" in query:
say("shutting down the system")
os.system("sudo shutdown now")
else:
results = google_search(query) #google serach
print(results)
say(results)

if __name__ == "__main__":
try:
print("starting the code ")
GPIO.output(green_led_pin, GPIO.LOW)
GPIO.output(red_led_pin, GPIO.LOW)
start()
except Exception as e:
GPIO.cleanup()
print(e)

39
Home Automation Program

#include <WiFi.h>
#include <PubSubClient.h>

/**************
* Define Constants
**************/
#define TOKEN "BBUS-hLmxzEtc7efjDFE0AxufTAJVXyISXb" // Your Ubidots TOKEN
#define WIFINAME "DEMON" // Your SSID
#define WIFIPASS "a1b2c34d5e" // Your Wifi Pass
#define MQTT_CLIENT_NAME "RandomName"

#define VARIABLE_LABEL1 "r1"

#define VARIABLE_LABEL2 "r2"
#define DEVICE_LABEL "iot"
#define RELAY_PIN1 2
#define RELAY_PIN2 4
char mqttBroker[] = "things.ubidots.com";
char payload[100];
char topic[150];
char topicSubscribe1[100];
char topicSubscribe2[100];

WiFiClient ubidots;
PubSubClient client(ubidots);

void callback(char *topic, byte *payload, unsigned int length)

{
char p[length + 1];
memcpy(p, payload, length);
p[length] = '\0'; // Null-terminate the string
String message(p);

//code to extrct r1 or r2
String str = String(topic);
int startIndex = str.indexOf("/r");
int endIndex = str.indexOf("/", startIndex + 1);
String value = str.substring(startIndex + 1, endIndex);// r1 or r2
//------------------------------------
Serial.print("Message arrived [");
Serial.print(topic);
40
 Serial.print("] ");
for (int i = 0; i < length; i++) {
Serial.print((char)payload[i]);
}
Serial.println();
//---------------------------------------

if(value=="r1")
{
if (message == "1.0")
{
digitalWrite(RELAY_PIN1, LOW);
Serial.println("turned on");
}
else
{
digitalWrite(RELAY_PIN1, HIGH);
Serial.println("turned off");
}
}

if(value=="r2")
{
if (message == "1.0")
{
digitalWrite(RELAY_PIN2, LOW);
Serial.println("turned on");
}
else
{
digitalWrite(RELAY_PIN2, HIGH);
Serial.println("turned off");
}
}
}

void reconnect()
{
// Loop until we are connected
while (!client.connected())
{
Serial.println("Attempting MQTT connection ...");
if (client.connect(MQTT_CLIENT_NAME, TOKEN, ""))
41
 {
Serial.println("Connected");
client.subscribe(topicSubscribe1);
client.subscribe(topicSubscribe2);
}
else
{
Serial.print("Failed, rc=");
Serial.print(client.state());
Serial.println(" try again in 2 seconds");
delay(2000);
}
}
}

void setup()
{
Serial.begin(115200);
WiFi.begin(WIFINAME, WIFIPASS);
pinMode(RELAY_PIN1, OUTPUT);
pinMode(RELAY_PIN2,OUTPUT);
Serial.print("Wait for WiFi ...");
while (WiFi.status() != WL_CONNECTED)
{
Serial.print(".");
delay(500);
}

Serial.println("");
Serial.println("WiFi connected");
Serial.println("IP address:");
Serial.println(WiFi.localIP());
client.setServer(mqttBroker, 1883);
client.setCallback(callback);
sprintf(topicSubscribe1, "/v1.6/devices/%s/%s/lv", DEVICE_LABEL,
VARIABLE_LABEL1);
client.subscribe(topicSubscribe1);
sprintf(topicSubscribe2, "/v1.6/devices/%s/%s/lv", DEVICE_LABEL,
VARIABLE_LABEL2);
client.subscribe(topicSubscribe2);
}

void loop()
42
{
if (!client.connected())
{
client.subscribe(topicSubscribe1);
client.subscribe(topicSubscribe2);
reconnect();
}

client.loop();
delay(1000);
}

43