Multilingual Home Automation using

Google Assistant and Raspberry Pi

Abstract
One of the problems with a voice-controlled smart home automation system is the language barrier.
Most of the existing home automation systems support only a few languages, including English.

Hence, we aim to make an Internet of Things (IoT) home automation system that allows you to
control your home devices with voice commands in your preferred language like Hindi, Bengali,
Marathi, and so on.

It uses Google Assistant to control smart devices at home and also supports all the languages
supported by Google Assistant. Also an Android app for a smart touch control system will be
developed for remote touch control of the devices.
 Introduction
The Internet of things (IoT) describes physical objects (or groups of such objects) that are
embedded with sensors, processing ability, software, and other technologies that connect and
exchange data with other devices and systems over the Internet or other communications networks.
[1][2][3][4]

The field has evolved due to the convergence of multiple technologies, including ubiquitous
computing, commodity sensors, increasingly powerful embedded systems, and machine learning.[1]
Traditional fields of embedded systems, wireless sensor networks, control systems, automation
(including home and building automation), independently and collectively enable the Internet of
things. In the consumer market, IoT technology is most synonymous with products pertaining to the
concept of the "smart home", including devices and appliances (such as lighting fixtures,
thermostats, home security systems and cameras, and other home appliances) that support one or
more common ecosystems, and can be controlled via devices associated with that ecosystem, such
as smartphones and smart speakers. The IoT can also be used in healthcare systems.[5]

There are a number of concerns about the risks in the growth of IoT technologies and products,
especially in the areas of privacy and security, and consequently, industry and governmental moves
to address these concerns have begun, including the development of international and local
standards, guidelines, and regulatory frameworks.[6]

History

The main concept of a network of smart devices was discussed as early as 1982, with a modified
Coca-Cola vending machine at Carnegie Mellon University becoming the first ARPANET-
connected appliance,[7] able to report its inventory and whether newly loaded drinks were cold or
not.[8] Mark Weiser's 1991 paper on ubiquitous computing, "The Computer of the 21st Century", as
well as academic venues such as UbiComp and PerCom produced the contemporary vision of the
IOT.[9][10] In 1994, Reza Raji described the concept in IEEE Spectrum as "[moving] small packets
of data to a large set of nodes, so as to integrate and automate everything from home appliances to
entire factories".[11] Between 1993 and 1997, several companies proposed solutions like
Microsoft's at Work or Novell's NEST. The field gained momentum when Bill Joy envisioned
device-to-device communication as a part of his "Six Webs" framework, presented at the World
Economic Forum at Davos in 1999.[12]

The concept of the "Internet of things" and the term itself, first appeared in a speech by Peter T.
Lewis, to the Congressional Black Caucus Foundation 15th Annual Legislative Weekend in
Washington, D.C, published in September 1985.[13] According to Lewis, "The Internet of Things,
or IoT, is the integration of people, processes and technology with connectable devices and sensors
to enable remote monitoring, status, manipulation and evaluation of trends of such devices."

The term "Internet of things" was coined independently by Kevin Ashton of Procter & Gamble,
later MIT's Auto-ID Center, in 1999,[14] though he prefers the phrase "Internet for things".[15] At
that point, he viewed radio-frequency identification (RFID) as essential to the Internet of things,[16]
which would allow computers to manage all individual things.[17][18][19] The main theme of the
Internet of things is to embed short-range mobile transceivers in various gadgets and daily
necessities to enable new forms of communication between people and things, and between things
themselves.[20]
Defining the Internet of things as "simply the point in time when more 'things or objects' were
connected to the Internet than people", Cisco Systems estimated that the IoT was "born" between
2008 and 2009, with the things/people ratio growing from 0.08 in 2003 to 1.84 in 2010.[21]

Applications

The extensive set of applications for IoT devices[22] is often divided into consumer, commercial,
industrial, and infrastructure spaces.[23][24]

Consumer applications

A growing portion of IoT devices are created for consumer use, including connected vehicles, home
automation, wearable technology, connected health, and appliances with remote monitoring
capabilities.[25]

Smart home

IoT devices are a part of the larger concept of home automation, which can include lighting, heating
and air conditioning, media and security systems and camera systems.[26][27] Long-term benefits
could include energy savings by automatically ensuring lights and electronics are turned off or by
making the residents in the home aware of usage.[28]

A smart home or automated home could be based on a platform or hubs that control smart devices
and appliances.[29] For instance, using Apple's HomeKit, manufacturers can have their home
products and accessories controlled by an application in iOS devices such as the iPhone and the
Apple Watch.[30][31] This could be a dedicated app or iOS native applications such as Siri.[32]
This can be demonstrated in the case of Lenovo's Smart Home Essentials, which is a line of smart
home devices that are controlled through Apple's Home app or Siri without the need for a Wi-Fi
bridge.[32] There are also dedicated smart home hubs that are offered as standalone platforms to
connect different smart home products and these include the Amazon Echo, Google Home, Apple's
HomePod, and Samsung's SmartThings Hub.[33] In addition to the commercial systems, there are
many non-proprietary, open source ecosystems; including Home Assistant, OpenHAB and
Domoticz.[34][35]
 Literature Survey
I. Multi Language Voice Control Home Automation System – This paper describes remote
control of household electrical outlets using a mobile phone as a transmitter which is further
connected via bluetooth link to the receiver featuring a bluetooth receiver module,
microcontroller and relay board. The system targets disabled people who cant easily reach
the switch boards for controlling the devices. The system features multilingual voice control
by developing an android app for same.

II. Home automation system using internet of things – This paper focusses on achieving remote
control of household device using the IoT (Internet of Things) concept. The system features
complete home automation including – device control, surveillance, and WSN (wireless
sensor networks) for intrusion detection, monitoring etc. The system specifies use of MQTT
protocol for its operation.

III. Smart Home Automation System using BLE - This paper presents technology which deals
with low cost and efficient for example Bluetooth technology for controlling yeelight. This
is achieved by incorporating java code into artik board. Technology has been developed at a
high pace. Furthermore, it will never stop. Web of Things is getting to be prevalent
nowadays because of putting tremendous effects on life. To structure an item utilizing
current technology that ought to be gainful to the lives of others is an extraordinary test now
a world. This paper exhibits cost effective technology for example ble, an artik board,
yeelight, and test system simulator. The test system is utilized to control the shade of the
yeelight in the scope of ble. The wireless communication has been established between the
light and the board. This entire procedure will be incorporated according to popular
request/response in accordance with the JSON format. The advantages of the light are it
possesses low cost with less utilization of power.

IV. Literature Review on Home Automation System - This paper deals with discussion of
different intelligent home automation systems and technologies from a various features
standpoint. The effort targeted on the home automation concept of where the controlling and
monitoring operations are expediting through smart devices. Wide-ranging home automation
systems and technologies considered in review with central controller based (Arduino or
Raspberry pi), cloud-based, Bluetooth-based, SMS based, ZigBee based, mobile-based, RF
Module based, web based and the Internet with performance.
From above survey we analysed that most of the systems were designed utilizing
BT/BLE/XBEE/WiFi systems and while some of the systems do provide voice control but they lack
in providing multilingual voice control of the connected home appliances.

Further from the survey analysis we have concluded to provide below listed functionality to the
home automation system with administration features to add more device control capabilities.

1. 2 x Electrical appliance control via smartphone voice commands or touch controls.

2. Integration with Google Assistant for multilingual support including – Hindi, Kannada,
Marathi, Tamil, Malayalam, Gujarati, Bengali, etc.

3. Local WiFi operation of the complete system.

4. Additional device management and configuration via dedicated admin control panel.

5. Optional long distance remote control via dedicated VPN connection.

Fig. 1. Complete system block representation.

As seen from the block representation the voice commands input by the user is processed by Google
Assistant and further if a valid command is found then the same is forwarded to the Raspberry Pi
Zero W using a WiFi network or hotspot hosted by the android device itself. The same is processed
by the Raspberry Pi running the PHP script and switches ON or OFF the connected load via its
GPIO port using python scripts, further connected to a 2 channel relay board and the 230V AC load
connected to it.

We will be designing the system with 2 loads control for cost control factors and the same can be
further expanded to more number of loads, with minor hardware and software changes.

An administration interface is provided to change or manage device control settings such as device
name to be controlled and adding or removing devices to be controlled.
Software Design
The system software design is split in two different parts namely – System control software and
Android control app. The first one will be responsible for achieving complete voice and touch
control of the connected devices via Google Assistant and Android App, and includes a web server
running PHP server side script on the Raspberry Pi Zero W which also features a device
administration interface and local database in MySQL for storing device control parameters. The
later one is an android app featuring remote touch control of the connected devices.

The Google Assistant features routines for routing voice commands to any device or application and
is configured to access PHP scripts running on the Raspberry Pi Zero W, based on different voice
commands. Thus the system invokes PHP scripts based on voice commands and controls the
connected devices via the GPIO interface using python scripts of Raspberry Pi Zero W.

The administration interface features a secured web interface for configuring device names and
adding or removing the connected device to the system.

The Raspberry Pi runs an Apache web server instance for serving PHP scripts for control and
administration purposes and also runs the MySQL database server which stores the device
configuration and status.

Fig. 2. System Software Architecture.

I. Multi Language Voice Control Home Automation System, Parveen Kadegaon, Susmita Paranjape,
Sarita Patil, Priyanka Hiremath, Vishal Danawade, Journal of Advance Research in Mobile
Computing, Volume 2 Issue 2.

II. HOME AUTOMATION SYSTEM USING INTERNET OF THINGS, Shweta Singh, Kishore
Kumar Ray, International Journal of Computer Engineering and Applications, Special Edition
www.ijcea.com ISSN 2321-3469

III. Smart Home Automation System using BLE, Manchanda Sonam, Paul Aman, Thakral Bindu,
International Journal of Innovative Technology and Exploring Engineering (IJITEE)
ISSN: 2278-3075, Volume-8 Issue-10, August 2019

IV. Literature Review on Home Automation System, Neha Malik, Yogita Bodwade, International
Journal of Advanced Research in Computer and Communication Engineering, Vol. 6, Issue 3,
March 2017

[1] Gillis, Alexander (2021). "What is internet of things (IoT)?". IOT Agenda. Retrieved 17 August
2021.

[2] Brown, Eric (20 September 2016). "21 Open Source Projects for IoT". Linux.com. Retrieved 23
October 2016.

[3] "Internet of Things Global Standards Initiative". ITU. Retrieved 26 June 2015.

[4] Hendricks, Drew. "The Trouble with the Internet of Things". London Datastore. Greater London
Authority. Retrieved 10 August 2015.

[5] Laplante, Phillip A.; Kassab, Mohamad; Laplante, Nancy L.; Voas, Jeffrey M. (2018). "Building
Caring Healthcare Systems in the Internet of Things". IEEE Systems Journal. 12 (3): 3030–3037.
Bibcode:2018ISysJ..12.3030L. doi:10.1109/JSYST.2017.2662602. ISSN 1932-8184. PMC
6506834. PMID 31080541.

[6] "The New York City Internet of Things Strategy". www1.nyc.gov. Retrieved 6 September 2021.

[7] "The "Only" Coke Machine on the Internet". Carnegie Mellon University. Retrieved 10
November 2014.

[8] "Internet of Things Done Wrong Stifles Innovation". InformationWeek. 7 July 2014. Retrieved
10 November 2014.

[9] Mattern, Friedemann; Floerkemeier, Christian (2010). "From the Internet of Computer to the
Internet of Things" (PDF). Informatik-Spektrum. 33 (2): 107–121. Bibcode:2009InfSp..32..496H.
doi:10.1007/s00287-010-0417-7. hdl:20.500.11850/159645. S2CID 29563772. Retrieved 3
February 2014.

[10] Weiser, Mark (1991). "The Computer for the 21st Century" (PDF). Scientific American. 265
(3): 94–104. Bibcode:1991SciAm.265c..94W. doi:10.1038/scientificamerican0991-94. Archived
from the original (PDF) on 11 March 2015. Retrieved 5 November 2014.
[11] Raji, R.S. (1994). "Smart networks for control". IEEE Spectrum. 31 (6): 49–55.
doi:10.1109/6.284793. S2CID 42364553.

[12] Pontin, Jason (29 September 2005). "ETC: Bill Joy's Six Webs". MIT Technology Review.
Retrieved 17 November 2013.

[13] "CORRECTING THE IOT HISTORY". CHETAN SHARMA. 14 March 2016. Retrieved 1
June 2021.

[14] Ashton, K. (22 June 2009). "That 'Internet of Things' Thing". Retrieved 9 May 2017.

[15] "Peter Day's World of Business". BBC World Service. BBC. Retrieved 4 October 2016.

[16] Magrassi, P. (2 May 2002). "Why a Universal RFID Infrastructure Would Be a Good Thing".
Gartner research report G00106518.

[17] Magrassi, P.; Berg, T (12 August 2002). "A World of Smart Objects". Gartner research report
R-17-2243. Archived from the original on 3 October 2003.

[18] Commission of the European Communities (18 June 2009). "Internet of Things – An action
plan for Europe" (PDF). COM(2009) 278 final.

[19] Wood, Alex (31 March 2015). "The internet of things is revolutionizing our lives, but standards
are a must". The Guardian.

[20] Stallings, William (2016). Foundations of modern networking : SDN, NFV, QoE, IoT, and
Cloud. Florence Agboma, Sofiene Jelassi. Indianapolis, Indiana. ISBN 978-0-13-417547-8. OCLC
927715441.

[21] Dave Evans (April 2011). "The Internet of Things: How the Next Evolution of the Internet Is
Changing Everything" (PDF). CISCO White Paper.

[22] Vongsingthong, S.; Smanchat, S. (2014). "Internet of Things: A review of applications &
technologies" (PDF). Suranaree Journal of Science and Technology.

[23] "The Enterprise Internet of Things Market". Business Insider. 25 February 2015. Retrieved 26
June 2015.

[24] Perera, C.; Liu, C. H.; Jayawardena, S. (December 2015). "The Emerging Internet of Things
Marketplace From an Industrial Perspective: A Survey". IEEE Transactions on Emerging Topics in
Computing. 3 (4): 585–598. arXiv:1502.00134. Bibcode:2015arXiv150200134P.
doi:10.1109/TETC.2015.2390034. ISSN 2168-6750. S2CID 7329149.

[25] "How IoT's are Changing the Fundamentals of "Retailing"". Trak.in – Indian Business of Tech,
Mobile & Startups. 30 August 2016. Retrieved 2 June 2017.

[26] Kang, Won Min; Moon, Seo Yeon; Park, Jong Hyuk (5 March 2017). "An enhanced security
framework for home appliances in smart home". Human-centric Computing and Information
Sciences. 7 (6). doi:10.1186/s13673-017-0087-4.

[27] "How IoT & smart home automation will change the way we live". Business Insider. Retrieved
10 November 2017.
[28] Jussi Karlgren; Lennart Fahlén; Anders Wallberg; Pär Hansson; Olov Ståhl; Jonas Söderberg;
Karl-Petter Åkesson (2008). Socially Intelligent Interfaces for Increased Energy Awareness in the
Home. The Internet of Things. Lecture Notes in Computer Science. Vol. 4952. Springer. pp. 263–
275. arXiv:2106.15297. doi:10.1007/978-3-540-78731-0_17. ISBN 978-3-540-78730-3. S2CID
30983428.

[29] Greengard, Samuel (2015). The Internet of Things. Cambridge, MA: MIT Press. p. 90. ISBN
9780262527736.

[30] "HomeKit – Apple Developer". developer.apple.com. Retrieved 19 September 2018.

[31] Wollerton, Megan (3 June 2018). "Here's everything you need to know about Apple HomeKit".
CNET. Retrieved 19 September 2018.

[32] Lovejoy, Ben (31 August 2018). "HomeKit devices getting more affordable as Lenovo
announces Smart Home Essentials line". 9to5Mac. Retrieved 19 September 2018.

[33] Prospero, Mike (12 September 2018). "Best Smart Home Hubs of 2018". Tom's Guide.
Retrieved 19 September 2018.

[34] Chinchilla, Chris (26 November 2018). "What Smart Home IoT Platform Should You Use?".
Hacker Noon. Retrieved 13 May 2019.

[35] Baker, Jason (14 December 2017). "6 open source home automation tools". opensource.com.
Retrieved 13 May 2019.