Build A Web Twelve Python Tips ` 120

Application On Firebase For Beginners ISSN-2456-4885

Volume: 11 | Issue: 01 | Pages: 100 | November 2022

The World Of
Connected
Devices
How IoT Can Make IoT Analytics:
The Workplace Making Sense
Intelligent Of Big Data

Industry 4.0 And A Complete

Embedded Internet Guide To Getting
Of Things: An Started With
Introduction Arduino

Open Source India 2022: Simplified Logging And

Post-Show Report Analysis With Parseable
CONTENTS NOVEMBER 2022 ISSN-2456-4885

FOR U & ME
28
14 The Importance of Open Source
in the Metaverse

19 Brain inspired Computing Moving

from known to the Unknown
Worlds

FOCUS
Open Source India 2022: Where the Industry Met the Community
45 How IoT can make the Workplace
Intelligent 39

53 A Complete Guide to Getting

Started with Arduino

60 IoT Might Disrupt Lots of

Technologies for Greater Good

DEVELOPERS
Industry 4.0 and Embedded Internet of Things: An Introduction
64 Hooked on React Hooks!
48
66 R Series: Probability Distributions

75 Twelve Python Tips for Beginners

78 AI: Anaconda and More on

Probability

89 Code Generation: How the Lazy

Become Prolific
IoT Analytics: Making Sense of Big Data

ADMIN
REGULAR FEATURES
93 Simplified Logging and Analysis 07 FossBytes
with Parseable

4 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

THANK YOU

Speakers 90+ Experts Share Knowledge here

For more details, call on +91-98111-55335 or email us at [email protected]

EDITOR
RAHUL CHOPRA

EDITORIAL, SUBSCRIPTIONS & ADVERTISING

Delhi (HQ)
D-87/1, Okhla Industrial Area, Phase I, New Delhi 110020

56
Phone: +91-9811155335
E-mail: [email protected]

MISSING ISSUES
Phone: +91-9811155335
E-mail: [email protected]

BACK ISSUES
Phone: +91-9811155335
E-mail: [email protected]

NEWSSTAND DISTRIBUTION
Phone: +91-9811155335
E-mail: [email protected]

ADVERTISEMENTS
NEW DELHI (HEAD OFFICE)
Phone: +91-9811155335
E-mail: [email protected]

MUMBAI
E-mail: [email protected]

BENGALURU
E-mail: [email protected]
Connecting Internet of Things Devices on Thinger.io
CHINA
Worldwide Focus Media
E-mail: [email protected]

GERMANY
pms Plantenberg Media Service GmbH
71
E-mail: [email protected]

JAPAN
Tandem Inc.
E-mail: [email protected]

TAIWAN
J.K. Media
E-mail: [email protected]

UNITED KINGDOM
ASA Media
E-mail: [email protected]

UNITED STATES
E & Tech Media
E-mail: [email protected]

Printed, published and owned by Ramesh Chopra. Printed at Tara

Art Printers Pvt Ltd, A-46,47, Sec-5, Noida, on 28th of the previous
month, and published from D-87/1, Okhla Industrial Area, Phase I, New
How Contact Centres can Benefit from Artificial Intelligence
Delhi 110020. Copyright © 2021. All articles in this issue, except for
interviews, verbatim quotes, or unless otherwise explicitly mentioned,

83 95
will be released under Creative Commons Attribution-NonCommercial
3.0 Unported License a month after the date of publication. Refer to
http://creativecommons.org/licenses/by-nc/3.0/ for a copy of the
licence. Although every effort is made to ensure accuracy, no responsi-
bility whatsoever is taken for any loss due to publishing errors. Articles
that cannot be used are returned to the authors if accompanied by a
self-addressed and sufficiently stamped envelope. But no responsibility
is taken for any loss or delay in returning the material. Disputes, if any,
will be settled in a New Delhi court only.

SUBSCRIPTION RATES
Year Newstand Price You Pay Overseas
(`) (`)
Five 7200 4320 —
Three 4320 3030 —
One 1440 1150 US$ 120
Kindly add ` 50/- for outside Delhi cheques. Build a Web Application More Shell Programming Secrets
Please send payments only in favour of EFY Enterprises Pvt Ltd.
Non-receipt of copies may be reported to [email protected]—do mention
your subscription number.
on Firebase Nobody Talks About

6 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 FOSSBYTES
Google unveils FIWARE Foundation and
TensorStore, a library
Red Hat join hands to
develop a smart city
to help manage large
solution
Red Hat, Inc., has announced a

data sets
partnership with the FIWARE
Foundation, a non-profit
organisation that promotes the use
of open standards in the creation
of smart solutions, to create an
integrated platform that will use
data to make cities all over the
world more resilient and improve
the wellbeing of their residents.
Red Hat Open Innovation Labs
collaborated with the FIWARE
Foundation and Human Oriented
Products (HOPU), a solution
provider involved in the FIWARE
community, over the course of a
six-week residency to develop an
enhanced smart city solution that
The open source C++ and Python framework TensorStore, developed by Google, is simple to deploy, fully scalable,
aims to accelerate the design for reading and writing huge multi-dimensional reliable and runs on Red Hat
arrays. Multi-dimensional data sets that cover a single large coordinate system OpenShift.
are commonly used in contemporary computer science and machine learning Cities around the world
applications. These data sets are challenging to work with because customers are dealing with escalating and
frequently wish to conduct investigations involving numerous workstations complicated environmental,
operating in parallel, and may receive and output data at unpredictable intervals economic, and social issues that
and varied scales. have an impact on the daily lives
Google Research has developed TensorStore, a library that provides users of their residents. City officials and
with access to an API that can manage huge data sets without the requirement public sector decision-makers need
for sophisticated hardware, to address issues with data storage and manipulation. eco-smart technology solutions to
Numerous storage systems, including local and network file systems, Google Cloud solve these issues so they can use
Storage, and others are supported by this library. data to assist them make decisions
To load and work with enormous arrays of data, TensorStore provides a simple that are good for the environment
Python API. Any arbitrary big underlying data set can be loaded and updated and for their constituents.
without having to store it in memory, because no actual data is read or kept in The smart city platform
memory until the precise slice is required. powered by FIWARE is able
This is made possible by the indexing and manipulation grammar, which is to derive insights from data
quite similar to the syntax used for NumPy operations. TensorStore supports virtual processed by HOPU air quality
views, broadcasting, alignment, and other sophisticated indexing features like data sensors in order to adopt wiser
type conversion, down sampling, and haphazardly created arrays. decisions for the welfare of
Additionally, it includes an asynchronous API that enables read or write inhabitants. It can be deployed on
operations to work concurrently. any private or public cloud. This
platform and its base application
Legit Security announces new tool to secure for air quality monitoring have
GitHub configurations been implemented in Las Rozas
Legitify, an open source security tool to secure GitHub implementations, has and Cartagena, two Spanish cities,
been announced by Legit Security, a cyber security company with an enterprise as pilot use cases.
platform to secure an organisation’s software supply chain. Security teams and

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 7

FOSSBYTES

CERN develops a detailed DevOps engineers can monitor and enforce their GitHub configurations in a
open science policy secure and scalable manner with the aid of Legitify, a GitHub misconfiguration
CERN (European Council for Nuclear checker. With support for Windows, Mac, and Linux, Legitify is a cross-platform
Research) established a new open security solution that offers only a small portion of the features offered by the
science policy for itself, which comes larger Legit Security platform.
into immediate effect, during its 209th
session in September this year. The
goal of the policy is to make all CERN
research completely open, inclusive,
democratic, and transparent for other
researchers as well as for the general
public. The Open Science Strategy
Working Group (OSSWG), which
has representatives from each CERN
department, developed this policy. A
web page dedicated to detailing all
the open scientific projects at CERN
has been published together with the
policy statement.
The finalised policy was
developed in response to the European Software developers all across the world utilise GitHub, a very well-liked
Strategy for Particle Physics’ 2020 source code management (SCM) solution at the centre of many enterprises’ software
update, which emphasised the value supply chains. However, because GitHub is a complicated product, managers may
of open science, and the UNESCO fail to notice incorrect configurations and insecure default settings, which can lead
Recommendation on Open Science, to security flaws that jeopardise an organisation’s software supply chain.
which was released in 2021. Right Due to the distinct configurations and protections necessary for each repository,
since the CERN Convention was ensuring security across big GitHub installations was challenging and time consuming
ratified at UNESCO in 1952, prior to the advent of open source Legitify. It took a lot of manual work and the efforts
open science has been one of the of a large GitHub group to consistently enforce security were prone to error.
organisation’s guiding principles. Along with Legitify, Legit Security has given back to the online security
community by responsibly disclosing more GitHub flaws found by its in-house
security research team. Legit Security also participates actively in groups like
OpenSSL and the Linux Foundation, where it helps to advance the security of the
software supply chain and safe software development in general.

Managing security issues in the open source

world is important
Extensively used open source projects are today being maintained by a limited
number of engineers. These engineers are unable to provide voluntary time and
effort completely to the job since they also need to pay their bills.
All research publications, For more complex open source projects, this can be a problem. As an
experimental data, and research illustration, the Linux kernel project consists of more than 30 million lines of code,
software and hardware will be made contains hundreds of flaws that need to be resolved, and has close to 2000 active
publicly available under the new developers. Each active developer has written more than 15,000 lines of code.
policy, which also incorporates the According to recent research from the Linux Foundation, an application has
previous rules for open access, open an average of 5.1 significant vulnerabilities that are still open, and 41 per cent of
data, and open source software and enterprises lack confidence in the security of their open source software. To make
hardware. Additionally, it combines matters worse, only 49 per cent of businesses have an open source security policy.
other open science tenets such Since OSS updates and patches must be implemented to secure systems, this
as training, outreach, and citizen requirement can bring unique difficulties. Updating mission-critical software could
science, which aim to inform and result in functionality loss and/or unplanned downtime if the solution depends on
engage the public and the next a certain software version. When a situation is business-critical, it may be more
generation of researchers. elegant to hire a specialist to backport the patch, and maintain a version for longer
than the larger community is willing to.

8 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 FOSSBYTES

Lyra audio codec

gets an update
The open source Lyra audio codec,
which Google first released last
year, has been updated with Lyra
V2, which has a new architecture
that enables scalable bitrate
capabilities, greater performance,
higher audio quality, and
compatibility with more systems.
Lyra V2’s internal architecture
is based on the SoundStream end-
to-end neural audio codec, which
has a ‘residual vector quantizer’
(RVQ) sitting before and after the
Contributing to open source as it was intended, improving the code and transmission channel and the ability
publishing it for others, or hiring professionals to manage the OSS code and debug to adjust the amount of quantizers to
it as necessary are options. However, the industry cannot afford to not contribute be used at any time to alter the audio
anything at all. bitrate. The three available bitrates
are 3.2kbps, 6kbps, and 9.2kbps.
HFCL introduces open source Wi-Fi 7 access points Lyra V2 makes use of artificial
The world’s first open source Wi-Fi 7 access points were introduced by HFCL intelligence, and a TensorFlow
Limited, in collaboration with Qualcomm Technologies, Inc., on October 1, 2022, Lite model makes it possible for it
at the India Mobile Congress in Pragati Maidan, New Delhi. to function on Linux and Android
The access points are based on IEEE 802.11be, a ground-breaking Wi-Fi smartphones. Support for Mac and
technology that is intended to give extremely high throughput (EHT), increased Windows is experimental. At this
spectrum efficiency, better interference mitigation, and support for real-time time, iOS and other embedded
applications (RTA). In order to provide a better user experience while using less platforms are not supported;
power, Wi-Fi 7 uses faster connections with 320MHz and 4kQAM, numerous however, things might change later.
connections with multi-link operation, and adaptive connections for adaptive As compared to other audio
interference puncturing. codecs like Lyra (V1) and Opus,
the new audio codec offers a higher
quality (MUSHRA score) than those
at a given bitrate. Lyra V2 at 9.2kbps
offers roughly the same quality as
Opus at 14kbps.

This second generation codec’s

latency has improved from 100ms
to 20ms, making it similar to Opus
for WebRTC, which typically has
delays of 26.5ms, 46.5ms, and
66.5ms. In order to provide real-
Wi-Fi 7 promises a significant technological advance over all prior Wi-Fi time audio encoding/decoding and
standards, providing a more immersive user experience and paving the way for a reduce power consumption, Lyra
more robust digital future. The peak data speeds supported by HFCL’s Wi-Fi 7 APs V2 also encodes and decodes five
will exceed 10Gbps, and they will have latency under 2ms as opposed to the 5Gbps times quicker than Lyra V1.
and 10ms of existing Wi-Fi 6 products.

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 9

FOSSBYTES

Scout APM releases Being the first Wi-Fi 7 access points in the market to embrace open standards,
TelemetryHub all Wi-Fi 7 variants will come pre-installed with open source software. The
Application performance monitoring introduction of Wi-Fi 7 will primarily support the country’s upcoming 5G rollout,
(APM) service provider Scout particularly for enhancing inside coverage. Additionally, the technology will make it
APM has announced the launch of easier to construct a variety of apps because of its increased throughput, dependable
TelemetryHub via its Early Access network performance, and lower latency.
Program (EAP). By utilising the open
source initiative OpenTelemetry, Elon Musk may turn Twitter into an open source platform
TelemetryHub offers observability that encourages free speech
solutions for small- to medium-sized Elon Musk’s Signal text exchanges, which were made public in advance of the
engineering teams. trial, reveal his goal to turn Twitter into an open source, democratically beneficial
Scout, which was first developed platform for free speech. Elon Musk had conversations with a few people in an
as a server monitoring tool in 2009, effort to determine the
has since expanded to offer insight value of the company
into the performance of applications before making the US$ 44
for a growing number of supported billion offer to purchase
languages. It is currently developing Twitter.
into an observability platform His Signal text
that will allow teams to diagnose communications with a
problems and shorten the mean time number of well-known
to resolution (MTTR) by providing figures in business and
a straightforward, reasonably priced, technology have just been
and practical single pane of glass made public. Mathias
view across distributed systems. Döpfner, the CEO of Axel Springer, the parent company of Business Insider,
Users can ingest telemetry data texted Elon Musk at one point, pleading with him to buy Twitter and promising to
from platforms instrumented with the “manage” the firm on his behalf.
OpenTelemetry CNCF initiative using Since Musk works to promote freedom of speech in the broadest sense, it is
TelemetryHub. As an early adopter obvious that these remarks are exactly what got him. People who raise crucial
of OpenTelemetry, TelemetryHub problems are frequently silenced on platforms like Twitter. Many Twitter users who
has the support of a project that has backed Musk noticed that their tweets were being deleted. These tweets earned a
amassed over 100,000 contributors small number of likes and responses despite interaction statistics that were high
from the community. TelemetryHub (often in the thousands). Additionally, it was observed that some tweets, particularly
is dedicated to evangelising and those that were critical of Twitter, were displayed to a group of Twitter users entirely
supporting the project through unrelated to the concerns or problems they were discussing and who were not
contribution and education. This gives followers of the account that sent them.
users the chance to employ an ever- Musk and former Twitter CEO Jack Dorsey had another text conversation that
evolving, vendor-neutral solution showed Musk was sincerely dedicated to making Twitter a high-quality and open
that gives them complete control and platform. Dorsey reportedly proposed that Twitter employ an open source protocol,
freedom with their observability data. similar to that used by the Signal messaging platform. Musk promised Dorsey that,
if possible, he would work towards it.

Red Hat rolls out OpenStack Platform 17

OpenStack, an open source Infrastructure-as-a-Service (IaaS) cloud, is still seen
by some as being obsolete. They are utterly mistaken. OpenStack is the most
widely used open source cloud, according to Statista. The fact that OpenStack
TelemetryHub will enable users is the lifeblood of the telecom industry, including Verizon cellular and 5G
with a stake in the stability and health networks, is even more significant for Red Hat. So it should come as no surprise
of their systems to take advantage that Red Hat highlighted the relationship between OpenStack and telecom while
of the comprehensive diagnostic announcing Red Hat OpenStack Platform (RHOP) 17 at the Mobile World
tools offered by the new platform to Congress in Las Vegas this year.
increase understanding and decrease Red Hat also emphasised how closely it is collaborating with its Kubernetes
MTTR thanks to usage-based pricing based OpenShift release with this release. Some people believe that OpenStack and
and no-seat licences. Kubernetes are competitors. They aren’t. Red Hat is demonstrating how they may
cooperate to benefit from both virtual machines and containers.

10 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 FOSSBYTES

“We have found over the last couple of years that the conversation is not a The Clearing House joins
religious dispute between the two platforms but rather where their workloads are OIN as a community
served best,” said Maria Bracho, lead product manager for OpenStack. Therefore, member
a virtualized control plane that uses OpenShift is now supported by OpenStack The Clearing House (TCH) has joined
version 17. This makes as a community member of the Open
it possible for customers Invention Network (OIN), which was
to run virtual machines founded to protect open source software
(VMs) and Kubernetes (OSS) and is currently the largest patent
based cloud native non-aggression group in history. TCH is
services on the same reaffirming its commitment to OSS as an
cloud. enabler of cutting-edge financial services
Dynamic resource payment platforms. It runs US-based
allocation is another payments networks that clear and settle
feature of the revised more than US$ 2 trillion in payments
OpenStack 17 that daily through wire, ACH, check image,
supports “lighter” and real-time payments.
deployments. These are Through royalty-free cross-
deployments that don’t need as many services, such as edge computing. licensing of Linux System patents,
The new RHOP will primarily appeal to the telecom industry, but it is still a the OIN community upholds patent
reliable platform for other use cases. These consist of edge deployments, private non-aggression in core Linux and
clouds, and public clouds. Anyone who prefers to manage their own cloud over related open source technology. Similar
entrusting their services to a hypercloud provider like Amazon Web Services, Azure, royalty-free licences are granted for
or Google Cloud should give RHOP some thought. Open Invention Network patents to any
company that pledges not to oppose
the Linux operating system with its
Google launches OpenXLA, an open source own patent claims. On http://www.j-
ML compiler project oin.net, the OIN licence can be signed
Google Cloud has introduced the OpenXLA open source machine learning compiler electronically
ecosystem at its Next ’22 conference. A community-led and open source ecosystem “Advancements in financial
of ML compiler and infrastructure projects, OpenXLA has been created by Google services and FinTech increasingly rely
in collaboration with AMD, Arm, Meta, NVIDIA, AWS, Intel, Apple, and other AI/ on open source technologies,” said
ML developers. By allowing developers to select the frameworks and hardware Keith Bergelt, CEO of Open Invention
they want for their ML projects, OpenXLA attempts to address the problem of Network. “As the most experienced
framework and hardware incompatibilities, which can hinder the development of payment company in the US, and a
machine learning algorithms. keystone for the financial services
The OpenXLA project’s goal is to enable effective lowering, optimisation, and industry, we are pleased that The
deployment of machine learning models from the majority of major frameworks, Clearing House is committed to patent
including PyTorch and TensorFlow, to any hardware backend (particularly CPUs, non-aggression in core Linux and
GPUs, and ML ASICs), through collaboration with major ML frameworks and adjacent open source technologies.”
hardware vendors. “We remain focused on integrating
Its objectives are: promote XLA industry collaboration and create a thriving the best technologies in our payment
OSS community; exchange ideas and solicit input on OpenXLA’s technical networks, which help to deliver services
direction to make sure it satisfies that our banks’ customers value and
the requirements of its key users trust. We enable all financial institutions
and contributors; create a new to offer safer, faster, and smarter digital
XLA repository or organisation transaction services, and open source is
that is independent of hardware a significant innovation driver in these
and framework, has infrastructure offerings,” said Sean Reilly, senior vice
to more easily accept PRs, and has independent build and test; and ensure that the president and associate general counsel
removal of XLA from TensorFlow causes the fewest possible disruptions for current at The Clearing House. “We are pleased
users and contributors. to join the Open Invention Network
The XLA compiler, which was created to simplify modelling in TensorFlow by and support its role in protecting open
accelerating the training process and lowering overall memory usage, will be the source software.”
first goal of the new community project, said Sachin Gupta, Google VP and GM of

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 11

FOSSBYTES

Intel and DataStax partner infrastructure, in a blog post. In order to create StableHLO, a portable ML compute
to enhance the performance operation set that serves as a portability layer across machine learning frameworks
of Apache Cassandra and and compilers, OpenXLA will endeavour to isolate the compiler from TensorFlow.
Apache Pulsar
Real-time data startup DataStax has Open RAN platform for simulation of
announced a strategic partnership with 5G networks is now available
Intel to improve the performance of The first open source platform for end-to-end simulation of 5G wireless networks,
systems based on the Apache Cassandra powered and optimised by Open Radio Access Network (Open RAN) components
database and the Apache Pulsar and ns-O-RAN (Open Radio Access Network Simulator), has been made available to
messaging and streaming technology the public by researchers at the Institute for the Wireless Internet of Things (WIoT)
on the most recent Intel Xeon Scalable at Northeastern University.
processors. In addition to providing
enterprise and cloud solutions based
on these potent open source initiatives,
the companies intend to make
improvements to each project. DataStax
has been chosen for the Intel Disruptor
Initiative, in which Intel works with a
limited group of cutting-edge businesses
that are redefining innovation.
Largely scalable databases, such
as Apache Cassandra, function best
on machinery designed specifically to
support database workloads. Through
this partnership, Intel and DataStax are
working together to provide Apache
Cassandra with CPUs that can support
this utilisation in order to contribute to
its long-term success. Apache Cassandra
and Apache Pulsar’s strength, along with The O-RAN standards-compliant Near-RT RIC is interfaced with using the ns-
Intel’s long-lasting processors, will pave O-RAN emulator, which combines ns-3 with O-RAN-compliant features including
the path for efficient database operations the E2 application protocol and service models. This project is the product of a
and quick access to crucial data. partnership between Mavenir, a member of the WIoT Industry Consortium and the
The Intel-DataStax partnership Open6G initiative led by WIoT, and the Institute for the Wireless Internet of Things
will also concentrate on introducing (WIoT) of Northeastern University.
the most recent Cassandra and Pulsar The widely used open source network simulator ns-3, which simulates all the
technologies to academic institutions layers of the RAN protocol stack as well as core network operations and mobile
around the world in order to speed up traffic applications, and its 5G module from the University of Padova and NYU,
innovation and train the next generation which enables the LTE-5G NR dual connectivity RAN protocol stack on ns-3, serve
of programmers and data scientists on as the foundation for the ns-O-RAN simulator.
these potent open source initiatives. With the help of extensive modelling of the entire 3GPP RAN protocol stack
With well-known products like and large-scale 5G simulations based on 3GPP channel models, ns-O-RAN has
Astra DB, DataStax’s cloud database been conceived and developed to enable the integration of O-RAN software, such as
built on Apache Cassandra, the company Mavenir’s Near-RT RIC. This enables RAN key performance metrics (KPMs) data
is already a significant supporter of collection at scale, in various simulated circumstances, and with various applications
the Cassandra and Pulsar open source (e.g., multimedia streaming, web browsing, wireless virtual reality, etc). Two E2 service
projects. Astra DB makes it simple for models (E2SM), E2SM-Key Performance Metrics (KPM) monitoring and E2SM-RAN
customers to access and utilise all of Control (RC), which enable a closed-loop control, are implemented by ns-O-RAN and
Cassandra’s capabilities without having support an O-RAN-compliant E2 interface (for example, of traffic steering and mobility).
to deal with the hassle of managing
infrastructure. End-to-end security, global Dutch minister in favour of using open source
scalability, and streamlined operations are for digital ID wallets
features of this serverless platform. The Dutch minister for digitalization, Alexandra van Huffelen, recently spoke about
the need for open source in promoting ID wallets as secure and user-friendly.

12 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 FOSSBYTES

The algorithms are intended to

serve as a decision-making tool at each
stage of farming, from sowing the seeds
through reaping the harvest. According
to Siegel, this involves forecasting
temperature and wind speeds,
determining the ideal depth to plant
seeds based on soil conditions, advising
on crop types, and following best
practices to keep carbon sequestered in
the soil. It also involves determining the
right amount of fertiliser or herbicide to
use on which areas of the fields.
The artificial intelligence (AI)
component of Microsoft’s FarmVibes
project includes features like Async
Fusion, which combines sensors and
During an appearance on news outlet Euractiv’s ‘The Tech Brief’ podcast, Van satellite imagery to create maps of soil
Huffelen stated that The Netherlands has reached an agreement on the usage of open moisture and nutrient heat; SpaceEye,
source in most or all components of a digital ID wallet. which uses AI to remove clouds from
Everyone involved in a digital wallet programme gains trust as a result of the satellite imagery; DeepMC, which
transparency. The software is superior to closed source programmes since anybody uses sensor data and weather forecasts
may access the source code and make changes. Open source software may be safer to predict the temperatures and wind
for the same reason. According to van Huffelen, it is also more effective. She said speeds of a farm’s microclimate, and a
that she “believes in not being super-naive,” echoing decades of prior justifications “what if” analysis tool that speculates
for open source. on how farming practices might change.
Andrew Nelson, a fifth-generation
Microsoft Research open sources Project FarmVibes farmer and software engineer, has
In an effort to increase farmers’ yields while lowering costs and assisting the tested the Project FarmVibes suite on a
agriculture sector in accepting responsibility for its role in climate change, 7,500-acre farm under his management.
Microsoft Research is open sourcing its agricultural data and networking Nelson collaborated with Microsoft
technology. The technology platform, known as Project FarmVibes, incorporates Research to evaluate the technology
the cloud provider’s most recent research in precision and sustainable agriculture, portfolio. He said that the project’s
according to a blog post by Microsoft’s Jake Siegel. This builds on data analysis and AI will improve damage control
integration work done with Microsoft’s clients like Land O’ Lakes and Bayer. capabilities while also saving him time
and money in agricultural settings.
Nelson is now piloting FarmVibes.
Connect, which will utilise the unused
TV white space spectrum to provide
internet connectivity to rural and remote
locations. He is also experimenting with
FarmVibes.Edge to cleverly compress
the enormous volumes of data collected
by drones. This approach helps to focus
on one specific problem, such as weeds
in a field, while ignoring other issues.
Microsoft intends to open source
additional FarmVibes parts related to
edge and connectivity services in the
future, in addition to the AI.

For more news, visit

www.opensourceforu.com

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 13

For U & Me Overview

The Importance of Open Source

in the Metaverse

T
The Metaverse is no he Metaverse has four pillars, can reach new audiences unlike
longer fiction…this virtual namely, virtual spaces, virtual before. All this can be achieved due to
world is set to take over assets, virtual interactions 5G, augmented reality (AR), virtual
our lives, making people and digital payments. In this new reality (VR), and edge computing.
all over the globe connect virtual world, co-workers or students These ultra-reliable, low latency
with each other in ways can gather for a chat over coffee services ensure the rendering of 3D
irrespective of what part of the globe graphics and content, and make virtual
never experienced
they belong to. Musicians or artists experiences faster. “Open source will
before. But how relevant
can also interact with fans from around play a huge role in the Metaverse, and
or important is the role
the world in a large digital venue, will help it evolve faster,” says Dr
of open source in this which would be impossible in the Lokesh Boregowda, director, Samsung
new virtual world? That is physical world. Today, conferences R&D Institute India.
the question this article
attempts to answer.

14 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 InSight For U & Me

A need for standards

“It becomes very interesting for us
to witness this seamlessly connected
virtual world as part of the Metaverse.
It opens the door to a wide number of
possibilities and we have yet to see
how far we can truly go with it. It is
important to adopt at scale; otherwise, it
all falls flat. We have seen some of the
standards that ensure interoperability,
such as Razer’s open source virtual
reality (OSVR). We have seen that
it encourages experimentation with
the development of headsets as well. Dr Lokesh Boregowda, director, Vinay Rajagopal, ISV technology lead archi-
But if you look at the flip side, on the Samsung R&D Institute India tect for partner ecosystem at Red Hat.
device layer you will see the OpenXR
standard being badly adopted by
manufacturers,” says Vinay Rajagopal,
ISV technology lead architect for
partner ecosystem at Red Hat.
“Encouraging the development
of a decentralised, distributed and
interoperable virtual world built on
the foundation of open standards also
creates multiple opportunities. We have
already started seeing a proliferation of
3D environments that have been created
by the users themselves. This is where
the blockchain has become imperative
for the details surrounding a transaction Gandhali Samant, director, cloud Dr Sumit Kalra, assistant professor in the
and the digital assets or the avatars. solution architecture, digital and Department of Computer Science and Engineering,
It has become important for us to put app innovation, Microsoft Indian Institute of Technology, Jodhpur
these assets on multiple ledgers, while
making sure that you understand who solution architecture, digital and app of Computer Science and Engineering,
has built, sold or purchased a particular innovation, Microsoft, India. “A good Indian Institute of Technology, Jodhpur.
digital asset. This is important from a aspect of working with open source is “Today whatever we do with the
legal aspect too,” he adds. that there will always be components Metaverse is going to have a tremendous
“Open source is the ground zero for ready to be used, which saves a lot of impact on the future of its development.
the development of technologies like time as you don’t need to start from If we are going to continue Metaverse
the Metaverse. All the biggest projects scratch. As the Metaverse continues to development with open source then it is
in the last five to ten years that have gain traction, more tools and platforms crucial for us to adopt open standards.”
been able to garner the most attraction will appear to assist the developers.”
are the ones that are open sourced — “Even though open source comes The right hardware
artificial intelligence, blockchain, and with a lot of benefits such as quick Syed Mohammed Khasim, senior
containerisation, to name a few. Open development and large communities, it manager, software development -
source comes with a great number of is important to note these communities Sitara ARM MPU, Texas Instruments,
benefits. It allows you to collaborate own the software itself. In 2014, a bug notes: “In the last couple of decades
with the smartest minds from every was detected in Unison. Its source code I have seen the evolution of open
corner of the world; your work gets had been open for so long that it made source, starting from Linux. It all
constantly reviewed and used, and it hard to estimate the time hackers had essentially started for shared software
there is a constant feedback loop,” been exploiting it,” says Dr Sumit Kalra, development. The cost of developing
says Gandhali Samant, director, cloud assistant professor in the Department software turns out to be a big challenge

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 15

For U & Me InSight

has woken up to the vulnerability

issues. Everyone today, including the
US government, is trying to spend
more money on the detection of
vulnerabilities and securing them.
Working together as a community can
allow us to solve these issues together.”

The advantages of open source

in the Metaverse
So what is the main foundation and
strength of open source, which will play
a key role in the development of the
Syed Mohammed Khasim, Biju K. Nair, Metaverse?
senior manager, software development - founding partner, “Open source is both flexible and
Sitara ARM MPU, Texas Instruments Legalitech agile. It is important to note that the
advantages of using open source far
as it requires a unique skillset. You need “Today, things have evolved outweigh the disadvantages. GitHub is
to have proper architecture to build tremendously. We are seeing multi-core the largest open source community in
software, a design and the necessary architecture and multi-core programming the world or, you can say, it is the home
components that will allow you to even in a simple smart watch today. It of open source itself. At GitHub our
get started with the coding. You then requires a great amount of skill to do it focus is to continuously improve the
have to validate and commercialise all. Not only are you not going to work platform and add features so that the
everything before it goes into the for free while working on open source collaboration and open source projects
hands of developers and, ultimately, software, but your contribution to open can thrive,” says Gandhali Samant.
the consumers themselves. It all boils source will also provide you with the “Today, every new security feature
down to sufficient investment. But there relevant validation and introduce you that we build is available for free
is a limit to which everyone can invest to new features. Hence, companies for public repositories. It is the main
in software and often it is only for a always try to build an ecosystem for source of developing open source
particular period of time.” open source development as it tends to software. If you look up the top 10
“Open source is no free lunch as be more robust and reliable. It is time Metaverse projects on Google, you
somebody is indeed paying for it. When to define the hardware ecosystem that will find that they are all available
we use the term open source, we mean we are going to enter into, and the kind on GitHub. The role of GitHub in
that we are enabling a shared software of software skillsets that are required, the Metaverse will be very similar
development model where we want to in order to assess what we need to build to the role that GitHub has played in
collaborate with similar entities that want the community around. We also need to the large scale success and adoption
to invest in the development of software. find whether it can scale and sustain on of technologies like TensorFlow
Hardware is also a fundamental aspect its own without a huge sum of money for AI, or Docker and Kubernetes
of building software, as without proper being pumped from big organisations,” for containerisation, orchestration,
hardware in place software cannot be adds Khasim. Hyperledger, etc. GitHub will continue
developed. We need low cost hardware to provide the best platform for
that can be afforded by everybody. The legal angle software development,” she adds.
This is why the Intel ecosystem picked Biju K. Nair, founding partner, But are there any limitations and
up, because everyone used a personal Legalitech, says: “There is also a concerns for the collaborators?
computer and these systems did not very important legal aspect to open According to Samant, “On one side
require a huge investment. Over time, source from an intellectual property we have multiple open source projects,
embedded systems have also evolved (IP) perspective. IP is different in the while on the other we have tech giants
and reduced a lot of the burden of Metaverse from how it is considered like Google and Microsoft that are already
software development. With the entry of in the traditional ecosystem. So this investing big dollars in open source. No
Android all the necessary components creates opportunities to maintain and one company holds the monopoly for
of open source development have been safeguard the IP within the Metaverse. any component today. They are bound
stitched together,” he says. In the last two years the whole globe to come together and work, as each one

16 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

For U & Me InSight

brings a different value to the table. We features. Every smart device today is what rights have been provided to
will see more standardisation across some connected to a network, which is also them, and plan accordingly. They
of the underlying fundamental blocks exposing it to various security threats. also need to check if they have
to ensure compatibility. For instance, So semiconductor companies, too, are filed for the trademark for virtual
Linux started as an open source project focusing on resolving these security goods or not, because if they don’t
while Microsoft Windows was totally threats,” says Khasim. do that they will be at risk of losing
closed source; yet, they still have a There could be trademark issues too. the rights for those goods in the
number of partnerships.” Biju Nair says, “Let us say you have a Metaverse workspace.”
trademark in the physical world but have There are a lot of legalities that
Data privacy no presence in the virtual world. Then need to be looked at and addressed
There are already a lot of copyright and the chances of you getting an injunction separately in the Metaverse.
patent infringement cases around data for trademark misuse are very low. Such Nonetheless, the way forward is
privacy as many open source projects a case has already been brought before a exciting and interesting as there is still
are being used without the consent of US court — E.S.S. Entertainment versus a lot to be explored and developed in
the original creator. Although we have Rock Star Videos. The court ruled that the this new virtual world.
better control over such scenarios in depiction of a physical world trademark
the real world, this is not the case in the in any video game does not infringe on
This article is put together from a
Metaverse as the person is not physically the original trademark, as the video game panel discussion at ‘Open Source
present there. It is not clear yet if the is an artistic expression protected by the Summit: Frontiers of Innovation 2022’,
virtual property in the Metaverse will First Amendment. So you may not be an event co-hosted by Samsung and
IEEE Bangalore.
have any analogy in the physical world. protected in the Metaverse unless you
“When it comes to hardware, have some sort of presence there.”
technology has evolved exponentially Nair adds, “Companies need to Transcribed and curated by
in terms of security, power and revisit their existing agreements, see Laveesh Kocher

18 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 InSight For U & Me

Brain Inspired Computing

Moving from Known to the
Unknown Worlds
Over the last few decades technology has integrated itself into our daily lives in
ways which make us totally reliant on it for our day-to-day tasks. Undoubtedly,
technology has only gotten smarter. Considering the trajectory it currently
follows, it is only a matter of time until it is able to mimic the way we think and
perform our daily tasks. Continue reading to know more.

W
e are living in an the perspective of the technology to the real world which is highly
extraordinary time when landscape. It shows that a personal dynamic in nature. By combining
we are trying to recreate a computer and a wired internet these two dimensions we envision
human brain in machine. Human brain connection can be used to connect to intelligent systems.
is the most sophisticated organ that the external world.
nature has given us. Building something The horizontal dimension in the Intelligence and artificial
which can equal that is a very difficult figure shows that with the help of larger intelligence
feat to achieve. systems we are able to integrate data The concept of intelligence has been
Over the last few years, a lot of altogether on a large scale storage and discussed by people for many years,
studies and findings have been done process that data through cloud. The even as far back as the 18th century.
but that does not seem to be enough by vertical direction contains the devices According to Wilhelm Wundt, “Our
any means. Our limited understanding and technologies that provide a plethora mind is so fortunately equipped that
of this complex organ makes it even of functionalities. it brings us the most important bases
harder to mimic all of its functions For example, our cell phones for our thought with our having
precisely. Yet, we are determined to can sense, display, communicate via the least knowledge of this work of
make something that can at least be wireless systems and the internet elaboration.” Clearly, we are curious
close to the real thing. while providing a great computational about intelligence, but we do not know
power for data processing applications. much about our brains and where does
Technology landscape The computing domain can change it actually come from.
Figure 1 shows an analogue from from a static and virtual environment People have become more curious
about the intelligence
T. Hylton, ‘Perspectives on Neuromorphic
itself through the years of
Dynamic, Online,
Computing,’ 2016 Real World exploration, which has led
• Sensing • IoT them to build intelligent
• Display
• Wireless Com-
• Robotics
• Industrial Internet
systems we today call
munication & • Self-driving Cars artificial intelligence (AI).
Internet • Smart Grid
• Computing at the • Secure Autonomous The history of AI can be
Edge Networks
• Real-time Data to
traced back to a summer
Program
Mobile Phone Decision
Learn
workshop back in 1956.
Intelligent Systems
It is widely considered
• Data Integration as the founding event of
• Personal Com- • Large-scale Storage
puting • Large-scale Computing
AI as a field.
• Wired Internet According to
Desktop/Workstation
Static, Offline Wikipedia, “It is a theory
Virtual World Data Center/Cloud
and development of
Figure 1: Technology landscape computer systems able

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 19

For U & Me InSight

to perform tasks normally requiring Million instructions per second (MIPS) Human
109
human intelligence, such as visual 100G Neurons

perception, speech recognition,

Monkey
decision-making, and translation 106 3G Neurons
between languages.”
Lizard
103 2M Neurons
Evolution of computing power Dr Raymond Kurzweil
The chart in Figure 2 is from Dr Worm
Raymond Kurzwell’s book that was 100 300 Neurons

published back in 1990, titled ‘The nes

age of intelligent machines.’ This chart 10-3 Bacteria
1 Neuron
summarises the computing power
measured as millions of instructions per
10-6
second. The dots in the chart represent
a curve.
This curve suggested that by the 10-9
1900 1920 1940 1960 1980 2000 2020
year 2020 the computing power could
reach the intelligence level of monkeys Figure 2: Evolution of computing power
and human beings. May be, right now
this claim is too bold to go with. But Channel-wise Shortcut
the bottom line is that a sufficiently
advanced computer program could
exhibit human-level intelligence.

Major limitations Shape-wise

Some limitations have emerged from the
Filter-wise Depth-wise
intrinsic architecture of the computing
system hardware. In the past thirty years, Figure 3: Structurally sparse deep neural network
people building computing systems have
had different objectives for improving
the RAM and processors. They aimed
to improve the execution speed for
processors while the major goal for the
memory part was to increase the storage
density. The two components have a
huge performance gap to this day.
Another important aspect is the
limited power efficiency ceiling. The
power efficiency ceiling is much lower Figure 4: Group lasso regulation
than the efficiency provided by human
brains. Human brain is indeed the most The gray matter consists of a neocortex, rigid operations.
efficient computing machine there is. which contains six layers where signals Fault tolerance: The cells within
So, let us take a quick look at it. travel within the layers. The fundamental the human brains are replaced with new
The human brain, on average, elements of the neural cortex are cells as the old ones die. The brain can
contains 15 to 30 billion neurons. It neurons and synapses. even heal and continue its functionalities.
essentially has very complex structures There are three characteristics that A silicon based computer system cannot
and offers complicated operations. But its can be found in human brain but not in heal itself at all.
power consumption is extremely low at the computers: Lack of need for the programs:
around 20 to 35 volts. Low power consumption: Only The human brain does not require a
Figure 3 shows that a human brain about 35 watts of power is consumed program in order to operate as it can
contains some gray matter for thinking by the brain versus roughly 250 watts learn and unlearn things. This is not the
and white matter for signal transitions. by a graphics processing unit (GPU) for case with a computer.

20 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 For U & Me InSight

WTA Masks prunings or structured prunings. We can

also consider activation sparsity where
ReLU has been widely adopted.
Our lab has proposed bit-level
sparsity, which basically decomposes
the weight parameter into binary
Input Hidden 1 Hidden 2 Output Input Hidden 1 Hidden 2 Output
(a) Original (b) With WTA masks
presentations that aims to remove
Feature Map
unwanted zeros from the computation.
Feature Map Input Matrix Input Matrix
Then there is inherent sparsity that
transposes the convolutions and expands
H
the flick necklace convolutional kernels.
Hence, we do not have to involve those
zeroes into the computation.
C W Col 1 Col 2 Col 3 Col 1 Col 2 Col 3
There is a need for structural
(a) Original (b) With WTA masks
sparsity because non-structured
COMPARISON WITH EXISTING FEATURE MAP PRUNING METHOD sparsity may not speedup traditional
platforms like GPUs. If we are able
to achieve structured sparse state,
then we are printing out the weight
parameters by rows or columns
that will be more friendly to the
hardware. These structures can be
squeezed into much smaller matrices
Figure 5: Comparison with existing feature map pruning method for the operations. Let us see how
we can do that.

Structurally sparse deep

neural network
The key idea here is to form a group.
A group can be any form of a weight
block, depending on what sparse
Figure 6: Mixing precision with bit-level sparsity structure you want to learn. For
example, in CNNs, a group of weights
Bit Representation Conversion FP/BP Loop: Each Bit as Float Trainable Variable can be a channel, a 3D filter, a 2D filter,
Scaling a filter shape fibre (a weight column),
STE Forward and even a layer (in ResNets), as
DNN FP
STE Backward shown in Figure 3.
Once the groups have been defined,
Binary Updated and
Quantisation
Conversion Trimmed
DNN BP we can apply lasso regulation to the
(a) (b) groups. For example, Figure 4 shows a
weight vector with five elements, which
Figure 7: Bit representation conversion we divide into two groups—one with
Let us now look at pruning and reduced so that we can reduce two and the other with three elements.
sparsification for deep neural networks the computation and bandwidth We can apply group lasso on each
(DNN) acceleration. Why do we need to requirement by eliminating that group by following the equations
sparse DNN models? Some answers are: redundancy. This raises the question illustrated in the figure itself. Then we
ƒ Because the ever-growing as to what solution do we have in sum the group lasso over all groups
size brought challenges to the order to achieve the goal. as a regulation and add it as a part of
deployment of the DNN models. When we talk about sparsification the optimisation terms of the overall
ƒ A lot of studies have shown that the and DNNs, there are multiple ways optimisation objectives. This forms
parameters of DNNs are redundant. to achieve the goal. We can sparse the procedure of our method called the
Many of them can be pruned and weight parameters through unstructured Structure Sparsity Learning.

22 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 InSight For U & Me

Sparsity-inducing regulariser for

DNN involes the following:
L1 regulariser (sum of absolute
values)
ƒ It has been used for sparsity since
1996.
ƒ It is differentiable, convex, and easy
to optimise
ƒ It is proportional to the scaling, which
will lead to scaling down of all the
elements with the same speed, which
is undesired
Figure 8: Processing engine with in-situ neuron activation
L0 regulariser (number of nonzero
values)
ƒ It directly reflects the sparsity by
definition
ƒ It is scale invariant. However, our
zero regulariser provides no useful
gradients
ƒ It needs additional tricks for applying
on DNN pruning (stochastic
approximation, ADMM), which makes
the problem complicated. This leads to
certain complexity in the design

Moving beyond lasso

We propose to move beyond L1 and L0
Figure 9: Spike conversion
and aim to find the sparsity-inducing
regulariser that is both differentiable and
scale-invariant for pruning. At this stage
the hoyer-square regulariser draws our
attention. It is represented by the ratio
between the L1 and L2, which is both
differentiable and scale-invariant.
For element-wise DNN pruning, we
can apply the square version of hoyer
regulariser, which has the same range and
similar minima structure as L0. We further
extend the hoyer-square regulariser to the
structure pruning settings. Figure 10: In-situ nonlinear activation function
We can apply hoyer-square technique to prune neuron activations quantisation.
regulariser over the L2 norm of each layer wise. Hence, the design introduces For a quantised matrix, when the
group of weight. The weight within the low overhead for the real-time interface most significant bits of all the elements
group can be induced to become zero (see Figure 5). are zero simultaneously, we can remove
simultaneously. Let us now look at the Mixing precision with bit-level the MSB directly (see Figure 6).
effect of the hoyer-square regulariser. sparsity: We know that fixed point For bit representation we follow the
Regulating the activation sparsity: quantisation is an important model dynamic scaling quantisation procedure.
In operations, the activation sparsity is compression technique. In the previous Here (see Figure 7) we use the element
dynamic and the sparsity level is not research it was found that some layers with the largest absolute value in each
identical for different inputs. Our group can retain higher precision while others layer that is the scaling factor and
has proposed what is called the Dynamic can be quantised to less precision. This uniformly quantise all the elements to
Winner Takes All (WTA) dropout is the whole idea of mixed precision embed the value in binary form.

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 23

For U & Me InSight

S1 S2
1

0.8

0.6

Voltage V
0.4
Voltage sampling
0.2 at spiking times

-0.2
0 50 100 150 200
(a)

1
0.8 Voltage sampling
at computation Output spike is
0.6 stage (99-100ns) generated at this

Voltage V
0.4 point

0.2
0 99 100

-0.2
0 50 100 150 200
Time/ns
(b)
(Z. Li et al., DAC’2020)

Figure 11: ReRAM based processing engine: ReSiPE

We consider the scaling factor line like proportional relations, but the network applications, the current can
‘s’ in Ws as an independent trainable measurements show that it could get either be several microamperes or
variable. This way we are able to saturated. This is mainly because of the several milliamperes—especially at
enable gradient based training of bit operation of the IFC (integrate-and-fire the early stage when the device is not
representation. During the training we case), which have an upper bound. very stable and could result in a larger
allow each bit to take a continuous Processing engine with in-situ range of resistance.
value between zero and two. neuron activation (ISNA): We integrate In-situ nonlinear activation
Memristor—rebirth of 64-kilobit RRAM arrays in this engine. function: In the following design (see
neuromorphic circuits: ReRam, The bottom part of this device is so Figure 10) we have two knobs. The
memristor, or metal-oxide resistive much like an irregular memory array. On reference voltage can control the read
random access memory is essentially the top part we add our neural processing voltage of the bit lines while the threshold
for simple two-terminal devices. components (see Figure 8). voltage is used to control the capacitors
When we apply voltage or pass current Spike conversion: The design still along with charging and discharging
through the devices, their resistance uses capacitors to generate spikes. speed. With these distant knobs we are
level could change. It is often also Over here (see Figure 9), we use the able to slightly tune the shape of neural
called the Programmable Resistance output of the op-amp to form a positive activation functionalities.
with analogue states. feedback to accelerate the responses Configurable activation precision:
The spike based neuromorphic from the current amplifier. We use We measured our chips on the
design: We have moved to spike based the current mirror to scale down the application level and explored trade-
neuromorphic design and minimisation bit-line current, because in the real offs between high precision weight and
of power consumption. It is very activation versus improved
similar to the previous structure, Net 1 Decision Net 2 Decision performance. Such exploration
Boundary Boundary
but here we use the frequency of is useful for system-level, fine-
the spike to represent the input data grained optimisation and here
amplitude. So, the information is are the results.
ADV
encoded in the spikes instead of As mentioned earlier, the
analogue voltage. Accordingly, fire designs were fabricated by
circuits are used to produce output 150-nanometer complementary
spikes along the bit line levels. metal-oxide semiconductor
LEG
We measure the output as (CMOS) and HfO RRAM
impulse number versus the input devices. They contain 64-kilobit
as the summation of the weighted RRAM devices. The energy
RAND
inputs. The initial expectation was efficiency is also high at 0.257
that these will follow a straight Figure 12: Transfer-based attacks pico joules (pJ/MAC).

24 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 InSight For U & Me

ReRAM based processing Error Rate vs

Attack Layer
Targeted Success
Rate vs Attack
engine: ReSiPE 1.00
0.75
Layer Depth
In the last two years we have moved to Layer L DN121 0.50 AA
RN50 0.25 ITCM
single spiking designs. The goal is to TMI/GSM
ƒw Dog
0.00
leverage the timing of the spike for data 1.00

representation and the matrix operation. For ƒw Plane DN121

0.75
0.50
that we divide the design into three stages. VGG19 0.25
ƒb
In the first stage we translate the Dog 0.00
1.00
0 10 20 0 10 20
timing of the input spikes to an analogue Activation Attack 0.75

voltage. Hence, we use a charging process ƒw Plane VGG19 0.50

RN50 0.25
of a capacitor Cgd for reference. When the 0.00
ƒb Plane
input spikes ‘S’ then the voltage will be 1.00
0.75
sampled at exactly the spiking time. VGG19 0.50
DN121 0.25
In stage two, the hinge signals are 0.00
fed into the ReRAM crossbar arrays to 0 5 10 0 5 10
Layer Depth
perform matrix operation and during the
computation the capacitor Ccog is charged Figure 13: Activation attack
continuously. The charging speed is shows two different models that are trained by Eli’s paper we found that models
determined by the bit-line current. with the data from the same distribution. entrained on the same data set capture a
The last stage is the output stage It implies the property of adversarial similar set of non-robust features. These
where Vout is translated to upper spike transferability. This is particularly are the features that are highly correlated
timing. The voltage of Cgd is again used relevant for a targeted attack. The region to the labels yet sensitive to noise. Such
as the reference and it generates the Vout for a targeted class in feature space must features open the door for vulnerabilities
here (see Figure 11). have the same orientation with respect to and adversary attacks.
There are three important components the source image for the transferred target We found that a large overlap in the
that are needed while building machine to be attacked successfully. In our set-up non-robust features leads to overlapping
learning techniques such as data, model, we train a white box model or source vulnerabilities across different models
and application. All these elements are model on the same task as the black box or which, in turn, leads to transferability of
connected and operate together. Although the target model. the attacks. For every source image and
we can say that a system is smart, it can Activation attack: Here we have tested set, we randomly sampled another
still be attacked, hence, further creating a designed an activation attack by target image and non-robust images that
lot of security problems. using the properties of adversarial look like the source while having the
Many researchers have shown how transferability. As shown in Figure 13, same hidden features as the target in the
well-trained deep learning algorithms can the white box model (Fw) and a black hidden layer of the model.
be fooled with adversary tags intentionally. box model (Fb) are initially correct. We To sum it all up, it is quite
Most of the work focuses on introducing perturb the source image, which drives understandable that the future computing
minimal perturbation of the data, which the layer L activation of the dog image systems will evolve making them more
results in a maximally devastating effect and perturbs the plane image. The results user-friendly, adaptive, and highly
on the model. In the case of image based can be seen on the right hand side. cost-efficient. It is a holistic scheme
data, it works to disrupt the network’s that integrates the efforts on device,
ability to classify with imperceptible Adversarial vulnerability circuit, architecture, system as well as
perturbations to the images. isolation algorithm. To achieve all of this there is
These attacks are of great concern To minimise the attack transferability, we a lot of work that needs to be done and a
because we do not have a firm studied why attacks even transfer. Inspired lot of challenges to be met.
understanding of the occurrences
taking place inside these interpretable By: Prof. Hai Li
deep models. However, attacks may The author is a professor at Duke University.This article is based on the keynote
also provide a way to study the inner address he gave at the VLSI Design Conference 2022.
workings of such models. There are
two types of attacks—the transfer based Transcribed and curated by Laveesh Kocher
attacks and activation attacks.
Transfer based attack: Figure 12 The article was originally published in the September 2022 issue of Electronics For You.

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 25

Open Source India 2022

Open Source India 2022:

WHERE THE INDUSTRY MET
THE COMMUNITY
Here’s a pictorial overview of all that took place at this mega event for open source
enthusiasts, held in Bengaluru in September this year.

S
eptember 2022 witnessed a mega
open source event – the 19th
edition of the annual, much-
awaited Open Source India (OSI) which
was a physical (on site) event after
a gap of three years. The event was
held on September 29 and 30, 2022,
at NIMHANS Convention Center,
Bengaluru, and focused on cutting-
edge technologies in open source. It
featured packed sessions by over 90
speakers in seven different conference
Powered By tracks, 11 workshops, and a co-hosted
SODA Forum, all of which ran in
Platinum Partners Gold Partners parallel over two days.
Apart from the talks by experts, the
event was also an occasion for exhibitors
to showcase their various community-
Silver Partner Track Partner – DevOps Track centric offerings to thousands of attendees.
The keynote sessions were
delivered by experts from Microsoft,
Associate Partners Exhibiting Partners
Shell, Google Cloud, AWS and SODA
Foundation. The panel discussion 'OSS
SaaS
is not just Code, it's a Community'
generated a lot of interest among
visitors. The other panel discussions
Community Partners were also attended well.
The hallmark of this year’s edition
of Open Source India was the presence
of community projects that were
conceptualised and started in India and
then went global.
Open Source India 2022 was
Media Partners
attended by a large number of open
source supporters from India and
around. The audience ranged from
CIOs and tech heads to evangelists and
academicians. Here is a pictorial round-
up for those who missed the event.

28 | NOVEMber 2022 | OPEN SOURCE For You  | www.OpenSourceForU.com

 Open Source India 2022

1 2

Date: September 29-30, 2022

3
Venue: NIMHANS Convention
Centre, Bengaluru

Total registrations:

10,340
Total attendees:

3,732 4
Total number of sessions:

50+ 4
Total number of speakers:

90+
Pictures of
01: Attendees visiting the Microsoft booth
02: Attendees visiting the Shell booth
03: Attendees visiting the SODA booth
04: Attendees at registration counter
05: Attendees at the Google booth

www.OpenSourceForU.com | OPEN SOURCE For You | NOVEmber 2022 | 29

 Open Source India 2022

8 9 10

6 The keynote sessions

06: Sandeep Alur, Director – Microsoft Technology Center, presenting his insight on open source at AWS
Microsoft Corporation India, talking about ‘Technology 09: Prashanth Subrahmanyam, Lead Developer Advocate,
Innovations leading to Digital Acceleration’ Google Cloud, talking about open source, open data,
07: The keynote session on ‘Digitalization and Disruptive and open cloud
Innovation in the Energy Industry’ by Michiel Stelten- 10: Keynote session on ‘Building Container and Multicloud
pool, Head of Software Engineering, Shell ODF for Tomorrow!’ by Sanil Kumar D., Huawei/SODA
08: Suman Debnath, Principal Developer Advocate, Data Foundation, Chief Architect & Head Data Management
Engineering and Analytics, Amazon Web Services, India, SODA Foundation TOC CoChair

11

The panel discussions 12

11: ‘OSS is not just Code, it's a Community’. Left to right: Ja-
nardan Revuru, JavaScript Evangelist; Vanshika Srivastava,
BoxyHQ; Frederick Noronha, Independent Journalist (Chair);
Vishal Biyani, InfraCloud; Karan MV, GitHub; Sanil Kumar D.,
SODA Foundation; Joshua Poddoku, Apache DevLake
12: ‘The Business of OSS’. Left to right: Prashant Ghildiyal, CEO
& Co-founder, Devtron Inc.; Divyanshu Verma, Co-Founder
and CEO, Redinent Innovations; Vinay Yadav, CEO, Webkul
Software; Dibya Prakash, CTO & Principal Consultant, Neural
Hub (Chair)
13: ‘Open Source Careers: Skills, Expertise and Hiring Trends’.
Left to right: Srivathsa NS, Senior Engineering Director,
Unisys (Chair); Bharat Joshi, Co-founder, VuNet Systems;
Sherin Mirza, Competency Center Lead - Full Stack Develop-
ment, Shell;
Suman Debnath, Principal Developer Advocate, Amazon Web
Services;
Deepak B.V., Founder & Principal Consultant, Cadence Con-
nections
14: ‘From Container to Multi-cloud: Trends and Opportunities’.
Left to right: Dr Keshava Munegowda, Goldman Sachs;
Kamesh Sampath, Harness; Veena Jayan, Siemens Health-
ineers; Sabari Girish S., Cisco; Sanil Kumar D. (Standing),
SODA Foundation
13

30 | NOVEMber 2022 | OPEN SOURCE For You  | www.OpenSourceForU.com

 Open Source India 2022

15

16

14

17 18 19 20

21 22 23

Presenting popular open source projects

24
15: Manoj Kumar, VP Developer Relations, LambdaTest 20: Ankit Nayan, CTO, SigNoz
16: Prashant Ghildiyal, CEO & Co-founder, Devtron Inc. 21: Navendu Pottekkat, Developer Advocate, API7.ai
17: SasiKumar Ganesan, Vice President of Engineering, MOSIP 22: Ruturaj Mohite, Maintainer, CircuitVerse
18: Vishal Biyani, CTO & Founder, InfraCloud 23: Tejaswini Chile, Product Engineer, Chatwoot
19: Saurav Pathak, FOSS Evangelist, Co-Founder Bagisto 24: Nikhil Bhaskaran, Founder-Director, Brainy Pi

www.OpenSourceForU.com | OPEN SOURCE For You | NOVEmber 2022 | 31

 Open Source India 2022

25 26 28

27 29

30 32

31

Sessions and workshopsprojects

25: Shubhang Vishwamitra, Senior Director, Y Media Labs 30: Sherin Mirza, Competency Center Lead, Shell, talking about ‘Using GitHub Actions to
26: A session on ‘Enterprise Class Observability’ by Rajesh K. Jeyapaul, Accelerate DevOps Adoption’
Engineering Lead, IBM Software Labs 31: Kshithij Iyer, Member of Technical Staff, Pure Storage, discussing ‘Enterprise
27: Workshop on ‘What is New on OWASP Top 10 – 2021’ by Lesley Tai, DevOps Strategies for Portworx and K8s’
Senior Solution Engineer, ThriveDX 32: ‘DevOps: Weaving the Maze in the Era of Digital Transformation’ was the topic that
28: Shankar Ramanathan, Site Reliability Engineering, Vonage, speaking Dr Biswajit Mohapatra, Head of Customer Solutions Management, AWS, spoke on
about ‘Running a Managed Service with K8s’ 33: Vivek Sridhar, Senior Developer Advocate, Microsoft, spoke about ‘Decoupling
29: An introductory session on Cloud Native Java, ‘java -jar myapp. Observability from the Application using Dapr!’
jar = CI + GitOps’ by Kamesh Sampath, Director of Developer 34: Workshop on ‘Open Source Workloads on Azure Cosmos DB’ being conducted by
Advocacy, Harness Microsoft

32 | NOVEMber 2022 | OPEN SOURCE For You  | www.OpenSourceForU.com

 Open Source India 2022

33

34
36

35 37

38

39
40

35: A session on ‘Adopting MLSecOps’ by Dibya Prakash, CTO & GitHub, speaking on ‘Let's build from here, in India’
Principal Consultant, Neural Hub 41: A session on ‘Open Source Projects and Legal Issues’ by Biju K. Nair, Lawyer,
36: Binitha MT, Software Principal Engineer, Dell Technologies spoke LegaliTech.in
about ‘Time Series Analysis: Anticipating Future with Darts’ 42: Nandakumar Edamana, Software Architect, Epixel Solutions Pvt Ltd, explain-
37: Subhankar Adak, Senior Software Engineer, Dell EMC ing ‘How the Lazy Become the Prolific using Code Generators’
38: Shivay Lamba, Contributor and Meshmate, Layer5 43: The SODA Forum team
39: Rohit Ghumare, Developer Advocate, Solo.io 44: Mohammad Asif Siddiqui, Staff Engineer, VMware, speaking on ‘Data Man-
40: Karan MV, Senior Manager, International Developer Relations agement in the Container and Multicloud Era’

www.OpenSourceForU.com | OPEN SOURCE For You | NOVEmber 2022 | 33

Open Source India 2022

41 43

42

44 45

46 47

45: A session on ‘Data Mesh – An Architecture Paradigm. Is it a

Must for all Enterprises?’ by Raghvendra Singh Dikhit, Senior
Solutions Architect, Impetus Technologies Pvt Ltd
46: Attendees at a workshop by Jeeva S. Chelladhurai, CEO,
Comorin Consulting Services
47: Workshop on ‘Kubernetes 101: Getting Started’ by Nee-
pendra Khare, Founder and Principal Consultant,
CloudYuga Technologies
48: Workshop on ‘Setting up a Production-ready k8s
Ecosystem in 2 Hours from Scratch’ by the Devtron team
49 49: Manish Kumar, Senior Software Engineer, Shell (Left), and
48 Rohit Srivastava, Senior Software Engineer, Shell (Right),
speaking on ‘Green Software@Shell powered by Open Source’

34 | NOVEMber 2022 | OPEN SOURCE For You  | www.OpenSourceForU.com

 Open Source India 2022

50 51 52

54 55

53

56 57

58 59
60

50: A session on ‘OCaml 5: Language, Platform and Ecosystem’ by Shakthi 55: Ashay Krishna, Director of Engineering, Microsoft (Left), and Madhava Gaikwad,
Kannan, Senior Software Engineer, Tarides India Pvt Ltd Senior Software Engineer, Microsoft (Right), introduced attendees to ‘SONIC: An
51: Kundan Kumar, Senior Member of Technical Staff, VMware (Left), and Operating System to Manage your Networking Hardware’
Prasad Paple, Operating System Developer, IBM (Right), speaking on 56: Sachin Apagundi, Software Principal Engineer, Dell Technologies, speaking on ‘Us-
‘Linux Synchronization Journey to Making Lockless Software’ ing Ansible and Terraform in System Management’
52: Closing remarks for SODA Forum by P Sainath Mathur, Practice Head 57: A session on ‘Centralized Authentication System in a HPC Cluster’ by Deepika K.,
Compute, Storage And Peripherals, ER&D, Wipro Technologies Principal Software Engineer, Dell R&D
53: ‘Getting Started with Bottlerocket’ was the topic for discussion by Mani 58: A workshop on ‘Event-Driven Microservices: Challenges and Strategies’ by Krishna
Chandrasekaran, Lead Architect, Amazon Internet Services Pvt Ltd Mohan Koyya, Proprietor & Principal Consultant, Glarimy Technology Services
54: Vinay Rajagopal, ISV Technology Lead, Red Hat, speaking on ‘Multicloud 59: Balaji Seetharaman, Senior Software Engineer, Extreme Networks
Communication for Kubernetes’ 60: Prashant Mishra, Senior Developer Programs Engineer, Google India, speaking on
‘Faster SELECT Queries Using Open Source java-bigquery’

www.OpenSourceForU.com | OPEN SOURCE For You | NOVEmber 2022 | 35

Focus
Internet Of Thing
Industry 4.0 and Embedded Internet of IoT Analytics: Making
Things: An Introduction Sense of Big Data
This article covers the characteristics
of IoT analytics and the challenges
faced when adopting it in the
industry. It also presents real world
use cases of IoT analytics.
Read more on page......................48

A Complete Guide to
Getting Started with
Arduino
The fourth industrial revolution, commonly referred to as Industry 4.0,
is all about making businesses smarter. It is focused on deepening the Building DIY projects with Arduino is
impact of digital technologies by making machines more autonomous super easy and a lot of fun. However, it
and enabling them to communicate with one another. These machines may not always be as straightforward
can now manage enormous amounts of data in ways that humans simply for beginners with absolutely no
cannot handle. This article introduces you to Embedded Internet of background knowledge in hardware
Things (EIoT) and Industry 4.0. Read more on page......................39 or programming. In this article, we
cover everything you need to know for
getting started with your first Arduino
How IoT can make the Workplace Intelligent project. That includes an overview
of the Arduino board, the IDE, the
programming language and a simple
project to wet your beak.
Read more on page......................53

Connecting Internet
of Things Devices on
Thinger.io
This article is about connecting
various IoT devices on Thinger.io,
an IoT platform. This step-by-step
tutorial will guide you on how to
connect a development board to
It’s well-known that an intelligent workplace enhances employee the platform and start sending data
productivity. The Internet of Things plays a big role in making a workplace from the device.
smart and intelligent. Let’s see how. Read more on page......................45 Read more on page......................56
 Focus

Industry 4.0
and Embedded Internet of
Things: An Introduction

The fourth industrial revolution,

commonly referred to as
Industry 4.0, is all about making
businesses smarter. It is focused
on deepening the impact of
digital technologies by making
machines more autonomous and
enabling them to communicate
with one another. These
machines can now manage
enormous amounts of data in
ways that humans simply cannot
handle. This article introduces
you to Embedded Internet of
Things (EIoT) and Industry 4.0.

T
he third industrial revolution manufacturing from planning to Some of the key technologies
was focused on converting delivery as part of a larger concept associated with Industry 4.0 are:
mechanical and analogue known as digital transformation. It ƒƒ Industrial Internet of Things
processes to digital ones to be ready includes solutions for deep analytics, ƒƒ Smart wearable gadgets
for advanced automation. Similar to shop floor data sensors, smart ƒƒ Smart cities and advanced
the transition from steam to electricity warehouses, simulated modifications, monitoring
in the second industrial revolution, as well as product and asset tracking. ƒƒ Advanced robotics
the advent of Industry 4.0 technology Industry 4.0 technologies assist ƒƒ Digital twins
marks a major transformation in how manufacturers in bridging the ƒƒ Big Data analytics
companies conduct their operations. gap between previously distinct ƒƒ E-governance
The effects of Industry 4.0 processes and a more transparent, ƒƒ Additive manufacturing (AM)
on production are extensive. It is viewable perspective across the ƒƒ Cloud computing
being used to increase operational entire business with a wealth of ƒƒ Augmented reality (AR) and virtual
effectiveness, improve demand useful insights. The emergence of reality (VR)
forecasting, eliminate data silos, digital solutions and cutting-edge ƒƒ Quantum computing based systems
engage in predictive maintenance, technology, which are frequently ƒƒ Simulation platforms and suites
provide virtual training to employees, linked to Industry 4.0, have made By integrating formerly
and more. Industry 4.0 encompasses this convergence possible. disjointed systems and processes with

www.OpenSourceForU.com | OPEN SOURCE For You | november 2022 | 39

Focus

interconnected computer systems

across the value and supply chain, these
technologies are assisting in the digital
transformation of manufacturing.
Adopting Industry 4.0, digital
manufacturing, and the associated Biometrics
interconnectivity enables businesses
to reap a variety of benefits, including
improved operational performance and
increased agility.

Embedded Internet of Things

(EIoT) and Industry 4.0
Embedded Internet of Things (EIoT)
refers to the integration of embedded
devices (including sensors and open
source hardware) with IoT based
platforms and communication channels
to enable real-time communication for
varied applications. EIoT with Industry
4.0 is a technology shift of traditional
engineering and governance to smart
deployments. The three technical
themes (connectivity, intelligence,
and flexible automation) driving
this transformation are collectively
referred to as Industry 4.0, or the Figure 1: Key segments of Industry 4.0
fourth industrial revolution with
embedded devices. Table 1: Platforms and software suites for simulation of Industry 4.0 applications
The fourth industrial revolution Platform/Software suite URL
represents breakthroughs in
Cloud MQTT cloudmqtt.com
technologies, such as robots, intelligent
systems, nanotechnology, quantum Mosquitto MQTT Broker mosquitto.org
computing, microbiology, the FlowHub flowhub.io
Internet of Things, 3D printing, and Sumo eclipse.org/sumo
autonomous cars.
NoFloJS noflowjs.org
In order to build a cyber-physical
environment, Industry 4.0 combines PraxisLive praxislive.org
OT (operational technology) and Netron github.com/lutzroeder/netron
IT. A number of software platforms Ryven gentle-sand-0114c7110.azurestaticapps.net/
and libraries can be used for the
PyFlow wonderworks-software.github.io/PyFlow/
simulation of EIoT and Industry 4.0
applications (Table 1). Wyliodrin wyliodrin.com
Flutter Wireless flutterwireless.com
Some use cases SiteWhere sitewhere.io
In Industry 4.0 and EIoT simulations,
DSA iot-dsa.org
there is a need to integrate lightweight
protocols so that low-powered and Zetta zettajs.org
smart gadgets can communicate Node Red nodered.org
effectively. MQTT is a key protocol IoTivity iotivity.org
that is used in Industry 4.0 and IoT
Open IoT openiot.eu
based deployments.

40 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

Focus

There are primarily two types Cloud based MQTT subscribe message queueing model
of objects in IoT communication — URL: cloudmqtt.com ƒ Fully managed Mosquitto servers in
publisher and subscriber. Publishing CloudMQTT, one of the most the cloud
refers to the transmission of data widely used MQTT brokers, can be To establish a real-time connection
signals — the process of moving data used for device communication and between CloudMQTT and a device,
from a computer to an external device. implementation of Industry 4.0 based IoT hardware or smartphones can
The receiver system is referred to as scenarios. It can be used to implement use authentication information from
the subscriber in IoT communication. IoT scenarios for a variety of applications CloudMQTT.
Publisher and subscriber are including smart toll plaza, smart cities,
terms used to describe the source smart homes, and smart offices. Traffic simulation using Eclipse SUMO
and recipient of a data signal, For researchers and academicians, URL: eclipse.org/sumo
respectively. The MQTT broker, CloudMQTT offers a free package Smart vehicular communication in
also known as an IoT server, serves called ‘Cute Cat’ that allows five smart cities and e-governance are
as the focal point and regulates users or connections for IoT devices. key application areas of Industry
communication between publishers Using an active Google account, 4.0. Eclipse SUMO (Simulation of
and subscribers. It manages the data free instances can be built on the Urban MObility) is very effective
that publishers broadcast for the CloudMQTT network. in simulating a traffic scenario. It
other devices known as subscribers. The features integrated in Cloud integrates a range of features to
For instance, information may be MQTT include: simulate smart vehicular environment,
sent from a temperature sensor to ƒ Open source message broker that and can work with multiple vehicles
a farmer’s mobile phone. In this implements the MQTT protocol of different types in different
instance, the temperature sensor ƒ 24x7 availability of MQTT broker environmental conditions.
(publisher) is the source of the data ƒ Cloud compatibility SUMO is free and open source.
signals, and the farmer’s gadget is ƒ Lightweight and high performance; Since its introduction in 2001, it
the receiver (subscriber). integrates with low power single has enabled the modelling of multi-
These MQTT broker applications boards like Raspberry Pi, Arduino, mode traffic networks, which include
can be installed on a variety of and ESP8266 as well as computers pedestrians, public transportation, and
specialised hardware, including and servers private vehicles. Numerous auxiliary
Raspberry Pi, Arduino, laptops, ƒ Integration of lightweight methods tools, such as network import, route
servers, and many others. for messaging using a publish and calculations, visualisation, and emission

Figure 2: Cloud based MQTT for simulating an IoT scenario

42 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 Focus

Figure 3: Interfacing with CloudMQTT and messaging panel

Figure 4: SUMO simulator for smart traffic scenarios

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 43

Focus

Figure 5: Smart vehicular simulation in SUMO

calculation, are included with SUMO ƒ Network import make wiser decisions. The only way
and automate fundamental processes ƒ Demand generation for manufacturers to stay ahead of
for the development, execution, and ƒ Performance the curve in this new digital era is by
evaluation of traffic simulations. Custom ƒ Portability embracing Industry 4.0.
models can be added to SUMO, and it The manufacturing sector is Researchers and practitioners
offers a number of APIs for controlling experiencing exciting times right now of Industry 4.0 can use free and
the simulation from a distance. as new digital technologies emerge. open source platforms to simulate
The key features of SUMO are: Industry 4.0 is also making it easier and experiment with its real world
ƒ Free and open source for humans and robots to collaborate, applications. Such simulation sessions
ƒ Automated driving enabling firms to gain more knowledge, will help analyse the outcome in real-
ƒ Vehicle communication lower the possibility of error, and time before actual deployment.
ƒ Traffic management
ƒ Multimodal traffic Dr Gaurav Kumar
ƒ Microscopic simulation The author is associated with various academic and research institutes for
ƒ Online interactions delivering expert lectures and conducting technical workshops on the latest
technologies and tools.
ƒ Traffic lights

THE COMPLETE MAGAZINE ON OPEN SOURCE The latest from the Open Source world is here.
OpenSourceForU.com
Join the community at facebook.com/opensourceforu
Follow us on Twitter @OpenSourceForU

44 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 Focus

How IoT can make

the Workplace Intelligent
It’s well-known that an intelligent workplace ƒ Air quality and temperature
enhances employee productivity. The Internet of management based on occupant needs
Things plays a big role in making a workplace smart ƒ Automatic switch off of air
and intelligent. Let’s see how. conditioning and heating to avoid
unnecessary energy use
ƒ Control of humidity levels
Networked lighting: Digital
integration of lighting fixtures with
sensors and software can create
an environment that maximises
occupants’ comfort and saves energy.
Such lighting helps to:
ƒ Adjust light to occupancy levels
ƒ Control lighting as per the personal
choice of occupants
ƒ Save energy
ƒ Adjust light based on ambient
conditions, and control
window shades
Occupancy sensors: These
identify the utilisation of a workplace
position through movement and/or

E
nterprises demand a flexible, IoT for an intelligent workplace thermal measurement. A sensor can be
intelligent, and connected An intelligent workplace uses automated positioned at any location occupied by a
workplace as it enables its processes to control an office’s operation person. It helps in monitoring:
occupants to work smarter. It also helps including heating, ventilation, air ƒ Office utilisation
in enhancing employee productivity conditioning, lighting, security and ƒ Desk utilisation
and experience, leading to better other systems. A smart office uses ƒ Meeting room utilisation
employee retention and helping attract sensors, actuators and microchips to ƒ Heat maps to illustrate occupancy
a new talent pool. Internet of Things collect data and manage it according of office
(IoT) enables an intelligent office to to business functions and services. ƒ People counter – counts real-time
utilise physical devices for collecting This infrastructure helps real estate occupancy
and sharing data. It helps enterprises and facility managers improve asset ƒ Room sensor – detects if a room is
to design flexible, activity-centric reliability and performance, which being used or not
workspaces to better meet employees’ reduces energy use, optimises how space Access and security: This involves
evolving needs by analysing IoT is used and minimises the environmental use of wearables (smartwatch, smart
sensors and connected system data on impact of buildings. badge, etc) to authenticate employees
how space is being utilised. It enriches Heating, ventilation, and air for access to buildings, meeting rooms,
employee journey maps (such as conditioning (HVAC): Enterprises equipment, devices and data. Smart
seamless access, automatic room and can digitally integrate an office HVAC wristbands could become ID badge
desk booking, and colleague or way system with sensors and software. They replacements (or smart ID badges could
finding) from the moment they enter can provide mobile applications and be created) in the office space. For
the office, and creates a conducive smart technologies that allow employees example, the building will recognise
work environment by using smart to customise the temperature and lighting your car when you arrive, authenticate
temperature and light controls. in their workspaces. This helps in: it, and then direct you to a parking

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 45

Focus

with their card, thus providing a seamless

experience. Sensors will recognise
when that person enters or exits a room,
which can assist in knowing when
meeting rooms are in use. In case people
don’t show up for a meeting or forget
to cancel it (which is common with
recurring meetings), these sensors can
automatically set the meeting rooms free
to be booked by other employees.
Wellness: Implementing wearable
programs to increase employee health,
wellness and productivity helps to:
ƒ Promote healthy habits
ƒ Gamify wellness for employees
Figure 1: IoT use cases for the intelligent workspace (making participation a fun and
conduct meetings, a smart workspace can engaging ‘game’)
spot, thus providing a seamless user proactively carry out such routine work. ƒ Give employees benefits for
experience. Networked cameras can Employees can use a digital workspace participation
stream live video straight to central mobile app to book rooms and seats. Eighty per cent of employees using
control, computer and mobile devices. Integration with employees’ calendars healthy wellness programs report being
They help to: enables the system to send smart prompts more engaged at work.
ƒ Monitor remote locations anywhere to devices. This helps to: Wayfinding and people finding:
ƒ Give office access to staff remotely ƒ Book a room anywhere and in real time Knowing the location of a person
in real-time ƒ Sense when a room is occupied or thing unlocks many contextual
ƒ Monitor real-time occupancy data ƒ Know when a room is available digital workplace opportunities;
Meeting room/workplace ƒ Eliminate ghost meetings examples include asset tracking,
reservation: This enables occupants ƒ Find the right room to fit the needs of improving worker safety, understanding
to simplify workplace meeting room every meeting employee behaviour, predicting
reservations. Instead of their wasting With sensor technology, an individual needs, and providing new services in
time looking for places to work and doesn’t have to manually sign in or out the smart office.
ThingsBoard Smart Office Solution

Workplace1

MQTT

Workplace2
Data Real time
Data
Dashboards Alarms

Meeting Room1
Storage Rule Engine

Server Room

Figure 2: ThingsBoard smart office solution

46 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 Focus

Figure 3: Smart office dashboard

ƒ Advanced and flexible visualisation metrics necessary for effective and
Open source intelligent of real-time and historical data proactive office management. As part
office solutions ƒ Customisable end user dashboards of this solution, there are smart office
There are many open source platforms to share the monitoring results dashboards that display data from
available that can be used to build ƒ On-premises and cloud multiple devices. These dashboards
intelligent office solutions. Mainflux, deployment options may be used to:
ThingsBoard, DeviceHive, Thinger, ƒ Remote control and OTA (over the ƒ Observe office sensors and their
Zetta, OpenRemote, Kaa, Fiware, air) updates for IoT devices location
ThingsConnect, WS02Rap are popular ƒ Customisable mobile applications ƒ Browse indoor temperature and
open source IoT platforms. with minimum coding efforts power consumption history
ThingsBoard: This platform ThingsBoard provides production- ƒ Monitor temperature alarms
dramatically reduces the effort needed ready server infrastructure to connect ƒ Control HVAC (requires
to create smart office solutions. It is IoT devices, as well as store, analyse connected device)
widely used for: and share the collected data. ƒ Observe specific details for
ƒ Smart energy management. The ThingsBoard smart office each sensor
ƒ Access logging and control template represents a basic workspace IoT devices also accept commands
ƒ Indoor tracking of employees monitoring and management solution. from the dashboard. For example,
ƒ Occupancy of meeting and wash With this layout (Figure 2), we get an HVAC sensors accept commands to
rooms interactive dashboard with the ability enable/disable air conditioning as well
ƒ Indoor air quality and HVAC to control the HVAC system, and as set target temperature. The commands
monitoring have a high-level report of the key are sent using the platform API.
By leveraging this IoT platform,
we can save up to 90 per cent of the By: Narasimha Sekhar Kakaraparthi
development time needed to build smart
The author has more than 27 years of software industry experience in product
office solutions. The benefits are: development for hybrid cloud and telecommunications. He is currently working as
ƒ Reliable and fault-tolerant data principal architect for virtual desktops and end user services at Wipro Technologies.
collection from IoT devices and He is a senior member of Wipro’s eminent architects group called Distinguished
Member of Technical Staff (DMTS).
sensors
ƒ Powerful rule engine to process
collected data for valuable insights Disclaimer: The views expressed in this article are that of the author and Wipro
like setting of alarms does not subscribe to the substance, veracity or truthfulness of the said opinion.

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 47

Focus

IoT Analytics:
Making Sense of Big Data
This article covers the characteristics of IoT analytics and the challenges faced when
adopting it in the industry. It also presents real world use cases of IoT analytics.

T
ƒ
he standard definitions of
IoT are:
“A network of physical objects or
ƒ “An IoT system is a network of
networks where, typically, a massive
number of objects, things, sensors
Internet of Things (IoT)
analytics assesses the wide range
of data collected from IoT devices.
things accessed through the internet, or devices are connected through It analyses vast quantities of data
as defined by technology analysts communications and information and produces useful information
and visionaries. These things infrastructure to provide value- called Big Data. It addresses large,
contain embedded technology to added services via intelligent data diverse, complex, longitudinal, and/
interact with internal states or the processing and management for or distributed data sets generated
external environment. In other different applications (e.g., smart from instruments, sensors, internet
words, when things can sense and cities, smart health, smart grid, smart transactions, emails, videos,
communicate, it changes how and home, smart transportation, and click streams, and/or all other
where the decisions are being made smart shopping).” — IEEE Internet digital sources available. The
and who makes them.” – Cisco of Things Journal processing and integration of this
ƒ “The IoT can be viewed as a global Overall, the Internet of Things heterogeneously generated data is
infrastructure for the information (IoT) is a network of devices which can done by IoT analytics.
society, enabling advanced services sense, accumulate, and transfer data IoT analytics helps gain
by interconnecting (physical and over the internet without any human insights into the operations of an
virtual) things based on existing intervention. IoT and data remain enterprise, boosts productivity,
and evolving interoperable intrinsically linked together. Data increases the quality and flexibility
information and communication consumed and produced keeps growing of operations, and minimises
technologies.” – ITU at an ever-expanding rate. material consumption.

48 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 Focus

Industry adoption of IoT demands highly elastic, highly available (information), and then contextualizing
According to Gartner, more than data processing and massive data the results (knowledge) and benefiting
30.9 billion IoT devices will be in storage solutions for the enterprise. from historical experience (wisdom).
use worldwide by 2025. A forecast Variety: IoT data is primarily The maturity of the processed raw
by International Data Corporation unstructured or semi-structured. Such data depends on the design of the
(IDC) estimates that there will be 41.6 data types require additional processing analytics solution.
billion IoT devices in 2025, capable of to derive meaning and the supporting Temporal correlation: Applying
generating 79.4 zettabytes (ZB) of data. metadata. Further complexity arises analytics closer to where the data
The global Internet of Things when data from a known source is generated reduces the burden of
market is estimated to rise from US$ changes without notice. correlation. It helps IoT systems to
478.36 billion in 2022 to US$ 2465.26 Veracity: Veracity refers to correlate the data between multiple sensors
billion by 2029 at a CAGR of 26.4 per the quality or fidelity of data. and process control system states.
cent during the forecast period, as per Assessment of data that enters IoT data Provenance: Performing analytics
Global Wire. environments for quality can lead to at a lower architecture tier maintains
According to McKinsey, by data processing activities to resolve the genuine sources of the data as it
2030, the IoT market will be valued invalid data and remove noise. Data progresses through the IoT layers.
between US$ 5.5 trillion and US$ 12.6 acquired in a controlled manner usually
trillion globally, including the value contains less noise than data acquired Principles of IoT analytics
captured by consumers of IoT products via uncontrolled sources. Data with Architecture principles provide a basis
and services. a high signal-to-noise ratio has more for decision-making when developing
veracity than data with a lower ratio. IoT data analytics solutions and design.
Key characteristics of Value: There is a range of They form a structured set of ideas that
IoT analytics quantitative and investigative collectively define and guide solution
IoT analytics integrates business techniques to derive value from data. architecture development, harmonising
intelligence and data analytics, The cost of data storage and computing decision-making across an organisation.
including conventional and Big Data. Its has fallen exponentially, thus providing The principles that guide enterprises
characteristics are explained briefly below. an abundance of data for statistical in the way they approach IoT analytics
Volume: The volume of data sampling and other techniques. data are:
processed by IoT data solutions is Response time and reliability: ƒ Data is an asset: Data is an asset
substantial and ever-growing. High data The essential requirement of IoT data that has a specific and measurable
volumes demand massive data storage, analytics is better response time and value for the enterprise.
data processing, data preparation, reliability. The solution is built on three ƒ Data is shared: Data must be shared
data curation, and data management fundamental factors: deterministic across the enterprise and its business
processes. Big Data requires processing analytics, data computation, and quick units. Users have access to the data
high volumes of data, that is, data of response time. that is necessary to perform their
unknown value — for example, network Bandwidth: The massive amount activities.
traffic, sensor-enabled equipment of data generated by IoT devices is ƒ Data trustees: Each data element has
capturing data, and many more. Some of gathered from control systems. Handling trustees accountable for data quality.
the data sources that are responsible for of this data requires better bandwidth. ƒ Common vocabulary and data
generating high data volumes are: The increased network and infrastructure definitions: Data definition is
ƒ Scientific and research experiments required to transfer data from one consistent, and the taxonomy is
ƒ Sensors, such as temperature, domain to another is compensated by understandable throughout the
pressure and GPS sensors creating valuable insights. enterprise.
ƒ RFIDs Security: Transferring raw data ƒ Data security: Data must be
ƒ Smart meters and telematics outside of controlled areas comes with protected from unauthorised users
Velocity: In IoT data analytics security concerns and the associated costs. and disclosure.
environments, massive data gets It may be more efficient to perform data ƒ Data privacy: Privacy and data
accumulated in a short time, as sensors analytics locally and share the necessary protection is considered throughout
collect data pretty quickly. The velocity summaries with other domains. the life cycle of a Big Data project.
of data arrival within an enterprise’s Analytics maturity: Analytics All data sharing conforms to the
perimeter is directly proportional to involves the processing of raw data relevant regulatory and business
the time taken to process the data. It (measurements) into descriptions requirements.

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 49

Focus

Data integrity and the transparency security requirements and managing ƒ AI-driven analytics: Enrich
of processes: Each party to a Big Data privacy sensitivities. The data generated and explore prescriptive and
analytics project must be aware of and by IoT data analytics needs to be predictive analytics data to deliver
abide by their responsibilities regarding encrypted, secured, and stored. actionable insights.
the provision of source data and the Data processing: IoT data analytics Figure 1 shows the logical
obligation to establish and maintain systems need to follow the steps listed architecture of the IoT analytics system
adequate controls over the use of below for data processing: with key components and layers.
personal or other sensitive data. ƒ Collection of data aggregated from While there exist many standard
a variety of sources, in a range of logical architectures for IoT analytics,
Key challenges of IoT analytics formats, and at multiple frequencies this one is based on my practical
applications ƒ Understanding the data and experience across various domains and
Heterogeneity: IoT data streams tend categorising it technologies.
to be multi-modal and heterogeneous ƒ Preparation of data The various layers for an enterprise
in their formats, semantics, and ƒ Processing data from different application are classified as:
velocities. To access IoT data sources, sources ƒ Data source layer
database drivers and adaptors need to ƒ Storing the information in time ƒ Edge layer
be deployed in Big Data applications. series for analysis ƒ Analytics layer
These applications incorporate features ƒ Analysing the results ƒ Application layer
like interoperability, transaction ƒ Building applications to fulfil Data source layer: This layer
management, and common data formats business processes consists of IoT devices and IoT
to address the heterogeneity. gateways. Sensors and actuators are
Data quality: Different IoT analytics architecture essential components of IoT.
manufacturers provide various sensor The requirements for an IoT analytics Sensors sense the environment
categories and types. Substandard and system with respect to its architecture to monitor and detect changes of a
unreliable sensors generate noisy and are as follows. particular attribute and collect the data,
incomplete IoT data streams. Usage of ƒ Context: Capture the context at all such as a change in temperature, heat,
these data streams leads to uncertainty times, 24 hours a day, 365 days a year. or cold. The output of the sensor is an
and inaccuracy in IoT analytics. ƒ Standards: Leverage standard electrical signal.
To improve the data quality and protocols of communication Actuators are devices used to control
eliminate the noise, statistical models between IoT devices and the system. They receive electrical
and probability based applications enterprise systems. output from sensors and produce a
are developed to feed into IoT data ƒ Scalability: Always respond even physical change. For example, if the
analytics applications to generate an during increased load; increase sensor detects a change in temperature in
accurate output. capacity proportionally as resources a room, the actuator can turn a device on
Real-time data sets: IoT devices are added. or off to maintain and control it.
provide high-speed and real-time ƒ Data management: Efficiently These sensors and actuators
data. In the Big Data ecosystem, data manage enormous volumes of data. establish a link between the physical
streaming platforms consume real-time ƒ Connectivity: Provide high network and digital world via an IoT gateway.
data sets to generate valuable insights. connectivity for large data payloads This layer basically deals with the
Location and time: IoT data comes and continuous streaming. gathering of specific data from data
with temporal and spatial metadata ƒ Security: Move and encrypt sources. This data is then passed to the
information, directly associated with information securely even as edge layer so that it can be securely
its business value in each application IoT introduces new risks and transmitted to the analytics layer.
context. These spatial and temporal vulnerabilities. Edge layer: The edge layer is an
dimensions can serve as a basis for ƒ Interoperability: Network all entry point for all things or devices
dynamically selecting and filtering systems together and ensure to capture and process the data. The
streams towards analytics applications interoperability of all data. layer consists of a data store, services
for specific timelines and locations. ƒ Device management: An ability to analyse the data streamed from
Security and privacy: IoT to connect to devices remotely the IoT devices and execute AI-
applications collect and process at scale to manage them such as driven analytics locally in real-time,
personal data and device data. IoT updating security credentials and security mechanisms (including data
analytics helps in exhibiting strong IoT firmware updates. encryption), and control applications.

50 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 Focus

Figure 1: IoT analytics logical reference architecture

It consists of an IoT gateway that Application layer: This layer real-time. It can facilitate the production
facilitates bi-directional communication acts as an interface to interact with flow in a manufacturing plant, as
with the platform and enterprise applications for end users. Devices like IoT devices automatically monitor
systems. It also implements edge smartphones, computers and tablets development cycles and manage
intelligence with different processing are used as a medium to make IoT inventories.
capabilities, enabling connectivity applications interactive. IoT for supply chain
between devices, routing, filtering, and management: Manufacturers use IoT
protocol and identity translation. Real world use cases to monitor their supply chain and get
Analytics layer: The primary of IoT analytics estimates of the available resources.
purpose of this layer is to interface with Implementation of IoT analytics based This includes information regarding
the edge layer to receive, store, process, solutions helps enterprises eliminate the status of the work on the floor,
and analyse the data. In addition to redundant assets and products, leading equipment collection, and the delivery
data acquisition capabilities, it offers to better operational efficiency and date of required materials.
AI-driven analytics for intelligence cost savings. IoT for smart homes: IoT helps
and visualisation, device management, IoT for digital government: to analyse and optimise use of devices
security, connectivity and network The vision of any government in the in a smart home. This can mean
management, and operations. digital era is to develop connected simple switching off or dimming of
The data from devices is collected and intelligent IoT based systems that lights, and modifying multiple home
and aggregated from the IoT device contribute to the economy, improve settings to optimise energy use. It also
gateway. Real-time AI-driven analytics citizen satisfaction, and lead to a safer helps discover problematic energy
and visualisation combined with society and environment sustainability. consumption by older and damaged
machine and deep learning models are Processing of data feeds from various appliances, or faulty system components.
applied to the data to derive intelligence. sources like weather and environmental IoT for media: IoT based
Machine learning is also performed sensors, traffic signal cameras, parking applications in media and advertising
over the collected device telemetry zones, etc, leads to improved government- lead to a customised experience
data, and this data is blended with IoT agency coordination and better in which the system analyses and
other enterprise data or secondary services to citizens. responds to the needs and interests
sources of telemetry from other IoT for manufacturing: IoT in of each customer. It helps in
systems or organisations. manufacturing allows richer insights in understanding the customer behaviour

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 51

Focus

patterns, buying habits, preferences, ƒ Data from other facilities: IoT ƒ Livestock monitoring: Farm
culture and other characteristics. also enhances the knowledge animals can be monitored using
IoT for healthcare: IoT based base used to establish education IoT devices to detect potential
applications/systems of healthcare standards and practices. It signs of disease. The data can
enhance the traditional technology introduces large, high-quality, be analysed from the central
used today. These devices help real-world data sets into repository, and the farmers can
increase the accuracy of the medical educational design. share relevant information.
data collected from a large set IoT for agriculture: IoT IoT analytics has a number of
of devices connected to various minimises human intervention in benefits that are helping transform
applications and systems. They farming analysis and monitoring. businesses and different industries.
also help gather data to improve IoT based systems detect changes in ƒ Greater transparency; transaction
the precision of the medical care crops, the soil, etc. histories are becoming
delivered through sophisticated ƒ Crop monitoring: IoT based more transparent
integrated healthcare systems. devices can be used to monitor ƒ Increased visibility and better
ƒ IoT devices give direct 24/7 x crops and the health of plants using control, resulting in quicker
365 access to the patient in a the data collected from them. These decision-making
less intrusive way. IoT based devices can also monitor pests and ƒ Flexible scaling of business needs
analytics and automation allow diseases proactively in the early and expansion in new markets
health providers to access patient stages of infestation. ƒ Reduction in operational costs
reports before they arrive at the ƒ Food safety: IoT helps connect the due to automation and better
hospital. They also help improve entire supply chain for ensuring resource utilisation
responsiveness in emergency the food is safe when it reaches ƒ New revenue streams,
healthcare. the customer. resulting from the clearing of
ƒ IoT-driven systems are used ƒ Climate monitoring: IoT based operational challenges
for continuous monitoring of devices are used to monitor ƒ Enhanced customer experience
a patient’s health status. These temperature, humidity, the based on an analysis of the
employ sensors to collect the intensity of light, and moisture of purchase history
physiological information the soil. The data is ingested into IoT is one of the most significant
analysed and stored on the cloud. the central repository to trigger sources of Big Data, which is
Doctors access this information alerts and automate water, air, and rendered useless without analytics
for further analysis and review. crop control. power. IoT analytics comes into play
This way, a continuous automated ƒ Logistics monitoring: Location when voluminous amounts of data
flow of information is provided. based IoT devices can be needs to be processed, transformed,
ƒ Patients’ health data is captured used to track vegetables and and analysed at a high frequency. It
using various sensors, analysed and other farm products during helps to provide advanced services
sent to the medical professional for transport and storage. across many domains.
proper remote medical assistance.
IoT for education: IoT customises
and enhances education by optimising
Acknowledgements
content and forms of delivery.
ƒ Student tracking: IoT facilitates the I would like to thank Santosh Shinde of BTIS, enterprise architecture division of HCL
Technologies Ltd, for giving the required time and support while writing this article
customisation of education to give as part of architecture practice efforts.
students access to what they need.
ƒ Instructor tracking: Educators
can use the IoT platform for By: Dr Gopala Krishna Behara
customised instructional designs
The author is an enterprise architect in BTIS, the enterprise architecture division of
for their students.
HCL Technologies Ltd. He has 26 years of experience in the IT industry.
ƒ Facility monitoring and
maintenance: IoT based
applications improve the Disclaimer: The views expressed in this article are that of the author and HCL
professional development Technologies Ltd does not subscribe to the substance, veracity or truthfulness of
of educators. the said opinion.

52 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 Focus

A Complete Guide to Getting

Started with Arduino
Building DIY projects with Arduino is super easy and a lot of fun. However, it may not
always be as straightforward for beginners with absolutely no background knowledge
in hardware or programming. In this article, we cover everything you need to know for
getting started with your first Arduino project. That includes an overview of the Arduino
board, the IDE, the programming language and a simple project to wet your beak.

a
rduino is an open source 300,000 Arduino boards were in that not only help make life better
project developed for circulation globally. By 2013, that but are also artistic and innovative
students, hobbyists, artists number was 700,000. in their approach.
and programmers who are interested Today, the Arduino project is a Consider life before Arduino. If
in combining hardware and software mammoth with 100+ devices consisting you wanted to build a light that turns on
to make DIY (Do It Yourself) devices of various types of boards, shields, automatically, you would have to design
and utilities. It was created in 2005 carriers, kits and other accessories. All the entire circuit diagram yourself.
by Massimo Banzi and his colleagues the information regarding the Arduino Solder it all together on a board. Write
and students at the Interaction Design project is available at its official low level machine code yourself and
Institute Ivrea, Italy. The primary website arduino.cc create your own mechanism to power the
purpose behind the creation of the So what makes Arduino so whole thing up.
Arduino project was to provide attractive? Go to YouTube and search With Arduino, you can do all that
enthusiasts and professionals with a ‘Cool Arduino Projects’ and the answer in 5 minutes. Just connect a few pins
platform consisting of a programming will become glaringly obvious. Arduino here and there using an Arduino and
language, an IDE, and a printed circuit offers developers the freedom and a breadboard. Write the code in a
board to create simple and effective autonomy to get creative with their language that is as simple as C and
DIY projects. The project quickly circuit design, and write innovative run the code with the click of a button.
gained steam and by 2011, an estimated code to make DIY devices and utilities That’s it! Arduino massively simplifies
your life as an engineer. What’s more,
it is designed keeping students in
mind, which means it is cheap to buy.

The anatomy of an
Arduino device
Before we get to the code part, it is
necessary to have a basic understanding
of the Arduino board. Every Arduino
board is differently constructed based
on the purpose it serves. For this article,
we shall be using the beginner-friendly
and most popular Arduino board, i.e.,
the Arduino Uno. Figure 1 shows the
anatomy of an Arduino Uno.
There are four major components
of an Arduino Uno board — digital
pins, analog pins, USB connector, and
battery power.
Digital pins: As shown in Figure 1,
Figure 1: Arduino Uno there are 14 digital pins in the Arduino

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 53

Focus

Uno. Digital pins can be used both Monitor window, as

as INPUT and OUTPUT ports. They shown in Figure 3.
have two states, i.e., HIGH and LOW.
A HIGH state indicates the presence Arduino
of current in the pin whereas the LOW programming
state indicates the opposite. Digital pins constructs
can be used for various purposes such Before we begin to
as blinking of LED lights. When set to write and execute our
HIGH, the light will turn on and when first Arduino code, let
set to LOW, it will turn off. This can be us get familiar with the
done in a loop to create a blinking effect. Arduino programming
Analog pins: There are six analog language. In this
pins in an Arduino Uno, as shown in section we will
Figure 1. They are indexed as A0 to A5 cover all the basic
on the board. These pins are used for constructs used in the
reading sensor data. A lot of the work Arduino programming
done in your Arduino projects will language. These are
require sensor data. For instance, if you Figure 2: Default view of the Arduino IDE more than enough to
are creating a light that turns on when it get you started with writing beginner
detects movement, then that will require each shortcut in a left to right order. to intermediate level code for your
you to use a sensor to detect motion. 1. Compile: This is the first button Arduino projects.
The analog pins can register readings from the left used for compiling and The basic programming constructs
between the values of 0 and 1023. verifying the code for any syntactic are listed below.
USB connector: The USB or logical errors. 1. Int: Used for integer values such as
connector helps you to connect your 2. Upload: The second button from the 12345
Arduino to the computer so you may left, it is used for uploading the code 2. Float: Used for decimal values such
upload your code. onto the Arduino device. The results as 1.34
Battery power: You can use the of each upload can be viewed in the 3. char[]: Used for string values such
battery power connector as shown in Status window at the bottom. as ‘Car’
Figure 1 to connect your Arduino Uno 3. New: This button is used for 4. HIGH: Denotes digital pin with
to a remote battery. This will make creating a new code document. current
your device portable and deployable to 4. Open: This button is used for re- 5. LOW: Denotes digital pin
places where you may not be able to opening an already existing code without current
connect it to a computer or a continuous document. 6. INPUT: To assign a pin to read
power source. 5. Save: This is used for saving the 7. OUTPUT: To assign a pin to set
progress on a code document. 8. A0 - A7: Constants used for each
The Arduino IDE 6. Serial monitor: This is the last analog pin
To write code for your Arduino project, button on the right that lets you 9. 0 – 13: Values used for each
you will have to download and install access the Serial Monitor window. digital pin
the latest Arduino IDE. You can Status window: Status window is 10. analogRead(): Returns the value of
download this from the official Arduino the pane shown at the bottom of Figure the analog pin
website 2. Whenever you compile or upload 11. analogWrite(): Sets the value of the
Figure 2 shows the default view your code to the Arduino device, the analog pin
of an Arduino IDE. Let us go through Status window shows the status of your 12. digitalRead(): Returns the value of
some of its major features. There are actions. It is also used for displaying the digital pin
three major components in the IDE, any error messages. 13. digitalWrite(): Sets the value of the
i.e., Toolbar, Status window, and Serial Serial Monitor window: The Serial digital pin
Monitor window. Monitor window logs all the messages 14. Serial.begin(): Initialises the serial
Toolbar: The Toolbar is the pane on generated by your code. Whenever we monitor
the top as shown in Figure 2. It contains use commands such as Serial.print() or 15. Serial.print(): Outputs a message on
shortcuts to the most commonly used Serial.println() to log any details of our the serial monitor
options in the IDE. We shall go through code, they will be displayed on the Serial 16. Serial.println(): Outputs a message

54 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 Focus

on the serial monitor in a new line

17. delay(): To add wait time in
processing in milliseconds
18. setup(): The standard function in
Arduino to be only called once
19. loop(): The standard function in
Arduino to be called repeatedly
20. if …. Else: Used for conditional
statements
21. //: Used for adding single line
comments
22. /* */: Used for adding multi-line
comments
23. #define: Used for defining a
constant value
24. #include: Used for including Figure 3: Output of sayHello program
external libraries
{ window. Figure 3 shows the output
Structuring the Arduino code Serial.begin(9600); // For on this window.
Arduino code should ideally be initializing the serial port Learning to use an Arduino
structured in three parts, i.e., external } device can be a rewarding
libraries, constants and variables, void loop() experience. Whether you end up
and functions. Each part follows the { liking IoT or not, just the feel of
other. We start by calling the necessary sayHello(); using an Arduino will give you a
external libraries for our project. Next, delay(3000); taste of programming hardware
we define any necessary constants or } from the ground up. The greatest
variables required. This is followed with challenge that beginners face with
the required functions. Executing the Arduino code Arduino is not the code as much as
There are two standard functions that Now that we have the code written the debugging. Many times, newbie
must be called in every program, i.e., and structured in the proper order, we developers write code without any
setup() and loop(). The setup() function shall start the execution process. In unit testing. My recommendation
is used for initialising the code while the order to run the code, you will need to if you are just getting started with
loop() runs certain methods for a certain do the following. Arduino is not to get frustrated if
number of iterations depending upon Step 1: Connect your Arduino your programs don’t run properly.
your requirements. device to the computer using the It can be challenging at first to find
Shared below is an example of what USB cable. out where you may have gone wrong
an ideal code structure should look like. Step 2: In your IDE, select the with your code or circuits. But over
We shall output the message ‘Hello Arduino board that you will be using time, you will learn to create checks
Internet of Things’ on the Serial Monitor by clicking Tools > Board. for your code at every level. Once
window every 3000 milliseconds. Step 3: Next, you will be selecting you get a hang of that, the sky’s the
the port by clicking on Tools > Port. limit! Every day, people are coming
// External Libraries Select the port that corresponds to the up with new and interesting ways of
#include <SPI.h> USB modem of your Arduino device. using an Arduino device in their DIY
// Constants and Variables Step 4: Click on the Upload projects. I look forward to seeing
char text = “Hello Internet of Things”; button and open the Serial Monitor what you do with it.
// Custom function(s)
void sayHello() By: Mir H.S. Quadri
{
The author is a research analyst with a specialisation in artificial intelligence and
Serial.println(text); machine learning. He is the founder of Arkinfo, which focuses on the research and
} development of tech products using new age technologies. He shares a deep love for
the analysis of technological trends and understanding their implications. Being a FOSS
// Standard functions
enthusiast, he has contributed to several open source projects.
void setup()

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 55

Focus

Connecting Internet of Things

Devices on Thinger.io

This article is about

connecting various
IoT devices on
Thinger.io, an IoT
platform. This step-
by-step tutorial
will guide you on
how to connect a
development board
to the platform and
start sending data
from the device.

D
evices connected by the 1 Physical objects: People, IoT platforms are gaining popularity by
Internet of Things (IoT) things, machines, vehicles blending hardware devices, APIs and
achieve higher functionality 2 Raw data: Reading of any online services for task management.
because they interface via a distributed parameter Thinger.io provides a scalable cloud
network with numerous physical base to connect devices. With the help
3 Sensors: Conversion of any
objects. They are equipped with of its admin console, it can be linked
physical quantity to an electri-
embedded software, sensors and cal signal to a project through various boards
connectivity choices with the objective like Raspberry Pi, Arduino, ESP8266,
4 Quick decision at the edge: Ur-
of collecting and sharing data with each and Intel Edison. Open source libraries
gent decision making through
other as well as with the central point limited resources imported by the user assure scalability,
through the internet. This opens the security and speed through a couple of
5 Storage: Physical/cloud based
doors to various challenges in terms of lines of code.
choices
data processing, storage and security. There are various components and
IoT platforms bridge this gap between 6 Data analysis: Study and pro- applications offered on the Thinger.io
the world of physical objects and of cessing of data menu bar. Our task for now is limited
actionable insights. to adding a device — the first and
Table 1: Components of IoT
Every IoT ecosystem has its own preliminary step to start an ecosystem.
unique set of input, processing and An IoT platform is an on-premises The first step involves signing up
output elements. However, all of them software suite or a cloud service (IoT to the platform. Enter your personal
have the typical flow setup shown platform as a service [PaaS]) that details for the profile settings.
in Figure 1, which consists of the monitors and may manage and control Remember this also becomes your user
components listed in Table 1. various types of endpoints. Open source name for all further processing on this

56 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 Focus

devices (Sigfox, Lora, the required libraries or development

cURL)/ MQTT device/ environment. For the present example,
Physical NB-IoT. The Device we will be using the Arduino IDE along
Objects ID is important and with an ESP8266 device.
you have to remember The ESP8266 chip (Figure 4) from
Data Raw it while interfacing Espressif is the new generation of low-
Analysis Data with your code. Device cost Wi-Fi chips after the TI CC3000/
information is optional CC3200. This small chip not only
Internet
of and is limited to the user integrates the entire Wi-Fi features,
Things (for identification). But but also a powerful programmable
Device Credentials are MCU. Depending on the board layout
Storage
Sensors vital. Each device has its (ESP-01, ESP-03, ESP-07, ESP12,
Cloud/Onsite
own identifier/credential, etc), it is attached to a programmable
so that a compromised flash, ranging from 512K to 4M. This
Quick
Inferences at device does not affect increases the available user code space,
the Edge other devices. All your and makes possible other cool features
passwords in the server like a small file system or OTA updates.
are stored securely using I am using the ESP 12 module for the
Figure 1: A typical Internet of Things setup PBKDF2 SHA256 with a present illustration.
32-byte salt generated with PRNG and The device can be directly
platform. Once you have entered your a non-depreciable number of iterations. programmed from the Arduino IDE.
profile settings, you can move on to the (Please remember, the password The very first requirement is to install
statistics, as shown in Figure 2. cannot be recalled later.) the board via the Arduino Boards
Go to the Devices tab and create a If the steps have been implemented Manager (Figure 5). From the File
device. This will ask for more details, correctly till now, you should see your menu, go to Preferences, and enter
as shown in Figure 3. device disconnected when you press the http://arduino.esp8266.com/stable/
The device type is generally Add Device button. package_esp8266com_index.json in the
a generic Thinger device (Wi-Fi, Using your new Device ID and Additional Boards Manager URL.
Ethernet, GSM). However, the the Device Credentials to connect the The next step involves installing the
user has the option to select HTTP new device, you will need to install ESP8266 board. Go to Tools > Boards

Figure 2: View of the statistics

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 57

Focus

> Board manager. Then search and

install the esp8266 package (Figure 6).
This should now enable any
program involving ESP8266 directly
from the Arduino IDE. By going to
Tools > Boards you should see the new
ESP8266 boards installed. Select your
board to be able to compile code for
the ESP8266.
You can find additional information
for the ESP8266 package in the
ESP8266 GitHub repository. The easiest
board to program is the NodeMCU,
which does not require pressing Flash
+ Reset buttons for uploading the
sketch. For other boards you will need
Figure 3: The Device tab from the menu to use a USB to serial converter (3v3!)
and flash the sketch by setting some
GPIOs to GND.
Use the code below to connect an
ESP8266 device to the cloud platform
in a few lines using the Wi-Fi interface.
Please remember to modify the
arduino_secrets.h file with your own
information.

#define THINGER_SERIAL_DEBUG
#include <ThingerESP8266.h>
#include “arduino_secrets.h”
ThingerESP8266 thing(USERNAME, DEVICE_
ID, DEVICE_CREDENTIAL);
Figure 4: The ESP8266 chip
void setup() {
// open serial for monitoring
Serial.begin(115200);
// set builtin led as output
pinMode(LED_BUILTIN, OUTPUT);
// add WiFi credentials
thing.add_wifi(SSID, SSID_PASSWORD);
// digital pin control example
(i.e. turning on/off a light, a relay,
configuring a parameter, etc)
thing[“led”] << digitalPin(LED_
BUILTIN);
// resource output example (i.e.
reading a sensor value)
thing[“millis”] >>
outputValue(millis());
// more details at http://docs.
thinger.io/arduino/
}
Figure 5: Adding the package for ESP8266 chip on Arduino IDE void loop() {

58 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 Focus

Figure 6: Installing the board for ESP8266 chip

Figure 7: Device ESP8266 is now connected on the Thinger.io IoT platform

thing.handle(); io>ESP8266>serial debug. We just saw how to connect

} Compile and upload the code as we ESP8266 to Thinger.io. Do follow
arduino_secrets.h do for any other Arduino code. Once the steps to connect it or similar
#define USERNAME “your_user_name” implemented, the inbuilt LED in the development boards to try out more
#define DEVICE_ID “your_device_id” ESP8266 should glow. possibilities with the platform.
#define DEVICE_CREDENTIAL “your_device_ If the steps are implemented
credential” correctly, Thinger.io should show that Reference
#define SSID “your_wifi_ssid” ESP8266 has been connected  https://docs.thinger.io
#define SSID_PASSWORD “your_wifi_ssid_ (Figure 7).
password”
By: Dr Geetali Saha
Alternatively, install the This author is the university program co-ordinator for Internet of Things, Minor
Thinger.io custom library. Follow Degree Program, CVM University, Anand, Gujarat. She is also a faculty member at the
Department of Electronics & Communication, GCET, Anand, Gujarat.
the path File>Examples>thinger.

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 59

Focus

IoT Might Disrupt

Lots of Technologies
for Greater Good

“I believe IoT will disrupt multiple industries in the coming decade

and beyond,” says Manish Kothari, senior vice president of software
development at Silicon Labs. In his keynote address at the VLSI
Design Conference 2022, Manish talked about the challenges and
future of IoT and its potential to be one of the most disruptive
technologies of this century.

i
n today’s time, safety and IoT is simple yet complex work for years without any human
comfort are paramount, and IoT solutions seem straightforward intervention. And not to forget about
automation is playing an active but are not as simple as they sound. the security, which in an IoT system
role in enhancing lifestyle, safety, For example, an embedded system, is paramount. Furthermore, there is
and comfort along with improved which is the backbone of an IoT a time-relevance aspect to the IoT
efficiency. And to realise automation, system, is often a low-powered device solution, that is, the product and its
gathering and processing data is the performing a simple task, which components should remain relevant
most crucial part, which is taken care can easily be performed by even a for at least 5 to 10 years.
of by the Internet of Things (IoT). low-cost smartphone, or a decade-old “The IoT solution has at least a
In a layman’s terms, IoT is basically microcontroller (MCU) combined 5-to-10-year life,” says Manish. “So,
connecting different applications and with a simple sensor that is easily there is a time relevance aspect of the
sensors to the cloud or connecting available in the market. hardware and software, both of which
them to the internet using a variety The real challenge here is not cannot go obsolete. You must be able
of different wireless protocols. The to complete the task or gather to upgrade the capability of your
inclusion of IoT on a large scale has not data but to do all of it in a space- devices, capabilities of your hardware
just improved the quality of life but has constrained, budget-constrained, and software, till the time an end
also increased the complexity manifold. power-constrained system that can customer has the product.”

60 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 Focus

Selecting the right SoC From a light bulb to a health monitor, feels that it will transform a variety of
An IoT system needs to sense, to machinery in a factory to an asset industries; it will transform even those
communicate, perform inference tracker, we have diverse applications industries that are considered low-tech,
and computation, while taking the and various micro verticals, with very such as the construction industry.
security of the system and data into different requirements for each one of Manish gives an example of how
consideration. These things require them in every vector. IoT can become beneficial for the
tight integration of hardware-software Therefore, designing a one- construction industry, which does
architecture. Designers need to design fit solution for all situations is not not usually require automation. He
compact products, running on minimum possible. But being a low-cost solution, claims that the maximum amount of
power, and utilising really low it is also not feasible for a company to time and resources are not efficiently
bandwidth while costing extremely low. do R&D from ground zero for every managed in the construction industry,
To achieve all of this the selection single product. You must do quick turn and deadlines often get extended due
of an optimum system-on-chip (SoC) derivatives in order to stay relevant. to multiple reasons, such as improper
is very critical. SoC is one of the main management of inventory, work
components that play a crucial role in getting delayed, or resources being
integrating all the components in an SOC SENSES wasted due to bad weather or improper
embedded system. AND CONTROLS, working conditions.
In addition to the discussed These are the areas where IoT
problems, the SoCs must last for years IT COMPUTES, can streamline things. IoT can help
and, in many cases, last over a decade IT INFERS, AND get live data of these tools, machines,
on a single coin-cell battery. So, very and equipment, the utilisation of
tight integration of all the aspects is IT ALSO SAFELY the equipment and tools, and their
necessary for a good IoT solution. To HANDLES THE efficiencies. All these parameters
overcome the problem, engineers are are very critical for delivering
trying to simplify inferencing, and COMMUNICATION projects on time.
to achieve that we can observe the PART OF IT. Manish also believes that there
development of Micro Edge. are other parameters, such as how and
when you service and maintain your
Inference at Micro Edge Furthermore, there are industrial equipment, or the status of the secure
The inference is turning raw data from and commercial customers with access for some of these construction
the sensors and other IoT devices into a massive quantity requirements who sites where there is a danger of
piece of actionable information. In the want to have their own ecosystems and operation, like in mining operations,
inference we have seen a development not of Google, Amazon, etc for privacy, where you have to take personal safety
of Micro Edge. Although sounding too security, and confidentiality. So, new into account. So, how do you ensure
fancy, Micro Edge is nothing but where players in the IoT segment have a wide secure access?
your sensors and devices are residing. variety of opportunities to tap in. There are clear problems right in
According to Manish, “When you “We have diverse applications and front of us that we need to identify
do inferencing right at the sensor or various micro verticals, with different and solve to improve lives, improve
near the device, you are taking care of requirements for each one of them businesses, their efficiencies, and the
security, you are sending that inferred in every vector, whether it is sense money that they make. This is just one
solution or the inference across as requirements, compute requirements, industry. Nobody’s going to give us a lot
opposed to the live data. So, it gives etc. Your hardware-software needs to be of money to just look cool. And unless
you lower latency. It gives you a more configurable,” adds Manish. we start solving real problems using IoT,
secure solution, and obviously, it is the technology will not take off.
cost-effective.” He adds, “This is a Finding the right opportunity
growing use-case where inferencing is IoT has the potential to change how What the future holds
occurring right at what we call Micro we do a lot of things. We are not just “IoT will disrupt multiple industries in
Edge by the sensors and devices out connecting things, or inferring things the coming decade and beyond.”
there in the field.” because it’s cool, but because it has Disruption occurs due to
the potential to change the world. technologies that are generally cheaper
Need for configurable system Apart from enhancing convenience and and simpler to use. Initially, in terms
The IoT applications are very diverse. improving the lifestyle, Manish also of performance, these disruptive

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 61

Focus

Phones Feature phones Smartphones

Smartphones were no more a novelty
1980-1990 1990-2000 2000-2010 2010-2020 2021+
but a necessity.
Communicate 1G (Analog 2G (Digital 3G (Mobile 4G (Native IP 5G
Cellular) Cellular) Broadband) Networks) We have just set foot in the 2020s, but
BT
NFC by now smartphones pack enough power
Wi-Fi
to compete with decent PCs, cameras, etc.
Communicate-Data ~1Mbps or less ~10Mbps or less ~100MBps or less ~1GBps or less ~10GBps or less
Some of the latest models have higher
Compute Single core ~ Single core ~ 2 and 4 cores ~ 4 and 8 cores ~ 8 cores+ ~
1Ghz or less 2Ghz or less 2Ghz+ 3Ghz 3Ghz+ RAM and processing power than most
Sense Camera
Touch
Fingerprint entry-level laptops. They offer much
ALS/PS
Accelerometers
higher resolution than most cameras, and
Gaming Gaming the 5G internet is capable of matching or
Location GPS
even exceeding the data transfer rate of
cabled internet available in many areas.
Inference Predictive text Facial and voice AR/VR
recognition Smartphones are now secure enough
Secure Arm TZ
to do banking transactions and, in
Software Symbian OS Full web browsing,
app store many countries, are being used as the
Process nodes -
(microns) -
3 or better 0.8 or better 0.1 or better 0.02 or better 0.01 or better primary mode for everyday transactions.
Moore’s Law Digital/RF separation However, it is too early to discuss the
Figure 1: Mobile phone evolution: Predictor for IoT evolution advancements that are likely to be made
by smartphones in this decade!
technologies are only suitable for a data transfer rate jumped from
small number of users. Over time, these 64kbps to over 1Mbps. The phones The future of IoT
technologies improve and become a now had cameras, touchscreens, IoT technology is in its second
necessity for most people. accelerometers, etc. This is when generation in terms of development.
This phenomenon has occurred time mobile phone technology really Compared to mobile phones, IoT is
and again. One such example is of the disrupted the market. The processor where the phones were in 1990s as
smartphone, which is making cameras, in smartphones began catching up feature phones. The IoT technology is
audio players, PCs, handheld GPS, and with the basic PCs, the phone cameras being adopted by a few, but it is yet to
many other things obsolete. affected the sale of low-level point- become a necessity for most.
According to many industry and-shoot cameras, and many different From a process node standpoint, in
stalwarts, IoT will disrupt multiple gadgets were affected due to these terms of our inferencing capability, in
industries in the coming decades. powerful smartphones. terms of our compute capability, we are
To predict the growth of IoT we may Smartphones were no longer merely nowhere close to gigahertz, right? We
look at the evolution of smartphones. The a tool for communication but became have the sub-gigahertz capability today.
timeline of phones is shown in Figure 1. much more than that. We saw a very So, these are very early days for IoT. Very
As can be seen, there was massive high level of integration of the various likely, as the Moore’s Law progresses, the
advancement with each successive functions, including computing prowess, sense of integration will also progress, and
decade. Alongside the features in the sense, and gaming—all of these started there will be dislocations that vendors and
phone’s hardware, the networking to disrupt several industries. players will benefit from.
technology kept improving. From The advancement made in the We normally think of Moore’s
1G in the 80s, which could transfer first decade caused major adoption of Law as digital integration, and we can
less than 10kbps, the growth in data smartphones, thus bringing a flood of say that the IoT is at the beginning of
transfer rate kept on improving by a manufacturers driving the cost lower Moore’s Law journey. It could possibly
factor of ten. and lower with each passing year. have a high level of function integration
The inception of digital cellular The rate of refinement and growth driven by Moore’s Law. When we
technology in 1990s changed the scope continued in the 2010s. The inception talk about sensors, there is a variety of
of mobile phones. From just a device to of 4G improved the internet speed and analogue front-ends on the sensors that
make calls the mobile device had added the processing power of mobiles was can get integrated. So, there’s going to
functionalities, such as gaming and now comparable to baseline PCs. be a high degree of integration.
GPS, and in terms of inference we got a The phone cameras started to It is again similar to the case of
new feature of predictive text. outperform the basic cameras. And smartphones, with a lot of constraints.
Then the next big development similar was the fate of music players, Compared to the technologies
happened in the 2000s, when the such as MP3 players and iPods. smartphones were battling - which

62 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 Focus

included PCs, servers, camera, etc - the

smartphone had constraints of power, High level of function (analog and digital) integration driven by Moore’s Law—sense/computte/secure/
infer/locate/communicate while preserving the power/area/memory budget advantages
space, memory, and budget. So, it is
Cloudless architectures for latency/privacy/cost etc
safe to say that the constraints of the
IoT are similar. On the fly, self organising networks using variety of wireless protocols ≥ intrusion detection,
locationing, mobility, networks that follow (shield)
We can expect a high level of
integration that will come in next few Scaling via Zero code SW for ‘infinite applications/use cases’—internal and customer SW development

decades. Some other expectations are:

Open source HW/SW ≥ IP vendors and business models (Imagine ‘Arduino projects’ 10+ years from now)
Cloudless architecture to
overcome design constraints: Massive SAAS opportunity for configurability of HW/SW, security and plethora of services for diverse
markets/applications
Cloudless technology will enhance
Large dense network of ‘powerful’ nodes in home/enterprise/cities—autonomous robots, blockchains,
various domains of the IoT. It will security architectures
provide greater choice and agility
while reducing cost for applications Figure 2: IoT future (Decade+) possibilities: Infinite and open to anyone’s imagination
that incorporate state-of-the-art
hardware, software, and data access. Open source software: The other there is a diverse market with specific
You can think of a variety of cloudless very interesting thing that is bound requirements, IoT needs to meet a
architectures, like mesh networks that to happen in software is scaling. In plethora of different demands, such as
offer better latency and privacy. In a world with infinite applications security and time relevance. So, there is
fact, there are some companies already and use-cases, it is not possible to an opportunity for a large dense network
thinking about piloting peer-to-peer serve each individual customer’s of powerful nodes. For example, every
cloudless architectures. Therefore, requirements. This is where a zero- lighting pole present in the smart grid
privacy and reduced latency will be code solution holds the potential to will have some sort of SoC that can self-
the biggest reason for the adoption of change the world. The zero-code organise and help you with location,
cloudless technology. solution will allow the customers sensing, etc. You can also think about
Self-organising network: A self- to take the code, drag and drop running blockchain on it.
organising network (SON) uses artificial connectors in a visual platform—and Although it seems far-fetched,
intelligence (AI), predictive analytics, the code is outputted and the product we are talking about these in the
and pre-optimised software algorithms is ready to go. future because we will have powerful
to offer automated functions, such as Growth with open source computing power in these nodes. We
self-configuration, self-optimisation, hardware: Open source hardware are thinking about the 5G equivalent
self-healing, and self-protection. (OSH) is free-to-use physical of IoT, which will have 8, 10, 12 cores
SONs simplify network administration artefacts of technology that can be offering significant compute power.
by enabling the creation of a plug- made and used by anyone without Memory is only going to grow in all of
and-play environment for a network any copyright infringement. There these massive networks that would be
task. There are over fifty different is a big open source community available in the future.
wireless protocols that serve a variety supported by electronics makers and These are all predictions, but what is
of different needs. Bluetooth has a DIYers all around the globe. One such certain is that the IoT holds the ability to
different role than Wi-Fi and so does the example is of Arduino projects for disrupt a lot of industries and change the
sub-gigahertz protocol that gives you a microcontrollers and NodeMCU, which world for greater good.
range of a few kilometres. is an open source IoT platform. OSH
The benefit of implementing SON is bound to grow and result in a variety By: Manish Kothari
is that it will allow us to change the of different business models that will
The author is senior VP of software
protocols on the fly. For example, the disrupt the industries. Furthermore, it development at Silicon Labs. This
transmitter radios can automatically will empower individuals to create IoT article is based on the keynote
address he gave at the VLSI Design
adjust parameters such as transmission solutions, thus democratising IoT.
Conference 2022.
power to avoid interference and Massive SAAS opportunity: The
maximise both coverage and capacity. need for configurability of hardware
It also provides better security as the and software creates a huge opportunity Transcribed and curated by
self-organising network automatically for software as a service (SaaS). Since Sharad Bhowmick
defends itself from penetration by
unauthorised users. The article was originally published in the September 2022 issue of Electronics For You.

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 63

Developers Let’s Try

Hooked on
React Hooks!
In this final part of the two-part in-depth series on React Hooks, we continue
to discuss a few more hooks and custom hooks.

I
n the first article in the series, which was carried in the useImperativeHandle. Also, you can customise the function
October 2022 issue of OSFY, we learnt what hooks and to behave in a different way than you would normally expect.
custom hooks are, and discussed a few hooks briefly. So, for example, in the focus function, we can update the
We will look at a few more hooks and custom hooks in this document title or log as seen in the code but notice that we
second and final article on React Hooks. have used forwardRef. In React, forwardRef is a method that
allows parent components to pass down ref to their children,
useImperativeHandle and it gives a child component a reference to a DOM element,
which is created by its parent component.
useImperativeHandle(ref, createHandle, [deps]); So, basically, useImperativeHandle is used when you
might not want to expose native properties to the parent or
function Button (props, ref) { (as in our case) want to change the behaviour of a native
const buttonRef = useRef(); function. As in the case of additional hooks, you’ll rarely use
useImperativeHandle(ref, () => ({ useImperativeHandle in your code base, but it’s good to know
focus: () => { how to use it.
console.log (“Inside focus”);
buttonRef.current.focus(); useLayoutEffect
}
})); const ref = React.useRef()
return ( React.useEffect (() => {
<button ref={buttonRef} {...props}> ref.current = ‘Lorem ipsum’
Button })
</button>
); React.useLayoutEffect (() => {
} console.log(ref.current) // This runs first, will log old
export default forwardRef (Button); value
})
Usually, when you use useRef, you are given the
instance value of the component the ref is attached to. This As we saw with the useEffect, the code inside of
allows you to interact with the DOM element directly. useLayoutEffect is fired after the layout is rendered.
useImperativeHandle is very similar but it lets you do two Although this makes it suitable for, let’s say, setting up
things -- it gives you control over the value that is returned; so subscriptions and event handlers, not all effects can be
instead of returning the instance element, you explicitly state deferred. So, basically, we should understand the difference
what the return value will be. Also, it allows you to replace between passive and active event listeners. Let’s say we
native functions such as blur and focus with functions of want to make changes to the DOM synchronously before we
your own, thus allowing side effects to the normal behaviour render the additional components. If we use useEffect, this
or different behaviour altogether. And the best point is can lead to inconsistencies in the UI. As a result, for these
you can call the function whatever you like. So, basically, types of events, React provides us with useLayoutEffect. It
useImperativeHandle customises the instance value that is has the same signature as useEffect but the only difference is
exposed to parent components when using ref. when it is fired. Your code runs immediately after the DOM
In the example above, the value we get from ref will has been updated but before the browser has had a chance to
only contain the function focus, which we declared in paint those changes.

64 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 Let’s Try Developers

For example, if you want to make DOM measurements window.localStorage.getItem(key) ||

like getting the scroll position or other styles of an element, String(default)
and then make mutations or trigger synchronous re-render, );
useLayoutEffect is our guy. If you don’t need to do all this, });
you should use useEffect. I’ll go as far as to say that 99 per useEffect (() => {
cent of the time, you would be using useEffect. window.localStorage.setItem(key, state);
}, [state]);
Custom hooks }
You may have heard a lot about custom hooks. Let us try and
understand what they are exactly. Custom hooks are normal function App() {
JavaScript functions that can use other hooks inside them, const [count, setCount] = useLocalStorage(‘counter-
and contain a common stateful logic that can be reused within value’, 0);
multiple components. These functions are prefixed with the return (
word ‘use’. You can name them anything as long as they have <div>
the ‘use’ prefix. Let’s say you have two functions or components <button onClick={() => {
which implement some common logic. You can create a third setCount(count + 1)
function with this common logic, and implement it in the other }}>{count}</button>
two functions. After all, hooks are just functions. They give us </div>
the advantage of fewer keystrokes and less repetitive code. I will );
show how this happens and we’ll create our very own hook now. }
Let’s say we have a count state variable and we want
to store it in the local storage whenever the value updates. As you see in the above code, we have created our very
useEffect is the perfect candidate here. own custom hook called useLocalStorage and added the
repeated code to this hook. Now, in all our 10 components we
function App() { can simply call the useLocalStorage hook, and pass in the key
const [count, setCount] = useState(() => { and the default value. It’s as simple as that and will work in
return JSON.parse( the exact same way as before.
window.localStorage.getItem(‘counter-value’) || With the introduction of custom hooks, people started
‘0’ creating hooks for the most common use cases. A simple
); search for a local storage hook in npm gave 60 already existing
}); packages for various use cases. So whenever you find the use
useEffect (() => { case for a custom hook in your React application, I recommend
window.localStorage.setItem(‘counter-value’, count); going through this list; maybe someone has already created a
}, [count]); package that you can reuse and save development time.
This is one of the strengths of having such a vast and
return ( active community and React has that -- you can surely use it
<div> to your advantage.
<button onClick={() => {
setCount(count + 1) useDebugValue
}}>{count}</button>
</div> function useLocalStorage (key, default) {
); const [count, setCount] = useState(0);
}
// ...
But what if we want to update this value in ten other
components? The highlighted code will be repeated in all // Show a label in DevTools next to this Hook
these 10 components. To avoid this repetition, we will use a // e.g. “LocalStorage: Empty” useDebugValue (count == 0 ?
custom hook. ‘Empty’: ‘Not Empty’);
useDebugValue(count == 0 ? ‘Empty’ : ‘Not Empty’);
function useLocalStorage(key, default) {
const [state, setState] = useState(() => { return count;
return JSON.parse( } Continued to page...70

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 65

Developers Let’s Try

R Series: Probability
Distributions
This fifteenth article in the R series will explore more probability distributions
and introduce statistical tests.

The displacement values are shown below:

> mtcars$disp
[1] 160.0 160.0 108.0 258.0 360.0 225.0 360.0 146.7 140.8
167.6 167.6 275.8

R
[13] 275.8 275.8 472.0 460.0 440.0 78.7 75.7 71.1 120.1
318.0 304.0 350.0
[25] 400.0 79.0 120.3 95.1 351.0 145.0 301.0 121.0

We can prepare two data sets, P and Q, from the

PROGRAMMING
displacement data, as follows:

> P <- mtcars$disp[0:9]

> P
[1] 160.0 160.0 108.0 258.0 360.0 225.0 360.0 146.7 140.8

> Q <- mtcars$disp[10:19]

W e will use R version 4.2.1 installed on Parabola GNU/

Linux-libre (x86-64) for the code snippets.
> Q
[1] 167.6 167.6 275.8 275.8 275.8 472.0 460.0 440.0 78.7 75.7

$ R --version The boxplot for the data sets is shown in Figure 1:

R version 4.2.1 (2022-06-23) -- “Funny-Looking Kid”

Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
400

R is free software and comes with ABSOLUTELY NO WARRANTY.

You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
300

https://www.gnu.org/licenses/.

Welch two sample t-test

Consider the mtcars data set in the lattice library:
200

> head(mtcars)
mpg cyl disp hp drat wt qsec vs am gear carb
Mazda RX4 21.0 6 160 110 3.90 2.620 16.46 0 1 4 4
Mazda RX4 Wag 21.0 6 160 110 3.90 2.875 17.02 0 1 4 4
100

Datsun 710 22.8 4 108 93 3.85 2.320 18.61 1 1 4 1

Hornet 4 Drive 21.4 6 258 110 3.08 3.215 19.44 1 0 3 1
1 2
Hornet Sportabout 18.7 8 360 175 3.15 3.440 17.02 0 0 3 2
Valiant 18.1 6 225 105 2.76 3.460 20.22 1 0 3 1 Figure 1: Boxplot mtcars$disp

66 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 Let’s Try Developers

> boxplot(P, Q) In wilcox.test.default(P, Q) : cannot compute exact p-value

with ties
The Welch two sample t-test can be performed using the
t.test() function, as demonstrated below: The empirical CDFs for the data sets can be drawn using
the plot() function:
> t.test(P, Q)
> plot (ecdf(P), verticals=TRUE)
Welch Two Sample t-test > plot (ecdf(Q), verticals=TRUE)

data: P and Q The wilcox.test() function accepts the following

t = -0.98055, df = 15.369, p-value = 0.342 arguments:
alternative hypothesis: true difference in means is not equal
to 0
Arguments Description
95 percent confidence interval:
-176.63004 65.16337 x A numeric vector of data values
sample estimates:
mean of x mean of y y Optional numeric vector of data values
213.1667 268.9000
alternative Character string for alternative hypoth-
esis (‘two.sided’, ‘greater’, or ‘less’)
We can also run the t-test assuming equal variances by
passing TRUE to the ‘var.equal’ argument, as follows: conf.int A logical value that indicates if confi-
dence interval should be computed
> t.test(P, Q, var.equal=TRUE)
conf.level Confidence level of the interval
Two Sample t-test
correct A logical value on whether to apply
continuity correction
data: P and Q
t = -0.95746, df = 17, p-value = 0.3518 data An optional matrix or data frame
alternative hypothesis: true difference in means is not equal
to 0 exact A logical value on whether p-value
95 percent confidence interval: should be computed
-178.54460 67.07793
mu Optional parameter for the null hypoth-
sample estimates:
esis
mean of x mean of y
na.action A function to handle data that contains
213.1667 268.9000 NAs

Two sample Wilcoxon test paired A logical value to indicate a paired test
The Mann-Whitney-Wilcoxon test is used for randomly
subset An optional vector specifying a subset
selected values from two populations. For our example, we of observations
can run the wilcox.test() function on the P and Q data sets, as
follows: tol.root A positive numeric tolerance

> wilcox.test(P, Q)
Kolmogrov-Smirnov test
Wilcoxon rank sum test with continuity correction The Kolmogrov-Smirnov test or KS test is used to test if
two samples come from the same distribution. It can also
data: P and Q be modified to be used as a ‘goodness of fit’ test. The two-
W = 32, p-value = 0.3059 sample Kolmogrov-Smirnov test for the P and Q data sets
alternative hypothesis: true location shift is not equal to 0 is as follows:

Warning message: > ks.test(P, Q)

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 67

Developers Let’s Try

ecdf(p) ecdf(Q)
1.0

1.0
0.8

0.8
0.6

0.6
Fn (x)

Fn (x)
0.4

0.4
0.2

0.2
0.0

0.0

100 150 200 250 300 350 400 100 200 300 400 500
x x

Figure 2: Empirical CDF (P) Figure 3: Empirical CDF (Q)

Exact two-sample Kolmogorov-Smirnov test The alternative hypothesis argument accepts

the string argument values: ‘two.sided’, ‘less’, and
data: P and Q ‘greater’. These indicate if the null hypothesis of the
D = 0.37778, p-value = 0.2634 distribution of ‘x’ is equal to, less than, or greater than
alternative hypothesis: two-sided the distribution of ‘y’. Any missing values are ignored
from both the samples.
You can also perform a one-sample Kolmogrov-Smirnov
test. The function accepts the following arguments: Chi-Square test
The Chi-Square test is a statistical hypothesis test to
Arguments Description determine the correlation between two categorical
variables. We can determine if the cylinder and horse
x Numeric vector of data values power variables from the mtcars data set are significantly
y Numeric vector or character string correlated.
name of a CDF
> data <- table(mtcars$cyl, mtcars$hp)
alternative Alternate hypothesis should be > print(data)
‘two-sided’, ‘less’ or ‘greater’

exact NULL or logical value to specify if 52 62 65 66 91 93 95 97 105 109 110 113 123 150 175
p-value should be computed 180 205 215 230 245 264
4 1 1 1 2 1 1 1 1 0 1 0 1 0 0 0
simulate.p.value A logical value to compute p-values
0 0 0 0 0 0
by Monte Carlo simulation
6 0 0 0 0 0 0 0 0 1 0 3 0 2 0 1
data An optional data frame 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0 0 0 0 0 2 2
subset A subset vector of data to be used 3 1 1 1 2 1
na.action A function to handle data that
contains NAs The chisq.test() function can be performed on the data
variables, as shown below:

68 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 Let’s Try Developers

> chisq.test(data) > p = seq(0, 5, length=50)

> plot(p, dbeta(p, 3, 5), type=’l’)
Pearson’s Chi-squared test

data: data
X-squared = 59.429, df = 42, p-value = 0.0393

2.0
A p-value of less than 0.05 shows that there is a strong
correlation between cylinder and horsepower.
The chisq.test function accepts the following arguments:

1.5
Arguments Description
x A numeric vector or matrix
y A numeric vector, or ignored by x is

1.0
a matrix
correct A logical value on whether to apply
continuity correction
0.5
p A vector of probabilities
rescale.p A logical scalar to sum to 1
simulate.p.value A logical value to whether compute
p-values by Monte Carlo simulation
0.0

Beta 0 1 2
p
3 4 5
The Beta function is mathematically defined by the integral
for two inputs, alpha and beta, as shown below: Figure 5: DBeta distribution

Gamma
The Gamma distribution is a continuous probability
distribution that has a shape (z) and scale parameter (k),
Figure 4: Beta distribution which is defined as shown in Figure 6.

You can use the beta() function for the P and Q data sets,
as follows:

> beta(P, Q) Figure 6: Gamma distribution

[1] 7.307338e-100 7.307338e-100 2.515827e-100 5.970038e-162
2.168131e-190 The dgamma() function provides the density function. In
[6] 8.246568e-192 1.139582e-245 1.211659e-144 2.186038e-63 the following example, a sequence of values is produced and
1.931142e-65 a plot for its gamma distribution is generated.

The lbeta() function provides the natural logarithm of the > a <- seq(0, 50, by=0.01)
beta function, as demonstrated below: > y <- dgamma(a, shape=3)
> plot(y)
> lbeta(P, Q)
[1] -228.2696 -228.2696 -229.3359 -371.2320 -436.7173 The lgamma() function provides the natural logarithm of
-439.9865 -564.0027 the absolute value of the gamma function. For example:
[8] -331.3803 -144.2808 -149.0099
> l <- lgamma(a)
The dbeta() function provides the density distribution. An > head(l)
example is given below with a plot (Figure 5). [1] Inf 4.599480 3.900805 3.489971 3.197078 2.968879

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 69

Developers Let’s Try
0.25
0.20
0.15
0.10
0.05
0.00

0 1000 2000 3000 4000 5000

Index

Figure 7: Gamma function

These functions accept the following arguments: Do read the manual pages for the above R functions to
Arguments Description learn more on their arguments, options and usage.
a, b Positive numeric vectors
x, n Numeric vectors By: Shakthi Kannan
k, d Integer vectors The author is a free software enthusiast.

Continued from page...65

As you may know, React dev tools are an amazing way Some rules
to debug apps. React shows a component tree with props Now that you know all about hooks, you should know two
values while running the app. useDebugValue is used to additional rules that are imposed by them. First, we have
display labels for custom hooks in the React dev tools. For to call hooks only at the top level. That is, don’t call hooks
example, if you look at a custom hook in the code above, inside loops, conditions, or nested functions. Second, call
we can display the state value of our custom book by using hooks only from React function components, and not from
useDebugValue, which will show us the label with the key regular JavaScript functions. There is just one other valid
as the hook name (it will just remove the ‘use’ word; so it place to call hooks, and that is your own custom hook
says local storage here instead of ‘useLocalStorage’) and the as we saw in the custom hook section. Basically, follow
value will be dependent on the state value. these two rules and everything will work as expected
We can pass an optional function to this hook solely for for you.
formatting purposes, i.e., for data to look good when we
inspect the components. Keep in mind that this is only for
debugging and it won’t be seen on the UI. This is just made By: Janhavi Dahihande
to make the lives of developers easy when they inspect The author is a software engineer based in San Francisco. She
elements. useDebugValue takes in the debug value and is interested in learning and talking about new technologies, and
returns the formatted display just for the developers to see likes contributing to open source. When not coding, she likes to
attend book club events, travel, and try out new cuisines.
the data. So the data looks pretty and is easier to understand.

70 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 InSight Developers

How Contact Centres can

Benefit from Artificial
Intelligence
Classifying text and processing it for inference
is very important in artificial intelligence based
solutions. This article is about how artificial
intelligence can be used in the contact centre
and the benefits it brings.

C
ontact centre automation collect some background information, customer with the right agent at the
creates hassle-free customer and try to solve their issues. AI chatbots right time can do a lot to improve the
support and helpdesk are good at solving very simple customer experience.
management. The application of problems, but the more complicated Insightful analysis: AI can also
artificial intelligence (AI) in a contact issues still need an agent’s touch. elevate analysis and provide business
centre is still new and evolving. The AI chatbot will pass along the leaders with information they can
However, it can reduce operational information it collected to make the act upon. For example, using AI
costs, personalise the customer transition as seamless as possible and to powered analysis, businesses can
experience, increase agent efficiency, boost agent efficiency. analyse customer behaviour, identify
and provide more actionable analysis in More intelligent routing: AI makes ones who are at risk of churning, and
the following ways. the routing of the ACD (automatic call reach out to them with a compelling,
Chatbots: Chatbots may be the distributor) even smarter. Now contact personalised offer.
most visible use of artificial intelligence centres can route enquiries based on AI can bring down costs in the
(AI) in the customer service process. additional criteria, such as customer contact centre due to increased self-
When customers choose to chat online personality and information gleaned service. It also helps to improve the
with a business, chatbots greet them, from previous contacts. Matching a customer experience, as customers can

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 71

Developers InSight

communicate using their channel of case, once AI is providing accurate the time and costs involved before
choice any hour of the day or night. information in a consistent manner, the implementing any kind of AI system.
However, organisations must realise technology can interact directly with To take this step, it is important to
that the role of a contact centre agent customers via a bot. ensure that effective training and
will become more challenging with the Use of AI to run analytics in coaching, workforce management,
introduction of AI. Automation remains the background: First, predictive and other technologies are already
most effective with interactions that contact routing analyses key pieces of in place. Customer satisfaction
involve simpler tasks and that do not information such as whether the caller should be a top priority.
pull in a lot of different pieces. Here, is a high spender, what kind of products A number of AI services for contact
the human touch is not that critical, they buy, their sales history, projected centres are available in the market. Let’s
and AI can provide straightforward future behaviour, and other key factors take a look at some of the best ones.
information to customers overnight, to route calls to the agent who has the
during holidays and other times when best set of skills to successfully work Amazon Connect
it can be costly to staff a contact centre. with the customer. AWS provides Amazon Connect as
This means that agents will receive a Second, contact centres can use a fully managed cloud solution for
larger percentage of calls that are more speech analytics technology to analyse contact centre automation.
complicated and challenging. recorded calls with customers (post-call Contact center AI is integrated with
When customers need information analytics). After listening to a call, AI DialogFlow, which uses natural language
about a specific problem, recovery from can identify when a voice inflection processing (NLP) for voice based
a mistake or a more detailed solution goes above a specific threshold level, heuristic search for customer queries.
that is too complicated for automation, and discover when and why customers It also helps to do contextual routing
there is a significant need for a human get angry and potentially churn. to the required support service, based
agent. Here, human interaction is Speech analytics technology can on the query typed by the end user. A
necessary to help relieve frustrations also be used on a real-time basis to dashboard and analytics are on offer for
and provide correct responses with analyse the tone and inflections of a query related metrics on a day-to-day
the appropriate empathy and emotions caller to indicate when they appear basis based on ticket status like ‘active’,
(which, at times, is more difficult than frustrated, show anger, and need ‘closed’, ‘waiting’, to name a few. It is a
just navigating a system). intervention from a higher skilled agent/ hybrid ‘build once and deploy anywhere’
supervisor. This is a more expensive use solution, and can work on smart devices
How AI can be used in a of AI in contact centres, but can provide including Alexa and Siri enabled devices.
contact centre an opportunity to avoid a negative AWS Connect helps to build
Use of AI to assist agents: While an interaction, making customers happier a scalable, resilient IVR solution
agent listens to a customer during and increasing company loyalty. without any worry about infrastructure,
a phone call, AI can run in parallel There will always be a percentage of deployment models and integration
and do some of the heavy lifting of calls that will be fully automated using with other services for a public cloud
searching a knowledge base for answers a form of AI. If an automated system environment. It is an omni-channel
while allowing the agent to focus on works well, customers may even reach cloud contact centre service with unified
the customer interaction. Once the out more frequently to get their answers machine learning (ML) capabilities to
information is presented to the agents, quickly and easily. This will reduce the manage the customer and contact centre
they can review the results in real- time spent waiting in a queue, eliminate client (agent) experience seamlessly. It
time, decide what will work best in the time involved for a human response, makes setting up a contact centre and
each situation, and communicate the and drive down overall costs. However, chatbots very easy. This can be done
response to the customer. on the flip side, agents will need to be without much technical experience
AI also works better if information better trained and require higher skills as Amazon takes care of most of
is coming in from the customer via to deal with the more complicated and the technical configuration based
the written word and/or images in sensitive calls that can’t be handled on your inputs.
the form of an email or chat session. through automation, which will offset If you don’t have much experience
This removes accents, translations, some of the savings. in contact centre development, you
and other questionable input so Businesses with large budgets can can make use of the Contact Center
that automated processes can easily test various AI processes to see what Intelligence (CCI) support facility in
understand the data and then respond works and what doesn’t. But smaller Amazon, which can help in designing a
to those interactions. In either contact centres really need to look at strategy for the centre. You can choose

72 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 InSight Developers

any AWS CCI partner for setting up Dialogflow CX: This helps with hybrid workplace models since
the contact centre, and integrate it with handling complex customer queries the beginning of the pandemic,
other AWS native services like data and supplemental questions by companies are investing in new
analytics, ML based data processing looking for the required responses technologies that allow them to
and data storage, to name a few. from a knowledge base. It supports provide high quality and personalised
human agents by providing the customer service experiences.
Azure Cognitive Services relevant details required to respond In response to the changing
Microsoft’s Azure Cognitive Services to a query. Dialogflow CX agents landscape, many companies are turning
can help you realise partial or full build on Google’s natural language to AI to power customer service
automation of telephony based customer understanding (NLU) algorithms to operations. Listed below are six trends
interactions, and provide accessibility handle these search capabilities. with respect to the use of AI in contact
across multiple channels. With its Agent Assistant: This is built with centres that promise to transform
language and speech services, you AI assisted conversational services customer service and experience.
can analyse call centre transcriptions, to prepare the recommended answer Alleviation of shortages in staff:
extract and redact personally identifiable choices for human agents to choose Labour shortages were a side effect
information (PII), summarise the from, for replying to customer queries. of the pandemic, and many industries
transcription, and detect the sentiments. It uses a centralised knowledge base found it difficult to manage high
Azure Cognitive Services can be to prepare ready-to-use responses. call volumes with low volumes of
implemented in call and contact centres It transcribes live calls in different agents. Many businesses are turning
in various ways. languages and helps train AI models to AI to account for the gaps within
ƒ Virtual agents: Conversational AI to analyse frequently asked questions, their workforces.
based telephony-integrated voice answer patterns and customer groups AI is helping to fill skills
bots and voice-enabled chatbots. based on interest. shortages of the existing workforce
ƒ Agent assistance: Real-time CCAI Insights: These services through career transition support
transcription and analysis of a call include text-to-speech and speech-to- tools. It is also helping employees
to improve the customer experience text conversion, sentiment analysis, do their jobs better and faster using
by providing insights and suggesting ranking of responses, integrated digital assistants and in-house AI-
actions to agents. IVR services, and natural language driven training programmes.
ƒ Post-call analytics: Post-call processing capabilities to provide Understanding the nuances of
analysis for creating insights into unified customer support powered by human conversations: AI solutions
customer conversations to improve Google Cloud AI services. that are perceptive to caller intent,
understanding and support continuous language, and sentiment are vital.
improvement of call handling. Contact centre AI: The trends Contact centre AI helps to understand
Contact centres are undergoing a the nuances of human conversation
Google Contact Center AI massive transformation. During the because it can pick up on the tone of
Google Cloud Platform (GCP) provides Covid-19 pandemic, call volumes voice, inflection, etc, to detect mood
loads of AI based solutions, which are increased but the number of customer and modify behaviour accordingly.
integrated with Google Contact Center service agents on duty fell. Calls Understanding voice nuances and
AI services for virtual assistance. GCP’s increased by nearly 300 per cent during serving customers on that basis
Contact Center AI is a hybrid solution the first few months of the pandemic, improves the overall experience.
suitable for on-premises private as well resulting in longer hold times and more Self-service: Self-service helps to
as the public cloud environment. escalations to human agents or other deliver customer support quickly and
Some of its important components, support representatives. efficiently since customers don’t have
which are now in general availablility With about 90 per cent of contact to wait for a human agent. Companies
(GA), are listed below. centres migrating to remote and continue to provide human backup, but
prioritise AI support for their contact
Google Contact Center AI centres. One way businesses can use
AI to deliver round-the-clock support
Dialogflow CX Agent Assist CCAI Insights is through chatbots and interactive
Virtual Agent Conversational AI Sentiment Analysis voice response (IVR) systems.
Chatbots automate a personalised
Figure 1: Google Contact Center AI experience, connect meaningfully with

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 73

Developers InSight

Amazon Transcribe Medical

In a crisis like Covid-19, remote medical consultation is a real need. Amazon Transcribe Medical is a machine learning
based service, which can convert audio instructions given by doctors into text, and can interpret the technical
medical terms used in the audio. It uses machine learning based automatic speech recognition (ASR) to capture the
conversation between the doctor and the patient for making an electronic medical/health record (EHR). This helps
doctors to attend to patients quickly.
As can be seen in Figure 2, a user connects to
Amazon Connect for interactive chats, which is a
User virtual assistant service that works with a given
AWS
knowledge base for answering queries (e.g., the
remedy for a dry eye problem). Lex (chatbot
Amazon Connect (Contract Center) service) is used to fetch the follow-up questions
Email Service (SES) that the user is asked. This is done through a
Rekognition
Lambda function invocation, which searches the
Lambada Function
knowledge base. If the required response is not
Comprehend
found, it moves into the next level of search
Amazon Lex through Rekognition, which helps identify the
Transcribe Medical
face of the user, and connects him or her to their
Step Function preferred doctor’s advice. The advice for various
Amazon ElastSearch
Lambada Function
medical problems is stored as an audio file and
can be searched through Transcribe Medical
service to get the solution for the given query.
S3 Bucket with IAM RDS DataStore
This is then given to Comprehend to convert as a
response to the user in Amazon
Figure 2: Architecture of Amazon Transcribe Medical based contact centre Connect or through email.

customers and deliver engaging content. The competitive advantage: Since deliver experiences that meet
IVRs are like ‘virtual receptionists’. customers like to solve issues at any customers’ demands. Contact center
Callers interact with them through a time and on their own, the quality AI technology also helps mitigate
series of menus, which direct them to the of a contact centre’s AI self-service stressors like long hold times and
most appropriate help for their needs. adds hugely to the value of a contact repetitive questions. Businesses that
24/7 customer service: Contact centre team. use AI for their contact centre achieve
centre AI enables customer service Contact centre AI is intuitive higher year-on-year revenue and profit
during every hour of the day, as it does and accurate, and helps businesses margins.
not need to sleep or take breaks.
Companies can strategise staff shifts References
for 24/7 service, taking the weight off  https://blog.hubspot.com/service/artificial-intelligence-call-center
of individuals expected to be on call at  https://callcenterstudio.com/blog/advantages-of-artificial-intelligence-in-call-
all hours of the night. Utilising AI for centers-blogs/
24-hour contact service helps companies  https://www.vonage.com/resources/articles/artificial-intelligence-contact-center/
 https://www.destinationcrm.com/Articles/CRM-News/CRM-Featured-Articles/AWS-
to allow fewer irregular work hours for
Expands-Its-AI-Powered-Contact-Center-Intelligence-(CCI)-144700.aspx
employees, while making sure customers  https://azure.microsoft.com/en-in/services/cognitive-services
receive help whenever they need.  https://cloud.google.com/solutions/contact-center
Enabling cost-effective pricing:  https://cloud.google.com/blog/products/ai-machine-learning/discover-advanced-
Like any investment in new technology, insights-with-google-cloud-natural-language-ai
 https://www.vonage.com/resources/articles/the-future-call-center-10-predictions-for-
businesses are wary of contact centre AI
the-next-10-years-2/
pricing too. However, research shows
that investing in quality AI for contact
By: Dr Anand Nayyar and Dr Magesh Kasthuri
centres will save businesses money
and increase the quality of service Dr Anand Nayyar is a PhD in wireless sensor networks and swarms intelligence.
He works at Duy Tan University, Vietnam, and loves to explore open source
provided over time. Contact centre AI technologies, IoT, cloud computing, deep learning and cyber security.
also helps generate revenue by allowing
Dr Magesh Kasthuri is a senior distinguished member of the technical staff and principal
allocation of employee hours to other consultant at Wipro Ltd. This article expresses his views and not that of Wipro.
key business functions.

74 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 Let’s Try Developers

Twelve
Python Tips for Beginners
In a previous article published in the March 2021 edition of Open Source For You
(‘Python: A Few Useful Tips for Beginners’), the author of this article had written some
simple but useful tips for those beginning to learn the Python language. He has now
compiled twelve more tips that are interesting and useful for Python programmers.
These tips will help write concise and easy-to-read code.

A
s per both the TIOBE index and the PYPL particular variables and the remaining values to some other
(PopularitY of Programming Language) site, as variable, the ‘*’ notation for unpacking can be used in Python.
on September 2022, Python is the most popular
programming language and has retained its popularity >>> numlist1 = [ 1,2,3,4,5 ]
for almost three years now. This is because of its ease of >>> num1, *numlist2 = numlist1
programming and the availability of a very large number of >>> print ( “num1 = “, num1 )
libraries covering a large number of areas, right from business num1 = 1
and finance to scientific computing and AI. It is also easy to >>> print ( “ numlist2 = “, numlist2 )
learn compared to other languages like C/C++ and Java. numlist2 = [ 2,3,4,5 ]
So let’s begin with the Python tips that will help you write
concise code easily. Unpacking is used in the assignment statement below to
assign values from the list to three variables simultaneously,
Use of underscore to separate digits and the ‘*’ is used to assign all the remaining values.
of long numbers
It is difficult to count the number of digits if a number, be >>> numlist1 = [ 1,2,3,4,5 ]
it an integer or floating-point number, is very long. Python >>> num1, *numlist2, num3 = numlist1
allows the use of underscores (_) as visual separators for >>> print (“num1 = “, num1)
grouping digits to make it easier to read the number. Hence, num1 = 1
the number 1000000000 can be written as 1_000_000_000. >>> print ( “numlist2 = “, numlist2 )
numlist2 = [ 2, 3, 4 ]
>>> num = 1_000_000_000 >>> print ( “num3 = “, num3 )
>>> num num3 = 5
1000000000
Use of underscore in unpacking
List unpacking and partial assignment The underscore (_) can also be used to ignore specific values
If we need to assign some of the elements of a list to while unpacking. The value that has to be ignored is assigned

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 75

Developers Let’s Try

to underscore (_). >>> numlist2 = [ i for i in numlist1 if i % 7 == 0 ]

>>> print ( numlist2 )
>>> num1, _, num2 = [ 1,2,3 ] [ 35, 49, 63, 84 ]
>>> print ( num1 )
1 zip() function
>>> print ( num2 ) The zip() function is an inbuilt function in Python that takes
3 two or more iterable contents passed as arguments like list,
dictionary, string or even iterables that are user defined, and
The ‘*’ symbol is used as a prefix to underscore (_) to combines them to create a single tuple, an iterator object. The
unpack multiple values and assign them to a list. zip function combines the elements with the same index in
pairs (or more if the number of iterators is more) in a tuple.
>>> num1, *_, numlist2 = [ 1,2,3,4,5,6 ] The first item of each argument iterator is combined, then the
>>> print ( num1 ) second is combined, and so on, until the length of the shorter
1 iterable is covered, by default, to create a tuple object. Given
>>> print ( numlist2 ) below is a simple example:
6
>>> names = [ “Ganesh”,”Siva Sundar”,”Gunasekhar”,”Santosh” ]
Getting a password from a user >>> marks = [ 90, 88, 75, 72 ]
An easy way to get a password as input from the user is to >>> marklist = tuple ( zip ( names, marks ) )
use the getpass function defined in the getpass module. The >>> print ( marklist )
function displays a prompt and then prevents echo on the ((‘Ganesh’, 90), (‘Siva Sundar’, 88), (‘Gunasekhar’, 75),
display when the user types the password. (‘Santosh’, 72))

!pip install getpass The zip function is extremely powerful as it can take
iterators like strings and dictionaries as arguments. It can also
from getpass import getpass take more than two iterables as arguments, and the iterables
uname = input ( “Username: “ ) can be of varying lengths. It even gives options for iterating
password = getpass ( “ Password: “ ) all the elements of the longer tuple if the iterable arguments
are not of the same length.
List comprehension
List comprehension offers an easy way to create a new list Combining two lists into a dictionary
in a single line, compared to the traditional methods. An We can combine two lists and create a dictionary by using the
example is given below. If we want to create a new list using zip function.
the for loop, the code will be as follows:
>>> names = [ “Ganesh”, “Siva Sundar”, “Gunasekhar”,
#create a list containing the cube of the first 9 integers “Santosh” ]
>>> cubelist = [ ] >>> marks = [ 90, 88, 75, 72 ]
>>> for i in range ( 1,10 ): >>> marklist = dict ( zip ( names, marks ) )
>>> cubelist.append ( i * i * I ) >>> print ( marklist )
>>> print ( cubelist ) {‘Ganesh’: 90, ‘Siva Sundar’: 88, ‘Gunasekhar’: 75,
[ 1, 8, 27, 64, 125, 216, 343, 512, 729 ] ‘Santosh’: 72}

The same can be written using list comprehension in the The pass keyword
following way in a single line of code: The pass keyword can be used as a placeholder for incomplete
code. This keyword is a statement by itself and results in NOP
>>> cubelist = [ i * i * i for i in range (1,10) ] (no operations) as it is discarded during the byte-compile phase
>>> print ( cubelist ) and nothing is executed. It is useful for code that you want to
add later. Two examples are given below.
You can even use it for conditional logic and filtering.
Example 1:
>>> numlist1 = [ 35, 49, 45, 75, 90, 15, 65, 63, 84, 54 ]
>>> #create a new list whose elements are divisible by 7 def passDemo ():

76 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 Let’s Try Developers

pass >>> y= 45
passDemo () >>> eval(‘x * y’)
315
Example 2:
>>> eval (‘2 ** 8’)
marks = 80 256
if marks >= 75:
pass >>> z = eval(‘x * y + 4’)
>>> print (z)
Counter function in collections module 319
The collections module of Python has different types of
containers. Counter, one of the containers in the collections Factorial
module, is a dict subclass that provides an efficient way Factorial of a non-negative integer ‘n’ is the product of all
to count objects. The keys component of the dictionary positive integers less than or equal to ‘n’, but greater than or
stores the object, whose occurrence has to be counted, and equal to 1. The math module of Python has a function named
the values component of the dictionary holds the object’s factorial to compute the factorial of an integer passed as an
count, that is, its frequency. Counter takes an iterable list argument to the function.
as an argument and returns a dict object containing each
distinct item in the iterable list and its frequency as a key- >>> import math
value pair. For example: >>> num = 9
>>> fact = math.factorial ( num )
>>> from collections import Counter >>> print ( fact )
>>> alphabets = [ ‘a’, ‘b’, ‘c’, ‘d’, ‘b’, ‘a’, ‘b’, ‘c’, 362880
‘a’, ‘b’, ‘c’ ]
>>> num_alphabets = Counter ( alphabets ) Playing sound notifications in Python
>>> print ( num_alphabets ) If you need to notify the user of the sound of some event, you
Counter({‘b’: 4, ‘a’: 3, ‘c’: 3, ‘d’: 1}) can use the chime module to generate the standard sounds.

Notice that the key-value pairs are ordered as per import chime
the descending order of the count of the occurrences in chime.info ()
the iterable. The Counter can also be used to merge two chime.success ()
dictionaries. The keys are preserved and the values of chime.error ()
matching keys are added. For example: chime.warning ()

>>> from collections import Counter The above tips will help programmers code faster, create
>>> marks1 = { ‘Ganesh’: 90, ‘Siva Sundar’: 88, ‘Gunasekhar’: more efficient and concise code, and will increase their
75, ‘Santosh’: 72 } productivity. The use of powerful features like zip, Counter
>>> marks2 = { ‘Ganesh’: 90, ‘Siva Sundar’: 88, ‘Gunasekhar’: and list comprehension makes the code concise and easy to
75, ‘Santosh’: 72, ‘Shashank’:78 } understand. Happy Pythoning!
>>> finalmarks = Counter( marks1 ) + Counter ( marks2 )
>>> print ( finalmarks )
Counter({‘Ganesh’: 180, ‘Siva Sundar’: 176, ‘Gunasekhar’:
References
150, ‘Santosh’: 144, ‘Shashank’: 78})  https://www.tiobe.com/tiobe-index/
 https://pypl.github.io/PYPL.html
 https://www.opensourceforu.com/2021/04/python-a-few-
useful-tips-for-beginners/
Expression evaluation using the eval function
The built-in eval function accepts a string as an argument. If
the string is an expression, then the expression is evaluated. By: Sathyanarayanan S.
This is useful for evaluating any Python expression that comes The author is working with Sri Sathya Sai University for
as an input dynamically. For example: Human Excellence, Gulbarga. He has more than 30 years
of experience in systems management and in teaching IT
courses. He is an enthusiastic promoter of FOSS.
>>> x = 7

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 77

Developers InSight

AI: Anaconda and

More on Probability
In the previous article in this series on AI and machine learning, we began by
exploring the intricacies of using TensorFlow, a very powerful library used for
developing AI and machine learning applications. Then, we discussed probability
and paved the way for many of our future discussions. In this article, the fifth in the
series, we will continue exploring concepts in probability and statistics.

AI market by these companies in the giants like Amazon, Meta (Facebook),

near future? Is it going to be just a Alphabet (Google), Microsoft, IBM,
couple of million, or a few billion or etc, are doing so. But, surprisingly,
maybe even a few trillion? companies like Shell, Johnson &
For the predictions and data Johnson, Unilever, Walmart, etc, whose
stated here, I depended on articles primary focus is neither computing nor
published in respected newspapers in information technology, are also in the list
the recent past rather than on scholarly of big companies heavily investing in AI.
articles which I couldn’t grasp well And yes! Amazon is a tech company and
to understand the complex dynamics not an e-commerce company.

T
o begin with, let us first install behind the development of an AI So it is clear that many of the giant
Anaconda, a distribution of based economy. An article published companies in the world believe that
Python for scientific computing by Forbes in had 2020 predicted that AI is going to play a prominent role
especially useful for developing AI, companies would spend 50 billion in the near future. But what changes
machine learning and data science dollars on AI in financial year 2020. and new trends are these leaders
based applications. Later, we will Well, that is a really huge investment, expecting in the future? Again, to
also introduce a Python library called even if we acknowledge that the find some answers, I depended on
Theano in the context of probability American billion (109) is just one- news articles and interviews, rather
theory. But before doing that, let us thousandth of the British billion (1012). than on scholarly articles. The terms
discuss the future of AI a little bit. An article published in Fortune, another frequently mentioned in context with
While going through the earlier American business magazine, reported future trends in AI include responsible
articles in this series for evaluation that venture capitalists are shifting some AI, quantum AI, IoT with AI, AI and
and course correction, I wondered of their focus from artificial intelligence ethics, automated machine learning,
whether my tone was a bit sceptical to newer and trendier fields like Web3 etc. I believe these are vast topics and
at times while discussing the future and decentralised finance (DeFi). But we will discuss some of these terms
of AI. By being brutally honest about The Wall Street Journal (an American (except AI and ethics, which we already
some of the topics we discussed, I may business-focused newspaper) in 2022 discussed in the last article) in detail, in
have inadvertently demotivated a tiny confidently predicted that, ‘Big Tech the coming articles in this series.
fraction of the readers and potential Is Spending Billions on AI Research.
practitioners of AI. This made me Investors Should Keep an Eye Out’. An introduction to Anaconda
research the financial aspects of AI and What about the Indian scenario? Let us now move to discussing the
machine learning. I wanted to identify Business Standard reported in 2022 that necessary tech for AI. From this article
the kind of companies involved in the 87 per cent of the Indian companies will onwards, we will start using Anaconda
AI market. Are the bigwigs heavily hike their AI expenditure by 10 per cent whenever necessary. Anaconda is
involved or are there just some startups in the next three years. So, overall, it a distribution of the Python and R
trying to move away from the pack? looks like the future of AI is very safe and programming languages for scientific
Another important question was: how bright. But which are the top companies computing. It helps a lot by simplifying
much money will be pumped into the investing in AI? As expected, all the package management.

78 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 InSight Developers

Now, let us install Anaconda. ‘shasum -a 256’. You will see a hash if the hash values match, execute the
First, go to the web page https:// value displayed on the terminal, following command on the terminal to
www.anaconda.com/products/ followed by the name of the installer. In begin the installation:
distribution#linux and download my case, the line displayed was:
the latest version of the Anaconda bash Anaconda3-2022.05-Linux-x86_64.sh
distribution installer. Open a terminal in a7c0afe862f6ea19a 596801fc138bde0463
the directory to which the distribution abcbce1b753e8d5c474b506a2db2d But in case you have downloaded
installer is downloaded. As of writing Anaconda3-2022.05-Linux-x86_64.sh a different installer, then use the
this article (October 2022), the latest appropriate name after the command
Anaconda distribution installer for Now, go to the web page https:// bash. Now, press Enter, scroll
systems with 64-bit processors is docs.anaconda.com/anaconda/install/ down to go through the agreement
Anaconda3-2022.05-Linux-x86_64.sh. hashes and go to the hash value of the and accept it. Finally, type ‘yes’ to
In order to check the integrity of the corresponding version of the installer start the installation. Whenever a
installer, run the following command on you have downloaded. Make sure prompt appears, stick to the default
the terminal: that you continue with the installation option provided by Anaconda
of Anaconda only if the hash value (unless you have a very good reason
shasum -a 256 Anaconda3-2022.05- displayed on the terminal and the hash not to do so). Now, the Anaconda
Linux-x86_64.sh value given in the web page are exactly installation is finished.
the same. If the hash values do not Anaconda installation by default
If you have downloaded a different match, then the installer downloaded also installs Conda, a package manager
version of the installer, then use the is corrupted. In this case, restart the and environment management system.
name of that version after the command installation process once again. Now, Wikipedia says that the Anaconda

Figure 1: Anaconda Navigator

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 79

Developers InSight

distribution comes with over 250 Introduction to Theano manually install Theano. But with the
packages automatically installed, with Now, let us try to work with Theano, help of Anaconda, all we need to do is
the option of installing over 7500 a Python library and optimising press the button Apply on the bottom
additional open source packages. The compiler for evaluating mathematical right corner of this pop-up menu. After a
most important thing to remember is that expressions. The installation of Theano while, the installation of Theano will be
whatever package or library you have is very easy since we have Anaconda completed smoothly. Now, we are ready
installed by using Anaconda can be used Navigator. First, open Anaconda to use Theano in our Jupyter Notebooks.
in your Jupyter Notebook also. Whatever Navigator. On the top right corner We are already familiar with the
updation is required by other packages or you will see the button Environments Python library called SymPy used
libraries during the installation of a new (marked with a red box in Figure 1). for symbolic calculation but Theano
package will be automatically handled by Press that button. You will go to a takes it to the next level. Let us see
Anaconda carefully, without bothering window showing the list of all currently a simple example to understand the
you in the least. installed packages. From the drop- power of Theano. Consider the code
So, finally, we have reached a place down list on top, choose the option Not shown in Figure 3. First, let us try to
in our journey where we no longer installed. Scroll down, find the option understand the code. Line 1 of the
need to worry about installing the new Theano, and click on the checkbox on code imports Theano. Line 2 imports
packages and libraries necessary to the left. Now press the green button tensor and names it T. We have already
continue our quest for developing AI and named Apply at the bottom right corner encountered tensors when we discussed
machine learning based applications. of the window. Anaconda will find TensorFlow. Mathematically, tensors
Notice that Anaconda only has a out all the dependencies for installing can be treated as multi-dimensional
command-line interface (CLI). Are we in Theano and show them in a pop-up arrays. Tensor is one of the key data
trouble? No! Our installation also gives menu. Figure 2 shows the pop-up menu structures used in Theano and can be
us Anaconda Navigator, a graphical user I got during the installation of Theano in used to store and manipulate scalars
interface (GUI) for Anaconda. Execute my system. You can see that in addition (numbers), vectors (1-dimensional
the command ‘anaconda-navigator’ on to Theano, one more new package is arrays), matrices (2-dimensional arrays),
the terminal to run Anaconda Navigator installed and eight other packages are tensors (multi-dimensional arrays), etc.
(Figure 1). Soon, we will see an modified. Imagine how difficult this In Line 3, a Theano function called
example of its power. would have been if I was trying to function( ) is imported. Line 4 imports
a Theano function called pp( ), which
is used for pretty-printing (printing in
a format appealing to humans). Line
5 creates a symbolic scalar variable of
type double called x. It is a bit tricky
to understand these sorts of symbolic
variables. Think of it as a 0-sized
double variable with no value or storage
associated. Similarly, Line 6 creates
another symbolic scalar variable called
y. Line 7 acts like a function in a sense.
This line tells the Python interpreter
something like the following, ‘if and
when symbolic scalar variables x and y
get some values, add those values and
store inside me’.
To further illustrate symbolic
operations at this point, let us try to
understand Line 8. From Figure 3, it
is clear that the output of this line of
code is (x+y). So, it is clear that actual
addition of two numbers is yet to take
place. Lines 9 to 11 similarly define
Figure 2: Installation of Theano symbolic subtraction, multiplication,

80 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 InSight Developers

and division, respectively. If you want Baby steps with

further clarity, use the function pp( ) to probability
find the values of b, c, and d. Line 12 is Now, let us continue discussing
very critical. It defines a function named probability and statistics. I had
f( ) using the function function( ) of suggested in the last article (while
Theano. The function function( ) takes introducing probability) that
two parameters. Parameter 1 is the data you carefully go through three
to be operated upon and parameter 2 is Wikipedia articles, and with that
the function to be applied on the data assumption of familiarity I tried
given as parameter 1. Here, the data to motivate you by discussing the
is the two symbolic scalar variables Figure 3: Our first code using Theano Normal distribution. However,
x and y. The function is given by [a we must revisit some of the
b c d] which represents symbolic basic notions of probability
addition, subtraction, multiplication, and statistics before we start
and division, respectively. Finally, in developing AI and machine
Line 13, the function f( ) is provided learning based applications. I
with actual values rather than hope all of you have heard about
symbolic ones. The output of this arithmetic mean and standard
operation is also shown in Figure 3. It deviation (SD). Arithmetic mean
is very easy to verify that the output can be thought of as the average
shown is correct. Figure 4: Manipulating matrices with Theano of a set of values. SD can be
Now, let us see how matrices can thought of as the variation or
be created and manipulated using dispersion of a set of values.
Theano. Consider the code shown in If the SD value is low then
Figure 4. It is important to note that the the elements in the set tend to
first three lines of code shown in Figure be closer to the mean. On the
3 should be added here also, if you are contrary, if the SD value is high
writing this program independent of Figure 5: Arithmetic mean and standard deviation then the elements of the set are
the programs we have discussed so far. spread out over a wider range.
Now, let us try to understand the code. is the function to be applied on the But how can we calculate arithmetic
Line 1 of the code creates two symbolic data given as parameter 1. But this mean and SD using Python? There is
matrices x and y. Think of x and y as time, the data is the two symbolic a module called statistics available
two 0-dimensional arrays. But this matrices x and y. The function is with Python which can be used to find
time these matrices are of type integer given by [a b c] which represents mean and standard deviation. However,
unlike the last time where the data type symbolic addition, subtraction, and experts are of the opinion that this
was double. Further, this time a plural multiplication, respectively. Finally, module is very slow and hence we
constructor (imatrices) is used so that in Line 6, the function f( ) is provided choose NumPy. Now, consider the code
more than one matrix can be constructed with actual values rather than symbolic shown in Figure 5.
at the same time. Lines 3 to 5 perform ones. The two matrices given as input The code prints the mean and
symbolic addition, subtraction, to f( ) are [[1, 2], [3, 4]] and [[5, 6], standard deviation of two lists C1 and C2
and multiplication, respectively, on [7, 8]]. The output of this operation (whose values are hidden from you for the
symbolic matrices x and y. Here again is also shown in Figure 4. It is very time being because of obvious reasons).
you can use print(pp(a)), print(pp(b)), easy to verify that the three output What interpretations can you make from
and print(pp(c)) to better understand matrices shown are correct. Notice these values? Nothing! They are just
the symbolic nature of the operations that, in addition to scalars and matrices numbers for you right now. But what if
being performed. If you add these (for both of which we saw examples I tell you the lists contain the marks of
lines of code, you will get (x+y), now), tensor also offers constructors six students, studying in 10th standard in
(x-y), and (x \dot y), respectively, as like vector, row, column, different school A and school B, for an exam in
output. Line 5 generates a function types of tensors, etc. Let us stop mathematics (the exam is out of 50 with
f( ) which takes two parameters as discussing Theano for now and revisit a pass mark of 20). The mean value tells
before. Again, parameter 1 is the data it while discussing advanced topics in us that students from both the schools
to be operated upon and parameter 2 probability and statistics. have relatively poor average marks with

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 81

Developers InSight

school B slightly outperforming school A. After the loop is iterated

But what does the standard deviation value a 1000 times, Line 8 will
tell us? To further your understanding, be executed. Line 8 prints
I will give you the two lists, ‘C1 = [20, the ratio between the
22, 20, 22, 22, 20]’ and ‘C2 = [18, number of occurrences of
16, 17, 16, 15, 48]’. Though hidden by the number 6 and the total
the mean, the large standard deviation Figure 6: Pseudo random number generation number of trials (1000).
value of 11.813363431112899 clearly Figure 7 also shows this
captures the mass failure in school B. This number as 0.179 (slightly more than the
example clearly tells us that we need more expected value 1/6 = 0.1666…which is
complicated parameters to understand the also printed in the output). Not that close
complex nature of the problems we deal to the expected value, right? In Line 2,
with. Probability and statistics will come set the value of n as 10000. Run the code
to our rescue here by providing more and again and observe the new output printed.
more complex models which can imitate Most probably you will get a number
complex and chaotic data. Figure 7: Illustrating the law of large numbers which is a little bit closer to the expected
Random number generation is an value. Notice that this could also be a
essential part of probability. But, in shows the output of this code, where number less than the expected value.
practice, only pseudorandom number the option ‘Tails’ is chosen for three Increase the value of n, in Line 2, for a
generation (a sequence of numbers consecutive times. few more times. You will observe that the
whose properties approximate the Now, let us try to illustrate a very output is inching closer and closer to the
properties of sequences of random popular theorem in probability, the law expected value. Thus, with a simple code,
numbers) is possible. Now, let us see a of large numbers (LLN), with a simple we have illustrated LLN.
few functions that will help us generate example. LLN states that the average Though a simple statement, you
pseudorandom numbers. Consider the of the results obtained from a large will be amazed to know that the list of
code shown in Figure 6. First, let us try number of trials should be close to the mathematicians who worked on a proof
to understand the code. Line 1 imports expected value. Further, this average for LLN or tried to refine one include
the random package of Python. In Line 2, tends to come closer and closer to the Cardano, Jacob Bernoulli, Daniel
the function random.random( ) generates expected value as more and more trials Bernoulli, Poisson, Chebyshev, Markov,
random numbers. This line of code, are performed. We all know that if a Borel, Cantelli, Kolmogorov, Khinchin,
‘new_list = [random.random() for i in fair dice is thrown, the probability of etc. All of them are mathematical giants
range(2)]’, uses a technique called list getting the number 6 is 1/6. We now in their own field.
comprehension to generate two random simulate this experiment with a simple We are yet to cover topics in
numbers and store them in the list Python code shown in Figure 7. Line 1 probability like random variables,
named new_list. Line 3 prints this list imports the random package of Python. probability distributions, etc, which
as output. Notice that the two random Line 2 sets the number of trials to be are essential in developing AI and
numbers printed change with each performed. In this case, it is 1000. Line machine learning based applications. Our
iteration of the code, and the probability 3 initialises the counter ct to zero. Line discussion of topics in probability and
of getting the same numbers printed 4 sets a loop, which iterates 1000 times statistics is still in the early stages and we
twice consecutively is theoretically zero. in this case. Line 5 uses the function will try to strengthen our knowledge of
The single line of code in the second random.randint(1, 6). In this case, the these subjects in the next article also. At
cell shown in Figure 6 uses the function function generates an integer between the same time, we will revisit two of our
random.choice( ). This function makes 1 and 6 (inclusive of both 1 and 6) old friends we met in this journey earlier,
an equally likely choice from all the randomly. The if statement in Line 5 the libraries Pandas and TensorFlow. We
choices given to it. In this case, the code checks whether the number generated will also befriend a relative of TensorFlow
fragment random.choice([“Heads”, is equal to 6; if yes the control goes to called Keras, a library which acts as an
“Tails”]) will make a choice between Line 7 and increases the counter ct by 1. interface for TensorFlow.
‘Heads’ or ‘Tails’ with equal probability.
Notice that this line of code also uses By: Deepu Benson
the technique called list comprehension The author is a free software enthusiast. His area of interest is theoretical computer
so that three successive choices between science. The open source tools of his choice include ns-2 and ns-3. He maintains
‘Heads’ or ‘Tails’ are made. Figure 6 his own technical blog too.

82 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 How To Developers

Build a Web Application

on Firebase
Firebase is a platform backed by Google for building, growing and deploying
web apps, mobile apps and games. This article illustrates developing a
complete single-page web application on Firebase using its services, namely,
FireStore, authentication and hosting. The readers can follow the steps given
here and develop a web application with ease.

T his article will handhold you to develop a simple web

application to maintain a journal of events.
Architecture
The application is designed around the Firebase cloud
platform. The platform offers services, scale, SDKs, tools and
Requirements well-maintained documentation. The following services are
To start with, let us specify a clear requirement. Requirement used for this application.
specification is one of the most important aspects of the ƒ FireStore: The real-time database service for storing the
development process. Here is the list of requirements. journal entries.
ƒ The journal should be accessible only via login. ƒ Hosting: The service to host the web application.
ƒ The login credentials are predefined by the system ƒ Authentication: The service to offer login-based access to
administrator, and consist of email and password. the application.
ƒ A logged-in user should be able to add a new ƒ Firebase SDK: The NPM module that offers commands and
entry into the journal. communication API. The JS client uses this API to interact
ƒ A logged-in user should be able to view the entries with the FireStore and authentication services at runtime.
in the journal. The architecture is depicted in Figure 1.
ƒ A logged-in user should be able to sort the entries So what do you need to get started with building the
chronologically. application? Here is the list of basic prerequisites.
ƒ Each entry must auto-fill the details like date of ƒ An account with Google (like a Gmail account)
entry and author. ƒ Node 10+ distribution on the development machine

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 83

Developers How To

ƒ An IDE like Visual Studio Code 1. Create the project

ƒ A web browser Visit https://firebase.google.com/ and sign in using your
Gmail account. It opens the Firebase console to add or create
Procedure a new project. We used glarimy-firebase as the name for the
The following sections give the step-by-step procedure to project. You may or may not select Google Analytics in the
be followed to configure Firebase, and develop and deploy project creation wizard. The Project Review page opens up
the application. once the project is created successfully. This page lists various
services and tools available in different categories. Expand the
Build category on the left pane of the overview page to locate
Authentication, Firestore Database and Hosting services that
we intend to use.

2. Set up the database

We intend to use Firestore Database to store the journal
entries. Note that Firestore is a NoSQL database. It organises
the data into collections of documents. Once the database is
enabled, you can start creating collections.
Click on the Firestore Database under the Build category.
It opens the Cloud Firestore page. Create a new database
by clicking on the Create Database button. Choose Start
in Production Mode and also choose an appropriate Cloud
Firestore location. We chose asia-south1. With this the
Figure 1: Architecture database is ready.

Figure 2: Database setup

84 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 How To Developers

Figure 3: Authentication setup

3. Set up authentication
We intend to control access to the application via a login.
For that, we need to configure authentication.
Click on Authentication under the Build category of
the Project Review pane. It opens the Authentication page.
Click on Get Started. Choose and enable email/password as
the sign-in provider. In other words, users must provide an
email-id and password to sign in to the web app.
We want to restrict the app to a few predefined users.
So, add new users under the User tab, as shown in Figure
3. We want only the users in this list to be able to sign
in to the application.
Do not worry about the privacy of the passwords. Once
the admin enters the details, no one (including the admin)
can view the password again in a readable format.

4. Develop the client

Click on Hosting under the Build category of the Project
Review pane. It opens the Hosting page. Click on Get
Started. The wizard looks as shown in Figure 4. Follow the
instructions to install Firebase SDK on your machine and
develop the client application.
At Step 4 of the wizard, make a note of the
generated firebaseConfig. In our case, it looks somewhat
like the following:

Figure 4: Firebase Hosting setup const firebaseConfig = {

apiKey: “AIzaSyXXXXXXXXXXXXXXwwvZgtbrnLiqKFPE”,
Now, create a collection with the name journal-entries. authDomain: “glarimy-firebase.firebaseapp.com”,
Create a dummy document with an auto-generated ID along projectId: “glarimy-firebase”,
with the structure, as shown in Figure 2. The document consists storageBucket: “glarimy-firebase.appspot.com”,
of three fields, namely, author (string), date (number to store the messagingSenderId: “7822123479139”,
time in UNIX epoch format) and description (string). appId: “1:782281879139:web:aad87vdvcdeee3914d256”,
All the journal entries will be stored in this journal-entries measurementId: “G-F86URYCS0X”
collection at runtime. };

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 85

Developers How To

By now, you will have a project folder created on <input type=’submit’ value=’Login’ id=’login’ />
your development machine with various folders and <input type=’reset’ value=’Cancel’ id=’resent’ />
files generated. Locate the folder named public in the </div>
project folder. We can develop the client application in </div>
this folder. The application consists of four important
kinds of files like in any other web client: 1) The HTML <div id=’form’>
files, 2) The CSS files, 3) The JS script files, 4) Other <h2>Record Your Story</h2>
assets like images, etc. <input type=’date’ id=’date’>
Here is the public/index.html file for the application: <div>
<textarea id=’description’></textarea>
<!DOCTYPE html> </div>
<html> <button id=’submit’>Record</button>
</div>
<head>
<meta charset=”utf-8”> <div id=’journal’></div>
<meta name=”viewport” content=”width=device-width, initial- </body>
scale=1”>
<title>Glarimy Journal</title> </html>
<link rel=”stylesheet” href=’./styles.css’>
<script defer src=”/__/firebase/8.4.1/firebase-app.js”></ Observe that the above HTML file is referring to a file
script> named index.js in the same folder. The index.js file consists
<script defer src=”/__/firebase/8.4.1/firebase-auth.js”></ of all the logic. It uses the Firebase SDK extensively. Note
script> that this is the file that also consists of the firebaseConfig.
<script defer src=”/__/firebase/8.4.1/firebase-database. Here is the code:
js”></script>
<script defer src=”/__/firebase/8.4.1/firebase-firestore. const firebaseConfig = {
js”></script> apiKey: “AIzaSyXXXXXXXXXXXXXXwwvZgtbrnLiqKFPE”,
<script defer src=”/__/firebase/8.4.1/firebase-messaging. authDomain: “glarimy-firebase.firebaseapp.com”,
js”></script> projectId: “glarimy-firebase”,
<script defer src=”/__/firebase/8.4.1/firebase-storage.js”></ storageBucket: “glarimy-firebase.appspot.com”,
script> messagingSenderId: “7822123479139”,
<script src=’./index.js’></script> appId: “1:782281879139:web:aad87vdvcdeee3914d256”,
</head> measurementId: “G-F86URYCS0X”
};
<body> var db = undefined;
<div id=’top’> var user = undefined;
<h1>Our Story</h1>
<div id=’current’></div> async function login(email, password) {
<div id=’nav’> await firebase.auth().signInWithEmailAndPassword(email,
<button id=’new’> New</button> password);
<button id=’refresh’>Refresh</button> user = firebase.auth().currentUser;
<button id=’latest’> Latest</button> }
<button id=’oldest’> Oldest</button>
</div> function activate() {
</div> document.querySelector(‘#date’).valueAsDate = new Date();
<div id=’auth’> document.querySelector(‘#login’).
<h2>Login Required</h2> addEventListener(‘click’, async function () {
<div><input type=’text’ placeholder=”email” id=’email’ let email = document.querySelector(‘#email’).value;
/></div> let password = document.querySelector(‘#password’).
<div><input type=’password’ placeholder=”password” value;
id=’password’ /></div> await login(email, password);
<div> await fetch(‘desc’);

86 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 How To Developers

document.querySelector(‘#current’).innerHTML = user. document.querySelector(‘#latest’).

email; addEventListener(‘click’, async function () {
document.querySelector(‘#email’).value = “”; await fetch(‘desc’);
document.querySelector(‘#password’).value = “”; document.querySelector(‘#journal’).style.display =
document.querySelector(‘#auth’).style.display = ‘block’;
‘none’; document.querySelector(‘#form’).style.display =
document.querySelector(‘#nav’).style.display = ‘none’;
‘block’; });
document.querySelector(‘#journal’).style.display =
‘block’; document.querySelector(‘#oldest’).
}); addEventListener(‘click’, async function () {
document.querySelector(‘#submit’). await fetch(‘asc’);
addEventListener(‘click’, async function () { document.querySelector(‘#journal’).style.display =
let date = document.querySelector(‘#date’).value; ‘block’;
let description = document. document.querySelector(‘#form’).style.display =
querySelector(‘#description’).value; ‘none’;
if(description.trim().length == 0) });
return; }
if (new Date(date).getTime() > new Date().getTime())
return; async function fetch(order) {
await db.collection(“journal-entries”).add({ let entries = await db.collection(“journal-entries”).
“description”: description, orderBy(‘date’, order).get();
“author”: user.email, document.querySelector(‘#journal’).innerHTML = “”;
“date”: new Date(date).getTime() entries.forEach((entry) => {
}); let record = entry.data();
document.querySelector(‘#date’).valueAsDate = new
Date(); let doe = document.createElement(“div”);
document.querySelector(‘#description’).value = “”; let date = new Date(record.date);
await fetch(‘desc’); doe.innerHTML = `&#128197; ${date.getDate()} ${date.
document.querySelector(‘#journal’).style.display = toLocaleString(‘default’, { month: ‘long’ })} ${date.
‘block’; getFullYear()} ${date.toLocaleDateString(‘default’, {
document.querySelector(‘#form’).style.display = weekday: ‘long’ })}`
‘none’; doe.classList = [‘date’];

}); let description = document.createElement(“div”);

description.innerHTML = record.description;
document.querySelector(‘#new’).addEventListener(‘click’, description.classList = [‘description’]
async function () {
document.querySelector(‘#journal’).style.display = let author = document.createElement(“div”);
‘none’; author.innerHTML = `&#9997; ${record.author}`;
document.querySelector(‘#form’).style.display = author.classList = [‘author’]
‘block’;
}); let div = document.createElement(“div”);
div.id = entry.id;
document.querySelector(‘#refresh’). div.classList = [‘entry’];
addEventListener(‘click’, async function () { div.appendChild(doe);
await fetch(‘desc’); div.appendChild(description);
document.querySelector(‘#journal’).style.display = div.appendChild(author);
‘block’;
document.querySelector(‘#form’).style.display = document.querySelector(‘#journal’).appendChild(div);
‘none’; });
});

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 87

Developers How To

Figure 5: User interface

} platform. The code used for this application is available

at https://bitbucket.org/glarimy/glarimy-university/src/
window.addEventListener(‘load’, async function () { master/glarimy-firebase/ for free access.
firebase.initializeApp(firebaseConfig); You can improve this application with the following
db = firebase.firestore(); additional features:
activate(); 1. Add authentication via sign-in providers like Google,
}); GitHub, Yahoo, Facebook, Twitter, etc.
2. Add a feature to edit the journal entries.
We also need to develop the stylesheet named public/ 3. Add a feature to upload images also along with the
styles.css which is being referenced in the HTML file. Refer journal entries.
to the repository specified towards the end of the article to 4. Add a feature to add tags to the entries and search
find the stylesheet. based on the tags.
5. Add a feature to export the journal entries into XML,
5. Deploy the web app JSON and PDF formats.
Once the application code is ready, deploy the application by 6. Redevelop the client using Typescript/React/Angular
running the following command from the root directory of the instead of basic JavaScript.
application on the development machine: I do hope this article will help you understand the
concepts and get started with the development of a single-
firebase deploy page web application on Firebase.

It provides the URL at which the application is ready. By: Krishna Mohan Koyya
Access the URL from the browser, and login with one of the The author is the founder and the principal technology
user credentials that are configured to use the application. consultant at Glarimy Technology Services, Bengaluru. He
Figure 5 depicts the user interface. has delivered more than 500 programs to corporate clients on
design and architecture in the last 12 years. Prior to that, he held
This is the whole process of developing a simple web various positions in the industry and academia for a decade.
app backed by database and authentication on the Firebase

88 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 InSight Developers

Code Generation:
How the Lazy Become Prolific
Code generators bring the fun back into coding. This article lists their benefits and tells
how they can be used in different ways, efficiently and effectively.

Code generation or better language?

Isn’t code generation like a workaround? Why not pick a
supreme language and use it directly instead of generating
code in multiple stages?
First, the goal of code generation is not always to
overcome language limitations. Just think about auto-
generating HTML documentation for your API. You are not
trying to overcome the limitation of Go or PHP here, right?
Second, general-purpose languages will always have
tradeoffs or weak areas and you’ll have to do something
about it. Think of Go before generics. Think of powerful
languages with unfriendly syntaxes. And even if there
is a superior language, what if you are working on an
existing codebase?
Also, code generators tend to be problem-specific. Not

W
hat does an enthusiastic programmer dream general-purpose, not even domain-specific, but problem-
of? Getting to spend every day trying to specific. That makes them so effective.
solve real-world problems using interesting Finally, who said code generators cannot be languages?
concepts and clever algorithms. And what do we really do There is m4, a macro processor that is Turing-complete, and
as professional developers? Spend hours on a vast set of there is PHP, a templating system, which everybody considers
boring and artificial problems like npm dependency issues. to be more of a programming language.
One of these boring and annoying problems is the The advantages of generative programming span from
creation and maintenance of boilerplate code. That’s when the ease and speed of development to maintainability,
we start exploring code generators. They first help us, then consistency, and memory safety and security. They can also
bring the fun back in, and transport us to a whole different be handy with shared code where librarification is hard or less
frontier. What’s more interesting? This is a rare case of the efficient (microservices, maybe?). And then there is fun.
lazy becoming prolific! Too much talk. Let’s see some examples.
Let’s see what code generators can offer, where they
are used, and how to make their use effective and efficient. Example: OpenAPI
Consider writing an API server. You not only have to
Code generators write the handlers, but add code for routing, security
What exactly do we mean by code generators? Compilers checks, parameter extraction, validation, etc, which most
are the most basic kinds of code generators, generating of the time don’t have anything to do with business logic.
Assembly or machine code from high-level language But things don’t end there. You also have to provide
instructions. There are in-language code generation documentation and client SDKs in various languages, keep
features too, like macros and generics. However, our them updated and in sync.
focus will be on the kinds like preprocessors and This is where OpenAPI comes in. With OpenAPI, you
transpilers, which generally convert between high- specify your API in a YAML or JSON file, and then use
level languages, and those which generate programs or different generators to generate boilerplate code, client SDKs,
snippets from the declarative specifications that we give documentation, etc. All you have to write manually is the
in YAML or JSON. business logic (well, most of the time).

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 89

Developers InSight

Figure 1 shows a sample OpenAPI workflow. Here some That’s it. The YAML file has 43 lines, and handwritten
Go code is auto-generated from the API spec api.yml, which is Go code has less than sixty lines. oapi-codegen generated
combined with manually written Go code (kept in a separate 351 lines that contain parameter extraction, validation, error
file) to build the final server binary. The same YAML file can be messages, etc. Here are the auto-generated validation and
fed into another tool to generate the HTML documentation. error messages in action:

+---------+ oapi-codegen +--------------+

| api.yml | ------------------> | api.gen.go | ---+ $ curl ‘localhost:8080/calc/diff?y=12’
+---------+ +--------------+ | +-----------+ parameter “x” in query has an error: \
| +---> | apiserv |
| +--------------+ | +-----------+ value is required but missing
| | handlers.go | ---+
| +--------------+ $ curl ‘localhost:8080/calc/diff?x=10&y=someString’
| parameter “y” in query has an error: \
|
| openapi-generator -g html +-----------+ value someString: an invalid integer: invalid syntax
+--------------------------------------------------> | html doc |
+-----------+
Example: enum stringification in C
Figure 1: OpenAPI workflow Many times we need to print the name of an enum member
1
in C, usually for logging or debugging purposes. But such
Here is a part from the YAML input, which is a spec for information is not available to the program at runtime. So
an API with two endpoints to find the sum and difference of we end up writing an array that contains string constants
two numbers: representing the enum members, like this:

/calc/sum: // XXX Manually keep in sync with the array opstr

get: typedef enum Operation {
description: Returns the sum of the given numbers. OP_SUM,
parameters: OP_DIFF,
- name: x OP_LARGEST,
in: query OP_SMALLEST,
required: true } Operation;
schema: // XXX Manually keep in sync with the enum
type: integer const char * opstr[] = {
# “y” needs quoting because y is boolean yes in YAML “OP_SUM”,
- name: “y” “OP_DIFF”,
in: query “OP_LARGEST”,
required: true “OP_SMALLEST” };
schema:
type: integer Now opstr[OP_LARGEST] would give us the string
responses: “OP_LARGEST”.
‘200’: But see how we had to put a comment asking future
description: The sum. maintainers to keep the blocks in sync? What if they (read:
us) missed those comments? That would depend on the
Here are the handwritten handlers in Go, which perform mistake. If changes were made in the beginning or middle
the actual logic: for either structure, we’d get incorrect logs, without ever
realising it. If the enum got a new value OP_DIV at the
func (calc Calc) GetCalcSum(w http.ResponseWriter, end and the array was never updated, our program would
r *http.Request, probably crash with a SEGFAULT (because opstr[OP_DIV]
params GetCalcSumParams) { points to invalid memory area now).
fmt.Fprintf(w, “%d\n”, params.X + params.Y) Now let’s write a Makefile that invokes grep and sed to
} automate the generation of the array.
func (calc Calc) GetCalcDiff(w http.ResponseWriter,
r *http.Request, opstr.gen.h: main.c
params GetCalcDiffParams) { (echo ‘// Do not edit this file!’ &&\
fmt.Fprintf(w, “%d\n”, params.X - params.Y) echo ‘const char * opstr[] = { ‘ &&\
} grep -E ‘^\s*OP_[A-Z]+,$$’ main.c|\

90 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 InSight Developers

sed -E ‘s/^\s*(OP_[A-Z]+),$$/ “\1”,/’ &&\ cases. (But you can write your own, right?)
echo ‘};’) > opstr.gen.h Compiler generators are also known for bad error
reporting. This could be one reason some major compilers
Once this file is in place, a simple make command will and interpreters use handwritten parsers.
make sure the array is generated from the enum if it is One could argue that the use of generators makes you
missing, or is regenerated if the enum has been updated. not know what’s happening behind the scenes. But this is
not a hard restriction.
go generate The declarative nature of the input to the generators
Go has parallels to both the stringification concept and can be an advantage and disadvantage at the same time.
make used in the last example. Since Go has its own build Advantage in the sense that it makes sure we have some
system instead of Makefiles, there is a special annotation sort of specification for our APIs, workflows, etc. But the
called go:generate and the go generate command to invoke problem is, we might end up ruining those same specs to
external generators. work around generator limitations.
Rob Pike, famous programmer and one of the creators of
Go, lists its uses in the Go codebase itself in an official blog Some tips
entry (go.dev/blog/generate): Okay, code generators are helpful. But what can we do on
“… generating Unicode tables in the unicode package, our part to make up for their shortcomings and make the
creating efficient methods for encoding and decoding arrays whole process even better? Here are some tips.
in encoding/gob, producing time zone data in the time ƒ Be aware of the licensing restrictions of the particular
package, and so on.” tools you use.
One tool used in combination with go generate is ƒ Use tools like indent and gofmt to reformat the generated
stringer, whose purpose is similar to the enum stringification code so that it becomes easy to read and debug. This will
we saw above. also get rid of formatting-related compiler warnings.
ƒ Use linters and other static analysis tools to check for
In the wild code quality and potential bugs.
Code generators are everywhere — from UNIX to Go and ƒ Make sure to write and run tests, especially for auto-
k8s ecosystems. Here is a quick listing to give you an idea: generated validators. If you’re auto-generating tests,
ƒ Compiler compilers: lex, yacc, etc check them manually and lock them.
ƒ Transpilers ƒ Although it’s an anti-pattern to commit and push
ƒ Build system generators: Automake, CMake, qmake, etc generated artefacts, treat auto-generated code like a
ƒ GUI builders: Glade, Qt Designer, etc regular source and include it in the repo. This will
ƒ Configuration generators: update-grub be helpful for others trying to build your project on
ƒ In the Web world a platform which the generator doesn’t support. It’ll
• CSS pre-processors: SaSS, LESS, Stylus be helpful to you too when the generator becomes
• k8s: kompose, kustomize, helm, etc obsolete in future.
ƒ Interface/binding generators: SIP for Python (used for ƒ Finally, and most importantly, try to make code generation
PyQt, wxPython, etc), PyBindGen, SWIG, etc. part of your pipeline, if it makes sense. I’ll explain.
ƒ Flexible: Telosys from Eclipse (DB/model to any kind of
code based on templates) Making it part of the pipeline
ƒ protoc with gRPC Go plugin Is code generation a one-time process or something that is to
ƒ m4 - a general-purpose macro processor from the 70s, be done every time you build the software?
part of UNIX For some cases, it doesn’t make sense to put the
I’m yet to try some of the above. generation in the pipeline. Maybe you are migrating your
project from one programming language to another, or
Drawbacks choosing a different set of supporting systems so that the
Code generators are not without drawbacks. To begin configuration files need to be migrated. Here it’s clear that
with, they can generate truly ugly code. They’re so you’ll be running the converter only once. The original
mechanical that not even debuggers like gdb can help. source files are archived, generated files become the new
Outputs of lex and bison are examples for this. Sometimes source, and the generator is never run again.
this is for better performance. But generators in general But in some other cases, you are not planning for
can be less efficient too. This is because tools have to be a migration. Maybe you need something in different
general and won’t be able to optimise for individual use formats, or maybe you need boilerplate related to

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 91

Developers InSight

your handwritten code. In such situations you won’t be overhead. Once mature enough, it could be used to generate
archiving or throwing away the original source. Here it C/GTK, C++/Qt, Android and Web apps.
makes sense to put things in a pipeline. The Makefile in Its C output is already mature and is being used to
our C enum stringification example is an example for this. self-host the compiler (i.e., ngg compiler is written in
The source file is checked for updates every time we build ngg itself). nguigen offers more expressiveness and
the software and the generation commands are run if there compile-time semi-automatic memory management when
is something new. generating C code. This is important since the C ecosystem
But what if you had to make manual changes to the remains far superior to many alternatives and one would
generated code and still want it auto-updated whenever the choose to stick with it.
source changes? Let’s see an example. nguigen is not released yet, but the plan is to release it
as free (libre) software. Learn more at nandakumar.co.in/
Don’t give up on the pipeline: kompose example software/nguigen/.
kompose is a tool that helps you generate Kubernetes
YAML files from Docker Compose files. It is intended Codegen in the nguigen ecosystem
to be a one-time process because there is no one-to-one The development of nguigen itself depends on code
mapping between Docker Compose and Kubernetes. So generation. Apart from being written in itself and self-
you are guaranteed to be making changes to the generated compiled to C, parts of the source are auto-generated from
files. But what if you want to keep both Docker Compose mappings written in TSV (tab-separated values) files.
and Kubernetes versions in parallel, auto-applying changes Makefiles with correct dependencies are auto generated
in the Docker Compose while still keeping your edits to using ngg itself and a bash script. This comes handy during
the generated configuration? major refactorings.
I had faced a similar challenge. These were my additions: The parser is also auto-generated from grammar specs.
1. k8s image pull policy and image pull secret, which were Usually lex and bison are used for this purpose, but for
missing in the Docker Compose version nguigen, I wrote my own.
2. Services ignored by kompose because the ports are not And, finally, there is h2ngg, a simple tool to auto-generate
exposed in the Docker Compose version ngg interfaces (like header files) for external libraries like GTK.
3. Configuration for initContainers, which was missing in
the Docker Compose version Write your own code generators
Instead of choosing to convert once and keep both Maybe you’ve already noticed it: this talk is not just about
versions in sync manually, I chose the following solutions, existing code generators. It’s about the advantages and fun
respectively, for each of the above: of generative programming, which can only be enjoyed at
1. Use kompose annotations kompose.image-pull-policy, its maximum if you write your own generators. They benefit
kompose.image-pull-secret in the Docker Compose file from being problem-specific.
itself, so that kompose knows to add them. How to do that? Maybe you can begin with Makefiles and
2. Write files for missing services manually. shell scripts. Use some Python. lex and bison will be overkill,
3. Use a patch file to add the initContainers section. unless you are trying to create a language.
This way, things will be reproducible while still keeping
customisations. Of course it doesn’t make sense to do this if
the differences outnumber similarities. References
 Slides of the talk: nandakumar.org/talks/
nguigen  Demo repo: github.com/nandedamana/lazy-becomes-prolific
I’ve been developing desktop applications for more than These resources cover several things not covered in this article.
a decade. Although trivial, they’ve been used by others, The demo repo contains working examples that you can try
yourself. It is supposed to be updated also.
which means I had to regularly update, port and package
these apps. nguigen (short: ngg) is a project that I started
as a solution to several difficulties I experienced as By: Nandakumar Edamana
part of this.
The author is a developer and technology writer who
It is a general-purpose programming language and primarily focuses on software freedom, privacy and other
a code generator that aims to generate code for multiple ethical issues. He has authored a couple of books and
languages, platforms and GUI toolkits from a single, mostly several articles for mainstream publishers. A GNU/Linux user
for 15+ years, he releases his software products under free
generic codebase. This is different from solutions like software licences.
Electron because we’re going native here and there is zero

92 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 How To Admin

Simplified Logging and

Analysis with Parseable

Parseable is a free and open source log storage and analysis

platform. It is written in the Rust programming language. This
article is a quick introduction to the topic.

W
ritten in Rust, Parseable leverages the advances ƒ Consumer apps thrive on user behaviour, access patterns
in data compression, storage and networking to and other data. All these are logged and passed through
bring you a simple but efficient platform. various ML pipelines to generate better responses,
But before we get into Parseable, let’s understand why notifications, and nudges.
you should even care about logging. ƒ IoT devices are growing and so is the data generated
from these devices. Crunching this data for predictive
Why logging? maintenance and analysis means all this data has to be
For us developers, sometimes it is not very clear as to why so stored for the long term first.
much fuss is made about logging. We debug the code, find out This is just a small list of possible scenarios where log
the issue, fix it and move on. Why does a log even have to be data is critical for business impact. Now that the need for
stored and persisted for a long term? logging and retention of data for long durations is established,
Log is a loaded term used for a large variety of data. But at a let’s take a deeper look at Parseable.
high level, any stream of events that represents the current state
of a software or hardware system can be called a log. You can What is Parseable?
already see the horizon widening now. Consider these scenarios: Parseable is a free and open source, log storage and
ƒ SaaS companies are answerable for any downtime, data analysis platform, released under the AGPL-3.0 licence.
access, security breaches, data loss, and so on. In such Parseable repo is available on GitHub at https://github.com/
cases, it makes sense to retain every bit of log data, so that parseablehq/parseable.
you can not only avoid issues, but also fix them properly The idea behind Parseable is to rethink logging; it’s a
if something happens. modern, blazing fast, cloud native (stateless, easily deployable

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 93

Admin How To

across cloud providers) platform that is ridiculously easy to ingestion as easy as possible. This means you can simply use
deploy and use. Some core concepts on which Parseable is HTTP POST calls to send logs to Parseable; no complicated
built are listed below. SDKs are required. Alternatively, if you want to use a logging
No need for indexing: Traditionally, text search agent like FluentBit, Vector, LogStash, etc, almost all the
engines like Elastic, etc, have doubled up as log storage major log collectors already have support for HTTP; hence,
platforms. This made sense because log data had to be Parseable is already compatible with your favourite log
searched for it to be really useful. But this came at a high collection agent.
cost – indexing is CPU-intensive and slows down the
ingestion. Additionally, most of these indexing systems Getting started
generate index data in the same order of the raw log data. Let’s see how to get started with Parseable now. We’ll use the
This doubles the storage cost and increases complexity. Docker image to try out Parseable.
Parseable changes this. With columnar data formats We’ll use Parseable with a publicly accessible object
(parquet), it is possible to compress and query the log data storage, just to help you experience it. Here is the command:
efficiently without indexing it.
Ownership of both data and content: With parquet cat << EOF > parseable-env
as the storage format, stored on standard object storage P_S3_URL=https://minio.parseable.io:9000
buckets, users not only own their log data but also have P_S3_ACCESS_KEY=minioadmin
unabridged access to the actual content. This means they P_S3_SECRET_KEY=minioadmin
can simply point their analysis tools like Spark, Presto or P_S3_REGION=us-east-1
TensorFlow to extract more value out of the data. This is P_S3_BUCKET=parseable
an extremely powerful feature, opening up new avenues of P_LOCAL_STORAGE=/data
data analysis. P_USERNAME=parseable
Fluid schema: Logs are generally semi-structured by P_PASSWORD=parseable
nature, and they are ever evolving, e.g., a developer may start EOF
with a log schema like this:
mkdir -p /tmp/data
``` docker run \
{ -p 8000:8000 \
“Status”: “Ready”, --env-file parseable-env \
“Application”: “Facebook” -v /tmp/data:/data \
} parseable/parseable:latest
```
You can now log in to the Parseable UI using the
But as more information gets collected, the log schema credentials we passed here, i.e., ‘parseable’ , ‘parseable’. This
may evolve to: will have some data, because Parseable is pointing to the
publicly open bucket.
``` Note that this is a public object storage bucket, which we
{ are using for demo purposes only. Make sure to change the
“Status”: “Ready”, bucket and credentials to your object storage instance before
“Application”: { sending any data to Parseable.
“UserID”: “asdadaferssda”, To get a deeper understanding of how Parseable works
“Name”: “Facebook” and how to ingest logs, please refer to the documentation
} available at https://www.parseable.io/docs/introduction.
} I hope this article helped you get better insights into the
``` logging ecosystem, and that you find Parseable an interesting
project to try out and, hopefully, join the community.
Engineering and SRE teams regularly face schema related
issues. Parseable solves this with a fluid schema approach that
lets users change the schema on the fly. By: Nitish Tiwari
SDK-less ingestion: The current ingestion mechanism to
The author is the founder of Parseable. You can find more
logging platforms is quite convoluted, with several protocols
details at https://github.com/nitisht.
and connectors floating around. Parseable aims to make log

94 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 InSight Admin

More Shell Programming

Secrets Nobody Talks
About
This is the second
part of the article on
shell programming
secrets. (The first part
was carried in the
October 2022 issue of
Open Source For You.)
It covers some other
important behaviour of
bash, particularly that
of text expansions and
substitutions.

t
he famed text-processing abilities of the Bourne ƒ By default, bash will not stop for errors
shell (sh) have been vastly expanded in the I forgot to mention that if you use the -e option of the
Bourne-Again shell (bash). These abilities set command, as in set -e, then you will not be able to
are efficient to a fault and the code can be even more check for error codes of previous statements with the if
cryptic. This makes the bash command interpreter and [ $? -eq 0 ]; then construct. This option makes bash stop
programming language very powerful but also a minefield running a shell script if any of its statements encounter an
of errors. error. If you are doing error handling using if statements,
In the last article published in the October 2022 issue then begin your scripts like this:
of Open Source For You titled ‘Shell Programming Secrets
Nobody Talks About’, you learnt that: #!/bin/bash
ƒ sh and bash are not the same set -u
ƒ if statements check for exit codes, not Boolean values # The rest of your shell script
ƒ [, true and false are programs, not keywords
ƒ Presence/absence of space in variable comparisons and The -u option of the set command, if you remember
assignments makes quite a difference from the last article, ensures that undefined variables are
ƒ [[ is not the fail-safe version of [ not replaced with an empty string and instead cause an
ƒ Arithmetic operations are not straightforward as in other error.
languages In this article, we will focus on how bash performs
ƒ Array operations use cryptic operators text expansions and substitutions. I will only cover what

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 95

Admin InSight

I think are the most important text-processing features. For You can also use any number of modifiers, each preceded
comprehensive information, you will have to study the Bash by a colon (:).
Reference Manual. Bash and several commands such as
sed and grep also use regular expressions to perform text Display (but not execute) last com-
!?leword?:p
processing. ‘Regular expressions’ is a separate topic on its mand containing ‘leword’
own and I will not cover it either. Execute with last command containing
‘leword’ after removing all pathnames
!?leword?:t
History expansion character (!) of last argument (i.e., leave the tail
This feature is available when typing commands at the shell containing the file name)
prompt. It is used to access commands stored in the bash Execute with last command containing
history file. !?leword?:r ‘leword’ after removing the file exten-
sion from the last argument
Execute with last command containing
Execute nth command in bash ‘leword’ after removing pathname and
!n !?leword?:e
history filename from the last argument (leav-
Execute last command ing just the extension)
!!
(Equivalent to !-1) Execute last command containing ‘le-
Execute last command beginning !?leword?:s/
!leword word’ after replacing the first instance
with ‘leword’ search/replace
of ‘search’ with ‘replace’
Execute last command containing Execute last command containing
!?leword? !?leword?:as/
‘leword’ ‘leword’ after replacing all instances of
Execute last command after re- search/replace
‘search’ with ‘replace’
^search^replace placing first occurrence of ‘search’ If you omit the search text (‘leword’) and use the history
with ‘replace’
expansion character with the word designators and the
You can modify the history search using certain word modifiers, bash will search the last command. Until you
designators, preceded by a colon (:). become proficient in using the history expansion character,
use the modifier :p to display the command before you
Execute with 0th word (usually the com- actually execute it.
!?leword?:0 mand) executable in last command
containing ‘leword’ Text expansions and substitutions
Execute with second word of last com- These features are available at the shell prompt and in shell
!?leword?:2
mand containing ‘leword’ scripts.
Execute with last word in last command ƒ Tilde (~): In your commands, bash will expand instances
!?leword?:$
containing ‘leword’ of ~ with the value of the environmental variable
Execute with second word to sixth $HOME, that is, your home directory.
!?leword?:2-6 word in last command containing ƒ ? and *: These are metacharacters. In file descriptors, ?
‘leword’
matches any one character while * matches any number of
Execute with all words up to 6th word in
any characters. If they do not match any file names, bash
!?leword?:-6 last command containing ‘leword’
(Equivalent to !?leword?:0-6) will use their literal values.
Execute with all words of last com- ƒ Brace expansion: You can use comma-separated text
!?leword?:* mand (but not the 0th word) containing strings within curly brackets to generate combinations of
‘leword’ (Equivalent to !?leword?:1-$) strings with their suffixes and/or prefixes. When I start a
Execute with the second word to the new book, I create its folders like this.
last word in last command (but not the
!?leword?:2*
0th word) containing ‘leword’ mkdir -p NewBook/{ebook/images,html/images,image-
(Equivalent to !?leword?:2-$) sources,isbn,pub,ref}
Execute with all words from the 2nd
position to last but-one word and not This command creates folders like this.
!?leword?:2-
the 0th word in the command containing
‘leword’
NewBook
Remember that bash will execute whatever you have NewBook/ref
retrieved from the history with whatever you have already NewBook/pub
typed at the prompt. NewBook/isbn

96 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

 InSight Admin

NewBook/image-sources script) or with variables that you have created.

NewBook/html
NewBook/html/images Substitution Effect
NewBook/ebook If var1 is null or does not exist, var2 is
NewBook/ebook/images ${var1:-var2}
used
If var1 is null or does not exist, value
ƒ Parameter expansion: When bash executes a script, it ${var1:=var2}
of var2 is used and set to var1
creates these special variables for the script. If var1 is null or does not exist, msg is
${var1:?msg}
displayed as error
Shell variable Use If var1 exists, var2 is used but not set
${var1:+var2}
to var1
$0 Name of the shell script
Everything of var after offset number
${var:offset}
Positional parameters or arguments of characters
$1, $2,…
passed to the script Length number of characters of var
${var:offset:length}
Total count of arguments passed to after offset number of characters
$# ${!prefix*} All variables names beginning with
the script
${!prefix@} prefix
$? Exit status of last command
${!var[@]}
All indexes of array variable var
$* All arguments (double-quoted) ${!var[*]}
${#var} Length of value of var
All arguments (individually double-
$@ Value of var without prefix matching
quoted) ${var#drop}
Regex pattern drop
$$ Process ID of current shell/script Empty string if prefix matches Regex
${var##drop}
pattern drop
At the terminal, $0 will usually expand to the shell
program (/bin/bash). Value of var without suffix matching
${var%drop}
Regex pattern drop
On a terminal, you can use the set command to ipso facto
specify parameters to the current shell. Empty string if suffix matches Regex
${var%%drop}
pattern drop
# Displays 0 Changes first letter of var to uppercase
set $?
if it matches letter (any alphabet, * or ?)
${var^letter}
# Displays an empty string and causes a new line If letter is not specified, all first letter(s)
echo $* of var will be changed to uppercase
# Sets hello and world as parameters to current shell Changes any letter of var to uppercase
set -- hello world if it matches letter (any alphabet, * or ?)
# Displays 2 (the number of parameters) ${var^^letter}
If letter is not specified, all letter(s) of
echo $#
var will be changed to uppercase
# Displays world
echo $2 Changes first letter of var to lowercase
# Remove parameters to current shell if it matches letter (any alphabet, * or ?)
set --
${var,letter}
If letter is not specified, all first letter(s)
# Displays 0 (as earlier) of var will be changed to lowercase
set $?
Changes any letter of var to lowercase
The option -- represents the end of options and implies if it matches letter (any alphabet, * or ?)
${var,,letter}
that whatever follows it must be command parameters. If letter is not specified, all letter(s) of
ƒ Command substitution: Instead of backquotes, you can var will be changed to lowercase
use the form $(commands) to capture the output of those Value of var with instances of find
commands for use in some other commands or variables. replaced with replace.
It makes quoting and escaping much easier. ${var/find/replace} If find begins with ‘#’, then a match is
ƒ Variable substitution: You can use these substitutions made at the beginning. A ‘%’ makes it
with command parameters (created by bash for a shell match at the end.

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | NOVEMBER 2022 | 97

Admin InSight

Escaping in double-quoted strings may also be replaced with left and

You can escape: right double quotation marks. They look similar but will result
ƒ Special characters using the backslash (\). To escape the in an error if used in a shell script or in the command line.
backslash character, use double backslashes (\\). I write my books and articles in CommonMark
ƒ Literal text strings by wrapping them in single quotation (MarkDown) and output them as HTML, ODT and PDF
marks (‘ ‘). Bash will not perform any expansions or documents. These documents will not have such errors. When
substitutions. The single-quoted string should not have someone edits the document (before it goes to print) in a
any more single-quotation marks. Bash will not perform rich-text editor such as LibreOffice or Microsoft Office, the
any backslash-escaping either. editor’s autocorrect feature may change ordinary quotation
ƒ Literal text strings by wrapping them in double-quotation marks and backquotes with inverted quotation marks. Just
marks (“ “) but allowing for be aware that this can happen. To avoid mistakes, type the
• $-prefixed variables, expansions and substitutions commands by hand. Do not copy-paste them.
• backslash-escaped characters
• backquoted (` `) command strings Summary
• history-expansion characters I am sure you will also conclude that bash code can be very
cryptic. A lot of production code (industrial-strength shell
# Displays Hello World scripts) is hundreds of lines long. If bash was not so succinct
a=World; echo “Hello $a” and powerful, it would take forever to write the lines. If you
# Displays Hello $a are doing any kind of serious shell scripting, then it is best
a=World; echo ‘Hello $a’ you know all about bash’s myriad secrets. I think I have
# Displays Hello ‘World’ covered enough of them to kindle your interest. You are on
a=World; echo “Hello ‘$a’” your own now.
You will find all this and more in my book ‘Linux
Printer’s error Command Line Tips & Tricks’. It is free for download from
The Bash Reference Manual or even this article may use wrong most popular ebook stores.
characters for the quotation marks. The apostrophe or u+0027
used in single-quoted strings may be replaced with the right By: V. Subhash
single quotation mark or u+2019. The grave accent or u+0060 The author has written over two dozen books including, Linux
used in back quoted strings may be replaced with a left single Command-Line Tips & Tricks, CommonMark Ready Reference,
quotation mark or u+2018. The quotation mark or u+0022 used Cool Electronic Projects and How To Install Solar.

OSFY Magazine Attractions During 2022-23

Month Theme
March 2022 Security, Network Management and Monitoring
April 2022 Open Source Programming (Languages and tools)
May 2022 DevOps Special
June 2022 AI, Deep learning and Machine Learning
July 2022 Database management and Optimisation
August 2022 Cloud Special: Everything from management to implementation
September 2022 Mobile/Web App Development, Optimisation and Security
October 2022 Blockchain and Open Source
November 2022 Open Source and IoT and Edge
December 2022 Data science, Data security, Data Management,Storage and Backup
January 2023 Docker and containers
February 2023 Open Source on Windows and Best in the world of Open Source (Tools and Services)

98 | NOVEMBER 2022 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com