• Cognizant 20-20 Insights

The Internet of Things: QA Unleashed

To seize the IoT high ground, QA organizations need to view software
testing beyond devices and sensors, and think holistically about added
technical complexity that comes with the huge volumes, velocity and
variety of data generated across a smart and connected ecosystem.

Executive Summary Though the influence of IoT is already evident

across different verticals, emerging markets will
The wide gamut of IP-aware and -addressable
witness rapid growth as product designers dream
technology known as the Internet of Things (IoT)
up ways to exploit real-time machine-to-machine
is embedded deeply into our professional and
connectivity and intelligence. This white paper
personal lives. “The Internet of Things installed
captures the aspects of IoT that are relevant to
base will grow to 26 billion units by 2020,” says
QA organizations. Further, it elaborates the para-
Gartner.1 “IoT product and service suppliers will
digm shift required in QA in order to embrace the
generate incremental revenue exceeding $300
technology changes applicable to products and
billion, mostly in services, in 2020. It will result in
services offered in IoT’s wide spectrum.
$1.9 trillion in global economic value-add through
sales into diverse end markets.”
Core Components
The sensors at the heart of IoT are embedded Though multiple definitions exist for IoT, we define
in a wide range of applications, from tweeting it as a network of physical objects that contain
turbines (GE’s Brilliant Machine metaphor) and sensors or embedded technologies to interact
toothbrushes, through smart thermostats (think with the internal or external environment and to
Nest) and personal health monitoring devices take intelligent decisions. Core components of
(think Fitbit). These sensors are transforming how IoT include three different components: things,
we work and live. The combination of sensors and communication and computing (see Figure 2,
applications are making the new smarter cities, next page).
homes, railways, healthcare services and much
more. Further, data analytics from the evolving
• Things: Things are the real-world objects or
devices that include the sensors and embedded
connected world holds immense promise for con- software required to communicate with the
tinuously improving the system. external environment.

cognizant 20-20 insights | april 2015

The Many Manifestations of IoT

46%

Automobile
41% Healthcare

14%

The Intern
Internet of Things
Retail
12% Manufacturing

Figure 1

• Communication: Communication is the core • Computing: Computing is often done on a

networking component that helps the thing mobile device, desktop or server, based on the
communicate with another thing or the amount of data that needs to be processed and
external environment. Typically, communica- analyzed. Computing occurs at two levels: one,
tion protocol is based on the type of network to make intelligent decisions within the system,
such as the WAN, LAN or PAN. The network is and the other to form the vital link for business
4G in the case of WAN; WiFi or Wifi Direct in the to perform big data analysis for understanding
case of LAN; and BLE, Zigbee, ANT+, sub 1GHz IoT user behavior.
in the case of PAN. Sometimes it is possible
that the communication is wired as well.

IoT’s Key Components

Communication Infrastructure
Wired and wireless (Wi-Fi, 4G, Bluetooth,
Zigbee) networks connect Things to the
Internet and each other.

Things
Smart, connected products and other
Things combine processors, sensors
and software with connectivity.

Computing Infrastructure
Data capture and analytics tools, and
new business and software applications
create new forms of value.

Figure 2

cognizant 20-20 insights 2

Defining QA’s Role Addressing IoT’s QA Complexity
There is a plethora of applications under devel- A comprehensive QA strategy is required to cover
opment to expand the existing hyperintelligent, the depth and breadth of IoT testing. The strat-
connected world. However, organizations need egy must include the types of testing, test lab
to perform testing on these applications before setup, testing tools and simulators that should
they reach prime time. The role of QA is there- be deployed. Considering the difficulties in gen-
fore critical, because it involves testing hardware erating big data from the thing in a testing
and software and transmitting massive amounts environment, it is crucial to evaluate data simula-
of real-time intelligence. Top challenges and solu- tion and virtualization techniques. Stubs can be
tions that QA managers can enlist include: considered as options during early stages while
data recorders can serve as alternatives at later
• The hardware/software convergence of QA:
stages. Beyond test planning and data simulation,
Devices, sensors and applications form the
metrics-driven exhaustive test execution is per-
ecosystem. However, there is a phase shift from
formed to achieve a stable system.
sheer testing of applications in a defined envi-
ronment to testing the experience in a dynamic QA organizations can split IoT test areas into two
environment. With several million sensors layers, as described below. While the QA needs
and different types of devices providing the to be performed across both layers, it is always
experience in conjunction with intelligent easier to identify techniques and the types of test-
software, mere functionality validation is not ing that can be adapted to each layer to enhance
enough in an environment as complex as IoT. the QA strategy.
• A working system is not sufficient: To certify • The device interaction layer: This layer
an IoT system or service, it is not enough to is where the software and the hardware
merely ensure a working set of device software. components of a real-time IoT environment
For instance, a shipment tracking system with interact.
sensors and devices that communicate with
multiple software back-ends will need intelligent One typical example is a Bluetooth device
algorithms to ensure accurate product delivery. transmitting real-time data to a mobile device
This requires a very robust QA validation app. Oftentimes, there is a lot of interaction
process. The use cases could be extremely testing occurring on the functional side of QA.
complex in real time; the variety of real-time However, other types of testing could also be
scenarios can become a challenge for QA. required. The following are the broad types of
other required elements, in addition to typical
• Large number of sensor interactions: The software testing:
creation of an environment to test a real-time
IoT implementation is a challenge. It entails not >> Conformance with standards: These are
only configuring an environment to validate mostly device performance attributes that
the correctness, but also to assess scalability are specific to devices and sensors. These at-
and reliability. The system is built on numerous tributes must be validated against the stan-
analytics engines, and demands a significant dards of the device and its communications
experience in simulation to build out a test protocol. Hardware vendors perform most
environment. While the hardware and the of these tests, but there could be certain
protocols are typically well tested by device domain- or use-case-specific requirements
makers, understanding application intelligence such as the use of such devices in an envi-
and the device’s real-time complexity is an ronment that was not tested.
extremely new art and science for many appli- >> Interoperability: The ability of different de-
cation developers and QA experts. vices to support the required functionality
among themselves, other external devices
In a nutshell, the software testing challenges
and implementations.
go beyond devices and sensors to include the
added complexity that comes with big data (i.e., >> Security: With billions of sensors in the mak-
huge volume, velocity and variety), which makes ing, it’s crucial to tackle data privacy and the
validation of real-time IoT certification a major security concerns across the IoT ecosystem.
headache. The following are the different types of secu-
rity testing requirements:

»» Identity and authentication.

cognizant 20-20 insights 3

 »» Data protection. >> The IoT services and back-end IoT envi-
ronment: While integration testing of the
»» Data encryption. interfaces is key, there is a complex data
»» Storage data security in local and remote layer that comes into play. For example, a
clouds. typical IoT system packs a complex ana-
• The user interaction layer: This layer is the lytical engine to ensure an exceptional user
touch point between the thing and the user. experience.
The success of the overall system depends on Creating a QA environment to enable validation
the user receiving a seamless experience. Key of such an interface means addressing the grow-
testing areas in this layer include: ing data volume, velocity and variety challenges
>> Network capability and device level tests: of the IoT ecosystem. The front-end validation
The specific aspects of network communica- environment can be done by assembling data
tion such as connectivity are validated by recorders and simulators. The service and data
simulating different network modes in addi- layer validations will involve complex simulation
tion to device-level validation such as energy services such as the generation of millions of
consumption tests, etc. sensor hits, machine learning algorithms and the
ability to generate time-boxed traffic.
>> Usability and user experience: Usability
and user experience are important in terms There are a few methods to create such an eco-
of the real-time usability; it involves human/ system; for example, leveraging sandboxes of
machine interaction and also the real-time development services or creating mock envi-
experience that the IoT system provides in ronments using virtualization tools. However,
a specific interaction. For example, contact- numerous implementation synergies are required
less payments compared with a physical to establish a working set of environments for
card-based payment. a thorough services and back-end validation
platform.

IoT Testing Areas

Compatibility:
multiple configuration, protocol
versions, product versions
Connectivity: (backward compatibility), mobile OS.
between the things
and communication
infrastructure.

Functionality:
Web/UI, embedded and
back-end computing.

Exploratory:
Security: one-day-in-a-life
including privacy, scenarios, and beyond
autonomy and control. functional requirements
and structured testing.

Performance:
both of the network
communication and
internal computation.

Figure 3

cognizant 20-20 insights 4

IoT Testing Types identification of defects, thereby establishing
a new dimension within the validation process.
A simple IoT ecosystem borrows several software
testing approaches from regular QA parlance to
Looking Forward
validate IoT applications. It is important to focus
on all three core components of the IoT system to The investments made in IT infrastructure and
treat functional and connectivity testing as criti- marketing can take businesses only so far, partic-
cal elements of overall IoT ecosystem testing. ularly if they do not have a holistic approach to
testing the IoT ecosystem. As IoT expands beyond
The following types of testing must also be per- the periphery into the mainstream of consumer
formed across the IoT ecosystem: and enterprise markets, QA teams must gear up
to help their organizations take advantage of the
• Performance testing that covers the rapidity
tremendous opportunities created by the ongo-
of the communication network model, as well
ing business digitization.
as the internal computation capabilities of the
embedded software system. It’s time for QA organizations to empower their
• Security testing that covers privacy, autonomy companies with reliable IoT products and services
and spying. that make good on the promise of smart, connect-
ed devices that elevate everything from personal
• Compatibility testing with the possible com-
wellness/hygiene and driving, through manufac-
bination of device version, protocol version,
turing, logistics management and air travel. With
mobile devices and mobile OS version.
ever-greater blending of IoT into business and IT,
• Exploratory testing to test from the user’s it is necessary for the QA teams to upskill beyond
perspective and beyond predefined test traditional functional testing and gear up for inte-
procedures. grated testing of embedded software, IT solutions
and big data — and to understand their influence
Solutions and Framework Opportunities
on one another.
The IoT ecosystem puts forth a myriad of QA chal-
lenges. Importantly, the QA organization should To prepare for the Iot onslaught, QA organiza-
view these challenges as opportunities to build tions should focus on the following:
frameworks and solutions. The following are a
few such opportunities: • Orient people to gain an amalgamation of
skills, combining QA, quality engineering and
• Protocol simulators: One of the interesting
hardware validation to meet the demands of
aspects of IoT QA is the ability to work with
IoT QA.
multiple protocols. Protocol simulators can
come in handy when there is a huge variety of • Build collaborative QA teams with a view
device end-points and interfaces to validate. toward performing integrated tests spanning
hardware, software and big data layers, thereby
• Data recorders: Data recorders from different
augmenting the niche and broader aspects of
types of devices can be helpful in smart
IoT testing.
validation across device sets. The recorded
data can be played across different device • Look for tool-build opportunities: As the dis-
end-points automatically, which in turn can be tinctions between hardware and software blur,
a great enabler in compatibility testing of apps there is a plethora of opportunities to build
across different device sets and communica- solutions to enhance QA across the internal
tion layers. systems ecosystem.

• Virtualization: This is an important aspect • Build labs that serve the entire digital
of IoT validation. Due to highly complex portfolio, to experiment and simulate real-time
IoT interfaces, there is little opportunity for experiences that inform smarter ways of
real-time validation of application behavior. testing.
Therefore, it will still be beneficial to bring in • Build a culture of “test as a user” vs. a
an ample amount of virtualization into the mind-set in which the organization merely
services on which IoT applications are built. tests against requirements; this will ensure
Virtualization of an IoT ecosystem yields the the “experience” component of the IoT stream
benefits of faster turnaround and reduced costs is well-established for a comprehensive quality
due to minimal dependency on the production product or service.
environment for testing. It also leads to earlier

cognizant 20-20 insights 5

Footnote
1
http://www.gartner.com/newsroom/id/2636073.

About the Authors

Subbiah Muthiah is an Associate Director within Cognizant’s Quality Engineering & Assurance (QE&A)
Center of Excellence (CoE). He is responsible for techno-business development and building quality
assurance for trending themes such as digital services, connected devices, wearables and telematics.
Subbiah has more than 14 years of relevant industry experience with in-depth knowledge across the
mobile stack — from the application layer to RF. He holds a bachelor’s degree in engineering and works
out of Chennai in Tamil Nadu, India. Subbiah can be reached at [email protected].

Ramakrishnan Venkatasubramanian (Ram) is a Director within Cognizant’s QE&A Technology Center of

Excellence. He has over 15 years of experience in the information technology industry. Ram’s experi-
ence spans software development, product engineering, test automation, mobility, cloud, the Internet
of Things and technology consulting. He leads the Digital Stream of Technology CoE including R&D,
mobility, virtualization and Internet of Things streams within the Cognizant QE&A practice. Ram can be
reached at [email protected].

About Cognizant
Cognizant (NASDAQ: CTSH) is a leading provider of information technology, consulting, and business process out-
sourcing services, dedicated to helping the world’s leading companies build stronger businesses. Headquartered in
Teaneck, New Jersey (U.S.), Cognizant combines a passion for client satisfaction, technology innovation, deep industry
and business process expertise, and a global, collaborative workforce that embodies the future of work. With over 75
development and delivery centers worldwide and approximately 211,500 employees as of December 31, 2014, Cogni-
zant is a member of the NASDAQ-100, the S&P 500, the Forbes Global 2000, and the Fortune 500 and is ranked among
the top performing and fastest growing companies in the world. Visit us online at www.cognizant.com or follow us on
Twitter: Cognizant.

World Headquarters European Headquarters India Operations Headquarters

500 Frank W. Burr Blvd. 1 Kingdom Street #5/535, Old Mahabalipuram Road
Teaneck, NJ 07666 USA Paddington Central Okkiyam Pettai, Thoraipakkam
Phone: +1 201 801 0233 London W2 6BD Chennai, 600 096 India
Fax: +1 201 801 0243 Phone: +44 (0) 20 7297 7600 Phone: +91 (0) 44 4209 6000
Toll Free: +1 888 937 3277 Fax: +44 (0) 20 7121 0102 Fax: +91 (0) 44 4209 6060
Email: [email protected] Email: [email protected] Email: [email protected]

­­© Copyright 2015, Cognizant. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any
means, electronic, mechanical, photocopying, recording, or otherwise, without the express written permission from Cognizant. The information contained herein is
subject to change without notice. All other trademarks mentioned herein are the property of their respective owners. TL Codex 1233