The Outer Limits

Successfully Implementing
AI at the Edge

June 4, 2020
Alix Paultre | Editor, Evaluation Engineering
Agenda

1 The Smart Revolution 4 Looking at Vision

• The Internet of Things • The demands of vision applications
• Striking a Balance in the Cloud • System requirements for processing
• IoT Traffic Demands • Proper integration for smooth operation

2 Advanced Computing at the Edge 5 Software and Hardware Integration

• What is Artificial Intelligence? • Design Issues
• Artificial Intelligence in the Cloud • An Integrated Solution
• Fog Computing, IoT Platforms and More • Security

3 Hardware Selection 6 Wrap Up

• It All Begins with the Processor • AI Everywhere
• The Importance of Evaluation Kits • Key Takeaways
• NVMe-Over-Fabric SoC and More • Q&A Session
1
The Smart Revolution

The next stage in

intelligent systems
Introduction – Challenging Times
An Era of Technological Disruption
• New Technologies and Solutions
• Integrated Smart Products
• New Functionalities Create New Applications

The Shift Toward a “Smart” Lifestyle

• Web-oriented Functionalities
• New Work Paradigms
• Leveraging Advanced Productivity

The Explosive Growth of IoT

• Empowering Old Applications
• Creating New Applications
• Enhancing Traditional Infrastructures

The Fragmentation of IoT Into Sectors

• Industrial IoT
• Medical IoT
• Military IoT
Finding Balance in the Cloud

Mixing Hardware, Software, and Wireless

• The Proper Mix is Critical
• Multi-protocol Compatibility is Important
• Costs and Support

Performance is Key to Adoption

• Addressing Device Functionality
• Matching Capabilities Between Systems
• Recognizing Consumer Perceptions

The Importance of Proper Balance

• RF Usage is Not a Trivial Issue
• Bandwidth and Latency
• Device vs. Infrastructure Costs
IoT Application Traffic Demands

• Facial Recognition • Environmental Monitoring

• Pattern Identification • Intelligent Thermal Management
• Voice Recognition • Financial Trading
• Gesture Capture • Municipality Infrastructures
Beyond the Thin Client

The Emergence of Edge Computing

• Addressing Infrastructure Costs
• Improving Performance
• Increasing Resiliency
• Addressing Latency

Managing Data in the Cloud

• Addressing Millions of IoT Devices
• Bandwidth Use Reduction
• Improved Hardware Enables New Functionality
• Smart Nodes are More Secure

A Natural Migration
• Beyond the Thin-Client Paradigm
• More Power in the Hands of the User
• AI as an Edge Computing Enabler
• Self-managing Systems Empower Users
Advanced Computing
at the Edge
Pushing Computing to the
Point of Application
What is Artificial Intelligence, Anyway?

AI is Already Here
• Face Capture and Identification
• Voice Recognition
• Text-to-Speech
• Advanced Driver Assistance Systems (ADAS)

Levels of Artificial Intelligence

• Smart Algorithms
• Machine Learning
• Neural Networks

Artificial Intelligence is Empowering

• Increased Software Functionality
• Smarter Devices are More Usable
• AI Creates Intelligent Infrastructures
Artificial Intelligence Empowering the Cloud

The Next Iteration of Edge Computing

• Will Better Address Applications
• "Soft Logic" Solutions
• Secure Devices, Secure Network

AI System Configuration
• Establish Hardware Infrastructures
• Software Must be Appropriate
• Managing Sensors and Other Data Input

Applications
• Pattern recognition
• Automated System Management
• Smart Home/Facility/City
The Fog: Operating Between the Edge and Center

Fog Computing as a Deployment Alternative

• AI systems often involve many semi-intelligent

nodes transmitting data to a local server
based remotely, instead of a centrally located
solution.

• Fog computing blends the advantages of a

centralized system with a remotely based
system in a hybrid approach.

• Locating support assets closer to the

application point provides most of the benefits
of edge computing, while retaining some
centralized control.
IoT Platforms

Courtesy of Avnet

Resolves Key Implementation Issues Streamlines Application Management

• Security and privacy • Simplified management of the entire IoT system
• Interoperability and standards • Pre-integrated process modules: remote device
• IT and OT convergence provisioning, pre-processing, real-time analytics,
visualization, etc.
• Hosting model
• Often customizable for industry-specific
• Integrated development environment applications
Intelligence and Functionality

How much – and where?

Determining the level of AI functionality your edge
devices will require to support your application ...

• Simplifies the process of specifying the

computing resources that are needed in the
cloud and at the edge.

• Significantly reduces cost and effort by

ensuring system capabilities and other factors
like power are properly matched to the
application.
Edge, fog and cloud – how should your application split the work?

One size does not fit all.

The requirements for computational power, data handling, and speed of response varies widely
(>10*5) between applications.
Latency: A Key Design Factor

• An IoT application's ability to respond to its

inputs is largely determined by: network latency
(in both directions) as well as cloud and server
processing delays.

• Any system latency can be an eternity for

applications requiring near-real-time response
such as ADAS, critical environmental
monitoring, or industrial processing.

• 5G networks are expected to have latencies

averaging around 30msec, but this may still be
too slow for certain applications.

• Edge computing addresses latency by reducing

or completely eliminating the need for data
transfer at the point of application.
Hardware Selection

Half the job is having

the right tools.
Integration at the Edge

Latency, Bandwidth, Functionality

• Edge computing resolves latency
• Reducing data transfer reduces bandwidth needs
– plus cloud costs and power consumption
• Smarter devices are more functional

Decentralized Architecture
• More power at the point of application
• Improved response time
• Device-oriented data management
• More user-oriented functionality

A Natural Migration
• Enhancing device functionality
• Serving consumer demand
• More reliability and easier troubleshooting
• Security and reliability
Secure Processing at the Edge

Xilinx Zynq UltraScale+ MPSoC

• 64-bit processor scalability

• Real-time processor
• Programmable logic
• Hardware and software security
• Low Power Consumption
• Large Design Tool Portfolio

Three Variants
• Dual application processor
• Quad processor
• GPU video codec
Creating the Optimal Solution

ZCU102 Evaluation Kit

• Jump-start Designs
• Zynq UltraScale+ MPSoC device
• Quad-core ARM Cortex-A53
• Real-time Dual-core Cortex-R5
• Mali-400 MP2 graphics unit
• Supports all major peripherals
• DDR4 SODIMM
• PCIe Root Port Gen2x4
• Two FPGA Mezzanine Card interfaces
Integrated Platforms Ease Deployment

BittWare 250-SoC
• Leverages NVMe-over-Fabric Technology
• 64-bit ARM processors
• Zynq UltraScale+ FPGA SoC FPGA
acceleration card
• Enables Ethernet Just-a-Bunch-of-Flash
(EJBOF)
• PCIe Gen 4 host interface
Enabling Advanced Storage Infrastructures

EJBOF and NVMe

• Remote, disaggregated storage
• Xilinx Zynq UltraScale+ MPSoC
• Programmable logic
• NVM Express-over-Fabrics offload
• RDMA NIC protocol
• Low latency, high throughput
• Reduction in solution footprint
• Lower total cost of ownership
Looking at Vision

Advanced imaging
is taking off.
Advanced Vision Requires Sophisticated Processing

NXP Vision Processor S32V

• Image signal processor
• 3D graphic processor unit
• Dual APEX-2 vision accelerators
• Automotive-grade reliability
• Quad Cortex-A53 cores
• M4 core up to 133 MHz
• Hardware security encryption
• Embedded image processing
• 4 MB on-chip system RAM
• DRAM controller support
 NXP Vision Processor

Detail View
Camera Integration and System Evaluation

Avnet S32V Development Kit

• Supports NXP S32V23x processors
• Arm Cortex-A53 quad-core based
• NXP S32V234 MPU
• SDRAM, eMMC Flash, microSD card
• Interfaces for up to five cameras
• Six 100 Mb Ethernet ports
• A JTAG interface
• A PCIE interface
• A 2-channel VIU interface
Software & Hardware
Integration
Coordination is
a science.
System Integration Issues
Complexity
• The number of protocols and
methodologies for IoT and edge-intensive
applications is growing.
• Multiple RF solutions exist, each with
different advantages.

Interoperability
• All nodes must recognize and communicate
with every subsystem.
• Must also recognize and communicate with
every compatible IoT system.

Security Gaps
• Comprehensive, robust features like secure
boot and the related software are needed
now more than ever.
Plug-in Vision Acceleration for the Edge

Intel Movidius Neural Compute Stick

• A dedicated hardware accelerator for Deep
Neural Network (DNN) deep learning
• Low-power Myriad 2 VPU architecture
• Enables rapid prototyping, validation and
deployment of DNN inference at the edge
• First of its class to feature a Neural
Compute Engine
• Strong option for on-device deep neural
networks and computer vision applications
• Enables an entirely new segment of AI
applications
Computing Engines for Deep Neural Networks

Intel Movidius Myriad X VPU

• Neural Compute Engine, dedicated hardware accelerator for deep neural networks
• 16 SHAVE cores and an ultra-high throughput intelligent memory fabric
• Tiny fanless deep-learning device for AI programming at the edge
• Deploys Convolutional Neural Networking on low-power applications
An Intelligent Sensor Suite for AI

SensorTile.box Development Kit

• Evaluate system configurations
• Speeds prototyping
• Works with STM32-Nucleo devices
STMicroelectronics SensorTile.box
• Humidity and temperature sensors
• Sensor integration solution • 100 mAh Li-Ion Battery
• 80MHz STM32L476 MCU • SWD programming cable
• BlueNRG network processor • Plastic housing + Cradle
• Wide-spectrum sensing • SensorTile firmware
• Motion and environmental MEMS sensors • STM32Cube expansion software
• Microphone • STM32Cube support
Modular, Scalable Systems for AI at the Edge

Avnet SmartEdge Agile

• Sensor integration solution

• Turnkey AI-empowered IoT
• A full software stack
• Brainium’s zero-coding platform
• Hosted on Microsoft Azure
• Software and firmware updates
• Introduces meta-sensing
System Architecture

SmartEdge Agile

• A hardware and software combo that provides AI • Cloud-optimized for MS Azure

and security at the edge • BLE interface
• Developed at three software levels (embedded, • Certified hardware solution, offering a full
gateway and cloud) software stack with edge intelligence
• The gateway can be an Android, iOS, Raspberry
PI, or another IP device
Next-Gen Cybersecurity Can’t Wait

• Software-based firewalls and passwords

aren’t enough anymore

• The root of trust is protected in a secure

environment

• Key-based authentication to validate

transactions

• Pre-provisioned, pre-configured or fully

customizable secure elements

• Secure manufacturing facilities safely

provision keys
How CryptoAuthentication Works

• Hardware-based secure storage

• Keep secret keys hidden from
unauthorized users
• Works with any microcontroller or
microprocessor
• Authenticity protection and
anticounterfeiting
• Hardware-based cryptographic
accelerators
• Random-number generator and
EEPROM-based secure key storage
• Physical anti-tampering and side-
channel attack protections
Wrap Up

Plus ...
How AI is changing
everything, everywhere.
AI Everywhere
AI will change everything, everywhere. Here are just a few of the trends already in motion.

Endless Edge Extreme Efficiency

AI will be a force multiplier for edge computing, A completely AI-enhanced smart place, be it a house,
accelerating implementation of edge intelligence in building or city, promises to offer the highest levels of
everything. AI algorithms will enhance the operation operational efficiency possible in a human system
of systems, making most passive systems obsolete. through real-time response to continuous stimuli.

Accelerated Discovery Self-Optimizing Systems

Self-learning systems will create novel solutions to Safety and reliability will increase significantly in self-
existing problems across all fields of business, monitoring systems. Smart homes will be safer and
science and technology. What took years or decades more comfortable. Smart facilities will achieve levels
in the past will be reduced to a fraction of the time. of throughput that were previously unattainable.
Key Takeaways

The cloud and edge will function as an Create a full system integration plan.
ecosystem – find your system balance.
• Sensor type must match the data required to
• Smarter edge devices are more useful, but increase prevent waste and redundancy.
implementation complexity.
• Power requirements must be balanced to
• Cloud-based systems use significant bandwidth ensure cost-effective operation.
and data transfer can be expensive..
• Connectivity makes or breaks interoperability.

Successful design starts with truly knowing AI is a major force multiplier for edge
your system requirements. computing.

• Make no assumptions – get the facts. • AI significantly increases the business and
functional value of edge devices.
• Too much HW/SW capability is as dangerous as
not enough – wasted capacity is wasted money. • Intelligent algorithms are the fuel. They can be
tailored for almost any application.
The Outer Limits
Successfully Implementing
AI at the Edge

Q&A Session
 DOUBLE YOUR CHANCES

Intel Neural Compute Stick 2 BittWare 250-SoC by Molex

Download this slide from the event’s
Resource List, then click on the images
to enter the separate giveaways for
more chances to win!

NXP S32V ADAS Development Kit Xilinx Zync UltraScale+ Kit STMicroelectronics SensorTile.Box

SPONSORED BY