AWS IoT & ML Recap

Taka Wang
2018.04.23
 IoT Foundation
Digitization of the Physical World
Prerequisites
 Greenfield vs. Brownfield

VS
Greenfield Brownfield

Courtesy of arabianbusiness & Colliers International

 Clash of Two Worlds
Machine Camp vs. Internet Camp
(aka. OT vs. IT)
Machine Camp Internet Camp
‣ Brown field ‣ Green field
‣ Strong company heritage, risk aversion ‣ High-risk, VC-driven culture
‣ Corporate career is the norm ‣ Entrepreneurial management and employees
‣ Domain: Physics, engineering ‣ Domains: IT, services
‣Think big ‣ Focus on point solutions/MVP
‣ Waterfall approach ‣ Agile approach
‣ Standards like DIN/ISO ‣ Open source
‣ Long QA & release cycles (defect free) ‣ Perpetual beta (“Fast patches”)
‣ Long lead time

Reference: Enterprise IoT, 2nd

 IIoT vs. CIoT
Categories IIoT IoT

Impact Evolutionary Revolutionary

Focus Data Things

Connectivity Structured Ad Hoc

Mission Important but

Criticality
Critical not critical

Existing
New Devices
Standards Devices &
& Standards
Standards

Courtesy of National Instruments

Further reading: Compare IoT and IIoT: Find 11 Differences Reference: Industrial Internet of Things: PROFINET Intro
Why Edge? (or Cloud-Centric Assumption)

• There is sufficient bandwidth to push data to the Cloud

• Connectivity is not an issue. A device will (almost) always be connected

to the cloud

• The latency induced by cloud-centralized analytics and control is

compatible with the dynamic of the IoT system

• The connectivity cost is negligible

• Industrial companies are comfortable in exposing their data to the cloud

Reference: Fog Computing with Vortex

Fog Computing vs. Edge Computing

ata g
D sin
e s
i n g
proc
e ss
ro c
atap
D

ata g
D sin
e s
proc

a ta g a ta g
D sin D sin
c es c es
pro pro

Courtesy of Forest Giant

Which edge are you talking about?
 offload

Cloud,
4G, data ce
5G
app, se nters,
rvices

eNodeB
MEC Mobile Core

4G,
5G
MobileMulti-Access Edge Computing

o rs eNodeB
u at
/ Act
nsor s Customer Edge Network Edge
a l Se
Loc

Internet

ISP ISP Backbone

Local Area Network
Fog Node/IoT Edge Gateway
Which edge are you talking about?
AWS IoT
AWS IoT Architecture Recap

Local connectivity intermittent connectivity

Things Edge/Gateway
Cloud

intermittent connectivity

Things
 AWS IoT Product Timeline
AWS FreeRTOS Cloud
AWS
AWS IoT Device SDK GreenGrass
AWS IoT Core

Things Edge/Gateway
AWS IoT Device
AWS FreeRTOS Mgmt.

AWS IoT Device SDK

AWS IoT Core/SDK AWS Greengrass Amazon FreeRTOS

Amazon IoT Device Mgmt.

15’ Oct 17’ Jun 17’ Nov

Amazon FreeRTOS AWS IoT Device SDK AWS ML*

Amazon IoT Device

AWS IoT Core AWS GreenGrass
Mgmt.
 Amazon FreeRTOS
Device Software Stack
Embedded User Application

MQTT Shadow Greengrass Wi-Fi Mgmt.

Agent Library Discovery Library
FreeRTOS
Kernel
Amazon FreeRTOS Internal Libraries
(TLS, OTA.., etc)

Vendor Drivers

Hardware

Reference: Amazon FreeRTOS Architecture

 Amazon FreeRTOS
Lean Canvas
Problem
Problem Solution
Solution Unique
Unique Value Proposition
Proposition Unfair
Unfair Advantage Customer
Customer
Segments
Segments
Machine/Device 預先整合(pre-integrated) - MCU類 的 Cloud/Edge FreeRTOS project 的
Maker 想要開發 MCU AWS IoT Device SDK 的 connectivity 開發包 Founder 在 Amazon Greenfield
類的 IoT 產品需要⾃ FreeRTOS 開發包 負責這項產品 Machine/Device
- 預先整合好的 Security
Maker
⾏處理 Library
1. 安全性
Key Metrics
- 預先整合好的 secure
Channels
Key OTA update
Channels Early Adopters
2. 與Cloud串接
3. Wi-Fi 管理 - 導入的 chipset 數⽬ - 整合 AWS 的 Device - Chipset Vendor - Chipset Vendor
Shadows 解決網路斷斷 - FreeRTOS 社群 - 既有的FreeRTOS
4. 遠端裝置管理更新 - Device Management 服務
等的開發問題 註冊的 Device 數 續續問題 - Webinar, Workshop ⽤⼾

Alternatives
Azure Sphere
Cost Structure
Cost Structure Revenue Streams
Revenue Streams
- 開發⼈員⼈⼒成本 免費使⽤ FreeRTOS，透過 AWS IoT Device
- Device Management Service 維運成本 Management (OTA) 與 IoT Core 等服務收取
- Enablement/Engagement的⼈⼒成本 Subscription fee
Rev. 2
 AWS IoT Device Mgmt.

Onboard Organize Update Output

User can onboard a large Organize devices into groups which AWS IoT Device Management sends Firmware updates can be sent to a
number of devices at once can also be arranged into hierarchies a device job over-the-air (OTA), such group or individual devices
using device provisioning as a firmware update

Device Provisioning Fleet Indexing and Search Device Jobs

Reference: AWS IoT Device Management
Amazon FreeRTOS AWS IoT Device SDK AWS ML*

Amazon IoT Device

AWS IoT Core AWS GreenGrass
Mgmt.
 AWS IoT Core/SDK
Things AWS IoT Core AWS

Allows integration with

other AWS services (route) Actions
Rules Engine Amazon Amazon Amazon
CloudWatch Kinesis ES
MQTT over TLS
MQTT over WebSocket/TLS
HTTP
Device Amazon Amazon Amazon
Communicates with Gateway S3 DynamoDB Lambda
authenticates &
authorized devices

Amazon Amazon Amazon

Machine Learning SNS SQS

Thing Thing MQTT

HTTP
Registry Shadow (REST
)
Device SDK
Applications
Certificate
Manages Allows devices to seamlessly
device identity drop on/off connection
Reference: Smart Home Building Blocks with Intel Edison & AWS
 AWS IoT Rules Engine
SELECT DATA FROM TOPIC WHERE FILTER ACTIONS

{
Rule "awsIotSqlVersion": "2016-03-23",
"sql": "SELECT * FROM 'iot/test'",
"ruleDisabled": false,
Name "actions": [
{
"s3": {
Description "roleArn": "arn:aws:iam::123456789012:role/aws_iot_s3",
"bucketName": "my-bucket",
"key": "myS3Key"
SQL Statement }
}
Array of Actions ]
}

Reference: (MBL312) NEW! AWS IoT: Programming a Physical World w/ Shadows & Rules
 AWS IoT Thing Shadows
AWS IoT

Reported

Desired

Intermittent Delta
REST APIs
Connection

Thing Shadow Mobile App

• Report current state to one or multiple shadows • Set the desired state of a device
• Retrieve its desired state from shadow • Get the last reported state of the device
• Delete the shadow

Shadow reports delta, desired and reported states

along with metadata and version
Reference: The Lifecycle of an AWS IoT Thing
 Thing Shadow - Step0
Cognito Identity

Thing Shadow
Thing Shadow Permissions

68 Report
Reported State State Delta

68 Authenticated

Desired State
Thermostat

68

X.509 Certificate

*nest is trademark owned by Google LLC Reference: AWS Thing Shadow

 Thing Shadow - Step1

Thing Shadow

68 Reported State State Delta

68

Desired State
Thermostat Update
70
70
68

*nest is trademark owned by Google LLC Reference: AWS Thing Shadow

 Thing Shadow - Step2
Pub

Sub

Thing Shadow

68 Reported State State Delta

68 +2

Desired State Calculate

Thermostat
70
70
68

*nest is trademark owned by Google LLC Reference: AWS Thing Shadow

 Thing Shadow - Step3

Thing Shadow

68
70 2. Update
Reported State State Delta

70

1. React Desired State

Thermostat
70
70
68

*nest is trademark owned by Google LLC Reference: AWS Thing Shadow

 Thing Shadow - Step4

Thing Shadow

68
70 Reported State State Delta

70 Sub
Pub

Desired State
Thermostat
70
70
68

Confirm

*nest is trademark owned by Google LLC Reference: AWS Thing Shadow

 AWS IoT Device SDK

Embedded C
C++11 Android iOS
(Embedded OS)

Javascript Arduino Library

Node.js/Browser (Arduino Yún) Java 1.7+ Python

Reference: Getting Started with AWS IoT

Reference: AWS IoT Device SDK
Embedded C SDK - Layered Architecture
Customer Application Customer Application

Thing Shadow
aws_iot_shadow_interface.h

MQTT Interface
aws_iot_mqtt_interface.h

AWS IoT
Embedded C SDK AWS MQTT Library - derived from Paho Embedded C

Network(TLS) Interface Timer Interface

network_interface.h timer_interface.h

OpenSSL/MbedTLS/… Linux/RTOS Timers

Hardware / OS Linux/LwIP TCP Sockets

Reference: AWS IoT Embedded C Device SDK

Amazon FreeRTOS AWS IoT Device SDK AWS ML*

Amazon IoT Device

AWS IoT Core AWS GreenGrass
Mgmt.
 Types of Machine Learning
Classification
Dimensionality
Reduction

Machine Supervised
Learning
Unsupervised
Learning
Learning
Regression

Clustering

Reinforcement
Learning

Reference: Machine Learning Algorithm - Backbone of emerging technologies

 Machine Learning Workflow
Iterations Prediction

Training Data
Machine
Machine Learning
Learning
Algorithms Model
Test Data

Production
Data
 Training vs. Inference
Forward “car”

Large N
Backward Error
TRANING

Forward
“car”
Smaller,
varied N
INFERENCE

Reference: Discover the Difference Between Deep Learning Training and Inference
 AWS Machine Learning Blackbox

binary classification multiclass classification Regression

(binary attributes) (categorical attributes) (numeric attributes)

AWS ML Process

Review model & Use model to

Prepare Data Create a Training Create a ML
Set a score generate
(Transform) DataSource model
threshold predictions
 AWS SageMaker First Launch: 07’ Nov.

‣ build-in algorithms
‣ Inference code image
‣ Interactive Environment ‣ pre-built train/inference images ‣ Scalable CPU/GPU Cluster
‣ Trained model
‣ your own images

Jupyter
Algorithms Training Hosting
Notebook
 Amazon SageMaker Amazon ECR
Client Application

Inference code

Training code

Model Training (on EC2)

Reference: Amazon Web Services

 Amazon SageMaker Amazon ECR
Client Application

Inference code

Training Data

Training code Helper code Training code

Model Training (on EC2)

Reference: Amazon Web Services

 Amazon SageMaker Amazon ECR
Client Application

Model artifacts

Inference code

Training Data

Training code Helper code Training code

Model Training (on EC2)

Reference: Amazon Web Services

 Amazon SageMaker Amazon ECR
Client Application

Model artifacts

Inference code Helper code

Inference code

Model Hosting (on EC2)

Training Data

Training code Helper code Training code

Model Training (on EC2)

Reference: Amazon Web Services

 Amazon SageMaker Amazon ECR
Client Application

Inference request
Inference Endpoint
Inference response

Model artifacts

Inference code Helper code

Inference code

Model Hosting (on EC2)

Training Data

Training code Helper code Training code

Model Training (on EC2)

Reference: Amazon Web Services

 Amazon SageMaker Amazon ECR
Client Application

Inference request
Inference Endpoint
Inference response
Ground Truth

Model artifacts

Inference code Helper code

Inference code

Model Hosting (on EC2)

Training Data

Training code Helper code Training code

Model Training (on EC2)

Reference: Amazon Web Services

Amazon FreeRTOS AWS IoT Device SDK AWS ML*

Amazon IoT Device

AWS IoT Core AWS GreenGrass
Mgmt.
 AWS

Edge Gateway
Greengrass core Greengrass core /Fog Node
(Runtime) (Runtime)

Device SDK
Device SDK Edge Devices

Factory Home
Why Edge? (or Cloud-Centric Assumption)

• There is sufficient bandwidth to push data to the Cloud

• Connectivity is not an issue. A device will (almost) always be connected

to the cloud

• The latency induced by cloud-centralized analytics and control is

compatible with the dynamic of the IoT system

• The connectivity cost is negligible

• Industrial companies are comfortable in exposing their data to the cloud

Reference: Fog Computing with Vortex

 AWS Greengrass Features

Concerns Connectivity Latency Data

Data and Local Local Local

state sync Actions Triggers Inference

Local Device Local Lambda

Local Messaging ML Inference
Shadow function
 Local Actions
Arbitrary IPC Event-Driven Programming Dataflow Programming

Trigger Trigger Process

Process Process

Pod of actions Engine Process

Process Process Python JS …

new Running Running Running Process

Action Action Action Process
Process new

Process

Programming paradigms
 Local Inference
Local Inference

Training Training
Data Machine Data
Machine Learning Machine
Learning
Machine Learning
Learning
Algorithms Algorithms
Model Model
Test Data Test Data

Cloud Infrastructure Cloud Infrastructure

Deployment

Machine
Production Inference Serverless Production Inference
Learning
Data function Data
Model

Predictive Analytics Edge Infrastructure Predictive Analytics Edge Infrastructure

Reference: How Edge Computing And Serverless Deliver Scalable Machine Learning Services
Greengrass ML inference Workflow
AWS Greengrass
Core Device Device data

Result from
Data inference processing

ML model

Prediction Transfer trained AWS AWS Bring your own

ML model Greengrass service SageMaker ML model
Trained ML models

Device performs inference Transfer ML models to Build and train ML models

locally to action quickly Greengrass devices in the cloud

Edge/Gateway Cloud

Reference: Perform Machine Learning Inference

 Local Triggers
Group Definition
Service Discovery
Core Devices Lambda functions
Group Name
Service
IP address of Core Core Subscriptions
Group's root CA Cert. Deploy
Lambda funcs. Source Destination
REST Request
Subscriptions

GG
Device Device
Core

MQTT Client MQTT Broker MQTT Client

 OPC-UA Bridge
PLC1

Topic namespace for PLC1

/opcua/server/node/plc1
Processing Lambda

OPCUA
Bridge OPCUA
Server Adapter

/opcua/server/node/plc2

PLC2 Topic namespace for PLC2

Reference: AWS & Sample

Summary - simplified model
AWS IoT AWS
AWS IoT Core
Device SDK GreenGrass

MQTT Client MQTT Broker MQTT Client API Gateway

Function MQTT broker

TLS Library OTA agent
Watchdog clusters

State Mgmt. Daemon AWS Ecosystem

 Discussions of Greengrass
‣ Price (3 Free Core / 1 Year)

‣ Linux only proprietary implementation (sandbox: Namespace + Cgroup)

‣ Lack of local service discovery (Device Node should connect to cloud

once!)

‣ Complicated and cumbersome procedures

‣ Good for Greenfield applications

And Last, but not least
 Friendly Reminders
‣ Internet connection is not stable (in the context of IoT)

‣ Security is not easy

‣ Device Mgmt. is not easy: Partial OTA, reboot

‣ OTA is not easy: (Partition based, Package based, Version, OS, Die-Hard)

‣ OPC-UA is one of the OT protocols (esp. Greenfield)

‣ Certs & Keys deployment in volume manufacturing is not easy

‣ Windows is still matter (in the context of IoT)

Further reading: Top 10 Reasons People Aren't Embracing the IoT
Takeaways

Courtesy of CNN Money & AZ Quotes

 Takeaways

Customer Segments

IoT or IIoT
分類思考法簡化問題
Greenfield or Brownfield

Where is your edge

 Readings
‣ Taking the pulse of enterprise IoT - Mckinsey & Company, July 2017
Comprehensive Marketing survey

‣ Top 10 Reasons People Aren't Embracing the IoT - IoTI, April 2016
Know your customers

‣ Compare IoT and IIoT: Find 11 Differences - RAMAX Group, Feb 2018
Narrow Your Focus

‣ Enterprise IoT, 2nd - Strategies and Best Practices for Connected

Products and Services - Dirk Slama et al., Nov 2015
Fundamentals for every IoT Practitioners

‣ The Lifecycle of an AWS IoT Thing - AWS, Oct 2016

Comprehensive AWS IoT Core Introduction
Supplements
Differences between MQTT broker
& AWS IoT Device Gateway
• No retained message support

• No QOS2 support

• No message persistence

Reference: Message Broker for AWS IoT > Protocol

 AWS IoT Core
Port/Protocol/Authentication combinations
Protocol Authentication TCP Port

MQTT Client Certificate 8883, 443*

HTTP Client Certificate 8443

HTTP AWS Signature Version 4 443

MQTT over Websockets AWS Signature Version 4 443

Reference: AWS IoT Core Now Supports MQTT Connections with Certificate Based Client Authentication On Port 443
Nvidia’s updates EULA to ban the use of gamer-oriented
GPUs in data centers

Courtesy of License For Customer Use of NVIDIA GeForce Software

Reference: Nvidia: Using cheap GeForce, Titan GPUs in servers? Haha, nope!
Reference: Nvidia’s updates EULA to ban the use of gamer-oriented GPUs in data centers
 Where is your Edge?
Device Edge Application Services
Device
ex. IoT gateway, Edge
Compute Storage Network
PACs (Controller)
AWS GreenGrass
Sensors & Applications Cloud Infrastructure
Azure IoT Edge

Application Services
Cloud
Cloud Edge Edge
Compute Storage Network
ex. MEC
Sensors & Applications Cloud Infrastructure

Reference: Demystifying Edge Computing -- Device Edge vs. Cloud Edge

Courtesy of SpiderCloud Inc
Courtesy of WinSystems