Warp10: A Time Series Platform

MHD Samy ALNAIMY

 Outlines
o Warp 10
 Introduction
 Server Types
 Data Structure
 Benchmarks
o Interacting with Warp 10
 Introduction
 Endpoints
o Warp Script
 Introduction
 Examples
o Demo
2
 Warp 10
Introduction 1 of 2

o A complete time series platform

o Apache 2.0 Open Source License
o Collect, store and analyze sensor and Time Series data
o Sensors, IoT, Transportation, IT monitoring ….

3
 Warp 10
Introduction 2 of 2

o Geo Time Series database

o Analytics engine

4
 Warp 10
Server Types

o Standalone
o Distributed

5
 Warp 10
Server Types : Standalone

o Raspberry Pi to a multi CPU box

o Highly Available via Datalog
o LevelDB

6
 Warp 10
Server Types : Distributed 1 of 2

7
 Warp 10
Server Types : Distributed 2 of 2

o Ingress
o Directory o Webcall
o Store o Runner
o Egress o Fetcher
o Plasma backend
o Plasma frontend

8
 Warp 10
Data Structure

GTS: Structure

9
 Warp 10
Benchmarks

o Standalone with LevelDB

o 100k datapoints per second
o Raspberry Pi with SD storage
o 10k datapoints per second
o In-memory data store
o 120M datapoints per second

10
 Interacting with Warp 10
Introduction

o HTTP protocol with payloads

o Access control
o Read token
o Write token

11
 Interacting with Warp 10
Endpoints 1 of 6

o Ingress
o Fetch
o Find
o Delete
o Meta
o WarpScript

12
 Interacting with Warp 10
Endpoints 2 of 6
Ingress
URL /api/vX/update
Target Sending data to the Warp 10 platform

Data format
959860945800000/
TS/LAT:LON/ELEV CLASS{LABELS} VALUE 37.399386:-3.071806/
214748
Timestamp Elevation engine.torque
(microsecond)
{vehicletype=x21,serial=VF43}
Geographic coordinates
206

13
 Interacting with Warp 10
Endpoints 3 of 6
Fetch
URL /api/vX/fetch
Target Retrieving raw Geo Time Series

Input format
Selector
Start/Stop or Now/Timespan

14
 Interacting with Warp 10
Endpoints 4 of 6
Fetch
URL /api/vX/fetch
Target Retrieving raw Geo Time Series

Output format [{
"c":"class",
c class name "l": {
"label0":"value0",
l labels },
"label1":"value1“

"a": {
a attributes (key/value) "attr0":"value0“
},
i an id which is unique per Geo Time Series "i":"1",
"v":[[1380475081000000,3.14]]
v Array of Geo Time Series™ readings. }]

15
 Interacting with Warp 10
Endpoints 5 of 6
Find
URL /api/vX/find
Target Retrieving Geo Time Series matching search criteria

No Values Just Meta Data

16
 Interacting with Warp 10
Endpoints 6 of 6
Delete
URL /api/vX/delete
Target Erasing data from the Storage Engine
Meta
URL /api/vX/meta
Target Setting Geo Time Series attributes
WarpScript
URL /api/vX/exec
Target Executing analysis expressed in the WarpScript language

17
 Warp Script
Introduction

o Simple statistics to complex algorithms

o Data flow programming language
o Stack-oriented language
o Calling Web Services trigger webhooks or push data to
other services
o Support Macros
o Go Focus on business not data manipulation

18
 Warp Script
Examples 1 of 4

Input

Output

19
 Warp Script
Examples 2 of 4

Input Output

20
 Warp Script
Examples 3 of 4

Patterns recognition

Input Time Series

21
 Warp Script
Examples 4 of 4

Patterns recognition

Output Patterns

22
Demo
References

o [Web Docs]. Retrieved Feb, 2018, from

https://www.warp10.io/content/03_Documentation
o [Web Blog Post]. Retrieved Feb, 2018, from
https://medium.com/@PierreZ/engage-maximum-warp-speed-in-
time-series-analysis-with-warpscript-c97a9f4a0016

24
THANKS!
Any questions?

25