Apache

 Ignite  -­‐  In-­‐Memory  Data  Fabric  

Ultimate  Speed  and  Scale  for  Transactions  and  Analytics

NIKITA  IVANOV  
Founder,  PMC  

www.ignite.incubator.apache.org #apacheignite

©  2014  GridGain  Systems,  Inc.

 Agenda
• Why  In-­‐Memory  Computing?  
• In-­‐Memory  Data  Fabric  
• Advanced  Clustering  
• In-­‐Memory  Compute  Grid  
• In-­‐Memory  Data  Grid  
• In-­‐Memory  Service  Grid  
• In-­‐Memory  Streaming  &  CEP  
• Plug-­‐n-­‐Play  Hadoop  Accelerator  
• Customer  Use  Cases  
• Q  &  A

©  2014  GridGain  Systems,  Inc.

 Why  In-­‐Memory
“In-­‐memory  computing  will  have  a  long  term,  disruptive  impact  by  radically  changing  users’  
expectations,  application  design  principles,  product  architectures,  and  vendor  strategies.”

Data Growth DRAM Cost, $

Less  than  2  zetabytes  in  2011,  8  in  2015 Cost drops 30% every 12 months

©  2014  GridGain  Systems,  Inc.

 In-­‐Memory  Data  Fabric  
Strategic  Approach  to  IMC

• Supports Applications of
various types and
languages

• Open Source – Apache 2.0

• Simple Java APIs
• 1 JAR Dependency
• High Performance & Scale
• Automatic Fault Tolerance
• Management/Monitoring
• Runs on Commodity Hardware

• Supports existing & 


new data sources
• No need to rip & replace

©  2014  GridGain  Systems,  Inc.

 In-­‐Memory  Data  Fabric  
Main  Capabilities

Performance   Transactions  
– High  Throughput   – Fully  ACID  Compliant  
– Low  Latencies   – Optimistic  &  Pessimistic  
Scalability   Persistence  
– Add  Cluster  Members  (cores)   – SQL,  NoSQL,  Hadoop  
– Add  Memory  (RAM)   Security  
High  Availability   – Authentication  
– Data  Backups   – Authorization  
– Datacenter  Replication – Tracing  &  Auditing

©  2014  GridGain  Systems,  Inc.

 In-­‐Memory  Data  Fabric  
Advanced  Clustering
• Ease  of  Getting  Started  
– Automatic  Discovery  
• Any  Environment  
– Public  Cloud  
– Private  Cloud  
– Hybrid  Cloud  
– Local  Laptop  
• Zero-­‐Deployment  
– Auto-­‐Deploy  Code  
• Full  Cluster  Management  
• Pluggable  Design
©  2014  GridGain  Systems,  Inc.
 In-­‐Memory  Compute  Grid
• Direct  API  for  MapReduce  
• Zero  Deployment  
• Cron-­‐like  Task  Scheduling  
• State  Checkpoints  
• Load  Balancing  
• Automatic  Failover  
• Full  Cluster  Management  
• Pluggable  SPI  Design

©  2014  GridGain  Systems,  Inc.

 In-­‐Memory  Data  Grid
• Distributed  In-­‐Memory  Key-­‐Value  Store  
• Replicated  and  Partitioned  data  
• TBs  of  data,  of  any  type  
• On-­‐Heap  and  Off-­‐Heap  Storage  
• Highly  Available  In-­‐Memory  Replicas  
• Automatic  Failover    
• Distributed  ACID  Transactions    
• SQL  queries  and  JDBC  driver  
• Collocation  of  Compute  and  Data

©  2014  GridGain  Systems,  Inc.

 In-­‐Memory  Service  Grid
• Distribute  Any  Data  Structure  
– Available  Anywhere  on  the  Grid   Node Singleton

– Automatic  Remote  Access  via  Proxies  

• Controlled  Deployment  
– Support  for  Cluster  Singleton  
– Support  for  Node  Singleton  
– Support  for  Custom  Topology   Grid Singleton
– Load  Balanced  
• Guaranteed  Availability  
– Auto  Redeployment  in  Case  of  Failures

©  2014  GridGain  Systems,  Inc.

 In-­‐Memory  Streaming  and  CEP
• Streaming  Data  
Never  Ends  
• Branching  Pipelines  
• Pluggable  Routing  
• Sliding  Windows  for  
CEP/Continuous  
Query  
• Real  Time  Analysis

©  2014  GridGain  Systems,  Inc.

 In-­‐Memory  Hadoop  Accelerator
• Plug  and  Play  installation  
• 10x  to  100x  Acceleration  
• In-­‐Memory  Native  
MapReduce  
• In-­‐Process  Data  Colocation  
• GGFS  In-­‐Memory  File  
System  
• Pure  In-­‐Memory  
• Read-­‐Through  from  HDFS  
• Write-­‐Through  to  HDFS    
• Sync  and  Async  Persistence

©  2014  GridGain  Systems,  Inc.

 In-­‐Memory  Hadoop  Accelerator
• In-­‐Memory  Native  
Performance  
• Zero  Code  Change  
• Use  existing  MR  code  
• Use  existing  Hive  queries  
• No  Name  Node  
• No  Network  Noise  
• In-­‐Process  Data  Colocation  
• Eager  Push  Scheduling

©  2014  GridGain  Systems,  Inc.

 Management  &  Monitoring

✴ Enterprise  Edition  Only

©  2014  GridGain  Systems,  Inc.
 Customer  Use  Cases

> Automated Trading Systems
 > Big Data Analytics


Real time analysis of trading positions & market risk. Customer 360 view, real-time analysis of KPIs, up-to-
High volume transactions, ultra low latencies. the-second operational BI.

> Financial Services
 > Online Gaming 


Fraud Detection, Risk Analysis, Insurance rating and Real-time back-ends for mobile and massively parallel
modeling. games.

> Online & Mobile Advertising
 > SaaS Platforms & Apps

Real time decisions, geo-targeting & retail traffic High performance next-generation architectures for
information. Software as a Service Application vendors.

©  2014  GridGain  Systems,  Inc.

Use  Case:  
Largest bank in Eastern Europe, and the third largest in Europe

• Open  tender  won  by  GridGain  

1 Billion
– Goal:  Real-­‐time  risk  and  leverage  
reporting  on  their  global  financial   Transactions per Second
trading  portfolio    
– Performed  a  detailed  evaluation  and  
software  assurance  test   10 Dell R610 servers

< $25K
– Delivered  best  performance,  scale   1 TB Memory
and  high  availability  

©  2014  GridGain  Systems,  Inc.

 ANY  QUESTIONS?

www.ignite.incubator.apache.org #apacheignite

©  2014  GridGain  Systems,  Inc.