0% found this document useful (0 votes)
21 views65 pages

Elastic Masterclass

The document outlines the agenda for an Elastic Partner Technical Enablement event scheduled for April 2024, covering topics such as Elastic's platform overview, GenAI use cases, observability, and security. It highlights the capabilities of the Elasticsearch platform, including its applications in AI, security, and observability, and emphasizes the importance of integrating various data sources for enhanced analytics. Additionally, it discusses the deployment models and licensing options available for Elastic products.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
21 views65 pages

Elastic Masterclass

The document outlines the agenda for an Elastic Partner Technical Enablement event scheduled for April 2024, covering topics such as Elastic's platform overview, GenAI use cases, observability, and security. It highlights the capabilities of the Elasticsearch platform, including its applications in AI, security, and observability, and emphasizes the importance of integrating various data sources for enhanced analytics. Additionally, it discusses the deployment models and licensing options available for Elastic products.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 65

Elastic Partner Technical

Enablement
Apr 2024
Topics Time
Opening 09.00 - 09.15
Elastic Overview (High Level) and Licensing 09.15 - 10.30
Elastic GenAI use cases 10.30 - 12.00
Lunch Break 12.00 - 13.00
Elastic Observability AIOps Slides + Demo / Short Workshop 13.00 - 14.30
Break 14.30 - 14.40

Elastic Security Slides + Demo / Short Workshop 14.40 - 16.00


Elastic Cloud Positioning and Licensing
https://play.instruqt.com/elastic/invite/p3qngotopalf

https://ela.st/lambda-2404
Agenda
- Elastic Platform Overview
- Elastic Search x GenAI
- Elastic Observability
- Elastic Security
- Deployment Models
- Licensing
- Discovery & Sizing
- Differentiation
Elastic powers about 90% of all search bars on the
internet (Forbes)

3.6B+ downloads and counting

Behind firewalls, the most popular


search engine (DB Engines)
One Search Analytics Platform
Two Out-of-the-Box Solutions
The Freedom to Build Anything

Out-of-the-Box Solutions Build Your Own

Elastic Observability Elastic Security Elastic Search

Elasticsearch™ Platform

Ingest Secure & Scalable Visualization Workflow


AI / ML Search
Storage Automation
Performance that Delivers
Relevant Results in Real-time
Out-of-the-Box Solutions Build Your Own

Any Data, Elastic Elastic Security Elastic Search Business Outcomes


Any Source Observability for Everyone
Generative AI Apps,
Logs, APM, Tracing,
SIEM, Endpoint, Cloud Product Search, Workplace
Metrics, Synthetics,
Search, Custom Search Apps
Profiling, RUM

Databases
Elasticsearch™ Platform

Legacy 69%
Systems Improvement in
Ingest & Secure Storage AI / ML & Search Visualization & Automation
customer and
Public Cloud employee satisfaction
Applications Data Extraction Full-Text / Vector Search Share & Collaborate
60%
Transformation / Machine Learning Data Exploration Reduction in risk
SaaS Apps Data Visualization
Normalization Correlations
On-Premises
Custom Dashboards 62%
Enrichment Analytics & Aggregations
Reduction in
Web Services 3rd Party Integrations revenue disruption
Loading / Indexing Data Manipulation

Intelligent Data Storage Federated Searches & Queries Workflow Automation


Files *Validated by
Security / Governance third-party research
Lead with:
SIEM
Expand across:
DevSecOps

Expand within:
+ Endpoint
+ Cloud
+ XDR

Lead with:
Log analytics

Expand across:
Expand within: Trusted Customer
+ APM Experiences
+ Infrastructure
+ Digital exp
(RUM + synth)

Expand within:
+ eCommerce
Expand across: + Workplace search
Observability for + Customer support
Customer Experiences

Journey With Lead with:


Search apps

Elastic
ElasticSearch x GenAI
How do we bridge Enterprise KB with GenAI Apps?

Enterprise Private KB LLMs

Sensitive databases
Multi-system / cloud information
Private knowledge bases
Case histories

`
An Analogy with Chicken Rice!

Private Fridge LLMs

Relevant Ingredients
Chicken, White Rice, Garlic

Irrelevant Ingredients
Pork, Beef
Elastic provides the bridge between
private data and GAI

Enterprise Private KB LLMs

Sensitive databases
Multi-system / cloud information
Private knowledge bases
Case histories

`
Retrieval Augmented Generation

Your Question
Your Question The Right
+
Context Window
Answer

GAI / LLM

Your Business’s Data Public Internet Data

documents images audio


Why not send all business data?

● Token Ceilings limitations


○ A 3-page document = 2,171 tokens
○ Current GPT-4 model limits: 8,192
○ Full token consumption: 4 documents

● Cost
○ Requests per day: 86,400 (1 request / sec)
○ Average cost (GPT-4): $0.045 / 1k tokens
○ Estimated cost per day: $3,888*
○ Estimated cost per month: $116k
* Assume 1k tokens per request
Enter: Elasticsearch Relevance Engine™ (ESRE)

Your Question
Your Question The Right
+
Context Window
Answer

GAI / LLM

Your Business’s Data Public Internet Data

documents images audio


Let’s See It - DEMO Time!!
Elastic is a Vector Database

Most Vector Databases


Hybrid Search
(text [BM25] + vector)
Store & Search
Vector
Some Vector Databases Choice & Flexibility
Embeddings
of embedding
models

Filtering & Faceting


Elasticsearch Create Vector
Embeddings
Aggregations
Autocomplete

Search Optimized for text,


Trained model
Analytics geo, date ranges &
out-of-the-box
other data
Elasticsearch provides the full scope of RBAC Ingest Tools (web crawler,
connectors, Beats, Agent,
necessary capabilities for Generative AI Document-level
API framework)
Security
applications, beyond those provided by
On-prem / Cloud /
point-solution vector databases Hybrid
Elastic Learned Sparse Encoder (ELSER v2)
Out of the box relevance, across domains
● Allows devs to implement semantic
search without the need to train
their own model

● Out-performs on: question-answer


pairs, weather records, medical text

● Generalises across domains without


training

● Great relevance, out of the box

● Currently English-only
Elastic Observability
Accelerate from Insights to Action with Elastic Observability

Out-of-the-Box Solutions Build Your Own

Any Data, Elastic Business Outcomes


Any Source Observability for Everyone
Logs, APM, Tracing, Metrics,
Synthetics, Profiling, RUM

Traditional Signals
Elasticsearch™ Platform
Logs
10x
Metrics Faster response times
Ingest & Secure Storage AI / ML & Search Visualization & Automation for customers
Traces

Data Extraction Full-Text / Vector Search Share & Collaborate 33%


Improvement in IT
Transformation / Machine Learning Data Exploration labour efficiency
Other Data
Normalization Correlations Data Visualization

SLI/SLOs Custom Dashboards 63%


Enrichment Analytics & Aggregations
Acceleration in
Business KPIs 3rd Party Integrations decision making
Loading / Indexing Data Manipulation

IOT data Intelligent Data Storage Federated Searches & Queries Workflow Automation
*Validated by
Security / Governance third-party research
Teams need AI-powered observability
Challenges of the old world Transformation New world

Evolution of
observability

Monitoring Observability AI-powered


● Siloed tools ● Tool sprawl observability
● Monolithic apps & infra ● Cloud native app & ● Unified Visibility
infra complexity
● Manual processes ● Open & extensible platform
● Exponential data growth
● Interactive, actionable insights
● AIOps
Unified full stack visibility: Context-aware insights
Increase productivity & improve collaboration

Log monitoring & analytics Integrated full stack views

Cloud & infrastructure


monitoring

Application Performance
Monitoring

Digital Experience
Monitoring

Universal Profiling
Open and flexible: Any data, any source
Integrate with the technology ecosystem you rely on

Data architecture based


on OpenTelemetry

Common data model Bring your own


ML Models
(OTel + ECS)

API-driven, Elastic Common


350+ integrations Schema (ECS)

Open ML models,
plus bring your own

Future proof your investments


AI & Analytics: Correlate any type of data
Accelerate problem resolution and improve predictability

Anomaly detection &


Democratize data and analytics
correlations
There's a spike in
hourly revenue

Log categorization

100+ out-of-the-box
customizable ML models

Generative AI powered
by ESRE
Typically, we make about
$400, but this hour we
made over $2500
Let’s See It - DEMO Time!!

AI Assistant for
Observability
Powered by
Machine Learning
Observability
AI Assistant for
Observability
Powered by
AI Assistant for
Observability
Powered by
AI Assistant for
Observability
Powered by
Custom Dashboards
Observability
Collect data
Centralize information from across your environment

Any data, any source


Apps & Services
Cloud Infra
Databases Public cloud

Hosts Data store


Users On-premises

Network
IoT & OT

Collect Normalize Enrich Detect Analyse Respond


Normalize data
Streamline analysis with a common schema

Search without a common schema Search with a common schema

src: 10.10.10.10 source.ip: 10.10.10.10


OR client_ip: 10.10.10.10
OR apache2.remote_ip: 10.10.10.10
OR context.user.ip: 10.10.10.10
OR src_ip: 10.10.10.10

Collect Normalize Enrich Detect Analyse Respond


Enrich data
Contextualize data automatically and on demand

IP: 10.10.10.10 File hash: 98j4h3 Account: john.doe


● Reputation: Risky ● Known malware: No ● Risk score: 71
● Geolocation: Russia ● Org: Finance
● ISP: Konkat ● Permissions: Admin

Collect Normalize Enrich Detect Analyse Respond


Detect Anomaly
Automate detection with correlation, AI and ML

Anomaly Detection Machine Learning Alerts

Collect Normalize Enrich Detect Analyse Respond


Detect Anomaly
Automate detection with correlation, AI and ML

Log Anomalies Machine Learning Models


“Auto-learn the pattern and set smart baseline,
“Automatically detect anomalies, outliers from group, and
Identify and alert if any anomalies detected”
rare events”

Collect Normalize Enrich Detect Analyse Respond


Analyse Root Cause
Classifying bad behaviors in Infra and Apps describing service degradation

Collect Normalize Enrich Detect Analyse Respond


AIOps Labs Features
Automated real time root cause analysis

Analyze Log Spikes Log Pattern Analysis Metric Change Point Detection

Collect Normalize Enrich Detect Analyse Respond


Universal Profiling
Optimise code performance with whole-system visibility
Unobtrusive. Frictionless Deployment
Powered by eBPF. Requires zero-instrumentation, no code changes
or app restarts. Gain faster ROI

Whole-System Visibility
Unlock unknown-unknowns - from the kernel through userspace
into high-level code, across multi-cloud workloads

Polyglot Visibility
C/C++, Rust & Go (without debug symbols on host) + PHP, Python,
Java (or any JVM language), Ruby, Perl & NodeJS

Extremely Low Overhead


Continuous profiling in production with negligible overhead. 1% <
CPU, ~250MB of RAM

Collect Normalize Enrich Detect Analyse Respond


Unified Observability
Single platform for all business and operational data

Correlate logs, metrics and traces for Isolate problems quickly across
faster investigation complex architectures

Collect Normalize Enrich Detect Analyse Respond


Respond quickly
Resolve problem before damage occurs

Automate downstream workflows


and resolve problems

Collect Normalize Enrich Detect Analyse Respond


Elastic Security
Accelerate from Insights to Action with Elastic Security

Out-of-the-Box Solutions Build Your Own

Any Data, Elastic Security Business Outcomes


Any Source for Everyone

SIEM, Endpoint, Cloud


Bring Your Data

Detect, Investigate, and Respond


Cloud Elasticsearch™ Platform
Network
65%
Host Risk reduction of
User Ingest & Secure Storage AI / ML & Search Visualization & Automation customer churn from
security incidents
More

Data Extraction Full-Text / Vector Search Share & Collaborate 41%


Reduction in employee
Transformation / Machine Learning Data Exploration disruption
Native Protection
Normalization Correlations Data Visualization
Block threats with Elastic Agent 18%
Enrichment Analytics & Aggregations Custom Dashboards Reduction in security
Windows, macOS, Linux operations costs
Loading / Indexing Data Manipulation 3rd Party Integrations
Servers and VMs
Intelligent Data Storage Federated Searches & Queries Workflow Automation
Containers & Kubernetes
*Validated by
Cloud Providers Security / Governance third-party research
Future proof your SOC
A proven track record of innovation

Elastic Threat
Cloud Intel SIEM
Kubernetes Integration
Elasticsearch/ Integrating Found Machine Integrating
Kibana (Elastic Cloud) Learning OSQuery Elastic build.security
is born platform Manager Defend and Cmd
Elasticsearch
as a service SIEM is GA XDR ES|QL

2010 2012 2015 2016 2018 2019 2020 2021 2022 2023

Integrating Endgame

Automated anomaly Endpoint prevention,


ELK Stack detection & time detection, and Searchable Elastic ML Model Cloud AI Assistant
is formed Packetbeat series forecasting response Snapshots Agent Management Security
Elastic named a
Leader in The
Forrester Wave™
Security Analytics
Platforms Q4
2022
The Forrester Wave™ is copyrighted by Forrester Research, Inc. Forrester and
Forrester Wave™ are trademarks of Forrester Research, Inc.

The Forrester New Wave™ is a graphical representation of Forrester’s call on a market.


Forrester does not endorse any vendor, product, or service depicted in the Forrester
Wave™.

Information is based on best available resources. Opinions reflect judgment at the


time and are subject to change.
Elastic Security
Bring your data
3rd-party integrations

Cloud Network Host SIEM Endpoint Cloud

Connectors
Ticketing SOAR
Dashboards & Advanced entity Automation &
User Threat intel Threat detection
reporting analytics remediation

LLMs Notification
Enabled by ML + AI
Data lake And more
Elastic Limitless Data

Data ingest
Data search, processing, and normalization

Normalized to ECS / OTel semantic convention


Native protection

Elastic Security AI Assistant


Elastic cross-cluster search
Windows macOS Linux

Securely leverage generative AI,


coupled with the Elasticsearch
Containers Servers
Relevance Engine, to accelerate triage,
investigation, adoption, and more

Cloud providers And more


Google Amazon Microsoft Local

Cloud storage S3 Azure blob NAS

Elastic searchable snapshots


AI Assistant for
Security
Powered by

● Alert Summarization
● Query Generation
● Workflow Suggestions
● Data Ingestion Helper
● Query Conversion
● Agent Integration Advice
Deployment
Elastic Deployments

SaaS Self-Managed

Elastic Cloud Elastic Stack Elastic Cloud Elastic Cloud


Enterprise on Kubernetes

Across AWS, GCP,


Azure Regions
worldwide Kubernetes
VMs Docker
ASEAN: SG, ID Services

Also available On-Premise


on Marketplace / Cloud
Licensing
Resource-based Pricing

Elastic Enterprise Search Elastic Observability Elastic Security

Site Search APM SIEM


No query-based pricing No agent-based pricing No seat/ingest-based pricing

App Search Logs Endpoint Security


No docs-based pricing No ingest-based pricing No endpoint-based pricing

Workplace Search Metrics


No user-based pricing No host-based pricing
Basic vs Platinum vs Enterprise

Basic (Free) ECE


ELK

Observability ECK
SSO
Search Security
Platinum Frozen
CCR
Data Tier
Community
Support Graph
Per Node Analytics Searchable
Enterprise Snapshots
Machine
AIOps Doc & Field
Learning
Security
Elastic AI
Enterprise Integration
Assistant
Support (Jira, SOAR)
ELSER
RRF K/CSPM
Per ERU
More info at
EDR
https://www.elastic.co/subscriptions Access to Elastic
Account Team
Threat Intelligence
Mgmt
Maps Server
Data Tiering &
Searchable Snapshots
Frozen Data Tier
Reduce more than 50% of costs by leveraging
on object storage options

Search across snapshots


No rehydration required, always real-time
search across all tiers
Data Tiering &
Searchable Snapshots
Frozen Data Tier
Reduce more than 50% of costs by leveraging
on object storage options

Search across snapshots


No rehydration required, always real-time
search across all tiers
Region X Region Y Region Z

Cross Cluster
Search & Replication
Cross Cluster Search
Search across deployments, reduce data
transfer costs
Overview Cluster
Cross Cluster Replication
Disaster recovery and resilience against failure
for business continuity
Optimize security spend
Flexibility to handle hybrid and multi cloud complexities

Other SIEM Challenges Elastic SIEM

?
EU Cloud A $$$ ? $$$$ AP Cloud C EU Cloud A AP Cloud C

Centralized Centralized
SIEM SIEM

Cloud Costs Minimize data access challenges


● Data egress charges? ● Secure distributed search
Hybrid Cloud Complexity ○ Correlation across instances & workloads
● Feature parity? ○ Maintain data residency / sovereignty
● Data residency concerns? ○ Full role / attribute based access control
● Cloud availability? ○ Minimize data egress charges
● Cloud / self managed feature parity
● 50+ Elastic Cloud regions
Discovery + Sizing
Discovery!!
Additional Discovery
Discover, Discover, Discover Questions
● Who’s the customer contact and can you
identify their pain points?
● What happened and what are they trying ● Is this a new solution or is this replacing an
to solve? existing one? If applicable, what are you looking
● Who will benefit if this issue is resolved? to improve over the existing solution?
● Budget? ● What are your goals?
● Timeline? ● How are you measuring the success of this
● Can they continue to operate without project?
fixing this problem and why? ● What kind of source data are you trying to
● Is this On-Prem or Cloud? Which CSP? collect?
● What percentage growth do you expect from
● Daily Ingest your data volumes over the next 12-24 months?
● Retention Period
○ How long to store in Hot, Warm,
Cold and Frozen tiers?
Sizing Calculator
Discover, Discover, Discover
● Who’s the customer contact and can you
identify their pain points?
● What happened and what are they trying
to solve?
● Who will benefit if this issue is resolved?
● Budget?
● Timeline?
● Can they continue to operate without
fixing this problem and why?
● Is this On-Prem or Cloud? Which CSP?
Sizing Estimation Guide
● Daily Ingest
Retention Period

○ How long to store in Hot, Warm,
Sizing Calculator
Cold and Frozen tiers?
Differentiation
Elasticsearch Platform Advantages

Elasticsearch™ Platform
Elasticsearch™ Platform | 3.6B Downloads

Accurate, relevant, business-specific insights. Works


with any LLM for context-aware output you can trust.

Scalable On-prem Single


economics + cloud platform

● Performant & ● Flexible deployment ● Observability +


cost-efficient tiered options, 50+ global security in one
storage cloud locations platform
● Predictable ● HA & data locality with ● Seamless
consumption billing cross cluster cross-cluster search
replication & analytics
Elastic’s Differentiation

Observability Security
Architecture
Building Blocks
Elastic Building Blocks

More info at
https://www.elasticpartneracademy.com
High-level functional Architecture

1. Ingestion from Beats directly to Elastic Cluster


2. Ingestion from Beats to Logstash
3. Ingestion from Beats to Apache Kafka messaging queue
4. Ingestion from Elastic Agent to Logstash / Elasticsearch
5. Ingestion from Apache Kafka messaging queue to Logstash
6. Ingestion from Logstash to an Elasticsearch cluster
Please note: steps 2-6 are optional

7. Elasticsearch cluster to index, store and analyze


8. Data Analysis visualization in Kibana
9. Creating Snapshots
10. Fleet Server managing the Elastic Agents
What’s Next

Try now on Cloud now! Get your free


Accreditations on
Partner Academy
Thank You

You might also like