UNIT-VI Log Management Through Splunk

1
Introduction
• Splunk is a powerful platform for analyzing machine-generated data.
• It helps organizations search, monitor, and analyze big data via a
web-based interface.
• Used in data centers and marketing departments for valuable
insights.
What is Splunk?
• Splunk is a software platform widely used for
• monitoring, searching, analyzing and visualizing the machine-generated data
in real time.
• It performs
• capturing, indexing, and correlating the real time data in a searchable
container and produces graphs, alerts, dashboards and visualizations.
• Splunk provides easy to
• access data over the whole organization for easy diagnostics and
• solutions to various business problems.

3
What is Splunk? Contd..
• Splunk is log analyzing and monitoring tool
• which communicates with the different log files and
• stores file’s data in the form of events into local indexes
• Splunk has the capability
• to show data in different forms of dashboards
• which is useful for the application users and higher leadership

4
What is Splunk? Contd..
• In other words, It is a proprietary software used by companies to
collect and analyze the data they produce.
What is Splunk? Contd..
Why Splunk?
• We need to monitor large amount of data being generated to improve
the functioning of our systems.

7
Why Splunk? (Contd..)
• We need to improve the large amount of data that is being generated
to improve functioning of the systems.
Uses of Splunk
Need of Splunk

10
Contd..
• Splunk Monitoring tool offers plenty of benefits for an organization.
• Some of the benefits of using Splunk are:
• Offers enhanced GUI and real-time visibility in a dashboard
• Reduces troubleshooting and resolving time by offering instant results.
• It is a best-suited tool for root cause analysis.
• Splunk allows you to generate graphs, alerts, and dashboards.
• You can easily search and investigate specific results using Splunk.
• It allows you to troubleshoot any condition of failure for improved
performance.
• Helps you to monitor any business metrics and make an informed decision.
• Splunk allows you to incorporate Artificial Intelligence into your data strategy.

11
Contd..
• Allows you to gather useful Operational Intelligence from your
machine data.
• Summarizing and collecting valuable information from different logs
• Splunk allows you to accept any data type like .csv, json, log formats,
etc.
• Offers most powerful search analysis, and visualization capabilities
to empower users of all types.
• Allows you to create a central repository for searching Splunk data
from various sources.

12
 Key Features of Splunk
• Important key features of Splunk
are:
• Real-time Data Processing
• Machine Learning & AI for Threat
Detection
• Custom Dashboards and Alerts
• Security Information and Event
Management (SIEM)
• Business Analytics and Performance
Monitoring

13
Who Uses Splunk?
• System Administrators & IT Staff
– Monitor configurations and user activity.
• Network Engineers
– Troubleshoot network issues and misconfigurations.
• Security Analysts & Incident Response Teams
– Detect security threats and fraud.
• Application Developers & Support Staff
– Investigate performance issues.
• Managers & CIOs
– Build reports and dashboards for monitoring IT health.
Different ways of using Splunk

15
Splunk Enterprise
• Splunk Enterprise collects, analyzes and acts on the value of the data
generated by technology infrastructure, security systems and business
applications
• It gives the insights to drive operational performance and business results

16
Splunk Cloud
• Splunk cloud delivers all the features of Splunk Enterprise, as a cloud
based service
• The platform provides access to Splunk Enterprise Security and the Splunk
App for AWS and it enables centralized visibility across cloud, hybrid and
on-premises environments

17
Splunk Light
• Splunk Light is a solution for small IT environments that automates log search and analysis

• It speeds troubleshooting by gathering real-time data from your distributed applications and
infrastructure in one place to enable
• powerful searches, dynamic dashboards, alerts and reporting for real-time analysis all at an attractive
price well within the budget.

18
How Does Splunk work?
Splunk Architecture
• Two type of Splunk Architecture:
• Single- Server Environment
• Distributed Environment
Splunk Architecture- Single Server Environment
• Single server is not used for actual deployment. A single component
of Splunk is installed to test Splunk component or to have a proof of
concept or for personal use and learning.
Splunk Architecture-Distributed Environment
Splunk Components

23
Processing Components

24
Management Components

25
Splunk Architecture
Splunk Data Flow
Splunk Data Pipeline
Splunk Data Pipeline
Splunk Data Pipeline
Splunk Flow
Splunk License management
How Licensing Works
• Every Splunk software component requires a license.
• Splunk license will specify the amount of data a given component can
work with and what kind of features it will be able to have.
Identify License types
License requirements
Major Component of Splunk Architecture

36
Universal Forward (UF)
• Universal forward or UF is a lightweight component which
• pushes the data to the heavy Splunk forwarder.
• You can install Universal Forward at client side or application server.
• The job of this component is only to forward the log data.
• It’s a light weight component
• Can not generate report, dashboard etc.
• Only forward the data.

37
Load Balancer (LB)
• Load balancer is default Splunk load balancer.
• However, it also allows you to use your personalized load balancer.

38
Heavy forward (HF)
• Heavy forward is a heavy component.
• This Splunk component allows you to filter the data.
• Example:
• collecting only error logs.

39
Indexer
• Indexer helps you to store and index the data.
• It improves Splunk search performance.
• By default, Splunk automatically performs the indexing.
• For example,
• host, source, and date & time.

40
Search head (SH)
• Search head is used to gain intelligence and perform reporting.

41
Deployment Server(DS)
• Deployment server helps to deploy the configuration.
• For example,
• update the UF configuration file.
• We can use a deployment server to share between the component

42
License manager (LM)
• The license is based on volume & usage —
• for example,
• 50 GB per day.
• Splunk regular checks the licensing details.

43
Use Cases: Domino’s

• Dominos has the following challenges like

• omni channel presence, several touch points, multiple systems for delivery, huge
customer database, manual search, error prone, Less visibility, and reactive mode
• so answer was Splunk. 44
Contd..

• So how this challenges were solved using Splunk?

• how we have solved that problem with help of interactive map, real time
feedback, Dashboard, Payment process, Promotional support and performance
monitor. 45
Contd..
• Interactive Map • Payment process
• It shows all the orders coming from • Analyze the speed of different payment
across us in real time modes
• It brought employee satisfaction • Determine error free payment modes
• Real time feedback • Promotional support
• So employee constantly see what • It tracks how various promotional offers are
customers are seeing and impacting in real time
• It help them to understand customer • Initially determining the impact of
expectations promotions took almost a day but
• With the help of Splunk it was so much faster
• The Dashboard
• Performance monitor
• It is used to keep score and set targets • It monitor the performance of domino’s in-
• Compare performance with previous house developed point of sale system
week.
46
Use Cases: Domino’s
• Problem Statement
❖ Domino’s had no clear visibility into what offer works the
best –in terms of
✓Offer type ( for e.g. 10% or $2 off)
✓Cultural differences at a region level
✓Device used
✓Time of purchase
✓Order revenue
❖They required insights on consumer behaviour and
customer to offers
47
Data Sources: Domino’s

48
Indexer For Data Storage and Processing

49
Search Head for Analysis and Visualization

50
Customer’s Benefits

51
Contd..
• So customer’s benefit include that by keeping mission critical
application and infrastructure
• up and running 50% faster time to market for new apps provided
• By modernizing and strengthening cyber defenses
• 70% of the risk of data breach, IP theft and fraud was lowered
• By releasing apps faster and improving developer efficiency
• 80% of the down time was reduced.

52
UNIT -VII: Incident Response and Management