Search Optimization

Search Optimization
turn data into doing™ 1 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Document Usage Guidelines
• Should be used only for enrolled students
• Not meant to be a self-paced document, an instructor is needed
• Lab Exercise slides reference the hands-on lab exercise guide
• Do not distribute

12 August 2022
Search Optimization
turn data into doing™ 2 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Course Goals
• Identify methods for optimizing searches
• Explain search scheduler precedence
• Define the three types of accelerations available in Splunk
• Use the tstats and datamodel commands to query data

Search Optimization
turn data into doing™ 3 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Course Outline
• Optimize Search
• Report Acceleration
• Data Model Acceleration
• Use the tstats Command

Search Optimization
turn data into doing™ 4 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
 Optimize Search

Search Optimization
turn data into doing™ 5 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Topic Objectives
• Understand how search modes affect performance
• Examine the role of the Splunk Search Scheduler
• Review general search practices

Search Optimization
turn data into doing™ 6 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Search Modes and Search Performance
• The Splunk user interface provides 3 search modes
• Search modes determine how much field data is returned and as a
result, affects how fast the search completes

Search Optimization
turn data into doing™ 7 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Search Modes
• Smart Mode
– Defaultsearch mode
– Balances speed and completeness

• Fast Mode
– Prioritizes
speed over completeness
– Disables Field Discovery

• Verbose Mode
– Prioritizescompleteness over speed
– Returns all extracted fields

Search Optimization
turn data into doing™ 8 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
General Search Practices
• As events are stored by time, _time is the most efficient filter
• After time, most powerful fields to filter on: index, host, source,
and sourcetype
• Create efficient searches by including specific search terms in the
basic search (i.e. search criteria)
sourcetype=access_combined failure failure
More specific & completes faster Less specific & less efficient

• Inclusion is generally better than exclusion

"access denied" NOT "access granted"
Inclusion Exclusion

Search Optimization
turn data into doing™ 9 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Splunk Search Scheduler
• Manages scheduled reports and alerts (i.e. scheduled searches)
• Prioritizes searches if too many are scheduled to run concurrently
– Concurrentsearch limit is determined by system configuration
– Searches can be skipped
Priority Type Examples
First Ad hoc historical searches Searches run manually
Manually scheduled searches User-saved scheduled reports and alerts
Second
with real-time scheduling that use real-time scheduling
Manually scheduled searches User-saved scheduled reports and alerts Note
Third
with continuous scheduling that use continuous scheduling "Real-time
The searches behind report acceleration scheduling" has
nothing to do with
Last Automatically scheduled searches and data model acceleration; "auto- whether a search
summarization" reports "runs in real time."

Search Optimization
turn data into doing™ 10 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
 Report Acceleration

Search Optimization
turn data into doing™ 11 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Topic Objectives
• Define acceleration and acceleration types
• Understand report acceleration and create an accelerated report
• Reveal when and how report acceleration summaries are created
• Search against acceleration summaries

Search Optimization
turn data into doing™ 12 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
What is Acceleration?
• A Splunk feature that relies on summaries of event data to speed
up search performance
• There are 3 acceleration
methods:
– Reportacceleration
– Summary Indexing
– Data model acceleration

Note
Data model acceleration is the easiest
and most efficient acceleration option
and should be your first choice.

Search Optimization
turn data into doing™ 13 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
How Summaries Make Searches Efficient
• Searches run against summaries should complete much
faster because:
– Summaries are considerably smaller than the original data set from
which they are generated
– Summaries contain only the data needed to fulfill the searches run
against them
• Summaries can be automatically or manually created
– Determined by what searches are being accelerated and which
acceleration method is chosen

Search Optimization
turn data into doing™ 14 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
"Acceleration" vs "Summary"
• The terms "acceleration" and "summary" are not interchangeable
• Report acceleration, data model acceleration, and summary
indexing are all acceleration methods that rely on summaries
• The differences in these methods are:
– How they are made
– How they are maintained
– How they are used

• These differences are discussed in more detail in the next topics

Search Optimization
turn data into doing™ 15 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Report Acceleration
• Reports that span a large volume of data can:
– Takea long time to complete
– Consume a lot of system resources
• Accelerated reports run off acceleration summaries which:
– Storeonly the data needed to fulfill the report
– Are automatically populated in the background

Accelerated reports run faster because they are running off

curated, updated data
Search Optimization
turn data into doing™ 16 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Report Acceleration: How to Qualify
• Reports must meet certain guidelines to be eligible for
report acceleration:
– Users must have the schedule_search privilege and
accelerate_search capabilities (power and admin users
have this by default)
– Search mode should be set to either 💡💡Smart or ⚡Fast
– Search must include a transforming command

Search Optimization
turn data into doing™ 17 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
 Report Acceleration: Commands
An accelerated report must include transforming commands and
may include streaming and non-streaming commands:
Must be included May be included, however order is important

Transforming Commands Streaming commands Non-streaming commands

• Order results into a • Operate on each event as it • Execute after all events
data table is returned by search are returned
...| stats ...| eval ...| table

...| timechart ...| search ...| sort

...| top ...| fields ...| fillnull

...| rename

Search Optimization
turn data into doing™ 18 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Report Acceleration: Streaming Commands
Distributable streaming
commands typically run on Centralized streaming
the indexers and are the only commands always execute
command type allowed at the search head and are
before a transforming only allowed after a
command (they are allowed
Search
Head
transforming command
after too)
...
| <transforming command> ...
| <distributable streaming command> ... ...
Execution depends Indexers Always | <transforming command> ...
on command order | <centralized streaming command> ...
...
| <distributable streaming command> ...
| <transforming command> ...

Search Optimization
turn data into doing™ 19 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Report Acceleration: To Summarize
If there are any commands that come before the transforming
command, they must be distributable streaming commands

| eval ... | transaction ...

| stats ... | stats ...

If there are any commands that come after the transforming

command, they can be streaming (distributable or centralized) or
non-streaming commands
| stats ... | stats ...
| eval ... | eventstats ...

Search Optimization
turn data into doing™ 20 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
What Reports Qualify for Acceleration?
index=web sourcetype=access_combined action=purchase status=200
| stats sum(price) as revenue by productId
| eval revenue = "$".revenue

index=web sourcetype=access_combined action=purchase status=404

index=web sourcetype=access*
| fields price action host
| chart sum(price) over action by host

index=web sourcetype=access_combined
| transaction clientip startswith="view" endswith="purchase"
| stats avg(duration) as avgDuration

Search Optimization
turn data into doing™ 21 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
 Report Acceleration Flowchart
What is your role? Yes
Are there any
commands before the
transforming What is the search
Does your search have a
command? If yes, are mode?
transforming command?
Power Admin they distributable
User
streaming commands?

No Yes
Run a regular search Splunk changes
(or check with admin) mode to Smart
when you
about being granted accelerate.
schedule_search and
accelerate_search No
privileges.
You can accelerate the report!

Search Optimization
turn data into doing™ 22 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
 Accelerating a Report
Create a qualifying search
1
1 index=sales sourcetype=vendor_sales

and save as a report

| stats values(Vendor) as Vendor sum(price) as
revenue by VendorCity
| sort 10 -revenue
2 After report saved,
click Acceleration
3 Check the box next to
Accelerate Report and
choose the 3

Summary Range 2

4 Click Save
4

Search Optimization
turn data into doing™ 23 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Report Acceleration: Summary Range
• Determines how much time the
acceleration summary spans
relative to now
• Searches within the time range
only use summary data
• Splunk automatically removes
older summary data that ages
out of range
Note
Report acceleration features automatic
backfill. If for some reason you have a data
interruption, Splunk software can detect
this and automatically update or rebuild
your summaries as appropriate.

Search Optimization
turn data into doing™ 24 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
 Accelerating a Previously Saved Report
A previously saved report
can be accelerated too 1
1 Click on Reports in the app
navigation bar and select a
saved report
2 Edit > Edit Acceleration and
enable the qualifying report 2

Search Optimization
turn data into doing™ 25 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Viewing Accelerated Reports
Once accelerated, a lightning bolt appears next to the saved
report in Settings > Searches, Reports, and Alerts

Search Optimization
turn data into doing™ 26 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Acceleration Summary Not Created
Even if report acceleration is enabled, Splunk may not create an
acceleration summary

Suspend summary
Is the number of events creation for 24 hours
and check again
returned > 100,000 hot Yes
bucket events?
Will summary
No be too large? Yes

Run as normal search

Note Create acceleration
No
A summary is too large if it will
exceed 10% of your total
summary
bucket size in your deployment.

Search Optimization
turn data into doing™ 27 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Acceleration Summary Not Created (cont.)
• Some searches run faster without a summary if:
– There
are fewer than 100K events in the summary range
– Summary size is projected to be too big

• If acceleration summary was defined and not created for the

above reasons, Splunk:
– Continues to check periodically
– Creates a summary if/when the report meets the requirements

Search Optimization
turn data into doing™ 28 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Acceleration Summary Created
• Acceleration is a good option for reports that call on 100k or more
events for the summary range selected
• Splunk automatically populates acceleration summaries every
10 minutes
• Report acceleration summaries are stored by time alongside
buckets in your indexes
– Buckets are filesystem directories that store events within indexes

Search Optimization
turn data into doing™ 29 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Acceleration Summary Created (cont.)
Splunk automatically shares summaries with users who have
access to the accelerated report

Users of an accelerated
Any searches run by these
shared report benefit from
users pull data from the
having access to the
acceleration summary
acceleration summary for
when possible
that report

Search Optimization
turn data into doing™ 30 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Searching an Acceleration Summary
• In addition to saved accelerated reports, ad hoc searches can
use the summary when:
– Search criteria matches the base saved search
– The user executing the ad hoc query has permission to the
acceleration summary
• You can also append the search string with additional commands,
for example:
index=web sourcetype=access_combined
index=web sourcetype=access_combined
| stats count by price
| stats count by price
| eval discount = price/2
Populating Search Ad Hoc Search

Search Optimization
turn data into doing™ 31 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Using Summaries

• The Job Inspector shows when summaries are being used

• Deleting all reports that use an acceleration summary
automatically deletes the acceleration summary
Note
The Job Inspector will show summaries being used for a
search even if the search time range is longer than the
summary range. This is because the portion of the search
within the summary range will still benefit from acceleration.
The rest of the search will run over raw data.

Search Optimization
turn data into doing™ 32 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Viewing Report Acceleration Summaries
Settings > Report Acceleration Summaries
– Summary ID and Normalized Summary ID: unique hashes assigned to the
summary (clicking these hashes loads the summary details page)
– Reports Using Summary: saved reports associated with the summary

Search Optimization
turn data into doing™ 33 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Viewing Report Acceleration Summaries (cont.)
Settings > Report Acceleration Summaries Note

– Summarization Load: calculation of effort to update summary If Summarization Load

is high and Access
Count is low, consider
SL = time to run populating report / interval of populating report deleting the summary.

– Access Count: how often summary used

Search Optimization
turn data into doing™ 34 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
 Viewing Report Acceleration Summaries (cont.)
Settings > Report Acceleration Summaries
– Summary Status: either % of summary complete at that moment, or a
status value
• Summarization not started
• Pending: the search head
about to schedule new update
for the summary
• Building summary
• Complete
• Suspended: summary size too
big to be useful
• Not enough data to summarize:
summary size too small (fewer
than 100K events)

Search Optimization
turn data into doing™ 35 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Viewing Summary Details
• Click on Summary ID to view
Summary Details
– Size on Disk: how much storage
space the summary takes up
– Summary Range: range of time
spanned by the summary, relative to
present moment
– Timespans: size of data chunks
comprising the summary
– Buckets: number of index buckets the
summary spans
– Chunks: number of data chunks
comprising the summary
Search Optimization
turn data into doing™ 36 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Viewing Summary Details (cont.)
• Actions
– Verify: examines a subset of the
summary and verifies that all
examined data is consistent
– Update: updates the summary
– Rebuild: rebuilds the summary
from scratch
– Delete: deletes the summary
Note
If accelerated report isn't returning expected
results, it may be that an underlying tag, event
type, or field extraction rule was changed. If
that happens, use Verify to determine whether
data is consistent. If verification fails, use
Rebuild to recreate the summary.

Search Optimization
turn data into doing™ 37 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Report Acceleration Lab Exercise
Time: 20 minutes
Tasks:
• Verify a search will qualify for report acceleration
• Accelerate a search
• Run an accelerated report

Search Optimization
turn data into doing™ 38 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
 Data Model
Acceleration

Search Optimization
turn data into doing™ 39 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Topic Objectives
• Understand data model acceleration
• Accelerate a data model
• Use the datamodel command to search data models

Search Optimization
turn data into doing™ 40 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Data Model Acceleration
• Generates summaries to speed pivot and report completion times
• Takes the form of inverted time-series index (tsidx) files that have
been optimized for speed
• Two types:
– Ad hoc data model acceleration
– Persistent data model acceleration

Note
Ad hoc data model acceleration occurs
automatically when accessing a data model
through Pivot. Ad hoc data model acceleration
and Pivot are outside the scope of this course.
Persistent data model acceleration will be the
focus of this course.

Search Optimization
turn data into doing™ 41 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
What are Data Models?
• Hierarchically
structured datasets
made up of search constraints = search
and fields
• Represents specific
categories of data
• Can be accelerated
for faster performance

Search Optimization
turn data into doing™ 42 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Persistent Data Model Acceleration
• Persistent data model acceleration builds dedicated summaries in
indexes and exists as long as the data model exists
• Once accelerated, Splunk maintains the dedicated summaries
• Reports and dashboard panels generated from persistently
accelerated data models complete more quickly
• Summaries can be used by Pivot, datamodel, and tstats
• Multiple users can access the summary at the same time
Note
The datamodel and tstats
commands are discussed in a
later topic. Pivot is outside the
scope of this course.

Search Optimization
turn data into doing™ 43 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Comparing Data Model Accelerations
Ad Hoc Persistent
The acceleration is built every time the Pivot
Explicitly defined before using
editor is accessed

Exists only for the duration of user’s Pivot

Exists as long as data model exists
editor session

Runs over all time (i.e., can’t be scoped to

Can be scoped to specific time ranges
specific time range)

Reports run without any acceleration Reports run faster and perform better overall

Note
The Splunk Pivot editor allows a user to create
a table, chart, or visualization.

Search Optimization
turn data into doing™ 44 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
 Accelerating a Data Model
1 Click Settings > Data Models
2 Select a data model and click Edit > Edit Acceleration

3 Click the Accelerate check box and choose a Summary Range

4 Click Save

Search Optimization
turn data into doing™ 45 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
After Accelerating a Data Model
Splunk builds an acceleration summary for the specified summary
range in the form of time-series index (tsidx) files

.tsidx raw

Files are stored in the

index containing events Index Bucket Files
that have fields specified
A ...
in the data model
B Acceleration summaries for
Index persistently accelerated data
.tsidx
.tsidx models are stored parallel to
.tsidx
their corresponding buckets

Data Model Acceleration Summary

Search Optimization
turn data into doing™ 46 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Time Series Index (tsidx) Files
• Exist inside buckets alongside raw data files
• Consist of a lexicon and a posting list and the indexed
field::value combinations (host, source, and sourcetype)
Lexicon
The lexicon is an alpha-
… Accepted djohnson Failed for from invalid numerically ordered list of terms
nobody password port ssh2 sshd sysadmin … found in the data at index time
Posting List
.tsidx
Accepted
The posting list is an array of
djohnson pointers that match each term to
. events in the raw data files
.
.

sshd[87755]: Accepted password for djohnson from 10.3.10.46 port 2988 ssh2 Splunk uses the pointers to
sshd[3954]: Failed password for invalid user sysadmin from 10.3.10.46 port 4759 ssh2
sshd[1268]: Failed password for mail from 10.3.10.46 port 1617 ssh2
search just the events that
raw
sshd[4816]: Failed password for nobody from 10.3.10.46 port 4412 ssh2 match the terms, making the
sshd[5744]: Failed password for sync from 10.3.10.46 port 4664 ssh2 search much more efficient

Search Optimization
turn data into doing™ 47 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Searching the Acceleration Summary
• Indexer retrieves
1

information about the data

model that has been 2

stored on disk in the tsidx

.tsidx raw

files that make up the ...

Index Bucket Files

acceleration summary
Index
• Indexer pulls additional
2
Indexers .tsidx
.tsidx
.tsidx

events from bucket files if 1

search is outside data

Data Model Acceleration Summary

summary range

Search Optimization
turn data into doing™ 48 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Accelerated Data Model Considerations
• The acceleration summary always contains a store of data that at
least meets the summary range (may slightly exceed)
• Splunk updates tsidx files every 5 minutes and removes
outdated summary data every 30 minutes
• Accelerated data model summaries can be accessed through:
– Pivoteditor (outside the scope of this course)
– Searches using pivot, tstats or datamodel

Search Optimization
turn data into doing™ 49 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Comparing Acceleration Methods
• Uses automatically created summaries to speed completion times for
qualified reports
Report • Easier to create than summary indexes and backfills automatically
Acceleration • Depending on the defined time span, periodically ages out data
• Can correct gaps and overlaps from the UI “rebuild” feature
• Cannot create a “data-cube” and report on smaller subsets

Data Model • Uses automatically created summaries to speed completion times for pivots
Acceleration • Takes the form of time-series index (tsidx) files

Search Optimization
turn data into doing™ 50 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
datamodel Command
• Used to display the structure of a | datamodel

data model or to search against it

• Returns a description of all or a
specified data model and its objects
• A generating command
– Must follow a leading | pipe
– Must be the first command in a search

Note
Use the datamodel command by itself (without
arguments) to display all the data models in your
deployment that you have access to.

Search Optimization
turn data into doing™ 51 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
datamodel Command (cont.)
• If the name of the data | datamodel AccButtercup_Games_Online_Sales

model is included as the

first argument, Splunk
shows the details of that
data model in JSON format
| datamodel [data_model_name]

• Click the + next to

objectNameList to show
all dataset names in the
selected data model
Search Optimization
turn data into doing™ 52 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
datamodel Command (cont.)
| datamodel AccButtercup_Games_Online_Sales

The successful_purchase object is

the third object in this data model

View details of a dataset

within a data model by
expanding objects A

The successful_purchase object is

B the third object in this data model

Search Optimization
turn data into doing™ 53 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
datamodel Command (cont.)
Alternatively, you can display a dataset within a data model by
using the dataset name as the second argument
| datamodel [data_model_name] [dataset_name]

| datamodel vsales apac

Search Optimization
turn data into doing™ 54 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
datamodel Command: Options
To view the events associated with the specified dataset, use the
search option
| datamodel [data_model_name] [dataset_name] search

| datamodel vsales apac search

Search Optimization
turn data into doing™ 55 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
datamodel Command: Options (cont.)
The flat option returns the same results as search but field
names are "flattened" by stripping hierarchical information

| datamodel vsales apac search | datamodel vsales apac flat

Search Optimization
turn data into doing™ 56 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
datamodel Command: Options (cont.)
• The dataset name and search argument aren’t valid unless
preceded by the data model name
• When using the datamodel command, the data model name and
dataset name are case sensitive

Search Optimization
turn data into doing™ 57 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
datamodel Command: summariesonly
| datamodel [data_model_name] [dataset_name] search summariesonly=<bool>

• summariesonly=true returns results only from the tsidx data

generated by the acceleration summary, i.e. summarized data
– Maximizes speed of search execution
• summariesonly=false (default) returns summarized and
unsummarized data
| datamodel vsales apac search | datamodel vsales apac search summariesonly=true

Search Optimization
turn data into doing™ 58 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Data Model Acceleration Lab Exercise
Time: 15 minutes
Tasks:
• Use the datamodel command to search a data model
• Use the summariesonly function of the datamodel command to
view the event count of two data models over the last 5 minutes

Search Optimization
turn data into doing™ 59 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
 Use the
tstats Command

Search Optimization
turn data into doing™ 60 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Topic Objectives
• Explore the tstats command
• Search acceleration summaries with tstats
• Search data models with tstats
• Compare tstats and stats

Search Optimization
turn data into doing™ 61 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
tstats Command
| tstats <stats-func> [summariesonly=<bool>]
[from datamodel=<data_model_name>]
[where <searchQuery>] [by <field-list>]

• Performs statistical queries on tsidx files

• Requires use of a statistical function
• Use a from clause to pull events from a specific data model
• Use where clause to filter results
• Group results with a by <field-list> clause Note

• Generating command Generating commands must

follow a pipe character and be the
first command in a search.

Search Optimization
turn data into doing™ 62 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
tstats Command (cont.)
• When querying indexed data, tstats searches tsidx files which
means:
– Search is limited to only indexed fields in the tsidx
– tstats searches execute very fast because it does not
open or read raw events

.tsidx
| tstats values(sourcetype) as sourcetype by index

Note raw

tstats use acceleration

summaries if they are available or
This search was run over All Time and
if summariesonly=true.
completed in 0.17 seconds!

Search Optimization
turn data into doing™ 63 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
tstats Command: stats-func
Scenario ? | tstats values(sourcetype) as sourcetype by index
ITOps wants a list of all source
types by index.

Most functions available to

stats can be used with tstats

Search Optimization
turn data into doing™ 64 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
tstats Command: from Clause
Use the from clause to search through tsidx files not created at
index time such as data model acceleration summary tsidx files
Scenario ? | tstats count from datamodel=AccButtercup_Games_Online_Sales
TechOps wants a count of all web
requests during the last 24 hours.

Search Optimization
turn data into doing™ 65 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
tstats Command: Without from Clause
Scenario ? | tstats count by index
User wants to count the events | sort -count
per index, for all indexes to which
they have access.

If you don’t use a from

clause, a search is
performed on indexed
fields in the index tsidx

Note
Statistical queries can only be
performed on indexed fields, not
search time fields.

Search Optimization
turn data into doing™ 66 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
tstats Command: by Clause
Group by any number of fields using by <field-list>
Scenario ?
TechOps is reconfiguring the web
servers and wants a count of all | tstats count from datamodel=AccButtercup_Games_Online_Sales by host
web requests per web server over
the last 24 hours.

Search Optimization
turn data into doing™ 67 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
 tstats Command: summariesonly
| tstats <stats-func> [summariesonly=<bool>] [from datamodel=<data_model_name>]

• true or t returns results only from the tsidx data generated by

the acceleration and does not include non-summarized data
• false or f (default) generates results from both summarized and
non-summarized data
| tstats count from (All time) 348,217 results by scanning
datamodel=AccButtercup_Games_Online_Sales 348,223 events in 0.335 seconds

| tstats count from

datamodel=AccButtercup_Games_Online_Sales (All time) 348,199 results by scanning
summariesonly=t 348,199 events in 0.107 seconds

Search Optimization
turn data into doing™ 68 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
tstats Command: summariesonly (cont.)
• When running a search with summariesonly set to false, you
might notice a larger result count because:
– Some of the index data may not have been added to the summary yet
– The search range may be greater than the summary range

• If used with an unaccelerated data model, summariesonly=t

produces no results
| tstats count from datamodel=AccButtercup_Games_Online_Sales summariesonly=t
Accelerated Data Model

| tstats count from datamodel=Buttercup_Games_Online_Sales summariesonly=t

Unaccelerated Data Model

Search Optimization
turn data into doing™ 69 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
 Data Model Field Names with tstats
• Use a data model field with tstats by | datamodel AccButtercup_Games_Online_Sales

referencing its location in the data model

with dot notation (owner.fieldName)
• Use the datamodel command to return
details of the data model and its objects
– Then,note the owner for the field you want
to access
| tstats sum(http_request.price) from
datamodel=AccButtercup_Games_Online_Sales

Search Optimization
turn data into doing™ 70 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
datamodel Notation With tstats
If a data model has more than one accelerated root dataset, you
must specify the dataset you want by using dot notation
datamodel.dataset
| tstats sum(http_request.price) as tsales from
datamodel=AccButtercup_Games_Online_Sales.http_request
where (http_request.action=purchase AND http_request.status=200) by http_request.product_name

Note
This example also uses
owner.fieldName notation as shown
on the previous slide. The owner
name is the name of the dataset.

Search Optimization
turn data into doing™ 71 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
 datamodel Notation With tstats Example
Scenario ? | tstats sum(http_request.price) as tsales from
The Online Sales manager datamodel=AccButtercup_Games_Online_Sales.http_request
launched a new campaign where (http_request.action=purchase AND http_request.status=200)
yesterday. Provide her with the by http_request.product_name
total sales for yesterday. | sort - tsales
| eval tsales="$".tostring(tsales,"commas")
| rename http_request.product_name as Product, tsales as "Daily Sales"
| fields Product, "Daily Sales"

Search Optimization
turn data into doing™ 72 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Searching Unaccelerated Data Models
• tstats can search | tstats count from datamodel=Buttercup_Games_Online_Sales
unaccelerated data models by host
| sort -count
– However, searches run the 3 results by scanning 272,746 events in 4.885 seconds
same as a normal search
with no performance benefit | tstats count from datamodel=AccButtercup_Games_Online_Sales
by host
• A best practice is to use | sort -count

tstats with accelerated 3 results by scanning 272,746 events in 0.745 seconds

data models

Search Optimization
turn data into doing™ 73 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
tstats Command: span Option
• If you group by _time, use
Search Time Range Default Span
span (e.g., span=3m) to 5 minutes 5 seconds
group into time buckets 15 minutes 10 seconds

• If you don’t specify a span, 60 minutes 1 minute

4 hours 5 minutes
the value set by the time 24 hours 30 minutes
picker determines the range 7 days 1 day

Search Optimization
turn data into doing™ 74 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
tstats Command: Wildcards
• tstats does not support wildcarded fields, however the wildcard
can be used in the where clause to search on field values
• You can specify:
| tstats count where host=w* by source
| sort -count

• But not these:

| tstats count(source*)

| tstats count where host=w* by source*

Search Optimization
turn data into doing™ 75 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
 tstats versus stats for Indexed Fields
Scenario
IT is doing resource planning and
?
When working with a massive amount of data and
wants the event load for the
security index. Count the events
using indexed fields, consider using tstats
for all time by source, sourcetype,
and host. Sort descending on
count and format with commas.

| tstats count as events index=security

where index=security by source, sourcetype, host | stats count as events by source, sourcetype, host
| sort -events | sort –events
| eval events = tostring(events,"commas") | eval events = tostring(events, "commas")

11 results by scanning 971,466 events in 0.08 seconds 11 results by scanning 971,016 events in 1.59 seconds

| tstats count as events index=*

where index=* by source, sourcetype, host | stats count as events by source, sourcetype, host
| sort -events | sort –events
| eval events = tostring(events,"commas") | eval events = tostring(events, "commas")

101 results by scanning 12,945,032 events in 0.39 seconds 101 results by scanning 12,944,443 events in 58.8 seconds

Search Optimization
turn data into doing™ 76 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
stats to tstats Search Optimization
• Any datamodel search using the stats command is converted
automatically to use tstats
Since stats to tstats
| datamodel vsales us search optimization is already | tstats sum(us.sales) from
| stats sum(us.price) by us.product_name enabled, this search datamodel=vsales.us by us.product_name
will become:

• A similar optimization is available for stats searches but, by

default, this feature is not enabled
If stats to tstats
index=web optimization is
enabled, this search
| tstats count where index=web
| stats count
will become:

• Greatly increases speed of searches that rely solely on indexed

fields or simple counts
Search Optimization
turn data into doing™ 77 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Using the tstats Command Lab Exercise
Time: 15 minutes
Tasks:
• Use the tstats command to return a count of all events
• Use the tstats command to search and transform summarized
data from a data model

Search Optimization
turn data into doing™ 78 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
 Wrap-up Slides

Search Optimization
turn data into doing™ 79 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Community
• S p l u n k C o mmu n i t y P o rt a l • Splunk Dev Google Group
community.splunk.com groups.google.com/forum/#!forum/splunkdev

– Answers • Splunk Docs on Twitter

– Discussions twitter.com/splunkdocs
– Splunk Trust • Splunk Dev on Twitter
– User Groups twitter.com/splunkdev
– Ideas • Splunk Live!
splunklive.splunk.com
• Splunk Blogs
splunk.com/blog/ • .conf
conf.splunk.com
• Splunk Apps
splunkbase.com

Search Optimization
turn data into doing™ 80 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Support Programs
• Web
– Documentation: dev.splunk.com and docs.splunk.com
– Wiki: wiki.splunk.com

• Splunk Lantern
Guidance from Splunk experts
– lantern.splunk.com
• Global Support
Support for critical issues, a dedicated resource to
manage your account – 24 x 7 x 365
– Web: splunk.com/index.php/submit_issue
• Enterprise, Cloud, ITSI, Security Support
– Web: splunk.com/en_us/about-splunk/contact-us.html#tabs/customersupport
– Phone: (855) SPLUNK-S or (855) 775-8657

Search Optimization
turn data into doing™ 81 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Learning Paths (cont.)
Knowledge Manager - Recommended Courses
Free eLearning courses are in blue and courses with an * are
present in both learning paths.
• What is Splunk * • Enriching Data with Lookups
• Introduction to Splunk * • Data Models
• Using Fields * • Introduction to Dashboards
• Introduction to Knowledge Objects • Dynamic Dashboards
• Creating Knowledge Objects • Using Choropleth
• Creating Field Extractions • Search Optimization *

Search Optimization
turn data into doing™ 82 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Learning Paths
Search Expert - Recommended Courses
Free eLearning courses are in blue and courses with an * are
present in both learning paths.
• What is Splunk * • Result Modification
• Introduction to Splunk * • Leveraging Lookups
• Using Fields * and Subsearches
• Scheduling Reports and Alerts • Correlation Analysis
• Visualizations
• Search Under the Hood
• Statistical Processing
• Working with Time • Multivalue Fields
• Comparing Values • Search Optimization *
Search Optimization
turn data into doing™ 83 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
 Splunk Certification
Offerings & Requirements

Search Optimization
turn data into doing™ 84 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Splunk Core and Beyond
Regardless of which Splunk product you use, it all starts with Splunk Core
Splunk Cloud

Splunk Core
Recommended

Splunk Enterprise

Search Optimization
turn data into doing™ 85 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Splunk Core Certified User
This entry-level certification demonstrates an individual's basic
ability to navigate and use Splunk software

Prerequisite Certification(s): Splunk Core Certified User Exam Congratulations! You are a...
• None Time to study! We suggest candidates looking to prepare for
this exam complete Fundamentals 1 or the following courses:
Prerequisite Course(s): • What is Splunk?
• None • Intro to Splunk
• Using Fields
• Scheduling Reports and Alerts
• Visualizations
• Statistical Processing
• Working with Time
Recommended Next Step
• Leveraging Lookups and Subsearches
• Search Optimization • Splunk Core Certified Power User
• Enriching Data with Lookups
• Data Models
See here for registration assistance.

Search Optimization
turn data into doing™ 86 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Splunk Core Certified Power User
This entry-level certification demonstrates an individual's
foundational competence of Splunk’s core software

Prerequisite Certification(s): Splunk Core Certified Power User Exam Congratulations! You are a...
• None Time to study! We suggest candidates looking to prepare for
this exam complete Fundamentals 2 or the following courses:
Prerequisite Course(s): • Visualizations
• None • Statistical Processing
• Working with Time
• Comparing Values
• Result Modification
• Correlation Analysis
• Search Under the Hood
Recommended Next Steps
• Introduction to Knowledge Objects
• Creating Knowledge Objects • Splunk Core Certified Advanced Power User
• Creating Field Extractions • Splunk Enterprise Certified Admin
• Data Models
• Using Choropleth • Splunk Cloud Certified Admin
See here for registration assistance.

Search Optimization
turn data into doing™ 87 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Splunk Core Certified Advanced Power User
This certification demonstrates an individual's ability to generate complex searches,
reports, and dashboards with Splunk’s core software to get the most out of their data

Prerequisite Certification(s): Splunk Core Certified Advanced Power User Exam Congratulations! You are a...
• Splunk Core Certified Power User Time to study! We suggest candidates looking to prepare for this exam
complete Fundamentals 3, Creating Dashboards, and Advanced
Searching & Reporting or the following courses:
Prerequisite Course(s):
• Using Fields
• None • Working with Time
• Comparing Values
• Result Modification
• Leveraging Lookups and Subsearches
• Correlation Analysis
• Search Under the Hood
Recommended Next Steps
• Multivalue Fields
• Search Optimization • Splunk Enterprise Certified Admin
• Creating Field Extractions
• Splunk Cloud Certified Admin
• Enriching Data with Lookups
• Data Models
• Using Choropleth
• Introduction to Dashboards
• Dynamic Dashboards
See here for registration assistance.

Search Optimization
turn data into doing™ 88 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022
Thank You

Search Optimization
turn data into doing™ 89 Copyright © 2022 Splunk, Inc. All rights reserved | 12 August 2022