KQL Query Optimization Guide
KQL Query Optimization Guide
Secops.com
Sentinel
Query
Optimization
Guide
How to write faster searches in Sentinel
11 KQL best practices with 11 real life examples
inspired by:
https://learn.microsoft.com/en-us/kusto/query/best-practices
Modern
Secops
Filter first
Bring the where statement as close to the
data source as possible.
This speeds up the search by making the
dataset smaller before applying other
transformations.
SigninLogs
| join OfficeActivity on AppId
| where Location == ‘US’
SigninLogs
| where Location == ‘US’
| join OfficeActivity on AppId
Modern When possible, replace contains with
Secops has. Has matches the full token, any word
seperated by special characters.
For example:
full_token
✅ has token and ✅contains token
fullToken
🚫 !has token and ✅contains token
SigninLogs
| where LocationDetails contains “Maryland”
SigninLogs
| where LocationDetails has “Maryland”
Use “has”
over
“contains”
Start with
a limit
Syslog
| take 10
Modern
Secops
Modern
Secops Use the case sensitive
operator whenever you can.
Operators like:
has_cs, contains_cs, ==
SinginLogs
| where tolower(UserDisplayName) ==
‘seyed nouraie'
SinginLogs
| where UserDisplayName =~ ‘seyed nouraie’
Be careful
about cases
Filter on a
dynamic
column, then
parse it
If accessing data in a dynamic
column, first filter with a ‘has’
operator on the column, then
parse the element you want to
filter on.
SigninLogs
| where LocationDetails has 'maryland'
| where LocationDetails.state =~ 'maryland'
Modern
Secops
Modern
Secops
Broadcast and
shuffle your
joins If the left left side of
your join is smaller, do
a broadcast join.
When both sides are
large, try a shuffle join.
Be careful...
If the left side is too
large, the broadcast will
fail.
AADNonInteractiveUserSignInLogs
| join kind=inner hint.strategy=broadcast (
SigninLogs
| where UserDisplayName has "seyed")
on UserDisplayName
Modern
Secops Lookup
instead of
join
If the left side of the join is small, try a
lookup instead.
Be careful...
Just like a broadcast join, a lookup will
fail if the left side is too large.
SigninLogs
| where UserDisplayName has "seyed"
| lookup kind=inner AADNonInteractiveUserSignInLogs
on UserDisplayName
When reusing the same tabular variable
in a query multiple times, try
Modern materializing it.
Secops
Be careful...
Materialize uses more memory, so if the
calculation is too small or the variable
isn’t reused often, then the tradeoff might
not be worth it.
Also if the data being materialized is too
large, the query will fail.
Materialize
Parse,
don’t
extract
When parsing a single column into
multiple parts, use a single parse
statement instead of using multiple
extract statements.
AzureActivity
| extend ResourceProvider =
extract(@"(.*?)\/",1, OperationNameValue),
Endpoint =
extract(@"(.*?)\/(.*)", 2, OperationNameValue)
AzureActivity
| parse OperationNameValue with
ResourceProvider:string "/"
Endpoint:string
Modern
Secops
Modern When possible, filter on
Secops
columns that exist in the
table schema.
AzureActivity
| parse OperationNameValue with
ResourceProvider:string "/"
Endpoint:string
| where Endpoint has ‘write’
AzureActivity
| where OperationNameValue
has ‘write’
| parse OperationNameValue with
ResourceProvider:string "/"
Endpoint:string
Filter on real
columns
Shuffle your
summaries
If you’re summarizing by a
column that has high
cardinality (over 1M unique
elements) use a shuffle
summary.
AADNonInteractiveUserSignInLogs
| summarize hint.strategy=shuffle
dcount(IPAddress) by UniqueTokenIdentifier
Modern
Secops