Scribd
Upload
15 views

KQL Query Optimization Guide

Uploaded by

Aaditya Arora
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
15 views

KQL Query Optimization Guide

Uploaded by

Aaditya Arora
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

Modern

Secops.com

Sentinel
Query
Optimization
Guide
How to write faster searches in Sentinel
11 KQL best practices with 11 real life examples

inspired by:
https://learn.microsoft.com/en-us/kusto/query/best-practices
Modern
Secops

Filter first
Bring the where statement as close to the
data source as possible.
This speeds up the search by making the
dataset smaller before applying other
transformations.

SigninLogs
| join OfficeActivity on AppId
| where Location == ‘US’

SigninLogs
| where Location == ‘US’
| join OfficeActivity on AppId
Modern When possible, replace contains with
Secops has. Has matches the full token, any word
seperated by special characters.
For example:
full_token
✅ has token and ✅contains token
fullToken
🚫 !has token and ✅contains token
SigninLogs
| where LocationDetails contains “Maryland”

SigninLogs
| where LocationDetails has “Maryland”

Use “has”
over
“contains”
Start with
a limit

When exploring a new dataset, get


started with a limit.
Once you have the query you want,
remove that limit.

Syslog
| take 10

Modern
Secops
Modern
Secops Use the case sensitive
operator whenever you can.
Operators like:
has_cs, contains_cs, ==

If you need a case insensitive


search, use the case
insensitive operator, don’t
convert to lowercase.

SinginLogs
| where tolower(UserDisplayName) ==
‘seyed nouraie'

SinginLogs
| where UserDisplayName =~ ‘seyed nouraie’

Be careful
about cases
Filter on a
dynamic
column, then
parse it
If accessing data in a dynamic
column, first filter with a ‘has’
operator on the column, then
parse the element you want to
filter on.

SigninLogs
| where LocationDetails has 'maryland'
| where LocationDetails.state =~ 'maryland'

Modern
Secops
Modern
Secops

Broadcast and
shuffle your
joins If the left left side of
your join is smaller, do
a broadcast join.
When both sides are
large, try a shuffle join.

Be careful...
If the left side is too
large, the broadcast will
fail.

AADNonInteractiveUserSignInLogs
| join kind=inner hint.strategy=broadcast (
SigninLogs
| where UserDisplayName has "seyed")
on UserDisplayName
Modern
Secops Lookup
instead of
join
If the left side of the join is small, try a
lookup instead.

Be careful...
Just like a broadcast join, a lookup will
fail if the left side is too large.

SigninLogs
| where UserDisplayName has "seyed"
| lookup kind=inner AADNonInteractiveUserSignInLogs
on UserDisplayName
When reusing the same tabular variable
in a query multiple times, try
Modern materializing it.
Secops
Be careful...
Materialize uses more memory, so if the
calculation is too small or the variable
isn’t reused often, then the tradeoff might
not be worth it.
Also if the data being materialized is too
large, the query will fail.

let RareLocations = materialize(SigninLogs


| summarize hits = count() by state =
tostring(LocationDetails.state)
| top 10 by hits asc
| project state);
SigninLogs
| extend state = tostring(LocationDetails.state)
| join RareLocations on $left.state == $right.state
| union (
AADNonInteractiveUserSignInLogs
| extend state = tostring(parse_json(LocationDetails).state)
| join RareLocations on $left.state == $right.state)
union...

Materialize
Parse,
don’t
extract
When parsing a single column into
multiple parts, use a single parse
statement instead of using multiple
extract statements.

AzureActivity
| extend ResourceProvider =
extract(@"(.*?)\/",1, OperationNameValue),
Endpoint =
extract(@"(.*?)\/(.*)", 2, OperationNameValue)

AzureActivity
| parse OperationNameValue with
ResourceProvider:string "/"
Endpoint:string
Modern
Secops
Modern When possible, filter on
Secops
columns that exist in the
table schema.

Avoid filtering on columns


that you generate
dynamically.

AzureActivity
| parse OperationNameValue with
ResourceProvider:string "/"
Endpoint:string
| where Endpoint has ‘write’

AzureActivity
| where OperationNameValue
has ‘write’
| parse OperationNameValue with
ResourceProvider:string "/"
Endpoint:string

Filter on real
columns
Shuffle your
summaries

If you’re summarizing by a
column that has high
cardinality (over 1M unique
elements) use a shuffle
summary.

AADNonInteractiveUserSignInLogs
| summarize hint.strategy=shuffle
dcount(IPAddress) by UniqueTokenIdentifier

Modern
Secops

You might also like