SQL Server 2016 / 2017 Mission-Critical (DB Engine)

Performance Security Availability / Platform Scalability

Operational Analytics Always Encrypted SQL Server 2017 on Linux Machine Learning Services
Insights on operational data; Works with in- Sensitive data remains encrypted at all times Enhanced AlwaysOn R Scripting along with Python scripting from
memory OLTP and disk-based OLTP with ability to query Three synchronous replicas for auto failover the SQL Server Engine
In-memory OLTP Enhancements across domains
Row-Level Security Round robin load balancing of replicas
Greater T-SQL surface area, terabytes of Apply fine-grained access control to table Graph DB Support
memory supported, and greater number of rows Automatic failover based on database health
parallel CPUs For modeling many-to-many relationships
DTC for transactional integrity across
Dynamic Data Masking database instances with AlwaysOn
Live Query Statistics Real-time obfuscation of data to prevent Enhanced Database Caching
unauthorized access Support for SSIS with AlwaysOn
Cache data with automatic, multiple TempDB
files per instance in multi-core environments
Query Store Advanced Threat Detection Stretch Database
Monitor and optimize query plans Ability to find unusual login patterns, track Archive historical data transparently and
usage behavior in an auditing database, track securely to Azure New Programmatic Improvements
SQL injection vulnerability, and more New TSQL Functionality, Maintenance Plan
Automatic Database Tuning Queries stretch across local data as well as Improvements, New ALTER DATABASE
Other Enhancements Azure data
Provides insight into potential query Options
performance problems, recommends Audit success/failure of database operations Expanded support for JSON data
solutions, and can automatically fix identified TDE support for storage of in-memory OLTP New PolyBase query engine integrates SQL
problems tables Server with external data in Hadoop or Azure
Enhanced auditing for OLTP with ability to Blob storage
DMV Improvements track history of record changes
Temporal Database Support
Query data as points in time
Adaptive Query Processing
A feature family that introduces a new
generation of query processing
improvements
New in SQL Server 2017
SQL Server 2016/2017 Monitoring and Tooling
Traditional Troubleshooting
Extended Events is scalable

Performance Monitor Query Store is persisted and improving

Counters
Performance Dashboard Reports

Dynamic Management Views Live Query Statistics

Graphical / Text • dm_exec_requests
SHOWPLAN • dm_exec_query_stats Lightweight Query Profiling
• dm_os_wait_stats polling
Expanded Query Plan Diagnostics

SQLProfiler and SSMS Dump Analysis (Preview)

SQL Trace
SQL Server Vulnerability Assessment

SQL Data Discovery and Classification

“A Bad Plan is not the one which
failed, but the one which
~Anonymous DBA
succeeded at the Greatest Cost.”
Why Use Plan Guides?
3rd party
applications

Hints

OPTION
Plan Guides Stored Procedures
sp_create_plan_guide

sp_control_plan_guide

sys.plan_guides
Common Query Hints Used in Plan Guides
• OPTIMIZE FOR (Value, Unknown)
• RECOMPILE

• NULL
Plan Guides from Cache
• Also known as ‘Plan Freezing’
• Prevents a current cached plan from changing
• Example: -- Create a plan guide for the query by specifying the query plan in the plan
cache.
DECLARE @plan_handle varbinary(64);
DECLARE @offset int;
SELECT @plan_handle = plan_handle, @offset = qs.statement_start_offset
FROM sys.dm_exec_query_stats AS qs
sp_create_plan_guide_from_handle
CROSS APPLY sys.dm_exec_sql_text(sql_handle) AS st
[ @name = ] N'plan_guide_name'
CROSS APPLY sys.dm_exec_text_query_plan(qs.plan_handle,
, [ @plan_handle = ] plan_handle
qs.statement_start_offset,
, [ [ @statement_start_offset
qs.statement_end_offset) AS qp = ] {
statement_start_offset
WHERE | NULL } ]p.Name, OrderQty, DueDate%';
text LIKE N'SELECT WorkOrderID,

EXECUTE sp_create_plan_guide_from_handle
@name = N'Guide1',
@plan_handle = @plan_handle,
@statement_start_offset = @offset;
GO
-- Verify that the plan guide is created.
SELECT * FROM sys.plan_guides
WHERE scope_batch LIKE N'SELECT WorkOrderID, p.Name, OrderQty, DueDate%';
GO
USE PLAN

SELECT *
FROM Sales.SalesOrderHeader h, Sales.SalesOrderDetail
OPTION (USE PLAN N'<ShowPlanXML
xmlns="http://schemas.microsoft.com/sqlserver/2004/07/showplan"
Version="0.5"
Build="9.00.1187.07">
<BatchSequence>
<Batch>
<Statements>
…
</Statements>
</Batch>
</BatchSequence>
</ShowPlanXML>
')
 Query and Query Plan Fingerprints
• •

•
When performance is not good…
• Database is not • Impossible to • Regression
working predict / root caused by
cause upgrade

Website / App is Temporary Perf. System

down issues Upgrade

Plan choice change can cause these problems

Have You Ever…?
…Had your system down/slowed down and everyone waiting for you to
magically fix the problem ASAP?

Query Plan
…Upgraded an application to the latest SQL Server version
choice changes
and had an issue with a plan change slowing your application down?
can cause all of
these problems!
…Had a problem with your SQL database performance and been unable
to determine what was going wrong?
What are you doing today?
• Most solutions are reactive in nature
• Flush the bad plan from the cache with sp_recompile
• Flush the entire plan cache with DBCC FREEPROCCACHE
• Force the plan to recompile every time
• Restart OS / SQL Server (It works for some reason?)

• Proactive solutions are challenging

• Often takes a long time to even detect there is a plan problem
• Only the latest plan is stored in the cache
• Need to catch both the good and the bad plan in order to troubleshoot
• Information is stored in memory only
• Reboot or memory pressure causes data to be lost
• No history or timing available – stats are aggregated for what is currently in cache
Addressing Plan Choice Regressions

• Can you modify the query to hint it?

• Can you figure out how to make a plan guide?
Tackling the Problem – What Could We Do?
1. Store the history of plans for each query
2. Baseline the performance of each plan over time
3. Identify queries that have “gotten slower recently”
4. Find a way to force plans quickly and easily
5. Make sure this works across server restarts, upgrades,
and query recompiles
This is what the Query Store does for you!
Introducing the Query Store
• Plan store persists execution plans per database
• Runtime stats store persists execution statistics per
database
• New views and graphical interface allow you to quickly and
easily troubleshoot query performance
• Quickly find query plan performance regressions
• Fix plan regressions by forcing a previous plan
• Determine the number of times a query was executed in a given time window
• Identify Top N Queries in the past X hours
• Audit the history of query plans for a given query
• Analyze the resource usage patterns for a particular database
Demonstration:
Enabling Query Store in
SQL Server 2016
• Query Store Properties
Key Usage Scenarios
Find and fix Identify top Reduce risks Deep analysis
query plan resource with server of workload
regressions consumers upgrade patterns/perf
SQL Query Execution
Query Execution

Found Plan Cache

lookup

Not
Found
Get forced plan
Compile and
Fetch Plan from Cache
Optimize Query
Send text and plan
Query Store
Recompile
Check for
Recompile

Not Needed

Execute Query

Finish Execution
 Query 1 - n 1 - n Runtime
Query Plan Stats
Text
Compile stats:
query_store_query_text
query_context_settings
Context Runtime Stats
Interval query_store_query
Settings query_store_plan
sys.query_store_wait_stats (2017)

Runtime stats:
One Row Per Query Text Per One Row Per Plan One Row Per
Plan Affecting Option (for each query) Plan Per Time query_store_runtime_stats_interval
(example: ANSI NULLS on/off) Interval query_store_runtime_stats
(example: 5 min)
Key DMVs for Query Store
SELECT * FROM sys.query_store_query_text ▪ The DMVs shown here are
SELECT * FROM sys.query_store_query enabled and populated for
SELECT * FROM sys.query_store_plan each database when Query
Store is turned on
SELECT * FROM sys.query_store_runtime_stats
ORDER BY runtime_stats_id

SELECT * FROM
sys.query_store_runtime_stats_interval

SELECT * FROM sys.query_store_wait_stats

SELECT * FROM sys.query_context_settings

Query Store Details
• Plans and execution data are stored on disk in the user
database
• Query store data persists reboots, upgrades, restores etc.
• Plans and statistics are tracked at the database level rather than the server level

• Query Store is configurable

• Settings such as MAX_SIZE_MB, QUERY_CAPTURE_MODE, CLEANUP_POLICY allow
you to decide how much data you want to store for how long
• Can be configured either via the SSMS GUI or T-SQL scripts

• Query Store can be viewed and managed via scripting or

SSMS
What does Query Store Track?
• Query Texts start at the first character of the first token of
the statement; end at last character of last token
• Comments before/after do not count
• Spaces and comments inside *do* count

• Context_settings contains one row per unique combination

of plan-affecting settings
• Different SET options cause multiple “queries” in the Query Store
• Plan caching/recompilation behavior unaffected
What Gets Captured?

• Count of executions of each captured plan

• For each metric: average, last, min, max, stddev
• Metrics: duration, cpu_time, logical_io_reads, logical_io_writes,
physical_io_reads, clr_time, DOP, query_max_used_memory,
rowcount
• Data is recorded when a query execution ends
 Keeping stability while upgrading to
SQL Sever 2016/2017

Upgrade to
SQL vNext Run Query Move to 130 Monitor perf.
Keep 110/120 Store Compat Level and fix
CompatLevel (establish perf. and unfreeze regressions with
Freeze plans baseline) plans plan forcing
(optional)
Troubleshooting with Query Store

Enable Set
Let Query Search for
Query FORCE
Store “Problem”
Store PLAN
collect queries
(ALTER policies
the data
DB)
Monitoring Performance with Query Store
• The Query Store
feature provides
DBAs with insight
on query plan
choice and
performance
Working with Query Store
• DB-level feature exposed
/* (6)
(1) Performance
Turn ON Query
analysis
Store */
using Query Store views*/
SELECT q.query_id, qt.query_text_id, qt.query_sql_text,

through T-SQL extensions

ALTER DATABASE MyDB SET QUERY_STORE
SUM(rs.count_executions) AS total_execution_count
= ON;
FROM
/* (2) Review current Query qt
sys.query_store_query_text Store
JOINparameters */
SELECT * FROM sys.database_query_store_options
sys.query_store_query q ON qt.query_text_id =

• ALTER DATABASE
q.query_text_id JOIN
/* (3) Set new parameter
sys.query_store_plan p ONvalues
q.query_id
*/ = p.query_id JOIN
ALTER DATABASE MyDB
sys.query_store_runtime_stats rs ON p.plan_id = rs.plan_id
SET QUERY_STORE
GROUP BY q.query_id,
( qt.query_text_id, qt.query_sql_text

• Catalog views (settings,

ORDER
OPERATION_MODE
BY total_execution_count
= READ_WRITE, DESC
CLEANUP_POLICY = (

compile & runtime stats)

/* (7)
STALE_QUERY_THRESHOLD_DAYS
Force plan for a given query
= 30 */
exec
), sp_query_store_force_plan
12DATA_FLUSH_INTERVAL_SECONDS
/*@query_id*/, 14 /*@plan_id*/
= 3000,
MAX_SIZE_MB = 500,

• Stored Procs (plan forcing,

INTERVAL_LENGTH_MINUTES = 15
);

query/plan/stats cleanup)
/* (4) Clear all Query Store data */
ALTER DATABASE MyDB SET QUERY_STORE CLEAR;

/* (5) Turn OFF Query Store */

ALTER DATABASE MyDB SET QUERY_STORE = OFF;
Troubleshooting Query Store
• Plan forcing does not always work
• Example: If you drop an index, you can’t force a plan that
uses it.
• Query Store will revert to not forcing if it fails
• This keeps the application working if the hint breaks
• You can see which plans are failing to force by looking at the
Plan Table:
SELECT * FROM sys.query_store_plan
WHERE is_forced_plan = 1 AND
force_failure_count > 0
Demonstration:
Using Query Store in SQL Server
2016 / 2017
Queries with Forced Plans in SQL Server 2017

Waits Reports
Coming Soon!
Demonstration:
Query Store in SQL Server 2017

• Query Store Waits

• Queries with Forced Plans
• Queries with High Variations
 SQL Server 2017 – Modern and Intelligent

Query Store – Wait Stats and “Cloud Learnings”

Automatic Tuning and Plan Correction

Query Plan Analysis in SSMS

Adaptive Query Processor

SQL Server 2017 – Query Store Improvements

ALTER DATABASE AdventureWorks2017

SET AUTOMATIC_TUNING ( FORCE_LAST_GOOD_PLAN = ON );

• sys.query_store_wait_stats
• sys.dm_db_tuning_recommendations
• sys.database_automatic_tuning_mode
• sys.database_automatic_tuning_options
SQL Server 2017 Automatic Tuning
ALTER DATABASE CURRENT
SET AUTOMATIC_TUNING (FORCE_LAST_GOOD_PLAN = ON);

Detect with dm_db_tuning_

recommendations and force
manually

Turn on Auto and system corrects

Reverts back to
“Last Known Good”

Perfect to help with parameter sniffing

Demonstration:
Automatic Tuning in SQL Server 2017

• Query Store Waits

• Queries with Forced Plans
• Queries with High Variations
Risks of Misestimation

Slow Query Excessive Resource T-SQL Refactoring

Reduced Throughput
Response Time Due Utilization for Off-Model
and Concurrency
to Bad Plans (CPU, Memory, IO) Statements
Cardinality Estimation and Plan Quality
SELECT [fo].[Order Key], [fo].[Description], [fo].[Package], [fo].[Quantity],
[foo].[OutlierEventQuantity]
FROM [Fact].[OrderHistory] AS [fo]
INNER JOIN [Fact].[WhatIfOutEventQuantity]('Mild Recession', '1-01-2013', '10-15-2014’)
AS [foo] ON [fo].[Order Key] = [foo].[Order Key]
AND [fo].[City Key] = [foo].[City Key]
AND [fo].[Customer Key] = [foo].[Customer Key]
AND [fo].[Stock Item Key] = [foo].[Stock Item Key]
AND [fo].[Order Date Key] = [foo].[Order Date Key]
AND [fo].[Picked Date Key] = [foo].[Picked Date Key]
AND [fo].[Salesperson Key] = [foo].[Salesperson Key]
AND [fo].[Picker Key] = [foo].[Picker Key]
INNER JOIN [Dimension].[Stock Item] AS [si]
ON [fo].[Stock Item Key] = [si].[Stock Item Key]
WHERE [si].[Lead Time Days] > 0
AND [fo].[Quantity] > 50;
Adaptive Query Processing (SQL 2017)
Interleaved Execution Batch-mode Memory Batch-mode
Grant Feedback Adaptive Joins
• Materialize
estimates for • Adjust memory grants • Defer the choice of hash
multi-statement based on execution join or nested loop until
table valued functions feedback after the first join input
(MSTVFs) has been scanned
• Remove spills and
• Downstream operations improve concurrency for • Uses nested loop for small
will benefit from the repeating queries inputs, hash joins for large
corrected MSTVF inputs
cardinality estimate
Demonstration:
Adaptive Query Processing

• Interleaved Execution
• Batch-Mode Memory Grant Feedback
• Batch-Mode Adaptive Join