Microsoft SQL 2016 (SP1)

&
SQL in Azure Inspire
Agenda
• Nyheder i SQL Server 2016 og SP1
• Performance og scalability
• Security
• High Availability
• Troubleshooting og support
• SQL in Azure
• Stretch Databases
• Backup til Azure
• SQL i Azure
• Spørgsmål og evaluering
Praktisk info
• Timing
• Vi forventer af slutte ca. kl 15:00
• Frokost er kl. 12:00 til 13:00

• Slides, mv på: www.it-craft.dk/events

• Klik på Event ressourcer
it-Craft A/S
• er SQL håndværkere af både navn og gavn
• arbejder KUN med Microsoft SQL – og gør det 24/7 365
• har hovedkontor i Jylland men har kunder i hele landet
• har kunder i alle segmenter
• eksisteret siden 2002
it-Craft A/S
Brian Aborg Weje
• SQL Specialist
• MCT
• MCSE: Data Platform
• 14 år med SQL som speciale, fra SQL Server 2000 og frem
Microsoft vision for a new era
United platform for the modern service provider

CUSTOMER
Enterprise-grade
DATACENTER
Global reach, scale, and security to meet
business demands

Hybrid cloud
CONSISTENT
Consistent platform across multiple
PLATFORM environments and clouds

MICROSOFT SERVICE People-focused

AZURE PROVIDER
Expands technical skill sets to the cloud for
new innovation
 The evolution of Microsoft SQL Server

SQL Server
XML ● KPIs 2000

SQL Server
Management Studio ● Mirroring 2005

Compression ● Policy-Based Mgmt ● Programmability SQL Server

2008

PowerPivot ● SharePoint Integration ● Master Data Services SQL Server

2008 R2

SQL Server
AlwaysOn ● ColumnStore Index ● Data Quality Services ● Power View ● Cloud Connectivity
2012

In-Memory Across Workloads ● Performance & Scale ● Hybrid Cloud Optimized ● HDInsight ● Cloud BI SQL Server
2014

Enhanced Always On ● In-Memory OLTP ● Stretch DB ● Temporal Tables ● Enhanced Backup to Cloud ● Polybase ● Real- SQL Server
Time Operational Analytics ● Row-Level Security ● Query Store ● R Services● Always Encrypted ● Mobile BI 2016
Do more. Achieve more.
 Mission-critical performance
Performance Security Availability Scalability
In-Memory OLTP Always Encrypted Basic Availability Groups Windows Server Support
enhancements Sensitive data remains encrypted With SQL Server 2016 Standard Support of new features in
at all times, with ability to query Edition Windows Server 2016
Greater T-SQL surface area,
terabytes of memory supported, Dynamic Data Masking Enhanced Always On Live migration
and higher number of parallel Real-time obfuscation of data to Distributed Availability Groups, Faster live migration, live
CPUs prevent unauthorized access automatic replica seeding, migration for non-clustered VMs
distributed transactions, automatic
Operational Analytics Row-Level Security failover, load balancing, Scalability enhancements
Insights on operational data; Fine-grained access control for manageability Hardware acceleration for TDE,
works with In-Memory OLTP and table rows parallelized decryption, TempDB
disk-based OLTP Backup enhancements optimization, and more
Other enhancements Managed backup to Azure,
Query Store Audit success/failure of database
Monitored, optimized query plans Database Recovery Advisor
operations
Temporal Tables TDE support for storage of
Query data as points in time and In-Memory OLTP tables
recover from accidental data Enhanced auditing for OLTP with
changes and application errors ability to track history of record
changes
 Mission-critical performance
Performance Security Availability Scalability
In-Memory OLTP Always Encrypted Basic Availability Groups Windows Server support
enhancements Sensitive data remains encrypted With SQL 2016 Standard Edition Support for Windows Server Core
at all times, with ability to query and Windows Server ReFS
Greater T-SQL surface area, Enhanced AlwaysOn
terabytes of memory supported, Dynamic Data Masking Distributed availability groups, Live migration
and higher number of parallel Real-time obfuscation of data to automatic replica seeding, Faster live migration, live
CPUs prevent unauthorized access distributed transactions, automatic migration for non-clustered VMs
failover, load balancing,
Operational Analytics Row-Level Security manageability Scalability enhancements
Insights on operational data; Fine-grained access control for Hardware acceleration for TDE,
works with In-Memory OLTP and table rows Backup enhancements parallelized decryption, TempDB
disk-based OLTP Managed backup to Azure, optimization, and more
Other enhancements Database Recovery Advisor
Query Store Audit success/failure of database
Monitored, optimized query plans operations
Temporal Tables TDE support for storage of
Query data as points in time and In-Memory OLTP tables
recover from accidental data Enhanced auditing for OLTP with
changes and application errors ability to track history of record
changes
In-Memory OLTP
enhancements
Performance and Scaling Improvements

No limit on the size of memory-optimized tables though the tables

do need to fit in memory
Multiple threads to persist memory-optimized tables
Parallel Support
• Parallel scan for memory-optimized tables and HASH indexes

• Parallel plan support for accessing memory-optimized tables

Query Surface Area in Native Modules
Disjunction (OR, NOT) LOB types for parameters and variables.
UNION and UNION ALL Natively compiled inline table-valued
functions (TVFs)
SELECT DISTINCT
EXECUTE AS CALLER support
OUTER JOIN
Built-in security functions
Subqueries in SELECT statements
(EXISTS, IN, scalar subqueries) Increased support for built-in math
functions
Nested execution (EXECUTE) of
natively compiled modules
Transact-SQL

Support with memory-optimized tables for:

NULLable index key columns.

LOB types [varchar(max), nvarchar(max), and varbinary(max)]

UNIQUE indexes in memory-optimized tables.

FOREIGN KEY constraints between memory-optimized tables.

CHECK and UNIQUE constraints

Triggers (AFTER) for INSERT/UPDATE/DELETE operations.

 ALTER support
ALTER TABLE Sales.SalesOrderDetail
ALTER INDEX PK_SalesOrderID Full schema change support: add/alter/drop
REBUILD column/constraint
WITH (BUCKET_COUNT=100000000) Add/drop index supported

The ALTER TABLE syntax is used for making changes to the table schema, as well as for
adding, deleting, and rebuilding indexes

Indexes are considered part of the table definition

Key advantage is the ability to change the BUCKET_COUNT with an ALTER INDEX
statement

Performance
Altering natively compiled stored procedures
CREATE PROCEDURE [dbo].[usp_1]
WITH NATIVE_COMPILATION, SCHEMABINDING, EXECUTE AS OWNER
You can now perform ALTER
AS BEGIN ATOMIC WITH operations on natively compiled
stored procedures using the
(
TRANSACTION ISOLATION LEVEL = SNAPSHOT, LANGUAGE =
N'us_english'
)
ALTER PROCEDURE statement
SELECT c1, c2 from dbo.T1
END Use sp_recompile to
GO
recompile stored procedures on
ALTER PROCEDURE [dbo].[usp_1]
WITH NATIVE_COMPILATION, SCHEMABINDING, EXECUTE AS OWNER the next execution
AS BEGIN ATOMIC WITH
(
TRANSACTION ISOLATION LEVEL = SNAPSHOT, LANGUAGE =
N'us_english'
)
SELECT c1 from dbo.T1
END
GO

Performance
Transact-SQL

Full support for all Collation and Unicode Support

(var)char columns can use any code page supported by SQL Server

Character columns in index keys can use any SQL Server collation

Expressions in natively compiled modules as well as constraints on memory-

optimized tables can use any SQL Server collation

Scalar User-Defined Functions for In-Memory OLTP

Create, drop, and alter natively compiled, scalar user-defined functions

Native compilation improves performance of the evaluation of UDFs in T-SQL

Performance
Cross-Feature Support

System-Versioned Temporal Tables

Query Store
Row-Level Security (RLS)
Multiple Active Result Sets (MARS)
Transparent Data Encryption (TDE)

Performance
Support for Transparent Data Encryption (TDE)
Windows Operating System
Level Data Protection In SQL Server 2016, the
DPAPI encrypts the Service Master Key
storage for memory-
optimized tables will be
SQL Server
Created at a time of SQL Server setup
Instance Level Service Master Key

Service Master Key Encrypts the Database

master Key for the master Database

Master Statement:
encrypted as part of enabling
TDE on the database
Database Level Database Encryption Key CREAT MASTER KEY…

Database Master Key of the master Database

creates a certificate in the master database

Simply follow the same steps

Statement:
CREATE CERTIFICATE…

as you would for a disk-based

The certificate encrypts the database
Encryption Key in the user database

User Database Statement:

Level Database Encryption Key

The entire user database is secured by the

CREATE DATABASE ENCRYPTION KEY…
database
Datbase Encryption Key (DEK) of the user
database by using transparent database
encryption

Statement:
ALTER DATABSE… SET ENCRYPTION

Performance
Improvements in Management Studio

Lightweight performance analysis

Transaction Performance Analysis report pinpoints hotspots in the application

Generating migration checklists

Migration checklists show unsupported features used in current disk-based tables and interpreted T-SQL
stored procedures

Generated checklists for all or some tables and procedures

Use GUI or PowerShell

Performance
Improved scaling

Other enhancements include: In-Memory OLTP

engine has been
enhanced to scale
linearly on servers up to
4 sockets

7x

Performance
New Transaction Performance Analysis Overview report

New report replaces the

need to use the
Management Data
Warehouse to analyze
which tables and stored
procedures are candidates
for in-memory
optimization

Performance
Summary: In-Memory OLTP enhancements

Capability
ALTER support for memory-optimized tables

Greater Transact-SQL coverage

Benefits
Improved scaling: In-Memory OLTP engine has been enhanced to scale linearly on servers up to 4 sockets

Tooling improvements in Management Studio

MARS (multiple active result sets) support

TDE (Transparent Data Encryption)-enabled: all on-disk data files are now encrypted once TDE is enabled

Performance
Operational Analytics:
disk-based and in-
memory tables
Traditional operational/analytics architecture

BI analysts Key issues

Complex implementation

Requires two servers (capital

expenditures and operational
IIS Server expenditures)

Data latency in analytics

High demand;
requires real-time analytics

Performance
Minimizing data latency for analytics

BI analysts
Challenges
Analytics queries are resource intensive and can
cause blocking

Minimizing impact on operational workloads

IIS Server
Sub-optimal execution of analytics on relational
schema

Benefits
No data latency

No ETL

No separate data warehouse

Performance
 Operational Analytics
The ability to run analytics queries concurrently with operational
workloads using the same schema
Goals:
• Minimal impact on operational workloads with concurrent analytics
• Performance analytics for operational schema

Not a replacement for:

• Extreme analytics performance queries possible only using customized schemas (e.g.
Star/Snowflake) and pre-aggregated cubes
• Data coming from non-relational sources
• Data coming from multiple relational sources requiring integrated analytics

Performance
Operational Analytics with columnstore index

Key points
B-tree index
Create an updateable NCCI for analytics
queries

Drop all other indexes that were created

for analytics

Delta row groups

No application changes

Columnstore index is maintained just like

any other index

Nonclustered columnstore index (NCCI) Query optimizer will choose columnstore

index where needed

Performance
Using Availability Groups instead of data warehouses
Always On Availability Group Key points
Mission-critical operational workloads
typically configured for high availability
using Always On Availability Groups
Secondary
replica You can offload analytics to readable
secondary replica

Primary Secondary
replica replica

Secondary
replica

Performance
Operational
Analytics:
In-Memory Tables
Operational Analytics: columnstore on In-Memory Tables
Hash index
No explicit delta row group
Rows (tail) not in columnstore stay in In-Memory OLTP table

nonclustered index
No columnstore index overhead when operating on tail

Background task migrates rows from tail to columnstore in

Deleted Rows Table tail chunks of 1 million rows

In-Memory OLTP table

Deleted Rows Table (DRT) – Tracks deleted rows

Columnstore data fully resident in memory

Persisted together with operational data

No application changes required

Columnstore Index

Performance
Summary of improvements
Improvements SQL Server 2014 SQL Server 2016
clustered Master copy of the data (10x compression) Master copy of the data (10x compression)
Only index supported; simplified analytics
columnstore Additional B-tree indexes for efficient equality, short-range searches, and PK/FK
No PK/FK constraints
index Uniqueness can be enforced through materialized views
constraints
Locking granularity for UPDATE/DELETE at row group level Locking granularity at row level using NCI index path
DDL: ALTER, REBUILD, REORGANIZE DDL: ALTER, REBUILD, REORGANIZE

updateable non- Introduced in SQL Server 2012 Updateable

NCCI is read-only: no delete bitmap or delta store
clustered index Ability to mix OLTP and analytics workload
Optimizer will choose between NCCI and NCI(s)/CI or
heap-based on the cost-based model Ability to create filtered NCCI
Partitioning supported Partitioning supported

equality and Row group elimination (when possible) Optimizer can choose NCI on column C1; index points directly to row group
short-range Partition-level scan (somewhat expensive) No full index scan
queries Full index scan (expensive) Covering NCI index

string predicate Retrieve 10 million rows by converting dictionary encoded Apply filter on dictionary entries
value to string Find rows that refer to dictionary entries that qualify (R1)
pushdown
Find rows not eligible for this optimization (R2)
Apply string predicate on 10 million rows Scan returns (R1 + R2) rows
Filter node applies string predicate on (R2)
Row returned by Filter node = (R1 + R2’)

Performance
Support for index maintenance
Operation SQL Server 2014 SQL Server 2016
Removing deleted rows Requires index REBUILD Index REORGANIZE
Remove deleted rows from single compressed RG
Merge one or more compressed RGs with deleted rows
Done ONLINE

Smaller RG size resulting from: Index REBUILD Index REORGANIZE

Smaller BATCHSIZE
Memory pressure
Index build residual

Ordering rows Create clustered index No changes

Create columnstore index by dropping clustered index

Query Row group granularity Support of SI and RCSI (non-blocking)

No support for RCSI or SI
Recommendation: use read uncommitted

Insert Lock at row level (trickle insert) No changes

Row group level for set of rows

Delete Lock at row group level Row-level lock in conjunction with NCI

Update Lock at row group level Row-level lock in conjunction with NCI
Implemented as Delete/Insert

AlwaysOn Failover Clustering (FCI) Fully supported Fully supported

AlwaysON Availability Groups Fully supported except readable secondary Fully supported with readable secondary

Index create/rebuild Offline Offline

Performance
Query Store
Your flight data recorder
for your database
Problems with query performance
Website
Database
is not
Fixing query plan choice regressions is difficult
Is down
working • Query plan cache is not well-suited for performance troubleshooting

Temporary Impossible
Long time to detect the issue (TTD)
perf issues to predict / • Which query is slow? Why is it slow?
root cause
• What was the previous plan?

DB Regression Long time to mitigate (TTM)

upgraded caused by
new bits
• Can I modify the query?
• How to use plan guide?

Performance
The solution: Query Store

Dedicated store for query workload performance data

Captures the history of plans for each query
Captures the performance of each plan over time
Persists the data to disk (works across restarts, upgrades, and recompiles)

Significantly reduces TTD/TTM

Find regressions and other issues in seconds
Allows you to force previous plans from history

DBA is now in control

Performance
Query Store Architecture
Collects query texts (plus all relevant properties)

Stores all plan choices and performance metrics

Compile Plan store
Works across restarts / upgrades / recompiles
Query
Store Dramatically lowers the bar for performance
schema troubleshooting
Runtime
Execute
stats
New Views

Intuitive and easy plan forcing

Durability latency controlled by DB option
DATA_FLUSH_INTERNAL_SECONDS

Performance
Monitoring performance by using the Query Store

The Query Store

feature provides DBAs
with insight on query
plan choice and
performance

Performance
Temporal Tables

Query back in time

Why temporal
Data changes over time
Tracking and analyzing changes is often important

Temporal in DB
Time travel Data audit Automatically tracks history of data changes

Enables easy querying of historical data states

Advantages over workarounds

Simplifies app development and maintenance
Slowly changing Repair record-level
dimensions corruptions Efficiently handles complex logic in DB engine

Performance
How to start with temporal
ANSI 2011 No change in programming model New Insights
compliant

CREATE temporal SELECT * FROM temporal Temporal

TABLE PERIOD FOR DML
Querying
SYSTEM_TIME…
INSERT / BULK INSERT
ALTER regular_table FOR SYSTEM_TIME
TABLE ADD
UPDATE AS OF
PERIOD…
FROM..TO
DELETE BETWEEN..AND
CONTAINED IN
DDL MERGE
Querying

Performance
Temporal database support: BETWEEN

SELECT * FROM Provides correct information

Person.BusinessEntityContact about stored facts at any
FOR SYSTEM_TIME BETWEEN @Start AND @End
WHERE ContactTypeID = 17 point in time, or between
two points in time
There are two orthogonal sets of scenarios with
regards to temporal data:

System (transaction)-time

Application-time

Performance
How does system-time work?

Temporal table (actual data) History table

* Include historical
version

Regular queries Temporal queries *

(current data) (Time travel, etc.)

Performance
Application-time temporal
CREATE
SELECT
ALTER
UPDATE
(
TABLE
FROM
TABLE Employee
* Employee
Employee
Employee
Limits of system-time
FOR[EmployeeNumber] Time flows ‘forward only’
WHERE
ADD VALID_TIME
CONSTRAINT
PORTION int NOT
CONTAINS NULL,
'2013-06-30'
OF FK_Employee_Department
VALID_TIME
[Name]
FOREIGN nvarchar(100)
FROM '2010-01-01'
KEY (LocationId, NOT NULL,VALID_TIME)
TO '2012-01-01'
PERIOD
[LocationId]
REFERENCES int NOT
Location NULL,
(LocationId, PERIOD VALID_TIME); System-time ≠ business-time
[Position]
SELECT * FROM varchar(50)
Employee NOT NULL,
(sometimes)
SET [Position] = 'CEO'
WHERE[AnnualSalary]
WHEREEmployeeNumberdecimal
EmployeeNumber (10,2) NOT NULL,
= =1 1AND

ValidFrom
VALID_TIME datetime2
OVERLAPS NOT ('2013-06-30',
PERIOD NULL, '2014-01-01') Immutable history, future does not
ValidTo datetime2
DELETE FROM Employee NOT NULL, exist
PERIOD FOR VALID_TIME (ValidFrom,ValidTo),
/*FOR
Temporal
PORTIONjoin */
OF VALID_TIME
SELECT * FROM Employee
FROM '2012-01-01' E
TO '2013-01-01'
CONSTRAINT PK_Employee App-time = new scenarios
JOIN PRIMARY
PositionKEY
D CLUSTERED
ON E.Position = D.Position AND
WHERE EmployeeNumber
(EmployeeNumber,
= 1
VALID_TIME WITHOUT OVERLAPS)
D.VALID_TIME CONTAINS PERIOD E.VALID_TIME Correct past records as new info is
available (HR, CRM, insurance,
)
banking)
Consistency
Project future events (budgeting,
what-if, loan repayment schedule)
Easy time
Temporal
Batch DW loading (with delay)
travel
querying edits

Performance
Temporal data continuum

SQL Database

Performance
 In-Memory OLTP and temporal

Extreme OLTP with

cost-effective data history
Disk-based history table
Super-fast DML and current data querying Fast DML
Temporal querying in interop mode

Internal
data
retention

Performance
Summary: Temporal Tables

Quickly add historical versioning with minimal developer effort

Add temporal data to existing tables without downstream impact

Support for temporal queries, auditing, and change tracking

 Mission-critical performance
Performance Security Availability Scalability
In-Memory OLTP Always Encrypted Basic Availability Groups Windows Server support
enhancements Sensitive data remains encrypted With SQL 2016 Standard Edition Support for Windows Server Core
at all times, with ability to query and Windows Server ReFS
Greater T-SQL surface area, Enhanced AlwaysOn
terabytes of memory supported, Dynamic Data Masking Distributed availability groups, Live migration
and higher number of parallel Real-time obfuscation of data to automatic replica seeding, Faster live migration, live
CPUs prevent unauthorized access distributed transactions, automatic migration for non-clustered VMs
failover, load balancing,
Operational analytics Row-Level Security manageability Scalability enhancements
Insights on operational data; Fine-grained access control for Hardware acceleration for TDE,
works with In-Memory OLTP and table rows Backup enhancements parallelized decryption, TempDB
disk-based OLTP Managed backup to Azure, optimization, and more
Other enhancements Database Recovery Advisor
Query Store Audit success/failure of database
Monitored, optimized query plans operations
Temporal Tables TDE support for storage of
Query data as points in time In-Memory OLTP tables
Enhanced auditing for OLTP with
ability to track history of record
changes
Always Encrypted
The need for Always Encrypted

Data disclosure Queries on Application

prevention encrypted data transparency
Client-side encryption of Support for equality Minimal application changes
sensitive data using keys that comparison, including join, via server and client library
are never given to the group by, and distinct enhancements
database system operators

Allows customers to securely store sensitive data outside of their trust boundary.
Data remains protected from high-privileged, yet unauthorized, users.

Security
How it works
Help protect data at rest and in motion, on-premises and in the cloud
Encrypted sensitive data and corresponding keys
are never seen in plaintext in SQL Server
Client SQL Server or SQL Database

"SELECT Name FROM Customers "SELECT Name FROM Customers

WHERE SSN = @SSN","111-22-3333" WHERE SSN = @SSN",0x7ff654ae6d

ciphertext

Result Set Result Set

ADO .NET
Name Name
Wayne Jefferson 0x19ca706fbd9a

dbo.Customers
Name SSN Country
trust boundary 0x19ca706fbd9a 0x7ff654ae6d USA

ciphertext
Security
Types of encryption for Always Encrypted
Randomized encryption
Encrypt('123-45-6789') = 0x17cfd50a
Two types of encryption
Repeat: Encrypt('123-45-6789') = 0x9b1fcf32
Allows for transparent retrieval of encrypted
available
data but NO operations Randomized encryption uses a method
More secure that encrypts data in a less predictable
manner
Deterministic encryption
Encrypt('123-45-6789') = 0x85a55d3f Deterministic encryption uses a method
Repeat: Encrypt('123-45-6789') = 0x85a55d3f that always generates the same encrypted
Allows for transparent retrieval of encrypted value for any given plaintext value
data AND equality comparison
E.g. in WHERE clauses and joins, distinct,
group by

Security
Example
Client (trusted) CMK store SQL Server (untrusted)
using (SqlCommand cmd = new SqlCommand( exec sp_describe_parameter_encryption
"SELECT Name FROM Customers WHERE SSN = @params = N'@SSN VARCHAR(11)'
, @tsql = N'SELECT * FROM Customers WHERE SSN = @SSN'

Encryption metadata
@SSN“
, conn))
{ Encryption CMK Store
cmd.Parameters.Add(new SqlParameter( Plaintext Type/ Encrypted Provider
Param Algorithm CEK Value Name CMK Path
"@SSN", SqlDbType.VarChar, 11).Value =
"111-22-3333");
CEK
SqlDataReader reader = Cache @SSN DET/ AES CERTIFICATE_ Current User/
cmd.ExecuteReader(); 256 STORE My/f2260…

EXEC sp_execute_sql
N'SELECT * FROM Customers WHERE SSN = @SSN'
, @params = N'@SSN VARCHAR(11)', @SSN=0x7ff654ae6d

Encryption metadata
Result set (plaintext) Encryption CMK Store
Name
Enhanced Type/ Encrypted Provider
Jim Gray
ADO.NET Param Algorithm CEK Value Name CMK Path

@Name Non-DET/ CERTIFICATE_ Current User/

AES 256 STORE My/f2260…
Name
Result set (ciphertext)
0x19ca706fbd9

Security
Summary: Always Encrypted
Protect data at rest and in motion, on-premises and in the cloud

Data remains encrypted Capability

during query ADO.NET client library provides
transparent client-side encryption,
while SQL Server executes T-SQL
Apps TCE-enabled SQL Server
Encrypted queries on encrypted data
query ADO .NET library
No app
changes
Benefits
Master Columnar
key key

Security
Dynamic Data Masking
SQL Server 2016
SQL Database
Dynamic Data Masking Table.CreditCardNo

4465-6571-7868-5796

Prevent abuse of sensitive data 4468-7746-3848-1978

by hiding it from users 4484-5434-6858-6550

Configuration made easy in new Azure portal Real-time data masking,

SQL Database partial masking
Policy-driven at table and column level, for SQL Server 2016
defined set of users

Data masking applied in real time to query

results based on policy

Multiple masking functions available, such as full

or partial, for various sensitive data categories
(credit card numbers, SSN)

Security
Benefits of Dynamic Data Masking

Regulatory Sensitive data Agility and

compliance protection transparency
Data is masked on the fly, with
underlying data in database
remaining intact (transparent to
application and applied
according to user privilege)

Limit access to sensitive data by defining policies to obfuscate specific database fields, without
affecting database integrity

Security
Dynamic data masking walkthrough
1) Security officer defines dynamic data masking policy in T-SQL over sensitive data in Employee table
2) Application user selects from Employee table
3) Dynamic data masking policy obfuscates the sensitive data in the query results

ALTER TABLE [Employee] ALTER COLUMN [SocialSecurityNumber]

ADD MASKED WITH (FUNCTION = ‘SSN()’)

ALTER TABLE [Employee] ALTER COLUMN [Email]

ADD MASKED WITH (FUNCTION = ‘EMAIL()’)

ALTER TABLE [Employee] ALTER COLUMN [Salary]

ADD MASKED WITH (FUNCTION = ‘RANDOM(1,20000)’)

GRANT UNMASK to admin1

SELECT [Name],
[SocialSecurityNumber],
[Email],
[Salary]
FROM [Employee]

Security
Summary: Dynamic Data Masking

Capability
Protects against unauthorized disclosure of sensitive data in application

Benefits
Enables setup of policies at table and column level that provide multiple masking functions
Allows certain privileged logins to see data unmasked

Security
Row-Level Security
SQL Server 2016
SQL Database
The need for Row-Level Security
Protect data privacy by ensuring
appropriate access across rows
Fine-grained access control over specific rows in Customer 1
database table Customer 2

Blocking of unauthorized access when multiple Customer 3

users share tables, or connection filtering in multi- SQL Database

tenant applications
Administration via SQL Server Management Studio
or SQL Server Data Tools
Enforcement logic inside database, with schema
bound to table

Security
Benefits of Row-Level Security

Fine-grained Application Centralized

access control transparency security logic
Keeping multi-tenant RLS works transparently at Enforcement logic inside
databases secure by limiting query time, without requiring database that is schema-
access by other users who app changes bound to protect table
share same tables
Compatible with RLS in other Reduced application
leading products maintenance and complexity

Store data intended for many consumers in a single database/table while also restricting
row-level read-and-write access based on user execution context

Security
Common RLS use cases
Traditional RLS workloads
Custom business logic to determine which rows each user can SELECT, INSERT, UPDATE, and
DELETE based on role, department, and security level

Target sectors: Finance, insurance, health care, energy, and government

Multi-tenant databases
Ensuring tenants can access only their own rows of data in a shared database, with
enforcement logic in database rather than app tier

For example: Multi-tenant shards with elastic database tools in SQL Database

Reporting, analytics, and data warehousing

Different users access same database through various reporting tools, and work with
different subsets of data based on their identity/role

Security
RLS in three steps
Two
Three
One

Security
App
Policyuser
Policy
(e.g., transparently
manager nurse)
createsselects rewrites
from
a filter Patients
query
predicate andtable
tosecurity
apply filter predicate
policy
Nurse
Database Policy Manager

Security Filter
Predicate:

Policy INNER
JOIN…

Patients CREATE FUNCTION dbo.fn_securitypredicate(@wing int)

RETURNS TABLE WITH SCHEMABINDING AS

Application return SELECT 1 as [fn_securitypredicate_result] FROM

SELECT *StaffDuties
FROM Patients
d INNER JOIN Employees e
SEMIJOIN APPLY dbo.fn_securitypredicate(patients.Wing);
ON (d.EmpId = e.EmpId)
WHERE e.UserSID = SUSER_SID() AND @wing = d.Wing;
SELECT * FROM Patients
SELECT
CREATE Patients.* FROM Patients,
SECURITY POLICY dbo.SecPol
StaffDuties d INNER JOIN Employees e ON (d.EmpId = e.EmpId)
ADD FILTER PREDICATE dbo.fn_securitypredicate(Wing) ON Patients
WHERE e.UserSID =
WITH (STATE = ON)SUSER_SID() AND Patients.wing = d.Wing;

Security
Summary: Row-Level Security

Capability
Row-Level Security provides fine-grained access control
over rows in a table based on conditions you set up

Benefits
Store data for many users in same databases and tables
while limiting access by other users who share same tables

Security
Security enhancements

Built-in tools for enabling compliance: SQL Server audit tools

Create server audits, with server audit specifications (audited events
can be written to event logs or to audit files)
User-defined audit: Allows middle-tier application to write custom events into audit log, which
enables more flexibility to store audit information

Audit filtering: Provides greater flexibility to filter wanted events in audit log

Audit resilience: Audit logging is now tolerant to loss of connectivity to target directory and will
recover automatically once network connection is re-established

Security
Transparent Data Encryption (TDE)

Encrypt data when it is stored on disk, and decrypt

it when read into memory
Developers can encrypt database files, log files, and backup files without
changing existing applications

Intel AES-NI hardware encryption acceleration

Support for storage of memory-optimized OLTP tables (new)

Exclusive to SQL Server Enterprise edition

Security
Transparent Data Encryption (TDE)

Encrypt backup by specifying encryption algorithm and encryptor

Supports on-premises and Azure storage locations
Configurable for Managed Backup to Windows Azure

Backup encryption now supported with compression, using AES-

NI hardware acceleration

Security
Summary: Security

Always encrypted: Sensitive data always encrypted (and queryable)

Dynamic Data Masking: Real-time obfuscation of data
Row-Level Security: Fine-grained access control of table rows
Audit success/failure of database operations
TDE support for storage of In-Memory OLTP tables
Enhanced auditing for OLTP with ability to track history of record
changes
Security
 Mission-critical performance
Performance Security Availability Scalability
In-Memory OLTP Always Encrypted Basic Availability Groups Windows Server support
enhancements Sensitive data remains encrypted With SQL 2016 Standard Edition Support for Windows Server Core
at all times, with ability to query and Windows Server ReFS
Greater T-SQL surface area, Enhanced Always On
terabytes of memory supported, Dynamic Data Masking Distributed Availability Groups, Live migration
and higher number of parallel Real-time obfuscation of data to automatic replica seeding, Faster live migration, live
CPUs prevent unauthorized access distributed transactions, automatic migration for non-clustered VMs
failover, load balancing,
Operational analytics Row-Level Security manageability Scalability enhancements
Insights on operational data; Fine-grained access control for Hardware acceleration for TDE,
works with In-Memory OLTP and table rows Backup enhancements parallelized decryption, TempDB
disk-based OLTP Managed backup to Azure, optimization, and more
Other enhancements Database Recovery Advisor
Query Data Store Audit success/failure of database
Monitored, optimized query plans operations
Temporal database TDE support for storage of
In-Memory OLTP tables
support
Query data as points in time Enhanced auditing for OLTP with
ability to track history of record
changes
Mission-critical availability

▪ Detects failures reliably

▪ Handles multiple failures at once
▪ Provides unified, simplified solution
▪ Streamlines deployment,
management, and monitoring

▪ Reuses existing investments

▪ Offers SAN/DAS environments
▪ Allows use of HA hardware resources
▪ Supports Fast, transparent failover

Availability
AlwaysOn
Failover Cluster Instances Availability Groups
for servers for groups of databases

▪ Failover on SQL Server instance level ▪ Failover on database level

▪ Shared storage (SAN/SMB) ▪ Direct attached storage
▪ Failover can take minutes based on load ▪ Failover takes seconds
▪ Multi-node clustering ▪ Multiple secondaries
▪ Passive secondary nodes ▪ Active secondaries

Availability
Failover Cluster Instances

WSFC node WSFC node Server failover

SQL SQL Server 2016 SQL Server 2016
Shared storage
Server
Failover
Cluster Multi-node clustering
Instance
Passive secondary nodes
Shared storage
Failover in minutes

Availability
Availability Groups
Availability
Group Listener
Multi-database failover
SQL Server 2016 SQL Server 2016

Direct attached storage

Multiple secondaries

SQL Server 2016 Active secondaries

Failover in seconds

Availability
Availability Groups + Failover Clustering
Windows Server Failover Clustering (WSFC) Cluster

Node
Network Subnet
Node Node
Network Subnet
Node Node
AlwaysOn:
WSFC WSFC WSFC WSFC WSFC

Failover Cluster Instances

Configuration Configuration Configuration Configuration Configuration

and Availability Groups

SQL Server SQL Server SQL Server AlwaysOn SQL Server
Instance Instance Instance Failover Cluster Instance

work together to ensure

Instance Instance Instance Instance
Network Name Network Name Network Name Network Name

AlwaysOn Availability Group data is accessible despite

failures
Primary Replica Secondary Replica Secondary Replica Secondary Replica

Availability Group Listener Virtual Network Name

Storage Storage Storage Share Storage

Availability
Basic Availability
Groups
Basic Availability Groups

Available in SQL Server 2016 Standard Edition or higher

Provides failover support for single database
Replaces database mirroring feature (now deprecated)
Single replica for primary database, using either synchronous or
asynchronous commit mode
Support for hybrid environments, spanning on-premises or Azure
https://msdn.microsoft.com/en-us/library/mt614935.aspx

Availability
Enhancements in
Always On
Availability Groups
Improvements in Always On Availability Groups
Greater scalability
Unified HA solution
Load-balancing readable secondaries
Increased number of automatic failover targets
Log transport performance
AG_Listener

AG Improved manageability
Hong Kong DTC support with limitations
Asynchronous data (Secondary)
movement
Database-level health monitoring
AG Group Managed Service Account
AG
New Jersey Domain-independent Availability Groups
Synchronous data (Secondary)
New York movement
(Primary)

Availability
Load balancing in readable secondaries

In SQL 2014, read-only READ_ONLY_ROUTING_LIST = (( SERVER02 , SERVER03', SERVER04'), SERVER05')

transactions routed by the

Listener went to first available
secondary Load Balanced Failover

Read-only routing lists can be

configured to round-robin
among specific set of secondaries SERVER01 SERVER02 SERVER03 SERVER04 SERVER05

(for each primary)

Availability
Database-level failover trigger

In SQL Server 2014, Availability Groups only monitor

health of the instance
Database can be offline or corrupt, but will not trigger failover as long as instance
itself is healthy

SQL Server 2016: Option to also monitor health of

databases in Availability Group
Databases going offline trigger change in health status

You can now configure AlwaysOn Availability Groups for

failover when database goes offline
This change requires setting DB_FAILOVER option to ON in CREATE AVAILABILITY
GROUP (Transact-SQL) or ALTER AVAILABILITY GROUP (Transact-SQL) statements
Availability
gMSA support
Group Managed Service Accounts (gMSA)
Automatically set domain scope for Managed Service Accounts
Automatic password rotation
Much more secure than regular domain accounts
Enables cross-system security context

Why would I want a gMSA?

No need to manually change passwords on all AlwaysOn instances

How does it work?

Passwords are managed by domain

What versions will it be supported in?

Supported in SQL Server 2014 and SQL Server 2016

Availability
Cross-Database Transactions and Distributed Transactions
Support for cross-database transactions within the same SQL
Server Instance
Cross-database transactions within the same SQL Server instance are not supported for Always
On Availability Groups

Support for distributed transactions

Distributed transactions are supported with Always On Availability Groups between databases
hosted by two different SQL Server instances. It also applies to distributed transactions between
SQL Server and another DTC-compliant server
Availability Groups must be running on Windows Server 2016 or Windows Server 2012 R2. For
Windows Server 2012 R2 you must install the update in KB3090973
Availability Groups must be greated with the CREATE AVAILABILITY GROUP command and the
WITH DTC_SUPPORT = PER_DB clause

Distributed transactions are not supported for database mirroring

https://msdn.microsoft.com/en-us/library/ms366279.aspx
Availability
More than two auto-failover targets

Increasing scale of solution

Increasing resiliency
Now any sync secondary can be target for automatic failover
Total of three (up from two) auto-failover targets

Availability
Domain-independent Availability Groups
Environments supported:
Cross domains (with trust)
Cross domains (no trust)
No domain at all

Cluster management via PowerShell only

SQL management as normal
Use of certificate-secured endpoints like DBM

Availability
Summary: Enhanced Always On

Capability
For scalability, SQL Server 2016 adds in load balancing of readable secondaries
Increases number of auto-failover targets from two to three

Benefits
Log transport performance has been improved
Support for Distributed Transaction Coordinator (DTC): Enrolled transactions for
Availability Group databases with limitations
Database-level health monitoring
gMSA: Domain-level accounts that are automatically managed

Availability
Backup
enhancements
Database Recovery Advisor

SQL Sever Management Studio Database Recovery Advisor

facilitates construction of restore plans that implement
optimal correct restore sequences
Restore-plan algorithm: Improved for complex restore scenarios

Point-in-time restores: Simplified restoration of database to given point in time, and

automatically includes backups relevant to desired restore point

Availability
Backup to Azure block blobs

Backup to Azure (SQL Backup to Azure block blobs

Server 2012) (SQL Server 2016)
Benefits: 2x cheaper storage
Near “bottomless” storage Backup striping and faster restore
Off-site, geo-redundant Maximum backup size is 12 TB+
No device management Granular access and unified credential story
Remote accessibility (SAS URIs)
Limitations: Supports all existing backup/restore features
(except append)
Backup size up to 1 TB
Restore speed

Availability
Managed Backup

In SQL Server 2016, Managed Backup to Microsoft Azure uses new

block blob storage for backup files
Stripe backup sets, enabling backup file sizes up to 12.8 TB
Other changes and enhancements to Managed Backup:
Managed Backup used for system databases

Support for databases in full, bulk logged, and simple recovery model

Support for both automated and custom scheduling of backups

Customized backup schedules – full backup and log backup

Availability
Backups and Stretch Databases

Backup of Stretch DB is a “shallow” backup only

(backup/restore of local SQL Server hot data only)
Stretch DB feature ensures remote data is transactionally
consistent with local data after each restore
Upon completion of local restore, SQL Server reconciles with
remote using metadata
For more information:
https://msdn.microsoft.com/en-us/library/dn934993.aspx
Availability
 Mission-critical performance
Performance Security Availability Scalability
In-Memory OLTP Always Encrypted Basic Availability Groups Windows Server support
enhancements Sensitive data remains encrypted With SQL 2016 Standard Edition Support for Windows Server 2016
at all times, with ability to query Scalability Enhancements
Greater T-SQL surface area, Enhanced AlwaysOn
terabytes of memory supported, Dynamic Data Masking Distributed availability groups, Live migration
and higher number of parallel Real-time obfuscation of data to automatic replica seeding, Faster live migration, live
CPUs prevent unauthorized access distributed transactions, automatic migration for non-clustered VMs
failover, load balancing,
Operational analytics Row-Level Security manageability Scalability enhancements
Insights on operational data; Fine-grained access control for Hardware acceleration for TDE,
works with In-Memory OLTP and table rows Backup enhancements parallelized decryption, TempDB
disk-based OLTP Managed backup to Azure, optimization, and more
Other enhancements Database Recovery Advisor
Query Data Store Audit success/failure of database
Monitored, optimized query plans operations
Temporal database TDE support for storage of
In-Memory OLTP tables
support
Query data as points in time Enhanced auditing for OLTP with
ability to track history of record
changes
Windows Server
support
Windows Server Scalability

Massive scale for in-

Unparalleled scalability memory performance
with Windows Server 2016
Simple, flexible HA
and DR
No domain join needed

Unparalleled security
Fine-grained security controls
12 TB of memory WS 2016 max Built-in anti-malware
cores

Scalability
Support for Windows Server Core

Windows Server edition with smallest footprint

Reduced memory and disk requirements
Fewer running processes and services: greater stability
Simplified management

Requires less maintenance and fewer OS patches,

greatly reduced downtime

50‒60 percent less patching and fewer OS reboots

Scalability
Microsoft Storage Spaces Direct
Hyper-V cluster with local storage

What is Storage Spaces Direct? Why Storage Spaces Direct?

Evolution of Storage Spaces New device types
Servers with local storage Lower-cost flash storage with SATA SSDs
Highly available and scalable Better flash performance with NVMe SSDs

Storage for Hyper-V virtualization and private Simplicity

cloud Ethernet/RDMA network as storage fabric
No need for complex multi-initiator fabric
Seamless capacity and performance expansion

Scalability
Domain-independent Availability Groups

New feature in Windows Server 2016

Environments supported:
• Cross domains (with trust)
• Cross domains (no trust)
• No domain at all

Windows 2016 clusters use certificates for intra-cluster authorization

Uses certificate-secured endpoints like DBM

Scalability
Live migration

Simultaneous migration of multiple SQL Server virtual machines

Maintain availability of SQL Server while decreasing downtime
Migrate many virtual machines (using priority settings) in a clustered environment
Use up to 10 GB of network bandwidth

Live migration for non-clustered virtual machines

Centrally shared and non-shared virtual machine storage scenarios
Reduced cost and complexity of SQL Server deployments in virtualized environments, with
availability during planned downtime

Scalability
Clustering enhancements

Cluster-aware updating
Applies updates automatically to host operating system—or to other system components in
a clustered SQL Server environment—while maintaining availability
Increases SQL Server availability during update process in both virtualized and non-
virtualized environments

Dynamic Quorum
Enables AlwaysOn cluster to dynamically adjust number of required quorum votes
Increases availability of cluster in failover scenarios with ability to recalculate quorum as
needed and still maintain working cluster

Scalability
SQL Server
scalability
enhancements
Encryption enhancements

Hardware accelerated encryption/decryption for TDE

Implements next generation of Microsoft cryptography
Takes advantage of specialized microprocessor instructions
Improves performance as much as 3x to 10x

Parallelizable decryption
Decryption now supported as parallelizable (used to be sequential only)
Dramatically improved response times for queries with encrypted data columns

Scalability
Distributed Replay

SQL Server Distributed Replay

Use multi-threaded replay utility

Simulate and test production workload scenarios

Protect production performance during changes

Integrate with Microsoft SQL Server Upgrade Assistant to help assess

impact of future SQL Server upgrades

Scalability
TempDB optimization

Scale up databases with enhanced data caching

Enables multiple TempDB files per instance for multi-core environments

Reduces metadata and allocation contention for TempDB workloads

Improves performance and scalability

Specifies multiple volumes for TempDB files

Scalability
Core engine scalability

Dynamic partitioning of thread-safe memory objects by

non-uniform memory access (NUMA) node or by CPU
Enables greater scalability of high-concurrency workloads running on NUMA hardware

Dynamically promotes CMemThread to be partitioned by NUMA node or by CPU based on

workload characteristics and contention factors

Eliminates need for trace flag, but also dynamically determines partition based on contention

Scalability
Summary: Scalability

Windows Server
12 TB RAM, WS2016 max cores
Server Core supported
Storage Spaces Direct
Domain-independent Availability Groups
Improved live migration

Enhanced scalability
Hardware acceleration for TDE
Distributed Replay
TempDB optimization

Scalability
Want high
performance ?
=> Upgrade !!!
“It Just Works Faster”
– Performance Improvements “under the hood” in
SQL Server 2016 Database Core Engine
Driving Issues

Increase in Larger
High Speed High Speed Data
CPUs per Memory
I/O Network Explosion
Node Footprints
 Highlights
DBCC Spatial
TEMPDB

Automatic Soft NUMA Vector Instructions

Deficit Scheduling

Dynamic CMemThread Partitioning AG Compression

Indirect Checkpoint

Larger Data File Writes

AG Transport
Multiple Log Writers
Non-Volatile Memory
Hardware Encryption

SOS_RWLock
Delay Start MSDTC
Parallel Recovery

Batch Mode Operators In Memory Optimized Checkpoint

Default Database File Sizes

Stamped LDF XEvent Linq Reader

Core improvements apply to all SKUs. Some features are SKU specific.
 DBCC Scales 7x
Old: MultiObjectScanner

New: CheckScanner

Primitives from In Memory Optimized

Checks Moved to Extended Checks

https://blogs.msdn.microsoft.com/psssql/2016/02/25/sql-2016-it-just-runs-faster-dbcc-scales-7x-better
https://blogs.msdn.microsoft.com/psssql/2016/03/01/sql-2016-it-just-runs-faster-dbcc-extended-checks
TEMPDB

-T1117 Default

-T1118 Default

Multiple Files

Metadata Access (SH)

https://blogs.msdn.microsoft.com/psssql/2016/03/15/sql-2016-it-just-runs-faster-t1117-and-t1118-changes-for-tempdb-and-user-databases
https://blogs.msdn.microsoft.com/psssql/2016/03/17/sql-2016-it-just-runs-faster-automatic-tempdb-configuration
 Auto-soft NUMA Batch Requests / Sec

Automatic Soft NUMA (20%) Enabled

Disabled
1.20 Million
0.74 Million

More 8 CPUs 2014 =

400K
sp_configure controlled

Scheduling

Partitioning

Logical vs Physical

https://blogs.msdn.microsoft.com/psssql/2016/03/30/sql-2016-it-just-runs-faster-automatic-soft-numa
Deficit Scheduling

Quantum Tracked

Group Level Tracking

Per Switch Check

Multi-CPU Considerations SQL Server 2014 21423ms

SQL Server 2016 32103ms

Azure Since Mar 2014

https://blogs.msdn.microsoft.com/psssql/2016/04/01/sql-2016-it-just-runs-faster-updated-scheduling-algorithms
 Dynamic Memory Object Partitioning

Replaces –T8048
+250% batch/sec
Dynamic

Instance Wide

To Node ~0 waits

To CPU
https://blogs.msdn.microsoft.com/psssql/2016/04/06/sql-2016-it-just-runs-faster-dynamic-memory-object-cmemthread-partitioning
 Indirect Checkpoint

Tracks Dirty Pages

Per Database

Database Default

Avoids BPool Sweep

https://blogs.msdn.microsoft.com/psssql/2016/04/12/sql-2016-it-just-runs-faster-indirect-checkpoint-default
Multiple Log Writers

4 Maximum

Multiple Nodes

Share Responsibility

Flash / SSD Targets

CPU Bound

https://blogs.msdn.microsoft.com/psssql/2016/04/19/sql-2016-it-just-runs-faster-multiple-log-writer-workers
 Spatial (2000x)
Geometry Geography TVP Before the Fix: 8000 rows/sec
Filter Filter TVP After the Fix: 120,000 rows/sec
MakeValid MakeValid

Native Invocations STAsBinary STAsBinary

AsBinaryZM AsBinaryZM STDistance
STBuffer STBuffer

Line String Vector Sort STContains STContains

STDistance STDistance
STEnvelope STEquals
TVP – Spatial Columns STEquals STIntersects
STIntersects STPointN
STRelate STWithin
Index / Tesselation STPointN
STWithin
STBoundary

https://blogs.msdn.microsoft.com/psssql/2016/03/03/sql-2016-it-just-runs-faster-native-spatial-implementations
https://blogs.msdn.microsoft.com/psssql/2016/03/08/sql-2016-it-just-runs-faster-tvps-with-spatial-columns
https://blogs.msdn.microsoft.com/psssql/2016/03/10/sql-2016-it-just-runs-faster-spatial-index-builds-faster
 Vector Instructions

SSE or AVX or Software

Used by ->

- Column Store

- Bulk Insert

With Hardware Speed

https://blogs.msdn.microsoft.com/psssql/2016/04/22/sql-2016-it-just-runs-faster-column-store-uses-vector-instructions-sseavx
https://blogs.msdn.microsoft.com/psssql/2016/04/27/sql-2016-it-just-runs-faster-bulk-insert-uses-vector-instructions-sseavx
 Always On
Throughput MB/s Average CPU utilization (secondary) MB sent on wire/sec

Reduced Context Switches (5x) Today 82 17 35

2016 540 36 230

Parallel Compression

Improved Compression

Parallel Recovery (6x)

Hardware Encryption (20x)

https://blogs.msdn.microsoft.com/psssql/2016/04/28/sql-2016-it-just-runs-faster-alwayson-log-transport-reduced-context-switches
https://blogs.msdn.microsoft.com/psssql/2016/05/03/sql-2016-it-just-runs-faster-alwayson-parallel-compression-improved-algorithms
https://blogs.msdn.microsoft.com/psssql/2016/05/05/sql-2016-it-just-runs-faster-alwayson-aes-ni-encryption
XEvent Linq Reader

10x Faster

Reduced CPU

SSMS Improvement

.NET Class Improvement

WITHOUT FIX 8000 Events/sec
WITH FIX 80,000 Events/sec

Ports
SQL 2012 - https://support.microsoft.com/en-us/kb/3112710
SQL 2014 - https://support.microsoft.com/en-us/kb/3075950
Future Considerations

Sort Memory Object Caching

Backup/Restore Compression

Reader Writer Spinlock

… and MORE
 Hyperscale cloud

Hyperscale features Simplicity Consistency

Stretch Database Easy migration of on-premises Common development,
Stretch operational tables in a secure manner SQL Server management, and identity tools
into Azure for cost-effective historic data Simple point-and-click migration to Azure Including Active Directory, Visual Studio, Hyper-
availability. Works with Always Encrypted and V, and System Center
Row-Level Security.
Suite of advisors for upgrading to SQL
High availability Server 2016 Consistent experience from SQL
Distributed availability groups add flexibility to SQL Server 2016 Upgrade Advisor in the Server on-premises to Microsoft
HA/DR. Add Azure Replica Wizard makes it easy adoption of new SQL Server features Azure IaaS and PaaS
to implement.

Enhanced backup to Azure

Enhanced backup includes faster restore times
and 50% reduction in storage. Supports larger
DBs with block blobs and custom backup
schedule with
local staging.
 Hyperscale cloud

Hyperscale features Simplicity Consistency

Stretch Database Easy migration of on-premises Common development,
Stretch operational tables in a secure manner SQL Server management, and identity tools
into Azure for cost-effective historic data Simple point-and-click migration to Azure Including Active Directory, Visual Studio, Hyper-
availability. Works with Always Encrypted and V, and System Center
Row-Level Security.
Suite of advisors for upgrading to SQL
High availability Server 2016 Consistent experience from SQL
Distributed availability groups add flexibility to SQL Server 2016 Upgrade Advisor in the Server on-premises to Microsoft
HA/DR. Add Azure Replica Wizard makes it easy adoption of new SQL Server features Azure IaaS and PaaS
to implement.

Enhanced backup to Azure

Enhanced backup includes faster restore times
and 50% reduction in storage. Supports larger
DBs with block blobs and custom backup
schedule with
local staging.
Stretch
Database
 Ever-growing data, ever-shrinking IT

Massive tables (hundreds of

millions/billions of rows, TBs size) What to do?
Users want/need to retain data Expand server and storage
indefinitely
Cold data infrequently accessed Move data elsewhere
but must be online
Datacenter consolidation
Delete
Maintenance challenges
Business SLAs at risk

Hybrid solutions
 Stretch SQL Server into Azure
Securely stretch cold tables to Azure with remote query processing
Capability
Stretch large operational tables
from on-premises to Azure with
the ability to query

Azure Benefits

SQL
SERVER
2016

Hybrid solutions
 Stretch Database architecture
How it works

Internet boundary
Creates a secure linked server definition
in the on-premises SQL Server

Targets remote endpoint with linked

Local
database
Remote
endpoint
server definition

Local Provisions remote resources and begins

data Linked servers to migrate eligible data, if migration
Remote
Eligible data is enabled
data
Queries against tables run against both
local database and remote endpoint
Azure

Hybrid solutions
 Typical workflow to enable Stretch Database
-- Enable local server
High-level steps
EXEC sp_configure 'remote data archive' , '1'; Configure local server for remote data
RECONFIGURE; archive
-- Provide administrator credential to connect to Create a credential with administrator
-- Azure SQL Database permission
CREATE CREDENTIAL <server_address> WITH
IDENTITY = <administrator_user_name>, Alter specific database for remote data
SECRET = <administrator_password> archive
-- Alter database for remote data archive Create a filter predicate (optional) to
ALTER DATABASE <database name> select rows to migrate
SET REMOTE_DATA_ARCHIVE = ON (SERVER = server name);
GO Alter table to enable Stretch for a table

-- Alter table for remote data archive Stretch Wizard in SQL Server
ALTER TABLE <table name> Management Studio makes all this
ENABLE REMOTE_DATA_ARCHIVE easy (does not currently support
WITH ( MIGRATION_STATE = ON ); creating filter predicates)
GO;

Hybrid solutions
 Queries continue working

Business applications continue

working without disruption
DBA scripts and tools work as
before (all controls still held in local
SQL Server)
Developers continue building or
enhancing applications with existing
tools and methods

Hybrid solutions
 Advanced security features supported

Data in motion always via secure

channels (TLS 1.1/1.2)
Always Encrypted supported if
enabled by user (encryption key
remains on-premises)
Row-Level Security and Auditing
supported

Hybrid solutions
 Backup and restore benefits

DBAs only back up/restore local

SQL Server hot data
StretchDB ensures remote data is
transactionally consistent with local
Upon completion of local restore,
SQL Server reconciles with remote
using metadata operation, not
data copy
Time of restore for remote not
dependent on size of data

Hybrid solutions
Current limitations that block stretching a table
• Tables with more than 1,023 columns or more than 998 indexes cannot be
stretched
• FileTables or FILESTREAM data not supported
• Replicated tables, Memory-optimized tables
• CLR data types (including geometry, geography, hierarchyid and CLR user-
defined types)
• Column types (COLUMN_SET, Computed columns)
• Constraints (Default and check constraints)
• Foreign key constraints that reference the table in a parent-child relationship.
You can stretch the child table (for example Order_Detail)
• Full text indexes
• XML indexes
• Spatial indexes
• Indexed views that reference the table
 Microsoft
Azure

Jim Gray ox7ff654ae6d 3/18/2005

Order
Order history
history Stretch to cloud
Name SSN Date
Name
Jane Doe SSN Date
cm61ba906fd 2/28/2005
Jim Gray
Jane Doe
Customer data
cm61ba906fd
ox7ff654ae6d 2/28/2005
3/18/2005 Query
John Smith
Jim Gray i2y36cg776rg
ox7ff654ae6d 4/10/2005
3/18/2005
Product data
Bill Brown nx290pldo90l 4/27/2005
John Smith i2y36cg776rg 4/10/2005
Sue Daniels
Bill
SarahBrown
Jones
✓
ypo85ba616rj 5/12/2005
Order History
nx290pldo90l 4/27/2005
bns51ra806fd 5/22/2005 App
Jake Marks mci12hh906fj 6/07/2005
Eric Mears utb76b916gi 6/18/2014
Rachel Hogan px61hi9306fj 7/1/2014
Sam Johnson ol43bi506gd 7/12/2014
Hybrid
Davidsolutions
Simon tx83hal916fi 7/29/2014
High availability
Enhanced Always On Availability Groups
Greater scalability
Unified HA solution Load-balancing readable secondaries
Increased number of automatic failover targets
Log transport performance
AG_Listener

AG
Improved manageability
Hong Kong DTC support with limitations (see Mission Critical
Asynchronous data (Secondary) section for details)
movement
Database-level health monitoring
AG
Group Managed Service Account
AG
New Jersey
Synchronous data (Secondary) Domain Independent Availability Groups
New York movement
(Primary)
Domain Independent Availability Groups
Environments supported:
Cross domains (with trust)
Cross domains (no trust)
No domain at all

On-premises databases can use AG with:

Azure Blob Storage
Azure VM with SQL Server 2016
 Simplified Add Azure Replica Wizard
Automatic listener configuration
Previously listener
configuration in Azure was
manual
SQL Server 2016

Simplicity
 Simplified Add Azure Replica Wizard

Add Azure Replica Wizard adds a replica of your databases to

Azure Blob Storage
Group listener is created and configured within the wizard
Clients can seamlessly connect to the Azure replica after
failover, as soon as the wizard completes its setup and without
additional complex steps

Simplicity
Enhanced
backup
Enhanced backup to Azure

Managed backup Backup to Azure block blobs Azure Storage snapshot backup
Granular control of the backup Cost savings on storage Fastest method for creating
schedule backups and running restores
Significantly improved restore
Local staging support for faster performance SQL Server database files on Azure
recovery and resiliency to transient Blob Storage
More granular control over Azure
network issues
Storage
Support for system databases
Support for simple recovery mode

Hybrid solutions
 Managed backup

Support for system databases

Support for databases in simple recovery mode
Leveraging backup to block blobs: more granular control
Allows customized backup schedules: full backup and log backup

144
Hybrid solutions
 Customized scheduling
Step1: Run the Scheduling SP to configure custom scheduling

EXEC Managed_Backup.sp_backup_config_schedule
@database_name = 'testDB'
,@scheduling_option= 'Custom'
,@full_backup_freq_type = 'weekly’
,@days_of_week = 'Saturday'
,@backup_begin_time = '11:00'
,@backup_duration = '02:00'
,@log_backup_freq = '00:05'

Step2: Run the Basic SP to configure Managed Backup

EXEC msdb.managed_backup.sp_backup_config_basic
@database_name= 'testDB',
@enable_backup=1,
@container_url='https://storage account name.blob.core.windows.net/container name',
@retention_days=30

Hybrid solutions
 Backup to Azure block blobs
2x cheaper storage
Backup striping and faster restore
Maximum backup size is 12 TB+
Granular access and unified credential story (SAS URIs)
Support for all existing backup/restore features (except append)

CREATE CREDENTIAL [https://<account>.blob.core.windows.net/<container>]

WITH IDENTITY = 'Shared Access Signature',
SECRET = 'sig=mw3K6dpwV%2BWUPj8L4Dq3cyNxCI‘

BACKUP DATABASE database TO

URL = N'https://<account>.blob.core.windows.net/<container>/<blob1>',
URL = N'https://<account>.blob.core.windows.net/<container>/<blob2>'

Hybrid solutions
 Backup to Azure with file snapshots

MDF MDF
Database BAK
LDF LDF
Instance

BACKUP DATABASE database TO

URL = N'https://<account>.blob.core.windows.net/<container>/<backupfileblob1>‘
WITH FILE_SNAPSHOT

Hybrid solutions
 Backup to Azure with file snapshots

Available to users whose database files are

located in Azure Storage
Copies database using a virtual snapshot within
Azure Storage
Database data does not move between storage system and server
instance, removing IO bottleneck

Uses only a fraction of the space that a traditional

backup would consume
Very fast
Hybrid solutions
 Summary: Enhanced backup

Hybrid solutions
 Hyperscale cloud

Hyperscale features Simplicity Consistency

Stretch Database Easy migration of on-premises Common development,
Stretch operational tables in a secure manner SQL Server management, and identity tools
into Azure for cost-effective historic data Simple point-and-click migration to Azure Including Active Directory, Visual Studio, Hyper-
availability. Works with Always Encrypted and V, and System Center
Row-Level Security.
Suite of advisors for upgrading to SQL
High availability Server 2016 Consistent experience from SQL
Distributed availability groups add flexibility to SQL Server 2016 Upgrade Advisor in the Server on-premises to Microsoft
HA/DR. Add Azure Replica Wizard makes it easy adoption of new SQL Server features Azure IaaS and PaaS
to implement.

Enhanced backup to Azure

Enhanced backup includes faster restore times
and 50% reduction in storage. Supports larger
DBs with block blobs and custom backup
schedule with
local staging.
Migrate
databases to
Azure
 Easily migrate to Microsoft Azure
Simple single-click migration experience
Capability
User DB System objects SQL settings
Along with schema and data, migrate other
system objects (logins, jobs, and certificates)
Migrate SQL Server settings (trace flags,
default language, and memory settings)

Migration Wizard Benefits

Literally as simple as point and click

On-premises
Microsoft Azure

Simplicity
 Migration methodologies
Method 1 Method 2 Method 3
SQL Server SQL Server
SQL Server
Visual
1. Import Studio
Source
Source Source database
DB
DB DB project
1. Export 4. Copy
1. 2. Transform
Generate 5. Publish
SSMS OR 1. Deploy SQL Azure SQL Azure
SSMS Copy
Migration T-SQL *.sql Migration
.bacpac DB
Wizard Wizard
2. Import 2. Execute
3. Edit, build
6. Export/
and test
import
Target or deploy 4. Publish (schema only)
Target Target
DB DB DB

SQL Database SQL Database SQL Database

Migrate a compatible database Migrate a near-compatible Update database schema offline using
using SSMS database using SAMW Visual Studio and SAMW, and then
deploy it with SSMS

Simplicity
 Migration Cookbook
Migrate an on-premises SQL Server database
to SQL Database (v12)

The Migration Cookbook describes various

approaches to migrate an on-premises SQL
Server database to the latest SQL Database
update (v12)

Download: http://aka.ms/azuresqlmigration

Simplicity
Suite of advisors
for upgrading
 Upgrade workflow today
Discover
Upgrade is a complicated process with many considerations

Most upgrades are actually “migration projects”

Monitor Assess
Wave of SQL 2005 upgrades coming

Upgrade Advisor focused on the “assess” phase: find

functional blockers

Review Plan

Implement

Simplicity
 Suite of advisors
Modern, scenario-driven user
experience

Suite of advisors

Rules built on DACFx

Modern telemetry and feedback

pipeline

Designed to move to a
community project

Upgrade Advisor and Stretch

Database initial scenarios

Independent install via WebPI or

download center

Auto-update notifications

Simplicity
 Upgrade Advisor Analysis Wizard

Extract Analyze Report

Built on DACFx as a standard runtime

Cataloged and ported rules from all known tools
Designed to have community-written rules
Support for SQL Server 2005 (compat level 80)
HTML and CSV reports

Simplicity
 Advisor for upgrade issues and recommendations
Analyzes instance for potential upgrade issues:
Pre-upgrade issues
Pre-upgrade recommendations
Post-upgrade considerations

Delivers feature advisors

First scenario is Stretch Database Advisor

Download: https://www.microsoft.com/enus/download/details.aspx?id=48119

Simplicity
 Guidance and customer feedback
New engine for defining
guidance rules
Self-contained HTML results as well as CSV

Smaller, more frequent releases

New delivery methods

Built-in auto-update notifications

Customer feedback
Modern telemetry pipeline
Built-in feedback feature

Simplicity
 Hyperscale cloud

Hyperscale features Simplicity Consistency

Stretch Database Easy migration of on-premises Common development,
Stretch operational tables in a secure manner SQL Server management, and identity tools
into Azure for cost-effective historic data Simple point-and-click migration to Azure Including Active Directory, Visual Studio, Hyper-
availability. Works with Always Encrypted and V, and System Center
Row-Level Security.
Suite of advisors for upgrading to SQL
High availability Server 2016 Consistent experience from SQL
Distributed availability groups add flexibility to SQL Server 2016 Upgrade Advisor in the Server on-premises to Microsoft
HA/DR. Add Azure Replica Wizard makes it easy adoption of new SQL Server features Azure IaaS and PaaS
to implement.

Enhanced backup to Azure

Enhanced backup includes faster restore times
and 50% reduction in storage. Supports larger
DBs with block blobs and custom backup
schedule with
local staging.
Consistent
platform
 Consistent platform

Consistent and
integrated platform
Virtualization

Complete data platform

Unified management and DevOps

Flexible development paradigm

Common identity

Consistency
 Consistent tools

Consistency across:
On-premises, private cloud, public cloud

SQL Server local, VM, SQL Database

Scalability, availability, security, identity, backup and restore, and replication

Plethora of data sources

Reporting, integration, processing, and analytics

All supports hybrid cloud

Consistency
 SQL Server, Azure VMs, SQL Database

On-premises
Hybrid cloud Options:
Shared
SQL Server on physical machines
lower cost Platform as Software as
a service a service
SQL Server in on-premises VMs
(private cloud)
Infrastructure SQL
SQL Database
as a service Virtualized databases
SQL Server in Azure VMs
SQL Server in Azure VM (public cloud)
Virtual SQL Virtualized machines

SQL Database (public cloud)

SQL Server Private Cloud
Physical Virtualized machines +
appliances

SQL Server
Dedicated Physical machines (raw iron)
SQL
higher cost
Off-premises

Higher administration Lower administration

Consistency
SQL Server 2016 SP1 – Nye features
SQL Server 2016 SP1 – Nye features
Standard Edition og Memory med SP1

Eksempel: SQL 2016 Standard SP1

• 25 DBs – Heraf 2 med In-Memory, og 1 med Columnstore
• Reporting services og SSAS i MOLAP mode i brug

• Resultat: 352 GB Max

➢ 128 GB Core Engine
➢ 64 GB til In memory
➢ 32 GB til Columnstore
➢ 64 GB til SSRS
➢ 64 GB til SSAS
Andre væsentlige forbedringer
• Database Cloning
• Cloning af en database
• Schema, Metadata, Statistics, men uden data
• Med SP1 support for CLR, Filestream/Filetable, In-Memory og Query Store
• Tjek om en database er clonet med DATABASEPROPERTYEX(‘clonedb’, ‘isClone’)
• DBCC CLONEDATABASE (source_database_name, target_database_name)
• CREATE OR ALTER
• Fungere til alle gængse objekttyper som Stored Procedures, trigger, functions etc.
• Query Hints
• Improved AlwaysOn Latency Diagnostics
• Removing noisy Hekaton logging messages from Errorlog
• M. fl. (Læs mere her)
SQL 2016 – Editions
Further Readings
The Data Driven Series
https://www.microsoft.com/data-driven

Alt om de forskellige editions

https://www.microsoft.com/da-dk/sql-server/sql-server-editions
https://technet.microsoft.com/da-dk/windows/cc645993
it-Craft SQL-specialisterne

[email protected] +45 70 27 70 22

www.it-Craft.dk

www.linkedin.com/company/itcraftdk

www.facebook.com/itCraftDK

twitter.com/itCraftDK