DP 300notes241025
Uploaded by
omar.alzahraniDP 300notes241025
Uploaded by
omar.alzahraniDP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
Plan and Implement Data Platform Resources
1. deploy database offerings on selected platforms
• SQL Server Virtual Machine
o Go to Azure SQL
▪ This is not a single service, but a group of SQL services.
- +Create, SQL database.
o Server name needs to be unique throughout Azure.
o Admin account password must be at least 12 characters long.
o In additional settings, add existing data if required, either from a backup or sample data.
o You should use Managed Service Accounts (MSA) for a single computer running a service.
▪ A Group Managed Service Account (gMSA) is used for assigning the MSA to multiple
servers.
• Azure SQL Managed Instance
o Go to Azure SQL, +Create, SQL managed instance.
o You need:
▪ Subscription and Resource group,
▪ Managed instance name,
▪ Region,
▪ Managed instance admin login and password.
• Azure VM with SQL Server
o Create a Virtual Network.
▪ In IP Addresses – default (subnet) – Edit subnet , add the Service Endpoint
"Microsoft.Sql"
o Create the Azure VM
▪ In Networking, select "Private endpoint", then "+Add private endpoint" and select
the subnet from above.
o When created, in "Firewalls and virtual networks", click "+Add client IP", and "Allow Azure
services and resources to access this server".
2. configure customized deployment templates
• See topics 9, 10 and 11.
3. apply patches and updates for hybrid and IaaS deployment
• IaaS deployment means VM, a hosted infrastructure.
Page 1 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
o PaaS SQL Database and Managed Instance have built-in patching, and they always use the
latest stable Database Engine version.
• Operating systems updates are as per standard Microsoft Windows Update.
o Windows Update may do some updates automatically.
• Patches are as per standard SQL Server Patches.
o You have full control of the database engine, e.g. when to apply patches.
o However, you can also enable “Automated Patching”
▪ You need SQL Server 2008 R2 or above, and Windows Server 2008 R2 or above.
▪ In SQL Server settings which creating the VM, or
▪ In the Azure resource, go to Settings – SQL Server configuration – Patching.
▪ It may take a few minutes for this to be configured.
▪ Settings include day, start hour, window duration
(number of minutes to download and install).
o You can configure it when:
▪ creating a new VM (under SQL Server Settings tab – Automated patching), or
▪ for existing VMs, by going to Azure Portal – the relevant VM – Settings – SQL Server
configuration – Patching.
▪ By using PowerShell, with:
- "New-AzVMSqlServerAutoPatchingConfig -Enable" setting the schedule,
and
- "Set-AzVMSqlServerExtension" installing the extension with the schedule.
o You can also enable automatic registration.
▪ This daily checks whether there are any unregistered VMs in the subscription, and if
so, registers them in lightweight mode.
- To take advantage of all of the features, you would still need to manually
upgrade.
▪ To do this, go to Azure Portal – SQL virtual machines (plural) – and at the top, click
on "Automatic SQL Server VM registration".
Page 2 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
-. evaluate requirements for the deployment
5. evaluate the functional benefits/impact of possible database offerings
Page 3 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
5. evaluate the scalability of the possible database offering
5. evaluate the HA/DR of the possible database offering
Page 4 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
6. evaluate the security aspects of the possible database offering
7, 12. evaluate table partitioning strategy
• Data can be partitioned – physically divided into different data stores.
o Scalable – there are hardware limits, but if you divide data into partitions, each on a separate
server, it can be scaled out.
o Increase performance – Smaller amount of data in a single partition, and multiple data stores
can be accessed at the same time.
o Security considerations – apply different security to sensitive and non-sensitive partitions.
o Administration – have different strategies of monitoring, management and backup per
partition. Backups for a single partition are quicker than for the entire data.
o Have different hardware or services – Premium or Standard where needed.
o Increase availability – if one instance fails, only that partition is temporarily unreadable.
• Partitions can be:
o Horizontal partitioning (sharding).
▪ All partitions have the same schema.
▪ Each partition, or shard, holds a subset of the data (rows).
▪ If some data is fairly static or small, consider replicating it in all partitions, to reduce
cross-partition access.
o Vertical partitioning.
▪ Each partition holds a subset of the columns.
▪ Some columns may be needed less often, and they could be separated away, and
used only when needed.
▪ Some columns may also be more sensitive, and could be separated away.
▪ All partitions would need to be capable of being joined – for instance, by the same
primary key in each.
o Functional partitioning.
▪ Different tables in each partition relating to function.
Page 5 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
▪ Store data could be in one partition, and employee data in another.
▪ Some tables could be more sensitive, and could be separated away into another
partition.
o These techniques can be combined.
o Keep the data, where possible, geographically close to the users.
• Consider the backup, archiving (including deleting) and High Availability, Disaster Recovery
requirements for each partition.
8. evaluate database partitioning techniques, such as database sharding
• Why partition?
o Storage space limitations
▪ Maximum storage capacity can be reached on a server.
o Computing resource limits
▪ Exceeding this may result in time out.
o Network bandwidth
▪ Exceeding this can result in failed requests.
• You can scale vertically:
o Add disk capacity, processing power, memory and network connections.
o However, you may reach the same problem later.
• You can partition the data store horizontally into shards.
o Each shard has its own subset of the data.
o It runs on a server acting as a storage node.
• You can:
o Scale out by adding further shards
o Use off-the-shelf hardware for each storage node
o Balance the workload across shards
o Locate shards near to the users who will use it.
• You can do it by implementing:
o Lookup strategy
▪ Have a shard key (an ID), and a map which shows where the data is stored.
▪ Offers more control.
▪ Requires additional overhead.
o Range strategy
▪ Use sequential shard keys in ranges (e.g. one per month).
Page 6 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
▪ Similar data is kept on the same storage node, so it can retrieve multiple items in a
single operation.
▪ Doesn’t necessarily provide optimal balancing.
▪ Rebalancing shares is difficult.
o Hash strategy
▪ Data distributed evenly among the shards. Reduces hotspots (high loads for an
individual server) by using some random element for distribution.
▪ More even data and load distribution.
▪ Computing the hash might increase overhead requirements.
▪ Rebalancing shards is difficult.
9. configure Azure SQL Database for scale and performance
• Azure SQL Database can be:
o Single database.
o Elastic pool.
o Compute tier:
▪ Provisioned – for regular usage patterns, or multiple databases with elastic pools.
▪ Serverless computer – on-off usage with a relative low average compute.
- Supports automatic pausing and resuming.
- When the database is paused, you only pay for storage.
• Azure SQL Database allows for:
• vCore purchasing model
o Specify separate amount of Number of vCores, memory, and amount/speed of storage. Look
at:
▪ Data Maximum Memory size,
▪ Log size (30% of the Data Max size),
▪ iOPS,
▪ Concurrent workers, and
▪ Backup retention.
o Maximum of:
▪ 80 vCores at Gen5,
▪ 4 Tb memory, and
▪ 4 Tb database size (apart from Hyperscale, which has up to 100 Tb).
o Azure Hybrid Benefit and/or reserved capacity
Page 7 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
▪ Azure Hybrid Benefit allows you to bring in your existing on-prem licenses to the
cloud.
▪ Reserved capacity is paying in advance at a discount.
o Uses local SSDs. Provision in 1 Gb increments.
o Choose from:
▪ General purpose (scale computer and storage) – For most business workloads.
Storage latency of 5-10 ms (about the same as SQL Server on a VM).
▪ Business critical (high transaction rate and high resiliency) – uses
- When you need low-latency I/O (1-2 ms) or frequent communications
between app and database.
- Large number of updates, or long running transactions that modify data.
- Higher resiliency, availability and fast geo-recovery and recovery from
failures, and advanced data corruption protection.
- Free-of-charge secondary read-only replica.
▪ Hyperscale (on-demand scalable storage) – Only for Azure SQL Database – say 100
Tb+ storage.
- You cannot subsequently change out of Hyperscale. Cost the same as Azure
SQL Database.
o Backup Storage Redundancy
▪ Geo-redundant backup storage (default and recommended),
▪ Zone and Local Redundancy are cheaper for single region data resiliency.
o Max data size:
▪ 512 Gb for GP_S_Gen5_1
▪ 1,024 Gb for GP_S_Gen5_2, 4, 6
▪ 1,536 Gb for GP_S_Gen5_10 and
▪ 3,072 Gb for GP_S_Gen5_12, 14, 16, 18, 20.
▪ 4,096 Gb for GP_S_Gen5_24, 32 and 40.
o Tempdb
▪ Azure SQL Database creates 1 file per vCore with 32Gb per file, with caps of up to 32
files for serverless computing only.
• DTU-based purchasing model
o For light to heavy database workloads.
o Offers bundles of maximum number of compute, memory and I/O (reads/writes) resources
for each class (cannot separate them).
o Uses Azure Premium disks. Provision in increments of 250 Gb to 1 Tb, and 256 Gb thereafter.
Page 8 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
o Choose from:
▪ Basic (for less demanding workloads)
▪ Standard (for typical performance)
▪ Premium (for I/O-intensive workloads)
▪ Please note: Basic and Standard S0, S1 and S2 have less than 1 vCore, and cannot
use "Change data capture".
- Consider Basic, S0 and S1, where database files are stored in Azure
Standard Storage (HDD), for development, testing and infrequently
accessed workloads.
o Consider changing to vCores if > 300 DTUs
▪ Might reduce costs – no downtime when converting.
o Calculates available for calculating DTUs needed:
▪ See https://dtucalculator.azurewebsites.net/
• Can change service tier on demand (but not out of Hyperscale).
o Don’t do it when you have a long job running!
o For the DMVs to have accurate figures, you may need to flush the Query Store after re-
scaling. Use:
▪ EXEC sp_query_store_flush_db;
• Choose from server or serverless model:
o Server.
▪ This is a logical server, which includes logins, firewall and auditing rules, policies and
failover groups.
▪ The server name must be unique in Azure.
▪ Don't need to manage the instance.
▪ Cannot use "USE" to change database context.
o Serverless model.
▪ Bills for compute per second.
▪ Pauses databases and billing in inactive periods.
• Configure network:
o No access.
o Public/private endpoint.
▪ You can "Add current client IP address".
o Choose whether to "Allow Azure services and resources to access this server" (for other
Azure services).
Page 9 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
▪ Or if not, you could allow specific Virtual Networks to have access.
• Connection policy
o Proxy – uses Azure SQL Database gateways,
o Redirect – Establish directly to the database node,
o Default – Redirect if connection originates inside Azure, and Proxy if outside Azure.
• You can have sample data, or data based on the restore from a geo-replicated backup.
• Choose database collation:
o CS/CI = case-[in]sensitive,
o AS/AI = accent-[in]sensitive.
• You can select a free trial of Azure Defender.
o Identify and mitigate potential database vulnerabilities and threat detection.
• It uses the Full Recovery model.
o This should not be changed for Azure SQL Database.
Page 10 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
10. configure Azure SQL Managed Instance for scale and performance
• Service Tier:
o General Purpose
▪ Most workloads (default option),
o Business Critical
▪ low-latency workloads
▪ High resiliency to failures
▪ Fast Failovers
• Hardware Generation
o Compute and memory limits.
o Up to 80 vCores,
o 400 Gb memory,
o Up to 100 databases, and
o up to 16 Tb database size.
• Features in Azure MI not in Azure SQL Database include:
o Cross-database queries,
o Common language runtime (CLR)
▪ The execution environment for .NET framework code (also known as "managed
code").
▪ CLR in SQL Server is called CLR integration.
o SQL Agent, and
Page 11 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
o The msdb system database.
o You can manually make a copy-only backup of a database (not instance).
• vCore compute model
o SQL Managed Instances does not support the DTU-based purchased model.
o See topic 9 for more details of the vCore compute model.
o Cannot use Hyperscale in Azure SQL MI.
• Deploys a dedicated ring (aka "Virtual cluster") for your data.
• Tempdb
o MI creates 12 files, regardless of the number of vCores.
11. configure SQL Server in Azure VMs for scale and performance
• SLA for Virtual Machines
o 95% (18 days) - Standard HDD Managed Disks,
o 99.5% (1.8 days) - Standard SSD Managed Disks,
o 99.9% (8 hours) - Premium SSD or Ultra Disks,
o 99.95% - 2+ instances in the same Availability Set
▪ Different computers in the same datacenter
o 99.99% - 2+ instances in 2+ Availability Zones in the same Azure region
▪ Different physical datacenters, with independent power, cooling and networking.
• When to use SQL Server in Azure VMs:
o When you need an older version of SQL Server or access to a Windows Operating System.
o When you need SSAS (Analysis), SSIS (Integration) or SSRS (Reporting) (non Azure services),
Page 12 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
o When you need features not available in Azure SQL Database or Azure MI.
• Best practices – VM Size:
o Choose at least 4 vCPUs.
o Memory optimized VMs for best SQL Server workloads.
o Higher memory-to-vCore ratio for mission critical or data warehouses.
o Azure VM marketplace images are configured for optimal SQL Server performance.
• Best practice – Storage:
o Place data, log and tempdb files on separate drives.
▪ Data drives should be put on Premium P30 and P40 disks for cache support.
▪ Log drive should be put on Premium P3o to P80 disks, or Ultra disks for
submillisecond latency.
▪ Tempdb should be placed on the D drive on the SSD.
o Stripe multiple data disks using Storage Spaces (similar to RAID, but done in software) to
increase I/O bandwidth. 3+ drives form a storage pool. This should be done by:
▪ Creating the individual disks,
▪ Creating a storage pool,
▪ Creating a single virtual disk, from these resiliency types.
- Simple
o Needs at least 1 physical disk
o Stripes data physical disks.
o Maximizes disk capacity and increases throughput.
o No resiliency (does not protect from disk failure)
o Use for high-performance where resiliency is not required by
striping.
- Mirror
o Needs at least 2 physical discs to protect from single disk failure.
o 2-3 copies of the data.
o Increases reliability, but reduces capacity.
o Greater data throughput and lower access latency.
o Use for most deployments.
- Parity
o Needs at least 3 physical discs to protect from single disk failure.
o Stripes data and parity information across disks.
Page 13 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
o Increases reliability, but reduces capacity.
o Increases resiliency.
o User for archive and backups.
▪ Creating a volume.
o Use Local Redundant Storage, not Geo-redundant storage, on the storage account.
• Six different series are available:
o General purpose – balanced CPU-to-memory.
▪ Good for testing and development, small-medium databases or traffic web servers.
o Compute optimized – high CPU.
▪ Good for medium traffic web servers, network appliances, batch processes, and
application servers.
o Memory optimized – high memory (up to 4 Tb).
▪ Good for relational database servers, medium to large caches, and in-memory
analytics.
o Storage optimized – high disk throughput
▪ Good for Big Data, SQL, NoSQL databases, data warehousing and large transactional
databases.
o GPU – specialized virtual machines
▪ heavy graphic rendering and video editing, as well as model training and inferencing
(ND) with deep learning.
o High performance compute – fastest machines
▪ Most powerful CPU virtual machines
• Best practices – SQL Server:
o Enable database page compression where appropriate.
o Enable backup compression and instant file initialization.
o Limit autogrowth and disable autoshrink.
o Use one tempdb data file per core, up to 8 files.
o Apply any cumulative updates for your version of SQL Server.
o Register with the SQL IaaS Agent Extension for:
▪ Automated backup,
▪ Automated patching,
▪ Azure Key Vault integration,
▪ View information in Azure Portal about your SQL Server configuration, and more.
Page 14 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
▪ It is installed when you deploy an SQL Server VM from the Azure Marketplace.
o Enable Autoshutdown for development and test environments.
11. configure storage and infrastructure resources
• Virtual Machines:
o When creating a VM, the "SQL Server settings – Change configuration" shows the storage.
o Under "Configure storage":
▪ Select "Transactional processing" or "Data warehousing" – this changes your stripe
configuration, optimising it for traditional OLTP workloads or analytic/reporting
workloads.
o All of the SQL Server VM marketplace images follow default storage best practices.
▪ See topic 11.
o After setting the VM, when using disk caching for Premium SSD, you can select the disk
caching level (by going to Settings – Disks):
▪ It should be ReadOnly for SQL Server data files, as this improves reads from cache
(VM memory and local SSD), which is much faster than from disk (Azure Blob
storage).
▪ It should be None for SQL Server Log files, as the data is written sequentially.
▪ ReadWrite caching should not be used for the SQL Server files, as SQL Server does
not support data consistency with this cache type. However, it could be used for the
O/S drive, but it is not recommended to change the O/S caching level.
▪ Any changes will require a reboot.
• For Azure SQL Database and Managed Instance:
o You can configure the storage by going to Settings – "Compute + storage".
-. calculate resource requirements
• Purchasing models:
o DTU Model – a package of compute, storage and IO resources.
o Simple, pre-configured resources.
• vCore-based model:
o Independent scaling of compute, storage and IO resources.
o Flexible, control and transparency
o Use with Azure Hybrid Benefit for cost savings.
o Business Critical service tier includes 3 replicas (and about 2.7x price)
• For both model:
o Single database.
Page 15 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
▪ They can be moved in/out of elastic pool.
▪ They are isolated from others and is portable.
▪ They can be dynamically (i.e. manually) scaled (but not autoscaled) up and down.
o Elastic pool.
▪ Assign resources which are shared by all pool databases.
▪ Can dynamically scale or autoscale resources up/down.
▪ This is for multiple databases, good when they have variable usage patterns.
▪ Can add databases by going to the pool and clicking on "+Add databases".
• Storage costs:
o Based on amount provisioned – that’s your maximum database size.
• Calculate costs from https://azure.microsoft.com/en-us/pricing/details/azure-sql-database/single/
• For DTU model, consider the following factors when determining how many DTUs you need:
o Maximum storage bytes [for all databases in the pool],
o Average DTU utilization x Number of databases,
o Number of concurrently peaking databases x Peak DTU Utilization per database.
o Note: Unit price for eDTU pools is 1.5x the DTU unit price for a single database.
▪ Price for v-Core pools is at the same unit price as for single databases.
13. evaluate the use of compression for tables and indexes
• Why compress?
o Reduced space – useful for data which is infrequently used.
o However, it requires extra time and CPU, both to compress and retrieve data.
o You can compress at the row level, the page (8,192 characters) level, or none.
▪ For row compression
- Numeric types (apart from tinyint) storage will be reduced, maybe down to
1 byte. Tinyint already takes 1 byte.
- Some date types will be compressed: datetime, datetime2 and
datetimeoffset. The others would not benefit from compression.
- char and nchar will be compressed, up to 50% in English, German, Hindi
and Turkish, but only up to 40% in Vietnamese and 15% in Japanese.
varchar and nvarchar types would not benefit from compression.
▪ Page compression consists of three operations in this order:
- Row compression
- Prefix compression
Page 16 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
o If values in the same column start with the same characters, this
can be optimised.
o A common prefix per column is moved to the Compression
Information structure immediately after the page header.
o A reference is made in that value to the prefix, and the number of
characters which are in common.
- Dictionary compression
o If values after prefix compression in any column are the same, this
can be optimised.
o Again, these common values are moved to the Compression
Information structure, and a reference is made.
- Pages are uncompressed at first. Page compression is only used when
additional rows can be fitted on a full page.
o Compression does not affect backup and restore.
• Available in:
o Azure SQL Database, Azure SQL MI and
o SQL Server on VMs
▪ from SQL Server 2016 SP1 in all editions, and
▪ before than, only in the Enterprise edition.
• You can compress:
o Tables stored with a clustered index or without (a heap).
▪ You cannot use data compression with tables which have SPARSE columns.
▪ To change the compression option in a clustered index, you need to drop the
clustered index, preferably OFFLINE, and then rebuild the table.
o A complete nonclustered index.
▪ By default, they are not compressed.
o A complete indexed view.
• You cannot compress system tables.
• Different partitions can be compressed using different settings.
• Would compression be useful?
o The following storage procedure cannot be used in Azure SQL Database.
▪ EXEC sp_estimate_data_compression_savings
- 'SchemaName',
- 'TableName',
Page 17 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
- Index_ID – either zero for a Heap, 1 for a clustered Index, or >1 for Non-
clustered Index. NULL if a table, and not an index,
o To get the index number, use:
◼ SELECT name, index_id
◼ FROM sys.indexes
◼ WHERE OBJECT_NAME (object_id) = N'TableName';
- PartitionNumber (or NULL)
o To get the partition number, use:
◼ SELECT *
◼ FROM sys.partitions
◼ WHERE OBJECT_NAME (object_id) = N'TableName';
- 'ROW' (or 'PAGE', 'NONE')
• To enable compression:
o In SSMS
▪ Right-hand click on the table or index, and go to Storage – Manage Compression
▪ Click next, and select the compression type for each partition.
- You can also click on "Use same compression type for all partitions".
- You can also click on "Calculate" to calculate space requirements (not
Azure SQL Database).
▪ Select whether to run immediately or to create a script (to a file, clipboard, or new
query window).
- If using this on a VM, you may also get "Schedule – you could select: one
time, recurring (Daily, Weekly or Monthly), when SQL Server Agent starts,
or whenever the CPUs become idle.
o In T-SQL - table
▪ ALTER TABLE Schema.TableName
▪ REBUILD PARTITION = 1 | ALL WITH (DATA_COMPRESSION = PAGE | ROW | NONE);
o In T-SQL – index
▪ ALTER INDEX IndexName ON Schema.TableName
▪ REBUILD PARTITION = ALL WITH (DATA_COMPRESSION = PAGE | ROW | NONE);
• For columnstore objects:
o Initially used in data warehouses, but then expanded.
o Columns are always compressed.
o Indexes work best when you scan large amounts of data, like fact tables in data warehouses.
Page 18 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
▪ They are generally clustered. Non-clustered only uses when you have a data type
not supported by a clustered index – e.g. XML, text and image.
o They can be further compressed using archival compression.
▪ Best used when the data is not often read, but you need the data to be retained for
regulatory or business reasons.
▪ Saves space, but there is a high CPU cost to uncompressing it, which is more than
any I/O saving.
14. evaluate requirements for the migration
o What workloads you intend to migrate.
o The actual resource requirements:
▪ Hard drive space,
▪ Compute size (processing power),
▪ Version of SQL Server needed.
- This will impact on whether you can use Azure SQL Database/Managed
Instance, or whether you need a VM.
▪ Version of Windows Server needed (if any).
o Downtime allowances
▪ Are you allowed any downtime at all? If not, you need to do an online migration.
o Dependences between databases, and between databases and applications.
o Security requirements
o Backup and restore requirements
o Current limitations, and future limitations.
o Location for data storage (e.g. GDPR, California Consumer Privacy Act, or similar
requirements)
15. evaluate offline or online migration strategies
• Do you need instead to lift and shift SQL Server to a Virtual Machine?
o If so, use Azure Migrate.
o It can also discover and assess SQL data estate at scale (across your data center).
o Get Azure SQL deployment recommendations, target sizing and monthly estimates.
• Do you need to migrate non-SQL objects, such as Access, DB2, MySQL, Oracle and SAP ASE databases
to SQL Server or Azure SQL?
o If so, use the SQL Server Migration Assistant (SSMA).
• Do you need to migrate SQL Server objects to SQL Database/Managed Instance? If so:
o Do you need to migrate and/or upgrade SQL Server?
Page 19 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
▪ If so, use Data Migration Assistant (DMA).
▪ It can help migrate to Azure SQL Database, or to a VM or to another on-prem
server.
▪ It can also discover and assess SQL data estate, and recommend performance and
reliability improvements for your target environment.
▪ Detect compatibility issues between your current database and a target version of
SQL Server or Azure SQL.
▪ Move your schema, data, and uncontained objects.
o Do you need to compare workloads between the source and target SQL Server?
▪ If so, use the Database Experimentation Assistant (DEA).
▪ Capture the workload of a source SQL Server environment.
▪ Identity compatibility issues.
o Do you need to migrate open source databases, such as MySQL, PostgreSQL or MariaDB?
▪ If so, use the Azure Database Migration Service (DMS).
▪ Minimal downtime (especially if online using the Premium pricing tier). Good for
large migrations.
▪ You need:
- To allow outbound point 443 (HTTPS) – you may also need 1434 (UDP).
- Enable the TCP/IP protocol.
- Create an Azure SQL Database instance, have a server-level firewall rule to
allow DMS access, and have CONTROL DATABASE permission on the target
database.
- Have CONTROL SERVER permissions on the source.
- Does not initiate any backups, and uses existing full and log backups (not
differential).
Page 20 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
16, 21. implement an online migration strategy
• See topic 20.
16, 21. implement an offline migration strategy
• Migrating from SQL Server to Azure SQL Database - Prerequisites:
o Enable TCP/IP protocol on your SQL Server instance.
o Download the Data Migration Assistant.
o Create a Virtual Network for the Azure Database Migration Service using either ExpressRoute
or VPN.
o Enable outbound port 443 of ServiceTag for ServiceBus, Storage and AzureMonitor.
o Allow database engine access in Windows firewall, and open the Windows firewall to TCP
port 1433 (unless you have changed it). You may also need to have UDP port 1434.
▪ Note: port 3306 is for MySQL and 5432 is for PostgreSQL.
o Create a server-level IP firewall rule to allow Azure Database Migration Service access.
o Your credentials need CONTROL SERVER on the SQL Server instance, and CONTROL
DATABASE on Azure SQL.
• Check for blocking issues:
o In Data Migration Assistant, select +New and Assessment, and enter a project name.
o Select Database Engine, SQL Server and Azure SQL Database, and either/both:
▪ Check database compatibility – identifies partially supported or unsupported
features which may block migration. There will be recommendations.
▪ Check feature parity – recommendations, different approached, and mitigating
steps.
o Click Next and provide connection details to SQL Server.
o Select databases, click Add and Start Assessment.
• To migrate sample schema:
o In the DMA, click +New, Project Type, Migration.
o Add a project name, SQL Server, and Azure SQL Database
Page 21 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
o Select “Schema only” in “Migration scope”.
o Click Create, and enter your SQL Server details.
o Click Next, then enter your Azure SQL details.
o Click Connect, and select the relevant database.
o Click Next, and specify the schema objects to be deployed.
▪ By default, all of them are selected.
o Click ”Generate SQL script” then “Deploy schema”.
• Register the Microsoft.DataMigration resource provider
o In the Azure portal, go to Subscriptions.
o In the Azure Database Migration Service subscription, click on Resource providers.
o Find “Microsoft.DataMigration” and Register it.
• Create an instance of the Azure Database Migration Service
o In the Azure portal, go to this service and click Create, and select:
▪ Subscription, Resource Group, Name
▪ Location, ”Azure” as Service mode, and
▪ Pricing tier (Standard is free)
- Standard is free for offline (one-off) migration only.
- Premium is about $1 for 3 hours. Allows for online (continuous migration)
and offline migrations, and faster speeds.
- You can have the 4 vCore Premium DMS free for 6 months. You can use it
for a total of 1 year, and create 2 DMS services per subscription.
o On ”Create Migration Service”, select an existing VN or create a new one.
• Create a migration project
o In the Azure portal, go to “Azure Database Migration Services”, select the relevant instance,
and select ”New Migration Project”.
o Add a project name, SQL Server, Azure SQL Database, and Data migration.
o Click on “Create and run activity”.
• Enter source settings
o Enter connection details (Fully Qualified Domain Name or IP Address),
o If you haven’t a trusted certificate, check “Trust server certificate”.
o Select databases, note the Expected downtime, and click “Next: Select target:.
• Enter target settings
o Enter target details.
Page 22 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
o Click “Next: Map to target databases”. This will be mapping to new databases, unless you
have a database with the same name.
o Click “Next: Configuration migration settings and select tables to be affected.
o Click “Next: Summary” and enter an Activity Name for the migration.
• Run the migration:
o Click “Start migration”. You can monitor the migration from there.
o Once complete, verify that the target database has been migrated.
• Other options:
o Bulk Copy Program (bcp) can be used for connecting from on-prem or a VM to Azure SQL.
o BULK INSERT – loading data from Azure Blob storage.
o SSIS packages – ETL (extract, transform and load).
o Spark or Azure Data Factory
18. Perform post migration validations
• Create validation tests
• Use SQL queries against both source and target databases.
• Create performance tests
• Against source and target databases.
• Check your apps
• They might need updating (e.g. references to the target database)
• Investigate what effect the database compatibility level may have had
• Azure SQL Database and Azure SQL MI will always use the latest version.
• Are existing queries using the best plan under the new compatibility level?
• Are there regressions? If so, force the last known good plan.
• What about parameter sniffing? Do stored procedures need to be recompiled?
• Look for features which work better in the source database but not in the target.
• Have a look for missing indexes.
• Are there new features in the latest database compatibility level?
• Some features may only be available once the database compatibility level has
changed.
• What about missing features?
• Azure SQL Database has fewer features than Azure SQL MI, which has fewer features than
on-prem or Azure VM.
Page 23 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
20. set up SQL Data Sync
• Azure SQL Data Sync allows you to synchronize data across multiple databases.
o Tables need to have a primary key, which cannot be changed (rows can be deleted/recreated
instead).
o Does not work with Azure SQL Managed Instance.
• You define one Azure SQL Database as the Hub Database.
• Sync Metadata Database contains the metadata and log for Data Sync. It is an Azure SQL Database in
the same region as the Hub Database.
o It should be an empty database. Data Sync creates tables and runs a frequent workload.
• Member databases are either Azure SQL Database or on-prem (not Managed Instance).
o If you are using on-prem, you will need to install and configure a local sync agent.
▪ Download it from https://www.microsoft.com/en-
us/download/details.aspx?id=27693
• A sync group has these properties:
o Sync Schema shows what data is synchronized.
o Sync Direction can be both ways or only one direction.
o Sync Interval is how often synchronization happens.
o Conflict Resolution Policy is “Hub wins” or “Member wins”
▪ But if there are several members, this depends on which member syncs first.
• Use in:
o Hybrid Data Synchronization.
o Distributed Applications, including Globally Distributed Applications.
• To set up a database and Sync Metadata Database:
Page 24 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
o Go to Azure portal – SQL databases.
o Go to the Hub database.
o Go to “Sync to other databases”.
o Go to “New Sync Group”, and select:
▪ Sync Group Name (not the database name),
▪ Sync Metadata Database (recommended that a new one is created),
▪ Automatic Sync (If on, choose from Seconds, Minutes, Hours or Days in Sync
Frequency),
▪ Conflict Resolution (Hub win or Member win),
▪ Use private link (a service managed private endpoint). If yes, you will later need to
approve the Private Endpoint Connection.
• To add sync members:
o Open the Sync Group – Databases.
o Click on “Add an Azure Database” or “Add an On-Premises Database”.
• For Azure Database, select:
o Sync Member Name (not the database name),
o Subscription,
o Azure SQL Server and Database,
o Sync Directions (To the Hub, From the Hub, or Bi-directional Sync),
o Existing Username and Password for the member database,
o Use private link.
• For on-prem SQL Server database:
o Select “Choose the Sync Agent Gateway”.
o Select “Existing agent” or “Create a new agent”. If new:
▪ Download the “Azure SQL Data Sync Agent”,
▪ Enter an Agent Name,
▪ Select “Create and Generate Key”, and copy it to the clipboard, then click OK.
• On the on-prem SQL Server:
o Run the Client Sync Agent app.
o Click “Submit Agent Key”.
o In the “Sync Metadata Database Configuration”, enter credentials for the metadata database
server.
▪ If automatically created, this will be the same server as the hub database.
Page 25 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
▪ You may need a firewall rule, created in the portal or in SSMS.
o Click Register.
o In the “SQL Server Configuration” box, connect using SQL Server or Windows authentication.
o Click Test connection and Save.
• In the Portal, in “Configure On-Premises page”:
o Select “Select the Database”.
o Provide a name for the new sync member (not the database name) and the Sync Directions.
• To see if it works, go to the Database Sync Group page – Tables, and click on Refresh schema.
o It may take a while for the data to be refreshed.
22. Implement a migration between Azure SQL services
• Migrating from SQL on Azure Virtual Machine to Azure SQL Database or Azure SQL Managed Instance
• Exactly the same as migrating from SQL On Prem.
• Migrating from Azure SQL Database or Azure SQL Managed Instance
• You can export the data:
• Right-hand click on the database in SSMS.
• Go to Tasks – Export Data. This will open the SQL Server Import and Export Wizard
(using SSIS).
• Select your source and destination.
• Use the Data Source of SQL Server Native Client 11.0.
• This will copy data, but not views, stored procedures, functions etc.
• Using SSMS Data-tier applications (DAC):
• Right hand click on the source database in SSMS and go to Tasks – Export Data-tier
Application.
• This will create a bacpac (back-up package), an archive containing schema and data.
• Afterwards, go to the destination server in SSMS, right-hand click on the word
"Databases" (not a particular database), and select Tasks – Import Data-tier
Application.
• Using the SQLPackage CLI utility, which also creates a bacpac.
• Open a Command Prompt, and run:
• cd C:\Program Files\Microsoft SQL Server\160\DAC\bin
• The number might need to be changed, depending on your view
of SQL Server
• sqlpackage.exe /a:Export
/SourceServerName:servername.database.windows.net
Page 26 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
/SourceDatabaseName:dbname /SourceUser:username
/SourcePassword:password
/TargetFile:C:\Users\user\Desktop\backup150.bacpac
• Details need to be filled in.
• Then to upload it, assuming you are still in the Command Prompt, run:
• sqlpackage.exe /a:Import
/TargetServerName:ManagedInstancename.appname.database.windows.n
et /TargetDatabaseName:dbname /TargetUser:username
/TargetPassword:password
/SourceFile:C:\Users\user\Desktop\backup150.bacpac
• Using the Azure Portal
• Not supported for exporting from or importing into MI.
• Uses Bacpac.
• Go to the database, and click on Export Database.
• Select a previously created Standard Storage.
• You can check the export status by going to the Azure SQL Server (not the database)
and go to Import/Export history.
• After it has exported, you can then use this for importing into MI using SSMS, or
create a new Azure SQL Database using the Azure Portal.
• Go to the Azure SQL Server (not the database), and click Import database.
• You can also use PowerShell
• To export, you can use the New-AzSqlDatabaseExport cmdlet
• $exportRequest = New-AzSqlDatabaseExport -ResourceGroupName
$ResourceGroupName -ServerName $ServerName `
• -DatabaseName $DatabaseName -StorageKeytype $StorageKeytype -
StorageKey $StorageKey -StorageUri $BacpacUri `
• -AdministratorLogin $creds.UserName -AdministratorLoginPassword
$creds.Password
• To import, you use the New-AzSqlDatabaseImport cmdlet:
• $importRequest = New-AzSqlDatabaseImport -ResourceGroupName
"<resourceGroupName>" `
• -ServerName "<serverName>" -DatabaseName "<databaseName>" `
• -DatabaseMaxSizeBytes "<databaseSizeInBytes>" -StorageKeyType
"StorageAccessKey" `
• -StorageKey $(Get-AzStorageAccountKey `
Page 27 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement Data Platform Resources
• -ResourceGroupName "<resourceGroupName>" -
StorageAccountName "<storageAccountName>").Value[0] `
• -StorageUri
"https://myStorageAccount.blob.core.windows.net/importsample/sample.
bacpac" `
• -Edition "Standard" -ServiceObjectiveName "P6" `
• -AdministratorLogin "<userId>" `
• -AdministratorLoginPassword $(ConvertTo-SecureString -String
"<password>" -AsPlainText -Force)
• You can also import use Azure CLI.
• Use az sql db import:
• # get the storage account key
• az storage account keys list --resource-group "<resourceGroup>" --account-
name "<storageAccount>"
• az sql db import --resource-group "<resourceGroup>" --server "<server>" --
name "<database>" `
• --storage-key-type "StorageAccessKey" --storage-key
"<storageAccountKey>" `
• --storage-uri
"https://myStorageAccount.blob.core.windows.net/importsample/sample.
bacpac" `
• -u "<userId>" -p "<password>"
22a. implement Azure SQL Managed Instance database copy and move
• You can copy or move one or more databases from one Azure SQL Managed Instance to another(s).
This is useful when:
o You want to manage database size and performance.
o Balance workloads and resources across several Managed Instances.
o Copy/move databases between development, test, and production environments.
o Combine databases from multiple instances.
• You are able to copy/move one or many databases from one Managed Instance to one or many
Managed Instances, without possibility of data loss.
o If you copy a database, the original remains online. There is no further synchronization, and
the databases on both instances are able to be read and written to.
o If you move a database, the original is dropped afterwards.
• You need read permissions from the source and write permissions for both source and the
destination databases.
Page 28 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
• There must also be sufficient network connectivity between the two Managed Instances.
• To copy/move:
o Go to the Managed Instance.
o Go to Data management – Databases.
o Select the database(s), and click Copy or Move at the top.
o In the Source details pane, select the source database(s) and Managed Instance.
o In the Destination details pane, select the destination Managed Instance.
o Click Review + Start, and then click Start.
• While the copy/move is happening:
o You will see “Copy/Move in progress” in the “Operation details” column.
o You can cancel it.
• After the data is transferred, the status changed to “Copy/Move ready for completion”.
• To complete the copy/move, click “Ready for completion” and “Complete”.
o If it is not completed in 24 hours, then the copy/move is cancelled, and the destination
database is dropped.
Implement a Secure Environment
23. configure authentication by using Active Directory and Entra ID
• Azure SQL Database supports:
o SQL Server authentication (user name and password, sent in plain text), and
o Microsoft Entra ID (previously known as Azure Active Directory or AAD) authentication.
▪ Microsoft Entra ID can sync with on-prem Windows Server AD.
• Azure AD authentication supports:
o Cloud-only identities,
o Hybrid identities that support cloud authentication with Single Sign-On (SSO), using
password hash or pass-through authentication.
o Hybrid identities that support federated authentication.
• Decision tree:
o Cloud-only identities
▪ Azure AD to handle sign-in completely in the cloud
▪ Do not want to enforced AD security policies during sign-in.
▪ Do not have a sign-in requirement not natively supported by Azure AD.
o Federated authentication
Page 29 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
▪ If you want to integrate with an existing federation provider, or
▪ Have a sign-in requirement not natively supported by Azure AD.
o Pass-through authentication
▪ All other cases
- Do not have a sign-in requirement not natively supported by Azure AD.
- No integration with an existing federation provider, OR want to enforce
user-level AD security policies during sign-in.
• Other authentications:
o Apps running on an Azure VM – passwordless authentication.
o Apps running on a non-Azure machine that is domain-joined: use managed identities
o Apps running on a non-Azure machine that is not domain-joined: use certificate
o Admin tools on a non-Azure machine that is not domain-joined: use Azure AD integrated
authentication, or Azure AD interactive authentication with multifactor authentication.
o Older apps where you can't change the connection string: SQL authentication.
• Microsoft Entra ID can allow additional security such as Multi-Factor Authentication (MFA).
o Go to the Azure Portal – Active Directory – (The relevant active directory, if more than one),
and Authentication methods. These include:
▪ FIDO2 (Hardware) Security Key
▪ Microsoft Authenticator (Phone App)
▪ Text message and
▪ Temporary Access Pass
• 23. configure Azure AD authentication
• To add a new user:
o In Azure portals – go to Microsoft Entra ID.
o Go to Users – New user.
o Enter:
▪ Name
▪ User name (similar to an email address), either @[DomainName].onmicrosoft.com,
or a custom domain name.
▪ Groups (Optional).
▪ Azure AD role (Optional),
▪ Job info (Optional).
o Azure will give you an autogenerated password in Password.
Page 30 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
o Click Create.
o You can delete a user from the same place.
24. create users from Entra ID identities
• You should create a second admin account as an Azure AD account, with the db_owner database
roles.
• This is for Azure SQL Managed Instance.
o Azure SQL Databases use a separate login, e.g.:
▪ CREATE LOGIN MyLogin
▪ WITH PASSWORD = 'MyComplexPassword';
▪ CREATE USER MyLogin FOR LOGIN MyLogin;
▪ GO
o Logins can:
▪ Do SQL Agent management and jobs execution,
▪ Database backup and restore operations,
▪ Auditing,
▪ Trigger logon triggers, and
▪ Setup Service Brokers and DB mail.
• Database users cannot be created using the Azure Portal.
• However, you can create logins from Azure AD users, groups or apps.
◼ Syntax for Azure SQL Managed Instance
▪ CREATE LOGIN login_name [FROM EXTERNAL PROVIDER] { WITH <option_list> [,..]}
o The parameters are:
▪ login_name is an existing Azure AD UserPrincipalName of the user, DisplayName
group or app when used with the “FROM EXTERNAL PROVIDER” indicates Azure AD
Authentication.
▪ <option_list> ::=
- PASSWORD = {'password’} – this cannot be used when FROM EXTERNAL
PROVIDER is used.
- | SID = sid
- | DEFAULT_DATABASE = database
- | DEFAULT_LANGUAGE = language
• Create user:
o CREATE USER user_name
Page 31 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
▪ [ { FOR | FROM } LOGIN login_name ]
▪ | FROM EXTERNAL PROVIDER
▪ [ WITH <limited_options_list> [ ,... ] ]
o [;]
o <limited_options_list> ::=
▪ DEFAULT_SCHEMA = schema_name
▪ | DEFAULT_LANGUAGE = { NONE | lcid | language name | language alias }
▪ | ALLOW_ENCRYPTED_VALUE_MODIFICATIONS = [ ON | OFF ] ]
• Both SQL Server Administrators and Microsoft Entra ID Administrators for SQL Server can create:
o Users based on SQL Server Authentication logins.
o Contained database users based on SQL Server Authentication (without logins)
• Microsoft Entra ID Administrators for SQL Server only can create:
o Contained database users based on Azure AD users and groups
• create SQL Server logins
• You cannot create an SQL Server login from the Azure portal.
o But you can create an Azure AD admin.
• Create a login using SSMS (Managed Instance only):
o [Name of server] – Security – New – Login…
o Enter the user name in the Login name, or click Search…
o Select type of authentication:
▪ Windows authentication,
▪ SQL authentication (you will need a password if so),
▪ “Microsoft Entra Universal with MFA [Multi-Factor Authentication] support”
- Strong verification.
▪ “Microsoft Entra Password”
- Uses identities in Azure AD. You can use it when your computer is logged
into Windows but it is not federated with Azure.
▪ “Microsoft Entra Integrated”
- When connecting from a Windows which is a federated domain.
▪ Special purpose logins, which cannot connect to SQL Server, but which can own
objects and have permissions:
- “SQL user without login” (no password),
- “Mapped to [stand-alone security] certificate”,
Page 32 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
- “Mapped to [stand-alone] asymmetric key”,
- “Mapped to [security] Credential”.
o Select Default database (Master is the default), and
o Select Default language.
• Create a user using SSMS (Managed Instance and Azure SQL Database):
o [Name of server] – Security – New – User…
o Select one of the following user types:
▪ “SQL user with login”
- Used when a person outside of your organization connects.
▪ “SQL user with password”. Also called a "contained database user". You can select
- User must change password at next login
- Enforce password expiration, and
- Enforce password policy.
- Used when a person outside of your organization connects.
▪ “SQL user without login” (no password),
- Can make your database more portable. Allowed in Azure SQL Database
and in a contained database in SQL Server.
- Cannot login but can be granted permissions.
▪ “Mapped to [stand-alone security] certificate”,
- Cannot login to a server, but can be granted permissions and can sign
modules
▪ “Mapped to [stand-alone] asymmetric key”,
- Cannot login to a server, but can be granted permissions and can sign
modules
▪ “Windows user”.
25. configure security principals
• Principal is that which receives permissions.
o Server-level: logins and server roles
o Database-level: users, database roles, application roles.
• Securables are that which can be secured.
o In a server, in a database, in a schema.
• Fixed Server-wide Login permissions (MI and SQL Server in VM only)
o sysadmin – any activity.
Page 33 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
o serveradmin – change server-wide configuration options and shut down the server.
o securityadmin – GRANT, DENY and REVOKE server-level permissions, and any database-level
permissions if they have access to the database.
▪ This allows them to assign most server permissions.
o processadmin – end processes.
o setupadmin – add/remove linked servers.
o bulkadmin – can run the BULK INSERT statement
o diskadmin – managing disk files.
o dbcreator – create/alter/drop/restore any database
o public – includes all users, group and roles. When you want the same permission(s) for
everyone.
• There are also Fixed Database Roles:
o db_owner – all configuration and most maintenance activities (in Azure SQL Database, some
activities require server-level permissions), including DROP database.
▪ However, if you give them db_denydatareader or DENY permissions, you can deny
read access to data.
o db_securityadmin – can modify role membership for custom roles only and manage
permissions. Can elevate own permissions.
o db_accessadmin – add/remove access to the database for logins and groups.
o db_backupoperator – can back up the database in MI or VM (not applicable in Azure SQL
Database, as BACKUP/RESTORE commands not available).
o db_ddladmin – run DDL command.
o db_[deny]datareader – [cannot] read all data from all user tables and views.
o db_[deny]datawriter - [cannot] add/delete/change data in all user tables.
• You can also add custom roles.
• In Azure SQL Databases, there are also two special database roles in the "master" database only:
o dbmanager – can create/delete databases. Connects as the dbo (database owner) user.
o loginmanager – create/delete logins in the "master" database (as per securityadmin server
role in on-prem SQL Server)
• You can use:
o sp_helprotect – returns user permissions for an object (or all objects) in the current
database.
▪ Does not list standard fixed server/database role permissions.
▪ Not available for Azure SQL Database.
o sp_helprole – lists the database roles.
Page 34 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
o sp_helprolemember – direct members of a role.
▪ Not available for Azure SQL Database.
• There are also role-based access control (RBAC), which are security rights outside of databases, which
include:
o SQL DB/Managed Instance/Server Contributor – manage SQL Databases, Mis or Servers, but
not get access to them. Cannot manage security-related policies.
o SQL Security Manager – mange security-related policies for servers and databases, but no
access to them.
• When deploying, Azure uses the "server admin", which is a principal in Azure SQL Database, and a
member of the sysadmin role in MI.
26. configure database and object-level permissions using graphical tools
• This is for MI and VM (not Azure SQL Database).
• In a particular login:
o Click Search.
o Select:
▪ “The server”,
▪ “Specific objects”. If so, click “Object Types” and select Endpoints, Logins, Servers,
Availability Groups and/or Server roles.
▪ “All objects of the types” – select Endpoints, Logins, Servers, Availability Groups
and/or Server roles.
• Objects that can be secured include:
o Server
▪ Availability group, Endpoint, Login, Server role and Database
o Database
▪ Application role, Assembly, Asymmetric key, Certificate, Contract, Fulltext catalog,
Fulltext stoplist, Message type, Remote Service Binding, (Database) Role, Route,
Schema, Search property list, Service, Symmetric key, User
o Schema
▪ Type
▪ XML schema collection
▪ Object: Aggregate, Function, Procedure, Queue, Synonym, Table, View, External
Table
Page 35 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
27. apply principle of least privilege for all securables
• Users should have the least privilege that is necessary for them to do their job.
o This is called the Least-privileged User Account (LUA).
• You can use Roles to assigned permissions to roles, and then users to roles.
o This makes security administration more easy.
• You can use the following permissions:
o GRANT
o REVOKE (the opposite of GRANT)
▪ Why use REVOKE instead of GRANT? It doesn’t give permissions, but it doesn’t stop
permissions if they have it through another role.
o DENY (this overrides a GRANT).
▪ DENY does not apply to sysadmin members or object owners.
▪ If DENY is applied to the public role, no non-sysadmin will have this permission.
• You can also prevent users from querying objects directly by allowing only access to procedures or
functions.
• Objects are chained together.
o If two objects have the same owner, then permissions in a second object called from the first
are not separately checked.
• You should still reduce the number of objects affected by permissions.
o SELECT permission in a database includes all (child) schemas, and the tables and views.
o SELECT permissions on a schema includes all of the tables and views.
o SELECT on a table gives SELECT permission only.
o CONTROL gives ownership-like permissions and includes all other permissions, including
ALTER, SELECT, INSERT, UPDATE.
29. manage certificates
• To create a self-signed certificate:
o CREATE CERTIFICATE CertificateName
▪ ENCRYPTION BY PASSWORD = ‘ComplicatedPassword’
- If this is not used, the private key is encrypted using the database master
key.
▪ WITH SUBJECT = ‘CertificateSubjectName’,
- This is a field in the certificate metadata.
▪ EXPIRY_DATE = ‘20291231’;
Page 36 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
- You can also have a START_DATE (in UTC). If not specified, START_DATE
defaults to current date, and EXPIRY_DATE (UTC) is one year after
START_DATE.
o GO
• By default, this certificate is stored in the master database.
o The Azure Key Vault can store customer-managed certificates ("Bring your own Key – BYOK")
• To restore a previously-created certificate, you can also use CREATE CERTIFICATE with FILE = 'path'
o Azure SQL Database does not support creating a certificate from a file or using private key
files.
• You can also ALTER CERTIFICATE
o ALTER CERTIFICATE CertificateName
o WITH PRIVATE KEY (ENCRYPTION BY PASSWORD = ‘ComplicatedPassword’)
o You can change the password, but not the SUBJECT or DATEs.
• You can also DROP CERTIFICATE.
29. manage security principals
• (Use “GO” after each statement).
• For MI and SQL Server in VMs:
o To create a login for a local Microsoft Entra ID account:
▪ CREATE LOGIN [login_name] FROM EXTERNAL PROVIDER -- the last 3 words indicate
Azure AD.
o To check
▪ SELECT * FROM sys.server_principals;
o To add members into a server role:
▪ ALTER SERVER ROLE server_role ADD MEMBER [login_name]
o To create a user:
▪ USE <Database Name>
▪ CREATE USER [user_name] FROM LOGIN [login_name]
• For Azure SQL Database:
o To create a user based on a local Microsoft Entra ID account:
▪ USE <Database Name>
▪ CREATE USER [user_name] FROM EXTERNAL PROVIDER
o You can create users in the master database, then create a user based on it, but it is better
practice to do the above:
Page 37 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
▪ [In Master]
CREATE LOGIN demo WITH PASSWORD = 'Pa55.w.rd'
- To check
o SELECT * FROM sys.sql_logins
▪ [In database]
CREATE USER demo FROM LOGIN demo
• To check users:
o SELECT * FROM sys.database_principals
• (Use “GO” after each statement).
• To grant permissions:
o AUTHORIZATION PERMISSION ON SECURABLE::NAME TO PRINCIPAL;
▪ For example:
GRANT SELECT ON OBJECT::Region TO Ted [WITH GRANT OPTION];
o AUTHORIZATION can be GRANT, REVOKE or DENY.
▪ REVOKE is the opposite of a GRANT.
▪ DENY beats all GRANTs from other roles.
o PERMISSION can be
▪ For tables and views, SELECT, INSERT, UPDATE and DELETE.
- They can also be CONTROL (all rights), REFERENCES (view foreign keys),
TAKE OWNERSHIP, VIEW CHANGE TRACKING and VIEW DEFINITION.
▪ For schema, ALTER permission on a schema is wide-ranging. You can alter, create or
drop any securable in that schema. However, you cannot change ownership.
▪ For functions and stored procedures, ALTER (change definition), CONTROL,
EXECUTE, VIEW CHANGE TRACKING and VIEW DEFINITION.
▪ You can give permissions on a stored procedure/function without giving
permissions on the underlying tables/views through ownership chaining (see topic
27).
▪ ALL (deprecated, maintained for backward compatibility)
- For databases, that means BACKUP DATABASE and LOG, CREATE
DATABASE, FUNCTION, PROCEDURE, RULE, TABLE, and VIEW (note – not
DROP or ALTER).
- For tables and views, ALL means DELETE, INSERT, REFERENCES, SELECT, and
UPDATE.
- For procedures, ALL means EXECUTE.
- For scalar functions, ALL means EXECUTE and REFERENCES.
Page 38 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
- For table-valued functions, ALL means DELETE, INSERT, REFERENCES,
SELECT and UPDATE
o SECURABLE is OBJECT::, SCHEMA:: or DATABASE:: or SERVER::
o PRINCIPAL is a login, user or role.
o The optional [WITH GRANT OPTION] allows you to grant that permission to others.
• To check permissions:
o SELECT * FROM sys.fn_my_permissions (NULL, 'DATABASE') -- login permissions.
o SELECT * FROM sys.fn_my_permissions ('MyMITestDB','DATABASE') – user permissions.
• To test user permissions:
o CREATE PROCEDURE proc_name WITH EXECUTE AS user_name AS …
o or if sysadmin in MI or VM:
▪ EXECUTE AS LOGIN = ‘login_name’
o or EXECUTE AS USER = 'user_name'
29. configure permissions for users to access database objects
• To add permissions to access database objects:
o AUTHORIZATION PERMISSION ON SECURABLE::NAME TO PRINCIPAL;
o For example:
▪ GRANT SELECT ON OBJECT::Region TO Phillip;
▪ GRANT SELECT ON OBJECT::Customer(CustomerName) TO Phillip; -- This is a
column.
o AUTHORIZATION can be GRANT, REVOKE or DENY.
o PERMISSION is:
▪ CONTROL (ownership-like capabilities).
▪ ALTER – allows for ALTER, CREATE and DROP.
▪ ALTER ANY [Server_Securable] – CREATE, ALTER and DROP things such as LOGIN.
▪ DELETE/INSERT/SELECT/UPDATE
▪ TAKE OWNERSHIP – allows grantee to take ownership, but doesn’t automatically
take it.
▪ IMPERSONATE Login/User – allows principal to impersonate, but doesn’t
automatically do it.
▪ CREATE Server-/Database-/Schema-Securable.
▪ VIEW DEFINITION – access to metadata.
▪ REFERENCES – permission to create a FOREIGN KEY constraint.
Page 39 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
o OBJECT can be a database, schema or object
o PRINCIPAL is a login, user or role.
• Check permissions using:
o SELECT * FROM fn_builtin_permissions(default);
▪ All permissions
o SELECT * FROM fn_builtin_permissions('assembly’);
▪ Specific database.
o SELECT * FROM fn_my_permissions('Orders55', 'object’);
▪ Specific object for a specific role.
o SELECT * FROM sys.database_permissions WHERE major_id = OBJECT_ID('Yttrium');
▪ Specific object.
29. configure permissions by using custom roles
• To create a role (database-level securable):
o CREATE ROLE role_name [ AUTHORIZATION owner_name ]
o If AUTHORIZATION is not given, it will be the current user.
• To add a user to a database-level role:
o USE <Database Name>
o ALTER ROLE db_datareader ADD MEMBER database_principal
o You can also use DROP MEMBER instead of ADD MEMBER.
o database_principal is a database user or user-defined role, but not a fixed database role or a
server principal.
o You need ALTER permission on the role, or ALTER ANY ROLE on the database, or
db_securityadmin or db_owner.
• To alter the name of the role:
o ALTER ROLE OriginalRoleName WITH NAME = NewRoleName;
30. implement Transparent Data Encryption (TDE)
• TDE (de-)encrypts data at the page level at rest. It is encrypted when written, and de-encrypted at
read.
o Don't confuse this with TLS – transparent layer security – which encrypts when in transit.
• It uses a symmetric Database Encryption Key (DEK).
o You can Bring Your Own Key (BYOK).
• It is protected by the TDE protector, using a service-managed certificate or an asymmetric key in the
Azure Key Vault.
Page 40 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
o For Azure SQL Database, it is set at the server level. New databases are encrypted by default
(but not ones created through restore or database copy).
o For Azure SQL Managed Instance, it is set at the instance level and is inherited to all
encrypted databases.
o It cannot be used to encrypted system databases.
• To enable it in Azure SQL Database only, go to the Azure Portal, then the relevant database, then go
to “Transparent data encryption” and set “Data encryption” to ON.
o You cannot do this in Azure SQL Managed Instance.
• In T-SQL, you can use:
o ALTER DATABASE DatabaseName SET ENCRYPTION ON/OFF.
o This can be used in Azure SQL Managed Instance.
o However, you can’t switch the TDE protector to a key in Key Vault in T-SQL.
• You can also use PowerShell.
o Set-AzSqlServerTransparentDataEncryptionProtector
▪ Change to ServiceManaged or Azure Key Vault
o Add-AzSqlServerKeyVaultKey
▪ Adds a Key Vault key to a SQL server
o Set-AzSqlDatabaseTransparentDataEncryption
▪ Modifies TDE property for a database.
• You can also use REST API.
31. implement object-level encryption
• See topic 33.
32. configure server and database-level firewall rules
• By default, all connections to the server and database are rejected.
o SQL Database communicates over port 1433. You need that opened on your own
computer/server.
• For the most secure connection:
o Set “Allow access to Azure services” to NO, then
o create a reserved IP (classic deployment) for the resource that needs to connect, then
o allow access through the IP address.
o A public IP address is required for each resource.
• What the difference?
o Server-level firewall rules are for users/apps to have access to all databases.
Page 41 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
o Database firewall rules are for an individual or app.
o Database rules are checked before server-level rules.
• To set up a server-level firewall rule:
o This applies to all databases in the server on Azure SQL Database only, whether single or
pooled databases. It does not apply to Azure SQL Managed Instance.
o You will need SQL Server Contributor or SQL Security Manager role, or the owner of the
resource that contains the Azure SQL Server.
o In Azure portal, go to your database.
o On the database Overview page, click “Set server firewall”.
o Select “Add client IP” to add your current IP address. This opens port 1433.
▪ A firewall rule of 0.0.0.0 enables all Azure services to bypass the server-level
firewall rule – but in the portal, you need to turn on "Allow Azure services and
resources to access this server" instead.
o Click OK. The rules are then stored in the master database.
• In T-SQL:
o To check the current server-level IP firewall rules:
▪ SELECT * FROM sys.firewall_rules
o To add a server-level IP firewall rule:
▪ EXECUTE sp_set_firewall_rule @name = N'MyFirewallRule',
▪ @start_ip_address = '192.168.1.1', @end_ip_address = '192.168.1.200'
o To delete a server-level IP firewall rule:
▪ EXECUTE sp_delete_firewall_rule @name = N'MyFirewallRule'
• You can also manage using PowerShell, CLI (Command Line Interface) or REST API.
• To set up a database firewall rule.
o It can only be done using T-SQL statements, and you need CONTROL DATABASE permission
at the database level.
o You need to have set up a server-level firewall rule first.
o Run a query such as:
▪ EXECUTE sp_set_database_firewall_rule N'Example DB Rule','0.0.0.4','0.0.0.4’;
o This rule is stored in that individual database.
• In T-SQL:
o To check the current database-level IP firewall rules:
▪ SELECT * FROM sys.database_firewall_rules
o To add a database-level IP firewall rule:
Page 42 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
▪ EXECUTE sp_set_database_firewall_rule @name = N'MyDatabaseFirewallRule',
▪ @start_ip_address = '192.168.1.1', @end_ip_address = '192.168.1.200'
o To delete a database-level IP firewall rule:
▪ EXECUTE sp_delete_database_firewall_rule @name = N'MyDatabaseFirewallRule'
33. implement Always Encrypted
• You can encrypt sensitive data using Always Encrypted in Azure SQL Database and MI.
• If you wish to use an Azure Key Vault, then you need to create it first
o Use the Azure Portal – Key Vault to create it.
▪ You need the following permissions:
▪ Cryptographic Operations: Decrypt, Encrypt, Unwrap Key, Wrap Key, Verify and
Sign.
▪ Key Management Operations: create, get, list.
o It costs $0.03 for 10,000 transactions. The Premium version allows for a Hardware Security
Module (HSM).
• To encrypt columns in SSMS:
o Go to Databases – NameOfDatabase – Tables – NameOfTable.
o Right-hand click and go to “Encrypt Columns…”.
o Select the columns and choose “Encryption Table”, either Deterministic or Randomized.
▪ Deterministic requires the string to be in a _Bin2 collation (e.g.
Latin1_General_BIN2).
▪ Deterministic allows equality joins, GROUP BY, indexes and DISTINCT. Randomized
prevents this.
o In “Master Key Configuration”, you can go to “Select an Azure Key Vault” and select the Key
Vault.
o The next three stages are Validation, Summary and Results.
• When the columns are encrypted, then when connecting, go to the “Additional Connection
Parameters” tab, and enter:
o Column Encryption Setting=enabled
• Database Permissions are:
o ALTER ANY COLUMN MASTER KEY
▪ Needed to create/delete a column master key.
o ALTER ANY COLUMN ENCRYPTION KEY
▪ Needed to create/delete a column encryption key.
o VIEW ANY COLUMN MASTER/ENCRYPTION KEY DEFINITION
Page 43 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
▪ Needed to access/read the metadata of the column master/encryption keys to
manage keys or query encrypted columns.
o Use GRANT VIEW ANY COLUMN MASTER KEY DEFINITION TO NameOfUser
• Do you need role separation?
o Security Administrator generates columns encryption keys and column master keys.
▪ Needs access to the keys and the key store, but not the database.
o Database Administrator (DBA) manages metadata about the keys in the database.
▪ Does not need access to the keys or the key store.
o Should they be different people?
▪ If not, you can use either SSMS or PowerShell.
▪ If so, then you can only use PowerShell.
• For the Security Administrator
o # Create a column master key in Windows Certificate Store.
▪ $storeLocation = "CurrentUser"
▪ $certPath = "Cert:" + $storeLocation + "\My"
▪ $cert = New-SelfSignedCertificate -Subject "AlwaysEncryptedCert" -
CertStoreLocation $certPath -KeyExportPolicy Exportable -Type
DocumentEncryptionCert -KeyUsage DataEncipherment -KeySpec KeyExchange
o # Import the SqlServer module
▪ Import-Module "SqlServer"
o # Create a SqlColumnMasterKeySettings object for your column master key.
▪ $cmkSettings = New-SqlCertificateStoreColumnMasterKeySettings -
CertificateStoreLocation "CurrentUser" -Thumbprint $cert.Thumbprint
o # Generate a column encryption key, encrypt it with the column master key to produce an
encrypted value of the column encryption key.
▪ $encryptedValue = New-SqlColumnEncryptionKeyEncryptedValue -
TargetColumnMasterKeySettings $cmkSettings
o # Share the location of the column master key and an encrypted value of the column
encryption key with a DBA, via a CSV file on a share drive
▪ $keyDataFile = "Z:\keydata.txt"
▪ "KeyStoreProviderName, KeyPath, EncryptedValue" > $keyDataFile
▪ $cmkSettings.KeyStoreProviderName + ", " + $cmkSettings.KeyPath + ", " +
$encryptedValue >> $keyDataFile
o # Read the key data back to verify
▪ $keyData = Import-Csv $keyDataFile
Page 44 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
▪ $keyData.KeyStoreProviderName
▪ $keyData.KeyPath
▪ $keyData.EncryptedValue
• For the DBA
o # Obtain the location of the column master key and the encrypted value of the column
encryption key from your Security Administrator, via a CSV file on a share drive.
o $keyDataFile = "Z:\keydata.txt"
o $keyData = Import-Csv $keyDataFile
o # Import the SqlServer module
o Import-Module "SqlServer"
o # Connect to your database.
o $serverName = "<server name>"
o $databaseName = "<database name>"
o $connStr = "Server = " + $serverName + "; Database = " + $databaseName + "; Integrated
Security = True"
o $database = Get-SqlDatabase -ConnectionString $connStr
o # Create a SqlColumnMasterKeySettings object for your column master key.
o $cmkSettings = New-SqlColumnMasterKeySettings -KeyStoreProviderName
$keyData.KeyStoreProviderName -KeyPath $keyData.KeyPath
o # Create column master key metadata in the database.
o $cmkName = "CMK1"
o New-SqlColumnMasterKey -Name $cmkName -InputObject $database -
ColumnMasterKeySettings $cmkSettings
o # Generate a column encryption key, encrypt it with the column master key and create
column encryption key metadata in the database.
o $cekName = "CEK1"
o New-SqlColumnEncryptionKey -Name $cekName -InputObject $database -ColumnMasterKey
$cmkName -EncryptedValue $keyData.EncryptedValue
Page 45 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
33a. implement Always Encrypted with VBS enclaves
• You can expand Always Encrypted with secure enclaves.
o An enclave is something within a bigger something, such as some territory inside bigger
territory.
o You can use this in SQL Server 2019 or later, or Azure SQL Database.
• Always Encrypted protects sensitive data from malware and users who should have access to the
database but not the data by encrypting it on the client, not allowing it to be in plaintext in the
Database Engine.
• However:
o because the data is encrypted, you can only do comparison based on values being the same
(or not), if you are using deterministic encryption.
o you cannot do data encryption, key rotation, or pattern matching in the database.
• To solve this problem, you can use Always Encrypted with secure enclaves. This create a protected
part of the memory, which can do computations on plaintext data in the secure enclave.
o It’s like a black box - You cannot view the data or code inside the enclave, even if you used a
debugging system.
• You need to use either:
o Intel Software Guard Extensions (Intel SGX) enclaves.
▪ You would need an Azure SQL Database using a DC-series.
Page 46 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
▪ This also requires Microsoft Azure Attestation with an Attestation administrator,
which verifies the trustworthiness of the Azure SQL Database, together with an
attestation provider. However, this is not required for the DP-300 exam.
o Virtualization-based Security (VBS) enclave.
▪ This is available for all versions of Azure SQL Database, including Elastic Pools, or
SQL Server 2019 or later.
▪ Microsoft Azure Attestation is not needed.
▪ Currently it is not available in Jio India Central.
▪ It provides some additional protection against OS-level threats. You also have Azure
protection, such as just-in-time-access, multifactor authentication, and secure
monitoring.
▪ However, VBS enclave cannot defend itself from bigger attacks, such as replacing
the enclave program with malware, so if you need strong security isolation, you may
wish to consider the Intel SGX enclave instead.
• To enable VBS enclave:
o When creating the database or elastic pool:
▪ go to the Security tab.
▪ in the Always Encrypted section, set “Enable secure enclaves” to On.
o For an existing database or elastic pool:
▪ Go to Security – Data Encryption (for databases) or Configuration (for elastic pools).
▪ In the Always Encrypted tabs, set “Enable secure enclaves” to On.
• Please note – you cannot switch it Off after it has been On.
• You can also enable it in SSMS by right-hand clicking on the database, select Properties, and change
“Enable Secure Enclaves” to On.
• You can also enable it in Azure PowerShell or by using Azure CLI.
34. implement Dynamic Data Masking
• Dynamic Data Masking is for both Azure SQL Database and Azure SQL Managed Instance.
o It prevents access to sensitive data by putting a mask, with none or part of the data (e.g. last
4 digits of a credit card).
o This encrypts the column (column-level encryption – CLE).
• To implement Dynamic Data Masking:
o In the Azure Portal, go to the Database, then go to Dynamic Data Masking.
o You may see recommended fields to mask.
▪ If so, you can click on “Add mask” (and then Save).
o To create a custom rule, click “Add mask”.
Page 47 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
o You can select the Schema, Table and Column to define the columns for masking.
o You can select the mask to be displayed:
▪ Default value (0, xxxx, 01-01-1900),
- XXXX for string data types. You can use fewer Xs if it less than 4 characters.
- Use 0 for numeric data types.
- Use 01-01-19000 for date and time data types.
▪ Credit card value (xxxx-xxxx-xxxx-1234),
- Exposes the last 4 digits of the credit card, with a constant string prefix.
▪ Email ([email protected]),
- Exposes the first letter, but replaces everything else with a constant string
prefix.
▪ Number (random number range),
- A random number between two boundaries.
▪ Custom string (prefix [padding] suffix).
- Shows the first X characters, the last Y characters, and a custom padding
string in the middle.
o Click “Add” to save this rule.
▪ You can also “Discard” changes and “Delete” the mask.
o In T-SQL, this is done by using:
▪ ALTER TABLE Schema.Table ALTER COLUMN ColumnName
▪ ADD MASKED WITH (FUNCTION = 'partial(1, "xxxxx", 1)') – or 'email()' or 'random(1,
1000)' or 'default()'
• You can select specific SQL users who were excluded from masking.
o Multiple users are separated by semicolons.
o Note: Administrators are always excluded for Dynamic Data Masking – they can always read
the data.
o In T-SQL, this is done by using:
▪ GRANT UNMASK to MyCustomRole;
▪ GRANT UNMASK to MyUser;
▪ REVOKE UNMASK to MyUser;
34. implement Azure Key Vault and disk encryption for Azure VMs
• To encrypt disks for Azure VMs:
o In the Azure Portal, go to the VM.
Page 48 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
o Select Disks (left-hand side),
o Select Additional Settings (at the top).
o In “Encryption settings – Disks to encrypt”, select “OS and data disks”.
o Then click “Select a key vault and key for encryption”.
o Next to “Select key from Azure Key Value: Key vault”, select “Create new”.
o Add a name (unique amongst Azure Key Vaults) and Resource Group.
o Go to the “Access Policies” tab, click “Enable Access to: Azure Disk Encryption for volume
encryption”.
o After creating the Key Vault, leave the Key field blank, click Select, and Save.
35. Configure Transport Layer Security (TLS)
• Transport Layer Security (TLS) seamlessly encrypts data between SQL Server and a client (such as
yourself).
• Packages of data are encrypted from one side and then decrypted by the other side.
• Each version has more properties
• TLS 1.0 was defined in January 1999, and TLS 1.1 was defined in April 2006.
• It was widely deprecated by web sites around the year 2020. Microsoft no longer supported
them in Microsoft Teams Desktop as of July 7, 2021.
• TLS 1.2 was defined in August 2008, with stronger SHA-256 encryption, improved reliability and better
performance.
• This is the most commonly used TLS version, and creates a secure connection.
• TLS 1.3 was defined in August 2018.
• It takes less time to connect.
• Why not use TLS 1.2 or later all the time?
• Some non-Microsoft drivers don't, by default, use TLS.
• To configure TLS:
• In the Azure portal, go to the SQL Server (not the database).
• If you are in the database, click on the server.
• Go to Security – Networking – Connectivity.
• Change the "Minimum" TLS version.
• Alternatively, you can do it in PowerShell:
• $SecureString = ConvertTo-SecureString "password" -AsPlainText -Force
• Set-AzSqlServer -ServerName sql-server-name -ResourceGroupName sql-server-
group -SqlAdministratorPassword $SecureString -MinimalTlsVersion "1.2"
• or Azure CLI:
Page 49 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
• az sql server update -n sql-server-name -g sql-server-group --set
minimalTlsVersion="1.2"
36. apply a data classification strategy
• Sensitive data includes:
o Data privacy, regulatory and national requirements (e.g. GDPR),
o Security scenarios, including controlling access.
• To apply a data classification strategy:
o Go to the database.
o Under Security, go to “Data Discovery & Classification”.
o Go to the Classification tab.
o At the bottom of the screen, you may have “X columns with classification
recommendations”.
▪ Check whatever recommendations you want to accept/dismiss.
▪ Click “Accept” or “Dismiss selected recommendations”.
o To create a new classification:
▪ Click on “+ Add classification”.
▪ Select the Schema, Table and Column name.
▪ Select the Information type:
- [n/a], Other
- Networking
- Personal data: Contact Info, Name, National ID, SSN, Health, Date of Birth,
- Credentials
- Financial records: Credit Card, Banking, Financial
▪ Select the Sensitivity Label:
- [n/a] – Data from your own personal life.
- Public – Freely available business data, or information that has been
released to the public.
- General – Business data not meant for the public, such as emails,
documents and files which do not include confidential data.
- Confidential or Confidential – GDPR, Highly Confidential or Highly
Confidential – GDPR – Business data that would cause harm or extensive
harm to your company if overshared.
▪ You cannot select [n/a] for both Information Type or Sensitivity Label.
▪ Then click “Add classification”.
Page 50 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
• The following roles can modify and read a database’s data classification:
o Owner,
o Contributor,
o SQL Security Manager.
• Additionally, the following roles can read (but not modify) a database’s data classification:
o Reader, and
o User Access Administrator.
• You can use Audit to drill down into "Security Insights", "Access to Sensitive Data" etc.
o You can also see it in Intelligent Insights.
• You can also use T-SQL, REST API or PowerShell to manage classifications.
• In T-SQL:
o To add a sensitivity classification:
o ADD SENSITIVITY CLASSIFICATION TO
o [schema.]table.column1[, schema.table.column2]… etc.
o WITH (
o LABEL='Highly Confidential’, -- you could also use LABEL_ID
o INFORMATION_TYPE='Financial’, -- you could also INFORMATION_TYPE_ID
▪ Networking, Contact Info, Credentials, Credit Card, Banking, Other, Name, National
IS, SSN, Health, Date of Birth
o RANK=NONE, LOW, MEDIUM, HIGH or CRITICAL
o )
o To check sensitivity classifications:
▪ SELECT * FROM sys.sensitivity_classifications
o To drop a sensitivity classification:
▪ DROP SENSITIVITY CLASSIFICATION FROM [schema.]table.column1[,
schema.table.column2 ]…
37. configure server and database audits
• You can use auditing to:
o Retain a trail of selected database actions,
o Report on database activities, using pre-configured reports and a dashboard.
o Analyse reports for suspicious events, unusual activity and trends.
• Notes:
o It is not supported for premium storage or hierarchical namespace.
Page 51 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
o Under high activity, Azure will prioritise other actions and may not record some audited
events.
o They are written to Append Blobs in Blob storage.
• Server policy audits:
o For all existing and newly created databases.
o Server policy audits always applies to the database, regardless of any database-level auditing
policies. They can sit side-by-side.
o Microsoft recommends using only server-level auditing, unless you want to audit different
event types/categories for a specific database.
o The default auditing policy includes:
▪ BATCH_COMPLETED_GROUP
- All queries and stored procedures,
▪ SUCCESSFUL_DATABASE_ and FAILED_DATABASE _AUTHENTICATION_GROUP
- Success and failed logins
o It stores 4,000 characters of data in an audit .
• To do this:
o Go to the Azure portal – NameOfServer or NameOfDatabase – Security – Auditing.
o Click “Enable Azure SQL Auditing” to track these events for a particular database or server.
You can select the details to be stored in:
▪ An existing or new Storage account
- The Advanced settings allow you to choose the retention period (the
default, zero days, is unlimited),
- This Advanced setting only applies to new audits.
▪ An existing Monitor Log Analytics workspace, and/or
▪ An existing Event Hub.
o If you are in the database, you can click on “View server settings”.
o If you are in the server, you can also audit Microsoft support operations.
• If you audit to an existing Monitor Log Analytics workspace:
o You can add it by:
▪ Creating an Azure Storage Container, going to Overview – Blobs, and “+Container”.
▪ Give the container a name, set the Public access level to Private and click OK.
▪ In the Properties, click on Properties and copy the URL for future use.
▪ Go to the Storage Account, and click on “Storage Settings – Shared access
signature”.
Page 52 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
▪ Add “Blob” to “Allowed services”, choose the Start date as yesterday (to avoid
timezone related problems), and an End date.
▪ Click “Generate SAS” and copy this token for future use. (are the highlighted
needed?)
o Choose the “Log Analytics” in the Auditing.
o You can “View audit logs”.
o You can then either:
▪ Click on “Log Analytics” to go to the workspace, or
▪ Click “View dashboard” to see a dashboard of audit logs.
• If you audit logs to the Event Hub, then:
o You would need to set up a stream to consume these events and write them to a target.
• If you audit logs to an Azure storage account, then:
o You can explore them in Azure Storage Explorer.
o You can click on “View audit logs”:
▪ Filter on specific dates,
▪ Look at Server or Database audit policy.
o You can use T-SQL:
▪ SELECT * FROM sys.fn_get_audit_file (‘NameOfFile.sqlaudit',default,default);
o You can use SSMS, going to File – Open – Merge Audit Files.
o You should change your storage keys periodically.
▪ In advanced properties, you can change to the secondary access storage key.
▪ Then you can go to your Storage Account – Settings – Access keys, and click the
regenerate icon on the primary access key.
▪ You can then go back to the audit, and change it to the primary key.
▪ You can then go to your Storage Account – Settings – Access keys, and click the
regenerate icon on the secondary access key.
38. implement data change tracking
• Change Tracking is supported in Azure SQL Database only.
o Change Tracking tracks whether the column was changed.
o However, it does not track how many times nor does it track historic data. Therefore, it more
lightweight and requires less storage than CDC (Change Data Capture).
o It therefore enables applications to determines which rows have changed, and request those
rows. (But you cannot see the previous data.)
Page 53 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
o The data is stored in an in-memory rowstore, and flushed on every checkpoint to the internal
data.
o You may wish to consider using snapshot isolation for the database, so that changes made
while getting the data are not visible within the transaction:
▪ ALTER DATABASE AdventureWorksLT
▪ SET ALLOW_SNAPSHOT_ISOLATION ON;
▪ SET TRANSACTION ISOLATION LEVEL SNAPSHOT;
▪ BEGIN TRAN / COMMIT TRAN
• To enable Data Change Tracking on a database:
o In SSMS
▪ Right-hand click on the database, select Properties and go to Change Tracking.
▪ Change “Change Tracking” to True.
▪ Select the Retention Period and Units (by default, 2 Days) – the minimum is 1
Minute; there is no maximum.
▪ Select whether data is “Auto Cleanup” in that retention period.
- If true, Change Tracking data will be removed periodically. If an App has
not got updated information in time, all data will needed to be refreshed.
- If False, change tracking information will not be removed and will continue
to grow.
o In T-SQL
▪ ALTER DATABASE MyDatabase
▪ SET CHANGE_TRACKING = ON
▪ (CHANGE_RETENTION = 2 DAYS, AUTO_CLEANUP = ON)
• However, you still need to enable it in a particular table.
• To enable Data Change Tracking on a table:
o In SSMS
▪ Right-hand click on the database, select Properties and go to Change Tracking.
▪ Change “Change Tracking” to True.
▪ If True, you can also change “Track Columns Updated” to True. This will indicate
whether UPDATEs to individual columns will be tracked.
o In T-SQL
▪ ALTER TABLE Schema.Table
▪ ENABLE CHANGE_TRACKING
▪ WITH (TRACK_COLUMNS_UPDATED = ON)
Page 54 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
• You can also disable Change Tracking on tables and databases
o However, to disable it on the database, all track changing of tables needs to be disabled first.
o In SSMS, change True to False.
o In T-SQL for tables:
▪ ALTER TABLE Schema.Table
▪ DISABLE CHANGE_TRACKING
o In T-SQL For databases:
▪ ALTER DATABASE MyDatabase
▪ SET CHANGE_TRACKING = OFF
• To check which tables/databases have Change Tracking enabled:
o SELECT * from sys.change_tracking_databases
o SELECT * from sys.change_tracking_tables -- this uses the current database. You need:
▪ SELECT permission on the primary key columns for the tables
▪ VIEW CHANGE TRACKING permission for the relevant table.
• To use it:
o To get the initial sync version
▪ declare @last_sync bigint;
▪ SET @last_sync = CHANGE_TRACKING_CURRENT_VERSION();
o After changes have happened:
▪ SELECT CT.ProductID, CT.SYS_CHANGE_OPERATION,
▪ CT.SYS_CHANGE_COLUMNS, CT.SYS_CHANGE_CONTEXT
▪ FROM CHANGETABLE(CHANGES Schema.Table, @last_sync) AS CT
o Check that you don’t have to refresh the entire table:
▪ IF (@last_sync < CHANGE_TRACKING_MIN_VALID_VERSION(
OBJECT_ID('Schema.Table'))
• Change Data Capture (CDC) is supported in Azure SQL Database, Azure SQL Managed Instance and
SQL Server on VM.
o CDC tracks historic data.
o Needs a minimum of 1 vCore or 100 DTUs or eDTUs.
o Cannot be used in Azure SQL Database Free, Basic or Standard tier Single Database (S0, S1,
S2).
o Cannot be used in Azure SQL Database Elastic Pool with vCore < 1 or eDTUs < 100.
• Before you can enable it for a table, you must switch it on for the database.
Page 55 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
o EXEC sys.sp_cdc_enable_db
▪ Returns 0 for Success, 1 for Failure.
▪ Only use it on user databases, not system databases.
▪ It creates the Change Data Capture objects, including metadata tables and DDL
triggers.
▪ You need sysadmin to run it.
• Then you can enable it for a table:
o EXEC sys.sp_cdc_enable_table . Some of the arguments are:
▪ @source_schema = N'HumanResources'
▪ , @source_name = N'Department'
- The name of the table.
▪ , @role_name = N'cdc_admin'
- The database role used to gate access to change data. Could be a new role.
▪ , @captured_column_list = N'DepartmentID, Name, GroupName'
- Columns to be captured. Needs the primary key.
- Cannot use encrypted columns.
▪ Returns 0 for Success, 1 for Failure.
• You can query your configuration using:
o EXECUTE sys.sp_cdc_help_change_data_capture
• You can view changed rows by using:
o DECLARE @from_lsn binary(10), @to_lsn binary(10);
o SET @from_lsn = sys.fn_cdc_get_min_lsn('HR_Department');
o SET @to_lsn = sys.fn_cdc_get_max_lsn();
o SELECT * FROM cdc.fn_cdc_get_all_changes_HR_Department (@from_lsn, @to_lsn, N'all');
▪ _$operation = 1 (delete), 2 (insert), 3 or 4 (update)
39. perform a vulnerability assessment
• Azure Defender for SQL costs around $15/server/month.
• In the Azure portal, go to the SQL database.
• Go to Security – Security Center.
• Next to “Enabled at the subscription level”, click “Configure”.
o Select your storage account.
o Select “Periodic recurring scans” to On if you want weekly scans.
Page 56 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
o Enter an email address for your scan reports and alerts.
▪ You can also send emails to admins and subscription owners.
• To view details of the findings, go to Security – Security Center - “View additional findings in
Vulnerability Assessment”.
o Findings include an overview, number of issues found, severity risk summary, and findings
list.
▪ You can click on an issue for more details.
▪ You can “Approve as Baseline” specific results. Any similar results are put in the
“Passed” section.
▪ You can also “Disable rule”
o You can also click “Scan” to do an on-demand scan.
o Click “Scan History” to view previous scans.
• Click on “Export Scan Results” to download an Excel report.
40. Manage database resources by using Azure Purview
• Data can be anywhere on-premises or in the cloud.
• It is hard to make sure data is compliance if you don't know where it is.
• Others may not know what data your company has access to.
• Azure Purview catalogs your data, whether it is on-premises, in a machine on the Internet, or in a
cloud using Software-as-a-Service (SaaS).
• It calls itself a Unified Data Governance solution. Cost from US$300 for 10 Gb of metadata.
• There are three main elements to the Azure Purview Studio:
• Azure Purview Data Map captures metadata (information about data) from the various
sources, by scanning and classifying it.
• Azure Purview Data Catalog helps you to find data with classification or metadata filters.
• Azure Purview Data Insights allow you to see where sensitive data is and how it flows from
one data source to another.
• Data can be classified into (for example):
• Location (City, Country, Place),
• Person First and Last Name,
• Bank account, business, company, driver's license, medial accounts, passport, social security,
tax file, and other identification numbers.
• Date of Birth,
• Email,
• Ethnic group,
Page 57 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
• IP (Internet Protocol) Addresses.
• You can create scan rule sets which group together the classifications and file types.
• To manage database resources:
• In the Azure SQL Database, go to "Server Firewall", and click on "Allow azure services and
resources".
• Create an Azure Key Vault.
• Create a new Secret with the SQL password.
• Create a Purview account
• Click on "Open Purview Studio".
• In Management – Credentials, click on "Manage Key Vault connections".
• Then click on "+New" for a new connection.
• Fill in a connection Name, and select the "Key Vault name", which should be the Key
Vault you have just created.
• Go back to Management – Credentials, Then add a "+New" credential.
• Enter the Name and User Name.
• The Authentication methods include Managed identity, and SQL/Windows
authentication.
• Select your Key Vault connection, and select your Secret.
• In Data Map – Sources, you may create a collection (optional).
• In Data Map – Sources – Register [a new source] – Azure SQL Database.
• Enter a name, and Select the Subscription, Server name, and collection.
• For this database, click on the new scan (a lot of Cs with a little pencil).
• Enter a name, and database name.
• Important: change the credential to the credential you have set up earlier.
• Unless you want information such as Stored Procedures executions, turn off Lineage
extraction.
• Additional steps are needed if you want this on.
• Select a collection, and click "Test connection".
• Select a scan frequency.
• You can delete the reference to the database, or click on "View details".
• You can view the results of the scans by going to the Data catalog.
• You have filter by Object Type, Classification (such as Person's Name), Contact,
Label or Assigned Terms.
Page 58 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
41. Implement Azure SQL Database ledger
• You may have data that you need to know has not been tampered with – for example, in the financial
and medical field.
• A Database Ledger protects your data:
• Preserves historical data, by maintaining previous values in a history table, which can
support T-SQL queries for auditing and forensics.
• Manages the process transparently, not requiring application changes.
• Providing cryptographic (secure communication techniques) proof of data to auditors,
reducing time needed to audit data.
• Any modification is hashed (cryptographically using SHA-256), to create a root hash.
• Root hashes are stored in blocks, which are closed after 30 seconds or 100,000
transactions.
• This block is then hashed along the root hash of the previous block, forming a
blockchain.
• The latest block hash is called the "database digest".
• They can be stored in immutable Azure Blob storage (Write Once, Read
Many or WORM) or Azure Confidential Ledger.
• You can then verify the database's integrity by comparing the database
digest hash against the database calculated hashes.
• You can create ledger databases in SQL Server 2022 and Azure SQL Database.
• You can create "updatable ledger tables". When doing so, the following are created:
• The table itself
• It includes the 4 GENERATED ALWAYS columns ledger_start/end_transaction_id and
ledger_start/end_sequence_number.
• The transaction_id columns are the unique transaction ID (which may contain
multiple rows).
• The sequence_number shows the order the values are inserted in each transaction
(restarting at zero for each transaction).
• A history table, showing the previous version of a row when it has been updated or deleted.
• The 4 GENERATED ALWAYS columns are also created in this table.
• Data cannot be deleted from this table.
• If you don't give it a name, it will generally have the suffix
.MSSQL_LedgerHistoryFor_(GUID).
• A view, which joins the updatable ledger table with the history table.
• It shows the transaction ID, together with whether it was a DELETE or INSERT (an
UPDATE is both).
• Microsoft recommends querying the history of changes using the ledger view,
instead of the history table.
• You can also create "append-only ledger tables".
• You can insert data.
• Updates and deletions are denied, even by system administrators or DBAs.
Page 59 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
• You get the error message "Updates are not allowed for the append only Ledger
table 'NAMEOFTABLE'."
• No history table is created, as there are no updates/deletes. However, two GENERATED
ALWAYS columns are automatically added in the main table: ledger_start _transaction_id
and _sequence_number.
• A view is created provides information about the transactions and the user which inserted
the data. However, it is more helpful for updatable ledger tables instead of append-only, as
you cannot UPDATE or DELETE, and is provided for consistency.
• You can also create ledger databases.
• All your tables are ledger tables (either Updatable or Append-only).
• By default, every table is an Updatable ledger table.
• To do this when creating a database in the Azure Portal:
• go to Security – Ledger, click "Configure ledger", and select "Enable for all future
tables in this database".
• You can also "Enable automatic digest storage", to store the digests automatically in
an Azure Storage account or Azure Confidential Ledger.
• To do this in the Azure Portal for all future tables:
• Go to the database in Azure Portal, and go to Security – Ledger, and select "Enable
for all future tables in this database".
• To do this in T-SQL, end the CREATE DATABASE command with "WITH LEDGER = ON"
• Transaction and block data is stored in:
• sys.database_ledger_transactions – information about each transaction, and
• sys.database_ledger_blocks – a row for every block.
• To create ledger tables in T-SQL:
• You need to have the ENABLE LEDGER permission.
• To create an updatable table in T-SQL, add at the end of the CREATE TABLE statement:
• WITH (SYSTEM_VERSIONING = ON
• (HISTORY_TABLE = [Schema].[TableName]),
• LEDGER = ON);
• Note – LEDGER = ON is optional for ledger databases.
• To create an append-only ledger table in T-SQL, use:
• WITH (LEDGER = ON (APPEND_ONLY = ON));
• You cannot convert existing (non-ledger) tables to ledger tables.
• You would need to create new ledger tables, copy the data across, and then (optionally)
rename the ledger tables. You can copy using:
• The stored procedure sp_copy_data_in_batches @source_table_name = N'NAME',
@target_table_name = N'NAME'.
• This splits the copy operation into batches of 10,000-100,000 rows per transaction.
As this is done in parallel, this can speed the copying.
• Alternatively, you can use SELECT INTO or BULK INSERT.
• To verify the ledger database, use:
Page 60 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
• T-SQL
• DECLARE @digest_locations NVARCHAR(MAX) = (SELECT * FROM
sys.database_ledger_digest_locations FOR JSON AUTO,
INCLUDE_NULL_VALUES);SELECT @digest_locations as digest_locations;
• BEGIN TRY
• EXEC sys.sp_verify_database_ledger_from_digest_storage @digest_locations;
• SELECT 'Ledger verification succeeded.' AS Result;
• END TRY
• BEGIN CATCH
• THROW;
• END CATCH
• This script can be found in the Azure portal – [Name of database] – Security – Ledger – Verify
database.
• If successful, you get a message. The output includes:
• path – the digest locations,
• last_digest_block_id – the last block ID, and
• is_current – whether the "path" is the latest (true) or previous (false) location.
• If unsuccessful, the database has been tampered with. Ideally, you should restore to a point
in time that can be verified, and using manually creating any future transactions through
investigating backups.
42. Implement row-level security
• Row-level security (RLS) is available for SQL Server 2016 or later, and Azure SQL Database.
• It restricts what your users can see/do based on their group membership or execution context:
• You can filter the rows available to read, using SELECT, UPDATE and DELETE.
• You can block write operations that users are not able to read, using either AFTER INSERT,
AFTER UPDATE to block write operations on new or existing data, or BEFORE UPDATE and
BEFORE DELETE to block update/deletes on existing data.
• To implement RLS, you need various objects:
• It is recommended to create separate schemas for Row-Level Security objects
• to reduce effort for maintaining permissions.
• Create an inline table-valued function, returning a 1 when the User should see the result.
• Add SELECT permissions to the function.
• Create a security policy with the function as a FILTER PREDICATE (for SELECT) or BLOCK
PREDICATE (for INSERT/UPDATE/DELETE).
• To create the necessary security policies, you need:
• ALTER ANY SECURITY POLICY permission
• SELECT and REFERENCES permissions to any relevant functions, target tables and
columns.
• -- In this example, we will create 3 user accounts:
• CREATE USER BOSS WITHOUT LOGIN;
Page 61 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
• CREATE USER User1 WITHOUT LOGIN;
• CREATE USER User2 WITHOUT LOGIN;
• -- and a Table with values:
• GO --Create schema must be the first statement in a batch
• CREATE SCHEMA Customers
• GO
• CREATE TABLE Customers.Customers
• (Customer nvarchar(10),
• Status nvarchar(10),
• UserLead nvarchar(10))
• INSERT INTO Customers.Customers VALUES
• ('John', 'A', 'User1'), ('Fred', 'B', 'User2'), ('Trevor', 'A', 'Boss') , ('Alfred', 'B', 'Boss')
• -- Function
• GO
• CREATE SCHEMA RLS;
• GO
• CREATE FUNCTION RLS.rls_security(@User as nvarchar(10), @Status as
nvarchar(10)) RETURNS TABLE
• WITH SCHEMABINDING
• AS
• RETURN SELECT 1 AS rls_security_result
• WHERE @User = USER_NAME() or (USER_NAME() = 'BOSS' AND @Status = 'A') ;
• GO
• -- Add SELECT permissions to the function and the table:
• GRANT SELECT ON RLS.rls_security TO [Boss]
• GRANT SELECT ON RLS.rls_security TO [User1]
• GRANT SELECT ON RLS.rls_security TO [User2]
• GRANT SELECT ON Customers.Customers TO [Boss]
• GRANT SELECT ON Customers.Customers TO [User1]
• GRANT SELECT ON Customers.Customers TO [User2]
• GRANT INSERT ON Customers.Customers TO [Boss]
• -- Create the security policy
• CREATE SECURITY POLICY RLSPolicy
• ADD FILTER PREDICATE RLS.rls_security(UserLead, Status)
• ON Customers.Customers,
• ADD BLOCK PREDICATE RLS.rls_security(UserLead, Status)
• ON Customers.Customers AFTER INSERT
• WITH (STATE = ON); -- To enable the policy
Page 62 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Implement a Secure Environment
• GO
• -- Then you can test:
• EXECUTE AS USER = 'User1'
• SELECT * FROM Customers.Customers
• REVERT
• -- Second test
• EXECUTE AS USER = 'Boss'
• SELECT * FROM Customers.Customers
• INSERT INTO Customers.Customers
• VALUES ('Sally', 'A', 'User1')
• SELECT * FROM Customers.Customers
• INSERT INTO Customers.Customers
• VALUES ('Susan', 'B', 'User1')
• REVERT
• -- Turn off security policy
• ALTER SECURITY POLICY RLSPolicy
• WITH (STATE = OFF);
• -- Do third test
• EXECUTE AS USER = 'User1'
• SELECT * FROM Customers.Customers
• REVERT
• EXECUTE AS USER = 'Boss'
• SELECT * FROM Customers.Customers
• INSERT INTO Customers.Customers
• VALUES ('Susan', 'B', 'User1')
• SELECT * FROM Customers.Customers
• REVERT
43. Configure Microsoft Defender for SQL
• Advanced Threat Protection is for Azure SQL Database, Azure SQL Managed Instance, SQL Server on
Azure VMs and more.
• It alerts customers to potential threats when they happen.
• You receive alerts on suspicious database activities (including access and query patterns),
possible vulnerabilities, and SQL injection attacks.
• Alerts are integrated with Microsoft Defender for Cloud, which includes recommended
actions.
• You may wish to enable auditing, for writing database events to an Azure log.
• See separate "configure server and database audits" topic for more details.
Page 63 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
• To set it up:
• In the Azure Portal, go to the SQL Server – Security – Microsoft Defender for Cloud.
• Click "Enable Microsoft Defender for SQL" if it is not enabled.
• Click on Configure.
• In the "Advanced Threat Protection Settings", click "Add your contact details to the
subscription's email settings in Defender for Cloud", and provide which roles should receive
the email notifications, together with any additional address.
• The emails will provide information on the activities, database, server and
application name, and the event time.
• There will be links for "View recent SQL alerts", Investigation steps and Remediation
steps.
• If you want to, click the "Notify about alerts with the following severity (or higher)", and
select a level.
• You will see alerts in the Overview – Notifications, and in Security – Advanced Threat Protection.
Monitor, Configure and Optimize Database Resources
44. prepare an operational performance baseline
• Metrics are numerical values that are collected at regular intervals and have a timestamp, name,
value and other labels.
o They are stored in a time-series database which is suitable for alerting and fast detection of
issues.
o It is lightweight and allow for near-real time alerting.
• View in Metrics Explorer.
o Go to Azure Portal – database – Monitoring – Metrics.
o Select:
▪ Scope,
▪ Metric Namespace,
▪ Metric (e.g. "Data space used"), and
▪ Aggregation (Min/Max/Avg or Sum/Count).
o To change the date/time range, go to the top-right hand corner (where it says "Local time").
▪ You can also change the "Show time as" from Local to UTC/GMT, and change the
"Time granularity" (how often it does the aggregation).
▪ Only a maximum of 30 days visible at once, but you can use the arrow at the left-
right to go back up to 93 days in the past.
o You can:
Page 64 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
▪ Change the color of a line (by clicking on the color in the legend – not the line, but
the legend).
▪ Edit the title,
▪ Split or filter a metric, if it has a dimension (not applicable to Azure SQL Database).
▪ Add a second metric onto the same chart (e.g. "Date space allocated").
▪ Change the chart type (from Line to Area, Bar, Scatter and Grid).
▪ Move the chart up, down, clone it, delete it, or see more settings (in the … to the
right-hand side).
▪ Add a new chart
▪ Share it by Downloading it to Excel or copy to Link.
• Logs are events in the system, which may contain other (non-numerical) data and may be structured
or free-form, with a timestamp.
o View in Log Analytics.
• Areas which could affect SQL Server Performance include:
o Hardware/compute/memory,
o The Operating System (VM),
o Database applications and
o Client applications.
• Azure Monitor allows you to monitor resource metrics, such as processor, memory and I/O resources.
o You may need more CPU or I/O resources if you have high DTU/processor percentage or high
I/O percentage. Alternatively, your queries may need to be optimized.
• You can also use T-SQL:
o SELECT * from sys.dm_db_resource_stats -- CPU, IO and memory
▪ You get a row for every 15 seconds for about the past hour.
o SELECT * FROM sys.dm_user_db_resource_governance -- storage
o SELECT * FROM sys.resource_usage
▪ You get a row showing the hourly summary of resource usage data for user
databases. Historical data is retained for 90 days.
▪ However, this is currently in a preview state. It says "Do not take a dependency on
the specific implementation of this feature because the feature might be changed
or removed in a future release."
45. determine sources for performance metrics
• The sources for performance metrics include:
o Azure Tenant – Tenant-wide services such as Microsoft Entra ID.
Page 65 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o Subscription
▪ Azure Activity log includes service health records and records of configuration
changes.
▪ Azure Service Health has information about your Azure services’ health
o Resources
▪ Most Azure services submit Platform metrics to the metrics database
▪ Resource logs are created internally regarding the internal operation of an Azure
resource.
o Guest operating system in Azure, other clouds, and on-prem
▪ Azure Diagnostic extension for Azure VM, when enabled, submits logs and metrics
▪ Log Analytics agents can be installed into your Windows or Linux VMs, running in
Azure, another cloud, or on-prem
▪ VM insights (preview) provides additional Azure Monitor functionality on Windows
and Linux VMs
o Other sources
▪ In Application code, you can enable Application Insights to collect metrics and logs
relating to the performance and operations of the app.
▪ Monitoring Solutions and Insights provide additional insights of a particular service
or app.
▪ Container insights provide data about Azure Kubernetes Service (AKS).
▪ VM Insights allow for a customized monitoring of VMs.
▪ In VM, you can also look at Windows Performance Monitor (perfmon).
- There are specific counters for SQL Server.
45. determine sources for performance metrics
• Metrics available for MI are:
o Average CPU percentage in a selected time period,
o I/O bytes read/written,
o I/O requests counts,
o Storage space reserved/used,
o Virtual core count (4-80 vCores).
• Metrics available for Azure SQL Database are:
o Blocked by firewall,
o Deadlocks,
o CPU %,
Page 66 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o Data I/O % or Log I/O %,
o Data space used/allocated/used %,
o DTU Limit, Used, %
o Failed connections, Successful connections,
o In-memory OLTP storage %,
o Sessions %,
o Workers %,
o SQL Server process core/memory %,
o Tempdb Data/Log File Size Kilobytes,
o Tempdb % Log Used.
46. interpret performance metrics
• If any of the following is high (close to 100%), consider upgrading to the next service tier.
• Space/components used
o DTU percentage – CPU, memory and I/O for vCores (not DTU-based model)
▪ If this is low, then you may save money downgrading.
o Other Component metrics:
- CPU percentage (avg_cpu_percent),
o When high, query latency increases and queries may time out.
o Maybe increase the compute size, or optimise queries to reduce
the CPU requirements.
- Data IO percentage (avg_data_io_percent),
- Log IO percentage (avg_log_write_percent),
o In-memory OLTP storage percent (xtp_storage_percent)
▪ Returns zero if in-memory OLTP (memory-optimized tables, indexes, and table
variables) is not used.
▪ If this hits 100%, then INSERT, UPDATE, ALTER and CREATE operations will fail
(SELECT and DELETE are fine).
o Data space used percent If this is getting high, then upgrade to the next service tier, shrink
the database, or scale out using sharding.
▪ If in an elastic pool, consider moving it out of the pool.
o Memory percentage (avg_memory_usage_percent)
▪ This is used for caching. If you get out-of-memory errors, Increase service tier, or
compute size, or optimize queries.
• Connections/requested used
Page 67 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o Sessions percentage
▪ Maximum concurrent sessions divided by the service tier limit.
(max_session_percent)
o Worker percentage
▪ Maximum concurrent requests divided by the service tier limit.
(max_worker_percent).
o Increase service tier, or compute size, or optimize queries.
• Top queries can be shown in Intelligent Performance – Query Performance Insight.
o Review top CPU-consuming queries
o Individual query details
o Top queries per duration or execution count (Custom – Metric type: Duration or Execution
Count)
o You may see icons showing performance recommendations.
47. configure and monitor activity and performance at the infrastructure, server,
service, and database levels
• See topic 38.
• You can go to a database, can click on:
o Metrics,
o Performance Overview,
o Performance recommendations, or
o Query Performance Insight.
• You can also monitor using Dynamic Management Views:
o https://docs.microsoft.com/en-us/azure/azure-sql/database/monitoring-with-dmvs
Page 68 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
48. Monitor by using SQL Insights
• SQL Insights uses DMVs to monitor health, diagnose problems, and tune performance.
• It supports:
• SQL Server 2012 or later,
• Azure SQL Database (but not with elastic pools, or Basic, S0, S1 or S2 service tiers),
• Azure SQL Managed Instance,
• SQL Server on Azure VMs.
• It can be gathered for the serverless tier, but will prevent the database from pausing.
• It does not support:
• Monitoring of more than one secondary replica per database,
• Authentication with Azure AD.
• Monitoring agents on dedicated VMs connect to your SQL resources and obtain the data.
• Microsoft recommends 1 Standard_B2s VM for every 100 connection strings.
• This data is now stored in Log Analytics workspace, and you can use Azure Monitor for
analysis.
• You can view this data from the SQL Insights workbook template or through log queries.
• The cost for SQL Insights are for the dedicated VMs, the Log Analytics workspaces, and any
alert rules.
• To enable SQL Insights:
• Create a Log Analytics workspace to store the data.
• Create a login/user and grant the required permissions:
• In Azure SQL Database, in the relevant (not "master") database, create a user with
a strong password, and grant the required permissions:
• CREATE USER [SQLInsightsUser] WITH PASSWORD = N'P@ssw0rdStr0ng';
• GO
• GRANT VIEW DATABASE STATE TO [SQLInsightsUser];
• In Azure Managed Instance and SQL Server on a VM:
• USE master
• GOCREATE LOGIN [SQLInsightsUser] WITH PASSWORD =
N'P@ssw0rdStr0ng';
• GO
• GRANT VIEW SERVER STATE TO [SQLInsightsUser];
• GO
• GRANT VIEW ANY DEFINITION TO [SQLInsightsUser];
• Create an Azure Virtual Machine:
• Operating system: Ubuntu 18.04 using Azure Marketplace image.
• Recommended VM: at least Standard_B2s (2 CPUs, 4GB)
• Not currently valid in South Africa West, US Gov Non-Region, DoD Central or East,
China Non-Regional, China East, China North, China North 2, West India.
Page 69 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
• Then you need to configure your database:
• For Azure SQL Database, in the Azure Portal, go to Set server firewall, and then add
a firewall rule.
• For Azure SQL MI, either connect inside the same Vnet, or connect in a different
Vnet using Azure Vnet peering or Vnet-to-Vnet VPN gateway.
• For on-premises, you need to use a Site-to-site VPN connection, or an Azure
ExpressRoute connection.
• You can choose to store your SQL user login passwords in a Key Vault.
• To create your SQL monitoring profile.
• In the Azure portal, go to Monitoring, then Insights – SQL.
• Then go to Manage profile and click "Create new profile".
• Enter:
• Name (cannot be edited later),
• Log Analytics workspace,
• Frequency collection (.5, 1, 2, 5 or 10 minutes).
• The higher the frequency and/or the more measures, the higher the cost.
• What to collect:
• Wait statistics,
• Memory clerks,
• Database I/O,
• Server properties,
• Performance counters,
• Requests,
• Schedulers,
• For Azure SQL Database and Azure SQL MI:
• Resource statistics,
• Resource governance.
• For SQL Server (on VM or on prem):
• Volume space,
• SQL Server CPU,
• Availability Replica States and
• Availability Database Replicas.
• Then click "Create monitoring profile", then "Create SQL monitoring profile".
• Add a monitoring machine
• Click on "Add monitoring machine".
• Select your VM.
• Add connection strings.
• For Azure SQL Database, enter in the format:
Page 70 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
• sqlAzureConnections": [
"Server=mysqlserver.database.windows.net;Port=1433;Database=mydatab
ase;User Id=$username;Password=$password;"
• ]
• Note: if you are not using Azure Key Vault, then you dno't need the
semicolon or the dollar sign surrounding "$password;".
• For Azure SQL Managed Instance, enter in the format:
• "sqlManagedInstanceConnections": [
• "Server= mysqlserver.<dns_zone>.database.windows.net;Port=1433;User
Id=$username;Password=$password;"
• ]
• For SQL Server, enter in the format:
• "sqlVmConnections": [
• "Server=SQLServerInstanceIPAddress;Port=1433;User
Id=$username;Password=$password;"
• ]
• Setting monitoring may take a few minutes. Afterwards, the Status column should change
"Collecting".
• To open SQL Insights:
• In the Azure portal, go to Azure Monitor, then Insights – SQL, and select a tile.
• You can enable alert rules by:
• Clicking on "Alerts".
Go to Alert templates, find a template, and click "Create rule".
• Select:
• the alert threshold (in percent),
• the name and severity for the alert, and
• an action group, creating notifications and alerts.
• Click "Enable alert rule", then "Deploy alert rule".
48a. Monitor by using database watcher
• Database watcher is a centralised store for performance, configuration and health data for Azure SQL
Database and Azure SQL Managed Instance.
• The data is stored in either:
o An Azure Data Explorer cluster – a highly scalable data service for fast input and analytics, or
o Real-Time Analytics within Microsoft Fabric.
• You can query data using KQL (Kusto) or T-SQL, in Azure Data Explorer dashboards, Power BI, Grafana
or Excel.
• Creating watchers and dashboards are free. They are no charges for per resource or per user.
However, you would have to pay for storage.
• The database watcher dashboards include:
Page 71 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o Estate dashboards – a high level view:
▪ Heatmaps allow you to find databases which use most resources.
▪ Top queries to find the highest resource consuming queries.
- It may also give you index recommendations.
▪ You can use filters to filter subscription, resource group and resource name.
▪ You can also drill through to other dashboards.
o Resource dashboards, allow you to see:
▪ Active sessions,
▪ Backup history,
▪ Storage size and performance, and
▪ Common performance counters.
o Filters by time.
• You can also download data to Excel and query the data in KQL using Azure Data Explorer.
• You will need to set up access in each database target. The script can be generated in Configuration –
SQL targets, and click on “+ Add”. It should be run in the “master” database.
o You can use either Microsoft Entra or SQL Authentication.
o For SQL Authentication, you will need a Key Vault.
• You can have either private or public connectivity from the database watcher to the databases. To
manage a private endpoint, go to Configuration – Managed private endpoints, and click on “+ Add”.
• To start monitoring, go to Overview – Start. The status then changes to “Running”.
• You can click on Monitoring – Dashboards to show all of the monitored resources.
• The datasets which are captured for SQL Database, Managed Instance and Elastic Pool are:
o Memory utilization
o Out-of-memory events
o Performance counters (common and detailed)
o Properties (Database and Instance properties for Managed Instance)
o Resource utilization
o SOS schedulers
o Storage IO
o Storage utilization (Database storage utilization for Managed Instance)
o Wait statistics
• The datasets which are captured for SQL Database and Managed Instance (not Elastic Pool) are:
o Active sessions
Page 72 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o Backup history
o Change processing and errors
o Connectivity
o Geo-replicas (Database geo-replicas for Managed Instance)
o Index metadata
o Missing indexes
o Query runtime and wait statistics
o Replicas (Database replicas for Managed Instance)
o Session statistics
o Table metadata
• The datasets which are captured for SQL Managed Instance (not Elastic Pool) are:
o SQL Agent job history
o SQL Agent job state
50, 51. configure Query Store to collect performance data
• What is Query Store?
o It contains 3 stores:
▪ Plan store for execution plan data.
▪ Runtime stats store for execution statistics data.
▪ Wait stats store.
• When would you use Query Store?
o Fix queries which are regressed due to changes in the execution plan.
o How many times has a query been executed?
o What are the Top X queries, by execution time, memory consumption, waiting on resources?
o Look at query plans for a given query.
o Look at CPU, I/O and memory used for a particular database.
o What are the waits for a query?
• Query Store is:
o Disabled by default for new SQL Server databases (e.g. on a VM), but
o Enabled by default for new Azure SQL Databases.
• To enable Query Store:
o In SSMS:
▪ Right-hand click on a database, and go to Properties.
Page 73 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
▪ Go to the Query Store tab.
▪ To enable Query Store generally, change "Operation Mode (Requested)" to "Read
write".
▪ To enable wait stats, change "Wait Statistics Capture Mode" to "On".
o In T-SQL:
▪ ALTER DATABASE Database_Name SET QUERY_STORE = ON (OPERATION_MODE =
READ_WRITE);
▪ ALTER DATABASE Database_Name SET QUERY_STORE = ON (
WAIT_STATS_CAPTURE_MODE = ON );
• It may up to a day to collect sufficient data to represent your workload.
• Options:
o Is it collecting runtime stats?
▪ Use SELECT actual_state, actual_state_desc, readonly_reason FROM
sys.database_query_store_options;
▪ If actual_state = 2, then it is READ_WRITE. If actual_state = 1, then it is READ_ONLY.
o To change how often it collects stats: ("Statistics Collection Interval"):
▪ In T-SQL:
- ALTER DATABASE DatabaseName
- SET QUERY_STORE (INTERVAL_LENGTH_MINUTES = 15);
▪ You can choose from 1, 5, 10, 15, 30, 60 or 1440 minutes. A query will have a
maximum of 1 row collected for this time period.
o You can change multiple options in T-SQL:
o ALTER DATABASE DatabaseName SET QUERY_STORE (
o MAX_STORAGE_SIZE_MB = 500,
◼ The space allocated to the Query Store. The default is 100
Mb in SQL Server 2016/2017, and 1 Gb in SQL
Server2019.
◼ If it reaches the limit, Query Store no longer collects new
data and changes to read-only mode. This will reduce the
performance accurate, because the Query Store will
become stale.
- SELECT actual_state, actual_state_desc, readonly_reason FROM
sys.database_query_store_options;
- readonly_reason would = 65536 if Query Store reached the
MAX_STORAGE_SIZE_MB.
Page 74 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
◼ To prevent it from reaching the limit, increase the
MAX_STORAGE_SIZE_MB. If you can't allocate extra
space, then decrease the Data Flush time.
o DATA_FLUSH_INTERVAL_SECONDS = 3000,
◼ how long (in seconds) the data is retained in memory
before being saved to disk.
▪ Have a higher value if you don't have a large number of queries running being
generated. However, if the SQL Server crashes or restarts, then anything new will
not be saved.
▪ Having a lower value may have a negative impact of performance, as it will save
more often.
o SIZE_BASED_CLEANUP_MODE = AUTO,
◼ whether automatic data cleanup occurs when size limit is
reached.
▪ When Query Store data reaches about 90% of MAX_STORAGE_SIZE_MB, a clean-up
begins. It will remove oldest/least expensive query data, and stops when size is
about 80% of MAX_STORAGE_SIZE_MB.
▪ If you need it checking more quickly, reduce the DATA_FLUSH_INTERVAL_SECONDS
period.
o OPERATION_MODE = READ_WRITE,
o CLEANUP_POLICY = (STALE_QUERY_THRESHOLD_DAYS = 30),
◼ how long data is retained in days.
▪ You can automatically delete Query data that you don't need.
o INTERVAL_LENGTH_MINUTES = 15,
◼ in this number of minutes, each query has a maximum of
1 row. Statistics are aggregated for each query during this
time.
o QUERY_CAPTURE_MODE = AUTO,
◼ capture "All" queries, None, Custom or Auto (ignore
infrequent queries and queries with small
compile/execution times). Default was "All", but in SQL
Server 2019 and Azure SQL is "Auto".
o MAX_PLANS_PER_QUERY = 1000,
o WAIT_STATS_CAPTURE_MODE = ON);
• To clear:
o ALTER DATABASE DatabaseName SET QUERY_STORE CLEAR;
o or click the "Purge Query Data" button in SSMS.
Page 75 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
52. identify sessions that cause blocking
• Blocking can occur when:
o Session 1 locks a resource (e.g. row, page or entire table), and then
o Session 2 requests that resource.
• Blocking is caused by:
o Poor transactional design, or
o Long running transactions.
• To emulate this, we are going to have an explicit transaction.
o Implicit transactions automatically add a BEGIN and COMMIT TRANSACTION.
o Explicit transactions require you to add the BEGIN, and COMMIT/ROLLBACK TRANSACTION.
• Session 1
o BEGIN TRANSACTION
o UPDATE [SalesLT].[Address]
o SET City = 'Toronto ON'
o where City = 'Toronto'
• Session 2
o BEGIN TRANSACTION
o UPDATE [SalesLT].[Address]
o SET City = 'Toronto'
o where City in ('Toronto ON', 'Toronto')
• To view locks:
o SELECT * FROM sys.dm_tran_locks
• To view blocking:
o SELECT session_id, blocking_session_id,
o start_time, status, command,
o DB_NAME(database_id) as [database],
o wait_type, wait_resource, wait_time,
o open_transaction_count
o FROM sys.dm_exec_requests
o WHERE blocking_session_id > 0
• For the session_id, look at the numbers in brackets at the top of SSMS.
• To reduce blocking, you can change the TRANSACTION ISOLATION LEVEL of a session:
Page 76 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o SET TRANSACTION ISOLATION LEVEL …
o READ UNCOMMITTED – No blocking, but would have dirty reads.
o READ COMMITTED – No dirty reads, as would not read statements that have been modified
but not committed.
▪ If READ_COMMITTED_SNAPSHOT is OFF (the default on SQL Server), may block.
▪ If READ_COMMITTED_SNAPSHOT is ON (the default on Azure SQL Database), uses a
snapshot and therefore does not block.
o REPEATABLE READ – No dirty reads, but blocks.
o SNAPSHOT – The data read remains the same until the end of the transaction. No blocks
unless the database is being recovered.
▪ Needs ALLOW_SNAPSHOT_ISOLATION to be ON.
o SERIALIZABLE - No dirty reads, as would not read statements that have been modified but
not committed. However, blocks updates/inserts.
• To see the current level, use
o DBCC USEROPTIONS
• There are database options as well.
o ALTER DATABASE NameOfDatabase SET ALLOW_SNAPSHOT_ISOLATION ON
▪ DML statements start generating row versions – allows snapshots but doesn't
enable it.
o ALTER DATABASE NameOfDatabase SET READ_COMMITTED_SNAPSHOT ON
▪ DML statements start generating row versions – means that TRANSACTION
ISOLATION LEVEL READ COMMITTED does not block.
• You can use Extended Events in MI and SQL Server on VM.
o Lightweight tracing system, used for:
▪ Troubleshooting blocking and deadlocking performance issues
▪ Identifying long-running queries
▪ Monitoring Data Definition Language (DDL) operations
▪ Logging missing column statistics
Page 77 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
▪ Observing Memory Pressure in your database
▪ Long-running physical I/O operations
• To create a new session:
o In SSMS, go to Management – Extended Events – and right-hand click on Sessions and go to
"New Session Wizard" or "New Session…"
o Give the session a name.
o Use a template if applicable, from:
▪ Locks and Blocks,
- Count Query Locks
▪ Profiler Equivalents,
- SP (Stored Procedure) Counts,
- Standard (Stored Procedures and T-SQL batches),
- TSQL (Debug client applications),
- TSQL_SPs (analyze the component steps of SPs),
- TSQL_Duration (identify slow queries),
- TSQL_Locks (deadlocks),
- TSQL_Replay (benchmark testing),
- Tuning (Stored Procedures and T-SQL batches).
▪ Query Execution,
- Query Batch/Detail Sampling (20% of active sessions),
- Query Batch/Detail Tracking (understand query flow),
- Query Wait Statistic (query hash and query plan hash)
▪ System Monitoring
- Activity Tracking (general activity),
- Connection Tracking (connection activity),
- Database Log File IO Tracking (for database log files).
• Select Events to Capture
o You can filter by Category.
• Capture Global Fields
o Such as "session_id".
• Set Session Event Filters.
o So you don't have to capture every event.
• Specify Session Data Storage.
Page 78 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o Event Tracing for Windows (ETW)
▪ Correlates SQL Server events with Windows OS events. Processes data
synchronously
o Event Counter
▪ Counts how many times each event occurs. Processes data synchronously
o Event File – disk file.
▪ Creating large records (asynchronous)
o Histogram
▪ Counts how many times events occurs, for event fields and actions separately
(asynchronous).
o Pair Matching
▪ Detect start events without an corresponding end event (asynchronous).
o Ring Buffer
▪ Smaller data sets or continuous data collection (asynchronous).
53. determine the appropriate Dynamic Management Views (DMVs) to gather query
performance information
• You can retrieve the last execution plans using:
o SELECT *
o FROM sys.dm_exec_cached_plans AS cp
o CROSS APPLY sys.dm_exec_sql_text(plan_handle) AS st
o CROSS APPLY sys.dm_exec_query_plan_stats(plan_handle) AS qps;
• You can also store execution plans using:
o Extended Events
Page 79 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
▪ Lightweight profiling
▪ Add at the end of a query OPTION(USE HINT ('QUERY_PLAN_PROFILE') would add it
to Extended Events
• Find top N queries ranked by average CPU time using sys.dm_exec_query_stats
o SELECT TOP 5 query_stats.query_hash AS "Query Hash",
o SUM(query_stats.total_worker_time) / SUM(query_stats.execution_count) AS "Avg CPU
Time",
o MIN(query_stats.statement_text) AS "Statement Text"
o FROM
o (SELECT QS.*,
▪ SUBSTRING(ST.text, (QS.statement_start_offset/2) + 1,
▪ ((CASE statement_end_offset
▪ WHEN -1 THEN DATALENGTH(ST.text)
▪ ELSE QS.statement_end_offset END
- QS.statement_start_offset)/2) + 1) AS statement_text
o FROM sys.dm_exec_query_stats AS QS
o CROSS APPLY sys.dm_exec_sql_text(QS.sql_handle) as ST) as query_stats
o GROUP BY query_stats.query_hash
o ORDER BY 2 DESC;
• Find which queries use the most cumulative CPU:
o SELECT
o highest_cpu_queries.plan_handle,
o highest_cpu_queries.total_worker_time,
o q.dbid, q.objectid, q.number, q.encrypted, q.[text]
o FROM
o (SELECT TOP 50 qs.plan_handle, qs.total_worker_time
o FROM sys.dm_exec_query_stats qs
o ORDER BY qs.total_worker_time desc) AS highest_cpu_queries
o CROSS APPLY sys.dm_exec_sql_text(plan_handle) AS q
o ORDER BY highest_cpu_queries.total_worker_time DESC;
53. identify performance issues using DMVs
• Long running queries that consume CPU are still running
Page 80 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o SELECT TOP 10 req.session_id, req.start_time, cpu_time 'cpu_time_ms',
OBJECT_NAME(ST.objectid, ST.dbid) 'ObjectName',
SUBSTRING(REPLACE(REPLACE(SUBSTRING(ST.text, (req.statement_start_offset / 2)+1,
((CASE statement_end_offset WHEN -1 THEN DATALENGTH(ST.text) ELSE
req.statement_end_offset END-req.statement_start_offset)/ 2)+1), CHAR(10), ' '), CHAR(13),
' '), 1, 512) AS statement_text
o FROM sys.dm_exec_requests AS req
o CROSS APPLY sys.dm_exec_sql_text(req.sql_handle) AS ST
o ORDER BY cpu_time DESC;
• Data from your database
o USE master
o GO
o SELECT * FROM sys.resource_stats
o WHERE database_name = 'X'
o ORDER BY start_time DESC;
• Current active sessions
o SELECT * FROM sys.dm_exec_connections
o SELECT @@SPID gives the current session.
54. identify and implement index changes for queries
• Requirements for Indexes:
o Big tables (small tables may use a Scan anyway).
o Small column size (the best are numeric, but smaller text columns are OK too).
o Use columns which are in WHERE (SARGable columns) and JOIN clauses.
▪ If using LIKE '%text%', then an index (apart from a full-text index) will not help.
▪ Additional columns can be included using INCLUDE (covered queries). This can make
the index key smaller and more efficient.
o Clustered or Non-clustered?
▪ Only one clustered index per table. It also used in PRIMARY KEYs. It re-sorts the
table. Use for frequently used queries and range queries.
- Should be used with the UNIQUE property – but it is possible to create one
which doesn't.
- Should be either UNIQUE or have many distinct values.
- Accessed sequentially (in ranges).
- IDENTITY
- Frequently used.
Page 81 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
▪ As many non-clustered indexes as you want. It creates a separate index.
o Columnstore indexes are available in almost all service tiers.
o Only need a small part indexes?
▪ Use a Filtered Index
o Do you need room to grow?
▪ Use the FillFactor option to leave space for growth.
o Ascending or Descending, based on how you want the results to appear.
• Too many indexes?
o If you INSERT, UPDATE, DELETE or MERGE, then all indexes need to be adjusted.
• Create in T-SQL:
o CREATE [UNIQUE] [NONCLUSTERED/CLUSTERED] INDEX [Name] ON Schema.Table (Columns)
[INCLUDE (Columns)] [WHERE … - filtered index]
• Create in SSMS:
o Right-hand click on Indexes in the relevant table and select "New Index" – "[Non-]Clustered
Index".
55. recommend query construct modifications based on resource usage
• Missing indexes:
o SELECT * FROM sys.dm_db_missing_index_details
▪ In Azure SQL Database, only gives information about databases to which user has
access.
▪ In creating the index, put equality before inequality – both of these should be the
key – and INCLUDE the included columns.
• Full query from https://docs.microsoft.com/en-us/azure/azure-sql/database/performance-guidance:
o SELECT
o CONVERT (varchar, getdate(), 126) AS runtime
o , mig.index_group_handle
o , mid.index_handle
o , CONVERT (decimal (28,1), migs.avg_total_user_cost * migs.avg_user_impact *
▪ (migs.user_seeks + migs.user_scans)) AS improvement_measure
o , 'CREATE INDEX missing_index_' + CONVERT (varchar, mig.index_group_handle) + '_' +
▪ CONVERT (varchar, mid.index_handle) + ' ON ' + mid.statement + '
▪ (' + ISNULL (mid.equality_columns,'')
▪ + CASE WHEN mid.equality_columns IS NOT NULL
Page 82 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
▪ AND mid.inequality_columns IS NOT NULL
▪ THEN ',' ELSE '' END + ISNULL (mid.inequality_columns, '') + ')'
▪ + ISNULL (' INCLUDE (' + mid.included_columns + ')', '') AS create_index_statement
o , migs.*
o , mid.database_id
o , mid.[object_id]
o FROM sys.dm_db_missing_index_groups AS mig
o INNER JOIN sys.dm_db_missing_index_group_stats AS migs
o ON migs.group_handle = mig.index_group_handle
o INNER JOIN sys.dm_db_missing_index_details AS mid
o ON mig.index_handle = mid.index_handle
o ORDER BY migs.avg_total_user_cost * migs.avg_user_impact * (migs.user_seeks +
migs.user_scans) DESC
56. assess the use of hints for query performance
• Hints only affect one particular DELETE, INSERT, SELECT, UPDATE or MERGE query.
o Microsoft says that the Query Optimizer typically selects the best execution plan, so only use
this as a last resort.
• Join hints can be LOOP, HASH, MERGE JOIN.
o For example, INNER LOOP JOIN instead of INNER JOIN.
▪ LOOP cannot be specified with a RIGHT or FULL Join.
• For query hints, end your query with OPTION (<hints>)
o For example, OPTION (MERGE JOIN)
o {HASH | ORDER } GROUP
o {MERGE | HASH | CONCAT} UNION
o {LOOP | MERGE | HASH} JOIN
o KEEPFIXED PLAN
▪ The query won't be recompiled because the statistics change. It will only recompile
if the schema of the underlying tables changes or sp_recompile is run against these
tables.
o KEEP PLAN
▪ Recompiles less often when statistics change
o OPTIMIZE FOR UNKNOWN
▪ Uses the average selectivity of a predicate, as opposed to the runtime parameter
used when the query is compiled and optimized.
Page 83 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o ROBUST PLAN
▪ Creates a plan that works for the maximum potential row size. If it isn't, then
performance may be impaired.
• If using parameters in a stored procedure, you can say
o OPTION (OPTIMIZE FOR (@param 2))
o or
o OPTION (OPTIMIZE FOR (@param UNKNOWN))
o You could also have WITH RECOMPILE before "AS BEGIN".
• Otherwise, the stored procedure will be optimised as per the first running.
• See also topic 57.
57. determine the appropriate type of execution plan
• To display Execution Plans in SSMS:
o Click on "Display Estimated Execution Plan" (using query optimizer)
▪ This happens immediately without running the query.
o Click on "Include Actual Execution Plan" (including additional runtime statistics).
▪ This is displayed as a separate tab when the query is run.
▪ or use
- SET SHOWPLAN_ALL ON;
- GO
◼ or SHOWPLAN_TEXT
- Neither version executes the statement, but displays execution
information.
o Live Query Statistics (updated while the query is running).
• Execution plans run from the right to the left.
o The arrow thickness represent number of rows.
▪ You can hover over them for more details.
▪ More estimated rows = more memory reserved.
o Note the degree of parallelism (also shown in properties).
• There are 3 main types of joins between tables:
o Nested Loops joins.
▪ Use when
- Input1 is small.
- Input2 is large.
Page 84 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
- Input2 is indexed on the join.
▪ Uses least I/O and fewest comparisons.
▪ It uses the top input (in the execution plan) and takes 1 row.
▪ It then searches for matches rows in the bottom input.
o Merge joins
▪ Use when
- Input1 and Input 2 is not small.
- Input1 and Input2 are sorted on their join – or if not, possibly when Input1
and Input2 are of a similar size. Then, the Sort might be worth the time
compared with the Hash Join.
▪ Can be very fast.
o Hash joins
▪ For large, unsorted, nonindexed inputs.
▪ Also used in the middle of complex queries, as intermediate results are often not
indexed or suitably sorted.
• In SQL Server 2017, a Batch mode Adaptive Join was introduced.
o This converts into a Hash Join or Nested Loops join after the first input has been scanned,
when it uses Batch mode.
o More in topic 57.
57. identify problem areas in execution plans
• Are you using SELECT *?
o Can you narrow down the columns? If so, maybe you can then use indexes.
• Have a look at the cost of each operation.
o Is there one that can be improved?
▪ Is there a Sort? It's expensive – do you really need it? If so, could you have an Index
which has already sorted on those columns?
▪ Do you use parameters? If so, and the performance is based, can you WITH
RECOMPILE the stored procedure, or use OPTION (RECOMPILE) for queries.
o Is there a Scan when you are using a WHERE?
▪ If so, could a Seek be better? It may need an index.
▪ Are you using a Heap? Do you need a clustered index?
▪ Could you use a SARGable predicate in the WHERE clause?
- e.g. don't use YEAR, use BETWEEN for dates.
- don't use LEFT – use LIKE for strings.
Page 85 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
- don't use ISNULL(X, 'Y') function – use (X IS NULL or X = 'Y')
o Is there a RID Lookup or a Key Lookup Operator?
▪ If so, could you use an INCLUDE with the index? This writes the data into the index,
but in a separate part of the index away from the Key – so it's quicker, but doesn't
slow down the index much.
- It's also useful for Unique indexes – INCLUDE columns are included in the
Uniqueness test.
o Are the field types too wide?
▪ This will increase the row size, increasing time to retrieve the data.
• Different loops
o Are you using a Hash Join when, with some changes, a Merge Join or Nested Loop could be
used?
▪ Maybe need an index?
• Are you using a cursor?
o Use a set-based operation instead.
57. extract query plans from the Query Store
• To use in T-SQL:
o The query plans are stored in:
▪ SELECT * FROM sys.query_store_plan
o Statistics about it can be found:
▪ SELECT * FROM sys.query_store_runtime_stats
o However, you can see the queries:
▪ SELECT Txt.query_text_id, Txt.query_sql_text, Qry.*
▪ FROM sys.query_store_query AS Qry
▪ INNER JOIN sys.query_store_query_text AS Txt
▪ ON Qry.query_text_id = Txt.query_text_id ;
• To use in SSMS:
o Note – you can click on "Configure" to change the time period. You can also click on "Track
the selected query in a new window".
o Go to the database – Query Stores:
▪ Regressed Queries
- Have your query speed got worse? Have a look at Duration, CPU Time,
Logical Reads, Physical Reads, and more
▪ Overall Resource Consumption
Page 86 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
- Are the resources used more during particular days, or daily/night?
▪ Top Resource Consuming Queries
- The most extreme values in Duration, Execution Count, CPU Time etc.
▪ Queries with Forced Plans
▪ Queries with High Variation
- Varied amount of duration, CPU time, I/O and memory.
▪ Queries Wait Statistics
- You can click on the categories (e.g. High Memory, Lock, Buffer I/O or CPU
waits) to get detail on that category.
- Data from sys.query_store_wait_stats
▪ Tracked Queries
- Track individual queries – you need to enter the Query ID.
- You can force query to use a particular plan ("Force Plan")
- Circle = query completed. Square = cancelled by client. Triangle = Failed
due to an exception aborted execution.
- Look for any missing indexes in the Query view.
57a. assess database performance by using Intelligent Insights for Azure SQL Database
and Managed Instance
• Not available in some regions
o West Europe, North Europe, West US 1 and East US 1.
o Not available for VMs.
• Compares current database workload (last hour) with the last 7-days.
o e.g. Most repeated and largest queries.
o Uses data from the Query Store (see topic 48), which is enabled by default in Azure SQL
Database.
• Monitors using Artificial Intelligence operational thresholds, detects issues with high wait times,
critical exceptions, and query parameterizations
o Impacted metrics are increase to query duration, excessive waiting, timed-out or errored-out
requests.
o Includes a “root cause analysis” in a readable form. May also contain a recommendation.
• Can be streamed to:
o Log Analytics workspace, can be used with Azure SQL Analytics (cloud-based only monitoring
solution) to see insights in the Azure portal. The typical way to view insights.
Page 87 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
▪ To add Azure SQL Analytics, go to Home in the Azure portal, click “+Create a
resource”, and search for “Azure SQL analytics”
▪ Can query using the Kusto Query Language.
o Azure Event Hubs, for custom monitoring and alerting
▪ Stored in Avro format, a binary JSON format
o Azure Storage, for custom app development.
▪ Stored in Extended Events format.
• How to connect
o Connect the Intelligent Insights to the log. OR
o Go to the database in the Azure Portal, and go to Monitoring – Diagnostic settings – Add.
▪ Add all the Category Details (log and metric), and in “Destination details” check
“Send to Log Analytics workspace”.
• To view Intelligent Insights:
o Go to Azure Portal – database – Query Performance Insight
• To view more Intelligent Insights:
o Go to the Log Analytics workspace – Workspace summary
• Intelligent Insights looks for things which affect Database performance:
o Reaching resource limits
▪ CPU reaching resource limits for Managed Instance, and
▪ DTUs, worker threads and login sessions reaching resource limits for Azure SQL
Database.
o Workload increase
o Memory pressure
▪ Workers (request) waiting for memory allocations
o Data locking
o Increase Maximum Degree of Parallelism option (MAXDOP)
▪ When there are more Parallel workers than there should have been.
o Missing indexes
o New queries affected performance.
o Increased Wait Statistic.
o Multiple threads using the same TempDB resource.
o For DTU-model, shortage of available eDTUs in the elastic pool.
o New plan, or change in existing plan.
Page 88 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o Pricing tier downgrade.
58. implement index maintenance tasks
• Missing indexes can be found in:
o SELECT * FROM sys.dm_db_missing_index_details
• In Azure SQL Database, in the Azure Portal – you can go to Intelligent Performance – Automatic
tuning.
o You can click on a “Create index” or “Drop index” and implement it.
o You cannot do it in Azure SQL MI.
• For VMs, you can also use the Database Engine Tuning Advisor.
o You open it by going to Tools – Database Engine Tuning Advisor.
o You then need to give it details such as the Query Store or a T-SQL file (.sql extension) with
your workload.
o You can then click "Start Analysis".
59. implement statistics maintenance tasks
• Statistics are used to create query plans to improve the speed of queries.
o The statistics contain information about the distribution of values in tables or indexed views’
columns.
o It uses it to estimate the cardinality, or number of rows, in a result.
o This enables the Query Optimizer to create better quality plans (e.g. seek vs scan).
• Usually, the Query Optimizer determines when statistics might be out of date and then updates them.
However, you may wish to manually update them if:
o Query execution times are slow,
o Insert operations occur on ascending or descending key columns, such as IDENTITY or
timestamp columns.
o After maintenance operations, such as a bulk insert (but not rebuilding or reorganizing an
index, as they do not change the data distribution).
• The stored procedure sp_updatestats updates statistics for all user-defined and internal tables.
• To update a particular table or indexed view, you can use:
o UPDATE STATISTICS Schema.Table
o You can add:
▪ WITH FULLSCAN – This scans all of rows. It is the same as SAMPLE 100 PERCENT
▪ WITH SAMPLE X PERCENT or ROWS. This is the approximately percentage of
number of rows to be used for updating statistics.
▪ WITH RESAMPLE – its most recent sample rate.
Page 89 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o You can append this with
▪ PERSIST_SAMPLE_PERCENT = ON or OFF – If ON, this will then be the default for
future statistics updates (unless you specify the number of rows).
60. evaluate database health using DMVs
• See topic 61.
• To use database DMVs, you need to have VIEW DATABASE STATE permission on the database.
o It is not enough to have VIEW SERVER STATE.
• Overall resource usage:
o SELECT * from sys.dm_db_resource_stats –
▪ CPU, IO and memory
▪ SLO is the Service Level Objective, which includes deployment option, service tier,
hard and compute amount.
▪ You get a row for every 15 seconds for about the past hour.
▪ Use sys.server_resource_stats for MI.
o SELECT * from sys.dm_user_db_resource_governance
▪ Storage in the current database or elastic pool.
▪ In MI only, you can also use sys.dm_instance_resource_governance ).
o SELECT * FROM sys.dm_os_job_object
▪ CPU, memory and I/O resource at the SQL Server level.
o SELECT * FROM sys.dm_io_virtual_file_stats(null, null)
▪ I/O statistics for data and log files
o SELECT * FROM sys.dm_os_performance_counters
▪ Performance counter information, including:
- SQL Server: Databases
- SQL Server: General Statistics
- SQL Server: Query Store
- SQL Server: SQL Statistics
• Waiting on resources:
o SELECT * FROM sys.dm_os_wait_stats OR
o SELECT * FROM sys.dm_db_wait_stats (Azure SQL Database / MI)
▪ Returns information about all the waits encountered by threads that executed.
▪ Top wait types
- Governor
Page 90 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o LOG_RATE_GOVERNOR – waits for Azure SQL Database
o POOL_LOG_RATE_GOVERNOR – Elastic Pools
o INSTANCE_LOG_GOVERNOR – MI waits
o RBIO* - Hyperscale log governance.
o HADR_THROTTLE_LOG_RATE – Business Critical and geo-
replication latency
- IO
o PAGEIOLAATCH_* - data file I/O issues
o PAGELATCH_* - tempdb I/O issues
o WRITE_LOG – transaction log I/O issues
- Memory Grant Wait performance issues
o RESOURCE_SEMAPHORE – waiting for memory to become
available
- Parallel
o CXPACKET – Max Degree of Parallelism may be too high, or
indexes may needed to be created.
o SOS_SCHEDULER_YIELD – high CPU utilization, maybe caused by
missing indexes – often seen with CXPACKET waits.
o Possible blocking
▪ SELECT * FROM sys.dm_exec_requests
- Active queries, and what resource they are waiting on.
▪ SELECT * FROM sys.dm_os_waiting_tasks
- Wait types for a particular task for a specific query.
60. evaluate server health using DMVs
• See topic 61.
• To use Server-scoped DMVs, you need VIEW SERVER STATE permission on the server.
• In addition:
o SELECT * from sys.databases
▪ msdb, tempdb and model are not listed in Azure SQL Database.
o SELECT * from sys.objects
▪ All tables, queries and other objects.
o SELECT * FROM sys.dm_os_schedulers where STATUS = 'VISIBLE ONLINE';
▪ Shows the vCores.
Page 91 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o SELECT SERVERPROPERTY('EngineEdition');
▪ Returns 5 for SQL Database, 8 for Managed Instance, and <5 for on-prem/VM.
60. perform database consistency checks by using DBCC
• DBCC CHECKDB checks the logical and physical integrity of all objects in a specific database. It:
o Runs DBCC CHECKALLOC, which checks the consistency of disk space allocation structures
o Runs DBCC CHECKTABLE for all tables and index views. The DBCC checks the integrity of all
pages and structures in a particular table or index view, including:
▪ Data pages are correctly linked.
▪ Indexes are in the correct sort order.
▪ Every row in a table has a matching row in a nonclustered index (and the other way
round), and is in the correct partition.
▪ DBCC CHECKTABLE ('TableName' OR 'ViewName') checks a table/view (Note: it is a
string.)
o Runs DBCC CHECKCATALOG which checks for catalog consistency, using an internal database
snapshot to provide transaction consistency to perform these checks.
▪ Does not work on tempdb or Filestream data (binary large objects or BLOBs on the
file system).
▪ DBCC CHECKCATALOG checks the current database.
▪ DBCC CHECKCATALOG (NameOfDatabase) checks a particular database. (Note: it is
not a string.)
o Validates the contents of every indexed view in the database, and link-level consistency
between table metadata and file system directories and files.
o DBCC CHECKDB syntax is similar as DBCC CHECKCATALOG.
60. perform database consistency checks by using DBCC
• Arguments:
o Relevant Database
▪ DBCC CHECKDB (MyDatabase)
- Note the lack of quote marks.
▪ DBCC CHECKDB or DBCC CHECKDB(0)
- The zero indicates that the current database should be used.
o What it should do?
▪ DBCC CHECKDB (0, NOINDEX)
- Detect errors only. Smaller execution time, as it does not do intensive
checks of nonclustered indexes for user tables
▪ DBCC CHECKDB (0, REPAIR_REBUILD)
Page 92 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
- Do only repairs which have no chance of data loss. Includes quick repairs
(e.g. missing rows in non-clustered indexes), and time-consuming repairs
(building an index).
- Needs to be in single user mode beforehand, i.e.
ALTER DATABASE MyDatabase SET SINGLE_USER WITH ROLLBACK
IMMEDIATE
GO
DBCC CHECKDB …
GO
ALTER DATABASE MyDatabase SET MULTI_USER
GO
▪ DBCC CHECKDB (0, REPAIR_FAST)
- No repair actions are performed. Only use for Backward compatibility
reasons only.
▪ DBCC CHECKDB (0, REPAIR_ALLOW_DATA_LOSS)
- Repairs any found errors.
- REPAIR_ALLOW_DATA_LOSS may cause data loss.
o Suggest that you create physical copies of the database files
beforehand.
o Needs to be in single user mode beforehand. Additionally, before
that, run
ALTER DATABASE MyDatabase SET EMERGENCY
This marks it as READ_ONLY, logging is disabled, and access is
limited to sysadmins.
• WITH Arguments
o DBCC CHECKDB (0, REPAIR_REBUILD) WITH …
o ALL_ERRORMSGS – displays all reported errors per object.
o EXTENDED_LOGICAL_CHECKS – performs logical consistency checks on indexed views, XML
indexes and spatial indexes.
o NO_INFOMSGS – does not show informational messages.
o TABLOCK – obtains exclusive locks, which will speed it up, but reduce concurrency.
o ESTIMATEONLY – No database checks are done, but displays the amount of tempdb space
needed to do it.
o PHYSICAL_ONLY – limits checking to page structure integrity, record header integrity, and
consistency of the database.
o MAX_DOP = number_of_processors – overrides the max degree of parallelism in
sp_configure.
• Best practices:
Page 93 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o BEGIN TRANSACTION beforehand, so the user can confirm that they want to accept the
results.
o For best repairing errors, Microsoft recommends restoring from a backup.
o After using DBCC CHECKDB, you need to inspect the referential integrity of the database –
DBCC CHECKCONSTRAINTS. This checks the integrity of a constraint or all constraints in a
table, or all constraints.
61. configure database auto-tuning
• Auto-tuning is a process which learns about your workload and identifies potential issues and
improvement: Learn – Adapt – Verify – repeat.
• You can configure database auto-tuning by:
o In the Azure Portal, go to the database or server.
o Go to “Automatic tuning”.
• In Azure SQL Database and Azure SQL MI:
o You can configure FORCE_LAST_GOOD_PLAN (it is enabled by default)
o This says that the last good plan should be forced whenever some plan change regression is
found – when the estimated gain is >10 seconds, or the number of errors in the new plan is >
recommend plan.
• In Azure SQL Database only, you can automate index maintenance by:
o You can change “Create Index” and “Drop Index” from Inherit (from server) to OFF or ON
[the default for servers is OFF for both of these]. These will override the server settings.
o Indexes will only be auto-created if the CPU, data I/O and log I/O are lower than 80%.
o You can view which indexes are auto-created by going to:
▪ SELECT * FROM sys.indexes WHERE auto_created = 1
o The performance of queries using the auto-created index will be reviewed. If is doesn’t
improve performance, it is automatically dropped.
o You can do this in T-SQL for a single database as follows:
▪ ALTER DATABASE DatabaseName SET AUTOMATIC_TUNING = AUTO | INHERIT |
CUSTOM
▪ ALTER DATABASE DatabaseName SET AUTOMATIC_TUNING
(FORCE_LAST_GOOD_PLAN = ON, CREATE_INDEX = ON, DROP_INDEX = OFF)
- CREATE_INDEX and DROP_INDEX cannot be done in Azure SQL MI.
o Recommendations, if any, can be found in:
▪ SELECT * FROM sys.dm_db_tuning_recommendations
62. configure server and service account settings for performance
• See topic 9.
Page 94 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
63. configure Resource Governor for performance
• Resource Governor is used in Azure SQL Database. However, it is not configurable.
• In VMs and Azure SQL MI, you can use Resource Governor to balance resources used by different
sessions.
o You can divide resources (CPU, physical I/O, and memory) differently, based on which
workload it is in. This can improve performance on critical workloads.
• Terminology:
o Resource pool – the physical resources. Two resource pools are created when SQL Server is
installed: internal and default.
▪ Without Resource Govenor enabled, all new sessions are classified into the default
workload group, and system requests into the internal workload group.
o Workload group – a container for requests which have similar criteria, and
o Classification – that criteria.
• To enable/disable Resource Governor:
o In SSMS
▪ Right hand click Management – Resource Governor, and select Properties.
▪ Click "Enable Resource Governor", and click OK.
o In T-SQL
▪ ALTER RESOURCE GOVERNOR RECONFIGURE [or DISABLE];
▪ GO
• Create a Resource Pool
o In SSMS
▪ Right hand click Management – Resource Governor, and select Properties.
▪ Click on the first column in the empty pool. It now has a *.
▪ Double-click the empty cell in the Name, and enter the resource pool Name.
▪ Add other values.
o In T-SQL:
▪ CREATE RESOURCE POOL myPool
▪ WITH (MAX_CPU_PERCENT = 20); -- If you want to delete, use DROP RESOURCE
POOL X
▪ GO
▪ ALTER RESOURCE GOVERNOR RECONFIGURE;
▪ GO
o Settings:
Page 95 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
▪ MIN_CPU_PERCENT and MAX_CPU_PERCENT
- Guaranteed average and maximum average CPU.
- The Max_CPU_PERCENT only applies if there is >100% if all requests were
honoured.
- e.g. Department A has min of 60%, and Department B has max of 40%.
▪ CAP_CPU_PERCENT
- Hard limit for CPU (not maximum average)
▪ MIN_ and MAX_MEMORY_PERCENT
- Memory may remain in a pool, even with no requests.
▪ MIN_ and MAX_IOPS_PER_VOLUME
- The physical I/O operations per seconds (IOPS).
• Workload Groups:
o Requests go into the default group if:
▪ There are no criteria,
▪ The resource pool specified is non-existent.
▪ There is a general classification failure.
• Create a Workload Group
o In SSMS
▪ Right hand click Management – Resource Governor, and select Properties.
▪ Click on the relevant resource pool.
▪ Go down to the "Workload groups for resource pool", and enter a name, with any
other values.
o In T-SQL
▪ CREATE WORKLOAD GROUP myGroup -- or ALTER, if you wish to change it, or DROP
to delete it.
▪ USING myPool; -- or [default];
▪ GO
• Create a classifier function in T-SQL:
o CREATE FUNCTION fnClassifierTime()
o RETURNS sysname
o WITH SCHEMABINDING
o AS
o BEGIN
Page 96 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o if DATEPART(HOUR,GETDATE())<8 or DATEPART(HOUR,GETDATE())>17
▪ BEGIN
- RETURN 'gOutsideOfficeHours';
▪ END
o RETURN 'gInsideOfficeHours';
o END
• Register this classified function:
o ALTER RESOURCE GOVERNOR with (CLASSIFIER_FUNCTION = dbo.fnClassifierTime);
o ALTER RESOURCE GOVERNOR RECONFIGURE;
o GO
• T-SQL
o SELECT * FROM sys.resource_governor_configuration
▪ Returns the stored Resource Governor state.
o SELECT * FROM sys.dm_resource_governor_resource_pools
▪ Returns information about the current resource pool state, the current
configuration of resource pools, and resource pool statistics.
o SELECT * FROM sys.dm_resource_governor_workload_groups
▪ Returns workload group statistics and the current in-memory configuration of the
workload group.
64. implement database-scoped configuration
• See also topic 51.
• In SSMS, you can:
o Right-hand click on a database, and go to Properties and go to Options.
o The settings in the topic are under "Database Scoped Configurations".
• In T-SQL, you can use:
o ALTER DATABASE SCOPED CONFIGURATION [FOR SECONDARY] SET … = ON/OFF; -- for
secondary is geo-replicated secondary database(s) (they all have the same settings).
• GLOBAL_TEMPORARY_TABLE_AUTO_DROP
o Drop global temporary tables when not in use by any session.
▪ Set in individual databases in Azure SQL Database.
▪ Set in tempdb in MI and VMs.
• LAST_QUERY_PLAN_STATS
o Enables/disables actual execution plans in sys.dm_exec_query_plan_stats.
Page 97 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
• LEGACY_CARDINALITY_ESTIMATION
o The query optimizer cardinality estimation model changed in SQL 2014. Should only be
turned on for compatibility purposes.
• MAXDOP – intra-query parallelism; the maximum number of parallel threads
o Having parallel threads should increase query speed.
o Too high a MAXDOP may cause performance problems when executing multiple queries at
the same time, as it may stave new queries of resources. Could reduce MAXDOP if this
happens.
o The default for new Azure SQL Databases is 8, which is best of most typical workloads.
• 55. implement database-scoped configuration
• OPTIMIZE_FOR_AD_HOC_WORKLOADS
o Stores a compiled plan stub when a batch is compiled for the first time, which has a smaller
memory footprint. When it is compiled/executed again, it will be replaced with a full
compiled plan.
• PARAMETER_SNIFFING
o Evaluates Stored Procedures to create an execution plan.
o On subsequent runnings, the computer uses the same execution plan.
▪ No need to spend time and CPU evaluating. However, may be suboptimal for certain
parameters.
• QUERY_OPTIMIZER_HOTFIXES
o Regardless of the compatibility level, enables or disables query optimization hotfixes.
▪ So you can have a compatibility level for SQL Server 2012, but have query
optimization hotfixes that were released after this version.
• There are many more, but these are the main ones.
64. review database configuration options
• These SET options are started with ALTER DATABASE DatabaseName SET = ON/OFF … // GO
o AUTO_CLOSE ON/OFF
▪ Whether the database is shut down after the last user exists.
o AUTO_CREATE_STATISTICS ON/OFF
▪ Creates statistics on single columns in query predicates, to improve query plans and
performance.
o AUTO_UPDATE_STATISTICS[_ASYNC]
▪ Query Optimizer updates statistics when they are used by a query and might be out-
of-date, after insert/update/delete/merge operations change the data distribution.
_ASYNC specifies whether it is done asynchronously or not.
Page 98 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o AUTO_SHRINK ON/OFF
▪ Shrinks when more than 25% of the file contains unused space. Recommended to
leave OFF.
o READ_ONLY / READ_WRITE
▪ Can users only read from the database (not modify it).
o SINGLE_USER / RESTRICTED_USER / MULTI_USER
▪ Only one user at a time, or only db_owner fixed database roles and dbcreator and
sysadmin fixed server roles (any number), or all users which have appropriate
permissions.
o RECOVERY FULL / RECOVERY BULK_LOGGED / RECOVERY SIMPLE
▪ Changes the recovery option. FULL uses transaction log backups. BULK_LOGGED
only minimally logs certain large-scale (bulk) operations. Simple only allows for
complete backups.
o COMPATIBILITY_LEVEL = 100 (SQL Server 2008 and R2), 110, 120, 130, 140, 150 (SQL Server
2019)
▪ In Azure SQL Database and MI and SQL Server 2014, you cannot set it below SQL
Server 2008 (100).
65. configure compute resources for scaling
• See topics 9-11.
65. assess proper database autogrowth configuration
• To look at current settings:
o SELECT * FROM sys.sysfiles
• To adjust auto-growth setting:
o ALTER DATABASE MyDB
o MODIFY FILE
o (NAME=NameFile,FILEGROWTH=40MB or 40%);
• You can also autogrow files in a particular filegroup.
o ALTER DATABASE MyDB
o MODIFY FILEGROUP FilegroupName
o AUTOGROW_ALL_FILES
▪ If any file in a filegroup meets the autogrow threshold, all files in the filegroup will
grow.
65. report on database free space
• To display space used, run:
Page 99 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o EXEC sp_spaceused
o It shows the unallocated space.
• To display it by file, run:
o SELECT file_id, name, type_desc, physical_name, size, max_size
o FROM sys.database_files
o A max_size of -1 means that it is unlimited.
• To view the number of pages used as well as total free space for a particular database, you can use
o SELECT allocated_extent_page_count, unallocated_extent_page_count FROM
sys.dm_db_file_space_usage
▪ Returns space usage information for each data file in the database.
• You can also use:
o DBCC SQLPERF (LOGSPACE)
o However, it only shows transaction log space statistics.
• You can also go to Reports – Standard Reports – Disk Usage on Azure VM.
• For tempdb only, you can use:
o SELECT * FROM sys.dm_db_session_space_usage
▪ Number of pages allocated/deallocated by each session.
o SELECT * FROM sys.dm_db_task_space_usage
▪ Pages allocated/deallocated by each task
66. configure Intelligent Query Processing (IQP)
• IQP is a suite of new features, improving performance. It is supported in Azure SQL Database, Azure
SQL Managed Instance for compatibility level 150. For SQL Server VM, this is SQL Server 2019 and
level 150.
o There are 7 different features, some of which are also available on lower levels.
o You can check which settings are enabled by:
▪ SELECT * FROM sys.database_scoped_configurations
o Note: server-wide configuration options can be found in:
▪ SELECT * FROM sys.configurations
▪ These can be configured (but not in Azure SQL Database) by using
- EXEC sp_configure 'Configuration', X
• You can disable any of them (except APPROX_COUNT_DISTINCT) for all queries in a single database,
or for a single query:
o All queries: ALTER DATABASE SCOPED CONFIGURATION SET X = OFF. 'X' is the first heading.
Page 100 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Monitor, Configure and Optimize Database Resources
o One query - add at the end of the query: OPTION (USE HINT('Y'). 'Y' is the second heading.
o To disable APPROX_COUNT_DISTINCT, don't use that function!
• [DISABLE_ ] BATCH_MODE_ADAPTIVE_JOINS
o For Azure SQL Database, and SQL Server 2017 or higher. Needs a Columnstore index in the
query or a table being referenced in the join, or batch mode enabled for rowstore.
o Selects the Join type (Hash Join or Nested Loops Join) during runtime based on actual input
rows, when it has scanned the first input.
o It defines a threshold (where the small number of rows makes a Nested Loops join better
than a Hash join) that is used to decide when to switch to a Nested Loops plan.
o Enabled by default in SQL Server 2017 under compatibility level 140, and Azure under
compatibility level 140.
• APPROX_COUNT_DISTINCT
o You can use the new aggregation APPROX_COUNT_DISTINCT.
o Provides an approximate COUNT DISTINCT for big data – decreases memory and
performance requirement. It guarantees up to a 2% error rate (within a 97% probability).
▪ Use where absolute precision is not important, but responsiveness is.
o Available in all compatibility levels of Azure SQL Database, and in SQL Server 2019 or higher.
• BATCH_MODE_ON_ROWSTORE / DISALLOW_BATCH_MODE
o For Data Warehouse workloads.
o Queries can work on batches of rows instead of one row at a time, when cached.
o This happens automatically when the query plan decides it is appropriate in Compatibility
Mode 140 for Batch Mode, and Mode 150 (SQL Server 2019+) for Row mode. No changes are
required.
• [DISABLE_ ] INTERLEAVED_EXECUTION_TVF
o Enabled by default in (Azure or SQL Server 2017+) and Compatibility Level 140+.
o Use the actual cardinality of a multi-statement table valued functions on first compilation,
rather than a fixed guess (100 rows from SQL Server 2014).
o Statements must be read-only – so no INSERT, UPDATE or DELETEs.
• [DISABLE_]BATCH_MODE_MEMORY_GRANT_FEEDBACK (Batch or Row mode)
o Enabled by default in (Azure or SQL Server 2017+) and Compatibility Level 140+.
o SQL Server looks how much memory is allocated to a cached query, and then allocates same
amount of memory next time (instead of guessing, then adding more, more, more).
▪ If a query spills to disk, add more memory for consecutive executions. If it wastes
50+% of the memory, reduce memory for consecutive executions.
• [DISABLE_ ] TSQL_SCALAR_UDF_INLINING
Page 101 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
o Enabled by default in (Azure or SQL Server 2019+) and Compatibility Level 150+.
o Scalar UDFs often perform poorly due to:
▪ Running multiple times, once per row.
▪ Unable to actually work out the performance cost.
▪ Unable to optimize more than one SELECT statement at once.
▪ No parallelism in queries which invoke UDFs.
o Scalar UDFs are transformed into equivalent relational expressions inlined into the query,
often resulting in performance gains.
▪ Does not work with all UDFs, including those which have multiple RETURN
statements.
▪ Can also be disabled for a specific UDF by adding "WITH INLINE = OFF" before "AS
BEGIN".
• [DISABLE_ ] DEFERRED_COMPILATION_TV
o Similar to INTERLEAVED_EXECUTION_TVF, but for Table Variables.
o Use the actual cardinality of the table variable encountered on first compilation instead of a
fixed guess (1 row).
Configure and manage automation of tasks
67. manage schedules for regular maintenance jobs
• For Azure SQL Database, see topic 46.
• This is for SQL Server on a VM, and Azure SQL MI, but not Azure SQL Database, as it uses SQL Server
Agent.
o SQL Server Agent doesn't need to be enabled on Azure SQL MI – it is always running.
o It doesn't have all of the functionality of on-prem SQL Server, but it has most of it.
• To create a new job:
o Go to SQL Server Agent (right-hand click it and Start if needed on a VM) – Jobs.
o Right-hand click and go to "New Job".
o Enter a job name.
o Go to the Steps tab and click New.
o Enter the First Step name, select the database, and which user is running the command, and
enter your T-SQL command.
o Click "Parse" to check the syntax.
o Add additional sets as needed.
• To create a schedule for a job:
Page 102 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
o Go to SQL Server Agent (right-hand click it and Start if needed on a VM) – Jobs.
o Right-hand click and go to Properties.
o Go to Schedules and click New.
o Enter:
▪ A name,
▪ Whether it is:
- One time,
- Recurring (Daily, Weekly, Monthly – and when),
- "Start whenever the CPUs become idle" or
- "Start automatically when SQL Server Agent starts" – this setting is not
supported in MI.
o If you subsequently want to Edit or Remove it, you can click those buttons.
o If you want to import a previously made schedule, click "Pick" and then choose the schedule.
• To do this in T-SQL:
▪ USE msdb ;
▪ GO
▪ EXEC sp_add_schedule
▪ @schedule_name = N'ScheduleName' ,
▪ @freq_type = 4,
◼ 1 = Once, 4 = Daily, 8 = Weekly, 16 = Monthly (day of
month), 32 = Monthly (Xth Sunday, for example), 64 =
When SQL Agent service stats, 128 = When computer is
idle.
▪ @freq_interval = 1, -- Fairly complex
▪ @active_start_time = 012345 ;
▪ GO
▪ EXEC sp_attach_schedule
▪ @job_name = N'JobName',
▪ @schedule_name = N'ScheduleName' ;
▪ GO
• To view schedules:
o USE msdb ;
o GO
Page 103 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
o select *
o from sysschedules
• You can see when jobs have run by:
o Going to SQL Server – Job Activity Monitor
68. configure notifications for task success/failure/non-completion
• For Azure SQL Database, see topic 46.
• To create an operator:
o Go to SQL Server Agent (right-hand click it and Start if needed on a VM) – Operators.
o Right-hand click and select "New Operator".
o Enter Name and e-mail name and/or pager e-mail name (and pager timings).
▪ Pager functionality has been deprecated, and will be removed in a future version.
• In T-SQL, use:
o USE msdb ;
o GO
o EXEC dbo.sp_add_operator
▪ @name = N'OperatorName',
▪ @enabled = 1, -- 1 = enabled, 0 = not enabled.
▪ @email_address = N'EmailAddress'
o There are also pager arguments as well.
• To configure notifications:
o Go to SQL Server Agent (right-hand click it and Start if needed on a VM) – Jobs.
o Right-hand click a job and go to Properties.
o Go to Notifications, and
▪ Select Email, Page(r), "Write to the Windows Application event log" and
"Automatically delete job"
▪ When the job fails, succeeds, or completes (fails or succeeds).
▪ This is for the entire task, not individual steps.
▪ The email and pager need to be already created.
• In T-SQL, use:
o USE msdb ;
o GO
o EXEC dbo.sp_add_notification
o @alert_name = N'NameOfAlert',
Page 104 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
o @operator_name = N'OperatorName',
o @notification_method = 1 ;
▪ 1 = Email, 2 = Pager, 4 = Net Send.
▪ Pager and Net Send have been deprecated.
• To configure Database Mail, you need to:
o Enable Database Mail.
▪ This is not necessary on Azure SQL MI, as it is enabled by default.
▪ In SSMS, go to the Server – Management.
▪ Right-hand click on "Database Mail", and click Configure Database Mail.
o Create a Database Mail account for the SQL Server Agent service account to use.
▪ In SSMS, go to the Server – Management.
▪ Double-click "Data Mail".
▪ In the "Database Mail Configuration Wizard", select "Manage Database Mail
accounts and profiles".
▪ Select "Create a new account".
o Create a Database Mail profile for the SQL Server Agent service account to use and add the
user to the DatabaseMailUserRole in the msdb database.
▪ As above, but select "Create a new profile".
▪ For Azure MI, your profile must be called AzureManagedInstance_dbmail_profile if
you want to send e-mail using SQL Agent jobs.
o Set the profile as the default profile for the msdb database.
▪ In the "Manage Profile Security", "Default Profile" should say "Yes".
70, 71, 72. perform automated deployment methods for resources
• Use an ARM Template to deploy resources.
o Written in JSON.
o https://docs.microsoft.com/en-us/azure/azure-sql/virtual-machines/windows/create-sql-
vm-resource-manager-template?tabs=CLI
• You can also use Azure Cloud Shell
o Using PowerShell
▪ https://docs.microsoft.com/en-us/azure/azure-sql/database/single-database-
create-quickstart?tabs=azure-powershell
▪ https://docs.microsoft.com/en-us/azure/azure-sql/managed-
instance/scripts/create-configure-managed-instance-powershell
o Or CLI (command line interface)
Page 105 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
▪ https://docs.microsoft.com/en-us/azure/azure-sql/database/single-database-
create-quickstart?tabs=azure-cli
• To deploy a database, you can use:
o If you are using an Azure Pipeline, you can use a DACPAC (data-tier application portable
artifact)
▪ This gets added to your azure-pipelines.yml (yml stands for “Yet Another Markup
Language”).
o SQL scripts, together with PowerShell.
70. Automate deployment by using Azure Resource Manager templates (ARM
templates) and Bicep
• An ARM template is a JSON (JavaScript Object Notation) file that defines your project.
• Bicep, a Domain Specific Language (DSL), uses declarative syntax to deploy Azure resources.
• It is an extension to the ARM template language.
• You can use Bicep Extension for VS Code to create and deploy your files.
• To deploy your own custom template:
• Go to the Azure Portal, and search for "Deploy a custom template".
• For ARM, you can use a Quickstart template such as:
• "quickstarts/microsoft.sql/sql-database",
• "quickstarts/microsoft.sql/sqlmi-new-vnet", or
• "quickstarts/microsoft.sqlvirtualmachine/sql-vm-new-storage".
• Alternatively, you can click "Build your own template in the editor".
• Note the "type":
• Microsoft.Sql/servers and Microsoft.Sql/servers/databases or
• Microsoft.Sql/managedInstances
• Microsoft.Computer/virtualMachines and
Microsoft.SqlVirtualMachine/sqlVirtualMachines
• For Bicep:
• For SQL Database, use resource sqlServer
• For SQL MI, use resource managedInstance
• You can convert Bicep to ARM by using the Bicep Playground.
• You can also go from ARM to Bicep by clicking on the "Decompile" button, or in Azure CLI:
• az bicep decompile --file myfile.json
Page 106 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
71. Automate deployment by using PowerShell
• To create an SQL database using PowerShell, use:
Write-host "Creating resource group..."
$resourceGroup = New-AzResourceGroup -Name "PowerShellResourceGroup" -Location
"eastus"
$resourceGroup
Write-host "Creating SQL Database Server..."
$server = New-AzSqlServer -ResourceGroupName "PowerShellResourceGroup" `
-ServerName "sqldatabase220714-5ps" `
-Location "eastus" `
-SqlAdministratorCredentials $(New-Object -TypeName
System.Management.Automation.PSCredential `
-ArgumentList "phillipburton", $(ConvertTo-SecureString -String "MyP@ssw0rd!" -AsPlainText
-Force))
$server
Write-host "Creating SQL Database..."
$database = New-AzSqlDatabase -ResourceGroupName "PowerShellResourceGroup" `
-ServerName "sqldatabase220714-5ps" `
-DatabaseName "mydatabase" `
-Edition Basic
$database
• For SQL MI, use New-AzSqlInstance
• For Azure Virtual Machine, use New-AzVM
72. Automate deployment by using Azure CLI
• To create an SQL database using Azure CLI, use:
echo "Creating resource Group"
az group create --name "CLIResourceGroup" --location "East US"
echo "Creating Server"
az sql server create --name "SQLDatabase220714-2" --resource-group "CLIResourceGroup" --
location "East US" --admin-user "phillipburton" --admin-password "MyP@ssw0rd!"
echo "Creating SQL Database $database"
az sql db create --resource-group "CLIResourceGroup" --server "SQLDatabase220714-2" --name
"MyDatabase" --edition Basic
Page 107 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
• For Azure Managed Instance, use az sql mi
• For Azure Virtual Machine, use az vm create
74. automate database maintenance tasks
• A lot of database maintenance tasks are already automated in Azure, such as updates, backups and
creation of indexes.
• You can create elastic job agents to automate maintenance tasks and/or run T-SQL queries.
o You could: manage credentials, collect performance data or telemetry data.
o Update reference data or load or summarise data from databases or Azure Blob storage.
o Targets can be in different servers, subscriptions or regions, but must be in the same Azure
cloud.
▪ One or more databases, all databases in a server or elastic pool or shard map.
o This is the equivalent of SQL Agent Jobs, which are available in SQL MI, but are not available
in Azure SQL Database.
• You need:
o Elastic Job agent – the Azure resource which runs the jobs. This is free.
o Job database – an existing Azure SQL Database stores job related data, such as metadata,
logs, results and job definitions. It also contains stored procedures and other objects for jobs.
▪ This is charged for as an Azure SQL Database.
▪ You need a Standard (S0 or above) or Premium service tier. S1 or above is
recommended, but if you run frequent jobs or against a big target group, you may
need more.
o Target group – servers, elastic pools, databases and databases of shard map(s) which are
affected.
▪ If a server or elastic group, all databases in the server at the time of running the job
will be affected. You will need to give the master database credential, so the
databases can be enumerated. You can also exclude individual databases or all
databases in an elastic pool.
o Job – unit of work which contained job steps, each of which specify the T-SQL script and
other details.
▪ Scripts must be "idempotent", capable of running twice with the same result.
o Job output – this can be saved in a table.
o Job history – stored for 45 days in jobs.job_executions
• Create the job database.
o An empty S0 or higher database.
o Create a credential for running the jobs in the Job database.
Page 108 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
▪ CREATE MASTER KEY ENCRYPTION BY PASSWORD='<an6?%9++Vyd%Ut9';
▪ CREATE DATABASE SCOPED CREDENTIAL MasterCred WITH IDENTITY = 'MasterU'
SECRET = '<an6?%9++Vyd%Ut9'
▪ CREATE DATABASE SCOPED CREDENTIAL RunJob WITH IDENTITY = 'JobU' SECRET =
'<an6?%9++Vyd%Ut9'
• Create an Elastic Job agent in the Portal.
o Go to the Azure Portal
o Search for Elastic Job agents
o Click Add, enter the name, and click OK.
o Select the Azure SQL job database.
• Define the target group in T-SQL (or PowerShell).
o In the job database:
o EXEC jobs.sp_add_target_group 'GrpDatabase';
o EXEC jobs.sp_add_target_group_member
▪ @target_group_name = 'GrpDatabase',
▪ @target_type = 'SqlDatabase'
- or 'SqlServer', -- or 'PoolGroup'
◼ if wanting to exclude, @membership_type = 'Exclude'
◼ If targeting a server or pool, @refresh_credential_name =
'RefreshPassword',
▪ @server_name = 'DataBaseName.database.windows.net';
o To view the recently created target group and target group members
▪ SELECT * FROM jobs.target_groups WHERE target_group_name='GrpDatabase';
▪ SELECT * FROM jobs.target_group_members WHERE
target_group_name='GrpDatabase';
• In each database, you will need a job agent credential in each affected database. You could use
PowerShell for this.
o In the Master Database:
▪ CREATE LOGIN MasterU WITH PASSWORD ='<an6?%9++Vyd%Ut9'
▪ CREATE USER MasterU FROM LOGIN MasterU
▪ CREATE LOGIN JobU WITH PASSWORD = '<an6?%9++Vyd%Ut9'
o In the target user database:
▪ CREATE USER JobU FROM LOGIN JobU
▪ ALTER ROLE db_owner ADD MEMBER JobU
Page 109 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
• Create a job in T-SQL (or PowerShell) in the Elastic Job database
o EXEC jobs.sp_add_job @job_name='My first job', @description='Look at objects'
• Create job steps in T-SQL (or PowerShell).
o EXEC jobs.sp_add_jobstep @job_name='My first job',
• @command='SELECT * FROM sys.objects',
• @credential_name='RunJob',
• @target_group_name='GrpDatabase'
• Run/schedule the job in T-SQL.
o EXEC jobs.sp_start_job 'My first job' -- run now
o EXEC jobs.sp_update_job
▪ @job_name='Sample T-SQL',
▪ @enabled=1,
▪ @schedule_interval_type='Minutes' – Or Hours, Days, Weeks, Months or Once,
▪ @schedule_interval_count=1
• Monitor job execution in the Portal or T-SQL (or PowerShell).
o select * from jobs.job_executions
75. configure multi-server automation
• See topic 46 for Azure SQL Database.
• For MI and VM, you need a master server and one or more target servers.
o A target server can be linked to only one master server.
Page 110 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
• To create a Master server:
o Right-hand click on SQL Server Agent, and go to Multi Server Administration – Make this a
Master.
o Enter any addresses for notifications.
o Add your target servers (by clicking on "Add Connection", if they are not already registered).
o After checking that the servers are compatible, you can "create a new login if necessary and
assign it rights to the MSX".
▪ MSX is the "Master Server".
• To create a Target server:
o Right-hand click on SQL Server Agent, and go to Multi Server Administration – Make this a
Target.
o Select the Master Server.
o You can "create a new login if necessary and assign it rights to the MSX".
• When creating jobs:
o You can go to the Targets page and select "Target local server" or "Target multiple servers".
75. automate backups
• For VMs, you can create automated backups.
o This is done through the installation of the Sql Server IaaS Agent Extension to enable
automated backups (this can be done through the “Create a Virtual Machine” process).
o Needs to be:
▪ Windows Server 2012 and SQL Server 2014 Standard/Enterprise (for Automated
Backup version 1), or
▪ Windows Server 2012 R2 or higher, and SQL Server 2016 or higher
Standard/Enterprise/Developer (for Automated Backup version 2).
o You can specify:
▪ Retention period – up to 30 days.
▪ Storage account.
▪ Encryption (with password).
▪ Whether system databases (Master, Model and msdb) are backed-up.
▪ Whether you configure a manual or automated backup schedule. Automated backs-
up depends on log growth. If manual, you specify:
- Frequency – Weekly or Daily. If weekly, it will back up each database once
a week, even if it needs to span over several days to do so.
- Backup start time,
- Backup time window (hours), and
Page 111 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
- Log backup frequency (minutes).
o You must use the Full recovery model.
o You can back up the default instance or a single named instance. If there is no default
instance and multiple named instances, it will fail.
75. automate performance tuning and patching
• For PaaS (Azure SQL Database and MI), patching happens automatically.
o You have no control over when it happens, but it has minimal impact if you use “retry logic”.
o If you have a database quorum, there should be at least one primary replica online.
o Business Critical and Premium databases should also have at least one secondary replica
online.
• On VMs, you could use "Automated Patching" – see topic 3.
75. implement policies by using automated evaluation modes
• This is not relevant to Azure SQL Database or MI.
• To create a new policy:
o Go to Management – Policy Management – Policies.
o Right-hand click and go to "New Policy".
o Enter a name for the policy.
o If you are intending to have this run to a schedule, click "Enabled" if you want the schedule
to be enabled.
o Next to "Check condition", click on "New condition".
o In this new box, enter a name, a facet, and what you are checking (at least one field, an
operator and a value).
▪ Note - != means <>.
▪ These conditions are what SHOULD be – the policy will fail if this is NOT the case.
o In the Against targets, select target types. If this is blank, then it will be targeted against the
server.
o In the Evaluation Mode, select:
▪ "On demand",
▪ "On change: prevent" – uses DDL triggers
▪ "On change: log" – logs to event notification.
▪ "On schedule" – select/pick an existing or create a new schedule.
• Once created, if you want it to be run, right-hand click on it and go to Evaluate.
• To edit it, right-hand click and go to Properties.
Page 112 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
76. Automate database workflows by using Azure Logic Apps (not needed from 7
February 2024)
• Logic Apps automate workflows from one connection to another.
• A workflow is multiple steps which define an overall process. IT starts with a trigger, and
continues with multiple actions.
• A similar user interface to Microsoft Automate, part of the Power Platform.
• Examples:
• Send an email when an item in a Sharepoint list is modified
• Get daily reminders emailed to you
• When a new file is created in Dropbox, copy it to OneDrive
• Email yourself (using Outlook) new tweets about a certain keyword
• Get a notification email when Microsoft Defender for Cloud
• detects a threat
• creates a recommendation
• creates a regulatory compliance assessment
• You have a choice of two plans:
• Standard – from around US$180 per month.
• Consumption – about US$1 for every 40,000 actions, and US$1.25 for 10,000 standard
connector executions per day.
• The first 4,000 actions are free.
• Logic Apps are based on triggers and actions.
• Triggers are the first step – why should the workflow start?
• It could be because new data has been added
• When an item is created/modified in SQL Server
• It could be scheduled – every hour, for example.
• Actions
• This is what should happen next. In SQL Server, you have:
• Delete, Get, Insert, Update row
• Get rows or tables
• Execute an SQL query
• Execute stored procedure
• Other actions include:
Page 113 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
• Control – Conditions and Switch (An If statement), For each and Until
(loop).
• Connections
• This is how you get to a data source.
• For Azure SQL type could be:
• Service principal (Azure AD applicationServer, the Authentication ),
• Logic Apps Managed Identity,
• Azure AD Integrated,
• SQL Server/Windows Authentication.
• If it is Azure SQL Database, then you don't need a Gateway. This is for SQL Server on
prem.
• You can combine this with other connections – for example, Azure Storage.
• To view connections afterwards:
• Go to the Logic App – API connections to view the API connections used by the Logic App.
• In the Azure Portal, go to API connections (not Logic App) to view all connections.
• Sample code for Azure SQL Database
CREATE TABLE SalesLT.NewTable
(intvalue int,
messagetext varchar(20),
currenttime datetime DEFAULT GETDATE());
CREATE PROCEDURE HowManyRows
(@numberRows int output) AS
BEGIN
SELECT @numberRows = COUNT(*)
FROM SalesLT.NewTable
END
declare @MyOutput int
exec HowManyRows @numberRows = @MyOutput OUTPUT
select @MyOutput
77. create event notifications based on metrics
• This is for SQL Server VM, as it uses the SQL Server Agent. It is not relevant for Azure SQL Database or
MI (which has SQL Server Agent, but doesn't allow for event notifications).
• To create an event notification in SQL Server.
Page 114 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
• Go to SQL Server Agent (right-hand click it and Start if needed on a VM) – Alerts.
• Right-hand click and go to "New Alert".
• Enter a Name for this alert.
• Select an Alert type:
o SQL Server event
▪ This is based on an error number or severity.
o WMI event alert
▪ This uses the Windows Management Instrumentation to monitor events in SQL
Server.
o SQL Server performance conditions. You select the:
▪ Object, such as Databases or General Statistics.
▪ Counter, such as Transactions/sec.
▪ Instance – a database.
▪ Alert if counter falls below, becomes equal to, or rises above a Value.
• In the Response page, you can:
o Execute an SQL Server Agent job.
▪ You can click New Job, or View [Existing] job (once you have selected one),
o and/or Notify an operator
▪ You can click "New Operator", or View [Existing] operator (once you have selected
one).
• In the Options page, you can:
o Include the alert error text in email or pager,
o Add an additional notification message, and
o Have a delay between responses. 0 minutes and 0 seconds indicate that you want a response
for every occurrence of the alert.
77. create event notifications for Azure resources
• To create an event notification:
o Go to the Azure Portal, and the specific database.
o Go to Monitor – Metrics or Alerts.
o If "Metrics":
▪ Select a metric.
▪ Click "+ New alert rule".
o If "Alerts":
Page 115 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
▪ Click "+ New alert rule".
▪ Select a metric.
o Click on Conditions:
▪ Optionally, select your signal type and monitor service
▪ Select your signal (measure).
▪ Select the Threshold – static or dynamic.
- Dynamic thresholds learns the data and models it using algorithms and
methods, detecting pattern such as seasonality (hourly, daily, weekly).
▪ If static, select:
- the operator (>, >=, < or <=),
- The aggregation type (Avg, Min, Max, Count, Sum), and
- the threshold value.
▪ If dynamic, select
- Select the operator (greater than the upper threshold and/or below the
lower threshold)
- The aggregation type, and
- The Threshold sensitivity:
o High – more alerts based on small deviations. greater than the
upper threshold and/or smaller than the lower threshold).
o Medium (default), and
o Low – fewer alerts based on large deviations.
- You can also select, in Advanced settings:
o The evaluation period, and
o The number of violations to occur within that evaluation period in
order to trigger the alert.
o Under "Evaluated based on", select:
▪ Aggregation granularity period – how often the measures are grouped together,
▪ The frequency of evaluation – how often should it be checked.
• Under to the "Actions" section,
o Select an existing action group, or
o Click on "Create action group", and select:
▪ Email,
▪ SMS (text message),
▪ Azure app Push Notifications, and/or
Page 116 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
▪ Voice.
o And/or select Actions:
▪ Automation Runbook, Azure Function,
▪ ITSM, Logic App, Webhook.
• Under the "Alert rule details", enter:
o The name
o Description (optional),
o Subscription and resource group,
o Severity – from 0 (Critical) to 4 (Verbose),
o Whether it is enabled on creation, and
o Whether the automatically resolve alerts.
▪ The alert period is shown in a different color when "unresolved"
- The line turns from blue to red dots, and the background turns light red as
well.
• To create a Kusto query from logs:
o Go to the Azure Portal, and the specific database.
o Go to Monitoring – Logs.
o Select a query and click "Run".
o Have a look at the results.
Page 117 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Configure and manage automation of tasks
• To create an alert rule
o Click "New alert rule"
o Under Measurement, select:
▪ Measure,
▪ Aggregation type (Average, Total, Maximum or Minimum),
▪ Aggregation granularity (5, 10, 15, 30 or 45 minutes, 1-6 hours, or 1-2 days).
o You can also split by dimensions.
o Enter your "Alert logic":
▪ Operator (>, >=, <, <= or =),
▪ Threshold value, and
▪ Frequency of evaluation (5, 10, 15, 30 or 45 minutes, 1-6 hours, or 1-2 days).
• Go to the "Actions" tab and:
o Select an existing action group, or
o Click on "Create action group", and select:
▪ Email,
▪ SMS (text message),
▪ Azure app Push Notifications, and/or
▪ Voice.
o And/or select Actions:
▪ Automation Runbook, Azure Function,
▪ ITSM, Logic App, Webhook
• Go to the "Details" tab, and:
o Select the severity from 0 (Critical) to 4 (Verbose).
o Enter an "alert rule name" and description (optional).
• You can also do all this by:
o Going to the Azure Portal, and the specific database, and go to Monitoring – Alerts, "+New
alert rule", and selecting:
▪ Resource,
▪ Condition and Alert logic,
- The signal could be a platform metric, or an activity log (an administrative
operation).
▪ Actions Groups, and
▪ Alert Details.
Page 118 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
77. create alerts for server configuration changes
• In MI or SQL Server on VM, changes are made to a server configuration by using sp_configure:
• This tracing may already be enabled. To enable the tracing:
o sp_configure 'show advanced options', 1
o GO
o RECONFIGURE
o GO
o sp_configure 'default trace enabled', 1
o GO
o RECONFIGURE
o GO
• You can see what changes have been made by:
o In SSMS, you can right-hand click on the server instance (not the database),
o Go to Reports – Standard Reports – Configuration Changes History.
• For SQL Database, see topic 77.
77. create tasks that respond to event notifications
• As per 76, using an Error event.
Plan and Implement a High Availability and Disaster Recovery
Environment
79. recommend HADR strategy based on RPO/RTO requirements
• Azure SQL Database offers a Service Level Agreement (SLA) of:
o If using Hyperscale tier:
▪ 99.9% for a zero replicas (8 hours 45 minutes over a year, or 43 minutes 48 seconds
over a month),
▪ 99.95% for one replica (4 hours 22 minutes over a year, or 21 minutes 54 seconds
over a month).
o Other Azure Database tiers:
▪ 99.99% (52 minutes over a year, or 4 minutes 23 seconds over a month) – this is for
other Azure SQL Database tiers and Azure SQL Managed Instance.
▪ However, if you are in Business Critical/Premium tiers, and you have Zone
Redundant Deployments, this increases to 99.995% (26 minutes over a year, or 2
minutes 11 seconds over a month).
- A Database that includes multiple synchronized replicas provisioned in
different Availability Zones
Page 119 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
• For VMs:
o The maximum SLA is 99.99% for the VM itself.
o However, the SQL Server may fail, even though the VM is healthy – so the actual SLA will
lower.
• Terminology:
o RPO – Recovery Point Objective of 5 seconds (how much data you can lose)
o RTO – Recovery Time Objective of 30 seconds (how long until you can use it again –
maximum "Failover" time)
▪ If exceeded, you get a credit of 100% of the total monthly cost of the Secondary
• If you have geo-replication, then you have a guarantee of:
o Geo-restore for geo-replicated backups
▪ RPO – 1 hour, RTO – 12 hours
o Auto-failover groups
▪ RPO – 5 seconds, RTO – 1 hour
o Manual database failover (to geo-replicated secondary)
▪ RPO – 5 seconds, RTO – 30 seconds
4, 80. evaluate HADR for hybrid deployments
• Availability groups
o 2-9 SQL Server instances on VMs or VMs and on-premises data center.
o Data is committed on a primary, then sent out to secondaries.
o You can use synchronous commit for secondary replica in on-prem network.
▪ Transactions are not committed on the primary until they can be committed on the
secondary.
▪ However, asynchronous is also available, for lowest patency or for geographically
spread secondaries.
o You need a domain controller VM, as it requires an Active Directory domain.
o Availability replicas running Azure VMs allow for DR. It uses an asynchronous commit.
▪ Transactions are committed on the primary before being replicated on the
secondary.
o You also need a VPN connection for the entire failover cluster, using a multi-subnet failover
cluster.
o For DR purposes, you also need a replica domain controller at the disaster recovery site.
o You fail over at the database level, not the instance.
• Database mirroring
Page 120 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
o An Azure VM running at least SQL Server 2012, and another SQL Server running on-prem
running at least SQL Server 2008R2, using server certificates.
o No VPN required, and they don't have to be in the same Active Directory domain (but you
can – but you will ned a VPN and a replica domain controller).
• Backup and restore using Azure Blog storage
• Replicate and fail over SQL Server to Azure with Azure Storage.
• Log shipping
o An Azure VM and a SQL Server on-prem.
o As log shipping requires Windows file sharing, you would need a VPN tunnel.
o The secondary database is stored on a secondary server or warm standby.
• These are also available for Azure VMs only configurations.
• Evaluate HADR for hybrid deployments
Availability groups
• Evaluate HADR for hybrid deployments
Database mirroring / Azure Blob Storage / Azure Site Recovery
Page 121 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
• Evaluate HADR for hybrid deployments
Log shipping
• Failover Cluster Instances (FCI) for Azure VMs
o Allows for HA, but not DR.
▪ Designed to protect against network card or disk failure – but there are other
solutions in Azure.
o There are 5 different configurations:
▪ Azure shared disks for Windows Server 2019.
Page 122 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
- Attach a managed disk to multiple VMs at the same time.
▪ Storage Spaces Direct (S2S) for a Storage Area Network for Windows Server 2016 or
later.
▪ Premium File Share for Windows Server 2012 or later. Uses SSD, have low latency,
supported for Failover Cluster Instances.
▪ Using third-party solutions.
▪ Using Azure ExpressRoute.
• For Azure SQL Database and MI:
o They use locally redundant availability as standard.
• For MI:
o You can also configure a Failover Group
• For Azure SQL Database:
o For more, see topic 94.
o Automatic Asynchronous Replication
▪ Secondary Database populated with primary database ("seeding").
▪ Updates are then replicated automatically.
o Readable secondary databases (replication).
▪ To create a replica, go to Data Management – Replicas – Create replica.
▪ You can have up to 4 per primary database.
- Can be in different regions.
- Want more? You can have a secondary of a secondary. Replication to that
secondary takes longer.
▪ They need to have at least the same service tier as the primary.
- Need to change? When upgrading, upgrade the secondary first. When
downgrade, downgrade the primary first.
- You don't need to disconnect the secondaries unless you change between
General Purpose and Business Critical.
▪ More than 1 secondary means that, even if one fails, there will still be at least one
until it is recreated.
▪ Uses snapshot isolation mode, so updates from the primary are not delayed by
long-running queries on the secondary.
• Azure Site Recovery service
o Simple DR (but not HA) of Azure VMs from a primary to a secondary region.
o Can also replicate on-prem VMs/servers to Azure or a secondary on-prem datacenter.
Page 123 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
o Not SQL Server solution, but can be used with SQL Server on the VMs.
o Provides continuous replication.
o Can replicate using recovery point snapshots; they capture disk data, data in memory, and
transactions in process.
o Run DR drills, planned failovers with zero-data loss, or unplanned failovers.
o Useful to protect against ransomware.
• identify resources for HADR solutions – Replicas/ geo-replication
• identify resources for HADR solutions – failover groups
Page 124 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
82. test HA by using failover – FAILOVER GROUP
82. test DR by using failover or restore
• To create a failover group
o Go to the SQL Database server (not the database).
o Go to Settings – Failover groups – Add groups.
o Create a failover group.
▪ Enter a unique failover group name,
▪ a server (or create a new server),
▪ The Read/Write failover policy (automatic or manual), and
▪ the Read/Write grace period (1-24 hours).
Page 125 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
o You can then add eligible databases into the failover group.
• To update the failover group settings:
o Click on Settings – Failover groups – and the name of the failover group.
o You can "edit the configuration" (read/write failover policy and grace period).
o You can "add databases" into the failover group.
o Once you have done all of your changes, click "Save" or "Discard".
• To test the failover:
o Click on Settings – Failover groups – and the name of the failover group.
o Click on Failover (or Forced [manual] failover).
▪ Forced failover risks possible data loss.
o All databases within a failover group are then fail-overed.
o You can also use the PowerShell cmdlet:
▪ Invoke-AzSqlInstanceFailover -ResourceGroupName "ResourceGroup01" -Name
"ManagedInstance01"
- This failovers an Azure SQL Managed Instance.
▪ Invoke-AzSqlDatabaseFailover -ResourceGroupName "ResourceGroup01" -
ServerName "Server01" -DatabaseName "Database01"
- This failovers a single database in an Azure SQL Database, which could be a
single database in an elastic pool (without affecting the other databases in
the same elastic pool).
▪ Invoke-AzSqlElasticPoolFailover -ResourceGroupName "ResourceGroup01" -
ServerName "Server01" -ElasticPoolName "ElasticPool01"
- This failovers all databases in an elastic pool.
82. test HA by using failover – AVAILABILITY GROUP
82. test DR by using failover or restore
• To manually fail over an availability group:
o Go to SSMS and the server which hosts a SECONDARY replica of the availability group.
o Go to AlwaysOn High Availability – Availability Groups.
o Right-hand click the availability group to be failed over, and click on "Failover".
o If the Introduction page of the wizard says "Perform a planned failover for this availability
group", then you can do this without data loss.
o In the "Select New Primary Replica" page, you can view the status of:
▪ The primary replica,
▪ The Windows Server Failover Cluster quorum status:
Page 126 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
- Normal quorum
- Forced quorum
- Unknown quorum, and
- Not applicable.
o The secondary replicas can say:
▪ "No data loss" – this is a planned manual failover,
▪ "Data loss, Warnings (X)", where X shows the number of warnings – this would have
to be a forced failover.
o The relevant secondary replica will then become the new primary replica.
o In the "Connect to Replica" page, you can connect to the failover target.
• In T-SQL, you can use:
o ALTER AVAILABILITY GROUP MyAg FAILOVER;
84. perform a database backup with options
• Performing a database backup with options is primarily for VMs.
o MIs can do some options, but it will be a Copy Only backup.
• The General page shows:
o Recovery model.
o Backup type (Full, Differential or Transaction Log [not for Simple]),
o Copy-only backup (this is independent of the sequence of other backups).
o Backup component:
▪ Database (all the database), or
▪ Specific files and filegroups.
o Back up to:
▪ Disk (System file or disk-based logical backup device),
▪ Tape (local or tape-based logical backup device) – this is deprecated.
▪ URL – Microsoft Azure Blob storage
▪ Contents shows the media contents for the selected disk/tape (not URL).
• The Media Options page shows:
o Backup to the existing media set
▪ Append to the existing backup set, preserving any prior backups
▪ Overwrite all existing backup set, replacing prior backups with the current backup.
▪ Check media set name and backup set expiration – requires the backup operation to
verify name and expiration date.
Page 127 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
- Optionally, set the media set name.
o Backup to a new media set, and erase all existing backup sets.
▪ Optionally, set the new media set name and description.
o Reliability
▪ Verify backup when finished.
▪ Perform checksum before writing to media
▪ Continue on error (even after encountering one or more errors)
o Transaction log
▪ Backup the transaction log and truncate it to free log space. The database remains
online.
▪ Backup the transaction log tail (tail-log backup), and leave the database in a
restoring state (not available to users until it is completely restored).
o Tape drive (deprecated)
▪ Unload the tape after backup, and
▪ Release and rewind the tape before unloading.
• The Backup set page shows:
o Name and description of the backup set name
o Expiration duration or date:
▪ After X days; if X is zero, the backup set will never expire.
▪ Specific date.
o For the Enterprise edition, you can select backup compression.
o Encrypt backup, using AES 128, AES 192, AES 256 and Triple DES.
▪ Only enabled if you append to existing backup set. Backup your certificate or keys to
a different location.
• If you have a VM with IaaS Extension can configure backups in the Azure Portal.
• You can:
o Enable/Disable the backups of system databases, and
▪ Configure a backup schedule – Automated or Manual,
▪ Backup frequency – Daily or Weekly,
▪ Backup start time (local VM time),
▪ Full backup time window (hours),
▪ Log backup frequency (minutes).
• Restores need to be configured from within SQL Server.
Page 128 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
85. perform a database restore with options
• The user must have CREATE DATABASE permissions.
o This exists in sysadmin and dbcreator fixed server roles, and dbo (owner) for existing
databases.
• The General page shows:
o Source
▪ Database – this list only contains databases backed up, based on the msdb backup
history.
▪ Device – tape, URL or file. This is required if the backup was taken on a different SQL
Server instance.
- You can select up to 64 devices that belong to a single media set.
▪ Device database – backups on the selected device.
o Destination
▪ Database to restore.
- Enter a new database, or choose an existing database from the dropdown
list, which includes all databases on the server, excluding master and
tempdb.
▪ Restore to.
- It is "To the last backup taken" by default.
- Alternatively, you can select [Backup] Timeline, which shows the database
backup history as a timeline.
o Restore plan
▪ Backup sets to restore.
- The default is the recovery plan suggested to achieve the goal.
o Verify backup media.
▪ Check the integrity of the backup files prior to restoring them.
• The Files page shows:
o Relocate all files to [a particular] folder, showing
▪ Logical and Original File Name,
▪ File Type,
▪ The file path/name to "Restore As".
• The Options page shows:
o Overwrite the existing database
▪ Overwrite database files.
Page 129 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
o Preserve the replication settings
▪ Only relevant if a database was replicated when the backup was created, and when
restoring a published database to a different server (other than the creation server).
o Restrict access to the restored database.
▪ Only for db_owner, dbcreator and sysadmin members.
o Recovery state:
▪ Restore with recovery. Default option.
- Only choose this option in a full or bulk-logged recovery model if you are
also restoring all log files at the same time.
▪ Restore with NoRecovery
- Left in the Restoring state. Allows for additional backups.
▪ Restore with Standby
- Limited read-only access.
- Need to specify a Standby file, which allows the recovery effects to be
undone.
o Tail-log backup.
▪ Take tail-log backup before restoring.
- You can specify a backup file for the tail-log.
o Server connections
▪ Restore options may fail if there are active connections to the database.
o Prompt before restoring each backup.
▪ The "Continue with Restore" dialog box will be displayed after each backup is
restored.
▪ If you click "No", the database will be left in the Restoring state.
86. perform a database restore to a point in time
• For VM, see topic 91.
• For Azure SQL MI, to restore an Azure SQL database to a different region:
o Go to the MI, click on "+New database", select the database name, and change "Use existing
data" to "Backup" and select the backup.
• Database backups for Azure SQL Database and Azure SQL MI are done automatically.
o Full backups every week,
o differential backups every 12-24 hours, and
o transaction log backups every 5-10 minutes.
o You can do a:
Page 130 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
▪ point-in-time restore (PITR) of existing or deleted databases – 7 days by default
- You can change it to 1-35 days optionally (apart from Hyperscale and Basic
tier databases – basic has a maximum of 7 days).
- Note: In MI, PITR is available for individual databases, but not for the entire
instance.
▪ recover to another geographic region - same
▪ restore from a long-term backup – up to 10 years.
o The first backup is scheduled immediately after a new database is created or restored.
o For non-large database, it usually completes within 30 minutes.
• To restore a database:
o Go to the Database overview page.
▪ If recovering a deleted database, go to the server or MI, and click "Deleted
database" (on the left-hand side).
o Click "Restore".
o Choose backup source.
o Select the Point-in-time backup point, and then OK.
o You cannot restore over an existing database (but you can rename it afterwards).
o You can use PowerShell cmdlets to restore an existing database, but again, you can't restore
over an existing database.
• To restore an Azure SQL database to a different region:
o Create the database.
o In the "Additional settings" tab, change "Use existing data" to "Backup", and select a backup.
87. configure long-term backup retention
• This is for both Azure SQL Database and Azure SQL MI.
o It is in Public Preview in SQL MI in Azure Public regions only.
• Backups can also be configured for Long-Term Retention (LTR).
• LTR backups are done by Azure.
Page 131 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
o You cannot control the timing nor manually create a LTR backup.
o It may take up to 7 days before the first LTR backup will be shown in the list of available
backups.
o Ensure that you have a LTR policy on secondary databases, only to be created when they
become primary.
o Backups are stored in Azure Blob storage – a different storage container weekly.
• To configure this, go to Azure portal – the server – Backups – Retention policies – select the
database(s), and configure the LTR:
o Weekly backups,
o Monthly backups,
o Yearly backups and
o WeekOfYear backups.
• To view backups, go to Azure portal – the server – Backups – Available backups – and next to the
relevant database, under “Available LTR backups”, select Manage.
o You can click on an LTR backup, and select Restore (which creates a new database) or Delete.
• Once the original database is deleted…
o No more backups are made.
o As backups expire, they are deleted.
88. perform transaction log backup
• This only applies to VMs.
• You need to be using full or bulk-logged recovery models.
o Simple recovery models do not use transaction log backups.
o To change recovery model:
▪ ALTER DATABASE NameOfDatabse
▪ SET RECOVERY FULL | BULK_LOGGED | SIMPLE
▪ GO
• You need the BACKUP DATABASE and backup log PERMISSIONS.
o They are already granted in the sysadmin fixed server role, and the db_owner and
db_backupoperator fixed database roles.
• Use the following command:
o BACKUP LOG NameOfDatabase
o TO MyPreviouslyCreatedNamedBackupDevice
o NORECOVERY, NO_TRUNCATE
Page 132 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
▪ NORECOVERY backups the tail of the log and leaves the database in the RESTORING
state.
- Useful when failing over to a secondary database or when saving the tail
before a RESTORE.
▪ NO_TRUNCATE causes SQL Server to attempt to backup, regardless of the state of
the database.
- Useful if the database is damaged.
▪ Suggest using NO_TRUNCATE and NORECOVERY together.
o GO
88. perform restore of user databases
• In SQL Server in an Azure VM, you can:
o Perform a complete restore or partial report.
o Restore to a point in time.
o You can also do the following, which we don't need to cover:
▪ Restore specific file(s), filegroup(s) or page(s).
▪ Restore a transaction log, or
• Use:
o RESTORE DATABASE NameOfDatabase
o FROM MyPreviouslyCreatedNamedBackupDevice
o [WITH RECOVERY | NORECOVERY]
o [FILE = BackupSetFileNumber]
▪ NORECOVERY is useful when you are restoring a single file, but you need to restore
more.
▪ Use RECOVERY when you have finished restoring, and you want the database to be
online.
o [STOPAT = { 'datetime'| @datetime_var }
▪ | STOPATMARK or STOPBEFOREMARK = { MarkName | LSNNumber } [ AFTER
'datetime']
• For example:
o RESTORE … WITH FILE = 6 NORECOVERY, STOPAT = 'Jun 19, 2024 12:00 PM';
o RESTORE … WITH FILE = 9 RECOVERY;
o RESTORE VERIFYONLY FROM …
▪ Verifies the backup but does not restore it.
• You can only use T-SQL in an MI when doing a complete restore from an Azure Blob Storage Account:
Page 133 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
o RESTORE DATABASE NameOfDatabase
o FROM URL = 'https:// … ' , 'https:// … ' (etc)
o It can only be restored onto another MI – not an on-prem SQL Server.
88. perform database backups with options
• MIs have automatic backups. You can create full database COPY_ONLY backups, but not differential,
log or file snapshot backups.
o BACKUP DATABASE NameOfDatabase
o TO URL = 'https:// … ' , 'https:// … ' (etc)
▪ The URLs is for the Microsoft Azure storage service.
▪ Maximum backup stripe size (blob size) is 195 Gb.
- If you want more space, add additional files.
o WITH COPY_ONLY
▪ Does not interfere with the normal sequence of backups.
o [COMPRESSION | NO_COMPRESSION]
▪ This overrides the server-level default. The default is no backup compression.
o [STATS = X]
▪ Displays a message every X% percentage. The default is 10 per cent.
• This is for VMs (and Mis if using COPY_ONLY). The syntax is:
o BACKUP DATABASE NameOfDatabase
o [FILEGROUP = 'X', FILEGROUP = 'Y' …]
▪ Useful for backing-up only part of a database.
o TO MyPreviouslyCreatedNamedBackupDevice
▪ If backing up to disk, then use TO DISK = 'FileLocation'.
▪ You can also use TO TAPE = or TO URL = 'https://…'
o [MIRROR TO AnotherBackupDevice]
▪ Only for the Enterprise edition of SQL Server.
▪ Must be the same time as the Primary Backup.
▪ You can have up to 3 secondaries.
o [WITH
▪ COPY_ONLY
- Creating a full backup, but is not treated as a full backup for purposes of
future DIFFERENTIAL or TRANSACTION LOG backups.
▪ DIFFERENTIAL
Page 134 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
- All the changes since the last FULL backup.
- Without DIFFERENTIAL or LOG, it is a FULL backup.
▪ COMPRESSION | NO_COMPRESSION
- Only use in the Enterprise edition. Overrides the server-level default
▪ DESCRIPTION – up to 255 characters
▪ NAME – up to 128 characters. Default is blank.
▪ CREDENTIAL
- Used only when creating backups to Azure Blobs.
▪ ENCRYPTION
- Choose from AES_128, AES_192, AES_256, TRIPLE_DES_3KEY or
NO_ENCRYPTION (default).
- If you encrypt, you will also need to use SERVER CERTIFICATE or SERVER
ASYMMETRIC KEY.
▪ FILE_SNAPSHOT [EXPIREDATE = 'date' | RETAINDAYS = days]
- Used when creating a snapshot of the database files and storing them into
Azure Blobs.
• [WITH
o NOINIT | INIT
▪ Whether the backup operation appends to/overwrites the existing backup sets on
the backup media. The default is NOINIT (append).
o NOSKIP | SKIP
▪ Checks whether a backup operation checks the expiration date and time of the
backup sets on the media before overwriting them. The default is NOSKIP (Check
the date/time).
o NOFORMAT | FORMAT
▪ Whether the media header should be written on the volumes used for the backup
operation, overwriting any existing media header and backup sets. The default is
NOFORMAT.
- Be careful from using FORMAT, as it renders the entire media set unusable.
- FORMAT implies SKIP.
o NO_CHECKSUM | CHECKSUM
▪ Whether backup checksums are enabled – this validates the backup. The default is
NO_CHECKSUM (no generation of backup checksums).
o STOP_ON_ERROR | CONTINUE_AFTER_ERROR
Page 135 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
▪ Whether BACKUP stops if there is a page checksum error. The default of
STOP_ON_ERROR (stop if it doesn't verify). CONTINUE_ON_ERROR is best used
when there are checksum errors.
o STATS = X
▪ Whether a percentage is displayed every X% (the default is 10%).
o REWIND | NOREWIND
▪ Whether a TAPE device is released and rewound. Default is REWIND (yes).
o UNLOAD | NOUNLOAD
▪ Whether the tape is rewound and unloaded. Default is UNLOAD (yes).
• [WITH
o When using BACKUP LOG:
▪ NORECOVERY
- Backs up the tail of the log and leaves the database in the RESTORING
state. Useful when failing over to a secondary database or when saving the
tail of the log before a RESTORE operation.
▪ STANDBY = standby_file_name
- Backs up the tail of the log and leaves the database in a read-only and
STANDBY state.
- The file holds the rolled back changes.
▪ NO_TRUNCATE
- The log is not truncated and requires SQL Server to attempt to backup
regardless of the state of the database.
- Is generally used if the database is SUSPENDED – when the database has
failed.
88a. backup and restore to and from cloud storage
• This can be using either Azure SQL Managed Instance or an SQL Server.
• You can create cloud storage using Azure Blob Storage.
• Create a Storage Account.
• Go to Data storage - Containers, and click on "+ Container", and create a new
container.
• Then you can create a credential.
• Right-hand click on your database and go to Tasks – Back up…
• In Destination – Back up to, choose URL, then select Add.
• Click on "New container", and then sign-in, and select your subscription,
storage account and container.
• Click "Create Credential". This generates a Shared Access Signature (SAS).
• Then click OK.
• Select the Blob container, and click OK, and OK.
Page 136 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
• To restore the database:
• Right-hand click on Databases and select Restore Database.
• Click on … next to Device.
• Select URL, and click Add.
• Select the container, and click OK.
• Expand Containers and select the relevant container, and the backup files.
• If you can't see any backup files, you may be using the wrong SAS key.
• Click OK, then OK.
90. create an Availability Group
• You need:
o A resource group with a domain controller.
o One or more domain-joined VMs in Azure running SQL Server 2012+ Enterprise, or SQL
Server 2016+ Standard in:
▪ The same availability set (different computers in the same datacenter), or
▪ Different availability zones (physical datacenters).
▪ They need to be registered with the SQL IaaS Agent extension in full manageability
mode and are using the same domain account for the SQL Server service on each
VM.
o Two available (not used by any entity) IP addresses.
▪ One for the internal load balancer.
▪ One for the availability group listener within the same subnet as the availability
group.
• An availability group supports:
o A set of primary replica (which host the primary databases), and
o 1-8 sets of secondary replicas (only 1 allowed in SQL server Standard), each of which hosts
the secondary databases (this does not replace backups).
o There must be at least 2+ failover partners.
• Note:
o The primary replica send transaction log records to every secondary database ("data
synchronization").
o Individual primary/secondary databases can be suspended or fail without affecting other
primary/secondary databases.
o You can configure 1+ secondary replicas to support read-only access to secondary databases,
and/or to permit backups on secondary databases.
• Availability modes are:
Page 137 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
o Asynchronous-commit mode. The primary replica commits without acknowledgement that
the secondary replica has committed.
▪ Minimizes transaction latency, but there is a lag for when the data is committed
onto the secondaries.
o Synchronous-commit mode. The primary replica does not commit until the secondary replica
has hardened the log.
• Failover:
o This is when the target secondary replica transitions to being the new primary replica.
o The former primary databases becomes secondary databases.
o The failovers are as follows:
▪ Planned manual failover – no data loss – secondary replica needs to be
synchronized – for synchronous-commit mode only.
▪ Automatic failover – no data loss – occurs when there is a failure to the primary
replica – for synchronous-commit mode only. Needs to have a Windows Server
Failover Cluster quorum and be synchronized.
o Forced manual failover (also known as "forced failover"). For asynchronous-commit mode.
This is a DR option.
▪ The only type of failover that is possible if the target secondary replica is not
synchronized with the primary replica.
o After failover, Azure SQL connections are automatically redirected to the new primary node.
• To create a new cluster in the Portal:
o In Azure portal, go to the VM – Settings – High Availability.
o Click on "+ New Windows Server Failover Cluster".
o Name the cluster, and give a Storage Account which is the Cloud Witness.
▪ Storage Account name: 3-24 characters using numbers and lower-case letters.
o In "Windows Server Failover Cluster", provide credentials for:
▪ The SQL Server service account,
▪ The cluster operator, and
▪ The Bootstrap account.
o Select the VMs to be added into the cluster.
▪ A restart may be required.
▪ Only relevant VMs will be shown.
o Click Apply.
• To create an availability group:
o In Azure portal, go to the VM – Settings – SQL Server configuration – Open - High Availability.
Page 138 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
o Click on "+ New Always On availability group".
o Name the availability group.
o Click "Configure listener".
▪ Use an existing load balancer, or
▪ Click "Create new".
▪ For the Load balancer:
- Type: "Internal" allows apps in the same Virtual Network to connect to the
availability group.
- Type: "External" allows apps to connect to the availability group through a
public internet connection.
- "IP address assignment" should be Static.
- The "Resource group" and "Location" should be that where the SQL Server
instances are in.
▪ For the Listener:
- The Listener name should not exceed 15 characters.
- The Listener Port defaults to 1433.
- The Probe Port is for the internal load balancer, which is 59999 by default.
- DHCP (Dynamic Host Configuration Protocol) is not recommended in a
production environment.
o Click "Apply".
o Click "+Select replica".
o Select the VMs to be added into the availability group.
o Click "Apply".
• In the Azure Portal – Settings – High Availability, the status of the availability group(s) are shown.
• However, you can also use the SQL Server to do this as well – and this is the way I do this in the videos
to this course.
90. prepare databases for Always On Availability Groups
• A secondary database need to be identical to the primary database.
o Therefore, do a BACKUP and RESTORE, including tail-log backups.
• Then join it – for example:
o ALTER DATABASE Db1 SET HADR AVAILABILITY GROUP = MyAG;
90. integrate a database into an Always On Availability Group
• The database must exist on the server instance that hosts the primary replica.
o The primary database should use the full recovery model.
Page 139 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
o The name of the secondary database must be the same as the primary database.
o Secondary databases do not exist until backups of the new primary database are restored to
the secondary replicas (use RESTORE WITH NORECOVERY).
• In SSMS
o Connect to one of your SQL Server VMs using (for example) RDP.
o In SSMS, go to your SQL Server instance – Always On High Availability – Availability Groups.
o Right-hand click on your availability group and select "Add Database…".
o Add the database(s) to your availability group.
o Click "OK".
• In T-SQL (see topic 108)
o ALTER AVAILABILITY GROUP MyAG ADD DATABASE MyDb3;
o GO
o ALTER DATABASE Db1 SET HADR AVAILABILITY GROUP = MyAG;
90. configure an Always On Availability Group listener
• See topic 95.
• To configure it in SSMS,
o Go to the server instances that hosts the primary replica.
o Go to Always On High Availability – Availability Groups.
o Right-hand click on "Availability group Listeners", and click on "Add Listener".
o Enter the listener DNS name – in SSMS, that is up to 15 letters, numbers, hyphens and
underscores.
o The TCP port used by the listener.
o Select the TCP Protocol used by the listener, either:
▪ Dynamic Host Configuration Protocol (DHCP) – not recommended, or
▪ Static IP.
- You must specify a static IP address for every subnet that hosts an
availability replica, including Subnet and IP Address.
91. configure auto-failover groups
• Used in both Azure SQL Database and MI (but not VM).
• Auto-failover will not happen for at least 1 hour.
o It may be only a minor outage.
o When it happens, failover itself takes a few seconds.
• Use auto-failover groups when:
Page 140 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
o It is mission critical,
o Your SLA does not allow for 12+ hours of downtime
▪ Geo-restore has a Recovery Time Objective of 12 hours.
o You don’t want to lose up to 1 hour of data (as per Geo-restore).
▪ Auto-failover groups has a Recovery Point Objective (RPO) of 5 seconds.
o Geo-replication is more costly than needed for the requirements.
92. configure quorum options for a Windows Server Failover Cluster
• Always On availability groups and Failover Cluster Instances rely on the underlying Windows Server
Failover Clustering (WSFC) service.
• It monitors network connections and the health of the nodes (clustered servers)
• A two-node cluster will function without a quorum resource, but its use is recommended.
o This then provides an odd number of votes, and a 3 quorum votes minimum.
• To configure:
o Open Failover Cluster Manager.
o Right-hand click the cluster, and go to More Actions – Configure Cluster Quorum Settings.
o In the wizard, select "Select the quorum witness".
o Select the type of witness.
• There are 3 options:
o Cloud Witness
▪ Needs Windows Server 2016+
▪ Uses Microsoft Azure to provide a vote on cluster quorum.
▪ Ideal for deployments in multiple sites, zones and/or regions.
▪ Only 1Mb.
▪ Recommended to use whenever possible, unless you have a failover cluster solution
with shared storage.
▪ Use General Purpose and Standard Storage (Blob storage and Premium storage are
not supported).
▪ Use "Locally redundant storage" for Replication type.
▪ Uses port 443 (HTTPS) for communication.
▪ You will need:
- The Azure Storage Account Name,
- Primary Access Key corresponding to the Storage Account, and
Page 141 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
- The endpoint server name, if you are using a different Azure service
endpoint, such as Microsoft Azure in China.
▪ Once finished, you can see this witness in the Failover Cluster Manager snap-in.
o Disk Witness.
▪ A small clustered disk in the Cluster Available Storage group.
▪ The disk is highly available (most resilient) and can fail over between nodes.
▪ Less than 1Gb.
▪ Only can be used with a cluster which uses Azure Shared Disks.
▪ Cannot be a Cluster Shared Volume.
o File share witness
▪ Configured on a file server running Windows Server.
▪ A file share on a separate VM in the same Virtual Network.
▪ Needs to be separate from the cluster workload, to allow equal opportunity to
other clusters.
▪ Only use if you can't use the other 2 options.
• In "Advanced quorum configuration", you have these options:
o Select Voting Configuration.
▪ By default, all nodes have a vote, but you can assign votes to only some nodes.
▪ You could also have "No nodes", which then the same as "No majority (disk witness
only)" – see below.
• Your cluster will then be configured in:
o Node majority (no witness).
▪ The cluster quorum is the majority of voting nodes in the active cluster
membership.
o Node majority with witness ("Node and File Share Majority" or "Node and Disk Majority")
▪ Nodes have a vote, and the witness also has a vote.
o No majority (disk witness only).
▪ Only the disk witness has a vote.
▪ Not recommended because it is a single point of failure.
Page 142 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
93. Configure failover cluster instances on Azure VMs
VM for domain controller
• Create new resource – SQLHADR
• Virtual machine name – VMDOMAINCONTROLLER
• Image – Windows Server 2019 Datacenter – Gen1
• Select inbound ports – RDP and HTTP
• Virtual network – SQLHADR-vnet
VM for SQLSERVER1
• Virtual machine name: SQLSERVER1
• Resource group – SQLHADR
• Image – Windows Server 2019 Datacenter
• Virtual network – SQLHADR-vnet
VM for SQLSERVER2
• Virtual machine name: SQLSERVER2
• Resource group – SQLHADR
• Image – Windows Server 2019 Datacenter
• Virtual network – SQLHADR-vnet
Connect to VMDOMAINCONTROLLER
• Go to VM – VMDOMAINCONTROLLER – Connect
• Enter credentials.
• Do you want your computer to be discoverable by other PCs and devices on this network? Yes
• Server Manager – Manage – Add/Remove Roles and Features
Page 143 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
• Server Selection – note IP Address (X.Y.0.4)
• Server Roles – Active Directory Domain Services
• Promote VM as a Domain controller.
• Add a new forest – FILECATS.CO.UK
• Enter the Directory Restore password.
Join VM1 to DC
• Go to SQLSERVER1 (then SQLSERVER2) – Networking – Network Interface hyperlink – DNS servers –
Custom – enter the DNS server IP address – X.Y.0.4
• Start SQLSERVER1 and SQLSERVER2.
• Then connect to the computers.
• Do you want your computer to be discoverable by other PCs and devices on this network? Yes
• Change to the domain FILECATS.CO.UK – go to Windows Explorer - My PC
Install Windows Failover Cluster Role in SQLSERVER1 and SQLSERVER2
• In SQLSERVER1 and SQLSERVER2: Server Manager – Manage – Add/Remove Roles and Features
• Features – Failover Clustering
• After reboot, go to All Services, Right click on SQLSERVER1 and select “Failover Cluster Manager”.
• (There are no clusters created).
• Actions – Create cluster.
• Add SQLSERVER1 and click Browse and add SQLSERVER2.
• Enter a cluster name: SQLCLUSTERNAME
• Create the cluster.
• Close the window. There is a new cluster.
• You can click on Nodes to see the 2 computers.
SQL Server management
• SQL Server Configuration Manager 2019
• Right-hand click on SQL Server (in SQL Server Services) – go to the “Always On Availability Groups”
and check “Always on Availability Group”.
• Restart SQL Server (in SQL Server Configuration Manager)
• Go to SQL Server Configuration Manager – SQL Server Network Configuration – Protocols – TCP/IP
and Enable.
• Restart SQL Server (in SQL Server Configuration Manager)
• Go to Windows Defender Firewall – New Rule – Port – TCP 1433 (all others as default).
• Do the same in SQLSERVER2.
Page 144 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
• Can you open SSMS in SQLSERVER1 and connect to SQLSERVER2 database?
Create Storage Account for cloud witness
• Portal – Storage Account
• New
• Storage account name: sqlstoragewitness.
• Select “Local redundancy option”.
• Create review + create.
• Go into the Storage account – Access keys.
Configure Witness
• Go to Failover Cluster Manager – the actual failover cluster (in my case, SQLCLUSTER.filecats.co.uk)
• On the right-hand sideMore Actions – Configure Cluster Quorum Settings
• Click Next.
• Click “Select the quorum witness”
• Click “Configure a cloud witness”.
• In the Storage account:
• Copy storage account name and storage account key.
• Click Next x 3.
• The Cloud Witness is now in the “Cluster Core Resources”.
Prepare databases for Always On Availability Groups – SQLSERVER1 (97, 107)
• Create database with table with demo data.
• Go to Always On High Availability – right hand-click on Availability Groups – go to “New Availability
Group Wizard”.
• Next
• Enter an availability group name: SQLAVAILABILITYGROUP
• Leave the cluster type as “Windows Server Failover Cluster”.
• Click next – find we need a full backup.
• Create the backup (right-hand click on the database – Tasks – Back Up…)
• Redo the “New Availability Group Wizard”.
• Select the databases.
• Select the replicas… Click “Add Replica” and log into SQLSERVER2.
• Look at availability mode, automatic failover, and readable secondaries. (Synchronous good if you
have close physical distance.)
Page 145 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
• Select Initial Data Synchronization – automatic seeding, full database and log backup, join only, or
skip.
• Finish the wizard (It’s OK for the purposes of the DP-300 course if the listener configuration has a
warning).
Add listener.
• In SSMS, go to Always On High Availability – Availability Groups – NameOfGroup – right-hand click on
Availability Group Listener – Add a Listener.
• Give it a name -SQLAG
• Port – 1433
• Network Mode – Static – click Add.
• Enter an iPv4 Address – X.Y.0.20.
Test failover
• Go to Always On High Availability – Availability Group – SQLAVAILABILITYGROUP
• Check which is Primary and which is Secondary.
• “Start Failover Wizard”.
• Start in the Primary.
• Select the new Primary Replica (which is currently a secondary).
• Click “Connect to Replica”, and click “Connect” to enter credentials.
• Finish.
• Check which is Primary and which is Secondary.
94. configure replication
• You can use transactional replication to push changes made in an Azure MI to:
o A SQL Server database (on prem or Azure VM),
o An Azure SQL Database, or
Page 146 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
o An instance database in Azure SQL MI
• Useful for:
o Distributing changes to one or more databases in SQL Server, Azure SQL MI or Azure SQL
Database.
o Keep several distributed databases synchronized.
o Migrate databases from SQL Server/Azure SQL MI to another database by continuously
published the changes.
• The components of a transactional replication are:
o Publisher
▪ Publishes changes made on some tables ("articles"), and send the updates to the
Distributor.
▪ Can be Azure SQL MI or an SQL Server instance.
▪ Cannot be Azure SQL Database (need to use Data Sync – topic 14 – for this).
o Distributor
▪ Collects changes from Publisher and distributes them to Subscribers.
▪ Can be Azure SQL MI or an SQL Server instance.
- Can be the same Azure SQL MI as the Publisher, but a different database.
▪ If SQL Server instance, version needs to be the same or higher than the Publisher
version.
o Pull subscriber
▪ Can be Azure SQL MI or an SQL Server instance, but needs to the same type as the
Distributor.
o Push subscriber.
▪ Can be an Azure SQL Database.
- However, it only supports Standard Transactional and Snapshot.
▪ Can be an Azure SQL MI.
- It supports Standard Transactional, Snapshot and Bidirectional.
▪ Can be an SQL Server instance.
- Needs to be more recent than the Publisher, or no more than 2 versions
earlier.
• Create a Publication:
o In SSMS, go to the Server – Replication, and right-hand click "Local Publications".
o Click "New Publication".
o Specify a Distributor.
Page 147 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
▪ If you don't, the Publisher will act as its own Distributor.
- You will need to specify a default snapshot folder, a directory that agents
can read from and write to this folder.
▪ If you do, you will need to enter the Publisher password.
o Choose a publication database.
o Select a publication type.
▪ Transaction replication – changes occur in near real time, applied to the Subscribe in
the same order as they occurred on the publisher.
- For Azure SQL Database, MI, on-prem VM
▪ Merge replication – Data can be changed on both the Publisher and Subscriber.
- When connected to the network, all rows which have changed between
Publisher and Subscriber are synchronised. For on-prem, VM and MI
▪ Snapshot replication – distributes data at a specific moment of time, and does not
monitor for updates to the data.
- For Azure SQL Database, MI, on-prem VM
▪ Peer-to-peer – allows for changes in near real-time on multiple server instances.
- For on-prem and VM
▪ Bidrectional – allows two servers to exchange changes with each other.
- For on-prem, VM and MI
▪ Updatable subscriptions (deprecated) – when data updated at a Subscriber, it is
propagated to the Publisher and then to the other Subscribers.
o Select data, database objects and filter columns and rows from table articles to publish.
o Set the Snapshot Agent schedule.
o Specify the credentials for:
▪ Snapshot Agent for all publications.
▪ Log Reader Agent for all transactional publications.
▪ Queue Reader Agent for transactional publications that allow updating
subscriptions.
o Optionally, script the publication.
o Specify a publication name.
• How to create a Push subscription:
o In SSMS, go to the Server – Replication, and right-hand click "Local Subscriptions".
o Click "New Subscriptions".
o Select Publisher and publication.
Page 148 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
o Select where replication agents will run.
▪ Run all agents at the Distributor (push subscriptions) or
▪ Merge Agent Location
o Select subscribers and subscription databases.
o Enter the logins and passwords for connections made by replication agents.
o Select a synchronization schedule and when the Subscriber should be initialized.
o Select additional options for merge or transactional publications.
• You can create readable secondary databases in the same or different region.
o If in the same region, not as good for DR.
o This is for Azure SQL Database, not for Azure SQL MI
▪ Azure SQL Database and Azure SQL MI can both use auto-failover groups.
o Up to 4 secondaries are supported in the same or different regions.
▪ They can be part of an elastic pool.
o They can be used for read-only access queries.
o It replicates changes by streaming database transaction logs (unlike transactional replication,
which replicates changes by executing DML commands, such as INSERT/UPDATE/DELETE).
o You can failover to secondary databases.
• You can use it for:
o Database migration from one server to another with minimum downtime, and
o Creating an extra secondary as a fail back copy during application upgrades.
o It uses asynchronous replication, so the transactions are committed on the primary before
being replicated.
o Planned and unplanned failover.
• To configure geo-replication:
o Go to the Azure Portal – the database – Data management – Replicas
94. Configure log shipping
• Log shipping continually backs up your on-prem or SQL Database on an Azure VM onto one (or more)
secondary databases, each on a different SQL Server instance.
• You only have limited read-only access to secondary databases.
• The two instances need to communicate with each other.
• From on premises to SQL Server running in an Azure VM, you need a site-to-Site VPN or
ExpressRoute between on prem and the Azure VM.
• From an Azure VM to another Azure VM, being in the same VNet with a domain controller,
or a VPN.
• The process is:
Page 149 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
Plan and Implement a High Availability and Disaster Recovery Environment
• Primary backs up the transaction log,
• Transaction log is copied to the secondary(s),
• Log backup is restored on secondary(s).
• You cannot read the secondary(s) when restoring occurs.
• You can also have a monitor server (optional). It tracks:
• when the transaction log was backed up (primary database),
• when the secondary servers copied and restored the backup files,
• Any backup failure alerts.
• Failover occurs manually (not automatically).
• To create the log shipping:
• Create a network share for the transaction log backups, and a path for where the transaction
logs backups should be copied (one per secondary).
• On the primary database:
• Right-hand click on the primary database, and go Properties.
• Go to the "Transaction Log Shipping" page.
• Click on "Enable this as a primary database in a log shipping configuration".
• Click on "Backup Settings"
• Enter a "Network path to the [transaction log] backup folder".
• Or you could enter a local path.
• Change, if necessary, the "Delete files older than and "Alert if no backup
occurs within" values.
• Change the Schedule, if necessary, in Backup job – Schedule.
• Choose the backup compression:
• Use the default server setting, Compress backup, or Do not
compress backup
• Click OK.
• To create the log shipping:
• Go to"Secondary server instances and databases" – Add.
• Click Connect, connect to a secondary server.
• Choose a "Secondary Database" or type the name of a new database.
• Select how you want to "Initialize Secondary database".
• Enter the transaction log backup copy path.
• For "Database state when restoring backups", select:
• "No recovery mode" , or
• "Standby mode" – not available if the secondary server major version is
greater than the primary.
• If "standby mode", select whether users should be disconnected
from the secondary while restoring.
• If necessary, choose a "Delay restoring backups at least X minutes".
Page 150 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
No longer tested in DP-300
• Useful if someone deletes a row on the primary, and you want time to
check the version on the secondary.
• Choose "Alert if no restore occurs within X minutes".
• Change the Schedule, if necessary, in Restore job – Schedule.
• Click OK.
• If required, add settings for the "Use a monitor server instance".
• Choose the SQL Instance.
• Choose the "Monitor connections" method.
• Select the "History retention" time.
• In T-SQL, the procedure is:
• On secondary, restore a full backup of the primary.
• On primary:
• run the stored procedures:
• sp_add_log_shipping_primary_database
• sp_add_jobschedule
• sp_add_log_shipping_alert_job
• sp_add_log_shipping_primary_secondary.
• enable the backup job.
• On secondary, run the stored procedures:
• sp_add_log_shipping_secondary_primary
• sp_add_jobschedule and sp_attach_schedule
• sp_add_log_shipping_secondary_database
• Then enable the jobs.
• sp_update_job
No longer tested in DP-300
evaluate requirements for the upgrade
• You can upgrade to SQL Server 2016 or 2017 from SQL Server 2008+.
• You can upgrade to SQL Server 2019 from SQL Server 2012+.
o If using an earlier version, databases can be migrated.
• You can also upgrade to a higher version of the same year.
• You can also upgrade to a higher version except from Enterprise (the highest):
o SQL Server 2012+ Business Intelligence can upgrade to Enterprise.
o Standard (or in older versions, Workgroup or Small Business) can upgrade to Standard or
Enterprise.
o Web can upgrade to Web, Standard or Enterprise.
Page 151 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
No longer tested in DP-300
o Developer can upgrade to Developer, or SQL Server 2019 (only) Web, Standard or Enterprise.
o Express can upgrade to Express, Web, Standard or Enterprise.
o Evaluation can upgrade to Evaluation, Express, Web, Standard or Enterprise.
• If an application requires a previous version, you can use that version’s compatibility level.
o For example, SQL Server 2016 is compatibility level 130.
evaluate offline or online upgrade strategies
• Offline
o You can upgrade SQL Server.
o However, you cannot do an offline upgrade a 32-bit instance to a 64-bit instance.
o You cannot add new features during the upgrade (but you can do it afterwards).
• Online
o You need to do a side-by-side installation, and then decommission the previous SQL Server.
o You can choose what features to use, and you can install a 64-bit instance, even if your
previous version is 32-bit.
implement an online upgrade strategy
• Side-by-side upgrade.
o Verify that the hardware and software you intend to use is supported.
o Backup databases, and check they can be restored.
o Identify reports to be used to check that upgrade has been a success.
o Install new instance of SQL Server.
▪ If using Analysis Services, make sure you install the correct server mode – tabular or
multidimensional.
o Attach/restore each database.
o Run DBCC to check for database integrity.
o Test upgrade has been a success using relevant reports.
o Backup and restore.
• Databases run with the previous compatibility level setting.
implement an offline upgrade strategy
• Upgrade strategy:
o Go to SQL Server Installation Media – Maintenance – Edition Upgrade.
o Enter Product Key (if applicable) and accept license terms.
o Select the SQL Server instance to upgrade.
Page 152 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
No longer tested in DP-300
o On the screen ”Read to Upgrade Edition”, click Upgrade and Close.
o Reboot if necessary.
• There may be additional steps if upgrading from SQL Server Express.
manage storage capacity
• This applies to Azure SQL Database, not Azure SQL Managed Instance.
• You may need:
o Add more space or decrease the maximum capacity to a database elastic pool.
o Change to a different service tier.
• Terminology:
o Data space used
▪ Generally increases with inserts and decreases with deletes, but dependent on
fragmentation.
o Data space allocated
▪ Can grow automatically, but does not automatically decrease after deletes.
o Data space allocated but unused
▪ Can be reclaimed when data files are shrinked.
o Data maximum size.
▪ The maximum that "Data space allocated" can be.
• This applies to Azure SQL Database, not Azure SQL Managed Instance.
o Display the allocated space:
▪ [Single database – In the master database]
- SELECT database_name, allocated_storage_in_megabytes FROM
sys.resource_stats
▪ [Elastic pool – In the master database]
- SELECT elastic_pool_name, elastic_pool_storage_limit_mb,
avg_allocated_storage_percent FROM sys.elastic_pool_resource_stats
o Display maximum size:
▪ [Go to the relevant database. If using Master, results be NULL]
▪ SELECT DATABASEPROPERTYEX('DatabaseName', 'MaxSizeInBytes')
o To shrink a transaction log file
▪ SELECT file_id, size FROM sys.database_files WHERE type = 1 -- "1" = Log file. Size is
in 8 Kb pages.
Page 153 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
No longer tested in DP-300
▪ DBCC SHRINKFILE (2); --Where "2" is the file_id.
▪ Will impact database performance when running; should be done when less used.
▪ DBCC SHRINKDATABASE(MyDatabase) will shrink all the data and log files in the
MyDatabase database. (Note the lack of quote marks.)
assess growth/fragmentation of databases and logs
• You can assess the growth in a database by using the following. It shows the growth over a time
period:
o For a database
▪ SELECT database_name, start_time, storage_in_megabytes
▪ FROM sys.resource_stats
▪ ORDER BY database_name, start_time
▪ Historical data is aggregated every 5 minutes and is retained for approximately 14
days.
o For an elastic pool
▪ SELECT start_time, elastic_pool_name, elastic_pool_storage_limit_mb,
avg_allocated_storage_percent FROM master.sys.elastic_pool_resource_stats
▪ ORDER BY start_time
o To view the current log size:
▪ SELECT file_id, type_desc, size, max_size, growth
▪ FROM sys.database_files
▪ WHERE type = 1
• Fragmented indexes can:
o Degrade query performance, because
o more I/O requests (with smaller data in each) are required.
o Each page can be fragmented between 0% and 100%.
• To assess fragmentation of database indexes:
o SELECT db_name(database_id) as DBName, object_name(object_id) as ObjectName,
avg_fragmentation_in_percent, page_count, *
o FROM sys.dm_db_index_physical_stats(NULL,NULL,NULL,NULL,NULL)
▪ --The arguments are: database_id (use db_id to look it up), object_id (use object_id
to look it up), index_id, partition_number and mode (the scan level).
o order by avg_fragmentation_in_percent * page_count desc
Page 154 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
No longer tested in DP-300
o You can also check it by right-hand clicking on the index in SSMS, going to Properties –
Fragmentation.
• You can also use:
o DBCC SHOWCONTIG
o However, this has been deprecated.
• To assess columnstore indexes, look at
o SELECT deleted_rows, total_rows FROM sys.dm_db_colum_store_row_group_physical_stats
▪ Where more than >20% of rows have been deleted (due to DELETE or UPDATE),
reorganize. This removes rows marked as deleted.
• Indexes can be reorganized or rebuilt
• ALTER INDEX [IndexName or ALL] ON Schema.Table …
o REORGANIZE
▪ REORGANIZE is always ONLINE.
o REBUILD
▪ [WITH ONLINE = ON]
- An offline rebuild is generally quicker, but locks the index during this time.
- Online rebuilds only require a lock right at the end.
▪ [WITH (FILLFACTOR = 70)] -- FILLFACTOR leaves free space for inserted/updated
rows.
▪ [MAX_DURATION = 30 RESUMABLE = ON]-- it pauses after 30 minutes – cannot be
used with "ALL". Used in SQL Server 2017+ or Azure SQL Database.
o PAUSE / ABORT / RESUME – pauses/stops/restarts an ONLINE REORGANIZE or REBUILD.
Used in SQL Server 2017+ or Azure SQL Database.
• Generally, REORGANIZE if >10% and <30%, and REBUILD is >30% - but this is a guide only.
assess performance-related database configuration parameters
• Auto Create Statistics
o The database generates information about the contents of each column. Can be useful for
deciding whether to use a scan or seek.
• Auto Shrink
o ALTER DATABASE CURRENT SET AUTO_SHRINK ON; -- will enable auto-shrink.
o Not recommended, as while it is less impactful to database performance, it is less effective.
o Also, what happens when it needs to grow again?
• See also topic 55.
Page 155 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
No longer tested in DP-300
identify data quality issues with duplication of data
• Minimize duplication of data
• Reduce data modification issues
• Simplify queries
identify normal form of database tables
• 1st Normal Form (1NF)
o Requirements
▪ The Values in each column must be atomic (indivisible),
▪ Each value contains only a single value.
o Actions
▪ Eliminate repeating groups in individual tables
▪ Create a separate table for each set of related data
▪ Identify each set of related data with a primary key
• 2nd Normal Form (2NF)
o Requirements
▪ Is in 1st Normal Form
▪ Reduce repeating information
o Actions
▪ Create separate tables for values that apply to multiple records.
▪ Relate tables with a foreign key
• 3rd Normal Form (3NF)
o Requirements
▪ Is in 2nd Normal Form
▪ Values that are not part of a record’s key are to be removed from the table.
o Action
▪ Remove fields that are not dependent on the key
• 67. identify normal form of database tables
• 4th Normal Form (4NF)
o Requirements
Page 156 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
No longer tested in DP-300
▪ It should be in 3rd Normal Form.
▪ It should have no multi-valued dependency
o Actions
▪ Separate out non-trivial multivalued dependences
• 5th Normal Form (5NF)
o Requirements
▪ It should be in 4th Normal Form.
▪ It should have no join dependency
o Actions
▪ Remove join dependences to remove redundancy
assess index design for performance
• See topic 63.
validate data types defined for columns
• Exact numerics
o bigint, int, smallint, tinyint, bit
o decimal/numeric
o money, smallmoney
• Approximate numerics
o float, real
• Date and time
o date, datetime, datetime2, datetimeoffset, smalldatetime, time
• Character strings
o char, varchar, text, varchar
o nchar, nvarchar, ntext, nvarchar(max)
• Other data types
o binary, varbinary
o cursor, geography, geometry, hierarchyid, rowversion, sql_variant, table, uniqueidentifier,
xml
Page 157 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
No longer tested in DP-300
recommend table and index storage including filegroups
• This is for Azure SQL MI and VM.
o Azure SQL Database supports only one database file (except in Hyperscale).
• There are three type of database files:
o Primary file
▪ Start-up information.
▪ There is only one primary file per database.
▪ Recommended filename extension ".mdf".
o Secondary file
▪ Additional, but optional, user-defined data files (zero to multiple). Cannot be used
in Azure SQL Database.
▪ Can be on separate disks.
▪ Recommended filename extension ".ndf".
o Transaction Log
▪ Information needed to recover database.
▪ One to multiple transaction logs.
▪ Recommended filename extension ".ldf".
o A file can only be used by one database.
o Simple databases can have a single data file and a single transaction log file.
• Have two different file name:
o Logical file name – used in T-SQL statements.
o O/S file name – its location, including directory path (you can set this on VM only).
• Storage size
o Can grow automatically, by a percentage or a fixed file size ("growth increment").
o Has a maximum size.
• Filegroups
o Contains multiple files for admin, data allocation or storage purposes. Not used in Azure SQL
Database.
o By default, the "default" filegroup is the PRIMARY filegroup. However, you can change it.
▪ ALTER DATABASE DatabaseName
MODIFY FILEGROUP FileGroupName DEFAULT;
o The primary filegroup contains the primary file, system tables. The default filegroup (which
may be the same) contains any other objects where you have not specified a filegroup.
o Other filegroups are called "User-defined" filegroups.
Page 158 of 159
DP-300: Administering Microsoft Azure SQL Solutions
From 26 July 2024, with minor revisions from 25 October 2024
No longer tested in DP-300
▪ There are other filegroups, called "Memory Optimized Data" and "Filestream".
o A file can only be contained in one filegroup.
o A filegroup can only be used by one database.
o Transaction logs are not part of a filegroup.
o If you use multiple data files, Microsoft recommends that you create a second file group for
the other files and make that filegroup the default filegroup.
• You can create files and filegroups in T-SQL and in SSMS.
o ALTER DATABASE [MyDatabase]
▪ ADD FILEGROUP [NewFileGroup]
o GO
o ALTER DATABASE [MyDatabase]
▪ ADD FILE (NAME = N'NewData',
▪ FILENAME = N'C:\PathToData\NewData.ndf' ,
▪ SIZE = 8192KB , FILEGROWTH = 65536KB )
◼ or FILEGROWTH = 10%
▪ TO FILEGROUP [NewFileGroup]
• For information on a table, use:
o sp_help 'Schema.TableName'
Page 159 of 159
You might also like
- Oracle Test-King 1z0-908 v2021-02-24 by Jude 45qNo ratings yetOracle Test-King 1z0-908 v2021-02-24 by Jude 45q29 pages
- SQL Server Database Administration in AzureNo ratings yetSQL Server Database Administration in Azure29 pages
- Azure SQL Database & SQL DW: Data Platform AirliftNo ratings yetAzure SQL Database & SQL DW: Data Platform Airlift46 pages
- 604 CA Final ISCA MCQ - by CA Swapnil Patni PDF50% (2)604 CA Final ISCA MCQ - by CA Swapnil Patni PDF73 pages
- Azure SQL Database Administration-OverviewNo ratings yetAzure SQL Database Administration-Overview7 pages
- Who Manages What?: Figure 1.1 Lists The Tasks That You (The Dba) and Microsoft Manage For Azure SQL PaasNo ratings yetWho Manages What?: Figure 1.1 Lists The Tasks That You (The Dba) and Microsoft Manage For Azure SQL Paas24 pages
- SQL Server 2014 Platform For Hybrid Cloud Level 300 DeckNo ratings yetSQL Server 2014 Platform For Hybrid Cloud Level 300 Deck47 pages
- Azure Database Administrator Learning Pathway 1122No ratings yetAzure Database Administrator Learning Pathway 11221 page
- Choosing Technologies For A Big Data Solution in The Cloud: James SerraNo ratings yetChoosing Technologies For A Big Data Solution in The Cloud: James Serra58 pages
- Putting Your Head in The Cloud - A Beginner's Guide To Cloud Computing and SQL Azure - New PortalNo ratings yetPutting Your Head in The Cloud - A Beginner's Guide To Cloud Computing and SQL Azure - New Portal45 pages
- PASS - 19 - Corp - Template - v2 - IaaS - Vs - PaaS FinalNo ratings yetPASS - 19 - Corp - Template - v2 - IaaS - Vs - PaaS Final33 pages
- Plan and Implement Data Platform ResourcesNo ratings yetPlan and Implement Data Platform Resources6 pages
- 8 DBA Tasks For Azure SQL: What's Different From On-Prem100% (1)8 DBA Tasks For Azure SQL: What's Different From On-Prem33 pages
- Describe Azure SQL Services and CapabilitiesNo ratings yetDescribe Azure SQL Services and Capabilities9 pages
- 765 Provisioning SQL Databases: Exam DesignNo ratings yet765 Provisioning SQL Databases: Exam Design3 pages
- AWS Solutions Architect Cheat Sheet Feb 2025No ratings yetAWS Solutions Architect Cheat Sheet Feb 202565 pages
- Planning and Implementing Data ServicesNo ratings yetPlanning and Implementing Data Services33 pages
- Exam DP 300 Administering Relational Databases On Microsoft Azure Skills MeasuredNo ratings yetExam DP 300 Administering Relational Databases On Microsoft Azure Skills Measured6 pages
- Microsoft Certified Azure Database Administrator Associate Skills MeasuredNo ratings yetMicrosoft Certified Azure Database Administrator Associate Skills Measured6 pages
- Technical Brochure 128 - Dell Connectrix MDS 9148T SpecSheetNo ratings yetTechnical Brochure 128 - Dell Connectrix MDS 9148T SpecSheet6 pages
- Sample: Audit Program For As/400 (Iseries, System I) and Os/400 (I5/Os, Ibm I)No ratings yetSample: Audit Program For As/400 (Iseries, System I) and Os/400 (I5/Os, Ibm I)9 pages
- Installing MySQL On Unix Linux Using Generic BinariesNo ratings yetInstalling MySQL On Unix Linux Using Generic Binaries6 pages
- Exam AZ 900: Azure Fundamental Study Guide-1: Explore Azure Fundamental guide and Get certified AZ 900 examFrom EverandExam AZ 900: Azure Fundamental Study Guide-1: Explore Azure Fundamental guide and Get certified AZ 900 examNo ratings yet
- MICROSOFT AZURE ADMINISTRATOR EXAM PREP(AZ-104) Part-3: AZ 104 EXAM STUDY GUIDEFrom EverandMICROSOFT AZURE ADMINISTRATOR EXAM PREP(AZ-104) Part-3: AZ 104 EXAM STUDY GUIDENo ratings yet
