Scribd
Upload
77 views41 pages

Implementing An Azure SQL Data Warehouse

This document provides an overview of implementing an Azure SQL Data Warehouse. It covers advantages of Azure SQL Data Warehouse including scalability, cost, security and availability. It also discusses migrating data to an Azure SQL Data Warehouse using tools like the Data Warehouse Migration Utility. The document includes lessons on creating and configuring an Azure SQL Data Warehouse database, developing databases in Azure SQL Data Warehouse including handling concurrency and distributions, and copying data between on-premises and Azure SQL Data Warehouse using Azure Data Factory. It concludes with a lab scenario to implement an Azure SQL Data Warehouse database and migrate and copy data using the tools covered.

Uploaded by

Richie Poo
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
77 views41 pages

Implementing An Azure SQL Data Warehouse

This document provides an overview of implementing an Azure SQL Data Warehouse. It covers advantages of Azure SQL Data Warehouse including scalability, cost, security and availability. It also discusses migrating data to an Azure SQL Data Warehouse using tools like the Data Warehouse Migration Utility. The document includes lessons on creating and configuring an Azure SQL Data Warehouse database, developing databases in Azure SQL Data Warehouse including handling concurrency and distributions, and copying data between on-premises and Azure SQL Data Warehouse using Azure Data Factory. It concludes with a lab scenario to implement an Azure SQL Data Warehouse database and migrate and copy data using the tools covered.

Uploaded by

Richie Poo
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 41

Module 5

Implementing an Azure SQL Data


Warehouse
Module Overview

Advantages of Azure SQL Data Warehouse


Implementing an Azure SQL Data Warehouse
Database
Developing an Azure SQL Data Warehouse
Migrating to an Azure SQL Data Warehouse
• Copying Data with the Azure Data Factory
Lesson 1: Advantages of Azure SQL Data
Warehouse

What is Azure SQL Data Warehouse?


Scalability and Cost
Security and Availability
PolyBase
• Hybrid Cloud
What is Azure SQL Data Warehouse?

• Cloud-based database
• Relational and nonrelational
• Enterprise workloads
• Integrated with Azure
• Fully managed service

• Benefits include:
• Massive parallel processing
• Advanced query optimization
• Columnstore indexes
• PolyBase integration
• Auditing
• Scalability
Scalability and Cost

• No upfront cost
• Storage
• Adjusts automatically
• Cost based on storage used

• Compute
• Determines execution performance
• Data Warehouse Unit (DWU)
• Increase or decrease DWU
• Cost based on DWU used
• Pause and start
Security and Availability

• Security
• Firewall
• Add logins
• Set authorisation
• Auditing

• Availability
• Can restore in different region
• Can choose restore point in last seven days
PolyBase

• Can access unstructured data in other systems


• Set up external table to link to data source
• Query external table as normal table
Hybrid Cloud

• Can integrate between on-premises, cloud, and


unstructured data sources
• Use PolyBase to query and copy data with
Transact-SQL
• Schedule data copy using Azure Data Factory
Lesson 2: Implementing an Azure SQL Data
Warehouse Database

Creating a Server
Creating a Database
Configuring the Server Firewall
Connecting to Azure Database Using SQL Server
Management Studio
• Demonstration: Creating and Configuring an
Azure SQL Data Warehouse Database
Creating a Server

• Logical server
• Specify:
• Server name that has not been used
• Server admin logon
• Password
• Location nearest to you

• Create database in same process


Creating a Database

• Create database
• Name of database
• Drag slider to change DWU performance
• Create a new server or use existing server
• Source
• Create a new resource group or use existing resource
group
• DWU settings
• Scale
• Pause/start
Configuring the Server Firewall

• Add client IP address before connecting


• Client IP address may change
• Specify rule:
• Range of IP addresses to allow for change
• IP addresses for other client computers
Connecting to Azure Database Using SQL Server
Management Studio

• Fully qualified server name


• Connect to server using SSMS
• USE statement not supported
• Right-click database, New Query
• Most Transact-SQL supported in Azure SQL Data
Warehouse databases
Demonstration: Creating and Configuring an
Azure SQL Data Warehouse Database

In this demonstration, you will see how to:


• Create an Azure SQL Data Warehouse Database
and server
• Change the performance settings
• Configure the Azure firewall
• Connect to the Azure server with SQL Server
Management Studio
Lesson 3: Developing an Azure SQL Data
Warehouse

Concurrency and Memory Allocation


Data Distribution
CREATE TABLE AS SELECT
GROUP BY Limitations
Temporary Tables
• User Defined Schemas
Concurrency and Memory Allocation

• Resource class
• Concurrency slots
• Query may use more than one concurrency slot
• Dependent on resource class and DWU service level

• Concurrent queries
• Maximum of 32 queries
• Maximum slots dependent on DWU service level

• Memory allocation
• Dependent on resource class and DWU service level
Data Distribution

• Data in tables allocated to distributions


• Round-robin distribution
• Random distribution allocation
• Hash distribution
• Choose hashed column
• Distribution determined by function of column value
• Ensure hashed column has even spread of data
CREATE TABLE AS SELECT

• Makes copy of a table


• Can set index properties and distribution type

CREATE TABLE Countries_New


WITH
(
CLUSTERED COLUMNSTORE INDEX,
DISTRIBUTION = HASH(CountryKey)
)
AS SELECT * FROM Countries
;

• Use to work around unsupported features


GROUP BY Limitations

• GROUP BY clause is supported


• GROUPING SETS, CUBE and ROLLUP subclauses
are not supported
• UNION ALL operator is supported
• When migrating to Azure SQL Data Warehouse,
ensure queries containing unsupported clauses
are amended
Temporary Tables

• Local temporary tables can be accessed


anywhere within session
• Global temporary tables are not supported
User Defined Schemas

• All data in one database


• Use schemas to identify legacy databases
Lesson 4: Migrating to an Azure SQL Data
Warehouse

The Data Warehouse Migration Utility


Migrating Data with the Data Warehouse Migration
Utility
Other Migration Tools
Differences Between SQL Server and Azure SQL
Data Warehouse Schemas
Updating Transact-SQL
• Demonstration: Migrating a Database to Azure
SQL Data Warehouse
The Data Warehouse Migration Utility

• Advantages
• Straightforward
• Multiple tables
• Specify distribution type
• Notification of incompatibility

• Download from Internet


• Must have BCP and Excel installed
Migrating Data with the Data Warehouse
Migration Utility

• Check compatibility
• Migrate schema
• Migrate data
• bcp commands to export and import
Other Migration Tools

• Options for loading data into an Azure SQL Data Warehouse include:
• Azure Feature Pack for Integration Services (SSIS)
• Downloadable extension for SSIS that facilitates the movement of data between on-
premises and cloud
• SSIS
• Add Azure SQL Data Warehouse connection in data flows
• Use SQL Agent to schedule regular transfer of data
• Bulk Copy Program (bcp)
• Useful for small data, use bcp to copy data to flat files and load into the data warehouse
destination
• AZCopy
• Copy data from flat files into Blob storage, and use PolyBase to load into data warehouse
• Import/Export
• For data larger than 10TB, bcp data to files, copy to disks and ship to Microsoft
• PolyBase and T-SQL
• Move UTF-8 formatted data in text files to Azure Blob storage or HDInsight, then use T-
SQL command to load into the data warehouse
• PolyBase uses the massively parallel processing (MPP) architecture for fast loading
Differences Between SQL Server and Azure SQL
Data Warehouse Schemas

• Some table features not supported


• Primary Keys
• Foreign Keys
• Unique Indexes
• Constraints

• Some data types not supported


• numeric
• nvarchar(max)
• varchar(max)
Updating Transact-SQL

• Some Transact-SQL not supported


• Rewrite to achieve same result
Demonstration: Migrating a Database to Azure
SQL Data Warehouse

In this demonstration, you will see how to:


• Install the Data Warehouse Migration Utility
• Check compatibility of the legacy database
• Migrate the schema
• Migrate the data
Lesson 5: Copying Data with the Azure Data
Factory

The Azure Data Factory


Creating a Data Factory
Setting Up a Data Gateway for the On-Premises
Server
Setting up a Linked Service
Setting Up a Dataset
Setting Up a Pipeline Activity to Copy Data
• Data Factory Diagram
The Azure Data Factory

• Capabilities and application to Azure SQL Data


Warehouse
• Entities
• Activity
• Pipeline
• Dataset
• Linked service

• Scheduling
• JSON templates
• Edit parameters in script
• Replace \ with \\
Creating a Data Factory

• Factory contains entities for activities


• Specify
• Name
• Resource group name
• Region
Setting Up a Data Gateway for the On-Premises
Server

• Access data factory from on-premises server


• Create new gateway
• Install on computer
Setting up a Linked Service

• New data store


• Edit parameters in JSON script
• name
• connectionString
• Integrated Security
• User ID
• Password
• gatewayName
• userName: Use for Windows authentication
• password: Use for Windows authentication
Setting Up a Dataset

• New dataset
• Edit parameters in JSON script
• name
• linkedServiceName
• tableName
• frequency
• interval
Setting Up a Pipeline Activity to Copy Data

• New pipeline
• Edit parameters in JSON script
• name
• start
• end

• Add activity script


Data Factory Diagram

• Shows data flow


• Pipeline properties
• Activities
• Datasets

• Dataset properties
• Data slices
Lab: Implement an Azure SQL Data Warehouse

Exercise 1: Create an Azure SQL Data Warehouse


Database
Exercise 2: Migrate to an Azure SQL Data
Warehouse Database
• Exercise 3: Copy Data with the Azure Data Factory

Logon Information
Virtual machine: 20767C-MIA-SQL
User name: ADVENTUREWORKS\Student
Password: Pa55w.rd

Estimated Time: 60 minutes


Lab Scenario

A data warehouse containing food orders might


need to rapidly expand; however, this is not
definite, so the executive board have decided not
to purchase the hardware to support the
expanded database and instead wish to
implement an Azure SQL Data Warehouse. You
have been asked to implement a preliminary test
system that uses a cutdown version of the data
warehouse.
Lab Scenario (Continued)

In this lab, you will create an Azure SQL Data


Warehouse database on a new Azure logical server.
You will then use the Data Warehouse Migration
Utility to migrate the data in the FoodOrdersDW
database on the MIA-SQL server to the new Azure
SQL Data Warehouse database. Finally, you will test
the capabilities of the Azure Data Factory by setting
up a scheduled pipeline activity that copies data
from another database on the MIA-SQL server to a
table in the new Azure SQL Data Warehouse
database.
Lab Review

• In this lab, you have used the tools provided by


Microsoft to manage data between on-premises
and Azure Data Warehouses. You will be able to
make better decisions around where, and how, to
store your organizations data.
Module Review and Takeaways

• Review Question(s)

You might also like