Azure Data Engineering hands-on training

Azure
Data Engineering

Contact-Us

[email protected]

www.raxicube.com

+91-6376969731

www.raxicube.com [email protected] +91-6376969731

 Azure Data Engineering hands-on training

Data Engineering Introduction:

 What is data engineering
 Introduction to Azure
 Comparison of Azure vs AWS Data Engineering services
 Bigdata introduction
Azure Basics:
 What is Azure and Cloud
 How to Create Resource Group in Azure
 Services Offered in Azure
 Azure Portal Walk Through
 SDKs or Tools for Azure Resources
 Create Free Azure Subscription
Azure Storage and Data Lake:
 Detail knowledge of Blob
 Create a container
 Upload a blob to Azure Storage
 Create a Blob
 List of the blobs in a container
 Delete a container
 Download the blob to your local computer
 Detail knowledge of Data Lake
 Create Azure Data Lake Gen 2 Account
 Create Folders
 Upload data
 Secure data
 Delete Azure Data Lake
Azure Key Vault:
 Introduction to Azure Key Vault
 Store Secrets in Azure Key Vault using Azure Portal

www.raxicube.com [email protected] +91-6376969731

 Azure Data Engineering hands-on training

Azure Data Factory:

 Introduction to Azure Data Factory
 Top level Concepts in Azure Data Factory
 Create your First Azure Data Factory
 Different ways to work with Azure Data Factory
 Pipelines and Activities in Azure Data Factory
 Linked Services and Datasets in Azure Data Factory
 Triggers in Azure Data Factory
 Schedule Trigger in Azure Data Factory
 Tumbling Window Trigger in Azure Data Factory
 Tumbling Window Trigger Dependency in Azure Data Factory
 Event based Triggers in Azure Data Factory
 Integration runtime in Azure Data Factory
 Azure Integration runtime in Azure Data Factory
 Self Hosted Integration runtime in Azure Data Factory
 Setting up Self Hosted Integration runtime in Azure Data Factory
 Shared Self Hosted Integration runtime in Azure Data Factory
 Parameterize Linked Services in Azure Data Factory
 Parameterize Datasets in Azure Data Factory
 Parameterize Pipelines in Azure Data Factory
 System Variables in Azure Data Factory
 Connectors Overview in Azure Data Factory
 Supported File Formats in Azure Data Factory
 Copy Data Activity in Azure Data Factory
 Copy Data Activity in Azure data factory Continuation
 Monitor Copy Data Activity in Azure Data Factory
 Delete Activity in Azure Data Factory
 Variables in Azure Data Factory
 Set Variable Activity in Azure Data Factory
 Append Variable Activity in Azure Data Factory
 User Properties in Azure Data Factory
 Execute Pipeline Activity in Azure Data Factory
 Filter Activity in Azure Data Factory
 ForEach Activity in Azure Data Factory
 Get Metadata Activity in Azure Data Factory

www.raxicube.com [email protected] +91-6376969731

 Azure Data Engineering hands-on training

 If Condition Activity in Azure Data Factory

 Wait Activity in Azure Data Factory
 Until Activity in Azure Data Factory
 Web Activity in Azure Data Factory
 WebHook Activity in Azure Data Factory
 Switch Activity in Azure Data Factory
 Validation Activity in Azure Data Factory
 Lookup Activity in Azure Data Factory
 Transform Data Activities Overview in Azure Data Factory
 Stored Procedure Activity in Azure Data Factory
 Data flow in Azure data factory
 Mapping Data Flow in Azure Data Factory
 Data Flow Activity in Azure Data Factory
 Mapping Data Flow Debug Mode in Azure Data Factory
 Filter Transformation in Mapping Data Flow in Azure Data Factory
 Aggregate Transformation in Mapping Data Flow in Azure Data Factory
 JOIN Transformation in Mapping Data Flow in Azure Data Factory
 Conditional Split Transformation in Mapping Data Flow in Azure data factory
 Derived Column Transformation in Mapping Data Flow in Azure Data Factory
 Exists Transformation in Mapping Data Flow in Azure Data Factory
 Union Transformation in Mapping Data Flow in Azure Data Factory
 Lookup Transformation in Mapping Data Flow in Azure Data Factory
 Sort Transformation in Mapping Data Flow in Azure Data Factory
 New Branch in Mapping Data Flow in Azure Data Factory
 Select Transformation in Mapping Data Flow in Azure Data Factory
 Pivot Transformation in Mapping Data Flow in Azure Data Factory
 Unpivot Transformation in Mapping Data Flow in Azure Data Factory
 Surrogate Key Transformation in Mapping Data Flow in Azure Data Factory
 Window Transformation in Mapping Data Flow in Azure Data Factory
 Alter Row Transformation in Mapping Data Flow in Azure Data Factory
 Flatten Transformation in Mapping Data Flow in Azure Data Factory
 Parameterize Mapping Data Flow in Azure Data Factory
 Validate Schema in Mapping Data Flow in Azure Data Factory
 Schema Drift in Mapping Data Flow in Azure Data Factory
 Wrangling Data Flow Overview in Azure Data Factory

www.raxicube.com [email protected] +91-6376969731

 Azure Data Engineering hands-on training

 Merge Queries in Wrangling Data Flow in Azure Data Factory

 Group By in Wrangling Data Flow in Azure Data Factory
 Different Author Modes in Azure Data Factory
 Setup GitHub Code Repository for Azure Data Factory
 Setup Azure DevOps Git Code Repository for Azure Data Factory
 Use Azure Key Vault Secrets in Azure Data Factory
 Continuous integration and deployment in Azure Data Factory
 How to read JSON output of one Activity in to another Activity in Azure Data
Factory
 Annotations in Azure Data Factory
 Templates Overview in Azure Data Factory
 Global Parameters in Azure Data Factory
 Rank Transformation in Mapping Data Flow in Azure Data Factory
 Cache Sink and Cached lookup in Mapping Data Flow in Azure Data Factory
 Session log in Copy Activity | Log Copied File names in Copy Activity in Azure
Data Factory
 Write Cache Sink to Activity Output in Azure Data factory
 Parse Transformation in Mapping Data Flow in Azure Data Factory
 Fail Activity in Azure Data Factory
 Inline Dataset in Azure Data Factory
 Stringify transformation in Mapping Data Flow in Azure Data Factory
 Assert Transformation in Mapping Data Flows in Azure Data Factory
 Flowlets in Mapping data flow in Azure Data Factory
 Script Activity in Azure Data Factory or Azure Synapse
 User defined Functions in Mapping data flows in Azure Data Factory

www.raxicube.com [email protected] +91-6376969731

 Azure Data Engineering hands-on training

Azure Synapse Analytics:

 Introduction to Azure Synapse Analytics
 Create Azure Synapse Analytics Workspace
 Basic Concepts in Azure Synapse Analytics
 Analyze data with a server less SQL pool and dedicated SQL Pool
 Analyze data with Server less Spark Pool
 Analyze data in Storage Account in Azure Synapse Analytics
 Integrate Pipelines in Azure Synapse Analytics
 Monitor your Azure Synapse Analytics Workspace
 Add an Administrator to your Azure Synapse Workspace
 Azure Synapse SQL Architecture
 Distributions(Hash, Round Robbin & Replicate)
 Server less SQL Pool Overview
 Create External Data source
 Create External File Format
 CETAS with Synapse SQL
 CTAS with Synapse SQL
 External Tables with Synapse SQL
 Create and query external tables from a file in ADLS
 Types of External Tables(Hadoop & Native) in Synapse SQL
 Administrative accounts in Synapse SQL
 Create Login and User for Server less SQL Pool
 Create Login and User for Dedicated SQL Pool
 Temporary Tables in Synapse SQL
 Using IDENTITY to create surrogate keys using dedicated SQL pool
 OPENROWSET() function in Synapse SQL
 Apache Spark in Azure Synapse Analytics
 Create a Spark Pool with Azure Portal
 Create a Notebook in Azure Synapse Analytics
 Pandas to read/write Azure Data Lake Storage Gen2 data in Apache
Spark pool in Synapse Analytics
 Use FSSPEC to read/write ADLS Gen2 data in Apache Spark pool in
Synapse Analytics
 Different ways to Create Notebooks in Azure Synapse Analytics
 Use multiple languages in Synapse notebook using magic commands

www.raxicube.com [email protected] +91-6376969731

 Azure Data Engineering hands-on training

 Use temp tables to reference data across languages in Synapse

notebooks in Azure Synapse
 %run command to reference another notebook with in current
notebook
 Parameterize Synapse notebook in Azure Synapse Analytics
 Run Synapse notebook from pipeline | Pass values to Notebook
parameters from pipeline in Synapse
 IPython Widgets in Synapse Notebook in Azure Synapse Analytics |
ipywidgets in Synapse notebook
 Introduction to Microsoft Spark Utilities(MSSparkutils)
 Microsoft Spark File System(mssparkutils.fs) Utilities in Azure Synapse
Analytics
 Microsoft Spark Utilities Notebook Utilities(mssparkutils.notebook)
 exit() function of notebook module in MSSparkUtils package
 run() function of notebook module in MSSparkUtils package
 Environment utilities (mssparkutils.env) in MSSparkUtils
 Configure access to Azure Data Lake Gen2(ADLS Gen2) for Synapse
Notebook
 MSSparkUtils Runtime Utils in Synapse Notebook
 How to Mount ADLS Gen2 Storage using Linked Service
 How to Mount ADLS Gen2 Storage using Account Key or SAS Token in
Synapse Notebook
 How to Unmount the Mount Point in Synapse Spark
 Creating a Spark Job Definition and Submitting it in Azure Synapse

Azure Data Bricks:

 Introduction to Azure Databricks
 Create an Azure Databricks Workspace using Azure Portal
 Create Databricks Community Edition Account
 Workspace in Azure Databricks
 Workspace assets in Azure Databricks
 Working with Workspace Objects in Azure Databricks
 Create and Run Spark Job in Databricks
 Azure Databricks architecture overview
 Databricks File System(DBFS) overview in Azure Databricks
 Databricks Utilities(dbutils) in Azure Databricks

www.raxicube.com [email protected] +91-6376969731

 Azure Data Engineering hands-on training

 Data Utility(dbutils.data) in Azure Databricks in Databricks utilities

 File System utility(dbutils.fs) of Databricks Utilities in Azure Databricks
 exit() command of notebook utility(dbutils.notebook) in Azure
Databricks
 run() command of notebook utility(dbutils.notebook) in Databricks
Utilities in Azure Databricks
 Widgets utility(dbutils.widgets) of Databricks Utilities in Azure
Databricks
 Pass values to notebook parameters from another notebook using run
command in Azure Databricks
 Parameterize SQL notebook using widgets in Azure Databricks |
Widgets in SQL in Azure Databricks
 Create Mount point using dbutils.fs.mount() in Azure Databricks
 Mount Azure Blob Storage to DBFS in Azure Databricks
 Delete or Unmount Mount Points in Azure Databricks
 mounts() & refreshMounts() commands of File system Utilities in
Azure Databricks
 Update Mount Point(dbutils.fs.updateMount()) in Azure Databricks
 Secret Scopes Overview in Azure Databricks
 Install Databricks CLI and configure with your workspace | Azure
Databricks
 Create an Azure Key Vault backed secret scope using the UI in Azure
Databricks
 Create a Databricks backed secret scope in Azure Databricks
 Secrets Utility(dbutils.secrets) of Databricks Utilities in Azure
Databricks
 Access ADLS Gen2 storage using Account Key in Azure Databricks
 Configure access to Azure storage with an Azure Active Directory
service principal
 Access Data Lake Storage Gen2 or Blob Storage with an Azure service
principal in Azure Databricks
 Access ADLS Gen2 or Blob Storage using a SAS token in Azure
Databricks

www.raxicube.com [email protected] +91-6376969731

 Azure Data Engineering hands-on training

Azure Application Insights:

 Azure Application Insights Tutorial
 What is Azure Application Insights?
 Monitoring without code or Codeless Monitoring
 Monitoring with Code or Code based Monitoring
 Features of Azure Application Insights
 What exactly does Azure Application Insights monitor?
 How to create Azure Application Insights?
 Prerequisites
 How to enable Application Insights for Application?

Projects
Interview Questions
Recordings
Materials

www.raxicube.com [email protected] +91-6376969731

 Azure Data Engineering hands-on training

For Enrolment

www.raxicube.com [email protected] +91-6376969731