Azure Data Engineering hands-on training

Azure
Data Engineering

Contact-Us

[email protected]

www.raxicube.com

+91-9652299920

www.raxicube.com [email protected] +91-9652299920

 Azure Data Engineering hands-on training

Data Engineering Introduction:

✓ What is data engineering
✓ Introduction to Azure
✓ Comparison of Azure vs AWS Data Engineering services
✓ Bigdata introduction
Azure Basics:
✓ Define Cloud Computing
✓ Services Offered by Microsoft Azure
✓ Benefits of Cloud Computing
✓ Different kinds of cloud deployments
✓ Different kinds of cloud services
✓ Differences between AWS and Microsoft Azure
✓ Create Azure Account
✓ Azure Services & Architecture
✓ Create Resource Groups and Resources in the Microsoft Azure
✓ Create Virtual Machine
✓ Azure Class Deployment vs. Azure Resource Manager
Azure Storage and Data Lake:
✓ Detail knowledge of Blob
✓ Create a container
✓ Upload a blob to Azure Storage
✓ Create a Blob
✓ List of the blobs in a container
✓ Delete a container
✓ Download the blob to your local computer
✓ Detail knowledge of Data Lake
✓ Create Azure Data Lake Gen 2 Account
✓ Create Folders
✓ Upload data
✓ Secure data
✓ Delete Azure Data Lake
✓ Access Azure Storage using Python

www.raxicube.com [email protected] +91-9652299920

 Azure Data Engineering hands-on training

Azure Key Vault:

✓ Introduction to Azure Key Vault
✓ What is Azure Key Vault Used For?
✓ Store Secrets in Azure Key Vault using Azure Portal
✓ Retrieve your secrets from Azure Key Vault using Python

Azure Logic Apps:

✓ Getting Started with the Logic Apps
✓ Creating Workflows through Logic Apps in Azure
✓ Sending Mail Logic App in Azure Data Factory

Azure Data Factory:

✓ Azure Data Factory Architecture
✓ Creating ADF Resource and Use in azure cloud
✓ Pipeline Creation and Usage Options
✓ Copy Data Tool in ADF Portal, Use
✓ Linked Service Creation in ADF
✓ Dataset Creation, Connection Reuse
✓ Staging Dataset with Azure Storage
✓ ADF Pipeline Deployments
✓ Pipeline Orchestration using Triggers
✓ ADF Transformations and other tools integration.
✓ Processing different type’s files using ADF.
✓ Integration Runtime
✓ Monitoring ADF Jobs
✓ Manage IR’s and Linked Services.
Azure Synapse Analytics:
✓ Introduction to Azure Synapse Analytics
✓ Create Azure Synapse Analytics Workspace
✓ Basic Concepts in Azure Synapse Analytics
✓ Analyze data with a server less SQL pool and dedicated SQL Pool
✓ Analyze data with Server less Spark Pool
✓ Analyze data in Storage Account in Azure Synapse Analytics

www.raxicube.com [email protected] +91-9652299920

 Azure Data Engineering hands-on training

✓ Integrate Pipelines in Azure Synapse Analytics

✓ Monitor your Azure Synapse Analytics Workspace
✓ Add an Administrator to your Azure Synapse Workspace
✓ Azure Synapse SQL Architecture
✓ Distributions(Hash, Round Robbin & Replicate)
✓ Server less SQL Pool Overview
✓ Create External Data source
✓ Create External File Format
✓ CETAS with Synapse SQL
✓ CTAS with Synapse SQL
✓ External Tables with Synapse SQL
✓ Create and query external tables from a file in ADLS
✓ Types of External Tables(Hadoop & Native) in Synapse SQL
✓ Administrative accounts in Synapse SQL
✓ Create Login and User for Server less SQL Pool
✓ Create Login and User for Dedicated SQL Pool
✓ Temporary Tables in Synapse SQL
✓ Using IDENTITY to create surrogate keys using dedicated SQL pool
✓ OPENROWSET() function in Synapse SQL
✓ Apache Spark in Azure Synapse Analytics
✓ Create a Spark Pool with Azure Portal
✓ Create a Notebook in Azure Synapse Analytics
✓ Pandas to read/write Azure Data Lake Storage Gen2 data in Apache
Spark pool in Synapse Analytics
✓ Use FSSPEC to read/write ADLS Gen2 data in Apache Spark pool in
Synapse Analytics
✓ Different ways to Create Notebooks in Azure Synapse Analytics
✓ Use multiple languages in Synapse notebook using magic commands
✓ Use temp tables to reference data across languages in Synapse
notebooks in Azure Synapse
✓ %run command to reference another notebook with in current
notebook
✓ Parameterize Synapse notebook in Azure Synapse Analytics
✓ Run Synapse notebook from pipeline | Pass values to Notebook
parameters from pipeline in Synapse
✓ IPython Widgets in Synapse Notebook in Azure Synapse Analytics |

www.raxicube.com [email protected] +91-9652299920

 Azure Data Engineering hands-on training

ipywidgets in Synapse notebook

✓ Introduction to Microsoft Spark Utilities(MSSparkutils)
✓ Microsoft Spark File System(mssparkutils.fs) Utilities in Azure Synapse
Analytics
✓ Microsoft Spark Utilities Notebook Utilities(mssparkutils.notebook)
✓ exit() function of notebook module in MSSparkUtils package
✓ run() function of notebook module in MSSparkUtils package
✓ Environment utilities (mssparkutils.env) in MSSparkUtils
✓ Configure access to Azure Data Lake Gen2(ADLS Gen2) for Synapse
Notebook
✓ MSSparkUtils Runtime Utils in Synapse Notebook
✓ How to Mount ADLS Gen2 Storage using Linked Service
✓ How to Mount ADLS Gen2 Storage using Account Key or SAS Token in
Synapse Notebook
✓ How to Unmount the Mount Point in Synapse Spark
✓ Creating a Spark Job Definition and Submitting it in Azure Synapse

Azure Data Bricks:

✓ Introduction to Azure Databricks
✓ Databricks Architecture
✓ Databricks Components overview
✓ Benefits for data engineers and data scientists
✓ Workspace – Creation and managing workspace.
✓ Notebook – creating notebooks, calling and managing different
notebooks.
✓ Library - installing libraries, managing libraries
✓ Databricks File System. - DBFS commands copy and manage files using
DBFS.
✓ Database - Creating database, tables and managing databases and
tables.
✓ Table - Creating Tables, dropping tables, loading data ..
✓ Metastore - managing metadata and delta tables creation, managing
delta tables.
✓ Cluster -- Creating Clusters , managing clusters
✓ Pool - creating pools and using pools for Auto scaling.
✓ Databricks Runtime - understanding and using Databricks runtimes

www.raxicube.com [email protected] +91-9652299920

 Azure Data Engineering hands-on training

based on requirement.
✓ Jobs - creating jobs from notebooks and assigning types of clusters for
jobs.
✓ Workload - monitoring jobs and managing loads.
✓ Execution Context – understanding context.
✓ Databricks Workflows
✓ Calling one notebook into another notebook.
✓ Creating global variables (widgets) and using into Azure ADF pipeline.
✓ How to implement parallelism in notebooks execution.
✓ Mounting azure blob storage and data lake storage accounts.
✓ Integrating source code (notebooks) with GitHub
✓ Calling DataBricks notebooks into Azure Data factory.
✓ Databricks Clusters logs monitoring flow.

Databricks Delta Lake:

✓ Delta Lake Architecture
✓ Delta Lake Storage Understanding
✓ Delta lake table creation and API options
✓ Delta Lake DML Operations usage.
✓ Delta Lake partitions
✓ Delta Lake Schema Enforcement
✓ Delta Lake Schema Evolution
✓ Delta Lake Versions
✓ Delta Lake Time Travel
✓ Delta Lake Vaccum
✓ Delta Lake Merge (SCD Type 1 and SCD Type2)

Azure Application Insights:

✓ Azure Application Insights Tutorial
✓ What is Azure Application Insights?
✓ Monitoring without code or Codeless Monitoring
✓ Monitoring with Code or Code based Monitoring
✓ Features of Azure Application Insights
✓ What exactly does Azure Application Insights monitor?

www.raxicube.com [email protected] +91-9652299920

 Azure Data Engineering hands-on training

✓ How to create Azure Application Insights?

✓ Prerequisites
✓ How to enable Application Insights for Application?

Python:
✓ Python Overview
✓ Python Environment Setup
✓ Data Types
✓ Operators
✓ Flow Control & Loops
✓ Functional Programming
✓ Collections
✓ Files I/O
✓ Object Oriented Programming
✓ Exception Handling
✓ modules
✓ Connecting to Databases
✓ Calling APIs

Spark & PySpark:

✓ Spark Introduction
✓ Spark Components or modules
✓ Spark Architecture
✓ Spark Core
✓ RDD Transformations and Actions
✓ Data Frames Transformations and Actions
✓ Spark SQL

Projects
Interview Questions
Recordings
Materials

www.raxicube.com [email protected] +91-9652299920

 Azure Data Engineering hands-on training

For Enrolment

www.raxicube.com [email protected] +91-9652299920