Scribd
Upload
16 views

MYC - Data Engineering Course Details

MYC - Data Engineering Course Details

Uploaded by

editor.kolkataheadlines
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
16 views

MYC - Data Engineering Course Details

MYC - Data Engineering Course Details

Uploaded by

editor.kolkataheadlines
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

MAKE YOUR CRAFT

C R A F T Y O U R F U T U R E
( 2 0 2 4 - 2 0 2 5 )

DATE ENGINEERING COURSE


Roadmap to Data Scientist
AI - Powered
Unleash your potential as a Data Engineer

Session 1-2: Introduction to Data Engineering


Data Warehouse Architecture
Data Driven Platform
OLTP | OLAP
Data Modelling - Star and SnowFlake Schema
ETL flow
DataMart

Session 3-4: Introduction to Microsoft Azure | Big Data | Data Lake


How Azure influenced the Data warehousing
Introduction to Azure Data Engineeirng
Azure Free account creation
DatLake | Blob | DataBase | Structured | Unstructuered data
Big Data to AzureDataLake Stoarge Gen2
DataLake Gen2 storage Architecture - data Storage Explorer
Data storing pattern and Data archival startegy

Session 5-8: Azure Key Vault Engineering


Azure Key Vault Understanding
Security principals
Secrets and Keys
Kay value Policy
Scoped Credential
SAS token
Understanding of Different Azure Data tools
ADLS2 connection approach to different data tools

Page 1
MAKE YOUR CRAFT
C R A F T Y O U R F U T U R E
( 2 0 2 4 - 2 0 2 5 )

DATE ENGINEERING COURSE


Roadmap to Data Scientist
AI - Powered
Unleash your potential as a Data Engineer

Session 9-10: Azure Data Factory - ADF


Understanding of ADF and it's architecture
ADF as ELT mechanism
Understanding Pipeline | Data flow | Control flow
Understanding of Linked Service | authorization mechanism
Build our first basic pipeline with thorough understanding (incl. variable and parameter)
Understanding of varoaus activities of ADF - Copy, Lookup, Store Procedure,set, IF- Else,
Loop, Try-Catch, Web etc
Understanding of Integration RunTime - Auto resolve, Self Hosted (On Premise)

Session 11-12: Hands-On End to End Data Pipeline building in ADF

Build Data pipeline in ADF using important activities


Understanding of Data Flow in detail in pipeline
Triggers in ADF

Session 13-14: Azure Monitoring & Error logging


Monitoring in ADF in detail
Error Logging
ADF Ingestion dynamic Framework in pipeline building

Page 2
MAKE YOUR CRAFT
C R A F T Y O U R F U T U R E
( 2 0 2 4 - 2 0 2 5 )

DATE ENGINEERING COURSE


Roadmap to Data Scientist
AI - Powered
Unleash your potential as a Data Engineer

Session 15-16: Introduction to SPARK & Databricks


Understanding of Spark
Spark in Databricks
Stage | Jobs | Notebook
Understanding PySpark and Databricks
Our first notebook in Databricks.

Session 17-18: Understanding Azure Databricks (ADB) & PySpark


Databricks Architecture (ADB)
Play with Notebooks for data ingestion using PySpark
PySpark - Python + SQL
DataBricks - Data ingestion from external file
Datbricks - connect to ADLS2
Understanding ADB utility

Session 19-20: Introduction to Synapse and its architecture


Why using Synapse?
Synapse vs Databricks
Different Pools in Synapse
Understanding of Serverless pool
Data analysis in serverless pool in detail

Page 3
MAKE YOUR CRAFT
C R A F T Y O U R F U T U R E
( 2 0 2 4 - 2 0 2 5 )

DATE ENGINEERING COURSE


Roadmap to Data Scientist
AI - Powered
Unleash your potential as a Data Engineer

Session 21-22: Synapse and SPARK Pool (Microsoft Spark utility)


Synapse Dedicated pool and pipeline
End to End Data load pipeline to a data warehouse in Fact and Dimension modelling.
Different kind of distribution
Introduction of Spark Pool (understanding Microsoft spark utility)
How spark pool helps in data analysis

Session 23-24: Conclusion


Synapse spark pool conclusion using a notebook
Logic App understanding and alert mechanism
Azure Stream analytics and Event hub high-level understanding for data engineering

NOTE: This course outline provides a structured learning path from foundational skills to more advanced
concepts, including hands-on projects and career development sessions. Each session can be further
broken down into lectures, hands-on exercises, and discussions to enhance learning and engagement.
This is an initiative taken by CLIFAV TECHNOLOGIES PRIVATE LIMITED - We Make IT Happen

Page 4

You might also like