Data Engineering Pune
Data Engineering Pune
Module1- Core Cloud • Overview of roles and • Introduction to roles and responsibilities of Lab 1: Practice
Concepts and Azure responsibilities of Data various roles in the market related to data navigating the Azure
engineers engineering. portal and creating a
Fundamentals
• Introduction to Cloud • Define cloud computing and its key web application.
Computing characteristics, including scalability,
• Azure Services elasticity. Lab 2: creating different
Overview • Discuss the differences between public, resources using azure,
• Azure Regions and private, and hybrid clouds. for example- ADF
Availability Zones • Explore core Azure services, including Databricks, SQL 15
• Cloud Deployment compute networking and storage databases.
Models • Understand the significance of Azure regions
and availability zones in ensuring redundancy
and availability for applications.
• Discuss the various cloud deployment models
(IaaS, PaaS, SaaS) and their appropriate use
cases.
You will learn the basics of cloud computing, Azure services, and deployment models, laying a solid foundation for using Azur e.
Module 2: Core Data Concepts and Relational Data in Azure
Module 2: Core Data • Introduction to Data • Learn why data is important for Lab 1: Create an Azure SQL
Concepts and • Types of Data (Structured, businesses and how it supports Database and run basic
Relational Data in Unstructured) decision-making. queries.
Azure
• Overview Azure • Understand different types of
• Azure SQL Database data, including structured (like Lab 2: Design a simple
• Basic SQL Queries (T-SQL) tables) and unstructured (like data model and
• Difference between Azure videos). implement it in Azure SQL.
SQL sever and azure • Get to know Azure SQL Database 15
Database. and its features for storing and
managing data.
• Learn the basics of T-SQL, the
language used to query and
manipulate data in SQL
databases.
You will explore the significance of data, different data types, and how to manage relational data using Azure SQL Database a nd T-SQL.
Module 3: Non-relational Data in Azure and Analytics
Module 3: Non- • Introduction to NoSQL • Discover what NoSQL databases are and their Lab 1: Set up Azure
relational Data in Databases advantages for handling large amounts of Cosmos DB and perform
Azure and Analytics • Azure Cosmos DB diverse data. basic data operations.
• Basics of Data • Explore Azure Cosmos DB, which supports
Analytics various data models and is designed for high Lab 2: Create an Azure
• Introduction to Azure performance. Synapse workspace and
Synapse Analytics. • Learn why data analytics is important for analyze sample data.
• Overview of Azure gaining insights from data.
storage account. • Get an overview of Azure Synapse Analytics, Lab 3: end to end 15
which combines big data and data Synapse pipeline.
warehousing..
You will discover NoSQL principles and Azure Cosmos DB, along with the importance of data analytics and Azure Synapse Analytics.
Module 4: Data Visualization, Governance, and Compliance
Module 4: • Data Visualization with • Learn how to create reports and Lab 1: Build and publish
Data Visualization, Power BI dashboards using Power BI to visualize a Power BI report with
• Azure Governance data effectively. visual data insights.
Governance, and
Features • Understand governance tools in Azure,
Compliance
• Security and Compliance like Azure Policy and Role-Based Access Lab 2: Configure
in Azure Control, to manage resources. governance policies in
• Monitoring and • Explore security features in Azure to Azure and monitor
Management Tools protect data and comply with regulations. resource performance.
• Discover monitoring tools like Azure 15
Monitor that help keep track of resource
performance.
You will learn to visualize data with Power BI, manage resources using governance tools, and ensure security and compliance in Azure.
Semester 2
(Implementing Analytics Solutions Using Microsoft Fabric)
Introduction to Data Data Integration and Data Analysis and Security, Governance, and
Architecture and Microsoft Transformation Visualization Performance
Fabric Optimization
Bonus Module 5
Introduction to Databricks
advance Course
Module 1: Introduction to Data Architecture and Microsoft Fabric
Module1- • Overview of Data • Understand the importance of data Lab 1: Set up a Microsoft
Introduction to Data Architecture architecture in modern data solutions and Fabric environment and
Architecture and • Microsoft Fabric analytics. navigate the interface.
Microsoft Fabric
• Overview Key • Explore Microsoft Fabric as an integrated
Components of analytics platform for data engineering, data Lab 2: Create a simple
Microsoft Fabric science, and business intelligence. data model using
• Data Modeling • Learn about the key components of Microsoft Microsoft Fabric tools.
Concepts Fabric, including Data Factory, Dataflows, and
Power BI. 15
• Discuss essential data modeling concepts,
including star schema, snowflake schema,
and normalization.
Understanding the architecture and capabilities of Microsoft Fabric sets a solid foundation for effectively leveraging its an alytics tools.
Module 2: Data Integration and Transformation
Module 2: Data • Data Integration • Learn about data integration techniques and Lab 1: Create and
Integration and Techniques their significance in building cohesive configure data
Transformation • Azure Data Factory in analytics solutions. integration pipelines
Microsoft Fabric • Explore Azure Data Factory within Microsoft using Azure Data
• Dataflows and ETL Fabric for orchestrating data workflows. Factory.
Processes Data • Understand dataflows and the difference
• Transformation Best between ETL (Extract, Transform, Load) and Lab 2: Implement a
Practices ELT (Extract, Load, Transform) processes. dataflow that performs
• Discuss best practices for data ETL operations on 15
transformation and handling different data sample datasets.
formats.
Mastering data ingestion and transformation techniques is crucial for ensuring high-quality data that drives accurate analytics.
Module 3: Data Analysis and Visualization
Module 3: Data • Data Analysis • Explore various data analysis techniques and Lab 1: Build a Power BI
Analysis and Techniques their application in decision-making. report using data from
Visualization • Power BI Integration • Learn how Power BI integrates with Microsoft Microsoft Fabric and
with Microsoft Fabric Fabric to enable rich visualizations and publish it.
• Creating analytics.
Visualizations • Understand how to create interactive Lab 2: Create a
• Dashboards and visualizations and utilize DAX (Data Analysis dashboard with multiple
Reports Expressions) for advanced calculations. visualizations to
• Discuss best practices for designing represent key metrics. 15
dashboards and reports that convey insights
effectively.
Developing efficient data models and implementing analytics solutions enables effective decision-making based on actionable insights.
Module 4: Security, Governance, and Performance Optimization
Module 4: Security, • Data Security • Learn about essential data security principles, Lab 1: Configure security
Governance, and Principles in Microsoft including authentication, authorization, and settings and access
Performance Fabric data encryption. controls within Microsoft
Optimization
• Governance and • Understand governance practices in Fabric.
Compliance Microsoft Fabric to ensure compliance with
• Performance Tuning data policies and regulations. Lab 2: Set up monitoring
Techniques • Explore performance tuning techniques for alerts and analyze
• Monitoring and optimizing data workflows and queries. performance metrics for
Maintenance • Discuss monitoring tools and maintenance data pipelines. 15
strategies to ensure the reliability and
efficiency of data solutions. Lab 3: Implement
governance policies to
manage data resources
effectively.
Successful deployment and ongoing monitoring of analytics solutions are essential for maintaining performance and delivering value to users.
Bonus Module 5: Introduction to Databricks advance Course
4 Hours.
• Overview of Azure Databricks environment: what is databricks, key components like- Workspaces, Clusters, Notebook.
• Overview of Community Edition: the community edition is a free, open-source version of a software product.
• Introduction to different Databricks Capabilities: Data Engineering- data ingestion, data processing, data pipelines. and AI ML- data
exploration, data visualization, machine learning.
• High level overview of unity catalog: Centralized metadata management, Data discovery and research, integration with Databricks
workspaces and other data sources.
• Introduction to credence’s Advance course on databricks.
• Databricks Certification- Data engineering Associate certification.
APPENDIX
Combining knowledge from this course with insights from other Azure certifications enhances your ability to design and implement comprehensive data
solutions. Leveraging Microsoft Fabric’s capabilities alongside Azure Cosmos DB for advanced analytics and data processing. Building a strong skill set
across these certifications positions you for roles in data engineering, analytics, and solution architecture.